# Guidelines for the reliable use of high throughput sequencing technologies to detect plant pathogens and pests

Sebastien Massart, Ian Adams, Al Rwahnih, Steve Baeyen, Guillaume J Bilodeau, Arnaud G Blouin, Neil Boonham, Thierry Candresse, Anne Chandellier, Kris de Jonghe, et al.

<figure><img src="image_1.png" /></figure>

## To cite this version:

Sebastien Massart, Ian Adams, Al Rwahnih, Steve Baeyen, Guillaume J Bilodeau, et al. Guidelines for the reliable use of high throughput sequencing technologies to detect plant pathogens and pests. Peer Community Journal, 2022, 2, pp.e62. 10.24072/pci.infections.100002. hal-03915292

HAL Id: hal-03915292

https://hal.inrae.fr/hal-03915292

Submitted on 29 Dec 2022

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>
RESEARCH ARTICLE

Published
2022-10-25

**Cite as**
Sebastien Massart, Ian Adams, Maher Al Rwahnih, Steve Baeyen, Guillaume J. Bilodeau, Arnaud G. Blouin, Neil Boonham, Thierry Candresse, Anne Chandellier, Kris De Jonghe, Adrian Fox, Yahya Z.A. Gaafar, Pascal Gentit, Annelies Haegeman, Wellcome Ho, Oscar Hurtado-Gonzales, Wilfried Jonkers, Jan Kreuze, Denis Kutjnak, Blanca B. Landa, Mingxin Liu, François Maclot, Marta Malapi-Wight, Hans J. Maree, Francesco Martoni, Natasa Mehle, Angelantonio Minafra, Dimitre Mollov, Adriana G. Moreira, Mark Nakhla, Françoise Petter, Alexander M. Piper, Julien P. Ponchart, Robbie Rae, Benoit Remenant, Yazmin Rivera, Brendan Rodoni, Marleen Botermans, J.W. Roenhorst, Johan Rollin, Pasquale Saldarelli, Johanna Santala, Rose Souza-Richards, Davide Spadaro, David J. Studholme, Stefanie Sultmanis, René van der Vlugt, Lucie Tamisier, Charlotte Trontin, Ines Vazquez-Iglesias, Claudia S.L. Vicente, Bart T.L.H. van de Vossenberg, Marcel Westenberg, Thierry Wetzel, Heiko Ziebell and Benedicte S. M. Lebas (2022) Guidelines for the reliable use of high throughput sequencing technologies to detect plant pathogens and pests, Peer Community Journal, 2: e62.

**Correspondence**
sebastien.massart@uliege.be

**Peer-review**
Peer reviewed and
recommended by
PCI Infections,
https://doi.org/10.24072/pci.infections.100002

<figure><img src="image_3.png" /></figure>

This article is licensed
under the Creative Commons
Attribution 4.0 License.

Guidelines for the reliable use of high throughput
sequencing technologies to detect plant pathogens
and pests

Sebastien Massart<fnref n="1" />, Ian Adams², Maher Al Rwahnih³, Steve Baeyen⁴, Guillaume J. Bilodeau⁵, Arnaud G. Blouin<fnref n="1" />, Neil Boonham⁶, Thierry Candresse⁷, Anne Chandellier⁸, Kris De Jonghe⁴, Adrian Fox², Yahya Z.A. Gaafar⁹, Pascal Gentit¹⁰, Annelies Haegeman⁴, Wellcome Ho¹¹, Oscar Hurtado-Gonzales¹², Wilfried Jonkers¹³, Jan Kreuze¹⁴, Denis Kutjnak¹⁵, Blanca B. Landa¹⁶, Mingxin Liu¹⁷, François Maclot<fnref n="1" />, Marta Malapi-Wight¹⁸, Hans J. Maree¹⁹,²⁰, Francesco Martoni²¹, Natasa Mehle¹⁵,²², Angelantonio Minafra²³, Dimitre Mollov²⁴, Adriana G. Moreira²⁵, Mark Nakhla²⁶, Françoise Petter²⁷, Alexander M. Piper²¹, Julien P. Ponchart<fnref n="1" />,²⁸, Robbie Rae²⁹, Benoit Remenant¹⁰, Yazmin Rivera²⁶, Brendan Rodoni²¹, Marleen Botermans³⁰, J.W. Roenhorst³⁰, Johan Rollin<fnref n="1" />, Pasquale Saldarelli²³, Johanna Santala³¹, Rose Souza-Richards³², Davide Spadaro³³, David J. Studholme³⁴, Stefanie Sultmanis⁵, René van der Vlugt³⁵, Lucie Tamisier<fnref n="1" />, Charlotte Trontin²⁷, Ines Vazquez-Iglesias², Claudia S.L. Vicente³⁶, Bart T.L.H. van de Vossenberg³⁰, Marcel Westenberg³⁰, Thierry Wetzel³⁷, Heiko Ziebell⁹, and Benedicte S. M. Lebas<fnref n="1" />

Volume 2 (2022), article e62

https://doi.org/10.24072/pcjournal.181

Abstract

High-throughput sequencing (HTS) technologies have the potential to become one of the most significant advances in molecular diagnostics. Their use by researchers to detect and characterize plant pathogens and pests has been growing steadily for more than a decade and they are now envisioned as a routine diagnostic test to be deployed by plant pest diagnostics laboratories. Nevertheless, HTS technologies and downstream bioinformatics analysis of the generated datasets represent a complex process including many steps whose reliability must be ensured. The aim of the present guidelines is to provide recommendations for researchers and diagnosticians aiming to reliably use HTS technologies to detect plant pathogens and pests. These guidelines are generic and do not depend on the sequencing technology or platform. They cover all the adoption processes of HTS technologies from test selection to test validation as well as their routine implementation. A special emphasis is given to key elements to be considered: undertaking a risk analysis, designing sample panels for validation, using proper controls, evaluating performance criteria, confirming and interpreting results. These guidelines cover any HTS test used for the detection and identification of any plant pest (viroid, virus, bacteria, phytoplasma, fungi and fungus-like protists, nematodes, arthropods, plants) from any type of matrix. Overall, their adoption by diagnosticians and researchers should greatly improve the reliability of pathogens and pest diagnostics and foster the use of HTS technologies in plant health.

<footnote n="1">¹Plant Pathology Laboratory, Terra-Gembloux Agro-Bio Tech, University of Liège (Belgium), ²FERA Science Ltd, York Biotechnology Campus, Sand Hutton, York (United Kingdom), ³Department of Plant Pathology, University of California-Davis, Davis, CA 95616 (USA), ⁴Flanders Research Institute for Agriculture, Fisheries and Food (ILVO), Plant Sciences Unit, Merelbeke (Belgium), ⁵Canadian Food Inspection Agency, 3851 Fallowfield Road, Ottawa, ON (Canada), ⁶Institute for Agrifood Research Innovations (IAFRI), Newcastle University (United Kingdom), ⁷Univ. Bordeaux, INRAE, UMR Biologie du Fruit et Pathologie, Villenave d'Ornon (France), ⁸CRA-W, Walloon Agricultural Research Centre (Belgium), ⁹Institute for Epidemiology and Pathogen Diagnostics, Julius Kühn Institute (JKI) - Federal Research Centre for Cultivated Plants, Braunschweig, Lower Saxony (Germany), ¹⁰ANSES, Plant Health Laboratory, Unité de Bactériologie, Virologie et détection des OGM (France), ¹¹Plant Health Diagnostic Laboratory, Ministry for Primary Industries (New Zealand), ¹²USDA-APHIS, PPQ, Plant Germplasm and Quarantine Program (USA), ¹³Bejo Zaden BV, Warmenhuizen, Noord-Holland (The Netherlands), ¹⁴CGIAR, Consultative Group for International Agricultural Research, ¹⁵Department of Biotechnology and Systems Biology, National Institute of Biology, Večna pot 111, Ljubljana (Slovenia), ¹⁶Institute for Sustainable Agriculture, CSIC, Av. Menedez Pidal s/n, Campus Alameda del Obispo, 14004 Córdoba (Spain), ¹⁷School of Natural Sciences, University of Tasmania (Australia), ¹⁸USDA-APHIS, BRS, Biotechnology Risk Analysis Programs (USA), ¹⁹Department of Genetics, Stellenbosch University, Private Bag X1, Matieland, 7602 (South Africa), ²⁰Citrus Research International, PO Box 2201, Matieland, 7602 (South Africa), ²¹Agriculture Victoria, AgriBio Centre for AgriBiosciences, 5 Ring Road, Bundoora, VIC 3083 (Australia), ²²School for Viticulture and Enology, University of Nova Gorica, Vipava (Slovenia), ²³IPSP-CNR, Institute for Sustainable Plant Protection, Consiglio Nazionale delle Ricerche, via Amendola 122/D, 70126 Bari (Italy), ²⁴USDA-ARS, Horticultural Crops Research Unit (USA), ²⁵Food and Agriculture Organization of the United Nations (FAO-UN), International Plant Protection Convention (IPPC), ²⁶USDA-APHIS, PPQ, Science and Technology (USA), ²⁷European and Mediterranean Plant Protection Organization, 21 Bd Richard Lenoir, 75011 Paris (France), ²⁸BeeOdiversity, Thines (Belgium), ²⁹Liverpool John Moores University, School of Biological and Environmental Sciences, Byrom Street, L33AF (United Kingdom), ³⁰Netherlands Food and Consumer Product Safety Authority, National Plant Protection Organization (the Netherlands), ³¹Plant Analytics, Finnish Food Authority (Finland), ³²International Seed Federation, Chemin du Reposoir 7, 1260 Nyon (Switzerland), ³³Department of Agricultural, Forest and Food Sciences and AGROINNOVA - Centre of Competence for the Innovation in the Agro-environmental Sector, University of Torino (Italy), ³⁴Biosciences, University of Exeter, Exeter EX4 4QD (United Kingdom), ³⁵Wageningen University and Research, Wageningen (the Netherlands), ³⁶Instituto Nacional de Investigação Agrária e Veterinária (INIAV I.P.), Quinta do Marquês, Oeiras (Portugal), ³⁷DLR Rheinpfalz, Institute of Plant Protection, 67435 Neustadt an der Weinstrasse (Germany)</footnote>
## Introduction

High-throughput sequencing (HTS), also known as next generation sequencing (NGS) or deep sequencing, is one of the most significant advances in molecular diagnostics together with the advent of ELISA in the late 1970s or the PCR methods in the 1980s. HTS technologies can theoretically detect the presence of nucleic acids from any organism present in a sample, including distant variants and uncharacterized organisms, without any *a priori* information on the sample infectious status (Massart et al., 2014).

The adoption of HTS technologies to detect plant pathogens and pests (further referred both as pests in the document) by researchers has been growing steadily for more than a decade. Their most frequent applications correspond to (i) the identification of pests causing novel diseases or diseases of unknown aetiology, with for example 91 novel viruses discovered in fruit trees (Hou et al., 2020), (ii) the sequencing of genomes from known pests to improve knowledge of genomic diversity (Fuentes et al., 2019), (iii) the surveillance, monitoring or source tracking of pests (Aguayo et al., 2018; Bruni et al., 2015; Chandelier et al., 2021; Elbrecht et al., 2019; Núñez et al., 2017; Piper et al., 2019) or (iv) the study of the virome composition at ecosystem scale (Maclot et al., 2020).

Nowadays, the use of HTS technologies for regulated plant pest diagnostics is becoming a reality and is at the agenda of the Commission on Phytosanitary Measures within the International Plant Protection Convention. As described previously (Olmos et al., 2018), HTS technologies may be applied for routine diagnostics to (a) trace the presence of a pest in a region through baseline surveillance programmes, (b) certify nuclear stock and plant propagation material, (c) prevent the introduction of pests into a country or area by (post-entry) quarantine testing, and (d) monitor imported commodities for new potential risks.

The HTS technologies currently rely on a complex succession of steps in the laboratory but also on the bioinformatic analysis of the generated sequences. For example, each step of the bioinformatic analysis (e.g., quality control, target identification and control evaluation) can be influenced by the bioinformatics triade: the algorithms, their parameters and thresholds, and the reference database(s). The steps of the laboratory and bioinformatics components have been described in detail in the companion paper proposing guidelines for the preparation of laboratories before the use of HTS technologies in plant pest diagnostics (Lebas et al., 2022). The large number of operations and the high complexity of some of them increase the risks of creating bias during the analysis and can cause a lack of reliability of the pest detection, e.g., false positive or false negative, which could have dramatic consequences for regulated plant pests.

So, ensuring the reliability of the results generated by HTS technologies in any laboratory is becoming a priority among the scientific community and plant health stakeholders as well as one of the main bottlenecks for their widespread adoption in plant health diagnostics. A cornerstone for overcoming this challenge is the availability of internationally recognized, harmonized guidelines covering both laboratory and bioinformatics components (Adams et al., 2018). Therefore, in the frame of the EU-funded project VALITEST, an international consortium of plant health stakeholders has been formed to write such guidelines and to disseminate them to the scientific community and diagnostics stakeholders.

This publication details guidelines for the reliable use of HTS technologies to detect plant pests. These guidelines are relevant for diagnostic and research laboratories and include all the key phases to ensure reliable use of HTS technologies: definition of intended use, risk analysis, test development, optimisation, validation and verification as well as the quality check requirements when using HTS test in routine diagnosis. The guidelines also describe the impacts of the HTS tests specificities on performance criteria evaluation and on the use of controls, corresponding to appropriate reference material.

## Intended Use and HTS test selection

First, the laboratory should clearly define the intended use of an HTS test, as the number of potential applications can be broad. Indeed, an HTS test can be used as a standalone test or as part of a series of tests for the detection and identification of specific pests (e.g., detection of quarantine pests at import or export, or as part of a phytosanitary certification programme) or for the broad detection of groups of
organisms (e.g., some viruses, bacteria, fungi and other microorganisms) with the likely detection and identification of uncharacterized or unexpected organisms (e.g., surveillance studies).

A range of questions and factors to consider when defining the intended use of an HTS test are provided in section 5.4.3 of EPPO standard PM 7/98 (2019) ('PM 7/98 (4) Specific Requirements for Laboratories Preparing Accreditation for a Plant Pest Diagnostic Activity', 2019). It is also recommended to define the desired taxonomic level for the HTS test: strains/isolates, pathovar, race, *formae speciales*, species, group of species, genus, family, or higher levels of taxa (see paragraph on special consideration for analytical specificity).

Further on, the protocol of the HTS test should be selected according to the intended use, the laboratory constraints (e.g., IT infrastructure, equipment, personnel competence/availability), the time constraints (e.g., expected timeline from sampling to result and acceptable delay of the results), the availability of validated tests and the consulted literature. A scientific and technical review of the protocols (laboratory and bioinformatics) is particularly important as each protocol has its own strengths and limits. In addition, in the last decade, sequencing technologies and protocols have evolved very quickly with regular improvements making previously available protocols outdated. The selection of the HTS test can also rely on the results of the risk analysis.

For the selection of an HTS test, a diagnostic laboratory should follow appropriate international recommendations like those of the EPPO standard PM 7/98 ('PM 7/98 (4) Specific Requirements for Laboratories Preparing Accreditation for a Plant Pest Diagnostic Activity', 2019). Specifically, "*tests described in the legislation (e.g., European Union or national legislation) are mandatory for the countries concerned. If no test is mandatory, tests published as international, regional or national standards should, preferably, be used. Whenever such tests are not available or whenever performance could be improved, laboratory-developed or adapted tests can be considered (ISO/IEC 17025:2017, points 7.2.1.1 and 7.2.1.4).*" In the latter case, the laboratory should select and define the most appropriate HTS test(s), including laboratory and bioinformatic components, for further development and/or optimization followed by validation. Figure 1 provides a decision tree to determine the successions of phases (development, validation, verification, routine use, update management) once the HTS test has been selected.

## Risk analysis

The risk analysis should start as soon as the HTS test has been selected and should be updated whenever needed throughout the development, validation, verification, or routine use of an HTS test (Figure 1).

The risks associated with running HTS tests should therefore be identified before their implementation as diagnostics tests. Some risks linked to the management of the laboratory (e.g., equipment, personnel, consumables, environment, organisation of the laboratory) or the management of the documentation (e.g., quality management system with procedures, records, traceability of measures) are identical to the risks associated with other molecular tests. However, some additional risks and their associated metrics are specific to HTS tests. For example, specific points related to the sequencing platforms and methods (e.g., nucleic acids isolation, nucleic acids enrichment, read length, number of generated reads, platform flexibility and scalability, platform error rate and the type of errors produced) or to the bioinformatics components (e.g., the amount of generated data, the bioinformatic triade (i.e., software, parameters/thresholds and databases used) and the IT infrastructure) should be considered.

Given the complexity of an HTS test, it is recommended to assess holistically the factors influencing the results. The severity of their impact should be estimated, and appropriate measures should be implemented to reduce, minimise or when possible, eliminate the risk (Hébrant et al., 2017; Jennings et al., 2017).

The risks can be analysed through the methodologies described in EPPO standard PM 7/98 (2019). An Ishikawa diagram, adapted for HTS technologies from a previous publication (Mehle et al., 2014), is proposed in Figure 2. Alternatively, an operational risk assessment framework with tools for the assessment and management of risks for a plant health laboratory has been proposed (Murugan & Kumarasinghe, 2018). The risk analysis should be conducted by competent personnel. A non-exhaustive list of risks associated with HTS and their corresponding metrics/controls to monitor/evaluate is presented in Supplementary Material 1. All the risks listed in the Supplementary material are not relevant to all
possible HTS tests. Instead, this indicative table aims to guide laboratories in their risk analysis and should be adapted and complemented by each laboratory to fit the intended use of their HTS test. Specific considerations for the risk analysis for the test selection/test development, for the validation/verification of the test and for ensuring the validity of results during routine diagnostics are discussed further in the text.

The risk analysis will be the basis for establishing the critical parameters and quality checks of the HTS test for routine use. The thresholds, acceptable range, and proper interpretation should be defined in the procedure used during routine analysis and be used for continuous monitoring of test performance through time.

Importantly, the risk analysis should be regularly updated (e.g., change in the level of and/or type of risks) depending on the results of the quality checks obtained during the development, validation, or verification phases but also during the routine use of the HTS test and its performance monitoring. The risk analysis should be documented as it will be useful in the decision-making for verification or validation after modification of an HTS test as well as when troubleshooting errors that may arise (Hébrant et al., 2017; Jennings et al., 2017).

<figure><img src="image_4.png" /><figcaption>**Figure 1:** Decision tree related to the development, validation or verification of an HTS test and to the updates when used in routine diagnostics. The blue stars correspond to the steps where the use of reference material is needed. The grey circles with XP mean technological and scientific expertise needed</figcaption></figure>
<figure><img src="image_5.png" /><figcaption>**Figure 2.** Ishikawa diagram representing the cause and effect of each component of HTS tests. Acronyms: NA: nucleic acids, PCR: polymerase chain reaction, bioIT: bioinformatics</figcaption></figure>
# Specific consideration on reference material and sample panels

## Reference material

Reference material should be used in the sample panel for the validation or verification of HTS tests, as controls to monitor the performance of HTS tests and during the second line (blind test within the laboratory) or third line (proficiency testing) quality checks. It is also highly recommended to use reference material during HTS tests development. In plant health, the commercial availability of reference material or certified reference material is very limited and consequently reference material often needs to be produced by individual diagnostic laboratories ('PM 7/147 (1) Guidelines for the Production of Biological Reference Material', 2021). The production of biological reference material should follow the EPPO standard PM 7/147 ('PM 7/147 (1) Guidelines for the Production of Biological Reference Material', 2021) which aims to support laboratories in describing and producing biological reference material in a consistent manner.

Due to the broad scope of any HTS test, it is generally impossible to have reference material for every expected target nor for every single combination of target(s)/matrix. The reference materials should therefore reflect the diversity of the targeted organisms (e.g., RNA viruses; DNA viruses; viroids; bacteria; eumycetes, fungi-like protists such asoomycetes and mold such asmyxomycete(further collectively referred as fungi); phytoplasmas; nematodes; insects; plants). Table 1 provides examples of reference materials that can be used in HTS tests. Importantly, taking into account the diversity of targets, several reference materials, each infected by specific targets, can be pooled together in a single composite control.

Biological reference material can be fresh, frozen, dried, or lyophilized matrix containing known target(s) and their stability over time should be ensured. The identity and purity of the biological reference material should be checked by the laboratory to maintain confidence in their status. Biological reference material can be used to generate a range of working reference materials with associated data (e.g., annotation of sequences) which can be used as controls to monitor specific step(s) or the whole protocol of the HTS test: subcultures or host tissue from single or several species, nucleic acid extracts, prepared libraries and reference reads datasets. Given the importance of bioinformatic analyses in HTS, the availability of reference reads datasets is recommended. Read datasets generated from reference material can be re-analysed over-time to ensure the reliability of the HTS bioinformatics pipeline and can be used when verifying or validating any pipeline update. To complement biological material, additional artificial reference materials specific to HTS tests may also be used, for example artificially manufactured DNA/RNA or artificially generated sequencing datasets (generated by computer algorithms) used alone or added to datasets of biological origin (semi-artificial). Such reference datasets should be stored and maintained properly and preferably be publicly available.

## Sample panels for development, validation or verification

Designing the panels of samples to be used during the development, validation and/or verification phases is a key step in establishing an HTS test. The type of matrix, the number of samples, the concentration range of targets, the type of genome (DNA vs RNA genomes for viruses), and the type of targets (e.g., including closely related organisms that might be difficult to differentiate) need to be defined depending on the intended use of the HTS test (Hébrant et al., 2017; Jennings et al., 2017).

The panel should contain reference material(s) that could later be used as external or internal control(s) for the continuous monitoring of the test performance. The reference material can correspond to a pool of several reference materials to achieve a better diversity of targets in a single material (see previous point).
<table><caption>**Table 1.** Examples of material that can be used as reference material for HTS depending on the test and targeted pests.</caption><thead><tr><th>Reference material</th><th>HTS test</th><th>Pest range</th><th>Notes/remarks</th></tr></thead><tbody><tr><td>(Mix of) pure culture(s) of strains belonging to representative species expected in the matrix</td><td>Amplicon sequencing</td><td>Bacteria, fungi</td><td>- Strains can be combined into a mock community<br/>- The relative proportion of the strains could vary, with some strains close to the limit of detection to evaluate the analytical sensitivity<br/>- The composition could also allow the continuous evaluation of the analytical specificity by combining taxonomically related strains</td></tr><tr><td>Individuals from taxonomically characterized species</td><td>Amplicon sequencing</td><td>Insects, nematodes</td><td>- A mix of individuals or of their extracted nucleic acids can be combined into a mock community<br/>- The relative proportion of the individuals could vary, with some species close to the limit of detection to evaluate sensitivity<br/>- The composition could also allow the continuous evaluation of the analytical specificity by combining taxonomically related individuals</td></tr><tr><td>Individual plant or seedor pollen from taxonomically characterized species</td><td>Amplicon sequencing</td><td>Invasive plants Weeds</td><td>- A mix of individuals or of their extracted nucleic acid can be combined into a mock community<br/>- The relative proportion of the individuals could vary, with some species close to the limit of detection to evaluate sensitivity<br/>- The composition should also allow the continuous evaluation of the analytical specificity by combining taxonomically related individuals</td></tr><tr><td>Matrix infected with known pest(s).</td><td>Shotgun sequencing for pest detection</td><td>Viruses, viroids, bacteria, fungi, nematodes</td><td>- Pest concentration in plants can be variable in different plant organs, over space and time. Lyophilisation of the reference material can provide stable material<br/>- Matrix could be infected by a single or by multiple pests reflecting the diversity of expected targets<br/>- Several plants/matrices could be combined into a single composite reference sample<br/>- Different concentrations of the pest: at least one high and at least one close to the limit of detection</td></tr><tr><td>A pure culture of a strain or an individual (plant, insect) whose genome is fully sequenced and annotated</td><td>Shotgun sequencing for genome characterization and sequencing</td><td>Bacteria, fungi</td><td>- Need a representative set of major genetic variants at species, subspecies or pathovar level<br/>- Spike the pure culture in plant material as a mock infection</td></tr><tr><td>Reference synthetic nucleic acids</td><td>All HTS tests</td><td>All targets</td><td>- Artificial nucleic acids manufactured<br/>- Artificial nucleic acids could be spiked into nucleic acids purified from plant material.</td></tr></tbody></table>
The variety of organisms that may be detected and identified by HTS tests from a wide range of matrices makes it impossible to include each target (which may be a species or a group of organisms) and matrix combination in a sample panel. For an HTS test targeting multiple organisms of the same group or from different groups, the panel of samples should be composed at a minimum with the different types of samples (matrix) expected to be tested routinely and includes a range of known targets representative of the diversity of organisms that may be detected (e.g., viroids, viruses, bacteria, fungi, insects...). For example, for an HTS test targeting virus, known viruses representing the different types of viral genomes (e.g., ssRNA, dsRNA, DNA) should be selected (Claverie et al., 2018; Y. Z. A. Gaafar & Ziebell, 2020). Similarly, for metabarcoding of macroorganisms such as insects or plants, individuals should be selected that represent a range of species from the expected main target groups (Piper et al., 2019). The concentration of the targets should mimic those in real samples and, for some, be close to the limit of detection. Depending on the test purpose, the inclusion of microbial community standards as reference material (Bakker, 2018), some being commercially available, may also be a relevant option for validation of the workflow.

The results obtained with this representative panel can be completed by a risk-based approach in which the validation or verification data are complemented with the use of other tests performed during the overall diagnostic process, as well as with the use of controls at each step of the HTS process during routine testing. These steps are assessed for their impact and how they can be managed during the risk analysis (Piper et al., 2019; Roenhorst et al., 2018).

The reference samples used during validation or verification, should preferably include several targets, most of which at a concentration close to the limit of detection to evaluate the analytical sensitivity (through a serial dilution of reference sample(s)) and to minimise the risk of false positives due to cross-contamination. During validation, the proper combination of targets in such composite samples can allow the simultaneous evaluation of several criteria with fewer samples.

## Specific consideration for the use of controls

Controls, corresponding to reference materials, are crucial in any diagnostic test, including HTS tests, as they provide essential traceability and validity in testing. They can be used, for example, to validate or verify tests or to monitor the performance of the test and/or of the laboratory (EPPO standard PM 7/98, 2019). These controls can correspond to commercially, publicly or internally available reference materials as defined by EPPO standard PM 7/147 (2021) or to material characterized properly by the laboratory itself. Table 2 provides a description of first line controls that can be used in HTS tests and their application in relation to the main steps of the HTS process. The purpose of each category of control in an HTS context is further detailed below.

Four types of first line controls can be used with HTS tests: three external controls that correspond to positive control, negative control, and alien control, as well as internal positive controls. It should be emphasised that each step of the HTS test should be monitored during each run, from nucleic acids extraction to identification of targets (2^nd^ bioinformatic step), allowing a proper evaluation of controls (3^rd^ bioinformatic step).

Regardless of the type of control, the absence of targets (i.e., no targets in negative controls, absence of other unexpected targets in positive and alien controls) or the presence and abundance of target(s) (i.e., positive and alien controls) should be known unequivocally in each control and should be stable over time (see Reference material section).

## Positive controls

A positive control is an external control used to monitor the correct detection of targets. Positive controls are processed alongside the samples to be analysed. A positive control will usually contain a small but representative fraction of the possible targets an HTS test could detect. It can be prepared as a mix of individual positive controls each containing different targets. It is recommended to use positive controls for which at least some target concentrations are close to the limit of detection. Low level targets are well suited to check the analytical sensitivity of the sequencing run and their low concentration limits the risk of contaminating other samples. A positive control containing target(s) at high concentration is not
recommended due to the much higher risk of detecting these target(s) at low level(s) in the analysed samples because of cross-contamination from the positive control.

Positive controls can also be used as negative controls. Indeed, the detection of an unexpected target in the positive control (in addition to the expected target(s)) may be a signal of contamination from another sample (contamination n->1 from any other sample or the environment (n) to the positive control (1)). This result can be further analysed, for example by comparing the percentage of nucleotide identity with the potential source of contamination.

To optimise the monitoring of target presence and the contamination level in a batch of samples, positive controls can be replaced by or used in addition to an alien control in HTS tests.

## Negative controls

Negative controls of HTS tests are the same as for any other molecular test and correspond to a matrix (or purified water for a blank) which does not contain any target. Negative controls can be used to monitor for contaminants as the detection of target(s) in the negative controls indicates that contamination has occurred during the HTS test (contamination n->1 from any other sample or the laboratory environment to the negative control).

A very low amount of contamination by target sequences will often be present in the data generated from negative controls. These traces of contamination might not be detectable by conventional or real-time PCR. Contamination can be more prevalent in amplicon sequencing because the amplification of traces of contaminant DNA, originating from the environment or the kits used, will be very efficient in the absence of other targeted DNA in the sample. This phenomenon might lead to the risk of overestimating the contamination level, as compared to a sample or control in which abundance of contaminant DNA is extremely low compared to sample DNA and is not likely to be efficiently amplified. For this reason, the use of positive and/or alien controls containing a DNA quantity similar to the analysed samples could allow a better estimation of contamination.

## Alien controls

The third type of external control, alien control, is a new type of control for plant pest diagnostics. An alien control is processed alongside the samples to monitor the detection of its target(s) (role of positive control) and to check for cross contamination between it and the other samples (role of a negative control). Alien controls have been recently used for the first time in plant virus diagnostics by HTS technologies in the frame of the validation of an HTS test (Rong et al., 2022).

An alien control corresponds to a matrix containing one or several targets (called alien targets) which belong to the same group as the target organism(s) but cannot be present in the samples to be tested. So, the detection of reads from an alien target in a sample or another control can be unequivocally considered as a contamination from the alien control. This alien target can be a pest or not. For example, an alien control can be a bacterial or fungal strain from a species or genus restricted to an ecological niche that is not related to the analysed matrix (e.g., extremophile species with plant samples or spore trapping). For insects or plants, a species/genus restricted to temperate climates could be used as an alien control when analysing tropical crops or environments (through traps) and vice versa. For viruses, a wheat sample infected by barley yellow dwarf virus-PAV (BYDV-PAV), an aphid-borne virus infecting only Poaceae, can be used as alien control when analysing viruses infecting potato or banana samples. In this case, the detection of BYDV sequences in the analysed potato or banana samples would indicate that cross contamination has occurred, and it can be quantified. For sequencing of RNA targets another possible alien control is plant RNA spiked with commercialised RNA spike-in controls or, in case of DNA sequencing, plant DNA spiked with gBlocks. When these spikes contain nucleic acids with low and high abundance, they can be used to monitor the sequencing efficiency (analytical sensitivity) as well as contamination to other samples. In case of identification of nematodes and arthropods using mitochondrial genes (e.g., COI, COIII, nad2, nad5) or shotgun sequencing, a possible alien control is commercially available DNA of any mammalian species (e.g., cat, dog, pig) from which the mitochondrial genome is sequenced and assembled to monitor the sequencing efficiency. With an alien control, the contamination analysis takes into account the contamination from a single sample (containing the alien control) to all the other samples (1->n) and is complementary to the classical contamination analysis from any sample to the negative control (n->1).
Calculating the abundance of alien target(s) in alien controls is also important for determining a quantitative threshold for contamination. This threshold can be an absolute number of reads and/or can be calculated as a relative proportion of reads between the alien targets in the samples and in the alien control. For example, 100 reads of the alien target have been detected as contamination in a sample or another control. The relative level of contamination will be different if, within the run, the number of reads of this target in the alien control is 1,000 (meaning 10% contamination) or 10,000,000 (meaning 0.01% contamination).

The alien control should preferably contain at least a high concentration of one of the alien targets (for example a plant with a high virus concentration (the higher the better), purified viruses or a pure isolate of bacteria or fungi). A high concentration of the alien target allows a better detection and quantification of alien contamination in the analysed samples. The number and/or proportion of the alien target sequences in the samples can be analysed (e.g., maximum, average, standard deviation, distribution) and compared to the number and proportion of alien target sequences in the alien sample (relative quantification of contamination).

If the alien control is also used as a positive control, in addition to the alien target(s) at high concentration, at least another alien target should be present at a low concentration, close to the limit of detection to monitor the detection of low abundance target.

As the composition of the alien control is known, the presence of an unexpected target in the generated sequence data from the alien control would also indicate a potential contamination from a sample or another control analysed in the same batch (Galan et al., 2016). This represents a contamination analysis n->1 (also possible with positive control and negative control).

## Internal positive controls

In addition to external controls, internal positive controls (IPC) may also be used in an HTS test. Internal positive controls can correspond to sequences that are expected to be always present in the nucleic acids extracted from the sample (endogenous nucleic acids), for example a plant gene (e.g., nad5 gene, 18S gene (not in case of ribodepletion), COI) constitutively expressed when analysing RNA shotgun sequencing data from plants to identify pests (Bester et al., 2021). Ideally, the selected sequences should be present at a stable and low level in the analysed matrix but above the level of detection to ensure a proper monitoring of the analytical sensitivity. For the detection of targets in complex matrices, this internal positive control presents the limitation of not being a target itself. For identification of isolated pests and pure cultures, non-target loci for identification (e.g., the insect nuclear rDNA where the mitochondrial cox1 gene is used for identification) and the ratio between these loci (e.g., copy numbers of nuclear ribosomal DNA compared to those of nuclear single copy barcode loci in fungal cultures) can be used to monitor if data is generated as expected.

Preferably, each sample can be spiked with synthetic nucleic acids (after nucleic acids extraction) or, even better, before nucleic acids extraction by a matrix containing a known target not expected to be found in the samples to be analysed (this target could therefore be another alien control). An advantage of using synthetic nucleic acids is that they are more readily quantifiable than total nucleic acids. The spiked material should be easily and unambiguously detected by the HTS test. It should be spiked at a low concentration (ideally close to the detection limit) to evaluate the analytical sensitivity of the test and to avoid masking the targets present in the sample. For example, black bean tissue containing an endornavirus has been used to spike grapevine samples to monitor the analytical sensitivity of the assay and set a threshold for the presence or absence of the target (Kesanakurti et al., 2016). An advantage of spiked alien is the proper monitoring of the extraction step that is more difficult when adding synthetic nucleic acids.

In metabarcoding, synthetic 16S rRNA gene spike-in controls have been used to aid in sample tracking and to detect and quantify cross-contamination that may have occurred during the laboratory processes. A distinct spike-in or mixtures of spike-ins were added in low concentration(s) in each sample before starting the DNA extraction (Tourlousse et al., 2018). Similarly, synthetic ITS spike-in controls (mock communities) were used in metabarcoding of forestry fungi. These synthetic controls proved to be useful for monitoring index-hoping and parameterizing the bioinformatic pipelines (Palmer et al., 2018).

In shotgun sequencing, a synthetic community of artificial microbial genomes called sequins (standing for sequencing spike-ins) mimicking the microbial community of the real samples, can be added to environmental DNA samples prior to library preparation. This enables the measurement and mitigation of
technical variation (e.g., library preparation protocols) that can influence sequencing. Sequins also provide a consistent reference that can be used during the development and optimization of HTS tests (Hardwick et al., 2018). Synthetic RNA spike-ins sets have also been used on zebrafish total RNA extracts for monitoring size-selection of RNA and for sample-to-sample normalization of RNA in small RNA sequencing. This improves the technical reproducibility of the test (Locati et al., 2015) but such an approach has not yet been evaluated in plant pest diagnostics.

Internal sequencing controls designed by the HTS technology manufacturers, are available for some sequencing platforms. Manufacturer's instructions should be followed when using these controls. For example, for the Illumina technology, the PhiX phage is used to monitor the sequencing run and is included in sequencing reagents and is always spiked in any sequencing reaction. Its genome sequence is known and it is therefore used to automatically evaluate the accuracy of sequencing (e.g., the proportion of sequencing errors). Similarly, Oxford Nanopore Technologies have a control sequence that can be spiked in.

Commercialised spike-in controls are now becoming available. For example, a common set of external RNA controls called ERCC RNA spike-in mix, has been developed ¹². This control, containing transcripts at very different concentrations, has been used routinely in some plant health diagnostic laboratories.

When rRNA depletion is used for RNA sequencing, the rRNA depletion efficiency could be monitored either by mapping the reads to nuclear, cytoplasmic and mitochondrial rRNA genes or by classification algorithms such as SortMeRNA (Kopylova et al., 2012). The proportion of rRNA reads can give an idea about the efficiency of the rRNA depletion step.

## Suggestion for the use of controls in two intended uses

Currently, one of the most advanced applications of HTS test for regulated plant pest diagnostics is the detection of plant viruses in plant tissue by shotgun sequencing. For such application, the use of the following controls can be suggested:

* an internal control corresponding to a matrix containing an alien target at a concentration close to the limit of detection could be spiked in all the samples;

* an alien control with at least one target at very high abundance could be used to monitor the contamination (1->n and n->1);

* a positive control with all the target(s) close to the limit of detection 13 could be used if needed;

* a negative control corresponding to a non-infected plant or to water is not required.

Whole genome shotgun sequencing tests for multi-locus identification of isolated specimens and pure cultures of pest species also hold increasing interest in routine diagnostics. HTS overcomes the need for independently amplifying and sequencing individual informative loci. For such applications, the following controls can be suggested:

* an alien control containing the loci of interest in copy numbers that could be expected in the target species;

* the relative coverage of the different target (and non-target) loci for identification could be used as an internal control;

* a separate external positive control is not needed in these cases as the sample itself acts as positive control. When target DNA is extracted from these materials, presence of barcode loci is expected at an expected coverage range given the genome size and HTS read yield;

* a separate external negative control is not needed as the sample itself serves as a negative control. High copy targets from the alien control should be monitored in the diagnostic samples under investigation.

Importantly, these suggestions should be evaluated by each laboratory in the frame of the risk analysis and can be adapted depending on the specific context of the laboratory and the intended use of the test.
<table><caption>**Table 2.** Description of first line controls <fnref n="1" /> that may be used in HTS tests and their application in relation to the main steps of the HTS process. (n->1) represents the contamination from n samples to one sample and (1->n) the contamination from one sample to n samples.</caption><thead><tr><th></th><th>Negative controls</th><th>Positive controls</th><th>Alien controls</th><th>Internal controls</th></tr></thead><tbody><tr><td>Aim/Monitoring</td><td>Contamination (n-&gt;1) for any target</td><td>Detection of specific targets in the control<br>Detection of target(s) present at low concentration in the control<br>Contamination (n-&gt;1) for targets absent in the control</td><td>Detection of specific alien targets in the control<br>Detection of target(s) present at low concentration in the control<br>Contamination (n-&gt;1) for targets absent in the control<br>Contamination (1-&gt;n) of the alien target(s) in any sample</td><td>Detection of specific alien targets in any sample (only when an alien control is spiked)<br>Detection of target(s) present at low concentration in the control</td></tr><tr><td>Description</td><td>Same type of matrix (e.g., same plant species) of the analysed samples but free of the target(s) or extraction buffer (NIC), or molecular grade water (NAC)</td><td>Same matrix and range of target(s) expected to be detected in the analysed samples and processed alongside the samples and preferably at low concentration (naturally infected or spiked)</td><td>Same type of matrix (e.g., plant tissue) with target(s) not expected to be found in the analysed samples (i.e., alien target) and processed alongside the samples</td><td>Same type of matrix with alien target or artificial target (e.g., synthetic nucleic acids) spiked at low concentration in the samples or Non-target nucleic acids naturally present (e.g., plant genes) and not related to the sample targets</td></tr><tr><td>Analysis</td><td>Absence or low abundance (e.g., below a set threshold) of target(s) (*) (**)</td><td>Detection of positive control targets<br>Absence or low abundance (e.g., below a set threshold) of any other target (*) (***)</td><td>The matrix can contain alien target at low concentration and at high concentration<br>Absence or low abundance (e.g., below a set threshold) of the alien target in the analysed samples (***)<br>Detection of expected alien target at low concentration in the alien control</td><td>Detection of internal control in each sample<br>Not applicable as included in the analysed samples</td></tr><tr><td>Sampling and nucleic acids extraction</td><td>NIC: matrix without target(s), if not available, extraction buffer</td><td>PIC: matrix containing target(s) from a single or pooled individual(s)</td><td>Matrix containing alien target(s) processed alongside samples</td><td>Spiked nucleic acids to be analysed with alien target or non-target nucleic acids of natural or synthetic origin</td></tr><tr><td>Library preparation</td><td>Nucleic acids previously extracted from a NIC during another HTS test. It will be used as NAC (***)<br>Molecular grade water to verify the absence of contamination (*) (****)</td><td>Nucleic acids previously extracted from a PIC during another HTS test. It will be used as PAC (***)</td><td>Nucleic acids previously extracted from an alien control during another HTS test</td><td></td></tr><tr><td>Sequencing</td><td>Previously prepared libraries from the respective controls can be sequenced for specific monitoring of sequencing<br>DNA sequence of the positive controls designed by the HTS technology manufacturer, present in the sequencing reagents.</td><td></td><td></td><td></td></tr><tr><td>Bioinformatic analysis</td><td>Raw sequencing data generated during previous HTS tests from respective controls or artificially generated data can be used to specifically monitor the bioinformatic analysis step</td><td></td><td></td><td></td></tr></tbody></table>

<footnote n="1">¹Abbreviations of first line controls: NAC: negative amplification control, NIC: negative isolation control, PAC: positive amplification control, PIC: positive isolation control.</footnote>

<footnote>(*) the complete absence of target sequences in a negative control (or sequences in sequenced water) is practically impossible with HTS sequencing.</footnote>

<footnote>(**) if an unexpected target is detected in any control or an alien target is detected in the samples, their presence should be quantified and compared with the controls and samples infected by the target.</footnote>

<footnote>(***) the Positive Amplification Control (PAC) and the Negative Amplification Control (NAC) can be used for amplicon sequencing and shotgun sequencing. They can be considered as "positive library control" or "negative library control" as they check the quality from the library preparation step.</footnote>

<footnote>(****) for shotgun sequencing, the same matrix as the analysed samples but free of the target(s) is preferred over molecular grade water as negative control.</footnote>
# Specific consideration on the performance criteria

## Analytical sensitivity

The analytical sensitivity of several HTS tests has already been compared to (RT-)PCR or real-time (RT-)
PCR test for plant or animal pathogens. The relative performance of the compared HTS and PCR-based
tests were dependent on the tests themselves and very diverse results were obtained. Some publications
found a similar analytical sensitivity between both tests (Hagen et al., 2012; van Boheemen et al., 2020)
while others concluded that HTS-based tests were less (Chandelier et al., 2021) or more sensitive (Santala
& Valkonen, 2018) than PCR-based tests. In addition, this is also dependent on threshold settings for
detection and identification (e.g., number of reads, percentage of genome coverage, average read
coverage on the genome) which can be different between laboratories.

The analytical sensitivity of an HTS test is theoretically very low as a single read from a target can be potentially identified by an appropriate bioinformatics pipeline. Nevertheless, the analytical sensitivity is limited by the contamination level between samples that can vary between sample batches and runs and, within a batch or a run, between target organisms. The analytical sensitivity will depend on the contamination threshold fixed for the run or the batch. For example, 10 reads of a target have been detected in a sample. If there is at least one other sample in the batch with a very high abundance of this target (for example 500,000 reads), there is a risk of cross-contamination from this sample. If this target is not detected in any other sample from the batch or the previous run, the 10 reads are more likely to represent a true infection at a very low level.

Even after optimization of the different steps of the protocol, determining the analytical sensitivity is particularly challenging for an HTS test. This is because the ability of an HTS test to detect a target depends on the number of reads generated from the sample (Bester et al., 2021; Culbreath et al., 2019; Santala & Valkonen, 2018). Noteworthy, the total number of reads generated during the sequencing run, also called sequencing yield or sequencing depth, will vary between runs. Within a sequencing run, when several samples are pooled together, the number of sequences per sample will vary and this variability can increase with the level of pooling. This can have a significant impact on the limit of detection. Indeed, if only 1,000 sequences are generated for the sample and a target is present in the nucleic acid extract at a very low proportion, for example 1:10,000 (target nucleic acids: total nucleic acids), it will most probably not be detected and a false negative would be generated. The probability of detecting the target will rise with the increase of the number of generated reads. Indeed, if one million reads are generated, around 100 reads from the target could be expected to be detected, potentially leading to a true positive result. This has been demonstrated for virus detection (Massart et al., 2019; Pecman et al., 2017; Visser et al., 2016). For example, during an inter-laboratory test performance evaluation of bioinformatics pipelines, the diagnostic sensitivity dropped from 87% when analysing 2.5M reads to 46% when analysing 50,000 reads per sample (Massart et al., 2019).

However, increasing the number of reads per sample also increases the probability of detecting reads originating from cross-contamination between samples and usually present at very low abundance. There is therefore a subtle balance needed between detecting low level targets infecting the sample and background contamination of targets from other samples. Other metrics should also be considered, such as sequence duplication levels or abundance of host reads in case of shotgun sequencing. A sample can have many reads, but the diversity of the sequenced molecules could be low (and the duplication rate very high) due to a poor library preparation. This phenomenon lowers the ability of the test to detect low level targets.

During validation, the minimal sequencing depth can be evaluated by applying the bioinformatic analysis pipeline to different numbers of reads from the same sample. Indeed, the generated reads for a sample can be rarefied by randomly selecting a portion of them. This rarefaction will generate subsamples of reads corresponding to variable lower sequencing depths. The bioinformatic analyses of all these subsamples will identify the sequencing depth(s) at which some targets are no longer detected. It is therefore recommended to generate a very high number of reads per sample during the development, validation and/or verification phases.

Another observed phenomenon is the impact of co-infecting organisms on the ability of the HTS test to detect a target. This is the case when an organism infecting a sample can be missed because another organism is present at a very high concentration and “masks” that organism (called dilution factor (Maclot
et al., 2020)). This situation has been observed for shotgun sequencing in samples co-infected with viruses or fungi (Trontin et al., 2021) and has been shown in amplicon sequencing for fungi using dedicated controls (Chandelier et al., 2021). For amplicon sequencing, the composition of the community determines whether certain species will be detected or not, for example due to competition for the primers in the PCR reaction, or differences in copy number of the used barcode. Furthermore, the DNA extraction efficiency also plays an important role. This has been shown with artificial mixes of nematodes where some species were hard to detect by amplicon sequencing, even if they were the only species present in the mix. Because of poor cell lysis, contaminants could be more easily picked up and amplified than the target (Herren et al., 2020). Nevertheless, this phenomenon cannot be currently anticipated for all the combinations of targets tested. To mitigate this risk, the validation could include reference samples with different proportions/quantities of the targets, some very abundant while others at very low level. Such a series of controls have been recently used for amplicon sequencing to survey the presence of fungal species in spore traps (Chandelier et al., 2021).

To conclude, determining the analytical sensitivity of a developed HTS test requires the determination of thresholds, at least concerning the minimal number of reads per sample and the determination of contamination but which can also include the duplication rate or the abundance of the host sequences. Whatever the situation, and depending on the intended use of the HTS test, a confirmation might or should be carried out. Further scientific developments are expected to improve the determination of the analytical sensitivity of HTS tests.

## Analytical specificity

The analytical specificity of an HTS test should be aligned with the taxonomic resolution needed for the intended use of the test. It will depend on the laboratory protocol used to generate the reads and the genetic variability of the targeted organisms (i.e., the ability to achieve the required taxonomic resolution based on the generated sequences. The bioinformatic triade (i.e., software, parameters/thresholds and databases) of each bioinformatic step is also key in this performance criterion. The desired taxonomic resolution (e.g., genes, isolates/strains, pathovars, *formae speciales*, species, genera or families relevant to plant health) should be determined when describing the scope and intended use of the test. For example, in addition to the identification at species level, when relevant and wherever possible, identification of putative genes and the prediction of relevant gene products and functions (especially those associated with potential phytosanitary risks) could be determined (Budowle et al., 2014). This is particularly important when an organism/agent new-to-science is detected, allowing to differentiate between known pathogenic and non-pathogenic organisms or strains. For example, virulence genes were found in three bacterial species consistently detected in the necrotic stem lesion of acute oak decline disease (Denman et al., 2018). Yet, using additional genetic marker sequences from other genes should be considered carefully, as the scientific knowledge on their association with the targeted organisms might not be exhaustive and homologous genes could be present in the genome of harmless organisms.

For amplicon sequencing, the analytical specificity of the target region can be at least partially evaluated theoretically by analysing all the targeted regions accessible in sequence databases, considering the intended use of the HTS test. The discrimination of closely related organisms based on certain genomic/sequence regions should be studied in depth through bioinformatic analyses. If sequence similarities exist between organisms that could potentially be present in the samples and might interfere with pest detection and identification, those organisms would need to be included in the development and/or validation phases. The taxonomic resolution based on a sequence region can also vary. For example, a genomic region may discriminate against all the species in one genus but may be unable to discriminate against the species of another genus because of the lack of divergence between these species in that genomic region. The analytical specificity could be evaluated by the use of artificial reads datasets with known pest composition or of positive controls containing a mix of targets whose presence has been confirmed by different methods (Tamisier et al., 2021). Ideally, the concentration of the target(s) should reflect as much as possible the concentration in real samples that will be tested.

When applying a shotgun sequencing protocol to a sample composed of multiple organisms, the analytical specificity might depend on the number of sequences generated from each organism, the percentage of the genome covered, the genomic regions that have been sequenced (conserved or specific) and their read depth. As for analytical sensitivity, the taxonomic resolution of a shotgun sequencing will
depend on the sequencing depth and the appropriate target coverage to achieve the intended taxonomic resolution. Sufficient and reproducible sequence coverage and quality needs to be obtained, and a minimal number of generated sequences per sample needs to be clearly stated during the development and/or validation phases.

For bacteria, determination of the analytical specificity can be complicated when applying shotgun sequencing based HTS for specific detection of pathogenic bacteria from a complex sample (not a single colony). This is because of their genome size and the presence of commensal bacterial species which may be related to the pathogenic ones present in the samples. For many bacteria, the appropriate discrimination between family, genus, species or strains may rely on a few specific genes, from which sequences need to be obtained. In cases where no specific genes are available for the identification to species level, the full genome should be obtained, which can be complicated for target organisms that are not isolated. Currently, the sequence databases are not yet representative of the diversity of bacterial species and the limited availability of genome sequences for bacteria will hamper their identification and may lead to the false positive detection of a related species whose genome is in the database(s). The use of curated databases such as the Genome Taxonomy Database (https://gtdb.ecogenomic.org/) is encouraged.

For fungi, protists, nematodes, arthropod pests, invasive plants or weeds, the determination of the analytical specificity is even more difficult than for bacteria. This is because they have larger genomes than bacteria and of the limited availability of genomic sequences in current sequence databases. In 2020, it is estimated that only a very small proportion of fungal DNA is described in databases with about 1% of fungal species having DNA sequences annotated. In addition, low-quality reference genomes or sequences can be contaminated by microbial sequences (some fungi host bacterial cells, for example *Paenibacillus* spp. can live inside fungi) which could interfere with the determination of the analytical specificity.

For viruses, determination of the analytical specificity can be achieved because of their small genomes that can be fully sequenced and of the sequence divergence that exists between and across species. However, the sequence variability of the envelopes or coat proteins of viral species is sometimes close to the species threshold. This can be an issue for establishing the limit between divergent isolates and closely related species. For example, according to the International Committee on Taxonomy of Viruses (ICTV), four molecular discrimination criteria exist for the family *Betaflexiviridae*: nucleotides and amino acids percentage for the coat protein and the replication polymerase genes. Therefore, wherever possible, the full genome should be sequenced or at least, several genomic regions should be sequenced although some uncertainties can remain and the demarcation criteria may be only partially met.

## Selectivity

According to EPPO standard PM 7/98 (2019), the selectivity of molecular methods aims to "*Determine whether variations of the matrix affect the test performance*". The variation of the matrix can correspond to different cultivars of the same plant species, other plant species, different types of soil, different sources of water, or to the community present (e.g., insects, spores, traps).

The presence of inhibitors due to variation in the matrix can be monitored using an appropriate internal control at low concentration. The use of an internal control is recommended as it can help in monitoring this criterion during routine diagnostics as the validation process cannot take into account any matrix variation. On the other hand, if there might be any inhibitors present in the nucleic acid extract, they should have an effect on all nucleic acids in the sample and not only on the target. So, as long as the desired minimal sequencing depth is reached, there might be no inhibitory effect. During validation, this could for instance be investigated by sequencing nucleic acid extract from healthy plant species known to have a high concentration of inhibitors (e.g., *Fragaria* sp., *Rosa* sp.) and from healthy host plants or indicator plants (e.g., *Nicotiana benthamiana*) all with identical nucleic acid concentrations and spiked with the same amount of nucleic acid extract containing the target. Comparison of the percentage of sequence reads mapped to the target sequence will show if matrices influence the HTS test performance.

A further confounding factor is that the genetics of host plants (e.g., different resistance/tolerance to organism(s)), the composition of soil/water samples, or the selectivity of an insect or spore trap, can influence the concentration of the target(s) and change their proportion in the community mix. This can impact the detection of target(s).
## Repeatability and reproducibility

The repeatability and reproducibility of HTS tests has already been evaluated in plant and animal health for some or all the steps (Bester et al., 2021; Chandelier et al., 2021; Cummings et al., 2016; Massart et al., 2019; Miller et al., 2019; Schlaberg et al., 2017; Soltani et al., 2021; van Boheemen et al., 2020). Overall, both performance criteria were evaluated as very good with up to 100% of repeatability and/or reproducibility of target detection.

For the evaluation of the repeatability of molecular methods, the EPPO standard PM 7/98 (2019) recommends analysing three replicates of a sample with a low target concentration. For the reproducibility, the same approach is recommended as repeatability, but with different operator(s) if possible, and on different days and with different equipment when relevant. Whenever possible, HTS tests should be evaluated in the same way.

## Diagnostic sensitivity

Several publications have reported the diagnostic sensitivity (i.e., = number of true positive/( number of true positive + number of false negative)) of various HTS tests when using reference samples infected by a target. The diagnostic sensitivity of HTS-based tests for detecting plant or animal pests was evaluated at 84% and 100% (Schlaberg et al., 2017), 73% and 92%(Miller et al., 2019), 98%(Soltani et al., 2021), 100%(Hanafi et al., 2022) or 95%(Di Gaspero et al., 2022). The diagnostic sensitivity depends on many parameters including the detection thresholds set during the test. For example, it has been shown that lowering the detection threshold of a target to a single read increases the sensitivity to 100% (Di Gaspero et al., 2022). Nevertheless, in such a case, the False Discovery Rate (FDR) will also rise substantially, complicating the interpretation and confirmation of the results.

## Diagnostic specificity and false discovery rate

Determining the diagnostic specificity (= number of true negative/(number of true negative + number of false positive)) of an HTS test is particularly complex due to their broad range of detection and the potentially very high number of targets that can be detected by a test. In addition, the number of potential targets is often unknown, making the determination of true negatives questionable. It is therefore recommended to calculate the false discovery rate (FDR) instead of the diagnostic specificity. The FDR was measured at 21% in a study on grapevine virus (Soltani et al., 2021) while it was calculated at 2.1% during a test performance study of bioinformatic analysis between 21 laboratories (Massart et al., 2019). The FDR will be strongly influenced by the minimal threshold of number of reads to consider the detection of a target and will evolve in opposition to the diagnostic sensitivity: diminishing the threshold will worsen the FDR while improving the diagnostic sensitivity.

## Development and optimisation of HTS test

In the absence of an official HTS standard protocol, the laboratory can be required to develop its own HTS test or to adapt a previously published HTS test to fit the intended use. The HTS test will need to be optimized to ensure it provides the appropriate level of confidence in its results (Hébrant et al., 2017; Maree et al., 2018). The development phase includes the following goals:

* gaining the necessary experience with the test by identifying the critical steps, parameters and quality metrics that may affect the test performance;

* defining the most appropriate controls and their continuous monitoring strategy (see section on Ensuring the validity of results);

* establishing the quality metrics thresholds and acceptable ranges;

* preparing a detailed protocol describing the optimized test conditions and analysis settings for validation and subsequent routine use.

The development phase should cover both the laboratory and the bioinformatics components of the HTS test. Iterative cycles of protocol development or updates should be performed until all HTS test conditions and analysis protocols meet the minimal predefined performance requirements. Designing the panel of samples to be used during the development phase is a key step (see previous point on sample panel). The results obtained with the optimized HTS protocol, including its quality metrics, and thresholds
should be documented (Aziz et al., 2015; Budowle et al., 2014; Hébrant et al., 2017; Jennings et al., 2017; Roy et al., 2018).

During this phase, the laboratory should also determine the number of samples that can be pooled per sequencing run to achieve the desired minimal number of sequencing reads (see validation of HTS test) and establish baseline cost and turnaround time projections (Rehm et al., 2013).

## Validation and verification of HTS test

The ISO/IEC 9000:2015 standard provides a definition of validation: confirmation, through the provision of objective evidence, that the requirements for a specific intended use or application have been fulfilled. The ISO/IEC 17025:2017 standard also recommends that "*the validation shall be as extensive as is necessary to meet the needs of the given application or field of application*". The EPPO standard PM 7/98 (2019) recommends the validation of the following performance criteria for a diagnostic test: analytical sensitivity, analytical specificity which includes inclusivity and exclusivity, selectivity, repeatability and reproducibility.

Before performing the validation or verification of an HTS test, a risk analysis should be carried out to identify which performance criteria need to be evaluated and to what extent (see section on Risk analysis). For tests that have been developed and optimized internally, this analysis can be based on the results of the risk analysis performed during the test development and optimisation phase. Figure 2 provides guidance on the decision-making process for validation or verification of an HTS test.

As described in EPPO standard PM 7/98 (2019), the laboratory interested in using an existing validated HTS test as a routine diagnostic test should demonstrate its ability to perform the test according to the relevant performance characteristics (section 5.4.2 of EPPO standard PM 7/98, 2019). The laboratory should prepare a verification plan based on the outcomes of the risk analysis (see Risk analysis section). The performance criteria to be evaluated according to EPPO standard PM 7/98 (2019) are the same.

It is advised to evaluate the performance criteria by covering all the steps of an HTS test, starting preferably from biological material. The bioinformatic pipeline can also be validated using sequence datasets obtained from biological reference material or artificial reference datasets (see Reference material and First line controls sections) containing known target(s) (Brinkmann et al., 2019; Budowle et al., 2014; Massart et al., 2019; Tamisier et al., 2021).

For HTS tests targeting a broad range of organisms, including uncharacterized organisms (e.g., detection of viruses from Solanaceae), it is not possible to develop and validate protocols for the analysis of all possible combinations of organisms, hosts or matrix. This means that a validation of a HTS test according to the conventional definition 'detection of organism x in matrix y by method z' is beyond the capability of a diagnostic laboratory. Therefore, a more generic approach should be developed based on initial validations and on analysis of the results and the laboratory's 'experiences with the aim of substantiating extrapolation to wider applications (scope). The sample panel used during the validation of the HTS test should include reference material containing key representatives of the targets/pests and mimicking the concentration and composition of real samples expected to be tested routinely. This approach is similar to what is currently proposed for the validation of HTS tests in oncology, where only a representative set of mutations are included in the validation (Hébrant et al., 2017; Jennings et al., 2017; Roy et al., 2018).

To exclude false negatives caused by inhibition (selectivity issue), a selection of representative hosts and cultivars has to be tested. Since it is not possible to test all possible combinations, extrapolation of the results of representative targets and hosts is needed. However, this might not be the most efficient and reliable basis, since it never guarantees the absence of inhibition in another host/cultivar. As an alternative approach, there might be no need to determine selectivity during validation, when making use of an internal alien control in each test. Setting a minimum number of reads for the alien control as a criterion to decide on the validity of the results, abolishes the need to address selectivity during validation and subsequent extrapolation of these data.
## Impact of changes made on a validated HTS test

The rapid pace of development of nucleic acids preparation protocols (e.g., library kits), sequencing platforms (e.g., length of reads, chemistry/technology) and bioinformatic triade (e.g., novel algorithms, updated database) may require updates of HTS tests already validated and used for pest detection in the laboratory. A kit might even be discontinued during the performance criteria evaluation of an HTS test as succeeded for *Musa* viruses (Rong et al., 2022). As recommended by EPPO in standard PM 7/98 (2019), the impact of the update should be evaluated by competent personnel who will decide whether a verification or a validation is required. The management of changes is included in the decision tree presented in figure 1. This decision should be documented by the laboratory.

The laboratory should determine during a risk analysis under which conditions (e.g., changes of reference materials or positive control, starting a new batch/version of critical reagents, changes in a kit composition, changes to algorithms and parameters) and for which HTS steps (e.g., library preparation, bioinformatic analyses) verification or validation is required to maintain the performance of the HTS test after an update (Hébrant et al., 2017).

In some instances, such as a new version of the library preparation protocol or sequencing kit or an update of bioinformatic software, verification or validation may be not required. If a verification or validation is not required *a priori*, the results obtained with the controls using the modified laboratory protocol and/or bioinformatic pipeline during routine analysis should be used to check for the reliability of results. The laboratory should document the decision, including the reason(s) for which validation/verification is not required and how the modified HTS test will be monitored.

To check the bioinformatic pipeline, the same sequence datasets used during the validation of the HTS test can be used. These sequence datasets can also be completed by datasets generated during routine use of the HTS test. In any case, the sequence datasets must be representative for the analysed samples, e.g., with a realistic range of targets at different concentrations, including concentrations close to the limit of detection. In addition, the same sequence datasets should be run regularly through the bioinformatic pipeline to make sure that updates to packages or the operating system do not affect the results. This verification and the comparative analysis of the sequence datasets should be documented. After completion of the verification of the bioinformatic pipeline, the datasets from a previous HTS run can be reanalysed to check that they were analysed correctly.

## Ensuring the validity of results

The recommendations on ensuring the validity of test results stated in EPPO standard PM 7/98 (2019) are valid for HTS tests. The validity of test results should be ensured at different levels, i.e., for each test and diagnostic process, as well as for global quality control of the laboratory (EPPO PM 7/98, 2019). They can be ensured by using reference material as internal and external quality controls.

The quality checks should be performed at different levels. The first line of control monitors the actual performance of each HTS run and its evolution over time (using external and/or internal quality controls), the second line of control checks the performance of a single operator within a laboratory (e.g., blind testing) and the third line of control evaluates the performance of the laboratory (e.g., inter-laboratory comparison) (EPPO standard PM 7/98, 2019).

The types and frequency of quality checks depend on the frequency of use and of the intended use of the HTS test (EPPO standard PM 7/76, 2018) and should be defined during the risk analysis. If there is any issue revealed by the quality checks, the origin of the issue should be investigated and addressed to prevent its recurrence and thus prevent the reporting of incorrect results (Roenhorst et al., 2018).

Quality metrics should be monitored for each sequencing run and routinely collected and compared to those of an optimal validated run. For example, the results obtained by a non-template control and a positive control have been monitored over 20 sequencing runs for pathogen detection in cerebrospinal fluids (Gaafar & Ziebell, 2020). Any significant deviation should be investigated and may require the repetition of the test (for example when one of the targets from the positive controls is not detected in the sequence data). Such data checks can also help investigate the source of the problem in an underperforming test (Hébrant et al., 2017)
## First line controls

The first line controls rely on the use of proper external and/or internal controls (see previous section) and on their analysis (3rd step of the bioinformatics component).

## Replicates

Biological and/or technical replicates can be used to validate the results although the costs can be prohibitive, for example for shotgun library preparation. Technical or biological replicates could be more affordable for amplicon sequencing due to the lower cost per sample.

Additive processing (i.e., pooling the replicates) can be useful for overcoming sampling stochasticity and controlling for false-negative results, while restrictive processing (i.e., only retaining sequences present in several replicates) effectively controls for cross-contamination. To balance the merits of both approaches, it may be best to include a minimum number of technical or biological replicates to allow a majority-rules approach (e.g., 2/3 replicates count as a detection (Piper et al., 2019)). The processing of replicates could be systematic for only a few samples or the controls and would be limited by their costs.

## Second and third line controls and performance monitoring

The second line control checks the performance of a single operator within a laboratory (e.g., blind testing) and the third line control evaluates the performance of the laboratory (e.g., proficiency tests). A list of second and third line controls can be found in EPPO standard PM7/98 (2019).

The types and frequency of quality checks depend on the frequency of use and of the intended use of the HTS test (EPPO PM 7/76, 2018) and should be defined during the risk analysis. If there is any issue resulting from the quality checks, the source of the issue should be investigated and addressed to prevent its recurrence and thus preventing reporting of incorrect results ('PM 7/98 (4) Specific Requirements for Laboratories Preparing Accreditation for a Plant Pest Diagnostic Activity', 2019; Roenhorst et al., 2018).

## Confirmation, biological interpretation and reporting

### Confirmation of the identity of the pest(s)

The confirmation of the detection or identification of pest(s) obtained by HTS tests is similar to any other diagnostic tests. In some cases, there might be a need to confirm the identity of a pest depending on the context of and the level of confidence in the analysis and on the type of organism identified. For regulated pests (i.e., quarantine pest or regulated non-quarantine pest) the results should be confirmed for the critical cases described in EPPO standard PM 7/76 (5), 2018), although, when HTS is used as an identification test, confirmation may not be required and the laboratory should document this decision. These critical cases are "the detection of a pest in an area where it is not known to occur, cases where a pest is identified by a laboratory for the first time", and "detection of a pest in a consignment originating from a country where the pest is declared to be absent" (5), 2018).

The identity of any uncharacterized organism with potential risks to plant health should also be confirmed and should be documented. For example, an apparently virulent strain of Xanthomonas sontii, a species that is normally considered to be a harmless endophyte, was identified taxonomically by HTS. Further testing and Koch's postulates were required, given that it was a surprising candidate for causing disease (Mirghasempour et al., 2020).

For non-regulated organisms commonly found in a particular host, the requirement of confirmatory tests may not be necessary (for example viruses with a wide host range and geographical distribution such as cucumber mosaic virus). However, a confirmatory test of such organisms could be useful in some situations, for example for targeted pest management. Similarly, a confirmatory test can be useful when a non-regulated organism is found associated with unexpectedly severe or unusual symptoms or a new host. Some bacteria and fungi can cause very different severity of symptoms depending on the host plant species. Some examples are Calonectria pseudonaviculata, Diplodia corticola and Xanthomonas sp. (Bérubé et al., 2018; Constantin et al., 2017; Malapi-Wight et al., 2016).

When confirmation is needed, it is recommended to use a test or a combination of tests based on different biological principles and/or to extract again nucleic acids from the same matrix and re-test by other molecular methods. When possible and relevant, the viability or pathogenicity of the pest should be confirmed (e.g., for the critical cases listed above). If available, validated tests should be preferred. General
characteristics of methods have been reviewed for plant virology (Roenhorst et al., 2018), for plant-
parasitic nematodes (Castagnone-Serreno et al., 2011) and for fungi, based on a case study of
*Phytophthora* (Martin et al., 2000). Diagnostic protocols developed by EPPO or International Plant
Protection Convention (IPPC), are available for a range of regulated pests
([https://www.eppo.int/RESOURCES/eppo_standards/pm7_diagnostics](https://www.eppo.int/RESOURCES/eppo_standards/pm7_diagnostics)
and
[https://www.ippc.int/en/publications/593/](https://www.ippc.int/en/publications/593/)).

If no other test is available to confirm the identity of the pest (i.e., an organism that has never been detected/identified before), primers should be designed based on the HTS sequence data and available sequence information in sequence databases. Whenever possible, the primers should be designed to maximise the inclusivity of the test. Alternatively, generic primers allowing the detection of several pests, including the detected one(s), followed by Sanger sequencing of the amplicon could be used to confirm the identity of the pest or, if possible, a new HTS test could be carried out.

Sometimes it may not be possible to confirm the presence and the identity of a pest in a sample. In such case(s), the laboratory should document the results and its decision for quality assurance purposes and if relevant further work should be conducted.

The laboratory should have a procedure describing when a confirmatory test is required (Aziz et al., 2015). Table 3 provides examples of situations when a confirmatory test can be required.

Interpretation of the biological relevance of the identified target(s))

HTS data do not provide any information on the biological relevance of the sequences identified, whether they correspond to a pathogenic organism with associated risks or whether the detected nucleic acids come from living organisms.

Understanding the biological relevance of the target(s) identified by an HTS test is important for evaluating the potential risk the detected organism(s) could pose to plant health (Pest Risk Assessment - PRA). This will provide useful information to the national or international bodies conducting risk evaluation. It applies mainly to poorly characterized and uncharacterized organisms and, in some cases, to known organisms unexpectedly found in a new host. However, the biological characterisation may take time or may not be possible for various reasons (e.g., mixed infection, complex biology, lack of human and/or financial resources) or be carried out by another laboratory.

The determination of the viability of an organism can be required in some instances (depending on the intended use of HTS tests), e.g., bacteria, nematode cysts or insects after phytosanitary treatment. It can also be the case of organisms that could become viable, e.g., virus sequences integrated in plant genome leading to a replicative form. If the organism is a regulated pest, recommendations provided in EPPO standard PM 7/76 (2018) on the viability of organisms should be followed. Appropriate viability tests (when available) should be conducted (e.g., agar plating for bacteria and fungi, fluorescent viability stain for bacteria such as BacLight). This is particularly important when a pest is detected outside of its host(s) as it was shown for plant viruses in wastewater (Bačnik et al., 2020), for the bacterium, *Ralstonia solanacearum* in water courses (Caruso et al., 2005), *Phytophthora* sp. in soils (Riddell et al., 2019) and for the nematode, *Pratylenchus penetrans* (Orlando et al., 2020).

In addition, detected viral sequences may correspond to a bona fide virus infecting other organisms associated with the sample, including bacteria, fungi or arthropods (Al Rwahnih et al., 2011; Marzano & Domier, 2016) or to viral sequences integrated into the plant genome (Baizan-Edge et al., 2019; Massart et al., 2017; Rong et al., 2022). Bacterial, fungal or viral sequences attributed to a pest species might be originating from closely related species (see analytical specificity issues) that are not pathogenic but living as endophytes without causing any harm under the specific environmental conditions. Relevant scientific expertise is essential to biologically interpret HTS results and their implications, in particular in case of the identification of any target at a low concentration, a poorly characterized organism or an uncharacterized organism (Massart et al., 2019).
<table><caption>Table 3. Examples of situations when a confirmatory test can be required.</caption><thead><tr><th>Scenario</th><th>Confirmatory test required?</th><th>Example</th><th>Comment</th></tr></thead><tbody><tr><td>Characterized organism found on known host</td><td>Confirmatory test of regulated organisms (*)</td><td>Any potato pest on EPPO A1 list, identified on potato in post entry quarantine testing EPPO standard PM 3/21 (2019) would need to be confirmed; <i>Xylella fastidiosa</i> (EPPO A2 list) can be identified by HTS (Bonants et al., 2019) but should be confirmed by molecular or serological tests (EPPO PM 7/24, 2019)</td><td>Known organisms with existing test</td></tr><tr><td></td><td>Confirmatory test of non-regulated pests may be required in other situations such as export certification, seed and reproduction material certification, diagnostics from field samples or environmental samples like spore or insect traps</td><td>Southern tomato virus, a non-regulated virus was detected by HTS in tomato from Germany and its presence was confirmed by conventional RT-PCR (Y. Gaafar et al., 2019)</td><td>Potentially emerging pathogens with existing test</td></tr><tr><td>Characterized organism found on known host with unusually severe or novel symptoms</td><td>Confirmatory test on non-regulated pests may be required in some situations such as crop protection, pest management</td><td>An isolate of pepino mosaic virus (CH2) responsible for different symptoms on tomato plants (Hanssen et al., 2009) Resistance breaking isolates of barley yellow mosaic virus Y and barley mild mosaic virus in barley (Rolland et al., 2017)</td><td>Known organisms with existing test</td></tr><tr><td>Characterized organism found on novel host</td><td>Confirmatory test of regulated organisms (*)</td><td>Seven new hosts identified for Physostegia chlorotic mottle virus (PhCoMV) (Temple et al., 2022)</td><td>No existing tests</td></tr><tr><td></td><td>Confirmatory test on non-regulated pests may be required in some situations such as crop protection, pest management</td><td>Several uncharacterized viruses detected in <i>Ullucus tuberosus</i> by HTS and confirmed by real-time RT-PCR (Fox et al., 2019)</td><td>No existing tests</td></tr><tr><td>Uncharacterized organism or poorly characterized organism with potential risk to plant health</td><td>Confirmatory test may be required</td><td>Mycoviruses, some endornaviruses and partitiviruses usually non-pathogenic to the plants (Marzano & Domier, 2016)</td><td>Any sequence data related to uncharacterized organisms with unknown/unlikely risk to plant health should be kept for future analysis as sequence databases evolve and identification may become possible</td></tr><tr><td>Uncharacterized organisms with unknown/unlikely risk to plant health</td><td>Confirmatory test optional</td><td></td><td></td></tr></tbody></table>

<footnote>(*): For regulated pests [i.e., quarantine pest or regulated non-quarantine pest] the results should be confirmed for the critical cases described in EPPO standard PM 7/76 ((5), 2018), although, when HTS is used as an identification test, confirmation may not be required and the laboratory should document this decision.</footnote>
The extent of the biological characterisation depends on the potential risks the detected organism(s) would pose to plant health. For example, the scaled and progressive scientific framework proposed by Massart et al. (Massart et al., 2017) is a useful tool that can be used by plant health stakeholders to perform the biological characterisation and the risk assessment of an uncharacterized or poorly characterized plant virus detected by HTS. If carried out partially or totally by the diagnostic laboratory, it should always document the decisions related to the biological characterisation of the identified organism(s).

Evidence of disease association is especially important when dealing with diseases potentially caused by several organisms (Lamichhane & Venturi, 2015). For example, several organisms (pathogenic and non-pathogenic) were identified in the disease of acute decline of oak (caused by bacteria) (Denman et al., 2018) and in the disease of carrot internal necrosis (caused by viruses) (Adams et al., 2014). Some complex diseases may also be influenced by abiotic factors, such as temperature, moisture, stage of host development (Denman et al., 2018). The understanding of organism interactions with the influence of abiotic factors and the evidence on the causative agent(s) will assist in minimising the potential risk to plant health by developing appropriate management strategies and by taking informed decisions in terms of phytosanitary action.

Fulfilling Koch's postulates, where one pathogen causes one disease, can be impractical in the HTS era and does not apply to diseases caused by several organisms and abiotic factors. Instead, Denman et al. (Denman et al., 2018) used a combination of sequencing and cultivation-based approaches to determine the biotic components of a complex decline-disease, acute decline of oak. Similarly, Adams et al. (Adams et al., 2014) used a combination of molecular tests (conventional PCR and HTS) with a statistical approach to determine which viruses were associated with internal necrosis in carrots. Fox has recently proposed a systematic integrated approach for plant virology, combining epidemiological observations supported by statistical analysis (Fox, 2020). The proposed approach may be extended, with some modification to other plant health disciplines.

## Official reporting of the results

### General recommendations

Regardless of the organism detected by an HTS test, either a known pest (expected or unexpected), a poorly characterized organism or an uncharacterized organism, the reporting of the diagnostic results should follow the recommendations of the EPPO standard PM 7/77 (PM 7/77 (3), 2019), i.e., "*the result of a diagnosis should be reported accurately, clearly, unambiguously and objectively.*" The diagnostic report should be adapted to the need of the client (e.g., different information may be required for a grower and the NPPO) and the confidentiality of the results to the client should be guaranteed (PM 7/77 (3), 2019; 'PM 7/98 (4), 2019). The reporting of HTS test results should be accompanied with an expert judgement and with other confirmatory tests, when needed. This is particularly important for the reporting of uncharacterized organisms.

The laboratory should have a procedure to ensure that the findings of regulated pests or new pests are reported to the relevant NPPO, as recommended in EPPO standard PM 7/98 (2019). The laboratory should also have a procedure on reporting to the NPPO the finding of any uncharacterized organisms with a potential risk to plant health. Information to consider in the report to NPPO includes (if relevant):

* relationship with other organisms in the same taxon (e.g., closely related to an economically important pest)

* relationship with its host (e.g., mycovirus, insect virus)

* potential risk of causing damage to its host

* potential risk for other hosts (economically and/or ecologically important)

* potential risk of spreading

* location risk (e.g., horticultural area versus isolated area)

* viability of the organism (e.g., bacteria alive or dead, virus sequence integrated in plant genome leading to replicative form)

* possible influence of abiotic factors

* presence of other organisms in the same host (e.g., symbiotic or antagonistic effect)

* recommendation for re-sampling/re-testing or other extended analyses
### Inconclusive results

As for any other molecular diagnostic test, inconclusive results (e.g., unable to confirm the presence of an organism because of the HTS results falling in the “grey zone” and/or lack of confirmatory tests) may be obtained with an HTS test (Boukari et al., 2020). Their reporting should follow the recommendations of EPPO standard PM 7/76 (2018), i.e., an explanation on the source of the uncertainty should be provided. The sources of uncertainty in an HTS test can be that the level of the pest is close to the limit of detection, it is present only in a single technical replicate (e.g., one out of two or three replicates), the poor sample quality, the difficulty in distinguishing between episomal and integrated viruses, the discovery of a previously uncharacterized organism for which it is not clear if the tested plant is the host, the lack of completeness of the database, the lack of knowledge whether a bacterium/fungus can be endophytic for some plant species but harmful for others, the limitations of the barcode used. The repeat of the HTS test and/or resampling should be discussed with the client and the decision documented.

### Detection of unexpected organisms

The detection of an unexpected organism is likely to happen during an HTS test targeting a group of organisms. The decision whether to report should be made by competent personnel. The laboratory should have a policy on how to deal with the detection of unexpected organisms. The policy should be agreed with the NPPO and follow local, national or international guidelines, when available, similarly to what is proposed with incidental findings in human genetics (Christenhusz et al., 2013).

Any unexpected organism detected by HTS that may pose a potential risk to plant health should be reported as per EPPO standard PM 7/77 after the recommended confirmatory testing. For example, the detection of an unexpected organism in imported plants (e.g., a known organism of economic significance or a new to science organism that may pose a risk to plant health) could potentially have trade issues (Massart et al., 2017). A study on the screening for fungi by HTS (with confirmation by real-time PCR) in environmental samples originating from air and insect traps in Canada showed the importance of maintaining surveillance of the genus *Heterobasidion* and be able to differentiate the different species, a genus of economic concern, although it is not part of the Canadian regulated pest list (Tremblay et al., 2018).

The reporting of unexpected organisms that may not pose a risk to plant health (e.g., endophytes, beneficial insects) would depend on the context of the analysis (e.g., metagenomics, metabarcoding). This is the case for example of cryptic viruses such as mycoviruses, endornaviruses, partitiviruses and some viruses infecting insects, where some interactions between the virus and its host can be beneficial (Roossinck, 2015). Similarly, insect or environmental DNA metabarcoding analyses could enable the record of unexpected beneficial insects (such as pollinators, parasitoids or biological control agents), previously unknown to be present in the area (Thomsen & Sigsgaard, 2019). A study on aerial spore samples by metabarcoding in Canada revealed the presence of *Diplodia corticola*, a fungal species considered an opportunistic plant endophyte capable of living asymptotically for several years before changing to a pathogen when conducive conditions arise (Bérubé et al., 2018).

### Additional remarks and disclaimers

The laboratory should include in the report additional remarks and disclaimers related to any limitation in the HTS test (for example, the impossibility to distinguish viable and dead pests) and in the performance analysis of the sample (Hébrant et al., 2017). Indeed, the HTS test results depend on the algorithms and sequence databases used. If confirmatory tests have been carried out (like bioassay or viability assays), some limitations of the HTS test may not be relevant.

As for any other diagnostic test, the HTS test results may be affected by the quality of the sample received. In this case, the report may state that the results apply to the sample as received as stated in ISO/IEC 17025:2017 standard.

## Conclusion

These comprehensive guidelines have been proposed to foster the reliable use of HTS technologies for plant pest diagnosis by diagnostic or research laboratories. Together with their companion publication
(Lebas et al., 2022), they cover all the phases and steps required for adopting HTS technologies in a laboratory.

These guidelines should be long-standing as they are applicable whatever the sequencing technology and platform and they are adapted to any HTS test used for the detection and identification of any plant pest, from any type of matrices, by plant health diagnostic laboratories.

## Acknowledgements

Preprint version 3 of this article has been peer-reviewed and recommended by Peer Community Infection (https://doi.org/10.24072/pci.infections.100002).

## Data, scripts and codes availability

Not applicable

## Supplementary material

Supplementary material 1 table is available at the end of this article

## Conflict of interest disclosure

The authors declare that they comply with the PCI rule of having no financial conflicts of interest in relation to the content of the article. Sebastien Massart is a recommender of Peer Community In Infection, Peer Community In Microbiology and Peer Community In Genomics.

## Funding

This article is based upon work from the work package 2 of the project VALITEST (https://www.valitest.eu/), supported by the European Union's Horizon 2020 research and innovation programme under grant agreement no. 773139. J. Rollin is supported by the INEXTVIR project Marie Sklodowska-Curie Innovative Training Network (MSCA-ITN n°813542). A. Blouin was supported by the Seviplan project from the Public Federal Service for health, food chain safety and environment (Belgium).

## References

Adams, I. P., Fox, A., Boonham, N., Massart, S., & De Jonghe, K. (2018). The impact of high throughput sequencing on plant health diagnostics. *European Journal of Plant Pathology*, *152*(4), 909–919.
https://doi.org/10.1007/s10658-018-1570-0

Adams, I. P., Skelton, A., Macarthur, R., Hodges, T., Hinds, H., Flint, L., Nath, P. D., Boonham, N., & Fox, A. (2014). Carrot yellow leaf virus is associated with carrot internal necrosis. *PLoS ONE*, *9*(11).
https://doi.org/10.1371/journal.pone.0109125

Aguayo, J., Fourrier-Jeandel, C., Husson, C., & loos, R. (2018). Assessment of passive traps combined with high-throughput sequencing to study airborne fungal communities. *Applied and Environmental Microbiology*, *84*(11).
https://doi.org/10.1128/AEM.02637-17

Al Rwahnih, M., Daubert, S., Urbez-Torres, J. R., Cordero, F., Rowhani, A., Rwahnih, M. A., Daubert, S., Úrbez-Torres, J. R., Cordero, F., & Rowhani, A. (2011). Deep sequencing evidence from single grapevine plants reveals a virome dominated by mycoviruses. *Arch Virol*, *156*(3), 397–403.
https://doi.org/10.1007/s00705-010-0869-8

Aziz, N., Zhao, Q., Bry, L., Driscoll, D. K., Funke, B., Gibson, J. S., Grody, W. W., Hegde, M. R., Hoeltge, G. A., Leonard, D. G. B., Merker, J. D., Nagarajan, R., Palicki, L. A., Robetorye, R. S., Schrijver, I., Weck, K. E., &
Voelkerding, K. V. (2015). College of American Pathologists' laboratory standards for next-generation sequencing clinical tests. *Archives of Pathology & Laboratory Medicine*, **139**(4), 481-493.
https://doi.org/10.5858/ARPA.2014-0250-CP

Bačnik, K., Kutnjak, D., Pecman, A., Mehle, N., Tušek Žnidarič, M., Gutiérrez Aguirre, I., & Ravnikar, M. (2020). Viromics and infectivity analysis reveal the release of infective plant viruses from wastewater into the environment. *Water Research*, **177**(115628). https://doi.org/10.1016/J.WATRES.2020.115628

Baizan-Edge, A., Cock, P., MacFarlane, S., McGavin, W., Torrance, L., & Jones, S. (2019). Kodoja: A workflow for virus detection in plants using k-mer analysis of RNA-sequencing data. *The Journal of General Virology*, **100**(3), 533-542. https://doi.org/10.1099/JGV.0.001210

Bakker, M. G. (2018). A fungal mock community control for amplicon sequencing experiments. *Molecular Ecology Resources*, **18**(3), 541–556. https://doi.org/10.1111/1755-0998.12760

Bérubé, J. A., Gagné, P. N., Ponchart, J. P., Tremblay, É. D., & Bilodeau, G. J. (2018). Detection of Diplodiaï corticola spores in Ontario and Québec based on High Throughput Sequencing (HTS) methods. *Canadian Journal of Plant Pathology*, **40**(3), 378-386.
https://doi.org/10.1080/07060661.2018.1498394

Bester, R., Cook, G., Breytenbach, J. H. J., Steyn, C., De Bruyn, R., & Maree, H. J. (2021). Towards the validation of high-throughput sequencing (HTS) for routine plant virus diagnostics: measurement of variation linked to HTS detection of citrus viruses and viroids. *Virology Journal* 2021 18:1, 18(1), 1-19.
https://doi.org/10.1186/S12985-021-01523-1

Bonants, P., Griekspoor, Y., Houwers, I., Krijger, M., Van Der Zouwen, P., Van Der Lee, T. A. J., & Van Der Wolf, J. (2019). Development and evaluation of a triplex taqman assay and next-generation sequence analysis for improved detection of Xylella in plant material. *Plant Disease*, **103**(4), 645-655.
https://doi.org/10.1094/PDIS-08-18-1433-RE

Boukari, W., Mollov, D., Wei, C., Tang, L., Grinstead, S., Tahir, N., Mulandesa, E., Hincapie, M., Beiriger, R., & Rott, P. (2021). Screening for sugarcane yellow leaf virus in sorghum in Florida revealed its occurrence in mixed infections with sugarcane mosaic virus and a new marafivirus. *Crop Protection*, **139**, 105373.
https://doi.org/10.1016/j.cropro.2020.105373

Brinkmann, A., Andrusch, A., Belka, A., Wylezich, C., Höper, D., Pohlmann, A., Petersen, T. N., Lucas, P., Blanchard, Y., Papa, A., Melidou, A., Oude Munnink, B. B., Matthijnssens, J., Deboutte, W., Ellis, R. J., Hansmann, F., Baumgärtner, W., van der Vries, E., Osterhaus, A., ... Nitsche, A. (2019). Proficiency Testing of Virus Diagnostics Based on Bioinformatics Analysis of Simulated In Silico High-Throughput Sequencing Data Sets. *Journal of Clinical Microbiology*, **57**(8). https://doi.org/10.1128/JCM.00466-19

Bruni, I., Galimberti, A., Caridi, L., Scaccabarozzi, D., De Mattia, F., Casiraghi, M., & Labra, M. (2015). A DNA barcoding approach to identify plant species in multiflower honey. *Food Chemistry*, **170**, 308-315.
https://doi.org/10.1016/j.FOODCHEM.2014.08.060

Budowle, B., Connell, N. D., Bielecka-Oder, A., Colwell, R. R., Corbett, C. R., Fletcher, J., Forsman, M., Kadavy, D. R., Markotic, A., Morse, S. A., Murch, R. S., Sajantila, A., Schmedes, S. E., Ternus, K. L., Turner, S. D., & Minot, S. (2014). Validation of high throughput sequencing and microbial forensics applications. *Investigative Genetics*, **5**(1). https://doi.org/10.1186/2041-2223-5-9

Caruso, P., Palomo, J. L., Bertolini, E., Álvarez, B., López, M. M., & Biosca, E. G. (2005). Seasonal variation of Ralstonia solanacearum biovar 2 populations in a Spanish river: recovery of stressed cells at low temperatures. *Applied and Environmental Microbiology*, **71**(1), 140-148.
https://doi.org/10.1128/AEM.71.1.140-148.2005

Castagnone-Sereno, P., Skantar, A., Robertson, L. (2011). Molecular Tools for Diagnostics. In: Jones, J., Gheysen, G., Fenoll, C. (eds) Genomics and Molecular Genetics of Plant-Nematode Interactions. Springer, Dordrecht. https://doi.org/10.1007/978-94-007-0434-3_21

Chandelier, A., Hulin, J., San Martin, G., Debode, F., & Massart, S. (2021). Comparison of qPCR and Metabarcoding Methods as Tools for the Detection of Airborne Inoculum of Forest Fungal Pathogens. *Phytopathology*, **111**(3), 570-581. https://doi.org/10.1094/PHYTO-02-20-0034-R

Christenhusz, G. M., Devriendt, K., & Dierickx, K. (2013). To tell or not to tell? A systematic review of ethical reflections on incidental findings arising in genetics contexts. *European Journal of Human Genetics*: *EJHG*, **21**(3), 248-255. https://doi.org/10.1038/EJHG.2012.130

Claverie, S., Bernardo, P., Kraberger, S., Hartnady, P., Lefeuvre, P., Lett, J. M., Galzi, S., Filloux, D., Harkins, G. W., Varsani, A., Martin, D. P., & Roumagnac, P. (2018). From Spatial Metagenomics to Molecular
Characterization of Plant Viruses: A Geminivirus Case Study. *Advances in Virus Research*, **101**, 55–83.
https://doi.org/10.1016/BS.AIVIR.2018.02.003

Constantin, E. C., Haegeman, A., Van Vaerenbergh, J., Baeyen, S., Van Malderghem, C., Maes, M., & Cottyn, B. (2017). Pathogenicity and virulence gene content of Xanthomonas strains infecting Araceae, formerly known as Xanthomonas axonopodis pv. dieffenbachiae. *Plant Pathology*, **66**(9), 1539–1554.
https://doi.org/10.1111/PPA.12694

Culbreath, K., Melanson, S., Gale, J., Baker, J., Li, F., Saebo, O., Kommedal, O., Contreras, D., Garner, O. B., & Yang, S. (2019). Validation and Retrospective Clinical Evaluation of a Quantitative 16S rRNA Gene Metagenomic Sequencing Assay for Bacterial Pathogen Detection in Body Fluids. *The Journal of Molecular Diagnostics*, **21**(5), 913–923. https://doi.org/10.1016/j.jmoldx.2019.05.002

Cummings, L. A., Kurosawa, K., Hoogestraat, D. R., SenGupta, D. J., Candra, F., Doyle, M., Thielges, S., Land, T. A., Rosenthal, C. A., Hoffman, N. G., Salipante, S. J., & Cookson, B. T. (2016). Clinical Next Generation Sequencing Outperforms Standard Microbiological Culture for Characterizing Polymicrobial Samples. *Clinical Chemistry*, **62**(11), 1465–1473. https://doi.org/10.1373/clinchem.2016.258806

Denman, S., Doonan, J., Ransom-Jones, E., Broberg, M., Plummer, S., Kirk, S., Scarlett, K., Griffiths, A. R., Kaczmarek, M., Forster, J., Peace, A., Golyshin, P. N., Hassard, F., Brown, N., Kenny, J. G., & McDonald, J. E. (2018). Microbiome and infectivity studies reveal complex polyspecies tree disease in Acute Oak Decline. *The ISME Journal*, **12**(2), 386–399. https://doi.org/10.1038/ISMEJ.2017.170

Di Gaspero, G., Radovic, S., De Luca, E., Spadotto, A., Magris, G., Falginella, L., Cattonaro, F., & Marroni, F. (2022). Evaluation of sensitivity and specificity in RNA-Seq-based detection of grapevine viral pathogens. *Journal of Virological Methods*, **300**, 114383.
https://doi.org/10.1016/j.jviromet.2021.114383

Elbrecht, V., Braukmann, T. W. A., Ivanova, N. V, Prosser, S. W. J., Hajibabaei, M., Wright, M., Zakharov, E. V, Hebert, P. D. N., & Steinke, D. (2019). Validation of COI metabarcoding primers for terrestrial arthropods. *PeerJ*, **2019**(10). https://doi.org/10.7717/PEERJ.7745

Fox, A. (2020). Reconsidering causal association in plant virology. *Plant Pathology*, **69**(6), 956–961.
https://doi.org/10.1111/ppa.13199

Fox, A., Fowkes, A. R., Skelton, A., Harju, V., Buxton-Kirk, A., Kelly, M., Forde, S. M. D., Pufal, H., Conyers, C., Ward, R., Weekes, R., Boonham, N., & Adams, I. P. (2019). Using high-throughput sequencing in support of a plant health outbreak reveals novel viruses in Ullucus tuberosus (Basellaceae). *Plant Pathology*, **68**(3), 576–587. https://doi.org/10.1111/PPA.12962

Fuentes, S., Jones, R. A. C., Matsuoka, H., Ohshima, K., Kreuze, J., & Gibbs, A. J. (2019). Potato virus Y; the Andean connection. *Virus Evolution*, **5**(2). https://doi.org/10.1093/ve/vez037

Gaafar, Y., Lüddecke, P., Heidler, C., Hartrick, J., Sieg-Müller, A., Hübert, C., Wichura, A., & Ziebell, H. (2019). First report of Southern tomato virus in German tomatoes. *New Disease Reports*, **40**(1), 1.
https://doi.org/10.5197/J.2044-0588.2019.040.001

Gaafar, Y. Z. A., & Ziebell, H. (2020). Comparative study on three viral enrichment approaches based on RNA extraction for plant virus/viroid detection using high-throughput sequencing. *PLOS ONE*, **15**(8), e0237951–e0237951. https://doi.org/10.1371/JOURNAL.PONE.0237951

Galan, M., Razzauti, M., Bard, E., Bernard, M., Brouat, C., Charbonnel, N., Dehne-Garcia, A., Loiseau, A., Tatard, C., Tamisier, L., Vayssier-Taussat, M., Vignes, H., & Cosson, J. F. (2016). 16S rRNA amplicon sequencing for epidemiological surveys of bacteria in wildlife. *MSystems*, **1**(4), 1–22.
https://doi.org/10.1128/mSystems.00032-16

Hagen, C., Frizzi, A., Gabriels, S., Huang, M., Salati, R., Gabor, B., & Huang, S. (2012). Accurate and sensitive diagnosis of geminiviruses through enrichment, high-throughput sequencing and automated sequence identification. *Arch Virol*, **157**(5), 907–915. https://doi.org/10.1007/s00705-012-1253-7

Hanafi, M., Rong, W., Tamisier, L., Berhal, C., Roux, N., & Massart, S. (2022). Detection of Banana Mild Mosaic Virus in Musa In Vitro Plants: High-Throughput Sequencing Presents Higher Diagnostic Sensitivity Than (IC)-RT-PCR and Identifies a New Betaflexiviridae Species. In *Plants* (Vol. 11, Issue 2).
https://doi.org/10.3390/plants11020226

Hanssen, I. M., Paeleman, A., Vandewoestijne, E., Van Bergen, L., Bragard, C., Lievens, B., Vanachter, A. C. R. C., & Thomma, B. P. H. J. (2009). Pepino mosaic virus isolates and differential symptomatology in tomato. *Plant Pathology*, **58**(3), 450–460. https://doi.org/10.1111/j.1365-3059.2008.02018.x
Hardwick, S. A., Chen, W. Y., Wong, T., Kanakamedala, B. S., Deveson, I. W., Ongley, S. E., Santini, N. S., Marcellin, E., Smith, M. A., Nielsen, L. K., Lovelock, C. E., Neilan, B. A., & Mercer, T. R. (2018). Synthetic microbe communities provide internal reference standards for metagenome sequencing and analysis. *Nature Communications* 2018 9:1, 9(1), 1–10. https://doi.org/10.1038/s41467-018-05555-0

Hébrant, A., Froyen, G., Maes, B., Salgado, R., Le Mercier, M., D'haene, N., De Keersmaecker, S., Claes, K., Van Der Meulen, J., Aftimos, P., Van Houdt, J., Cuppens, K., Vanneste, K., Dequeker, E., Van Dooren, S., Van Huysse, J., Nollet, F., Van Laere, S., Denys, B., ... Van Den Bulcke, M. (2017). The Belgian next generation sequencing guidelines for haematological and solid tumours. *The Belgian Journal of Medical Oncology*, 11(2), 56–67.

Herren, G. L., Habraken, J., Waeyenberge, L., Haegeman, A., Viaene, N., Cougnon, M., Reheul, D., Steel, H., & Bert, W. (2020). Effects of synthetic fertilizer and farm compost on soil nematode community in long-term crop rotation plots: A morphological and metabarcoding approach. *PLOS ONE*, 15(3), e0230153–e0230153. https://doi.org/10.1371/JOURNAL.PONE.0230153

Hou, W., Li, S., & Massart, S. (2020). Is There a 'Biological Desert' With the Discovery of New Plant Viruses? A Retrospective Analysis for New Fruit Tree Viruses. *Frontiers in Microbiology*, 11. https://doi.org/10.3389/FMICB.2020.592816

Jennings, L. J., Arcila, M. E., Corless, C., Kamel-Reid, S., Lubin, I. M., Pfeifer, J., Temple-Smolkin, R. L., Voelkerding, K. V., & Nikiforova, M. N. (2017). Guidelines for Validation of Next-Generation Sequencing-Based Oncology Panels: A Joint Consensus Recommendation of the Association for Molecular Pathology and College of American Pathologists. *The Journal of Molecular Diagnostics : JMD*, 19(3), 341–365. https://doi.org/10.1016/J.JMOLDX.2017.01.011

Kesanakurti, P., Belton, M., Saeed, H., Rast, H., Boyes, I., & Rott, M. (2016). Screening for plant viruses by next generation sequencing using a modified double strand RNA extraction protocol with an internal amplification control. *Journal of Virological Methods*, 236, 35–40. https://doi.org/10.1016/J.JVIROMET.2016.07.001

Kopylova, E., Noé, L., & Touzet, H. (2012). SortMeRNA: fast and accurate filtering of ribosomal RNAs in metatranscriptomic data. *Bioinformatics*, 28(24), 3211–3217. https://doi.org/10.1093/bioinformatics/bts611

Lamichhane, J. R., & Venturi, V. (2015). Synergisms between microbial pathogens in plant disease complexes: a growing trend. *Frontiers in Plant Science*, 6(MAY), 1–12. https://doi.org/10.3389/FPLS.2015.00385

Lebas, B., Adams, I., Rwahnih, M. al, Baeyen, S., Bilodeau, G. J., Blouin, A. G., Boonham, N., Candresse, T., Chandelier, A., Jonghe, K. de, Fox, A., Gaafar, Y. Z. A., Gentit, P., Haegeman, A., Ho, W., Hurtado-Gonzales, O., Jonkers, W., Kreuze, J., Kutjnak, D., ... Massart, S. (2022). Facilitating the adoption of high-throughput sequencing technologies as a plant pest diagnostic test in laboratories: A step-by-step description. *EPPO Bulletin*, 52(2), 394–418. https://doi.org/10.1111/EPP.12863

Locati, M. D., Terpstra, I., De Leeuw, W. C., Kuzak, M., Rauwerda, H., Ensink, W. A., Van Leeuwen, S., Nehrdich, U., Spaink, H. P., Jonker, M. J., Breit, T. M., & Dekker, R. J. (2015). Improving small RNA-seq by using a synthetic spike-in set for size-range quality control together with a set for data normalization. *Nucleic Acids Research*, 43(14), e89–e89. https://doi.org/10.1093/nar/GKV303

Maclot, F., Candresse, T., Filloux, D., Malmstrom, C. M., Roumagnac, P., van der Vlugt, R., & Massart, S. (2020). Illuminating an Ecological Blackbox: Using High Throughput Sequencing to Characterize the Plant Virome Across Scales. *Frontiers in Microbiology*, 11. https://doi.org/10.3389/fmicb.2020.578064

Malapi-Wight, M., Salgado-Salazar, C., Demers, J. E., Clement, D. L., Rane, K. K., & Crouch, J. A. (2016). Sarcococca blight: Use of whole-genome sequencing for fungal plant disease diagnosis. *Plant Disease*, 100(6), 1093–1100. https://doi.org/10.1094/PDIS-10-15-1159-RE

Maree, H. J., Fox, A., Al Rwahnih, M., Boonham, N., & Candresse, T. (2018). Application of HTS for Routine Plant Virus Diagnostics: State of the Art and Challenges *Frontiers in Plant Science* 9: 1082. https://doi.org/10.3389/fpls.2018.01082

Martin, R. R., James, D., & Levesque, C. A. (2000). Impacts of molecular diagnostic technologies on plant disease management. *Annual Review of Phytopathology*, 38, 207–239. https://doi.org/10.1146/annurev.phyto.38.1.207
Marzano, S. Y. L., & Domier, L. L. (2016). Novel mycoviruses discovered from metatranscriptomics survey of soybean phyllosphere phytobiomes. *Virus Research*, **213**, 332–342.
https://doi.org/10.1016/J.VIRUSRES.2015.11.002

Massart, S., Candresse, T., Gil, J., Lacomme, C., Predajna, L., Ravnikar, M., Reynard, J.-S., Rumbou, A., Saldarelli, P., Škoric, D., Vainio, E. J., Valkonen, J. P. T., Vanderschuren, H., Varveri, C., & Wetzel, T. (2017). A framework for the evaluation of biosecurity, commercial, regulatory, and scientific impacts of plant viruses and viroids identified by NGS technologies. *Frontiers in Microbiology*, **8**(JAN).
https://doi.org/10.3389/fmicb.2017.00045

Massart, S., Chiumenti, M., De Jonghe, K., Glover, R., Haegeman, A., Koloniuk, I., Komínek, P., Kreuze, J., Kutnjak, D., Lotos, L., Maclot, F., Maliogka, V., Maree, H. J., Olivier, T., Olmos, A., Pooggin, M. M., Reynard, J.-S., Ruiz-García, A. B., Safarova, D., ... Candresse, T. (2019). Virus detection by high-throughput sequencing of small RNAs: Large-scale performance testing of sequence analysis strategies. *Phytopathology*, **109**(3). https://doi.org/10.1094/PHYTO-02-18-0067-R

Massart, S., Olmos, A., Jijakli, H., & Candresse, T. (2014). Current impact and future directions of high throughput sequencing in plant virus diagnostics. *Virus Research*, **188**. https://doi.org/10.1016/j.virusres.2014.03.029

Mehle, N., Dreo, T., Jeffries, C., & Ravnikar, M. (2014). Descriptive assessment of uncertainties of qualitative real-time PCR for detection of plant pathogens and quality performance monitoring. *EPPO Bulletin*, **44**(3), 502–509.
https://doi.org/10.1111/EPP.12166

Miller, S., Naccache, S. N., Samayoa, E., Messacar, K., Arevalo, S., Federman, S., Stryke, D., Pham, E., Fung, B., Bolosky, W. J., Ingebrigtsen, D., Lorizio, W., Paff, S. M., Leake, J. A., Pesano, R., DeBiasi, R., Dominguez, S., & Chiu, C. Y. (2019). Laboratory validation of a clinical metagenomic sequencing assay for pathogen detection in cerebrospinal fluid. *Genome Research*, **29**(5), 831–842.
https://doi.org/10.1101/gr.238170.118

Mirghasempour, S. A., Huang, S., Studholme, D. J., & Brady, C. L. (2020). A Grain Rot of Rice in Iran Caused by a Xanthomonas Strain Closely Related to X. sacchari. *Plant Disease*, **104**(6), 1581–1583.
https://doi.org/10.1094/PDIS-01-20-0179-SC

Murugan, S., & Kumarasinghe, L. (2018). An operational risk assessment framework for a plant health diagnostic laboratory. *EPPO Bulletin*, **48**(2), 225–233.
https://doi.org/10.1111/EPP.12477

Núñez, A., de Paz, G. A., Ferencova, Z., Rastrojo, A., Guantes, R., García, A. M., Alcamí, A., Gutiérrez-Bustillo, A. M., & Moreno, D. A. (2017). Validation of the Hirst-Type Spore Trap for Simultaneous Monitoring of Prokaryotic and Eukaryotic Biodiversities in Urban Air Samples by Next-Generation Sequencing. *Applied and Environmental Microbiology*, **83**(13). https://doi.org/10.1128/AEM.00472-17

Olmos, A., Boonham, N., Candresse, T., Gentit, P., Giovani, B., Kutnjak, D., Liefting, L., Maree, H. J., Minafra, A., Moreira, A., Nakhla, M. K., Petter, F., Ravnikar, M., Rodoni, B., Roenhorst, J. W., Rott, M., Ruiz-García, A. B., Santala, J., Stancanelli, G., ... Massart, S. (2018). High-throughput sequencing technologies for plant pest diagnosis: challenges and opportunities. *EPPO Bulletin*, **48**(2), 219–224.
https://doi.org/10.1111/epp.12472

Orlando, V., Grove, I. G., Edwards, S. G., Prior, T., Roberts, D., Neilson, R., & Back, M. (2020). Root-lesion nematodes of potato: Current status of diagnostics, pathogenicity and management. *Plant Pathology*, **69**(3), 405–417.
https://doi.org/10.1111/PPA.13144

Palmer, J. M., Jusino, M. A., Banik, M. T., & Lindner, D. L. (2018). Non-biological synthetic spike-in controls and the AMPtk software pipeline improve mycobiome data. *PeerJ*, **2018**(5).
https://doi.org/10.7717/PEERJ.4925

Pecman, A., Kutnjak, D., Gutiérrez-Aguirre, I., Adams, I., Fox, A., Boonham, N., & Ravnikar, M. (2017). Next Generation Sequencing for Detection and Discovery of Plant Viruses and Viroids: Comparison of Two Approaches. In *Frontiers in Microbiology* **8**:1998. https://doi.org/10.3389/fmicb.2017.01998

Piper, A. M., Batovska, J., Cogan, N. O. I., Weiss, J., Cunningham, J. P., Rodoni, B. C., & Blacket, M. J. (2019). Prospects and challenges of implementing DNA metabarcoding for high-throughput insect surveillance. *GigaScience*, **8**(8), 1–22.
https://doi.org/10.1093/GIGASCIENCE/GIZ092

PM 7/77 (3) (2019). Documentation and reporting on a diagnosis. EPPO Bulletin, 49(3), 527–529.
https://doi.org/10.1111/epp.12628

PM 7/76 (5) (2018). Use of EPPO Diagnostic Standards. EPPO Bulletin, 48(3), 373–377.
https://doi.org/10.1111/EPP.12506
PM 7/98 (4) (2019). Specific requirements for laboratories preparing accreditation for a plant pest diagnostic activity. *EPPO Bulletin*, **49**(3), 530-563. https://doi.org/10.1111/epp.12629

PM 7/147 (1) (2021). Guidelines for the production of biological reference material. *EPPO Bulletin*, **51**(3), 499-506. https://doi.org/10.1111/epp.12781

Rehm, H. L., Bale, S. J., Bayrak-Toydemir, P., Berg, J. S., Brown, K. K., Deignan, J. L., Friez, M. J., Funke, B. H., Hegde, M. R., & Lyon, E. (2013). ACMG clinical laboratory standards for next-generation sequencing. *Genetics in Medicine : Official Journal of the American College of Medical Genetics*, **15**(9), 733-747. https://doi.org/10.1038/GIM.2013.92

Riddell, C. E., Frederickson-Matika, D., Armstrong, A. C., Elliot, M., Forster, J., Hedley, P. E., Morris, J., Thorpe, P., Cooke, D. E. L., Pritchard, L., Sharp, P. M., & Green, S. (2019). Metabarcoding reveals a high diversity of woody host-associated Phytophthora spp. in soils at public gardens and amenity woodlands in Britain. *PeerJ*, **7**, e6931-e6931. https://doi.org/10.7717/PEERJ.6931

Roenhorst, J. W., de Krom, C., Fox, A., Mehle, N., Ravnikar, M., & Werkman, A. W. (2018). Ensuring validation in diagnostic testing is fit for purpose: a view from the plant virology laboratory. *EPPO Bulletin*, **48**(1), 105-115. https://doi.org/10.1111/EPP.12445

Rolland, M., Villemot, J., Marais, A., Theil, S., Faure, C., Cadot, V., Valade, R., Vitry, C., Rabenstein, F., & Candresse, T. (2017). Classical and next generation sequencing approaches unravel Bymovirus diversity in barley crops in France. *PLOS ONE*, **12**(11), e0188495-e0188495. https://doi.org/10.1371/JOURNAL.PONE.0188495

Rong, W. Rollin J., Hanafi M., Roux N., Massart S. (2022). Validation of high throughput sequencing as virus indexing test for Musa germplasm: performance criteria evaluation and contamination monitoring using an alien control. In press https://doi.org/10.1094/PHYTOFR-03-22-0030-FI

Roossinck, M. J. (2015). Plants, viruses and the environment: Ecology and mutualism. *Virology*, **480**, 271-277. https://doi.org/10.1016/j.virol.2015.03.041

Roy, S., Coldren, C., Karunamurthy, A., Kip, N. S., Klee, E. W., Lincoln, S. E., Leon, A., Pullambhatla, M., Temple-Smolkin, R. L., Voelkerding, K. V, Wang, C., & Carter, A. B. (2018). Standards and Guidelines for Validating Next-Generation Sequencing Bioinformatics Pipelines: A Joint Recommendation of the Association for Molecular Pathology and the College of American Pathologists. *The Journal of Molecular Diagnostics : JMD*, **20**(1), 4-27. https://doi.org/10.1016/J.JMOLDX.2017.11.003

Santala, J., & Valkonen, J. P. T. (2018). Sensitivity of Small RNA-Based Detection of Plant Viruses. In *Frontiers in Microbiology* 9:939. https://doi.org/10.3389/fmicb.2018.00939

Schlaberg, R.; Chiu, C. Y.; Miller, S.; Procop, G. W.; Weinstock, G. (2017) Validation of Metagenomic Next-Generation Sequencing Tests for Universal Pathogen Detection, Archives of Pathology & Laboratory Medicine, Volume 141 no. 6, pp. 776-786. https://doi.org/10.5858/arpa.2016-0539-RA

Soltani, N., Stevens, K. A., Klaassen, V., Hwang, M.-S., Golino, D. A., & al Rwahnih, M. (2021). Quality Assessment and Validation of High-Throughput Sequencing for Grapevine Virus Diagnostics. In *Viruses* (Vol. 13, Issue 6). https://doi.org/10.3390/v13061130

Tamisier, L., Haegeman, A., Foucart, Y., Fouillien, N., Al Rwahnih, M., Buzkan, N., Candresse, T., Chiumenti, M., De Jonghe, K., Lefebvre, M., Margaria, P., Reynard, J. S., Stevens, K., Kutnjak, D., & Massart, S. (2021). Semi-artificial datasets as a resource for validation of bioinformatics pipelines for plant virus detection. *Peer Community Journal, 1*. https://doi.org/10.24072/pcjournal.62

Temple, C., Blouin, J., Arnaud G., Jonghe, K. de, Foucart, Y., Yoika, Botermans, Ms. M., Westenberg, Mr. M., Schoen, Mr. R., Gentit, Dr. P., Visage, Ms. M., Verdin, Mr. E., Wipf-Scheibel, Ms. C., Ziebell, Dr. H., Gaafar, Dr. Y. Z. A., Kutnjak, Dr. D., Vučurović, Dr. A., Rivarez, Mr. M. P. S., Richert-Pöggeler, Dr. K. R., Ulrich, Mrs. R., Zia, Mr. A., ... Massart, S. (2022). Biological and genetic characterization of Physostegia chlorotic mottle virus in Europe based on host range, location, and time. *Plant Disease*. https://doi.org/10.1094/PDIS-12-21-2800-RE

Thomsen, P. F., & Sigsgaard, E. E. (2019). Environmental DNA metabarcoding of wild flowers reveals diverse communities of terrestrial arthropods. *Ecology and Evolution*, **9**(4), 1665-1679. https://doi.org/10.1002/ECE3.4809

Tourlousse, Di. M., Ohashi, A., & Sekiguchi, Y. (2018). Sample tracking in microbiome community profiling assays using synthetic 16S rRNA gene spike-in controls. *Scientific Reports* 2018 8:1, 1-9. https://doi.org/10.1038/s41598-018-27314-3
Tremblay, É. D., Duceppe, M. O., Bérubé, J. A., Kimoto, T., Lemieux, C., & Bilodeau, G. J. (2018). Screening for Exotic Forest Pathogens to Increase Survey Capacity Using Metagenomics. *Phytopathology*, *108*(12), 1509–1521. https://doi.org/10.1094/PHYTO-02-18-0028-R

Trontin, C., Agstner, B., Altenbach, D., Anthoine, G., Bagińska, H., Brittain, I., Chabirand, A., Chappé, A. M., Dahlin, P., Dreo, T., Freye-Minks, C., Gianinazzi, C., Harrison, C., Jones, G., Luigi, M., Massart, S., Mehle, N., Mezzalama, M., Mouaziz, H., ... Vučurović, A. (2021). VALITEST: Validation of diagnostic tests to support plant health. *EPPO Bulletin*, *51*(1), 198–206. https://doi.org/10.1111/EPP.12740

van Boheemen, S., van Rijn, A. L., Pappas, N., Carbo, E. C., Vorderman, R. H. P., Sidorov, I., van 't Hof, P. J., Mei, H., Claas, E. C. J., Kroes, A. C. M., & de Vries, J. J. C. (2020). Retrospective Validation of a Metagenomic Sequencing Protocol for Combined Detection of RNA and DNA Viruses Using Respiratory Samples from Pediatric Patients. *The Journal of Molecular Diagnostics*, *22*(2), 196–207. https://doi.org/10.1016/j.jmoldx.2019.10.007

Visser, M., Bester, R., Burger, J. T., & Maree, H. J. (2016). Next-generation sequencing for virus detection: covering all the bases. *Virology Journal*, *13*, 85. https://doi.org/10.1186/s12985-016-0539-x
<table><caption>**Supplementary table 1.** Potential sources of errors (non exhaustive) that may negatively affect the performance of HTS tests with their causes, consequences and mitigation (other mitigation options may exist). For details on controls, see section "First line controls"</caption><thead><tr><th>HTS step</th><th>Risk</th><th>Cause</th><th>Consequence</th><th>Mitigation</th></tr></thead><tbody><tr><td rowspan="2">Sampling</td><td>Sampled material not fit for purpose</td><td>Unclear or absence of procedure for sampling.</td><td>False negative</td><td>Protocol describing the sampling procedure with sample acceptance criteria (e.g. type of matrix, material received in sufficient quantity, quality of material suitable for the HTS test).</td></tr><tr><td>Cross-contamination between samples</td><td>Inappropriate hygiene procedure.</td><td>False positive</td><td>Protocol describing the measures to minimise cross-contamination. Use of appropriate high concentration alien controls and/or low concentration positive controls. Use of appropriate threshold values to discard contamination sequences.</td></tr><tr><td>Sample handling</td><td>Inappropriate transport and storage conditions leading to sample degradation</td><td>Absent or unclear procedure for transport and storage.</td><td>False negative</td><td>Protocol describing transport and storage conditions, including the acceptance criteria.</td></tr><tr><td rowspan="4">Nucleic acid extraction</td><td>Cross-contamination between samples</td><td>See cross-contamination between samples during sampling.</td><td></td><td></td></tr><tr><td>Inappropriate integrity and purity of nucleic acids</td><td>Inhibitors in the matrix, reagent lot number or inappropriate handling and/or storage conditions of samples or reagents will affect the nucleic acid extraction.</td><td>False negative</td><td>Protocol describing the control of the integrity and purity of nucleic acids with appropriate thresholds. Use of appropriate internal positive controls.</td></tr><tr><td>Low quantity of nucleic acids</td><td>Low quantity of input biological material or inappropriate nucleic acid extraction procedure</td><td>False negative</td><td>Protocol describing the minimal quantity of biological material required and the control of the quantity of nucleic acids with appropriate thresholds. Use of appropriate internal positive controls.</td></tr><tr><td>Bias in presence and relative abundance of targets</td><td>Protocol favouring specific targets (e.g. VANA or dsRNA protocol under representing some viral taxa, spore forming bacteria or fungal spores difficult to extract with certain extraction protocols and that may be missed or under represented).</td><td>False negative</td><td>Proper definition of the intended use of the HTS test and selection of the most appropriate extraction protocol. Use of appropriate positive control with relevant targets at known concentration.</td></tr><tr><td rowspan="2">Library preparation for shotgun sequencing</td><td>Cross-contamination between samples</td><td>See cross-contamination between samples during sampling.</td><td></td><td></td></tr><tr><td>Content bias (fragment size and concentration of nucleic acids) related to the protocol used</td><td>Library protocol favouring some targets (e.g. siRNA/VANA/dsRNA sequencing favouring different viruses, PCR amplification favouring amplicons of smaller size, primers not inclusive enough).</td><td>False negative</td><td>Proper definition of the intended use of the HTS test and selection of the most appropriate protocol with appropriate threshold.</td></tr></tbody></table>
<table><thead><tr><th>HTS step</th><th>Risk</th><th>Cause</th><th>Consequence</th><th>Mitigation</th></tr></thead><tbody><tr><td rowspan="5">PCR for amplicon sequencing and some shotgun sequencing</td><td>Amplification error</td><td>Fidelity of the polymerase can create errors in the generated strand, resulting in false mutation that can bias the sequence annotation.</td><td>False positive</td><td>Use high fidelity polymerase. Check target sequences from positive controls.</td></tr><tr><td>Bias during amplification</td><td>Modification of proportion between targets. Creation of large proportion of duplicated reads.</td><td>False negative Bias in relative proportion for metabarcoding</td><td>Appropriate primer selection. Set a maximum number of PCR cycles. Use of appropriate positive controls with target(s) at low concentration. Monitor the reads duplication rate (under threshold set)</td></tr><tr><td>Missing low abundance target due to low input quantity of nucleic acids</td><td>The quantity of nucleic acids used for PCR is usually very small and rare targets might be missed</td><td>False negative</td><td>Rising the quantity of nucleic acids in PCR reaction while avoiding bias Use of replicates for PCR reactions</td></tr><tr><td>Cross-contamination between samples</td><td>See cross-contamination during sampling.</td><td></td><td></td></tr><tr><td>Unequal representation of samples in the pool with samples sequenced at very low depth.</td><td>Unreliable quantification of prepared nucleic acids between samples or favoured sequencing of target(s) from some samples.</td><td>False negative</td><td>Check the quantity of prepared nucleic acids using appropriate protocol. Monitor sequencing depth per sample after demultiplexing and set a minimal sequencing depth.</td></tr><tr><td rowspan="3">Sample indexing and pooling</td><td>Index-hopping (e.g. a small proportion of sequence is not tagged with the appropriate index)</td><td>Presence of free index in the reaction getting tagged randomly to sample.</td><td>False positive</td><td>Use highly distinct and unique index between two sequencing runs. Dual labelling index should be preferred for Illumina sequencing technology. Monitor index switching and cross-contamination between library preparation and between sequencing runs. Check the number or % of sequences unexpected in controls.</td></tr><tr><td>Unequal size of the prepared nucleic acids between samples</td><td>The size of the prepared nucleic acids may influence the sequencing yield of the sample or the amplicon. For example, low sequencing yields are obtained with long fragments (&gt;500-600 nucleotides) with Illumina technology.</td><td>False negative</td><td>Define acceptable range of sequence length in prepared libraries. Pool PCR products of similar length or shear samples at appropriate size.</td></tr><tr><td></td><td></td><td></td><td></td></tr></tbody></table>
<table><thead><tr><th>HTS step</th><th>Risk</th><th>Cause</th><th>Consequence</th><th>Mitigation</th></tr></thead><tbody><tr><td rowspan="7">Sequencing</td><td>Not enough sequences</td><td>Presence of inhibitors in one of the samples Sequencing reagents Under- or overloading the sequencing device with DNA.</td><td>False negative</td><td>Set minimal sequencing depths per sample.</td></tr><tr><td>Low quality of nucleotide bases</td><td>Sequencing reagents, accuracy of signal measurement, background signal Low quality and incorrect sequences will be generated and can be eliminated during reads analysis or introduce errors during bioinformatic analyses.</td><td>False negative False positive</td><td>Monitor the number of reads eliminated or remaining per sample after reads analysis.</td></tr><tr><td>Sequencing errors</td><td>Type of sequencing platform and sequencing method (e.g. sequencing reagents).</td><td>False negative False positive</td><td>Use of appropriate controls to evaluate the sequencing fidelity (e.g. positive control, alien control, PhiX control with Illumina technology). Use appropriate threshold. Sequencing the DNA library in both direction to have overlap between reads.</td></tr><tr><td>Inter-run contamination</td><td>Traces of libraries from a previous run are sequenced.</td><td>False positive</td><td>Proper washing of the sequencing machine and alternate the indexes used between runs. Detecting indexes used during previous run (when alternating indexes between runs)</td></tr><tr><td>Data storage capacity</td><td>IT computational system not adapted for handling and storing HTS data.</td><td>Corrupted and/or loss of data</td><td>Procedure detailing the required storage capacity including back up, the safe storage of HTS data with disclosure clause and the retention period.</td></tr><tr><td>Corrupted data Loss of data Disclosure of data</td><td>Target sequences with high GC content are currently not well sequenced (e.g. some bacterial taxa, operon). The sequence coverage of these regions can be very low or absent compared to the other regions.</td><td>False negative</td><td>Determine the extent of GC bias for HTS test targeting organisms with high GC content. Follow GC content statistics of the sequencing.</td></tr><tr><td>GC bias</td><td>Inappropriate demultiplexing software and parameters might fail to assign a small proportion of the sequences to the proper sample. inappropriate combination of indexed samples within a run.</td><td>False positive</td><td>Verify the compatibility of demultiplexing parameters and software with the used indexes. Use highly distinct and unique index between two sequencing runs.</td></tr><tr><td>First step of the bioinformatic analyses</td><td>Inappropriate demultiplexing</td><td></td><td></td><td></td></tr></tbody></table>
<table><thead><tr><th>HTS step</th><th>Risk</th><th>Cause</th><th>Consequence</th><th>Mitigation</th></tr></thead><tbody><tr><td rowspan="5">Second step of the bioinformatic analyses</td><td>Low quality of reads assembly</td><td>The use of too stringent parameters during the assembly of reads can hamper reads assembly while a higher tolerance to mismatches can create assembly artefacts.</td><td>False negative<br>False positive</td><td>Set appropriate thresholds for the parameters of reads assembly quality.<br>Monitor the parameters of reads assembly quality in each run using appropriate controls.</td></tr><tr><td>Low mapping quality</td><td>The reference database used (and its completeness and curation level) and the parameter of mapping can create bias: incorrect mapping of reads on a reference from another species or absence of mapping due to the sequencing of a distant isolate of the pest.</td><td>False negative<br>False positive</td><td>Set appropriate thresholds for the parameters of mapping quality, such as percentage of identity with reference sequence, quality of assembled genomes.<br>Monitor the parameters of mapping quality in each run using appropriate controls<br>Check the suitability of the reference sequence and/or sequence database.</td></tr><tr><td>Non-uniform coverage of the pest genome</td><td>Variation in read depth or absence of coverage can be due to low abundance of the pest, the use of distant reference genome, the difficulty for sequencing some regions of the genome.</td><td>False negative</td><td>Set the minimum criteria for the level of coverage across the sequenced regions (which may be partial or full genome).<br>Monitor the uniformity of coverage in each run using appropriate controls.</td></tr><tr><td>Wrong taxonomic and functional annotation</td><td>Database completeness and accuracy are essential for proper annotation of the sequences. Software methodology and threshold parameters could also influence the annotation.<br>The particular case is outside the knowledge/know how of the diagnostician/bioinformatician.</td><td>False negative<br>False positive</td><td>Set appropriate threshold parameters to ensure the reliable identification of targets according to species demarcation criteria.<br>Monitor the taxonomic and/or functional annotation using appropriate controls.<br>Check the suitability of the reference sequence and/or sequence database.<br>Use latest official disciplinary taxonomy rules for each species/group of species.<br>Personnel competent in the bioinformatic analyses and the relevant plant health discipline.</td></tr><tr><td>Variant not identified or false variant identified</td><td>The identification of variants can be done according to different methodologies, each depending on several parameters. It also depends on the quality control of the reads (e.g. SNPs can originate from low quality reads or sequencing errors).</td><td>False negative<br>False positive</td><td>Set appropriate threshold parameters (e.g. minimal SNP frequency, coverage, base-calling quality, strand bias) to ensure the reliable identification of variants.<br>Controlled artificial datasets could be used.<br>Sequencing replicates of a single sample.</td></tr><tr><td>Third step of the bioinformatic analyses</td><td>Targets of control samples not retrieved</td><td>One or several threshold(s) of parameters from the previous bioinformatic steps of the HTS test have not been appropriately set.</td><td>False negative<br>False positive</td><td>Set appropriate threshold parameters to ensure the reliable identification of controls.<br>Controlled artificial datasets could be used.</td></tr></tbody></table>