Requirements for NAHLN Deployment of New Diagnostic Assays

FINAL DOCUMENT 06.21/2023

Submission Checklist for Feasibility Studies 

This document is intended to provide guidance to commercial manufacturers of reagents and testing 

platforms regarding the minimum validation requirements for acceptance into the NAHLN. Additional 

approvals by the National Veterinary Services Laboratories (NVSL), VS’ Center for Epidemiology and 

Animal Health, and the appropriate VS commodity program may be required prior to deployment of 

new test methods to NAHLN laboratories.

For the initial stage of approval, the following information is requested. See Appendix A for additional 

detail on each of these topics: 

1. Intended purpose of the assay including specific use cases

2. Side-by-side comparison of current NAHLN protocol/SOP using an initial test panel obtained 

from NVSL [subject to test panel availability*]. Data from this panel should include: 

a) Operating range of assay: One or two high positive reference samples that have been 

serially diluted to extinction.

b) Assay specificity: for nucleic acid-based assays, an in-silico analysis should be provided 

for new primer and/or probe sequences. (Recommended but not required.)

c) Assay Standardization: comparison of NAHLN SOP to the new method. 

a. Test panel must contain a minimum of 30 samples and cover high pos, low pos 

and negative reference samples.

d) Matrix Inhibition: 

b. Data must be provided for each different sample or matrix type that will be used 

for the assay. Recommended to include all currently approved sample types. 

e) Robustness

c. Data must be provided for at least two lots of reagents, and preferably tested by 

two different people. 

d. Specific test parameters for nucleic acid-based tests:

 Test results provided using at least one and preferably all, commercial 

extraction and thermal cycler instruments used by NAHLN laboratories. 

e. Specific test parameters for commercial primer/probe reagents:

 Test results provided using a minimum of two and preferably three 

common commercial master mixes. 

f. Specific QC controls for nucleic acid-based PCR tests:

 No-template control (NTC)

 Analyte-positive control

 Inhibition control or extraction control 

 For RNA assays – reverse transcription control

 
Requirements for NAHLN Deployment of New Diagnostic Assays

FINAL DOCUMENT 06/21/2023

2

Submission Checklist for Final Validation Study 

Once the initial feasibility study data set has been reviewed and approved by the NAHLN MTWG, a 

second, larger panel suitable for generating the following data will then be made available through NVSL 

[subject to test panel availability*]. A side-by-side comparison of the second panel to the current NAHLN 

SOP must be conducted for all the following parameters and provided for final review and approval by 

NVSL and NAHLN prior to deployment. See Appendix B for additional details. 

1. Analytical Performance: 

a. Repeatability: Test three to five currently circulating field samples representing the analyte 

activity within the full operating range of the assay. 

b. Analytical Sensitivity (limit of detection): a dilution series of the target pathogen diluted in 

all the different sample matrices. The dilution series should be to extinction.

c. Analytical Specificity: refers to the ability of an assay to measure a particular organism or 

substance, rather than similar organisms, in a sample.

2. Diagnostic Performance: See Appendix B for additional details on each of these requirements. 

a. Diagnostic Sensitivity (estimation of false negatives): Ability of the test to identify the 

presence of disease correctly. 

b. Diagnostic Specificity (estimation of false positive): Measure of diagnostic test accuracy.

c. Diagnostic Reproducibility: Agreement of multiple laboratories using the same diagnostic 

procedure. 

*If test panels are not available through NVSL, manufacturers may choose to provide their own sample 

panels; however, prior approval of panel composition will be required from NVSL and the NAHLN 

Program Office before initiating any validation studies. 

For questions regarding this process, please contact the NAHLN Program Office at NAHLN@usda.gov. 

For information on test panel availably contact NVSL.info@usda.gov 
Requirements for NAHLN Deployment of New Diagnostic Assays

FINAL DOCUMENT 06/21/2023

3

Appendix A: Dossier Requirements for Feasibility Studies

More detail on specific experimental design parameters for each of the required data sets is provided in 

this Appendix. For feasibility studies only, sample panels may contain biological samples (live or 

inactivated) or spiked samples containing either biological or synthetic analytes.

Intended Purpose of Assay

- Must provide intended purpose for the assay.

o This includes a qualitative and quantitative assessment of the ability of a positive or 

negative test result to accurately predict the infection or exposure status of the animal 

or population of animals.

o Examples of intended purpose(s):

- Demonstration of freedom from infection in a defined population 

(country/zone/herd).

- Confirm freedom from of infection for trade/movement purposes.

- Confirm diagnosis of infected animals.

- Estimate prevalence of infection for risk analysis.

- Determine the immune status of an animal or animal populations postvaccination.

o Must identify target animal species, target diseases, and sample types/sample matrices.

- Data for the feasibility testing must be provided comparing the new assay to the equivalent 

procedure currently deployed to the NAHLN.

Side-by-side comparison to NAHLN SOP

- Feasibility data must include a side-by-side comparison of the current NAHLN protocol using the 

same sample panels. The following data must be provided:

Operating range of the assay

- Use of a high positive reference sample that has been serially diluted to extinction, based on the 

assay’s response curve. Sample(s) must be diluted in an analyte-negative matrix of the same 

constitution as the sample matrix and be from animals targeted by the assay. The results should 

be plotted as a ‘response-curve’, with the response (e.g., optical density, cycle threshold, 

counts, etc.) as a function of analyte concentration (amount). Spiked samples are permitted for 

this study. 

Specific feasibility studies for Ab based assays:

- Test panel must be comprised of at least 30 samples spanning the operating range of the assay.

o Must include one or more reference standards if required for data normalization.

o Must demonstrate adequate separation of results between negative, low positive and 

high positive samples. 

- The lower OD range should be 0.1 or less for negative controls in an indirect 

ELISA,

- OD values should not exceed 2.0 for strong positive controls in 

competitive/blocking ELISAs to avoid inaccuracy in plate readers at this range. 

- Test panel must contain at least two contemporary field strains and closely related strains to 

test for cross-reactivity. 
Requirements for NAHLN Deployment of New Diagnostic Assays

FINAL DOCUMENT 06/21/2023

4

Assay Standardization 

- Test panel must contain high pos, low pos and negative reference samples. 

o If more than one sample matrix will be approved for use with the assay, standardization 

data for each sample matrix must be provided. 

o In general, it is recommended that at least 30 samples are tested, and include a range 

high pos, low pos, and negative sample types. 

- Reference samples must represent both known infected and uninfected 

animals.

- Reference samples must be well characterized by at least one, and preferably 

two, alternate methodologies. 

o The alternative of preparing reference samples spiked with cultured 

agents or positive sera is inferior, as these samples do not truly 

represent the naturally occurring matrix-agent interaction.

o When no other alternative exists, spiking a sample with a known 

amount of the analyte or agent derived from culture, or diluting a high 

positive serum in negative serum of the same species may be all that is 

available.

o The matrix into which the analyte is placed or diluted, must be

identical to or resemble as closely as possible the samples that 

ultimately will be tested in the assay.

Specific feasibility studies for nucleic acid-based assays:

- Test results must include a comparison of at least one, and preferably all, commercial extraction 

and thermal cycler platforms approved for use in the corresponding NAHLN SOP (e.g., ABI 7500, 

QuantStudio5)

- Test results must include a comparison of all commercial master mixes approved for use in the 

corresponding NAHLN SOP, and include the following:

o Information on assay components (primer/probe sequences, probe reporter and 

quencher dyes, master mix composition and internal controls) should be provided. 

o For multiplex real-time PCR assays, reporter dyes should be chosen for compatibility 

with common commercial real-time PCR instruments. 

o For all real-time PCR assays, threshold determination methods and positive/negative 

cutoff values should be provided. 

Optimization

Matrix inhibition studies

- Data must be provided for each different matrix type that will be used for the assay. 

Robustness

- Data must be provided for at least two lots of reagents, and preferably tested by two different 

NAHLN laboratories. 

- Quality control data must be provided for at least two lots of reagents used during the feasibility 

studies. Reagents used for quality control must be fully described. (Are these commercially 

available? Will these be provided to external customers either in the kit or as a separate 

reagent?)

o Specific QC controls for nucleic acid-based tests:
Requirements for NAHLN Deployment of New Diagnostic Assays

FINAL DOCUMENT 06/21/2023

5

- No-template control (NTC) – approximately 5% of wells should be NTC and

distributed randomly within the assay or over the plate when 96 or 386 well 

plate formats are used.

- Analyte-positive control –must be tested with each plate (at least one well) and 

Ct activity must be within the defined operating range of the assay.

- Inhibition control or extraction control – must be tested with each plate.

- For RNA assays – reverse transcription control must be tested with each plate.

Specific feasibility studies for Ag based assays:

- Results must be able to distinguish between samples with and without the analyte.

- Data must show minimal cross-reactivity when non-target antigen is present in the sample or 

sample matrix.

- Data must assess repeatability for a range of control samples over several days.

o Recommend running all critical reagents against each other in a checkerboard titration.

 
Requirements for NAHLN Deployment of New Diagnostic Assays

FINAL DOCUMENT 06/21/2023

6

Appendix B: Dossier Requirements for Final Assay Approval 

Once data has been submitted for the feasibility studies and has been approved by the NAHLN, the 

following data will then be requested for final review and approval prior to deployment to the NAHLN. 

Analytical Performance 

- Ideally, the assistance of a statistician and disease expert should be engaged to ensure sample 

size and experimental approach are valid. 

Repeatability testing

- A minimum of three samples and preferably five samples should be tested, with samples 

representing the analyte activity within the full operating range of the assay. 

o All replicates must be tested through all steps of the assay, including any extraction 

procedures and all working dilutions. It is not acceptable to prepare a final working dilution 

of a sample in a single tube from which diluted aliquots are pipetted into reaction vessels, or 

to create replicates from one extraction of nucleic acid rather than to extract each replicate 

before dilution into the reaction vessels. 

- Specific repeatability testing requirements For Ab based assays:

o A reference panel containing no less than 3 (preferably 5) samples covering the operating 

range of the assay, and of sufficient quantity for at least 20 runs of the assay over several 

days.

- At least one reference sample should be included in an indirect ELISA (pos serum 

control) to normalize test samples against.

- CV should not exceed 15% for strong positive samples. 

- Specific repeatability testing requirements for nucleic acid-based assays, 

o A reference panel containing a minimum of three and preferably five samples covering the 

operating range of the assay must be tested using the entire assay procedure, including 

nucleic acid extraction.

o Intra-assay data must be provided that has a minimum of five replicates of each sample in 

one assay run.

o Inter-assay data must be determined by testing samples over several days, using multiple 

operators and at least 20 runs. 

Analytical Specificity (ASp) 

Evaluation should reflect the test purpose and assay type. 

- Specific analytical specificity parameters for antibody detection assays:

o Sera from animals infected with related species should be used to evaluate analytical 

specificity. 

- Specific analytical specificity parameters for antigen-based assays:

o ASp should be determined by testing well characterized samples from similar or related

pathogens, which produce similar lesions as the target pathogen or are frequently found in 

samples containing the target pathogen. For example, to assess the ASp of a FMD antigen 

detection ELISA for one particular serotype (e.g., serotype O), it is necessary to assess its 
Requirements for NAHLN Deployment of New Diagnostic Assays

FINAL DOCUMENT 06/21/2023

7

reactivity of all sub-strains within this serotype (e.g., O Campos, O Manisa, etc.) to assess 

inclusivity. At the same time, it is important to show that the test does not cross-react with 

other serotypes such as A, Asia 1, C, SAT 1, 2 and 3. Finally, there is also a need to assess 

whether the test cross-reacts with agents from diseases which may produce similar signs, 

e.g., vesicular stomatitis, swine vesicular disease and swine vesicular exanthema.

- Specific analytical specificity parameters for nucleic acid-based assays:

o Data demonstrating the discriminatory power of the assay must be provided by testing 

pathogens that cause similar clinical syndromes which are likely to be found in the target 

sample.

o Example: if the assay is designed to detect avian influenza A virus, pathogens such 

as Newcastle disease, infectious bursal disease, etc. should be tested to 

demonstrate analytical specificity. 

o Data will also be required for pathogens with known variants circulating in animal 

populations, such as HPAI and FMD. In these cases, the assay must be evaluated against as 

many well-characterized isolates of the virus as available to ensure all strains from a variety 

of geographical areas and hosts are detected. 

o How the assay will be used and in which geographical regions (example: North 

America) will determine whether it is necessary to evaluate some or all available 

lineages

Analytical Sensitivity (ASe)

- Limit of Detection (LOD)

This is the estimated amount of analyte in a specified matrix that would produce a positive 

result at least a specified percent of the time.

- Specific LOD parameters for Ab based assays: 

o Data must be provided demonstrating the LOD using an endpoint dilution in which 10 

replicates of each dilution in a log2 dilution series are run in the assay. 

- Specific LOD parameters for Ag based assays: 

o Data must be provided demonstrating the LOD using an endpoint dilution in which 10 

replicates of each dilution in a log2 dilution series are run in the assay. 

o Screening assays or assays designed to detect sub-clinical infections should have a very high 

ASe. 

o If available, serial samples from experimentally infected animals could provide temporal 

information about the assay’s capacity to detect antigen over the course of the infection. 

- Specific LOD parameters for nucleic acid-based assays:

o Data is required showing a dilution series of the target pathogen diluted in all the different 

sample matrices. Dilution series should be to extinction – i.e., not detected by the assay in 

any of the replicate samples at that dilution. 

o Samples must be tested by both the assay and by another standard method of detection.
Requirements for NAHLN Deployment of New Diagnostic Assays

FINAL DOCUMENT 06/21/2023

8

Diagnostic Assay Performance

Diagnostic Sensitivity and Specificity (DSe and DSp)

- Ideally, samples used for determining diagnostic assay performance are derived from testing a 

panel of samples from reference animals, of known history and infection status relative to the 

disease/infection in question and relevant to the country or region in which the test is to be 

used.

- An estimate of the area under the receiver operating characteristic (ROC) curve is a useful

adjunct to DSe and DSp estimates for a quantitative diagnostic test because it assesses its global 

accuracy across all possible assay values.

- The designated number of known positive and known negative samples will depend on the likely 

values of DSe and DSp of the candidate assay and the desired confidence level for the estimates. 

- The table below provides two panels of the theoretical number of samples required, when 

either a 5% or 2% error is allowed in the estimates of DSe or DSp.

Theoretical number of samples from animals of known infection status required for establishing diagnostic 

sensitivity (DSe) and specificity (DSp) estimates depending on likely value of DSe or DSp and desired error margin 

and confidence. 

2% error allowed in 

estimate of DSe and DSp

5% error allowed in 

estimate of DSe and DSp

Confidence Confidence

Estimated 

DSe or DSp 90% 95% 99% 90% 95% 99%

90% 601 864 1493 98 138 239

92% 466 707 1221 75 113 195

94% 382 542 935 61 87 150

95% 372 456 788 60 73 126

96% 260 369 637 42 59 102

97% 197 279 483 32 45 77

98% 133 188 325 21 30 52

99% 67 95 164 11 15 26

- Use of experimentally infected or vaccinated reference animals

o Multiple serially acquired pre- and post-exposure results from individual animals are not 

acceptable for establishing estimates of DSe and DSp because the statistical 

requirement of independent observations is violated.

o A single time-point sampling of individual experimental animals can be acceptable (e.g., 

one sample randomly chosen from each animal). 

o In cases where the near impossibility of obtaining suitable reference samples from 

naturally exposed animals necessitates the use of samples from experimental animals, 

the resulting DSe and DSp measures should be considered as less than ideal estimates of 

the true DSe and DSp. 

Diagnostic Reproducibility 

This is a measure of the agreement between results obtained in different NAHLN laboratories using the 

same protocol, similar equipment, and the same panel of samples. 
Requirements for NAHLN Deployment of New Diagnostic Assays

FINAL DOCUMENT 06/21/2023

9

- For all assays:

o A panel consisting of 20-30 samples, including at least three that are present in 

quadruplicate, should be tested.

- The panel should consist of samples covering the range of the test, with several 

that are very close to the test positive/negative cut-off values. 

o The same panel used for repeatability testing can be used for reproducibility, but with 

enhanced numbers of replicates. 

References:

JACOBSON R.H. (1998). Validation of serological assays for diagnosis of infectious diseases. Rev. sci. tech. Off. int.

Epiz., 17, 469–486.

WOAH– Manual of Diagnostic Tests and Vaccines for Terrestrial Animals 2022. Ch. 1.1.6. Principles and Methods of 

Validation of Diagnostic Assays for Infectious diseases. Accessed online 2/1/2023.https://www.woah.org/en/what-wedo/standards/codes-and-manuals/terrestrial-manual-online-access/

WOAH- Manual of Diagnostic Tests and Vaccines for Terrestrial Animals 2022. Ch. 2.2.1. Development and 

optimisation of antibody detection assays. Accessed online 2/1/2023.https://www.woah.org/en/what-wedo/standards/codes-and-manuals/terrestrial-manual-online-access/

WOAH- Manual of Diagnostic Tests and Vaccines for Terrestrial Animals 2022. Ch. 2.2.2. Development and 

optimisation of antigen detection assays. Accessed online 2/1/2023.https://www.woah.org/en/what-wedo/standards/codes-and-manuals/terrestrial-manual-online-access/

WOAH- Manual of Diagnostic Tests and Vaccines for Terrestrial Animals 2022. Ch. 2.2.3. Development and 

optimisation of nucleic acid detection assays. Accessed online 2/1/2023.https://www.woah.org/en/what-wedo/standards/codes-and-manuals/terrestrial-manual-online-access/