Chapter 2.2.4. – Measurement uncertainty

Chapter 1.1.5. – Quality management in veterinary testing laboratories

Chapter 1.1.5.

quality management in veterinary testing laboratories

SUMMARY

Valid laboratory results are essential for diagnosis, surveillance, and trade. Such results are ~~achieved by the use~~assured through implementation of ~~good~~a management ~~practices, valid~~system that supports accurate and consistent test and calibration methods~~, proper techniques, quality control and quality assurance, all working together within a quality management system~~. Laboratory quality management includes technical, managerial, and operational elements of ~~testing~~performing, interpreting and ~~the interpretation of~~reporting test results. A quality management system enables the laboratory to demonstrate both competency and an ability to generate consistent technically valid results that meet the needs of its customers. ~~The need for~~Mutual recognition and acceptance of test results for international trade, and the ~~acceptance~~accreditation of tests to international standards such as ISO/IEC[footnoteRef:2] 17025~~:2005~~(General Requirements for the Competence of Testing and Calibration Laboratories) (ISO/IEC, ~~2005~~2017b) requires ~~good~~suitable laboratory quality management systems. This chapter is not intended to reiterate the requirements of ISO/IEC 17025, nor has it been endorsed by accreditation bodies. Rather, it outlines the important issues and considerations a laboratory should address in the design and maintenance of its quality management system, ~~whether or not it has been formally accredited~~regardless of formal accreditation status. Chapter 1.1.1 Management of ~~veterinary diagnostic laboratories gives an introduction to~~veterinary diagnostic laboratories introduces the components of governance and management of veterinary laboratories that are necessary for the effective delivery of diagnostic services, and highlights the critical elements that should be established as minimum requirements. [2: 	ISO/IEC: International Organization for Standardization/International Electrochemical Commission.] 

a.  KEY CONSIDERATIONS FOR THE DESIGN AND MAINTENANCE OF A LABORATORY QUALITY MANAGEMENT SYSTEM

To ensure that the quality management system is appropriate and effective, the design must be carefully ~~thought out~~planned and, where accreditation is sought, must address all criteria of the appropriate quality standard. The major ~~categories of~~considerations and ~~the~~their associated key issues and activities~~within each of these categories~~ are outlined in the following eight sections of this chapter.

1.	The work, responsibilities, and goals of the laboratory

Many factors affect the necessary elements and requirements of a quality management system~~. These factors include~~, including:

i)	Type of testing ~~done~~performed, e.g. research versus diagnostic work;

ii)	Purpose and requirements of ~~the~~test results, e.g. ~~for~~import~~or~~/export quarantine testing, surveillance, emergency disease exclusion, declaration of freedom from disease post-outbreak;

iii)	Potential impact of a questionable~~or~~, erroneous or unfavourable result, e.g. detection of foot and mouth disease (FMD) in an FMD-free country;

iv)	~~The tolerance level of~~Risk and liability tolerance, e.g. vaccination ~~vs~~versus culling~~or~~/slaughter;

v)	Customer ~~needs (~~requirements, e.g. sensitivity and specificity~~of the test method~~, cost, turnaround time, ~~strain or genotype~~level of characterisation~~), e.g. for surveillance, or declaration of freedom after outbreak~~;

vi)	~~The role of the laboratory~~Role in legal work or in regulatory programmes, e.g. for disease eradication and declaration of disease freedom to the WOAH;

vii)	~~The role of the laboratory~~Role in assisting with, confirming, or overseeing the work of other laboratories (e.g. as a reference laboratory);

viii)	Business goals~~of the laboratory~~, including the need for any third-party recognition or accreditation.

2.	Standards, guides, and references

The laboratory should ~~choose reputable and accepted~~follow globally recognised standards ~~and guides~~to assist in designing the quality management system. For laboratories seeking ~~accreditation~~formal recognition of testing competency, and for all WOAH Reference Laboratories, the use of ISO/IEC 17025 (ISO/IEC, ~~2005~~2017b) or equivalent ~~will be~~is essential. This standard ~~includes~~specifies managerial and technical requirements and accredited laboratories~~that are compliant~~ are regarded as competent. Further information on standards may be obtained from the national standards body of each country, from the International Laboratory Accreditation Cooperation (ILAC)[footnoteRef:3], and from accreditation bodies, ~~e.g. the National Association of Testing Authorities (NATA), Australia, the United Kingdom Accreditation Service (UKAS), the American Association for Laboratory Accreditation (A2LA), etc~~. Technical and international organisations such as AOAC International (The Scientific Association Dedicated to Analytical Excellence; formerly the Association of Official Analytical Chemists) and the International Organization for Standardization (ISO) publish useful references, guides, application documents and standards that supplement the general requirements of ISO/IEC 17025. Other relevant documents may include guide https://nata.com.au/files/2021/05/Animal-Health-ISO-IEC-17025-Appendix-effective-March2021.pdf; Newberry & Colling, 2021. [3:  	ILAC: The ILAC Secretariat, PO Box 7507, Silverwater, NSW 2128, Australia; http://ilac.org/ ] 

The ISO International Standard 9001 (ISO, 2015)~~, is a certification standard~~specifies the requirements for quality management systems and while it may be a useful ~~supplement~~framework to ~~a~~underpin a laboratory quality system, fulfilment of its requirements does not ~~necessarily ensure or imply~~assure technical competence (in the areas listed in Section 3 *Accreditation*). Conformance to the requirements of ISO 9001 is assessed by a certification body that is accredited ~~to undertake such assessments~~by the national accreditation body to undertake such assessments. When a laboratory meets the requirements of ISO 9001*,*the term *registration* or *certification* is used to indicate conformity, not *accreditation*. 

With the advent of stronger alliances between medical and veterinary diagnostic testing under initiatives such as “One Health”, some laboratories may ~~wish to~~choose to follow other ISO standards such as ISO 15189 Medical Laboratories – Requirements for Quality and Competence (ISO/IEC, ~~2012), which include~~2022), for testing of human samples, e.g. for zoonotic diseases. ~~It should be noted that for veterinary laboratories, limited availability of suitable material may render validation difficult; under these circumstances it is necessary to highlight the limited validation status when reporting results and their interpretation (Stevenson *et al.,* 2021).~~

3.	Accreditation

If ~~the laboratory decides to proceed with~~formal recognition of ~~its~~a laboratory’s quality management system and testing~~, then~~is sought, third party verification of its conformity with the selected standard(s) ~~will be~~is necessary. ILAC has published specific requirements and guides for laboratories and accreditation bodies. Under the ILAC system, ISO/IEC 17025 is to be used for laboratory accreditation of testing or calibration activities. Definitions regarding laboratory accreditation may be found in ISO/IEC International Standard 17000: Conformity Assessment – Vocabulary and General Principles (ISO/IEC, ~~2004a~~2020). Accreditation is ~~tied to~~dependent on demonstrated competence, which ~~is~~encompasses significantly more than having and following documented procedures. Providing a competent and customer-oriented service also ~~means that the laboratory~~requires:

i)	Adequate facilities and environmental controls;

ii)	~~Has~~Appropriately qualified and trained personnel with a depth of technical knowledge commensurate with appropriate level of authority;

iii)	~~Has appropriate~~Equipment ~~with planned~~that is appropriately verified and managed in accordance with the relevant maintenance and calibration schedule;

~~iv)	Has adequate facilities and environmental control;~~

~~v)	Has procedures and specifications that ensure accurate and reliable results;~~

~~vi)	Implements continual improvements in testing and quality management;~~

~~vii)	Can assess the need for and implement appropriate corrective or preventive actions, e.g. customer satisfaction;~~

~~viii)	Accurately assesses and controls uncertainty in testing;~~

iv)	Appropriate sample and materials management processes;

v)~~Has~~Technically valid and validated test methods, procedures and specifications~~that are~~, documented in accordance with the requirements of the applicable standard or guidelines, e.g. Chapter 1.1.6 *Principles and methods of validation of diagnostic assays for infectious diseases*~~and~~*,*chapters 2.2.1 to 2.2.8 *Recommendations for validation of diagnostic tests*and Special Issue of the *Scientific and Technical Review*(2021)[footnoteRef:4]; [4:  	Available at: https://doc.woah.org/dyn/portal/index.xhtml?page=alo&aloId=41245] 

vi)	~~Demonstrates~~Demonstrable proficiency in the applicable test methods ~~used~~(e.g. by regular participation in proficiency ~~tests on a regular basis~~testing schemes);

vii)	Accurate assessment and control of the measurement of uncertainty in testing;

viii)	Good documentation practices, e.g. ALCOA+ principles (i.e. Attributable, Legible, Contemporaneous, Original, Accurate, Complete, Consistent, Enduring, Available);

ix)	Non-conformance management process, including detection, reporting, risk-assessment and implementation of effective corrective and preventive actions;

x)	Complaints management;

xi)	Adequate control of data and information; 

xii)	Appropriate reporting and approval process;

xiii)	Culture of continual improvement.

xiv)	Has demonstrable competence to generate technically valid results.

4.	Selection of an accreditation body

To facilitate the acceptance of the laboratory’s test results for trade, the accreditation standard used must be recognised by the international community and the accreditation body recognised as competent to accredit laboratories. Programmes for the recognition of accreditation bodies are, in the ILAC scheme, based on the requirements of ISO/IEC International Standard 17011: Conformity Assessment – General Requirements for Accreditation Bodies Accrediting Conformity Assessment Bodies (ISO/IEC, ~~2004b~~2017a). Information on recognised accreditation bodies may be obtained from the organisations that recognise them, such as the Asia-Pacific Accreditation Cooperation (APAC), the Inter-American Accreditation Cooperation (IAAC), and the European Co-operation for Accreditation (EA).

Accreditation bodies may also be signatory to the ILAC and regional (e.g. APAC) mutual recognition arrangements (MRAs). These MRAs are designed to reduce technical barriers to trade and further facilitate the acceptance of a laboratory’s test results in foreign markets. Further information on the ILAC MRA may be obtained from the http://www.ilac.org.

5.	Determination of the scope of the quality management system or of the laboratory’s accreditation

The~~~~scope of the quality management system should ~~cover all areas of activity affecting all~~include all activities that impact testing ~~that is done at~~performed by the laboratory. Whilst only accredited laboratories are obliged to meet ~~the~~requirements of the relevant standard~~as detailed below, these~~, the guiding principles should be considered best practise and are relevant to all testing laboratories.

~~Laboratories accredited~~A laboratory’s accreditation to ISO/IEC 17025 ~~have~~includes a ~~specific~~list of ~~those~~accredited tests~~that are accredited, called~~, referred to as the schedule or scope of accreditation~~or the scope. Veterinary testing facilities include government and private facilities, veterinary practices, university veterinary schools, and other laboratories for the testing of animals and animal products for the diagnosis, monitoring and treatment of disease~~. In principle, if new testing methods are introduced these must be assessed and accredited before they can be added to the scope, however a flexible scope can be implemented that assesses the laboratory as competent to add tests to scope, which are then formally added at the next accreditation visit. ~~The quality management system should ideally cover all areas of activity affecting all testing that is done at the laboratory. However, it is up to the laboratory to decide which tests are to be accredited and included in the scope.~~If an accredited laboratory also offers ~~unaccredited~~non-accredited tests, these must be clearly indicated as such on any reports that claim or ~~make~~reference ~~to~~accreditation. ~~Factors~~It is ultimately the decision of the laboratory to decide which tests require inclusion in the scope of accreditation, and factors that might affect ~~the laboratory’s choice of tests for scope of accreditation~~this decision include:

~~i)	The impact of initial accreditation on resources within a given deadline;~~

i)	Associated risks and opportunities; 

ii)	Initial investment required (e.g. time, resources);

iii)	~~A~~Contractual requirement for accredited testing (e.g. for international trade, research projects);

iv)	~~The~~Importance of the test and the potential impact of an incorrect result;

v)	The cost of maintaining an accredited test versus frequency of use;

vi)	Availability of personnel, facilities and equipment;

vii)	Availability of appropriate materials and reference standards (e.g. ~~standardised~~reagents, ~~internal quality control samples~~controls, reference cultures)~~and~~

viii)	Access to proficiency testing schemes;

ix)	The quality ~~assurance~~control processes necessary for materials, reagents and media;

x)	The validation status, e.g. access to field samples from infected and non-infected animals, technical complexity and reliability of the test method;

xi)	~~The~~Potential for subcontracting of accredited tests.

6.	Quality assurance, quality control and proficiency testing

Quality assurance (QA) is the ~~part~~element of quality management focused on providing confidence that ~~quality~~defined requirements ~~will be~~are fulfilled. The requirements may be internal or defined in an accreditation or certification standard. QA is process-oriented and ~~ensures~~provides the ~~right things are being done in the right way~~appropriate inputs to prevent problems arising.

Quality control (QC) is the systematic and planned monitoring of outputs to ensure ~~the~~minimum ~~levels of quality~~requirements have been met. For a testing laboratory, this ~~is to ensure test processes~~ensures tests are ~~working correctly~~performing consistently and reliably, and results are within ~~the expected~~acceptable parameters and limits. QC is ~~test orientated and ensures the~~results~~are as expected~~-oriented and ensures detection of any problems that arise.

Proficiency testing (PT), sometimes referred to as external quality assurance ~~or~~(EQA), is the ~~determination~~assessment of a laboratory’s performance ~~by~~when testing a standardised panel of specimens of undisclosed content. Ideally, PT schemes should be ~~run~~managed by an external independent provider. Participation in proficiency testing schemes enables the laboratory to assess and demonstrate ~~the~~their testing reliability ~~of results by~~in comparison with ~~those from~~other participating laboratories.

All laboratories should, where possible, participate in external proficiency testing schemes appropriate to ~~their testing. Participation~~the suite of tests provided; participation in such schemes is a requirement for accredited laboratories. This provides an independent assessment of the testing methods used ~~and~~as well as the level of staff competence. If such schemes are not available, valid alternatives may be used, such as ring trials organised by reference laboratories, inter-laboratory testing, use of certified reference materials or internal quality control samples, replicate testing using the same or different methods, retesting of retained items, ~~and~~or correlation of results for different characteristics of a specimen. 

Providers and operators of proficiency testing programmes should be accredited to ISO/IEC 17043 – Conformity Assessment – General Requirements for Proficiency Testing (ISO/IEC, 2010).

Proficiency testing material from accredited providers ~~has been~~is well characterised and any spare material, once the proficiency testing has been completed, can be useful to demonstrate staff competence or for test validation. Information about selection and use of reference samples and panels is available in Chapter 2.2.6 *Selection and use of reference samples and panels*. Proficiency testing and reproducibility scenarios are described by Johnson & Cabuang (2021) and Waugh & Clark (2021), respectively.

7.	Test methods

ISO/IEC 17025 requires the use of appropriate test methods and has requirements for their selection, development, and validation to ~~show~~demonstrate fitness for purpose.

This *Terrestrial Manual*provides recommendations on the selection of test methods for trade, diagnostic and surveillance purposes in the chapters on specific diseases. Disease-specific chapters include, or will include in the near future, a table of the tests available for the disease, graded against the test’s fitness for purpose; these purposes are defined in the WOAH Validation Template (chapter 1.1.6)*,* which identifies six main purposes for which diagnostic tests may be carried out. The table is intended ~~to be~~as a general guide to test application~~.~~; the fact that a test is recommended does not necessarily mean that a laboratory is competent to perform it. The laboratory quality system should incorporate provision of evidence of competency.

In ~~the~~veterinary ~~profession~~laboratories, other standard methods (published in international, regional, or national standards) or fully validated methods (having undergone a full collaborative study and that are published or issued by an authoritative technical body such as the AOAC International) may be preferable to use, but ~~may~~not ~~be~~available. Many veterinary laboratories develop or modify methods, and most laboratories have test systems that use non-standard methods, or a combination of standard and non-standard methods. In veterinary laboratories, even with the use of standard methods, some in-house evaluation, optimisation, or validation is generally ~~must be done~~required to ensure valid results. 

Customers and laboratory staff must have a clear understanding of the performance characteristics of the test, and customers should be informed if the method is non-standard. Many veterinary testing laboratories will therefore need to demonstrate competence in the development, adaptation, verification and validation of test methods.

This *Terrestrial Manual* provides more detailed and specific guidance on test selection, optimisation, standardisation, and validation in chapter 1.1.6. ~~Chapter 1.1.6 refers to~~chapters 2.2.1–2.2.8~~*Recommendations for validation of diagnostic tests* that~~ deal with the development and optimisation of fundamentally different assays such as antibody, antigen and nucleic acid detection~~s~~ tests, measurement uncertainty, statistical approaches to test validation, selection and use of reference samples and panels, validation of diagnostic tests for wildlife, and comparability experiments after changes in a validated test method. 

The following are key test method issues for those involved in the quality management of the laboratory.

7.1.	Selection of the test method

Valid results begin with the selection of a test method that meets the needs of the laboratory’s customers in addressing their specific requirements (fitness for purpose). Some issues relate directly to the laboratory, others to the customer. 

7.1.1.	Considerations for the selection of a test method

i)	International acceptance;

ii)	Scientific acceptance;

iii)	Appropriate or current technology;

iv)	Suitable performance characteristics (e.g. analytical and diagnostic sensitivity and specificity, repeatability, reproducibility, isolation rate, limits of detection, precision, trueness, and uncertainty);

v)	Suitability of the test in the species and population of interest;

vi)	Sample type (e.g. serum, tissue, milk) and its expected quality or state on arrival at the laboratory;

vii)	Test target (e.g. antibody, antigen, live pathogen, nucleic acid sequence);

viii)	Test turnaround time;

ix)	Resources and time available for development, adaptation, evaluation;

x)	Intended use (e.g. export, import, surveillance, screening, diagnostic, confirmatory);

xi)	Safety ~~factors~~and biocontainment requirements;

xii)	Customer expectations;

xiii)	~~Throughput of test~~Sample numbers and required throughput (automation, robot);

xiv)	Cost of test, per sample;

xv)	Availability of reference standards, reference materials and proficiency testing schemes. (See also chapter 2.2.6.).

7.2.	Optimisation and standardisation of the test method

Once the method has been selected, it must be set up at the laboratory. Additional optimisation is necessary, whether the method was developed in-house (validation) or imported from an outside source (verification). Optimisation establishes critical specifications and performance standards for the test process as used in a specific laboratory. 

7.2.1.	Determinants of optimisation 

i)	Critical specifications for equipment, ~~instruments~~consumables, ~~and~~reagents (e.g. chemicals, biologicals), reference standards, reference materials, and internal controls;

ii)	Robustness – critical control points and acceptable ranges, attributes or behaviour at critical control points, using statistically acceptable procedures;

iii)	Quality control activities necessary to monitor critical control points;

iv)	The type, number, range, frequency, and arrangement of test run controls;

v)	Criteria for ~~non-subjective~~objective acceptance or rejection of ~~a batch of~~test results;

vi)	Criteria for ~~the~~interpretation and reporting of test results;

vii)	~~A~~Documented test method and reporting procedure~~for use by laboratory staff~~; 

viii)	Evidence of technical competence for those ~~who~~performing the test ~~processes~~methods, authorising test results and interpreting results.

7.3.	Validation of the test method 

Test method validation evaluates the test for ~~its~~fitness for ~~a given use~~purpose by establishing ~~test~~performance characteristics such as sensitivity, specificity, and isolation rate; and diagnostic parameters such as positive or negative cut-off, repeatability, reproducibility and titre of interest or significance. Validation should be ~~done~~performed using an optimised, documented, and fixed procedure. The extent and depth of the validation process will depend on logistical and risk factors~~. It~~ and may involve any number of activities and amount of data, with subsequent data analysis using appropriate statistical methods (Chapter 1.1.6.). Acknowledging diagnostic test validation science as a key element in the effective detection of infectious diseases, WOAH recently published a Special Issue representing an up-to-date compilation of the relevant standards (WOAH and non-WOAH) and guidance documents for all stages of diagnostic test validation and proficiency testing, including design and analysis, as well as clear, complete and transparent reporting of validation studies in the peer-reviewed literature (Colling & Gardner, 2021). It is important to note that the current version of ISO 17025:2017 specifies that personnel must be authorised to perform validation and related activities, which means that training in validation and verification methods, including results interpretation, is likely to become more important to prove competence (Colling & Gardner, 2021). It should also be noted that for veterinary laboratories, limited availability of suitable material may render validation difficult; under these circumstances it is necessary to highlight the limited validation status when reporting results and their interpretation (Stevenson *et al.,* 2021).

7.3.1.	Activities that validation might include

~~i)	Field or epidemiological studies, including disease outbreak investigations and testing of samples from infected and non-infected animals;~~

~~ii)	Development of testing algorithms for specific purposes, e.g. surveillance, outbreak investigations, etc.;~~

~~ii~~i)	Repeat testing in the same laboratory to establish the effect of variables such as operator, reagents, equipment;

ii~~v~~)	Comparison with other, preferably standard methods and with reference standards (if available);

iii~~v~~)	Collaborative studies with other laboratories using the same documented method. Ideally organised by a reference laboratory and including testing a panel of samples of undisclosed composition or titre with expert evaluation of results and feedback to ~~the~~participants to estimate reproducibility;

iv~~i~~)	Reproduction of data from an accepted standard method, or from a ~~reputable~~peer-reviewed publication (verification);

v~~ii~~)	Experimental infection ~~or disease outbreak~~studies;

vi~~ii~~)	Analysis of internal quality control data.

vii)	Field or epidemiological studies, including disease outbreak investigations and testing of samples from infected and non-infected animals;

viii)	Development of testing algorithms for specific purposes, e.g. surveillance, outbreak investigations, etc.;

Validation is always a balance between cost, risk, and technical possibilities. There may be cases where ~~quantities such as~~only basic accuracy and precision can ~~only~~be ~~given~~determined, e.g. when the disease is not present in a ~~simplified way~~country or region. Criteria and procedures for the correlation of test results for diagnosis of disease status or for regulatory action must be developed. The criteria and procedures developed should account for screening methods, retesting and confirmatory testing.

~~Test validation is covered in chapter 1.1.6.~~

~~7.4.	Uncertainty of the test method~~

Statistically relevant numbers of samples from infected and non-infected animals are discussed in chapter 1.1.6. test validation and chapter 2.2.5 statistical approaches to validation. 

7.4.	Estimation of Measurement Uncertainty

Measurement ~~of~~Uncertainty (MU) is “a parameter associated with the result of a measurement that characterises the dispersion of values that could reasonably be attributed to the measure” (Eurachem, 2012). Uncertainty of measurement does not imply doubt about a result but rather increased confidence in its validity. It is not the equivalent to *error*, as it may be applied to all test results derived from a particular procedure.

Laboratories must estimate the MU for each test method resulting in a quantitative measurement~~included in their scope of accreditation~~, and for any methods used to calibrate equipment, included in their scope of accreditation (ISO/IEC 17025, ~~2005~~2017b).

Tests can be broadly divided into two groups: quantitative (e.g. biochemical assays, enzyme-linked immunosorbent assays [ELISA], titrations, real-time polymerase chain reaction [PCR], pathogen enumeration, etc.); and qualitative (bacterial culture, parasite identification, virus isolation, endpoint PCR, immunofluorescence, etc.).

The determination of MU is well established in *quantitative* measurement sciences (ANSI, 1997). It may be given as a numeric expression of reliability and is commonly shown as a stated range. Standard deviation (SD) and ~~confidence~~reference interval (~~C~~RI) are examples of the expression of MU, for example the optical density result of an ELISA expressed as ± *n*SD, where *n* is usually 1, 2 or 3. The confidence interval (usually 95%) gives an estimated range in which the result is likely to fall, calculated from a given set of test data. For quantitative measurements, example for a top-down or control-sample approach are provided for an antibody ELISA in chapter 2.2.4, and by the Australian government webpage[footnoteRef:5]. An example for a quantitative PCR hydrolysis probe~~(TaqMan)~~assay is provided by Newberry & Colling (2021). [5:  	Australian Government, Department of Agriculture, Fisheries and Forestry. Worked examples of measurement uncertainty. Measurement uncertainty in veterinary diagnostic testing – DAFF (agriculture.gov.au) (accessed 15 March 2023).] 

The ISO/IEC 17025 requirement for “quality control procedures for monitoring the validity of tests” implies that the laboratory must use quality control procedures that cover all major sources of uncertainty. There is no requirement to cover each component separately. Laboratories may establish acceptable specifications, criteria, ranges, etc., at critical control points for each component of the test process. The laboratory can then implement appropriate quality control measures at these critical points, or seek to reduce or eliminate the uncertainty effect of each component.

7.4.1.	Potential sources of uncertainty include:

i)	Sampling;

ii)	Contamination;

iii)	Sample transport and storage conditions;

iv)	Sample processing;

v)	Reagent quality, preparation and storage;

vi)	Type of reference material;

vii)	Volumetric and weight manipulations;

viii)	Environmental conditions;

ix)	Equipment effects;

x)	Analyst or operator bias;

xi)	Biological variability;

xii)	Unknown or random effects.

Systematic errors or bias determined by validation must be corrected by changes in the method, adjusted for mathematically, or have the bias noted as part of the report statement.

If an adjustment is made to a test or procedure to reduce uncertainty or correct bias then a new source of uncertainty is introduced (the uncertainty of the correction). This must be assessed as part of the MU estimate. 

The application of the principles of MU to *qualitative* testing is less well defined. The determination and expression of MU has not been standardised for veterinary (or medical, food, or environmental) testing laboratories, but sound guidance exists and as accreditation becomes more important, applications are being developed. The ISO/IEC 17025 standard recognises that some test methods may preclude metrologically and statistically valid calculation of uncertainty of measurement. In such cases the laboratory must attempt to identify and estimate all the components of uncertainty based on knowledge of the performance of the method and making use of previous experience, validation data, internal control results, etc.

Many technical organisations and accreditation bodies (e.g. AOAC International, ISO, NATA, A2LA, Standards Council of Canada, UKAS, Eurachem, the Cooperation of International Traceability in Analytical Chemistry) teach courses or provide guidance on MU for laboratories seeking accreditation. 

~~The ISO/IEC 17025 requirement for “quality control procedures for monitoring the validity of tests” implies that the laboratory must use quality control procedures that cover all major sources of uncertainty. There is no requirement to cover each component separately. Laboratories may establish acceptable specifications, criteria, ranges, etc., at critical control points for each component of the test process. The laboratory can then implement appropriate quality control measures at these critical points, or seek to reduce or eliminate the uncertainty effect of each component. Measurement Uncertainty is covered in chapter 2.2.4.~~

~~7.4.1.	Components of tests with sources of uncertainty include:~~

~~i)	Sampling;~~

~~ii)	Contamination;~~

~~iii)	Sample transport and storage conditions;~~

~~iv)	Sample processing;~~

~~v)	Reagent quality, preparation and storage;~~

~~vi)	Type of reference material;~~

~~vii)	Volumetric and weight manipulations;~~

~~viii)	Environmental conditions;~~

~~ix)	Equipment effects;~~

~~x)	Analyst or operator bias;~~

~~xi)	Biological variability;~~

~~xii)	Unknown or random effects.~~

~~Systematic errors or bias determined by validation must be corrected by changes in the method, adjusted for mathematically, or have the bias noted as part of the report statement.~~

~~If an adjustment is made to a test or procedure to reduce uncertainty or correct bias then a new source of uncertainty is introduced (the uncertainty of the correction). This must be assessed as part of the MU estimate.~~

Additional information on the analysis of uncertainty may be found in the Eurachem Guides to Quantifying Uncertainty in Measurement (Eurachem, 2012) and Use of ~~uncertainty information in compliance assessment~~Uncertainty Information in Compliance Assessment (Eurachem, 2007).

7.5.	Implementation and use of the test method

Training should be a planned and structured activity with steps to ensure adequate supervision is maintained while analysts are being trained. Depending on the complexity of the test and the experience of the analyst, training may include any combination of reading and understanding the documented test method, initial demonstration, performance of the test under supervision and independent performance. Analysts should ~~be able to~~demonstrate proficiency in using the test method prior to ~~producing~~being authorised to produce reported results, and on an ongoing basis.

The laboratory must be able to demonstrate traceability for all accredited tests and the principle should apply to all tests whether accredited or not. This covers all activities relating to test selection, development, optimisation, standardisation, validation, verification, implementation, reporting, personnel, quality control and quality assurance (see also Section 7.3.1. point vi). Traceability is achieved by using appropriate documented project management, record keeping, data management and archiving systems.

8.	Strategic planning

Laboratories should have evidence of continual improvement, which is an obligatory requirement for accredited laboratories. The laboratory must ~~be knowledgeable of and stay~~maintain current ~~with~~knowledge of the relevant quality and technical~~management~~ standards and with methods used to demonstrate laboratory competence and establish and maintain technical validity. Evidence of this may ~~be provided by~~include:

i)	Attendance at conferences, organisation of in-house or external meetings on diagnostics and quality management;

ii)	~~Participation in~~Membership of local and international organisations;

iii)	~~Participation in writing~~Contribution to national and international standards (e.g. on ILAC and ISO committees);

iv)	Maintenance of current awareness ~~of publications, writing~~through review of and ~~reviewing publications about diagnostic methods~~contribution to relevant literature;

v)	Participation in training programmes, including visits to other laboratories;

vi)	Conducting research;

vii)	Participation in cooperative programmes (e.g. Inter-American Institute for Cooperation in Agriculture);

viii)	Exchange of procedures, methods, reagents, samples, personnel, and ideas;

ix)	Planned, continual professional development and technical training;

x)	Management reviews;

xi)	Analysis of customer feedback; 

xii)	Root cause analysis of anomalies and implementation of corrective, preventive and improvement actions, as well as effectiveness reviews.

REFERENCES

ANSI (AMERICAN NATIONAL STANDARDS INSTITUTE) (1997). ANSI/NCS[footnoteRef:6] Z540-2-1997 (R2012), US Guide to the Expression of Uncertainty in Measurement, First Edition. American National Standards Institute, www.ansi.org. [6:  	NCSL: The National Conference of Standards Laboratories.] 

COLLING A.& GARDNER I.A., EDS (2021). Diagnostic test validation science: a key element for effective detection and control of infectious animal diseases. *Rev. Sci. Tech. Off. Int. Epiz.,***40**, Available at https://doc.woah.org/dyn/portal/index.xhtml?page=alo&aloId=41245

EURACHEM (2007). Eurachem/CITAC Guide: Use of Uncertainty Information in Compliance Assessment, First Edition. Eurachem Secretariat. www.eurachem.org.

EURACHEM (2012). Eurachem/CITAC[footnoteRef:7] Guide (CG4): Quantifying Uncertainty in Analytical Measurement, Third Edition. Eurachem Secretariat. www.eurachem.org. [7:  	CITAC: The Cooperation of International Traceability in Analytical Chemistry.] 

ISO (2015). ISO 9001:2015. Quality Management Systems – Requirements. International Organization for Standardization (ISO), www.iso.org.

ISO/IEC (~~2004a~~2020). ISO/IEC 17000:~~2004~~2020. Conformity Assessment – Vocabulary and General Principles. International Organization for Standardization (ISO), www.iso.org.

ISO/IEC (~~2004b~~2017a). ISO/IEC 17011:~~2004~~2 017. Conformity Assessment – General Requirements for accreditation Bodies Accrediting Conformity Assessment Bodies. International Organization for Standardization (ISO), www.iso.org.

ISO/IEC (~~2005~~2017b). ISO/IEC 17025:~~2005~~2017. General Requirements for the Competence of Testing and Calibration Laboratories. International Organization for Standardization (ISO), www.iso.org.

ISO/IEC (2010). ISO/IEC 17043:2010. Conformity Assessment – General Requirements for Proficiency Testing. International Organization for Standardization (ISO), www.iso.org

ISO/IEC (2022~~2012~~). ISO/IEC 15189:~~2012~~2022. Medical Laboratories – Requirements for Quality and Competence. International Organization for Standardization (ISO), www.iso.org

JOHNSON P. & CABUANG L. (2021). Proficiency testing and ring trials. *Rev. Sci. Tech. Off. Int. Epiz*., **40**, 189–203.

NEWBERRY K. & COLLING A. (2021). Quality standards and guidelines for test validation for infectious diseases in veterinary laboratories. *Rev. Sci. Tech. Off. Int. Epiz*., **40**, 227–237. 

STEVENSON M., HALPIN K. & HEUER C. (2021). Emerging and endemic zoonotic diseases: surveillance and diagnostics. *Rev. Sci. Tech. Off. Int. Epiz.*, **40**, 119–129. 

WAUGH C. & CLARK G. (2021). Factors affecting test reproducibility among laboratories. *Rev. Sci. Tech. Off. Int. Epiz.,***40**, 131–141. 

**   *

NB: FIRST ADOPTED IN 1996 AS *GOOD LABORATORY PRACTICE, QUALITY CONTROL AND QUALITY ASSURANCE*. MOST RECENT UPDATES ADOPTED IN 2017.

WOAH *Terrestrial**Manual* 2024	1

1	WOAH *Terrestrial**Manual* 2024

WOAH *Terrestrial**Manual* 2024	1