NIST Handbook 150-31

2021 Edition

NVLAP

Health Information

Technology Testing

Bradley Moore

Asara Clark

This publication is available free of charge from: 

https://doi.org/10.6028/NIST.HB.150-31-2021 
NIST Handbook 150-31

2021 Edition

NVLAP

Health Information

Technology Testing

Bradley Moore

National Voluntary Laboratory Accreditation Program

Standards Coordination Office

Laboratory Programs

Asara Clark

Health IT Certification Program

The Office of the National Coordinator for Health Information Technology (ONC)

Health and Human Services (HHS)

This publication is available free of charge from:

https://doi.org/10.6028/NIST.HB.150-31-2021

November 2021

U.S. Department of Commerce

Gina M. Raimondo, Secretary

National Institute of Standards and Technology

James K. Olthoff, Performing the Non-Exclusive Functions and Duties of the Under Secretary of Commerce for 

Standards and Technology & Director, National Institute of Standards and Technology
Certain commercial entities, equipment, or materials may be 

identified in this document in order to describe an experimental procedure 

or concept adequately. Such identification is not intended to imply 

recommendations or endorsements by the National Institute of Standards 

and Technology, nor is it intended to imply that the entities, materials, or 

equipment are necessarily the best available for the purpose.

National Institute of Standards and Technology Handbook 150-31 2021 Edition

Natl. Inst. Stand. Technol. Handbook 150-31 2021 Ed., 25 pages (November 2021) 

This publication is available free of charge from: 

https://doi.org/10.6028/NIST.HB.150-31-2021
NVLAP AND THE NVLAP LOGO

The term NVLAP and the NVLAP logo are 

registered marks of the Federal Government, 

which retains exclusive rights to control the use 

thereof. Permission to use the term and symbol 

(NVLAP logo with approved caption) is granted 

to NVLAP-accredited laboratories for the 

limited purpose of announcing their accredited 

status, and for use on reports that describe only 

testing and calibration within the scope of 

accreditation. NVLAP reserves the right to 

control the quality of the use of the NVLAP 

term, logo, and symbol.

______________________________________________________________________________________________________ 

This publication is available free of charge from: https://doi.org/10.6028/NIST.HB.150-31-2021
NIST Handbook 150-31:2021 i 

Contents

Acknowledgments ........................................................................................................................................ iii

Foreword ...................................................................................................................................................... iv

Introduction ................................................................................................................................................... v

1 General information ............................................................................................................................... 1

1.1 Scope ............................................................................................................................................. 1

1.2 Organization of handbook ............................................................................................................. 1

1.3 Program description...................................................................................................................... 1

1.4 References..................................................................................................................................... 2

1.5 Terms and definitions ................................................................................................................... 3

1.6 Program Documentation ............................................................................................................... 3

2 LAP establishment, development, and implementation ......................................................................... 4

3 Accreditation process............................................................................................................................. 4

3.1 General.......................................................................................................................................... 4

3.2 Management system review.......................................................................................................... 5

3.3 Onsite assessment ......................................................................................................................... 6

3.3.1 General.................................................................................................................................... 6

3.3.3 Nonconformities and comments .............................................................................................. 8

3.4 Proficiency testing ........................................................................................................................ 8

3.5 Accreditatoin Decision.................................................................................................................. 9

3.6 Granting Accreditation.................................................................................................................. 9

3.7 Renewal of accreditation............................................................................................................... 9

3.8 Monitoring visits........................................................................................................................... 9

3.9 Changes to the scope of accreditation ......................................................................................... 10

3.10 Adverse accreditation actions (suspension, revocation) ............................................................. 10

4 General requirements ........................................................................................................................... 10

4.1 Impartiality .................................................................................................................................. 10

4.2 Confidentiality ............................................................................................................................ 10

5 Structural requirements ........................................................................................................................ 10

6 Resource Requirements ........................................................................................................................ 11

6.1 General ........................................................................................................................................ 11

6.2 Personnel ..................................................................................................................................... 11

6.3 Facilities and environmental conditions...................................................................................... 11

6.4 Equipment ................................................................................................................................... 11

6.5 Metrological traceability ............................................................................................................. 11

6.6 Externally provided products and services ................................................................................. 12

______________________________________________________________________________________________________ 

This publication is available free of charge from: https://doi.org/10.6028/NIST.HB.150-31-2021
NIST Handbook 150-31:2021 ii

7 Process Requirements ........................................................................................................................... 12

7.1 Review of requests, tenders and contracts .................................................................................. 12

7.2 Selection, verification and validation of methods ....................................................................... 12

7.3 Sampling ..................................................................................................................................... 12

7.4 Handling of test or calibration items ........................................................................................... 12

7.5 Technical records ........................................................................................................................ 12

7.6 Evaluation of measurement uncertainty ...................................................................................... 12

7.7 Ensuring the validity of results ................................................................................................... 12

7.8 Reporting of results ..................................................................................................................... 13

7.9 Complaints .................................................................................................................................. 13

7.10 Nonconforming work .................................................................................................................. 13

7.11 Control of data and information management ............................................................................ 13

8 Management system requirements ....................................................................................................... 13

8.1 Options ........................................................................................................................................ 13

8.2 Management system documentation (Option A) ........................................................................ 14

8.3 Control of management system documents (Option A) .............................................................. 14

8.4 Control of records (Option A) .................................................................................................... 14

8.5 Actions to address risks and opportunities (Option A) ............................................................... 14

8.6 Improvement (Option A) ............................................................................................................ 14

8.7 Corrective actions (Option A) ..................................................................................................... 14

8.8 Internal Audits (Option A) .......................................................................................................... 14

8.9 Management Review (Option A) ................................................................................................ 14

Annex A ...................................................................................................................................................... 15

Annex B ...................................................................................................................................................... 16

______________________________________________________________________________________________________ 

This publication is available free of charge from: https://doi.org/10.6028/NIST.HB.150-31-2021
NIST Handbook 150-31:2021 iii

Acknowledgments

The authors would like to thank the persons who reviewed and contributed to this document, especially the 

members of the healthcare information technology working group from the Office of the National 

Coordinator for Health Information Technology at the U.S. Department of Health and Human Services 

(ONC/HHS); Lisa Carnahan and Amy Phelps from the National Institute of Standards and Technology 

(NIST); and assessors Mike Glickman and Bob Yencha. 

______________________________________________________________________________________________________ 

This publication is available free of charge from: https://doi.org/10.6028/NIST.HB.150-31-2021
NIST Handbook 150-31:2021 iv

Foreword

The NIST Handbook 150 publication series sets forth the procedures, requirements, and guidance for the 

accreditation of testing and calibration laboratories by the National Voluntary Laboratory Accreditation 

Program (NVLAP). The series is comprised of the following publications:

• NIST Handbook 150, NVLAP Procedures and General Requirements, which contains the general 

procedures and requirements under which NVLAP operates as an unbiased third-party accreditation 

body; 

• NIST Handbook 150-xx program-specific handbooks, which supplement NIST Handbook 150 by 

providing additional requirements, guidance, and interpretive information applicable to specific 

NVLAP laboratory accreditation programs (LAPs).

The program-specific handbooks are not stand-alone documents, but rather are companion documents to 

NIST Handbook 150 and ISO/IEC 17025:2017 (hereafter referred to as ISO/IEC 17025). They tailor the 

general criteria found in ISO/IEC 17025 and NIST Handbook 150 to the specific tests, calibrations, or types 

of tests or calibrations covered by a LAP.

NIST Handbook 150-31, NVLAP Health Information Technology Testing, presents the technical 

requirements and guidance for the accreditation of laboratories under the NVLAP Health Information 

Technology (Health IT) LAP. The handbook is intended for use by accredited laboratories, assessors 

conducting onsite assessments, laboratories seeking accreditation, laboratory accreditation systems, users 

of laboratory services, and others needing information on the requirements for accreditation under this 

program. All statements in this handbook are meant to supplement ISO/IEC 17025 and NIST Handbook 

150 and by no means contradict them. In all cases of ambiguity, the requirements stated in effective federal 

regulations supersedes all other documents. This November 2021 edition of the NIST Handbook 150-31 

supersedes and replaces the July 2017 edition. 

The November 2021 edition of NIST Handbook 150-31 was revised with the participation of technical 

experts in the field of health information technology testing. The following main changes have been made 

to this handbook with respect to the previous edition: 

• the numbering has been updated to reflect that used by ISO/IEC 17025:2017, General requirements for 

the competence of testing and calibration laboratories; 

• all references to applicable international guides and standards have been updated; 

• redundant requirements for specific equipment, certifications, records, etc., found in the various 

standards for test methods used in the program were removed. 

This handbook is also available on the NVLAP web site, https://www.nist.gov/nvlap and also from the 

NIST Research Library, https://doi.org/10.6028/NIST.HB.150-31-2021. 

Questions or comments concerning this handbook should be submitted to NVLAP, National Institute of 

Standards and Technology, 100 Bureau Drive, Stop 2140, Gaithersburg, MD, 20899-2140; phone: 301-975-

4016; fax: 301-926-2884; e-mail: nvlap@nist.gov. 

______________________________________________________________________________________________________ 

This publication is available free of charge from: https://doi.org/10.6028/NIST.HB.150-31-2021
NIST Handbook 150-31:2021 v 

Introduction

NIST Handbook 150-31 augments ISO/IEC 17025, General requirements for the competence of testing and 

calibration laboratories, and NIST Handbook 150, NVLAP Procedures and General Requirements, by 

gathering technical requirements of the laboratory accreditation program (LAP) for functional and 

conformance testing of health information technology (IT) capabilities to regulatory requirements of the 

Office of the National Coordinator for Health Information Technology ONC Health IT Certification 

Program. Technical requirements are explained to indicate how NVLAP criteria are applied for 

accreditation under the Health IT LAP.

Any domestic or foreign laboratory (including commercial; manufacturer; academic; and federal, state or 

local government laboratories) that performs testing covered by the Health IT LAP may apply for NVLAP 

accreditation. Accreditation will be granted to a laboratory that complies with the conditions for 

accreditation as defined in NIST Handbook 150. Accreditation does not imply a guarantee of laboratory 

performance or of test results; it is a finding of laboratory competence and proficiency in conducting testing.

The testing services related to this health IT accreditation program are defined by the “Health Information 

Technology Standards, Implementation Specifications, and Certification Criteria and Certification 

Programs for Health Information Technology,” Title 45 Code of Federal Regulations, Part 170, amended 

May 1, 2020. (see 1.4 for related references).

______________________________________________________________________________________________________ 

This publication is available free of charge from: https://doi.org/10.6028/NIST.HB.150-31-2021
NIST Handbook 150-31:2021 1 

1 General information

1.1 Scope

1.1.1 This handbook specifies the technical requirements and provides guidance for the accreditation of 

laboratories under the NVLAP Health IT LAP. It supplements the NVLAP procedures and general 

requirements found in NIST Handbook 150 and ISO/IEC 17025 by tailoring the general criteria found in 

NIST Handbook 150 to the specific tests and/or types of tests covered by the Health IT LAP.

1.1.2 NIST Handbook 150, ISO/IEC 17025, this handbook, Office of the National Coordinator for Health 

Information Technology (ONC) certification regulations, and ONC-approved methods and procedures, test 

tools, and test data associated with this program constitute the collective body of requirements that must be 

met by a laboratory seeking NVLAP accreditation for the Health IT LAP.

1.1.3 Any interpretive comments and additional requirements contained in this handbook complement 

the general NVLAP criteria for specific application in the Health IT LAP.

1.2 Organization of handbook 

The numbering and titles of the clauses and subclauses of this handbook match those of ISO/IEC 17025. 

The primary subclauses four through eight (e.g., 4.1, 4.2, etc.) are also numbered and titled to correspond 

with those of ISO/IEC 17025, even when there are no requirements additional to those in ISO/IEC 17025. 

Annex A (informative) provides a list of acronyms and abbreviations used in this handbook. Annex B 

(normative) contains provisions to which it is necessary to conform in order to claim compliance with the 

handbook requirements. 

1.3 Program description

1.3.1 In response to the Health Information Technology for Economic and Clinical Health (HITECH) 

Act of 2009, the U.S. Department of Health and Human Services along with ONC requested establishment 

of the Health Information Technology LAP by NVLAP to accredit laboratories that perform technical and 

conformance testing of health information technology to ONC regulatory requirements. See 1.4 References 

for a complete list of currently accepted standards and certification criteria. 

1.3.2 NVLAP reserves the right to expand the Health IT LAP and offer to interested laboratories 

additional testing not listed in this handbook. Laboratories are advised to review the Health IT LAP’s 

website for the most current information, <https://www.nist.gov/nvlap/healthcare-information-technologylap>. 

1.3.3 The Health IT LAP offers accreditation for all active editions of ONC Certification Criteria. 

Depending on the breadth of its testing capabilities, the applicant laboratory may select a scope of 

accreditation to an entire certification edition (e.g., 2015 Edition) or to a subset of a certification edition.

For additional information regarding the scope of accreditation available for selection in this program, 

please contact NVLAP. 

1.3.4 Accreditation to this LAP is a requirement for application to obtain ONC-Authorized status as 

noted in § 170.520(b) ONC-ATL application. Note that although NVLAP accreditation may be granted, 

work cannot be offered or commence under the ONC Health IT Certification Program until ONC-

______________________________________________________________________________________________________ 

This publication is available free of charge from: https://doi.org/10.6028/NIST.HB.150-31-2021
NIST Handbook 150-31:2021 2 

Authorization is granted. For additional information on ONC-Authorization, please contact the ONC Health 

IT Certification Program. 

1.3.5 Accreditation in this LAP is also a requirement to maintain ONC-Authorized status as noted in 

§ 170.524 Principles of Proper Conduct for ONC-ATLs. ONC requires laboratories to adhere to these 

Principles of Proper Conduct to remain in good standing with ONC. 

1.4 References

The following documents are referenced in this handbook. For dated references, only the edition cited 

applies. For undated references, the latest edition of the referenced document (including any amendments) 

shall apply within one year of publication or within another time limit specified by regulations or other 

requirement documents.

General

— NIST Handbook 150, NVLAP Procedures and General Requirements

— ISO/IEC 17025, General requirements for the competence of testing and calibration laboratories

ONC Health IT Certification Program Regulations 

— “Information Blocking and the ONC Health IT Certification Program: Extension of Compliance

Dates and Timeframes in Response to the COVID-19 Public Health Emergency,” Federal Register 

(effective: 4 November 2020): 70064-70085 (codified at 45 CFR Part 170); available online 

<https://www.federalregister.gov/documents/2020/11/04/2020-24376/information-blocking-andthe-onc-health-it-certification-program-extension-of-compliance-dates-and>

— “21st Century Cures Act: Interoperability, Information Blocking, and the ONC Health IT 

Certification Program,” Federal Register (effective: 1 May 2020): 25642-25961 (codified at 45 

CFR Part 170); available online <https://www.govinfo.gov/content/pkg/FR-2020-05-01/pdf/2020-

07419.pdf>.

— “ONC Health IT Certification Program: Enhanced Oversight and Accountability,” Federal 

Register (effective: 10/19/2016): 72404-72471; (codified at 45 CFR Part 170); available online 

<https://www.govinfo.gov/content/pkg/FR-2016-10-19/pdf/2016-24908.pdf> 

— “2015 Edition Health Information Technology (Health IT) Certification Criteria, 2015 Edition Base 

Electronic Health Record (EHR) Definition, and ONC Health IT Certification Program 

Modifications; Final Rule,” Federal Register (effective: 16 October 2015): 62602-62759 (codified 

at 45 CFR Part 170); available online <https://www.gpo.gov/fdsys/pkg/FR-2015-10-16/pdf/2015-

25597.pdf>.

— “Establishment of the Permanent Certification Program for Health Information Technology; Final 

Rule,” Federal Register (effective: 7 January 2011): 1262-1331 (to be codified at 45 CFR Part 

170); available online at <https://www.gpo.gov/fdsys/pkg/FR-2011-01-07/pdf/2010-33174.pdf>.

______________________________________________________________________________________________________ 

This publication is available free of charge from: https://doi.org/10.6028/NIST.HB.150-31-2021
NIST Handbook 150-31:2021 3 

ONC Health IT Certification Program Approved Test Methods

— ONC-Approved 2015 Edition Test Method; available online at <https://www.healtHealth 

IT.gov/policy-researchers-implementers/2015-edition-test-method>.

— ONC-Approved 2015 Edition Cures Update Test Method; available online at

<https://www.healthit.gov/topic/certification-ehrs/2015-edition-cures-update-test-method>. 

1.5 Terms and definitions

For the purposes of this handbook, the terms and definitions given in NIST Handbook 150 apply unless a 

term is redefined in this handbook or in ONC regulations (§ 170.102 Definitions.) The definitions provided 

in this handbook are specific to the Health IT LAP, and when applicable, they supersede the definitions 

given in NIST Handbook 150. For a list of acronyms, see Annex A. 

1.5.1 

Base Electronic Health Record (EHR)

A baseline assurance that certified health IT which satisfies the Base EHR definition has been developed 

to possess, at a minimum, a key set of capabilities. See 2015 Edition Cures Update – Base Electronic Health 

Record (EHR) Definition <https://www.healthit.gov/topic/certification-ehrs/2015-edition-testmethod/2015-edition-cures-update-base-electronic-health-record-definition>. 

1.5.2 

health information technology

Defined in the regulations cited in Section 1.4 References. Involves the processing, storage, and exchange 

of health information in an electronic environment. 

1.5.3 

test data

Data used to verify that a given set of inputs to a given function produces the expected result. 

1.5.4 

test method

The set of ONC-approved test procedures, test tools, and test data used for evaluating conformance of health 

IT to regulatory requirements, technical requirements, and standards as approved by ONC. 

1.5.5 

test procedure

The ONC-approved testing document used to guide testing laboratories as they conduct health IT testing in 

the ONC Health IT Certification Program to provide traceability from the certification criterion or criteria 

to testing activities, and to ensure consistency throughout the certification process. 

1.5.6 

test tool 

A software tool used to perform automated testing of specific functionality or conformity to requirements. 

1.6 Program Documentation 

1.6.1 General

______________________________________________________________________________________________________ 

This publication is available free of charge from: https://doi.org/10.6028/NIST.HB.150-31-2021
NIST Handbook 150-31:2021 4 

This handbook details the health IT-program-specific requirements and technical procedures while 

interpreting, detailing, and expanding portions of NIST Handbook 150 for Health IT LAP use. Both the 

NIST Handbook 150 checklist and the NIST Handbook 150-31 checklist are used in conducting an 

assessment in the Health IT LAP. Assessor use of the NVLAP checklists is to ensure that each laboratory 

receives an assessment comparable to that received by other laboratories. Checklists assist assessors in 

documenting the assessment to NVLAP requirements found in NIST Handbook 150 and in this handbook. 

Checklists contain definitive statements or questions about all aspects of the NVLAP criteria for 

accreditation and form part of the assessment report (see NIST Handbook 150). The current version of each 

checklist is available upon request or on the NVLAP website, https://www.nist.gov/nvlap

1.6.2 NVLAP General Criteria Checklist

All NVLAP programs use the NVLAP General Criteria Checklist (ISO/IEC 17025), formerly called the 

NIST Handbook 150 Checklist, which contains the requirements published in ISO/IEC 17025 and NIST 

Handbook 150. The checklist items are numbered to correspond to clauses 4 through 8 of ISO/IEC 17025 

and annexes A, B, and E of NIST Handbook 150. The current version of the checklist is available from

NVLAP upon verification of ownership of ISO/IEC 17025. 

1.6.3 NIST Handbook 150-31 Checklist

The NIST Handbook 150-31 Checklist (also referred to as the health IT program-specific checklist) 

addresses the requirements specific to health information technology testing given in NIST Handbook 

150-31. The checklist contains the requirements provided in this handbook, including testing requirements, 

and additional details and notes for the assessors (e.g., the names of the key personnel), with an emphasis 

on observing test performance, accuracy, instrumentation, calibration, personnel competency, and 

reporting. 

The current version of the checklist is available from the Health IT LAP website, 

<https://www.nist.gov/nvlap/nvlap-checklists.cfm>.

1.6.5 NVLAP Lab Bulletins

NVLAP Lab Bulletins are issued to laboratories and assessors, when needed, to clarify program-specific 

requirements and to provide information about program additions and changes. 

2 LAP establishment, development, and implementation

This clause contains no information additional to that provided in NIST Handbook 150, clause 2. 

3 Accreditation process

3.1 General

3.1.1 An overview of the laboratory accreditation process is provided in NIST Handbook 150, clause 3, 

and includes information pertaining to application for accreditation, onsite assessment, proficiency testing, 

or similar programs, accreditation decision, granting accreditation, renewal of accreditation, changes to 

______________________________________________________________________________________________________ 

This publication is available free of charge from: https://doi.org/10.6028/NIST.HB.150-31-2021
NIST Handbook 150-31:2021 5 

scope of accreditation, monitoring visits, and suspension, denial, revocation, and voluntary termination of 

accreditation.

3.1.2 The ONC has determined that access to review the accreditation records for laboratories in this 

accreditation program is needed. To that end, this program has an additional condition of accreditation that 

requires the attestation in Annex B be executed and returned to NVLAP with the lab’s application. 

In addition, under § 170.524(f)(2) the ONC may also request all records related to the testing of Complete 

EHRs and/or Health IT Modules within the scope of § 170.524(f)(1).

3.2 Management system review

3.2.1 Management System Implementation

3.2.1.1 A laboratory interested in accreditation for any type of test offered under the Health IT LAP shall 

review and become familiar with the requirements listed in ISO/IEC 17025, NIST Handbook 150 and this 

handbook, review the Health IT LAP website at <https://www.nist.gov/nvlap/healthcare-informationtechnology-lap>, and contact NVLAP for the most current updates on the requirements and application 

process.

3.2.1.2 Prior to applying to NVLAP for accreditation, a laboratory shall have a fully implemented 

management system.

3.2.2 Activities prior to the assessment 

Upon NVLAP determining that an application is complete, the next steps are evaluation of the laboratory’s 

management system documentation and administration of the proficiency written/oral exam, as needed. 

a) Management system documentation evaluation

Prior to the onsite assessment, NVLAP and the assigned assessor(s) review all relevant management 

system documentation for conformity with NVLAP requirements, including the requirements of this 

handbook, NIST Handbook 150, and ISO/IEC 17025. During this review, the assessor may request 

additional management system and/or technical documents and/or records. 

b) Proficiency written and/or oral exam

The Health IT LAP requires demonstration of domain knowledge and technical competence of the 

technologies, standards, practices, and implementation architectures commonly deployed in the 

healthcare information technology domain as well as technical expertise in the application of the 

technical requirements found in 45 CFR Part 170, et al.

Prior to the initial onsite assessment and after it is determined that the management system 

documentation meets the minimum requirements of this handbook, NIST Handbook 150, and ISO/IEC 

17025 a written exam may be provided to the applicant laboratory depending upon the intended scope 

of accreditation. This exam evaluates the laboratory personnel’s technical expertise and knowledge of 

the standards and test procedure(s) applicable to the scope of accreditation for which the laboratory is 

applying. In some instances, an oral exam may be necessary to demonstrate proficiency. A technical 

assessor(s) and/or expert(s) from the associated technical program conducts this exam via a 

teleconference with the laboratory personnel prior to the assessment.

______________________________________________________________________________________________________ 

This publication is available free of charge from: https://doi.org/10.6028/NIST.HB.150-31-2021
NIST Handbook 150-31:2021 6 

3.3 Onsite assessment 

3.3.1 General

3.3.1.1 The purpose of the onsite assessment is to gather evidence of the laboratory’s compliance with 

ISO/IEC 17025, NIST Handbook 150, this handbook, and its own management system and to assess the 

capability and competence of the testing activities for which accreditation is being requested.

3.3.1.2 The provisions of CFR 45 170.524(e) allow ONC, or its authorized agent(s), to periodically observe 

onsite (unannounced or scheduled), during normal business hours, any testing performed pursuant to the 

ONC Health IT Certification Program. ONC may request observer access to the remote collaboration 

software when remote testing is being utilized. 

3.3.1.3 Prior to the onsite assessment, the NVLAP assessor(s) will provide a preliminary agenda.

3.3.1.4 The assessment is scheduled by NVLAP at a mutually agreed-upon date and time; The time span 

for the assessment is dependent upon the applicant’s scope of accreditation. Typically, the assessment will 

span two to three days and will be performed by two or more NVLAP assessors. Observations made by the 

assessors during the assessment are held in the strictest confidence.

3.3.1.5 The laboratory shall make available all supporting technical information including the management 

system cross-reference document (see 8.2). All relevant documentation shall be provided to NVLAP and 

its assessor(s) in English.

3.3.1.6 Additionally, the assigned assessors shall evaluate any new or updated requirements that are 

documented in a NVLAP lab bulletin and on the Health IT LAP’s website,

<https://www.nist.gov/nvlap/nvlap-healthcare-information-technology-lap>, but are not yet incorporated 

into this handbook and assessment checklist.

3.3.2 Assessment activities

3.3.2.1 The activities covered during a typical onsite assessment are described below. 

a) Opening meeting: The assessors meet with laboratory management and supervisory personnel to 

explain the purpose of the onsite assessment and to discuss the schedule for assessment activities. 

Information provided by the laboratory on its application form may be discussed during this 

meeting. At the discretion of the laboratory manager, other staff may attend this meeting.

b) Staff interviews: The assessors will ask the laboratory manager to assist in arranging times for 

individual interviews with laboratory staff. The assessors interview staff filling key positions 

(e.g., laboratory manager, authorized representative, and any authorized signatories) and staff who 

have an effect on the outcome of testing. It is not necessary for the assessors to talk to all staff; 

however, they will select staff representing all aspects of the laboratory.

These interviews are conducted to determine whether staff are properly trained, assigned, 

supervised, and technically competent for the tasks assigned to them (i.e., the health IT 

conformance testing methodologies). 

c) Records review: The assessors review laboratory documentation, including the management 

system documentation, equipment and maintenance records, record-keeping procedures, testing 

______________________________________________________________________________________________________ 

This publication is available free of charge from: https://doi.org/10.6028/NIST.HB.150-31-2021
NIST Handbook 150-31:2021 7 

procedures, laboratory test records and reports, personnel competency records, personnel training 

plans and records, procedures for updating pertinent information, and safeguards for the protection 

of vendor-sensitive and proprietary information. 

Assessors do not need access to employee information that may be considered sensitive or private 

such as salary, medical information, or performance reviews for work done outside the scope of 

the laboratory’s accreditation. However, this information is often stored together with personnel 

information the assessors need to check (e.g., job descriptions, resumes, and technical 

performance reviews). In these cases, the assessors work with the laboratory to ensure that they 

can perform their review without violating individual privacy. At the discretion of the laboratory, 

a member of its human resources department may be present during review of personnel 

information. 

d) Internal audit and management review: The assessors review and discuss with staff the laboratory’s 

internal audit and management review activities. The discussion will include all aspects of those 

activities including the management system procedures, the audit findings, the actions taken to 

resolve problems identified, and the results of the management review.

e) Equipment, Environment, and Demonstrations: The assessor examines the equipment, facilities, 

and observes the demonstration of selected procedures by the appropriate personnel assigned to 

conduct the tests and apply the health IT conformance testing methodology. The testing 

environment is evaluated for adequacy (e.g., space, security, separation, and storage). 

Demonstrations requested may be selective or all-inclusive and may include the laboratory staff

preparing and using test tools, including loading, configuring, and running of the tools; preparing 

test reports; and performing updates, if necessary. The assessors will also review the test data and 

examine the hardware/software for functionality and appropriateness. Test personnel are requested 

to demonstrate their competence to perform the test methods for which the laboratory is seeking 

accreditation. 

f) Proficiency testing: When applicable, the assessors discuss all aspects of proficiency testing with 

staff. Test methodology and the records documenting the laboratory’s execution of the testing are 

reviewed and discussed.

g) Onsite assessment report: The assessors prepare an onsite assessment report that summarizes their 

findings (nonconformities and comments). This report normally consists of the Signature Sheet, 

Onsite Assessment Narrative Summary, the NVLAP General Criteria Checklist, and the Health 

IT (HIT) Program-Specific Checklist.

h) Closing meeting: At the end of the onsite assessment, the assessors hold a closing meeting with 

the laboratory manager and staff to discuss the onsite assessment report and the laboratory’s plans 

for resolution of nonconformities. The process for resolving nonconformities is documented in 

NIST Handbook 150. 

At the conclusion of the discussion, the report is finalized, and the assessors and the laboratory’s 

authorized representative sign the report. A copy of the complete report is given to the laboratory 

representative and the assessors submit the report documentation to NVLAP. Disagreements 

between the laboratory and the assessors shall be referred to NVLAP for arbitration and final 

resolution. 

______________________________________________________________________________________________________ 

This publication is available free of charge from: https://doi.org/10.6028/NIST.HB.150-31-2021
NIST Handbook 150-31:2021 8 

3.3.2.2 The laboratory personnel expertise will be evaluated in the technical area(s)/specific criteria for 

which the laboratory is applying for accreditation. Staff member interaction and knowledge distribution 

among team members are key factors that will be observed by the assessors. 

3.3.3 Nonconformities and comments

3.3.1 The process for resolving nonconformities identified during the onsite is documented in NIST 

Handbook 150, 3.3.4.. 

3.3.3.2 Comments in the report should be given serious consideration by the laboratory, but no action is 

mandated and changes may be made at the laboratory’s discretion. Comments are those areas of concern 

where a nonconformity may arise; however, no objective evidence is available to support citing a 

nonconformity. Comments noted in the assessment will be reviewed at the next assessment to ensure that 

these issues have not risen to the level of nonconformities. 

3.3.3.3 Positive feedback may also be recorded in the assessment report.

3.3.3.4 Upon completion of the assessment and corrective action responses to nonconformities, if any, the 

accreditation process ends with NVLAP’s decision regarding the laboratory’s accreditation.

3.4 Proficiency testing

3.4.1 General

3.4.1.1 Approved Test Methods are designed as standalone methods that most often have defined outcomes 

of “demonstrates conformance” or “does not demonstrate conformance” (pass/fail) that can be sequenced 

to optimize test plans to the functional scope of the product under test. 

3.4..1.2 Proficiency testing for this program may include written examinations, oral examinations, and 

evaluation of artifacts. Proficiency testing is performed prior to initial assessment. Written examinations, 

oral examinations, and evaluation of artifacts may also be conducted at the request of the ONC. 

3.4.1.3 The laboratory will be informed of any proficiency testing schedules developed for the Health IT

LAP. When a proficiency testing schedule is published, participation in the proficiency testing is 

mandatory for laboratories accredited in the program. 

3.4.2 Types of proficiency testing

NVLAP utilizes proficiency testing programs that are consistent with the requirements contained in 

ISO/IEC 17043, Conformity assessment – General requirements for proficiency testing, where applicable. 

a) Demonstration of a solid background, knowledge, and technical expertise in the area of the ONCapproved test procedures for the scope of accreditation. For this demonstration, the laboratory 

shall be provided with a proficiency evaluation exam to be completed by appropriate personnel 

including all those performing testing. 

b) Demonstration of correct identification and use of any ONC-approved test tools. The laboratory 

shall demonstrate that all appropriate personnel, including those performing testing, understand 

the test tools and/or component use and operation. This shall be demonstrated by laboratory 

______________________________________________________________________________________________________ 

This publication is available free of charge from: https://doi.org/10.6028/NIST.HB.150-31-2021
NIST Handbook 150-31:2021 9 

personnel exercising use of the ONC-approved test tools either under the assessors’ direct 

observation or through a written exam. 

c) Demonstration of correct use and/or development of test data for use in testing. The laboratory 

shall demonstrate that all appropriate personnel, including those performing testing, understand 

test data use and operation. This shall be demonstrated by laboratory personnel exercising use 

and/or development of test data either under the assessors’ direct observation or through a written 

exam.

d) demonstration of an understanding and correct interpretation of all data and test results reported 

by the test tools or reported in the course of implementing appropriate test procedure(s). This 

shall be demonstrated by laboratory personnel during an assessment under the assessors’ direct 

observation or in response to a request from NVLAP. 

e) demonstration of report generation of accurate results in an approved format. This shall be 

demonstrated by laboratory personnel exercising use of the laboratory’s report generation process 

either during the assessment under the assessors’ direct observation or by written request from 

NVLAP.

3.4.3 Proficiency testing analysis and reporting

Information regarding proficiency testing (PT) analysis and reporting may be found in section 3.4.3 of 

NIST Handbook 150. 

3.4.4 Proficiency testing nonconformities

Information regarding proficiency testing (PT) nonconformities may be found in section 3.4.4 of NIST 

Handbook 150. 

3.5 Accreditation Decision 

There are no requirements additional to those set forth in NIST Handbook 150.

3.6 Granting Accreditation 

It is important to note that the laboratory is granted initial accreditation after it has effectively 

implemented the management system, produced appropriate records of all management system activities, 

including conducting at least one internal audit and one management review, and successfully completed 

the initial Proficiency Testing (PT) exam/activity.

3.7 Renewal of accreditation

There are no requirements additional to those set forth in NIST Handbook 150.

3.8 Monitoring visits 

There are no requirements additional to those set forth in NIST Handbook 150.

______________________________________________________________________________________________________ 

This publication is available free of charge from: https://doi.org/10.6028/NIST.HB.150-31-2021
NIST Handbook 150-31:2021 10

3.9 Changes to the scope of accreditation 

There are no requirements additional to those set forth in NIST Handbook 150.

3.10 Adverse accreditation actions (suspension, revocation) 

3.10.1 Reasons for suspension or revocation of a laboratory’s accreditation may include, but are not 

limited to: 

a) Failure to appropriately address and resolve complaints from customers or other interested 

parties. 

b) Loss of key personnel without immediate adequate replacement;

c) New personnel prove to be unqualified for authorized testing; 

d) The facilities become inadequate to support testing; 

e) Failure to demonstrate continued competence to perform Health IT conformance testing. 

3.10.2 All issues surrounding the need to suspend and/or revoke a laboratory’s accreditation are reviewed 

on a case-by-case basis by NVLAP.

NOTE Laboratories are subject to the provisions of the “ONC Health IT Certification Program: Enhanced Oversight 

and Accountability” Final Rule.

4 General requirements

4.1 Impartiality

There are no requirements additional to those set forth in ISO/IEC 17025. 

4.2 Confidentiality

There are no requirements additional to those set forth in ISO/IEC 17025. 

NOTE See 3.1.2 for information related to ONC access to laboratory accreditation records.

5 Structural requirements

There are no requirements additional to those set forth in ISO/IEC 17025. 

______________________________________________________________________________________________________ 

This publication is available free of charge from: https://doi.org/10.6028/NIST.HB.150-31-2021
NIST Handbook 150-31:2021 11

6 Resource Requirements

6.1 General

There are no requirements additional to those set forth in ISO/IEC 17025. 

6.2 Personnel

6.2.1 The testing laboratory shall retain responsible, competent personnel who are knowledgeable and 

capable of demonstrating competencies in the ONC Health IT Certification Program regulations and test 

methods listed in section 1.4. 

6.2.2 The testing laboratory shall have staff members with at least a bachelor’s degree in computer 

science, information systems, or similar technical discipline or equivalent experience – such as three years 

experience – in the area of health IT testing, health IT interoperability, health IT standards and technologies, 

and events relevant to health IT. 

6.2.3 The laboratory’s training program shall be relevant to health IT testing, health IT standards, 

technologies, and events relevant to health IT testing. The laboratory shall document its training program 

including, at a minimum, the ONC Health IT Certification Program regulations and test methods listed in 

section 1.4 of this handbook.

In addition, the laboratory shall also participate in required training as directed by the ONC Health IT 

Certification Program. (§ 170.524(b)). 

6.3 Facilities and environmental conditions

Where there are no physical testing locations, the laboratory shall ensure that the requirements of ISO/IEC 

17025, section 6.3 shall be met when conducting testing of health IT. 

6.4 Equipment

The laboratory may install and maintain a locally controlled installation of the ONC-approved testing 

tool(s) to produce test results. Local instantiations shall be validated in accordance with section 7.2 and 

demonstrate that they achieve the same results as the ONC-designated hosted tools. 

6.5 Metrological traceability

The testing laboratory shall ensure: 

a) management system documentation is in place to trace localized test scripts and test data back to 

the ONC-Approved Test Method. 

b) local installations of the test tools are documented and traceable back to the ONC-Approved Test 

Method. 

______________________________________________________________________________________________________ 

This publication is available free of charge from: https://doi.org/10.6028/NIST.HB.150-31-2021
NIST Handbook 150-31:2021 12

c) where permitted, laboratory or developer-supplied test data meet the functional and interoperable 

requirements identified in the certification criteria and can be adequately evaluated for 

conformance. 

NOTE For health IT testing, traceability is interpreted to mean that the ONC-Approved Test Method (test 

procedures, test tools, and required test data) shall be traceable back to the underlying requirements of the ONC health 

IT certification criteria requirements in the applicable section(s) of 45 CFR Part 170. 

6.6 Externally provided products and services 

There are no requirements additional to those set forth in ISO/IEC 17025. 

7 Process Requirements

7.1 Review of requests, tenders and contracts

There are no requirements additional to those set forth in ISO/IEC 17025. 

7.2 Selection, verification and validation of methods 

There are no requirements additional to those set forth in ISO/IEC 17025. 

7.3 Sampling 

There are no requirements additional to those set forth in ISO/IEC 17025. 

7.4 Handling of test or calibration items 

There are no requirements additional to those set forth in ISO/IEC 17025. 

7.5 Technical records 

There are no requirements additional to those set forth in ISO/IEC 17025. 

7.6 Evaluation of measurement uncertainty 

There are no requirements additional to those set forth in ISO/IEC 17025. 

7.7 Ensuring the validity of results 

7.7.1 Laboratories shall demonstrate and document on-going activities to maintain their proficiency in 

health IT to ensure no methods are compromised during execution. 

______________________________________________________________________________________________________ 

This publication is available free of charge from: https://doi.org/10.6028/NIST.HB.150-31-2021
NIST Handbook 150-31:2021 13

7.2.2 The laboratory shall have satisfactorily participated in all required proficiency testing (PT) during 

its previous accreditation period or prior to accreditation being granted, if initial accreditation.

NOTE Required PT will be identified on the program page on the NVLAP website. If the laboratory performs 

unsatisfactorily in any proficiency test, the laboratory shall take corrective action to investigate and resolve 

nonconformities in a timely manner according to the requirements of ISO/IEC 17025 for the control of nonconforming 

work. (See also section 3.5 of this handbook). 

7.8 Reporting of results 

7.8.1 Common requirements

Testing laboratories shall document the specific laboratory or developer-supplied test data utilized for 

testing, when applicable. 

7.8.2 Reporting opinions and interpretations

7.8.2.1 Whenever test procedures are such that an analysis of the observations is required in order to 

interpret the results before stating them in a test report, the laboratory shall have a defined process to ensure 

that the repeatability, reproducibility, and objectivity of the test results can be maintained. 

7.8.2.2 The testing laboratory shall have and maintain a policy for handling interpretations of test results.

7.9 Complaints 

There are no requirements additional to those set forth in ISO/IEC 17025. 

7.10 Nonconforming work 

The laboratory’s procedure shall ensure that when nonconforming work is identified and recalled for an

ONC-ACB (Authorized Certification Body) certified product that is listed on the Certified Health IT 

Product List (CHPL), the laboratory shall immediately notify NVLAP, ONC, and any associated 

certification bodies, as well as the developer, in writing. 

7.11 Control of data and information management 

There are no requirements additional to those set forth in ISO/IEC 17025. 

8 Management system requirements

8.1 Options

There are no requirements additional to those set forth in ISO/IEC 17025. 

NOTE No distinction is made during the assessment of laboratory management systems under §8.1.2 Option A or 

§8.1.3 Option B for conformance to the Health IT LAP; laboratory management systems under Option B must, at a 

minimum, meet all the requirements of Option A and this handbook. 

______________________________________________________________________________________________________ 

This publication is available free of charge from: https://doi.org/10.6028/NIST.HB.150-31-2021
NIST Handbook 150-31:2021 14

8.2 Management system documentation (Option A) 

The laboratory shall create a cross-reference document that facilitates verification that all program 

requirements, including scheme (regulatory) requirements, have been addressed by the management 

system, which includes clauses 4 through 8 of ISO./IEC 17025, annexes A, B, and E of NIST Handbook 

150, and NIST Handbook 150-31. 

8.3 Control of management system documents (Option A) 

There are no requirements additional to those set forth in ISO/IEC 17025. 

8.4 Control of records (Option A)

The laboratory shall retain all records as defined in 170.524(f) Records retention. 

8.5 Actions to address risks and opportunities (Option A) 

There are no requirements additional to those set forth in ISO/IEC 17025.

8.6 Improvement (Option A) 

There are no requirements additional to those set forth in ISO/IEC 17025. 

8.7 Corrective actions (Option A)

Should a nonconformity(s) be identified regarding a product tested within the laboratory, the laboratory 

shall initiate its corrective action process to investigate the validity of the test results issued. If further 

actions are warranted as a result of this investigation process (e.g., it was determined that the test results 

are not correct or the laboratory deviated from its testing process), those actions shall be taken in accordance 

with the laboratory’s management system.

8.8 Internal Audits (Option A) 

There are no requirements additional to those set forth in ISO/IEC 17025. 

8.9 Management Review (Option A) 

There are no requirements additional to those set forth in ISO/IEC 17025. 

______________________________________________________________________________________________________ 

This publication is available free of charge from: https://doi.org/10.6028/NIST.HB.150-31-2021
NIST Handbook 150-31:2021 15

Annex A

(informative)

Acronyms and abbreviations

The following acronyms and abbreviations are used throughout this handbook: 

CFR Code of Federal Regulations 

CHPL Certified Health Information Technology Products List 

EHR Electronic Health Record

HHS Department of Health and Human Services

Health IT Health Information Technology 

HITECH Health Information Technology for Economic and Clinical Health 

ILAC International Laboratory Accreditation Cooperation 

ISO International Organization for Standardization 

IT Information Technology 

LAP Laboratory Accreditation Program 

MRA Mutual/Multilateral Recognition Arrangement 

NIST National Institute of Standards and Technology 

NPRM Notice of Proposed Rulemaking 

NVLAP National Voluntary Laboratory Accreditation Program 

ONC Office of the National Coordinator for Health Information Technology 

RWT Real World Testing 

______________________________________________________________________________________________________ 

This publication is available free of charge from: https://doi.org/10.6028/NIST.HB.150-31-2021
NIST Handbook 150-31:2021 16

Annex B 

(normative)

Additional conditions for accreditation in the Health Information Technology 

(Health IT) Laboratory Accreditation Program (LAP)

The National Voluntary Laboratory Accreditation Program (NVLAP) established and is conducting its 

laboratory accreditation program for health information technology testing laboratories in support of the 

responsibilities for NVLAP-accredited testing under the final rule published in 45 CFR Part 170 dated 

January 7, 2011. 

Under the NVLAP Healthcare Information Technology Laboratory Accreditation Program (Health IT

LAP), NVLAP evaluates the competence of laboratories to test the services and/or products related to 

electronic health information (health IT) products and systems. In support of its regulatory program, the 

Office of the National Coordinator (ONC) has determined access to review the accreditation records for 

laboratories in this accreditation program is needed. 

As the Authorized Representative for ____________________________________(laboratory’s name), 

Lab Code ___________________, I grant NVLAP permission to the sharing of my application for 

accreditation information, as well as any records collected by NVLAP in support of my laboratory’s 

accreditation activities. The laboratory information shared by NVLAP with the ONC representatives will 

be limited to access to the laboratory’s information that is maintained at the NVLAP offices at NIST in 

accordance with NVLAP policies and procedures for record retention. 

Signature _______________________________________ Date ____________________ 

Printed Name ________________________________________________________________ 

______________________________________________________________________________________________________ 

This publication is available free of charge from: https://doi.org/10.6028/NIST.HB.150-31-2021