N I S T H A N D B O O K 1 5 0 - 3 1

2 0 1 1 E d i t i o n

National

Voluntary

Laboratory

Accreditation

Program

HEALTHCARE

INFORMATION

TECHNOLOGY

TESTING

Dana S. Leaman and Lawrence I. Knab

National Voluntary Laboratory Accreditation Program

Standards Coordination Office

November 2011

U.S. Department of Commerce

John Bryson, Secretary

National Institute of Standards and Technology

Patrick D. Gallagher, Director
NVLAP AND THE NVLAP LOGO

The term NVLAP and the NVLAP logo are registered 

marks of the Federal Government, which retains 

exclusive rights to control the use thereof. Permission 

to use the term and symbol (NVLAP logo with 

approved caption) is granted to NVLAP-accredited 

laboratories for the limited purpose of announcing 

their accredited status, and for use on reports that 

describe only testing and calibration within the scope 

of accreditation. NVLAP reserves the right to control 

the quality of the use of the NVLAP term, logo, and 

symbol.
DEPARTMENT OF COMMERCE

National Institute of Standards and Technology

National Voluntary Laboratory Accreditation Program

ISSUE DATE:

February 12, 2015

LAB BULLETIN

NUMBER: LB-84-2015

LAP: Healthcare IT LAP

SUBJECT: Addition of the 2014 Edition Release 2 Certification Criteria for EHR 

Technology to the HIT LAP

On September 11, 2014, the 2014 Edition Release 2 Electronic Health Record (EHR) 

Certification Criteria and the ONC HIT Certification Program; Regulatory Flexibilities, 

Improvements, and Enhanced Health Information Exchange; Final Rule was published. As 

stated by the Office of the National Coordinator (ONC), this rule “introduces regulatory 

flexibilities and general improvements for certification to the 2014 Edition EHR certification 

criteria (2014 Edition).” Approved Test Procedures regarding these revisions in criteria were 

released on January 26, 2015.

Based on this rule and the release of approved test procedures, NVLAP has updated its 

Healthcare Information Technology (HIT) Laboratory Accreditation Program (LAP) to include 

the additional 2014 Edition Release 2 EHR certification criteria for testing of EHR technology. 

For laboratories currently accredited in the NVLAP HIT LAP, addition of these methods to the 

scope of accreditation must be requested in writing (e-mail is a sufficient form of written 

request). With the written request, a signed self-declaration regarding implementation of the 

new test procedures within the laboratory’s management system (attached to this bulletin) is 

required. (Note: The declaration must be signed by the Laboratory’s Authorized Representative 

as designated in the NVLAP records.) Each applicant laboratory must also supply a detailed 

description of how this additional testing has been added into its management system and 

communicated to its testers. 

Once this information is received, NVLAP will proceed with a review of the information and a 

decision regarding the addition of the new criteria to the scope of accreditation. A revised scope 

of accreditation will be issued to each applicant that meets the requirements within the HIT 

program and this bulletin.

Questions should be directed to Dana S. Leaman at 301-975-4679, or dana.leaman@nist.gov.

Attachment:

Self-Declaration of Implementation

NVLAP LAB BULLETIN NUMBER: LB-84-2015 PAGE: 1 of 1
UNITED STATES DEPARTMENT OF COMMERCE 

National Institute of Standards and Technology 

Gaithersburg, Maryland 20899

NVLAP-Accredited Testing Laboratory

Healthcare Information Technology Laboratory Accreditation Program

Self-Declaration of Implementation:

45 CFR 170, Health Information Technology: 2014 Edition Release 2 Electronic Health Record 

(EHR) Certification Criteria and the ONC HIT Certification Program; Regulatory Flexibilities, 

Improvements, and Enhanced Health Information Exchange, Final Rule and its associated ONCapproved 2014 R2 Test Procedures, Test Data and Test Tools.

Laboratory Name: Lab Code:

As the laboratory’s Authorized Representative, I attest that this criteria and its associated test 

procedures, data and tools have been implemented into our laboratory’s quality management 

system. The appropriate training to conduct this testing to the 2014 R2 criteria has also been 

performed and documented as required in our system. These statements are correct to the 

best of my knowledge and are made in good faith.

Signature_______________________________________ Date______________________

Printed Name___________________________________

2015-02-09

NIST/NVLAP • 100 Bureau Drive, Stop 2140 • Gaithersburg, MD 20899-2140

http://www.nist.gov/nvlap
DEPARTMENT OF COMMERCE

National Institute of Standards and Technology

National Voluntary Laboratory Accreditation Program

ISSUE DATE:

November 22, 2013

LAB BULLETIN

NUMBER: LB-79-2013

LAP: Healthcare IT LAP

SUBJECT: Addition of Test Reporting Requirements to the HIT LAP

The purpose of this bulletin is to publish a minor revision to NIST Handbook 150-31 for the 

subclause noted below. This bulletin becomes a part of NIST Handbook 150-31, NVLAP 

Healthcare Information Technology Testing, until such time as the next edition of the handbook 

is published.

Overview and background

On September 4, 2012, the 2014 Edition Standards and Certification Criteria (S&CC) final rule 

was published outlining the Stage 2 of the Medicare and Medicaid Electronic Health Record 

(EHR) Incentive Programs, updates to Stage 1, and other program modifications. Under Section 

170.523(f)(8), publication of the test results used to make the certification decision is required. 

To ensure consistency among the Accredited Test Laboratories (ATLs), NVLAP, with input from 

the ATLs, supplied information to the Office of the National Coordinator (ONC) on a test report 

summary template.

In September 2013, ONC released the 2014 EHR Test Results Summary Template for immediate 

implementation and use. Based on the release of this template, NVLAP has updated its 

Healthcare Information Technology (HIT) Laboratory Accreditation Program (LAP) to include

the required use of the ONC-approved template for reporting test results.

Implementation of changes

Effective upon the issuance of this bulletin, the first paragraph of NIST Handbook 150-31, 

subclause 5.10.3 is revised as shown in the italicized text:

5.10.3 For test report information that is being generated by the testing laboratory for use by the 

certification body to make the certification decision, the laboratory shall use the ONC-approved 

test report summary template. The testing laboratory shall may also create a an additional test 

report template for its own use that includes other information necessary for reporting to describe 

the system/module being certified and shall ensure policies and procedures exist to meet the 

requirements of the vendor, the certification body, or the ONC.

The second paragraph of subclause 5.10.3 is deleted.

Questions regarding the changes to the NVLAP HIT LAP requirements should be directed to 

Dana Leaman, NVLAP Program Manager, at dana.leaman@nist.gov, or 301-975-4679.

NVLAP LAB BULLETIN NUMBER: LB-79-2013 PAGE: 1 of 1
NVLAP LAB BULLETIN NUMBER: LB-71-2013 PAGE: 1 of 2

DEPARTMENT OF COMMERCE

National Institute of Standards and Technology

National Voluntary Laboratory Accreditation Program

ISSUE DATE:

January 2, 2013

LAB BULLETIN

NUMBER: LB-71-2013

LAP: Healthcare IT LAP 

SUBJECT: Amendments to NIST Handbook 150-31, NVLAP Healthcare Information 

Technology Testing

The purpose of this NVLAP Lab Bulletin is to publish revisions to the Healthcare Information 

Technology laboratory accreditation program (HIT LAP) handbook to reflect the addition of the 2014 

certification criteria for EHR technology to the program. NVLAP is now accrediting testing applicants to 

the 2014 Edition Test Method. This is in support of the Office of the National Coordinator (ONC) 

requirements published in the Federal Register on September 4, 2012.

The following method has been added to the HIT LAP Test Method Selection List:

31/2014 Testing based on criteria in 45 CFR Part 170, Health Information Technology: Standards, 

Implementation Specifications, and Certification Criteria for Electronic Health Record 

Technology, 2014 Edition; Revisions to the Permanent Certification Program for Health 

Information Technology; Final Rule (September 4, 2012).

Effective immediately, NIST Handbook 150-31, NVLAP Healthcare Information Technology Testing,

sections 1.4, 5.2, and 5.3 are amended as follows:

1.4 References

1.4.1 Publications referenced in this handbook

*****

— “Health Information Technology: Standards, Implementation Specifications, and Certification 

Criteria for Electronic Health Record Technology, 2014 Edition; Revisions to the Permanent 

Certification Program for Health Information Technology; Final Rule,” Federal Register

(4 September 2012): 54163-54292 (codified at at 45 CFR Part 170); available online 

<http://www.gpo.gov/fdsys/pkg/FR-2012-09-04/pdf/2012-20982.pdf>.

— ONC-Approved 2014 Edition Test Method (effective December 14, 2012); available online at 

<http://www.healthit.gov/policy-researchers-implementers/2014-edition-final-test-method>.

5.2 Personnel

5.2.1 The testing laboratory shall maintain responsible personnel and competent technical staff 

who are knowledgeable of the following (see documents list in 1.4 for complete citations):

*****

— “Health Information Technology: Standards, Implementation Specifications, and Certification 

Criteria for Electronic Health Record Technology, 2014 Edition; Revisions to the Permanent 

Certification Program for Health Information Technology; Final Rule,” Federal Register
NVLAP LAB BULLETIN NUMBER: LB-71-2013 PAGE: 2 of 2

(4 September 2012): 54163-54292 (codified at at 45 CFR Part 170); available online 

<http://www.gpo.gov/fdsys/pkg/FR-2012-09-04/pdf/2012-20982.pdf>.

— ONC-Approved 2014 Edition Test Method (effective December 14, 2012); available online at 

<http://www.healthit.gov/policy-researchers-implementers/2014-edition-final-test-method>.

5.3 Accommodation and environmental conditions

*****

5.3.2 (second paragraph is revised to add the words, “where necessary.”)

*****

The testing laboratory shall ensure a secure electronic communication channel exists for remote 

testing to protect the confidentiality and integrity of the testing process, where necessary.

*****

This bulletin should be maintained with your copy of NIST Handbook 150-31 until the next edition of the 

handbook is released, at which time these changes will be incorporated into the handbook.

Questions regarding the changes to the HIT program should be directed to Dana S. Leaman, NVLAP 

Program Manager, at 301-975-4679, or dana.leaman@nist.gov.
NIST Handbook 150-31:2011 iii

Contents

Acknowledgments.........................................................................................................................................v

Foreword......................................................................................................................................................vi

Introduction.................................................................................................................................................vii

1 General information ...............................................................................................................................1

1.1 Scope.............................................................................................................................................1

1.2 Organization of handbook.............................................................................................................1

1.3 Program description ......................................................................................................................1

1.4 References.....................................................................................................................................2

1.5 Terms and definitions....................................................................................................................3

1.6 Program documentation ................................................................................................................3

2 LAP establishment, development and implementation ..........................................................................4

3 Accreditation process .............................................................................................................................5

3.1 General..........................................................................................................................................5

3.2 Application for accreditation ........................................................................................................5

3.3 Activities prior to initial on-site assessment .................................................................................6

3.4 On-site assessment ........................................................................................................................6

3.5 Proficiency testing ........................................................................................................................9

3.6 Suspension of accreditation ..........................................................................................................9

4 Management requirements for accreditation ........................................................................................ 10

4.1 Organization................................................................................................................................ 10

4.2 Management system.................................................................................................................... 10

4.3 Document control........................................................................................................................ 10

4.4 Review of requests, tenders and contracts .................................................................................. 10

4.5 Subcontracting of tests and calibrations...................................................................................... 10

4.6 Purchasing services and supplies................................................................................................ 10

4.7 Service to the customer............................................................................................................... 11

4.8 Complaints.................................................................................................................................. 11

4.9 Control of nonconforming testing and/or calibration work ........................................................ 11

4.10 Improvement............................................................................................................................... 11

4.11 Corrective action......................................................................................................................... 11

4.12 Preventive action......................................................................................................................... 11

4.13 Control of records ....................................................................................................................... 11

4.14 Internal audits.............................................................................................................................. 11

4.15 Management reviews .................................................................................................................. 11
iv NIST Handbook 150-31:2011

5 Technical requirements for accreditation ............................................................................................. 12

5.1 General........................................................................................................................................ 12

5.2 Personnel..................................................................................................................................... 12

5.3 Accommodation and environmental conditions.......................................................................... 13

5.4 Test and calibration methods and method validation.................................................................. 14

5.5 Equipment................................................................................................................................... 15

5.6 Measurement traceability............................................................................................................ 15

5.7 Sampling ..................................................................................................................................... 15

5.8 Handling of test and calibration items ........................................................................................ 15

5.9 Assuring the quality of test and calibration results..................................................................... 16

5.10 Reporting the results ................................................................................................................... 16

6 Additional requirements....................................................................................................................... 16

Annex A (informative) Acronyms and abbreviations................................................................................. 17
NIST Handbook 150-31:2011 v

Acknowledgments

The authors would like to thank the persons who reviewed and contributed to this document, especially 

the following individuals who are members of the healthcare information technology working group: 

Carol Bean and Asara Johnson of ONC/HHS; David Alderman, Lisa Carnahan, Ken Gebhart, Jeffrey

Horlick, and Amy Phelps of NIST.
vi NIST Handbook 150-31:2011

Foreword

The NIST Handbook 150 publication series sets forth the procedures, requirements, and guidance for the 

accreditation of testing and calibration laboratories by the National Voluntary Laboratory Accreditation 

Program (NVLAP). The series is comprised of the following publications:

NIST Handbook 150, NVLAP Procedures and General Requirements, which contains the general 

procedures and requirements under which NVLAP operates as an unbiased third-party accreditation 

body;

NIST Handbook 150-xx program-specific handbooks, which supplement NIST Handbook 150 by 

providing additional requirements, guidance, and interpretive information applicable to specific 

NVLAP laboratory accreditation programs (LAPs).

The program-specific handbooks are not stand-alone documents, but rather are companion documents to 

NIST Handbook 150. They tailor the general criteria found in NIST Handbook 150 to the specific tests, 

calibrations, or types of tests or calibrations covered by a LAP.

NIST Handbook 150-31, NVLAP Healthcare Information Technology Testing, presents the technical 

requirements and guidance for the accreditation of laboratories under the NVLAP Healthcare Information 

Technology (HIT) LAP. The handbook is intended for information and use by accredited laboratories, 

assessors conducting on-site visits, laboratories seeking accreditation, laboratory accreditation systems, 

users of laboratory services, and others needing information on the requirements for accreditation under 

this program. All statements in this handbook are meant to supplement NIST Handbook 150 and by no 

means contradict it. If any ambiguity unintentionally arises, the NIST Handbook 150 requirements are to 

be followed.

The 2011 edition of NIST Handbook 150-31 was developed with the participation of technical experts in 

the field of healthcare information technology testing and was approved by NVLAP. The handbook 

incorporates the information and requirements found in the latest editions of ISO/IEC 17025, General 

requirements for the competence of testing and calibration laboratories, NIST Handbook 150, and the 

test procedures associated with this program. The requirements of NIST Handbook 150, the 

interpretations and specific requirements in NIST Handbook 150-31, and the requirements set forth by the 

technical standards have been combined to produce the criteria for accreditation in the NVLAP HIT LAP.

This handbook is also available on the NVLAP web site, <http://www.nist.gov/nvlap>.

Questions or comments concerning this handbook should be submitted to NVLAP, National Institute of 

Standards and Technology, 100 Bureau Drive, Stop 2140, Gaithersburg, MD, 20899-2140; phone: 

301-975-4016; fax: 301-926-2884; e-mail: nvlap@nist.gov.
NIST Handbook 150-31:2011 vii

Introduction

NIST Handbook 150-31 augments NIST Handbook 150, NVLAP Procedures and General Requirements, 

by gathering the technical requirements of the Laboratory Accreditation Program (LAP) for functional 

and conformance testing of electronic health record products to nationally recognized meaningful use 

requirements for healthcare IT products. Technical requirements are explained to indicate how the 

NVLAP criteria are applied for accreditation under the HIT LAP.

Any domestic or foreign laboratory (including commercial; manufacturer; academic; and federal, state or 

local government laboratories) that performs test methods covered by the HIT LAP may apply for 

NVLAP accreditation. Accreditation will be granted to a laboratory that complies with the conditions for 

accreditation as defined in NIST Handbook 150. Accreditation does not imply a guarantee of laboratory 

performance or of test results; it is a finding of laboratory competence and proficiency in conducting 

testing.

The services and/or products related to electronic health information products and systems referred to 

within this accreditation program are defined by the “Health Information Technology Standards, 

Implementation Specifications, and Certification Criteria and Certification Programs for Health 

Information Technology,” Title 45 Code of Federal Regulations, Part 170, 2010 ed. (see 1.4.1).

NIST Handbook 150-31:2011 1

1 General information

1.1 Scope

1.1.1 This handbook specifies the technical requirements and provides guidance for the accreditation of 

laboratories under the NVLAP Healthcare Information Technology Testing (HIT) LAP. It supplements 

the NVLAP procedures and general requirements found in NIST Handbook 150, by tailoring the general 

criteria found in NIST Handbook 150 to the specific tests and/or types of tests covered by the HIT LAP.

1.1.2 NIST Handbook 150, this handbook, and the test procedures associated with this program

constitute the collective body of requirements that must be met by a laboratory seeking NVLAP 

accreditation for the HIT LAP.

1.1.3 Any interpretive comments and additional requirements contained in this handbook complement 

the general NVLAP criteria for specific application in the HIT LAP.

1.2 Organization of handbook

The numbering and titles of the first five clauses of this handbook are patterned after NIST Handbook 150 

to allow easy cross-reference. The primary subclauses in clauses 4 and 5 (e.g., 4.1, 4.2, etc.) are also 

numbered and titled to correspond with those of NIST Handbook 150, even when there are no additional 

requirements to those in NIST Handbook 150.

Annex A (informative) provides a list of acronyms and abbreviations used in this handbook.

1.3 Program description

1.3.1 In response to the Health Information Technology for Economic and Clinical Health (HITECH) 

Act of 2009, the U.S. Department of Health and Human Services along with the Office of the National 

Coordinator for Health Information Technology (ONC) requested establishment of the Healthcare 

Information Technology (HIT) Testing LAP by NVLAP to accredit laboratories that perform functional 

and conformance testing of electronic health record (EHR) technology products to meaningful use 

requirements as defined in the nationally recognized EHR products testing standards. See 1.4, 

References, for a complete list of the currently accepted standards.

1.3.2 NVLAP reserves the right to expand the HIT LAP and offer to interested laboratories additional 

test methods not listed in this handbook. Laboratories are advised to review the HIT LAP’s website for 

the most current information, <http://www.nist.gov/nvlap/hit-lap.cfm>.

1.3.3 The HIT LAP offers a set of test procedures for accreditation. Depending on the breadth of its 

testing capabilities, the applicant laboratory may select test(s) from the list of offered test procedures. 

For additional information regarding the test procedures available for selection, refer to the Test 

Procedure Selection List available on the HIT LAP website, <http://www.nist.gov/nvlap/hit-lap.cfm>.
2 NIST Handbook 150-31:2011

1.4 References

1.4.1 Publications referenced in this handbook

The following documents are referenced in this handbook. For dated references, only the edition cited 

applies. For undated references, the latest edition of the referenced document (including any amendments) 

shall apply within one year of publication or within another time limit specified by regulations or other 

requirement documents.

— “Certified Health IT Product List (CHPL),” Office of the National Coordinator for Health 

Information Technology; available online at <http://healthit.hhs.gov/chpl>.

— “Establishment of the Permanent Certification Program for Health Information Technology; Final 

Rule,” Federal Register (7 January 2011): 1262-1331 (to be codified at 45 CFR Part 170); available 

online at <http://www.gpo.gov/fdsys/pkg/FR-2011-01-07/pdf/2010-33174.pdf>.

— “Health Information Technology: Initial Set of Standards, Implementation Specifications, and 

Certification Criteria for Electronic Health Record Technology; Final Rule,” Federal Register

(28 July 2010): 44590-44654 (codified at 45 CFR Part 170); available online at 

<http://www.gpo.gov/fdsys/pkg/FR-2010-07-28/pdf/2010-17210.pdf>.

— “Health Information Technology Standards, Implementation Specifications, and Certification Criteria 

and Certification Programs for Health Information Technology,” Title 45 Code of Federal 

Regulations, Part 170 (2010 ed.): 873-890; available online at <http://www.gpo.gov/fdsys/pkg/CFR2010-title45-vol1/pdf/CFR-2010-title45-vol1-part170.pdf>.

— “Medicare and Medicaid Programs; Electronic Health Record Incentive Program; Final Rule,” 

Federal Register (28 July 2010): 44314-44588 (codified at 42 CFR Parts 412, 413, 422, and 495); 

available online at <http://www.gpo.gov/fdsys/pkg/FR-2010-07-28/pdf/2010-17207.pdf>.

— NIST Handbook 150: 2006, “NVLAP Procedures and General Requirements”; available online at

<http://www.nist.gov/nvlap>.

— ONC-Approved Test Method, “Approved Test Procedures Version 1.1,” with each associated 

erratum, where applicable (effective October 24, 2010); available online at 

<http://healthit.hhs.gov/portal/server.pt/community/healthit_hhs_gov__certification_program/2884>. 

(Click on NIST link under Additional Information.)

1.4.2 Informative references

The following documents contain information intended to assist the reader with the understanding and use 

of this handbook.

— “Electronic Health Records and Meaningful Use,” Office of the National Coordinator for Health 

Information Technology; available online at 

<http://healthit.hhs.gov/portal/server.pt?open=512&objID=2996&mode=2>.

— “Proposed Establishment of Certification Programs for Health Information Technology; Proposed 

Rule,” Federal Register (10 March 2010): 11328-11373; available online at 

<http://www.gpo.gov/fdsys/pkg/FR-2010-03-10/pdf/2010-4991.pdf>.
NIST Handbook 150-31:2011 3

1.5 Terms and definitions

For the purposes of this handbook, the terms and definitions given in NIST Handbook 150 apply unless a 

term is redefined in this handbook. The definitions provided in this handbook are specific to the HIT 

LAP, and when applicable, they supersede the definitions given in NIST Handbook 150. For a list of

acronyms, see Annex A.

1.5.1

test method

The set of test procedures, test data, and test tools to ensure compliance with the meaningful use technical 

requirements and standards as approved by ONC.

1.5.2

test procedure

The document used to determine conformance to a specific criterion within the ONC Rule. 

1.5.3

test tool 

A software tool used to perform automated testing of specific functionality.

1.6 Program documentation

1.6.1 General

This handbook details the HIT-program-specific requirements and technical procedures, while 

interpreting, detailing and expanding portions of NIST Handbook 150 for HIT LAP use. Both the NIST 

Handbook 150 checklist and the NIST Handbook 150-31 checklist are used in conducting an assessment

in the HIT LAP. Assessor use of the NVLAP checklists is to ensure that each laboratory receives an 

assessment comparable to that received by other laboratories. Checklists assist assessor(s) in 

documenting the assessment to the NVLAP requirements found in NIST Handbook 150 and in this 

handbook. Checklists contain definitive statements or questions about all aspects of the NVLAP criteria 

for accreditation, and form part of the On-Site Assessment Report (see NIST Handbook 150). The current 

version of each checklist is available upon request or on the NVLAP website,

<http://www.nist.gov/nvlap>.

1.6.2 NIST Handbook 150 Checklist

All NVLAP programs use the NIST Handbook 150 Checklist, which contains the requirements published 

in NIST Handbook 150. The checklist items are numbered to correspond to clauses 4 and 5 and annexes 

A and B of NIST Handbook 150. The current version of the checklist is available from the NVLAP 

website, <http://www.nist.gov/nvlap>

1.6.3 NIST Handbook 150-31 Checklist

The NIST Handbook 150-31 Checklist (also referred to as the HIT Program-Specific Checklist) addresses 

the requirements specific to Healthcare Information Technology testing given in NIST Handbook 150-31. 

The checklist contains the requirements provided in this handbook, including testing requirements and 

additional details and notes for the assessor(s) (e.g., the names of the key personnel), with an emphasis on 
4 NIST Handbook 150-31:2011

observing and performing tests, testing accuracy, instrumentation, calibration, personnel competency, and 

test reporting. The current version of the checklist is available from the HIT LAP website,

<http://www.nist.gov/nvlap/nvlap-checklists.cfm>.

1.6.4 HIT Template for Oral Quizzing

The assessor(s) use(s) the HIT Template for Oral Quizzing as a means to document the information 

gathered during the oral quizzing conducted during the on-site assessment. The template captures the 

questions asked, the laboratory personnel participating in the quiz and any assessor(s) comments 

regarding the responses provided by the laboratory personnel. The current version of the checklist is 

available from the NVLAP LAP website, <http://www.nist.gov/nvlap/nvlap-checklists.cfm>.

1.6.5 NVLAP Lab Bulletins

NVLAP Lab Bulletins are issued to laboratories and assessors, when needed, to clarify program-specific 

requirements and to provide information about program additions and changes.

2 LAP establishment, development and implementation

2.1 Basis for establishment

There are no requirements additional to those set forth in NIST Handbook 150.

2.2 Development of technical requirements

All technical requirements mandated for a laboratory under accreditation tailor the requirements discussed 

in clauses 4 and 5 which are derived from the elected scope of accreditation and associated test 

procedures for which a candidate requests accreditation.

2.3 Announcing the establishment of a LAP

There are no requirements additional to those set forth in NIST Handbook 150.

2.4 Adding to or modifying a LAP

Upon identifying the need for additional tests or test types, NVLAP reserves the right to add or modify 

the HIT LAP either by adding new subsidiary programs or new test procedures to existing programs, or 

modifying the existing test procedures. All changes will be published in a timely manner in a NVLAP 

Lab Bulletin and will be reflected on the NVLAP website, <http://www.nist.gov/nvlap>.

2.5 Termination of a LAP

There are no requirements additional to those set forth in NIST Handbook 150.
NIST Handbook 150-31:2011 5

3 Accreditation process

3.1 General

An overview of the laboratory accreditation process is provided in NIST Handbook 150, clause 3, and 

includes general information pertaining to application for accreditation; activities prior to on-site 

assessment; on-site assessment; proficiency testing; accreditation decision; granting accreditation; 

renewal of accreditation; changes to scope of accreditation; monitoring visits; suspension, denial, and 

revocation of accreditation; voluntary termination of accreditation; and appeals.

The flowchart in Figure 1 describes the accreditation process for a laboratory seeking accreditation for the 

HIT LAP.

Figure 1. Accreditation process flowchart.

3.2 Application for accreditation

3.2.1 The accreditation process begins with the submission of the laboratory’s application, including 

supporting documents, and payment of fees. A laboratory interested in accreditation for any type of test

offered under the HIT LAP shall review and become familiar with the requirements listed in NIST 

Handbook 150 and this handbook, review the HIT LAP website at <http://www.nist.gov/nvlap/hitlap.cfm>, and contact NVLAP for the most current updates on the requirements and application process.

3.2.2 Prior to applying to NVLAP, the laboratory shall have a fully implemented management system. 

The quality manual and related documentation shall contain or refer to documentation that describes and 

details the implementation of procedures covering all of the technical requirements in NIST Handbook 

150 and this handbook. A copy of the quality manual and related documentation shall be sent to NVLAP 

with the application forms.

See 3.3.1 a)

See 3.3.1 b)

See 3.4.3
6 NIST Handbook 150-31:2011

3.3 Activities prior to initial on-site assessment

3.3.1 Once NVLAP determines that an application is complete, the next steps are evaluation of the 

laboratory’s quality manual and associated documentation and administration of the proficiency 

written/oral exam.

a) Quality manual evaluation

NVLAP reviews the laboratory quality manual and associated documentation and determines whether 

the management system meets the requirements. Nonconformities and recommendations for 

management system enhancements will be discussed during the on-site assessment. However, if the 

quality manual and documentation are evaluated as unsatisfactory, the on-site assessment will be 

postponed.

b) Proficiency written and/or oral exam

For an initial accreditation, when it is determined that the quality manual meets the minimum 

requirements of NIST Handbook 150 and this handbook, a written exam may be provided to the 

applicant laboratory depending upon the intended scope of accreditation. This exam evaluates the 

laboratory personnel’s technical expertise and knowledge of the standards and test procedure(s) 

applicable to the scope of accreditation for which the laboratory is applying. In some instances, an 

oral exam may be necessary to demonstrate proficiency. A technical assessor(s) and/or expert(s) 

from the associated technical program conduct(s) this exam via a teleconference with the laboratory 

personnel prior to the on-site assessment.

3.3.2 It is important to note that a laboratory applying for initial accreditation cannot proceed to the onsite assessment phase of the accreditation process until successful completion of the quality manual 

evaluation and the proficiency written and/or oral exam. For all applicants, the on-site assessment is not 

scheduled until it is determined that the management system meets the requirements found in NIST 

Handbook 150 and this handbook.

3.4 On-site assessment

3.4.1 General

3.4.1.1 The on-site assessment is scheduled by NVLAP at a mutually agreed-upon date and time at the 

laboratory facility. The time span for the assessment is dependent upon the applicant’s scope of 

accreditation. Typically, the assessment will span about two to three days and will be performed by two 

or more NVLAP assessors. All observations made by the assessor(s) during the assessment are held in 

the strictest confidence.

3.4.1.2 In addition to the NIST Handbook 150 checklist, the assessor(s) will use the HIT ProgramSpecific Checklist (NIST Handbook 150-31 Checklist), which is derived from the technical criteria 

contained in this handbook.

3.4.1.3 Additionally, the assigned assessor(s) shall evaluate any new or updated requirements that are 

documented in a NVLAP Lab Bulletin and on the HIT LAP’s website, <http://www.nist.gov/nvlap/hitlap.cfm>, but are not yet incorporated into the checklist.
NIST Handbook 150-31:2011 7

3.4.2 On-site assessment activities

The on-site assessment activities include:

a) a proficiency/round-table quiz (see 3.4.3);

b) an evaluation of the laboratory staff’s understanding of and competence to apply the HIT 

conformance testing methodology;

c) an evaluation of the exercised management system and associated records of all management system 

activities;

d) a demonstration, for the selected test(s), that the required set of tools and test procedures are 

available and the testing environment is adequate (e.g., space, security, separation, and storage); and

e) a demonstration, if required, of the competence of the laboratory staff to prepare and use test tools, 

which will include loading, configuring and running the tools; preparing the test reports; and 

performing updates, if necessary.

3.4.3 On-site assessment and proficiency/round-table quiz

During the on-site visit, the laboratory’s personnel will be quizzed and team dynamics observed for 

proficiency and expertise in the technical area for which the laboratory is applying for accreditation. Staff 

member interaction and knowledge distribution among team members are key factors that will be 

monitored by the technical assessor(s) and/or technical expert(s). The laboratory staff shall provide 

satisfactory responses that demonstrate sufficient knowledge to support competence for the scope of 

accreditation.

3.4.4 Agenda

The agenda for a typical on-site assessment is given below.

a) Opening meeting: During the on-site visit, the assessor(s) conduct(s) an entry briefing with 

laboratory management and supervisory personnel to explain the purpose of the on-site assessment 

and to discuss the schedule for the assessment activities. Information provided by the laboratory on 

the accreditation application form may be discussed during this meeting. At the discretion of the 

laboratory manager, other staff may attend this meeting.

b) Staff interviews, discussions, quizzes: The assessor(s) will ask the laboratory manager to assist in 

arranging times for individual interviews with laboratory staff members and/or proficiency/roundtable quizzes of staff. While it may not be necessary for the assessor to talk to all staff members if 

individual interviews are requested, he/she may select staff members representing all different 

aspects of the laboratory. If proficiency/round-table quizzes are to be conducted on-site, all members 

of the relevant staff shall be scheduled to participate. Also, after the completion of the round-table 

quizzing and/or individual interviews, further interviews may be requested. 

c) Records review: During the on-site visit, the assessor(s) will also review the laboratory’s 

documentation, including, but not limited to:

• organizational structure;
8 NIST Handbook 150-31:2011

• contract review records; 

• purchasing records;

• equipment and maintenance records; 

• laboratory test records/reports; 

• personnel competency evaluation records; 

• personnel training records including, but not limited to, training plans, areas of training, and 

training materials; and

• version of the test tools and/or other test program-specific software.

Laboratory staff shall be available to answer questions; however, the assessor(s) may wish to review 

the documents alone. Under some circumstances, the assessor(s) may remove some documents from 

the laboratory during the assessment. Specifically, the assessor(s) may remove for review documents 

related to the quality system, such as a revised quality manual or new procedures. The material will 

be returned or destroyed at the laboratory’s direction.

The assessor(s) will check personnel information for job descriptions, resumes, training records and 

technical performance reviews. The assessor(s) shall not be given information which violates 

individual privacy such as salary, medical information, or individual performance reviews outside 

the scope of the laboratory’s accreditation. At the discretion of the laboratory, a member of its 

human resources department (or equivalent) may be present during the review of personnel 

information.

d) Internal audit and management review: The laboratory shall perform a complete internal audit and 

management review of its management system and related management system records prior to the 

full on-site assessment visit. The assessor(s) will review and discuss the laboratory’s internal audit 

and management review activities with the laboratory staff. The discussion will include all aspects of 

those activities including the management system procedures, the audit findings, the results of the 

management review, and the actions taken to resolve any problems identified.

e) Equipment: The assessor(s) will examine test procedure-specific computer hardware, software, 

supporting test equipment, and facilities for appropriateness, capability, adherence to specifications, 

etc.

f) Laboratory walk-through: The assessor(s) will inspect the laboratory in the following areas during a 

walk-through: 

• physical layout of the laboratory including entrance and exit points;

• all test equipment and tools, including computer hardware, servers used for records retention and 

physical storage area;

• work environment in regard to providing adequate testing work space (including adequate 

separation of work activities as appropriate or by programmatic requirement), heating, lighting, 

etc.; and

• physical security including access control procedures and records. 

g) Proficiency evaluations: Although the written examination is provided prior to the initial on-site 

assessment, the group round-table quizzes and individual demonstrations conducted during the initial 

and renewal on-site assessments are considered part of the proficiency evaluations.
NIST Handbook 150-31:2011 9

h) Closing meeting: At the end of the on-site visit, a closing meeting is held with the laboratory 

manager and staff to discuss any nonconformities documented by the assessor(s) during the visit. 

See NIST Handbook 150, 3.3.3 for more information regarding the assessment report, 

nonconformities, and the final resolution.

3.4.5 On-site assessment report 

The assessor(s) completes the On-Site Assessment Report that summarizes the findings. Copies of the 

completed checklists are attached to the report at the closing meeting. The report is signed by the 

assessor(s) and the laboratory’s Authorized Representative. The original report and checklists are 

forwarded to NVLAP as required by NIST Handbook 150, 3.3.2.3. A copy of the report and a copy of the 

checklists are given to the laboratory representative for retention. The decision to grant or renew 

accreditation is not made by the assessor team but is made by NVLAP in accordance with the procedures 

described in NIST Handbook 150.

3.4.6 Nonconformities, comments, and recommendations

3.4.6.1 A nonconformity that has been corrected during the on-site assessment by the laboratory using its 

corrective action process and any recommendations will be specifically noted on the on-site assessment 

report by the assessor(s). The assessor(s) will also note how the nonconformity was resolved and attach a 

copy of the objective evidence supporting the actions taken.

3.4.6.2 Comments in the report should be given serious consideration by the laboratory, but no action is 

mandated and changes are made at the laboratory’s discretion. Comments are those areas of concern 

where a nonconformity may arise; however, no objective evidence is available to support citing a 

nonconformity. Historically, it has been noted that comments often rise to the level of nonconformities on 

subsequent assessments. As such, comments noted in the assessment will be reviewed at the next on-site 

assessment to ensure that these issues have not risen to the level of nonconformities since the previous onsite visit.

3.4.6.3 Positive feedback will also be recorded on the on-site assessment report.

3.4.6.4 Upon completion of the on-site assessment and corrective action responses to nonconformities, if 

any, the accreditation process ends with NVLAP’s decision regarding the laboratory’s accreditation.

3.5 Proficiency testing 

Proficiency testing for this program may include written examinations, oral examinations, and evaluation 

of artifacts. The laboratory will be informed of proficiency testing schedules.

3.6 Suspension of accreditation

3.6.1 Failure to appropriately address and resolve complaints from customers or other interested parties 

may result in a NVLAP surveillance activity and/or suspension or revocation of accreditation.

3.6.2 Significant changes in a laboratory’s key technical personnel or facilities may result in a NVLAP 

monitoring visit(s), and/or suspension of accreditation of the affected test procedure(s) on the scope of 

accreditation if the new personnel prove inadequately prepared or unsuited for the job or if the facilities 

are inadequate to support the testing. Loss of key personnel without immediate adequate replacement may 
10 NIST Handbook 150-31:2011

result in suspension of the laboratory’s accreditation for the test procedure(s) affected by the loss of key 

personnel.

3.6.3 If the laboratory does not demonstrate continued competence to perform HIT conformance 

testing, NVLAP may suspend or revoke the laboratory’s accreditation. 

All issues surrounding the need to suspend and/or revoke a laboratory’s accreditation are reviewed on a 

case-by-case basis by NVLAP.

4 Management requirements for accreditation

4.1 Organization

If any services are offered by the laboratory other than the testing defined in the HIT scope of 

accreditation, the laboratory shall have a policy and procedure for maintaining separation of those 

services from its testing activities. The procedure shall demonstrate how separation is maintained. As an 

important example, if testing and certification are conducted in the same organization, the organization 

shall develop and implement policies and procedures to maintain separation of those functions.

4.2 Management system

The laboratory shall create and maintain a cross reference document mapping clauses 4 and 5, annexes A 

and B of Handbook 150 and clauses 4 and 5 of NIST Handbook 150-31 to the laboratory’s management 

system documentation.

4.3 Document control

There are no requirements additional to those set forth in NIST Handbook 150.

4.4 Review of requests, tenders and contracts

There are no requirements additional to those set forth in NIST Handbook 150.

4.5 Subcontracting of tests and calibrations

Subcontracting of tests is the use of laboratory services outside of the HIT laboratory’s management 

system to perform tests. When unforeseen circumstances occur, any subcontracted tests (from the 

laboratory’s scope of accreditation) shall be performed by a laboratory accredited under the NVLAP HIT 

LAP and recognized by the ONC.

4.6 Purchasing services and supplies

There are no requirements additional to those set forth in NIST Handbook 150.
NIST Handbook 150-31:2011 11

4.7 Service to the customer

There are no requirements additional to those set forth in NIST Handbook 150.

4.8 Complaints

The laboratory shall maintain a log of all complaints received regarding its testing activities. The log 

shall include information regarding the content of the complaint as well as activities for resolution of the 

complaint.

This log shall be provided to NVLAP at least on an annual basis or upon request.

4.9 Control of nonconforming testing and/or calibration work

If any nonconforming work is identified and recalled for an approved product listed on the CHPL, the 

laboratory shall immediately notify NVLAP, ONC, and any associated certification body(s), as well as 

the vendor, in writing.

4.10 Improvement

There are no requirements additional to those set forth in NIST Handbook 150.

4.11 Corrective action

There are no requirements additional to those set forth in NIST Handbook 150.

4.12 Preventive action

There are no requirements additional to those set forth in NIST Handbook 150.

4.13 Control of records

All records shall be maintained for a minimum of five years, unless other regulatory requirements specify 

a longer retention period.

4.14 Internal audits

There are no requirements additional to those set forth in NIST Handbook 150.

4.15 Management reviews

There are no requirements additional to those set forth in NIST Handbook 150.
12 NIST Handbook 150-31:2011

5 Technical requirements for accreditation

5.1 General

The quality manual shall contain or refer to documentation that describes and details the testing 

laboratory’s implementation of the procedures covering all of the technical requirements in this 

handbook.

5.2 Personnel

5.2.1 The testing laboratory shall maintain responsible personnel and competent technical staff who are 

knowledgeable of the following (see documents list in 1.4 for complete citations):

• “Health Information Technology Standards, Implementation Specifications, and Certification 

Criteria and Certification Programs for Health Information Technology,” 45 CFR Part 170;

• “Medicare and Medicaid Programs; Electronic Health Record Incentive Program; Final 

Rule,” Federal Register (28 July 2010): 44314-44588 (codified at 42 CFR Parts 412, 413, 

422, and 495);

• health IT standards including security, messaging, ePrescribing, quality, content-related 

standards as well as vocabularies and code sets found in sections §170.205, §170.207, and 

§170.210 of 45 CFR Part 170;

• health IT terminology, ONC certification criteria as well as security and privacy requirements 

found in sections §170.302, §170.304, and §170.306 of 45 CFR Part 170;

• ONC-Approved Test Method;

• using testing tools per the ONC-Approved Test Method; and

• “Establishment of the Permanent Certification Program for Health Information Technology; 

Final Rule,” Federal Register (7 January 2011): 1262-1331 (to be codified at 

45 CFR Part 170).

5.2.2 The laboratory’s training program shall be relevant to health IT testing, health IT standards and 

technologies, and events relevant to health IT and electronic health record (EHR) testing. 

The laboratory shall have a detailed, documented description of its training program for new and current 

staff members. Current staff members shall receive additional training when test procedures are modified 

or developed, when responsibilities have changed, or when technical requirements within the certification 

criteria have been modified or developed. The training shall include applying the new test procedures and 

performing required tests. The training shall be conducted through either on-the-job training, formal 

classroom training, or another appropriate training mechanism.

The testing laboratory shall ensure adequate training for the laboratory staff as identified below. The 

personnel shall possess knowledge of, or be trained prior to accreditation in the following areas (see 

documents list in 1.4 for complete citations):

• requirements of the ONC-Approved Test Method;

• health IT standards and interoperability concepts in accordance with 45 CFR Part 170;
NIST Handbook 150-31:2011 13

• health IT security and privacy concepts and requirements in accordance with 45 CFR 

Part 170;

• familiarity with health IT terminology in accordance with 45 CFR Part 170;

• health IT standards found in sections §170.205, §170.207, and §170.210 of 45 CFR Part 170; 

and

• operation of test tools found in the ONC-Approved Test Method.

5.2.3 The testing laboratory shall maintain a list of the key personnel designated to satisfy the technical 

requirements within this document, including their assigned roles and a brief summary of their latest 

training qualifications. The list shall include, but is not limited to:

• Authorized Representative;

• Laboratory Director;

• Approved Signatories; and

• key technical personnel in the laboratory (including team leaders and testers).

All testing laboratory staff having an effect on the outcome of testing shall be treated as personnel

regardless of their employment status. This includes, but is not limited to, full-time employees, 

temporary employees, and contracted subject matter experts.

5.2.4 The testing laboratory shall identify a staff member as quality manager with overall responsibility 

for quality assurance and maintenance of the quality manual. An individual may be assigned to serve in 

more than one position; however, to the extent possible, the laboratory director and the quality manager 

positions should be independently staffed.

The quality manager shall be knowledgeable in all aspects of ISO/IEC 17025. 

5.2.5 The testing laboratory shall have staff members with at least a bachelor’s degree in computer 

science, information systems or similar technical discipline or equivalent experience – such as three years 

experience – in the area of health IT testing, health IT interoperability, health IT standards and 

technologies, and events relevant to health IT.

The laboratory shall have a competency review program and procedures for the evaluation and 

maintenance of competency of each staff member for the specific test procedures the staff member is 

authorized to conduct. The evaluation shall be conducted annually by the immediate supervisor or 

designee appointed by the laboratory director. A record of the evaluation shall be dated and signed by the 

employee.

Changes to key personnel shall be reported to NVLAP within 30 days. Notification of a personnel change 

shall include an up-to-date copy of the person’s resume. NVLAP reserves the right to require a 

reassessment if considered necessary.

5.3 Accommodation and environmental conditions

5.3.1 The testing laboratory shall have adequate facilities to meet the requirements for accreditation. 

This includes facilities for conformance testing, record-keeping, document storage, and software storage. 
14 NIST Handbook 150-31:2011

If a testing laboratory conducts conformance testing at the customer site or other locations outside of the 

laboratory facility, the environment shall conform, as appropriate, to the requirements for the laboratory 

site. The laboratory shall have a policy and procedure regarding any conformance testing conducted 

outside of the laboratory facility.

5.3.2 The testing laboratory shall provide a secure environment capable of safeguarding proprietary 

software, test data, electronic and paper records, and other materials. This environment/system shall 

protect all proprietary materials and information from laboratory personnel not authorized to perform 

conformance testing and result reporting, and/or visitors to the laboratory.

The testing laboratory shall ensure a secure electronic communication channel exists for remote testing to 

protect the confidentiality and integrity of the testing process.

The testing laboratory shall have its internal networks protected from unauthorized access by external 

entities, as well as protection against malicious software, worms, viruses, etc.

5.3.3 If the testing laboratory is conducting multiple simultaneous tests, a process of total separation of 

products from different customers and conformance testing activities shall be maintained.

5.3.4 The testing laboratory shall have Internet access for obtaining the most current documentation 

and test tools from the ONC certification program and secure e-mail capabilities for communication with 

the ONC certification program, the certification body, NVLAP, and the laboratory’s customers.

The testing laboratory shall ensure that, where applicable, the correct version of the test tools per the 

ONC-Approved Test Method are used and that the tools have not been altered in any way that might lead 

to incorrect results.

The testing laboratory shall have policies and procedures to reset the system under test to a prior known 

state.

5.4 Test and calibration methods and method validation

5.4.1 Tests may be conducted at the testing laboratory or other mutually agreed upon site. When 

testing is performed outside the laboratory, all requirements pertaining to the test environment shall apply. 

The personnel of the recognized testing laboratory shall conduct the tests and record the results including 

the loading, compiling, configuring, and execution of any of the mandated testing tools.

5.4.2 A laboratory shall use the ONC-Approved Test Method applicable to the test procedures listed on 

its scope of accreditation along with any associated ONC-Approved testing tool(s).

5.4.3 For electronic health record testing, traceability is interpreted to mean that the testing tools shall 

be traceable back to the underlying requirements of the normative health IT standards found in sections 

§170.205, §170.207, and §170.210 of 45 CFR Part 170. Each test procedure is traceable to a specific 

electronic health record test criterion listed in accordance with 45 CFR Part 170, and the test procedures 

are achieved via the assertions and associated derived test requirements in the testing tools in use.

The testing laboratory shall ensure procedures and instructions are in place to trace localized test 

procedures back to the ONC-Approved Test Method.
NIST Handbook 150-31:2011 15

5.4.4 Testing laboratories shall use the test methods described in the program’s specific derived testing 

requirements. The testing laboratory shall have policies and procedures for exceptions that are deemed 

necessary for technical reasons, such as departures from the test data. When exceptions are deemed 

necessary, the customer and the certification body shall be informed and details shall be described in the 

test report.

5.5 Equipment

5.5.1 For conformance testing, the testing laboratory shall load and run a copy of the testing tool(s) and 

produce test results using the tool(s), as appropriate.

5.5.2 Records shall be maintained of each item of equipment and testing tool(s) significant to the tests 

performed. The records shall include:

• the identity of the item of equipment and testing tool(s);

• testing tool name, type, and version number or other unique identification;

• checks that the equipment complies with the specifications;

• the current location, where appropriate; and

• the instructions or reference to their location.

5.5.3 Whenever updates are made to any testing tool, the testing laboratory shall have procedures to 

assure the accurate execution and correct performance of the test tool. 

5.5.4 The testing laboratory shall document and follow appropriate procedures whenever a test tool is 

suspected to contain errors. These procedures include establishing that there is a genuine error, reporting 

the error to the appropriate maintenance authority, and withdrawing the test tool or test case(s) from 

service. If the conformance testing results differ from the original testing results for the system under test 

after correcting the test tool, the information shall be transmitted to the customer and certification body.

5.6 Measurement traceability

Testing laboratories shall ensure that any instantiation, enhancements, or modifications to the test tools 

are documented and traceable back to the ONC-Approved Test Method. Testing laboratories that locally 

instantiate the ONC-Approved Test Method shall have documented traceability back to the ONCApproved Test Method.

5.7 Sampling

Testing laboratories shall ensure that input test data meet the functional and interoperable requirements 

identified in the certification criteria and can be adequately evaluated for conformance. Testing 

laboratories shall document the specific vendor-supplied test data utilized for testing, when applicable.

5.8 Handling of test and calibration items

5.8.1 Testing laboratories shall protect all products under testing and test tools from modifications of 

any kind.
16 NIST Handbook 150-31:2011

5.8.2 Before the testing laboratory begins conducting a test, the laboratory shall ensure the ONCApproved Test Method and any associated testing tool(s) have not been corrupted, the test data is correct, 

and that the laboratory is using the appropriate tool. 

5.8.3 The testing laboratory shall ensure that a configuration management plan is in place for the 

system under test to prevent inadvertent modifications. This configuration management shall uniquely 

identify each system under test, as well as control and document modifications to any of the software 

components.

5.9 Assuring the quality of test and calibration results

There are no requirements additional to those set forth in NIST Handbook 150.

5.10 Reporting the results

5.10.1 The testing laboratory shall issue test reports which accurately, clearly, and unambiguously 

present the test conditions and the test setup when they vary from the standard protocol. Any deviations 

from the standard protocol shall be clearly indicated. 

5.10.2 Whenever test procedures are such that an analysis of the observations by the testing staff is 

required in order to interpret the results before stating them in a test report, the testing laboratory shall 

have objective procedures to be followed by the test operators performing the analysis, sufficient to 

ensure that the repeatability, reproducibility, and objectivity of the test results can be maintained. 

5.10.3 The testing laboratory shall create a test report template that includes information necessary to 

describe the system/module being certified and shall ensure policies and procedures exist to meet the 

requirements of the certification body or the ONC.

Test report information collected during testing of each test procedure shall be maintained by the 

laboratory. The outcome of each test procedure shall be contained in the test report.

5.10.4 A testing laboratory may submit either a printed or an electronic report as instructed by the 

certification body. The electronic version shall have the same content as the printed report and shall be 

generated using a software application that is acceptable to the certification body. A controlled copy of 

the report shall be placed in the testing laboratory’s records. 

5.10.5 The testing laboratory shall maintain a policy for handling interpretations of test results.

5.10.6 For test reports created for validation purposes and submitted to the certification body, the testing 

laboratory shall issue corrections or additions to a test report only by a supplementary document that is 

suitably marked and that meets the requirements of the test method.

6 Additional requirements

There are no additional requirements beyond NIST Handbook 150 and its associated normative annexes, 

and any other normative references cited in this handbook.
NIST Handbook 150-31:2011 17

Annex A

(informative)

Acronyms and abbreviations

The following acronyms and abbreviations are used throughout this handbook:

CFR Code of Federal Regulations

CHP Certified Health Information Technology Products List

EHR Electronic Health Record

HHS Department of Health and Human Services

HIT Healthcare Information Technology

HITECH Health Information Technology for Economic and Clinical Health

ILAC International Laboratory Accreditation Cooperation

ISO International Organization for Standardization

IT Information Technology

LAP Laboratory Accreditation Program

MRA Mutual/Multilateral Recognition Arrangement

NIST National Institute of Standards and Technology

NPRM Notice of Proposed Rulemaking

NVLAP National Voluntary Laboratory Accreditation Program

ONC Office of the National Coordinator for Health Information Technology