United States Government Accountability Office

GAO Report to the Chairman, Committee on 

House Administration, House of 

Representatives 

ELECTIONS

Federal Programs for 

Accrediting 

Laboratories That Test 

Voting Systems Need 

to Be Better Defined 

and Implemented 

September 2008 

 

GAO-08-770 
What GAO Found

United States Government Accountability Office

Why GAO Did This Study

Highlights

Accountability Integrity Reliability

September 2008

ELECTIONS

Federal Programs for Accrediting Laboratories That 

Test Voting Systems Need to Be Better Defined and 

Highlights of GAO-08-770, a report to the Implemented 

Chairman, Committee on House 

Administration, House of Representatives 

The 2002 Help America Vote Act

(HAVA) created the Election

Assistance Commission (EAC) and 

assigned both it and the National

Institute of Standards and

Technology (NIST) responsibilities 

for accrediting laboratories that

test voting systems. NIST assesses

a laboratory’s technical 

qualifications and makes 

recommendations to EAC, which 

makes a final accreditation 

decision. In view of the continuing

concerns about voting systems and 

the important roles that NIST and

EAC play in accrediting the 

laboratories that test these

systems, GAO was asked to 

determine whether each 

organization has defined an 

effective approach for accrediting 

laboratories that test voting

systems and whether each is 

following its defined approach. To 

accomplish this, GAO compared 

NIST and EAC policies, guidelines, 

and procedures against applicable

legislation and guidance, and 

reviewed both agencies’ efforts to

implement them. 

What GAO Recommends

GAO is making recommendations 

to NIST and EAC aimed at further

defining and implementing their 

respective accreditation programs 

in a way that better ensures that

voting system laboratory

accreditations are performed 

consistently and are verifiable. 

NIST and EAC generally agreed 

with the need for their respective

programs to continuously improve 

and both sought clarification on the

report’s recommendations, which 

GAO has added. 

NIST has largely defined and implemented an approach for accrediting voting 

system testing laboratories that incorporates many aspects of an effective

program. In particular, its approach addresses relevant HAVA requirements 

and reflects relevant laboratory accreditation guidance, including standards 

accepted by the international standards community. However, NIST’s defined 

approach does not, for example, cite explicit qualifications for the persons 

who conduct accreditation technical assessments, as called for in federal 

accreditation program guidance. Instead, NIST officials said that they rely on 

individuals who have prior experience in reviewing such laboratories. Further, 

even though the EAC requires that laboratory accreditation be based on 

demonstrated capabilities to test against the latest voting system standards, 

NIST’s defined approach has not always cited these current standards. As a 

result, two of the four laboratories accredited to date were assessed using

assessment tools that were not linked to the latest standards. Moreover, 

available documentation for the four laboratory assessments was not 

sufficient to determine how the checklists were applied and how decisions 

were reached. According to NIST officials, the four laboratories were 

consistently assessed. Moreover, they said that they intend to evolve NIST’s 

accreditation approach to, for example, clearly provide for sufficient 

documentation of how accreditation reviews are conducted and decisions are 

reached. However, they had yet to develop specific plans for accomplishing

this. 

EAC recently developed a draft laboratory accreditation program manual, but 

this draft manual does not adequately define all aspects of an effective 

approach, and it was not used in the four laboratory accreditations performed 

to date. Specifically, while this draft manual addresses relevant HAVA 

requirements, such as the requirement for the commissioners to vote on the 

accreditation of any laboratory that NIST recommends for accreditation, it 

does not include a methodology governing how laboratories are to be 

evaluated or criteria for granting accreditation. Because the manual was not 

approved at the time EAC accredited four laboratories, these accreditations 

were governed by a more broadly defined accreditation review process that 

was described in correspondence sent to each laboratory and a related 

document receipt checklist. As a result, these accreditations were based on 

review steps that were not sufficiently defined to permit them to be executed 

in a repeatable manner. According to EAC officials, including the official who 

conducted the accreditation reviews for the four laboratories, using the same 

person to conduct the reviews ensured that the steps performed on the first 

laboratory were repeated on the other three. However, given that both the 

steps and the results were not documented, GAO could not verify this. EAC 

officials stated that they intend to evolve the program manual over time and 

apply it to future accreditations and reaccreditations. However, they did not 

have specific plans for accomplishing this. Further, although EAC very 

recently approved an initial version of its program manual, this did not occur 

until after EAC provided comments, and GAO had finalized, this report. 

To view the full product, including the scope 

and methodology, click on GAO-08-770. 

For more information, contact Randolph C. 

Hite at (202) 512-3439 or hiter@gao.gov. 
Contents 

Letter 1

Results in Brief 2 

Background 4

NIST Has Defined and Implemented an Accreditation Approach 

That Reflects Relevant Standards but Is Missing Details Needed

for Consistent and Verifiable Implementation 17 

EAC’s Recently Drafted Accreditation Approach and Its Earlier 

Performed Laboratory Accreditations Lack Key Effectiveness 

Factors and Features 25 

Conclusions 32

Recommendations for Executive Action 32 

Agency Comments and Our Evaluation 33 

Appendix I Objectives, Scope, and Methodology 37 

Appendix II Comments from the National Institute of Standards 

and Technology 40 

Appendix III Comments from the Election Assistance Commission 45 

Appendix IV GAO Contact and Staff Acknowledgments 49 

Tables 

Table 1: EAC and NIST Responsibilities under HAVA 11 

Table 2: Summary of Extent to Which EAC Draft Approach 

Addresses NIST-Identified Accreditation Factors 28 

Table 3: Summary of Extent to Which EAC Satisfies Features 

Common to Federal Accreditation Programs 29 

Page i GAO-08-770 Accreditation of Voting System Testing Laboratories
Figures 

Figure 1: A Voting System Life Cycle Model 6 

Figure 2: Overall NIST and EAC Accreditation Processes 12 

Figure 3: VSTL Accreditation Program Activities 16 

Abbreviations 

EAC Election Assistance Commission 

HAVA Help America Vote Act 

ISO International Organization for Standardization 

NIST National Institute of Standards and Technology 

NVLAP National Voluntary Laboratory Accreditation 

Program 

VSTL voting system testing laboratory 

VSS Voting Systems Standards 

VVSG Voluntary Voting System Guidelines 

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately. 

Page ii GAO-08-770 Accreditation of Voting System Testing Laboratories
United States Government Accountability Office

Washington, DC 20548 

September 9, 2008 

The Honorable Robert A. Brady 

Chairman 

Committee on House Administration

House of Representatives 

Dear Mr. Chairman: 

In the wake of the 2000 and 2004 general elections, we issued a series of 

reports and testified1

 on virtually every aspect of our nation’s overall 

election system, including the many challenges and opportunities 

associated with various types of voting systems. In this regard, we 

emphasized that voting systems alone were neither the sole contributor 

nor solution to the problems that were experienced during the 2000 and 

2004 elections, and that the overall election system depended on the 

effective interplay of people, process, and technology and involved all 

levels of government. Among many things, we specifically reported in 

20012

 that no federal entity was responsible for accrediting the 

laboratories that tested voting systems, and we raised the establishment of 

such an entity as a matter for congressional consideration. 

Subsequently, Congress passed the Help America Vote Act (HAVA), which 

created the Election Assistance Commission (EAC) and assigned both it 

and the National Institute of Standards and Technology (NIST) separate 

but related responsibilities for accrediting laboratories that test voting 

systems.3

 In general, NIST is responsible for assessing a laboratory’s 

technical qualifications and making an accreditation recommendation to 

 1

See, for example, GAO, Elections: Perspectives on Activities and Challenges Across the

Nation, GAO-02-3 (Washington, D.C.: Oct. 15, 2001); Elections: Status and Use of Federal

Voting Equipment Standards, GAO-02-52 (Washington, D.C.: Oct. 15, 2001); Elections: A

Framework for Evaluating Reform Proposals, GAO-02-90 (Washington, D.C.: Oct. 15, 

2001); Elections: Federal Efforts to Improve Security and Reliability of Electronic Voting

Systems Are Under Way, but Key Activities Need to Be Completed, GAO-05-956 

(Washington, D.C.: Sept. 21, 2005); and Elections: All Levels of Government Are Needed to 

Address Electronic Voting System Challenges, GAO-07-576T (Washington, D.C.: Mar. 7,

2007). 

2

GAO-02-52. 

3

42 U.S.C. § 15371. 

Page 1 GAO-08-770 Accreditation of Voting System Testing Laboratories
EAC, while EAC is to use the assessment results and recommendation, 

along with its own review of related laboratory capabilities, to reach an 

accreditation decision. In 2004 and 2007, NIST and EAC established voting

system testing laboratory accreditation programs, respectively. To date, 

EAC has accredited four laboratories. In view of the continuing concerns 

about voting systems and the important roles that both NIST and EAC play 

in accrediting the laboratories that test these systems, you asked us to 

determine whether NIST and EAC have each defined an effective 

laboratory accreditation approach and whether each is following its 

defined approach. 

To accomplish this, we reviewed NIST and EAC policies, guidelines, and 

procedures governing voting system testing laboratory accreditation, 

deaccreditation, and reaccreditation and compared them, as appropriate,

to applicable statute, such as HAVA, and guidance published by NIST, the 

International Organization for Standardization, and us. We then compared

NIST and EAC actions and artifacts that were used for accrediting four 

voting system testing laboratories to their respective policies, guidelines, 

and procedures. We did not review a fifth laboratory because NIST was in 

the process of assessing it when we started our review, and had yet to 

recommend the laboratory to EAC for final accreditation. In addition, we 

interviewed officials from NIST, EAC, and the four laboratories to 

understand and clarify approaches taken, documentation provided, and

decisions reached. 

We conducted this performance audit at EAC and NIST offices in 

Washington, D.C., and Gaithersburg, Maryland, respectively, from 

September 2007 to September 2008 in accordance with generally accepted 

government auditing standards. Those standards require that we plan and

perform the audit to obtain sufficient, appropriate evidence to provide a 

reasonable basis for our findings and conclusions based on our audit 

objectives. We believe that the evidence obtained provides a reasonable

basis for our findings and conclusions based on our audit objectives. 

Further details of our objective, scope, and methodology are included in 

appendix I. 

NIST has largely defined and implemented an approach for accrediting 

voting system testing laboratories that incorporates many aspects of an 

effective program. In particular, its approach addresses relevant HAVA 

requirements and reflects relevant laboratory accreditation standards that

have been accepted by the international community. However, NIST’s 

defined approach does not cite explicit qualifications or training 

Results in Brief 

Page 2 GAO-08-770 Accreditation of Voting System Testing Laboratories
requirements for accreditation technical assessors, which, according to 

NIST, is a characteristic of an effective accreditation program. Instead, the 

program has relied, in part, on a small and specialized field of potential 

assessors in the voting system arena. Further, until recently, NIST’s 

laboratory accreditation guidance cited different versions of required 

standards. As a result, two of the four accredited laboratories were 

assessed using checklists linked to the current standards and two were 

not. Moreover, NIST’s documentation of these assessments was not 

sufficient to determine how the checklists were applied and how decisions 

were reached. According to NIST officials, the four laboratories were 

consistently assessed, and they intend to ensure that sufficient 

documentation is produced to show how assessments are conducted and 

decisions are reached. However, they said that they do not have a 

documented plan for accomplishing this. 

EAC recently developed a draft of a voting system testing laboratory 

accreditation program manual, but the draft manual does not adequately 

define all aspects of an effective approach and was not used in accrediting 

the four laboratories. EAC’s draft manual addresses applicable HAVA 

requirements, but does not include either a methodology governing how 

laboratories are to be reviewed or certain criteria relevant to granting 

accreditation. Because the draft manual was not available for laboratory 

assessments until recently, EAC instead used a more broadly defined 

accreditation review process contained in correspondence with the 

laboratories and a related checklist that were not specific enough to 

ensure that review steps were executed in a repeatable manner. According 

to EAC officials, using the same person to conduct the reviews ensured 

that the reviews were consistently performed. However, because both the

steps and the results were not documented, we could not verify this. In 

addition, EAC officials told us that they intend to evolve the program 

manual over time and apply it to future accreditations and 

reaccreditations; however, they do not have a documented plan for 

accomplishing this.4

To assist NIST and EAC in evolving their respective voting system testing 

laboratory accreditation programs, we are making recommendations to 

NIST and EAC to develop and execute plans, with specific tasks, 

 4

In August 2008, after EAC provided comments on, and we had finalized, this report, the

commission announced that it had approved its draft accreditation program manual. As a 

result, we did not review the approved manual. 

Page 3 GAO-08-770 Accreditation of Voting System Testing Laboratories
milestones, resources, and measures, which are aimed at adding 

consistency and specificity to their defined approaches and ensuring that 

the approaches are fully implemented and documented. 

NIST and EAC provided written comments on a draft of this report, signed 

by the Deputy Director of NIST and the Executive Director of EAC, 

respectively. More specifically, NIST stated that it appreciated our careful 

review of its voting system testing laboratory accreditation program and 

added that it generally concurs with our findings that this program must 

continue to evolve and improve. NIST also provided comments that were 

intended to clarify the current status of the program relative to three of 

our findings. For various reasons discussed in the agency comments 

section of this report, we do not believe that these comments affect any of 

the three findings. Therefore, we have not modified the report’s 

presentation of them. In light of NIST’s recent actions to address one of 

the findings, we updated our report to reflect these actions and have 

removed the associated recommendation from our final report. Further, in 

order to avoid the possibility of any misunderstanding about the actions 

needed to address one other finding, we have slightly modified the 

recommendation associated with it.

With respect to EAC’s comments, the commission described our review 

and report as helpful to the commission as it works to fully develop and 

implement its voting system testing laboratory accreditation program. It 

also stated that it agrees with the report’s conclusions that additional 

written internal procedures, standards, and documentation are needed to

ensure more consistent and repeatable implementation of the program. 

Further, it stated that it generally accepts our recommendations, adding 

that it will work hard to implement them. However, it sought clarification 

about two of the recommendations. In response, we have slightly modified 

both recommendations to avoid any confusion as to their intent. Both the 

NIST and EAC comments are discussed in detail in the agency comments 

section of this report, and are reprinted in their entirety in appendixes II 

and III, respectively. 

All levels of government share responsibility in the overall U.S. election 

system. At the federal level, Congress has authority under the Constitution 

to regulate presidential and congressional elections and to enforce 

prohibitions against specific discriminatory practices in all federal, state, 

and local elections. Congress has passed legislation that addresses voter 

registration, absentee voting, accessibility provisions for the elderly and 

persons with disabilities, and prohibitions against discriminatory 

Background 

Page 4 GAO-08-770 Accreditation of Voting System Testing Laboratories
practices.5

 At the state level, individual states are responsible for the 

administration of both federal elections and their own elections. States 

regulate the election process, including, for example, the adoption of 

voluntary voting system guidelines, the state certification and acceptance 

testing of voting systems, ballot access, registration procedures, absentee

voting requirements, the establishment of voting places, the provision of 

election day workers, and the counting and certification of the vote. 

In total, the overall U.S. election system can be seen as an assemblage of 

55 distinct election systems—those of the 50 states, 4 U.S. territories, and 

the District of Columbia. Further, although election policy and procedures 

are legislated primarily at the state level, states typically have

decentralized election systems, so that the details of administering 

elections are carried out at the city or county levels, and voting is done at

the local level. As we reported in 2001,6

 local election jurisdictions number 

more than 10,000, and their sizes vary enormously—from a rural county 

with about 200 voters to a large urban county, such as Los Angeles County, 

where the total number of registered voters for the 2000 elections 

exceeded the registered voter totals in 41 states. Further, these thousands 

of jurisdictions rely on many different types of voting methods that employ 

a wide range of voting system makes, models, and versions. Because of the 

prominent role played by electronic voting systems, testing these systems 

against national standards is critical to ensuring their security and 

reliability. Equally critical is ensuring that the laboratories that perform 

these tests are competent to carry out testing activities. 

The Overall U.S. Election 

System Depends on 

Effective Interactions 

among People, Processes, 

and Technology 

In the United States today, most votes are cast and counted by electronic 

voting systems, and many states require use of systems that have been 

certified nationally or by state authorities. However, voting systems are 

but one facet of a multifaceted, continuous overall election system that 

involves the interplay of people, processes, and technology during the 

entire life of a system. All levels of government, as well as commercial 

voting system manufacturers and system testing laboratories, play key 

roles in ensuring that voting systems perform as intended. 

Electronic voting systems are typically developed by manufacturers, then 

purchased as commercial, off-the-shelf products and operated by state and

 5

GAO-02-3. 

6

GAO-02-3. 

Page 5 GAO-08-770 Accreditation of Voting System Testing Laboratories
local election administrators. Viewed at a high level, these activities make 

up three phases of a system life cycle: product development, acquisition, 

and operations. (See fig. 1.) Key processes that span these life cycle

phases include managing the people, processes, and technologies within 

each phase and across phases, and testing the systems and components 

during and at the end of each phase. Additionally, voting system standards 

are important through all of the phases because they provide criteria for 

developing, testing, and acquiring voting systems, and they specify the 

necessary documentation for operating the systems. 

Figure 1: A Voting System Life Cycle Model 

Source: GAO analysis of EAC, NIST, and Institute of Electrical and Electronics Engineers (IEEE) publications.

Product Acquisition Operations

development

Management

Testing

Standards

• The product development phase includes activities such as establishing 

requirements for the system, designing a system architecture, developing

software, and integrating components. Activities in this phase are 

performed by the system vendor. 

• The acquisition phase includes activities such as publishing a solicitation, 

evaluating offers, choosing a voting technology and a vendor, and 

awarding and administering contracts. For voting systems, activities in this 

phase are primarily the responsibility of state and local governments but 

entail some responsibilities that are shared with the system vendor (e.g., 

entering into the contract). 

• The operations phase consists of activities such as ballot design and 

programming, setup of systems before voting, pre-election testing, vote 

capture and counting during elections, recounts and system audits after 

elections, and storage of systems between elections. Responsibility for 

activities in this phase typically resides with local jurisdictions, whose 

officials may, in turn, rely on or obtain assistance from system vendors for 

aspects of these activities. 

• Standards for voting systems, as will be discussed in a later section, were 

developed at the national level by the Federal Election Commission in 

Page 6 GAO-08-770 Accreditation of Voting System Testing Laboratories
1990 and 2002 and were updated by EAC in 2005. In the product 

development phase, voting system standards serve as requirements to 

meet for developers to build systems. In the acquisition phase, they also 

provide a framework that state and local governments can use to evaluate

systems. In the operations phase, they specify the necessary 

documentation for operating the systems. 

• Testing processes are conducted throughout the life cycle of a voting 

system. Voting system vendors conduct product testing during 

development of the system and its components. Federal certification 

testing of products submitted by system vendors is conducted by national 

voting system testing laboratories (VSTL). States may conduct evaluation 

testing before acquiring a system to determine how well products meet 

their state-specific specifications, or they may conduct certification testing 

to ensure that a system performs its functions as specified by state laws 

and requirements. Once a voting system is delivered by the system vendor, 

states and local jurisdictions may conduct acceptance testing to ensure 

that the system satisfies functional requirements. Finally, local 

jurisdictions typically conduct logic and accuracy tests related to each 

election and sometimes subject portions of the system to parallel testing 

during each election to ensure that the system components perform 

accurately.

• Management processes ensure that each life cycle phase produces a 

desirable outcome. Typical management activities that span the system life 

cycle include planning, configuration management, system performance 

review and evaluation, problem tracking and correction, human capital

management, and user training. These activities are conducted by the 

responsible parties in each life cycle phase. 

In 2004, we reported7

 that the performance of electronic voting systems, 

like any type of automated information system, can be judged on several 

bases, including their security, accuracy, ease of use, efficiency, and cost.

We also reported that voting system performance depends on how the 

system was designed, developed, and implemented. 

 7

GAO, Elections: Electronic Voting Offers Opportunities and Presents Challenges, 

GAO-04-975T (Washington, D.C.: July 20, 2004). 

Page 7 GAO-08-770 Accreditation of Voting System Testing Laboratories
Since the passage of HAVA, the use of electronic voting systems has 

increased and become the predominant method of voting. However, 

concerns have been raised about the security and reliability of these 

systems. As we have previously reported,8

 testing and certifying voting 

systems is one critical step in acquiring, deploying, operating, and 

administering voting systems, which better ensures that they perform 

securely and reliably. Among other things, rigorous execution and careful

documentation of system testing is a proven way to help ensure that 

system problems are found before the systems are deployed and used in 

an election. To accomplish this, it is vital that the organizations that test 

the systems be qualified and competent to do so. For voting systems, a key 

testing organization is a federally accredited, national VSTL.

In general, accreditation is the formal recognition that a laboratory is 

competent to carry out specific types of tests or calibrations. Federally 

accredited laboratories perform many different types of testing and related 

activities on various products, ranging from inspecting grain to certifying 

maritime cargo gear. The genesis of laboratory accreditation programs 

owes largely to agencies’ need to assure themselves of the competency of

the organizations responsible for testing products or services that involve 

the use of federal funds.

To provide national recognition for competent laboratories, the NIST 

Director established the National Voluntary Laboratory Accreditation 

Program (NVLAP) in 1976 at the request of the private sector. Under this 

program, which is based on internationally accepted standards, NIST 

accredits laboratories that it finds competent to perform specific types of 

tests or calibrations. In June 2004, NVLAP announced the establishment, in 

accordance with HAVA, of an accreditation program for laboratories that 

test voting systems using standards determined by EAC. 

Enacted in October 2002, HAVA affected nearly every aspect of the voting

process, from voting technology to provisional ballots and from voter 

registration to poll worker training.9

 In particular, the act authorized $3.86

billion in funding over several fiscal years to replace punch card and 

mechanical lever voting equipment, improve election administration and 

accessibility, train poll workers, and perform research and pilot studies. 

Laboratory Accreditation 

Plays an Important Role in 

Ensuring Accurate, 

Reliable, and Secure 

Voting Systems 

HAVA Assigned EAC and 

NIST Responsibility for 

Accrediting VSTLs 

 8

GAO-05-956. 

9

Help America Vote Act, Pub. L. No. 107-252 (Oct. 29. 2002). 

Page 8 GAO-08-770 Accreditation of Voting System Testing Laboratories
HAVA also established EAC, provided for the appointment of four 

commissioners, and specified the process for selecting an executive

director. Generally speaking, EAC is to assist in the administration of 

federal elections and provide assistance in administering certain federal 

election laws and programs. 

Since the passage of HAVA in 2002, the federal government has taken 

steps to implement the act’s provisions. For example, after beginning 

operations in January 2004, EAC updated the existing federal voluntary 

standards for voting systems, including strengthening provisions related to

security and reliability. Additionally, EAC established an interim VSTL 

accreditation program that leveraged a predecessor program run by the 

National Association of State Elections Directors, and EAC and NIST then 

established companion accreditation programs that replaced the interim 

program. 

Federal standards for voting systems were first issued in 1990 when the 

Federal Election Commission published standards.10 These federal 

standards identified minimum functional and performance requirements, 

which states were free to adopt in whole, in part, or not at all, for 

electronic voting equipment, and specified test procedures to ensure that 

the equipment met those requirements. In 2002, the Federal Election 

Commission issued its Voting System Standards (VSS), which updated the 

1990 standards to reflect more modern voting system technologies. In 

2005, we reported11 that these standards identified minimum functional 

and performance requirements for voting systems but were not sufficient 

to ensure secure and reliable voting systems. As a result, we 

recommended that EAC work to define specific tasks, measurable 

outcomes, milestones, and resource needs to improve the voting system 

standards. Until then, election administrators were at risk of relying on

voting systems that were not developed, acquired, tested, operated, or 

managed in accordance with rigorous security and reliability standards—

potentially affecting the reliability of future elections and voter confidence 

in the accuracy of the vote count. 

Following the enactment of HAVA in 2002 and the establishment of EAC in 

2004, EAC adopted the Voluntary Voting System Guidelines (VVSG) in 

EAC Updated the Federal 

Voluntary Standards for Voting 

Systems 

 10Federal Election Commission, Performance and Test Standards for Punchcard, 

Marksense, and Direct Recording Electronic Voting Systems (January 1990). 

11GAO-05-956. 

Page 9 GAO-08-770 Accreditation of Voting System Testing Laboratories
2005.

12 The VVSG specify the functional requirements, performance 

characteristics, documentation requirements, and test evaluation criteria 

for the national certification of voting systems. Accredited testing 

laboratories are to use the VVSG to develop test plans and procedures for 

the analysis and testing of systems in support of EAC’s voting system 

certification program.13 The VVSG are also used by voting system 

manufacturers as the basis for designing and deploying systems that can 

be federally certified. 

We reported in 200114 that the National Association of State Elections 

Directors was accrediting independent test authorities to test voting 

equipment against the Federal Election Commission standards. Under this 

program, three laboratories were accredited. Under HAVA, NIST is to 

recommend laboratories for EAC accreditation. In 2006, NIST notified 

EAC that its initial recommendations might not be available until 

sometime in 2007. As a result, EAC initiated an interim accreditation 

program and invited the three laboratories accredited by the state 

elections directors to apply.15 As part of the interim program, laboratories 

were required to attest to a set of EAC-required conditions and practices, 

including certifying the integrity of personnel, the absence of conflicts of 

interest, and the financial stability of the laboratory. In August and 

September 2006, EAC granted interim accreditation to two of the three 

laboratories invited to apply. EAC terminated its interim program in March 

2007. 

HAVA assigned responsibilities for laboratory accreditation to both EAC 

and NIST. In general, to reach an accreditation decision, NIST is to focus 

on assessing laboratory technical qualifications, while EAC is to use those 

assessment results and recommendations and augment them with its own 

review of related laboratory capabilities. See table 1 for the two agencies’ 

HAVA responsibilities. 

EAC Established an Interim 

Accreditation Program 

EAC and NIST Have 

Established Separate but 

Related Laboratory 

Accreditation Programs 

 12The VVSG did not take effect until December 2007. 

13We have ongoing work to review EAC’s certification program for the House Committee 

on House Administration. 

14GAO-02-90. 

15The state elections directors’ accreditation program was discontinued in July 2006. 

Page 10 GAO-08-770 Accreditation of Voting System Testing Laboratories
Table 1: EAC and NIST Responsibilities under HAVA 

 Responsible entity

HAVA responsibility EAC NIST 

Prov fication, decertification, and 

rece

accredite

ide for the testing, certi X 

rtification of voting system hardware and software by 

d laboratories. 

Conduct boratories 

and d for

accredita

evaluations of independent, nonfederal la X 

submit to EAC a list of those laboratories propose

tion.

Vote on into 

con

may be a

vote of E

the accreditation of any laboratory, taking X 

sideration the NIST recommendation for it. No laboratory 

ccredited unless its accreditation is approved by a 

AC.

Publish a

labo

recomm

n explanation for the accreditation of any X 

ratory not included on the list submitted for 

endation by NIST. 

Monitor of laboratories 

accr ith

respect t

including

laborato

and review the performance X 

edited by EAC and make recommendations to EAC w

o the continuing accreditation of laboratories, 

recommendations to revoke the accreditation of 

ries. 

Revoke ly when 

appr

the accreditation of a laboratory on X 

oved by a vote of EAC. 

Source: GAO an

in meeting HAVA’s requirements are 

ent 

ncies will 

The NIST and EAC accreditation programs can be viewed together as 

f 12

alysis of United States Code.

The tasks that NIST is to perform 

addressed in an annual interagency agreement executed between the 

institute and EAC each year. For example, the 2008 interagency agreem

states that NVLAP will continue to assess VSTLs and will coordinate with 

EAC to continually monitor and review the performance of the 

laboratories. Additionally, the agreement states that the two age

coordinate to maintain continuity between their respective accreditation 

programs. 

forming a federal VSTL accreditation process that consists of a series o

complementary steps. These steps are depicted in figure 2, where the 

numbers correspond to a detailed narrative description below. 

Page 11 GAO-08-770 Accreditation of Voting System Testing Laboratories
Figure 2: Overall NIST and EAC Accreditation Processes

Source: GAO analysis. 

1. Laboratory submits

an application and initial 

documentation to NIST

6. NIST recommends laboratory 

to EAC for accreditation 

7. EAC invites

laboratory to apply 

for accreditation and 

requests application 

materials

2. NIST conducts a

preassessment review

3. NIST conducts an 

on-site assessment of 

the laboratory facilities

and capabilities

4. Laboratory responds

to any nonconformities

identified by NIST

5. NIST completes its review 

and makes a determination 

regarding accreditation 

8. EAC staff 

evaluates

application

9. Laboratory 

responds to any 

nonconformities

identified by EAC

11. EAC commissioners

publicly vote on 

accreditation

12. Laboratory 

is granted 

accreditation

10. EAC staff completes

the review and 

recommends laboratory 

for accreditation

NIST determines

laboratory must further 

address nonconformities

EAC determines

laboratory must further 

address nonconformities

NIST does not recommend accreditation due to nonconformities

that were not addressed within the alloted time

EAC does not recommend accreditation due to nonconformities

that were not addressed within the alloted time

NIST

EAC

1. Laboratory Application to NIST

The accreditation process begins when a laboratory submits a completed

application to NIST, along with administrative information about the 

laboratory, the scope of accreditation being applied for, and an agreement 

to the conditions of accreditation (i.e. practices that must be followed to 

obtain and maintain accreditation). In addition, the laboratory submits 

documentation that supports the application, including the laboratory’s 

quality control manual. 

2. NIST Preassessment

Using the application and supporting documentation, NIST conducts a 

preassessment review. Among other things, this review includes 

Page 12 GAO-08-770 Accreditation of Voting System Testing Laboratories
comparing the quality control manual against the requirements in NIST 

accreditation program guidance. If deficiencies in the documentation are

found, NIST requests corrections to satisfy program requirements. 

3. NIST On-site Assessment 

Following satisfactory completion of the preassessment review, a NIST 

team visits the laboratory facilities to conduct an on-site assessment. This 

assessment includes staff interviews, reviews of laboratory records and 

audit reports, and demonstrations of staff competence to execute planned 

test methods and procedures. It concludes with the NIST team presenting 

its findings to laboratory management. While conducting the on-site 

assessment, the team records its observations and comments. 

4. NIST Nonconformity Resolution 

NIST prepares a final report, including a list of any nonconformities, and 

provides it to the laboratory. The laboratory has 30 days to respond as to 

how it will address the areas of nonconformity. NIST evaluates the 

laboratory’s response and determines whether the nonconformities have 

been sufficiently addressed. If so, NIST renders an accreditation decision. 

If not, NIST may contact the laboratory for additional information or may 

deny accreditation. If a laboratory is denied accreditation, it may reapply

to NIST. 

5. NIST Accreditation Decision 

When a laboratory has no areas of nonconformity, the voting systems 

program manager makes an accreditation recommendation to the Chief of 

NVLAP, who is responsible for all NVLAP accreditation decisions and 

issues all NVLAP accreditation certificates. 

6. NIST Recommendation to EAC 

In addition to granting the NVLAP accreditation, the Chief provides a 

recommendation to the Director of NIST. The recommendation is 

reviewed by NIST’s general counsel, and then a letter is sent to EAC that 

recommends the laboratory for accreditation as a VSTL in accordance 

with HAVA.

Page 13 GAO-08-770 Accreditation of Voting System Testing Laboratories
7. Accreditation Application to EAC 

After receiving the NIST recommendation, EAC sends the recommended 

laboratory an invitation to apply to the EAC accreditation program. In the 

letter, EAC specifies a list of information and documentation that the 

laboratory must provide. 

8. EAC Accreditation Review 

The laboratory submits an application and supporting information to EAC. 

EAC staff review the application package for completeness. In addition, 

staff review the supporting materials vis-a-vis accreditation program 

requirements. During the course of the review, staff may contact the 

laboratory to clarify the information provided or to inform the laboratory 

of requirements that are not sufficiently addressed. 

9. EAC Nonconformity Resolution 

EAC submits correspondence (generally through e-mail) to the laboratory 

identifying areas of nonconformity. The laboratory then provides the 

missing and/or clarifying documentation. EAC staff determines if the 

provided information adequately addresses the nonconformity issues, 

contacting the laboratory as needed.

10. Recommendation to EAC Commissioners 

The EAC accreditation program director, through the EAC executive 

director, makes a recommendation to the EAC commissioners as to 

whether the laboratory should be accredited by EAC. Along with the 

recommendation, the program director provides the review results, as well 

as laboratory-provided materials. 

11. Vote by EAC Commissioners 

The commissioners review the material provided and may request 

additional clarification, as needed. At a public meeting, the commissioners 

vote on whether to accredit the laboratory. Should the EAC 

commissioners vote to deny the accreditation, the laboratory must wait for 

EAC to invite the laboratory to reapply. 

12. EAC Accreditation Granted 

Page 14 GAO-08-770 Accreditation of Voting System Testing Laboratories
When the commissioners vote to accredit a laboratory, EAC’s executive 

director issues an accreditation certificate identifying the scope and 

effective dates of the VSTL accreditation. In addition, the program director 

makes information about the laboratory’s accreditation publicly available 

via the EAC Web site. At this point, the laboratory is authorized to operate 

as a VSTL under EAC’s testing and certification program. 

Once a laboratory has been accredited, both NIST and EAC are to monitor 

its compliance with the terms of its accreditation. In doing so, NVLAP staff 

may visit a laboratory at any time, whether for cause or on a random 

selection basis, and these visits can be either scheduled in advance with 

the laboratory or unannounced. If a laboratory is found to not be in 

compliance, the accreditation may either be suspended16 or revoked, 

depending on the nature of the issues involved. A suspension provides the 

opportunity for the laboratory to address the identified issues. EAC also 

monitors the procedures and practices of accredited laboratories through 

documentation reviews and visits. If a VSTL is unable to remedy identified

compliance issues, the EAC program director can propose that the 

accreditation be suspended and ultimately revoked. As provided for under 

HAVA, the EAC commissioners would vote on any proposed revocation. 

Postaccreditation Monitoring 

Activities 

Status of Completed VSTL 

Accreditation Activities 

As of May 2008, EAC has accredited four laboratories. These laboratories 

are SysTest Labs, LLC; Wyle Laboratories, Inc.; iBeta Quality Assurance; 

and InfoGard Laboratories, Inc. A fifth laboratory, CIBER Inc., has been 

granted NVLAP accreditation and has been recommended to, but not yet 

accredited by, EAC. InfoGard Laboratories, Inc., whose NVLAP 

accreditation expires in June 2008, has recently notified NIST and EAC 

that it would not apply to renew its accreditation, citing the volatility of 

the voting system environment as one reason. The timeline for each of 

these accreditations, and other accreditation program activities, is found 

in figure 3. 

 16A suspension provides the laboratory an opportunity to address the identified issues. 

Page 15 GAO-08-770 Accreditation of Voting System Testing Laboratories
Figure 3: VSTL Accreditation Program Activities 

Source: GAO based on NIST- and EAC-provided data.

Applied to NIST

Preassessment began

On-site assessment began

NVLAP accreditation/recommendation to EAC

Lab applied to EAC

EAC accreditation

NIST-related activities

EAC-related activities

1

2

3

4

5

6

2005 2006

July A S D O N JF MAM J J A S D O N

2007 2008

J F MAM J J A S D O N

Laboratories

Wyle

SysTest

InfoGard

CIBER

iBeta

2005 2006

J FMAM J J A S D O N J FMAM O J J A S D N

2007

J FMAM O J J A S D N

2002 2003

Sept. O N J FMAM O D J J A S D N

2004

J FMAM O J J A S D N

HAVA enacted Federal Register notice 

published announcing 

that laboratories can 

apply to NIST program

EAC receives operational 

level of funding

EAC establishes

EAC begins operation accreditation program

Accreditation milestones for individual laboratories

Federal Register notice published 

announcing NIST program

2008

J FMAM J

1

1

1

2 3

2

2

3

4

3

3

3 4

4 5 6

5 6

4 5 6

4 5 6

1

1

2

2

Page 16 GAO-08-770 Accreditation of Voting System Testing Laboratories
NIST’s defined approach to accrediting voting system laboratories largely

reflects applicable HAVA requirements and relevant international 

standards, both of which are necessary to an effective program. However, 

this approach is continuing to evolve based on issues realized during 

NIST’s implementation experience to date. In particular, because NIST’s 

defined program does not, for example, specify the nature and extent of 

assessment documentation to generate or retain or specify the version of 

the voting system standards to be used, our analysis of NIST’s efforts in 

accrediting four laboratories could not confirm that the agency has 

consistently followed its defined accreditation program. NIST officials 

stated that these limitations are due in part to the relative newness of the 

program and that they will be addressed by updating the accreditation 

program handbook. However, they said that they do not have documented 

plans to accomplish this. Until these limitations are addressed, NIST will 

be challenged in accrediting voting system laboratories in a consistent and 

verifiable manner. 

NIST has defined its voting system accreditation program to address 

relevant HAVA requirements. According to HAVA, NIST is to 

• conduct reviews of independent, nonfederal voting system testing 

laboratories and submit to EAC a list of proposed voting system testing 

laboratories and 

NIST Has Defined and 

Implemented an 

Accreditation 

Approach That 

Reflects Relevant 

Standards but Is 

Missing Details 

Needed for Consistent 

and Verifiable

Implementation 

NIST Voting System 

Accreditation Program 

Reflects HAVA

Requirements 

• monitor and review the performance of those proposed laboratories that 

EAC accredits, including making recommendations to EAC regarding 

accreditation continuance and revocation. 

NIST’s defined voting system accreditation program satisfies both of these 

requirements. With respect to the first, NIST announced in June 2004 the 

establishment of its voting system testing laboratory accreditation 

program as part of NVLAP, a statutorily created program for unbiased, 

third parties to establish the competence of national independent 

laboratories. As such, NIST adopted its NVLAP handbook17 as the basis for 

 17NIST, NIST Handbook 150: National Voluntary Laboratory Accreditation Program 

Procedures and General Requirements (Gaithersburg, Md.: February 2006). 

Page 17 GAO-08-770 Accreditation of Voting System Testing Laboratories
its defined approach to reviewing VSTLs and has supplemented it with a 

handbook that is specific to voting system testing.18

With respect to the second HAVA requirement, the supplemental 

handbook cited above states that the NIST Director will recommend 

NVLAP-accredited VSTLs to EAC for subsequent commission 

accreditation. Additionally, NIST’s handbooks provide for both monitoring 

accredited laboratories and for making recommendations regarding a 

laboratory’s continued accreditation. For example, the handbook states 

that a monitoring visit may occur at both scheduled and unscheduled 

times and the scope may be limited to a few items or include a full review. 

It also states that a reaccreditation review shall be conducted in 

accordance with the procedures used to initially accredit laboratories. 

Further, the handbook also identifies accreditation or reaccreditation 

decision options, including granting, denying, or modifying the scope of an 

accreditation. 

According to NIST officials, these HAVA requirements are relevant and 

important to defining an effective voting system testing laboratory 

accreditation program. By incorporating them, NIST has reflected one key 

aspect of an effectively defined program. 

NIST’s VSTL accreditation program reflects internationally recognized 

standards for establishing and conducting accreditation activities. These 

standards are published by the International Organization for 

Standardization (ISO), and the two that are germane to this accreditation 

program are (1) ISO/IEC 17011,19 which establishes general requirements 

for accreditation bodies and (2) ISO/IEC 17025,20 which establishes the 

general requirements for reviewing the competence of laboratories. 

According to NIST program documentation, this allows NVLAP to both 

operate as an unbiased, third party accreditation body and to utilize a 

quality management system compliant with international standards. As a 

NIST Has Incorporated 

Relevant International 

Accreditation Standards 

into Its VSTL Accreditation 

Program 

 18NIST, NIST Handbook 150-22: National Voluntary Laboratory Accreditation Program: 

Voting System Testing (Gaithersburg, Md.: December 2005).

19ISO, ISO/IEC 17011: Conformity Assessment: General Requirements for Accreditation 

Bodies Accrediting Conformity Assessment Bodies (Geneva, Switzerland: Feb. 15, 2005).

20ISO, ISO/IEC 17025: General Requirements for the Competence of Testing and

Calibration Laboratories (Geneva, Switzerland: May 15, 2005). 

Page 18 GAO-08-770 Accreditation of Voting System Testing Laboratories
result, NIST has incorporated key aspects of an effective accreditation 

body into its voting system accreditation program. 

ISO/IEC 17011 requires that an accrediting body have, among other things, 

(1) a management system for accreditation activities, (2) a policy defining

the types of records to be retained and how those records will be 

maintained, (3) a clear description of the accreditation process that covers

the rights and responsibilities of those seeking accreditation, and (4) a 

clear description of the accreditation activities to be performed. 

NIST VSTL accreditation program-related documentation, including its 

program handbooks, satisfies each of these requirements. In fact, NIST has 

cross-referenced its documentation to each ISO/IEC 17011 requirement. 

Specifically, the first requirement is cross-referenced to the NVLAP 

Management System Manual,21 which describes the overall accreditation 

program’s management policies and control structure, and the second is 

cross-referenced to the program’s record keeping policy, which specifies 

what types of records should be maintained and how they should be 

maintained. The third and fourth requirements are cross-referenced to the

accreditation process descriptions in both the Management System 

Manual and the general handbook. Together, these documents contain, for 

example, (1) the rights of laboratories applying for accreditation and (2) 

the scope of accreditation activities to be performed, including a 

preassessment review, an on-site review, and a final on-site assessment 

report. 

ISO/IEC 17025 requires that accreditation reviews cover specific topics. 

These include (1) laboratory personnel independence and conflicts of 

interest; (2) a laboratory system for quality control (i.e., a framework for 

producing reliable results and continuous improvement to laboratory 

procedures); and (3) a laboratory mechanism for collecting and 

responding to customer complaints. Additionally, the standard establishes 

basic technical requirements that a laboratory has to meet, and thus that 

reviews are to cover, including (1) competent laboratory personnel who 

are capable of executing the planned tests, (2) appropriate tests and test 

methods, and (3) clear and accurate test result documentation. 

NIST Program Meets ISO 

Accreditation Body 

Requirements 

NIST Program Meets ISO 

Laboratory Accreditation 

Review Requirements 

 21NIST, National Voluntary Laboratory Accreditation Program: Management System 

Manual, Revision 2 (Gaithersburg, Md.: Mar. 25, 2008). 

Page 19 GAO-08-770 Accreditation of Voting System Testing Laboratories
NIST voting system testing laboratory accreditation program-related 

documents, including its program handbooks, satisfy these requirements. 

First, the general handbook defines the requirement for a laboratory to 

have personnel that are independent and free of any conflict of interest. 

Second, the handbook requires that a laboratory have a management 

quality control system and that this system provide for reliable results and 

continuous improvement to laboratory procedures. Third, the handbook 

requires that a laboratory have a mechanism for receiving and responding 

to customer complaints. Last, the handbook establishes certain technical 

requirements that a laboratory must meet, such as having competent 

laboratory personnel capable of executing the planned tests, using 

appropriate tests and test methods, and documenting test results in a clear 

and accurate manner. 

For several of these requirements, NIST’s voting-specific supplemental 

handbook augments the general handbook. For example, this

supplemental handbook requires laboratories to submit a quality control 

manual, as well as information to demonstrate the competence of 

laboratory administrative and technical staff. Further, it requires that a 

laboratory’s training program be updated so that staff can be retrained as 

new versions of voting system standards are issued. 

NIST has reported on the importance of ensuring that those persons who 

perform accreditation assessments are sufficiently qualified and that the 

assessments themselves are based on explicitly defined criteria and are 

adequately documented. Nevertheless, NIST has not fully reflected key 

aspects of these findings in its defined approach to accrediting voting 

system testing laboratories. For example, it has not specified the basis for 

determining the qualifications of its accreditation assessors, and while a 

draft update to its handbook now includes the specific voting system 

standards to be used when performing an accreditation assessment, this 

handbook was only recently approved. According to NIST officials, these 

gaps are due to the newness of the accreditation program and will be 

addressed in the near future. Because these gaps have confused 

laboratories as to what standards they were to meet, and may have 

resulted in differences in how accreditations have been performed to date, 

it is important that the gaps be addressed. 

NIST Voting System 

Accreditation Program 

Does Not Reflect Its Own 

Findings on the Need for 

Assessor Qualifications 

and Training and Key 

Assessment Criteria 

Page 20 GAO-08-770 Accreditation of Voting System Testing Laboratories
NIST has reported22 on the importance of having competent and qualified

human resources to support accreditation programs. According to these 

findings, an accreditation program should, among other things, provide for 

NIST Has Not Specified 

Requirements for Assessor 

Qualifications and Training 

• having experienced and qualified assessors to perform accreditation 

activities; 

• demonstrating an assessors’ qualifications using defined documentation 

and explicit criteria that encompass the person’s education, experience,

and training; and 

• training (initial and continuing) for assessors. 

NIST’s defined approach to VSTL accreditation does not provide for all 

these requirements. To its credit, its program handbook identifies the need 

for experienced and qualified assessors in the execution of accreditation 

activities and provides for each assessor’s qualifications to be

documented. Further, it has defined generic training that applies to all of 

its accreditation assessors. For example, the NVLAP Assessor Training 

Syllabus includes training on ISO/IEC 17011 and 17025, as well as training

on the NVLAP general handbook. In addition, the VSTL accreditation 

program manager stated that new assessors receive training on the 2002 

VSS and 2005 VVSG and that periodic training seminars are provided to 

assessors on changes to either the general handbook or the 2005 VVSG. 

In addition, the program manager told us that candidate assessors must 

submit some form of documentation (e.g., a resume), and that this 

documentation is used to evaluate, rank, and select candidates that are 

best qualified. The NIST VSTL assessors that we interviewed confirmed 

that they were required to submit such documentation at NIST’s request.

However, NIST’s defined approach does not cite the explicit capabilities 

and qualifications that an assessor must meet or the associated 

documentation needed to demonstrate these capabilities and 

qualifications. According to the program manager, this is because the field

of potential assessors in the voting system arena is small and specialized 

and because they focused on defining other aspects of the program that 

were higher priorities. Further, NIST has not defined and documented the 

specific training requirements needed to be a VSTL lead assessor or a 

 22NIST, The ABC’s of the U.S. Conformity Assessment System, NISTIR 6014 (Gaithersburg, 

Md.: April 1997). 

Page 21 GAO-08-770 Accreditation of Voting System Testing Laboratories
technical assessor for the VSTL program. According to the program 

manager, this is because these assessors receive all the training they need 

by working on the job with more experienced assessors. Not specifying 

criteria governing assessor qualifications and training is of concern 

because differences in assessors’ capabilities could cause inconsistencies 

in how assessments are performed. 

NIST recognizes the importance of specifying explicit criteria against 

which all candidate laboratories will be assessed and fully documenting 

the assessments that are performed. Specifically, the general handbook 

provides the criteria and requirements that will be used to evaluate basic 

laboratory capabilities. It also states that technical requirements specific 

to a given field of accreditation are published in program-specific 

handbooks. To that end, NIST published a supplemental program-specific 

handbook in December 2005 that provided the voting-specific 

requirements to be used to evaluate VSTLs, additional guidance, and

related interpretive information.23

NIST’s 2005 supplemental handbook does not contain sufficient criteria 

against which to evaluate VSTLs. It identifies specific requirements that 

laboratories are to demonstrate relative to the 2002 VSS but not the 2005 

VVSG. For example, the handbook states that laboratories are expected to

develop, validate, and document test methods that meet the 2002 VSS.

However, it does not refer to the 2005 VVSG. In addition, the programspecific checklist that accompanies this version of the handbook does not 

identify all the 2005 VVSG standards against which laboratories are 

evaluated. Specifically, this checklist makes reference to the VVSG in 

relation to just a few checklist requirements. 

According to the NIST program manager, the 2005 handbook did not refer

to the 2005 VVSG requirements because only the 2002 VSS requirements 

were mandatory at the time it was published. He further stated that, 

despite the fact that the 2005 VVSG requirements were not included in that 

handbook, NIST assessors were expected to use them when performing

the first laboratory assessments. Representatives for two laboratories 

stated that because these requirements were not documented or identified

in the NIST handbooks, they did not learn that they would be required to 

demonstrate 2005 VVSG-based capabilities until the NIST on-site 

assessment teams arrived. 

NIST’s Approach Does Not 

Fully Specify Criteria for 

Evaluating and Documenting 

VSTL Capabilities 

 23NIST, NIST Handbook 150-22. 

Page 22 GAO-08-770 Accreditation of Voting System Testing Laboratories
In December 2007, NIST released draft revisions of the voting programspecific handbook and checklist, stating that labs are expected to meet 

both 2002 VSS and 2005 VVSG. In addition, the 2007 draft handbook clearly 

specifies that laboratories must demonstrate how developed test methods 

and planned tests trace back to and satisfy both the 2002 VSS and the 2005

VVSG. Taken together, the new handbook and checklist should better 

identify the requirements and criteria used to evaluate a laboratory and 

document the results. According to NIST, the new handbook and checklist 

have recently been finalized, and both are now in use.24

NIST has found that reliable and accurate documentation provides 

assurance that laboratory accreditation activities have been effectively 

fulfilled.25 However, in its efforts to date in accrediting four VSTLs, 

documentation of the assessments does not show that NIST has fully 

followed its defined accreditation approach. While we could not determine 

whether this is due to incomplete documentation of the steps performed 

and the decisions made during an assessment or due to steps not being 

performed as defined, this absence of verifiable evidence raises questions 

about the consistency of the assessments and the resultant accreditations.

Without adequately documenting each assessment, including all steps 

performed and the basis for any steps not performed, such questions may 

continue to be raised. 

To NIST’s credit, available documentation shows that it consistently 

followed some aspects of its defined approach in accrediting the four 

laboratories. For example, we verified that NIST received an application

from each of the laboratories as required, and our review of completed 

checklists and summary reports shows that preassessment reviews and 

on-site assessments were performed for each laboratory, as was required. 

According to a lead assessor, this review usually focused on the 

laboratories’ quality assurance manuals. Moreover, the completed 

checklists identified whether the requirement was met or not for each 

listed requirement, and included comments, in some cases, as to how a 

laboratory addressed a requirement. Also as required, NIST received 

Available Documentation 

Does Not Show That NIST 

Has Consistently Followed 

All Aspects of Its Defined 

Accreditation Approach 

 24NIST, Handbook 150-22 2008 Edition: National Voluntary Laboratory Accreditation 

Program: Voting System Testing (Gaithersburg, Md.: May 2008); NIST Handbook 150-22

Checklist: Voting System Testing Program (Rev. 2008-06-25).

25NIST, National Voluntary Laboratory Accreditation Program: Management System 

Manual, Revision 2. 

Page 23 GAO-08-770 Accreditation of Voting System Testing Laboratories
laboratory responses describing how unmet requirements were addressed 

within specified time frames, used the responses in making accreditation 

decisions, and notified EAC of its decisions via letters of recommendation. 

Furthermore, NIST has recently begun reaccreditation reviews at two 

laboratories, as required.

However, documentation does not show that NIST has consistently 

followed other aspects of its defined approach. Our analysis of the 

checklists that are to be used to both guide and document a given 

assessment, including identifying unmet requirements and capturing 

assessor comments and observations, shows some differences. For 

example: 

• One type of checklist (the supplemental handbook checklist) was 

prepared for only two of the four laboratory assessments. According to the 

program manager, this is because even though a draft revision of this 

checklist was actually used to assess the other two laboratories, the 

assessment results were recorded on a different checklist (the general 

handbook checklist). While this is indicated on one of the two checklists,

it is not indicated on the other. 

• On the checklist used for one laboratory, an assessor marked several 

sections as “TA” with no explanation as to what this means. Also, the 

checklist used for another laboratory did not identify whether most of the 

requirements were met or not met. Further, the checklist for a third 

laboratory had one section marked as “not applicable” but included no 

explanation as to why that section did not apply, while the checklist for a 

different laboratory marked the same section as “not applicable” but 

included a reason for doing so. 

Notwithstanding these differences, the program manager told us that each

laboratory was assessed using the same requirements and all assessments 

to date were performed in a consistent manner. On the basis of available 

documentation, however, we could not verify that this is the case. As a 

result, it is not clear that NIST has consistently followed its defined 

approach. 

Available documentation also does not show that NIST followed other 

aspects of its approach. For example: 

• The program handbook states that each laboratory is to identify the 

requested scope of accreditation in its application package. However, our 

analysis of the four application packages shows that two laboratories did 

Page 24 GAO-08-770 Accreditation of Voting System Testing Laboratories
not specify a requested scope of accreditation. According to the program 

manager, the scope of accreditation for all laboratories was the 2002 VSS 

and 2005 VVSG because, even though the latter standards were not yet in

effect at the time, they were anticipated to be in effect in the near future.26

However, NIST did not have documentation that notified the laboratories 

of this scope of accreditation or that indicated whether this scope was 

established by EAC, NIST, or the laboratories. 

• The program handbook states that after receiving a laboratory’s 

application package, NIST will acknowledge its receipt in writing and will

inform the laboratory of the next steps in the accreditation process. 

However, NIST did not have documentation demonstrating that this was 

done. According to the program manager, this was handled via telephone 

conversations. However, representatives for several laboratories noted 

that these calls did not clearly establish expectations, adding that some 

expectations were not communicated until the NIST team assessors 

arrived to conduct the on-site assessment. 

The program manager stated that these deviations from the defined 

approach are attributable to the relative newness of the program, but 

despite these discrepancies, each laboratory was assessed consistently. 

However, we could not verify this, and thus it is not clear that NIST has 

consistently followed its defined approach. According to this official, 

future versions of the program handbook would address these limitations. 

However, documented plans for doing so have not been developed. 

EAC has recently defined its voting system laboratory accreditation 

approach in a draft program manual. However, this draft manual omits 

important content. While addressing relevant HAVA requirements, the 

draft manual does not adequately define key accreditation factors that 

NIST has identified, and a key accreditation feature that we have 

previously reported as being integral to an effective accreditation 

program. Moreover, not all factors and features that the draft manual does 

include have been defined to a level that would ensure thorough, 

consistent, and verifiable implementation. Because this manual was not 

available for EAC to use on the four laboratory accreditations that it has 

completed, the accreditations were performed using a largely 

undocumented series of steps. As a result, the thoroughness and 

consistency of these accreditations is not clear. According to EAC 

EAC’s Recently 

Drafted Accreditation 

Approach and Its 

Earlier Performed 

Laboratory 

Accreditations Lack 

Key Effectiveness 

Factors and Features 

 26As mentioned earlier, these guidelines became effective in December 2007. 

Page 25 GAO-08-770 Accreditation of Voting System Testing Laboratories
officials, these gaps are due to the agency’s limited resources being 

focused on other issues, and will be addressed as its accreditation 

program evolves. However, they said that they do not yet have

documented plans to accomplish this. Until EAC fully defines a repeatable 

VSTL accreditation approach, it will be challenged in its ability to treat all 

laboratories consistently and produce verifiable results. 

In February 2008, EAC issued a draft version of a VSTL accreditation 

program manual27 for public comment. According to HAVA, EAC’s 

accreditation program is to meet certain requirements. Specifically, it is to 

provide for voting system hardware and software testing, certification, 

decertification, and recertification by accredited laboratories. 

Additionally, it is to base laboratory accreditation decisions, including 

decisions to revoke an accreditation, on a vote of the commissioners, and 

it is to provide for a published explanation of any commission decision to 

accredit any laboratory that was not first recommended for accreditation 

by NIST. 

To EAC’s credit, its draft accreditation program manual addresses each of

these requirements. First, the manual defines the role that the laboratories 

are to play relative to voting system testing, certification, recertification 

and decertification, and it incorporates by reference an EAC companion 

voting system certification manual28 that defines requirements and process

steps for voting system testing and certification-related activities. 

With respect to the remaining three HAVA requirements, the draft EAC 

accreditation manual also requires (1) that the commissioners vote on the 

accreditation of laboratories recommended by NIST for accreditation, (2) 

that EAC publish an explanation for the accreditation of any laboratory 

not recommended by NIST for accreditation, and (3) that the 

commissioners vote on the proposed revocation of a laboratory’s 

accreditation. 

According to EAC officials, its draft approach incorporates HAVA 

requirements because the commission is focused on meeting its legal 

obligations in all aspects of its operations, including VSTL accreditation. In

EAC Has Defined a Draft 

Accreditation Approach 

that Meets HAVA

Requirements 

 27EAC, Voting System Test Laboratory Accreditation Program Manual (draft, Washington,

D.C., February 2008). 

28EAC, Testing and Certification Program Manual (Washington, D.C.: January 2007). 

Page 26 GAO-08-770 Accreditation of Voting System Testing Laboratories
doing so, EAC has addressed one important aspect of having an effective 

accreditation program. 

Beyond addressing relevant HAVA requirements, EAC’s draft accreditation 

manual defines an accreditation process, including program phases, 

requirements, and certain evaluation criteria. However, it does not do so in 

a manner that fully satisfies factors that NIST has reported can affect the 

effectiveness of accreditation programs.29 Moreover, it does not adequately 

address a set of features that our research shows are common to federal 

accreditation programs30 and that can influence a program’s effectiveness. 

According to EAC officials, these factors and features are not fully 

addressed in the draft program manual because its accreditation program 

is still in its early stages of development and is still evolving. Until they are 

fully addressed, EAC’s accreditation program’s effectiveness will be 

limited. 

According to NIST, having confidence in and ensuring appropriate use of

an accredited testing laboratory requires that accreditation stakeholders 

have an adequate understanding of the accreditation process, scope, and 

related criteria. NIST further reports that confidence in the accreditation 

process can be traced to a number of factors that will influence the 

thoroughness and competence of accreditation programs, and thus these 

factors can be viewed as essential accreditation program characteristics. 

They include having

EAC Draft Approach Does 

Not Adequately Define Key 

Accreditation-Related 

Steps and Decision Criteria 

Key NIST Accreditation Factors 

Not Fully Addressed 

• published procedures governing how the accreditation program is to be 

executed, such as procedures for granting, maintaining, modifying, 

suspending, and withdrawing accreditation; 

• specific instructions, steps, and criteria for those who conduct an 

accreditation assessment (assessors) to follow, such as a test methodology 

that is acceptable to the accreditation program; 

• knowledgeable and experienced assessors to execute the instructions and 

steps and apply the related criteria; and 

 29NIST, The ABC’s of the U.S. Conformity Assessment System, NISTIR 6014. 

30GAO, Laboratory Accreditation: Requirements Vary Throughout the Federal 

Government, GAO/RCED-89-102 (Washington, D.C.: Mar. 28, 1989). 

Page 27 GAO-08-770 Accreditation of Voting System Testing Laboratories
• complete records on the data collected, results found, and reports 

prepared relative to each assessment performed. 

EAC’s draft accreditation program manual addresses one of these factors 

but it does not fully address the other three. (See table 2.) For example, 

while the manual requires that EAC maintain records, it only addresses the 

retention of records associated with the testing of voting systems and not 

those associated with the accreditation of laboratories. EAC officials told 

us that testing records are meant to include accreditation records, 

although they added that this is not explicit in the manual and needs to be 

clarified. Further, the manual is silent on the steps to be followed and 

criteria to be applied in reviewing a laboratory’s application and the 

qualifications required for accreditation reviewers. By not fully addressing

these factors, EAC increases the risk that its accreditation reviews will not 

be performed consistently and comprehensively.

Table 2: Summary of Extent to Which EAC Draft Approach Addresses NISTIdentified Accreditation Factors 

Accreditation program factor Addressed by EAC?

Published accreditation program procedures Yes 

Specific accreditation instructions No 

Established accreditation personnel qualifications No

Adequate maintenance of records Partially 

Source: GAO analysis of EAC data. 

As we have previously reported,31 the nature and focus of federal programs 

for accrediting laboratories vary, but nevertheless include certain common 

features. In particular, these programs require laboratories to provide 

certain information to the accrediting body, and they provide for 

evaluation of this information by the accrediting body in making an 

accreditation determination. As we reported, the required information is to 

include, among other things, the laboratory’s (1) organizational 

information, (2) records and record-keeping policy, (3) test methods and 

procedures, (4) conflict of interest policy, and (5) financial stability. 

To its credit, EAC’s draft accreditation manual provides for laboratories to 

submit information relative to each of these features that are common to 

federal accreditation programs. For example, it provides for laboratories 

All but One Key GAO-Reported 

Accreditation Program Feature 

Has Been Addressed 

 31GAO/RCED-89-102. 

Page 28 GAO-08-770 Accreditation of Voting System Testing Laboratories
to submit organizational information, such as location(s), ownership, and

organizational chart; a written policy for maintaining accreditation-related

records for 5 years; conflict of interest policies and procedures; testrelated polices and procedures, as well as system-specific test plans; and 

financial information needed to demonstrate stability. Moreover, for four 

of the five features, the manual identifies the specific types of information 

needed for accreditation and how the information is to be evaluated, 

including the criteria that are to be used in evaluating it. However, for the 

financial stability feature, the manual does not describe what specific 

documents are required from the laboratory to satisfy this requirement, 

nor does the manual indicate how information provided by a laboratory 

will be evaluated. 

Table 3: Summary of Extent to Which EAC Satisfies Features Common to Federal 

Accreditation Programs 

Accreditation program 

feature 

EAC requires 

information? 

EAC specifies

information scope

and level of detail?

EAC specifies how

information is to be 

evaluated and

criteria to be used?

Organizational 

information

Yes Yes Yes

Records and recordkeeping Yes Yes Yes

Test methods and 

procedures 

Yes Yes Yes

Conflict of interest policy Yes Yes Yes 

Assurance of financial 

stability 

Yes No No

Source: GAO analysis of EAC data. 

At the time of our review, EAC’s Director of Voting System Testing and 

Certification32 told us that the draft accreditation manual was to be 

submitted for approval and that this draft did not address all of the 

limitations cited above.33 For example, it would not contain the 

information needed and the evaluation approach and criteria to be used in

 32The EAC Director of Voting System Testing and Certification also manages EAC’s VSTL 

accreditation program.

33On August 4, 2008, EAC reported that it approved an initial version of its program manual.

However, we did not evaluate this initial version to determine the extent to which it 

addresses limitations that we found in the draft manual because it was approved after EAC

provided comments on a draft of, and we had finalized, this report. 

Page 29 GAO-08-770 Accreditation of Voting System Testing Laboratories
making determinations about financial stability because this decision is to

be based on what the director referred to as a “reasonableness” test that 

involves EAC evaluation of the information relative to that provided by 

other laboratories. Further, while EAC officials said that they plan to 

evolve their approach to VSTL accreditation and to address these gaps, 

EAC does not have documented plans for accomplishing this. Without 

clearly defining information to be used and how it is to be used, EAC 

increases the risk that financial stability determinations will not be 

consistently and thoroughly made. 

As of May 2008, EAC has accredited four laboratories,34 but the 

documentation associated with each of these accreditations is not 

sufficient to recreate a meaningful understanding of how each evaluation 

was performed and how decisions were made, and thus, the bases for each 

accreditation were not clear. Specifically, each of the accreditations 

occurred before EAC had defined its approach for conducting them. 

Because of this, EAC performed each one using a broadly defined process 

outlined in a letter to each laboratory and an associated checklist that only

indicated whether certain documents were received. Our analysis of these 

letters showed that the correspondence sent to each laboratory was all the 

same, identifying three basic review steps to be performed and citing a list 

of documents that the laboratories were to provide as part of their 

applications.

35 However, the letters did not describe in any manner how 

EAC would review the submitted material, including the criteria to be 

used. 

According to EAC officials, the review steps were not documented. 

Instead, they were derived by a single reviewer using (1) the applications

and accompanying documents submitted by the laboratories, (2) 

familiarity with the materials used by the state election directorssponsored accreditation program, and (3) the judgment of each reviewer.

Further, while the reviews were supported by a checklist that covered 

Available Documentation 

Does Not Demonstrate 

EAC’s Basis for 

Accrediting Laboratories 

to Date 

 34According to the NIST program manager, a fifth laboratory has been accredited by NVLAP

but not yet recommended to EAC. 

35The steps were to: 1) provide information, such as the laboratory’s conflict of interest 

policy, evidence of insurance coverage limits, and audited financial statements; 2) provide

a signed, standardized letter of agreement to abide by the EAC program terms; and 3) 

provide a signed certification of laboratory practices and conditions, such as having 

policies in place with respect to personnel practices, record-keeping requirements, and 

financial stability. 

Page 30 GAO-08-770 Accreditation of Voting System Testing Laboratories
each of the items that was to be included in the laboratory applications 

and provided space for the reviewer(s) to make notes relative to each of 

these items, the checklists did not include any guidance or methodology, 

including criteria, for evaluating the submitted items. Rather, the EAC 

accreditation program director told us that he was the reviewer on all the 

accreditations and he applied his own, but undocumented, tests for 

reasonableness in deciding on the submissions’ adequacy and

acceptability. 

Our analysis of the checklists for each laboratory accreditation showed 

that while the same checklist was used for each laboratory, the checklists 

did not provide a basis for evaluating and documenting the basis for the 

sufficiency of those documents. In some cases, additional communications 

occurred between the reviewer and the laboratory to obtain additional 

documents. However, no documentation was available to demonstrate 

what standards or other criteria the laboratories were held to or how their 

submissions were otherwise reviewed. For example, each of the checklists 

indicated that each laboratory provided “a copy of the laboratory’s conflict 

of interest policy.” However, they did not specify, for example, whether 

the policy adequately addressed particular requirements. Nevertheless, for 

three of the four accredited laboratories, documentation shows that EAC 

sought clarification on or modification to the policies provided, thus 

suggesting that some form of review was performed against more detailed

requirements. Similarly, while the checklists indicate that the laboratories 

disclosed their respective coverage limits for general liability insurance 

policies, and in one case EAC communicated to the laboratory that the 

limits appeared to be low, no documentation specifies the expected

coverage limits. According to the EAC Director of Voting System Testing 

and Certification, this determination was made after comparing limits 

among the laboratories and was not based on any predetermined 

threshold. Further, while the checklists indicate that each laboratory 

provided audited financial statements, there is no documentation 

indicating how these statements were reviewed.

According to the EAC program director, the lack of documentation 

demonstrating the basis for EAC’s laboratory accreditations is due to the 

need at the time to move quickly in accrediting the laboratories and the 

fact that use of the same individual to review the accreditation evaluation

negated the need for greater documentation. Without such documentation, 

however, we could not fully establish how the accreditations were 

performed, including whether there was an adequate basis for the 

Page 31 GAO-08-770 Accreditation of Voting System Testing Laboratories
accreditation decisions reached and whether they were performed 

consistently.

The effectiveness of our nation’s overall election system depends on many 

interrelated and interdependent variables, including the security and 

reliability of voting systems. Both NIST and EAC play critical roles in 

ensuring that the laboratories that test these two variables have the 

capability, experience, and competence necessary to test a voting system 

against the relevant standards. NIST has recently established an 

accreditation program that largely accomplishes this, and while EAC is not 

as far along, it has a foundation upon which it can build. 

However, important elements are still missing from both programs. 

Specifically, the current NIST approach does not define requirements for 

assessor qualifications and training or ensure that assessments are fully 

documented. Additionally, EAC has not developed program management 

practices that are fully consistent with what NIST has found to be 

hallmarks of an effective accreditation program, nor has the agency

adequately specified how evaluations are to be performed and 

documented. As a result, opportunities exist for NIST and EAC to further 

define and implement their respective programs in ways that promote 

greater consistency, repeatability, and transparency—and thus improve 

the results achieved. It is also important for NIST and EAC to follow 

through on their stated intentions to evolve their respective programs, 

building on what they have already accomplished through the 

development and execution of well-defined plans of action. If they do not,

both will be challenged in their ability to consistently provide the 

American people with adequate assurance that accredited laboratories are 

qualified to test the voting systems that will eventually be used in U.S. 

elections. 

To help NIST in evolving its VSTL accreditation program, we recommend 

that the Director of NIST ensure that the accreditation program manager 

develops and executes plans that specify tasks, milestones, resources, and

performance measures that provide for the following two actions: 

Conclusions 

Recommendations for 

Executive Action 

• Establish and implement transparent requirements for the technical 

qualifications and training of accreditation assessors. 

Page 32 GAO-08-770 Accreditation of Voting System Testing Laboratories
• Ensure that each laboratory accreditation review is fully and consistently 

documented in accordance with NIST program requirements. 

To help EAC in evolving its VSTL accreditation program, we recommend 

that the Chair of the EAC ensure that the EAC Executive Director 

develops and executes plans that specify tasks, milestones, resources, and

performance measures that provide for the following action:

• Establish and implement practices for the VSTL accreditation program 

consistent with accreditation program management guidance published by 

NIST and GAO, including 

• documentation of specific accreditation steps and criteria to guide 

assessors in conducting each laboratory review;

• transparent requirements for the qualifications of accreditation 

reviewers; 

• requirements for the adequate maintenance of records related to the 

VSTL accreditation program; and 

• requirements for determining laboratory financial stability. 

Both NIST and EAC provided written comments on a draft of this report, 

signed by the Deputy Director of NIST and the Executive Director of EAC, 

respectively. These comments are described below along with our 

response to them. 

In its comments, NIST stated that it appreciates our careful review of its 

VSTL program and generally concurs with our conclusions that its 

program must continue to evolve and improve. However, NIST also 

provided comments to clarify the current status of the program relative to

three of our findings. 

Agency Comments 

and Our Evaluation 

• With respect to our finding that NIST’s defined approach for accrediting 

VSTLs does not cite explicit qualifications for the persons who conduct 

the technical assessments, the institute stated that it does explicitly cite

assessor qualifications for its overall national laboratory accreditation 

program, adding that this approach to specifying assessor qualifications 

has a proven record of success. It also stated that the overall program’s 

management manual requires all assessors to meet defined criteria in such

areas as laboratory experience, assessment skills, and technical 

knowledge, and that candidate assessors must submit information 

Page 33 GAO-08-770 Accreditation of Voting System Testing Laboratories
addressing each of these areas as well as factors addressing technical 

competence in a given laboratory’s focus area (e.g., voting systems). 

Further, it stated that candidate assessors’ qualification ratings and 

rankings are captured in work sheets. 

In response, we do not disagree with any of these statements. However, 

our finding is that NIST’s defined approach for VSTL accreditation does 

not specify requirements for persons who assess those laboratories that 

specifically test voting systems. In this regard, NIST’s own written 

comments confirm this, stating that specific requirements for assessors 

are not separately documented for each of its national laboratory 

accreditation programs, such as the VSTL program. Therefore, we have

not modified this finding or the related recommendation. 

• Regarding our finding that NIST’s defined approach for accrediting VSTLs 

has not always cited the current voting system standards, the institute 

affirmed this in its comments by stating that the VSTL program handbook 

that it provided to us only cites the 2002 system standards, as these were 

the only standards in place when the handbook was published. However, 

NIST also noted that when the 2005 system guidelines were adopted in 

December 2005, it began the process of updating the handbook and 

associated assessment checklist, and that the handbook update was 

recently finalized for publication and is now being used. 

In response, we stand by our finding that NIST’s defined approach has not

always cited the current voting system standards, which NIST 

acknowledges in its comments. However, we also recognize that NIST has 

recently addressed this inconsistency by finalizing its new handbook and 

the associated assessment checklist. In light of NIST’s recent actions, we 

have updated the report to acknowledge the finalization of the handbook

and checklist, and removed the associated recommendation that was 

contained in our draft report for NIST to ensure that its defined approach

addresses all required voting system standards. 

• Regarding our finding that available documentation from completed 

accreditations does not show that NIST has consistently followed all 

aspects of its defined approach, the institute stated that, among other 

things, all required documents for its VSTL accreditation program are 

currently in use and reflect the recent update to its handbook and 

checklist, and that all these documents are securely maintained. 

In response, we do not question these statements; however, they are not 

pertinent to our finding. Specifically, our finding is that the four completed 

accreditations that we reviewed were not consistently documented. As we 

Page 34 GAO-08-770 Accreditation of Voting System Testing Laboratories
state in our report, we reviewed the documentation associated with the 

accreditation assessments for these four laboratories, and we found that 

all four were not documented in a similar manner, even though they were

based on the same version of the program handbook. For example, neither 

the laboratory notifications of the scope of the assessment nor the next 

steps in the accreditation process were consistently documented. 

Therefore, we have not modified our finding, but have slightly modified 

our recommendation to make it clear that its intent is to ensure that all 

phases of the accreditation review are fully and consistently documented.

In its comments, EAC described our review and report as being helpful to 

the commission as it works to fully develop and implement its VSTL 

program. It also stated that it agrees with the report’s conclusions that 

additional written internal procedures, standards, and documentation are 

needed to ensure more consistent and repeatable implementation of the 

program. The commission added that it generally accepts our 

recommendations and will work hard to implement them. To assist it in 

doing so, it sought clarification about two of our recommendations, as 

discussed below. 

• EAC stated that the recommendation in our draft report for the 

commission to develop specific accreditation steps and criteria was 

broadly worded, and thus the recommendation’s intent was not clear. EAC 

also stated that it interpreted the recommendation to mean that it should 

define internal instructions to guide assessors in performing an 

accreditation, and that the recommendation was not intended to have any

impact on its published requirements and procedures governing, for 

example, granting, suspending, or withdrawing an accreditation. We agree 

with EAC’s interpretation, as it is in line with the intent of our

recommendation. To avoid the potential for any future misunderstanding, 

we have modified the wording of the recommendation to clarify its intent. 

• EAC stated that the recommendation in our draft report for the 

commission to develop transparent technical requirements for the 

qualifications of its assessors may be confusing because, as we state in our 

report, only NIST performs a technical accreditation review, as EAC’s 

review is administrative, non-technical in nature. To avoid the potential for 

any confusion, we have modified the wording of the recommendation to 

eliminate any reference to technical qualification requirements. 

Page 35 GAO-08-770 Accreditation of Voting System Testing Laboratories
We are sending copies of this report to the Ranking Member of the House 

Committee on House Administration, the Chairman and Ranking Member 

of the Senate Committee on Rules and Administration, the Chairmen and 

Ranking Members of the Subcommittees on Financial Services and 

General Government, Senate and House Committees on Appropriations, 

and the Chairman and Ranking Member of the House Committee on 

Oversight and Government Reform. We are also sending copies to the 

Chair and Executive Director of EAC, the Secretary of Commerce, the 

Deputy Director of NIST, and other interested parties. We will also make 

copies available to others on request. In addition, this report will be 

available at no charge on the GAO Website at http://www.gao.gov. 

Should you or your staffs have any questions on matters discussed in this 

report, please contact me at (202) 512-3439 or at hiter@gao.gov. Contact 

points for our Offices of Congressional Relations and Public Affairs may 

be found on the last page of this report. Key contributors to this report are 

listed in appendix IV. 

Sincerely yours, 

Randolph C. Hite 

Director, Information Technology Architecture 

 and System Issues 

Page 36 GAO-08-770 Accreditation of Voting System Testing Laboratories
Appendix I: Objectives, Scope, and Appendix I: Objectives, Scope, and Methodology 

Methodology 

Our objectives were to determine whether the National Institute of 

Standards and Technology (NIST) and the Election Assistance 

Commission (EAC) have defined effective voting system testing laboratory 

(VSTL) accreditation approaches, and whether each is following its 

defined approach. 

To determine whether NIST has defined an effective accreditation 

approach, we reviewed documentation from its VSTL accreditation 

program, such as handbooks and program manuals for the National 

Voluntary Laboratory Accreditation Program (NVLAP), of which the VSTL 

accreditation program is a part. In doing so, we compared these 

documents with applicable statute, guidance, and best practices, primarily 

the Help America Vote Act of 2002 (HAVA), internationally recognized 

standards from the International Organization for Standardization (ISO), 

and federal accreditation program management guidance published by 

NIST. We compared program documentation with HAVA’s NIST-specific 

accreditation requirements to determine the extent to which the agency

was fulfilling its HAVA responsibilities. We also reviewed program 

documentation against ISO/IEC 17011,1

 which establishes general 

requirements for accreditation bodies, and ISO/IEC 17025,2

 which 

establishes the general requirements for assessing the competence of 

laboratories, to determine the extent to which NIST’s accreditation 

program was based on internationally recognized standards. We also 

compared the documentation against NIST publication NISTIR 6014,3

which contains sections that provide guidance for laboratory accreditation 

programs, to determine whether the VSTL accreditation program had 

defined other elements of effective accreditation programs. We also 

interviewed the voting accreditation program manager to determine how 

these documents were used to guide the program. 

To determine whether NIST has followed its defined approach, we 

examined artifacts from the accreditation assessments of five VSTLs, 

including one laboratory accredited by NVLAP, but not yet recommended 

to EAC. This material included completed assessment checklists derived 

 1

ISO, ISO/IEC 17011: Conformity Assessment: General Requirements for Accreditation 

Bodies Accrediting Conformity Assessment Bodies (Geneva, Switzerland: Feb. 15, 2005).

2

ISO, ISO/IEC 17025: General Requirements for the Competence of Testing and

Calibration Laboratories (Geneva, Switzerland: May 15, 2005). 

3

NIST, The ABC’s of the U.S. Conformity Assessment System, NISTIR 6014 (Gaithersburg, 

Md.: April 1997). 

Page 37 GAO-08-770 Accreditation of Voting System Testing Laboratories
Appendix I: Objectives, Scope, and 

Methodology 

from the accreditation program handbooks, additional documents 

supporting the assessments, and laboratory accreditation applications and 

supporting documentation. We compared artifacts from these assessments 

to program guidance to determine the extent to which the defined process 

was followed. In addition, we interviewed officials from NIST and NIST 

contract assessors and officials from EAC and the four EAC-accredited 

VSTLs to understand how the NIST process was implemented and how it

related to the process managed by EAC. 

To determine whether EAC has defined an effective accreditation 

approach, we reviewed documentation from its VSTL accreditation 

program, such as the draft Voting System Test Laboratory Accreditation 

Program Manual.

4

 In doing so, we compared this document with 

applicable statute and best practices, primarily HAVA and federal 

accreditation program management guidance published by NIST. We 

compared the draft program manual with HAVA’s EAC-specific 

accreditation requirements to determine the extent to which the agency

was fulfilling its HAVA responsibilities. We also compared the 

documentation against the accreditation guidance in NISTIR 6014 to 

determine whether the accreditation program had defined other elements

of effective accreditation programs. We also interviewed the EAC voting 

program director and executive director to determine how these 

documents were used to guide the program and to understand EAC’s 

defined accreditation approach prior to the development of the draft 

manual. 

To determine whether EAC has followed its defined approach, we 

compared artifacts from the accreditation reviews of four VSTLs. We did 

not review a fifth laboratory, which had been accredited by NVLAP, but 

not yet recommended to EAC. The materials reviewed included checklists 

completed by EAC in the absence of an approved program manual. In 

doing so, we compared the review artifacts to accreditation program 

requirements, as communicated to the laboratories, to determine the 

extent to which the agency followed its process, as verbally described to 

us. We did not compare accreditation submissions or EAC review artifacts 

with the draft accreditation manual because agency officials stated that 

the draft manual had not been used in the review of any laboratory. In 

 4

EAC, Voting System Test Laboratory Accreditation Program Manual (draft, Washington,

D.C.: February 2008). EAC approved the program manual for publication in July 2008; 

however this was accomplished too late for GAO to review the manual’s contents for this 

report. 

Page 38 GAO-08-770 Accreditation of Voting System Testing Laboratories
Appendix I: Objectives, Scope, and 

Methodology 

addition, we interviewed officials from NIST, EAC, and the four EACaccredited VSTLs5

 to understand how the EAC process was implemented 

and how it related to the process managed by NIST. 

To assess data reliability, we reviewed program documentation to 

substantiate data provided in interviews with knowledgeable agency

officials. We have also made appropriate attribution indicating the data’s 

sources. 

We conducted this performance audit at EAC and NIST offices in 

Washington, D.C., and Gaithersburg, Maryland, respectively, from 

September 2007 to September 2008 in accordance with generally accepted 

government auditing standards. Those standards require that we plan and

perform the audit to obtain sufficient, appropriate evidence to provide a 

reasonable basis for our findings and conclusions based on our audit 

objectives. We believe that the evidence obtained provides a reasonable

basis for our findings and conclusions based on our audit objectives. 

 5

In June 2008, NIST granted NVLAP accreditation to a fifth laboratory, CIBER Inc., and 

recommended it for EAC accreditation. As of August 2008, that accreditation had yet to be 

granted. 

Page 39 GAO-08-770 Accreditation of Voting System Testing Laboratories
Appendix II: Comments from the National Appendix II: Comments from the National Institute of Standards and Technology 

Institute of Standards and Technology 

Page 40 GAO-08-770 Accreditation of Voting System Testing Laboratories
Appendix II: Comments from the National 

Institute of Standards and Technology 

Page 41 GAO-08-770 Accreditation of Voting System Testing Laboratories
Appendix II: Comments from the National 

Institute of Standards and Technology 

Page 42 GAO-08-770 Accreditation of Voting System Testing Laboratories
Appendix II: Comments from the National 

Institute of Standards and Technology 

Page 43 GAO-08-770 Accreditation of Voting System Testing Laboratories
Appendix II: Comments from the National 

Institute of Standards and Technology 

Page 44 GAO-08-770 Accreditation of Voting System Testing Laboratories
Appendix III: Comments from the Election

Appendix III: Comments from the Election Assistance Commission 

Assistance Commission 

Page 45 GAO-08-770 Accreditation of Voting System Testing Laboratories
Appendix III: Comments from the Election

Assistance Commission 

Page 46 GAO-08-770 Accreditation of Voting System Testing Laboratories
Appendix III: Comments from the Election

Assistance Commission 

Page 47 GAO-08-770 Accreditation of Voting System Testing Laboratories
Appendix III: Comments from the Election

Assistance Commission 

Page 48 GAO-08-770 Accreditation of Voting System Testing Laboratories
Appendix IV: 

A

GAO Contact and Staff 

cknowledgments 

Page 49 GAO-08-770

Appendix IV: GAO Contact and Staff 

Acknowledgments 

Randolph C. Hite, (202) 512-3439 or hiter@gao.gov

Paula Moore, Assistant Director; Justin Booth; Timothy Case; Neil 

Doherty; Timothy Eagle; Nancy Glover; Dave Hinchman; Rebecca LaPaze; 

Freda Paintsil; Nik Rapelje; and Jeffrey Woodward made key contributions 

to this report. 

Accreditation of Voting System Testing Laboratories

GAO Contact 

Staff 

Acknowledgments 

(310646) 
GAO’s Mission The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and policies; 

and provides analyses, recommendations, and other assistance to help

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of 

accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost

is through GAO’s Web site (www.gao.gov). Each weekday, GAO posts 

newly released reports, testimony, and correspondence on its Web site. To

have GAO e-mail you a list of newly posted products every afternoon, go 

to www.gao.gov and select “E-mail Updates.” 

The first copy of each printed report is free. Additional copies are $2 each. 

A check or money order should be made out to the Superintendent of 

Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. Orders 

should be sent to: 

U.S. Government Accountability Office 

441 G Street NW, Room LM 

Washington, DC 20548 

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470 

Ralph Dawn, Managing Director, dawnr@gao.gov, (202) 512-4400 

U.S. Government Accountability Office, 441 G Street NW, Room 7125 

Washington, DC 20548 

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800 

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548 

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Mail or Phone 

To Report Fraud,

Waste, and Abuse in 

Federal Programs 

Congressional 

Relations 

Public Affairs 

PRINTED ON RECYCLED PAPER