U. S. ELECTION ASSISTANCE COMMISSION VOTING SYSTEM TESTING AND CERTIFICATION PROGRAM

 1225 New York Avenue, NW, Suite 1100 

 Washington, DC. 20005 

May 2, 2008 

To: Election Officials 

From: Brian Hancock, Director 

 United States Election Assistance Commission 

 Testing and Certification Program 

RE: An update on the state of the EAC’s Testing and Certification Program 

All, 

It has been 15 months since the EAC opened the doors to its voting system testing and 

certification program, and although much has been accomplished in that time, we have 

also heard several concerns regarding our program. I wanted to take this opportunity to 

address some of the concerns that have been expressed to me by state and local officials 

in recent months and to make you aware that an annual report outlining all our activities, 

in detail, will be forwarded to you within the next several weeks. 

Current Program Challenges 

Since the initiation of our Certification Program, there has been much anticipation 

surrounding the EAC certifying its first system. There have been many concerns 

expressed as to why the EAC has not yet certified a voting system and if any systems will 

be certified in time for the 2008 General Election. I want to take this opportunity to 

address those concerns and explain why we are being deliberate in our review of 

applicant voting systems. 

The EAC Certification Program represents the first time the Federal Government is 

testing and certifying voting systems. Prior to the creation of the EAC’s program, voting 

systems were, as many of you know, qualified by the National Association of State 

Election Directors (NASED). NASED operated with little to no funding or staff 

dedicated to this effort. When the EAC’s program was established in January of 2007 the 

EAC made the difficult but correct decision not to “grandfather” NASED qualified 

systems. Instead the EAC chose to start anew and require that all systems applying for 

EAC testing and certification must be fully tested as a new system under our program in 
order to receive an EAC certification. This decision not to grandfather NASED qualified 

systems had the predictable effect of lengthening the initial testing and certification 

process. It is important to note that after a system receives an EAC certification any 

modifications to that system will be tested under a more streamlined process which will 

test the modification (delta testing) and those systems or subsystems altered or impacted 

by the modification (regression testing). The system will then be subjected to integration 

testing to ensure overall functionality. In addition, the EAC program has developed 

procedures to deal with De Minimus changes to a voting system and to deal with 

emergency Pre-Election modifications to EAC certified voting systems. These 

procedures are outlined in Sections 3.5 and 3.6 of the Certification Program Manual. 

During our first year of operation, the EAC’s Certification Program has experienced 

reluctance from many of the participating parties to accept the need to change the very 

culture surrounding the testing and certification of voting systems. The EAC has 

intentionally moved to shift the paradigm away from the reliance on previous practices 

and towards a more structured and rigorous process. The EAC understands the need to 

have certified systems available to jurisdictions and appreciates that many states rely on 

EAC certification as a prerequisite for implementing a voting system in their State. That 

is precisely why the EAC is committed to fostering a certification environment that fully 

evaluates the voting systems against the applicable standards in order to ensure that they 

are capable of performing securely, accurately and reliably in an election. 

As a result of this new certification paradigm, the EAC has seen several predictable 

issues arise that have had the effect of slowing the certification process. Voting system 

manufacturers were slow at times to adjust their business practices to the new 

requirements of the EAC’s program. Over the past year the EAC has issued three notices 

of non-compliance to manufacturers who have failed to abide by the EAC’s program 

requirements and requested them to come into compliance with the program or face 

possible suspension of their registration. In addition the EAC has issued numerous letters 

to manufacturers regarding the key differences between the EAC and NASED program 

and the need for manufacturers to implement additional measures to meet the EAC’s 

program standards. Also, the EAC has held several meetings with its registered 

manufacturers in order to create a better understanding of our expectations and to better 

open the lines of communication between the two groups. All official correspondence 

between the EAC and the manufacturers can be found at 

http://www.eac.gov/voting%20systems/voting-system-certification/correspondence. The 

EAC has been pleased to see that in general the manufacturers have been willing to 

cooperate and even more willing to offer ideas on how better to streamline the EAC’s 

process. 

The paradigm shift has also had a profound effect on the Voting System Test 

Laboratories (VSTL). The differences between the NASED and EAC’s program has 

greatly affected the way the VSTLs (particularly those who had formerly participated in 

the NASED program) prepared test plans, issued test reports, and evaluated the voting 

systems. As evidenced by the multiple drafts of test plans received for each voting 

system, the VSTLs had trouble adjusting to the level of detail required for an EAC 

2
approved test plan. However, the VSTLs have shown continuing improvements to their 

processes and procedures and are now offering test plans that have the level of detail 

necessary to receive approval from the EAC. 

Another challenge the VSTLs have encountered is the current lack of a standardized test 

methodology. The EAC and NIST are currently pursuing ways to expedite the test 

methodology development process in order to provide the VSTLs a set of test methods 

for the 2005 VVSG in the near future. This set of test methods will greatly increase the 

efficiency of the test plan and test report review process since the general test method 

will remain consistent from system to system with changes incorporated to accommodate 

the unique variations between different voting systems. 

The VSTLs are currently undergoing the required reaccreditation review by NVLAP. 

The first laboratory accredited by EAC/NVLAP, SysTest Labs. Inc., underwent its 

review in early March and the second laboratory, iBeta Quality Assurance Inc., had its 

review last week. It is the EAC’s intent that the NVLAP reassessments will serve to 

validate the EAC’s expectations for test methodology, experience and training of test 

engineers, and quality monitoring for systems while in the testing processes. The EAC 

remains extremely confident in its VSTLs and appreciates their willingness to cooperate 

with the EAC and in many cases change their laboratory practices in order to meet the 

expanded requirements of the EAC’s program. 

Conclusion

The passage of HAVA and the creation of the EAC’s Certification Program has created a 

new environment for the testing and certification of voting systems. The basic premise 

and procedures of the certification program are sound and borrow important aspects from 

successful certification efforts in other agencies. In addition, the certification program 

model is based upon U.S. and international Standards (ISO/IEC 65) used in numerous 

other industries. The full benefits and value of the EAC program will only emerge with 

time and experience. 

Other Federal certification efforts reflect the importance of allowing due time to develop 

mature programs and processes. The Federal Communications Commission (FCC) was 

created by the Communications Act of 1934 and has been testing and certifying 

communications equipment since that time. Depending on the FCC workload, 

certification of products averages from 60 to 100 days. To speed up certifications (and 

time to market for manufacturers) in 2000, the FCC allowed manufacturers of certain 

kinds of equipment to self-certify their products under a Declaration of Conformity 

program. It also instituted a program of allowing contract examiners working at 

organizations called Telecommunications Certification Bodies (TCBs) to perform the test 

report and certification function under FCC authority and supervision. There are 

approximately 13 TCBs and over 500 test labs doing the testing and preparing the reports 

that they review. The TCB examiners now average approximately 35 days to review a 

test report and certify a product. It is important to remember that these products need 

only pass 30 or fewer tests to achieve certification. Voting systems, on the other hand, 

3
must conform to approximately 1100 separate testable requirements and each test report 

is a very custom document, often exceeding several thousand pages. 

Non-Federal certification efforts also reflect efficiencies made possible only by hard won 

experience and continuous work with the program participants. The Nevada Gaming 

Control Board certifies all gaming devices used in the State of Nevada. While gaming 

systems are significantly different than voting systems, they are nevertheless similar in 

that they must engender the utmost trust and confidence by their users and be highly 

secure from tampering. The Gaming Control Board started testing gaming devices with 

the emergence of microprocessor controlled slot machines in the late 1980’s and early 

1990’s. Today, the Board runs its own testing laboratory supported by a Technology 

Division of over 60 employees. New gaming system approvals (certifications) take 

anywhere from 6- 18 months. Additionally, before the Board grants final approval to 

anew gaming system, that system must undergo a field trial (run in a sanctioned casino) 

for no less than 60 nor more than 180 days. 

These examples show that conformity assessment testing is neither an easy nor a quick 

process, but that a commitment to continuously improve the process does ultimately lead 

to successful certification programs and confidence by the end user of the system. 

 Unfortunately for the EAC, VSTLs, manufacturers, and election officials, the 2008 

General election cannot be postponed to allow the EAC’s program to grow and mature. 

The EAC is in the difficult position of needing to develop its program while actively 

testing and certifying voting systems. Because this challenge is so great the EAC is being 

very deliberate in creating a program that will be effective, useable and cost effective for 

the States in the long term. Simply rubber stamping systems through our certification 

program would only serve to undermine the confidence of both election officials and 

voters in our program. The EAC understands that election officials across the country are 

depending on us to get certified systems to them as soon as possible. We are committed 

to proceeding expeditiously but prudently to certify voting systems. Efficiency will 

improve as the program matures and all involved become more familiar with our 

systematic and rigorous approach to voting system certification. Currently, we are 

pursuing with States ways to facilitate cost efficiencies in the testing and certification 

process at both the Federal and State levels. The ultimate goal of this project is to affect 

cost savings for the States by moving as much repetitive (and often expensive) testing 

into the Federal process, hopefully saving States both time and money. 

I hope this letter serves to inform you on some of the issues facd by the EAC 

Certification Program over the last year. Please check the EAC’s website www.eac.gov

for any information you may need, we work very hard to ensure that any information we 

have is provided to you. If you have any questions regarding this letter or the EAC 

program in general please do not hesitate to contact me. 

4