EAC Future VVSG Working Group EAC Future VVSG Working Group

Future VVSG Development Goals

July 15, 2015

1. The purpose and scope of the VVSG must be defined and confirmed.

2. The VVSG (and supporting process) must be consistent with Federal statute 

and rule.

3. The VVSG must reflect the bottom-up reality of election administration by 

incorporating common State requirements to inform future VVSG 

development. 

4. The application of the VVSG must benefit election administration.

5. The VVSG must be implementable.

6. The VVSG should accommodate the interoperability of election systems.

7. The VVSG should permit jurisdictional options by incorporating a planned 

transition between Standards in a measured and predictable manner. 

8. The VVSG should not impose unanticipated costs onto organizations.

9. The VVSG must include a cost analysis estimate of conformance testing to 

the standard.

10. The VVSG requirements should be performance based and technology 

neutral.

11. The VVSG should allow maximum flexibility to incorporate new/revised 

requirements including those from other Standards setting bodies.

12. The VVSG should be accompanied by education and outreach efforts to the 

election community.
1 www.eac.gov

United States

Election Assistance Commission

1335 East West Highway, Suite 4300, Silver Spring, MD 20910

EAC Future VVSG Working Group White Paper:

The Goals for Future Federal 

Voting System Standards 

Development Efforts
2 www.eac.gov

Introduction/Background: 

One of the tangible suggestions noted during the June 12, 2014 Election Assistance Commission (EAC) Roundtable 

Meeting on “Reforming the Testing and Certification Process” http://www.eac.gov/roundtable_-

_reforming_the_testing_and_certification_process/ was for the EAC to begin discussions with members of the 

election community regarding how a future Voluntary Voting System Guidelines (VVSG) document should be 

developed, designed and structured.

The EAC envisioned that the primary job of the Future VVSG Working Group (“The Working Group”) would be to 

provide thoughts and ideas to outline the high level direction for the next VVSG development effort. The EAC began 

this effort with outreach to States via the annual Conference on State Certification of Voting Systems meeting as well 

as engaging in conversations with numerous individual State and local election officials. The two primary goals for the 

Working group were:

• To explore how future VVSG efforts can support innovation and allow for flexible product solutions while still 

maintaining clear and testable requirements within the standard

• To define a strategy and to develop priorities for producing an effective VVSG in order to ensure that 

standards consider the interests of all stake holders as well as the real-time operational needs of election 

officials.

The Working Group intentionally did not discuss specific testable VVSG requirements since that process is delegated 

to the TGDC process called out in the Help America Vote Act.

Process and Members:

The EAC brought together a diverse and uniquely experienced group of stakeholders to debate and discuss the goals 

for future VVSG development efforts. The Working Group included:

• Ann McGeehan, Former Texas Director of Elections & PCEA Member

• Juan Gilbert, Professor of Computer Science, University of Florida

• Christi Coburn, Director of Elections, Jefferson County, Colorado

• Lori Augino, Director of Elections, Washington

• Paul Aumayr, Voting System Director, Maryland State Board of Elections

• Jay Bagga, Professor of Computer Science Ball State University., Indiana VSTOP 

• Jack Cobb, Lab Director, Pro V & V Laboratories

• Merle King, Executive Director, Kennesaw State University Center for Election Systems

• Cliff Tatum Executive Director, DC Board of Elections 

• Steve Pearson, Vice President, Certification, Election Systems and Software (ES&S)

• David Wallick, Product Manager, Everyone Counts

• Steve Trout, Director of Election Innovation, Clear Ballot

• Mary Brady, Voting Project Manager, (NIST)
3 www.eac.gov

Group Consensus on Future VVSG Development Goals:

1. The purpose and scope of the VVSG must be defined and confirmed.

The Working Group felt that one of the fundamental purposes of the VVSG was determining what policy 

objectives the guidelines were trying to achieve and to define and describe a voting system. HAVA has 

its definition of voting system incorporated into the current VVSG, but the reality of state supremacy in 

election administration and recognition of state diversity and variety are not recognized. While states 

currently have different definitions of what constitutes a voting system, they should be recognized as an 

integral part of this process. EAC’s Voting System Testing and Certification Program is only one part of 

the overall conformity assessment process that includes equally important companion efforts at the 

State and local levels. The process to ensure that voting equipment meets specific technical 

requirements is a distributed, cooperative effort of Federal, State, and local officials in the United States. 

Finally, the Working Group urged the EAC to clearly define what components (e-pollbooks? UOCAVA 

ballot delivery systems? election night reporting systems??) are to be included in the definition of a 

voting system and to identify what should be tested and certified. Many Working Group members 

expressed the need for a true federal standard-, i.e. a confederation of state standards. 

2. The VVSG (and supporting process) must be consistent with Federal statute and rule. 

While perhaps obvious, the working group felt it was necessary to include the indisputable fact that the 

next VVSG and its individual requirements must not be in conflict with federal laws and regulations. 

3. The VVSG must reflect the bottom-up reality of election administration by incorporating common 

State requirements to inform future VVSG development.

Working Group members reiterated that state and local officials should continue to decide what their 

rules and requirements are for voting systems and what they should test to. Members suggested that in 

order to make the next VVSG more relevant and more useful to the states, the EAC should map the 

VVSG with individual state requirements as step one in the next VVSG development process. It was also 

noted that states currently have a difficult time looking at Federal test reports to determine where state 

requirements might overlap with VVSG requirements and that a more bottom-up approach to the VVSG 

might help to alleviate this problem. 

EAC staff noted that they are currently working with several states to map requirements to the VVSG 

and are working on a way to build state mapping into the EAC’s Virtual Review Tool (VRT) (The VRT is the 

secure web-based portal developed and used by EAC to work through and track progress in certification 

test campaigns. NASED representatives on the working group noted that mapping requirements is one 
4 www.eac.gov

of NASED’s recommendations to the EAC and that they would be drafting a follow up letter to EAC 

Commissioners on this subject. 

Ultimately, the Working Group agreed that the real utility of the VVSG was as a core set of requirements

that would be useful to as many states as possible as part of the overall certification process for voting 

systems . 

4. The application of the VVSG must benefit election administration.

The Working Group agreed that the next VVSG must be a practical document and must be fit for 

purpose. For this paper, the general definition of “fit for purpose” should be thought of as: A document 

well equipped or well suited for its designated role or purpose. Most members agreed that the 2005 VVSG

document was not entirely fit for purpose in hindsight. The Group noted that how well the current 

VVSG is fit for purpose can be debated and that how to determine if a future VVSG is fit for purpose

would be a challenge depending upon which individuals or groups were looking at the document. The 

Group did agree, however that the VVSG serves a larger purpose outside the confines of the 

requirements stipulated within the document. Group members were adamant that the next VVSG 

should not only provide for basic voting system functionality but should also mention and explicitly 

recognize that the end users of the products designed to meet VVSG requirements conduct elections in 

various ways from State to State, and that the VVSG remains voluntary. It is this emphasis on, and 

recognition of, the practical application of the VVSG that will better ensure its fit.

5. The VVSG must be implementable. 

Any standards development effort is successful only to the extent that the requirements contained 

within the document can actually be implemented by product manufacturers and accurately tested by 

laboratory personnel. The group noted that the Help America Vote Act (HAVA) called out 6 general 

attributes of a voting system. 

SEC. 301. VOTING SYSTEMS STANDARDS. (a) REQUIREMENTS.—Each voting system used in an election for 

Federal office shall meet the following requirements: 

(1) IN GENERAL.— (A) Except as provided in subparagraph 

(B), the voting system (including any lever voting system, optical scanning voting system, or direct recording electronic 

system) shall— 

(i) permit the voter to verify (in a private and independent manner) the votes selected by the voter on the 

ballot before the ballot is cast and counted; 

(ii) provide the voter with the opportunity (in a private and independent manner) to change the ballot or 

correct any error before the ballot is cast and counted (including the opportunity to correct the error 

through the issuance of a replacement ballot if the voter was otherwise unable to change the ballot or 

correct any error); and 

(iii) if the voter selects votes for more than one candidate for a single office— 
5 www.eac.gov

(I) notify the voter that the voter has selected more than one candidate for a single office on the 

ballot; 

(II) notify the voter before the ballot is cast and counted of the effect of casting multiple votes for the 

office; and 

(III) provide the voter with the opportunity to correct the ballot before the ballot is cast and 

counted. 

(B) A State or jurisdiction that uses a paper ballot voting system, a punch card voting system, or a central count voting 

system (including mail-in absentee ballots and mail-in ballots), may meet the requirements of subparagraph (A)(iii) by— 

H. R. 3295—40 (i) establishing a voter education program specific to that voting system that notifies each voter of the 

effect of casting multiple votes for an office; and (ii) providing the voter with instructions on how to correct the ballot 

before it is cast and counted (including instructions on how to correct the error through the issuance of a replacement 

ballot if the voter was otherwise unable to change the ballot or correct any error).

(C) The voting system shall ensure that any notification required under this paragraph preserves the privacy of the voter 

and the confidentiality of the ballot. 

(2) AUDIT CAPACITY.— (A) IN GENERAL.—The voting system shall produce a record with an audit capacity for such 

system. 

(B) MANUAL AUDIT CAPACITY.— 

(i) The voting system shall produce a permanent paper record with a manual audit capacity for such system. 

(ii) The voting system shall provide the voter with an opportunity to change the ballot or correct any error before the 

permanent paper record is produced. 

(iii) The paper record produced under subparagraph (A) shall be available as an official record for any recount 

conducted with respect to any election in which the system is used. 

(3) ACCESSIBILITY FOR INDIVIDUALS WITH DISABILITIES.— The voting system shall— 

(A) be accessible for individuals with disabilities, including nonvisual accessibility for the blind and visually impaired, in a 

manner that provides the same opportunity for access and participation (including privacy and independence) as for 

other voters; 

(B) satisfy the requirement of subparagraph (A) through the use of at least one direct recording electronic voting system 

or other voting system equipped for individuals with disabilities at each polling place; and

(C) if purchased with funds made available under title II on or after January 1, 2007, meet the voting system standards 

for disability access (as outlined in this paragraph). 

(4) ALTERNATIVE LANGUAGE ACCESSIBILITY.—The voting system shall provide alternative language accessibility 

pursuant to the requirements of section 203 of the Voting Rights Act of 1965 (42 U.S.C. 1973aa–1a). 

(5) ERROR RATES.—The error rate of the voting system in counting ballots (determined by taking into account only 

those errors which are attributable to the voting system and not attributable to an act of the voter) shall comply with the 

error rate standards established under section 3.2.1 of the voting systems standards issued by the Federal Election 

Commission which are in effect on the date of the enactment of this Act.

(6) UNIFORM DEFINITION OF WHAT CONSTITUTES A VOTE.— Each State shall adopt uniform and nondiscriminatory 

standards that define what constitutes a vote and what will be H. R. 3295—41 counted as a vote for each category of 

voting system used in the State

HAVA also gives a very specific definition of a voting system in Section 301 (b). The Working Group 

noted that the intent of the legislation as well as the very specific language should be taken into account 
6 www.eac.gov

in future VVSG development. The group asked if a more expansive definition for the VVSG that 

encompassed “election system” as opposed to “voting system”, could become the baseline definition 

for VVSG development.

The Working Group also explored the concept of implementing varying levels of standards into the next

VVSG to increase flexibility for jurisdictions with varying needs. An example of this would be where level 

A includes testing to all standards, level B is testing to a large portion of the standards , and level C is 

testing only to a few high level requirements. This concept would be relevant for cost savings and for 

those states that may not need a complete and comprehensive standards testing process for the 

requirements of their particular state. Ultimately, the Group dismissed this idea since many members 

said that states that don’t use the federal testing and certification now aren’t likely to do so simply 

because of the inclusion of varying test levels. Other members thought that the concept was also bad 

for voters since concerns would be raised from voters regarding their “lower” testing and certification 

impacting their voting process more so than a system which has undergone a “higher” testing and 

certification of a neighboring state. 

Finally, several members were concerned about states that accept EAC testing and do little or no further 

testing before certifying and implementing a new voting system. These members noted that if the 

document was not practical and implementable from the perspective of those jurisdictions, those states 

might not use the document at all and consequently be worse off than they were currently. 

6. The VVSG should accommodate the interoperability of election systems.

The Working Group agreed that while perhaps not currently practicable; the next VVSG should 

accommodate the interoperability of the wide variety of election systems and peripheral devices 

currently on the market and the numerous new peripheral devices sure to be on the market in coming 

years. 

Several Working Group members noted that some set of “core” requirements are generally mandatory 

in other industries. Other members noted that one option might be to include guidance language for

optional peripherals in the next VVSG in order to assist State and local election officials without making 

this a requirement. While this idea was generally well received, many members noted that putting non 

standards (non-normative requirements language) in a standards document is concerning and that some 

States might feel compelled to include the guidance as a requirement for their state.

Most working members agreed that component testing is a certification/process issue not a VVSG issue. 

EAC staff noted that current VVSG language already allows and does not prohibit the development of a 

component certification process. 

VVSG 1.1, Section 1.5.2 (Implementation Statement) notes that:
7 www.eac.gov

“An implementation statement documents the requirements that have been implemented by the voting 

system, the optional features and capabilities supported by the voting system, and any extensions (i.e., 

additional functionality beyond what is defined in the VVSG) that it implements.” 

One final comment on this issue suggested that while the VVSG would permit component certification, 

additional standards for integration will need to be developed.

7. The VVSG should permit jurisdictional options by incorporating a planned transition between 

Standards in a measured and predictable manner. 

The Working Group was in unanimous agreement that any future VVSG must be compatible with 

existing standards, professional practice and policy. The group felt that future versions of the VVSG 

must be operationally and technologically bridged to preceding guidelines and practices.

Many members of the Working Group noted that without a planned transition, future VVSG 

implementation would be as disruptive, if not more disruptive to election administration than was 

encountered in the transitional years after the implementation of the 2005 VVSG 1.0. The majority of 

members felt that the EAC should let the market drive future VVSG implementations and that it should 

be up to the states/counties to make the choice. 

While all Working Group members understood the need to transition away from old, out-of-date 

standards documents at some point, they agreed that clear understanding of the transition process was 

vital. A majority stated that old standards needed to be retired. All Working Group members recognized 

that jurisdictions may have plans for their old systems (certified to previous iterations of the VVSG) and 

that they need plenty of time to plan for sunset these old systems. Essentially the group stated that they 

would be looking for an articulated transition plan from the EAC. 

8. The VVSG should not impose unanticipated costs onto organizations.

Because budgetary concerns are a reality for almost all election officials in the United States, the 

Working Group agreed that any future VVSG should not impose unanticipated cost burdens on 

organizations. The group intentionally used the term “organizations” in goal #8 so that the financial 

impact on voting system manufacturers, election officials, test laboratories and others would be 

recognized and considered during the VVSG development process. The Working Group realized that it is 

unrealistic to expect that future VVSG documents might not contain some requirements that produce 

unanticipated costs; no matter how carefully the standards development process considers cost impact. 

The group nevertheless felt that it was imperative to urge the TGDC, NIST and the EAC to carefully 

consider cost factors when developing standards. By considering cost impact, the TGDC, NIST and the 

EAC can better evaluate competing standards when addressing a policy objective of the standard.
8 www.eac.gov

9. The VVSG must include a cost analysis estimate of conformance testing to the standard.

The Working Group felt that a cost analysis estimate of future draft VVSG standards should be included 

as the next logical goal in order to operationalize goal #8. Because doing a cost analysis is not typically 

part of the TGDC/NIST standards development process, the EAC agreed that they would develop this 

cost estimate in conjunction with the Voting System Test Laboratories (VSTLs) and make the results 

available to the election community during the VVSG public comment process.

10. The VVSG requirements should be performance based and technology neutral.

The Working Group had perhaps its longest discussions around the topic of whether future VVSG 

standards documents should be primarily performance based or primarily design based documents. A 

majority of the group felt that future VVSG standards should be functional in nature so that the 

document can more easily be re-defined as technology changes, or as new approaches are presented. 

The group noted that technology neutral statements have longer lives and in order to keep a future 

VVSG document “evergreen” detailed description of any technology should be excluded. 

The Working Group determined that the most promising avenue for pursuing such a document would 

be to develop high level (performance based) standard and have details contained in some lower level 

document that can more easily change and adapt as technology changes. The group did recognize, 

however, that labs will need guidance on how to test performance based standards. NIST and EAC staff 

noted that such “lower level” detail was already being done to a large extent in the Test Assertions

development working group led by Mark Skall. Similar to a process envisioned by the Working Group, 

the test assertions are not design requirements but details on how those individual requirements will be 

tested. 

11. The VVSG should allow maximum flexibility to incorporate new/revised requirements including those 

from other Standards setting bodies. 

Working Group members stated that future VVSG development efforts must allow maximum flexibility 

to incorporate new/revised requirements, potentially including those from States themselves as well as 

from other Standards setting bodies. NIST is currently providing support to the IEEE VSSC 1622, with the 

goal of creating an XML-based common data format (CDF) for election systems. A CDF will facilitate 

interoperability among voting devices and certain types of automated testing. At this time the IEEE 

scope of the work includes:

• Exports of election management system databases including for election results

• Voter registration data and precinct information processed by electronic pollbooks

• Candidate and ballot definition information
9 

• Voted ballot information and tabulations

• Device logs

• Other information that may be produced by election management systems

12. The VVSG should be accompanied by education and outreach efforts to the election community.

Although not a goal directly affecting the TGDC standards development process, the Working Group 

nevertheless was in unanimous agreement that any future VVSG development must be accompanied by 

educational outreach efforts for the election administration community, other stakeholders, and the 

public. The group felt that including this as a goal would serve as a reminder to those writing the 

standards that people need to understand the requirements contained in the VVSG document at their 

respective level of technological sophistication in order for it to be most useful to the widest audience. 

Next Steps:

This paper is ultimately intended for submission to the EAC Commissioners for review. The Future VVSG 

Working Group members believe that the Commissioners should then task the TGDC and NIST to use the 

twelve stated goals contained in the accompanying document as a baseline for developing and implementing 

the next VVSG standards development effort in order to achieve a standards document that allows for 

reliable testing and certification of election systems through implementable and technology neutral 

requirements. 

www.eac.gov