1

October 29, 2009

National Institute of Standards and Technology

100 Bureau Drive, Stop 1070 

Gaithersburg, MD 20899-1070

We, the undersigned, participated in a working meeting on vote tabulation audits hosted by the 

American Statistical Association (ASA) on October 23 and 24, 2009. We write to emphasize that 

future iterations of the Voluntary Voting System Guidelines (VVSG) should facilitate effective 

vote tabulation audits. We applaud the VVSG II's requirement for independent voter-verifiable 

records (IVVRs). This requirement is necessary to enable verification of election outcomes 

independently of the tabulation systems; it should be adopted as soon as possible. However, if 

election outcomes are to be verified efficiently, vote tabulation systems must meet requirements 

that go well beyond the draft VVSG 1.1.

Overview

Two key goals of vote tabulation audits are 

i) To verify that the election outcomes implied by the reported vote totals are correct, and

ii) To provide data for process improvement: specifically, to identify and quantify various 

causes of discrepancies between voter intentions and the originally reported vote totals.

Difficulty in obtaining subtotals of the machine tallies to compare with manually-derived totals 

from small batches of ballots is a major problem. Efficient vote tabulation audits require – in 

addition to software-independent audit trails – timely, comprehensive, detailed, standardized, 

machine-readable subtotals of the votes as recorded by the vote tabulation systems. For greatest

efficiency, individual ballot interpretations should be available to support emerging methods that 

audit at the ballot level (that is, batches of size 1) without breaching confidentiality. 

Future VVSGs should contain audit-related requirements for all voting systems, designed in 

consultation with experts in election auditing, to ensure that the next generation of voting 

systems facilitate election audits. 

Key areas for standards include:

• Usability of the paper record

• Comprehensive reporting of all important data elements

• Small-batch or individual ballot reporting capability

• Machine-readable, standard election result reporting formats, with support for 

standardized identification of contests and candidates, that facilitate aggregation for 

electoral contests spanning multiple jurisdictions

• Machine-readable, standard audit result reporting formats, including audit units selected 

and discrepancies found
2

Detail

A “batch” is a physically identifiable collection of paper records (or ballots) that is separately 

tallied in the electronic vote totals. Every vote record must be assigned to one and only one 

batch. Batches may be organized by geographic boundaries (e.g., one or more batches from the 

same precinct or polling place) or by other forms of proximity (e.g., batches formed from among 

ballots received at the same central location, or cast on the same early voting machine). Most 

currently used batches, such as whole precincts, are too large to support efficient audits. We 

strongly urge that voting equipment support collecting and reporting votes for arbitrarily many, 

arbitrarily small batches, down to individual ballots, in a standard, machine-readable format. 

Ballot confidentiality, however, must always be ensured. 

Voter-verifiable records should meet usability standards. For instance, DRE voter-verifiable 

paper audit trails (VVPATs) produced on spools of thin, narrow paper can be difficult to handle 

and audit. Further research may be needed to determine appropriate standards. There should also 

be standards for digitizing and storing ballot-image data.

Voting system reports should facilitate accounting for each ballot accepted and the outcomes of 

every contest choice on each ballot. Reporting should capture rich information about accepted 

ballots, including: 

• Batch identifier

• Precinct identifier

• Equipment used (e.g., make, model and machine serial number) 

• Ballot style identifier

• Voting mode (e.g., early-vote, mail-in absentee, election day, provisional)

• Place of voting (e.g., physical location where the vote was cast or the mailed in ballot 

received)

• Whether the ballot is invalid or spoiled; and, for valid ballots, all votes recorded, 

undervotes and overvotes, for each listed contest.

Voting systems should make it easy to create detailed reports with subtotals by contest, by ballot 

batch, by precinct, or by scanner or tabulation machine.

One common, standardized data format is needed for reporting audit results, as well as initial 

election results. Implementation details are outside the scope of this letter; election auditing 

experts should participate in specifying these requirements. 

In summary, we strongly recommend that the next version of the VVSG support auditing 

election outcomes by facilitating small-batch reporting in standardized electronic reporting 

formats, and usable voter-verifiable cast vote records.
3

The following individuals have endorsed this statement.*

Robert Adams, Deputy County Clerk, Bernalillo County New Mexico

Vittorio Addona, Assistant Professor, Mathematics, Statistics, and Computer Science, Macalester 

College

Arlene S. Ash, PhD Professor, University of Massachusetts Medical School, Worcester, MA and 

ASA Fellow

Lonna Atkeson, Professor of Political Science, University of New Mexico

Mary Batcher, PhD and ASA Fellow 

Sean Flaherty, Research Assistant, Verified Voting

Lynn Garland

Ed Gracely, PhD, Drexel University College of Medicine and School of Public Health

Mary W. Gray, Chair, Department of Mathematics and Statistics, American University, 

Washington, DC and ASA Fellow

Joseph Lorenzo Hall, Postdoctoral Research Associate, UC Berkeley School of 

Information/Princeton University Center for Information Technology Policy

Mark Halvorson, Founder and Director, Citizens for Election Integrity Minnesota

Mark Lindeman, Assistant Professor of Political Studies, Bard College

David Marker, Senior Statistician and Associate Director, Westat

Neal McBurnett, Election Auditing Consultant

John McCarthy, Computer Scientist (retired), Lawrence Berkeley National Laboratory

Walter Mebane, Professor of Political Science and Professor of Statistics at the University of 

Michigan, Ann Arbor

Ron Olson, member, Citizens Alliance for Secure Elections

Jane Platten, Director, Cuyahoga County Board of Elections, Ohio

Prof. Ronald L. Rivest, EECS Department, MIT

Ion Sancho, Supervisor of Elections, Leon County, Florida

Alexander Shvartsman, Professor of Computer Science and Engineering and Director of the 

Voting Technology Research Center at the University of Connecticut

Pamela Smith, President, Verified Voting Foundation

Philip Stark, Professor of Statistics, University of California, Berkeley

Luther Weeks, Executive Director, CTVotersCount

*Affiliations for identification purposes only.