2010-05-21 14:00 EDT  Auditability Working Group telecon
Minutes rev. 2010-05-27

                                    - * -

Pursuant to the Help America Vote Act of 2002, the Technical Guidelines
Development Committee (TGDC) is charged with directing NIST to conduct voting
systems research so that the TGDC can fulfill its role of recommending
technical standards for voting equipment to the Election Assistance
Commission (EAC).  This teleconference discussion is for the purposes of the
Auditability Working Group of the TGDC to direct NIST staff and coordinate
voting related research relevant to the development of Voluntary Voting
Systems Guidelines (VVSG).

Discussions on this teleconference including referenced discussion papers are
preliminary and pre-decisional, and do not necessarily reflect the views of
the National Institute of Standards and Technology, or the TGDC.

                                    - * -

Agenda:
  * Finalize minutes of previous meeting.
  * Review available records of the VVPAT accessibility decision(s).
  * Reach a consensus regarding whether paperless approaches are in scope.
  * Start narrowing the scope for auditability.

Pending items:
  * Awaiting more information on EBM-scanner hybrid devices.

Attendees:
David Flater
Diane Golden
Nelson Hastings
Marty Herman
Ann McGeehan
Russ Ragsdale
Andrew Regenscheid
David Wagner
Karen Yavetz


-- Finalize minutes of previous meeting

Forgot to do this--deferred to next meeting.


-- Review available records of the VVPAT accessibility decision(s)

In 2003, the Department of Justice published an opinion that the
presence of a contemporaneous paper record on a DRE does not violate
HAVA or the Americans with Disabilities Act, but they did not address
the specific question of whether that remains true if the VVPAT is the
ballot of record and/or is part of a process in which the voter is
expected to verify recorded-as-cast.  [Where HAVA Sec. 301
(a)(1)(A)(i) said "permit the voter to verify," it was in the context
of reviewing one's selections for voting errors--the summary screen on
a paperless DRE being sufficient for that purpose.]  On the "shall"
requirement in VVSG 1.0 Req. I.7.9.7.b, EAC counsel's office did not
issue a decision as was previously believed.  The Commissioners
requested the change based on comments received during the public
comment period for the VVSG.  Thus, we have no explicit and
incontrovertible legal opinion on the question of how HAVA and/or the
Americans with Disabilities Act would apply.

A Discussion Draft of the "Holt Bill" (111th Congress, 1st Session)
dated March 13, 2009, that was provided by Diane Golden, clearly takes
the position that accessible readback of a VVPAT (as well as a
solution to paper handling) should be required.

Diane explained the softer, socio-political side of the issue.  To the
extent that voters in general are expected to participate in the
process through some substantive activity (in this case, directly
verifying their ballots), that activity must be implemented in a way
that does not block voters with disabilities from participating in it.
Though we may agree that the voter-verification act is not really for
the individual doing it but is rather for the integrity of the
election as a whole (for which purpose it is not necessary that every
voter do it), that act is still a form of substantive participation in
the process, and to implement it in a way that excludes voters with
disabilities specifically "smacks of discrimination" and therefore
"won't fly."

Diane confirmed that providing a readback from the paper record would
satisfy the objection at a general level [assuming that the paper
handling issue was also addressed somehow], but the acceptability of
shortcuts such as pulling the text from memory instead of OCRing it
from the ballot or relying on a bar code that is not what gets
tabulated remains controversial.


-- Path forward

The working group discussed a strawman manifesto proposed by David
Flater.  Following is a version amended to reflect the consensus of
the working group, to be validated at the next meeting.

1.  It is not our job to write election law, resolve conflicting laws,
or translate election law which varies from one jurisdiction to the
next into VVSG requirements.

2.  It is not our job to design a voting system.

3.  Our job is
  To define auditability;
  To require auditability to the level needed for defensible elections;
  To ensure that drafted requirements are consistent with federal law
    such as HAVA and the Americans with Disabilities Act;
  To leave jurisdictions with choices of what systems to deploy;
  And to be candid about the compromises and risks inherent in each one.

As a specific case where this matters:  The auditability properties of
VVPAT have nothing to do with whether it is the legal ballot of
record.  The auditor gets multiple records and checks them for
consistency.  As previously agreed, it suffices to detect evidence of
material error.  We can do that much without regard to the legal
precedence of one record over the other.

However, there was not consensus that the legal term "ballot of
record" could be thrown entirely out of scope since it could still
affect the application of federal law with which we are required to
remain consistent.


-- Scope of auditability and paper vs. no-paper

David Flater asserted that parallel testing is an audit method for
recorded-as-cast, which is the same scope addressed by VVPAT.  Setting
aside the question of whether we consider parallel testing to be good
enough in the final analysis, does this not give us a way to talk
about auditability without excluding paperless systems a priori?

David Wagner agreed in principle that you could call it that, but the
practical concerns are substantial.
  - Parallel testing tends to be "de-adopted" because of cost.
  - Difficult to do it right.
  - Effectiveness hard to demonstrate.

Russ expressed concern that parallel testing was an election procedure
and therefore out of scope.

David Flater:  Granted, the VVSG cannot require parallel testing, but
the availability of parallel testing as an audit method means that we
can include paperless systems in the scope of what we talk about in
the context of auditability.

Russ / David Wagner:  Are you suggesting then that every voting system
is auditable?  So we will have no meaningful requirements?

David Flater:  Not like that.  Propose that we stop thinking about
auditability as yes or no and start thinking about it in relative
terms, and in terms of getting different assurances from different
systems that maybe aren't even directly comparable.  First we do the
analysis of the entire space; then we write the requirements.  The
requirements will be yes-or-no, but we draw the line somewhere that
leaves jurisdictions with more than one voting system to choose from,
and we don't draw it the same place as SI, which answer was already
rejected.

Diane Golden and David Wagner each expressed fears that this process
would lead to the same end as before, but were prepared to give it
diligence.


-- Next steps

Everyone begin analyzing the various kinds of auditability found in
all of the voting systems that we have heard about and prepare to
narrow down a definition of auditability.

Next meeting scheduled for June 4 at 2 PM EDT.

Adjourned at 3:20 PM EDT.