ES&S Comments on the Draft Standard for Voter Verifiable Audit Trails on
DRE Voting Systems (DRE:VVPAT), Draft Version March 2, 2005

+------------------+---------------------------------------------------+
| **Reference**    | **Comment**                                       |
+------------------+---------------------------------------------------+
| Correlation of   | There is no question that there needs to be a     |
| PAT ballot with  | one-to-one correlation of CAST ballots between    |
| electronic       | the electronic ballots and the PAT ballot.        |
| ballot           | However, the existing standard document imposes a |
|                  | correlation on the electronic ballot with         |
|                  | CANCELLED PAT ballots. A cancelled ballot is a    |
|                  | non-ballot. The entire concept of canceling a     |
|                  | ballot due to change of selections does not apply |
|                  | to the DRE electronic ballot since the ballot is  |
|                  | always changeable until cast. Cancellation under  |
|                  | these circumstances is a condition created by the |
|                  | printing of ballot selections for preview before  |
|                  | the ballot is cast. If these ballots are not to   |
|                  | be cast due to change, since there is no way to   |
|                  | unprint them, they must be cancelled. However,    |
|                  | there should not be a requirement that electronic |
|                  | ballots be created to correspond to them since    |
|                  | there is nothing that they will be used for. A    |
|                  | log record identifying the cancellation should    |
|                  | indeed be recorded that can be used as an audit   |
|                  | of the number of cancelled PAT ballots.           |
|                  |                                                   |
|                  | This concept of correlation permeates the         |
|                  | document and should be modified throughout.       |
|                  | (2.3.6, 3.3 R4 bullets 3 & 4, 3.3.4.5)            |
|                  |                                                   |
|                  | There are conditions where an electronic ballot   |
|                  | needs to be cancelled but it only needs to be     |
|                  | cancelled in such instances that a wrong ballot   |
|                  | was selected by the pollworker, the voter doesn't |
|                  | cast the ballot and state law requires            |
|                  | cancellation, etc. Even in these instances,       |
|                  | electronic ballots are NOT recorded. Log records  |
|                  | are recorded identifying these events.            |
+------------------+---------------------------------------------------+
| Printing/storing | 3.3.6.4 requires that a 10 digit random integer   |
| of Ballot Record | number be printed and stored on both the PAT      |
| ID               | ballot and the electronic ballot. Other sections  |
|                  | indicate that the voter & pollworker should not   |
|                  | be able to see this number and suggest it be      |
|                  | printed so that it is not human readable.         |
|                  |                                                   |
|                  | There seems to be conflicting                     |
|                  | requirements/specifications in different          |
|                  | sections. Some state that the voter can't handle  |
|                  | (under a window) while 2.3.1 indicates that it    |
|                  | can "as little as possible". 3.3.5.4.3 does state |
|                  | that the voter can't handle if the markings are   |
|                  | human readable.                                   |
|                  |                                                   |
|                  | -   The handling of this is design based and not  |
|                  |     performance based. Should not the standard be |
|                  |     a performance standard?                       |
|                  |                                                   |
|                  | -   The specification of a non-human readable     |
|                  |     code seems to defeat the human audit purpose  |
|                  |     of the DRE-VVPAT. If it takes a machine to    |
|                  |     read the codes on the paper, how is the paper |
|                  |     directly audited against the electronic       |
|                  |     record.                                       |
+------------------+---------------------------------------------------+
| Printing of bar  | A bar code is not directly human readable. Why is |
| code for         | there trust that the printed bar code (which      |
| candidate        | would be used in an electronic recount from the   |
| selections       | paper) indeed match the printed name? Could not   |
|                  | the same alleged tampering of the stored          |
|                  | electronic records compared to the displayed      |
|                  | selections cause the selected name printout to    |
|                  | not match the printed bar code?                   |
+------------------+---------------------------------------------------+
| Audit mode       | 3.3.6.8 requires that the DRE-VVPAT be able to    |
|                  | produce a listing of electronic records BEFORE    |
|                  | THE POLLS CLOSE for "surprise auditing". This may |
|                  | be illegal in most states in that it provides an  |
|                  | indication of vote before the polls close. There  |
|                  | are states that allow central tabulation of       |
|                  | election day ballots in an "early pickup" and/or  |
|                  | tabulation of absentee ballot prior to the close  |
|                  | of the polls. In all of these cases, tabulation   |
|                  | is allowed but NOT reporting or printing of       |
|                  | results.                                          |
+------------------+---------------------------------------------------+
| Format of PAT    | There seems to be conflicting specification       |
| ballot           | regarding the format of the PAT printout.         |
|                  |                                                   |
|                  | -   3.3.1.2 states that the paper record contains |
|                  |     all the warnings displayed on the DRE         |
|                  |     display. This would seem to indicate that the |
|                  |     paper is an audit log of vote activity and    |
|                  |     not just a list of ballot selection. This     |
|                  |     would include such things as overvote         |
|                  |     attempts that are prevented and the attempt   |
|                  |     to exceed the available text length on a      |
|                  |     write-in.                                     |
|                  |                                                   |
|                  | -   3.3.2.5.6 states that the PAT display and the |
|                  |     DRE display of the voted record "shall be as  |
|                  |     close in design and layout as possible". The  |
|                  |     implication is that the displayed summary     |
|                  |     matches the paper printout. This is in        |
|                  |     conflict with 3.3.1.2.                        |
|                  |                                                   |
|                  | -   3.2.1 indicates that the PAT is produced      |
|                  |     AFTER the voter indicates that selections are |
|                  |     complete. This would imply that transactional |
|                  |     information, such as identified in 3.3.1.2,   |
|                  |     could not be printed or would have to be      |
|                  |     stored in a fashion for after-the-fact        |
|                  |     printing.                                     |
+------------------+---------------------------------------------------+
| Reconciliation   | There is no recognition that there is a           |
| of conflict of   | non-system created poll record that is part of    |
| number of        | the normal polling place voting process. That is  |
| records between  | the log of who has actually appeared at the       |
| VVPAT and        | polling place to vote and identifies whether they |
| electronic       | have voted. State laws also usually require that  |
|                  | there are affidavits created for any anomalies    |
|                  | that occur for things such as voter leaving       |
|                  | without casting ballot. These records provide the |
|                  | means to reconcile the number of voters against   |
|                  | the number of ballots cast.                       |
+------------------+---------------------------------------------------+
| General comment  | The document, in many areas, has explicitly       |
|                  | identified how features and functions should be   |
|                  | implemented (design specification) instead of     |
|                  | specifying the functional/performance             |
|                  | requirements of any design that may be used. This |
|                  | severely restricts the creativity of the vendor   |
|                  | in designing better solutions than the ones       |
|                  | specified.                                        |
+------------------+---------------------------------------------------+
| 3.2.1 Voting     | ES&S will be introducing the concept of a **real  |
| perspective,     | time audit log printer** (**RTAL**) for use with  |
| 3.3.1.1          | the DRE to satisfy both the aims/purpose and      |
|                  | fundamental requirements of the VVPAT.            |
| Printing of the  |                                                   |
| paper audit      | The RTAL has been used for election auditing      |
| trail            | purposes, and is a requirement for certification  |
|                  | in the state of Texas, for many years now. In     |
|                  | reviewing the needs to audit an election, it was  |
|                  | determined that a tried and true method would be  |
|                  | easier to implement than a totally new concept    |
|                  | regarding the verifiability of a DRE ballot.      |
|                  |                                                   |
|                  | Furthermore, it is our experience as an election  |
|                  | vendor, that although the voter may make mistakes |
|                  | on a ballot occasionally, it is the poll worker   |
|                  | that generally does not follow the correct        |
|                  | procedures and is later determined to be the      |
|                  | cause of an election issue. These actions (poll   |
|                  | worker/supervisor), just as the action to print   |
|                  | the zero tape/record of the unit, should also be  |
|                  | logged for purposes of review in the case that    |
|                  | the entire electronic results and internal audit  |
|                  | log are destroyed.                                |
|                  |                                                   |
|                  | Explanation and differences of the original VVPAT |
|                  | and the RTAL methods:                             |
|                  |                                                   |
|                  | A DRE unit would require the RTAL to be           |
|                  | operational at all times and if not, the DRE is   |
|                  | halted until the printer is corrected. This       |
|                  | begins from the time the unit is used within the  |
|                  | precinct by the election official, through the    |
|                  | voting process, and through the close of polls    |
|                  | and printing of final results.                    |
|                  |                                                   |
|                  | The RTAL simply continuously runs and documents   |
|                  | every action taken on the unit, including the     |
|                  | poll workers activities. This eliminates the need |
|                  | for the Voter to become a necessary part of the   |
|                  | auditing process. The RTAL is there for them to   |
|                  | review and verify that their actions are being    |
|                  | recorded, but it does not FORCE them to be. This  |
|                  | is truly a Paper Audit trail that is voter        |
|                  | verifiable.                                       |
|                  |                                                   |
|                  | The RTAL will perform the same secrecy processes, |
|                  | i.e. clearing of the window to prevent the next   |
|                  | voter from viewing, keeping the audit paper tape  |
|                  | in secured captivity, allowing an election        |
|                  | official to change the paper if required, etc, as |
|                  | the VVPAT system.                                 |
|                  |                                                   |
|                  | Since there is no need for paper rejection, the   |
|                  | amount of paper used for the RTAL is considerably |
|                  | less. Take for instance the fact that the window  |
|                  | can be as small as 5 inches. This allows a voter  |
|                  | to see approximately 25-30 lines of actions. When |
|                  | a voter is done voting, there is no need to       |
|                  | display on the DRE all the instructional messages |
|                  | to print the paper record and verify.             |
|                  |                                                   |
|                  | The VVPAT requires a much longer window to review |
|                  | the final ballot in one viewing, and then it is   |
|                  | not guaranteed to fit in one window. Our          |
|                  | elections simply keep growing. Handling a         |
|                  | two-page paper review ballot will require even    |
|                  | more instructional text on the DRE screen for the |
|                  | voter to understand. All these instructions, mean |
|                  | more assistance from the poll worker to help the  |
|                  | voter understand how to use the new screens and   |
|                  | VVPAT.                                            |
|                  |                                                   |
|                  | With the RTAL, it's simply a review window of the |
|                  | actions taken by the voter in Real Time. When the |
|                  | voter is done, they cast the ballot and walk      |
|                  | away. NO added screens are necessary to implement |
|                  | the RTAL.                                         |
|                  |                                                   |
|                  | Section 2.3 points out major issues in providing  |
|                  | the VVPAT. The RTAL can actually reduce the       |
|                  | severity of several issues listed in this list.   |
|                  |                                                   |
|                  | **2.3.1 Handling and auditing physical paper      |
|                  | records**                                         |
|                  |                                                   |
|                  | The RTAL, since it use reduces paper usage, can   |
|                  | actually handle a single day without replacement  |
|                  | needed. This allows for the paper tape to be      |
|                  | secured all day and never handled. The paper tape |
|                  | roll can be inserted into a reel to reel          |
|                  | automated unit to actually read the barcodes of   |
|                  | each of the voter records, making re-counting or  |
|                  | auditing much easier with fewer handlers.         |
|                  |                                                   |
|                  | **2.3.2 VVPAT printer reliability issues**        |
|                  |                                                   |
|                  | This is an issue with any printer, but RTAL does  |
|                  | not require a high speed printer to "spit" the    |
|                  | voter record out quickly. The RTAL runs           |
|                  | constantly so speed is not an issue, thus         |
|                  | reducing the chance of critical paper jams.       |
|                  |                                                   |
|                  | **2.3.3 Time to vote and election space issues**  |
|                  |                                                   |
|                  | This issue is eliminated with the RTAL. The RTAL  |
|                  | does not require the voter to perform the         |
|                  | printing function as the end of the DRE ballot,   |
|                  | nor the need to reject the ballot if they find    |
|                  | they made a mistake or wish to change their       |
|                  | minds. The voter uses the same DRE process as if  |
|                  | there were no RTAL. The DRE screens do not need   |
|                  | to change. Since the RTAL prints in real time,    |
|                  | there are no election space issues to deal with   |
|                  | as the voters actions simply print in the window  |
|                  | as they are recorded.                             |
|                  |                                                   |
|                  | **2.3.4 Near-term accessibility and usability     |
|                  | issues**                                          |
|                  |                                                   |
|                  | The RTAL even reduces this issue. Take for        |
|                  | instance an individual that is blind and has a    |
|                  | text to speech hand held device to read text.     |
|                  | With the VVPAT, the ballot is generated and       |
|                  | displayed in the window from the top to the       |
|                  | bottom of the window of the VVPAT. It would be    |
|                  | extremely hard for the voter to use the handheld  |
|                  | text to speech device to follow the printed text  |
|                  | within the window. With the RTAL, each line is    |
|                  | displayed at the bottom of the window, until the  |
|                  | next line is printed, which is again at the       |
|                  | bottom, pushing the other lines up. The blind     |
|                  | individual could easily scan each line across the |
|                  | bottom of the window using it as the guide to     |
|                  | read each line as it is printed.                  |
|                  |                                                   |
|                  | **2.3.5 Usability issues affecting the value of   |
|                  | VVPAT**                                           |
|                  |                                                   |
|                  | The usability of the RTAL is increased by the     |
|                  | fact that the voter is taken out of the forced    |
|                  | roll of printing, reviewing, and rejecting a      |
|                  | ballot record. Quicker voter times allow more     |
|                  | voters, and less units required to be added to    |
|                  | pick up the slower pace of voting with a VVPAT.   |
|                  | Sealed audit log printer tape rolls means that    |
|                  | the audit records can be automatically scanned on |
|                  | a machine allowing an easier method of recount    |
|                  | ability.                                          |
|                  |                                                   |
|                  | **2.3.6 Issues involved in handling multiple      |
|                  | representations of votes**                        |
|                  |                                                   |
|                  | This is another issue that is eliminated due to   |
|                  | the fact that the RTAL does not have a need to    |
|                  | spoil a ballot record.                            |
|                  |                                                   |
|                  | **2.3.7 Issues in resolving disputes**            |
|                  |                                                   |
|                  | This is an ongoing issue that involves every      |
|                  | voting system ever introduced. The RTAL does      |
|                  | provide for an easier method of tracking where a  |
|                  | dispute may have happened by keeping the audit    |
|                  | tape roll for each unit in tact and sealed.       |
|                  | Automation should reduce the causes of disputes.  |
+------------------+---------------------------------------------------+
| 3.3.1.2.         | Some DRE warning messages are not pertinent to    |
|                  | voter verification on the VVPAT and not relevant  |
|                  | for purposes of audits or recounts. They do not   |
|                  | need to be printed on the VVPAT printer.          |
+------------------+---------------------------------------------------+
| 3.3.2.3          | Need greater clarification. Ballot designs are    |
|                  | not created uniquely per election.                |
+------------------+---------------------------------------------------+
| 3.3.1.2.         | Some DRE warning messages are not pertinent to    |
|                  | voter verification on the VVPAT and not relevant  |
|                  | for purposes of audits or recounts. They do not   |
|                  | need to be printed on the VVPAT printer.          |
+------------------+---------------------------------------------------+
| 3.3.2.4.1.       | The referenced standard was developed for an      |
|                  | electronic interface, not paper. The font for the |
|                  | VVPAT does not have to be as large as paper has a |
|                  | greater dpi and smaller font sizes can be read. A |
|                  | paper based standard needs to be defined.         |
+------------------+---------------------------------------------------+
| 3.3.2.4.2.       | What color? What shade? Why not black on white?   |
|                  | This statement is too general and can be          |
|                  | interpreted subjectively. A suggestion would be   |
|                  | to determine a contrast ratio (much like the DRE  |
|                  | requirement) that the printer and paper must      |
|                  | meet.                                             |
+------------------+---------------------------------------------------+
| 3.3.2.5.6.       | DRE displays have been developed to maximize      |
|                  | usability and interaction needed on a touch       |
|                  | screen. Replicating this design and layout on the |
|                  | VVPAT, which is a printer and not an interactive  |
|                  | display is nonsensical. This would also require a |
|                  | much larger and more costly printer.              |
+------------------+---------------------------------------------------+
| 3.3.3.5,         | This will be a costly, unwieldy, and undesirable  |
| 3.3.3.5.4.       | requirement from many perspectives. A requirement |
|                  | for a secondary audio playback system that can    |
|                  | read the printed output or the data coming from   |
|                  | the DRE, intended for the printer, will require a |
|                  | secondary set of technology. Unless prerecorded   |
|                  | audio is mapped to the corresponding printer      |
|                  | output data and then used by the secondary        |
|                  | channel, the system will need to rely on          |
|                  | text-to-speech technology, which does not always  |
|                  | perform well for formal names (e.g. candidate     |
|                  | names). Additionally text-to-speech engines do    |
|                  | not perform well for all multi-language character |
|                  | sets or non-English formal names. If different    |
|                  | data, either through mapping, or more conducive   |
|                  | to text-to-speech engines, were used to drive the |
|                  | secondary audio channel, the voter will be        |
|                  | listening to a different representation than what |
|                  | will appear on the paper -- and this will not add |
|                  | verifiability of the official paper record. From  |
|                  | a voter usability perspective, the secondary      |
|                  | audio channel requirement will change the voting  |
|                  | experience into an unusable process. The process  |
|                  | will require an audio ballot user to listen to    |
|                  | and vote on the DRE audio system (which already   |
|                  | has an audio ballot selection playback            |
|                  | capability), remove and exchange headsets or plug |
|                  | into the second audio system and then listen to   |
|                  | it. If it isn't quite what they remember they     |
|                  | voted, the user will have to switch headphone     |
|                  | connections back again and interact with the DRE  |
|                  | based audio system. In one case of a ballot in    |
|                  | South Carolina, the audio play of a single        |
|                  | referendum selection took over twenty four        |
|                  | minutes to be fully heard. Such a system design   |
|                  | and doubling of the voting period will be so      |
|                  | senseless that voters will become frustrated and  |
|                  | not use it.                                       |
+------------------+---------------------------------------------------+
| 3.3.5.5.         | This requirement is undesirable for many reasons. |
|                  | It may lead to a prohibitively expensive          |
|                  | solution. Recount of multiple independent slips   |
|                  | of paper, each of which can be of a different     |
|                  | length, will be arduous, time consuming, and      |
|                  | error prone. Paper that can be cut reliably in    |
|                  | the precinct may not have the qualities necessary |
|                  | to serve as a long lasting, archive-able medium.  |
|                  | Use of roll based paper that is fed from a roll,  |
|                  | not cut, and taken back up into a roll as it      |
|                  | operates, can help in the recount process, as     |
|                  | well as simplify the operational aspects in the   |
|                  | polling place.                                    |
+------------------+---------------------------------------------------+
| 1.1.1.1.         | There are many types of voter personal assistive  |
|                  | devices in use. Why would we prohibit a voter     |
|                  | from using their own personal assistive device?   |
|                  | It may prove to be difficult and costly for the   |
|                  | election administration to provide every type of  |
|                  | personal assistive device in every polling place. |
+------------------+---------------------------------------------------+