# Pre-Election and Parallel Testing

## [Introduction]{.underline}

Pre-election testing, also called Logic and Accuracy testing (L&A
testing), is the act of testing *every* ballot style and *every*
component of the voting system prior to the election. If you represent a
large jurisdiction with thousands of voting stations and dozens of
ballot styles, this will appear to be a daunting task. It is. However,
documenting pre-election testing is a critical component of the
election's audit trail, which ensures the election was administered in a
fair, accurate, and transparent manner.

Parallel testing is the act of duplicating, as nearly as possible, a
portion of the election under conditions that are identical to the
conditions that occur in a polling place. Parallel testing came about in
response to the contention that it is impossible to detect with 100 %
certainty the presence of so called, Trojan Horses that may be present
in the voting system.

The contention is that these Trojan Horses would contain malicious code
that would only become active after the polls are opened and would
disappear immediately after the polls are closed. This being the case,
their malicious activity would be impossible to detect. Thus, the idea
behind parallel testing is that we can fool these Trojan Horses into
thinking that the polls are open and then detect their malicious
activity with a carefully controlled test.

## [Pre-Election/Logic and Accuracy Tests]{.underline}

**Pre-test Planning:** Pre-election testing involves setting up the
voting system for *each* of your precincts and early voting locations,
loading the election definition, opening the election, casting a known
pattern of votes on each ballot style, closing the election, printing
the vote totals for the precinct, and then comparing the printed vote
totals with the known pattern of votes. In short, you are going to set
up and test your entire election.

Planning for the pre-election testing should begin several weeks before
the actual tests begin. This planning should include identifying the
space required to house the tests, the staff required to conduct the
tests, and the time required for the tests. If you represent a large
jurisdiction, time requirements may dictate that tests for several
voting locations must be conducted simultaneously.

Staff restrictions may require that you use contractors to assist you
with the pre-election testing. If at all possible, a member of the
election staff should be in charge of each contractor test team. One
staff person can supervise two or three test teams provided the tests
are conducted in the same location. Under no circumstances should a test
team be composed entirely of contractor personnel. Each team should
consist of at least two individuals. The EAC recommends that
representatives from voting system manufacturers ***do not*** assist
jurisdictions with pre-election testing.

As part of your pre-test planning prepare the test scripts that are to
be voted on each ballot style. Be sure that each test script includes at
least one vote for each candidate on the ballot and one 'yes' and one
'no' vote for each issue or question on the ballot. Be sure to include
audio ballots in your test scripts.

Any testing that requires printed ballots (i.e. optical scan ballots)
cannot actually begin until these ballots are available. This can be as
little as 45 days prior to the election. However, if early voting
locations and election-day precincts only use electronic voting stations
and do not require paper ballots this testing can be scheduled to begin
as soon as the election definition is completed.

As part of the pre-planning activities review your jurisdiction's
requirements as they relate to open access to the pre-election tests.
Many State election codes require that these tests must be open to the
public and advertised in your official record (newspaper) a specified
number of days before the tests begin.

*Recommendation: If your state or local laws and regulations allow make
the pre-election testing a media event.*

About a week before the pre-election tests are to begin invite the media
to a briefing. At this briefing provide them with the dates and times of
the tests. Show diagrams of the test locations and point out where the
media personnel can be during the tests. Provide written copies of any
rules and procedures that apply to media personnel attendance and/or
participation in the tests.

*Recommendation: At the test locations separate the test area from the
public and media area with a physical barrier; for example, a low wall
or a rope. Ensure this barrier is located so that the public and media
can observe the test but cannot touch the testers or any of the test
materials.*

**Pre-election testing:** Before beginning a test, be sure that all
required supplies are on hand, including the test deck of optical scan
ballots. Although your ballot printer can provide you with a 'Logic and
Accuracy Test Deck', the *Voluntary Voting System Guidelines* recommend
that the optical scan test deck consist of hand-marked ballots.

Conduct the pre-election test from beginning to end on one
precinct/voting location at a time. Set up all of the equipment for the
precinct/voting location and load the election setup on each device.
Open the election and cast the test scripts for each ballot style. If
DRE voting stations are used in the precinct/voting location, manually
enter the test script for *every* ballot style on *every* voting
station. If it is not feasible to vote every ballot on every voting
station, the minimum recommendation is to vote every ballot style on at
least one voting station and then visually inspect every ballot style on
each of the other voting stations.

If optical scan ballots are used in the precinct/voting location, run
the test deck for every ballot style used in the precinct/voting
location.

When all testing has been completed, zero out all vote totals and close
and seal all devices with a tamper proof seal. Record the seal numbers
on a transmittal sheet that will be used in the polling place to verify
that the devices have not been tampered with during transit.

Repeat the above pre-election test for every precinct/voting location.

*Remember: Test the absentee precincts!*

*Hint: When time is of the essence, you can begin pre-election tests for
voting locations that use only DRE voting stations prior to receiving
the optical scan ballots from the printer.*

When the tests in all of the precincts/voting locations have been
completed and uploaded to the election management system, close the
election and print the overall jurisdiction reports. Compare the vote
totals on the jurisdiction reports with the known results from the test
scripts.

## [Parallel Testing]{.underline}

Parallel testing is the act of duplicating, as nearly as possible, a
portion of the election under conditions that are identical to the time
and conditions that occur in a polling place. The rational for parallel
testing is that malicious code may have been introduced into the voting
system software at some point, and is lying dormant until the voting
station is setup for an election. Parallel testing is an attempt to
deceive this code into becoming active by simulating the exact
conditions of an election.

Successful parallel testing must take place during the hours of the
election, using software and hardware that was prepared to be used in
the election.

The mechanics of conducting a parallel test are as follows:

-   Conduct all of your pre-election testing in the normal manner,

-   Determine how many precincts and voting stations you are going to
    include in the parallel test,

-   Randomly select precincts/voting locations to be included in the
    parallel test,

-   Randomly select voting stations that have been prepared and sealed
    for use in the selected precincts/voting locations,

-   During the hours that the precinct/voting location is open, cast a
    known script on the selected voting stations,

-   Compare the resulting vote totals with the vote totals for the known
    script.

When selecting voting locations to be included in your parallel test,
stratify your sample so that the number of large, medium, and small
voting locations to be used in your test are in the same proportions as
the number of large, medium, and small voting locations in your
jurisdiction. Also, prepare voting scripts that reflect the demographics
of the voting locations used in the test. This includes the number of
ballots usually cast on a single voting station at that location and the
distribution of those ballots by political party affiliation and
language choice. *Note: Audio is a language choice.*

If possible select voting stations to be used in the parallel test that
were prepared and sealed to be used in the voting location. If this will
cause a shortage of voting stations in the precinct or voting location,
use voting stations from your inventory -- preferably voting stations
that have been prepared and sealed for the election as 'back up' or
'spare machines.'

*Important: On Election Day, conduct the parallel test in 'Official
Election Mode' and cast the test voting scripts during the times that
the voting location is open for voting.*

Observe the "two person rule" when casting votes on a DRE voting
station; have one person enter the votes while the other person calls
out the votes and verifies the votes on the screen. *Suggestion: Make a
video record of all voting. This record can be used in the event of a
discrepancy in the results to determine whether or not a data entry
error caused the discrepancy.*

If optical scan ballots or VVPATs are used, manually count the optical
scan ballots or the VVPATs and compare the results with both the
electronic vote totals and the known results of the voting script.

*Important: Be aware that the votes accumulated on the parallel test
system are compatible with the actual election. USE EXTREME CARE TO
ENSURE THAT THE RESULTS OF THE PARALLEL TEST DO NOT BECOME INCLUDED IN
THE OFFICIAL ELECTION RESULTS.*

## [Post-Election Testing]{.underline}

In order to be sure that nothing changed, either accidentally or on
purpose, during the course of an election, Logic and Accuracy testing
should be conducted again after the election.

## [Conclusion]{.underline}

Parallel testing and post-election testing will only detect problems in
an election *after* they occur. At this point, as the saying goes, "The
horse is already out of the barn." Certainly, if a problem occurs during
an election you want to detect it so that you can take remedial action.

However, the best way to detect a potential problem *before* it occurs
is with comprehensive, extensive pre-election (Logic and Accuracy)
testing.