ElectionArchive.org

The Election Integrity Audit

Kathy Dopp and Frank Stenger

1

WHY AUDIT ELECTION RESULTS?

The elections industry is the only major industry not routinely subjected to independent manual

audits. In any field, electronic mistakes or tampering2 would go undetected in the absence of audits.

Banks, churches and businesses are subjected to routine independent audits to detect and prevent errors and

embezzlement. Virtually all U.S. votes will be counted electronically by 2008. Electronic voting systems

make it possible for one person to rig a state-wide election and for electronic errors and failures to produce

wrong results.

3

 

Electronic vote counts determine who controls budgets of millions to trillions of dollars. Yet only perhaps

fifteen states conduct any audits – manual checks - of vote count accuracy and their procedures are not

sufficient to detect wrongfully altered outcomes in close races

4

. In some states audit procedures are not

independent; audit results are not made public; audit results are not used to correct discrepancies found

between manual and electronic vote counts; or the audits are performed after election results are certified. 

This is especially alarming in light of the fact that U.S. jurisdictions publicly report their vote counts

aggregated in a way that hides evidence of vote count errors, machine problems, and tampering.5 

In other words, U.S. election outcomes are wide open to undiscovered fraud and innocent error. 

The Accuracy of Any System Depends on the Ability to Detect and Correct Errors

The most accurate voting system employs both manual and electronic counts because manual counts can

detect errors and most types of tampering in electronic tallies; and electronic tallies can detect some types of

paper ballot box stuffing, tampering and voter errors. The more types of errors that can be prevented,

detected, and corrected, the more accurate the system. The integrity of election results depends on “trusting

no one” and verifying.

Requiring voter-verifiable paper ballot (VVPB) records with voting systems does little more than provide

false assurance if the VVPB records are not used in recounts and audits. In audits 100% of the paper ballot

records associated with randomly selected vote counts (precinct vote counts or digital electronic recording

(DRE) machine vote counts) are manually counted using the VVPB records and compared exactly to their

associated electronic vote counts.

Routine independent audits – manual counts of the voter-verifiable paper ballot records - of randomly

selected

6

 electronic vote counts could detect most types of electronic vote count error that could wrongfully

1

 Kathy Dopp, MS Mathematics, President, US Count Votes and Frank Stenger, PhD Mathematics, University of Utah

2

 Any system is most at risk of tampering by insiders. In elections, insiders include election officials, poll workers, voting

machine vendor technicians, programmers, and staff; and any county staff with access to voting machines or central tabulators.

3

 See the Brennan Center Report at ht tp://brennancenter.org/programs/dem_vr_hava_machineryofdemocracy.html “The

Machinery of Democracy: Protecting Elections in an Electronic World”

4

 Although some states require recounts when margins between candidates are under 1%, sometimes these states merely reconcile

the electronic records during a “recount”.

5

 See "What Election Data Can We Collect and Publicly Release to Monitor our Elections for Accuracy?"

http://uscountvotes.org/ucvAnalysis/US/election_officials/ElectionArchive_advice.pdf

6

 "Random selection" is a scientific process designed by an expert. Having election officials select the electronic counts to audit

does not qualify as "random" selection.

 National Election Data Archive Project September 25, 2006 ElectionArchive.org Page 1
alter election outcomes, to any desired confidence level if audit sample sizes are properly calculated.

Auditors must be independent. I.e. Auditors must not be insiders such as voting machine vendor staff or

election officials.

WHAT TYPES OF ERRORS ARE NOT DETECTABLE VIA AUDITS?

Most types of electronic tampering and error are detectable by manually counting voter-verifiable paper ballot

records. However, some types of deliberate vote tampering are not detectable. For instance:

• Paper ballot tampering before or after an election spoils ballots by marking too many candidates in one race,

thus creating over-votes thus the voter’s actual votes are not counted. Auditors should look for high or

unusual ballot spoilage rates and treat them with suspicion. 

• Paper ballot substitution – switches votes

• Paper ballot box stuffing – adds extra votes

• Programmers can make DRE voting machine electronic and paper records both wrong to subvert audits.

Studies show that only about 30% of voters take the time to verify their DRE touch-screen paper roll ballot

records7, so about 2/3rds of votes for a candidate could be shifted to an opponent by making the paper and

electronic record both wrong on the first try, and if the voter notices and cancels the ballot, get it right on the

second try. Voters should be educated to take the time to verify that their paper ballot records match their

selections and immediately notify officials if the paper does not match their touch-screen choices. A picture

should be taken, the offending DRE machine should be removed from service for the remainder of the

election, all votes previously cast on that machine included as possible discrepancies during an audit, and a

forensic technical investigation done on that DRE machine.

Vote count error or fraud can only be completely identified if voters verify the accuracy of their paper

ballot records and paper ballot frauds are prevented. 8 This paper focuses on manual audits to detect

discrepancy between electronic and manual counts sufficient to alter election outcomes.

WHICH VOTING SYSTEMS ARE INDEPENDENTLY AUDITABLE?

A voting system is audit-able if it provides hand-countable, voter-verifiable paper ballot records that can be used to

check the electronic counts. Auditable voting systems include

• Direct recording electronic (DRE) voting systems that provide voter-verifiable paper roll ballot records -

Auditing paper roll ballot records requires special equipment, called “paper-roll advancers” to be able to

conveniently read the paper rolls. If most voters do not bother to take the extra step to verify their paper roll

ballot record, then not all electronic errors are detectable via manual audits. Automatically auditing paper roll

ballot records using electronic bar codes is not an independent check of machine accuracy and does not

qualify as an “independent audit” of DRE vote count accuracy.

• Paper ballot optical scan voting systems - Optical scan paper ballots are verified by most voters in the process

of marking the ballots

9

. 

7

 From a survey of voters conducted in Las Vegas, Nevada by Lombardo Consulting Group in conjunction with political science

professor, Dr. Michael John Burton of Ohio University, Nevada http://e-grapevine.org/nvvotersurvey.pdf

8

From Roy G. Saltman’s I ndependent Verification: Essential Action to Assure Integrity in the Voting Process submitted to

the National Institute of Standards and Technology Gaithersburg, MD 20899 Aug. 22, 2006 “...protections against ballot frauds.

It was recommended that all of the official ballots printed and distributed must be accounted for, and there must be assurance

against use of counterfeit ballots. At each precinct, the number of blank ballots received must equal the sum of ballots voted,

unused, spoiled and otherwise employed (Saltman, 1975, pp. 41-45). This additional reconciliation is intended to prevent “ballot

stuffing,” i.e., the addition of extra voted ballots to a ballot box. Furthermore, if there are missing votes that do not complete the

total, or if there are more votes cast than voters signed-in to vote, these inconsistencies can be investigated and resolved. (p 22) If

reconciliations are thoroughly and correctly made, and prevention of counterfeit ballots is assured, then only the fraud of

electronic miscount can be perpetrated. Good ballot-handling and tracking procedures are critical.

9

 The exception is with touch-screen ballot marking devices used by disabled or foreign-language voters. Ballot marking devices

provide for electronic verifiability or read-back of the marked ballot.

 National Election Data Archive Project September 25, 2006 ElectionArchive.org Page 2
Mechanical lever machines and paperless DRE voting systems are not independently audit-able. As such, both lever

machines and paperless DRE voting systems are inherently inaccurate and open to undetectable tampering since no

method is provided for detecting and correcting errors.

ABSTRACT

This paper explains how to directly calculate a sufficient number of electronic vote counts (either precinct

vote counts or DRE machine counts) to audit – count manually - in order to detect an amount of electronic

miscount that could wrongfully alter an election outcome. Vote count audits of fixed 1 or 2% of electronic

counts provide no confidence that outcome-altering errors would be detected in close races. 

This new audit calculation method builds on the National Election Data Archive’s (NEDA) prior work on

the mathematics of vote count audits and the Brennan Justice Center’s threat analysis which assumes a

maximum amount of miscount that could occur in each electronic vote count without raising immediate

suspicion and investigation.10 Thus:

• The minimum number of corrupt electronic vote counts that could wrongfully alter an election

outcome depends on the margin between candidates and the assumed maximum miscount rate in any

vote count, and 

• If we solve the equation for calculating the probability of detecting one or more vote miscounts for the

audit sample size in terms of the desired probability, the total number of vote counts, and the number

of corrupt counts sufficient to wrongfully alter outcomes, then we can determine the audit sample size

that will ensure election outcome integrity.

The “election integrity” probability equation cannot be solved by hand, but it can be solved by computer. In

this paper, we provide a numerical (computer) solution to directly calculate vote count audit sample sizes to

detect electronic errors to any desired probability that could have altered election outcomes.

11

Success in detecting evidence of outcome-altering electronic vote miscount via audits is influenced by the

following relationships:

• As the margin between candidates decreases, the amount of vote miscount that could wrongfully alter

an election decreases and the audit amount must increase.

• For a fixed rate of corruption, as the total number of vote counts to be audited decreases, the required

audit amount does not similarly decrease. i.e. As the number of total vote counts decreases, the audit

percentage required to keep the same probability of detecting errors increases. For example if one in

ten electronic vote counts is corrupt, then at least ten vote counts must be audited in order to have a

good chance of detecting one corrupt count, regardless of how many total counts there are. 

• When auditing optical scan precinct counts (as opposed to DRE machine counts), if precinct sizes vary

significantly, it becomes possible for miscounts to be targeted to a fewer number of large precincts,

and larger audit sample sizes are needed.

• When auditing DRE machine counts variability of the number of ballots cast on DRE machines may

have a similar effect (to the effect of precinct-size variation on optical scan audits) on the probability

of detecting outcome-altering miscount. The conservative approach would be to use the number of

ballots cast on DRE machines to adjust audit sample sizes but research is needed.

• The total number of electronic vote counts in each race, rather than the total number of vote counts in

the entire jurisdiction, determines the sample size that must be used to audit that race outcome. 

10 VoteTrustUSA’s Howard Stanislevic suggests using 0.20 for the maximum vote shift per vote count that would not be

suspicious based on historical trends. We use 0.15 here but 0.20 is good because it is more conservative.

11 We make our new “election integrity” audit methodology publicly available on ElectionArchive.org and in Appendix B.

 National Election Data Archive Project September 25, 2006 ElectionArchive.org Page 3
Our numerical method is for calculating the audit sample size for actual vote count audits to find actual

discrepancies between machine and manual counts expected to be exactly equal.

12

 In other words, 100% of

paper ballot records associated with randomly selected vote counts are manually counted and compared

exactly to their corresponding electronic vote counts.

Future research is needed to determine what calculations and methodology to employ when discrepancies

between electronic and manual counts are uncovered during initial audits. Further research is needed to

investigate the effects of variability of the total number of ballots cast on individual DRE machines on

probabilities of detection. Systematic study is needed on the pros and cons of state-wide versus county-wide

audits such as reduction of information, efficiency of the overall process, and reductions in the probability

of detecting outcome-altering miscount in state-wide audits. State-level audit sample sizes could be easily

calculated in states having uniform voting systems, but would be difficult to correctly calculate in states

with mixed voting systems.

13

Background

In 1975, Roy G. Saltman

14

, then of the National Bureau of Standards (now called the National Institute of

Standards and Technology) wrote a report15 that, in Appendix B, recommends an almost identical

calculation for determining the number of corrupt vote counts that audits should be designed to detect.

Saltman also uses the margin between candidates, an assumed “maximum level of undetectability by

observation”16 and precinct sizes to calculate vote count audit sample sizes.

In June, 2005 NEDA publicly released a paper by Kathy Dopp and Ron Baiman, “How Can Independent

Paper Audits Correct & Detect Vote Miscounts” which recommended vote count audits, demonstrated the

basic equations, and made a spreadsheet available to calculate the probabilities for detecting one or more

vote miscounts, given various amounts of corrupt vote counts. 

Since June 2005, the Carter-Baker Election Reform Commission, the U.S. Government Accountability

Office, the National Science Foundation’s "A Center for Correct, Usable, Reliable, Auditable, and

Transparent Elections" (Project ACCURATE), Verified Voting, the Brennan Justice Center Voting System

Task Force, and the U.S. League of Women Voters have recommended routine vote count audits - manual

counts of randomly-selected voter-verifiable paper ballot records.

12 This is not a ballot sampling procedure. Most voting systems today do not use unique anonymous identifiers on paper ballots

that match a machine-tabulated record. Thus ballots are not individually auditable. Thus ballot sampling procedures are only

capable of finding statistical anomalies rather than actual discrepancies – leaving the debate open to “sampling error”; and making

scientific random sampling of ballots problematic. Scientific random ballot sampling of DRE machine paper roll ballot records

would be impractical, even if there were anonymous identifiers.

13

 It may be difficult to correctly calculate state-wide audits when some counties use DRE machine counts and other counties use

precinct optical scan counts because of the problem of not being able to randomly select comparable counts because DRE

machine counts often include multiple precincts. The total number of ballots cast in each count would be required to adjust the

calculation for size variation, and may cause some counties to be over-sampled. Even when states use uniform voting systems,

state-level audits reduce the information obtained and the probability of detecting outcome-altering miscount, and prolong the

audit process when discrepancies are found as compared to county-level audits as recommended here. More research is needed.

14 Roy G. Saltman was instrumental in creating the first federal engineering and procedural performance standards for voting

systems. See http://en.wikipedia.org/wiki/Certification_of_voting_machines#Work_of_Roy_Saltman

15 Saltman, Roy G. (1975), "Appendix B: Mathematical Considerations and Implications in Selection of Recount Quantities," in

Effective Use of Computing Technology in Vote-Tallying, Report NBSIR 75-687, (reprinted as SP 500-30, 1978), National

Institute of Standards and Technology, Gaithersburg, MD 20899. The report was paid for by the US General Accounting Office,

now called the US Government Accountability Office. http://csrc.nist.gov/publications/nistpubs/NBS_SP_500-30.pdf

16 On p. 115 Saltman defines this as “...the largest percent switch of votes in any one precinct that will fail to make the opposition

correctly suspicious that a switch has occurred in that precinct. The higher the maximum level of undetectability by observation,

the higher the number of switched votes that can be packed into a single precinct, and the fewer the number of misreported

precincts that are needed to reverse an election, the less likelihood there is of a vote-switching scheme being discovered by a

partial recount.”

 National Election Data Archive Project September 25, 2006 ElectionArchive.org Page 4
On July 16 and 17, 2006, the National Election Data Archive (NEDA) publicly revised its 2005 paper and

spreadsheet for calculating election audit amounts to explain how to calculate the number of corrupt vote

counts required to wrongly alter an election outcome, and began working to develop a method to directly

calculate audit amounts that would ensure election outcome integrity. 

On August 9, 2006 Howard Stanislevic of the VoteTrustUSA E-Voter Education Project publicly released a

paper independently confirming NEDA’s calculation method and pointing out the necessity of adjusting

calculations for precinct-size variation.17 Stanislevic and Dopp had independently derived equivalent audit

calculation methods, both taking precinct size into account, using slightly different assumed maximum

miscount rates per vote count. 

Saltman’s, Stanislevic’s and Dopp’s previous calculations for calculating audit sample size relied partially

on a trial and error, generate and test method that requires some expertise. 

In September, 2006 Ron Rivest presented a method for estimating vote count audit sample sizes that can be

performed easily on a calculator, is always conservative (because it is never less than the optimal value),

and is a better estimate than the “sampling with replacement” estimate.

18

The method presented in this paper by Dopp and Stenger makes it possible to automatically calculate the

optimal vote count audit sample size that gives at or just above any desired probability for detecting

outcome-altering amounts of vote miscount.

HOW TO INDEPENDENTLY AUDIT VOTE COUNTS?

Manual “hand-to-eye” counts of voter-verifiable paper ballots for randomly selected electronic vote counts

(precinct or machine counts) can achieve any desired probability for detecting most electronic errors that

could wrongly alter election outcomes, particularly if voters are encouraged to verify that their paper ballot

records match their choices.

19

 

An “auditable” vote count report must be publicly released prior to randomly selecting vote counts and

assigning auditors randomly to each count.

20

 After the auditable report is publicly released, then the

calculated number of electronic vote counts must be transparently, scientifically, randomly selected and

100% of the paper records associated with those electronic counts manually counted. If discrepancies

between the electronic and hand counts are found, then actions must be taken including investigations,

expanded audits, 100% hand counts, correction of the official results, or even possibly a new election. 

If the goal of elections is to ensure that voters decide who is sworn into office and what ballot measures pass

or fail, then audit amounts must be sufficient to detect any error that could wrongfully alter election

17 See Howard Stanislevic’s “Random Auditing of E-Voting Systems: How Much is Enough?”

http://www.votetrustusa.org/pdfs/VTTF/EVEPAuditing.pdf. The effect of precinct-size variation on vote count audit

probabilities, and other important points related to manual audits were also brought up on the Open Voting Consortium email

discussion list, a group of computer experts working on better open source voting system designs. See http://OpenVoting.org.

18

 September 2006, “Estimating the Size of a Statistical Audit” by Ronald L. Rivest PhD, Computer Science and Artificial

Intelligence Laboratory Massachusetts Institute of Technology, Cambridge, MA 02139 http://theory.csail.mit.edu/~rivest/RivestOnEstimatingTheSizeOfAStatisticalAudit.pdf Ron Rivest is a member of the Technical Guidelines Development Committee

which is advisory to the U.S. Election Assistance Commission.

19 A random manual audit, in combination with voter education and mathematical analysis of detailed precinct-level by vote-type

election results, would virtually ensure election integrity because mathematical analysis of election results could catch significant

errors or patterns that indicate problems that may not be detected with a manual audit. NY State, Subtitle V of Title 9 of the

Official Compilation of Codes, Rules and Regulations, Part 6209, Voting Systems Standards, Section 6209.2. F. (2) states: “There

shall be instructions for performing the verification process made available to the voter in a location on the voting system."

20 An “auditable report must enumerate votes cast for each race and measure on each DRE voting machine or optical scan

machine in each precinct for each race and candidate, including under-votes and over-votes, along with unique machine

identifiers. It might be helpful to include batch numbers and the dates votes are cast in case of possible investigations.

 National Election Data Archive Project September 25, 2006 ElectionArchive.org Page 5
outcomes. The smaller the margins between the winners and losers, the smaller the amount of vote

miscount that could wrongfully alter an outcome; and the larger the audit size must be to detect it. 

Elections are administered by county, parish, and township election jurisdictions, so subjecting each

jurisdiction to audits gives the highest assurance of integrity. 

Procedures that are Not Independent Vote Count Audits include:

• Comparing the barcodes on touch-screen voting machine paper-roll ballot records with electronic

counts. This checks whether two electronic records, neither of which can be verified by voters, match

and is not a check of whether electronic vote counts reflect voter intent.

• Printing out the paper audit tape for each machine’s memory card to see if the printout matches the

machine counts. This is a test of whether print programs work.21 They usually do.

• Randomly selecting ballots after the election and counting them by hand and by machine. This is

post-election machine testing and is not a measure of Election Day vote count accuracy.

What is the Probability of Finding Corrupt Vote Counts with a 2% Audit?

To calculate the probability of detecting at least one corrupt vote count we need to know the:

N - number of total vote counts (DRE machine memory card or optical scan precinct counts) eligible to cast votes

in a particular race or ballot measure

C – number of corrupt vote counts that we want to be able to detect

22

S - amount of machine or precinct vote counts that are randomly selected for audits

The average number of precincts in a U.S. Congressional district is 440. The table below (rightmost column) shows

probabilities that a 2% audit will find one or more corrupt vote counts, based on N=440 total vote counts with varying

hypothetical number of corrupt vote counts.

23

Total

Number

of Vote

Counts

Hypothetical

Number of

Corrupt Vote

Counts

Percent of

Corrupt

Vote

Counts

Number of

Audited

Vote Counts

for 2% Audit

Expected

Value of

Corrupt

Counts in

Audit

Standard

Deviation

Probability of

Finding a

Corrupt Vote

Count

440 220 50.00% 9 4.50 1.50 99.82%

440 198 45.00% 9 4.05 1.49 99.57%

440 176 40.00% 9 3.60 1.47 99.05%

440 154 35.00% 9 3.15 1.43 98.02%

440 132 30.00% 9 2.70 1.37 96.11%

440 110 25.00% 9 2.25 1.30 92.70%

440 88 20.00% 9 1.80 1.20 86.85%

440 66 15.00% 9 1.35 1.07 77.18%

440 44 10.00% 9 0.90 0.90 61.61%

440 22 5.00% 9 0.45 0.65 37.25%

440 18 4.00% 9 0.36 0.59 31.58%

440 14 3.00% 9 0.27 0.51 25.45%

440 5 1.00% 9 0.09 0.30 9.86%

440 3 0.50% 9 0.05 0.21 6.03%

440 1 0.10% 9 0.01 0.09 2.05%

440 1 0.05% 9 0.00 0.07 2.05%

21 The NV 2004 election “audit” used this method in some counties according to independent investigations by several groups.

22 A rule of thumb for determining the percentage of corrupt counts needed to alter an election result for a race is to divide the

margin between two candidates by 0.30 (twice the assumed maximum vote shift that would not be considered immediately

suspicious) to obtain an approximation of the smallest percentage of corrupted counts needed to alter the outcome of any race.

23 Details of the basic formula are in Appendix A. USCountVotes has made a spreadsheet audit calculator available.

http://ElectionArchive.org/ucvAnalysis/US/paper-audits/ and an “election integrity” audit calculator program.

 National Election Data Archive Project September 25, 2006 ElectionArchive.org Page 6
The probability (rightmost column above) for finding at least one corrupt vote count decreases dramatically

as the percentage of corrupt vote counts falls below 10%. Yet errors only need exist in a small percentage

of vote counts to wrongfully alter outcomes of close races. A 2% audit is not likely to detect the small

amount of vote miscount that could wrongfully alter outcomes of U.S. Congressional races with less than

5% margins between candidates.

Smaller jurisdictions and races with smaller number of total vote counts must audit approximately the same

amount of vote counts as larger jurisdictions. In other words, smaller jurisdictions with fewer total counts

must audit a larger percentage of their vote counts to have the same probability of detecting outcomealtering miscount.

Vote count audits of 1 or 2% provide no confidence that outcome-altering errors would be detected in close

races. The right number to use in election audit legislation is the desired probability of detecting outcomealtering vote miscount.

24

A nationwide 2% audit, such as that proposed in U.S. House of Representatives Bill 550 introduced in the

109th Congress is not only insufficient to ensure election outcome integrity, but worse, could practically

interfere with more efficient local audits by taking the paper ballot records for a federal audit, making them

unavailable for local audits.

How to Calculate Audit Amounts that will Detect “Outcome-Altering” Vote Miscount?

The purpose for auditing election results is to ensure that no outcome is wrongfully altered by error. 

Therefore, we introduce a calculation to determine the number of vote counts to audit in order to achieve

any desired probability of detecting outcome-altering error, for any margin between candidates. We begin

with the basic probability equation for detecting one or more corrupt vote counts:

Eq. 1

0

1

C N C

S

P

N

S

  − 

  

  

= −

 

 

 

 

where

P is the probability of detecting one or more vote counts

C is the number of corrupt vote counts needed to wrongfully alter an election outcome

S is the number of vote counts that we randomly select for our hand counted audit, and

N is the total number of vote counts for a particular race or measure to be audited in our county

For the moment, we put off the discussion of how to correctly calculate the number of corrupt vote counts,

C that we need to be able to detect in order to assure that no election outcomes are wrongly altered.

We want to solve our probability equation for S, the number of vote counts to audit, because we want to

specify our desired probability, P, calculate C from our margins and other assumptions or facts, and we

know N, the total number of vote counts for each race or outcome.

Attempting to solve for S, 

0

1

C N C

S

P

N

S

  − 

  

  

= −

 

 

 

 reduces to 

24 That could be set at a constant 99% for state-wide audits or perhaps at 95% for county-wide audits.

 National Election Data Archive Project September 25, 2006 ElectionArchive.org Page 7
Eq. 2:

(1 ) ! ( )!

0

( )! ( )!

P N N S

N C N S C

− −

− =

− − −

 and taking the natural log gives us:

 Eq. 3: ln(1− P) + ln((N − S −C)!) − ln((N − S)!) + ln(N!) − ln((N −C)!) = 0

It would be wonderful if there was an easy solution for S that would allow election officials to simply plug

numbers for N, P, and C into a simple formula. However due to the factorials, and real world values of the

variables involved, no exact algebraic solution for S in terms of the other variables has been found for this

equation. Hence we use the LNΓ(x) = GammaLN function to approximate N! where

1

0

( )

u x

x e u du

∞

− − Γ = ∫

where Γ(x +1) = x! for any non-negative integer x.

We numerically determine a real number S to desired accuracy, which solves the equation

Eq. 4:

ln(1− P) + gamma ln(N −C − S +1) − gamma ln(N − S +1) + gamma ln(N +1) − gamma ln(N −C +1) = 0

for given values of C, P, and N. Then, since vote counts `S' are integers, we replace S with the integer value

greater than our computed S. Our method thus calculates the correct number of vote counts to audit in order

to achieve at least the desired probability of detecting at least one corrupt vote count given C corrupt counts

out of a total of N vote counts..

25

How To Estimate The Minimum Number Of Corrupt Vote Counts Required To Wrongfully Alter

Election Outcomes

To calculate the correct audit sample size, we must first estimate the minimum number of corrupt vote

counts, C, required to wrongfully alter an election outcome.

What Is The Maximum Number Of Votes That Might Be Wrongfully Switched From One Candidate To

Another Within Any One Vote Count Without Raising Suspicion by the Opposition? 

The Brennan Center Task Force, a group of security and voting system experts, in their threat analysis of

U.S. voting systems, suggests a value of maximum wrongful vote shift of 15% of votes from one candidate

to another. This results in a maximum wrongful margin shift between the two candidates of 30% (when 15%

of votes are taken from one candidate and wrongfully given to another) in any one vote count. We assign,

therefore, a “MaxShift” value of 0.30 as the maximum wrongful margin shift we might expect within any

one vote count when the maximum vote shift is 0.15. Discrepancy data obtained from future audits should

be publicly released and used to adjust the assumed value of the maximum rate of margin shift per vote

count that was uncovered that was not noticed to be suspicious prior to the audit. 

From the minimum margin between any two candidates

26

 and the maximum margin shift on any one vote

count, we can estimate the amount of vote counts which must be corrupted to wrongfully alter the election

outcome for any race. Dividing the candidate margins by the 0.3 assumed maximum vote margin shift per

vote count gives

mN

C

MaxShift

= where m = margin between candidates in the race or ballot measure being

audited, N = #total vote counts in the for the particular race being audited, and MaxShift = the maximum

assumed margin shift per vote count that would not be immediately suspicious. To obtain at least the

25 This numerical method solution to Dopp’s “election integrity” equation was derived by Professor Frank Stenger, PhD expert in

numerical methods, University of Utah.

26 The margin between candidates that should be used for county-wide audits is the minimum of the county-wide or the state-wide

margin between candidates in any one race.

 National Election Data Archive Project September 25, 2006 ElectionArchive.org Page 8
desired probability of detecting outcome-altering vote miscount, the integer floor values for C are used to

calculate integer ceiling values for S. 

The smaller the margin between candidates, the smaller the percentage of corrupt vote counts that is needed

to wrongfully alter an election outcome and the larger the audit percentage that is needed to detect at least

one corrupt vote count. 

Should State-Wide Or County-Wide Margins Between The Candidates Be Used? The state-wide

margin determines the percentage of votes that could be wrongfully switched, on average, in each county to

wrongfully alter the state-wide outcome. Therefore the amount of vote counts audited in each county needs

to be sufficient to detect at least the amount of vote miscount that could have altered the state-wide margin.

However, in order to determine if the outcome in each county is correct and to reduce the need for expanded

audits when discrepancies are uncovered in other counties, each county should audit sufficient vote counts

to validate the outcome in their county. Therefore, whichever margin is less, the state-wide or the countywide margin, should be used to calculate the audit amount in each county in order to make the audit process

most efficient and informative.

27

Should State-Wide Or County-Wide Total Number Of Vote Counts Be Used? Audits should be

conducted for any race in each county or township using N = the total #vote counts within the county.

Although audits could be conducted with smaller sample sizes using the state-wide total number of vote

counts, this could prolong the audit process by requiring expanded audits in all counties whenever

discrepancies are found in any one county. If county-level audits are conducted initially, more information

is obtained from the first round of audits to make determinations that could avoid unnecessary expanded

audits in counties where no discrepancies are found during initial audits. County-wide audits could enable

the election process to be more quickly completed to announce final election winners.

When auditing precinct vote count audits (as opposed to machine DRE machine count audits) an

adjustment in the calculation of the number of corrupt counts that could wrongfully alter the

outcome is required whenever precinct sizes vary significantly.

When auditing precinct vote counts, the probability of detecting outcome-altering vote miscount is reduced

if vote miscount is targeted in fewer large-sized precincts. To adjust for precinct-size variation we directly

calculate the minimum number of corrupt vote counts that could alter outcomes using a list of the total

number of ballots cast in precincts.

28

 The following information is needed to adjust for precinct-size

variation:

PrecinctSize = (n0, n1,...,) the total number of ballots cast in each precinct in which votes were cast for a race.

T = total number of ballots cast in the county for the particular race to be audited (T should be equal to the sum of

the precinct sizes input above). 

On the other hand, DRE machine audits do not need adjustment for precinct-size variation because there are

more DRE machines in larger precincts, increasing the probability of selecting counts in larger precincts

29

.

At an average ten minutes per voter, 6 voters per hour could use one DRE times a 14 hour Election Day

would be 84 voters. Responsible election officials provide adequate DRE machines for voters, and provide

backup optical scan paper ballots. To be conservative DRE machine count audit sizes could be adjusted

using the number of total ballots cast on each DRE.

27 Email discussions with Howard Stanislevic led to the realization that the state-wide margin should be used whenever the statewide margin is smaller than the county-wide margin.

28 The method for calculating the number of corrupt precincts needed to wrongly alter an outcome is described in Appendix B.

The effect of precinct-size variation affects precinct count (not DRE machine count) audits. When auditing precinct counts, a

decrease in probability is avoided by calculating precisely the number of corrupt vote counts required to alter election outcomes

as shown in Appendix B. Because more DRE machine counts exist in larger counties, the audit amount does not need adjustment

for precinct-size when auditing DRE machine counts. DRE machine count size variability still needs further study.

29 Although there might be certain errors that kick in after a certain number of ballots are cast on a DRE machine.

 National Election Data Archive Project September 25, 2006 ElectionArchive.org Page 9
Some Real Life Examples – Election Integrity Vote Count Audit Amounts

Here are examples of calculated audit sample sizes using Lake County, OH and Multnomah County, OR.

Lake County, OH November, 2004 has N=217 total

county-wide DRE machine vote counts. 

The table to the left and the chart below show the audit

amounts required for a county-wide race for selected

margins between candidate (leftmost column in table

and horizontal axis in chart) in order to have at least

95% probability for detecting at least one corrupt count

if there is outcome-altering vote miscount, assuming

that at most 15% of vote counts are wrongly shifted per

vote count (i.e. assuming that candidate margins are

wrongly shifted by at most 30% per vote count). Lake

County uses DRE voting machines so no adjustment is

needed for precinct-size variation.

We check the resulting probabilities of detection using

the calculated audit sample sizes. The probabilities are

at or above our desired 95%. If we increase the desired

probability to 99%, then the audited number of vote

counts increases.

The smaller the margin between candidates, the fewer

corrupt counts could wrongly alter outcomes, the harder

is it to detect, so the larger the required audit size.

Whereas the larger the margin between candidates, the

larger the number of corrupt vote counts needed to alter

the race, and the smaller the audit required to detect it.

Lake County, OHDRECountsNov 2004

Audit Amount toDetect Outcome-AlteringVote

Miscountfor N=217,P=95%

0

50

100

150

0.0% 5.0% 10.0%

Marginsbetween Candidates

Audit Amt

Audit Amount

Corrupt Counts toAlter

Outcome

 National Election Data Archive Project September 25, 2006 ElectionArchive.org Page 10

mINaxPUTS

wrong

margin

shift per

vote count 0.3

total#of

vote

counts 217

probability

of

detecting

miscounts 0.95

OUTPUTS

Candidate

Margins

Number

of Vote

Countsto

Audit

Percentage

of Vote

Countsto

Audit

Number

Corrupt

Countsto

Alter

Outcome

Percent

Corrupt

Countsto

Alter

Outcome

Probability

Check

0.5% 137 63.1% 3 1.4% 95.11%

1.0% 75 34.6% 7 3.2% 95.12%

1.5% 55 25.3% 10 4.6% 95.00%

2.0% 41 18.9% 14 6.5% 95.19%

2.5% 32 14.7% 18 8.3% 95.03%

3.0% 28 12.9% 21 9.7% 95.29%

3.5% 24 11.1% 25 11.5% 95.57%

4.0% 21 9.7% 28 12.9% 95.29%

4.5% 18 8.3% 32 14.7% 95.03%

5.0% 16 7.4% 36 16.6% 95.11%

5.5% 15 6.9% 39 18.0% 95.42%

6.0% 14 6.5% 43 19.8% 95.93%

6.5% 12 5.5% 47 21.7% 95.10%

7.0% 12 5.5% 50 23.0% 96.07%

7.5% 11 5.1% 54 24.9% 96.06%

8.0% 10 4.6% 57 26.3% 95.60%

8.5% 9 4.1% 61 28.1% 95.20%

9.0% 9 4.1% 65 30.0% 96.23%

9.5% 8 3.7% 68 31.3% 95.35%

10.0% 8 3.7% 72 33.2% 96.28%

10.5% 7 3.2% 75 34.6% 95.12%

11.0% 7 3.2% 79 36.4% 96.03%

11.5% 7 3.2% 83 38.2% 96.78%

12.0% 6 2.8% 86 39.6% 95.38%

12.5% 6 2.8% 90 41.5% 96.18%

<< constant - let expert

analysts set

<<set as desired
Multnomah County, OR in its 2006, May primary election had N=128 total number of precincts

using paper ballots counted with optical scan machines.

The Democratic US Representative primary election had

N=102 total precincts with ballots cast in that race

30

, with

T=62,651 total ballots cast and the total number of ballots

cast in the largest precincts were (1292, 1267, 1239,

1192, 1188, 1157, 1144, 1109, 1060, 1054, 1053, 1047,

1028, 1028, 983, 964, 960, 959, 939, 921, 889, 888, 832,

832, 831, 826, 812, 793, 791, 783, 750, 740, 727, 723,

722, 717, 703, 691, 667, 666, 663, 652, 647, 639, 637,

631, 630, 614, 613, 600, 583, 580, 579, 569). For desired

probability P=95%, assuming a 15% maximum wrongful

vote shift per vote count (i.e. 30% margin shift) the table

on the left shows the minimum amount of vote counts to

audit for selected margins between candidates, adjusted

for precinct-size variation. 

The table at right and the chart below shows the audit

sample sizes for Multnomah County’s Democratic

primary for the U.S. House of Representatives. However,

the actual margin between the two leading candidates in

Multnomah County in the May 2006 election was about

80% (not shown on our chart) so that a 1% minimum

audit rate is sufficient.

The table below (lower right) shows how the probability

of detecting at least one corrupt count decreases if vote

miscounts are targeted to the largest precincts yet audit

size is not adjusted for precinct-size variation. Counties

with more divergent precinct sizes would have lower probabilities.

Multinomah County, ORPrecinct CountsMay 2006

Audit Amount to Detect Outcome-AlteringVote

Miscount for N=102, P=95%

0

50

100

0.0% 5.0% 10.0%

Margins between Candidates

Audit Amt

Audit Amount

Corrupt Counts toAlter

Outcome

30 There were 104 precincts eligible to vote in this primary race, but in two precincts no ballots were cast.

 National Election Data Archive Project September 25, 2006 ElectionArchive.org Page 11

Margins

between

candidates

Probability

of

detection

without

adjusting

for precinctsize

variation

Audit

Amount if

NOT

adjusted

for precinctsize

variation

0.5% 95.1% 97

1.0% 86.4% 64

1.5% 83.9% 46

2.0% 86.9% 40

2.5% 84.4% 31

3.0% 83.8% 26

3.5% 80.9% 24

4.0% 79.4% 20

4.5% 80.1% 18

5.0% 79.9% 16

5.5% 81.2% 15

6.0% 79.5% 13

6.5% 79.7% 12

7.0% 82.4% 12

7.5% 82.0% 11

8.0% 81.2% 10

8.5% 79.9% 9

9.0% 83.9% 9

9.5% 82.0% 8

10.0% 83.7% 8

INPUTS

0.3

102

0.95

Margin

between

candidates

Audit this

many vote

counts

Audit

percent

Number

corrupt

countsto

alter

outcome

Percent

corrupt

countsto

alter

outcome

Probability

Check

0.5% 97 95.1% 1 1.0% 95.10%

1.0% 79 77.5% 2 2.0% 95.09%

1.5% 64 62.7% 3 2.9% 95.09%

2.0% 53 52.0% 4 3.9% 95.01%

2.5% 46 45.1% 5 4.9% 95.41%

3.0% 40 39.2% 6 5.9% 95.43%

3.5% 40 39.2% 6 5.9% 95.43%

4.0% 35 34.3% 7 6.9% 95.29%

4.5% 31 30.4% 8 7.8% 95.15%

5.0% 28 27.5% 9 8.8% 95.17%

5.5% 26 25.5% 10 9.8% 95.52%

6.0% 24 23.5% 11 10.8% 95.63%

6.5% 22 21.6% 12 11.8% 95.54%

7.0% 20 19.6% 13 12.7% 95.23%

7.5% 19 18.6% 14 13.7% 95.55%

8.0% 18 17.6% 15 14.7% 95.75%

8.5% 17 16.7% 16 15.7% 95.85%

9.0% 15 14.7% 18 17.6% 95.75%

9.5% 14 13.7% 19 18.6% 95.55%

10.0% 13 12.7% 20 19.6% 95.23%

10.5% 13 12.7% 21 20.6% 95.99%

11.0% 12 11.8% 22 21.6% 95.54%

11.5% 12 11.8% 23 22.5% 96.21%

12.0% 11 10.8% 25 24.5% 96.25%

<< constant - let experts set

<<set as desired

<< total number of counts in race

max wrong margin

shift per vote count

total#of vote counts

probabilityof detecting

at least one miscount

OUTPUTS
Now consider the Multnomah County, OR May

2005 primary race for State Senate District #17

where there were only N = 9 precinct counts

eligible to vote in that race. To the left are the

audit sample sizes required for various margins.

Without even adjusting for precinct size variation, a

100% audit is required for margins up to 6.5% in

order to achieve a 95% probability for detecting

outcome-altering vote miscount when assuming a

15% maximum vote counts shift per vote count (i.e.

30% margin shift). 

The margin between the two OR State Senate

candidates was approximately 6% so that all nine

counts must be audited – hand counted - to detect

outcome-altering electronic vote miscount for this

race. 

Because only one corrupt count in nine could

wrongly alter the race outcome, all nine precinct

counts must be manually counted to have a good

chance to detect one corrupt count.

 National Election Data Archive Project September 25, 2006 ElectionArchive.org Page 12

Candidate

Margins

Number of

Vote

Countsto

Audit

Percentage

of Vote

Countsto

Audit

Number

Corrupt

Countsto

Alter

Outcome

Percent

Corrupt

Countsto

Alter

Outcome

0.5% 9 100.0% 1 11.1%

1.0% 9 100.0% 1 11.1%

1.5% 9 100.0% 1 11.1%

2.0% 9 100.0% 1 11.1%

2.5% 9 100.0% 1 11.1%

3.0% 9 100.0% 1 11.1%

3.5% 9 100.0% 1 11.1%

4.0% 9 100.0% 1 11.1%

4.5% 9 100.0% 1 11.1%

5.0% 9 100.0% 1 11.1%

5.5% 9 100.0% 1 11.1%

6.0% 9 100.0% 1 11.1%

6.5% 9 100.0% 1 11.1%

7.0% 7 77.8% 2 22.2%

7.5% 7 77.8% 2 22.2%

8.0% 7 77.8% 2 22.2%

8.5% 7 77.8% 2 22.2%

9.0% 7 77.8% 2 22.2%

9.5% 7 77.8% 2 22.2%

10.0% 5 55.6% 3 33.3%

10.5% 5 55.6% 3 33.3%

11.0% 5 55.6% 3 33.3%

11.5% 5 55.6% 3 33.3%

12.0% 5 55.6% 3 33.3%

MultinomahCounty, ORPrecinct CountsMay2006

Audit Amount toDetect Outcome-AlteringVote

Miscount for N=9, P=95%

0

5

10

0.0% 5.0% 10.0%

MarginsbetweenCandidates

Audit Amt

Audit Amount

Corrupt Counts toAlter

Outcome
SUMMARY

The National Election Data Archive (NEDA)’s Frank Stenger and Kathy Dopp developed a numerical

solution for calculating vote count audit sample sizes to ensure electronic vote count integrity in all

auditable voting systems, particularly when voters verify that their paper ballot records are correct.

Properly calculated and conducted audits – hand counts of paper ballots performed to check the

accuracy of electronic vote counts - can be performed in jurisdictions using audit-able voting systems.

31

 

To assure electronic election outcome accuracy in auditable voting systems, the following information is

required to calculate vote count audit sample sizes:

1. The desired probability for detecting outcome-altering miscount.

32

2. The minimum of the state-wide or county-wide margin between the winning and losing

candidates or ballot measures.

3. The assumed maximum vote miscount per electronic vote count not considered suspicious,

usually set by an expert analyst.

33

4. The number of total (DRE machine or precinct) electronic vote counts in the jurisdiction for

which votes were cast for the race being audited.34

5. In the case of auditing precinct counts, the total number of ballots cast in each precinct in which

votes were cast for the race. (Also possibly, in the case of auditing DRE machine counts, the

total number of ballots cast on each DRE eligible to be voted on in a particular race.)

NEDA’s new “election integrity” audit calculation method is shown in Appendix B. 

To calculate vote count audit size for a particular race requires knowing the margin between candidates,

the total number of electronic vote counts in a race for the jurisdiction being audited, an assumed

maximum wrongful rate of vote miscount per vote count that would not otherwise raise suspicion and

action35, the desired probability for detecting errors, and, in the case of optical scan precinct count

audits, the total number of ballots cast in each precinct. 

Combined with procedures to secure paper ballots and reconcile precinct counts, vote count integrity can

be assured by auditing sufficient vote counts of auditable voting systems to detect any miscount that

could wrongfully alter election outcomes and by educating voters to verify paper ballot records.

The National Election Data Archive is a national, not-for-profit nonpartisan organization devoted to

promoting scientific means to ensure election-outcome integrity. NEDA’s “Election Integrity Audit”

paper can be found at http://electionarchive.org/ucvAnalysis/US/paperaudits/ElectionIntegrityAudit.pdf. A program that can be used to calculate election audit amounts is

available on ElectionArchive.org: http://electionarchive.org/auditcalculator/eic.cgi

31 Paperless electronic voting machines and lever machines provide no method to independently check electronic vote count

accuracy. Some states, like Maryland, Georgia, and parts of Florida, Pennsylvania, and several other states use voting systems

that are not auditable. Any voting system that provides no routine measures to detect and correct errors is fundamentally

inaccurate and error-prone.

32 We tentatively suggest a minimum of 95% for county-wide audits.

33 We recommend initially using 0.15=15% or 0.20=20% (i.e. a 30% or 40% margin shift respectively). If an election official

uses 20% (i.e. a 40% margin shift) for the maximum vote shift considered not to raise suspicion, then any amount of shift in a

vote count that is more than 20% in comparison to prior comparable election results could be considered suspicious and its

count automatically included in an audit or investigation in addition to the randomly selected vote counts.

34 Assuming a county-wide audit, or alternatively for a state-wide audit the total number of vote counts in the state in a race.

35 The Brennan Justice Center has suggested a rate of 15% to 30% (i.e. a 30% to 60% wrongful margin shift) and Howard

Stanislevic of VoteTrustUSA suggests a rate of 20% (i.e. a 40% wrongful margin shift). NEDA adopts 15% (a 30% margin shift)

in this paper, but encourages people to use a larger number to produce more conservative (larger) audit samples.

 National Election Data Archive Project September 25, 2006 ElectionArchive.org Page 13
AUDIT PROCEDURES & WHAT IF DISCREPANCIES ARE FOUND?

Audits of electronic vote counts would be useless without consequences and procedures to follow when

discrepancies are found. This paper does not go into detailed audit procedures. However we suggest:

• If the “election integrity audit” calculation results in an audit amount of less than 1% of vote

counts, then a minimum audit rate of 1% should be used. If 1% of vote counts is an integer plus

a fraction, then the next larger integer value should be used.36

• Audits performed on a county level

37

 are most conservative and practical because elections are

administered separately in each county and errors may be specific to particular counties. Statewide audits would prolong the audit process in all counties whenever discrepancies are found in

any county due to the reduced information they provide of each county’s election integrity.

• Only after an auditable report of unofficial vote counts is publicly released by election officials

should vote counts be selected and auditors randomly assigned to count each.

• After randomly selecting vote counts, extra vote counts could be selected either by candidates or

by criteria (such as whenever absolute increases in vote count margins from comparable

previous elections exceed the assumed maximum non-suspicious margin shift).

• States and county vote count audit committees comprised of independent38 PhD mathematicians

and statisticians, PhD computer scientists, gaming experts with masters’ degrees, election

integrity activists and non-voting members representing local election officials; all representing

several political parties, could be appointed to determine audit policies and procedures specific

to local election systems. These committees could oversee audits, make decisions on what to do

when discrepancies are found, produce vote count audit reports, and work with local election

officials and state legislators to suggest improvements to voting systems.

39

• Audit procedures should be transparently open to public observation.

• Audits must be performed by independent auditors who do not work for election offices or for

voting machine equipment or service companies or otherwise have access to voting equipment.

Auditors could consist of state audit officers; election judges or poll workers who did not work

during the election to be audited; persons whose names are supplied by candidates and authors

of ballot measures; or independent accounting firms.

• At least two political parties should be present to verify the hand-count of each voter-verifiable

paper ballot record.

• Audit results should be made publicly available before vote counts are made official.

• Any discrepancy between electronic and paper ballots must be evaluated to determine:

o Can the discrepancy be explained or not?

36 Any audit of less than 1% of vote counts could potentially not detect outcome-altering miscount in state-wide contests even in

cases where calculations show less than 1% to be adequate.

37 Some states like MA have townships administer elections and audits should be designed per each township. State-wide audits

would reduce the sample size per county and may result in reduced and varying probabilities of detecting vote miscount in

different counties due to differing margins, number of vote counts, and other factors specific to each county. State-wide designed

audits could be less efficient when some counties use optical scan precinct counts and others use DRE machine counts because

precinct counts would have to be audited in order to pool counts together for random selection.

38 Independent means not related to any election officials or voting vendors.

39 For instance, a Vote Count Audit Committee may recommend the implementation of unique anonymous identifiers that match

paper ballot to machine tabulated records, in order to facilitate audit investigations and corrections.

 National Election Data Archive Project September 25, 2006 ElectionArchive.org Page 14
o Could the amount of discrepancy have altered the outcome or not? 

o Can the vote count be corrected using existing available records or not?

• The audit should be expanded, perhaps to a 100% manual count in some counties, to search for

additional errors whenever it is possible that errors could have wrongfully altered an outcome. 

• There may be cases when it can be determined that vote miscount may have wrongfully altered

election outcomes, yet vote counts cannot be corrected using available records. Electronic ballot

voting systems are susceptible to election tampering in ways that are not detectable by audits, or

sufficient paper ballot tampering may be detected to wrongfully alter outcomes. In such cases

elections may have to be re-held.

• Election results should not be certified official until after the independent vote count audit report

is publicly released.

• The National Election Data Archive stresses that vote count data must be broken out by precinct

and by vote type (Election Day, early, early- provisional, Election Day – provisional, absentee,

overseas, military, etc.) both for statistical analysis and for audits. Otherwise errors could be

hidden in aggregated data. Different audit procedures may need to be followed for different

types of votes and different races within one county, parish, or township.

• Random selection must be done scientifically. An informative paper on the subject of random

vote count selection is “The Role of Dice in Election Audits – Extended Abstract”.

40

More research by teams of computer scientists, election officials, mathematicians, and others are needed

to establish best practices and procedures for vote count audits.

40 June 16, 2006 “The Role of Dice in Election Audits – Extended Abstract” by Arel Cordero, David Wagner, and David Dill.

http://www.cs.berkeley.edu/~daw/papers/dice-wote06.pdf

 National Election Data Archive Project September 25, 2006 ElectionArchive.org Page 15
Appendix A:

The probability estimates are based on a “Hypergeometric” distribution which determines the

probability of finding:

a) x (target corrupt electronic vote counts) We let x = 0 to find the probability of detecting no

corrupted vote counts.

b) in an overall sample of n (Number of Audited Precincts) which in this case is

 20 (for 2% audit) or 50 (for 5% audit),

c) when there are X (Hypothetical Number of Corrupted Precincts) which in two tables above

range from 50% to 0.05% of 1000. However, the best way to calculate X is [the smallest margin

between any candidates in any race] divided by [the maximum percentage of corruption that would not

raise immediate suspicion in any one electronic vote count] to give the number of corrupt vote counts

that could wrongly alter an election outcome. When there are differences in the size of precincts, X can

be calculated using the total number of ballots cast in each precinct where votes are casts in the audited

race.

d) out of N (Total Number of Precincts) which in this case is 1000.

This distribution is calculated using the Excel Function:

HYPGEOMDIST(x, n, X, N) =

X N X

x n x

N

n

  − 

  

  − 

 

 

 

The hypergeometric function assumes all individual “picks” are random but adjusts this random

probability for each pick. The first row in Table 1 for example, assumes a 50% probability that the first

of 20 picks will be corrupted, a 499/999 probability that the second precinct chosen will be corrupted,

and so on for all 20 picks. 

The probability that one or more of the 20 vote counts will be corrupt is 1 or 100% minus the probability

that none of the 20 precincts will be corrupted. So, the probability that at least one of the 20 is corrupted

which equals:

 

0

1

X N X

n

P

N

n

  − 

  

  

= −

 

 

 

 This equation can be solved using numerical methods for the sample size n.

For any county and race or ballot measure, the probability distribution will depend on the exact values of

n, X, and N.

A spreadsheet calculator for determining the probabilities of detecting one or more miscounted precincts

is available on the USCV web site: http://ElectionArchive.org/ucvAnalysis/US/paper-audits/

 

 National Election Data Archive Project September 25, 2006 ElectionArchive.org Page 16
Appendix B: Numerical Calculations

The following Matlab program shows the basic “election integrity” numerical algorithm for vote count

audit amounts. It needs revision to be fully general and usable in all situations.

 

%

% `k_d2_revised5.m'

%

% This routine computes an S, via method of bisections to an accuracy EP, given 

% a probability P , 0 < P < 1, positive integers N, and C, with 0 < C < N, 

% and with 0 <= S <= N-C, such that dopp(P,N,C,S) = 0, with (*) dopp(P,N,C,S) =

% ln(1-P)-ln[(N-C)! (N-S)!/(N! (N-C-S)!)].

clear all

format long e

%

% Initialization 

%

% Set this to 1 for precinct count audits

%

PrecinctCounts = 0;

%

% These are input values supplied by the user.

%

P = .95;

N = 211;

Maxshift = 0.3;

marginmin = .005;

marginmax = .20;

marginstep = .005;

EP = .005;

%

% These input values are supplied for precinct count audits only

%

TotBallots = 16900;

Precincts = ...

[921, 755, 731, 719, 687, 679, 676, 675, 675, 672, 670, 668, 668, 667, 660, 660, 660, 658, 657, 656, 656, 639, 639, 638, 636,

627, 621, 620, 620, 617, 617, 611, 609, 609, 608, 606, 605, 604, 596, 596, 594, 593, 593, 592, 591, 589, 588, 588, 585, 585,

585, 580, 579, 574, 572, 569, 568, 567, 565, 560, 554, 552, 552, 551, 551, 547, 547, 545, 544, 541, 541, 540, 539, 538, 537,

536, 536, 533, 532, 530, 528, 526, 526, 525, 524, 522, 519, 519, 518, 518, 517, 517, 514, 509, 509, 499, 497, 496, 495, 495,

495, 494, 494, 488, 488, 486, 486, 484, 481, 481, 480, 479, 479, 479, 478, 478, 478, 478, 477, 477, 476, 476, 475, 475, 474,

470, 468, 465, 465, 461, 460, 456, 455, 455, 455, 455, 454, 454, 453, 451, 450, 449, 448, 446, 444, 444, 443, 443, 443, 443,

438, 435, 435, 435, 434, 431, 429, 429, 429, 428, 427, 425, 425, 424, 424, 422, 422, 419, 417, 416, 415, 415, 412, 411, 407,

407, 400, 399, 398, 396, 393, 392, 391, 383, 380, 374, 372, 369, 366, 365, 365, 363, 362, 352, 349, 347, 346, 346, 340, 330,

327, 320, 320, 316, 316, 314, 314, 314, 311, 310, 287, 261, 247, 241, 223, 114, 94];

ArraySize =size(Precincts,2);

%

% Compute CumulativeShift

%

if(PrecinctCounts == 1)

 CumulativeShift(1)=Precincts(1);

 for i=2:ArraySize

CumulativeShift(i) = Precincts(i)+CumulativeShift(i-1);

 end

 for i=1:ArraySize

CumulativeShift(i) = CumulativeShift(i)*Maxshift/TotBallots;

 end

end

%

% Compute max j

%

maxj = (marginmax - marginmin + .005)/marginstep; % = 40

 National Election Data Archive Project September 25, 2006 ElectionArchive.org Page 17
%

% Compute C values % C = margin*N/Maxshift

%

for j=1:maxj

 margin(j) = j * marginstep;

 if(Precinctcount==1)

 i=1;

 while (margin(j) > CumulativeShift(i))

 i=i+1;

 end

 c(j)=i;

 else

 c(j) = floor(margin(j)*N/Maxshift);

 end

end

%

% Computing S via method of bisections

%

% First, the lower and upper end values of S

%

for j=1:maxj

 C(j) = c(j);

% First, the lower and upper end values

% of S

%

 A=0;

 B = N-C(j);

%

 a = dopp(P,N,C(j),A);

 b = dopp(P,N,C(j),B);

 while(abs(A-B) > EP)

 S = (A+B)/2;

 cc = dopp(P,N,C(j),S);

 if a*cc > 0 

 A = S;

 a = cc;

 else 

 B = S;

 b = cc;

 end 

 end 

 s(j) = S;

 S0 = ceil(S);

 c0 = dopp(P,N,C(j),S0);

 if(c0 >= 0)

 ss(j) = S0;

 else

 ss(j) = S0+1;

 end

end 

%

plot(margin,s,'-',margin,ss,'.', margin,C,'x')

pause

title('Plot of C & S VS margin, N = 211, .005 < margin < .2, P = .95')

xlabel(' margin = values = C*Maxshift/N')

ylabel('S (--) Continuous & SS (.) integer values')

print -dps2 dopp_plot211.ps

%

% Data Files

for j=1:maxj

 TOT(j,:) = [margin(j), C(j), ss(j)];

 National Election Data Archive Project September 25, 2006 ElectionArchive.org Page 18
end

save('tot','TOT', '-ASCII');

The following subroutine, dopp.m, is used in the above program. It evaluates ln(1-P) - ln[(N-C)!(NS)!/(N!(N-C-S)!)] for input values of P, N, C, and S.:

%

% `dopp.m'

%

function y = dopp(P,N,C,S)

%

QQ = log(1-P);

AA = gammaln(N+1);

BB = gammaln(N-C+1);

CC = gammaln(N-S+1);

DD = gammaln(N-C-S+1);

y = QQ + DD-CC +AA-BB;

The algorithm for the adjustment for precinct-size variation in audits of precinct counts, in words, is:

• Sort the precinct size array in descending order (largest to smallest)

• Find the cumulative total number ballots cast for the largest precinct, the two largest precincts, the three largest

precincts, etc. CumulativePrecinct =(PrecinctSize(0),

1

0

Pr ( )

i

i

ecinctSize i

=

∑ =

, 

2

0

Pr ( )

i

i

ecinctSize i

=

∑ =

,...)

• For each margin between candidates (beginning with the smallest margin) find the smallest value of i for which

(margin) < (CumulativePrecinct(i)*Maxshift/T)

• Compare i to (margin*N/Maxshift) and use the smaller value as the amount of corrupt precincts to input into the

calculation of audit size, S.

41

• Once we find that a value of margin for which the calculation is not needed, we do not perform the calculation

for larger margins where it will not be needed.

41 We recognize that we are using a calculation which compares integer with decimal. We may want to bring the floor integer

value for corrupt counts, C, into the calculation prior to this point. Little details still need review.

 National Election Data Archive Project September 25, 2006 ElectionArchive.org Page 19
Appendix C: Example Results with N=150 and N=1000

Notice that the number of electronic vote counts that must be audited to detect a particular rate of

miscount, does not change as significantly as one might expect when the total number of vote counts

changes. Thus the percentage of vote counts to audit is much higher when the total number of counts is

less.

 

The first example is for races with N=150 total vote counts when assuming a 15% maximum vote counts

shift per vote count (i.e. 30% margin shift). 

Audit Amount to Detect Outcome-AlteringVote Miscount

for N=150, P=95%

0

100

200

0.0% 5.0% 10.0% 15.0% 20.0%

CandidateMargins

Audit Amount

Audit Amount

Corrupt Counts toAlter

Outcome

 National Election Data Archive Project September 25, 2006 ElectionArchive.org Page 20

Candidate

Margins

Number

of Vote

Countsto

Audit

Percentage

of Vote

Countsto

Audit

Number

Corrupt

Countsto

Alter

Outcome

Percent

Corrupt

Countsto

Alter

Outcome

Probability

Check

0.5% 117 78.0% 2.0 1.3% 95.28%

1.0% 67 44.7% 5.0 3.3% 95.09%

1.5% 52 34.7% 7.0 4.7% 95.30%

2.0% 38 25.3% 10.0 6.7% 95.16%

2.5% 32 21.3% 12.0 8.0% 95.05%

3.0% 26 17.3% 15.0 10.0% 95.09%

3.5% 23 15.3% 17.0 11.3% 95.06%

4.0% 20 13.3% 20.0 13.3% 95.39%

4.5% 18 12.0% 22.0 14.7% 95.25%

5.0% 16 10.7% 25.0 16.7% 95.45%

5.5% 15 10.0% 27.0 18.0% 95.68%

6.0% 13 8.7% 30.0 20.0% 95.21%

7.0% 11 7.3% 35.0 23.3% 95.22%

8.0% 10 6.7% 40.0 26.7% 95.99%

9.0% 9 6.0% 45.0 30.0% 96.38%

10.0% 8 5.3% 50.0 33.3% 96.46%
This next example is for a race with N=1000 total vote counts, much larger than average, except for

state-wide races, assuming a 15% maximum vote counts shift per vote count (i.e. 30% margin shift). 

Candidate

Margins

Number

of Vote

Counts to

Audit

Percentage

of Vote

Counts to

Audit

Number

Corrupt

Counts to

Alter

Outcome

Percent

Corrupt

Counts to

Alter

Outcome

Probability

Check

0.5% 170 17.0% 16.0 1.6% 95.05%

1.0% 86 8.6% 33.0 3.3% 95.11%

1.5% 57 5.7% 50.0 5.0% 95.08%

2.0% 43 4.3% 66.0 6.6% 95.03%

2.5% 34 3.4% 83.0 8.3% 95.01%

3.0% 29 2.9% 100.0 10.0% 95.50%

3.5% 24 2.4% 116.0 11.6% 95.00%

4.0% 21 2.1% 133.0 13.3% 95.17%

4.5% 19 1.9% 150.0 15.0% 95.58%

5.0% 17 1.7% 166.0 16.6% 95.55%

5.5% 15 1.5% 183.0 18.3% 95.29%

6.0% 14 1.4% 200.0 20.0% 95.70%

7.0% 12 1.2% 233.0 23.3% 95.94%

8.0% 10 1.0% 266.0 26.6% 95.53%

9.0% 9 0.9% 300.0 30.0% 96.03%

10.0% 8 0.8% 333.0 33.3% 96.14%

Audit Amount to Detect Outcome-AlteringVote Miscount

for N=1000, P=95%

0

100

200

300

400

500

600

700

0.0% 5.0% 10.0% 15.0% 20.0%

CandidateMargins

Audit Amount Audit Amount

Corrupt Counts toAlter

Outcome

Prepared by Kathy Dopp of

National Election Data Archive Project (NEDA) - ElectionArchive.org

501(c)(3) status approved

Contact: Kathy Dopp kathy@uscountvotes.org 435-608-1382

The National Election Data Archive is a scientific research project whose mission is to ensure the accuracy of

elections in America through development and promotion of mathematical means to ensure accurate vote counts

and the creation and analysis of a database containing precinct-level election data broken out by vote type for the

entire United States. NEDA is seeking funds to complete a nationwide system for mathematically monitoring

election results.

 National Election Data Archive Project September 25, 2006 ElectionArchive.org Page 21