Issues in Open Ended Vulnerability Testing---DRAFT

John Kelsey, NIST, March 2006

1.  Introduction

In resolution 17-05, the members of the TGDC agreed that evaluation of
voting systems in the future should involve **open-ended testing**.

> *The TGDC directs NIST to research and draft standards documents
> requiring testing of voting systems that includes a significant amount
> of open-ended research for vulnerabilities by an analysis team
> supplied with complete source code and system documentation and
> operational voting system hardware. The vulnerabilities sought should
> not exclude those involving collusion between multiple parties
> (including vendor insiders) and should not exclude those involving
> adversaries with significant financial and technical resources.*

We can most efficiently define open-ended testing by contrast with the
kind of testing that is common on most electronic systems, including
electronic voting systems. Most of the review and testing done by a
Voting System testing Laboratory (VSTL) on a voting system will
inevitably be testing for compliance. That is, the standards will
specify that some security control[^1] is to be used, and the VSTL will
verify that it is, in fact, being used. We use the term "open-ended
testing" to refer to testing which checks whether the security measures
in place are really enough to prevent an attack, most often by trying to
defeat those security measures in some way. For example:

-   Normal testing reviews source code to ensure that the coding
    standards are being followed, and that the code does what its
    documentation says it does. Open-ended testing reviews source code
    for vulnerabilities, both by automated search and by careful human
    review.

-   Normal testing reviews operating system security to ensure that
    event logs are kept, with the right events being logged. Open-ended
    testing reviews whether the event logs actually capture the events
    they're supposed to capture, and whether there's some way for an
    attacker to alter the logs before they can be reviewed.

-   Normal testing reviews communication security to ensure that
    appropriate standards are being followed---for example, that TLS is
    used to secure a link, and that the keys are managed properly.
    Open-ended testing examines the full system to see whether there is
    some way to bypass the cryptographic protection used.

Normal testing is relatively straightforward, and one properly done
evaluation should turn out very nearly identical to another. Different
labs reviewing the same system at different times should come to very
similar conclusions about the acceptability of the voting system. There
is a clear point at which the testing is finished, and this makes
budgeting for the test straightforward. When a system passes this test,
it means that the VSTL has verified that the system complies with the
standard. While there will be small variations in outcomes between
evaluations (for example, a more diligent reviewer may notice some
deviations from the standard that a more lax reviewer doesn't catch,
especially where the standard is itself somewhat broadly defined), a
voting system must be in substantial compliance with the standard if it
is to pass this kind of evaluation.

Open-ended testing is inherently much less straightforward. Two
different evaluations may not use the same attacks or tools, and are
thus rather hard to compare. There's often no clear point at which the
testing is finished, because while automated testing tools are useful,
human review is also necessary. When a system passes an open-ended
evaluation, it means that the VSTL has not found any substantial
weaknesses in the system, but this doesn't guarantee that another
reviewer would not have found some weaknesses.

Normal testing is like asking a policeman to check your doors, windows,
locks, and lighting to make sure they're good quality and are installed
correctly. Open-ended testing is like asking a professional burglar to
see if he can get into the house and steal your television. Both have
value, and each tests a somewhat different part of your security.

1.  Why Do Open-Ended Testing?

There are limitations to open-ended testing: It can be expensive, it
isn't repeatable, and it's hard to determine when enough of it has been
done. This raises the question: Why do open-ended testing at all?

The answer is that no checklist or standard can guarantee a secure
system. Because of this, open-ended testing is necessary at a number of
different levels in a voting system:

-   Because strong components can always be combined into a weak system,
    evaluators must look at the voting system as a whole, and attempt to
    find ways to subvert or disrupt the election or violate voter
    privacy. For example, the Diebold system reviewed by the RABA report
    used smart cards with passwords that should have protected them from
    duplication or misuse. However, the passwords appeared in the system
    source code (which had been posted to the internet by mistake), and
    so these strong components were bypassed and a variety of attacks
    became possible.

-   Because strong security controls are often used incorrectly or fail
    to fully protect important resources, evaluators must examine
    whether the protection mechanisms are effective at preventing
    various prohibited behaviors that could lead to an attack. For
    example, the RABA report discovered a misapplication of the SSL
    cryptographic protocol, which would have allowed an attacker to
    intercept the transmission of electronic results to the tabulation
    center and tamper with those results in transit, despite the
    cryptographic protection.

-   Because knowledge about attacks and weaknesses advances faster than
    standards change, evaluators must check that the security of the
    components of a voting system are still secure, despite any recent,
    newly disclosed attacks. For example, the RABA report described a
    number of known security issues with the GEMS server used in the
    Diebold system for collecting and tabulating the votes which had not
    been addressed. This left the system vulnerable to a number of
    attacks.

-   Because all voting systems depend critically on procedures carried
    out by election officials, evaluators must verify that the
    procedures spelled out in the vendor documentation are sufficient to
    ensure good election security, and that those procedures can
    reasonably be expected to be followed.

It's worth noting that any of these problems might have been detected by
checklist testing, *if the standard writers had anticipated the
problem.* Open-ended testing is valuable precisely because standard
writers can't anticipate every problem and specifically exclude it.

1.  Tradeoffs Between Open-Ended and Checklist-Type Testing

The main value of OEVT is to test parts of the voting system which can't
be standardized in a consistently secure way. Where security controls
like approved encryption algorithms can be specified and verified, it's
more efficient to do this. OEVT can then be used to check that the
encryption algorithms are being used correctly---more generally, to
check that the resources being protected by required security controls
are, in fact, being protected.

1.  Code Review

While OEVT isn't the same thing as code review, the OEVT team must be
given full access to the source code of the voting system, and it's
likely that the same people will sometimes be involved in both kinds of
testing. Code review involves verifying compliance with coding
standards, verifying the absence of obvious backdoors or hidden
functionality by inspection, and looking for software vulnerabilities
that might lead to attacks on the voting system. Of these, only the last
is part of OEVT. During the code review, off-the-shelf tools should be
used to check for common vulnerabilities, but during OEVT, the
evaluators will analyze specific parts of the source code for
vulnerabilities in depth.

2.  Existing Open-Ended Evaluations

    1.  Voting Systems

In the last few years, a number of high-profile published attacks have
taken place to demonstrate, in laboratory conditions, that voting
systems could be attacked. Probably the best documented and most
professional of these appear in the RABA report. In that report, a
number of vulnerabilities are discovered and analyzed, with the goal of
demonstrating attacks on a particular real-world voting system.

One striking thing about the RABA report and the other reports along
these lines is that it appeared that the evaluators ran out of time and
resources long before they ran out of vulnerabilities, and that
different evaluators, even working from the same documentation and
copies of one another's' reports, often came up with different attacks!
Another is that the evaluators spent some substantial amount of time
constructing full attacks out of vulnerabilities that, once
demonstrated, were sufficient to call the security of the voting system
into question. A third important lesson from these reports is that many
of the problems discovered were fairly simple to fix, once they were
discovered. Had this kind of analysis been done before the systems had
been sold to various states, those vulnerabilities would have been
closed, and hopefully, the voting systems would have been much harder to
attack as a result of this.

2.  Open-Ended Testing of Non-Voting Systems

The kind of analysis proposed in this document is new. However, various
forms of open-ended analysis and penetration testing are used in
specific areas of computer security. In addition, there are many tools
available for carrying out some level of automated security testing.

1.  Cryptographic Algorithms and Protocols

Cryptographic algorithms and protocols are evaluated in an informal way
within the cryptographic community for many years, before seeing
widespread use. Some examples of formal periods of evaluation used for
cryptographic mechanisms include:

-   RIPE

-   AES

-   NESSIE

In addition, standards organizations such as the IETF, IEEE, X9, and ISO
usually wait for at least an informal consensus to come about within the
cryptographic community before choosing to standardize an algorithm or
protocol.

1.  Cryptographic Devices and Implementations

Open-ended testing is commonly done against special-purpose
cryptographic modules, which are also evaluated against elaborate
checklists of requirements.

2.  Secured Networks

Penetration testing is reasonably common for networks, and some
companies offer this service commercially.

3.  Operating System Security Mechanisms

Operating systems and widely-used programs are subjected to a kind of
unstructured version of this, as people are asked to look for
vulnerabilities.

1.  Issues to Discuss in This Paper

This white paper is intended to describe our approach to OEVT at a high
level, and then to surface a number of important policy issues for
discussion and decisions by the TGDC and EAC. We will also surface some
technical issues, which we intend to resolve by further research and
consideration over the coming months.

2.  High Level OEVT Approach

    1.  Overview

At a high level, the OEVT process will proceed as follows: First, the
vendor will submit a voting system, including some sample machines, all
software (including source code for non-COTS software), all devices
required for the election, and a full set of security and documentation
and description of procedures as they are to be given to the election
officials. The evaluators will then review these things, looking for
potential attacks. They will develop a list of problems, and the details
of this list will determine whether the voting system fails
automatically, passes, or whether the decision of whether it passes or
fails requires more information or some policy decision from the EAC.
Where problems are obviously correctable, the evaluation team should
provide some minimal guidance for how to correct them, but it must not
become involved in the design of the voting system beyond that minimal
guidance. When all the critical problems have been corrected and the
system passes, a final report is produced about the findings of the
open-ended evaluation on the final system. This includes any attacks or
problems which are known, but which have been deemed not to undermine
the security of the full election system. At least some version of this
report should be made public.

2.  Inputs from Vendor and Rules of Engagement

The evaluation process begins when the vendor provides some sample
hardware sufficient to run a full (very small) election. Some minimal
number of voting machines will be needed, and they may suffer some
damage or even be destroyed in testing. (This may be necessary for
testing physical security, for example.) Full details of the system are
given to the evaluators, including source code for all voting system
software. The evaluators may be required to agree not to disclose the
internal details of the system. The vendor provides a full description
of the required procedures for running the election correctly, including
procedures for recounts. The vendor must also provide a thorough
description of how the election system works, including how all
information flows between the components, how software is updated on the
machines, and how results and event logs are recovered and checked after
an election.

Along with all the other details, the vendor also provides security
documentation. This explains how the specific security goals of the
system are met, providing a broad argument for the security of the
voting system. The security documentation should explain what defenses
prevent each broad kind of attack on the voting system (tampering with
the election outcome, violating voter privacy, disrupting the election,
and discrediting the election). For each intermediate attack goal (see
below) which applies to this voting system, the security documentation
should specify what security controls are intended to prevent the goal
being achieved.

The evaluators require some rules of engagement---basically, the limits
they put on themselves. These will include such questions as how long a
physical attacker is assumed to have to carry out an attack on a voting
machine, which roles the evaluator is allowed to take on in the voting
system, what inputs he is allowed to provide, etc[^2]; the more powerful
the evaluator, the more meaningful the evaluation. The rules of
engagement must be included exactly in the final public report from the
evaluation, because they determine how the results should be
interpreted.

The evaluation is only meaningful if the materials provided to the
evaluators are representative of the ones used in the field. It is
critical that the tested system is the same as the one that is fielded.
Some measures must be taken to ensure that this is the case, but these
are outside the scope of our standards.

3.  Attacks and Intermediate Attack Goals

Open-ended evaluation of a voting system involves determining whether
some action the evaluator manages to do represent a weakness in the
voting system. We propose two different ways to determine when the
evaluators have revealed an important security problem in the voting
system, both of which, when found, lead to failing the voting system:

-   Full attacks on the voting system are actions taken by the
    evaluators that, if carried out on Election Day, would lead to an
    attack on the voting system. The attacks we are concerned with for
    this analysis include:

    -   Insiders (as specified by our rules of engagement) changing the
        outcome of an election.

    -   Insiders determining how specific voters voted without their
        cooperation.

    -   Outsiders disrupting an election with a low probability of being
        caught at a crime.

    -   Outsiders discrediting an election by causing apparent false
        evidence of fraud to be produced, or preventing true evidence of
        correct operation from being created.

-   Intermediate attack goals are actions taken by the evaluator which
    violate some critical security goal of the voting system. Other
    components of the voting system may or may not prevent a full attack
    from working; however, this represents a major failure in security.
    Generating a list of these intermediate attack goals is an
    interesting technical issue. However, there are a number of
    straightforward examples:

    -   Given access to a voting machine and all its internal memory,
        reconstruct the order of voters.

    -   Given complete control of one voting machine, tamper with the
        operations of the tabulation center computer.

    -   Cause the execution or installation of any piece of software on
        a voting machine without proper authorization.

    -   Cause any event that is supposed to be logged to occur without
        an entry appearing in the event log.

    -   Given complete control of the tabulation center computer, tamper
        with the operations of the voting machine.

    -   Given physical access to the voting machine for N minutes
        unobserved, gain access to any critical internal parts of the
        system without leaving evidence of tampering.

    -   Tamper with any electronic records from the election either on
        the voting machine, in transit, or on the tabulation center
        computer, without detection.

There are two important reasons to use intermediate attack goals, as
well as full attacks:

-   The evaluators' time and resources are limited. By letting them stop
    at these intermediate attack goals, we more efficiently use their
    time; having discovered a weakness that should not be in a voting
    system, it makes more sense to have the vendor close that weakness
    than to spend more resources having the evaluator work out how to
    turn it into an attack. This translates into either a less expensive
    evaluation process, or more money available for other parts of the
    testing of the voting system.

-   Failing a system for intermediate attack goals encourages defense in
    depth; the voting system should have many levels of secure
    components which must be defeated to compromise an election, since a
    real-world attacker may be smarter and better funded than the
    open-ended evaluation team. Thus, we allow an intermediate attack
    goal like tampering with the operations of the tabulation center,
    given control of a voting machine. The attacker should never gain
    control of a voting machine (that's the subject of several other
    intermediate attack goals), but if he does, he had better not be
    able to take control of other computers from that one.

    1.  Interfacing with the Rest of the Standard

Intermediate attack goals also offer a way for the open-ended evaluation
to be linked into the rest of the standard. All the security controls
mandated for a given voting system are there to accomplish some goal;
those parts of the standard can specify not only the required security
control, but also an intermediate attack goal which, if reached, would
represent undermining that security goal.

The main requirements for an intermediate attack goal are:

-   Accomplishing it must demonstrate a failure in some part of the
    security of the voting system.

-   The evaluator must be able to unambiguously demonstrate that it has
    been accomplished.

-   The fact that it has been accomplished should be serious enough to
    cause the voting system to fail evaluation.

    1.  Process of Evaluation

The purpose of the evaluation is to take those inputs from the vendor,
and check the security claims of the voting system by trying to find
weaknesses or attacks. The following steps are listed in approximate
order, but information will flow between the steps. In addition, members
of the evaluation team may be involved in the normal testing of the
voting system.

1.  Overall Review of Voting System

The most open-ended part of the analysis is an overall review of the
voting system, based on the machines, software, and security
documentation. The goals here are:

-   To apply the results of the threat analysis document, and any other
    ideas the attacker may have, to attack the voting system.

-   To look for weaknesses in the design that allow an attacker to
    bypass the security controls that are in place.

-   To highlight particular points of concern in the voting system that
    can be more carefully analyzed later in the evaluation.

-   To determine whether there are flaws in the underlying security
    reasoning going into the security documentation. (For example, if a
    listed security control doesn't prevent some intermediate attack
    goal, this should at least be pointed out.)

If potential attacks on the design of the voting system are found, the
evaluators may try to determine if there are simple fixes, or obvious
errors in the documentation leading to the attack. If not, the initial
result needs to be reported back to the vendor, and depending on the
details, it may not make sense to continue the evaluation until the
design flaws are corrected.

If the vendor documentation is incomplete or unclear in some important
area, the evaluators can request further information from the vendor. It
will usually make sense to wait for the documentation to be complete
before continuing with the evaluation. We expect that it will be
reasonably common for the evaluation team to contact the vendor at the
end of this step, and request corrections and clarifications before
continuing the evaluation.

1.  Conditions for Failure

The voting system unambiguously fails if:

-   The evaluators can demonstrate a complete attack on the voting
    system, allowing insiders to substantially change the outcome of the
    election or outsiders to disrupt it without high probability of
    getting caught, on the equipment and with the procedures given. This
    will be based partly on the ongoing work on threats on voting
    systems being done by NIST and the Brennan Center.

-   The security documentation does not address all the high-level
    attacks, or leaves out any applicable intermediate attack goals.

The vendor should be contacted for corrections and clarifications where
there are questionable or incomplete areas of the security
documentation, as these may be easy to resolve at this point.

1.  Useful Intermediate Outputs from Analysis

The most important output of this step is likely to be an understanding
of which parts of the voting system need special scrutiny. The security
documentation also gives the evaluators some notion of what the
designers of the system think are the important security controls.

1.  Review of Procedures

The specified procedures are an integral part of the voting system. The
evaluators must review the procedures that are assumed for the security
of this voting system, and answer the following questions:

-   Do the procedures address the threats they're required to address?

-   Are the procedures reasonable for election officials to follow?

-   What are the consequences if the procedures are not followed?

-   What evidence would be produced by following these procedures or
    failing to follow them? Would anyone notice which was done?

This part of the open-ended evaluation seems to us the most likely
source of ambiguity in the evaluation. In particular, whether a set of
procedures is practical or not will inevitably be a matter of debate.

Note that the security documentation must address why the specified
procedures accomplish their specific security goals. Among the
procedures which must be specified:

-   Storing the voting machines safely

-   Setting up voting machines and loading ballot definitions.

-   Updating voting machine software

-   Running an election

-   Running a recount

-   Disaster recovery

    1.  Conditions for Failure

The voting system unambiguously fails if:

-   The procedures given can be shown not to provide the security
    claimed for them. For example, if the specified procedures for
    ensuring that the voting machines start the day with no stored votes
    do not ensure that, then the voting system fails.

-   The procedures given don't work with the hardware and software
    provided. This simply has to be fixed before the rest of the
    evaluation can continue.

-   The procedures are obviously impractical for the election officials
    to use.

    1.  Useful Intermediate Outputs from Analysis

This step of the analysis provides the evaluators with knowledge of what
procedures are critical for system security. This can guide later
analysis in which the software or hardware supporting those procedures
is attacked to subvert the critical procedures.

1.  Automated and Manual Searching for Known or Suspected
    Vulnerabilities

Each component of the system can be checked against lists of known weak
components. That is, each software component can be checked to see if it
has known weaknesses, the configuration of the operating system and the
installed services can be checked to see whether they include things
that are known to be weak, etc.

This is probably the least ambiguous, least open-ended part of the
analysis.

1.  Conditions for Failure

The voting system unambiguously fails if:

-   The evaluators find one or more components which both:

    -   Are known to have some weakness

    -   Through that weakness, allow either an attack on the full voting
        system or the achievement of one of the intermediate attack
        goals.

-   The evaluators find too many known weak components.

-   The evaluators find weak components, and the weakness applies
    directly to how the component is used. For example, if a component
    is known to be weak if the machine it is installed on has network
    access, and the voting machine has network access, then the voting
    system is failed.

The "open-ended" part of this analysis comes after known weak components
are found. The evaluators must determine whether the weaknesses in the
components make the system vulnerable. However, before that's done, the
evaluators should report these known weak components to the vendor, and
ask whether updated software can be made available. Ideally, the vendor
would respond with either patched software or explanations of why the
remaining known weak components could not lead to vulnerabilities in the
full voting system.

An interesting policy/technical question arises when there are known
vulnerable components for which no full attack on the voting system is
known. Our current plan is that known to be vulnerable components should
always be patched, replaced, or eliminated. However, in cases where no
patch or replacement is available, they may be allowed if there is no
chance of a vulnerability being introduced into the voting system. This
typically means that the weak component is shielded from any unverified
or untrusted input. For example, many software attacks are known based
on passing untrusted input to a regex or SQL engine. A known
vulnerability in one of these engines could be mitigated by ensuring
that untrusted input isn't able to get through. In this case, the
evaluator would fail the system if he could get any invalid or untrusted
input into the system.

1.  Useful Intermediate Outputs from Analysis

To the extent that there are huge numbers of known weak components, the
output from this analysis is too time-consuming to use. Most of the
useful output from this step should go directly to the vendor, and
should lead to his updating all the weak components for which updates or
patches are available.

1.  Intermediate Attack Goals and Analysis of Components

Using all the information provided by the vendor, the evaluators now
attempt to carry out any of the intermediate attack goals. Although the
evaluators will have physical access to the voting hardware and
passwords for accounts on the machines when these exist, the specific
intermediate attack goals and the rules of engagement will determine
what resource can be used in attempting to meet the intermediate attack
goal.

If there are a relatively small number of intermediate attack goals
which are relevant, then each should be attempted, and some notes about
the attempt recorded in the final report. If there are a large number of
intermediate attack goals, this may not be practical---in this case, the
evaluators must specify which goals they attempted to reach, and why
they chose these goals.

1.  Deciding Pass or Fail

After the evaluation has been completed, the evaluators may already be
certain that the system has either passed or failed. For example, if any
of the conditions for failing the system listed above have been met, the
system will be failed. If no real problems have been discovered, or all
such problems have been resolved, the voting system will pass
evaluation.

In some cases, the situation will be more ambiguous. For example, there
may be issues with the specified procedures that need further
consideration, or there may be a large number of problems that
individually don't lead to failing the system, but which may sensibly
result in failing the system. There will have to be some mechanism for
resolving this kind of case.

Ambiguity in the outcome of the evaluation is bad for everyone: Vendors
end up unsure what is required for success, evaluators end up uncertain
where to concentrate their efforts, and state election officials and the
public end up unclear on what the evaluation means. In the worst case,
the decision to pass or fail a voting system could be the result of a
negotiation between a voting system vendor who is writing checks and the
VSTL that is cashing them.

This is one of the main reasons for the intermediate attack goals on our
conception of OEVT for voting systems---they remove much of the
ambiguity in the results of the evaluation. They also give both vendors
and evaluators guidance about where to focus their efforts.

2.  Reporting Results to the Vendor

As described above, some intermediate results should be reported back to
the vendor, especially where a problem is likely to be easily fixed. For
example, software that hasn't been patched to close some known security
holes is almost always going to be smarter to patch than to analyze for
whether those security holes allow the compromise of the whole voting
system.

The final report to the vendor must specify what was looked at, and what
the results were. If the system is failed, each known cause of failure
is reported, along with any brief suggestions for how the problem could
be fixed. It is critical that the evaluators not become involved in the
design of the voting system, because they will almost certainly be
evaluating the fixes later. However, simple recommendations, along the
lines of "put individual passwords on the smartcards" or "turn on TKIP
on the wireless network" are entirely reasonable.

Additional concerns which may raise potential problems, but weren't
enough to cause the system to be failed, should also be reported to the
vendor. The vendor should be encouraged to fix those problems.

3.  Interaction and Resubmission of Systems

Systems which fail will usually be resubmitted. While there is no
requirement that they be resubmitted to the same evaluation team (it
would be better in security terms if many teams evaluated the system),
the cost of re-evaluating a system with which the team is already
familiar will be much lower than that of evaluating a new system.

The interaction between the evaluators and the vendors offers an
opportunity for large improvements in security.

4.  Final Public Output

The final report will be provided to the vendor, but will also include a
public report which must be made available at least to any election
official who considers buying the voting system, and probably should be
made available to the public.

This must include:

-   The documentation and procedures assumed for the evaluation.

-   The rules of engagement for the evaluation.

-   The list of unresolved non-critical issues in the voting system.

-   A full description of the hardware and software reviewed, including
    versions of COTS software used, and SHA256 hashes of all source code
    supplied by the vendor, and also including pictures and diagrams of
    the hardware sufficient to make it easy to tell if the same hardware
    is being described in the report as what is being used. COTS
    hardware components used in the voting system must be listed and
    described.

-   A high-level description of what was considered in the evaluation,
    and what conclusions were reached. This mustn't reveal proprietary
    details of the voting system which were given under NDA, but must
    describe what sorts of things were evaluated. For example, "Because
    the smartcards are critical parts of this voting system's security,
    we verified that the vendor's procedures would put different PINs on
    each smartcard, and that the smartcards would cease to function
    after ten incorrect PINs were entered in a row. We also attempted to
    compromise the information on the smartcard by communicating with it
    via a standard smartcard reader, using the interface provided by the
    manufacturer of the smartcards."

    1.  Conclusions

This section is intended as a high-level summary of our current approach
to OEVT. While many details are not yet understood, this summary should
be sufficient to motivate the discussion of policy and technical issues
that follows.

3.  Open Policy Issues

A number of policy decisions will need to be made regarding OEVT of
voting systems. In this section, we consider some important questions
that will have to be answered. Many of these issues overlap with
technical issues, and the answers will be based on an overlap of
technical and policy considerations.

1.  Goals

We propose a set of high-level goals for the open-ended testing process:

-   The process should be as transparent as possible, without revealing
    either proprietary information of the vendors or information that
    makes the voting systems more susceptible to attack in the field.

-   The process should use testing resources efficiently.

-   The process should improve the quality of voting systems that go
    through it.

-   The results of testing should be as consistent as possible.

Evaluation should improve justified public confidence in voting system
security. Making the testing program as transparent as possible will
help accomplish this goal. However, this must be balanced against the
problems of disclosing proprietary information of both the vendor and
the VSTL, and against the bigger problem of disclosing information that
would help real-world attackers.

The goal of open-ended testing is not to demonstrate that voting systems
are vulnerable to attack---we already know that, because essentially all
real-world computer systems are vulnerable to attack. Instead, the goal
is to improve the security of voting systems against real-world
attackers. This suggests that the process of preparing a submission
package for evaluation should lead vendors to a better understanding of
their own security, and that VSTL feedback from the evaluation, whether
it is passed or failed, should provide useful guidance on how to fix the
problems found.

Open-ended testing is inherently less repeatable than other kinds of
testing. A different set of evaluators will often find somewhat
different problems. However, the procedures and rules for testing can be
made consistent---the same or very similar rules of engagement,
pass/fail criteria, and high level goals are used by all VSTLs.

1.  Money and Resource Allocation Issues

There are three major issues involving money and resource allocation:

-   Testing a voting system's security can be expensive, and open-ended
    testing is especially expensive. Detailed requirements for OEVT
    cannot be written without an estimate of what resources will be
    available for security testing.

-   Once an approximate budget for security testing (as opposed to
    usability testing and other requirements testing) is determined, a
    tradeoff will exist between checklist testing and open-ended
    testing.

-   A business relationship will exist between a voting system vendor
    and a VSTL, and the ability to do business together in the future
    may be important to both. Arrangements must be found that will
    minimize the impact of that business decision on testing results.
    Among the measures that may be useful are clear fass/fail criteria
    (to minimize the amount of negotiation about results) and a lab
    accreditation process that encourages meaningful and independent lab
    results.

    1.  How Much Money is Available for Testing?

The most important question is how much money is likely to be available
for evaluation. It is hard to imagine this open-ended evaluation process
costing much less than \$100,000 if done properly, and a more thorough
job might cost two or three times this. This is in addition to other
testing, such as checklist security testing, usability testing, and
hardware reliability testing.

2.  How Should Resources Be Allocated in Security Testing

The second question is how this money should be allocated between the
more straightforward checklist-type testing and open-ended testing.
Checklist testing has a lot of value---it can make sure that systems are
using the required security controls, and to an extent verify that
they're used in the right ways, and in many cases, it is relatively
cheap to verify that a large set of potential attacks is blocked by the
use of the right kind of security controls.

On the other hand, the open-ended analyses we've seen of voting systems
demonstrate that the checklist approach is simply not sufficient to
provide security in the face of real attackers. The Hopkins report was
written by three academics with access to nothing but some leaked source
code, and yet it revealed weaknesses that had not been corrected as a
result of the ITA review. The SAIC report presumably found many
problems, given the long list of procedural defenses imposed. However,
most of their findings were redacted. The RABA report found a long list
of problems, many of which could have been used by themselves or in
combination with procedural attacks to undermine real elections. Better
ITA testing might have caught some of these problems, but would likely
not have found them all. Open-ended testing is the best way we can find
to tighten the security of these voting systems.

This is a policy question, but one that will have to be reconsidered
over time, as we get more insight into how effective open-ended testing
can be with more thorough standards. There is a wide range of choices
available here, ranging from giving the VSTL a couple days to do some
open-ended evaluation once all the checklist-type testing is done, to
spending many times the resources of the checklist-type testing on the
open-ended evaluation.

3.  Who Pays the Bill? Incentives and Testing

Many people have commented on the incentives facing the testing labs.
Voting system vendors and testing labs are often in a long-term business
relationship---the vendor plans to resubmit future revisions and new
systems to the lab, and the lab is paid for doing the testing. This
raises both the possibility and the appearance of a conflict of
interest.

At present, our best recommendation is to try to remove as much
ambiguity as possible from the evaluation results. When some requirement
is clearly not met, or some intermediate attack goal is clearly
demonstrated, there is little room for negotiation---a vendor may decide
not to use the VSTL in the future, but probably cannot avoid addressing
the problem that was demonstrated. This motivates the use of
intermediate attack goals in the analysis.

A second recommendation is to make the accreditation of the labs
dependent on the quality and independence of its work. Programs such as
FIPS-140-1 involve similar incentive issues, but appear to provide
substantially independent evaluation of submitted devices.

An interesting related financial question involves retesting. When a
voting system fails evaluation, it will presumably be updated and
retested. How is this paid for? A fixed price for the full evaluation
will limit the resources available for retesting. On the other hand, a
voting system that may have to retest many times could become
arbitrarily expensive to get through the VSTL if each retest is charged
separately.

1.  Interaction Between VSTLs and Vendors

Many policy issues revolve around the interaction between voting system
vendors and VSTLs. The interaction should be consistent, and should be
designed to allow the testing process to improve the security of the
final voting system. The final report to the vendor should help to
improve future voting systems. (A related public report will be
described below.)

The important policy issues center on three broad questions:

-   What information must the vendor provide in the evaluation package,
    and what, if anything, may be withheld?

-   How should the rules of engagement for the evaluation be negotiated?

-   What information should be provided back from the VSTL to the
    vendor?

    1.  The Submission Package

The evaluation can't start until the vendor puts together a package of
all the required information. We will call this the submission package.
Most of the contents of this submission package are determined by
technical considerations. For example, full source code from the
vendor's programs, a listing of all COTS software (including versions)
and hardware suppliers and versions, and security documentation as
discussed above, along with some sample machines, have to be provided.

1.  Ensuring the Honesty of Submissions

The most important policy issue with regard to the submission package is
how we can ensure that the submission is an accurate representation of
the system that will be used in running elections. This is a common
problem for any kind of testing (it arises in crash testing of cars, for
example), but for open-ended evaluation, it is especially severe,
because it is possible for fairly minor changes to the system to
substantially change the outcome of the evaluation. For example:

-   Changing OS parameters or system settings may make the system much
    harder to attack, at the cost of degrading performance or usability
    for voting officials.

-   Putting the same password on each machine by default, or having all
    the voting machine locks use the same key, may make the system
    cheaper to manufacture and use. It would be easy for the vendor to
    change this for the evaluation version.

-   Sturdier physical construction of voting machines may make them
    harder to tamper with, but may also increase production costs.

In all three examples, the assumption is that the vendor is knowingly
making a tradeoff between security and other goals, but doesn't want to
reveal this to the evaluation team. This kind of submission of an
"armor-plated" version of a product is an explicit concern of
organizations that crash test cars, because it's always possible to make
a car more resilient to crash damage by making it heavier and more
expensive to build.

This is a crucial issue for ensuring that the results of the testing are
meaningful, but it is outside the scope of our standards.

1.  Ensuring the Accuracy of Submissions

A related issue involves unintentional submission of bad information.
For example, an older version of the source code might be accidentally
provided to the evaluation team. Good procedures need to be developed to
minimize the chances of this.

2.  Attackers Within the Vendor

Finally, there is the possibility that an intentional attacker within
the voting system vendor will insert some extra code in the system.
While the open-ended evaluation team should look for this sort of thing,
this isn't the focus of the evaluation, and it is likely to be quite
hard for the evaluation team to detect it. Criminal penalties are
unlikely to do much to prevent this, since the attacker will already
know he is committing a crime. Similarly, incentives for the vendor not
to deceive the evaluation team aren't too useful, since the attacker is
also deceiving the vendor. Good version control procedures and internal
computer and network security within the vendor's development
environment can make this much harder, but it isn't clear how much of
this should be made into testable requirements. (Testing the vendor's
development environment for security is outside the scope of OEVT.)

3.  Other Issues

Other minor policy issues involve:

-   Whether there is any information the vendor can intentionally
    withhold from the evaluation team.

-   How to deal with minor changes to the system over time, such as new
    versions of COTS components.

    1.  Rules of Engagement

Rules of engagement determine what will and won't be tested, and what
limits the evaluators will accept on their own power. For example:

-   Evaluating the tamper-resistance of a voting machine will have some
    limits on the amount of time given to the attacker.

-   Insider attacks must be considered in the open-ended evaluation.
    However, where there are different roles enforced by the voting
    system or different machines involved, it makes sense to limit how
    many different entities the attacker is allowed to control.

-   Certain procedural defenses may be specific to the voting system,
    and may need to be assumed for the voting system to be secure. The
    rules of engagement would specify those defenses as an assumption,
    if the evaluators believed the assumption made sense.

Most of the rules of engagement should be determined by the standard.
All the rules of engagement must be included in the final public report,
so that anyone that buys or relies on the security of the voting system
can determine whether those assumed procedures are being followed or
not.

Sensible parameters for the rules of engagement will have to be
determined; this is a major technical issue.

1.  Feedback to the Vendor: The Confidential Evaluation Report

The result of the evaluation will be a confidential evaluation report,
which will specify either that the system passed (possibly with
outstanding minor issues that need to be resolved) or that it failed.

When a voting system fails, the VSTL should specify what would need to
be fixed in order for it to pass. The VSTL may specify some suggestions
about how to fix the problems, but it's important that the evaluators
don't get too involved in designing fixes, because they will presumably
be re-evaluating the system after the fixes are applied.

1.  Re-evaluation procedure

The main general policy issue here is what process a voting system with
a failed evaluation will go through to get fixed. In particular, we
assume there will be some standard procedure for fixing and resubmitting
a voting system to the VSTL. We need to determine some details of this,
including:

-   Will the reevaluation simply verify the fixes and retest those part,
    or will there be additional evaluation?

-   What disqualifies a voting system for going through the reevaluation
    process? That is, are there systems so flawed that they just need to
    be redesigned and evaluated again from scratch?

-   What limits are there on changes that may be made to address the
    fixes? It's clear that if the voting system is radically changed, a
    new evaluation needs to be done. But what are the limits on
    peripheral changes before resubmission?

    1.  Stopping and Starting the Evaluation

These evaluations are likely to be expensive. A second policy issue is
whether there's a point where the VSTL should stop the evaluation, and
wait for some major problem to be corrected before continuing. The most
likely ways for this to happen would be :

-   If some very fundamental problem were found with the system design,
    it might be wiser to simply stop the evaluation until that problem
    was fixed.

-   If some very easily fixed problems were found (such as many
    out-of-date and known-flawed versions of COTS software), it might
    make sense to let the vendor fix these, and then continue the
    evaluation.

-   If the security documentation is fundamentally flawed (in the sense
    that the security arguments given don't stand up), then it may make
    sense to stop the evaluation until those problems are resolved.

    1.  Deciding Pass/Fail

The decision of whether a voting system passes or fails evaluation is a
core part of making the evaluation meaningful---if the evaluation team
discovers major problems, but the system still passes, the open-ended
evaluation was probably a waste of money.

The policy decisions here resolve around three questions:

-   How should the final result of pass or fail be resolved?

-   Should there be an appeals process for disputed evaluations?

-   Are there ever cases where a passed evaluation should be revoked?

    1.  How is Pass/Fail Decided?

We expect that the results of evaluations will fall into three
categories:

-   Clearly passing voting systems

-   Clearly failing voting systems

-   Voting systems whose result is subject to some debate

Our current notion is that if any of the intermediate attack goals are
successfully reached, the system should fail evaluation unambiguously.
The evaluation team may wish to fail the system over bad arguments in
the security documentation or bad procedures, but this is likely to be
less clear cut.

A related policy decision involves whether the evaluation team or some
other body, perhaps appointed by the EAC, decide whether to pass or fail
a voting system. The main advantage of having the evaluation team or
VSTL make the decision is that they are closest to the facts, having
actually evaluated the voting system. The advantage of an outside body
making the decision is that this body would be further away from any
business relationship between the VSTL and vendor. However, it's not
clear how much this would change the result of the evaluation, since the
outside body would be basing their decision entirely on what the VSTL
reported.

1.  Appealing a Decision

Sometimes, the vendor is likely to disagree with the evaluator's
decision. More commonly, there may be some ambiguity in the standard
that requires an external decision about whether something is or is not
ground for failure.

The best answer we can see for this is some process to appeal decisions
to a body appointed to the EAC.

Also, some body appointed by the EAC needs to keep track of precedent in
decisions about how to interpret the standard, so that the same rules
are applied consistently. Something like this is done in common criteria
evaluations, and the interpretations are eventually used to modify the
standard.

1.  Public Disclosure: The Final Public Report

The final public report will be provided to purchases and prospective
purchasers of the voting system, and should be available to the public.
The goal of the public report is to tell people who rely on this voting
system what kind of evaluation was done on it, and most importantly, to
list exactly what assumptions were made in analyzing its security.

The interesting technical and policy issues here involve three
questions:

-   Should the report be fully public, or should it be restricted to
    only election officials considering buying the system?

-   What information should be included in the final public report?

-   How should we resolve disputes about the contents of the final
    report?

    1.  Fully Public?

There are two reasons to make the report public:

-   The report will specify the conditions under which the system was
    evaluated. Making these public will allow the public to convince
    themselves that the voting system is being used in a way that's
    consistent with the evaluation and the security analysis.

-   All the details in this report must be made available to election
    officials considering purchasing the voting system. This means that
    it already cannot include information that would undermine the
    voting system's security. (If the evaluation discovered information
    that would allow a practical attack on the voting system, the system
    should not have passed evaluation.)

    1.  Included Information

What information should be included in this final report? We think the
most important elements include:

-   The security documentation from the election official, specifying
    procedures and a security argument that was checked during
    evaluation of the system.

-   The rules of engagement, specifying what limitations were assumed on
    the attacker.

-   A list of specific security controls or intermediate attack goals
    checked, along with at least a brief summary of how they were
    checked. For example, the report might include many entries along
    the lines of the following:

> *Intermediate Attack Goal: Tamper with electronic records in transit*
>
> *Security Control: TLS cryptographic protocol between voting machine
> and tabulation center server*
>
> *Evaluation:*

-   *We verified use of FIPS-approved TLS implementation in software,
    restricted to only FIPS algorithms, by inspection.*

-   *We intercepted the encrypted communications and verified that they
    were in cipher text, and that tampering with them caused the
    communications to shut down and indicate an error.*

```{=html}
<!-- -->
```
-   A list of concerns that haven't been resolved, but which didn't
    cause the system to fail evaluation.

-   The dates when the evaluation was started and completed.

The first two of these are critical for both buyers and people trusting
the results of the voting system to see. Since it's not possible to
evaluate a voting system without knowing something about the procedures
that will surround it, specifying the procedures that were assumed is of
great value. Further, knowing what assumptions were made during analysis
allows any member of the public to convince himself that the voting
system is now being used in a way consistent with those assumptions.

The rules of engagement also allow members of the public to evaluate how
comprehensive the evaluation really was. An evaluation that assumed no
insiders were corrupt, even the volunteer poll workers who showed up on
Election Day, would inspire less confidence than one which allowed the
evaluators to try to attack the system while playing various
partially-trusted roles within that system. This might also allow a way
to slowly phase in use of these evaluations---early evaluations might
have less demanding rules of engagement. Or the different kinds of rules
of engagement might be used to certify voting systems to different
levels of security, much as FIPS-140 module validation is done at
different levels of security.

The list of security controls or attack goals considered lets members of
the public, and the potential buyer of the voting system, understand how
this system was evaluated. A long list of examined attack goals inspires
some confidence, though it's not possible to determine how talented or
diligent the evaluation team was.

The list of issues may be useful for someone considering buying the
system. For example, if the evaluators point out that the procedures are
possible to use, but cumbersome and difficult to do correctly, this
might be very valuable to a prospective buyer.

The dates for the evaluation simply allow a potential buyer to verify
that the testing has been done recently. Some form of renewal of the
evaluation would make sense, with evaluators going back and at least
checking for known-weak versions of COTS software and applying other
more up-to-date vulnerability scanning tools.

1.  Disputes

It's possible that the vendor will dispute some of the contents of the
report, though this seems like a minor concern. It probably makes sense
to have the same set of dispute resolution procedures available for this
as for the pass/fail decision.

4.  Open Technical Issues

    1.  Overview

The most important technical issues revolve around the novelty of this
proposed evaluation scheme. While there are many cases of this kind of
evaluation being done informally, we don't have a good example of a
functioning, real-world testing regime that works this way on a large
scale. This inclines us toward a desire to focus our testing on concrete
intermediate attack goals, and also toward a desire to use open-ended
evaluation as an addition to the more traditional testing, rather than
as the main device for testing the security of a voting system.

2.  Is This the Right Approach to Open-Ended Testing?

The approach we have described above is new; while it is derived from
existing systems, we cannot point to similar programs for operational
experience. This raises some real concerns that the general approach of
using open-ended evaluation as a major part of evaluating a voting
system may be very difficult to carry out. Specifically, we might ask
two questions:

-   Will evaluating these systems in the way we're describing simply be
    too hard for the VSTLs? That is, will the budget we can expect for
    this kind of evaluation support so little time and money for finding
    vulnerabilities that only the most obvious of vulnerabilities is
    ever discovered?

-   Will designing a voting system to resist attack even at the limited
    level possible prove too difficult or expensive for voting system
    vendors? (This ultimately would result in high prices for voting
    systems, as purchased by counties and states.)

    1.  Starting Slowly

The best way we can see to get open-ended testing working in a
meaningful way is to start slowly: To start by making most of the focus
of VSTL testing the checklist-style testing with which we have extensive
operational experience. Open-ended evaluation should be focused on the
parts of the system which can't be tested in other ways, and should
start up by focusing mainly on kinds of open-ended testing that can be
automated, and that are already capable of being done by the VSTLs.

At first, we should emphasize the parts of the open-ended analysis which
cover parts of the voting system which are least susceptible to being
checked using more traditional testing approaches. In particular:

-   Reviewing and evaluating the security documentation from the vendor
    will both force the vendor to think through how that document may be
    written, and give the evaluator a chance to see whether there are
    questionable assumptions being made.

-   Evaluating the procedures specified by the vendor is extremely
    important. Both the plausibility of these procedures and their
    effectiveness in accomplishing some security goal are very valuable.

-   In conjunction with the above two steps, some level of analysis of
    whether there are basic design flaws in the voting system can be
    done.

-   Direct physical inspection of the voting machine for exposed ports
    or related problems requires no special expertise, and can at least
    catch gross design flaws.

> We should similarly focus our initial open-ended testing efforts on
> the use of tools to search for possible vulnerabilities, as this is
> clearly possible, and doesn't require enormous expertise or resources
> to do well. This includes:

-   Automated scans of the source code for common patterns of software
    vulnerability.

-   Root kit detection tools to verify that nothing is hidden in the
    installation.

-   Scans of COTS software versions within the installation for
    known-weak mechanisms.

-   Vulnerability scanning of any machines which connect to a network.

Finally, we should list the intermediate attack goals from the standard,
even if the open-ended evaluation team will not always have the budget
or expertise to try to reach most of them. Even if the evaluators have
time only to check the two or three of these which seem most likely to
yield an attack, this will ensure that at least obvious vulnerabilities
are ultimately closed, and that voting system vendors have a strong
incentive to look for such vulnerabilities themselves as carefully as
possible, to avoid delay and extra expense that would be caused by
failing their evaluation.

1.  Not Setting the Bar Too High

The rules of engagement determine the starting assumptions for the
evaluation team. This includes what kind of access the evaluators are
allowed in evaluating different parts of the system. For example, when
attempting to attack a voting machine over a network, is the evaluator
allowed to communicate with the voting machine from another trusted
machine, such as another voting machine in the same polling place or the
server at the tabulation center? The choice of rules of engagement can
be used to avoid setting the bar for passing so high that no real-world
system can survive it. However, we urge caution in doing this, because
it would be easy to specify rules of engagement so restrictive that the
result of the evaluation was meaningless.

1.  How can intermediate attack goals be decided?

Intermediate attack goals are a major part of our approach to open-ended
evaluation. An important technical question is how we should determine
these goals. There appear to be two strategies:

-   Specify a small set of high level attack goals, which demonstrate
    components of attacks on voting systems, e.g., from the NIST
    "Threats to Voting" workshop. The evaluation team should then spend
    significant time and effort on each.

    -   Example Goal: Acting in any role that is not authorized to
        install new software on the voting system, install any new piece
        of software on the voting system.

-   Specify a large set of attack goals, most of which test the
    effectiveness of some specific security control. In this case, the
    evaluation only examines the attack goals that look the most likely
    to be vulnerable.

    -   Example Goal: Bypass the patch signing mechanism without
        knowledge of the signing key, and without tampering with
        software on the voting machine in order to get the attack to
        work.

Note that the first example goal is high-level, and relates to a
high-level security requirement (don't let the attacker install programs
on your voting machine). The second example is much more narrowly
focused at a security control (defeating code signing).

We are inclined toward the first strategy, because this allows a lot of
flexibility for the team. However, this also makes the evaluation less
consistently repeatable, since different evaluators will try to
accomplish these goals in different ways.

1.  How can we ensure that the open-ended evaluations are done properly?

These evaluations, even if somewhat limited in scope, will be consuming
a lot of time and money. We need to work out how to ensure that they're
being done competently. This resolves into three questions:

-   How can the EAC or whatever organization certifies VSTLs ensure that
    the VSTLs are capable of doing this kind of evaluation?

-   How can the EAC verify that the evaluations are really being done in
    a serious, meaningful way?

-   How can the public verify that the evaluations are being done in a
    serious and meaningful way?

    1.  Certifying VSTLs for Open-Ended Evaluation

How can the EAC verify that the VSTL has the resources to do this? We're
not sure yet---this is a hard problem. Some list of qualifications and
tools used can help, but this is still an open issue.

Perhaps the best answer comes from competence testing of VSTLs for
open-ended testing. Further work will be needed to see if this is
sufficient.

2.  Verifying Reports

Another major question is how the EAC can review reports to make sure
the analysis is being competently done. This is a broad general problem
in testing schemes, and maybe some people in NAVLAP will have some
insights. Open-ended testing results are harder to check for correctness
than other kinds of testing results.

5.  Conclusions

In this white paper, we have discussed the need for some level of
open-ended evaluation of voting system security, and outlined an
approach for doing that evaluation. We have also raised a number of
policy and technical issues that need to be addressed over the next few
years, in order to begin carrying out meaningful open-ended evaluation
as a component of the larger voting-system testing regime. Some of these
issues can be resolved only after we have gained some operational
experience doing open-ended evaluation on voting systems.

6.  Acknowledgements

The author would like to thank Alicia Clay, Barbara Guttman, Nelson
Hastings, Rick Kuhn, Ron Rivest, Adam Shostack, and Murugiah Souppaya
for useful discussions about issues surrounding open-ended testing of
voting systems.

7.  References

-   NIST SP 800-53 revision 1, "Recommended Security Controls for
    Federal Information Systems, Draft March 2006" , available at
    http://csrc.nist.gov

-   RABA Trusted Agent Report, available at
    http://www.raba.com/press/TA_Report_AccuVote.pdf

-   SAIC "Risk Assessment Report, Diebold Accuvote--TS Voting System and
    Processes," September 2003, available at http://www.dbm.maryland.gov

-   Kohno, Stubblefield, Rubin, Wallach, "Analysis of an Electronic
    Voting System," IEEE Symposium on Security and Privacy 2004,
    available at http:// avirubin.com/vote.pdf

-   "Preliminary Analysis of Threats to Voting Systems," John Kelsey,
    NIST, March 2005, available at
    [http://vote.nist.gov](http://vote.nist.gov/)

-   "Voting System Testing and Evaluation Addressing Resolution 17-05,"
    John Kelsey, NIST, March 2005, available at
    [http://vote.nist.gov](http://vote.nist.gov/)

[^1]: *Security controls* are defined in NIST SP800-53 as the
    management, operational, and technical safeguards or countermeasures
    prescribed for an information system to protect the confidentiality,
    integrity, and availability of the system and its information.

[^2]: More research on technical issues will be needed in order to
    produce a complete list of properties and resources specified in the
    rules of engagement, and this list will likely evolve as labs and
    the EAC gain operational experience with OEVT.