ES&S Comments on the Preliminary Analysis for Draft Recommendations for
Setup Validation, Draft Version March 2, 2005

  ------------------- ---------------------------------------------------
  **Reference**       **Comment**

  General comment     We would encourage NIST to more fully understand
                      the current processes used under VSS2002 by
                      vendors, ITAs, elections administrators,
                      jurisdictions, and the EAC. Once more fully
                      understood, we would suggest that NIST determine
                      the gaps in the current process that need to be
                      addressed, and recommend improvements in an
                      evolutionary fashion.

  Systems Validation  We suggest that NIST learn about current Source
                      Review and Logic and Accuracy processes that are
                      already in place and determine what low tech
                      options or alternatives might be fully sufficient
                      vs. prescribing wholly new processes.

  Elections results   Also, we would suggest that NIST review how
  validation          elections systems currently support dual path
                      tabulation to provide checks and balances to voting
                      processing throughout the election events
                      themselves.

  Today's reality and Many of the recommendations currently being
  a transition        proposed by NIST in this area could apply to newly
  requirement         designed systems but cannot apply to previously
                      designed and fielded systems. Many of today's
                      current voting systems are fully secure, including
                      an ability to audit the setup, installation, and
                      configuration, but were built in a manner not
                      conducive to the validation techniques now being
                      suggested by NIST. New standards and guidelines
                      should not disqualify these systems or eliminate
                      such systems from usage. We suggest that NIST look
                      to understand the current voting machines and
                      systems in the field and their need to remain in
                      the field because of customers' economic needs. We
                      further suggest that, based on acquired knowledge
                      of currently fielded voting systems technology,
                      NIST define secure, adequate and confident
                      alternatives for setup and validation, that can be
                      supported by the current generation of voting
                      systems in the field.

  Articulate          Suggest elimination of prescriptive solutions
  performance         within these standards and replace them with
  requirements,       performance requirements (e.g., specifying
  rather than         distribution media properties is prescriptive; it
  prescriptive        is a solution to a requirement to mitigate ability
  solutions           to alter programs through some combination of
                      distribution and validation processes, depending on
                      the technologies being used and who has
                      responsibilities for coding, distribution and
                      validation).

  Use and extent of   We assume the requirement for "write-once" media
  the "write-once"    pertains to distribution between vendor and
  media requirement.  customer and not customer to machine. This is an
                      important clarification point, because many of the
                      current voting systems in the field do not have
                      such media reading capabilities. Please clarify the
                      purpose and specific use intended for the
                      "write-once" media.

  FIPS standard for   Many of the machines currently in the field could
  cryptography        not adequately support this type of processing.
                      Consideration for these systems must be made and
                      alternatives defined which would provide the
                      necessary level of protection and security, and
                      which could be adequately supported on currently
                      and recently fielded systems.
  ------------------- ---------------------------------------------------

#  The following pages discuss the topic of Setup Validation and Software Distribution in greater detail, including examples of real world circumstances, issues, constraints and challenges. Specific examples and situations are discussed to help illustrate the challenges and opportunities. 

# Background

Setup Validation:

1.  Intended to provide setup validation method on all aspects of the
    Voting System (incl. EMS), not just poll place machines.

2.  Installation and initialization processes are out of scope.

3.  Primary focus on requirements using computing capability of system
    to validate.

4.  Secondary focus on inspection techniques and lists to determine
    proper initial state.

Software Distribution:

1.  Intended to focus on all software and configuration files that can
    exist on a voting machine -- includes software developed, third
    party (e.g., drivers) and COTS, and election data. Intended to
    include all aspects of voting system (EMS, EMR, tabulators, etc.).

2.  Out of scope items include validating the software performance,
    validating any platform performance outside the software and
    configuration files installed; loading is determined to be storage
    in non-volatile memory only (not load to RAM).

3.  Distribution of "golden" copies will be done through "write once"
    media.

# Comments

It would be helpful for NIST to more fully understand the current
processes used under VSS2002 by Vendors, Independent Testing Authorities
(ITAs) and the ECA; and then to determine what gaps exist in the current
processes that need to be addressed in an evolutionary fashion. For
example, items 1-3 in the Setup Validation recommendations we think are
adequately addressed through the submission of a Technical Data Package
(TDP) and the Software Inspection Process. The TDP includes various
documents inventorying all modules, libraries and software entities and
their purposes; how items are called (function/module call chains).
Addressing many of the items in the recommendations list, the TDP also
contains product maintenance instructions and operational user guides.
Within our documentation are also security guidelines and suggestions
for jurisdictions who must maintain our software on their networks and
setup secure poll places. We would like to see NIST work with the ITAs
and vendors to determine what gaps exist in the current documentation
and process and provide recommendations on how that could be enhanced.

The second area we would like to review in more detail with NIST is the
source review process currently conducted by the ITAs and the chain of
evidence processes used in an ITA testing and evaluation session as
prescribed by VSS2002. We are required in our source standards to limit
our usage of global variables, properly initialize all variables,
document within the code all functions, initializations of variables and
structure the code so it can be easily followed. The ITAs then inspect
all of the code submitted (in the case of a new product) or differences
in code submitted (for a previously approved and baselined product) for
compliance to standards, as well as Trojan Horses, malicious code, etc.
They use a combination of visual inspection and tools to accomplish this
task and it is the single longest event in the certification process.
Once this code is iteratively modified and subsequently approved, the
ITA brings that code base in their possession into the certification
event. Vendors are then required to construct their applications in the
presence of the ITAs (from witness compiles of modules to construction
of complete application installations), based on the TDP documentation
and inventories. This code is then the code used to perform the
functional tests and when approved becomes the baseline that can be
identified as approved. This chain of evidence allows a high degree of
confidence that the inspected code is what is included and validated
through the certification process.

During inspection, it could be proven with reasonable certainty that
proper hash , checksum or other file validation calls were being
performed properly within the code without modification. It can also
then be tested during the certification event that these techniques
return correct results. Once the software package(s) were tested and
validated and ready for approval, it would be possible for the ITA to
insert an additional file into the installation as documentation of the
versions and file hashes of all executables, and the certifications of
the release would then include a newly calculated hash for the
installation package by NIST, which would preclude programmatic
manipulation. It would then be possible to treat the validation process
as a programmatic exercise within the product and the installation
medium without external programmatic devices. This is an important
alternative to the NIST proposal because it allows a customer to seal
their machines and prevent access to OS level functions by third party
programs. Under the NIST proposal, item 13, where external programs are
allowed to run against pc's or tabulators to determine their "state" -
what is to prevent them from performing any modifications to the
machines they're supposed to protect?

The NIST procedures also ignore the Logic and Accuracy Testing process
that can be used to validate election data, machine configuration
options, tabulation processes and the reliability of the equipment (wear
and tear issues, etc.). During the election coding process, customers
manually validate all ballots to insure that they represent the election
data pertinent to that election. Configuration values can be inspected
in the coding, tabulation and reporting systems visually and through
system supplied reports that have been tested and validated by the ITAs.
Random voting or pattern voting can be conducted through pre-marked
paper test decks and DRE routines (validated by ITAs) to derive a
predetermined tabulation answer and then run through and collected by
the tabulation and reporting equipment to insure the same results. Once
validated, this equipment can be cleared of results (validated on
election day by printing zero tapes at poll open) and sealed and stored
ready for use. Broken seals would automatically retire a unit from an
election unless more testing and validation were performed. This same
process can be repeated post election in it's entirety or through
quality sampling techniques in some areas to revalidate the machines and
processes.

There is also the notion within an election event of dual paths of data
for unofficial (election night) and official (final count) results.
Transmitted or collected and delivered results for central reporting can
be validated against tabulated reports calculated at the precincts
(validated by ballots cast against the poll log) to determine that the
results processed agree. Election data collected can be tagged with
version information and hash or checksum validations. Processing can be
done through two different sources independently (the tabulators in the
precincts, the accumulation software in the central site) to prove that
the totals agree. This also confirms proper processing results.

Some of the procedures specified in the NIST recommendations rely on the
ability to run independent programs against the machines in the field
where firmware is installed. Some of these devices do not come with an
OS (all firmware including OS and drivers were custom developed) which
would make running another independent application problematic.
Secondly, opening a machine and running programs in any sort of an
intrusive fashion allows more opportunities for Trojan Horses and other
configurable integrity calls. Testing them out during the certification
event and then building a unique file into the final certifiable install
to guarantee a unique integrity value outside the control of the vendor
and would in our opinion be a preferable approach. We support tracking
file integrity for both the installation medium prior to the install and
for the installed components after the installation on the machine. And,
we support application modifications to allow printing of full inventory
and disclosure of all configuration options and files and integrity
checks by customers at their request that could be tested and validated
in the certification process.

Many of the recommendations currently being proposed by NIST could apply
to newly designed systems, but would have applicability problems with
many machines currently in the field that customers might not have been
anticipating having to upgrade (or do not have enough money to upgrade).
It is our position that NIST should spend more time understanding the
machines that currently exist in the field and what customers intentions
and finances are with regard to equipment upgrades (e.g., has money for
disability access machines but must keep current tabulators for several
years yet). Rather than stand behind the position that these are
voluntary guidelines and possibly created an unfunded mandate, it would
then be easier to adapt the guidelines to talk about requirements that
distinguish between the development of new systems (go forward
standards) and those that can be easily applied to all equipment in the
field. The best analogy to this would be adopting new car safety and
emissions standards on new cars and not mandating retrofits for older
cars still in use. As vendors, we welcome opportunities for new
standards to drive new equipment development and purchases, but we
understand that in our customer base economic realities preclude all
customers from being able to pursue this path over the same period of
time.

Additionally, there are some points within the NIST proposal that are
not requirements but prescriptions. For example, configuration
information can be coded through the EMS and passed into the system, and
most tabulators also contain options within the equipment to change or
override configuration options. These options can change within the
course of the election if done with proper authorization, configuration
reports in and of themselves would be meaningless. However, machines can
undergo logic and accuracy testing both before and after an election to
validate if the machine was configured properly, and examinations of the
systems audit\\events logs (validated during certification testing)
would determine whether any change had been completed before that time.
So testing and procedures and log files can be an additional
verification mechanism different from what NIST is proposing. The focus
should be that a validation mechanism is defined, evaluated and used to
mitigate a problem; not what it should be.

Finally, we understand the NIST software distribution requirement of
"write-only media" to be applied as being able to deliver a "golden"
copy (write protected) from Vendors to ITAs to NIST and these same kinds
of copies would be provided to jurisdictions. It would be possible to
perform this operation solely through products like CD-R, and then copy
these files into media that can be used and loaded onto tabulators by
jurisdictions. Current products in the field load firmware through
floppies, CD-rom, flash, programmable EPROM, PCMCIA and ZIP disks. Some
tabulators currently in the field load the firmware and election
directly into the tabulation device eprom together as part of the coding
and delivery functions, which a jurisdiction could currently choose to
outsource. We do not understand instead why audit capabilities on all
voting machines with the capability to validate executables back to
NIST/NSRL hashes would not be sufficient and credible. Prescriptively,
you also apply FIPS standards for cryptography to all voting devices --
when many of the machines currently in the field could not adequately
support this type of processing. Again, some level of distinction should
be made to go forward products and leave jurisdictions an opportunity to
use these evaluation criteria in their new systems purchases when they
have finances and opportunity.

# Summary

In Summary, we would ask NIST to :

1.  Review current VSS Standards and ITA procedures and focus changes on
    addressing gaps or improving aspects of the existing process;

2.  Understand more about Source Review and Logic and Accuracy processes
    and what low tech options might be alternatives to things being
    prescribed within these documents. Also, how elections systems can
    currently allow dual path tabulation to provide checks and balances
    to voting processing throughout the election events themselves;

3.  Look to understand the current machines in the field and their need
    to remain in the field because of customers' economic needs. Also,
    to distinguish based on the knowledge of these machines where new
    standards may not be applied

4.  Look to eliminate prescriptive solutions within these standards and
    replace them with requirements (e.g., distribution media properties
    is prescriptive; it is a solution to a requirement to mitigate
    ability to alter programs through some combination of distribution
    and validation processes, depending on the technologies being used
    and who has responsibilities for coding, distribution and
    validation).

We would welcome the opportunity to review our products and processes
with NIST personnel and our customers, and have a dialogue on how to
help create a successful transition plan to better standards for all
jurisdictional authorities tasked with making this happen. As vendors we
are delighted to make changes to improve and sell new equipment or
upgrade existing equipment as long as the market will support this.