MR. SALTMAN: Thank you, Mr. Rivest, Mr. Gannon, Mr. Green. Thank you for
inviting me. I appreciate your invitation. I am thankful to be on the
same panel with experts like Michael Shamos and Doug Jones.

I\'ll start off with a little biography which Dr. Rivest asked us to
produce. Since May, 1996, I have consulted individually on election
policy and technology for, among others, the International Foundation on
Election Systems and the American Civil Liberties Union. For the former,
I have consulted in a Brazil, Equador, Venezuela and Japan. For the
latter, I have served as an expert on the failings of pre-scored punch
card voting in California and Ohio. I was employed at NIST in this very
building for the last year. I was there as a computer scientist from
1969 until 1996. I was employed by the Sperry Rand Corporation from 1955
to 1964, and by IBM Corporation from \'64 to \'69. With both
organizations, I worked for the field of computer engineering.

I have two masters degrees; one from MIT in electrical engineering, and
the other from American University in public administration. I have two
additional degrees in engineering, one from Rensselaer Polytechnic
Institute and from Columbia University.

While employed at NIST, I authored two reports that are well known by
those concerned with the voting process. The first entitled, \"Effective
Use Of Computer Technology in Vote Tallying,\" was published in 1975.
The second entitled, \"Integrity, Accuracy and Security in Computerized
Vote Tallying,\" in 1988. At NIST, the author\'s name is printed on the
face of the publication, and anyone who has seen my reports knows that
this situation pertains to my reports.

The bottom line here is public confidence in announced results. A
national awakened occurred on November 7, 2000. Help America Vote Act
was enacted by the Congress and signed by the President in October,
2002. We hope the TGDC will develop recommendations that can be
implemented to improve election integrity and security. The bottom line
by which TGDC may be measured is the effect of its recommendations on
public confidence.

In 1975, I wrote, \"The assurance that steps are being taken to prevent
unauthorized computer program alteration or other computer-related
manipulation remains, nationwide, a continuing problem for the
maintenance of public confidence.\"

The issues of public confidence, concerns about elections have been
stolen goes back to the earliest days of the Republic. The three decades
after the Civil War, 1865 to 1895, were perhaps the heyday of election
fraud, involving the infamous Tweed Ring\'s New York frauds of 1868, the
stolen Presidential election of 1876, and the unsurpassed bribery and
intimidation of the Cleveland/Harris election of 1888. More recently,
the uncertainty and ambiguity surrounding the outcome of the 2000
Presidential Election in Florida has continued to resonate with many
citizens. Even though the concern now is placed on different subjects,
namely the correctness of software and the potential failure of
electronic equipment, rather than on hanging cad and inability of many
voters to correctly record their intentions, many voters have a
recurring nightmare that the turmoil generated in that election\'s
aftermath will happen again.

For example, I was told just last week by a professor at a Maryland
university that, quote, \"the real reason,\" unquote, that Maryland
State Board Of Elections wants to remove the State Director of Elections
is not just a difference in party affiliation, he said. It is so that
the current state director would not be able to manipulate the software
to benefit her party, and so that the new director would be able to
arbitrarily change the software to benefit the Board of Election\'s
majority.

This demonstrates that the TGDC will never be able to convince all the
people of the efficacy of what it will develop, but let us hope it will
be able to convince important molders of public opinion. These include
editorial writers of major newspapers who, after many years of neglect,
have suddenly discovered the issue of the voting process. It appears
that the issue of voting confidence now is closely tied to the issue of
software correctness.

New York Times, May 30, 2004, published an article, \"Who Tests Voting
Machines?\" The article makes no mention of NIST or the TGDC.
Nevertheless, the Times makes some good points and asks good questions.
The editorial quotes a well-known activist as stating, quote, \"The
standards do not require examination of any commercial, off-the-shelf
software used in voting machines, even though it can contain flaws that
put the integrity of the whole system in doubt.\"

I agree this is a problem. I wrote in 1975, quote, \"In order to
eliminate as many securities threats as possible, the least complex
operating system that provides the capabilities required by the vote
tallying program should be used to support the vote tallying process.\"

In my opinion, it is necessary to even test the entire software package
that exists in a voting machine that is used to record and count votes.
When I wrote the 1975 report, central processing of ballots was almost
universally carried out. That meant that the computers doing the vote
recording and counting were probably multi processors, and they have
been undertaking non-election processing at the same time.

Now, technology has changed. Much vote recording and counting has been
accomplished on precinct-located computers. Indeed, requirements in HAVA
imply that only precinct-located vote recording may be done in the
future to allow the voter the opportunity to correct overvotes. That
could not be accomplished with a central count process. When
precinct-counting is being done, the likelihood is very high that the
computer is dedicated to vote-recording and counting and is not
performing any other task.

Therefore, it makes no sense to use an operating system of such a
computer that is intended for multi-tasking operations for a human user.
Nevertheless, it appears that at least one election equipment
manufacturer has based its offerings on just such an operating system.
If software is based on an enormous operating system containing much
code that is unused for voting purposes, it cannot be effectively
tested. The size, that is, the number of lines of code, of the vote
tallying software presented by the vendor should be considered as a
parameter in the certification process. Either the cost of testing
should be increased accordingly to the vendor, or the product may be
summarily refused certification for being beyond a testable size or for
including extraneous functions.

Additionally, the certification process should be undertaken on the
software in its final object-code form, as well as in source-code form,
to include the effects of compiler software that may have contained
malicious code to be implemented in the vote recording and counting
software.

The New York Times article just quoted says, \"Government, not the
voting machine companies, must pay for the testing and oversee it,\"
unquote. Or quote, \"Voters should be told how testing is being done and
the testers\' qualifications,\" unquote. And final quote, \"rigorous
standards should spell out in detail how software and hardware are to be
tested, and fix deficiencies computer experts have found.\"

With regard to the first time, the Times is incorrect when it states
elsewhere in the editorial that, \"These labs are selected by the voting
machines companies, not the Government,\" unquote. However, the labs had
been selected by NASED, which is a private association of state election
officials. Thus, it had been reasonable for NASED, as well as the labs
themselves, to refuse to reveal the accreditation criteria and the
testing criteria. While I know of no law that would require Government
to pay for the testing process of vendors\' products, the remainder of
the Times\' recommendation are in agreement with HAVA.

Under HAVA Section 231, \"the Election Assistance Commission shall
provide for the testing, certification, decertification and
recertification of voting system hardware and software by accredited
laboratories.\" Further language of HAVA gives much responsibility to
NIST in this area. Thus, certification and follow-on work is to be done
by a public process, and TDGC must follow the law on revealing the
results of Government deliberations and decisions. TGDC needs to develop
laboratory accreditation criteria in detail and software testing methods
in detail, and should, upon their completion, submit them for public
review, and then, final publication as public documents.

There is practicality of testing for software correctness. It is often
stated by opponents of the use of DRE voting machines that it is
theoretically impossible to assure correctness of software. That is
true, but on the basis of theoretical impossibility, no bridge, dam, or
other large structure should ever be constructed. It can never be
assured that the maximum stress beyond which the structure will fail
will never occur. Engineering decisions on the strength and resilience
of a structure to be built must be a trade-off between cost and some
maximum expected stress.

There is a concept called due diligence, that is, that the latest
professional understanding of processes of construction and predictions
of stresses are to be utilized. Nevertheless, if there are several
levels of testing of voting quality software, we need high-quality
procedure that may cost a bit more but provides a higher level of
confidence.

The functions of vote-recording of counting are more like on-line,
real-time processes than they are Internet-connected multi-tasking
systems. We need to absolutely prevent external connections to voting
equipment that are unnecessary for the specific functions it performs.
This is necessary to minimizes complexity as well as to assure security.
We need to apply the type of testing procedures that are used by
airplane and automobile manufacturers for their respective vehicle
control systems. Planes and cars now contain programmed computer trips.
How do Boeing and Ford use them? We need to know. If we really believe
software can never be assured to be correct, we should never fly or
drive. One of the leading proponents of software correctness made that
statement at NIST last December after flying in from California. All of
our human brains are filled with these types of contradictions. We need
to do the best we can with the best professional quality we can find and
pay for.

The transition or precious or dangerous materials from the custody of
one organization to another is also the subject of administrative
procedures. Think of hospitals and controlled substances. Think of
nuclear materials and gold bars. Once testing has been completed on
administrative -- or voting software, administrative procedures must be
in place to assure the absence of tampering until the software is
actually used in the election for which it was designed.

Eric Fischer\'s statement that security involves people and procedures
and activities and equipment, it is not just equipment alone, it is an
extremely important point, and I once made a speech on that subject, and
I would be happy to give you a copy of it, if you would like to see it.

Cryptographic techniques may be used, as well as chain of custody
concepts. We must assure that the software that was tested for
correctness is the same software that is to be used. The necessary
procedures should be developed by the TDGC in cooperation with the
states and local governments that will be using the software, and will
be responsible for its protection.

In the voting process, it is difficult to separate security from other
aspects of integrity. For example, in 1975, I recommended that records
of overvotes and undervotes be retained and reported. I have been
informed recently there are still local jurisdictions in which that
procedure is not carried out. As in accounting, where double-entry
booking has been standard for about a century, there needs to be
response to total response to each category in voting. This cross
checking will produce a spreadsheet where the sum of the horizontal
summations will equal the sum of vertical summations. The contest total
number of ballots cast multiplied by the total number of legitimate
votes cast per ballot should equal the sum of votes assigned to each
candidate plus the number of overvotes, plus the number of undervotes.

In DRE systems, that process is possible with the use of electronic
ballots images now required to be implemented under the Federal
standards issued in 1990 and 2002. I first proposed to concept in DRE
systems in my 1988 report, referring to EBIs as voter choice sets.

There are other facets of assurance of public confidence that I might
discuss, but the number is too great for the presentation. For example,
in my 1975 report, Appendix B discusses the number of precincts needed
to be recounted based on the closeness of the contest. The adaptation of
this process to non-ballot systems requires discussion. In 20 addition,
the viability of different methods of supplying a voter-verified audit
trail needs evaluation. Recently, Richard Smolka\'s Election
Administrative Reports revealed that, in Nevada, with regard to the
printed, post voting receipt, quote, \"most voters ignored this
verification feature,\" unquote.

If voters are not checking the receipt for correctness, as would be
expected by human nature, then this receipt fails its purpose as an
audit trail. If the people aren\'t looking at the receipt, then what the
computer has produced in the receipt is just a computer printout of the
computer software. If the people, if the voters have not verified it,
then it is not an auditable document. Only a document that is reviewed
by the voter or filled out by the voter constitutes an auditable
document. Computer scientists, it seems, do not factor human nature into
their demands for their own solutions of system integrity.

And one other thing I would mention, that I disagree with the Craig
Burkhardt in his response on whether the same equipment should be used
in a recount. If the question in the recount is whether the software is
correct or not, clearly using the same computer with the same software
error will give the same answers.

So, if we\'re going to be assured that the software is correct and
that\'s an issue in a lawsuit, then other software needs to be used.

As a matter of fact, that came up \-- Paul Craft, if you remember, in
the suit against Palm Beach County back in the 1980s. All they could do
at that time, under their law, was run it through the same computer
again. Well, if the software was wrong, it would have given the same
wrong answer.

Thank you for your attention. That is what I have to say.

MR. CRAFT: Roy, one thing you said there that gave me the impression,
maybe I misinterpreted the way you said or maybe there is such out
there, are there DRE machines out there now that do not capture the
detail of candidate votes, overvotes, undervotes, and the ballot images
from each vote cast?

MR. SALTMAN: I don\'t know whether they are DRE machines, but some
machines, they don\'t record them. For example, some of the older,
full-faced DREs may not do that. I\'m sure that the EBIs and the newer
touch-convenient machines do it, and can record it. And I hope to serve
as a poll watcher this November, hoping to see whether this is done in
the county where I live, whether the EBIs are used, and to print out the
sum totals, or at least give the spread sheet that I talked about. I
think that\'s quite important for public confidence purposes, but I
think that in some of the older, full-faced DREs, that was not done.

MR. CRAFT: My concern, I haven\'t seen any machines that weren\'t
capable of that for probably 12 years. So I guess my concern is whether
there are machines out there in use that we need to be concerned about,
within our standards, that don\'t have that.

MR. SALTMAN: Well, that is a good question. I thought maybe the problem
that was talked about in Reba County in New Mexico was due to the fact
that there weren\'t EBIs recorded. If you recall, the issue was
something happened in their early voting in that county, and they did
not record the total votes for the major candidates in 2000. And I have
been trying to find out whether that machine does have EBIs or doesn\'t,
and whether they had been lost, as well as the total votes that were
cast. So I suspect that was the case, that the machine does not have it.
It might be an older machine.

The difference between lever machines and DREs is very interesting. In
lever machines, you weren\'t able to record EBIs. So, in lever machines,
you never really had real a audit trail at all. And it strikes me as
another one of these contradictions is that the New York Times has
suddenly, in 2002, discovered the need for paper audit trails. Yet, they
were surrounded in a city and state that used lever machines in New York
City since 1928. So there weren\'t any ballots in New York City or New
York state for over 75 years. Now, suddenly, in 2002, the New York Times
discovered this. Now, maybe all of their editorial writers live in New
Jersey. That is an interesting problem that they haven\'t addressed.

DR. RIVEST: Thanks. Other questions? Anybody on the phone?

MS. PURCELL: This is Helen Purcell. Again, on that topic, I\'m glad to
hear you make that comment because we talk about that quite often, how
the lever machines did not have any kinds of paper receipt. So we\'re
now talking about paper receipts. So I was glad to hear you make that
comment.

MR. SALTMAN: Thank you. DR. RIVEST: I wanted to follow-up on your offer
to give us copy of the speech that you gave on people and procedures.

MR. SALTMAN: yes.