**MEETING OF THE TECHNICAL GUIDELINES**

**DEVELOPMENT COMMITTEE**

**NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY**

**Wednesday, March 9, 2005**

(**START OF AUDIOTAPE**)

MALE SPEAKER 1: \-- the habit now of having these meetings on the
coldest day of the month so we decided not to break that and we'll try
to keep you warm inside with the topics that we cover today.

A couple of administrative things. First of all, welcome to the public
and not only here but on the Webcast. The Webcast will be archived and
can be accessed after the meeting as well. We would request that the
public here wear their name tags at all times for security reasons and
also so that we know who you are. If you would turn off or turn down
your cell phones that would be most appreciated as well. We have a tight
schedule so we will move right along during the morning and we have had
weather delays and actually postponements of capabilities for people to
be here because of the weather so, that can't be helped but we have a
number of members joining us on Tele-conference as well. So, we are
happy that that capability is available as well.

Does anyone, or if anyone, needs interpretation for the deaf our
interpreter is over here and you might want to move over to that part of
the Green Auditorium. Last time we had a little practice so you need to
really watch this. We are in the Green Auditorium and if you go to the
back doors, either one, you take a left and then a right and you get
right outside and you will know there's a fire drill here because the
Klieg lights start to go on and off and you will hear the sirens just
like in school so we are concerned for your safety but there are a
number of ways out of here. We have lunch at 12:30 today. That will be
in the main cafeteria which is just down the hall from where we are
here. There are several bathrooms around. There are several bathrooms
right here. In other words once you go out, take a left for the
handicapped and, as I said, if there is any other emergencies you will
hear the, the P.A. system will go on and we'll be advised on how to go
from there.

Briefly, pretty much what I have covered here on the fire, severe
weather and security emergencies and with that I think I'll say that I'm
here to assist you in any way during the conference, the public that's
here. There are handouts in the back for the public. All of that
information is on the web site as well. They are big green folders. Feel
free to take one with you. Power Point presentations will be on the web
as well early tomorrow morning. With that I would like to hand it over
to Dr. Smergian, the Chair of the Technical Guidelines Development
Committee.

DR. SMERGIAN: Thank you, Alan. Good morning everybody. I am Rach
Smergian. I am the acting director of NIST and I would like to welcome
all of you to NIST, to this meeting of the Technical Guidelines
Development Committee. I hereby call to order the third meeting of this
committee today, Wednesday, March 9, 2005. Let us now stand and pledge
allegiance.

GROUP: I pledge allegiance to the flag of the United States of American
and to the Republic for which it stands, one Nation, under God,
indivisible with liberty and justice for all.

DR. SMERGIAN: We have currently nine members here. We also have Dr.
Revest and Ms. Turner-Bowie on the phone with us. We are still expecting
a couple more people to join us. Given the weather conditions I am sure
some of the flights are delayed and two other members, Mr. Harding and
Mr. (undecipherable) will join us in the afternoon. They are actually
attending another meeting this morning so they will be joining us for
the afternoon session.

At this time, oh, I would also like to acknowledge Commissioner Martinez
who is here with us and sitting back there. He wants to make the point
that he is an observer. I believe that Commissioner DeGregorio just came
in. So, we are very pleased to have them with us at this meeting.

With that I now recognize Mr. Craig Burkhart as the TDGC Parliamentarian
and request that he determine if a quorum of the committee is present.
Mr. Burkhart.

MR. BURKHART: I call the roll. Williams. Williams is here. Burger.
Burger. Not here. Caldess. Caldess is here. Kraft. Kraft is here.
Davidson. Davidson is here. Eleckies. Not here. Gannon. Gannon is here.
Harding. Not here. Miller. Miller is here. Crystell. Crystell is here.
Quisenberry. Quisenberry is here. Revest. Revest is here. Schutzer.
Schutzer is here. Turner-Bowie. Turner-Bowie is here. Smergian. Smergian
is here.

Mr. Chair a quorum is present. You may declare a quorum is present.

DR. SMERGIAN: Thank you Mr. Burkhart. I also thank the members of this
committee for arranging your busy schedules to allow your participation
at this meeting. As I have said in the past and it certainly bears
repeating, your willingness to volunteer significant time to the work of
this committee is a mark of the highest ideals of citizenship and civic
responsibility. I am sure every American voter will benefit from your
commitment. At this time I will entertain a motion to adopt the March 9,
2005 meeting agenda for the Technical Guidelines Development Committee
which is in your binder with the tab "Agenda". Do I hear a motion?

FEMALE SPEAKER 1: (undecipherable).

DR. SMERGIAN: Do we have a second?

FEMALE SPEAKER 2: Second.

DR. SMERGIAN: Any questions or discussion? Hearing none, I would move
for a vote.

MR. BURKHART: Take a voice vote.

DR. SMERGIAN: I guess we'll take a voice vote on that. All in favor say
"aye". Any opposed? The agenda has been adopted.

At this time I will entertain a motion to accept the minutes of the
January 18 and 19, 2005 meeting of the Technical Guidelines Development
Committee. Do I hear a motion to accept the minutes of the last meeting?

MALE SPEAKER 2: I so move.

DR. SMERGIAN: We have a motion. Do I have a second? Any questions or
discussion? Having none, all those in favor say "aye". Any opposed?
Thank you. The minutes, as presented, have been accepted.

Having seen the material that was sent to you earlier I hope you agree
with me that these scientists have made significant progress. Mr. Kraft.

MR. KRAFT: Yes sir. We seem to have a recurring problem in seeing the
materials for the meeting. I received a blank C.D. from Mr. Ustus and
actually got my hands on the materials Monday night and spent the time
that I had blocked on my schedule to review the materials in a very
frustrating exercise downloading materials off the NIST web site, some
of which were in a PDF format that made it difficult to save them. I'm
beginning to feel like I'm trying to teach the cat to fetch. I'm not
going to object to the materials in this particular meeting in the
interest of getting the work done but we absolutely must have these
materials in hand five days before the meeting so we can review them. We
passed a Resolution on that last time and NIST needs to work on its
delivering.

DR. SMERGIAN: I believe the material was sent on Tuesday or Wednesday?
It was sent by Federal Express on Wednesday so that you all should have
gotten it Thursday. Is that correct?

MR. KRAFT: The material that I got was a blank C.D. I received an e-mail
Monday from you, Allan, after --

ALLAN: There was another one (undecipherable).

MR. KRAFT: I didn't get that one. My administrative assistant did a
search for any e-mails from you this morning.

DR. SMERGIAN: I believe we tried three different drafts by e-mail, by
hard copy through FedEx and by, which also included a C.D. Certainly our
intention was and is to fully comply with the Resolution. We appreciate
the fact that there is voluminous material and that people need to have
sufficient time to review them. On the other hand as you all know, since
you are all involved there is work being done until the very last minute
so its sort of a balance but we certainly have tried to meet the spirit
of the Resolution that was passed last time and to avoid any of these
communication problems we tried to do multiple, you know, parallel
processing, try to use as many of the channels available to us for
communicating the material to you. I assure you that we will try to do
better.

MR. KRAFT: Okay. I appreciate it.

DR. SMERGIAN: Thank you, Mr. Kraft.

As I was saying the scientists have made significant progress on
technical tasks defined in the thirty-one Resolutions adopted by this
committee at the January plenary meeting. The TDGC members will have an
opportunity today to provide further guidance to NIST on these tasks. In
addition, two new Resolutions have been submitted for consideration.

Proposed Resolutions number 36-05 and number 37-05 as well as
preliminary task reports were sent to the committee on March 2 in
accordance with the advanced notice required in Resolution number 1-05.
Obviously we did have some problems but certainly we tried to meet the
spirit of that Resolution. In addition this material has been posted on
the public web site "NIST.gov". As a brief review for the public in
attendance and hearing the web site, Webcast, Public Law 107-252, the
Help America Vote Act also know as HAVA, establishes the Technical
Guidelines Development Committee. HAVA charters the members of this
committee to assist the Election Assistance Commission with the
development of voluntary voting system guidelines. This committee's
initial set of recommendations for these guidelines are due to the
Executive Director of the Election Assistance Commission in accordance
with HAVA's nine month deadline. In the interim the 2002 voting system
standards adopted by the Federal Election Commission serve as the first
set of voluntary voting system guidelines under HAVA.

At this time I note that the latest revised version of the Roberts Rules
of Order was adopted on July 9, 2004 to govern Technical Guidelines
Development Committee and subcommittee proceedings. I call on Mr.
Burkhart to review the logistics of this third meeting of the TGDC. Mr.
Burkhart.

MR. BURKHART: Good morning. I'm Craig Burkhart and I'm the Chief Legal
Counsel for Technology at the Department of Commerce. I welcome you on
behalf of the Secretary of Commerce Gutierrez as well as President Bush
who again are so thankful to the TGDC members for being willing to spend
you very valuable volunteer time to serve on this committee.

I'll very briefly review the strategy that we will be following today to
see that we make as much progress as possible according to the TGDC's
agreement. We are doing three things today. There will be a brief review
of the prioritization of the work projects in response to one of your
Resolutions last meeting. The second thing is we will review the work
product which will be given to you, which was given to you. And then
finally there will be a couple of new Resolutions toward the end of the
meeting.

The goal, the most important goal is to review this work product which
has been given to you so that the NIST staff can continue to fine tune
and develop that into a finalized work product which you will be capable
of adopting during your April meeting. Probably one key concept to keep
in mind is that during this meeting we are not adopting or approving any
of this work product as part of your initial recommendations to the
Election Assistance Commission. We are just simply making a check on
whether or not the work product to date is following the spirit and
letter of the Resolutions which you have previously adopted. You will,
in any case, be able to take a look at the finalized work product which
you will get during your April meeting and that will be the critical
time when you will actually take vote to approve work product to
recommend to the Election Assistance Commission.

MALE SPEAKER 3: So, if I understood that the point of today's meeting is
primarily to give the NIST staff the committee's feedback on the work
product they've presented.

MR. BURKHART: That's correct. Here is the explanation for what we'll be
doing for the large bulk of this meeting. You, of course, have received
written work product and then you will be receiving brief presentations
by the senior NIST staff person who is charged with developing that
particular segment of the work product. The combination of that oral
presentation and the written work product we're calling that the
preliminary report to you on that particular subject matter. After the
live presentation there will be a question and answer session during
which you will be able to dialogue with the NIST staff person. After
that, among yourselves as a TGDC you can have a discussion as to whether
or not this suits you well, the way the project is being developed or
whether there are supplemental instructions that you would like to give
or even corrections if there is something about the development of the
work project which you disagree with, that you don't think meets the
letter of the Resolutions that you have previously passed, you want to
bring that up at that time.

The intent is that if the group is in agreement as to some supplemental
instructions or corrections that if it's agreed there can be unanimous
consent for some type of supplemental directions being given to the NIST
staff person. In fact, at the end of each presentation the chair will
read a brief statement saying we believe this is in response to the
certain Resolutions you have passed. Are there any corrections and that
will begin the discussion. If there are no corrections or supplemental
instructions, no vote whatsoever is taken and the NIST staff will
continue along the line that they have developed. If, however, there are
supplemental instructions, if it's by consensus that will be what's
called unanimous consent. It will have the same effect as though you had
taken a contested vote.

Finally, there may be some contested issues where there is not unanimity
of opinion on something and then a vote can actually be taken in the
normal manner where somebody moves a particular correction or a
particular supplemental instruction and it can be debated in the normal
manner.

So, that's what we will try to do today. You might say this is your
speak now or hold your peace. Make sure that you believe that the NIST
staff is following the directions that you have previously given.

Finally, of course, what you will get in April will be the finalized
draft work products. And you will remember that you will get two. One
will be, if you will the addendum to the 2002 standards and the second
product will be those things which are not currently embraced in the
2002 standards and are more the long term type projects. So, you will
actually get two work products which will be based upon the projects
that you are going to hear about today.

And then, finally, there will be a couple of new Resolutions at the end
which will be handled in the same manner that we have worked the last
meeting. That would be all Mr. Chair.

DR. SMERGIAN: Thank you Mr. Burkhart. The scientists and I look forward
to the committee's comments and questions on the preliminary reports
presented today and we have an ambitious agenda today as we did at your
January meeting so we will try to move as expeditiously as possible.
Last time we went through the introductions of the committee. We'll save
some time and not repeat that however we are very pleased that Secretary
Davidson joined us who was not with us at the last meeting. I'll ask her
if she would introduce herself and make a few comments. Secretary
Davidson.

SECRETARY DAVIDSON: Thank you very much. Well, as stated my name is
Donnetta Davidson and I'm the Secretary of State in the State of
Colorado. I'm very honored to be on this committee and I think that we
do have a lot of work. A lot of work has been done but there's a lot of
things that need to take place. I won't take anymore of your time so we
can move on with the important agenda today. Thank you.

DR. SMERGIAN: Thank you very much. So, as Mr. Burkhart pointed out today
as a committee we will review and, where appropriate, provide
supplemental direction to NIST scientists. This guidance is critical to
the completion of a draft of voluntary voting system guidelines that you
will receive for review in our April meeting.

The time required to accomplish the agenda items means that the
committee cannot take public comments at this meeting however; there
will continue to be opportunities for the public to comment on relevant
issues. Comments and position statements should be sent to; you want to
send an e-mail to [voting@NIST.gov]{.underline} where they will be
posted on the NIST voting website VOTE.NIST.gov. The comments we have
received to date have been posted and reviewed by NIST staff and TGDC
members.

As I mentioned in my introduction, this is the third plenary meeting of
the Technical Guidelines Development Committee. At the first and second
plenary sessions of the TGDC in July of 2004, in January of 2005,
Resolutions were adopted that have guided the committees and NIST work
up to this phase of the guideline development process mandated by HAVA.
Welcome, Mr. Burger.

From July 2004 until today three subcommittees engaged in information
gathering, research and analysis including the taking of public
testimony at a two day hearing in September 2004. In addition, public
testimony has been requested, accepted and posted electronically at the
NIST voting web site, VOTE.NIST.gov.

The work product of the subcommittees' work was presented at the January
2005 meeting as a series of Resolutions. The Resolutions were debated
and, in many cases, amended. Adopted Resolutions form the basis for the
document drafting phase of the guidelines development process. The
Resolutions assigned NIST specific technical tasks. NIST scientists are
here today to report back to the committee on their progress.

At this time I would like to call on Mark Scolo of the NIST Information
Technology Laboratory to review a prioritization of the committee's
adopted Resolutions and a proposed strategy for implementing them.

MARK SCOLO: Thank you Dr. Smergian. Welcome. Nice seeing you all again.
I would like to welcome the public and the EAC Commissioners.

DR. SMERGIAN: Mark, if I could interrupt you just for a minute.

MARK SCOLO: Sure.

DR. SMERGIAN: Just for the record I would like to welcome Dr. Burger to
the committee meeting.

DR. BURGER: (Undecipherable).

DR. SMERGIAN: We quite understand.

MARK SCOLO: So this morning I would like to talk to you about the
priorities we have developed for the Resolutions that you have passed
and the strategy for implementing those priorities.

As background as you well know this committee did yeoman work in passing
thirty-one --

DR. SMERGIAN: By the way, you have copies of these presentations I know.
You may get a pain in the neck trying to look back. So, if you do have
copies in front of you --

MARK SCOLO: That's right. They are in the small workbook. Thank you. So
this committee has done tremendous work in passing thirty-one
Resolutions at the January plenary and those Resolutions were very
substantive and there is an awful, awful lot of work to do. We have a
lot on our plate and we decided that we would prioritize the Resolutions
so that NIST can focus on the important ones early on.

So, before I go into the strategy for determining the priority I just
want to tell you a little bit about the NIST thinking and the TDGC
thinking at least as much as we have been able to communicate with
respect to how we are going to, or at least or plan or our approach, to
packaging the work products. We have a couple of goals that really drive
us. Goal number one is we want to produce the best standard possible.
This means, at least in our opinion, building on the strengths of the
2002 VSS but making changes when, in fact, those changes are appropriate
and the Resolutions refer to some of the things that need to be changed
or at least looked at to determine whether they need to be changed. The
coding standards, for instance, the exemption for COT software, quality
management standards, rewriting some requirements to make them more
precise and testable.

So, the new futuristic standard will probably not look a whole lot like
the 2002 VSS but it will clearly build upon the strengths. It will be
reorganized to reflect our thinking and how to better present this. At
the same time it is clear to us that the goal, and we've heard this from
the states, we've hear this from the commissioners; one of the goals is
to help the states get through the 2006 election. They have an election
to conduct and we're very sensitive to that. They need as much help as
possible. In order to do that, one can argue we should change the 2002
VSS as little as possible so that, in fact, they work they have already
done in getting systems qualified should stay but also give them
guidance on the gaps in the VSS 2002 such as voter verified paper audit
trail, such as wireless, such as human factors.

What we've done is look at both of these goals and developed a two
pronged approach to produce essentially two standards. A shorter term,
what we call augmented VSS, which we used to call an addendum but its
probably not going to be in the form of an addendum. So, an augmented
VSS which will in fact fill in the gaps that exist now in the VSS 2002
and correct errors that are in there while at the same time developing
the first draft of the longer term redesign standards. That is our
approach.

With that in mind the strategy for determining the priorities centers on
creating the highest priority items for the guidance needed for the
states to conduct the 2006 election cycle. The highest priority
Resolutions are addressed in the April work products that impact the
2006 election cycle and focus on, again, improving the 2002 VSS by
filling in the gaps, correcting errors and also addressing the issues
facing the states such as helping to ensure that installed voting
software is the same as the software that's been tested.

So, how with the strategy be implemented? This is what I just really
explained but to summarize, the highest priority items will be packaged
into an addendum or perhaps incorporated into a document that augments
and corrects the 2002 VSS while at the same time a draft of the new
redesign voting system standard will be developed in parallel.

So, looking at grouping the priorities, Group 1, the work targets the
highest priority Resolutions, it augments and corrects the 2002 VSS and
the guidance and requirements will be complete by April.

Group 2, that work targets the second highest priority Resolutions and
this is work in developing a new redesign standard. Again, it builds
upon the 2002 VSS but enhances many areas including security, human
factors, precision and testability of requirements. The initial work
product for this document will be produced in April with planned
completion in November.

Group 3 targets the remainder of the Resolutions. They will be addressed
after April. Plans for completion, again, in November.

So, getting into a little bit of detail, Group 1, targets voter verified
paper audit trail which we know is a burning issue for the states since
many states have enacted legislation to produce such an audit trail and
I try to tie in the Resolutions to the actual priorities. This is not
only a Resolution but specifically the AC has mentioned this to us as
has the TGDC off line. The second part of Group 1 is the accessibility
and usability requirements. The third, software distribution and set of
validation procedures. Use of wireless technology. Next the interim
conformance clause that's clearly needed for the existing 2002 VSS and
we are going to revise the glossary and update it.

Group 2 includes an analysis of the 2002 VSS to determine what things
need to be changed, an analysis of voter verifiability, additional
requirements and test methods for human factors and accessibility, test
strategies and analysis addressing COT software documentation
requirements, the issue of non-conforming voting systems, publicly
available test suits and test reports is part of this group of
priorities, strategy for addressing multiple representation of ballots.
We are going to analyze Federal standards that we have developed and see
whether, how much of those apply to voting systems and essentially draft
the next version of the VSS.

Group 3, the lowest priority, the usability of the standards, common
ballot format specifications, quality management standards, maintenance
of the VSS, sharing information and disqualification of voting systems
and the last assessment papers on recommendations for future work.

Now this is a document you have in your big folder, I think the sleeve.
Its, its in my handout as well. It's a separate document. This is sort
of an index of all the documents that you have obtained with references
to what committee the fall under, which of the subcommittees. They are
identified by tabs in the book, A through W. Its on the next line. Thank
you. The Resolution number the priority grouping and the disposition
whether in fact they are in the augmented VSS or in the new revised
version. For those of you who haven't memorized it yet this the second
half.

My last slide just introduces a couple of issue I'd like to bring up and
get some feedback on if at all possible. The first has to do with
packaging of the April work products that augment and correct the VSS
2002. It seemed to us at the beginning there were two alternatives.
Essentially do an addendum or do a new version of the 2002 VSS with
changes in line and perhaps highlight the changes. It seems to us there
are going to be a lot of changes and they will be in different places in
the 2002 VSS, interspersed throughout, not just in one place. So, it
seemed to us an addendum would be unwieldy. It would be referring to
many, many changes, page 3, line 6, and probably difficult to do. So,
our recommendation is to basically do the changes in line and highlight
the ones that are different.

A more, perhaps, minor question is what do we call the augmented VSS
2002 in the new redesigned standard? Some thoughts were there have been
two versions of the VSS so far, as you could think of the augmented
version as Version 2.1. The new redesign standards as 3.0. Other
suggestions are call the augmented version VSS 2002 revision 1. Its not
really important to us. If you have strong feelings we would like to
know.

The third is perhaps a little more substantive. Up until now all the
versions of the VSS, the first and second, have had requirements for
voting systems and, by implication, manufacturers and vendors of voting
systems as well as for testing labs. Clearly we are in little bit of a
different environment. Now, we are still going to have those
requirements but we are going to have additional requirements for voting
officials and if you u look at Resolution for wireless, for voter
verified paper audit trail, many of those Resolutions talk about what
has to be present in the technology but as well, certainly how that
technology is implemented in systems. I'm sorry, Paul, did you have a
question?

PAUL: Yes. I thought we had been around this issue of keeping election
administrative requirements. Requirements for voting officials out of
the voting system standards. That is a separate body of work that vests
mainly in state elections codes unless Congress chooses to pass
something that overrides. There was a best practices work product for
election administrations run by the EAC but election administrative
issues, in my belief, and I thought it was a consensus of the committee
have no place in the voting system standards and we continue to hit that
issue.

MARK SCOLO: Whitney, yes.

WHITNEY QUISENBERRY: Whitney Quisenberry speaking. I would like to
disagree. I think we went around this issue the last time and we talked
about the difference between the administration of an election and the
requirements for the deployment of a machine to effectively meet the
standards. Those are quite different things in my mind. I think that,
for instance, you could make a perfectly accessible machine but if you
place it a foot from the wall, backed up against the wall, you have
essentially rendered those entire accessibility requirements moot. So,
it is my understanding in talking at least with human factors and
privacy staff that what we are talking about is not how to run an
election but how to deploy a machine in the service of an election.

PAUL: Okay. I guess to respond to that Whitney, the concept that I
thought we had consensus on was, yes, that's true. How you use the
machine is a very important of it being able to do its job but how the
machine is properly used is one of the things that you put within the
system boundary of the requirements in the operator's manual and, of
course, if you are using a machine and you are not following the
manufacturer's procedures that were included in the certification, then
you are not using the certified system.

MARK SCOLO: Can I just respond to that? I guess I would concur with
Whitney's interpretation of what we had discussed. We had discussed this
further in the CRT group that I think TGDC did not have full access to.
I guess my question is, in my mind its not just the operator's manual.
For instance, how do we give guidance on voter verified paper audit
trail and wireless, for instance, if we don't give guidance? These don't
have to be in the form of requirements, they could be best practices, we
are not arguing how they should be said. Somehow these Resolutions talk
about implementing things well beyond the capabilities provided by the
voting system and the operator's manual. You need to know as a voting
official how to set up voter verified paper audit trail, to ensure
privacy, to ensure all the things we are talking about. These are
instructions for how to set them up. Wireless is the same thing. You can
have the capability of that wireless but you have to know when to turn
it on, when to turn it off, when it use it. So, my question is how do we
follow the Resolutions if we don't provide guidance and how that
guidance is provided is, in my mind, completely up to the TGDC. It could
be not mandatory, it could be best practices, but somehow we need to
document that, is my feeling. So, I'd love to hear --

PAUL: What needs to flow into the documentation requirements for the
system and that needs to be part of the evaluation of the system.

WHITNEY QUISENBERRY: I'm sorry but I believe that may be how it is
promulgated and how it is communicated and I would argue that it should
not be a separate system if you want the answer to the actual question
you posed Mark. I do think that what we're saying is that; for instance,
let's just stay with my mobility example. The system documentation shall
provide proper layout and shall meet that requirement. I'm not arguing
that we are saying that here is the measurements that a voting official
might use. If you would like to say it comes out of the documentation
that's fine but I believe that we, we set up a lot of requirements for
the design and manufacture of voting systems and this would be one of
them.

MARK SCOLO: Yes, Brit.

BRIT: We've got precedence for this sort of thing now. Voting system
vendors take the optical scan. They may specify in their documentation
that you use a certain pen to mark the ballots; usually it's a pen that
they sell. Fifty percent of the people that use that system use a number
two pencil. I don't know, I definitely think that in the set up for the
ITA's for the laboratories, we should, they should specify, the vendor
should specify how the system is to be used. It should be tested to that
standard. Now, if a jurisdiction chooses not to use it that way, that's
their choice but, they have to do that realizing that they are not then
running a certified system.

Now, here's what I would like to recommend. One of the real problems we
have right now is that the voting system standards and I don't think
this one's going to be any better are so convoluted and difficult to
read that your typical local election official can't read it. They don't
know what they are supposed to be doing. So, your suggestion down here
of putting that in a separate document is, I think, an excellent
suggestion. That's a heck of a lot of extra work but if you could do
that I think that's a great idea.

WHITNEY QUISENBERRY: I forget the number of the Resolution but I would
point us back to the human factors and privacy Resolution on the
usability of the standards and that was exactly what we were thinking
about when we wrote that Resolution.

MARK SCOLO: Yes. I think you will hear discussions, I think, in some of
the presentations following mine that will perhaps allude to some of
this and I think we need closure by the end of the day on this issue.
Perhaps we should take it up at the end of the plenary after you have
heard all of the discussions. I mean, I have a lot of questions about
just documentation because people producing printers. They are not
concerned with the privacy issue, for instance, so the documentation
wouldn't talk about how to use it to ensure privacy. That's clearance
guidance for voting officials that can only come from this group, I
believe. So, yes, I'm sorry.

FEMALE SPEAKER 3: I think we also need to keep in mind our limitations
on getting the polling places ready and even having polling places that
change at the last minute. We certainly need to keep those kind of
practical things in mind of what the election official deals with on
Election Day.

MARK SCOLO: Yes, absolutely. Thank you. I think Ron --

DR. SMERGIAN: Ah, yes. Dr. Revest?

DR. REVEST: (Undecipherable - too far from microphone).

DR. SMERGIAN: Thank you.

MARK SCOLO: Mr. Chairman I would suggest that we revisit this issue at
the end of all the presentations.

DR. SMERGIAN: Any other questions or comments?

MARK SCOLO: Thank you.

DR. SMERGIAN: Thank you, Mark. This leads the proceeding plenary report
on the prioritization of adopting Resolutions from the January 2005
meeting and the proposed prioritization of work product; technical
support response to all currently adopted Resolutions by the committee.
So, unless there are supplemental directions or corrections the
technical support and related work product will continue to be developed
consistent with this preliminary report. Are there any other questions,
further directions or corrections?

MALE SPEAKER 4: You need a motion to be adopted.

DR. SMERGIAN: I think we agreed that we don't need to have a motion
unless you are recommending a change in the direction. If you think we
are going, you know, in concert with the spirit of the Resolution, we
don't feel that we need a vote at this time.

MALE SPEAKER 4: Let me propose a compromise to my point that I think
will address Whitney's issue. I guess my problem with the Power Point
was the wording of it. Both standards will have requirements for voting
officials. I think perhaps requirements for proper use of the system as
certified or as developed would be a little clearer to the intent.

DR. SMERGIAN: I think I'll, just to move things along, I will suggest
that you perhaps work off line with Mr. Scolo and suggest the wording
that would be acceptable to you and we'll then agree to modify this
presentation so that it is part of the record, the language is part. Is
that acceptable?

MARK SCOLO: I just want to mention one thing. Certainly we will talk and
come to the correct wording. One of the things we do in the conformance
clause and that's the reason I put voting officials. I had no idea if
that's a proper term. We want to identify the entities that conform and
maybe requirement is too strong a word as well, recommendations, I don't
know. So in conformance we are very interested in making sure we know
who it is that's responsible for these things. That's why I have a
party, so let's talk off line and see if we can --

DR. SMERGIAN: Thank you. At this time I call on Dr. Sharon Lescosio of
the information technology laboratory to present a preliminary report on
this approach to usability and accessibility requirements for the
augmented 2002 VSS.

DR. LISKOWSKI: Thank you. Good morning commissioners and good morning
committee members. I'm going to be reporting on our usability and
accessibility progress thus far.

This serves as an outline of what I'm going to be talking about. I'll
briefly go over the definitions, talk about the gap analysis we did,
then the draft of the augmented use accessibility requirements, then the
outline of the augmented usability requirements for planning. I'm going
to talk very briefly as an intro to the next talk about accessibility
and privacy requirements that relate to voter verified paper audit
trail. Go through our road map and then I'm not going to talk about
this, I'm just going to mention it at this point because after April we
are going to be looking at usability test protocols for the system. We
need test ballots. I have been collecting, with the help of many state
election directors who kindly have sent me samples of actual election
ballots, a collection that we can analyze so that we get a good test set
of varying complexity, suitable for qualification testing.

First I would like to talk about a couple of issues that have arisen as
we've been doing the work and that is one is the difference between the
augmented versus the redesigned VSS with respect to usability and
accessibility? The current draft that you will be seeing finalized in
April, the augmented are small additions and corrections and for the
most part usability and accessibility will be just additions to the VSS.

Longer term the redesigned VSS due in November will be more
reorganization and also performance requirements and as those
performance requirements in the long run evolve, we see also that the
need for very specific design requirements will be lessen, hence this
reorganization will look somewhat different. As it evolves over the next
few years, we hope, will have a more of a performance based tack. But as
I say, that is beyond the April deadline.

We've already talked about the equipment standards versus what I call
the process standards. I'm not going to go into the detail there. Our
April deliverable basically focused on the equipment standards but there
are these other things, especially critical and accessibility which we
will talk about in the April deliverable about setting things up
properly. The deployment of the equipment. The design of the ballot. We
do see those within the scope of what we are doing.

The accessibility requirements that you see are based on a fairly well
established body of literature that has been evolving for accessibility
of IT products. Because it's a bit more mature than usability
requirements, specifically for voting systems, you'll see more design
requirements for accessibility so that we can firm action and coverage
across a range of disabilities. As always, my proviso is design
requirements do not capture everything. You still, even in this case,
need some usability testing.

The think I want to point out about the glossary definitions that we've
put in is first, accessibility -- voting systems because of the large
audience that's trying to understand what's going but a more conceptual
definition was important to get us started with accessibility so, hence,
why we diverted from that.

The other thing that you see with the focus of a lot of accessibility
recommendations is that we've talked about legally defined disabilities.
However, HAVA under the accessible umbrella does talk about
accessibility for Native American, Alaskan native citizens and voters
with limited English proficiency. So we put that under the scope of
accessibility although typically from other standards you don't see that
kind of coverage. I'll let you read these off line because that's a
pretty standard kind of definition.

The privacy definition or really, just what it is with respect to the
voting system is a little bit different than what we normally think
about privacy in everyday situations because here privacy in the voting
system means it makes it impossible for others to find out the voter
voted. Usually privacy is just concerned about keeping information you
don't want released private. Here, the additional thing that you can't
sell your vote so it has to be virtually impossible for others to find
out how you voted a subtle distinction specific to voting.

Okay, on to the large deliverable, our assessment of the current status
of human factors and privacy for voting and we relied on lots of
literature here. Certainly what is in the VSS with respect to
accessibility? The ADAG, section 508, the web contact accessibility
guidelines, HAVA, things like not allowing other voting. These are human
factors constraints.

MALE SPEAKER 5: (undecipherable).

DR. LISKOWSKI: Yes.

MALE SPEAKER 5: I had a need to web cast accessibility here. I'm just
getting a blank screen nor any PDF. Are you using slide
(undecipherable)?

DR. LISKOWSKI: Yes, I am.

MALE SPEAKER 4: Is there a file name or something? I don't have any -- I
can't see what you're showing. Some files are shown but yours aren't
included. Maybe (undecipherable). The web cast is going but I have
nothing.

DR. LISKOWSKI: Alan? Alan will send you the slides.

MALE SPEAKER 4: Sorry to interrupt.

FEMALE SPEAKER 3: What I would say Ron, that what she's doing is doing a
high level summary of the documents of the, in this case, of her gap
analysis documents.

MALE SPEAKER 5: Thank you.

DR. LISKOWSKI: So, there is quite a large body of literature for
accessibility. For usability there is the Appendix C of the VSS. What
we've outlined in the human factors report. Some things in the I Triple
E draft. Some things on testing that came out of a UPA workshop. The
usable guidebooks that the FEC did in conjunction with AIR. There's not
a whole lot of specific things with respect to usability hence our gap
analysis, the next slide, recommends a better organization for the VSS
to make the usability and accessibility clearer, improving the standards
and test for accessibility, putting in some performance standards for
usability, creating a test methodology for usability testing to be used
by the ITAs and one thing that we haven't done in a formal way is to
examine voter privacy issues of privacy concerning the voter's choices.
We will do a principled analysis of that in the next month, I hope.

Now, let me talk about some of the issues of the draft of the augmented
accessibility that you have in your handouts. As I said its composed of
accessibility requirements that have been gathered up from all the
resources identified in the assessment and gap analysis with the
exception of section 255 of the Telecommunications Act. There's some
hearing aid kind of things that we haven't looked at yet but its on our
list of things to do. As you know, this is just a preliminary draft so
its rough around the edges. Speaking of rough around the edges we do
need to go through the draft and make sure that we use consistent,
generally accepted, terminology both in the voting community and the
community of the disabled.

We are having some discussion of do we organize accessibility by
disability type or not. Its organized b y disability type at the moment
but we are going, for the final draft, decide whether that's the way we
want to go. As I mentioned before we do include the limited English
proficiency and Native American and Alaskan native in our umbrella for
accessibility. We need a major pass through because it is a little rough
around the edges and we have to talk more about alternate formats,
usability tests, absentee ballots because there's a question we had.
Also, accessibility of the smart card machine.

You also have an outline of the augmented usability VSS and here our
major issue is how much do we do with respect to putting in design
guidelines for usability which may be -- a little technical glitch here
on the screen. So, how hard do we work between now and April to put in a
whole lot of design guidelines that well go away when we get the
performance requirements? So, it's a delicate balance. I just lost my
view graphs, Alan. I'll do a little tap dance while we're waiting.

MALE SPEAKER 6 While you're waiting on that can I make a couple of
comments?

DR. LISKOWSKI: Yes.

MALE SPEAKER 6: On this privacy issue, selling votes in minuscule. The
main reason I want privacy is I don't want my wife to know how I voted.

DR. LISKOWSKI: Very good point.

MALE SPEAKER 6: Vote selling itself is pretty minuscule. The other is,
if you are collecting ballots, you may be collecting the wrong thing.
You might want to collect state laws that define ballots. These ballots
are developed in compliance with some very explicit laws.

DR. LISKOWSKI: That's a good point. We'll definitely look into that. I
do want to see actual ballots because we also want to develop the actual
test ballots so I want them to see how they are rendered across the
board, but absolutely, you're right.

So, right now we are discussing how much time do we put into pulling
together all of these design requirements versus getting to the
performance based usability testing and, so, my feeling, you know, given
the tight deadlines is that we sort of gather up what's out there now in
terms of usability and put that into the April version and then start
the new work.

I promised John Wack who is going to be talking about easy pack that I
would spend a couple of minutes talking about accessibility, usability
and privacy with respect to it and I view it as a three legged stool. If
you break one of the legs the stool falls over. With respect to
accessibility of a paper audit trail that is voter verifiable you need
to provide, its generally accepted that there should be an accessible
way to do that for people with disabilities. So, just to give you kind
of a generic example one might imagine an accessible VRE that prints out
an optic scan ballot. That ballot is fed into a separate reader that has
audio and reads it for those that are blind but also might have a screen
that lets you adjust the contrast and the larger print, etc. for other
kinds of disabilities or combination of disabilities. From the various
information I've gathered having this kind of secondary reader as long
as its separate from the voting system seems to be an acceptable way of
doing the verifiability. For example, the blind community typically
makes heavy use of readers in all their IT but suppose the audit trail
is under glass. How do you then feed this paper into something that's
accessible? One idea might be to have it cabled to the reader the same
feed that's going into the printer. Nothing like that is implemented at
the current time.

You might say: why don't we just do some alternate format paper audit
trial like larger print, etc. Then you get into privacy issues because
to preserve privacy you want the ballot, all the audit trails to be
indistinguishable from one another. You could print on the ballot output
the different choices and all the alternate formats of different font
sizes and languages and you get something very long. That doesn't sound
very usable. You have got to have a paper audit trail that's readable by
both the voter and the election officials if they are going to use it
for a recount and there's also kind of load if you are looking at your
voting work station, say a VRE. If you are looking over to the paper,
it's a different medium. It looks very different. How many errors do you
get if people who think when they try to audit that its different and it
really isn't, especially if you have something long.

There is a bunch of usability issues as well and that's the usability of
the set up, the printer, the paper, expert readers, etc. and then you
are back to pulling a whole bunch of things together and is the whole
thing accessible as well. There's a lot of issues fighting in there that
we have to think about very carefully and John's going to talk a little
bit more about that in a broader context.

I do think that there are definitely solutions out there but paper being
inherently inaccessible means we have to think about it because it's a
complex issue.

Briefly, our road map for April, the two augmented versions of additions
to the VSS for accessibility and usability focus on capturing the
indication of the voter's choice. Just a draft gathering up existing
guidelines for polling place, defining some usability performance
measures to set the stage for the next work, a plan for an assessment of
what we are going to use with respect to universal design. I forget the
Resolution number that talked about that but there is a Resolution, and
just kind of looking at sort of current, what's out there for ballot
design, instruction, error message collection, coming up with a plan to
then address that. There's a couple of requirements on testing
spreadsheet that you'll hear about later. Today, we need to look at the
human factors and privacy issues there and address them in that
spreadsheet and revise our VV path analysis as well.

For November we expect the redesign usability and accessibility
requirements. Some initial performance benchmarks for usability and a
test protocol and then some revised polling place guidelines, universal
design guidelines, again, using our verbiage today that are requirements
for the deployment of the voting equipment as well as ballot design,
instruction error message guidelines and just looking over the whole of
this work product and looking at any other general human factors and
privacy issues that may have arisen in what we've done thus far and some
usability evaluation, the redesigned VSS in general.

I guess I'll leave the road map up. That concludes my summary of where
we are thus far and I'd like any feedback or questions or
clarifications.

MALE SPEAKER 6: It looks like a good plan.

DR. LISKOWSKI: Thank you.

FEMALE SPEAKER 5: I just have a question for you Sharon.

DR. LISKOWSKI: Yes.

FEMALE SPEAKER 5: I know you have been getting some feedback and input
especially on the accessibility requirements in the community. Would you
speak a little bit about that?

DR. LISKOWSKI: Yes, thank you for reminding me. We are in close
consultation with the Access Board and, in fact, they have been very
kind to provide, I think we got it Thursday or Friday a lengthy
commentary on what we've done so far and so we are going to go back and
address all those comments and be in both e-mail and verbal discussions
with the Access Board.

FEMALE SPEAKER 5: I know I have seen some comments come in from
accessibility and disabilities commissions around the country.

DR. LISKOWSKI: Yes, we have read all the comments that have come in and
we are going to be addressing each one, one by one to make sure we
incorporate those into our accessibility documents.

DR. SMERGIAN: Thank you, Sharon.

DR. LISKOWSKI: You're welcome.

DR. SMERGIAN: This leaves the preceding preliminary report of technical
support titled "NIST Approach to Usability and Accessibility
Requirements for the Augmented 2002 VSS" and "Response to TGDC
Resolutions Number 2, 3, 4, 5, 6, 8, 9, 10, and 11\". They had their
work cut out for them. So, unless there are supplemental directions or
corrections the technical support and related work product will continue
to be developed consistent with this preliminary report. Are there any
questions, further directions or corrections?

MALE SPEAKER 7: No, I was very pleased with that.

DR. SMERGIAN: Looks like everybody is happy with what they've seen and
the direction they are following. Thank you.

At this time I call on John Wack of NIST Information Technology
Laboratory to present the preliminary report on NIST approach to voter
verified paper audit trails requirements for the augmented 2004 VSS.
John.

JOHN WACK: Thank you. Can you all hear me fairly well? Okay, well,
hello, all. I'm John Wack. I've worked in the computer security division
at NIST for a number of years and Mark Scull said this is a burning
issue and I can assure you that I am burning. I feel a little bit at a
loss discussing this with you because this issue has been discussed so
much that what can I really tell you that you don't already know and I
was hoping to plan a couple of things, you know, earth shaking, like it
reduces cholesterol or makes people feel better, you know, reduces
impotency, things like that. What I generally found out is that it
really being in security doesn't necessarily make me the best person to
address this issue. Usability is critical and there are a number of
small issues that need to be addressed that aren't always brought out to
light and that some of the state legislation is very good, some of it is
at a higher level and more than ever I think we really need your help.
We've come up with some requirements. We may have set the bar high,
maybe we need to go higher, maybe you want to take it down lower, but in
general, any comments, any suggestions, any criticisms, are very much
appreciated.

The other thing is I just want to briefly thank the members of the NIST
voting team. I've got a lot of assistance and the STS, especially Dr.
Ron Revest. Its been a pleasure to work with him. The I Triple E very
graciously allowed us to use their draft P1583 standard. I wanted to
also thank Leslie Reynolds. Ms. Leslie Reynolds of the NASS who helped
in some analysis of state legislation.

Lets see if this thing works. I've got about ten slides and I am really
just going to go through the first eight very quickly and just basically
to give you an idea that we did try to do some due diligence in figuring
out what the issues are. You know the genesis for this. We really are
basically working on a addendum to the existing 2002 VSS. We are really
addressing DRE systems that basically produce a printed summary of a
voter's choices. The voter can then compare that with the electronic
record. If the voter is satisfied with that, he can then do something
that causes the electronic record to be recorded and that is the record
that is counted. The paper record is used for audits and recounts.
That's the system we are talking about.

Let me go on to the next one. Basically, in addressing this issue we,
number one, there is plenty of literature out there, a number of strong
opinions, a number of good analyses that are out there. A number of
those are available on our web site. We have been discussing things with
some election officials. That's been what I've found critical. We can
sit and think about requirements and think about the most secure ways of
doing things and then we have to talk to the people that actually have
to pick up the paper, turn the machines on, do statistical analysis at
the end of the day in a hurried environment and so really getting back
to the security issue, it has to be secure, it has to do what its
supposed to do but if its not usable its really not practical and
workable. Then we looked at what material we had out there that we could
derive requirements from and found that in general some of the state
legislation was very good and from there we went on addressing the major
issues and came up with some requirements.

From looking at things, looking at the literatures out there,
discussions, dealing with people, its no surprise that the issues I have
up here are really the important issues, usability, accessibility,
robustness of the technology. The fact that the electronic and paper
records need to correspond with each other because they are two distinct
records of votes and they need to correspond carefully and be handled
appropriately. Then there are various issues that arise in resolving
complications and disputes. So, we looked at those issues. One other
thing that was brought out to me is that usability, we tend to think of
that as for the voter but its critical that it also be for the poll
worker, for the election official. Usability of the paper record has to
be looked at by the voter but somebody else has to audit that. That's a
very important issue.

The other thing is that DRE's with VV pack capability may be larger. It
may take up more space. If a poll site generally has a line of voters it
may increase the line of voters because it takes more time. I don't know
how much more time but it takes more time to verify your vote and cast
it that way. So, there are just some practical issues associated with
that as well.

Looking at the state legislation I have to admit I looked at it at first
thinking I'm not going to find anything and actually I found a wealth of
very good material. A number of states have done a very good job, I
think, in addressing some of the requirements and figuring out some of
the issues. Some states looked at it at a higher level and again I think
this is where we come in where I think we can do some good at least
specifying the core fundamental issues and requirements of best
practices or what have you that the states really need to take a look at
to make sure that they do this correctly.

A couple of things I'll point out. I found a couple of surprising things
that I didn't expect to find but some of the states actually wanted a
strict correspondence between the electronic and the paper records. They
wanted to be able to look at the paper record and go to the specific
electronic records. So, they wanted identifiers on the two to map them
back and forth. So wanted machine readability for the paper records.
Accessibility was covered; I thought, fairly well, at least mandates for
accessibility.

A couple of other things. I didn't delve too much into the state
legislation in other areas besides VV pack capability. I did find that
five states overall had enacted legislation with a lot of VV pack
requirements. At the most recent count I think twenty-three other states
were dealing with legislation. The legislation had been introduced and
was in various stages. We pretty much looked at what had been enacted
thus far. I don't know how the other legislation will turn out.

Again, no surprises. What did we find out based on our initial analysis
of issues and state legislation, again, usability, usability, usability.
Basically procedures needed to handle the paper. VV pack capability
should not simply just reintroduce paper problems. As much as possible
we should address that issue so that paper is easier to handle and so
that auditing is more accurate. It should not be a surprise.

I'm going to skip over the major features and the major benefits because
it sounds like I'm trying to sell you something here. I'll just go
directly to the requirements and what I would like to do is just kind of
go over them a little bit and, see if I can find my notes here, give you
a brief summary and some of the issues involved in that and, Alan, I've
got about five minutes left? I don't want to talk too much. Oh, I can
keep going. Okay.

What we did in our document was we came up with eight core requirements
and then we have a number of derived requirements for each of those
eight. Other documents I looked at came up with slightly different
requirements, slightly different language, so we are not wedded to this
but this is our way of looking at it.

The first one is a fundamental requirement of DRE capability, DRE VV
pack capability and it basically just says that a paper record has to be
produced and at the end of the day you end up with two distinct,
independently handled records of votes. You have two separate things
there. You don't necessarily have to use the paper record for auditing
and recount. I think HAVA says that the paper record is the record to be
used as the last resort but you do end up with two records that have to
be handled carefully.

The second one, permitting the voter to compare the paper record and the
electronic record with maximum uses. As you can figure out a usability
related requirement. Basically if the verification procedure is not
used, or if it draws complaints, obviously that's a bad thing. Let's say
one in every thirty voters actually performs the verification procedure.
That's statistically may be a significant figure depending on the size
of the election. Its still an indication that the usability is probably
pretty poor. I don't think if the usability is very good we are going to
have everybody verifying their vote but we should strive for more than
one in thirty.

Number three, the method for vote verification shall be accessible. I'm
going to point back to Dr. Liskowski there a little bit more. What I
know about this subject is just enough to make me dangerous. Basically I
do know that handling accessibility requirements is fundamentally a
little bit different from doing it otherwise. It requires special
features. A couple of things I find interesting, for example, if you
print out a paper record, let's say in Spanish, then is that going to
make it harder for somebody auditing the paper records to deal with
that. We have to design the paper records in certain ways so that they
are still accessible to auditors as well as voters. A number of issues
associated with that, privacy has to be safeguarded. If there is only
one Spanish speaking voter at that site then its going to be pretty easy
to figure out what their paper record is. We have to be careful about
that.

Number four. The DRE VV pack shall permit the voter to accept or reject
the paper record and reenter ballot choices. Basically, we need to keep
a strict correspondence of records even if they are spoiled. A high rate
of spoilage may indicate that there are some usability problems with
that system. In a sense with that particular requirement we need to come
up with a number of procedures that deal with spoiling limits, how
that's going to be handled. What we'll do in basically marking ballots
in the presence of voters that they are spoiled or accepted. A number of
issues are coming up there.

Number five. Voter's privacy and anonymity shall be preserved. Well, one
of the obvious things is when you have two records of the votes,
especially a paper records that can be handled and read, then you have
to address the privacy related issues with that. I'll talk a little bit
more about that when I deal with the next requirement. One of the
interesting things that we are struggling with is that the manner in
which the paper record is produced and then stored ought to safeguard
voter privacy. Ideally it would come off the printer and it would fall
in some sort of random fashion. Somebody could pick it up. They wouldn't
know what system it came from, what voter. That may be difficult to do.
It may be very difficult for every single DRE to do that. I don't know
but it seems to me that that could cause some problems and it may
involve some compromises such as people may need to handle the paper
ballot. They may need to walk it to a central ballot box. It may be
that, you know, one possible compromise is that it may be that the
papers all come out in the order in which people voted and then poll
workers need to safeguard the security of that or randomize it some way.
There are a number of issues there. Ideally, though, we would like the
technology to address the security issue automatically and not rely so
much on people but we may not be able to do that.

Number six. The DRE VV pack shall permit robust auditing, forensic
analysis, full recount capability. That's really the more interesting
one. That deals kind of with the situation of when a voter goes to
verify that the paper record and the electronic record match and if they
don't match, and then we have some interesting issues there. I'm not
sure what exactly to do whether we call in the armed guards, whether we
pull the plug on that voting system or what. Basically we need to be
able to provide a paper record that points back specifically to its
corresponding electronic record to improve auditing capability. We need
to do the same for spoiled ballots as well. We need to have a one to one
correspondence. It would be good if the paper ballots contained more
information such as an indication of the serial number perhaps of the
DRE. The name of the election. The precinct. A number of other things
that can be looked at after the fact to help identify problems,
forensics analysis, things of that sort. Let's see if I've left anything
out about that. Also got a couple of requirements in there for how
electronic records ought to be exported from voting systems. It would be
good if voting systems exported the electronic records in a format that
could then be used and analyzed on a different computing platform and if
you are going to export them it would be good to export them with some
sort of a hash, ideally a digital signature but provide some
authenticity.

MALE SPEAKER 8: I just wanted to make sure I understand because I'm a
little bit confused about what the readers really provide. So, let me
make sure I understand the process. I'm a voter. I see on my screen the
votes. I look down and I see on a piece of paper, under glass, let's
say, perhaps, a vote. I see they match. I'm happy. I walk away. Isn't it
likely that the meddling would be after it leaves the screen? It get
manipulated through the data base. What I see is not what necessarily
going to match. So to me I don't, it's a lot of effort and it doesn't
really solve the problem. Is that the way people are talking about it
right now? It would seem to be more likely to, let's say, have it in a
data base which is used for counting, put some random number, and I have
a random number here, and I can go through the process of accessing it
from the data base. Even then there's some risk but a little less risk.
At least it has physically left the screen and gone to some data base
for counting. Something like that.

JOHN WACK: Well, I guess as far as opinions about whether, you know,
this is a good thing to do or not, I'll probably refrain from that. I
think, though, that establishing some traceability between individual
records certainly allows someone to determine whether paper ballots have
been inserted or whether they have been removed. I think, I don't really
know, based on my research if that aspect of fraud is the biggest issue.
Its probably not. I do think making the records correspond to each other
strictly does help in that regard. Did I answer your question?

MALE SPEAKER 8: Yes.

JOHN WACK: Okay. Another question.

MALE SPEAKER 9: I think within the scope of what I understand we are to
work on and what the commission has an interest in is providing
standards for the states who now have mandated this technology. So, the
issue of how you spoil ballots, what you do with the spoiled ballot,
what you do when you reach a conflict, how you are going to run the
forensic analysis, you are going to have to alternately look to state
law for that and those states that have required this and haven't
addressed that will find themselves addressing it either in statute or
in the election contest activity.

I was interested in the use of the word auditing and I just checked the
prepared glossary and I agree with that definition there and I just
wanted to make sure in your analysis we are being consistent with that
definition which in the glossary it is the systematic, independent,
documented process of obtaining records, statements, fact or other
relevant information and assessing them objectively to determine the
extent to which specified requirements are being filled. That is very
different from the use of the word auditing or reference to recount type
proceedings.

JOHN WACK: Yeah, yeah and I think we are talking about the auditing, you
know, the aspect -- there's a possibility that definition does cover
that in a very general sense. The auditing I'm talking about more is the
correctness of the electronic record via auditing it with a paper
records. I'm talking about the usual one percent manual recount or
auditing that we generally see in state law.

MALE SPEAKER 9: So you are talking about what I think in some states is
called a discovery recount.

JOHN WACK: Right.

MALE SPEAKER 9: Once again, that's something that some states have and
some don't. Once again, you are going to have to look, even though I
know a lot of its high level and there's not much there, in some states,
you are going to have to look back to the state laws because that's,
those are the people who this particular standard is going to serve.

JOHN WACK: You raise a good point. I meant to bring it out. I don't
think we are in a position to say what auditing you ought to do. That's
clearly not our area. What I try to do is at least make the records more
accurate and more traceable and in a sense give people tools so that, if
they choose to use them, they can and we assert that that will result in
better accuracy especially in resolving disputes. As far as the rate of
auditing I've read a lot of material about that and opinions vary. I
don't want to get into that so much, its just basically here are the
tools so that you can do auditing.

MALE SPEAKER 9: Okay.

DR. SMERGIAN: If I can, John, we've talked about this. I think this is a
very important point. We are not advocating any of the particular uses
technologies, etc. The point here is, if a state decides to do certain
things we want to make sure that the standards and therefore the tools
exist or are present to be able to do those. This is strictly if all the
statements have to start with the "If" statement, you know. If a state
decides to do this or if they have a legislation to do such and such how
can that process be done in the best possible way. I think that's a very
important issue here. We don't want to give any impression that we are
in any way advocating, you know, this or that technology or to do a
particular procedure. I just wanted to emphasize that.

MALE SPEAKER 9: But we do want to have the tools available.

JOHN WACK: Yes, thank you.

MALE SPEAKER 9: I'm just a little sensitive about the misuse of the word
audit because, I mean, its something that election officials have been
beat up on by activists and interested parties who have complained that
they have no audit process when they have very good internal controls,
they have very good documentation of the process, they have separation
of functions and they have a whole bunch of stuff that you can audit but
those are election administration controls more so than audit. I think
the definition here in the proposed glossary speaks to that. I just
wasn't sure if we were being consistent.

JOHN WACK: Well, what I would like to do is talk with you off line. I
like the glossary. I just like the fact that we are trying to focus on
common language that we understand. That's another thing that I kind of
learned in state legislation is that people use terms differently
depending on what state and that must be confusing. It would be better
if we all stuck with the same definitions. Yes.

FEMALE SPEAKER 6: I just wanted to make sure and I think the question
was answered but I just want to make sure that when we are talking about
whether its in a roll, our receipt is in a roll and that the elector has
the ability to read that or if it is cut and given to or made available
where the elector can put it in a box. I don't want us to take a stand
on that because many of us really have concerns about the elector
because if they walk out with that receipt then it doesn't match.
Whether you call it auditing or recount or whatever the case might be,
we are in trouble. I want to make sure that we are not indicating one
way or another and leave that up to the states for their state law.

JOHN WACK: Okay, well, that's a good comment. I would say what we have
thus far is leaning towards the direction of basically saying that it is
easier when you have individual pieces of paper to make then random. Its
easier to feed them perhaps through a machine and automatically scan
them. If you keep them on a paper roll obviously you preserve the order
in which voters cast their votes. As to which way you want to go that's
really not, you know, our role to say. They both have advantages and
disadvantages.

FEMALE SPEAKER 6: I agree. I totally agree with you. I do think even if
its on a roll it can be handled by procedures and by rules that its not
marked because, I mean, if you only have one piece of equipment to meet
the disability requirement of the law in that polling place and even if
you have more than one, you are not going to know how the voter voted
even if you tried to track back unless you marked that, what machine
they voted on and obviously that's the thing we should be concerned
about, that there's never, ever an indication whether the individual was
voting a scanned ballot or on a DRE or which piece of equipment that
they were voting on. I think that's a real concern there because we have
to protect the privacy of the voter. That's my main concern when you get
into an issue like that. I agree with you totally but I think there is
ways to count those, whether its underneath a glass or a bigger ballot,
I mean, they are both coming up with ways of being able to count that
other than just by hand.

JOHN WACK: Could I respond real quickly. We, no, you are more important.
Go ahead.

FEMALE SPEAKER 7: Well, I guess, I actually wanted to start by
congratulating you on a really coherent and dispassionate presentation
of this issue. One that has caused raised voices in hallowed halls and
many meetings. So, I think that, in itself, is quite an achievement and
I congratulate NIST. This is exactly the sort of careful analysis I had
hoped we would be getting from the NIST staff. So, thank you very much.

I would also just like to pick up on what Secretary Davidson was saying
which is I was looking at the structure of the document as you have been
proposing. It was kind of hierarchal requirements and I think that there
is a lot of value in being able to say, if you do it this way, then here
are some requirements. If you have chosen a paper roll then you must
have surrounding administrative procedures or if you have, whatever, so
that we are not providing a single solution but are providing the
requirements in all of the appropriate permutations of solutions and I
think that one of the things that will do indirectly is provide guidance
on how difficult to administer or how difficult to design or prepare or
qualify or certify actually, both of those two things, a particular
design of system might be. I think that, in itself, would be a helpful
kind of indirect analysis that would help state officials make good
decisions.

JOHN WACK: Thank you.

FEMALE SPEAKER 7: Sure.

MALE SPEAKER 10: John, just two thoughts, you know, I didn't give it
much thought. With respect to the concern about the order in which
ballots are and seeing what people have designed, now this may not work
for all districts, it may not work for any but, if you had a large
enough number of people you could perhaps say have then sign their
books, have bunches of lets say five or six, use a randomization of the
individual before they go into the voting booth. Therefore what you are
seeing on the books, you know, isn't necessary the order in which they
voted.

FEMALE SPEAKER 9: That won't work in my precinct because we have a
single machine per precinct.

MALE SPEAKER 9: Okay.

FEMALE SPEAKER 9: I live in a very small town.

MALE SPEAKER 10: No, even if a single machine but you have to have at
least enough people so you couldn't, wouldn't make -- thirty minutes
until another three or four come along.

MALE SPEAKER 11: Another issue that you have is poll watcher who
basically are allowed to sit in a precinct and would be capable of
documenting the order in which people voted.

MALE SPEAKER 12: It is possible in the disability issue that even though
it's a standard paper, if you had feeding after that machine into
another special machine that could render it in an particular media or
so forth, you might be able to get around it that way. It's the same
paper that's now being scanned and read and it gives an audio voice or a
different print or whatever it is that you want from the machines in
which you have counted is a possible solution for the disability one.

MALE SPEAKER 13: John, could you address on number six to clarify. What
it seems to read is that both the electronic and paper records would be
used for a full recount? I thought there should be a precedence as to
which is used for recount as opposed to saying both.

FEMALE SPEAKER 10: I believe --

JOHN WACK: Well, that precedence --

FEMALE SPEAKER 10: Isn't that a matter of statute?

MALE SPEAKER 14: Yes, it would come from state law.

MALE SPEAKER 13: But that's my point. I think this recommendation seems
to imply that both, as opposed to saying that there is a precedence
here.

JOHN WACK: Okay, I understand. Yes.

MALE SPEAKER 13: That's all I'm saying is a clarification of how --

DR. SMERGIAN: John, do you mean both or do you mean either?

JOHN WACK: I guess really either.

DR. SMERGIAN: I think what you mean is that depending on the situation
either one may be used. Is that what you really mean?

JOHN WACK: Right. We should do our best to end up with a situation where
either record could be used. As to which one must be used or shall be
used that's not our decision but it is best if we design the records and
produce them in a way that if they both have to be used, the can. Really
when it comes to resolving disputes, if the two records disagree, I
think the more information we can draw about how the records were
created the better off we are and I wanted to come back to an issue you
addressed and it seems that from what I can understand there's this
tension between usability of the paper records, let's say, versus
auditing and information on the paper records to make auditing usable
and more accurate. If you don't include any identifiers on the ballots
or on the paper records, I should say, if you don't include the serial
number of the DRE or a unique identifier, things of that sort then you
can take those spools of paper, you can put them in a box, they can be
stored for a long time, nobody can go back and figure out who voted
first on a particular machine. At the same time it does make it a little
more difficult to audit the paper records against the electronic records
and the better the correspondence, the better you are able to do a
number of things and also just basically come up with post election
statistics and analyses that would be good for usability of systems.

One of the things that we proposed in the requirements is the
possibility of, you could have a paper ballot, let's say like this, and
its got a human readable part and then an encoded part. The encoded part
could be basically an encoded representation of what's on the human
readable content. That way you could hide some information such as
unique identifiers and things of that sort. It would have to be an open
format basically, the major shippers use similar sorts of encodings and
formats, easily scanned. It would require some auditing to make sure the
two representations worked.

FEMALE SPEAKER 11: Too far from microphone to understand what was being
said.

JOHN WACK: The major shippers. I was waiting for a package and a shipper
showed up with basically a box that had some dirt and scratches but they
were still able to use a hand scanner to go through and read off
information on the package. Its fairly robust technology. That would,
that may help in this situation. It may hide some information. It would
also making scanning the paper records more accurate. There is this
tension here, you know, I think the more of these sorts of things we can
do to make auditing perhaps more accurate we still have this tension
with usability of the records and perhaps reduce overall accuracy in
auditing. Maybe I'm misusing it a little bit but in recounts that may be
acceptable. I really don't know.

DR. SMERGIAN: Secretary Davidson and then Dr. Williams and then Mr.
Gant.

SECRETARY DAVIDSON: And I agree with you. I think that, you know, there
is so much that you can do with a bar code. You have to be very careful
not to put numbers on there because the public will view that that we
can identify how they voted. Its very, very contentious. A bar code that
could not ever identify how somebody votes. We don't want to ever do
that, be able to with a bar code or with numbers. We have to make sure
that they understand that it is only for auditing to make sure the votes
were there, not tracking how they voted. It is very important. When we
are wording it, evidently it has to cover that because we don't want the
public to take it the wrong way. It's got to be very common wordage
(sic) so that they understand that, I think

JOHN WACK: That's a good point, thank you.

DR. WILLIAMS: I have several points. One is on this issue of putting
information in bar codes. I have to point out that the voter can't
verify that bar code so you don't have a voter verified ballot anymore
if you use that bar code to count those paper ballots. You are not (END
OF TAPE 1) were introduced in this country voters have handled their
ballots. One of the criticisms I've heard of these VAT systems is the
fact that that things under glass and the voter has trouble reading it
and can't hold it in their hand and look at it. I'm glad to see you are
considering that.

The third thing that's kind of a nagging problem for me is that I
haven't heard any discussion of the impact that all of these innovations
are going to have on the voter. We know a little bit about voter
behavior and we know that most people who vote are in a hurry. If you
start putting impediments in the voting place, regardless of what that
impediment is, whether I've got to verify this or take it over to
another station, or what, you are going to have a drop off in your
participation so we don't want to create an unintended consequence here
where we come up with an enormously secure system that people stay away
from in droves.

JOHN WACK: Well, you raise a question and that is what if a voter
doesn't want to verify their vote. Should there be a way of saying, I'm
happy, press a button I'm out of here. Yeah. The issue about the voter,
in a sense not being able to verify the encoded representations that is
a good point and I've thought about that. That does require some manual
auditing. How much, one percent, I think it would probably be likely a
lot less than that but you do have to do that.

MALE SPEAKER 15: We could miss the whole underlying concept here is to
establish voter confidence. When we are in the back room verifying that
this bar code corresponds to that printed record the voter is not
participating in that process and it doesn't do anything for his
confidence.

JOHN WACK: I anticipated a question about this and all I can say is, I
agree at the same time I think its, if you handle a piece of paper you
can wrinkle it, the paper has got to be more robust, a number of things.
If there is a robust encoded bit of information on there that can be
machine scanned it makes it easier to handle the paper and you more
accurately scans so there is this, you've identified a conflict here.

Was there one other question?

MALE SPEAKER 16: John, I just wanted to kind of close up my question.
So, in the section 3.3.6 which is the detailed set of requirements that
you will add some text or another item to kind of clarify the recount as
being a different process than the audit as described there. In other
words there is nothing in there that really talks about the full recount
being based upon one or the other, based upon state law, etc. I just
want to be sure that that section gets clarified. That you are not
expecting that a recount is done with both.

JOHN WACK: Okay. All right. I understand and again, maybe afterwards I
can grab you for a minute or two and make sure I get the language down.
I think I may be running over and I just have one last thing to say. I
didn't address just two of the last requirements and I should do them in
maybe one sentence. They are fairly obvious. One is printer technology
must be very robust and even reloading a printer with paper or ink may
be too much maintenance to do on a printer during the day. So we are
addressing those issues and we, I'm a little sensitive to prescribing
procedures or requirements for procedures but we are identifying the
need for definition of procedures to handle all of the complication that
can occur because, you know, what can go wrong will go wrong. There are
a lot of complexities to providing this. So, people who have the proper
training, who can anticipate and answer questions, so on and so forth.
That's another important issue that can't be neglected. Yes.

SECRETARY DAVIDSON: I have one additional question. Have you thought
about and I apologize, but have you thought about or looked about
retrofitting old equipment. States are saying you've got to have the
paper verification on DREs that have already been bought and
retrofitting that. Are you moving into that area whatsoever in the
suggestions because that is coming up in states.

JOHN WACK: Well, that's a good point and we haven't addressed that
specifically. I think its possible we could not address it specifically
but still issue requirements. That may get back to what Whitney's
mentioned that we may have to do something like if you are going to do
it this way, if you are going to have separate sheets of paper do it
this way. If you are retrofitting do it this way. That will be a little
more work, I think, but I agree its an important issue.

Okay. Thank you very much.

DR. SMERGIAN: Thank you, John. Did we take care of all the questions,
comments? Well, I do believe the preceding preliminary report of
technical support titled "NIST Approach to VV Pat Requirements for the
Augmented 2002 VSS Response to TDGC Resolution 1205\". Unless there are
supplemental directions or corrections, certainly the comments made will
be taken into account. The technical support and related work product
will continue to be developed consistent with this preliminary report.
Let me ask again; are there any questions, further directions or
corrections? If not -- yes.

SECRETARY DAVIDSON: I do believe that we need to do something about the
retrofitting and that would be an additional requirement, I'm sure. I
think that states looking at your vendors right now are looking into it.
They could use some guidance I do believe because they are just
beginning to assess that, what it would take. Obviously, it would have
to go through certification and we feel like the equipment would have to
go back through certification if you are retrofitting an additional
piece on there. So, I think that should be something that we should add
to it.

DR. SMERGIAN: Do we need a motion for that? I think that's a minor
addition to the instructions. Do we have unanimous consent, basically,
to add retrofitting issue to the instructions?

MALE SPEAKER 16: That's governed in your present standards anyway
because the present standards say that any change to the voting system
requires re-qualification, re-certification.

DR. SMERGIAN: All right. So, hearing -- did you have another question?

SECRETARY DAVIDSON: No. I just, I agree with Britt on the certification
but on the, giving them guidelines, you know, what we think that
retrofitting should be, I think should be there.

DR. SMERGIAN: Okay. Well, not hearing any further --

MALE SPEAKER 17: Well, why don't we go ahead and make a motion, make it
a Resolution and then that kind of ends the question of whether we need
to or not.

DR. SMERGIAN: Well, we just did. I mean, by unanimous consent we will
add retrofitting to the instructions. Is that acceptable to everybody?
Okay.

Well, amazingly we are ahead of schedule. Instead of starting the next
presentation I suggest that we take a little earlier break. We basically
had allowed for a twenty minute break so I suggest we come back at 10:40
this time and we will start again. So, please try to get back a few
minutes before that so we will be able to get started at 10:40. Thank
you.

DR. SMERGIAN: Okay. If we could resume our meeting. At this time I call
on David Siefert of NIST Information Technology Laboratory to present a
preliminary report on NIST approach to wireless requirements for the
augmented 2002 VSS. David.

DAVID SIEFERT: Thank you.

DR. SMERGIAN: Can we sit down please?

DAVID SIEFERT: Based on the little discrepancy with whether you've had
time to review the document or not, my main concern is have you at least
had time to read the Executive Summary which is just a cover page? If
not, I'm going to read that so you have some idea of where my
presentation goes because the hope was, I wasn't going to rehash what I
have in the presentation or what I have in the document but rather
highlight some of the issues that I think need to be addressed.

I'm going to read from the Executive Summary from the document: "Based
on Resolution 35, number five, titled 'Wireless' NIST is directed to
research and draft standard documents for the use of wireless
communications in voting systems. Since a blanket statement about
wireless communication devices in voting systems is neither prudent nor
appropriate given the wide variety of wireless communication devices and
possible usage in the numerous and diverse voting systems the approach
to considering wireless communication devices in voting systems will be
on a case by case basis.

To this end this will create a guide showing where wireless
communication may potentially be placed in a particular voting system
and some of the associated security risks. This guide will contrast the
hype for wireless technology usage versus the real needs and/or
requirements for the wireless technology to improve the performance or
operation of a voting system. The placements described are not to be
construed as suggesting that wireless technology should be used in these
locations nor is the list exhaustive of all the current or future usage
of wireless technology placements, nor does it replace those
pre-existing wireless requirements currently stated in the VSS 202 or
the I Triple E draft." I'm going to skip the next paragraph and read the
last, final paragraph.

"An exhaustive investigation of all possible wireless technologies or,
more importantly, all implementations of wireless technologies will
never be practical. Therefore specific wireless technologies will only
be used as examples."

With that said,

SECRETARY DAVIDSON: Could you move the mike closer to your --

DAVID SIEFERT: Can't hear me? All right. Better. I'll talk louder. Okay.
So, the draft as it stands is to respond to 3505. First we need to reach
a common understanding of the subject of wireless communication devices
and I'm trying to do that by providing background information. Wireless,
as a term, is broad. I also try to identify any technical requirements
for the use of wireless communication devices for the voting system by
review VSS 202. The end goal is what you want to provide guidelines on
the use of wireless communication devices.

The progress so far is the draft before you. I've tried to do this in
three, trying to address three separate potential people who are
concerned with wireless usage. As was mentioned in the opening
statements by Mark Scolo, we have to prioritize these based on the
voting system itself, the users of the system and also the technologists
who have to try to define features that will be available to those
systems. So, if you look at the draft, its laid out in these areas. The
first section, Section 3, which actually goes through the VSS and notes
any possible location of where someone might interpret as a wireless
communication device being needed or being used.

Section 4 addresses the issue of the Election Day or preparation for
election. The equipment itself, how the election is run, wireless need,
where and when wireless may be used because there has been discussions
that, well, you can use wireless except for the Election Day. Well, not
necessarily, on Election Day there may be times when a system to use
wireless may be appropriate or may be convenient. Its not a time of day
that you can turn wireless on and off.

Finally, the major issue that I had reviewing VSS 202 and wireless,
whether its use, was in the realm of telecommunications.
Telecommunications has a section unto itself, Section 5, and it lists
some wireless technologies or wireless as an example of usage cases but
it doesn't go into any more detail. If I look at that strictly from a
requirements point of view, wireless has no more requirements than a
telecommunication network does. I can have that perspective but I want
to call that out that that's an assumption. Also, there is a discrepancy
between the telecommunications network being public or private. A
statement is made in VSS 202 that private networks have to be the same
or have to follow the same requirements as public networks. Now, I know
that public networks have a lot more regulation than private networks
and it may be that regulation that makes a public network more trusted
than a private network. So, I wish to call that out as well. They have
to be separated. The difference between telecommunications and wireless
and the difference between a public network and a private network. Okay.

Now, this is the most interesting picture and its hard to see but, I've
tried to place wireless anywhere that appears to be a place in the VSS
that wireless may be used. It doesn't say it has to be used. So, this
figure shows many different arrows and the arrows are to be
representatives of wireless communications. Along with each arrow is a
piece of information that may be transported by that wireless
communication. As part of wireless usage, and securing wireless usage
you need to know what is being transmitted and what do you need to
secure.

So, in the simplest case, we have the keyboard and the mouse going to a
DRE or a vendor system to program either the ballots or the program
itself. How secure or insecure are those links? When you think of mouse
click, who needs anything from a mouse click? What information could
someone obtain by having a mouse click? The same is true for a keyboard.
Well, guess what, passwords are put on keyboards. How many wireless
keyboards are encrypted so that no one can intercept the password? A
vital piece of information. Also we talked this morning about whether
the system has a capability and whether we need to state that a person
makes sure that that capability is implemented and followed correctly. A
password is a perfect example. I can make a requirement that a system
has to have a password but how do I force the user to make sure that
that password is used? We all have the default enter and that's a
password. Its not very efficient for security. Wireless is no different.
We can have the capability but unless we have policy and procedure as
well, its not going to do it.

MALE SPEAKER 17: You can get software that you have to have
(undecipherable).

MALE SPEAKER 18: If we do that you put in software that the person puts
in two characters requiring eights, it won't accept it as a valid
password.

DAVID SIEFERT: Okay and now, \--

MALE SPEAKER 18: You can set up a password that's not long enough and
doesn't have the requirements that you want to have.

DAVID SIEFERT: Okay.

MALE SPEAKER 18: If software -- someone couldn't do a space bar and set
that up as a password, they just couldn't on those kinds of systems. You
couldn't do it on our system.

DAVID SIEFERT: Okay. Now, does your system also require that that
password has to be changed frequently?

MALE SPEAKER 18: Yes.

DAVID SIEFERT: And how frequently is that?

MALE SPEAKER 18: Well, that's a matter of our policy. I think a voting
system might have a different kind of policy because they don't run it
every day. We do require every month, every two months. It depends upon
the policy of the particular business and the sensitivity of the systems
that can be accessed and so forth. But that also can be enforced by
software. That's what it is. I go in there and I go to enter my password
and I automatically get a new screen. I have to reenter, select a new
password. When I go to select that password, if it's an old password it
won't accept. If its less than eight characters, it won't accept it, and
on and on and on. I could do trailing zeros. Its whatever they put in
that software control to sort of force me to put it in. That certainly
can be done. It is done.

DAVID SIEFERT: Okay.

MALE SPEAKER 18: I mean, we don't get certified unless we do it.

DAVID SIEFERT: All right. Well. Again, that is, I guess the question
there becomes on password, eight characters is not very sufficient --

MALE SPEAKER 18: I was just giving that as an example.

DAVID SIEFERT: prep.

MALE SPEAKER 18: Actually there is work going on at NIST right now in
this area in support of GSA and we're doing a study with them and they
are looking at what we would have to be to be certified as a certifying
authority for our customers to access their sites if we were to
authenticate it. They have a whole set of tests, that part of NIST as to
what we would have to do in order to be qualified as a certifying
authority. Part of it deals with this whole issue of passwords.

DAVID SIEFERT: Okay. What happens if you forget your password and you
are locked out? But anyhow. Okay. So, again, back with the examples
present. We just heard, prior to the break, about the verified paper
audit trail. If the printer needs to be added and there's no printer
port to the device it may be easier to implement that connection by a
wireless connection. I'm not saying do it but its another place where
its available. The only place that I can find that has any kind of a
requirement and its an if requirement, its conditional, is for the
access for handicapped. Its marked as a wireless coupling if you are
using a telephone handset and that is required in the VSS and that is a
wireless communication. So, there is no way to make a blanket statement,
you can't have wireless because there's a perfect example of where
wireless exists and would have to be supported if that device was done.

The public, back on the public telecommunication network there may be
wireless inside the public telecommunication network through microwave
communication or satellite communication that is impossible for us to
control. To add security features there, we have no authority to do so.
So, wireless, in some cases, and that's why I said it's a case by case
basis, we are not going to be able to provide you with any help in those
areas because its beyond our control as well.

Okay. I guess I'm doing okay on time. Now here are the numbers trying to
address the issue of all the possible wireless communications. I've been
participating at I Triple E land man working group and there are five
different wireless standards that they have published. There are seven
different physical layers that use wireless technology, infra red or
radio frequencies and they are building more. That's asking a lot just
from the I Triple E, if I just look at the I Triple E standards.

Go to cellular technology and I can count more than eight standards
available for cellular technology and as cellular technology grows and
the shrinkage in electronics and the advancement in the capacity of
those devices more and more things will be able to be done on a cell
phone. We'll have to call them P.C.'s too.

Now, the last statement, I have a visual graphic, I didn't put it in my
slide.

MALE SPEAKER 19: Does it -- you've included Blue Tooth and everything
under that top five?

DAVID SIEFERT: Blue Tooth is the lower layers of the Blue Tooth
specification were standardized by 80215.1 but the Blue Tooth technology
itself is an entire seven layer stacked protocol. It does not follow the
I Triple E structure for putting layers above it. It's a completely
defined self-contained package.

MALE SPEAKER 19: Right, but isn't that becoming more common for these
detached keyboards and headsets and many of these wireless connections?

DAVID SIEFERT: Right. Blue Tooth is getting more and more places.
Correct. There are Blue Tooth mice. There are Blue Tooth keyboards.
Where Blue Tooth is the initial penetration point was for the cell phone
because it was pushed by a cell phone manufacturer. They were doing
wireless headsets for that purpose. Wireless is being put in top of the
line cars now too. I don't know why but most of the time just to connect
your phone. Okay?

So, this is a frequency allocation spectrum and its very hard to read.
This is October 2003 from NTIA, our sister organization. This is just a
radio frequency spectrum and if you count all the little colors you have
well over five hundred possibilities. If only one technology was present
in those and I know they are not because I Triple E 802 sticks a lot of
technologies in one frequency band. This is not a doable project. Yes.

MALE SPEAKER 20: A couple of things. One is -- they also -- IR is used a
lot.

DAVID SIEFERT: Sorry?

MALE SPEAKER 20: IR.

DAVID SIEFERT: Yes. That's not in this radio frequency band.

MALE SPEAKER 20: Right and IRE is very directional. Its very hard for
someone to intercept. It would seem to me that if you specify something
like IR, with some encryption, if you are really paranoid, then you have
solved the problem of the wireless doesn't provide any security
problems. As far as the wireless in the public telecommunications
system, I bet that every single precinct is going to go to the
telecommunication system does eventually hit some wireless links, I
agree. I would doubt that there is a single precinct that doesn't have
that situation if they are communicating externally they don't have
wireless somewhere along those lines. So, the bottom line is if you are
sending sensitive information out of the physical voting precinct by
telecommunications, public telecommunications to some other location and
you think its sensitive its got to be encrypted.

DAVID SIEFERT: Right. Encrypted is the requirement but the level of --

MALE SPEAKER 20: By except for the issues of a spectrum such as IR being
much more directional and therefore won't radiate away and therefore be
harder for someone to intercept, connection keyboards and everything
else, the frequency is sort of irrelevant, isn't it?

DAVID SIEFERT: Not necessarily. My example, the example for encryption
is the I Triple E 802.15.4 which is a very, very low rate wireless
network and its designed to send very infrequent and very small pieces
of information and its to fit within 128 or 256 K of memory. You cannot
put encryption into that system. However, without knowing what is being
transmitted on these voting systems, if I would go back a slide, you
would see that there was a place, an ability to just send a signal of
open the poll or close the poll. That would not require very much data
information and technology would fit in that case. If it had to be
encrypted because you didn't want somebody to be able to open the polls
or close the polls out of when they should be, you've got to have
encryption to protect that and you can't.

MALE SPEAKER 20: Well, I think this is very good. If you can follow
along these lines, you know, consulting with the experts here, I think
you could probably begin to determine which ones need protection. For
example, uploading a ballot and a program, maybe that doesn't need
protection if I have some procedures and check points in the system
afterwards to ensure that, you know, hashes, checks and something there
that hasn't been modified in transit. There's no vote, there's no
secrets going on, so, the question would just be, what is happens,
what's the threat to each one of these and what could you do to
eliminate that threat and does it require encryption and then you could
go through it and you could determine where, if they wanted to use
wireless, the would have to have encryption and that.

DAVID SIEFERT: Correct.

MALE SPEAKER 20: That's your intent?

DAVID SIEFERT: That's my intent. The main missing piece is wireless
gives you access to the systems. So, the protection of the voting
information that we think of absolutely needs to be protected but we
also have to protect the machines themselves from having anybody
interfere with them while the wireless link is open. Yes.

DR. SMERGIAN: I thought there might be somebody --

MALE SPEAKER 21: Can I take a turn?

DR. SMERGIAN: Yes, go ahead Dr. Revest.

DR. REVEST: I just want to support (undecipherable) approach to that and
David's response. I think that sets sort of wireless is definitely just
a threat that is encompassed there and if you divide them into
transmitting and receiving the -- any transmitting from a voting system
the concern is the sensitive information that might be disclosed
typically and the reception of information is a possibility of modifying
behavior of the machine in a way that's not authorized.
(Undecipherable), you know individual signatures and so on if the
information is otherwise public. That may be appropriately handled. I
think the threat base now is the use of wireless (undecipherable) and I
support the momentum in that direction.

DAVID SIEFERT: Ms. Quisenberry.

MS. QUISENBERRY: I just want to make sure I understand what's in the
actual draft standards. So you actually have an appendix draft standard
which covers private networks. Could you just briefly go over where
thinking on those requirements?

DAVID SIEFERT: Okay. Section B is a direct copy of Section 6, the
security section with wireless placed separately because I do believe
wireless needs to be addressed separately as opposed to being hidden in
the current requirements. As was stated previously, we didn't know
whether we were going to rewrite the draft or make suggested changes so
annex B at the moment is a suggested addendum to be put in the security
section for wireless.

MS. QUISENBERRY: Could you just run down what you are saying in that
section? I can read it but I don't necessarily understand the
implications of the decisions you are making at this point.

DAVID SIEFERT: Okay. I guess I don't know what -- the main thing is I'm
making a distinction between private and public which is one of the
items I'm calling out. I'm also calling out that we are definitely using
wireless communication as opposed to telecommunications. Those are the
two main reasons for this. There are no specific additions. There was, I
think, a subtraction because it said something about a physical, you
needed physical security which doesn't exist in wireless communications.
If I had the other text I could do a line by line comparison and tell
you but those are the three items I can remember at the moment of what
I'm trying to do. The real requirements, specific wireless requirements,
are in Section 5 of the main document. Again, trying to do three
different views of wireless in systems both from a VSS, an election
process and then from a technology point of view needs to be coordinated
but at the moment three separate -- I need to address the specific
issues but all three sections must be coordinated and list the same
information. Annex B may or may not be present depending on which
direction we go with the document. I was trying to cover all points of
view and say, okay, here's what it would look like if we did it this
way. Here's what it would look like if we did it another way. Yes, there
is information in Section 3 that's not in 5 or its in 4, its not in 5.
They all have to be coordinated. They all contain the same information.

DR. SMERGIAN: David, are you finished with your presentation or did you
have --

DAVID SIEFERT: I was finished with my presentation, yeah. My last slide
was the numbers.

DR. SMERGIAN: Then, Mr. Kraft, then Mr. Berger and them Dr. Williams.

MR. KRAFT: Okay. David, I think I heard you say that number 1 we are
using wireless. You can't go down the road of prohibiting the use of
wireless and two, its almost impossible to conceptualize and bring into
a measurable standard that will address every case and time is short so,
I guess, what do you recommend to the committee that we do to pursue
this issue. I guess one of the things that we have done in the past in
dealing with telecommunication and networking issues is actually address
which data can be transmitted. For example, there was a prohibition in
the 1990 FEC standards against unstipulated ballot data being
telecommunicated before it was, you know, tabulated and before there was
some controls. Where are you going with that?

DAVID SIEFERT: Well, I guess, from what I understood you to say, you
want me to tell you what pieces of information would be distributed
using wireless technology.

MR. KRAFT: Well, no, I was kind of giving that as a suggestion as one
alternative to trying to get our arms around standards for encryption
and protocols and all these different wireless technologies. One of many
options is actually evaluating what pieces of information or what level
of information risk you could take within the systems. I guess what I'm
asking is are you looking at other avenues to try to get some sort of a
measurable standard built around this? Are we kind of at the point where
we are adrift?

DAVID SIEFERT: Well, I guess, again, this is why I left this picture
here. This is where my view is. This is my understanding of the
information that may be distributed and therefore that's what I need to
examine for threat. Again, back to the questions of having everything
encrypted, that is not a possible statement either because, again, back
to the handicapped situation. That wireless technology cannot be
encrypted. It is at the voice level and therefore if you encrypt it I
can't hear it. So, having encryption required on every wireless link
doesn't work either. I guess, yes, I'm at the point where I don't know
where to go either. I can continue to do these analyses.

The other issue that was spoken of earlier, the upload of the program.
There is a requirement that that program has to be on right one media
which means it could never be -- that program could never be transmitted
over wireless. So, one may go away simply because of an existing
requirement of the information cannot be distributed in that manner.
That one may be restricted because of another requirement in the spec.
It really is going to be on a case by case basis and these are the links
that I'm going to have to try to address.

The printer point of view -- going to the printer I don't have to worry
about information necessarily coming back to me. Its an output device
although I don't want anybody to intercept the ballot. If they transmit
from voting precinct to the main voting location that information is a
tally vote and I need to encrypt it but if somebody knows the numbers
who cares as long as they don't modify it. The modification of those
numbers is really what's important in an election.

DR. KRAFT: Do you, or have you had good reliable information from the
industry in this process on what data they are actually moving because
some of the things in this model, while its very good, are very
actually, you know, somewhat hypothetical but I was wondering in keeping
with EACH's interest and I think, mandating the best of current
technology nationwide, perhaps we could look and get some cooperation
from the industry in disclosing exactly what data they are transmitting.

DR. SMERGIAN: That's a good suggestion. It could minimize your work if
you know what they are physically doing. One point in the handset area,
I'm not saying you should go to that expense but there is an older
technology that we used in the analog communications world which did
voice scrambling, frequency scrambling and them B scrambling can be used
to make it very difficult for someone to intercept and understand what's
being said. You can do that.

DAVID SIEFERT: But you are correct but in the application its designed
for that cannot be used. Its going to the hearing aid and unless I put
the encryption in the hearing aid as well, it doesn't work.

DR. SMERGIAN: Right. That's what you would have to do. You would have to
have a specially designed -- I'm not saying its worth it but it can be
done.

DAVID SIEFERT: Well, that's also an example of where what's the risk?
How feasible is it that someone is going to get between the loop and the
hearing aid.

MALE SPEAKER 22: I agree with you. For efficiency purposes I think you
ought to just scan this stuff and say this link's not a risk for these
reasons. You don't have to worry about it. This link isn't really being
used or contemplated to be used for wireless. Don't worry about that
one. You can get them down to a manageable number to work with.

DAVID SIEFERT: But I'm hearing immediate help from us are the industry.

MALE SPEAKER 23: In terms of the task here, David I saw you
(undecipherable) a big job here and to get started on. I think that the
committee is giving you good advice here on how to proceed. We have a
Resolution that says that wireless poses a severe risk and should be
approached with extreme caution. I think that the task is,
(undecipherable) the thoughts of no wireless and any wireless and
wireless where it is encrypted and they are trying to work on an
exception. But I think that the approach that's being suggested are
reasonable points for possible exception if you can identify and try to
see if you can (undecipherable). Its not a question of trying to cover
the whole base but just the pole position.

DAVID SIEFERT: Inherently very risky and a manufacturer that wants to
use wireless for this better be prepared to justify it very carefully
and show many of the risks that go along with it and to the extent,
excuse me, you can identify closed applications where wireless really
was justified we can standardize on those ahead of time. That would be
the right thing to do. A hand full of those where wireless was justified
either because there is already mitigating technology in place or
because there is such (undecipherable) it is really necessary or for
some other reason where the data is public and protection is probably an
issue. That's when we can standardize on those.

DR. REVEST: We understand that you might be able to make a case for
exception to handle the risk but I think we are trying to stop it. The
original wording of the Resolution ways that wireless does introduce
threats and must be approached with extreme caution. The thought here is
whether the (undecipherable) are exceptional. We've got the general
(undecipherable).

DR. SMERGIAN: Thank you Dr. Revest, Mr. Berger.

MR. BERGER: A couple of comments. Let me start with a little history on
the hearing aid compatibility issue because I think it points to
something important in this issue.

In the old Western Electric telephone days, receivers put out a magnetic
flux leakage. It was jut a leakage that happened naturally and someone
discovered that to be useful in coupling into hearing aids and help
people with hearing aids get better reception. As technology changed in
1988 the Congress mandated that that be continued and that's an FCC
requirement to this day and that's what you are talking about.

I think the important point is that wireless is always present over
every connection if there are electrons flowing, currents flowing there
is a wireless emanation. It is just a matter of degree at that point and
from a security standpoint and I know there is a lot of people in this
who know a lot more about this than I do, if all these wireless
connections are wired, all you've done is change the degree to which
someone can monitor what's going on. To that degree, I think, the
comments that each link needs to be analyzed as to what's the risk and
are those risks appropriately mitigated is real important.

I think there is some general technology trends that will help circle
this because to a large degree the wireless technology is irrelevant.
Let me go to your cloud there. Its not generally understood for those
who don't spend a lot of time in it but since the mid-90\'s there's been
a huge trend away from circuit switch traditional telephone system to
packet switch, to an IP based data transmission, particularly data
transmission. Its almost impossible to send data, wireless or wired, now
through the public network. Its not IP based. In some ways I think the
question is not wireless but is it possible to set up a packet switched
IP based data exchange that's sufficiently secure for our purposes. The
transport at that point is almost irrelevant.

Actually I had an interesting discussion with the chair of 802 and I
think his advice was germane to this work. He said, you ought to assume
wire to wireless that someone can tap into that wire and if you are
satisfied that you have appropriate security, you are okay.

MALE SPEAKER 25: I'm missing your point. Number one, I don't understand
the different between whether I'm tapping into an IP stream or any other
kind of stream. I still can have equipment that can read it. Your other
comment about the emissions, the biggest threat there if you don't
address it is you have emissions coming out from the screen which I can
pick up and replicate copies of every screen shot and I can demonstrate
the equipment that can do that. Its relatively inexpensive. So, if you
are really worried about capturing votes and so forth like that, and I
agree with you, it doesn't matter whether its wired or wireless, then
you would have to talk about either some design of the equipment that
its got emission controls or the cubicle in which you put that has some
kind of a screen in it that would prevent the radiation outside of that
screen. That wouldn't match some of the pictures I've seen. In some of
the precincts we have these PCs in open areas like that and it would
almost require you to have some kind of a ---

MR. BERGER: No. I think that's a great illustration. The only one I'd
add to it is, if all those wireless links are wired, the quality of the
shielding of the wire and all those could be very easy to pick up in a
similar manner.

I think the point is and I think you made it well is, what's the risk
and what degree of mitigation is appropriate, you know, to the risk.

DR. SMERGIAN: Dr. Williams.

DR. WILLIAMS: Yeah, this is a little too theoretical to be useful in the
short term. We need to look at how voting systems are using wireless
technologies and right now what happens is that the primary use of
wireless technology in a voting system right now is to program voting
stations, to load ballot definitions into voting stations. If I'm Fulton
and I've got three thousand voting stations and I'm having to load those
with PC cards I've got to sit down \--

(START OF TAPE 2). I'm in the precinct and I want to, its time to open
the polls and I've got anywhere from ten to thirty voting stations
sitting there. Its very convenient to be able to sit down at a console
and wirelessly open all thirty of those voting stations. The same thing
with closing the poll at the end of the day. Now, during the day that
wireless capability is not a threat because you turn it off. So, under
what circumstances can we use that and then if the close of the polls,
if I want to use that wireless facility to do my roll up in the
precinct. These are the issues; these are the questions that local
voting officials are asking us to address for them. They have systems
out there that are capable of doing that. They want to know what are the
risks if I use that capability and if I do choose to use it what
safeguards can I reasonably put in place. Whether my mouse communicates
wireless, I could care less or my keyboard or my printer. These are not
the issues. These are voting systems and the issues are transmitting
ballot images and opening and closing polls and transmitting election
results at the end of the day.

MALE SPEAKER 26: Let me give some thoughts on that for a second then. I
think then you do, let's do a top level kind of threat analysis. Let's
say one of the primary uses is in the administration and loading and so
forth. I think you might want to spend some time to address how we can
do that wirelessly and how to have the checks and balances so that
initial set up and close down. That's important to do. I think I would
tend to agree that those things that are just output types of devices
and those wireless (undecipherable) aren't really that much the issue
because, frankly, as we had that discussion, your biggest threat there
is privacy and knowing who voted what and there you would really have to
be most concerned about my running around with some box in my pocket
that could pick up the emissions from the CRT which is a very liable
type of thing to do. So the only other thing I think you would want to
protect on those links where you'd think people are actually using it is
on the input or transfer side because it is possible then. Let's say
that I could manipulate that DRE, you know, going to the counting
system, modify and insert, you know, what I saw and voted, what actually
was done. So, if you were to use wireless there, but on the other hand
if it turns out, so you might show that threat and if no one is
currently using it in that area then we might just say, it is
recommended not to use wireless in that area. If someone ever wants to
do that it would have to go through a whole threat analysis. Just focus
on the areas where they find it administratively efficient and what are
the procedures and the checks and safeguards to insure that.

DR. WILLIAMS: These systems, you have to have the capability of allowing
the poll manager, the poll workers to wirelessly activate voting
stations for voters. Nobody that I know of uses that capability because
you don't want to sit there eight hours or twelve hours with your polls
open with that wireless capability hot so they all use the physical
cards to activate voting stations. So, the people that use this and we
have some of them sitting in the audience here today, are concerned
about this wireless capability and the way they are using it and would
like guidance on the problems that they are dealing with, not the
theoretical problems that might happen in the next generation of
systems.

DR. SMERGIAN: What I'm hearing is that I think the guidance you are
getting is, you don't need to address all the possible permutations, the
possibilities but rather the first order lets look at the processes or
applications that are either being used now or where election officials
would like to use but they are concerned about the potential risks. So,
focus on those specific applications either current or potentially
imminent, you know applications and not address the broader very large
domain of possible applications. I think that a very constructive
guidance. It certainly limits the domain that we have to worry about and
make the job, hopefully, more manageable. Do we, I'm not the specialist
here, so I'm trying to sort of capture the general feeling here.

DR. WILLIAMS: We'd be happy to put you in contact with some of the
vendors that have these wireless systems. That way you can go look at
them and get comfortable with how they are capable of operating and how
they should be operated.

SECRETARY DAVIDSON: Or we could get on with getting the systems in house
for NIST to work with.

FEMALE SPEAKER 12: Sorry, we are on this nine month battle to get access
to systems for the experts at NIST so, --

DR. SMERGIAN: They would be available after nine months but -- I think
the concern here is clearly there is a huge, you know, domain out there
that you could look at and you could spend the next ten years looking at
them and that's not what the committee has in mind.

DR. WILLIAMS: We've got a group from Virginia sitting right here that
has one of those wireless systems. I'm sure they would be happy to have
you come down and go through how they use that system and look at it and
make recommendations and suggestions to them.

DR. SMERGIAN: Mr. Berger.

MR. BERGER: Well, I'm in complete agreement with what's being said here
and I would just like to highlight one point that I thinks been said but
very important. That is simply what constitutes acceptable turning off
of a wireless system. Is a software switch enough? Do you have to remove
the whole module? A simple question but I think its very important.

DR. WILLIAMS: These are the issues I think we can deal with in the time
frame that we are talking about.

MR. BERGER: Also, at the mention of the unavailable of equipment to NIST
I've made eye contact with some vendor representatives here who looked a
little confused. I think perhaps you could get with some of the vendor
representatives who are in the audience and go ahead and start making
some arrangements today. I'm see some nods.

DR. SMERGIAN: Any other questions, especially from the folks on the
phone?

DR. WILLIAMS: Let me comment on this a little further. Some of us have
got some leverage with the vendors. If you will let us know which
systems you would like to see we maybe can help you with that.

DR. SMERGIAN: I'm sure we all appreciate the help. Dr. Revest did you
have any other comments?

DR. REVEST: No. I think that direction here is a good one to focus in on
those near term things to the extent that we can have a vendor input as
to which application area they have seen the most pressing to them. That
would be helpful.

DR. SMERGIAN: Well, I consider these comments certainly very
constructive but also clarifying in nature. I don't really consider
these as a change in our direction. I think I consider these as focusing
on the more limited set of work products and outputs and I don't really
see the need for a Resolution or any kind of vote. I think that there
is, the committee seems to be in general agreement that this is an
important topic but we don't need to solve the world's problems. We need
to address the immediate issues of concern that are for systems that are
being used now and perhaps some of the things that may be coming down
the pike soon. I certainly consider that as a very constructive comment
and unless we have other questions or comments I think we'll sort of
take those comments to heart and proceed from there.

Thank you David.

DAVID SIEFERT: Thank you.

DR. SMERGIAN: So, let me go through my official process here. Certainly
NIST believes the preceding preliminary report of the technical support
titled "NIST Approach to Wireless Requirements for the Augmented 2002
VSS, Response to TGDC Resolution 35-05\". Unless there are supplemental
directions or corrections the technical support and related work product
will continue to be developed consistent with this preliminary report
and I should add with the comments made right now. Are there any further
questions, directions or corrections? I think this was a very productive
discussion. Are there any other questions or comments? If not, we'll
proceed to the next presentation and at this time I call on Nelson
Hastings of the NIST ITL to present a preliminary report on NIST
Approach to Software Distribution and Set up Validation for the
Augmented 2002 VSS. Nelson.

NELSON HASTINGS: Thank you. My presentation is going to be broken up
into two parts specifically addressing the Resolution on software
distribution and set up validation. Due to the time constraints we have
today this presentation will give you just highlights of the goals and
methodologies that we are thinking of using as well as the scope that
we've come to on those Resolutions as well as some sample draft
recommendations. At the end of the presentation I will open up the
floor, obviously, for discussion.

The goal for software distribution is to ensure that the software has
been distributed without modifications. The approach that we are going
to take to that is to develop reference information to that can be used
to check that the software hasn't been modified. That can take the form
of binary images, just to a binary to binary comparison of that software
as well as you could use hash values to do a comparison of the hash that
generated of the system of the software being inspected and that type of
technology is being used in the National Software Reference Library here
at NIST in their work with law enforcement. The last technology is
digital signatures which is used in the commercial space when they do
signed codes such as signed JAVA applets.

There are different places or different sources of this reference
information. Vendors and independent testing authorities supply
reference information in regard to the parts of the voting system
software that don't change based on where its installed or the election
that's being conducted whereas jurisdictions may be \-- would be able to
provide the reference information on the software that changes on
election day, that kind of thing. I think Georgia does that in the
program that they use. The National Software Reference Library could
generate whatever reference information for whatever software they
receive from the vendors, the ITAs or the jurisdictions themselves.

The scope is to look at all executable code and the associated
configuration files irregardless of whether its at the polling place,
the central counting system, or election management system. Included in
this is third-party software such as operating systems and drivers that
are critical to the proper operation of the voting system. However,
determining whether the software is operating correctly is beyond the
scope of this Resolution. It actually will be addressed by another
Resolution.

Some sample recommendations that we've put together. The first one is
related to a requirement found in the current VSS where it says that
ITAs shall witness the build of the executable that would be used to go
under test. This extends that and just says that once that system has
been qualified the ITAs will witness the final build of that system.

The second requirement there is directly from the Resolution itself that
basically says that right once media will be used to distribute the
software, the binary software.

The third requirement is basically just talking about labeling of the
write once media so that you can have traceability back to the
authoritative source that created it. There is a requirement in the VSS
something about product marking, marking products for identification.

The next three recommendations are actually related to the I Triple E
voting standard, specifically the section on software and firmware
installation. However, these requirements make, refine the requirements
found in the I Triple E standard and give it a crisper requirement.

The first one basically just says that Phipps approved functions can be
used to generate reference information.

The second one basically says a Phipps approved digital signature skink
will be used to generate reference information for digital signatures.

The final requirement calls for a Phipps 140 level one validated
cryptographic module be used to general hash values and digital
signatures that's used for reference information. By imposing this
requirement you bring in the testing that's done to get a cryptographic
module validated which insured that the cryptography is operating
properly as well as some other security features such as key management.

The goal of set up validation is to insure that unauthorized software is
not on the system. Authorized software is on the system and that the
system is in an initial proper state before it is being used. The
approach that we are going to take here is to develop items that need to
be inspected to determine that the voting machine is in the proper
state. These items not only include software but registers, the
accumulation registers that need to be checked that they are zeroed
before being used.

The method will be to leverage off the requirements of the software
distribution. It will be more process oriented or guidelines or best
practices for election officials and will probably manifest itself
mostly in the vendor documentation that's provide. However, there may be
some additional technical requirements. Because the software
distribution and set up validation are so heavily related to each other,
those two documents that you received previously will probably be merged
into one document for the final deliverable.

The set up validation is going to focus on the software itself. Once
again, regardless of whether it's a polling place system, a central
counting system or election management system, general hardware, voting
system hardware set up will not be included in this scope so you won't
see things such as one needs to verify that the voting system is plugged
into a proper back up power supply or that the voting system's screen is
set to the appropriate contrast level. So, those types of things are
beyond the scope of what this, what we are producing here.

Also beyond the scope is how the authorized software is installed on the
system. How unauthorized software is prevented from being installed on
the system and how the voting system reaches its initial state. All
those are very important in the assurance of the voting system. However,
its beyond the scope of this specific Resolution.

Here is some sample draft requirements that we have been working on. The
first one is a documentation requirement that basically says that
vendors need to identify all the software required for the system to run
properly including the software that needs to be modified by
jurisdictions. The second requirement talks about obtaining reference
information for that software and the final one says that jurisdictions
will use that reference information to verify that the system software
hasn't been modified.

These last three requirements are embedded are related to a section of
the VSS on, I think its called validation at the polling station and
validation at the central counting facility. Basically there is a
requirement in there that says that you need to check registers,
accumulation registers are zeroed and things like this. These
requirements just extend that concept to all registers that are used and
all variables that may be used in the voting system so that you can make
sure that those values are correct.

The first one talks about vendors documenting the variables that they
control. The second requirement says that jurisdictions shall document
the variables that they set for given elections and the final
requirement says that the jurisdictions will use the documented values
and check them on the system. So, this kind of gives you a flavor of the
types of requirements that we are looking at to put towards software
distribution and validation set up. At this point I am opening up for
discussion and feedback.

DR. SMERGIAN: Thank you, Nelson. Any questions or comments? That sounds
like time to sit down.

NELSON HASTINGS: Okay. I will take that. Thank you.

DR. SMERGIAN: Seriously, are there any questions. Dr. Revest, did you
have any comments?

DR. REVEST: (Too far from microphone to hear what he was saying).

DR. SMERGIAN: Well, not hearing any other questions I believe that, we
believe the preceding preliminary report of technical support titled
"NIST Approach to Software Distribution and Set Up Validation for the
Augmented 2002 VSS Response to TGDC Resolutions 15-05 and 16-05\". This
is your last chance. Unless there are supplemental directions or
corrections, the technical support and related work product will
continue to be developed consistent with this preliminary report. Any
other questions or comments? Hearing none. Thank you. We will proceed
and at this time I call on Lynn Rosenthal of our information technology
laboratory to present a preliminary report on Interim Conformance Clause
and Glossary for the Augmented 2002 VSS. Lynn.

LYNN ROSENTHAL: Thank you. Moving right along I thought I would be up
against lunch and have to do this very quickly. Now you've given me way
too much time. So, we'll finish early. I'm just going to give a very
highlight view of the conformance clause and then talk a little bit
about our glossary and for your pleasure Mark Spoce (sic) is going to be
my driver and we are actually going to show you a very early prototype
of the glossary. So, lets get started.

The conformance clause is Resolution number 24 where we were instructed
to draft a conformance clause for the VSS. The way we are approaching
this actually to have two conformance clauses. One for this April
deliverable of the Augmented 2002 VSS. That's being called the interim
conformance clause and then after April we will start working on the
more permanent performance clause that would go in this redesigned
standard.

Let me talk a little bit more about the interim conformance clause which
you have a copy of in your document folder. The idea there is that there
currently is no conformance clause in the VSS and just to remind you
what a conformance clause is, it's a very high level statement or
paragraphs or section that talks about who needs to conform, what needs
to conform, what defines conformance, how is it defined, how is it
specified. It defines any of the conditions, constraints, anything that
would be applicable to the voting systems, the tech labs, the vendor
with respect to conformance. Basically, what do you have to do and how
do you go about doing it. The conformance clause talks about the minimum
requirements. These are functional requirements, performance
requirements, document test evaluation and procedural requirements. Now
these procedural requirements, as we well know, is a different animal.
They may be requirements that have to do with the set up or the
operation of the equipment that would be in the documentation from the
vendor or it may be types of requirements that actually end up as a best
practice that the states would be able to adopt as they saw fit. It
would relate to aspects of the equipment, the setting up, the
maintenance, etc. In fact, the requirements would be and in fact the
conformance clause talks about requirements for those who design,
manufacture and sell voting systems, those test entities who do the
qualification testing or the certification at the Federal level of the
VSS.

DR. SMERGIAN: I think some people are trying to find your presentation.
They are actually at the end, the right side of your book. Sorry.

MALE SPEAKER 27: So, if I understand correctly what you are going to do
is just. We are going to be at the functional requirements level as
opposed to the how, the detailed design level that we are going to test.

LYNN ROSENTHAL: We are not going to get into -- the interim conformance
clause is going to take what the current VSS already says plus all the
new additions and changes. Its going to talk about, you know, how do you
put all that together and which applies to me, you know, what do I have
to concern myself with depending on which role I'm in. Some of those
requirements that are currently in the VSS are functional, some are
design and some are requirements on the testing people. So, it's a mixed
bag.

MALE SPEAKER 27: That (undecipherable) that it had some design ones but
I didn't see design but you really threw design in those cases where the
current spec actually goes into design.

LYNN ROSENTHAL: Correct. Thank you for the clarification. So, the other
people who may want to be using the document and that may be addressed
by the conformance clause as well would be other testing entities. For
instance, at the state level.

Then, finally as Mark indicated this morning, voting officials, we are
not quite sure what the correct term would be to incorporate all,
whether its poll workers, voting officials, election judges, those who
are in the warehouses that are technicians so we came up with just this
general name to include all of those people.

I'm going to move on to the glossary unless somebody wants to talk a
little bit more about conformance clause.

MALE SPEAKER 28: Lynn, I have a question. When you are talking about
coming up with definitions for conformance and you are primarily
referring to the voting system standards which my understanding will, in
some cases, reference other standards that come from other
organizations, whether its I Triple E or anyone else, is there a level
of criteria that -- so if you are referencing a specification from
another organization that they have achieved a certain level of
specificity in how they define conformance. Is that part of this, you
know, for any reference standard?

LYNN ROSENTHAL: To the best of my knowledge the VSS doesn't do that
right now. That is a good point in that within the performance clause we
do need to address those normative documents, those normative standards
and what role they would play when it comes to implementing the
requirements that are being imposed on a system let's say. That will be
something that we will need to consider. That's a good point. At this,
for the interim performance clause, I believe its not going to be a
question unless I'm missing something from my reading of the current
VSS. In a sense the conformance clause right now is fluid because we
can't put anything in concrete until we know what all the pieces are
going to be and how we are going to want to provide, if you will,
guidance on which piece is applied to which role and how they apply
whether they are mandatory requirements or whether they are optional
requirements or whether it's a good practice. Those are the things that
the performance clause will take care of and will be done for the April
document.

Moving to the glossary this is Resolution -- Yes, Ron?

RON: I just want to make sure I understand with respect to security
requirements and so on. We talked about what kind of (undecipherable)
where a vulnerability may be discovered after a system has been
qualified and so on. Does the conformance clause address issues like
that of situations where a security vulnerability is discovered?

LYNN ROSENTHAL: After the fact?

RON: Yes, I mean is it, I'm trying to figure out how to use the
terminology here. I suppose for conforming the system is on that, you
know, meets the requirements then its discovered not to meet after.
Security is funny that way because you could discover that, my goodness,
its broadcasting everybody's vote on radio if we didn't know that or
something.

LYNN ROSENTHAL: Typically the conformance clause does not address what
happens after a system has been tested or a new problem has arisen. Its
used to just identify what are the requirements that apply and at what
level do they apply, whether they are mandatory or an optional
requirement and if its optional and you chose to pick that option you
must do it correctly. What I think you are talking about has to do with
the testing and that aspect of once something is tested and determined
to be certified or qualified, then how do you go about decertifying or
disqualifying that system now that a vulnerability has been found. That
is not part of what goes on in the conformance clause. That is a very
important aspect that will be addressed in the, I would believe,
probably by the testing labs and the requirements for doing the testing.

DR. SMERGIAN: Yes and part of it could probably be addressed in the
maintenance and updating, how do you do that? That's really what happens
where you have a system, vulnerabilities get discovered, patches
eventually get addressed and from what I gather that's a particular
sensitive thing because you don't necessarily want to do an update of a
patch right as you get started to do an election but so there are some
thoughts behind it.

MALE SPEAKER 29: Yeah and I think in the larger scope that is part of an
intuitive process. There are risks that are discovered. There are
anomalies in systems that are not discovered in testing. On the other
side of the coin there are also new functionalities that election
officials and state legislators demand of systems. So, all of these
systems evolve as they go through time.

LYNN ROSENTHAL: Barbara just pointed out there are two Resolutions,
number 31 which is the maintenance of the VSS and 32 which is sharing
information and disqualification of voting systems. I think what we are
talking about now would be addressed under those Resolutions. I think,
Paul, you bring up an excellent point is that is with conformance, with
standards, with testing, it is a cycle in that which one feeds the other
and as things are found out you have to feed it back and maybe change
something or add something to the standard. Maybe change your testing so
that it does so in a sort of intuitive feed back loop here.

MALE SPEAKER 30: So to summarize I do agree that that's where it belongs
in those two sections and you know, just some thought on how we handle
that sort of thing which I think might be something you might consider
how you do because we also have to be pragmatic about when we do updates
and when we don't do updates. What we normally do is we \--. Normally we
absolutely want to know when a vulnerability gets discovered. We leave
it up to that business or in this case, the election district, to
determine when they are going to install the patch and what they are
going to do in terms of mitigation and remediation considering the fact
that they know they might be going into a system that isn't necessarily
entirely patched with all the vulnerabilities. So, the vulnerability
exists and there is no available patch or its not an appropriate time to
do a patch then they have to acknowledge that there is a risk and what
they are doing to compensate for that risk, operation procedurally and
then we also have procedures on how you do patches and what the order of
priority is and, you know, all those kinds of things. So, I'm sure you
have guidance like that so you might want to put some of that in those
appropriate sections subject to review, of course.

DR. SMERGIAN: Mr. Berger and then Secretary Davidson.

MR. BERGER: I would just like to highlight. We may want to consider as a
policy issue whether conformity and certification should be time
limited. That's being considered in some other arenas and it offers some
unique advantages if you know that a system is certified for some period
of time and then it has to be renewed and then that gives you an assured
checkpoint where you can make sure security patches are in, that sort of
thing. It's a larger policy issue but I think its worth considering at
some point.

MALE SPEAKER 30: The only problem with that it sometimes is not related
strictly to time. What happens is a vendor comes in. They put in an
application or some software. They continue to upgrade in different
versions and at some point they no longer support it. Its not longer to
be maintained. That's probably when you really want to consider retiring
that system. Exactly when that point is reached is quite variable
depending upon the vendor and the system and so forth.

MR. BERGER: In some of the radio work that's going on, one of the
advantages of an assured time limit is that you can take certain system
risks if you know that in two years this piece of equipment has to get a
re-authorization to continue to operate. So if any vulnerabilities get
discovered, any problems, you know, basically an upgrade can be required
at that point in time.

LYNN ROSENTHAL: Those are all things that are typically done in the
testing procedures and they are often done in both those ways in the
sense of a time in the sense of not necessarily that you have to get
retested but it has to come up for review to determine whether or not to
renew the certification and also when equipment is changed and I believe
its already in place, any time the system is changed it has to be
retested.

MARK: I just want to make it clear because it sounds like there is some
confusion. What the conformance clause does is define, excuse me,
instructions for what it takes for implementations to conform. This is
related to but separate than the way of proving conformance. One way of
proving conformance is through testing. There are other ways, self
declaration. So what Lynn is talking about are instructions. Much of the
discussion, a really good discussion about testing and things like that.
They are really outside the scope of the conformance clause.

MR. BERGER: I just wanted to make sure everyone understands it.

MALE SPEAKER 31: We recognize that. We were just digressing into those.

LYNN ROSENTHAL: Moving on to the other work products that we --

DR. SMERGIAN: Secretary Davidson.

MALE SPEAKER 32: May I ask a quick question? I'm still confused a little
bit about how things change. So if we talk a little bit about what if
the system changes say in the perception of security but when the
standards are changing and this is very relevant to our work now. If we
introduce new requirements that all systems must have a Future X then
existing systems that have been qualified already and don't have Future
X, does the conformance clause direct the issue of transition,
grandfathering (sic) and things like that?

LYNN ROSENTHAL: No. How's that for shorthand? No but the testing
procedures or the exceptions procedures that either the test labs or the
states would have would probably address that of when it needs to be
retested and how do you handle the phasing in and the grandfathering in
of systems.

The conformance clause is very specific and as Mark said it just has to
do with what are those requirements that are in this standard and who do
they apply to.

DR. SMERGIAN: Secretary Davidson.

SECRETARY DAVIDSON: You know, the only thing I want to say and it goes
to back to understanding the whole, you know, testing part of the
equipment and testing it before an election. As Paul said there is many
things that take place at the very end of a process that changes and you
have to change your whole program. For instance, a deceased candidate
and you can't count the votes. So you have to change your programming
and the testing and that type of thing at the very last minute or if a
candidate withdraws or a judge says you can't count something. So, I
think that's the other thing. I think we need to make sure that they do
proper testing at those times that something like that happens at the
last minute.

LYNN ROSENTHAL: I think these are all excellent point. I'll move on to
the glossary.

This is Resolution number 33 and this was to develop a glossary of terms
for the VSS and, in fact, its broader in scope that just the terms that
are in the VSS. It includes additional terms that are needed to
understand what voting's all about, what security is about, human
factors and testing. Right now what we are doing is we are just
collecting terms from various sources from the VSS, for the I Triple E
standards, from HAVA, from ISO, from various NIST publications, even
looking at what some of the states have defined in their standards and
legislation and using that. We are also, if you will allow me, inventing
a few terms as we need them as we are doing our research and discovering
that we have to define some of these terms and how does it most make
sense for the way we are using that term.

Once we get these terms our next step is to look at them, revise some if
need be, harmonize the various terms when you get more or less a similar
one from two different sources and they are close but they are not the
same, which one do you use and go from there.

Some highlights with the glossary. We've added some new terms related to
security, to human factors, terms related to the voter verified paper
audit trail. As Sharon point out, definitions for privacy,
accessibility, usability. The term conformity assessment which comes
from our NAV lab area.

Additionally, each term is more than just the term and a definition.
We've tried to indicate the source either that the term was extracted
directly from or is based on. We've added something called an
association. This is what is this term relevant to. Is it a voting term?
Is it a testing term? Is it an accessibility term? We also have added
the See also when there is related terms or other terms that are used in
either correctly or incorrectly in place of this term.

We are discovering lots of questions, lots of areas that we need to look
further into. For instance, what is a ballot? We are finding lots of
different ways of describing a ballot and needs some information from
Florida. Even there they have different definitions for what determines
a ballot. So we need to come up with an appropriate general definition
followed by any of the specifics.

Another area is clarification on terms like accreditation,
qualification, certification. These are not even used consistently
between HAVA, between what's currently in the VSS, between what the NAV
lab people here at NIST typically use. So, we need to come up with
either better definitions, maybe deprecate (sic) some of the current
definitions and replace them with new ones as we go into accredited
laboratories that are no longer being called ITAs but are now called
DSTLs, so here we have a transition. We are looking at all of those and
that will be in the glossary as well.

So, next step obviously review and revise the definitions. We keep
adding new terms. I'm a little bit behind in the sense that I have a
stack of terms that need to be added but they are not in there. Just to
give you a preview, we are developing a web version of this glossary and
Mark, you can just go there. The web address that you have on the slide
doesn't work yet. Its an alias that I'm going to try and see if they can
get it working by next week at the latest. But I did this morning for
those on the phone; I sent the real URL this morning to the TGDC list so
you should be able to get it.

This is a very early rough prototype. Not everything works and its not
fully populated but just to give you a view of the what it looks like
you can get the full glossary. You can jump to specific terms. Lets say
you want to see, you know, all the terms starting with B. Rather than
paging down, you can see there is an area that says "also see" and lists
the various other related terms. It has the source. It has association
of what its to. Now, lets say you don't want to look at the full
glossary, you want a different view. You can go over to the list of
sources on the side and say, give me all the terms that were extracted
or based on the I Triple E standard and you will get those, or you say,
give me all the terms that have to do with human factors accessibility.
You can go and you can pull out all those terms. So you can grab it from
different ways. As I said this is a very early prototype to just give
you a view of what we would like to have eventually. We will make sure
that its in XML, so its very easy to make sure that this is accessible,
has all the right ways of being presented and right now we are just
trying to work out the infrastructure behind it so that we can grab the
pieces we need to grab.

MALE SPEAKER 32: Two things.

LYNN ROSENTHAL: You're not allowed to talk.

MALE SPEAKER 32: Accessibility is not spelled right.

LYNN ROSENTHAL: Thank you. As I say, very rough, but thank you. We will
correct that. And thank you to my wonderful driver over here. Any
questions on the glossary?

DR. SMERGIAN: I have to go through my formal procedure here. So, NIST
believes --

MALE SPEAKER 33: Question.

DR. SMERGIAN: Yeah, I'll get there in a minute. NIST believes the
preceding preliminary report of technical support titled "NIST Interim
Conformance Clause and Glossary for the Augments 2002 VSS Response to
TGDC Resolutions 24.05 and 33.05\". So, at this time I open the
question. One thing I wanted to point out, all of the morning
presentations were related to the augmented 2002 VSS. So the ones in the
afternoon will be associated with the redesign 2002. So, the fact that
there is general agreement and I hope that will be the case for this one
also, I think puts us into a very good position to complete these tasks
by April. I didn't hearing any major change in direction. I think that's
very good. We are now open for questions.

LYNN ROSENTHAL: Thank you.

MALE SPEAKER 34: Lynn, on the glossary, this is excellent. I think it
will be very helpful. Question kind of on the scope of the terms. Most
of these appear to be more procedural terms. Will you be getting into
actually listing in there data elements? For instance, if there is a
record transfer between systems, the actual data elements, definitions
of those data elements.

LYNN ROSENTHAL: No. I don't believe so. If I understand the question
correctly. I'm open to put whatever term in the glossary that people
need to see in the glossary. So, the answer is "no" but, you know, its
fuzzy. If there is a terms that needs to be defined in order to relay an
understanding related to voting or related to the voting standards,
related to the testing of these according to the standards, etc., then
we will put the term in or if its just a general term that needs to be
understood in this community then I don't have a problem putting it in.

MALE SPEAKER 34: I guess, maybe an example is, if you are, if there is a
definition or a standard for electronic ballot, an electronic record for
a ballot and that record is made up of specific data elements, each of
those data elements is defined.

LYNN ROSENTHAL: Right now that wasn't envisioned but if that is a term
that and those data elements related to, you know, what the record is
need to be defined and there is a context for it then it makes sense to
put it in but right now its not.

MALE SPEAKER 35: I think what you are suggesting though is going down
the tricky path of setting standards by definitions and I think the
approach they have taken as a glossary is to list terms that you need
understand the standards.

(START TAPE 3)

DR. SMERGIAN: If we could come to order, we'll resume this meeting of
the TGDC. I understand two of our members had other meetings to go to
this afternoon so they will try to check in by phone, Ms. Miller and Mr.
Berger. Do we still have Dr. Revest and Ms. Turner Bowie on the phone?
Thank you. Ms. Turner Bowie are you? Currently we have ten members,
either here, present here or on the phone. So, we are a quorum so we
will proceed as planned.

At this time I call on John Kelsey of NIST Information Technology
Laboratory to present a preliminary report on NIST Approach to Direct
and Indirect Verification for the Redesigned VSS.

JOHN KELSEY: Okay. Can you hear me? Okay. So, let's see if I can make
this work. Good. Okay. I'm John Kelsey which I guess everybody heard.
Now, I'm going to try to address this in a little different way than
some of the other talks.

First I want to talk about this broad security framework we've been
trying to work out for how we write standards that will lead to secure
voting systems. Systems that can be, you know, that can be tested and
implemented and used in a secure way. I'm going to take about the DVID
Resolution and the (undecipherable) Resolution in light of that in a
broader security approach.

DR. SMERGIAN: John, just a minute. I thought I heard a click on the
phone. Do we have Ms. Turner Bowie on the phone?

MS. TURNER BOWIE: Yes, you do.

DR. SMERGIAN: Okay. Thank you. Please proceed John.

JOHN KELSEY: Okay. So kind of a high level view of this, of our security
framework. I'm going to start with a picture of a model voting. The high
level, okay, a summary of this is as we looked at the security
requirements for voting the first conclusion we came to is you don't
actually need paper to get a secure, to get a good voting system. You do
need some way of checking results. You need what I've been calling audit
ability (sic) and I gather now that I have been using that term
incorrectly for the voting world so I will have to find another term.
I'm probably going to say audit because otherwise I'm going to be
misreading my slides for the entire talk but to substitute in, what I'm
trying to get at with the notion of auditing is cross checking records,
cross checking independent records.

So, this is a picture of kind of the general voting, kind of a very
simple voting model. This part in the dotted box, this is the
interaction between the voter and voting system. The problem that you
can have here is whatever, the only chance we get to get the information
about what the voter wants is in this box. We are going to do this
process of capturing and verifying the vote. Verifying in this sense
does not mean in the voter verified paper trail sense. It just means in
the sense of you get a list of what you chose and you get a chance to
say yes or no. It's a usability issue. Then we produce a vote record.
Probably there's a better term for that as well but we'll produce some
final record of this interaction and at the end of the election, after
we have done this many times for many voters, this vote record is put
into the process of counting and kind of cross checking and auditing and
we get a preliminary count and eventually we can also get a recount from
that. So it's a very broad overview.

There are a couple of interesting properties of this, right? Its
actually really hard to catch errors or fraud in this interaction
between the voting system and the voter because I think pretty much all
the other steps in the voting process you can have observers watch
things. You can often rerun a process like if you think that the machine
has incorrectly read the paper ballot, you can do a recount. You can't
really do that with this interaction with the voter short of rerunning
an election which is a disaster, right? So, and the other thing is that
the privacy requirements for the voter really impose some limits on how
much auditing you can do. You couldn't put a video, a security camera in
here and watch everybody vote at least if you could observe the screen
because if you did, you would violate their privacy. You would get in
trouble. Right now how does, as I understand, at least, requires a step
where the voter gets a chance to verify his vote. That doesn't mean in
the security sense. That means in the sense that, for example, he gets
to look at the piece of paper he's marked or if this is a touch screen
machine or a derry (sic) of some kind, the derry (sic) presents his vote
back to him and gives him a chance to say that's what I meant or that's
not what I meant. It actually resolves a lot of usability problems and
actually even some security problems get caught with that if you are
doing something or mis-calibrating the screen to cause some votes to not
get counted that would actually help a lot. Its just having the
verification steps.

So in our framework what we want to do is use this step to provide
independent evidence of the voter's choices, the voter's interaction
with the voting system. That makes that step auditable or checkable,
right, in the broad sense.

MALE SPEAKER 36: What do you mean by independent?

JOHN KELSEY: I'll get to that about two slides from now. Okay?

MALE SPEAKER 36: Okay.

JOHN KELSEY: So, I've come up with this name, pretty much out of thin
air of a dual verification voting system. There are a gazillion of these
out there. This is not an invention of ours. This is just a term that
I'm using because I want to get at this idea that we produce two records
of independent validity. Typically one during the capturing, the process
of capturing the vote and one during the process of verifying the vote.
One of the requirements here is both of those go into that first step
where you are going to cross check and do a count. Its important that
you do that. Some voter verified paper audit trail ideas; the idea seems
to be that you keep the paper around just for recounts. That really
doesn't work well in security terms because it means that an attacker
who can tamper with the election in a way that doesn't quite cross the
threshold to cause a recount gets a way with it and maybe gets caught
later and maybe he doesn't. There is a reasonable chance of success
there. You can kind of see that there's a couple of interesting points
and I'll get a little more in this before when you were talking about an
independent record. This process and this process may be independent
enough that an attacker who could plausibly compromise this process
doesn't, we don't just assume that he must also be able to compromise
this one and vice versa. The whole system has to be designed in that
sense. For example, if you had paper audit trail plus electronic records
on a memory card you couldn't have the same people carry put those
together, put the memory card in the ballot box with no cryptographic
protection or anything and have them carry that to the central facility
to count because if you did they wouldn't be independent anymore. The
same one person would have had a chance to tamper with both records. You
might address that other ways like cryptographically but that's kind of
the broad idea.

Just kind of covering this a bit the idea is that each record is going
to give us some independent evidence of the voter's choices and here's
where I am going to try to address that a little bit, your question. By
independent evidence and I also say independent validity what I'm trying
to get at is two things. Number one its meaningful to audit one against
the other, to check one against the other. So its not just a matter of
saying -- an example of some place this wouldn't make any sense would be
a paper ballot where you scan it in electronically and you have an
electronic record derived from it. In that case there is not really a
cross check where the electronic record could ever tell you anything
about the incorrectness of the paper record. The errors can only go one
way. In this case the two records can be used to cross check each other.
So an attacker who wants to compromise an election has to compromise
both records.

MALE SPEAKER 37: Lets step back to your paper example because I think
its very similar. You have a paper marked ballot and you have a
cumulated totals from scanning repeatedly those ballots and the DRE
device you have an electronic image of ballot images and you have
registers that accumulate totals for each candidate off those registers
so, with the exception of electronic versus paper, I don't really see
any difference in the two processes.

JOHN KELSEY: Well, if you had just hand marked paper ballots then the
processes are rather similar in this view in the sense that in both
cases you are looking at a system that looks more like this, where
there's one process by which you are capturing and verifying the votes.
The process is you are using the DRE, you are selecting the votes and
its presenting you a verification screen and you say yes or no and
there's a single record. There's no way to cross check that record
against itself. The DRE could be producing multiple like independent
logs or something.

MALE SPEAKER 37: It (undecipherable) possible memories.

JOHN KELSEY: Right. But those independent memories are still being
produced fundamentally by the same process on the same machine. Maybe
there's some way of partitioning that out.

MALE SPEAKER 37: But they are not always exact.

JOHN KELSEY: I'm sorry.

MALE SPEAKER 37: They are not always exact.

JOHN KELSEY: They are not always exact copies of each other?

MALE SPEAKER 37: Right because sometimes machines malfunction and then
we actually go into the memories and discover what happened and find
which one has the correct totals. So the multiple memory actually exists
for a reason and they actually add value and independence to the
process.

JOHN KELSEY: I think in that case you are talking about a fairly narrow
range of things that can cause the error that involves physical failure
of the memory. If there were an error in the DRE itself or if somebody
compromised the DRE itself, then I don't believe that these multiple
memories would add a lot of value. There's not any reason to think if I
had taken over the DRE that I couldn't write the same incorrect data,
you know, show you that you voted for John Smith and then write the vote
for Mary Jones. There's no way to cross check that particular
interaction with a DRE and similarly with a paper ballot. With a paper
ballot there is sort of a similar issue. There you've got to verify it
but then there's a question --

MALE SPEAKER 37: But there are ways to validate that software before the
election and then make sure that you don't gain access to it during the
election.

JOHN KELSEY: Right. So, there are ways to try to prevent the attack. I'm
sorry, go ahead.

SECRETARY DAVIDSON: But I think the point is that in one case you are
checking the operational, you are checking the operational failures of
the machine and in another you are actually producing duplicate, sorry,
you are producing two different records of the same event as opposed to
simply writing the same thing twice to make sure there is no electrical
glitches.

JOHN KELSEY: Let me address that just a little bit, the basic idea
because I think its important to get. One way that you can secure any
system, basically there are a couple of different directions you can go
in trying to secure any system. One way is before the fact trying to
prevent attacks. Right. You lock the system down. I'm going to talk
about that actually in the next talk that I do on system evaluation.
What we do ahead of time to make sure that the box can't be broken into.
Oh, sorry.

MALE SPEAKER 38: Here's the concern and why I like this approach over
the other approach. Now, I can construct an attack that does the
following. You know, I built code that during the test performs properly
but at the time of the election what it does is it shows you on the
screen, the same code, the same machine. It shows you on the screen one
thing and it prints it the same as it shows on the screen but what it
enters in the electronic ballot is something else. Its an attack that I
can do.

Now, this approach says I have two different machines built by two
different vendors.

MALE SPEAKER 37: You said you can do. Have you done it?

MALE SPEAKER 38: Can do what? Could I do such a thing?

MALE SPEAKER 37: Can you modify a DRE voting system?

MALE SPEAKER 38: I believe you can.

MALE SPEAKER 37: Without being detected where it will take votes from
one -- have you done it?

MALE SPEAKER 38: Some of the virus software does that. They look good
for the scanners. They say there's no virus and then they hide
themselves and they come back later.

MALE SPEAKER 37: Once again, have you done it? Its very difficult to
start manipulating frame ware --

MALE SPEAKER 38: I haven't attempted to do this for this application.

MALE SPEAKER 37: That's not --

MALE SPEAKER 38: I have seen it done in the internet for other kinds of
attacks.

MALE SPEAKER 37: For other kinds of attacks but everybody who keeps
saying this is possible. No one comes and says we've done it. There
seems to be enough bright people and resources trying to prove that its
possible that I would like to see someone who can say they've done it.

JOHN KELSEY: May I suggest, I think -- in my threat analysis I wrote
this and I write this in some of the other stuff I've done here. Trying
to secure a system assumes that there are attackers who are a threat to
your system, right. One would try to address this a preventative, to
lock down the system so well that it can't be tampered with. I don't
know of any experimental data where we, where people have taken a
working voting system with full procedures and attacked it. I think
we've seen attacks on pieces or things that were leaked and I think that
there you have this problem that you are seeing one piece of the system
in isolation. Its another thing I will talk about in the testing stuff.
When you look at one piece in isolation you actually, you think you see
attacks that may or may not really be there when you see the whole
system as its used.

I think, first of all I would say, that in the computer security world
there's not much, there's a lot of history of people having systems they
were convinced couldn't be attacked, that had been done very well by
very competent people who were successfully attacked. The kind of common
wisdom you see in, like the Belvin and Cheswick book on fire walls and
internet security is something -- I wish I could remember the exact
quote but its something like, you know, if you haven't noticed traces of
an attack on your system and your system's been on the network for a
long time it probably means you are not checking your logs. So, its, you
know, I'm probably misquoting that but that's approximately right. You
can't assume that a system can't be attacked just because you haven't
seen evidence yet. Also we have a problem that we have to worry about
attackers in the future, not just the ones today. I think there's some
argument for that. I think maybe we can move to the next couple of
slides if that's okay.

The highlight of the idea here is we don't just want to try to prevent
the attacks from happening. That's very important for a lot of reasons.
We would also like to detect them if they happen. We can do that in a
lot of ways. This offers a chance to detect attacks in one area that I
don't believe existing procedures could reliably catch. You can't have
observers from different parties' watch the person cast their ballot and
then make sure that the ballot is recorded correctly. If you did that
then the person would, the observers would know your vote. I want to
make it clear we are not inventing something new here. There's a bunch
of things in the literature, a bunch of ideas that have been floated in
the world to do this. The MIT/Cal Tech report has a lot of nice
information in it. It talks about a thing called the Frog Protocol which
amounts to doing the same kind of protections you are trying to do with
a voter verified paper trail but without having to handle a bunch of
paper, doing it all electronically. There are a lot of good reasons to
like that. There are a bunch of other similar things. DREs with a
camera, DREs with an external screen, (undecipherable). He pointed out
that you could do that to get the same verifications and, of course,
voter verified paper another way of getting the same assurances.

Having given you kind of a broad overview of how we want to approach the
security and I think I can address these Resolutions. I'm way behind to
what I thought I'd be but I hope its okay. I'm glad you guys were fast
earlier.

Resolution 1205 introduces this DV/IV distinction. It basically says a
DV system is a system where the voter verifies some representation of
his vote that recorded with his own senses. In practice it seems like
that's always paper. In principle it could be other media but it seems
like its always actually paper. Then with IV its just everything that's
not DV basically. You verify your vote with the help of a computer. So,
in this kind of broad category, these broad categories something like
the Frog Protocol would be IV. You are verifying your vote with a
computer. DV would be like voter verified paper.

Resolution 2105 talks about multiple representation. This is an area
where I suspect we may run into problems between what's required for
security, what makes sense for security and what works for state
election law. I'm hoping to get some feedback on this. In multiple
representations we have voting systems, voter verified paper, the Frog
Protocol or any of these that produce, these voting systems produce two
different records or three different records of the vote and there's a
question of what you do if they disagree and is there a clean way to
resolve those disagreements. We'd really like a simple rule to resolve
the disagreements and if you look at it in terms of security you can't
have a simple rule to resolve the disagreements. This is potentially a
problem and I'll get into that in a bit.

So, I'll talk a little bit about DV and IV. In our framework these are
kind of the definition issues for DV and IV, for a DV system, right. DV
means directly verifiable. The idea is without any intermediary hardware
or software or anything you are verifying some representation of your
vote. Its then actually stored and recorded. So the idea is the voter
verifies the record with his own senses and generally its paper and then
you have both paper and electronic somehow used in this first count.
That's a little confusing. What I'm trying to get at there is, you cross
check them before you use the electronic count. You have to have an
auditing procedure to make sure that the paper agrees with the
electronic records. You don't just lock those paper records in a box
until it comes time for a recount. If you do that, again, there is an
easy attack where you just tamper with the electronic, and assuming you
can tamper with the electronic records is an easy attack where you just
bypass the paper and there's no point in having a printer there in the
first place.

Now, some interesting problems that arise here. First of all there's an
issue with verifying that the human readable thing equals the scanned in
electronic thing. I think Bert Williams talked about that a minute ago
or he talked about that earlier today. It came up in the voter verified
paper audit trail talk. The thing is, its not just an issue with bar
codes and we were talking about bar codes, putting a bar code or some
mechanical thing on the ballot to make it easier for the computer to
read or more robust. The natural thing is to think that bar codes are
the only place where this issue comes up. Actually even with optical
scan ballots there are known problems where if the thing, if its printed
in an offset way you can misaligned it, you can cause differential rates
of error because votes for John Smith get counted as errors a little
more often than votes for Mary Jones. There has to be some process by
which we verify that the thing that the voter thinks he verified and saw
is the same thing as the electronic record that was actually counted.
That's an independent kind of step that is needed in these systems and
is not generally needed in the IV systems.

There are some interesting issues of error and reliability rates of
printers and I think John really covered that pretty well. One of the
things that comes up in the security context is if you have something
where the printer goes out and a bunch of your records are blank, you
know, you have a bunch of ballot records that are blank on the paper and
they are there in the electronic records, that could mask an attack.
That happens very often, it will mask an attack if somebody does one.
Like if you were pulling out real ballots and putting in blank pieces of
paper. Its hard to distinguish between those two cases which is a good
\--. Go ahead, oh, sorry.

SECRETARY DAVIDSON: I'm just trying to understand what you were saying.
Could you repeat that last portion again?

JOHN KELSEY: Okay. So, suppose you have a voter verified paper audit
trail. We have something where the printer malfunctions and starts
producing just blank pieces of paper and we get through -- if we assume
that often voters don't actually look at these papers, at this paper --

SECRETARY DAVIDSON: Oh, now I understand.

JOHN KELSEY: we might get ten or fifteen of these ballots records or
whatever they are into the ballot box before anybody notices the
problem. We've now got fifteen ballots that will never match up with the
paper records and from an auditing perspective. Its very hard to
determine whether that's really what happened or what happened was
somebody pulled out fifteen real ballots and put in fifteen blank pieces
of paper. That's the thing we worry about and that really pushes us
towards needing very high reliability rates on these printers.

There are also privacy problems with unshuffled paper, of course if you
have the rolls of papers there are pretty dire privacy problems. I know
there was some discussion, swapping the rolls around to keep people
confused about which roll goes to which machine. I don't think that will
work very well with any kind of auditing procedures because you usually
would like to be able to recount.

A lot of times you have a requirement to recount a certain number of
machines. You say choose one machine from each precinct and recount it.
You have to know which ballots went with which machine or you are going
to get really strange answers out of that. You can't actually do a
verified; you can't actually check your work that way.

I think that there are a lot of issues there that need to be resolved
and then, of course, the whole accessibility and usability problems with
paper. If you are blind or if you just need a magnifying glass to see
small print, the voter verified paper can raise some problems with that.
Those can be addressed but they are problems that need to be addressed.

So, then IV systems in our framework, enormously wide range of systems.
It ranges from the Frog Protocol from the MIT/Cal Tech report to just a
camera with a DRE or a DRE with a separate screen for verification, all
the way to the cryptographic schemes and I'll show my bias here. I'm a
cryptographer. Its what I do. I think these schemes are just beautiful.
I would like to see these widespread. I don't think we are ready. I
think there are reasons why we aren't ready to standardize on them. They
are really cool schemes. They do fit in this and they actually fit kind
of poorly in this category. They provide a different set of assurance
than any of the other schemes. Long term I hope we can move toward those
but I think that's a direction that's, you know, provide some really
unique security but, some of the concerns that come up here and this is
something else that comes to your question of independence, is when
you've got independent processes there's a question of how much
independence you need.

In particular I've thought a lot about attacks on these systems. I've
spent a huge amount of time. I tend to attack things as a way of
designing. One of the problems you can run into is, for example, if you
have, the Frog Protocol you have a voting station and a verification
station and you move a token between them, like a memory card between
them to do you vote. The interesting issue that can come up. Suppose
that both of them have been resourced from independent vendors but they
are both running the same version and patch level of Win XP. Okay. If I
have a new exploit on Win XP, a new attack on it, these machines are
vulnerable to. If they are using wireless networking, it would be a
network attack perhaps. If they were using some other, you know,
mechanisms it might be some other thing. If I can find a way to attack
these systems, they are kind of in a mono-culture, if they come from the
same operating system or the same or similar designs or something there
may be a way to leverage that attack on both those machines at once and
defeat the independence. I'm sorry.

SECRETARY DAVIDSON: But don't you also have the problem if you get
somebody that's --

DR. SMERGIAN: Would you turn your mike on, please.

SECRETARY DAVIDSON: Excuse me, I'm sorry. If you have, the individual
that is doing the programming, and we do not let any of the vendors do
our programming in our state. If you've got an individual doing the
programming and they are trying to, you know, obviously throw the
election let's say, they are going to have the ability to program both
systems.

JOHN KELSEY: Okay. When you say programming, do you mean actually
writing code to run on the voting systems or do you mean doing ballots
or \--?

SECRETARY DAVIDSON: To do the ballots, I mean, you know, each time
before an election is when you are really concerned about this correct?

JOHN KELSEY: Right.

SECRETARY DAVIDSON: So, I mean, they may be also able to, if anybody
else can hack into it why can't they figure out how to do it? So they
are actually changing the code to where they could do it on both
systems.

JOHN KELSEY: Right. I think in the design of this kind of system as we
fill out the requirements of the design that's something we have to
address. First of all, making sure, and in my document I don't think I
got this in the slides but in the full document that I wrote on DV/IV I
talk about the fact that one of the requirements of this kind of system
is that there is no one election official or small group of election
officials who have the authority to do and no vendor, no one person or
small group of people have the authority to alter both machines, you
know, to alter the function of both machines. Now, that means you load
in a ballot design, we should be able to design a system that doesn't,
or somebody should be able to design a system that doesn't allow a
loaded in ballot design to compromise the security of the machine. If
you are actually talking about getting in and rewriting the programming
on the Diebold VRE or something, you know, I don't \--. Maybe I'm not
understanding, is that something your state would do is have somebody
rewrite some of the codes for the DRE?

SECRETARY DAVIDSON: Not necessarily but I mean, we are talking about
people hacking in and that could be, I mean, I think one of the main
concerns about somebody that's a hacker they've got more capability to
the equipment and everything else as an employee than somebody else. So,
that's where my concern came in is that, you know, we can try to protect
the whole world but we also have to remember we have very small counties
that only has maybe one person that even can write the program for the
election. So, say they can't write the program on each system if you
require two different systems. It would be virtually -- I don't know how
we would accomplish things like that in a lot of state and a lot of
counties.

JOHN KELSEY: Okay. I think maybe we should talk about that more off
line. I think clearly we need to make sure that whatever standards are
written here require that the machines be usable by the election
officials in ways that don't compromise the security. That's true, just
as its true with Nelson's things with software distribution and set up;
I think its true here too. I think that's a requirement. It will be
interesting to get to that.

DR. SMERGIAN: I might say that if you have a separate machine and if its
functions are really very minimal, you know, that is to say it can
receive a representation of a ballot and display it in a variety of ways
and that's all it does, you could design that where its really hard to
attack and you could also say that's not the area where I will tolerate
any changes that's in a read only, non-modifiable type of memory. You
could go to (undecipherable) with that device. It would not be very
expensive. It would be so stripped down that you could verifiably prove
it and you wouldn't allow anybody to modify it. It would be the hard
memory, the hard ground, it wouldn't be changeable. You could play a
game like that if you want to. It's really just a medium to render it in
different views for different disabilities and for someone to see and
perhaps have verified. The less you do the easier it is to prove.

JOHN KELSEY: I think you are ballooning the number of lines of code once
you start talking about alternate --

DR. SMERGIAN: Yeah, yeah, I know. I should have kept quiet.

JOHN KELSEY: I'm told I have very little time left to cover the rest of
this so. Let me get through the rest of these real quickly.

So, these are just a couple of the points here. Kind of the round up,
these are the things I really wanted to make as a point here. They are
somewhat contentious. I have given some thought to the idea that I was
going to have at least two large groups of people in the voting
community mad at me at the end of this talk.

The first thing to understand is the distinction between paper, you know
DV and IV. Its actually a little less clear than you think of at first
because even paper systems aren't really pure paper you almost always
scan them electronically and you have to deal with that electronic
record. Some of the cryptographic schemes, they look nothing like the
paper schemes but they actually provide, in some sense, stronger
assurances at the end than the paper ones do. Neither DV or IV systems
are inherently more secure. That's an important point to get because
there is a lot of idea that, you know, paper is this kind of magic thing
you are going to sprinkle on an election system and make it secure.

DV systems are probably harder to do a wide spread attack where you have
a small conspiracy of two or three people that actually tamper with
machines all over a state or something. There are operations reason that
are really hard in practice if you think about. The two things I want to
get at -- do I have time to cover them (undecipherable) really quickly.
Okay. I'll take it from some of my testing time.

So, the kind of the big thing with multiple record and this is something
we really need feedback. Okay. In the 1205, Resolution 1205, there is
this idea, the fundamental representations, the thing that gets counted
initially. There is also this idea of like a ballot of record. The thing
that's the final, the record that has the final say. The problem with
this sort of system its true with voter verified paper or with any of
the dual verification architectures, is both records have independent
validity. If you lost one and you knew it was an accident, you could
reasonably reconstruct the results from the other. You always have to
count them against each other in normal process. There's not one of
these records that can always be seen to dominate in a disagreement. I
gather that that may not work well with election law and I don't know
exactly how to address that because I think that's state by state. From
a security point of view, assuming the paper records are always right
just means that people who can stuff ballot boxes always win and
assuming the electronic record is always right just means that people
who can tamper with DREs always win. Neither of those are the thing you
wanted. One goal that we can do in the standards is, I'll make this
phrasing, disagreements between these records should be very rare and
they should be meaningful. So, if we see a disagreement between these
records we need to be able to figure out what the cause is and it needs
to be one of a fairly small, short list of possible causes, like the
printer failed or somebody dropped the DRE in a river or something. I
don't what you would have to do to destroy all the memory copies you
keep on a DRE. You may shoot it out a canon or something.

The other part of this is, we want to get down to zero misreads which
means we use error correcting codes, we make sure error detecting codes,
to make sure that the probability that you will ever read a vote for
John Smith as a vote for Mary Jones is negligible. That's existing
technology. Its off the shelf. Its well understood. You can make that
less probable than getting struck by lightning twice in your life or
something. Once we have the kind of zero misreads we can remove a lot of
the ambiguity about damage or error versus fraud. That's the real goal
here is to get down to a very, if you see a disagreement, to be able to
pretty quickly resolve whether its evidence of fraud or a major error or
whether its just a routine thing like the printer failure.

I think that's basically it. All the rest of the stuff is covered in my
document. Kind of the big thing is you need to keep the records under
separate control for this to work. I talked about this in the document a
lot more. I think that's everything. Our broad approach to security is
going to be to look at dual verification systems as the things that we
want to see in the standard in the future. I would like to move toward
having those be the things that we allow for electronic voting systems,
not having a system where's there's a single point of compromise.
Exactly how to specify that independence is kind of an interesting
question. We need to talk about that. Then we've addressed DV/IV and
multiple representations within that framework. That's everything.

MALE SPEAKER 38: Could you put your slide back up that show a diagram of
the system.

JOHN KELSEY: Yeah, this one? Okay.

MALE SPEAKER 38: You're saying that those two systems are independent.
That those two are independent and should be produced by separate
vendors.

JOHN KELSEY: Yes, that would be the ideal, yeah.

MALE SPEAKER 38: Okay. A person's going to vote on this one on the left.

JOHN KELSEY: Right.

MALE SPEAKER 38: And then he's going to go over and verify it on the
right.

JOHN KELSEY: Right.

MALE SPEAKER 38: Is going to vote again?

JOHN KELSEY: No. It's a --

MALE SPEAKER 38: All right. So how's the records going to get from here
to there?

JOHN KELSEY: Each of the different architectures does this differently.
I'll point out; it doesn't actually have to be that he interacts
directly with this process. One of the designs we've talked about is a
camera. People have talked about it. I don't know whether anybody's
implemented it. Is a camera over the DRE screen? The idea is there is
some point at which the voter has the verification screen or several
verification screens and the camera takes an independent record of that,
of what's being shown to the voter and then that can be reproduced if
there is some questions of whether or not the results are correct.

MALE SPEAKER 38: A camera over a voting system doesn't sound to bad but
we've got 25,000 voting systems. So we would have 25,000 cameras.

JOHN KELSEY: That's right.

MALE SPEAKER 38: which would operate with about the same reliability as
25,000 printers.

JOHN KELSEY: Right. I didn't say that the camera was the optimal way to
do this. I suspect that something like the Frog Protocol is better but
--

MALE SPEAKER 38: This is my concern. I'm not sure this is well thought
out. I'm not sure its as independent as you think it is because one set
is going to be generated from the other set. The voter's not going to
vote twice and if they do they are going to get different results
because, not many voters will vote a 38 or 40 race ballot and then walk
over to another voting station and vote it the same identical way.

JOHN KELSEY: That's not even on the table. Nobody's talking about having
him vote twice.

MALE SPEAKER 38: So, they are not independent in that sense. The other
question that I have is a lot of people we know from what little
experience we've had with these systems, that a lot of people do not
verify their ballot. They just don't. They are in a hurry. They want to
go in. They want to vote. They want to get out. They are on the way to
dinner. They are on the way home. They have to pick up the kids. Most
people that vote are in a hurry. So, you are going to have a lot of
unverified material in that verified bucket.

JOHN KELSEY: Right.

MALE SPEAKER 38: I appreciate your concept but to require this at the
stage we are at right now, I think would be a bad, bad mistake. This
needs a lot of careful thought. Something that I haven't seen much talk
about in any of these is the voter themselves. What I the voter's
reaction going to be to this?

JOHN KELSEY: Okay.

MALE SPEAKER 38: I think I said this morning, if we are not careful we
can have an unintended consequence here of designing a system that the
voters just despise and result in a drop off in participation that's
unacceptable. So, these are the variables we are juggling here. I like
your concept but I think it's a long, long way from any reasonable
implementation.

MALE SPEAKER 39: Well, I've spent even less time thinking about it than
he has but if you can envision some kind of a picture where, let's say,
in this voting booth or space is three pizza platters or boxes, all
driving the same display, let us say for the sake of argument. So, from
a user's point of view, he votes on the DRE machine on that screen and
the next thing the control shifts through a verification processor which
shows him that that vote which he confirms, and then there's a
comparison processor that compares them. So, he never really has to
leave or get in another line. Now, what happens if he chooses to not
really verify, just thick or thin. Well, if the verification processor's
representation didn't match the DRE representation then that vote is not
-- he tried to verify. We know that they didn't match. Like his blank
example so you have at least got a situation where you say, when that
voter leaves there's a red light or something. This guy can't leave yet
because he obviously didn't verify it because they don't match.

JOHN KELSEY: Well, that sounds like a bad (undecipherable).

MALE SPEAKER 39: So, now if you did this right you could conceive of how
we could overcome a lot of those kinds of objections.

MALE SPEAKER 38: I have trouble seeing the difference between that a lot
of what happens now where you are given a verification screen at the end
to look at and say yes this is the way I intended to vote. All of your
DREs have that now, every single one. I don't see a whole lot of
difference in that and what's up here because the original voting device
has got to create that verification image.

JOHN KELSEY: So, if it creates the verification image and that image
then goes to a separate processor, a separate machine, say, then
displays that image up and give the voter a chance to verify it on the
other machine. Under normal circumstances what should happen is, if the
voter doesn't want to verify if he just pushes yes, yes, yes, yes and
walks away. You actually don't need a lot of voters to verify this to
catch large scale fraud as long as we can produce the record. The other
side of this is that record that's been created on one machine is
displayed on a different machine. The idea is if this machine, if the
first machine that I voted on is sending a different record to the
second machine then I'll see it if I check and that will get recorded.
If its getting the same thing that it showed me but recording a
different thing then I have a chance of catching it and if enough people
complain, that should raise a red flag.

MALE SPEAKER 38: And then the last thing and I'll shut up, is this idea
of massive fraud. In reliability and security work you don't spend
enormous amounts of resource to protect against events that occur with a
minuscule probability. So far, in the forty years we have been using
electronic devices to count votes we have not had a single massive
electronic fraud. Lots of massive blunders but not any massive
electronic frauds. It looks to me like we are spending enormous
resources to protect ourselves against something that has a rather
minuscule probability of occurring.

JOHN KELSEY: I understand the point there. I think, maybe this is just
my bias, but I think that its hard to estimate the probability that
there will be fraud in the future and I think that I worry -- well, we
are doing the standards so we have, we're doing standards so we are
saying something about the future. We are saying we think we know how to
secure votes so that next year somebody won't come out, or two years
from now there will be a governor's race where there's evidence that
somebody's fixed the election and we don't know what to do. That's where
we are trying to go so I think that's a --

MALE SPEAKER 38: Well, don't get me wrong. I'm not saying that we
shouldn't pursue this but what I am saying is watch the expenditure of
resources and see if we can't pursue this with, well two things,
expenditure to resources and the demand on the voter because either one
of those can destroy this whole concept.

JOHN KELSEY: I think Sharon would beat me up if I foul usability
requirements. Ron wanted to talk?

DR. SMERGIAN: We have a question and comment on the phone. Dr. Revest.

DR. REVEST: Thank you.

DR. SMERGIAN: Can you speak up a little more?

DR. REVEST: (undecipherable) massive fraud or fraud attempted by
software modification. We are fortunate but he seems to be correct that
there haven't been any major tampering that we've seen today on a large
scale. However, one of the things this committee is hearing on a
consistent basis from the efforts in the field is that the complexity of
the system is making it much easier to mount such an attack and do so
undetectably. (Undecipherable) could sneak a Trojan horse through the
back door or something under the rug as well and in such a way that it
would be much harder to detect. We are raising the ease of doing this by
computerizing our voting system. (Undecipherable) making it much, much
easier to (undecipherable) (This gentleman was speaking over the
telephone and I could not really make any sense of what he was trying to
say).

DR. SMERGIAN: Secretary Davidson.

SECRETARY DAVIDSON: I only want to make a statement. I think that we are
dealing with voluntary standards and if we make then where they are so
cost prohibiting our states will not accept them. I definitely believe
that the states should be accepting them so I guess I'm putting up a red
flag, be careful of what we ask for or we won't get anybody that will do
them. I think there's so much good in our standards that we have to be
very careful of how we proceed.

JOHN KELSEY: I think cost is clearly a big issue.

DR. SMERGIAN: Mr. Kraft.

MR. KRAFT: Yes, I applaud the (undecipherable). I think, yes, that's
something that we need to be working on. With the current state of the
art there is a verification process in the works. As I said at the last
meeting and I mentioned it to some individuals who I had seen in the
audience, there are a number of people working on other ways to get more
independence into the process and do more verification. At the same time
we have to remember that we are in the business of prescribing standards
to be used in the real world and that means taking what we know about
the current state of the art, the current best practice, and putting
those into standards. I guess my caution here is while you are doing new
research and new ideas, you shouldn't be bringing those ideas into
standards until they research is complete and the science is mature.

JOHN KELSEY: Okay. That's helpful. Again, particularly that would apply,
especially the cryptographic schemes which are very new and clever but
--

MR. KRAFT: The other thing that I really have problems with and I
continue to have problems with this is within your assumptions that you
have stated, there are a lot of things that, frankly, I know you have no
evidence for. Its possible and easy to manipulate firmware on an
installed system and escape detection. That's a theory that some people
have advanced. It's a theory that if you are going to pursue, somebody
needs to go out and actually do some experiments and see what's really
involved in that.

You made a statement that optical scan based on ballots being misaligned
can create more read errors on one candidate than the other. My
experience is no. Basically when those machines get misalignment
problems they reject the ballot but they don't do read errors and that's
based on thousands of hours of testing those devices. You need to do
real research in those areas before relying on those assumptions.

JOHN KELSEY: Okay. I wasn't assuming that the attack as I understood it
was, you caused some of those ballots to be misread as incorrect, as
under votes, not as -- the misalignment problem is that they don't get
read to the proper vote, not that they --

MR. KRAFT: No. The misalignment problem and, as I say, we test for this,
the misalignment will cause the ballot to be rejected as being the wrong
ballot for the election and when you stick it back in it reads it.

JOHN KELSEY: Okay.

DR. SMERGIAN: Any other questions or comments? We believe that the
preceding preliminary report of the technical support titled "NIST
Approach to Direct and Indirect Verification for the Redesigned VSS
Response to the TGDC Resolution 1205 and 2105\". On this there are
supplemental directions or corrections. The technical support and
related work product will continue to be developed consistent with this
preliminary report taking into account the comments made here. Are there
any other questions or further directions or corrections? Do we need
substantial change in directions?

I think my interpretation was so the level of emphasis and some
additional parameters such as cost, etc. that we need to keep in mind as
we develop these standards.

MR. KRAFT: Well, I think that's practice and this dictates and analysis
of cost, a real analysis of risk rather than relying on antidotal
theories of risk and then moving to look at real world solutions that
can be implemented and tested that address those risks that are real. I
think that's good science and I think that's what you all are supposed
to do here.

JOHN KELSEY: Does that need to change.

DR. SMERGIAN: No. I think that is good advice. You are back on for a
presentation on the second preliminary report on NIST Approach to
Security Testing for the Redesigned VSS.

JOHN KELSEY: Okay. So, I'm me again and now I'm going to talk about
testing and evaluation.

We are trying to address the specific Resolution which is Resolution
1705. I'm going to kind of show you and overview and I'm going to talk
about the Resolution and then talk about kind of this idea of open-ended
evaluation of a system. That's really the innovated part of that or the
new part of that Resolution as opposed to what everybody's already done
which is saying we've got to test the system. We have to include
open-ended kind of search for vulnerabilities and we do this a lot in
cryptography and computer security. That's kind of the thing that people
think about a lot. I think we can, maybe, bring some nice tools to the
table here. We are going to talk about evaluating a design, evaluating
the pieces and then some issues that I think are going to come up.

One of the big issues related to the previous conversation is cost. I
think a good evaluation, where there's a lot of search for open-ended
vulnerabilities is going to cost a lot more than simply going down a
checklist. Fortunately that does actually get amortized over the entire
set of voting systems so, hopefully, that's not prohibitive to the
states.

Resolution 1705 and I didn't try to quote the whole thing because if I
tried to include it all in a slide, you couldn't read any of it. The
italicized part is the interesting part here. We are supposed to come up
with standards to deal with testing of voting systems that include a lot
of open-ended research for vulnerabilities. So, let's kind of think
about this. Open-ended evaluation, open-ended research for
vulnerabilities.

MR. KRAFT: I think it said significant amount.

JOHN KELSEY: A significant amount, yeah, did I --

MR. KRAFT: Not a lot of.

JOHN KELSEY: I quoted it right, I just said it wrong. A significant
amount. So, the idea here is a natural way to try to address security.
One that you actually use in any sensible way of addressing security,
testing something, is you go down a checklist. You say, okay, did you
lock all the doors, did you lock all the windows, did you turn the alarm
on, that sort of thing. You can apply it to computer systems. There
actually are some nice, I think its 870, that's the checklist for
securing computer systems. We have some really nice work that's been
done there. The point of this is a checklist approach gets you so far
but you still need somebody to just look at the design at a high level
and try to find weaknesses, try to find possible flaws. One of the kind
of core ideas here is that this evaluation needs to be adversarial. I've
done some of this and it's a strange thing if you are working as a
consultant because your customer is basically paying you to beat up on
them, to tell them what they did wrong. Maybe people in other fields
have had a different experience. The idea is your goal as an evaluator
is to find weaknesses before the system gets out the door. Its more like
you are looking for a reason to fail the system than a reason to pass
the system. Of course, the full process there is after you find the
flaws then they go back and fix them and you check and make sure that
they've got them fixed. This mind set has to be a little bit
adversarial. You are not looking for an excuse to let them through. Your
goal is really to find weaknesses and find design flaws at a high level
sometimes there will be a design, an idea for doing something that just
doesn't, you know, there's some fundamental reasons why the thing
doesn't work or where you can't secure it properly.

The other thing is you can't just trust vendor assertions about
verification. I think that's already in the VSS, the testing that's
done, right? You don't just say, did you do a good job with security and
they say yes. You say, okay, now let's look at that and check the code
and make sure everything works out. I think Brit used the term before,
making them open the kimono or something. I think that was a good,
that's the approach we want. They show us everything and we try to
attack the system.

FEMALE SPEAKER 15: In other words, always rely on saying we are doing an
acceptance testing, like we are accepting the equipment from the very
beginning because we want to prove that the vendors, if he has got any
flaws, we want to turn them back. So, every step of the way you should
be approaching it as if you are doing an acceptance testing of the
equipment.

JOHN KELSEY: Okay. Yeah, that makes sense. So, kind of the adversarial
model of the world that bring to this process is you are assuming the
existence of serious attackers. I talked about this in my risk document
a little bit and the idea here, and I know there is some question about
whether or not there are going to be attackers that really try to mount
a serious attack on the system and what's plausible but I think it's a
very big mistake in any system design to start with the assumption that
your attackers aren't as smart as you are or don't have full access to
the system internals. Now, of course, that doesn't mean you publish the
system's internals, it just means that when you are doing the security
analysis you don't assume that they don't know it. You have to assume
that they might be in on, they might be working for, they might have
some access to it, they might be working for one of the test labs or
something.

The big sin in designing a system and I think, also evaluating it, is
underestimating the attackers up front. So, an open-ended search for
weaknesses, just kind of a high level view of what we are going to be
expecting. The evaluation is going to get top level system design and
architecture; they are going to get a lot of documentation. It is
actually going to add some documentation requirements to the vendors.
That documentation is going to have to include some discussion of the
risks to the voting system and how those risks are addressed and give an
argument for why the voting system is secure. That's going to be where
the evaluation starts. What these documentations are, they are going to
document the procedures that they recommend for the election officials.
All that has to be evaluated. Somebody has to sit down and look at those
and say, here is how you tell us to set up the system. Does that
actually lead to a secure system? Here's how you tell us to handle a
recount. Does that actually get a meaningful recount done? Somebody has
to do that. The evaluator has to check all that and after you have
looked at the top level system, you are going to look at the pieces. You
are going to look at software, operating system configuration, COT
software, hardware, communications protocols, all this stuff. Again, you
are making these assumptions that the attacker really has a lot of
knowledge of the system, insider access and you can't just assume that
they are kind of small scale vandals.

When we evaluate the design we are looking at the whole system. One
thing that, this actually comes up a lot, a lot of the high level
discussions of the area systems. When you are looking at the top level
of the design you often find attacks that don't work when you get down
to the details. As I've said, I've done this as a consultant and its
very common to come up with something, I think I've broken your system.
Well, when you actually dig down a little you actually find out that
there's some reason why that doesn't really work. They really did think
about that. What you come up with is not so much attacks as lists of
possible attacks. Lists of ways you might be able to compromise the
system and that leads to some idea of the things you've got to look for
in the later parts of the analysis. Sometimes you are going to find
fundamental flaws in the assumptions of the design. I think the real
core of this is this first part where you are looking for attacks,
looking for places where the security might be vulnerable and where you
find out what you are really relying on for security.

This last part here about, the second part here about checking to see if
the procedures and documentation really address the problems. I've heard
from a couple of people in the election community that sometimes they
don't feel like they get procedure that tell them exactly how to address
some of the stuff that they are supposed to do in a real clear way. I
think that's an important thing. There needs to be clear guidance on how
to do each of these pieces. That has to be evaluated per design because
different designs will have somewhat different nature of, you know, what
does the recount mean if you are doing it on DREs? There's a way to do
that in some meaningful way by looking at audit records but it might be
a little different than what you do for paper. It might be a little
different than what you do Frog Protocol.

Once we get down to the pieces, once we get down past that pile of
evaluations then basically what we are going is looking at all the
pieces. Some of this is already covered in existing testing rules. I
think we are going to want to expand on that a lot but kind of the idea
here is that there is, at each stage of this, there's this mix of kind
of checklist. You know the sort of broad problems the system can have.
You know DRE voting system. You some things that you want to make sure
doesn't happen. You want to make sure somebody doesn't, you know, load
new code on the DRE and take it over. You are going to have some sort of
checklist for things that you want to make sure are being secured.

You are going to use some automated tools. For network vulnerabilities
they are like port scanners and things. There are automated tools to
look at problems at almost any level of this. You are also going to just
do an open-ended search. You are just going to look for places where you
might be able to get through the defenses and then check to see if you
can actually get through.

I want to talk a little bit about these. I will try and keep this a
little short. In communications, there are two sides of communication,
of securing communications. We talked about this a bit with wireless.
One side of it is figuring out how to secure those communications, you
know, encryption, authentication, origin identification. All those
things are kind of standard cryptographic mechanisms. If it's a closed
network maybe you don't need that but if its wireless or your if you are
transmitting over an untrusted network you, of course, got to have
encryption for things that are secret and authentication for things that
have got to be, you know, say ballot summaries or summary totals or
something from different precincts. That's got to go over an encrypted
and authenticated link.

The other side of that though, that's securing the communications. Kind
of think of the risks of somebody tampering with the communications or
reading them ahead of time. The other side of this though is you have
got to protect the systems from communications. Any time there is a
communications channel into a computer that's a potential avenue of
attack. That's one reason why I think people worry a little about
wireless systems in voting machines is because, okay, you're on a
network and there's no way for the election officials to see if
somebody's connected to you. Who knows what's going in there and that's
an avenue for an attack. There are a lot of attacks on general purpose
operating systems that start with network access. I can send you
packets, I can start, you know, talking to you on some port and see if I
can turn on a service and do something bad to you. That's kind of a
broad way to do this. The tools for this are going to be testing and
watching the communications. Doing kind of the same thing you do with a
network where you actually look at the packets going by and see if they
are really what they say they are and checking the software and the
configuration and everything.

The other layer to this is software. There are at least three places
here that I think are interesting to talk about. All this involves
open-ended search for vulnerabilities here. One part of this that I've
seen discussed a little bit is the development environment. If I were
doing a review of a system that was supposed to be secure, I would be
very interested in the development environment. I would want to know,
for example, did you actually secure your development network? Is that
open to the world? If it is, the attacker doesn't have to be a voting
system programmer. He could be somebody who came into the network. If
you do good version control, that actually makes a lot of attacks harder
because it means you've got to be responsible for whatever changes you
introduce, what tools you are using, what kind of quality assurance and
testing you do internally. Then, having seen it, and that's all vendor
assertion effectively, but you would want to look at that. Then having
seen that, you then want to talk about testing and vulnerability scans.
So there are some automated tools for looking for problems. Generally,
the program is language specific. There are things where you load the
program up and you run it and you look for memory leaks and things like
that. All of that is stuff you would want to do as part of the review.
Of course, you are going to end up with a code review and that's where
you are really doing an open-ended search for problems. You are looking
at the code and saying well does this really actually count the ballots?
Or does this really capture the ballots correctly? Is there something
weird going on?

The other two sides of this are operating system and COTS software
security. They are actually some interesting issues with the COTS
software as far as getting it approved but assuming that we have to,
that you have off the shelf operating systems and off the shelf software
in your voting system. There is typically going to be some checklist you
are going to go through to lock it down, to make sure that you're not
leaving any known vulnerabilities. You are going to check against kind
of databases of known vulnerabilities in different products and say, is
this the most recent patched version? Is this known to be vulnerable to
any attacks? Also, you could say, even if it is, maybe that's mitigated
because it's a network based attack and there is no networking access on
this machine. That needs to be checked. That's all stuff that has to be
checked.

One of the things that is more open-ended and this is a very old
procedure for locking down a box, is everything that is not actually
necessary for the functioning of the election system has to be taken off
the box. You don't leave anything running that's, you know, if you do
network security, that's a big deal.

The physical configuration security and I've just gone over this very
briefly because that's really way far outside of my level of expertise.
Some things that you would want to do would be making sure that, you
know, any hardware that's supposed to not be there, that's not supposed
to be there, is not there. Making sure that the ports are blocked, if
they are not supposed to be used. Making sure that the locks and seals
work and everything. That's more, somebody, we'll have to find other
people to look at the physical security issues. That's not an area I
feel real qualified to address.

I think that leads us to the place where we get potentially ugly
although, hopefully, this is not as bad because its spread out over a
lot of systems. An open-ended evaluation could cost a lot of money. We
are not talking, this is not a matter of running a couple of
vulnerability scans and going down a checklist and saying did you use a
AES? Yeah. Did you use Shaw 256? Yeah. Did you do the right things? This
is actually going through and evaluating the thing and looking for
vulnerabilities.

The \$64,000.00 number is probably way, way low. This is the smallest
amount that I could imagine. You are talking about a team of four
people. That's probably too small but I was thinking if you tried to
cover all those levels of expertise. If you are trying to evaluate
cryptographic protocols you are not going to want to bring in a general.
You are going to need somebody who knows something cryptographic
protocols. If you want to evaluate software security you are going to
need somebody who is good at that. You are talking about pretty highly
paid people who cost some money and you are talking about a fair amount
of to do this. I'm concerned that the cost issue could be big although
the good news is, its probably a one time cost, spread out over an awful
lot of machines.

There are also some interesting kind of issues. A good voting system
architecture can reduce how many things have to be checked. The idea
here is, if you had, for example, the voter verified paper trail. One of
the things you are checking, you are checking the DRE, you still have to
make sure that the DRE can't be compromised but you might have fewer
concerns about the integrity of that electronic record because there is
some way to check it. If you had the Frog Protocol, you would have a
different kind of evaluation. On the cryptographic voting schemes if
they ever get kind of wide spread would be a good example of something
where the architecture could really reduce some of the costs.

There is some interesting incentive in financing issues that basically
amount to the relationship between the labs and the voting vendors. I
don't know how easy it would be, I know in the Phipps 140 program, it
seems to work pretty well that the validation labs are willing to reject
things that come from different vendors and I don't have the piece of
paper but apparently some really large fraction of the first submissions
of these things get rejected. This is your cryptographic algorithms that
are well-defined, anybody can go out and look at the standard and the
still get them wrong fairly often. So, its kind of nice that somebody's
checking. If they weren't checking then these products would just not
work with each other. There is other issues revolving around like the
availability of the reports when you are done. There is a concern that
on one side you probably don't want to give, to produce a record that
says everything that the vendor did wrong and how it was corrected and
publish it on the web. That might be a little embarrassing for the
vendor. It might also; you might not want to guide other attackers who
are trying to attack other systems. You also might have some real issues
for the vendor because they might say well, look, we went through this
\$100,000.00 process to get a really secure system and now somebody else
is going read the report, never make any of our mistakes and walk away
with the same thing. I don't really know how to address that because you
would like the information on how to secure the system out there but I
think there are some incentive and financial issues there. That's it.

DR. SMERGIAN: Thank you John. NIST believes the preceding preliminary
report of technical support entitled "NIST Approach to Security Testing
for the Redesigned VSS, Response to TGDC Resolution 1705\". Unless there
are supplemental directions or corrections, the technical support and
related work product will continue to be developed consistent with this
preliminary report. Are there questions, further directions or
corrections? Mr. Kraft.

MR. KRAFT: Two things. One, well first off; I like a lot what you did on
that. That is very similar to the type analysis my program does and I
think Brit's program does in many areas. There was some intermingling
there between concepts that are applicable in certifying a voting system
for deployment and actually certifying at installed site. The voting
system standards really have to concentrate on those measurable criteria
for evaluating standards or evaluating systems. I think you need to sort
out a little bit of that.

Additionally, I don't know where, oh, there's Jeff back there. I wasn't
watching him during your presentation but I think he must have cringed a
little bit every time you said open-ended. We are going to have to for
purposes of accrediting labs, we are going to have to put a definition
around open-ended and we are going to have to define what due diligence
in doing a search for extraordinary and anomalies that you haven't
previously encountered is going to entail. We cannot really leave that
undefined in the standards.

JOHN KELSEY: I agree. That's something I didn't cover and I should. We
have to think in terms of their being a limit as to how long you keep
evaluating this. So, you don't say, spend a year on it and see if you
can find anything. I think that's a major issue.

MR. KRAFT: And you touched on it a little bit with the concept there of
one vendor being able to read the report of another vendor and benefit
from that person's \$100,000.00. There are best practices that are know
and for many of these areas once you verify that that best practice is
in place, you can move on and not spend a lot of resources.

JOHN KELSEY: Actually, I think that's the point of the open-ended
evaluation is to say, even when you've done the best practices, somebody
needs to look at this at a high level and make sure that there hasn't,
you can always combine strong pieces together in a weak way. That's just
an unpleasant fact about security. Maybe I misunderstand you, I think.
You would want to see best practices being used and once you saw that,
that would make your evaluation a little shorter but you would still
want to make sure that it was being done in a way that worked. There is
a concern that you use all the right tools and still end up with a bad
product.

MR. KRAFT: Right. I think we still have to put bounds around what an
open-ended piece is going to be.

JOHN KELSEY: Right. I agree.

DR. SMERGIAN: Any other comments, questions? Well, hearing none, I think
the work is in the right direction of what the appropriate comments will
be taken into account. With that, we are almost on schedule for our
break so, I suggest, let me see, why don't we plan on taking a fifteen
minute break. Come back at 3:00 and then we will hear our final report
from Alan Goldstein and then we will go to the Resolutions. Thank you.
See you in fifteen minutes.

BACK ON RECORD:

DR. SMERGIAN: Can we get other back please so that we can start quickly
and finish up the session today on time? Can we take our seats please so
that we can get started?

Ron, when you make comments, are you on a speaker phone? We can barely
hear you. When you want to make a comment you might want to get on the
regular phone, perhaps.

Okay. At this time I call on Alan Goldstein of NIST to present a
preliminary report and discussion of NIST analysis of the 2002 VSS for
the redesign VSS. Alan.

ALAN GOLDSTEIN: Thanks. Am I on yet?

DR. SMERGIAN: No.

ALAN GOLDSTEIN: Okay. Thanks. What I'm going to be talking about are
some documents, some products that are being developed with in the core
requirements and testing group. These tasks have to do with the analysis
and the revision of the 2002 VSS. Admittedly, this task is, perhaps, a
little bit less glamorous than some of the earlier presentations but,
nevertheless, is completely necessary and vital to the, you know, to the
successful completion of our charge.

Another slight different is that, unlike some of the previous
presentations, there are no overriding global well-defined, well-scoped
comments or decision points. What I am going to be talking about
specifically are documents, most of which are in the package, in the CDs
which some of you haven't been able to read, that are on the web and so
on. What my goal is is to invite comments as to our overall direction
and, most specifically, comments on the actual text, the actual work of
the documents that you have in front of you or that you or that you can
look at.

The first document is the notorious disposition of VSS Requirements
Table, referred to earlier as the spreadsheet, you know, whatever. Its
really just a Word document table. What this is is a, right now, a
complete first draft of a table that extracts requirements from the 2002
VSS, slightly oversimplified. We went through and took out every bit of
text within the 2002 VSS pretty much that had the word shall in it. That
was pretty much what we viewed as a requirement and put this in a table
and there are some additional things but primarily we proposed, or have
proposed, the disposition of each of these requirements.

MALE SPEAKER 42: (Too far from microphone to understand what was being
said).

ALAN GOLDSTEIN: I think its what's up there. Disposition of VSS
Requirements, Volume 1. That's what it is on the web. That's what I was
using. Pardon me?

MALE SPEAKER 43: Tab "O".

ALAN GOLDSTEIN: Tab "O". I never even got an unreadable CD so, I really
can't say. So are we all on page one. All on the same page. Basically, I
have fifteen minutes, I'm not going to, you know, go into detail. I'm
going to talk more about the structure and the intent of these
documents.

Basically we went through and for each identified requirements in the
VSS categorized them as "E" which is a requirement that can be extracted
as is. No need to rewrite it. "R" which is a requirement that was not
precise, clear or testable. It needs to be rewritten. There are quite a
few of these. Some of these refer to minor changes, others we anticipate
much larger, more major changes to these requirements.

MALE SPEAKER 44: (Too far from microphone to understand what was being
said).

DR. SMERGIAN: Welcome aboard, J.R.

ALAN GOLDSTEIN: Another category which a few fell into was that a
requirement that appeared in Volume 1 which this complete table is
referring to really was a testing requirement and should be moved to
Volume 2.

The fourth category is, for one reason or another, the requirement was
extraneous, irrelevant, obsolete, totally misleading, incorrect,
whatever and we're suggesting, proposing that this requirement be
deleted. There aren't that many of them.

Most importantly, in addition, I guess, there's a comments column which
currently has initial observations and questions. Questions for the
community, the TGDC, for anyone who reads this. For the most part these
are not rhetorical questions. These are real questions, you know, what
is the answer to this question? We need comments to be able, to enable
us to actually do the rewriting and so on.

Our next step is to incorporate the comments that we have received and
will receive and answers to questions, into this document to provide a
clear path to revising the VSS requirement by requirement. The precise
quantification of what will be done in April versus what will be done in
November, you know, I guess, remains to be seen. Certainly we have
encountered quite a few typos, misprints, ambiguities, bad references
and so on. Those can certainly be corrected in the April time frame.

A second document which is not on the CD, which doesn't cause many of
you any problem at all, didn't quite make it. We did put it up on the
web last Friday. Its up there. Its available for your perusal. I don't
think you have a hard copy representing it either, is a first draft a
similar table for the disposition of requirements for Volume 2, the
testing volume of the 2002 VSS. Its not quite complete, sections 2
through six I think there are one or two additional sections that need
to be processed. This was what we got at the deadline and please take a
look at this as well. I think the format is identical except that "M"
now refers to things that are in this volume that should have been in
the first volume, obviously.

The next document in the package is the latest updated version of the
process model. I think there are two files comprising this. One the
model diagram itself and another a text document describing the model.
Its our attempt, this was presented in at the previous plenary so, I
don't think there is anything startling or new about it. There are some
revisions. Take a look at it. Give us your comments. It's the, our
attempt at the election process modeled with activities, objects and
flows. We are using it as a reference for organizing and
cross-referencing the compliance points that we are developing to see
where they fit into the overall process.

Another document, some of these are a little bit awkward because even if
you have the hard copy and look at it there aren't any titles on some of
these. So, its still hard to figure out which document we are talking
about. There is one called the CRT rationale pieces. This, at least, is
what its called on the web which is a discussion and explanation of
issues, sort of fragmentary discussion impacting the CRT subgroup, you
know, various issues that we have dealt with. We've tried to summarize,
you know, what the discussion was, what the thinking is on a number of
these things. They have to do with the architecture of the revised VSS,
some testing issues, coding conventions, quality assurance and
configuration management, you know, things that fall under the domain of
the CRT group.

There is also a discussion of what's called here procedural
requirements. That begins to get into the area of election procedures
which, in turn, gets into the area of instructions for voting officials
and that particular can of worms. It's a bit of our thought on it and
why its necessary to, for us to talk about this in some form, the form,
I guess, to be determined.

The document CRT products standard pieces, what this is, is a set of
detailed compliance points relating to one of the eight organizing
principles that we discussed at the previous plenary. In particular, the
one dealing with accurately accumulate, count and report votes. It's a
work, its by no means finished. Attempts to extract, identify and
identify precise testable compliance points, clarify, remove redundancy,
organize coherently. If you view the table mentioned a few minutes ago,
as sort of like a horizontal analysis of the VSS, this is a vertical
drilled down deeply for those requirements, those compliance points that
deal with one particular topic, namely, accumulate, you know, accurately
accumulate, count and report votes. It deals specifically with those
issues rather than a broad, you know, one liner analysis of every single
requirement found in the document.

The document called CRT testing pieces presents a formal mathematical
model of voting systems developed by David Slater with terms and
definitions. It applies logic verification to or shows how it can be
applied to source code, to voting systems source code and also provides
examples of abstract test cases for functional tests, capacity tests,
some others, I think, perhaps are in there which would begin to give you
a flavor of what we are envisioning for the new testing document. What
types of tests, what type of test specification is at the right level is
appropriate for a VSS as opposed to, you know, what, perhaps needs to be
developed, needs to be customized by the testing labs. So, I urge you to
take a look at that.

Finally, there is a couple of documents which propose a rewrite of the
sections in the 2002 VSS dealing with software standards. We looked at
the VSS and it seemed that the chapters on software were probably the
ones that were in the most need of rewrite of all of these. There's a
lot of material in there that is, well, lets put it generously,
obsolete. It needs to be, it needs to have a major revision. This is a
proposed revision, now more red and blue ink than anything else. It
recommends changes to the coding conventions on the source code. It does
retain and expand conventions, addressing software integrity. It does
retain some size limits on the modules, purges outdated, optional
stylistic inventions, you know, irrelevant naming conventions and things
like that. Rather than trying to tell the reader what are the precise
practices to use it defines criteria for a published credible coding
convention. You sort of say use what is considered to be the best
practices in the world out there and hear are some criteria to help you
decide whether or not these are, in fact, the credible best practices.
It required some structured exception handling.

Those are the documents. None of them are finished although the process
model is probably, I think, pretty much complete unless we get some more
comments in. We are eager to get comments, to get answers to our
questions. I'm a particular advocate for the table in there. I have
tried to highlight the questions. We really do need people, qualified
people to go through and say, you know, we missed a point here, or this
is the answer, this is the definition of the term, this is the meaning
of the term that was used and this is what was intended and so on so we
can do the appropriate rewrite.

Okay. Any questions.

DR. SMERGIAN: Thank you Alan. Questions, comments. Don't go away.

MALE SPEAKER 45: Yes. On the left side is there a hand out document that
corresponds to this?

ALAN GOLDSTEIN: Yeah, its in the package. I think the whole draft --

MALE SPEAKER 45: I don't see anything that's titled Q1-S4.

ALAN GOLDSTEIN: I forget what the title is but its there. As a matter of
fact its more than an outline, it's the entire draft.

MALE SPEAKER 45: Well, okay, they were jumping back and forth. I'm just
trying to follow your order.

ALAN GOLDSTEIN: The order is slightly different from what turned out to
be the \--. Its all there. Everything is there except for the Table for
Volume 2 of the VSS.

DR. SMERGIAN: I assume Volume 1, Section 4, is under tab "P".

MALE SPEAKER 45: The letters correspond to the front.

DR. SMERGIAN: And then, Volume 2, Section 5, is under tab "Q".

ALAN GOLDSTEIN: These are both dealing with the software one in the
requirements part of the VSS and the second one is the equivalent
section in Volume 2 of the testing part.

DR. SMERGIAN: Any other questions, comments? Anybody on the phone, any
questions or comments?

MALE SPEAKER 46: You've done a good job. Thank you.

DR. SMERGIAN: Okay, well based on these comments NIST believes the
preceding preliminary report of the technical support titled "NIST
Analysis of the 2002 VSS for the Redesigned VSS Response to TGDC
Resolutions 25-05, 27-05 and 29-05\". Unless there are supplemental
directions or corrections which I have not heard, the technical support
and related work product will continue to be developed consistent with
this preliminary report. Obviously we continue to welcome, as Alan
pointed out, your comments and your additions and corrections and,
hopefully, some answers to the questions that he has put in the tables.
So, unless I hear any other comments, we are --

DR. FITZER: I just want to commend NIST for some very good work, a lot
of hard work. We appreciate it.

DR. SMERGIAN: Well, we appreciate your comments, Dr. Fitzer. So we are
basically finished with this section of the meeting and will proceed to
consider the two Resolutions, two new Resolutions which are in your
notebook. Its tabbed "Proposed New Resolutions". It's the last tab in
your binder.

So, we have received two new Resolution from TGDC members that were
circulated to the entire committee last week and I will call on Ms. Ann
Caldess to introduce Resolution number 36-05.

ANN CALDESS: Thank you. Before I do so I would like to clarify that the
Resolution is simply the first two paragraphs of these several pages.
The additional background is simply that background and perhaps not
appropriate to be part of the, to have been included as part of the
documents that reflect the Resolution. So, that was really background
information. I will also say as I read it that I'm going to edit a
phrase out of it. So, I would like to read the proposed Resolution.

DR. SMERGIAN: Please go ahead.

ANN CALDESS: The TGDC recognizes the time frames established in HAVA as
necessary to insure a prompt response to the Nation's voting system
issues which it was written to address. The TGDC further recognizes that
a robust, voluntary consensus standards system exists in this country
and should be relied on to the greatest extent possible to facilitate
the long term development and regular maintenance of voluntary voting
system guidelines and standards. Moreover the TGDC encourages the EAC to
rely not only on this recognized expertise but also on the U.S.
voluntary consensus and conformity assessment system as exemplified by
programs and standards that comply with the requirements set forth on
OMB circular A119 as part of the EAC's long term systemic approach to
addressing the Nation's continuing needs for up-to-date voluntary voting
systems guidelines and standards.

DR. SMERGIAN: Could you point out the changes because --

ANN CALDESS: I deleted the, because I understand people misinterpreted
its inclusion, reference to accredited by the American National
Standards Institute and others.

DR. SMERGIAN: So, and you added "and standards" I believe after
programs.

ANN CALDESS: Yes, programs and standards. Right.

DR. SMERGIAN: And then simply continues as which comply with the
requirements.

ANN CALDESS: Right. That's correct.

DR. SMERGIAN: Was that the only change?

ANN CALDESS: Yes.

DR. SMERGIAN: Did everybody catch those changes? Yes. Go ahead J.R.

J.R.: I am just confirming that I heard the changes and I'm comfortable
with them.

DR. SMERGIAN: You are not comfortable with the changes?

ANN CALDESS: No, they are.

DR. SMERGIAN: He is comfortable with them. Okay.

MR. BERGER: So the Resolution ends with the word standard, voting system
guidelines and standards.

DR. SMERGIAN: We are only talking about the first two paragraphs of that
Resolution. The rest of it is background information. Is that correct?

ANN CALDESS: Yes.

DR. SMERGIAN: And basically have taken out the third line of the second
paragraph starting at accredited.

ANN CALDESS: Right.

DR. SMERGIAN: And added after programs added and standards.

ANN CALDESS: That's right.

(START TAPE 4)

DR. SMERGIAN: So I have a second?

MALE SPEAKER 47: Yes.

DR. SMERGIAN: Questions, comments.

FEMALE SPEAKER 19: I understand this is a kind of general statement
because it seems to me that we've been doing this all along. I'm a
little curious about why this exists but I'm also curious about things
like relying on existing Federal regulations. For instance in the human
factors area we are relying rather heavily on Federal regulations known
as Section 508. Given this statement it might suggest that we should be
relying more heavily on say, the web accessibility initiative which is a
voluntary consensus standard over that Federal regulation.

ANN CALDESS: I think, perhaps, it's a, you know, a little bit about OMBA
119 should also be clear that it lists the criteria for standards that
are in the public interest as open, as balance, due process and
consensus but also recognizes that where its not practical to rely on
them, that you would rely on whatever works essentially. Whether that's
a standard written by one person or a standard written, you know, by
hundreds. Again, it was really for the long term process.

FEMALE SPEAKER 19: I guess the other point that I would like to raise is
that one of the issues with some of the, I don't know how to say the
word, proprietary consensus standards is that they can be expensive.
They are often not available and they are often, in fact, not in the
public domain. So, there are issues with using them at all.

ANN CALDESS: This does not in any way mandate it. It encourages the
reliance on them, where its appropriate, because such standards are in
the public interest because they involve all those who are materially
affected stakeholders have an opportunity to contribute in an
environment that is characterized by openness, due process and
consideration of all views. It is not; it does not exclude the use of
any, but for the long term, you know, recognizes the importance of the
existing system in this country.

I would also note that NIST participates on many of those standards
development activities. They are not separate and it was not meant to
imply that they were.

DR. SMERGIAN: If I could ask a question. Do you feel that this is meant
to apply to this immediate period, you know, for the next year or so or
are you talking about beyond that?

ANN CALDESS: I think it's the long term. HAVA reads as it does and
obviously has to be implemented as such. I think its important to raise,
you know, awareness, you know, that for the long term there is a process
that exists that is in the public interest.

DR. SMERGIAN: Mr. Berger.

MR. BERGER: I'll speak in favor of the Resolution. I think at its heart
it calls for a two stage system, one in the voluntary consensus arena
where the door is open, people can come in and join committees as fully
participating members. I think that's important especially in this arena
and that work product can be taken and considered in a process such as
this to see its appropriateness and, obviously, particularly in
situations where something that's appropriate isn't available or perhaps
where that kind of open consensus couldn't find a consensus then a
process like this can be used to make decisions where they are
necessary.

DR. SMERGIAN: Well, if I may, that was sort of part of the confusion I
had in my own thinking because I thought that this committee and the
team, the NIST team, certainly included and utilized I Triple E and
other, you know, standards or draft standards or any other informational
tool that was available. Is that the focus of this Resolution or is it
more than that. Basically saying that, well, a voluntary standards
process should take over this process in the long run because my
impression wasn't that, I was thinking about more like you pointed out a
two step process which feeds into, you know, this kind of a process or
are we basically talking about going to an entirely voluntary standards
based process? Could you --

MR. BERGER: In my view what's happened recently would be an example of
implementing this Resolution so that the first step was to see what was
out and available and to the degree its applicable and its sufficient
mature to be usable. Its been incorporated and then where there are
gaps, that's where the staff time is applied. So, I would see this
Resolution as already being exemplified in the current process.

DR. SMERGIAN: Is that your thinking?

ANN CALDESS: Yeah, yeah, this doesn't intend to imply that it would be
completely under the voluntary consensus process.

DR. SMERGIAN: Any other questions from the folks on the phone?

J.R. (Speaking from phone): Yes, this is J.R.

DR. SMERGIAN: Go ahead.

J.R. (Speaking from phone): If I understand it right, we are moving
forward on type of being able to screw the ultimate (undecipherable) and
then, if it (undecipherable) its going to be much higher in terms of the
expect ability to accomplish here and that it would be (undecipherable)
baseline for all national elections. Is that correct?

DR. SMERGIAN: I don't think we could hear your last few words. Could you
repeat yourself?

J.R. (Speaking from phone): The two step meaning (undecipherable) and
then indicating to the public that this is the threshold or the standard
will be on a higher level of expectation and that we are, in fact, on a
(undecipherable) national standard for inclusion of all people in the
voting system.

DR. SMERGIAN: Thank you. Mr. Kraft.

MR. KRAFT: Yeah, this is Paul Kraft and J.R. in partial response to you,
I am supporting the Resolution. I think what this Resolution does more
so than what you described is to point out that this is a long term,
ongoing process that won't end in '06 or '08 or, hopefully, ever. As
technology continues to evolve, as we learn more in the areas of
accessibility, usability and security, this work needs to be updated at
least every other year.

FEMALE SPEAKER 20: I'm sorry, while I completely agree with you; I don't
see how this Resolution addresses that.

MR. KRAFT: I'm locking in on that fourth sentence there in the first
paragraph, to facilitate the long term development and regular
maintenance which maybe we need to say that more clear, or you need to
say that more clear.

FEMALE SPEAKER 20: I mean, its not entirely clear to me that a voluntary
consensus standard is the best way to maintain this in the long term. It
might be that an open, fair and transparent process would be the best
way to maintain it in the long run but, I haven't, for instance, seen
the access board decide to turn back to voluntary industry consensus
standards for designing accessible web sites.

FEMALE SPEAKER 21: I feel somewhat conflicted because I agree with every
word in here but I don't really quite understand why we have this and so
I think that's some of the hesitation around here. If this is simply a
statement of good principles and I have to say that the human factors
and privacy group put out a certain number of statements of good
principles in our Resolution so I suppose the pot can't call the kettle
black, then I think I'm fine with it. I just want to make sure that we
are not, in fact, creating Resolution that says more or less than we
mean to say.

DR. SMERGIAN: Yes.

FEMALE SPEAKER 22: Yeah, I just, I don't think its inconsistent with
3105 which talks about the need to develop a process for the maintenance
and the interpretations and for due process and those are old provisions
that are, at least within Nancy's world, but in other worlds as well are
a part of the required aspects of standards development, a voluntary
consensus standards development.

DR. SMERGIAN: Do we have any other questions or comments?

MALE SPEAKER 42: I call (undecipherable)

DR. SMERGIAN: Do I have a second?

MALE SPEAKER 43: I second it.

DR. SMERGIAN: Okay, Sagreen would you please take the roll call?

SAGREEN: This is 3605. Williams.

WILLIAMS: For.

SAGREEN: Williams votes yes. Berger?

BERGER: For.

SAGREEN: Berger votes for. Caldess?

CALDESS: Yes.

SAGREEN: Caldess votes for. Kraft?

KRAFT: For.

SAGREEN: Kraft votes for. Davidson?

DAVIDSON: For.

SAGREEN: Davidson votes for. Aleckies?

ALECKIES: For.

SAGREEN: Aleckies votes for. Gannon?

GANNON: Yes.

SAGREEN: Gannon votes yes. Harding?

HARDING: For.

SAGREEN: Harding votes for. Miller? Miller?

DR. SMERGIAN: She's not present.

SAGREEN: Purcell?

PURCELL: Yes.

SAGREEN: Purcell votes yes. Quisenberry?

QUISENBERRY: Yes.

SAGREEN: Quisenberry votes yes. Revest?

REVEST: For.

SAGREEN: Revest votes yes. Shutzer?

SHUTZER: Yes.

SAGREEN: Shutzer votes yes. Turner-Bowie?

TURNER-BOWIE: Yes.

SAGREEN: Turner-Bowie votes yes. Smergian?

SMERGIAN: Yes.

SAGREEN: Smergian votes yes. That's fourteen votes in favor. One voter
absent.

DR. SMERGIAN: The motion, Resolution has been adopted, number 36-05. The
next Resolution is offered by Dr. Revest, Resolution number 37-05. Dr.
Revest would you like to make a few comments.

DR. REVEST: Yes. Can you hear me okay like this?

DR. SMERGIAN: Yes.

DR. REVEST (speaking on phone): Okay. I would like to read the
Resolution and describe it briefly. It's a simplification of an earlier
Resolution that we had talked at our last meeting regarding the
availability of source codes for review. The proposed Resolution reads
as follows:

The TGDC has reviewed the issue of the availability of voting system
source codes for review. The TGDC directs this draft requirement stating
that (a) voting system vendors shall supply source codes with any voting
system submitted for evaluation; (b) the source code shall be retained
and (c) the source code shall be made available for review upon request
to local, state or Federal election officials and/or their designated
representatives. (Undecipherable) handle third party software for which
source code may be unavailable other than its authorized access to the
source code, availability of required tools such as the (undecipherable)
for source code review and possible restrictions on reports resulting
from source code reviews. That is the proposed Resolution. This is in
line with previous discussions about transparency and building voter
confidence in the systems. The availability of the source codes and
specifics what the voting machine does in detail and having that
available to state election officials and representatives I think do a
lot to increase public confidence that these voting systems are in fact
doing the right thing and (undecipherable). I'm open for discussion on
this.

DR. SMERGIAN: Do I have a second?

FEMALE SPEAKER 24: Second.

DR. SMERGIAN: Discussion, comments? Go ahead, Dr Berger.
(Undecipherable).

MALE SPEAKER 44: This issue keeps coming up and we don't seem to be able
to drive a wooden stake in its heart. There are some of that have
concerns about open availability of the source code. We are concerned
that when you put this code in the hands of every teenage hacker in the
country good things aren't going to happen. In my experience which spans
over eighteen years of dealing with voting systems, I have never had a
vendor deny me access to the source code. Its been my experience that
everyone in this industry who has a reasonable need to know can gain
access to this source code.

So, my first recommendation would be to vote this down entirely but
since its on the table I would like to propose an amendment to it. I'd
like to amend section, propose that we amend paragraph (a) to read:
Voting system vendors shall provide source code with any voting system
submitted to an ITA for evaluation for compliance with the voting system
standards.

I'd like to amend (b) to say: The source code shall be retained by the
ITA and I would like to delete paragraph (c) entirely.

DR. SMERGIAN: Do I have a second?

MALE SPEAKER 45: I'll second.

FEMALE SPEAKER 25: Is it ITA or is there a different --

MALE SPEAKER 44: Whatever.

MALE SPEAKER 46: I think its now Voting System Test Lab --

MALE SPEAKER 44: I think we've agreed we're going to use those
interchangeably.

MALE SPEAKER 46: Yeah.

FEMALE SPEAKER 25: Okay, just checking.

DR. SMERGIAN: That was actually the question I was going to ask of Dr.
Revest because this didn't say who is supposed to be given to or who is
supposed to keep. What did you have in mind?

DR. REVEST (via telephone): This was intended to be left open for NIST
to make a resolution of. I'm happy to view those modifications to (a)
and (b) the present amendment. The proposal to drop (c), I'd view as a
very unfriendly.

DR. SMERGIAN: I didn't quite hear the comment about (c).

DR. REVEST (via telephone): I said I viewed the proposal to drop the
language in part (c) as an unfriendly proposal.

MALE SPEAKER 44: Well, then let me explain why I recommended that. It
talks about making available on request to local state or Federal
election representations and/or their designated representatives. That's
entirely too vague. Who --

FEMALE SPEAKER 26: Perhaps this could be worded as to direct NIST to
write criteria under which it can be made available to appropriate
officials.

DR. REVEST (via telephone): I think the question of who should have
access and how is a little bit vague here. It needs to be clarified by
NIST. If somebody wanted to hire you, Brit, to look at the source code,
this would authorize that. That's the kind of thing that we had in mind.

MALE SPEAKER 44: I don't need it. Ron, I have never had a vendor in
eighteen years deny me access to their source code.

DR. REVEST (via telephone): There are some states that (undecipherable)
that I wanted to come in look at the source code. I think we ought to
allow that.

MALE SPEAKER 44: Its allowed by the marketplace. All the Secretary of
State has to do is say if I don't look at your source code, you aren't
going to do business with us.

DR. REVEST (via telephone): (undecipherable).

DR. SMERGIAN: Mr. Kraft.

MR. KRAFT: Yes. The issue as to the availability of source code, the
fair use of the source code, the protection, the intellectual property
rights, that's primarily, its state law. In our state we have a law that
requires the source code to be provided to our office and there is also
a law that makes it a felony for me to disclose it for anyone's use
outside of its intended use. I think most states have that and also the
marketplace control. First off our whole concept of voting system
testing labs here, those labs cannot do their job if they don't have the
source code. Beyond that any state and any local jurisdiction who is
controlling the check to the vendor can see source code any time they
want to.

DR. SMERGIAN: I might add there is another reason beside all this why
you might want to consider doing it and, of course, I don't see the
difficulty if you have ever experienced a problem, what's the problem
putting in. Another reason why you might want to do it, which we write
in a lot of our contracts, and I think there's a distinct possibility
with some of the vendors, is that we like to hold the source code in
escrow in the event that something happens to that vendor, so if it was
not available we would like to at least maintain it and continue on.

MALE SPEAKER 44: Right. But that is a contractual. That's something
that's usually handled contractually at the time you buy the system
because what you want in escrow is the system you are using. This is
talking about asking everybody for everything. You are talking about
specifically escrowing the software you are using and I think everybody
addresses that at contract time.

FEMALE SPEAKER 27: I guess I want to agree with Mr. Schutzer which is
that if what we are talking about is embodying in the standards
something that is already common practice, I don't understand what the
problem is. If what we are talking about is creating new requirements
that people feel are unacceptable we'll see that when the requirements
come out from NIST.

MALE SPEAKER 44: Well, this expands what's already in the standards and
what's in practice. This basically requires the source code to made
available, upon request, to local, state or Federal election officials
and/or their designated representatives. It doesn't put any bounds
around what those jurisdictions may do with that code, who those
representatives might be.

FEMALE SPEAKER 28: I'm sorry but I think we have already proposed,
although we didn't get final language on it, that we bound this to say
not at this level of resolution that it doesn't just say to any official
in their thing but that part of the requirement shall specify who and
how and, you know, how that's determined.

MALE SPEAKER 44: What's a local official? If the dog catcher wants it
can he get it?

FEMALE SPEAKER 28: He's not an election official.

MALE SPEAKER 44: It doesn't say election official.

FEMALE SPEAKER 28; It does. It does.

MALE SPEAKER 44: Well, yeah, okay or their designated representative,
yeah.

FEMALE SPEAKER 28: I mean, I would take your point about the open-ended
of this but I would say that a part of what the requirements would have
to do is say who might be eligible.

MALE SPEAKER 44: Well, Dr. Revest has accepted my changes to (a) and (b)
and, so I will withdraw my recommendation to omit (c) if someone would
like to reword that.

DR. SMERGIAN: May I ask a question? If this Resolution passed exactly
what it is that you are asking NIST to do? I mean, it seems to me this
is already regular practice and it seems to me people are comfortable
with the definition of local, state and Federal election officials
and/or their designated representatives, so, exactly what it is that you
expect NIST to do?

MALE SPEAKER 44: Well, I guess what I would like to see is bounding this
closer to what is actually desired. The reason we get source code from a
vendor is number one, so we can review it and we can make sure that it
is properly prepared, decent code and we can do all the analysis and
including me that we don't do now that we will do in the future for the
purpose of determining whether or not a system can be qualified.

The second purpose for source code and the reason that we escrow it is
not so that we can go compile it and build our own voting system when
and if the vendor goes broke, but so that when he brings his next
revision into us, and he gives us technical release notes on what he has
changed in that code, we can do a code compare, the new code versus the
old code and determine if, in fact, he made changes outside of that and
we can then lay in test lands that are specific to those changes. That
is really the only utility in this program that source code has. That's
what the vendors make it available for. That's how its used and for
those people who are engaged in that particular work, the vendors have
no problem making it available. So to make it available outside that
scope, I think its something we shouldn't do.

FEMALE SPEAKER 28: Okay. So, I can propose some wording changes that, if
I do it slowly, I think are quite simple and don't require a lot of
redrafting. I'll need your help, Paul, (undecipherable).

PAUL: Okay.

FEMALE SPEAKER 28: So, back up at the first sentence, TGDC directs NIST
to draft requirements stating. You need to put the colon after stating
and drop the that.

PAUL: Okay.

FEMALE SPEAKER 28: The word that then needs to appear at the beginning
of (a), so stating that voting system vendor shall supply. Beginning of
(b) that the source code shall be retained, and (c) the conditions under
which the source code shall be made available for review, remove upon
request, and Paul, I'll let you take the rest of that paragraph. Hang on
for a second. Go down to the paragraph below it. These requirements
shall address a number of issues.

PAUL: Okay and NIST shall clarify these requirements.

FEMALE SPEAKER 28: That these requirements shall address. So the
question I have for you is do we need to change this for review, the
phrase after for review to local, state or Federal election officials to
clarify the point that you are making. I don't think that anybody is
implying that we should give this away to teenager hackers or even
people who just happen to be curious.

PAUL: How about if we change (c) to the source code shall be made
available for review upon request to local, state or Federal election
officials for their use in approval, purchase or certification
assistance.

DR. SMERGIAN: I have a fundamental question. What is this going to allow
you to do that you cannot do now?

PAUL: Nothing.

MALE SPEAKER 44: This does not contribute anything to the existing
standards. That's why I'm

DR. SMERGIAN: So, I guess then my question is --

FEMALE SPEAKER 28: Maybe that question should be addressed to Dr. Revest
about why he wanted --

DR. SMERGIAN: Dr. Revest would you like to comment on that?

DR. REVEST (via telephone): I think there is a perception that these
voting systems are black boxes that, you know, no one except the vendors
maybe know what's going inside. I understand that common practice has
the source code available to state election officials. I think part of
the point of having this as a requirement, even though if they codified
what is common practice today, is to increase public confidence that the
public's representatives do have the right type practice but the right
standards to have access to the source code and to allow critical of the
source code for their purposes.

MALE SPEAKER 44: I'm not sure they should be making the standards styled
in such a way to address things that everybody sitting at the table know
really are not an issue. These are not black box systems and they are
not reviewed and black box tested.

DR. SMERGIAN: Mr. Berger.

MALE SPEAKER 48: (Undecipherable) software system and a black box
standard except at the market forces you are talking about. I think the
standard is simple and the market forces to set expectations
(undecipherable) transparency to have in the marketplace. Its just a
matter of protecting the principle, I think. We have a statement of
principles here. Some of us are happy with principles and I'm sure
(undecipherable).

DR. SMERGIAN: Mr. Berger.

MR. BERGER: It seems to me the key point on (c) is if the vendor
protests the distribution of the software. Clearly if the vendor agrees
to a particular official or their designated representatives receiving
it, its going to happen anyway. It happens currently. I would suggest
some language be added in (c) that NIST recommend when, over vendor
protest, the software, the source should be distributed. I'm not sure
that such a case exists but someone ought to think that one through
carefully.

MALE SPEAKER 49: Well, I guess, looking at, basically as this is written
this would allow Brit Williams in Georgia, who has a Diebold state owned
system, statewide to force ES&S to hand over their source code.

MR. BERGER: Or the other example would be if the election official from
Cut and Shoot, Texas, had a brother-in-law who has a competing company,
he could get everyone else's source code under this. I don't think
that's our intent.

FEMALE SPEAKER 28: I think that's the point of being sure that we
specify what the conditions are and we didn't finish the sentence that
you started so we might want to finish that before Alan gets lost in the
ether.

For the use in evaluating systems for purchase something or something.

ALAN: For purchase, certification and I forget what the third one was.

DR. SMERGIAN: Let me ask another question. It seems to me what I'm
hearing is, this is current, the current practice is accomplishing what
you want to accomplish.

MALE SPEAKER 44: Yes. It's a little stronger than that. The current
standard specifies that the vendors shall submit the source code with
their technical data package when they submit the system for review.

DR. SMERGIAN: Its in the 2002 -- so then, if that is there, and if we
don't want to change the procedures or the requirements or the criteria,
is the idea, Dr. Revest, I guess maybe this is addressed to you, is the
idea to have a resolution that publicly states that this is what
happens, we don't have black boxes, that these are contractually as well
as part of the requirements of 2002 VSS that vendors provide, you know,
source code for whatever purposes, either for, you know, safekeeping or
whatever and that this practice should continue. I'm trying to see, when
you start saying draft, you direct NIST to draft requirements, I'm not
sure what we are developing requirements for. I'm trying to understand
what it is that I would ask our people to do in response to such a
resolution.

DR. REVEST (via telephone): (Undecipherable) standards merely codify
what is common practice and I think we are essentially in agreement here
that its common practice for source code to be available to election
officials and their team to evaluate for the purposes of prospective
purchase or maybe to evaluate what's going on with the system they've
already purchased and so on. I think that this (undecipherable) a
situation where we are essentially codifying common practice. It doesn't
go as far as the Clinton Boxer proposal did which would have made source
code public, available to the public with no control whatsoever. They
tried to draw reasonable boundaries on who would have access to the
source code. That boundary corresponds to common practice.

MALE SPEAKER 48: I believe the standards are useful sometimes even when
the do correspond to (undecipherable).

DR. SMERGIAN: I agree with that but what I'm told is that its already in
the standards. Its in the 2002 VSS.

DR. REVEST: (Undecipherable) in the standard for making it available to
the ITA. I don't believe its in the standard to make it available
prospective purchasers.

MALE SPEAKER 44: And, some software purchasers don't want it. Those that
do want it, get it because you are a purchaser they are going to give
you what you tell them you need.

MALE SPEAKER 50: I wouldn't have any objection to wording that said
purchasers.

FEMALE SPEAKER 28: You said what?

MALE SPEAKER 50: To purchasing the system. This says anybody whether you
are purchasing it or just want to look at it or what.

MALE SPEAKER 51: That's your objection. It's a system you haven't
purchased.

FEMALE SPEAKER 28: Yeah. So purchasers (undecipherable).

MALE SPEAKER 44: I think I'm going to have to speak against this because
I don't think, I was thinking that, you know, we could do a somewhat
friendly amendment but, I think this goes outside of the scope of voting
system evaluation. It goes outside the scope of the standards. It goes
more to a national policy of making source code available on a wider
basis than is needed for evaluating and certifying systems.

DR. SMERGIAN: Mr. Berger and then Secretary Davidson.

MR. BERGER: Actually I think we've brought up another issue that's
probably worth looking at and that is, are we, in changing the system,
because currently the source code is never given to any Federal entity.
In the system we are constructing is this now given to a Federal entity
and can it be protected even under the assumptions all of us would
assume, against a request under Freedom of Information. That's probably
worth a look to make sure that someone's not inadvertently making their
source code publicly available.

DR. SMERGIAN: Secretary Davidson. Just a minute.

SECRETARY DAVIDSON: Number one, I think there's two problems that I
really see. I don't want to leave it in the hands of the locals and
leave it there because those are the ones that programming the elections
and if you ever wanted to tamper, that gives them an edge.

Number two, why the Federal. The state is running the Federal election
so, who are you going to give it to at the Federal level?

DR. SMERGIAN: We don't want it. Yes, J.R., you had a question or
comment.

J.R.: Yes, I was going to suggest to our committee chairman that
(undecipherable) will be evolving the standards for some time. Let's get
past the (undecipherable) and then re-examine the issue if he still
thinks its needed.

DR. SMERGIAN: Dr. Revest would you --

DR. REVEST (via telephone): Yes, I think it is a long term issue in some
sense. I think, you know, it is my opinion that the earlier the better
just to make sure the source code is available for review. I think the
issue that Dr. Berger raised on the (undecipherable) issue is one I
hadn't considered. That's an interesting point. That might be sufficient
cause for (undecipherable) on that score but I think, and I'm looking
(undecipherable) J.R. (undecipherable). The point (undecipherable) the
transparency should be as high as possible. I think there's a lot of
mistrust among the public at large for voting officials and if the
source code is not available and we have attempted to mitigation that.
The goal here is to increase confidence and trust in the voting systems
and if the source code is not available except to (undecipherable) only
doing a limited review I think, in my mind that's something that
(undecipherable). I feel its an important point (undecipherable) this
committee could take a stand on this point (undecipherable) and I
appreciate the (undecipherable) to try to word it in a way that would
make it most acceptable.

DR. SMERGIAN: So, would you withdraw this and, perhaps, consider some of
the comments made here and if you want to bring up at a future meeting,
it would still be timely.

DR. REVEST (via telephone): I appreciate the comments. As I said, I
think the one about the FOIA is one I would not want to have a trap like
that sprung on us. So, yes, I'm going to withdraw it at this meeting and
proceed to take the comments under advisement and (undecipherable) to
TGDC for wording changes and (undecipherable) that can be brought into
this.

DR. SMERGIAN: Thank you. Did you want to make a comment, Ms. Purcell?

MS. PURCELL: I was just going to follow up on what Secretary Davidson
said. I don't believe the local officials want that responsibility
either. We have the same issues of public records law that come into
play a great deal.

DR. SMERGIAN: Well, yes, Secretary Davidson.

SECRETARY DAVIDSON: I think, just to add, every state has different open
record acts. They are very different throughout the United States and
that, I mean, its going to take lot of research.

DR. SMERGIAN: Okay. Then, we considered this Resolution withdrawn and
Dr. Revest will reconsider it, you know, especially in light of the
comments and see if he wants to bring it up again.

Well, believe it or not, that completes our tasks for today. A job well
done, I must say. Certainly the Resolution adopted resolutions and the
ideas expressed, adopted at this plenary meeting further instruct NIST
staff on the research and drafting of standards recommendations. The
adopted motions provide essential policy guidance on relevant voting
standard issues and NIST staff, in cooperation with the TGDC members
will continue to make best efforts to accomplish the critical tasks most
urgently needed by election community as part of their recommendations
to the EAS.

I appreciate the participation today of all of the committee members. I
want you to notice that all of the members were here at least part of
the or participated at least part of the meeting and I very much
appreciate that participation of all the members. I think that tells you
how seriously each one of you take this task and we look forward to
working with you in the coming month. My contact information is in your
meeting notebook so if you have any specific issues that you want to
discuss with me privately, you know where I can be reached. Its in the
list of membership.

I also want to thank the NIST scientists for their efforts to make this
meeting a success. I'm sure, its not just from the weight of the paper
that you received but I think from the quality of the work that they
have been working very, very hard and, as acting Director of NIST I'm
certainly extremely proud of the effort that they have put in. So, thank
you, great job.

Our next plenary session of the TGDC is scheduled for April 20 and 21. I
actually have a conflict that has developed and I was wondering if
instead of 20 and 21, which is a Wednesday and Thursday, if we could
move it to 21 and 22 which is Thursday and Friday. Would that be a
problem? Dr. Williams.

DR. WILLIAMS: (Cannot hear what is being said).

DR. SMERGIAN: Well, that sounds like maybe we'll have to leave it alone.
Is everybody going to be available so far as you know?

MALE SPEAKER 49: Wednesday and Thursday?

DR. SMERGIAN: Okay. Well, then we'll stick with that. Again, thank you
to the TGDC members and thank you to all the other folks who
participated and listened in and thank you to Commissioner Martinez and
Commissioner DeGregorio for sitting in. With that I adjourn this meeting
of the Technical Guidelines Development Committee. Thank you very much.

(**END OF AUDIOTAPE**)

\* \* \* \* \*

[CERTIFICATE OF AGENCY]{.underline}

I, Carol J. Schwartz, President of Carol J. Thomas Stenotype Reporting
Services, Inc., do hereby certify we were authorized to transcribe the
submitted cassette tapes, and that thereafter these proceedings were
transcribed under our supervision, and I further certify that the
forgoing transcription contains a full, true and correct transcription
of the cassettes furnished, to the best of our ability.

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

CAROL J. SCHWARTZ

PRESIDENT