TGDC MEETING

WEDNESDAY, JANUARY 19, 2005

**(START OF AUDIOTAPE)**

DR. SMURGIAN: Good morning everyone, and welcome back. For those of you
who were not here, I'm (Unintelligible) Smurgian, Acting Director of the
National Institute of Science and Technology, and Chairman of the
Technical Guidelines Development Committee.

I hereby call to order the second day of this committee's planerary
meeting, today, Wednesday, January 19, 2005.

> Now let's stand and pledge allegiance.
>
> (Pledge of Allegiance)
>
> Thank you. At this time I recognize Mr. Greg Burkhart

as the TGDC Parliamentarian and request that he determine if a quorum of
the committee is present. Mr. Burkhart, roll call for attendance.

MR. BURKHART: (Off microphone). You just want a roll call for
attendance?

DR. SMURGIAN: Roll call for attendance.

MR. BURKHART: Smurgian?

DR. SMURGIAN: Here.

MR. BURKHART: Davidson? Davidson?

MS. DAVIDSON: **(No response heard)**.

MR. BURKHART: Miller?

MS. MILLER: **(No response heard)**.

MR. BURKHART: Turnerbuoy?

MS. TURNERBUOY: (No response heard).

DR. SMURGIAN: Let's go ahead and we'll come back because it's
(unintelligible).

MR. BURKHART: Purcell?

MS. PURCELL: **(No response heard)**.

MR. BURKHART: Harding?

MR. HARDING: Here.

MR. BURKHART: Ellekese?

MR. ELLEKESE: Here by teleconference.

MR. BURKHART: Havas?

MR. HAVAS: **(No response heard**).

MR. BURKHART: Burger?

MR. BURGER: Here.

MR. BURKHART: Williams?

DR. WILLIAMS: **(No response heard)**.

MR. BURKHART: Kraft?

MR. KRAFT: Here.

MR. BURKHART: Revest?

DR. REVEST: **(No response heard)**.

MR. BURKHART: Schutser?

DR. SCHUSTER: **(No response heard)**.

MR. BURKHART: Gannon?

MR. GANNON: **(No response heard)**.

MR. BURKHART: Quesenberry?

MS. QUESENBERRY: Here.

MR. BURKHART: Going back, Davidson?

MS. DAVIDSON: **(No response heard).**

MR. BURKHART: Miller?

MS. MILLER: **(No response heard)**.

MR. BURKHART: Turnerbuoy?

MS. TURNERBUOY: **(No response heard)**.

MR. BURKHART: (Off microphone). I advise you that a quorum is present.

DR. SMURGIAN: Thank you. So we do have a quorum.

At this time I note that the latest revised version of Roberts Rules of
Order was adopted on our July 9, 2004 meeting to govern Technical
Guidelines Development Committee and Subcommittee proceedings. And I
call on Mr. Burkhart to see if we have any logistics to be reviewed for
this second meeting of the TGDC.

MR. BURKHART: (Off microphone).

DR SMURGIAN: I guess there are no other logistic issues.

Yesterday we started with resolutions put forth by the Human Factors and
Privacy Subcommittee and then moved on to Security and Transparency
Subcommittee resolutions.

However there were a couple of resolutions by the Human Factors and
Privacy Subcommittee that were to be modified and then put forth for
adoption this morning.

You have in your package that was handed to you this morning one page,
numbered 50, that has these resolutions, and perhaps the Chair, Ms.
Quesenberry can walk us through these.

MS. QUESENBERRY: I'd be happy to do so. Yesterday we voted and passed
Resolution 3-05. When we got to Resolution 7-05 the question was raised
about this being so close in nature to the previous resolution that
perhaps they should be combined.

So I offer as an amendment to 3-05, proposed new text, which is before
you. I'll read it out as before.

It's Human Factors and Privacy of Voting Systems at the Polling Place.
It reads; the TGDC has considered the issue of what is required to
insure both access to the voting system by voters with disabilities, and
usability and privacy for all voters.

It is included that usability, accessibility, and privacy are functions
of both the system used to vote and the environment at the polling
place.

The TGDC directs NIST to research and draft guidance on the deployment
and configuration of systems in the polling place to insure usability,
accessibility, and privacy. These guidelines should be combined with the
accessibility standards described in Resolution 2-05 or the standards
described in Resolution 4-05.

I think I've managed to preserve all of our notes from yesterday and
simply combined the two points.

DR. SMURGIAN: Do I have a second for Resolution 3-05 as modified?

MALE SPEAKER: I'll second (unintelligible).

DR. SMURGIAN: Okay. Any discussion, comments? Sounds like everybody's in
violent agreement.

FEMALE SPEAKER: I think I have to say something. Do we have to say at
least one thing?

DR. SMURGIAN: Well I just did.

MALE SPEAKER: I think what you said earlier is \--

FEMALE SPEAKER: Okay.

DR. SMURGIAN: So I'll move to a vote. Mr. Green, will you please take a
roll call?

MR. GREEN: This is to amend \--

DR. SMURGIAN: This is Resolution 3-05 as read this morning, which will
replace the Resolution 3-05 we voted on yesterday. Is that correct?

MS. QUESENBERRY: Yes.

MR. GREEN: Okay, roll call; Smurgian?

DR. SMURGIAN: Yes.

MR. GREEN: Davidson?

MS. DAVIDSON: **(No response heard)**.

MR. GREEN: Miller?

MS. MILLER: Yes.

MR. GREEN: Turnerbuoy?

MS. TURNERBUOY: Yes.

MR. GREEN: Purcell?

MS. PURCELL: Yes.

MR. GREEN: Harding?

MR. HARDING: Yes.

MR. GREEN: Ellekese?

MR. ELLEKESE: Yes.

MR. GREEN: Caldise?

MS. CALDISE: Abstain.

MR. GREEN: Burger?

MR. BURGER: Yes.

MR. GREEN: Williams?

DR. WILLIAMS: Yes.

MR. GREEN: Kraft?

MR. KRAFT: Yes.

MR. GREEN: Schutser?

DR. SCHUTSER: Yes.

MR. GREEN: Gannon?

MR. GANNON: Yes.

MR. GREEN: I missed Revest.

DR. REVEST: Yes.

MR. GREEN: And Quesenberry.

MS. QUESENBERRY: Yes.

MR. GREEN: That's 13 votes for, one vote absent, one voter abstaining.

DR. SMURGIAN: Thank you. The Resolution 3-05 as presented has been
adopted.

Would you like to go for the next one, Ms. Quesenberry?

MS. QUESENBERRY: Yes, thank you. I offer a substantial redraft, I guess
it's officially an amendment but it's in fact a substantial rewrite of
Resolution 6-05 following our discussion yesterday and hoping to
incorporate the wisdom we learned there.

The revised title of this resolution is Accommodating a Wide Range of
Human Abilities. And it reads; the TGDC recognizes that there is a wide
range of human abilities. The voting population includes not only people
with specifically identified disabilities but also the aging population,
language minorities, and people with other special needs.

A goal of voting systems should be to accommodate as much as possible
this wide range of abilities to insure the greatest usability and
accessibility of those systems. This approach is sometimes called
universal design or universal usability.

In drafting standards the TGDC directs this to; one,

consider what accommodations to voter abilities can be included in the
standards for all voting systems using currently available technology,
and number two, develop guidelines for universal design based on
existing Best Practices and other guidelines or standards such as 3-CFR
1194, known as Section 508, to guide future standards development to aid
in updating the voting systems standards.

DR. SMURGIAN: Ms. Quesenberry, you read the second paragraph, develop
guidelines instead of principals. Is that change?

MS. QUESENBERRY: No, I'm sorry. Develop principals for a universal
design.

DR. SMURGIAN: Okay. We have a motion on the table. Do we have a second?

MALE SPEAKER: Second.

DR. SMURGIAN: Thank you. Any questions, comments, discussion?

MALE SPEAKER: I think that resolves the questions that I had yesterday
so I support it.

DR. SMURGIAN: Thank you. Hearing no other comments I move to a vote. Mr.
Green, would you please take a roll call? We are voting on Resolution
6-05 as presented this morning and read for the record.

MR. GREEN: Roll call; Smurgian?

DR. SMURGIAN: Yes.

MR. GREEN: Davidson?

MS. DAVIDSON: (No response heard).

MR. GREEN: Miller?

MS. MILLER: Yes.

MR. GREEN: Turnerbuoy?

MS. TURNERBUOY: Yes.

MR. GREEN: Purcell?

MS. PURCELL: Yes.

MR. GREEN: Harding?

MR. HARDING: Yes.

MR. GREEN: Ellekese?

MR. ELLEKESE: Yes.

MR. GREEN: Caldise?

MS. CALDISE: Abstain.

MR. GREEN: Burger?

MR. BURGER: Yes.

MR. GREEN: Williams?

DR. WILLIAMS: **(No response heard)**

MR. GREEN: Kraft?

MR. KRAFT: Yes.

MR. GREEN: Revest?

DR. REVEST: Yes.

MR. GREEN: Schutser?

DR. SCHUTSER: Yes.

MR. GREEN: Gannon?

MR. GANNON: Yes.

MR. GREEN: Quesenberry?

MS. QUESENBERRY: Yes.

MR. GREEN: That's 13 votes for, one voter abstaining, one voter absent.

DR. SMURGIAN: Thank you. The motion is carried and Resolution 6-05 as
amended and as read for the record is adopted.

MALE SPEAKER: Mr. Chairman?

DR. SMURGIAN: Yes.

MALE SPEAKER: Do we need to do something to close out 7-05?

DR. SMURGIAN: Well I thought that that actually was withdrawn yesterday
so we never really took any action yesterday on that so I assume we
don't need to withdraw. That was done yesterday already.

MALE SPEAKER: Okay.

DR. SMURGIAN: So with that I believe all the resolutions presented by
the Human Factors and Privacy Subcommittee have been covered and thank
you for a great job by the committee, subcommittee, and staff supporting
their activities.

Unless there is any objection, we will now go back to the resolutions
presented by the Security and Transparency working group, where we left
off yesterday.

And again, for your information, there was one resolution yesterday,
which was numbered \-- well there were two resolutions numbered the same
and that has been renumbered as we did yesterday, and presented to you
in hard copy as Resolution 35-05, titled Wireless. So Dr. Revest, the
floor is yours.

DR. REVEST: Thank you, Chairman Smurgian. Good morning, fellow committee
members.

I'd like to continue with the list of resolutions that we had on the
table from the Security and Transparency Subcommittee.

The Committee had formulated 13 all together. We took care of three of
them yesterday. That leaves ten. I've decided to proceed in an order,
which I think would be most beneficial today should we perhaps not get
to all of them.

DR. SMURGIAN: May I make a comment on that, please? I think our
objective is to pass as many resolutions as we can today but I don't
want to shortchange so to speak the Core Requirements and Testing
Working Group. Therefore we will consider your highest priority
resolutions of the Security and Transparency Subcommittee until the
break this morning.

At that time we will stop and move on to the next subcommittee and then
if at the end of the day we have time, we will come back and consider
the remaining resolutions from your subcommittee.

If we don't get there we will then pick up the remaining resolutions
during our February meeting. Thank you.

DR. REVEST: Thank you, Mr. Chairman.

> The first resolution I'd like us to consider would be

16, which is Set Up Validation. Let me read it to you.

The TGDC has considered the issue of Electronic Voting Machines Set Up
Validation and has concluded the current standards and practices needs
substantial improvement in this regard. A setup validation method
insures that a voting system contains the authorized software, contains
no unauthorized software, and is in the proper initial state.

The TGDC requests NIST to do research and develop standards, one, that
specify the characteristics of acceptable setup validation methods such
as for example, that the setup validation method may not modify the
state of the system nor require the execution of any software, and let
me insert the word currently here, currently on the system, and two,
that require each voting system submission to specify an acceptable
setup validation method.

So I'd like to propose this as a resolution to be considered.

DR. SMURGIAN: Thank you. Do we have a second?

MR. KRAFT: I will second for the purpose of discussion.

To an extent this covers \--

DR. SMURGIAN: Excuse me. I would like to make sure that everybody gets a
chance so please raise your hand and I will try to recognize everybody
in order so that we have a little more orderly discussion. Mr. Kraft.

MR. KRAFT: Okay, this in part covers the territory that we covered in
July with Resolution 5 from the Technical Guidelines Development
Committee that met then, which charged NIST with setting up a validation
process through the National Software Reference Library.

I don't see any particular conflict here but I would like to see the
language be such that it makes it clear that the TGDC wishes NIST to
create a utility that will allow local election officials, interested
citizens, and the courts in local jurisdictions to validate their
software and their setup.

DR. REVEST: I think that would be intended that this would be a method
to be widely applicable, not just by say a vendor, and it certainly has
the possibility of using the National Software Reference Library as
developed by NIST. The characteristics in part one would presumably
cover those grounds.

DR. SMURGIAN: Dr. Williams.

DR. WILLIAMS: Yeah, I just wanted to comment on that. We want to keep in
mind that we're writing standards for systems and NIST is not a
developer of systems nor an ITA.

What we need here is to be sure that the standards contain language that
require the vendors to place in their software, the hooks that allow you
to perform these activities.

We have a problem with that right now. We can use the techniques of the
software library to test our election management system but we haven't
been able to get the vendor to provide us with the hooks to get the
software out of the voting station where we can perform those same tests
on it.

And so those are the kind of things I'd like to see in the standards
that require the vendors to provide us the facility to do this type of
testing.

DR. REVEST: If I could comment on that. I think one of the risks that is
being covered here is the risk that the vendor may have either
maliciously or accidentally somehow supplied faulty software and so the
point is to be able to trust the setup of the system without trusting
the software that's on the system currently.

DR. SMURGIAN: Mr. Kraft.

MR. KRAFT: May I suggest, Dr. Revest, since at this moment Resolution 5
is out there \-- I've been discussing with NIST staff and my staff, the
state of Georgia, and a couple of the vendors \--

DR. SMURGIAN: Clarification please. The Resolution 5, you're referring
to is from the July meeting?

MR. KRAFT: Correct. We are moving forward next week to begin a pilot
project for development of the software validation steps so I think
perhaps it would be most appropriate to table this resolution until a
future meeting of the TGDC \-- take a work product from the pilot
project, which will begin next week and readdress it in a future
meeting.

I mean that work is ongoing and I'm not sure that this resolution
necessarily contributes anything positive to it.

DR. REVEST: I think the point here if I may respond, is that this is a
testing method that's more then just, you know, how to compare hashes
and so on to \-- which is mostly what that addressed.

The point here is you need an active procedure for taking a machine,
which you're suspicious of having been modified somehow and may have had
operating system changes and so on too \--

MR. KRAFT: Right, we understand that.

DR. REVEST: Yes. And so I think that the goal here is somewhat broader
then what the previous resolution addressed.

Moreover in terms of the timeframe, I think the timeframe we're talking
about work for NIST to do, having that go concurrently with what you're
doing seems a perfectly reasonable procedure to me.

DR. SMURGIAN: Any other \-- Ms. Quesenbery, did you \--

MS. QUESENBERRY: No.

DR. SMURGIAN: Any other \--

MALE SPEAKER: I support Dr. (Unintelligible) on this, that this is a
critical area in the standards. You know, when we start prioritizing
things this is going to be up close to the top and so although I agree
with Paul that it's somewhat redundant, I don't see a problem with
re-passing this as an emphasis of the importance of this particular
activity.

DR. SMURGIAN: Any other comments, questions?

MALE SPEAKER: Mr. Chairman?

DR. SMURGIAN: Yes, Mr. (Unintelligible).

MALE SPEAKER: For those of us who are not quite as with it in terms of
the experts on this panel, what is then the end goal of tabling it or
the outcome that will help us get to some kind of library reference
system for local officials to verify the software or the setup
validations of the equipment?

DR. SMURGIAN: If I may comment, my understanding is that \-- I mean
there are really two issues.

One is the integrity of the software as it is delivered let's say. The
other one is the entire integrity of the system on which the software
resides along with other operating systems et cetera. Is that an
appropriate interpretation?

DR. REVEST: Yes. It's not just the delivery of the system to the local
officials but also the installation. If you're starting up in the
morning and you have specified for example a spot check that said that
one of the machines in every precinct should be randomly chosen and
carefully tested to see that it contains the software it's supposed
have, doesn't contain any software it's not supposed to have, and all
\-- the appropriate switch is set, the wireless is on or off or whatever
you want to have. All of these things are set up properly.

There needs to be a procedure that the vendor specifies that can be
followed by a local election official that allows the local election
official to confirm that this election machine is really representative
of what got qualified by the testing laboratory and certified by the
state.

DR. SMURGIAN: Mr. Kraft.

MR. KRAFT: I think we may have just crossed a line that I don't want to
go across there, that we are not going to prescribe a procedure of
election administration. We are simply going to require the systems as
presented to have the capability of being validated and provide a
resource through NIST for that validation and a process that's available
to the local election officials. We're not going to get in prescribing
\--

DR. REVEST: If I misspoke on that, I stand corrected. The intent is to
make sure there is at least an acceptable procedure for doing this
validation available should the election officials or other officials
want to do that.

MR. KRAFT: Okay. Well I'll support the amendment as on the table at this
point.

DR. SMURGIAN: Ms. Quesenberry.

MS. QUESENBERRY: I would just add one comment and I'm not proposing that
we merge the efforts, although at the time we might, that we just
considered a resolution that looked at the proper setup of accessibility
devices and so on, so there's certainly a line that's coming together
there about making sure that the documentation that comes with the
system is adequate to insure that someone who followed it would have a
well working system.

DR. SMURGIAN: I think that's certainly a very important point and I
think the comment was made earlier on that the subcommittees, they've
been working sort of independently, but clearly there are a lot of areas
of overlap and that once these resolutions have passed and the work
products proceed, there will have to be a review by the subcommittees as
well as by the entire committee to make sure that things that are
related to each other are treated as such rather then as independent
objectives.

MS. QUESENBERRY: Yes.

DR. SMURGIAN: So I think that's something that we perhaps need to keep
in mind and take a look at, at a later time once the work product is
developed a little further.

MS. QUESENBERRY: I would note that the subcommittee chairs have in fact
been passing material back and forth so that we're watching for places
where we could either help inform the process or where two work products
would in the end combine because in the end they go together even though
they might be developed separately.

DR. SMURGIAN: Okay, it sounds like that's the end of the discussion. Not
hearing any further comments I move for a vote. Mr. Green, would you
please take the roll call.

MR. GREEN: This is Resolution 16\--05 as amended?

DR. SMURGIAN: Yes, as presented with one minor edit, adding the word
currently after the word software in the sentence within the
(unintelligible). Reading, any software currently on the system.

MR. GREEN: Roll call; Smurgian?

DR. SMURGIAN: Yes.

MR. GREEN: Davidson?

MS. DAVIDSON: **(No response heard).**

MR. GREEN: Miller?

MS. MILLER: Yes.

MR. GREEN: Turnerbuoy?

MS. TURNERBUOY: Yes.

MR. GREEN: Purcell?

MS. PURCELL: Yes.

MR. GREEN: Harding?

MR. HARDING: Yes.

MR. GREEN: Ellekese?

MR. ELLEKESE: Yes.

MR. GREEN: Caldise?

MS. CALDISE: Abstain.

MR. GREEN: Burger?

MR. BURGER: Yes.

MR. GREEN: Williams?

DR. WILLIAMS: Yes.

MR. GREEN: Kraft?

MR. KRAFT: Yes.

MR. GREEN: Revest?

DR. REVEST: Yes.

MR. GREEN: Schutser?

DR. SCHUTSER: Yes.

MR. GREEN: Gannon?

MR. GANNON: **(No response heard)**.

MR. GREEN: Quesenberry.

MS. QUESENBERRY: Yes.

MR. GREEN: That's 12 votes for, two voters absent, one voter abstaining.

DR. SMURGIAN: Before I get to the vote, I understand we're getting
interference on the teleconference system because someone is calling on
a cell phone. Whoever that person is needs to get on a landline. That's
why we're getting the interference. So we're not quite sure who the
person is but will you please make sure that you are using a regular
phone? Thank you.

So the vote on the Resolution 16-05 is 12, and two abstaining, and one
absent, is that correct? Two absent and one abstaining, okay. So the
resolution has passed as presented and modified by the author. Thank
you. Dr. Revest.

DR. REVEST: Thank you, Chairman Smurgian.

I'd like to move on to the next resolution to be considered this
morning, which would be 17, on Testing.

Let me read it. The proposed resolution reads; the TGDC has concluded
that voting systems must be subject in general to the same tests used
for systems of critical importance in the U.S. federal government.

Accordingly these tests must include a "penetration analysis" of the
voting system, methods for system exploitation change as new software is
introduced, as new vulnerabilities are found. Therefore the "penetration
analysis" must rely on techniques as will evolve over time.

The TGDC directs NIST to research and draft standard documents requiring
testing of voting system that includes a significant amount of open
ended research for vulnerabilities by an analysis team supplied with
complete source codes, and system documentation and operational voting
system hardware.

The vulnerability sought should not exclude those involving collusion
between multiple parties including vendor insiders, and should not
exclude those involving adversaries with significant financial and
technical resources.

The point to this resolution, Mr. Chairman is to make sure that we have
the ability for a team to look broadly to look for things that might be
overlooked, to do more then just follow a checklist of sort of well
known vulnerabilities, but to look for new vulnerabilities that may be
presented in the architecture supplied, and to apply sort of the highest
possible critical thinking to the systems submitted for \--

DR. SMURGIAN: Okay, we have a motion on the floor and we have a second.
Comments, questions? Dr. Williams.

DR. WILLIAMS: We're introducing a third part here, this analysis team.
So far the standards have directed themselves toward the vendor that
produced the system and the ITA that inspects the system for conformance
with the standards.

Now if the standards are going to call out a third party, this analysis
team, then we need to define who that analysis team is, and then that
immediately raises such questions as well who pays for it, who owns its
output. This brings in a whole new level of complexity into the
standards and the qualification process.

DR. REVEST: The resolution doesn't specify that this is outside of the
body of (unintelligible). This is an analysis team. There's no way in
the implication here that this is outside of the lab if they have the
ability to perform such an analysis.

DR. SMURGIAN: Mr. Kraft and then Mr. Burger.

MR. KRAFT: Well to start with I don't agree that the TGDC has concluded
that voting systems must be subject in general to the same test used for
systems of critical importance to the U.S. federal government otherwise
we would have just adopted those tests and we wouldn't be spending all
this time here.

I think this walks across the importance of the voting system standards
as the key document in evaluating systems. Additionally I'm not sure
exactly who first coined the term penetration analysis but the
evaluation of voting systems security I think really is a higher calling
than just one media popular buzz word.

I think if you want to specifically charge NIST with looking at the
issues and practices of voting system security as those apply to voting
systems design, I will support that but I think this amendment tends to
make an assertion about the committee's conclusion that is untrue.

It tends to give legitimacy to a term that has been developed in the
popular press and it appears to attempt to yet bring additional
undefined third parties as Dr. Williams pointed out, into the process
and none of that I want to go into.

I am quite happy with NIST doing the analysis and giving us some sound
recommendations on voting system security.

DR. REVEST: If I could respond to that. I think that again I already
addressed the issue of the analysis team, you know, being perhaps in the
voting system testing laboratories as we talked about earlier so I think
that's not an issue brought in here.

The term "penetration analysis" is in quotes. It is a term that's there.
"White box testing" would be another phrase that could have been used
here.

The meat of this resolution is the second part of the resolution, which
says that we want to see a very open ended analysis. And that's the key
point here that we want to address, that it's not just a matter of
following the checklist, saying do the election officials have passwords
to use the system, is the plug carefully removed from voter reach, et
cetera, et cetera.

We have a list of known vulnerabilities you can go through but I think
you have to have somebody sit back and look at these systems and say,
you know, based on my expertise \-- the reason we've got people with
expertise within the testing labs that can do this kind of thing \--
based on their expertise, can they discover vulnerabilities that were
overlooked.

And I think that the complexity of voting systems is such that this is a
very important part of any kind of security analysis that needs to be
done. It needs to be people who are trained to think critically about
these systems and look for vulnerabilities that might have been
overlooked.

DR. SMURGIAN: Mr. Burger.

MR. BURGER: Well I strongly support the concept of rigorous system level
testing.

I'd like to raise an issue that this has implications for, and it's a
very difficult issue but I think we need to address it, and that is how
changes are handled and what level of testing is required.

The issue of course is there's a wide range of changes and if the
testing required to introduce changes is too onerous you can hold off
clearly needed changes that are absolutely positive for the system.

Typically this is done through some sort of a change review and \-- or
some changes more limited testing, and obviously for others more
rigorous testing, but see what your response is. But I think at some
point we need to address how we're going to address that issue.

DR. REVEST: If I could respond. Yes, I think that's a good point, Steve,
and I think we do need to work that in.

The phrase here is that it requires a significant amount of open-ended
research. What the word significant is, is for NIST to develop and when
the change comes in, what an appropriate significantly amount of work is
on that would be. So if you have a small change, a small amount of
looking it over critically would be advisable certainly.

DR. SMURGIAN: Ms. Quesenberry.

MS. QUESENBERRY: If I may, I think what I just heard in the discussion
was a general agreement on the need for the substance of this amendment
and perhaps some wording changes in the preamble would help let us move
to a vote fairly quickly if we are in fact in agreement.

I wonder whether the first sentence could be changed to; the TGDC has
considered the need for security evaluation of voting systems.
Accordingly certification tests must include \-- would that solve \--

DR. REVEST: I'd be happy to consider that a friendly amendment.

DR. SMURGIAN: Okay, could we capture that change please? Could you read
that again, Ms. Quesenberry?

MS. QUESENBERRY: The TGDC has considered the need for security
evaluation of voting systems.

DR. SMURGIAN: A little slower so that \--

MS. QUESENBERRY: I've got it written. I'll hand it to you.

The TGDC has considered the need for security evaluation of voting
systems. Accordingly certification tests must \-- so I'm striking the
first sentence and replacing it with the one I read, and adding a word
to the second sentence.

MALE SPEAKER: You want to delete the word certification. The word
certification \--

MALE SPEAKER: Qualification.

MS. QUESENBERRY: Okay. ITA testing \-- what's the right word?

MALE SPEAKER: Just say testing.

MS. QUESENBERRY: Just testing? Okay. Accordingly testing must include
\--

DR. REVEST: Yes, I'd be happy to consider that a friendly amendment. It
seems to cover Paul's objection.

DR. SMURGIAN: Okay, so modified. Mr. Kraft, did you have a \--

MR. KRAFT: Well yes, my suggestion was going to be as a friendly
amendment to strike the entire first paragraph and that leaves as Dr.
Revest pointed out, the meat of this amendment in the second paragraph
alive and well.

MS. QUESENBERRY: I'm sorry, I think you missed \-- the last sentence in
the first paragraph I think is an important concept that will tie into a
resolution I think we'll hear later about the ongoing maintenance of
these standards, which is to acknowledge that the hackers stay ahead of
us.

DR. REVEST: Yeah. The fact that in the second paragraph we talk about
open-ended research sort of implies the end of that first sentence,
somewhat a different \--

MALE SPEAKER: Well I basically do not want to lend legitimacy to
penetration analysis as being the be all end all in evaluating system
security. I want them to take a broader view of it.

DR. REVEST: Let me make that clear. That's a good point, Paul.

MALE SPEAKER: Security analysis (unintelligible).

DR. REVEST: Yeah, this is not intended to be the entire evaluation by
any means but it's an important part that the Security and Transparency
Committee worried would be omitted if we had a style that emphasized for
example, repeatability of these tests.

This is a kind of analysis that you don't necessarily get repeatability
on because you've got an analysis team and one analysis team may see a
vulnerability and another one may not. And so it's not as repeatable as
many tests are here but it is nonetheless very, very important to have
this style of analysis because it allows you to catch bugs in ways you
don't catch other ways.

DR. SMURGIAN: Are you suggesting a change?

DR. REVEST: No, I was just commenting on Paul's implication here that
this amendment somehow \--

DR. SMURGIAN: Would you find it more acceptable if that were called
security analysis, or ethical hacking, or something like that?

DR. REVEST: Security analysis.

DR. SMURGIAN: Would that be okay?

MALE SPEAKER: I mean ethical hacking is another very narrow \--

DR. REVEST: I have another tie here to term penetration analysis. It's
not key to this amendment. The point is the second paragraph, and if we
want to just delete the first paragraph altogether I'm happy with that
if that allows us to move on quickly here.

MALE SPEAKER: Well I'll offer that as a friendly amendment.

DR. SMURGIAN: What is the offer, to delete the whole new paragraph?

MALE SPEAKER: I'll offer a motion to delete the first paragraph in its
entirety.

DR. SMURGIAN: Is that acceptable to you, Dr. Revest?

DR. REVEST: That's fine. Like I said, the meat is in the second
paragraph here so \--

DR. SMURGIAN: Okay, then we delete your first paragraph and do we need a
change or a clarification by what you mean by the analysis team in the
second paragraph or is that acceptable?

DR. REVEST: It's left unspecified and I think that NIST can
appropriately specify how that team should work. I would trust them \--

MALE SPEAKER: I agree with Ron. We can leave that the way it is here
with the understanding that as we develop the standards this has got to
be clarified as to who that team is, how it's constituted, what sort of
agreement we've got to have among the various parties as to who the
members of that team are. So that's something that's got be addressed.

DR. SMURGIAN: Okay, any other questions or comments? All right, then
Resolution 17-05 has been modified by taking out the first paragraph in
its entirety and there are no changes made to the second paragraph. This
is acceptable to you, Dr. Revest?

DR. REVEST: Yes, it is.

DR. SMURGIAN: Any other comments? Hearing none, do I have a motion to
move to a vote?

MALE SPEAKER: So moved.

DR. SMURGIAN: Thank you. Mr. Green, would you please take a roll cal?

MR. GREEN: Resolution 17-05 as amended; Smurgian?

DR. SMURGIAN: Yes.

MR. GREEN: Davidson?

MS. DAVIDSON: **(No response heard)**.

MR. GREEN: Miller?

MS. MILLER: Yes.

MR. GREEN: Turnerbuoy?

MS. TURNERBUOY: Yes.

MR. GREEN: Purcell?

MS. PURCELL: Yes.

MR. GREEN: Harding?

MR. HARDING: Yes.

MR. GREEN: Ellekese?

MR. ELLEKESE: Yes.

MR. GREEN: Caldise?

MS. CALDISE: Abstain.

MR. GREEN: Burger?

MR. BURGER: Yes.

MR. GREEN: Williams?

DR. WILLIAMS: Yes.

MR. GREEN: Kraft?

MR. KRAFT: Yes.

MR. GREEN: Revest?

DR. REVEST: Yes.

MR. GREEN: Schutser?

DR. SCHUTSER: Yes.

MR. GREEN: Gannon?

MR. GANNON: **(No response heard)**.

MR. GREEN: That's a yes from Gannon?

DR. SMURGIAN: No.

MR. GREEN: I'm sorry.

DR. SMURGIAN: That didn't sound like Gannon.

MR. GREEN: Quesenberry?

MS. QUESENBERRY: Yes.

MR. GANNON: Gannon votes yes on that motion.

MR. GREEN: Gannon votes yes?

DR. SMURGIAN: Could we hear that again, please, Mr. Gannon.

MR. GANNON: Yes, Gannon votes yes.

DR. SMURGIAN: Thank you.

MR. GREEN: Is that a yes?

DR. SMURGIAN: Yes.

MR. GREEN: Recording that as a yes. That's 13 votes for, one voter
abstaining, one voter absent.

DR. SMURGIAN: Well Resolution 17-05 has passed as modified. Thank you.
Dr. Revest.

DR. REVEST: Thank you, Mr. Chairman. I'd like to move next to Resolution
18-05, which is on documentation.

Briefly this document requires that vendors supply significant
documentation of the security components of the system submitted for
evaluation.

This is a long resolution. I apologize for that. Would you like me to
read it all again or should I \--

DR. SMURGAIN: Let me ask the members if they feel that we need to read
it. I think all of you have had a chance to read it before. Do we need
to read it again? Does anybody feel it needs to be read?

DR. REVEST: I'd be happy to if somebody wanted but \-- I would like to
make one small change in the first paragraph. W=Where it says are
typically, I would like to say, may also be.

MALE SPEAKER: What line in the first paragraph?

DR. REVEST: The fourth line.

DR. SMURGIAN: The fourth line, so document \-- well documented.

MALE SPEAKER: May also be.

DR. REVEST: (Unintelligible).

MR. ELLEKESE: Mr. Chairman, Jim Ellekese, question.

DR. SMURGIAN: Yes, please, go ahead.

MR. ELLEKESE: This may be a small point but because I'm a member of an
election team here in New Jersey and totally blind, the documentation as
I'm reading this \-- at some point will the documentation be afforded an
alternative format accessible by those members of the voting teams on
the state level or on any level that may have either a physical or
sensory disability?

DR. REVEST: This resolution doesn't address that point and this
resolution specifies documentation that is to be submitted with the
other documentation submitted describing the voting system that is to be
evaluated.

I'm actually not sure what the standards are for that. I think that \--

DR. WILLIAMS: May I comment on that?

DR. SMURGIAN: Dr. Williams.

DR. WILLIAMS: This is more of a question then a statement. J.R. or some
of the advocates, would it suffice to say that all documentation has to
be presented in written and machine readable form and then you could
take that machine readable form and pass it through any of your
disability features?

DR. SMURGIAN: Mr. Harding.

MR. HARDING: Well my speaker's not working but I'll \-- thank you. That
would probably work because it would follow under my interpretation as
an employee working for the voting \-- you know, in the field for the
need for the accommodation and someone who has legal and legitimate
access to that information, and so the local (unintelligible) then or
the voting people could then put it in brail or an alternative format
and if they had it in electronic means that would easily be
accommodated.

DR. REVEST: If I can comment. There's no understanding or implication
here that this would be information that would be public necessarily.
That would be a subject of a different resolution and others thinking
about the standards that might apply.

This is intended to be material just to make sure that the documentation
that's submitted with the voting machine includes substantial coverage
of the security components of this voting system for evaluation
purposes.

DR. SMURGIAN: Mr. Burger.

MS. QUESENBERRY: But that's doesn't preclude the need for accessibility.

DR. REVEST: It does not, right.

DR. SMURGIAN: Mr. Burger.

MR. BURGER: I'll defer to our colleagues from the Access Board on this
but in my work I believe it's accurate to say that if the documentation
is available, I wouldn't say machine readable but in accessible HTML,
and what accessible means there is things like illustrations are
adequately described in text so someone who can't see them can get an
accurate description of the information they're conveying.

**(CROSS TALK)**

MR. ELLEKESE: Mr. Chairman, Jim Ellekese. Mr. Burger's comment is
perfectly acceptable and understandable, if somehow that could be
incorporated.

DR. SMURGIAN: Could you identify yourself please?

MR. ELLEKESE: Sure, it's Jim Ellekese from the Access Board.

DR. SMURGIAN: Could you repeat that again?

MR. ELLEKESE: Jim Ellekese from the Access Board.

DR. SMURGIAN: Could you repeat your comment?

MR. ELLEKESE: Sure. Mr. Burger's comment hit the nail on the head as far
as the information I'm seeking. If it's in an accessible HTML with
adequate narrative for any illustrations that would almost eliminate any
roadblocks to individuals that may have to access the documentation.

DR. SMURGIAN: Thank you.

MS. QUESENBERRY: If I might, since accessible format is described
elsewhere perhaps we could just say accessible format.

DR. REVEST: Yeah, I think that sounds like a friendly amendment, maybe
in the beginning of the second paragraph.

DR. SMURGIAN: Mr. Kraft.

MR. KRAFT: Yes. I think we have stepped a little bit out of scope.
Basically the provision of materials in accessible format by employees
of jurisdictions by ITAs who are using that material is already covered
under the ADA and the requirements to provide accessible features to
employees.

Additionally all of us in all phases of government already have very
strong regulations that dictate us making accessible materials available
to the public so I don't think that we need a resolution here that is
going to make all the information coming into the labs in an accessible
format because that's very costly and it may not be needed. But it
certainly should be needed if there are employees or members of the
public involved in the process.

So I think rather then making that a blanket requirement that the
materials be provided in accessible and non-accessible formats, we could
certainly easily go with putting in electronic and printed, but I think
we should leave the alternate formats to existing laws in those other
areas.

DR. SMURGIAN: Dr. (Unintelligible).

MALE SPEAKER: Yeah. I hope we don't get too distracted about the format
issues here because the main point of this is what the content of this
documentation should cover and maybe we could just defer that issue and
say that it is documentation in formats to be specified by NIST or in
the standards et cetera, and try to focus on the main point of this.

DR. SMURGIAN: Mr. Burger.

MR. BURGER: Paul, with all due respect I've got to disagree a bit. I
find that there's quite a bit of misunderstanding and confusion on what
are alternative formats and therefore many people go to a great deal of
extra effort, more then is really necessary.

So I think there's a real benefit in our clarifying that but I'm happy
to put that off to perhaps the next meeting. Maybe we can ask the Access
Board to give us those suggestions.

DR. SMURGIAN: Ms. Quesenberry.

MS. QUESENBERRY: I would agree with Mr. Burger and suggest that again
rather then scatter this through every time we say produce documents,
that we wrap this up and perhaps we can come up with a statement about
acceptable formats that could cover everything.

DR. SMURGIAN: Is that acceptable to you, Dr. Revest?

DR. REVEST: That's a wise recommendation to factor those considerations
out, definitely.

DR. SMURGIAN: It sounds like \-- I mean this applies to a lot of other
resolutions as well and perhaps a sort of well thought out blanket
resolution may serve better.

Yes, Mr. (Unintelligible).

MALE SPEAKER: I'd like to come back from that and offer an amendment to
the resolution that's presented and I hope it will be taken as friendly.

It has been said that this is a long resolution and I think it could be
very appropriately shorten. We already have a suggested change and
(unintelligible) may also striking (unintelligible).

Come to the last paragraph on page 29, strike the first sentence because
again we're setting requirements for documentation of voting system.
We're not going to rely on other federal standards per se otherwise we
wouldn't be here.

And then keep the next sentence after that. Put a period at the end of
documentation and strike the remainder of the resolution. That will
leave our opening statement. It will then direct NIST to research and
draft standard documents requiring voting system documentation. The rest
of that body of that document, NIST is certainly competent to make sure
that it's included and determine what is and isn't appropriate.

DR. SMURGIAN: Dr. Revest, is that \--

DR. REVEST: I'm concerned that this strikes a bit at the meat of this
resolution in the sense that there's important examples here of the
kinds of things that are intended to be included and I'm not sure that
the paragraphs that are left on page 29 carry the weight of that.

You know, for example, the risk assessment information. I mean certainly
it is the case that NIST could draft any of these but it leaves it sort
of hanging in the wind as to exactly what kind of thing is intended and
I would prefer the items that are listed continue to be listed.

DR. SMURGIAN: Dr. Williams.

DR. WILLIAMS: Can I ask a question of Ron? The last sentence on page 29
says, to include such items as. Do you mean this to be a definitive list
or are you putting this up as suggestions for NIST to review?

DR. REVEST: NIST would review these, would presumably take our
recommendations that these are the kinds of things we want to see. All
of the things that are seen here would presumably make it on the final
list unless they had good reason otherwise, and other things may be
added. This is a starting list.

DR. WILLIAMS: All right. So it would be within your intent that if in
developing these documentation standards NIST reached the conclusion
that certain of these things you've got listed are not appropriate and
maybe some things that you don't listed are.

DR. REVEST: That's correct.

DR. WILLIAMS: Okay. Then in that spirit I don't have a problem with
this.

MS. QUESENBERRY: So it would be including but not limited to?

DR. REVEST: Yes.

DR. WILLIAMS: Okay. Well I will change my motion to merely striking the
first sentence in the last paragraph on page 29 then.

DR. SMURGIAN: Dr. Revest, is that \--

DR. REVEST: I have no big problem with that. I think the meat is
elsewhere but I think it helps set the tone for the kind of
documentation required. If we can proceed quickly without the change,
sure.

DR. SMURGIAN: So we are taking out the first sentence of the last
paragraph on page 29, and then the paragraph that's on page 31, is that
correct?

DR. REVEST: No, I just limited it to \--

DR. SMURGIAN: So you're leaving \-- okay, so the only change is taking
out the first sentence of the last paragraph on page 29. Would you
consider that a friendly amendment, Dr. Revest?

DR. REVEST: Yes.

DR. SMURGIAN: Okay, any other questions, comments?

MALE SPEAKER: I believe he changes the word typical to another word.

DR. SMURGIAN: Yes, we already made that so we have two changes. One is
on the fourth line of the first paragraph. Now it reads; voting systems
that are not well documented may also be less secure in that, et cetera,
et cetera.

And then taking out the first sentence of the last paragraph that now
that paragraph simply reads that TGDC directs NIST to research, et
cetera, et cetera. And everything else is intact.

DR. SMURGIAN: Yes, Ms. Purcell.

MS. PURCELL: Mr. Chairman, I wonder \-- it was mentioned before to
include but not be limited to such items as, in the last \-- that
paragraph.

DR. SMURGIAN: Dr. Revest.

DR. REVEST: That's the intended meaning. If it's not clear without those
words actually there then we should put them in.

DR. SMURGIAN: So we are adding after \-- to include but not limited, is
that the motion?

MS. PURCELL: That's correct.

DR. SMURGIAN: Would you consider that a friendly amendment?

DR. REVEST: Yes.

DR. SMURGIAN: Thank you, Dr. Revest. I think we've covered the
territory. Do I have a motion to move to a vote?

FEMALE SPEAKER: Yes.

DR. SMURGIAN: Second?

MALE SPEAKER: Second.

DR. SMURGIAN: Mr. Green, would you please take a roll call?

MR. GREEN: Resolution 18-05 as amended; Smurgian?

DR. SMURGIAN: Yes.

MR. GREEN: Davidson?

MS. DAVIDSON: (No response heard).

MR. GREEN: Miller?

MS. MILLER: Yes.

MR. GREEN: Turnerbuoy?

MS. TURNERBUOY: Yes.

MR. GREEN: Purcell?

MS. PURCELL: Yes.

MR. GREEN: Harding?

MR. HARDING: Yes.

MR. GREEN: Ellekese?

MR. ELLEKESE: Yes.

MR. GREEN: Caldise?

MS. CALDISE: Abstain.

MR. GREEN: Burger?

MR. BURGER: Yes.

MR. GREEN: Williams?

DR. WILLIAMS: Yes.

MR. GREEN: Kraft?

MR. KRAFT: Yes.

MR. GREEN: Revest?

DR. REVEST: Yes.

MR. GREEN: Schutser?

DR. SCHUTSER: Yes.

MR. GREEN: Gannon?

MR. GANNON: Yes.

MR. GREEN: Quesenberry?

MS. QUESENBERRY: Yes.

MR. GREEN: Thirteen votes for, one voter abstaining, one voter absent.

DR. SMURGIAN: With that, Resolution 18-05 has been adopted as amended.
Thank you. Dr. Revest, would you move on to the next one, please?

DR. REVEST: Thank you, Chairman Smurgian.

I'd like us next to look at Resolution 22-05, entitled Federal
Standards.

This is a resolution in the briefest form intended to encourage us to
follow federal standards where appropriate.

I'd like to read it. It's one page. I'm going to make a couple of small
changes to the second sentence only to correct some factual errors.

Here's the proposed resolution. Voting systems while specialized in
their purpose often have many aspects in common with general information
technology (IT Systems).

The second sentence reads as follows with modification. Guidelines,
standards, and testing programs have been developed for U.S. government
civilian IT systems, including the cryptographic, that's a change of the
word crypto to cryptographic, cryptographic module validation program,
(CMVP) for analysis and testing of cryptographic, and inserting here
before the word software, modules, and modules and software, and the
National Voluntary Laboratory, and I'm changing the word assessment to
accreditation just to get that correct, an accreditation program NVLAP
for \-- I'm changing the word certification to accreditation of testing
laboratories.

So let me read that second sentence again because there were a number of
changes, to make sure they're correct.

Guidelines, standards, and testing programs have been developed for U.S.
government civilian IT systems including the cryptographic module
validation program, (CMVP), for analysis and testing of cryptographic
modules and software and the National Voluntary Laboratory Accreditation
Program, (NVLAP) for accreditation of testing laboratories.

This is currently creating an information security standard mandated by
the Federal Information Security Management Act, specifically
recommended security controls for federal information systems, draft
NIST special publication 800-53, affecting all federal government
systems.

This draft standard specifies the inclusion and proper use of security
related protections affecting many areas of IT system design and
development, management testing and operations, all of which have
relevance to voting systems.

This and other similar efforts by the U.S. government have the benefits
of having been developed in the public process and having had successful
track records with industry.

The TGDC therefore requests that NIST examine existing federal guideline
standards and testing programs for security and general information
technology systems for the relevance and applicability and standards to
voting systems, and to draft standards documents that follow such prior
guidelines, standards, and programs when possible and where appropriate.

DR. SMURGIAN: Thank you. We have a motion on the floor. Do we have a
second?

MALE SPEAKER: Second.

DR. SMURGIAN: We have a second. Questions, comments?

MALE SPEAKER: Yes. If this not redundant of what we have already passed?
We've asked NIST to look at voting system security and make
recommendations and now we're asking them in a little different
language. I mean is there another aspect to this that I fail to
understand or is this simply redundant from what we've already passed?

MALE SPEAKER: I think the earlier one was talking about the testing,
security analysis risk assessment. This is a general document which I
believe was handed out that talks about the whole management and
everything, all recommendations about IT system security. It's more
comprehensive then just that.

DR. SMURGIAN: Mr. Burger.

MR. BURGER: I'd like to offer an amendment. It pretty well established
federal policy to the maximum extent possible to rely on recognized
consensus voluntary standards, particularly ANCI accredited standards.

If my memory is correct that was originally done in 1996 Technology
Transfer Act and is reflected in OMB Circular 119.

So my amendment would be, and I'll invite a representative from ANCI
with a comment if she cares to, but in the third line where it says U.S.
government, to insert typically relying on ANCI accredited consensus
voluntary standards.

And then in the second paragraph, first line after federal guidelines,
to amend standards to read, ANCI accredited voluntary consensus
standards.

There is in fact quite a body and active effort going on to develop a
wide range of security standards for Homeland Security and other
purposes within the IEEE and elsewhere and I would like to make sure
that this includes those efforts.

MALE SPEAKER: So I'm not sure \-- the amendment is federal guidelines
and ANCI approved \--

MR. BURGER: ANCI accredited consensus voluntary standards. Would that be
the right wording?

DR. SMURGIAN: Where are we talking about specifically? Where would that
be inserted?

MR. BURGER: Two places, on line three after systems (unintelligible),
and then on the first line of the second paragraph.

DR. SMURGIAN: Could we read the specific insertion?

MR. BURGER: The insertion would be in the first instance, typically
utilizing ANCI accredited consensus voluntary standards. And in the
second paragraph simply ANCI accredited consensus voluntary standards.

MALE SPEAKER: To replace the word standard? Read the whole sentence for
us.

MR. BURGER: Okay. In the first case, it's the second sentence in the
first paragraph. Guidelines, standards, and testing programs have been
developed for the U.S. government civilian IT systems, new wording,
typically utilizing ANCI accredited consensus voluntary standards. And
then going on, including the cryto module validation program and so
forth.

Then in the second paragraph first sentence it would read, the TGDC
therefore requests that NIST examine the existing federal guidelines,
ANCI accredited consensus voluntary standards, and testing programs for
security, and the rest remains unchanged.

MALE SPEAKER: (Off microphone.) How about the last (unintelligible).
Okay.

MR. BURGER: I just want to make sure \-- because it actually is \-- it
leverages a lot of work and brings it into the federal sphere and it
saves a lot of effort.

**(END OF SIDE A, START OF SIDE B)**

DR. SMURGIAN: Yes.

FEMALE SPEAKER: Just the appropriate word would be approved as opposed
to accredited. We accredited standard developers such as IEEE and
approve the actual standards, and the reason the link is important is
OMBA-19 does encourage the use of voluntary consensus standards as an
alternative to private \-- you know, to government rule making. And an
American national standard is the standard that is developed in an
environment that is opened, balanced, characterized by due process and
consensus.

DR. SMURGIAN: Okay, so what we have is an amendment that would add in
the second sentence of the first paragraph after civilian IT systems \--
it would say, typically utilizing ANCI approved consensus voluntary
standards.

And then move on as shown before \--

MS. QUESENBERRY: A question for Ms. Caldise. Is it voluntary consensus
standards or consensus voluntary standards, or does it matter?

MS. CALDISE: Well it's voluntary consensus standards because they're by
definition voluntary, they're not law unless referenced \--

MS. QUESENBERRY: Just the wording looked awkward when I looked at and I
wondered if we had it \--

MS. CALDISE: (Unintelligible).

DR. SMURGIAN: I'm sorry, it should be voluntary consensus standards.

MS. CALDISE: Voluntary consensus standards, yeah.

FEMALE SPEAKER: If we're going to have a phrase people recognize it
might as well be the one they'll recognize.

DR. SMURGIAN: Yeah. Okay, and then in the first sentence of the last
paragraph after existing federal guidelines, inserting ANCI approved
voluntary consensus standards.

So we don't repeat the word standards twice, right? We're taking that
out if we're going to add the \-- okay. So Dr. Revest, is that a \--

DR. REVEST: I want to make sure I understand the spirit of this and that
we're getting the intended effect here. I would be happy to consider
this a friendly amendment if it's enlarging the scope of what NIST might
be looking at.

I'm concerned though that we want to make sure we retain the focus on
the effort that NIST currently has underway for developing these IT
standards.

It's an excellent effort. The publication (unintelligible) 853 shows
considerable work by NIST. It's excellent work and I certainly want to
make sure that we as a committee here can leverage that work in what's
happening here because NIST has the expertise and the standard is
evolving as we speak, so I want to make sure that effort can be
incorporated into what's happening and the actual status of that as a
standard is in early stage still I think.

DR. SMURGIAN: Mr. Burger.

MR. BURGER: Yeah, I think my purpose is to encompass the full scope of
that effort and I'll defer to any of the NIST folks that care to
comment, but a lot of that effort is being done very consciously out in
the ANCI accredited sphere.

DR. SMURGIAN: Do we have any issue? Mr. Skall.

MR. SKALL: Thank you. I think I would agree with Ron that clearly the
emphasis should be on the work we've done because we've spent a lot of
time.

However I think the way the amendment has been put in makes the task
more comprehensive. It makes our job a little bit more difficult but I
think it's good policy.

So I think the intent of the motion to concentrate on the proposed
federal standard but making sure that we do in fact look at voluntary
standard of ANCI and perhaps ISO as well certainly makes sense.

DR. SMURGIAN: I guess the amendment is being interpreted in the spirit
suggested by \--

DR. REVEST: Yes, which means I'll be happy to consider that a friendly
amendment.

DR. SMURGIAN: Okay. Any other comments? Any comments from the members on
the phone?

MR. GANNON: Yes, this is Gannon.

DR. SMURGIAN: Yes.

MR. GANNON: I just want to be sure that in the editing there that \-- I
think Dr. Revest suggested it broadens, it doesn't restrict NIST only
examining federal guidelines or ANCI standards, that there might be
other security related standards that should be looked at.

MALE SPEAKER: As written I guess it talks just about federal \-- with
the amendment, with federal and ANCI voluntary consensus standards and
other standards.

DR. SMURGIAN: How about if we simply add typically utilizing ANCI
approved and other voluntary consensus standards? Would that cover the
territory?

MALE SPEAKER: Yes.

DR. SMURGIAN: Simply inserting the word "other"?

MALE SPEAKER: ANCI accredited and other approved voluntary consensus.

DR. SMURGIAN: ANCI approved, not accredited.

MALE SPEAKER: Sorry, yes.

DR. SMURGIAN: ANCI approved and other voluntary consensus standards.
Would that do it?

DR. REVEST: That would cover it, yes.

DR. SMURGIAN: So we use that modification in both cases?

DR. REVEST: Yes, I think so.

DR. SMURGIAN: Any further comments or questions? If not, do I hear a
move to \--

DR. REVEST: I just want to make sure we've got federal guidelines, ANCI
approved voluntary consensus standards. So the word federal I think
previously modified guidelines, standards, and testing programs and now
we've got federal, maybe with syntax, depending how you parse it.
Modifying just guidelines and testing programs is open for example so I
think maybe federal \--

DR. SMURGIAN: Should that be then inserted in after programs so that it
read TGDC therefore requests that NIST examine existing federal
guidelines, standards, and testing programs and ANCI approved and other
voluntary consensus standards for security in general information
technology systems, et cetera. Does that read better, because you're
right, it decoupled the federal from the other?

DR. REVEST: The original intent was to make sure that federal
guidelines, standards, and testing were covered.

DR. SMURGIAN: So we could keep that intact, federal guidelines,
standards, and testing programs \--

DR. REVEST: And move the \--

DR. SMURGIAN: \-- And ANCI approved and other voluntary consensus
standards. Is that okay with you?

DR. REVEST: Excellent. Yes, it's fine.

DR. SMURGIAN: Okay. Well let me summarize the changes again to
Resolution 22-05.

We have on the third line of the first paragraph after U.S. government
civilian IT systems, we have added, typically utilizing ANCI approved
and other voluntary consensus standards.

And then continue to read, including the cryptographic, that's a
modification, module validation programs, CMVP, for analysis and testing
of cryptographic, and then we've added modules. And then it reads on,
software and national voluntary laboratory accreditation, this is simply
a correction, National Voluntary Laboratory Accreditation Program,
NVLAP, for accreditation. Correcting that, instead of certification,
accreditation of testing laboratories.

And then on the first sentence of the last paragraph we have inserted on
the second line after, and testing programs, we've inserted a comma,
ANCI approved and other voluntary consensus standards, and then continue
as it reads, for security and general information technology systems et
cetera.

DR. REVEST: Yes.

DR. SMURGIAN: I guess we don't have a comma after and ANCI approved and
other voluntary consensus standards.

Okay, with those \-- I guess the screen will catch up with us, but with
those changes, do we have a motion to vote?

MALE SPEAKER: I motion to vote.

DR. SMURGIAN: Okay. Do we have a motion and a second?

MALE SPEAKER: Second.

DR. SMURGIAN: So Mr. Green, will you please take a roll call for
Resolution 22-05?

MR. GREEN: 22-05 as amended; Smurgian?

DR. SMURGIAN: Yes.

MR. GREEN: Davidson?

MS. DAVIDSON: **(No response heard).**

MR. GREEN: Miller?

MS. MILLER: Yes.

MR. GREEN: Turnerbuoy?

MS. TURNERBUOY: Yes.

MR. GREEN: Purcell?

MS. PURCELL: Yes.

MR. GREEN: Harding?

MR. HARDING: Yes.

MR. GREEN: Ellekese?

MR. ELLEKESE: Yes.

MR. GREEN: Caldise?

MS. CALDISE: Abstain.

MR. GREEN: Burger?

MR. BURGER: Yes.

MR. GREEN: Williams?

DR. WILLIAMS: Yes.

MR. GREEN: Kraft?

MR. KRAFT: Yes.

MR. GREEN: Revest?

DR. REVEST: Yes.

MR. GREEN: Schutser?

DR. SHCUTSER: Yes.

MR. GREEN: Gannon?

MR. GANNON: Yes.

MR. GREEN: Quesenberry?

MS. QUESENBERRY: Yes.

MR. GREEN: That's 13 voting for, one voter absent, one voter abstaining.

DR. SMURGIAN: Thank you. Resolution 22-05 has been adopted as amended.
Dr. Revest, would you like to move on to the next resolution?

DR. REVEST: Thank you, Chairman Smurgian. The next resolution I'd like
to have us consider is Resolution 21-05, entitled Multiple
Representation of Ballots.

And the high level intent here is to have some work by NIST on the
situation commonly occurring now in electronic voting systems where a
single ballot is represented multiple times.

I'll read it to you. Proposed resolution is; The Help America Vote Act
of 2002, Section 15481, subdivision A-2-B, requires that any voting
system used in election for federal office must produce a paper record
\-- this paragraph should go. I'm sorry, I want to delete that.

Let me start with paragraph two. I know where I'm going with this.

DR. SMURGIAN: Okay, so we are deleting that first paragraph.

DR. REVEST: Yeah. I know what this committee will do with that
paragraph.

DR. SMURGIAN: You can calm down.

DR. REVEST: Let's start with the second paragraph, which will be the
first paragraph.

Voting systems may create one or more electronic representation of
ballots in addition to the paper record required for recount. For
example, three redundant electronic copies may be made for reliability
purposes.

As another example, the scanning of an un-scanned ballot may create
another electronic representation of the ballot.

A number of issues are related to the use of multiple representations,
both electronic and paper, that are in some cases relatively new and not
completely identified or understood and in another case need uniform
terminology and procedures.

These issues include detecting disagreements between the
representations, handling disagreements between the representations,
converting between representations, insuring that ballots are not
multiply converted and counted, use of multiple representations and
fraud analysis, authenticity of the representations, marking of ballot
representations with unique identifiers if and when possible to do so
while preserving voter privacy, and conversion to and from standard
formats.

The TGDC has concluded that further research is advisable in identifying
potential problems associated with voting systems that use multiple
representations of ballots and identifying best approaches for handling
such problems.

The TGDC thus requests that NIST perform such research and draft
standard documents that reflects NIST determination with the Best
Practices and best approaches for handling these problems.

DR. SMURGIAN: Thank you. We have a motion and a second and we're open
for discussion. Mr. Kraft.

MR. KRAFT: Okay, basically in the first sentence of the second paragraph
I don't know that we have agreement that there is in fact a paper record
required for recount. Voting systems may create one or more electronic
representations of ballots in addition to the paper record required for
recount. I don't think anyone has established that there is a paper
record required for recount.

MS. QUESENBERRY: If I may.

DR. SMURGIAN: Ms. Quesenberry.

MS. QUESENBERRY: I'm reading from HAVA; the voting system shall produce
a permanent paper record with manual audit capability for such system.

MR. KRAFT: Right. And that's not required to be used for recount.

DR. REVEST: It's required to be available for recount, yes.

DR. SMURGIAN: It's for the purpose of audit as opposed to recount?

MR. KRAFT: And HAVA recognizes that \--

MS. QUESENBERRY: No, I'm sorry.

MR. KRAFT: \-- The conduct of recounts and the records to be used for
recounts are the purview of state legislatures.

MS. QUESENBERRY: I'm sorry, I'd like to continue. In paragraph three,
iii, under B, Manual Audit Capacity; the paper record produced under
subparagraph A shall be available as an official record for any recount
conducted with respect to any election in which the system is used.

MR. KRAFT: And that is the summary totals that come out of machines that
are printed out and witnessed by the precinct workers.

MS. QUESENBERRY: This just says that voting systems may create one or
more representation in addition to the paper record required for
recount.

MR. KRAFT: Well I don't agree that that's a redundant copy of a ballot.

MS. QUESENBERRY: No, no, in addition to the paper record. They're
clearly saying that the paper record can be different from that
electronic representation.

DR. REVEST: This is a motivational paragraph, trying to get us to \--

MR. KRAFT: Okay. How about the paper record required by this site, and
then that will leave interpretation at that site open to state
legislatures, the courts, and other policy makers.

DR. SMURGIAN: Are you suggesting we refer to HAVA specifically and not
make this statement?

MR. KRAFT: Refer a direct site to that requirement rather than trying to
interpret the requirement.

MS. QUESENBERRY: So required under HAVA citation?

MR. KRAFT: Yes, yes.

DR. SMURGIAN: Is that acceptable to you, Dr. Revest?

DR. REVEST: Yes. The paper record required for recount under HAVA \--

DR. SMURGIAN: In the meantime, Dr. Williams.

DR. WILLIAMS: May I finish?

DR. SMURGIAN: Just a minute. Are we talking about this particular \--
let me take that one first.

DR. WILLIAMS: I don't think it's necessary for us to become lawyers and
interpret HAVA. I think the intent here is that whatever type records,
multiple records are produced in the voting system has to have these
characteristics.

So I think that we can end that sentence \-- we can take that first
sentence and say voting systems may create one or more electronic
representations of ballots in addition to any paper record produced.

DR. REVEST: That would be fine. This is a motivational sentence that
we've got at this point.

DR. WILLIAMS: You know, whether it's compliant with HAVA or not, we'll
leave to the purview of the legal staff. But if it's produced \-- the
intent here is that if the system is producing multiple records, whether
they be electronic or paper, then those records should have these
characteristics. Is that \--

DR. REVEST: Yes. We have systems now with computerization that are
capable of producing multiple records for a variety of purposes, some of
them may be reliability, some of them may be for the vote tally, others
may be for the recount and so on too, and we need to I think, think
about this issue at the highest level and make sure we have a clear
philosophy for moving forward with dealing with these kinds of issues
here.

DR. SMURGIAN: Okay, so it sounds like we have a friendly amendment
that's accepted by the author to modify that first sentence, of now the
first paragraph, which reads; voting systems may create one or more
electronic representations of ballots in addition to any paper record
produced period.

DR. REVEST: Yes.

DR. SMURGIAN: Okay, any other comments. Mr. Kraft.

MR. KRAFT: My only other comment is, we're almost making it acceptable
and expected for there to be disagreements between the representations
and very frankly any detectable occurrence of a disagreement between the
representations of ballots is a serious flaw and it's something that in
testing you really should not see with an occurrence of more then say
once in a million events, or perhaps ten million events.

DR. REVEST: This better be very rare, yes.

MR. KRAFT: I'm not sure \-- by basically talking about these issues
being how you're going to handle disagreements between representations
and that \-- we're almost making it sound as though these disagreements
are to be expected and acceptable.

DR. REVEST: That's not the intent.

MR. KRAFT: Okay.

DR. REVEST: If there's a word you can toss in there somehow that says
\--

MALE SPEAKER: How about something like we know the design should
preclude the occurrence in the event.

MS. QUESENBERRY: Yes, I would say that the unlikely occurrence is
actually the one for which we need to provide guidance because as a
voting official said to me once, it's good standards that take care of
the boundary, good procedural things that care of the boundary
conditions that make it possible for us to get out of problems when they
happen.

Things that happen normally and are \-- you know, we don't have to tell
people to go to lunch, they go to lunch but we do have to tell them
other things. I mean you have to handle the exceptions and handled the
problems and that's really what you need \--

DR. REVEST: Yes, that's exactly the point here and I think that making
sure that \--

DR. SMURGIAN: Okay, what is the specific amendment?

MR. KRAFT: Well I would recommend maybe before \-- these issues include
\-- just say something like, although the design should preclude the
occurrence of disagreements, these issues should include when they occur
\--

DR. REVEST: If I may, disagreements is only the first two of these
issues and \--

MR. KRAFT: Okay.

DR. REVEST: So maybe just adding \-- if I may suggest wording as we're
proceeding here collaboratively, disagreements between the
representations should they occur, is that enough to \-- or do you want
to have something that's more indicative of it being a rare event, in
the rare event that they should occur?

MALE SPEAKER: It needs to be a rare event. It also needs to circle back
to standards and testing. I mean in the last election cycle we had
events where we discovered conflicts between the multiple memories and
we traced those down, followed them up, and in ever case it was a matter
of actual hardware failure.

But in every case from my perspective as the authority over those
systems, I have to trace those down and make sure that it's not a coding
issue or a logic issue, that it is something that we understand and
something that's not going to recur.

MS. QUESENBERRY: It sounds like good practice and that what this
amendment suggests is that we should be identifying that practice and
providing guidance for how to repeat it in the other 49 states.

DR. SMURGIAN: I still haven't heard a specific wording.

DR. REVEST: Could I try to capture that then?

DR. SMURGIAN: Yes, please.

DR. REVEST: Maybe adding at the end of the first two items, detecting
disagreements between the representations in the rare event that they
should occur, and at the end of the second one, the same phrase. Does
that meet the \--

MALE SPEAKER: Well I think you say, preventing, and detecting, and
handling disagreements between representations in the rare event they
should occur and have them all lumped together under one number.

DR. SMURGIAN: Detecting and handling \--

DR. REVEST: Preventing, detecting, and handling disagreements between
the representations in the rare event that they should occur.

DR. SMURGIAN: So the amendment then is to modify item number one to
read; preventing, comma, detecting and handling disagreements between
the representations, comma, in the rare event that it should occur
period.

And then delete item number two and renumber the subsequent items, is
that the amendment? Dr. Revest, would you consider that a friendly
amendment?

DR. REVEST: Yes.

DR. SMURGIAN: Okay, we've made the changes accordingly. Any other
comments? Any comments from members on the phone?

MALE SPEAKER: None.

DR. SMURGIAN: Not having any further comments, do I hear a move to vote?

FEMALE SPEAKER: So moved.

MALE SPEAKER: Second.

DR. SMURGIAN: Thank you. Mr. Green, would you please take a roll call on
Resolution 21-05 as modified.

MR. GREEN: Smurgian?

DR. SMURGIAN: Yes.

MR. GREEN: Davidson?

MS. DAVIDSON: **(No response heard).**

MR. GREEN: Miller?

MS. MILLER: Yes.

MR. GREEN: Turnerbuoy?

MS. TURNERBUOY: Yes.

MR. GREEN: Purcell?

MS. PURCELL: Yes.

MR. GREEN: Harding?

MR. HARDING: Yes.

MR. GREEN: Ellekese?

MR. ELLEKESE: Yes.

MR. GREEN: Caldise?

MS. CALDISE: Abstain.

MR. GREEN: Burger?

MR. BURGER: Yes.

MR. GREEN: Williams?

DR. WILLIAMS: Yes.

MR. GREEN: Kraft?

MR. KRAFT: Yes.

MR. GREEN: Revest?

DR. REVEST: Yes.

MR. GREEN: Schutser?

DR. SCHUTSER: Yes.

MR. GREEN: Gannon?

MR. GANNON: Yes.

MR. GREEN: Quesenberry?

MS. QUESENBERRY: Yes.

MR. GREEN: That's 13 voting for, one voter absent, one voter abstaining.

DR. SMURGIAN: Thank you. Resolution 21-05 as amended has been adopted.
Dr. Revest, next motion.

DR. REVEST: Thank you, Chairman Smurgian. I'd next like to turn our
attention to Resolution 23-05, Common Ballot Format Specifications.

This ballot is intended to encourage NIST in the drafting of standards
to make use of existing common formats when available.

Let me read the resolution to you. The TGDC has concluded that the
adoption of standard formats for election related information such as
ballots, both blank and filled in, has many positive benefits and is
worth pursuing.

I'm going to have a couple of changes in this next sentence. I'll read
the corrected version. An example of such a standard is Oasis election
markup language (EML version 4.0), which is an XML based specification.
Just a couple of corrections there.

The TGDC therefore requests NIST to do research and develop standards
documents; one, specifying what existing election information format
standards or portions thereof or variations thereof are acceptable for
use in voting systems, and two, requiring that voting systems use such
standards wherever possible.

This evaluation of election information standards shall consider
security, ease and cost of implementation and other factors judged
relevant by NIST. If no existing election information format standards
or portions thereof or variations thereof are judged by NIST upon its
detailed examination to be acceptable for current use then NIST should
so recommend and this resolution will have no net effect at this time.

FEMALE SPEAKER: Second.

DR. SMURGIAN: Okay, we have a motion and a second on the floor. Mr.
Kraft.

MR. KRAFT: Dr. Revest, is it the intent to \-- in using the word format
are you addressing ballot layout issues, which I would hold are solely
in the purview of the state, or are you speaking only of basically the
various formats of computer codes such as the \--

DR. REVEST: Good question. The intent is the latter.

MR. KRAFT: The ladder, okay.

DR. SMURGIAN: Any other questions, comments?

MR. KRAFT: I guess I would suggest a friendly amendment then that one of
the outcomes or one of the criteria perhaps for NIST research and
evaluation would be to make sure that the formats represented would
still provide users with the ability to lay out and present ballots as
required by the various state laws.

MS. QUESENBERRY: So fit for function, fit for use?

MR. KRAFT: Fit for use, fit for function, yeah.

DR. SMURGIAN: So what is the specific amendment you would like to make?

MR. KRAFT: I haven't thought of that yet, sir.

DR. REVEST: Perhaps in the last paragraph in the list of things for NIST
to consider is fit for function along with security ease and cost of
implementation.

MR. KRAFT: Okay, that would work.

MALE SPEAKER: (Off microphone.) (Unintelligible) state law.

DR. SMURGIAN: Shall consider \-- after shall consider, fit for function?

MS. QUESENBERRY: Fitness for function.

MALE SPEAKER: And compliance with state election (unintelligible).

MS. QUESENBERRY: Do we really want to include compliance with 50 plus
election codes?

MALE SPEAKER: Pretty much because if they're fairly similar as to ballot
layout, the existing systems, Paul, I think need all of the state
election code requirements.

MS. QUESENBERRY: Isn't that something for the states to determine?

MALE SPEAKER: (Off microphone). (Unintelligible) and basically we want
people to be able to create legal ballots.

MS. QUESENBERRY: Could we ask them to consider state election codes
without requiring a full analysis of all 50 plus codes? I mean I'm just
concerned about the workload we're creating.

DR. REVEST: Yes. It says shall consider and then a variety of things, so
consider \--

DR. SMURGIAN: But this is evaluation it says. If we put it here it says
evaluation.

DR. REVEST: This evaluation shall consider.

MS. QUESENBERRY: So are you suggesting that we add another one that says
existing election codes?

DR. REVEST: I think that's important actually yes. I mean if we have \--

MALE SPEAKER: How about after cost of implementation and existing \--

DR. REVEST: We wouldn't adopt a standard that was not usable in some of
our major states.

MS. QUESENBERRY: Yeah, right.

DR. REVEST: So we're adding after, consider fitness for function and
existing election codes \--

MALE SPEAKER: Fitness for function under existing election codes.

DR. REVEST: Oh, fitness under existing election, then comma, security
ease of cost and implementation and other factors. So I would consider
that a friendly amendment.

MALE SPEAKER: Fitness for function under existing election codes.

DR. SMURGIAN: Okay, so the amendment is to insert on the last paragraph
after NIST evaluation of existing election information standards, shall
consider fitness for function under existing election codes, comma, and
then continue as it is, security, ease and cost of implementation and
other factors judged relevant by NIST.

DR. REVEST: Yes.

DR. SMURGIAN: So we are then interpreting this saying NIST staff will
keep those in mind. They will not do a full analysis of all the existing
election codes. Is that the appropriate interpretation?

DR. WILLIAMS: I think that's the appropriate interpretation and I think
the existing systems, you know, produce ballots now that address the
existing election codes so it's not something that requires a
significant amount of research.

DR. SMURGIAN: Well we want to make sure that we're not committing to
making a complete analysis of every code, that we'll simply take those
into account in the consideration process.

DR. REVEST: Yes.

DR. SMURGIAN: Do we have any other comments? Yes, Mr. Burger.

MR. BURGER: Well I completely agree. I'll just add the observation
though. I think one of the most helpful things the staff could do is be
consulting with the state officials and incorporate that experience into
their work. Just a comment.

DR. SMURGIAN: Thank you. Hearing no other discussion, do I have a motion
for a vote?

MALE SPEAKER: I so move.

FEMALE SPEAKER: Second.

DR. SMURGIAN: Thank you. We have Resolution 23-05, which has been
modified to read \-- the changes are on the third line of the first
paragraph. It reads; an example of such a standard is, and then it's
added Oasis, and acronym EML has been removed. And then it reads Oasis
election markup language and then in paren (EML version 4.0).

DR. REVEST: I think the close paren should go after EML, sorry.

DR. SMURGIAN: You want to leave the version \--

MS. QUESENBERRY: Move the version with the \--

MALE SPEAKER: Yeah, say Oasis EML version 4.0 parenthesis, election
markup language. Is that what you wanted?

DR. REVEST: Just like it is there I think is fine. EML is just an
abbreviation for Election Markup Language, and then version 4.0.

DR. SMURGIAN: So that's the only change on that line?

DR. REVEST: So the right parenthesis after EML and then \--

DR. SMURGIAN: Take the other one out, right.

DR. REVEST: Yeah, that's it. That's the only change there.

DR. SMURGIAN: Okay. And then \--

MALE SPEAKER: Also changing standards (unintelligible) at the end of
that sentence.

DR. SMURGIAN: And now the first sentence of the last paragraph reads,
NIST evaluation of existing election information standards \--

DR. REVEST: No, this is \--

DR. SMURGIAN: Yeah, go to the next one. NIST evaluation of existing
election information standards shall consider fitness for function under
existing election codes. That's the only language added. You don't need
"and" there. There is no "and". Delete the comma.

And then it goes on, as it was presented before, security, comma, ease
and cost of implementation, comma and other factors et cetera.

So those are the two changes made. With that Mr. Green, will you please
\--

MALE SPEAKER: Did we move for a vote?

DR. SMURGIAN: We already did that.

MR. GREEN: 23-05 as amended; Smurgian?

DR. SMURGIAN: Yes.

MR. GREEN: Davidson?

MS. DAVIDSON: **(No response heard).**

MR. GREEN: Miller?

MS. MILLER: Yes.

MR. GREEN: Turnerbuoy?

MS. TURNERBUOY: Yes.

MR. GREEN: Purcell?

MS. PURCELL: Yes.

MR. GREEN: Harding?

MR. HARDING: Yes.

MR. GREEN: Ellekese?

MR. ELLEKESE: Yes.

MR. GREEN: Caldise?

MS. CALDISE: Abstain.

MR. GREEN: Burger?

MR. BURGER: Yes.

MR. GREEN: Williams?

DR. WILLIAMS: Yes.

MR. GREEN: Kraft?

MR. KRAFT: Yes.

MR. GREEN: Revest?

DR. REVEST: Yes.

MR. GREEN: Schutser?

DR. SCHUTSER: Yes.

MR. GREEN: Gannon?

MR. GANNON: **(No response heard)**.

MR. GREEN: Quesenberry.

MS. QUESENBERRY: Yes.

MR. GREEN: That's 12 voting for, two voters absent, one voter
abstaining.

MS. TURNERBUOY: This is Sharon Turnerbuoy, I don't know if you heard my
response. It was yes.

MR. GREEN: We did, thank you.

DR. SMURGIAN: We did not hear from Mr. Gannon. Are you still on the
phone? I guess not.

Thank you. With that vote Resolution 23-05 has been adopted. Dr. Revest,
would you like to proceed?

DR. REVEST: Yes, thank you Chairman.

The next amendment I'd like us to consider would be Resolution 35-05,
which is the Wireless Resolution.

DR. SMURGIAN: This is the one that was presented to you with a new
number this morning. It's in the new package.

DR. REVEST: It's on the front of the \--

DR. SMURGIAN: I assume the members on the phone have received these
electronically.

FEMALE SPEAKER: Yes, we have.

MALE SPEAKER: Yes, we have.

DR. REVEST: The wording has not been changed since the original
publication under the incorrect number.

DR. SMURGIAN: Would you mind reading it again, please?

DR. REVEST: Yes. The wording of the proposed resolution is the
following; the TGDC has considered the advisability of using wireless
technology within voting systems from a security perspective. It has
concluded that for now the use of wireless technology introduces
excessive and unnecessary risk and should be avoided. The TGDC directs
NIST to do research. There's a word do missing there. Maybe it's just
fine like it is.

DR. SMURGIAN: I think it's okay.

MALE SPEAKER: That's a verb.

DR. REVEST: Yeah. To research and draft standards documents requiring
that all wireless transmitters and receivers including infrared
transmitters and receivers be excluded from voting systems.

DR. SMURGIAN: Okay, we have a motion and we have a second.

DR. REVEST: Could I make some comments before \--

DR. SMURGIAN: Yes.

DR. REVEST: So the point here, the sense of the committee was that
wireless does introduce a lot of new risks that need to be mitigated in
some way or other.

This resolution represents sort of an extreme

position on the use of wireless and I'm sure there will be amendments
offered to this here as this committee \-- because I'm beginning to know
this committee better and better.

But the point here is that wireless is a significant risk that is worth
identifying separately and trying to make sure it is well managed, and
the use of wireless, you know, casually without care to the security
implication should certainly be prohibited, and in an extreme position
like this, may not be supportable unless the vendors and others can come
up with a strong reasons why wireless should be included in the systems
because of the risk they involve.

DR. SMURGIAN: Mr. Burger, then Mr. Kraft, and then Dr. Williams.

MR. BURGER: Ron, you're good at predicating. I'd like to offer the first
amendment.

To support people with hearing aids \-- many people know but in the
interest of clarity, many hearings aids, roughly 20 percent have a mode
where instead of receiving sound they receive a localized magnetic field
and in fact it's called a tele-coil or T-Coil mode. The FCC requires
that for all telephones, that as well as sound, this low level magnetic
field down in the acoustic region be produced to support hearing aids.

I'd like to propose a note to this, specifically saying that this does
not preclude support of T-Coil signal for hearing aids and in fact that
that is encouraged.

DR. REVEST: I have a question about that. What's the range of those if
somebody had an antenna. I mean we're not broadcasting everybody's
voting in a way that somebody sitting outside the polling place with a
van and nice antenna could pick up are we?

MR. BURGER: It originally came from just magnetic flux leakage out of
the old Western Electric phones. It's a low level mileant per meter
field and it dies off very quickly. However, you know, as anyone who's
involved in wireless knows, wireless goes on and just degrades over time
so it's hard to say there's a brick wall anywhere to it. It's very low
frequency though.

DR. REVEST: Are there adapters that one could use if you had a \-- I'm
presuming that if you have it closer to your ear you can have a weaker
\-- are you talking about something that someone would hold up to their
ear like a phone as you say, or broadcasting in a stronger signal from
the voting machine itself.

MR. BURGER: Actually the current requirements in FCC Part 68 are about
10DB under many common environmental noise sources for this same kind of
emission and so those typical levels are almost too low for many
environments. So they're kept at quite low levels.

DR. REVEST: It seems like a well motivated proposal and it's certainly
within the scope of I think NIST staff to evaluate what security risks
we might run by including that.

MR. BURGER: Right. And I have provided wording to staff earlier this
morning so they have specific wording. It would go as a note to the
bottom of the resolution.

DR. SMURGIAN: Could you please read that? Yes, Ms. Quesenberrry.

MS. QUESENBERRY: If I could just add a note to this. I don't have the
material with me, but we did go through this in the IEEE process. And I
don't have the materials to cite or look at but we do have that history
of this essentially same discussion having gone on there.

DR. REVEST: My understanding was that that discussion was somewhat
contentious at the time and I don't know if IEEE 15-3 actually came to a
resolution about this point.

MS. QUESENBERRY: I'm sorry, on the T-Coil issue.

DR. REVEST: Yes, on that particular issue. Or maybe it wasn't, I \--

MS. QUESENBERRY: I thought we'd resolved that T-Coils were acceptable
but I don't have my notes and so I can't be sure.

DR. REVEST: That's useful background.

MALE SPEAKER: If I may, if we didn't do this I think we probably are in
violation of Section 508 federal law, which requires that federal IT
equipment be supporting of people with disabilities and specifically in
the 508 standards, T-Coil is called for.

MS. QUESENBERRY: It is included in the VSS-2002.

MALE SPEAKER: Right, it's Section C 2.27.2 DRE standards provides and
conforms with FCC Part 68 Wireless (Unintelligible) for assisted devices
used by persons with hard of hearing when a system utilizes a telephone
handset to provide audio (unintelligible) information.

MALE SPEAKER: If I can just throw out one more comment, Ron. From a
security standpoint there's no way you could get in and influence the
program with the machine through this connection. You're right that
there may be a privacy concern. I think that's pretty minimal but we
could leave it to NIST staff to evaluate.

MALE SPEAKER: May I have the floor, sir?

DR. SMURGIAN: I feel we're discussing it sort of in a vacuum because I
think we should read the language. Is this all?

DR. REVEST: Yes.

DR. SMURGIAN: Okay, the wording proposed is, this resolution excludes
T-Coil coupling to hearing aids. T-Coil coupling for hearing aid users
should be supported when the voter is provided an audio output. Is that
acceptable to you, Dr. Revest?

DR. REVEST: I would like to see the amendment amended to included
wording of the effect that the TGDC requests NIST to review the security
and privacy considerations that may arise here.

MS. QUESENBERRY: Security, privacy and legal considerations.

DR. REVEST: Yeah.

DR. SMURGIAN: So we add at the end of this \-- so then it should not
really be a note. Perhaps it should say \-- take out the note and leave
the rest alone, and at the end add, NIST should consider \--

DR. REVEST: This resolution excludes \-- The TGDC requests NIST to
review security, privacy and legal issues in such T-Coil coupling and
advises \-- presumably everything's hunky-dory and these are fine but I
would just like NIST to give us their judgment on that, and the privacy
issue is the main one I care about.

DR. SMURGIAN: Okay. Mr. Kraft.

MR. KRAFT: The resolution as drafted and as amended would outlaw use of
the ES&S IVO-Tronic voting system. It would make illegal the voting
system being used by 40 percent of the voters in Florida, and it would
jeopardize the nation's ability to conduct the '06 elections.
Accordingly, I have a proposed amendment, which I will move that we
adopt and read if it pleases the Chair.

DR. SMURGIAN: Okay, just for clarification let me make sure that we
understand that this amendment as presented is acceptable to you as a
friendly amendment, Dr. Revest?

DR. REVEST: Yes. The full wording isn't up there but \--

DR. SMURGIAN: Does TGDC request NIST to review security, privacy and
legal issues?

DR. REVEST: Issues related to the use of T-Coil coupling, yes.

DR. SMURGIAN: Okay.

MALE SPEAKER: If I may, I think at the end of the first sentence there
it should be hearing aid users, and then the second sentence should
start, T-Coil coupling should be supported.

DR. SMURGIAN: Okay, and take out the parenthesis. So is that okay with
you, Dr. Revest?

DR. REVEST: Yes.

DR. SMURGIAN: Okay, so that's considered a friendly amendment and
modified.

DR. REVEST: Yes.

DR. SMURGIAN: Now to the next amendment, Mr. Kraft.

MR. KRAFT: Okay, I'm not sure if all the members of the committee heard
my opening comment because we did digress a bit.

This amendment as drafted and as amended would outlaw the ES&S
IVO-Tronic system, which is used by about 40 percent of the voters in
Florida and widely across the nation. It would as drafted make it nearly
impossible for the country to conduct an election in '06 that would be
in compliance with the standards.

Additionally the prohibition against developing particular communication
technologies and systems is really not appropriate.

I would like to suggest the following amendment to the resolution. It
will read as follows, and since it's rather short I'll just read it in
its entirety.

The TGDC has considered the advisability of using wireless technology
within voting systems from a security perspective. It has concluded that
for now the use of wireless technology introduces severe risk and should
only be approached with extreme caution.

The TGDC directs NIST to research and draft standards document for the
use of wireless communications devices in voting systems.

I think that goes to the intent. It charges NIST with research. It
allows existing systems to continue to be used and it is very
specifically aimed at allowing the vendors and system developers to
continue research in how to appropriately and securely use wireless
technology.

DR. SMURGIAN: Would you repeat the last part?

MS. QUESENBERRY: I just have a certain point of the information that I'd
like clarified. Yesterday when we began this discussion there was some
talk about the difference between the use of wireless during setup or
administrative functions and the use of wireless during the voting
function.

And on top of any security concerns covering all of this, there's
certainly some privacy concerns around wireless during the voting
function and I wonder whether there would be some value in being clear
about the potential for there being different requirements in the two
cases.

DR. SMURGIAN: Yes, Dr. Williams.

DR. WILLIAMS: You're paraphrasing something I said. Paul mentioned
Florida. We have some representatives here from Virginia. The
Commonwealth of Virginia \-- I believe 19 counties use the ABS \-- 13
counties use the ABS system, which is a wireless based system.

And the way they use it is, that they use the wireless feature to
program their voting stations in their warehouse where number one, you
can insulate the warehouse and number two, even if someone does alter
the system within the warehouse you're doing that in front of your L&A
testing so anything that happened that corrupted the system would be
detected during L&A testing.

So they do not use the wireless feature and correct me if I'm wrong on
this but they do not use the wireless feature in the polling place where
you're sitting there with it on for a 12 hour period with the
vulnerability that results from that.

And, you know, in doing state certifications of that system, my
recommendation to states that want to use that particular system is just
that, that they use the wireless feature in their warehouse but that
they do not use it in the polling place.

MALE SPEAKER: And as point of clarification too, at least when I was
last in the business, you can have Wyatt connections and you can have
\-- if it's not appropriately secured and the boxes haven't got tempest
protection, it's just as much of a problem.

DR. REVEST: I wanted to comment on what Brit said. I think that if we're
going to add requirements that all warehouses store these things, be
tempest proof, I think that's a consideration cost and expense that
would be very surprising to \--

**(CROSS TALK)**

MS. QUESENBERRY: I think we're tying to solve the question here. I think
we're trying to \--

DR. REVEST: Well we need to get software into those machines in a way
that \-- I want to also \-- as a point that Brit said I strongly
disagree and I think it's important that the committee understand the
disagreement.

He said that if the software happened to have been changed in the
warehouse by whatever means including wireless that the following logic
and accuracy test would detect that.

I would to disagree with that in the strongest terms. I don't believe
that logic and accuracy tests \-- first I don't believe they're at all
adequate for determining whether software modification has happened,
that you can make software modifications that change the behavior of the
machines in ways that will occur later in the election and not be
detected during logic and accuracy tests. And so I think that trying to
rely on an assumption that logic and accuracy tests will pick up those
kinds of changes is improper.

On the other hand we have a resolution passed that talks about setup
validation tests. So if you load up the software and then execute a
proper setup validation test as we passed in this (unintelligible), then
with that correction I think that would be \-- I agree with what Brit
said on that point.

DR. SMURGIAN: Mr. Kraft.

DR. WILLIAMS: More and more you're seeing your validation test part of
the logic and accuracy testing.

DR. SMURGIAN: Mr. Kraft and then Ms. Quesenberry.

MR. KRAFT: Well the intent of my amendment is not to endorse relying on
any particular process including validation tests, including logic and
accuracy tests.

The intent of my amendment is to state that yes, there is severe risk
with wireless but at the same time \-- and, you know, in my personal
world we don't use wireless. The wireless drivers are not on this
machine even though it's got an antenna built into it.

However, in my travels I'm finding there are more and more hotels where
I cannot get high speed access in the room. The only thing offered by
the hotel is wireless and we're not testing cellular modems to see which
will give us the best performance to work around that.

So wireless is coming. It's part of the environment now. It's part of
the systems that are out there now. I think in the systems that are out
there now it has been used very responsibly. I think we need to
encourage future responsible use of wireless but we need to understand
that it is coming.

It will probably not be very far in the distant future before \-- in
many areas wireless becomes your only possible ability to communicate
and you're going to have to develop systems, you're going to have to
develop standards and testing modules that will work in that
environment.

MALE SPEAKER: For voting systems?

MR. KRAFT: Yes.

DR. SMURGIAN: Ms. Quesenberry.

MS. QUESENBERRY: (Unintelligible).

DR. SMURGIAN: Okay, let me just summarize because we actually have an
amendment on the floor. Did we have a second?

MALE SPEAKER: Yes.

DR. SMURGIAN: Okay. But basically the amendment is to modify this
resolution to read; the TGDC has considered the advisability of using
wireless technology within voting systems from a security perspective.
It has concluded that for now the use of wireless technology introduces
severe risk and should only be approached with extreme caution. We
probably don't need the only do we? Should be approached.

MS. QUESENBERRY: How about used?

DR. REVEST: Yeah, used or approached.

DR. SMURGIAN: Should be used with extreme caution.

MALE SPEAKER: Well I guess I wanted to get in there that the developers
who are bringing these systems out should be approaching this with
extreme caution in addition to the users. That's why I drafted the
language that I did, to introduce secure risk and should only be
approached with extreme caution.

DR. SMURGIAN: So we'll leave the original language then. Okay so it
starts \-- the third line goes on, technology introduces severe risks or
severe risk and should be approached with extreme caution. The TGDC
directs NIST to research and draft standards documents for the use of
wireless communication devices in voting systems. Is that correct?

MALE SPEAKER: Yes.

DR. SMURGIAN: So that is the motion on the table and we have a second.

MALE SPEAKER: Second.

DR. SMURGIAN: So do we have any other discussion before we vote on this
amendment? Dr. Revest.

DR. REVEST: Yes. So the rewriting of the first portion seems fine. You
know, approach with extreme caution certainly \--

DR. SMURGIAN: Well my assumption is that this amendment says we don't
have this second part. Is that true?

MALE SPEAKER: Yeah, the T-Coil issue becomes moot.

DR. SMURGIAN: Because you no longer are \--

DR. REVEST: I had a couple of questions. I think one is with respect to
the '06 election, you know, what the intent is. We've got a lot of
systems that have not been reviewed that are using wireless in a rather
open manner.

I'm concerned that, you know, we have to think about the impact on the
'06 election, and not only with respect to running the elections, which
of course we want to be able to do and we've got existing equipment out
there \--

MALE SPEAKER: We've got a lot of systems that have been reviewed that
have been very carefully scrutinized that were used in the '02 and '04
elections, which at the language as amended has not passed and the
original drafted resolution is passed. Those systems will no longer be
legal under the new standards that were set up.

DR. REVEST: I understand what you're saying. So we will be adopting
standards which will affect what can be purchased for the '06 elections
and part of that will be the review of wireless usage.

And so what you're suggesting if I understand this amendment correctly
is that we allow wireless but only under standards that are adopted by
us. I don't know what those are.

MALE SPEAKER: We are saying that right now there is a high level of risk
with wireless and there really are no specific standards for wireless.
We are urging everyone to approach wireless with extreme caution and we
are urging NIST to bring us standards for the use of wireless that will
insure the system they are on is secure. And I think that's a reasonable
approach.

DR. REVEST: I'm concerned that getting the standards in place and the
evaluation procedures in place to actually make wireless systems
valuable in time for '06 elections is a big task.

You've got not only the wireless portions themselves but also the fact
that they depend upon software that needs to take care of a lot of
additional issues.

MALE SPEAKER: We've got systems we are going to depend on in '06 that
are currently using wireless technologies that would be outlawed by the
original resolution.

DR. REVEST: I understand that but \--

MALE SPEAKER: I don't think I can say it any clearer.

DR. SMURGIAN: Mr. Burger.

MR. BURGER: I actually think we should not use that concern on this
resolution, although the concern \-- but I think later we have a
resolution on a de-qualification of systems and I think we've got a
general issue of when these standards come out how are they introduced
and what is the grandfathering period for equipment that's already in
the field. I think that's an issue in its own right.

DR. SMURGIAN: Dr. Williams.

DR. WILLIAMS: Let me put a dose of reality in this thing. When states
adopt these standards, they're going to grandfather in their existing
systems. If they've got a system that they've been running out there for
two, three, or four years, and they're comfortable with that system and
they've been running good elections, they're not going to go to those
counties and say okay county, you're going to throw out your voting
system and go by another voting system. They're going to simply
grandfather them in.

MS. QUESENBERRY: But does that mean that we mean future systems to \--

MALE SPEAKER: Sure.

MS. QUESENBERRY: No, I don't think so.

MALE SPEAKER: (Off microphone).

MS. QUESENBERRY: Yeah, I mean what you said is a reality on the ground,
which is that there are systems out there that are currently in use but
I just don't know that that should limit our ability to decide what
further systems should do although it might inform it.

DR. SMURGIAN: I'm not quite sure that it's constructive to say you
cannot use something rather then saying \-- you know, just like we
talked before, if you're going to use a particular system these are the
criteria or the requirements that such a system has to meet.

MALE SPEAKER: We have a lot of very fancy technology developing these
days and wireless is one of those. And from a security viewpoint you
want to keep systems simple, simple, simple and every time you add
another feature like wireless because it gives you a little bit of
convenience say at the loading of software or whatever, you know,
there's a security consideration that comes into play and you start down
the slippery slope.

So, you know, the language here certainly expresses the caution that's
appropriate if you're going to make use of those. But extreme caution I
guess is the wording, which is appropriate.

DR. SMURGIAN: Ms. Purcell.

MS. PURCELL: What we might add then if we're talking, because the states
will grandfather in the existing systems, is maybe in that last sentence
just put devices in future voting systems.

**(END OF TAPE 1, START OF TAPE 2)**

DR. SMURGIAN: Let me ask Dr. Revest, is that \--

DR. REVEST: Does that help future voting systems?

DR. SMURGIAN: Yes.

DR. REVEST: No, I think the principals and standards that NIST is
developing should be something you can evaluate in existing systems.

**(RECORDING INTERUPTED)**

MALE SPEAKER: (Off microphone). (Unintelligible) and standards.

DR. SMURGIAN: Mr. Kraft.

MR. KRAFT: Yes, Dr. Revest, I have to address one of your statements
that simpler is better. And I agree fully that simpler is better but we
don't have that luxury.

Congress has dealt us a hand, a more complex hand. In 2000 the Florida
legislature decided that we needed to get rid of punch card voting
systems and they took up action to do that. That has ended now with a
requirement that we have audio/video voting systems that both display a
video ballot and have an audio channel in three different languages. It
has not gotten simpler. It is much more complex.

Additionally the Access Board is working to come up with more ways to
make these systems more accessible. We have not even begun to see the
demands of what the public and our legislatures are going to put on
these systems. To cut off access to merging technologies and development
of systems is irresponsible and I can't support that.

DR. REVEST: If I could respond. I think that the distinction here I want
to make is between complexity and the requirements. And you're very
accurately describing that the requirements have gotten more complex
versus the complexity of the design decisions you make within those
requirements.

Once you've got the problem specified and it may be a complicated
problem, you have a choice to make within that, communication technology
perhaps being one of them.

Congress has not mandated the use of wireless to my knowledge and we may
have alternatives in many or most cases that we can take advantage of.

So given physically the problem, you may have choices to make. Simpler
is better in that case.

DR. SMURGIAN: Let me ask to see if members on the phone have any
comments or questions.

MR. ELLEKESE: Not at this time. This is Jim Ellekese.

MALE SPEAKER: I move we vote the amendment to the resolution.

DR. SMURGIAN: Dr. Williams, last comment?

DR. WILLIAMS: Yeah, I was simply going to call the question on the
amendment. I think that this is another one of those issues where
everybody on the panel has made up their mind and we can debate it for
several days without changing anybody's mind so I move to call the
question.

DR. SMURGIAN: Okay, so the amended version of the resolution reads \--
let me read it again.

The TGDC has considered the advisability of using wireless technology
within voting systems from a security perspective. It has concluded that
for now the use of wireless technology introduces severe risk and should
be approached with extreme caution.

The TGDC directs NIST to research and draft standards documents for the
use of wireless communication devices in the voting systems.

Okay, so we'll move for a vote. Mr. Green, would you please \--

MR. GREEN: 35-05 is amended using my wireless microphone. Smurgian?

DR. SMURGIAN: Yes.

MR. GREEN: Davidson?

MS. DAVIDSON: **(No response heard)**.

MR. GREEN: Miller?

MS. MILLER: Yes.

MR. GREEN: Turnerbuoy?

MS. TURNERBUOY: Yes.

MR. GREEN: Purcell?

MS. PURCELL: Yes.

MR. GREEN: Harding?

MR. HARDING: Yes.

MR. GREEN: Ellekese?

MR. ELLEKESE: Yes.

MR. GREEN: Caldise?

MS. CALDISE: Abstain.

MR. GREEN: Burger?

MR. BURGER: Yes.

MR. GREEN: Williams?

DR. WILLIAMS: Yes.

MR. GREEN: Kraft?

MR. KRAFT: Yes.

MR. GREEN: Revest?

DR. REVEST: Yes.

MR. GREEN: Schutser?

MR. SCHUTSER: Yes.

MR. GREEN: Gannon.

MR. GANNON: **(No response heard)**.

MR. GREEN: Quesenberry?

MS. QUESENBERRY: Yes.

MR. GREEN: Gannon?

MR. GANNON: Gannon votes yes.

DR. SMURGIAN: Would you repeat on the phone, please?

MR. GREEN: Gannon, are you there?

MR. GANNON: **(No response heard).**

MR. GREEN: That's 12 voting for, one voter abstaining, two voters
absent.

DR. SMURGIAN: So Resolution 35-05 as modified has passed.

With that we would like to take a break. And perhaps Dr. Revest can
review what's left of the resolutions but my plan is to go to the next
subcommittee's resolutions and then perhaps come back at the end of the
day if we have time. Is that acceptable?

DR. REVEST: Yes, that's fine. I ordered the presentation of the
resolutions considering this possibility. I'm happy to proceed as you
suggested.

DR. SMURGIAN: Okay, let's take a half hour break. Let's start at 11:45
a.m. Let's not come back at 11:45 a.m. Thank you.

**(SHORT BREAK)**

DR. SMURGIAN: Could I ask everyone to take your seats please.

We're going by the NIST atomic clock so nobody can question the accuracy
of the clock, which is good to five parts (unintelligible) 17 so I think
that's accurate now for this purpose.

DR. SCHUTSER: In the interest of time I will make my remarks brief and
get on to the resolutions. So I just want to tell you that in the Core
Requirements and Testing Group, our emphasis has been in providing a
good update to the current specifications and with some special emphasis
on insuring we have open transparent testing.

So accordingly our 11 resolutions deal with the following kinds of
things; examining the specs around the framework to insure nothing
important is missing, insuring the specs are sufficiently unambiguous
and detailed enough for testing, emphasizing the requirements is on the
core and common aspects that would record common and core
(unintelligible) election jurisdictions, recognizing that the testing
should be done openly and transparently, namely all the inputs, outputs,
test procedures, methodology should be available, recognizing that the
tests may not just involve pass or fail but could also involve
performance oriented measurements for which there could be a minimum
acceptable range set, and recognizing that the testing of the equipment
is only relevant in the intended operating environment, including the
appropriate setup configuration maintenance procedures as recommended by
the vendor.

So this list is already currently already engaged in the detailed
analysis of the current specs and is close to some deliverables. In
addition at the end I've added a resolution that addresses the need for
an assessment of several important technical issues that impact the
voting process but which fall outside the immediate needs and the scope
of the current specification and to be done afterwards.

DR. SMURGIAN: Just for clarification we are now in this session trying
to cover Resolution 24-05 through Resolution 34-05.

Three of these resolutions were not included in some of the packages
that were distributed yesterday but you should have them in front of
you. They were distributed this morning so you should all have copies. I
think they were actually distributed electronically earlier but for some
reason they were left out of the \--

MALE SPEAKER: So we're (unintelligible) 32, 33, and 34.

DR. SMURGIAN: Yes. And then we have 35, which was part of the security
group and that we voted on, on the wireless. So we have a total of 35
resolutions on the record. Sorry for the interruption, Dr. Schutser.

DR. SCHUTSER: Oh, no problem.

So the first resolution \-- we'll get to that and I will read it and see
\--

DR. SMURGIAN: Which one are we talking about?

DR. SCHUTSER: 24-05, Conformance Clause. It's very simple. The
conformance clause of a standard provides the answers to the important
question, what may conform and how. The conformance clause defines at a
high level what is required of implementers of the specification. The
clause may specify minimum requirements for certain functions as well as
extensibility, optional features and alternative approaches and how they
are to be handled.

The TGDC requests that NIST draft a conformance clause section for the
voting system standard.

DR. SMURGIAN: Okay, we have a motion on the floor. Do I hear a second?

MALE SPEAKER: I'll second.

DR. SMURGIAN: We have a second. Discussion, comments?

MALE SPEAKER: It seems pretty clear cut.

DR. SMURGIAN: Do we need that number one or should we simply \--

DR. SCHUTSER: You don't need the number one. I guess that's the only
thing I would say, take out the colon and take out the number one and
just add it to \--

DR. SMURGIAN: Make that a continuation of the statement.

DR. SCHUTSER: Yes.

MALE SPEAKER: Call the question.

DR. SMURGIAN: We have called the question so we will take a roll call on
Resolution 24-05 as edited for format.

MR. GREEN: 24-05 as edited, Smurgian?

DR. SMURGINA: Yes.

MR. GREEN: Smurgian votes yes. Davidson?

MS. DAVIDSON: (No response heard).

MR. GREEN: Davidson absent. Miller?

MS. MILLER: Yes.

MR. GREEN: Miller votes yes. Turnerbuoy?

MS. TURNERBUOY: Yes.

MR. GREEN: Turnerbuoy votes yes. Purcell?

MS. PURCELL: Yes.

MR. GREEN: Purcell votes yes. Harding?

MR. HARDING: Yes.

MR. GREEN: Harding votes yes. Ellekese?

MR. ELLEKESE: Yes.

MR. GREEN: Ellekese votes yes. Caldise?

MS. CALDISE: Abstain.

MR. GREEN: Caldise abstains. Burger?

MR. BURGER: Yes.

MR. GREEN: Burger votes yes. Williams?

DR WILLIAMS: Yes.

MR. GREEN: Williams votes yes. Kraft?

MR. KRAFT: Yes.

MR. GREEN: Kraft votes yes. Revest?

DR. REVEST: Yes.

MR. GREEN: Revest votes yes. Schutser?

DR. SCHUTSER: Yes.

MR. GREEN: Schutser votes yes. Gannon?

MR. GANNON: (No response heard).

MR. GREEN: Gannon not responding. Quesenberry?

MS. QUESENBERRY: Yes.

MR. GREEN: Quesenberry votes yes. That's 12 votes for, two voters not
responding, one voter abstaining.

DR. SMURGIAN: Resolution 24-05 is adopted as presented. Thank you. Okay,
Dr. Schutser.

DR. SCHUTSER: Thank you, Mr. Chairman. The next resolution I'd like to
discuss is 25-05, Precise and Testable Requirements, but I do have some
recommended changes to it. So I'll read it to you and indicate where the
changes are.

For qualification of voting systems to be consistent, fair and
meaningful it's necessary to control variability and conformance
assessment system. Both the requirements to be tested and the methods by
which they are to be tested must be specified with appropriate
precision.

The TGDC requests that NIST one, conduct a review and analysis of the
requirements in the 2002 VSS to insure that they are sufficiently
precise to enable meaningful testing. Two, include the requirement from
the 2002 VSS that are already precise testable. Three, write testable
requirements for those requirements that are not sufficiently precise.

And delete the end \-- the four \-- delete the end, there's going to be
a fifth. Four, expand the testing standards in the VSS to specify test
methods for those requirements. Delete a subset of. We thought we
wouldn't have sufficient capability to address them all but we will. And
I'm adding the fifth now. In addition, during this review NIST should
update the requirements where appropriate such as the reliability and
accuracy specifications.

DR. SMURGIAN: Could you read that part a little more slowly, please?
Update \--

DR. SCHUTSER: Sure. In addition \--

DR. SMURGIAN: Do we need in addition? I assume it will say and.

DR. SCHUTSER: Right, great. During the review NIST should update the
requirements where appropriate such as the reliability and accuracy
specifications.

FEMALE SPEAKER: Second.

DR. SMURGINA: We have second. Now we're open for discussion. Ms.
Quesenbery.

MS. QUESENBERRY: Just a point of clarification, in the Human Factors and
Privacy Subcommittee we've essentially proposed to re-write moving
forward some fairly large sections of VSS 2002. How does that intersect
with this?

DR. SCHUTSER: I think as you mentioned previously we're going to have to
coordinate these things so that clearly if you're taking up something
and rewriting it then that's what we would review.

MS. QUESENBERRY: Yeah, let's make sure that you're not reviewing old
requirements but reviewing the new ones.

DR. SMURGIAN: Yes, Mr. Burger.

MR. BURGER: I'd like to offer an amendment adding one item. I think it
would be helpful to the system on many of the tests to move them away
from some of the mill standards to some of the more commonly used
commercial tests for the same purpose, so I'd like to add a point that
to the extent NIST determines it is advisable commonly used commercial
tests be utilized.

DR. SCHUTSER: I would accept that, just make sure we get the wording
proper.

MR. BURGER: A lot of the reliability stuff calls on mill standard stuff
and there's comparable tests that are more commonly used in test
laboratories for commercial equipment.

DR. SCHUTSER: Okay, Mr. Chair.

DR. SMURGIAN: I'm trying to make sure that we have the language up there
before we go on.

DR. SCHUTSER: Okay.

MALE SPEAKER: No, it's five. You're adding that to five, is that what
you're saying?

**(CROSS TALK)**

DR. SMURGIAN: No, but I mean where do you want to add yours?

MALE SPEAKER: It would be number six.

MALE SPEAKER: We want to add another, number six.

MR. BURGER: I think so because I think the point here is that we want
the staff to specifically look at the reliability numbers to make sure
they don't inordinate system consequences.

DR. SCHUTSER: Right, so don't delete the five we added but just add a
new six that addresses that.

MR. BURGER: Right, because that's a different issue.

DR. SMURGIAN: Okay, number six.

MALE SPEAKER: Well we don't want any of these new measures to by their
nature impose new functional requirements, accuracy requirements or
physical design changes prior to '06.

MALE SPEAKER: Right, that's a separate issue.

DR. SCHUTSER: Right. No, the intent that was brought to my attention was
not the functional change but in some cases some of the reliability
requirements for example, if you examine them in the current
specifications aren't really stringent enough for what you could get.

MALE SPEAKER: Right, right.

DR. SMURGIAN: Okay, to the extent \--

MALE SPEAKER: Just the term, it is advisable.

MALE SPEAKER: Commonly used commercial test methods should be adopted.

**(CROSS TALK)**

MALE SPEAKER: What we're trying to capture there is to ride on many
NVLAP accredited labs in various purposes and use that infrastructure to
the maximum degree as possible.

DR. SMURGIAN: Dr. Schutser, would you consider these as friendly
amendments?

DR. SCHUTSER: I accept that. Yes.

MALE SPEAKER: I think it's number seven; in conducting this NIST will
take care to avoid changing any test measure, or criteria, or method
which would by its nature impose performance or functional standards
which are significantly different to prevent currently qualified voting
systems from being used in '06.

MS. QUESENBERRY: Could you just say that out? I mean not to the language
but what you mean.

MALE SPEAKER: What I mean is basically we do not \-- in going from a
mill standard to a commercially available standard, we do not want to
inadvertently impose a requirement that the best of the breed in voting
systems out there would not be able to meet in '06.

MS. QUESENBERRY: I guess I have two points. One is that I don't think
the testing should ever change the requirements. The test should be
there to support and validate the requirements, and also I just don't
know that \-- while I agree that the consideration of current voting
systems is important, I'm not sure that it's our job to insure that they
remain qualified if in fact we determine that they are not meeting our
new requirements.

DR. SCHUTSER: I think frankly the intent of moving away from the old
spec to commercially acceptable standards is probably to relax in some
cases some of the requirements, but in some instances it has been noted
that some of the requirements might be independent of six. That's what
we're saying in five in things like the reliability.

We think that future specs should specify the systems that are more
reliable then the current reliability specifications, which would allow
roughly one in ten systems to fail during the voting process.

So I'd just as soon \-- we're saying keep it where appropriate. So we're
trusting in this not to do something that would provide something so
onerous as to disqualify good and valid systems, but on the other hand
if they think in some cases the bars were set a little too low for
what's appropriate they should feel free to raise it and of course all
those changes they will be presenting, we'll be reviewing.

MALE SPEAKER: Right. And I'm all in favor of raising the bar. I think it
needs to be raised but I also think we are basically 11 months from the
beginning of the next major election cycle and we need to concentrate on
only those things that Congress has mandated be in place in '06, and
then we need to be doing a substantial amount of work to raise the bar
beyond that.

DR. SMURGIAN: Mr. Burger.

MR. BURGER: I think Paul's concern has come up a couple of times. I'm
not sure I'm comfortable addressing it here but perhaps Dan, the next
resolution we ought to look at what we're suggesting in terms of the
phase in for these recommendations because isn't that the heart of what
you're saying, Paul? We need to understand \--

MALE SPEAKER: Yes.

MR. BURGER: Are we talking about equipment going forward or are we
talking about a retrospective set of requirements.

MALE SPEAKER: Well we're talking about Congress has required basically
that NIST take over the voting systems certification process and that to
some extent the EAC take over from NASET the qualifications process,
which NASET has been doing for many years.

Congress in that Act gave specific requirements for '06. Those
requirements basically were based on congressional evaluation of the
best of the current voting systems that were out there. Now they went to
great pains to make sure that the stuff that's out there that's
recognized as being fairly good be adopted by more states. They also put
an emphasis on continued development research and improvement of
systems.

And that's the congressional intent behind this bill and that's what we
need to be focusing on is bringing the accessibility requirements
basically of how HAVA \-- and as a requirement for all currently
qualified voting systems, and then moving forward with improving those
in the future.

DR. SCHUTSER: Well if you can suggest something. All this is really
saying I think is that we should view the current specs where they're
not sufficiently precise for consistent testing, ask NIST to attempt to
update it and where appropriate to review the specs and modify those too
in cases where appropriate.

MALE SPEAKER: And all I'm saying is be mindful of the danger of imposing
a new specification inadvertently when you apply new testing methods.

DR. SCHUTSER: That's duly noted but is there any suggested changes you'd
want to make?

MALE SPEAKER: I think number seven does that.

DR. SCHUTSER: Read number seven?

DR. SMURGIAN: Current standards, is that what you want to say?

MS. QUESENBERRY: Significantly different from what is really the crux of
this question, especially since our committee is about to propose a raft
of new and updated standards to meet the accessibility requirements. I'm
particularly concerned about this.

MALE SPEAKER: Well I'm concerned that the test methods not by their
nature impose standards I guess. Standards should come without \-- and
the test method should be designed to test on \--

DR. SHCUTSER: Yeah, the test methods won't be from the standards,
they'll be \-- but they'll be restating the standards in such a way
that's precise enough to test.

**(CROSS TALK)**

MALE SPEAKER: Only the test method is an issue.

MS. QUESENBERRY: Perhaps number six should stay, determines it's
advisable \-- commonly used equivalent commercial test methods.

MALE SPEAKER: All right.

MS. QUESENBERRY: As I understand it your point is to switch from a
military standard, which is not widely adopted by test labs to
commercial versions of the test which would achieve the same result but
which are more widely known, rather then changing the underlying
requirements.

MALE SPEAKER: Precisely.

MS. QUESENBERRY: So I think it's the equivalency that's really the
issue.

MALE SPEAKER: Would that satisfy you?

MALE SPEAKER: Yeah.

DR. SMURGIAN: So we're taking out number seven?

MALE SPEAKER: Taking out number seven \-- we never finished it and I'm
settling for equivalent in number six.

MALE SPEAKER: We're going to need a resolution in its own \-- if we're
the body to handle it on how these standards phase in with what's in
existence today.

DR. SMURGIAN: So is this acceptable as a friendly amendment, Dr.
Schutser?

DR. SCHUTSER: Yes.

DR. SMURGIAN: Any other questions, comments? Dr. Revest.

DR. REVEST: I just wanted to support Steve's suggestion that we have a
separate phase in resolution. We need to really consider how changes to
the standards will be phased in and I think that's an important question
that deserves careful consideration as a separate amendment.

And I think also implicit in what was said here though, there was an
implication which I think is probably appropriate that the NIST needs to
consider the reliability portion of the VSS, in particular in other
parts of VSS for improvement at the standard level. And that's maybe a
separate resolution as well but I don't want to see that dropped.

MALE SPEAKER: I don't have anything serious here. I agree with that. I
think this says what we want it to say and that is that we want to use
the portions of the current standard that are good and improve the
portions that aren't, and if there's holes in it we want to fill in the
holes and I think this says that.

And as far as a transition strategy, I would recommend that we postpone
discussion of that to a later date when we've begun to evolve a standard
and can look and talk in terms of the differences.

I mean what are we transitioning to? So that's important but not for
this meeting.

MS. QUESENBERRY: Here, here.

DR. SMURGIAN: May I suggest that maybe that's \-- I mean I think that's
going to take some thinking and perhaps that's something we should think
about considering at the next meeting.

MALE SPEAKER: Yes.

DR. SMURGIAN: Okay. Mr. Burger.

MR. BURGER: Just for clarity and for the record, in number five one of
the specific concerns that we want the staff to look at is some of the
mean time between failure numbers that on an individual unit basis may
seem reasonable but when you look at the impact potentially on something
that just meets the minimum on a system deployment, you know, may be an
inordinately high failure rate if something just barely met the minimum.
So that's one of the concerns there.

DR. SMURGIAN: It looks like the resolution has been modified through
friendly amendments.

MS. QUESENBERRY: Excuse me, there's one more edit that \-- there's an
edit that Mr. Schutser made that hasn't been made here and that was in
number four to remove a subset of.

DR. SCHUTSER: Right. Eliminate \-- you didn't do that. A subset of under
number four should be removed.

DR. SMURGIAN: Just take out for those (unintelligible), that's it,
right.

I think there is a bit of an English to be worked out because all the
others start with a verb but if that's okay we'll handle that offline.

MALE SPEAKER: Move to question.

DR. SMURGIAN: So with that I hear a move to question so do we have a
second?

MALE SPEAEKR: Second.

DR. SMURGIAN: Thank you. Mr. Green, will you please take a roll call for
Resolution 25-05, as modified.

MR. GREEN: Smurgian?

DR. SMURGIAN: Yes.

MR. GREEN: Smurgian votes yes. Davidson? Davidson not responding.
Miller?

MS. MILLER: Yes.

MR. GREEN: Miller votes yes. Turnerbuoy?

MS. TURNERBUOY: Yes.

MR. GREEN: Turnerbuoy votes yes. Purcell?

MS. PURCELL: Yes.

MR. GREEN: Purcell votes yes. Harding?

MR. HARDING: Yes.

MR. GREEN: Harding votes yes. Ellekese?

MR. ELLEKESE: Yes.

MR. GREEN: Ellekese votes yes. Caldise?

MS. CALDISE: Abstain.

MR. GREEN: Caldise abstains. Burger?

MR. BURGER: Yes.

MR. GREEN: Burger votes yes. Williams?

DR. WILLIAMS: Yes.

MR. GREEN: Williams votes yes. Kraft?

MR. KRAFT: Yes.

MR. GREEN: Kraft votes yes. Revest?

DR. REVEST: Yes.

MR. GREEN: Revest votes yes. Schutser?

DR. SCHUTSER: Yes.

MR. GREEN: Schutser votes yes. Gannon? Gannon not responding.
Quesenberry?

MS. QUESENBERRY: Yes.

MR. GREEN: Quesenberry votes yes. That's 12 votes for, two voters not
responding, one voter abstaining.

DR. SMURGIAN: Resolution 25-05 has been adopted. Dr. Schutser, the next
one.

DR. SCHUTSER: I'd like to move to 26-05, Uniform Testing Materials.

For consistency and transparency of voting systems testing and to
increase the public trust and competence in the testing of voting
systems it is necessary that the same set of testing materials be used
by each testing organization.

The TGDC requests that NIST, and I would take out the colon and the
number and just say request that NIST draft guidance for how to develop
a public set of test materials.

FEMALE SPEAKER: Second.

DR. SMURGIAN: We have a motion and second. Any questions, comments? Dr.
Revest.

DR. REVEST: Yes, I just wanted a little bit of clarification as to what
was imagined to be within the scope of "test materials", and also the
question as to whether this presupposes that all the testing will sort
of be given with fixed inputs or whether, you know, tests that involve
dynamically generated random numbers would also be allowed.

DR. SCHUTSER: My thinking is that this would include both some standard
input test sets and also some standard procedures and algorithms for
generating the kind of random test that you were referring to. Do you
have any other thoughts to elaborate, Alan, on this?

MALE SPEAKER: I would agree but where \--

DR. SMURGIAN: Alan, would you please take the microphone there?

MALE SPEAKER: Yeah, I would agree, the thrust here is the consistency
among labs, that whatever instructions are provided or whatever
procedures are developed by one lab should be the same as the procedures
that are applied by other labs and, you know, to the extent that the VSS
provides these instructions, they should apply uniformly among the labs.

DR. SMURGIAN: Thank you. Ms. Quesenberry.

MS. QUESENBERRY: I had essentially the same question but from a slightly
different perspective of our committee. Are you assuming that this would
include, or I hope that you're assuming that this would include ballot
definitions, that is the elections to be tested so that where \-- sorry,
I just lost my ability to speak English there. That test materials would
include the definitions of the ballots, that is the races, the
candidates, the number of candidates and so on that would be included in
any testing that for instance included usability or accessibility and so
on.

DR. SCHUTSER: Yeah, to the extent that that's included in the special
occasions, yes, that would be tested.

DR. SMURGIAN: Dr. Williams.

DR. WILLIAMS: One of the problems if you want to use that word that we
have with the current ITA process is maintaining uniformity in the
testing between the various ITAs of which now there are exactly two.
We've lost Wiley.

NIST also has the responsibility for developing and maintaining the lab
functions and so I do think that uniformity among the laboratory testing
is extremely important. If that testing is not uniform, it has a lot of
implications, not the least of which is that the vendors will shop the
laboratories for the easier labs so to speak.

So the standard \-- this is not something we can do in the standards
other then just have a statement in the standards to the effect that
laboratory testing should be uniform and then it's going to be up to
whoever manages the labs, Mark, that probably are not doing anything on
Sunday afternoon now, to enforce that and see to it that the testing \--
but that is an ongoing thing that has to be monitored continuously by
whoever is managing the laboratory process.

MS. QUESENBERRRY: Just a question. Is that true that we can't in the
test specify the materials for that test?

DR. WILLIAMS: Yeah, we can test certainly \-- we can test that and
that's what I'm saying, this is a good amendment that moves toward that,
but when you get down where the rubber meets the road it's an
enforcement thing after the fact to keep the labs testing consistency.

MS. QUESENBERRY: Right. But if we had a test that said you have to do it
this way, and this way, and this way, and that a vote for one rays, for
two rays of it, you know, whatever, that that \-- never mind.

DR. SCHUTSER: You could have a standard if you wanted to modify \-- for
example let's talk about the availability mean time to failure type of
thing. You could specify a test, the test (unintelligible) failure and
specify in the specification that a voting system would have to be able
to perform that test and meet at least a certain number. In that sense
you would be building into the spec a test procedure.

DR. SMURGIAN: Mr. Burger and then maybe (unintelligible) comment. Go
ahead.

MR. BURGER: Yes, I just wanted to comment that while we certainly should
try and make the standards as specific and clear as possible, I think
over and over again we've found in all kinds of areas that there needs
to be an ongoing function of getting labs together, keeping them
updated, making sure that the way they read the English language is the
same and uniform.

DR. SMURGIAN: Mark, did you want to \--

MALE SPEAKER: (Off microphone).

FEMALE SPEAKER: I think to make a clarification here by what is meant
here as to draft guidance. It's not to be part of the standard itself.
It addresses the part of core requirements and testing and the objective
is provide guidance or a blueprint as to how to go about creating a
public set of test scenarios or test suites using what is already
existing in the states by the testing labs and collecting that so that
there is a common set. This could be a minimum set that everybody uses.

It doesn't mean that a testing lab or a state could \-- they are welcome
to do more but the idea here was to address can we have a common minimum
set that is publicly available so everybody including the public can see
what is being done.

DR. SMURGIAN: Thank you. Dr. Revest.

DR. REVEST: Reading the resolution as proposed I'm a little confused by
the wording, at least the understanding behind the wording.

Draft guidance for how to develop a public set of test materials. So
there's \-- the understanding I'm getting from this as to who is
developing this public set of test materials, whether it's expected that
NIST would be developing these or the testing labs or somebody else.

Also it says public set of test materials. If the goal is consistency
between the labs, I mean merely making it public doesn't necessarily
make them consistent. You could have several different public things so
it doesn't seem to hit the nail on the head in terms of the consistency
we're looking for. Just merely publication is not enough.

DR. SMURGIAN: Dr. Schutser, would you like to comment on that?

DR. SCHUTSER: I think what we're saying is we expect there will be a
companion document of the specifications that would provide a guardian
of instructions on how to conduct the test along with test A, test et
cetera, where possible, and we would as was mentioned expect that that
could be used to certify equipment and whatever additional tests that
the states might want.

DR. SMURGIAN: Mr. Kraft.

MR. KRAFT: Yes. I think this may help clarify it a little bit. One
thing, we're not talking test materials we're talking a set of tests
suites and test scripts and those things to an extent are going to be
developed and practiced.

So I think what we're looking for here is number one, taking the
existing art and voting system testing in the current standards that's
out there, putting that in the box, then having NIST look at a process
for putting new art in that box and managing the box.

Obviously as the labs encounter new systems and they attempt to apply
the standards at the time, and the test scripts and methods that are in
their inventory of standard methods, they're going to develop new stuff.

There has to be a way of those new tests steps being vetted, being
approved, having some sort of external review and becoming standard
practice and I think that's kind of what we're asking NIST to give us
guidance on.

I think within that guidance will come a recommendation for who should
do that, how it should be done, what kind of feedback loops are going to
be involved, and in that context I'm actually fairly happy with the
resolution as drafted except we could perhaps go from testing materials
to go to something like methods, and scripts, and processes instead of
materials.

DR. SMURGIAN: Past methods and procedures, something like that?

DR. SCHUTSER: So do we want to say a public set of tests \--

MR. KRAFT: Methods and procedures.

DR. SMURGIAN: Is that acceptable to you?

DR. SCHUTSER: That's acceptable. And I think what was said is that they
didn't go as far as to say to actually develop this because one was
conscious of the time available in order to get all this done. So that's
why it's guidance as opposed to actually developing this.

MR. KRAFT: Yeah, and I guess we don't really have anyone set out to
manage this program yet. Congress really has not created a federal
entity for that. I think the EAC is considering what its options will be
in managing this but what we do want NIST to do right now is draft
guidance on how to evolve that public set of methods and procedures for
adoption by whoever the management entity for this process is going to
be.

DR. SMURGIAN: So we have a friendly amendment to delete the last word in
the resolution, materials, and replace that with methods and procedures.

MALE SPEAKER: I would recommend in the title also.

DR. SMURGIAN: Okay.

MALE SPEAKER: And in the second word on the last line of the first
paragraph.

DR. SCHUTSER: Right. And also the title, instead of Uniform Testing
Materials, we'll call it Uniform Testing Methods and Procedures.

DR. SMURGIAN: Make the resolution consistent. Whenever we have testing
materials we'll replace that with test methods and procedures.

Any other questions, comments? Do we have any comments from members on
the phone?

FEMALE SPEAKER: Not at this time.

FEMALE SPEAKER: No.

MALE SPEAKER: Not at this time.

MS. DAVIDSON: And Donna Davidson is on the phone, been listening and I
thought maybe I should let you know I am on the phone.

DR. SMURGIAN: Thank you, good to have you with us.

MS. DAVIDSON: Thank you.

DR. SMURGIAN: Do I have a motion to move the \--

MALE SPEAKER: Call the question.

DR. SMURGIAN: Thank you. Mr. Green, will you please take the roll call.

MR. GREEN: This is for 26-05 as amended. Smurgian?

DR. SMURGIAN: Yes.

MR. GREEN: Smurgian votes yes. Davidson?

MS. DAVIDSON: You know, I haven't been on long enough to know so I'll
pass.

MR. GREEN: Davidson abstains. Miller?

MS. MILLER: Yes.

MR. GREEN: Miller votes yes. Turnerbuoy?

MS. TURNERBUOY: Yes.

MR. GREEN: Turnerbuoy votes yes. Purcell?

MS. PURCELL: Yes.

MR. GREEN: Purcell votes yes. Harding?

MR. HARDING: Yes.

MR. GREEN: Harding votes yes. Ellekese?

MR. ELLEKESE: Yes.

MR. GREEN: Ellekese votes yes. Caldise?

MS. CALDISE: Abstain.

MR. GREEN: Caldise abstains. Burger?

MR. BURGER: Yes.

MR. GREEN: Burger votes yes. Williams?

DR. WILLIAMS: Yes.

MR. GREEN: Williams votes yes. Kraft?

MR. KRAFT: Yes.

MR. GREEN: Kraft votes yes. Revest?

DR. REVEST: Yes.

MR. GREEN: Revest votes yes. Schutser?

DR. SCHUTSER: Yes.

MR. GREEN: Schutser votes yes. Gannon? Gannon not responding.
Quesenberry?

MS. QUESENBERRY: Yes.

MR. GREEN: Quesenberry votes yes. That's 12 votes yes, two voters
abstaining, one voter not responding.

DR. SMURGIAN: Therefore Resolution 26-05 has been adopted as amended.
Dr. Schutser.

DR. SCHUTSER: Next I'd like to present Resolution 27-05.

DR. SMURGIAN: May I suggest that this will be the last resolution we
will take up before lunch. Thank you.

DR. SCHUTSER: Title, Non-Conforming Voting System. A provision in the
2002 VSS allows qualification of voting systems that do not conform to
the requirements and then in parenthesis("any uncorrected deficiency
that does not involve the loss or corruption data shall not necessarily
be cause for rejection). If there are requirements that are frequently
unmet by qualified systems these requirements should be reviewed for
possible elimination.

The TGDCC requests that NIST, and take out the colon and the number one,
review the text of the 2002 VSS to determine if provision of the
qualification of voting systems that do not conform to requirements
should be deleted.

DR. SMURGINA: Do we have a second?

MALE SPEAKER: I second.

DR. SMURGIAN: Thank you, we have a second. Discussion?

MALE SPEAKER: Let me give you a little historical perspective here. That
sentence that you quoted originated in the 1990 standards, which were
written in the latter part of the '80s and at the time that those
standards were written there were no previous standards so every voting
system in use in the country had not been developed to standards.

So that was a recognition that when we started trying to bring these
Legacy systems into this process that we might find all kinds of little
discrepancies in them and that we didn't want to get into a thrashing of
insignificant discrepancies. And so that sentence was put in to allow us
to let insignificant discrepancies go through the net.

Now the situation we have now is that every voting system in place with
the exception of very, very few \-- most all of the DRE and all of the
optical scan systems were developed under the voting system standards so
they don't contain these discrepancies.

So it may well be that we can just delete that sentence from the
standard. It just is not applicable anymore.

DR. SMURGIAN: Are you saying that we should withdraw this or are you
saying that the language of the resolution be changed?

MALE SPEAKER: Well I think the resolution is fine exactly the way it is
and I'm just advising NIST that when they review this that one of the
possible outcomes may be to recommend that we just delete that sentence.

DR. SMURGIAN: Okay, thank you.

MALE SPEAKER: Call the question.

DR. SMURGIAN: Well sounds like everybody's hungry so we called the
question. Mr. Green, will you please take the roll call on Resolution
27-05?

MR. GREEN: Smurgian?

DR. SMURGIAN: Yes.

MR. GREEN: Smurgian votes yes. Davidson?

MS. DAVIDSON: Yes.

MR. GREEN: Davidson votes yes. Miller?

MS. MILLER: Yes.

DR. GREEN: Miller votes yes. Turnerbuoy?

MS. TURNERBUOY: Yes.

MR. GREEN: Turnerbuoy votes yes. Purcell?

MS. PURCELL: Yes.

MR. GREEN: Purcell votes yes. Harding?

MR. HARDING: Yes.

MR. GREEN: Harding votes yes. Ellekese?

MR. ELLEKESE: Yes.

MR. GREEN: Ellekese votes yes. Caldise?

MS. CALDISE: Abstain.

MR. GREEN: Caldise abstains. Burger?

MR. BURGER: Yes.

MR. GREEN: Burger votes yes. Williams?

DR. WILLIAMS: Yes.

MR. GREEN: Williams votes yes. Kraft?

MR. KRAFT: Yes.

MR. GREEN: Kraft votes yes. Revest?

DR. REVEST: Yes.

MR. GREEN: Revest votes yes. Schutser?

DR. SCHUTSER: Yes.

MR. GREEN: Schutser votes yes. Gannon? Gannon not responding. And
Quesenberry?

MS. QUESENBERRY: Yes.

MR. GREEN: Quesenberry votes yes. That's 13 votes for, one voter
abstaining, one voter not responding.

DR. SMURGIAN: With that vote Resolution 27-05 has been adopted as
presented. Thank you.

Well we are within 40 seconds of our slated time so I would propose that
we break for lunch and come back to start sharply at 1:30 p.m. Thank
you.

(Lunch Break)

DR. SMURGIAN: As soon as we can corral the rest of the members of the
committee we'll get going. So please, would everybody take your seats?

While we're waiting I should mention that the last shuttle from NIST to
the metro will be at 5:15 p.m. If you need a cab or any other help with
transportation please see Mary over there so that we can make the
appropriate arrangements for you.

MALE SPEAKER: Is there a shuttle going back to the Hilton? Are you going
back to the Hilton? Okay.

DR. SMURGIAN: Okay, I guess we need Dr. Schutser.

MALE SPEAKER: Yes, since it's his committee. Why don't we just go on and
pass some of these resolutions. We'll tell him about it.

DR. SMURGIAN: He may not appreciate that.

DR. SCHUTSER: I apologize. Sorry to be delayed.

DR. SMURGIAN: Okay, now that all the members are here I hope that all
the members that are on the phone have joined us and we'll proceed with
the resolutions that were put forth by the Core Requirements of Testing
Subcommittee. Dr. Schutser.

DR. SCHUTSER: Thank you, Mr. Chairman. The next resolution I'd like to
cover is 28-05, Publicly Available Qualification Data.

The TGDC recognizes that (unintelligible) are standard practice in many
arenas where public trust and/or safety are at stake. To the extent
possible qualification test reports should be released to the public as
evidence that the qualification process was responsibly executed.

To handle those tests \-- those cases where release of the entirety of
the report is problematic the TGDC requests that NIST, again I would
take away the quotes and the numbering system, recommends standards and
qualification data to be provided called "public information package"
that would set out minimum requirements on the information that must be
publicly available and published.

MALE SPEAKER: I second that.

**(CROSS TALK)**

DR. SMURGIAN: Okay, we have a motion and a second on the floor. Dr.
Williams, and then Mr. Kraft, and then Mr. Burger.

DR. WILLIAMS: This is a good resolution and I think it's one that we can
pretty much \-- everybody support the way it's written.

However I want to suggest that in view of the NIST workload that this is
an activity that we can put off until we know what the reports are going
to look like, and then at a later time after the standards are written
and we begin to work with laboratories we can sit down with that package
at that time and develop this information package.

So I recommend approval of this with the understanding that it is not a
priority item between now and April.

DR. SCHUTSER: That's fine with me.

DR. SMURGIAN: Mr. Kraft.

MR. KRAFT: I'd like to suggest a friendly amendment. As much as the
state of Florida appreciates the compliment, I'm not sure sunshine rules
is a universally applied term and I would suggest we insert the phrase
public records laws.

MALE SPEAKER: Recognizes the public record (unintelligible).

MR. KRAFT: Public records laws. And other then that I fully support the
resolution.

DR. SCHUTSER: I accept that change.

MALE SPEAKER: I'd like to call for a vote.

MALE SPEAKER: I second.

DR. SMURGIAN: So we've replaced the word sunshine rules with public
record laws.

MALE SPEAKER: Right, only take away the semi-colon and the I.

MALE SPEAKER: Since the question's been called up I defer my comment.

DR. SMURGIAN: Okay. Dr. Revest.

DR. REVEST: I just wanted to (unintelligible). I support the sense of
the resolution. We'd like to make a lot more information available about
the qualification process and the results, and there is a related
resolution that appears in the Security one to which we didn't get to
yet and I wanted to speak to a couple of issues that that other
resolution addressed that this one doesn't and maybe can consider how to
combine or address them there.

My first issue is whether this is within scope of \--

MALE SPEAKER: Point of order.

DR. SMURGIAN: Yes.

MALE SPEAKER: Well we've got a resolution on the floor. We've called a
question and now it sounds like we're beginning to entertain motions to
amend the resolution.

DR. SMURGIAN: No, we're simply discussing it. I'd like to hear what the
issues \--

DR. REVEST: I'm not introducing a motion at this point. I'm raising some
questions for the committee to consider about either this resolution or
other resolutions if we want to.

I'll proceed on that. So one is whether this is actually within scope of
what the committee is allowed to do.

One of the issues that we struggled with in our committee was the point
of view that requiring publication of qualification reports may be
outside the scope of what this committee is charted to do in the sense
that it's not specifying standards to be tested to but something
external to the qualification process. And I'm not sure how we address
that here.

DR. SMURGIAN: Given Dr. Williams' comments, and since we haven't had the
time to act on your resolutions, and since the point was made that
perhaps this is not a highest priority and that it could be dealt with
later on, may I suggest that if you feel that there are some overlapping
issues between this resolution and some of your resolutions, and that
since this is not an immediate issue, would you like to withdraw and
then work with Dr. Revest to explore ways of combining some of these and
then bring them up at the next meeting, or is that not \--

DR. SCHUTSER: (Off microphone). (Unintelligible).

MALE SPEAKER: I concur with that. I'd like to get this one on the books
and then that certainly doesn't prevent us from going back and expanding
it.

DR. SMURGIAN: Yes, Mr. Kraft.

MR. KRAFT: And while I agree with you Dr. Revest, that this may be
slightly out of scope for the actual voting system standards themselves,
it is an important part of the enforcement of the standards.

I mean ultimately this will probably wind up in the work product for ITA
accreditation labs and within the standards for labs, will be reporting
standards and work product standards for them but nonetheless I think
the statement that these documents are to be publicly available is
important and needs to be made.

DR. REVEST: May I ask, is that a statement of something that's
enforceable then or is that a statement of just sort desirability? It
would be nice if we were able to mandate that certain information be
made public.

MALE SPEAKER: Well I think it's a statement of desirability. We can't
really decide what's going to be made public until we define what the
work product of the labs will be.

There is going to be in the output from the labs trade secret
information, which at least in our state, release of trade secret
information is a third degree felony.

**(END OF SIDE A, START OF SIDE B)**

MALE SPEAKER: There will be things in the output of the labs that will
rise to that level and someone at some point is going to have go
through, okay, here is the records and the work product that the labs
must produce and then this particular part of it is going to be public.
I think it's premature to try to make that a narrow requirement but it's
certainly a good time to state the concept and the goal.

DR. SMURGIAN: Okay, next Mr. Burger, and then I would like to ask one of
our folks form NVLAP to see if this represents any kind of an issue for
their (unintelligible) process. Mr. Burger.

MR. BURGER: I support the motion and by way of reference, in a number of
industry areas this is standard practice. As one example, anyone can go
to the FCC website, look up the equipment grant for any wireless device
and what's restricted under confidentiality is very limited. In fact you
can find a block diagram for any wireless product that's FCC approved
and full test reports. There's nothing new here for many industries.

DR. SMURGIAN: Would you introduce yourself, please?

MR. HORLICK: Yes, I'm Jeffrey Horlick and I'm on the staff of the
National Voluntary Laboratory Accreditation Program.

What we would do, would be to see that the laboratories meet the
requirements that were assigned to them. So if the EAC for instance were
to require certain information to be made public, we would simply look
to see that the laboratories would do that. So it would not be a NVLAP
requirement. We would simply look for conformance to the requirement.

DR. SMURGIAN: Thank you. Any other questions or comments? From members
on the phone, do we have any questions or comments?

DR. REVEST: May I add another comment?

DR. SMURGIAN: Yes, Dr. Revest.

DR. REVEST: In the desire for maximum transparency in the process here,
this request that will set out the minimum requirements on the
information that must be publicly available, maybe we could have
something trying to set out the requirements for what is allowed to not
be public on the presumption that everything else is.

MALE SPEAKER: Yeah, that would be the better way to implement.

DR. REVEST: All right. To note, you know, the following kinds of
information \-- this would identify information that needs to be
restricted for some reason and all other information resulting from the
evaluation would be made public. So the default is public.

DR. SMURGIAN: Okay, did you want to suggest a change in the wording?

DR. REVEST: I could suggest wording. Maybe that's \-- is that the most
efficient way to do that?

DR. SMURGIAN: Well in the meantime we'll entertain a comment from Ms.
Quesenberry.

MS. QUESENBERRY: I'm thinking about usability and accessibility testing
and wondering \-- those generate a lot of data of which a subset of that
data if made public in an appropriate form becomes rather useful. So
just saying you don't have to keep it private actually isn't enough to
create a public information package.

So I'd be looking not just for what the data is but something about
format or how it was collected. So without going into lots of detail \--
but I'm not sure that just saying these things are excluded does enough
to solve the problem of allowing someone to look at the usability and
accessibility of a system that's been qualified and see what they
thought about that.

MALE SPEAKER; Because there's so much \--

MS. QUESENBERRY: Well it would be very easy in that case to drown the
real information by just pumping out lots of data and if what we were
doing was specifying a public information package that said, and this
test, which of course would have been defined by the standard, will be
reported on in the following way, those reports are usually summary
reports that wrap the data up in a consistent way so that you're not
creating hide in plain sight problems.

DR. SMURGIAN: Are you suggesting that we then keep the language of, set
of minimum requirements? Is that what you're \--

MS. QUESENBERRY: Yes.

MALE SPEAKER: The expectation here though is that the minimum is what
will be met and no more.

MS. QUESENBERRY: That's correct. I would imagine that's correct. And it
might be that we combine them, which is that you might want to know what
the minimum was and also what things could be excluded because from the
point of view of future development, the fact that we haven't just
omitted something but have specifically decided that it could be
excluded from public record would be good information to have since you
can't tell from just not saying it what you mean. That's the problem
we've wrestled with in a number of our resolutions.

DR. SMURGIAN: Dr. Williams.

DR. WILLIAMS: Yeah, we don't want to confuse public information with
need to know information. If you've got a need to know, like you're on
the Access Board or what have you, you can get access to information
that's not in this public package.

MS. QUESENBERRY: I understand.

DR. WILLIAMS: When we do certifications on voting systems we get all
kinds of stuff. We get source codes. I mean we make them open up the
kimono. So this is not what we're talking about here. We're talking
about that information which is just generally available whether you
need to know it or not.

MS. QUESENBERRY: Yes.

DR. SMURGIAN: So where are we? Are we making any amendments or are we
basically \--

DR. REVEST: If I could propose an amendment then to just \-- maybe add a
new sentence at the end. The TGDC encourages NIST to set minimum
requirements rather liberally. The intent being that a fair amount will
be made public.

DR. SMURGIAN: And that would be a sentence added to the very end? TGDC
encourage NIST to set requirements rather liberally.

DR. REVEST: Yeah, I'm not sure that's the best wording but \--

**(CROSS TALK)**

MS. QUESENBERRY: What if we just took out the word minimum?

DR. SMURGIAN: NIST staff having heard this discussion that perhaps we
should move towards identifying a minimum set and identifying those that
clearly don't belong in the public domain, can we then do exactly what's
suggested, take out the word minimum with the understanding that the
NIST staff will keep that in mind and draft the standard accordingly? Is
that acceptable, Dr. Schutser?

MALE SPEAKER: You're saying you won't have (unintelligible).

DR. SMURGIAN: No, we'll simply take the amendment \-- keep it as it was,
take out the word minimum from the \-- simply \-- that will set out
requirements on the information that must be publicly available and
published, with the understanding that they will also identify data that
is excluded from public domain.

DR. REVEST: I'm happy with that and I withdraw my suggestion for an
amendment then.

DR. SMURGIAN: Mr. Kraft.

MR. KRAFT: I support the resolution as amended but since we have
discussed in front of NIST staff this issue here, I think it's very
important that the disclosure and reporting part of the work product
look at what information is actually needed.

I mean as far as public records laws in Florida, as probably one of the
more liberal examples, anything that is not specifically prohibited by
statute, and most specifically trade secret information, must be
disclosed.

And trade secret information is very narrowly defined as that
intellectual property or information that an individual owns that would
basically erode his rights or economic circumstance, or give someone
else a competitive advantage if he lost control of his secrecy of it and
there's certainly adequate court cases around that.

I think the bigger issue here is structuring the information that is
published so that it's usable, so that decision makers at a variety of
levels can use it, and ultimately so that the curious member of the
public who has no official capacity but just wants to see what's there,
can muck around through all the data.

So there needs to be a standard for reporting specified \-- with the
required elements of perhaps an executive summary, a required layout of
findings that are in the beginning of the report, a required structure
for disclosing the tests and the findings that supported those higher
findings, and then references to the work papers from the testing.

And all that I think is kind of beyond our scope here today but that's
kind of what the NIST staff is going to have to deal with in working
through that.

DR. SMURGIAN: So was that for the benefit of the NIST staff here or did
you want to make an amendment to the motion?

MR. KRAFT: That was for the benefit of the NIST staff in light of us
having bantered around words such as be liberal in what's disclosed and
those kind of things. I mean we don't want to lose sight that you have
in reporting, a duty to communicate to various audiences. But I'm happy
with the resolution that's drafted.

DR. SMURGIAN: Any further comments or questions? Any of the members on
the phone?

MALE SPEAKER: No.

DR. SMURGIAN: Any questions? Not hearing any further questions, do I
have a call to motion?

MALE SPEAKER: Motion.

FEMALE SPEAKER: Second.

DR. SMURGIAN: Thank you. Mr. Green, will you please take the roll call
on Resolution 28-05, with two minor changes.

On the first line now it reads that TGDC recognizes that public record
laws, instead of sunshine rules, are standard practice, et cetera, et
cetera. And then the number one item is removed, and that text is made a
continuation of the original paragraph and the word minimum out of
minimum requirements is deleted. Let's go on with the roll call.

MR. GREEN: Resolution 28-05 as amended. Smurgian?

DR. SMURGIAN: Yes.

MR. GREEN: Smurgian votes yes. Davidson? Davidson no response. Miller?

MS. MILLER: Yes.

MR. GREEN: Miller votes yes. Turnerbuoy? Turnerbuoy no response.
Purcell?

MS. PURCELL: Yes.

MR. GREEN: Purcell votes yes. Harding?

MR. HARDING: Yes.

MR. GREEN: Harding votes yes. Ellekese?

MR. ELLEKESE: Yes.

MR. GREEN: Ellekese votes yes. Caldise?

MS. CALDISE: Abstain.

MR. GREEN: Caldise abstains. Burger?

MR. BURGER: Yes.

MR. GREEN: Burger votes yes. Williams?

DR. WILLIAMS: Yes.

MR. GREEN: Williams votes yes. Kraft?

MR. KRAFT: Yes.

MR. GREEN: Kraft votes yes. Revest?

DR. REVEST: Yes.

MR. GREEN: Revest votes yes. Schutser?

DR. SCHUTSER: Yes.

MR. GREEN: Schutser votes yes. Gannon? Gannon not responding.
Quesenbery?

MS. QUESENBERRY: Yes.

MR. GREEN: Quesenberry votes yes. That's 11 votes for, three members not
responding, one abstaining.

DR. SMURGIAN: With that, Resolution 28-05 as amended has been adopted.
Dr. Schutser, next one.

DR. SCHUTSER: I'd like to introduce 29-05, Insuring Correctness of
Software Code, Volume I, Section 4.2, and Volume II, Section 5.4 of the
2002 VSS Defined Coding Standards, as well as a source code review to be
conducted by independent testing authorities, ITAs to enforce those
coding standards.

These coding standards are a means to an end, the end being an ITA
evaluation of the codes correctness to a high level of assurance.

The TGDC requests that NIST, one, recommends standards to be used in
evaluating the correctness of voting system logic including but not
limited to software implementations, and two, evaluate the 2002 VSS
Software Coding Standards with respect to their applicability to the
recommended standards and either revise them, delete them, or recommend
new software coding standards as appropriate.

DR. SMURGIAN: We have a motion on the table. Do I have a second?

MALE SPEAKER: I'll second.

DR. SMURGIAN: We have a second. Discussion? Dr. Revest.

DR. REVEST: So I'm interpreting the phrase coding standards here to
refer not only to the way in which the code is written but in the way in
which it's documented?

DR. SCHUTSER: Yes.

DR. REVEST: Thank you.

DR. SMURGIAN: Is that understood or do we need to make a change?

DR. SCHUTSER: That's usually commonly understood, that the coding
practice includes good documentation.

DR. REVEST: I'm intending that question to mean not just the
documentation contained within the code but auxiliary documentation
describing the architecture and so on too, some of which you've already
talked about in a previous resolution.

DR. SMURGIAN: Any other questions, comments? Call the question. Second?

MALE SPEAKER: I'll second.

DR. SMURGIAN: Okay. Mr. Green will you please take the roll call for
Resolution 29-05 as presented?

MR. GREEN: Smurgian?

DR. SMURGIAN: Yes.

MR. GREEN: Smurgian votes yes. Davidson? Davidson not responding.
Miller?

MS. MILLER: Yes.

MR. GREEN: Miller votes yes. Turnerbuoy? Turnerbuoy not responding.
Purcell?

MS. PURCELL: Yes.

MR. GREEN: Purcell votes yes. Harding?

MR. HARDING: Yes.

MR. GREEN: Harding votes yes. Ellekese?

MR. ELLEKESE: Yes.

MR. GREEN: Ellekese votes yes. Caldise?

MS. CALDISE: Abstain.

MR. GREEN: Caldise abstains. Burger?

MR. BURGER: Yes.

MR. GREEN: Burger votes yes. Williams?

DR. WILLIAMS: Yes.

MR. GREEN: Williams votes yes. Kraft?

MR. KRAFT: Yes.

MR. GREEN: Kraft votes yes. Revest?

DR. REVEST: Yes.

MR. GREEN: Revest votes yes. Schutser?

DR. SCHUTSER: Yes.

MR. GREEN: Schutser votes yes. Gannon? Gannon not responding.
Quesenberry?

MS. QUESENBERRY: Yes.

MR. GREEN: Quesenberry votes yes. Eleven votes in favor, three voters
not responding, one abstaining.

DR. SMURGIAN: Resolution 29-05 has been adopted.

DR. SCHUTSER: Thank you, Mr. Chairman. I'd like to present the next
resolution 30-05, Quality Management Standards, Volume I, Section 7 and
8, and Volume II, Section 7 of the 2002 VSS. Require a Vendor to Follow
Certain Quality Assurance and Configuration Management Practices and
require the ITA to conduct several audits and documentation reviews to
insure that they were followed.

These are a means to insure that the vendor is capable of the following
responsible software engineering practices.

The TGDC requests that NIST one, review and analyze quality management
standards including ISO 9000 to determine the relevance to voting
systems, and two, recommend changes to the VSS Quality Assurance and
Configuration Management sections based on the findings above.

FEMALE SPEAKERL: Second.

DR. SMURGIAN: Thank you. We have a motion on the floor and a second.
Discussion? Yes, Mr. Burger.

MR. BURGER: This is one that I completely support the spirit of but I
have significant concern for the churn it may create in implementation.

If all that happens is ISO 9000 becomes required, I'm not sure that the
system can stand the cost of that. That's quite an undertaking.

I think I'd like to suggest an amendment to recommend that that analysis
particularly analyze the system and cost impact of requiring whatever
quality standards get put in place.

DR. SCHUTSER: So you would review and analyze the quality of management
standards including ISO 9000, determine their relevance to voting
systems and their impact on costs?

DR. BURGER: Yeah, after I say including ISO 9000, comma, with a
particular view to the cost and other impact to the quality system and
determine their relevance to the voting system.

DR. SCHUTSER: So you'd want to have a comma after voting system with
particular relevance \--

MR. BURGER: Yeah. When I've been through this before we typically put a
team of people in a fulltime job \--

DR. SCHUTSER: Okay, that's okay. I'm just trying to get the wording. It
would be comma with \--

**(CROSS TALK)**

MR. BURGER: Yeah. Comma with particular view to the cost and other
impact \--

DR. SCHUTSER: And other impacts.

MR. BURGER: Yes. And then close comma, to determine their relevance to
the voting system.

DR. SMURGIAN: Mr. Harding.

MR. HARDING: Thank you. While I don't object to our colleague's friendly
motion, but cost becomes an ambiguous term of where do you provide that
line and so I just think that word cost ought to come out. I'm not
suggesting what you're arguing is not valid but I think cost becomes a
very difficult line to define.

MR. BURGER: Yeah, what I'm trying to get at is this is an easy one. It's
like saying you're in favor of motherhood. It's easy to say yes to this
but when I've been involved in companies where we go through ISO
qualification, you assigned staff fulltime for a year or two just to
develop the documentation required and I'm not sure that there's that
kind of manpower out there. So that's the concern. I just want to make
sure that these requirements are levied with that intelligence involved.

DR. SMURGIAN: Mr. Kraft.

MR. KRAFT: Yeah, J.R., when we say costs here we're talking horrendous
costs. We're talking a cost that ultimately has to be borne by the
states and counties, which really could exceed the utility that they're
going to get out of this.

We're not talking small dollars or miserly looks at cost but whether
some very, very substantial big dollars would actually be appropriate
spending.

DR. SCHUTSER: Yeah, I think it's appropriate to put in at least that
concern.

DR. SMURGIAN: Ms. Quesenberry.

MS. QUESENBERRY: Just a clarification. Are you suggesting here that the
team will look at various existing quality management standards and
suggest one to be adopted that you might end up with the requirement
that any system vendor meet one or more existing standards, or are you
suggesting that those quality management standards be investigated as
source material for standards that we might write?

DR. SCHUTSER: Well I think particularly now that we're putting in
particular attention to cost impact, what probably will be an outcome is
that it would serving as a guideline and they'll be coming up with some
recommendations that would hopefully provide some of the needed quality
but not have as large an impact on the cost.

DR. SMURGIAN: It's not clear to me how NIST is going to address this
cost issue so Mark, would you like to make a comment on that?

MALE SPEAKER: Yeah, I had exactly the same thoughts that Dr. Smurgian
just expressed. Nowhere else were we asked to evaluate cost versus
benefit. Clearly everything has a cost and what worries me is we're
looking at the technical approach and whether to include something and
ask to evaluate the cost and not given a basis to make the final
determination as to whether to include how much is too much cost.
Clearly it's going to be expensive. I agree with everybody.

If we all feel that ISO 9000 is just going to be too expensive I suggest
just eliminating it from consideration, but to say we should look at it
and then make a cost benefit analysis, and then recommend changes to the
VSS based on its value versus the cost without being privy to all the
cost data of all the states is just something \-- I think that's not
something we at NIST should be doing.

DR. SCHUTSER: So you think it might be better to say review and analyze
quality management standards including ISO 9000 period, then say
recommend changes to VSS quality assurance configuration based on the
findings above, implying that you will use that information as an input
to provide specification?

MALE SPEAKER: Well I think someone perhaps is able to make a good cost
benefit analysis. All I'm saying is I don't think NIST has that
background and understands the cost of the states. So perhaps we can
make technical recommendations and someone much more qualified can look
at the cost and decide on whether that cost is worth the benefit.

MALE SPEAKER: Mark, would you like to have ISO 9000 deleted from this,
is that what you're saying?

MALE SPEAKER: That's better then the way it is now, yes. If it's between
that and the way it is now I'd rather see it deleted because I think
it's an open-ended question that we can't possibly answer.

DR. SCHUTSER: All right, so then you think it would be better to just
say, review and analyze quality management standards, determine their
relevance to voting systems and then recommend changes, blah, blah,
blah, based on the findings above?

MALE SPEAKER: Yes.

DR. SCHUTSER: So it would be just deleting, including ISO 9000 and not
worrying about the comment about cost impacts.

DR. SMURGIAN: Ms. Quesenbery.

MS. QUESENBERRY: I don't think I'm offering this as an amendment but
just a point for NIST. I think one of the things that would be very
interesting for me to hear as we consider whatever recommendation come
back would be something that assessed perhaps not the cost of conforming
to them because you can't necessarily know that but some indication of
the size and scale of the problem in terms of burden or numbers of
documents, or just something that would help us \-- some people are
obviously very familiar with quality standards, some are going to be
less familiar so that would be very helpful.

DR. SMURGIAN: Did you have a question, Dr. Williams? Just out of
curiosity, isn't this cost, the benefit ratio or any cost analysis going
to depend on the size of the community you're dealing with or the number
of customers let's say a particular vendor has? I mean in terms of per
unit cost, isn't that going to vary all over depending on those factors?

MALE SPEAKER: Not if the qualification \-- it's going to cost you the
same to qualify a system whether you've got one customer or a thousand.

**(CROSS TALK)**

DR. SMURGIAN: Yes, the absolute cost will not change but in terms of
cost impact, it will be very different depending on whether you're
dealing with a small \--

**(CROSS TALK)**

MS. QUESENBERRY: Well actually I would want to look at a very different
benefit ratio, not the cost to the vendor versus the number of units
they're selling but the benefit to the increasing quality of voting
systems versus the burden it might place.

MALE SPEAKER: Right. I think all those things are valid but I think what
we're hearing from NIST is that they don't feel equipped to do that.

MS. QUESENBERRY: But they can certainly quantify the burden and then let
us make our own decision about the relative merits of it.

DR. SMURGIAN: Mr. Burger.

MR. BURGER: Mark, I hear your comment and I'm not the expert on this but
I know there are federal requirements for regulatory impact statements
on rule makings. I'm not sure if this falls under that but that may not
be a requirement we can avoid in any case (unintelligible) to research.
And I would like to emphasize my comments are in no way meant to
eviscerate this from its intent. I think the intent is extremely
important.

DR. SMURGIAN: Okay, so far the only change we have is deleting,
including ISO 9000, is that correct?

MALE SPEAKER: Yes.

DR. SMURGIAN: And you're okay with that Dr. Schutser?

DR. SCHUTSER: I'm okay with that if everybody else is.

DR. SMURGIAN: Dr. Revest.

DR. REVEST: (Off microphone). I wanted to propose an amendment, which
was to add at the end of point one, grant (unintelligible) and their
security in parenthesis. There are many aspects of voting system quality
control which overlap (unintelligible) consideration one needs to take
with software development for purposes of evaluating the security.
Configuration management was already mentioned (unintelligible) kinds of
things one does. I think that that's certainly the kind of thing one
needs to care about when you're thinking about security issues and
development. So I just wanted to put that \--

DR. SMURGIAN: And their security characteristics or is security
sufficient?

DR. REVEST: (Off microphone).

DR. SMURGIAN: Dr. Schutser, do you \--

DR. SCHUTSER: That's fine with me.

DR. SMURGIAN: Is that acceptable to you?

DR. SCHUTSER: Yes.

DR. SMURGIAN: Okay, so we'll consider that a friendly amendment. Any
other comments or questions?

MALE SPEAKER: Call the question.

DR. SMURGIAN: Is there second? Thank you. Mr. Green will you take the
roll call on Resolution 30-05 with two friendly amendments, one deleting
the words including ISO 9000 from item one, and adding at the end of
item one in parenthesis (and their security).

MALE SPEAKER: (Off microphone). It's got the word procedure in there
which (unintelligible).

MALE SPEAKER: (Off microphone). Procedures (unintelligible).

MALE SPEAKER: (Off microphone). It's one of the versions
(unintelligible).

MALE SPEAKER: I don't think we had any of that in did we? Just the
relevance of voting systems.

MALE SPEAKER: (Off microphone). How did you propose the amendment on
(unintelligible)?

DR. SMURGIAN: And their security.

**(CROSS TALK)**

MALE SPEAKER: Procedures I think was something else we were writing \--

MS. QUESENBERRY: How about security implications?

MALE SPEAKER: (Off microphone). Voting systems on implications
(unintelligible).

MS. QUESENBERRY: Well standards have implications.

MALE SPEAKER: (Off microphone). Problems to voting systems
(unintelligible).

DR. SMURGIAN: Wait a minute. Are we adding properties or not?

DR. SCHUTSER: I don't think so.

MALE SPEAKER: No.

MS. QUESENBERRY: I'm not voting on that.

DR. SMURGIAN: Take out properties and just put a close paren there.
Leave that paren. Okay, I think we know what we want so let's proceed
with the roll call on Resolution 30-05.

MR. GREEN: Smurgian?

DR. SMURGIAN: Yes.

MR. GREEN: Smurgian votes yes. Davidson? Davidson not responding.
Miller?

MS. MILLER: Yes.

MR. GREEN: Miller votes yes. Turnerbuoy? Turnerbuoy not responding.
Purcell?

MS. PURCELL: Yes.

MR. GREEN: Purcell votes yes. Harding?

MR. HARDING: Yes.

MR. GREEN: Harding votes yes. Ellekese?

MR. ELLEKESE: Yes.

MR. GREEN: Ellekese votes yes. Caldise?

MS. CALDISE: Abstain.

MR. GREEN: Caldise abstains. Burger?

MR. BURGER: Yes.

MR. GREEN: Burger votes yes. Williams?

DR. WILLIAMS: Yes.

MR. GREEN: Williams votes yes. Kraft?

MR. KRAFT: Yes.

MR. GREEN: Kraft votes yes. Revest?

DR. REVEST: Yes.

MR. GREEN: Revest votes yes. Schutser?

DR. SCHUTSER: Yes.

MR. GREEN: Schutser votes yes. Gannon? Gannon not responding.
Quesenbery?

MS. QUESENBERRY: Yes.

MR. GREEN: Quesenberry votes yes. That's 11 votes for, three voters not
responding, one voter abstaining.

DR. SMURGIAN: Resolution 30-05 has been adopted. Dr. Schutser.

DR. SCHUTSER: Thank you. I'd like to now propose Resolution 31-05,
Maintenance of the VSS. All specifications contain ambiguities that are
discovered during testing of implementations. Similarly all
specifications contain requirements that can be subject to multiple
equally defensible interpretations.

The TGDC requests that NIST, I'd take out the colon in number one, draft
a strategy for maintenance of the VSS, which would address the issuance
of interpretations of the VSS, the resolution of disputes, and the
continuous improvement and revision of the VSS.

MALE SPEAKER: Second.

DR. SMURGIAN: Okay, we have a motion and a second. Any comments,
discussion?

MR. HARDING: Mr. Chairman.

DR. SMURGIAN: Yes, Mr. Harding.

MR. HARDING: The draft strategy for maintenance of the interpretations
and resolutions that disputes and so forth, what about a library or a
clearinghouse of that information because I would think people would
want to have access to it?

DR. SCHUTSER: That would be fine with me. Would you want to include
something like that, some wording?

MR. HARDING: Please, yes. Some kind of a clearinghouse of \--

DR. SCHUTSER: So we would say, comma, that would address the need for a
clearinghouse?

MR. HARDING: Correct.

MS. QUESENBERRY: Wouldn't it be public record? Wouldn't anything we did
to the VSS be public record in any event?

MR. HARDING: Yes, it would be but I guess I'm getting to the point of
what.

MALE SPEAKER: Well it's not so much there but knowing there's going to
be a need to disseminate that information from time to time and the need
to be able to say hey, I have a dispute, I need a body to rule on this
dispute.

DR. SMURGIAN: Dr. Williams.

DR. WILLIAMS: Let me talk about how it's done now, okay? Right now the
voting systems standards are administered by a committee of the National
Association of State Election Directors, referred to the as the Voting
System Committee and within that committee there's a subcommittee called
the Voting System Technical Committee.

And right now things like this go to that technical committee, which
reviews it, works with the vendor or the ITA, or whoever raises the
dispute or the question, recommends a resolution to the Voting System
Board and they act on. And then the appeals process \-- if the losers
don't like it they can appeal back to the Voting System Board and then
they can appeal to the full executive committee of NAVSET.

So I think that some structure like that is going to be necessary. Now
it's up to the EAC. Hopefully one day we're going to throw a big switch
and the EAC is going to be in charge of this process and when that day
comes they're going to have to set up some kind of a structure to
administer this and it will be that group then that will make their
rules about maintaining records, about their audit ability, what is the
challenge \-- you know what protocols and so forth.

So for us (unintelligible) to write these kinds of standards without
knowing how the EAC is going to structure this, it would be real
difficult.

MALE SPEAKER: So you would recommend leaving it without that?

DR. WILLIAMS: Yeah, I would leave this as a future thing that's got to
be addressed when the EAC begins to structure how they're going to
administer this process.

MALE SPEAKER: Is that okay with you?

MALE SPEAKER: It is, with the greater understanding of how it's
occurring now and how it might happen in the future, this then just is
one of those noted items to our four commissioners that this is a
clearinghouse resource vetting thing that they need to be ready to deal
with.

DR. SMURGIAN: Ms. Quesenberry.

MS. QUESENBERRY: Well while we're noting things then, I would say that
having listened to your description and thought more about Mr. Harding's
comment, that actually the notion of suggesting that there be a public
way of finding this material would be important.

For instance if you go now to the VSS 2002 you find that, but you don't
find the two NASET technical bulletins, which we had to find on the
NASET side. So bringing that together in an (unintelligible) system
would be useful.

DR. WILLIAMS: Yeah, there needs to be a more graceful way of finding it.
It's there, it's public information but, you know, let's face it until a
couple years ago nobody was interested in it. It wasn't a handful of us.
We could have had this meeting in a broom closet two years ago.

DR. SCHUTSER: In the next resolution we have something \-- it wasn't
intended to be quite that large but you might consider maybe amending it
that way, about sharing information.

MS. QUESENBERRY: The other point I would make and maybe this goes better
in your next resolution but I'll just say it here and you can decide, is
I totally agree with the process of continually monitoring the standards
and making sure that we're keeping them up-to-date and so.

I do think that we need to consider how to break a problem that we've
had raised here several times, which is the question of what do you do
with existing systems that have been qualified when you introduce
changes?

So I'd also somehow like to suggest whether this is just a point of
information that as whatever happens here happens, that we also think
about some structure for saying, and everybody has to come up to this
standard to remain certified within a certain period of time, or you
could be grandfathered in even if you're not, or whatever.

DR. WILLIAMS: I think that we've got if nothing else sort of a moral
obligation to develop a document like that for the EAC as we develop
these standards, to point out to them that, you know, we're sending this
thing your way and here's some of the things that you're going to have
to do and be prepared to deal with.

MALE SPEAKER: I think we discussed earlier that there was a need once we
developed the new specifications for discussing how to migrate that new
specification, so I think we could take that spirit along \-- how to
migrate the new one plus all the continuing improvements and we could
address that there.

DR. SMURGIAN: So far I have not heard any amendments. Let's take a vote
on Resolution 31-05 as presented.

MR. GREEN: Smurgian?

DR. SMURGIAN: Yes.

MR. GREEN: Smurgian votes yes? Davidson? Davidson not replying. Miller?

MS. MILLER: Yes.

MR. GREEN: Miller votes yes. Turnerbuoy? Turnerbuoy not replying.
Purcell?

MS. PURCELL: Yes.

MR. GREEN: Purcell votes yes. Harding?

MR. HARDING: Yes.

MR. GREEN: Harding votes yes. Ellekese?

MR. ELLEKESE: Yes.

MR. GREEN: Ellekese votes yes. Caldise?

MS. CALDISE: Abstain.

MR. GREEN: Caldise abstains. Burger?

MR. BURGER: Yes.

MR. GREEN: Burger votes yes. Williams?

DR. WILLIAMS: Yes.

MR. GREEN: Williams votes yes. Kraft?

MR. KRAFT: Yes.

MR. GREEN: Kraft votes yes. Revest?

DR. REVEST: Yes.

MR. GREEN: Revest votes yes. Schutser?

DR. SCHUTSER: Yes.

MR. GREEN: Schutser votes yes. Gannon? Gannon not replying. Quesenberry?

MS. QUESENBERRY: Yes.

MR. GREEN: Quesenberry votes yes. That's 11 votes in favor, one member
abstaining, three members not replying.

DR. SMURGIAN: Resolution 31-05 has been adopted. Dr. Schutser, would you
like to proceed to the next one?

DR. SCHUTSER: Yes, Mr. Chairman. I'd like to now present Resolution
32-05, titled Sharing Information and De-Qualification of Voting
Systems.

The TGDC recognizes that no conformance assessment process is prefect.
Systems with non-conformities even serious ones can be granted
qualification only to cause problems at the precinct level after they
are deployed. When a serious flaw is discovered in one jurisdiction,
other affected jurisdictions should be informed. At present however
there is no process to de-qualify voting systems that are discovered
after qualification has been granted, that have serious problems.

TGDC requests that NIST one, define a process as specification for
sharing information amongst jurisdictions concerning qualified voting
systems that have been discovered that have non-conformities, present
problems and known vulnerabilities, and two, develop a strategy for a
de-qualification process for voting systems.

DR. SMURGIAN: We have a motion. Second?

MALE SPEAKER: Second.

DR. SMURGIAN: If I could make a comment. This sounds like a good way for
NIST to make a lot of friends I guess. Yes, Dr. Williams.

DR. WILLIAMS: What this says is that we all agree that there ought to be
a bell on the cat and now the question is how do you put the bell on the
cat. Who's going to go do it?

We have de-qualified systems in the past and I'm here to tell you from
bitter experience that it is a bitter, bitter process. This is something
that is necessary but it's about as much fun as a root canal.

And the way this generally comes about or at least historically the way
this has come about is that the vendor will put strong pressure \-- a
jurisdiction will put strong pressure on a vendor to make a modification
to the system and the modification will result in the system being
non-compliant.

So we've got this situation where the jurisdiction now wants us to wink
at it and say okay it's a compliant system and here's the number. And
it's a really, really tough issue. I can't wait to see what Mark's going
to write about this.

MR. KRAFT: Mr. Chairman, if I may?

DR. SMURGIAN: Yes, Mr. Kraft.

MR. KRAFT: Well back to how we do it in Florida, we've got a different
legal structure. It is a very difficult process though.

We have to \-- when we find an anomaly like this we have to weigh the
risk that that presents to our future elections, when the next election
is scheduled, and the risk of trying to do system upgrades in a narrow
timeline, make sure that the county, their personnel, their training
resources, and vendor personnel are available to do a statewide upgrade.

And having done that exercise, getting the concurrence of the people
above me in the food chain, going out and de-certifying a system and
mandating that all counties upgrade to a new one, that is also very
difficult at a state level.

I'm not exactly sure how you're going to handle that problem at the
national level but I am pretty sure that that is more of a problem for
the EAC then for NIST and I'm not sure since it's business processed, it
is regulation more so then a standard. I'm not sure that's in the
purview of NIST.

So I would suggest as a friendly amendment that we delete item two. I
think NIST can be of value to finding the process and specifications of
information sharing and that possibly can go somewhat into the reporting
requirements life cycle that the ITAs are going to deal with but I'm not
sure that a strategy for de-qualification process is really within the
scope of what NIST does for a living.

DR. SMURGIAN: I'm sure Mark appreciates those comments.

MALE SPEAKER: You can be sure that if any of us were involved on one of
those committees I'm talking about and somebody came in and said we want
to de-qualify that system, we would say fine, we'll tell the
commissioners and just can't wait to see what they do about it because
that is a top level decision. That's not something that any of us want
to make.

MS. QUESENBERRY: Although I guess I would comment that if many of the
problems come from jurisdictions pressuring vendors to make changes it
would helpful if we could strengthen their backbone.

MALE SPEAKER: Yeah, we can do that. We can give lots of evidence, and
lots of support, and what have you but I'm agreeing with Paul that we
should not try to develop the process at this level. This should be in
that notebook we're going to prepare for the AC that we're going \--
there are going to be times when we're going to recommend to you that
you de-qualify a system so you need to be thinking about how you're
going to do that.

DR. SMURGIAN: Dr. Revest.

DR. REVEST: Yeah, the whole point if I understand Brit properly is that
we want to make it clear that de-qualification is envisioned and that
there will be some process for doing it, that it's not the case that a
system once it's qualified is qualified for ever but that there will be
some process for doing this that the commissioners may have to work out
on a case-by-case basis or whatever.

But there needs to be an understanding that systems may be re-reviewed
in light of changes of recommendations, changes of standards, or if
vulnerability is discovered, and that \-- you know, whether NIST
develops a strategy or the commissioners handle it on a line by line,
item by item basis.

MALE SPEAKER: (Off microphone). That first item that we're leaving in is
an important one. There needs to be a mechanism where \--

DR. SMURGIAN: Could you turn on your speaker, please?

MALE SPEAKER: Sure. You might discover a non-conformity that's not
serious enough that you would go to the extreme of de-qualifying the
system but you still need to notify everyone who's using that system
about that non-conformance so they can take whatever action they feel is
appropriate.

DR. REVEST: Can I make another comment?

DR. SMURGIAN: Yes.

DR. REVEST: This is addressing the first point. There's a fine line
between having a system which talks about vulnerabilities and having an
incident reporting system so \-- when you have systems where you have
repeated problems of some sort, whether they're reliability problems or
some security issue that comes up frequently.

One of the resolutions that we discussed to some extent within the
Security Committee was having an incident reporting system that would
allow information not only about vulnerabilities in some level of
prescription but also describe the specific times that we had problems
with these issues, and I'd like I guess the committee to consider
whether that might be advisable to introduce an amendment to address
that as well here.

MR. KRAFT: If I may?

DR. SMURGIAN: Mr. Kraft.

MR. KRAFT: One of the problems with the incident reporting system and
evaluating this information that's coming through is it's not all that
clear just what the problem is and most frequently \-- I mean I will
hear horror stories and news clips out of other states where someone is
complaining of a problem with a voting system and you read the report
and you say no, that's actually a problem with election administration.

And probably within the EAC there's going to have to be either a board
or some individuals with good election administration experience who can
make those calls, whether these incidents that are reported reflect
perhaps on an improvement that needs to be made to Best Practices, or
point to an election official who basically didn't follow Best
Practices, or whether a particular incident actually relates to an
anomaly in the system, which either needs to be distributed for
everyone's information or might lead to revocation.

But when you start getting into those things and I get to see a lot of
them from around the country, it's very difficult to determine what's
what.

DR. SMURGIAN: Okay, let me just make sure that we're on target. So far
what I have heard is we have a proposed friendly amendment to take out
item number two. Is that acceptable to you, Dr. Schutser?

DR. SCHUTSER: That's acceptable, yes.

DR. SMURGIAN: So therefore we take out item number two, and take out the
number one and make that part of the text.

DR. SCHUTSER: And take out the end \-- at the end.

DR. SMURGIAN: And do we take out the qualification after the title? I
thought that was \--

DR. SCHUTSER: I didn't think so, no.

**(CROSS TALK)**

DR. SCHUTSER: I think we're still addressing sharing information through
qualification. We're just not doing a strategy. We're talking about \--

DR. SMURGIAN: Yeah, because we're still talking about how there is no
process to de-qualify et cetera, so I think the issue was to define a
process but not to get into the logistics, the strategy of how to go
about doing that because that may be very much dependent on the
particular jurisdiction, particular local issues basically.

MALE SPEAKER: It's a circumstantial thing and what we would do is
document non-conformities and then pass that on to the commissioners and
say here are the non-conformities. And then they get those big bucks to
deal with the political implications of that and what action they want
to take on it.

DR. SMURGIAN: Okay, so basically the change we've made is simply take
out item number two and then edit to make the transition for item number
one. So do I hear a move? Do I have a second?

MALE SPEAKER: Second.

DR. SMURGIAN: Thank you. Mr. Green, will you take a roll call please on
Resolution 32-05 as amended where we've deleted item number two?

MR. GREEN: Smurgian?

DR. SMURGIAN: Yes.

MR. GREEN: Smurgian votes yes. Davidson? Davidson not replying. Miller?

MS. MILLER: Yes.

MR. GREEN: Miller votes yes. Turnerbuoy?

MS. TURNERBUOY: I abstain. I have just returned to the conference call.

MR. GREEN: Turnerbuoy abstains. Purcell?

MS. PURCELL: Yes.

MR. GREEN: Purcell votes yes. Harding? Harding has departed. Ellekese?

MR. ELLEKESE: Yes.

MR. GREEN: Ellekese votes yes. Caldise?

MS. CALDISE: Abstain.

MR. GREEN: Caldise abstains. Burger?

MR. BURGER: Yes.

MR. GREEN: Burger votes yes. Williams?

DR. WILLIAMS: Yes.

MR. GREEN: Williams votes yes. Kraft?

MR. KRAFT: Yes.

MR. KRAFT: Kraft votes yes. Revest?

DR. REVEST: Yes.

MR. GREEN: Revest votes yes. Schutser?

DR. SCHUTSER: Yes.

MR. GREEN: Schutser votes yes. Gannon? Gannon not responding.
Quesenberry?

MS. QUESENBERRY: Yes.

MR. GREEN: Quesenberry votes yes. That's ten voting for, three not
replying, two abstaining.

DR. SMURGIAN: Thank you. Resolution 32-05 has been adopted. Dr.
Schutser, would you like to proceed?

DR. SCHUTSER: Yes, Mr. Chairman. I'd like to propose Resolution 33-05,
Glossary and Voting Model. The 2002 VSS does not contain a voting model
depicting the entire voting process. The current glossary of terms needs
revision.

The TGDC requests that NIST, I'd take away the colon and number, update
the 2002 VSS glossary of terms and develop a voting process model that
incorporates terminology from the revised glossary to clearly depict the
entire voting process and determine where a voting system fits into this
large process model.

MALE SPEAKER: Second.

DR. SMURGIAN: We have a motion and a second. Comments or discussion? Dr.
Williams.

DR. WILLIAMS: (Off microphone). I would suggest that we put a period at
the end of (unintelligible) and make item number two (unintelligible)
process model (unintelligible) clarity purposes (unintelligible) instead
of one.

DR. SMURGIAN: So the first item would read update the 2002 VSS glossary
of terms; semi-colon, and item two, develop a voting process model et
cetera. Is that acceptable to you, Dr. Schutser?

DR. SCHUTSER: Yes, it is.

DR. SMURGIAN: Any other questions or comments?

MALE SPEAKER: Second.

DR. SMURGIAN: Okay, we'll take a roll call on Resolution 33-05 as
amended where item one has been broken down into two separate items. Mr.
Green, will you take a roll call?

MR. GREEN: Smurgian?

DR. SMURGIAN: Yes.

MR. GREEN: Smurgian votes yes. Davidson? Davidson not replying. Miller?

MS. MILLER: Yes.

MR. GREEN: Miller votes yes. Turnerbuoy?

MS. TURNERBUOY: Yes.

MR. GREEN: Turnerbuoy votes yes. Purcell?

MS. PURCELL: Yes.

MR. GREEN: Purcell votes yes. Harding? Harding's absent. Ellekese?

MR. ELLEKESE: Yes.

MR. GREEN: Ellekese votes yes. Caldise?

MS. CALDISE: Abstain.

MR. GREEN: Caldise abstains. Burger?

MR. BURGER: Yes.

MR. GREEN: Burger votes yes. Williams?

DR. WILLIAMS: Yes.

MR. GREEN: Williams votes yes. Kraft?

MR. KRAFT: Yes.

MR. GREEN: Kraft votes yes. Revest?

DR. REVEST: Yes.

MR. GREEN: Revest votes yes. Schutser?

DR. SCHUTSER: Yes.

MR. GREEN: Schutser votes yes. Gannon? Gannon not replying. Quesenberry?

MS. QUESENBERRY: Yes.

MR. GREEN: Quesenberry votes yes. Eleven votes for, three members not
voting, one member abstaining.

DR. SMURGIAN: Resolution 33-05 has been adopted. Dr. Schutser, would you
like to proceed with your last resolution?

DR. SCHUTSER: Yes, Mr. Chairman. Resolution 34-05. Title is Assessment
Papers on Recommendations for Future Work. Separate from the immediate
work effort to abate the 2002 VSS specifications.

The TGDC recognizes the need to develop a series of assessment papers
that address important issues related to the interrelation of election
management and VSS systems. These issues are likely to lead to future
specifications for VSS systems.

The TGDC requests that NIST develop the assessment papers that discuss
the need for:

1\) Standards and tests to support future systems built to support
election day verification of voters.

2\) Standards and formatting of registration information possibly using
extensible markup language XML to make it easer for states to share
information.

3\) Testing standard to validate compensating process, procedures, and
fixes and address known real test deficiencies.

4\) Better ways to integrate the voting registration process with the
rest of the voting process.

5\) Standards and tests to support systems that implement absentee
voting.

6\) Standards and tests to support systems that implement multi-day
voting.

7\) Standards specifying what existing election information format
standards (or portions thereof or variations thereof) are acceptable for
use in voting systems.

8\) Standards supporting voter interactions and issues are correctly
capturing voter intent. Leave out the and at the end.

MALE SPEAKER: Second.

DR. SMURGIAN: We have a motion and a second. Discussion? Mr. Kraft.

MR. KRAFT: Yes. I think a number of the items here are either really out
of scope for the Technical Guidelines Committee or close to being out
scope. However that said, I'm going to support the resolution because I
think there is a tremendous amount of future research that needs to be
done in election administration.

DR. SCHUTSER: Thank you.

MR. ELLEKESE: Mr. Chairman, Jim Ellekese.

DR. SMURGIAN: Yes, please go ahead. Identify yourself.

MR. ELLEKESE: Jim Ellekese from the Access Board. It's small matter but
number ten, which was just read, do we need to edit voter intent to
voter choice?

DR. SCHUTSER: Yes, we forgot to edit that one. It shouldn't say voter
intent it should say voter choice.

MALE SPEAKER: Good catch.

DR. SCHUTSER: Thank you.

DR. SMURGIAN: Thank you. Any other questions or comments?

MALE SPEAKER: Move the question.

FEMALE SPEAKER: Second.

MALE SPEAKER: (Off microphone). In the grand scheme of things between
now and March this should not \--

**(CROSS TALK)**

DR. SMURGIAN: This is definitely future.

DR. SCHUTSER: That's understood.

DR. SMURGIAN: Yeah, I assume that the committee acted with that in mind.

DR. SCHUTSER: It's post April.

DR. SMURGIAN: Right. Ms. Purcell.

MS. PURCELL: Yes, I would just like to make the comment that I'm
assuming that number five and number six, either one or both of those
would include early or mail in ballots.

DR. SCHUTSER: Yes.

DR. SMURGIAN: I believe we've moved the question, a second, so we will
be voting on Resolution 34-05. The only modification being the voter
intent will be placed with indication of \--

MS. QUESENBERRY: (Off microphone). Actually at least with my copy, is
there a number nine?

**(CROSS TALK)**

DR. SMURGIAN: It looks like we're maybe missing a page. There is a
number nine but it's not in our hard copy.

DR. SCHUTSER: It wasn't in the hard copy. (Unintelligible) is supporting
the interrelationship of polling place operation and usability,
accessibility, and policy.

FEMALE SPEAKER: That's privacy.

DR. SCHUTSER: Priority.

DR. SMURGIAN: Privacy.

**(CROSS TALK)**

DR. SCHUTSER: Privacy.

MALE SPEAKER: It was in the e-mail distribution.

DR. SCHUTSER: Yeah. Shall we leave that in? I just didn't see it on the
piece of paper, sorry.

DR. SMURGIAN: Yeah, there is a number nine. Okay, with that we'll take a
roll call. Mr. Green.

MR. GREEN: This is 34-05 as amended. Smurgian?

DR. SMURGIAN: Yes.

MR. GREEN: Smurgain votes yes. Davidson? Davidson not replying. Miller?

MS. MILLER: Miller votes yes. Turnerbuoy?

MS. TURNERBUOY: Yes.

MR. GREEN: Turnerbuoy votes yes. Purcell?

MR. GREEN: Purcell votes yes. Harding has departed. Ellekese?

MR. ELLEKESE: Yes.

MR. GREEN: Ellekese votes yes. Caldise?

MS. CALDISE: Abstain.

MS. CALDISE: Caldise abstains. Burger?

MR. BURGER: Yes.

MR. GREEN: Burger votes. Williams?

DR. WILLIAMS: Yes.

MR. GREEN: Williams votes yes. Kraft?

MR. KRAFT: Yes.

MR. GREEN: Kraft votes yes. Revest?

DR. REVEST: Yes.

MR. GREEN: Revest votes yes. Schutser?

DR. SCHUTSER: Yes.

MR. GREEN: Schutser votes yes. Gannon? Gannon not replying. Quesenberry?

MS. QUESENBERRY: Yes.

MR. GREEN: Quesenberry votes yes. That's 11 members voting for, three
members not voting, one member abstaining.

DR. SMURGIAN: Resolution 34-05 has been adopted. Thank you Dr. Schutser
for a great job in getting through these very quickly.

That actually leaves us some time to go back to the resolutions that we
did not address for Security and Transparency Subcommittee. Dr. Revest,
how many did we have left?

DR. REVEST: We have three left.

DR. SMURGIAN: Do you want to take these up at this time?

DR. REVEST: (Off microphone). Unintelligible. So I'd like to take up
number 15, Software Distribution now and we'll talk about the others \--

**(END OF TAPE 2, START OF TAPE 3)**

DR. REVEST: Software distribution, I guess I could read it. The TGDC has
concluded that generally speaking the manner in which software is loaded
onto voting systems, is not governed by existing standards and that it
is a significant security issue that warrants more stringent controls.

It is important to know which software has been installed on a voting
system, when the software has been installed and from what sources.
Without strict controls on these procedures, non certified software
could be loaded onto voting systems with potentially disastrous results.
The TGDC directs NIST to research and draft standards, documents
requiring:

1\) The distribution of any software to voting systems should only be
performed by means of physically distributed, read-only, and here I
would like to insert, or write-once before the word media, so it's
write-once media, including software such as a) operating system
required software, b) updates and patches, c) data files, and d) voting
system software.

2\) The electronic transmission of any software to voting machines, be
it networks or wireless be prohibited.

3\) That the software will include an integrity check such as a digital
signature that positively authenticates its source, that must be
verified as part of the process of loading the software.

4\) That the record of loading the software will be written permanently
to the system auto log, kept in write-once memory.

DR. SMURGIAN: We have a motion, do I have a second?

DR. WILLIAMS: Second and I have a question.

DR. SMURGIAN: Yes, Dr. Williams.

(INTERUPTION IN THE RECORDING)

MR. KRAFT: I do have a problem with two, particularly envisioning a
world where we are going to have good integrity check validation tools
available. I really don't care how they got the code onto the system as
long as they make a very positive absolute validation that it is the
code that they intended to load. So I would suggest a friendly amendment
of striking item two and with that I think at this point I could support
the rest of the resolution.

DR. WILLIAMS: You know also Ron, if you look at item one, when you say
only media such as the following, that's precluding via wireless. My
concern is that, that item \--

DR. REVEST: If it's redundant with one \--

DR. WILLIAMS: \-- Is a lightening rod that attracts attention and it's
applied by one, when you say that the only way to do it is the
following.

MR. KRAFT: And a lot of the configurations in systems in warehouses are
going to be done on small closed networks. You basically can't sneak a
NET7000 machine.

DR. REVEST: Well then you are arguing against one as well?

MR. KRAFT: I am reading one to mean that the vendor has to give the
customer, yeah a CD-rom which we require anyway. But as far as getting
how they actually put that onto the voting machines, I really have
problems with that, particularly if we're gong to require validation.

DR. REVEST: Let me remark that I consider the validation here, you know,
\-- the number three, helpful but not definitive in the sense that
usually when you have those kinds of validation, you are making the
assumption that the verifier is already \-- that's what you want, you
know that it hasn't been modified and so on to with the potentially
maliciously modification or erroneous modification. You might have the
part of the software that checks the signatures also being corrupted.
You would have controls about which software gets updated.

DR. SMURGIAN: Dr. Revest \--

DR. REVEST: So I guess to respond to the, if I may quickly, there was a
proposal for a friendly modification, I don't consider it friendly. It's
a modification.

DR. WILLIAMS: Let me describe a situation to you and you tell me what's
wrong with this picture. In our election we have a house, and there is
noting in it but our election center. And we're part of a university so
coming into that house is internet connections. Now within that house,
we have a hard wired intronet.

So if you go into some of our people's offices, you will see three
colored plugs, one of them is the telephone, one is them is the
internet, the other is our intronet, which is hard wired and does not go
outside that house. We use that intronet as a secure network. Are you
saying we shouldn't be doing that?

DR. REVEST: So this is for voting systems in the \--

DR. WILLIAMS: When we're developing ballots and ballot styles and doing
all sorts of work on the voting system. It's like I got a computer here,
you have got a computer there, there is a piece of wire that runs from
here to there, is that a vulnerable network.

DR. REVEST: It may be depending on what risks you are considering. It
may be broadcasting \--

DR. WILLIAMS: It's not. I'm telling you it's a piece of wire that we
installed. It goes to that point, to that point to that point and there
are no access points on that piece of wire.

DR. REVEST: None that you know of.

DR. WILLIAMS: I give up.

DR. SMURGIAN: Dr. Revest, may I suggest something? Would it be more
acceptable instead of saying prohibited, is strongly discouraged or
something like that so that it leaves some room for special cases like
this, for example?

MR. KRAFT: How about should be introduced as special risk, and should be
approached with extreme caution.

DR. REVEST: Yeah, this is the line with your earlier \--

MR. KRAFT: Same thing that we did with the earlier \-- where they
restricted standard.

DR. SMURGIAN: Would you consider that a friendly amendment?

DR. REVEST: I am not sure where it is intended to be put on this.

DR. SMURGIAN: I think it is at item two, instead of saying prohibited,
what was the language we used before.

MR. KRAFT: I would say introduces \--

MALE SPEAKER: Severe risk and should be approached with extreme caution.

MR. KRAFT: Okay.

DR. REVEST: There is a presumption that systems that work this way are
going to be acceptable in the end. I mean it might be better just to let
NIST investigate this a bit further and see if they view that as
workable. I think the \--

DR. SMURGIAN: But this does not \-- the way it's worded this does not
leave much judgment for NIST.

DR. REVEST: No I understand. I was proposing a replacement for that.

MR. KRAFT: You know Ron I shouldn't lose my tempers on boards like this
but confound it, when Brett described his scenario of two machines and a
wire running between them, you asked him how he knee there was nobody
else on the wire, this is being done by amateurs.

This is being done by IT professionals who actually are aware of
security issues, who do have some professional training. Some of the
advanced degrees and who take a great deal of care to make sure that
their networks are secure. And I wish you would give the profession a
little credit for having some common sense.

MALE SPEAKER: (Off microphone).

DR.SMURGIAN: Could you turn on your microphone?

DR. WILLIAMS: I said in the south we fall under the New York Times
definition of not being real computer scientist.

MALE SPEAKER: Yeah, you have your PhD but you're from Georgia.

MS. PURCELL: Mr. Chairman?

DR. SMURGIAN: Yes, Ms. Purcell.

MS. PURCELL: I might mention that we have the same situation that exists
as Dr. Williams just eluded to. We have also an intranet set up for our
ballot design, transmitting that ballot design and so forth.

DR.SMURGIAN: Dr. Revest what change \--

DR. REVEST: Well I think the sense of the committee is probably most
consistent with what I am hearing \-- most consistent with what Paul
earlier suggested which is that there be extreme caution used. So I
think there is a specific amendment there but I think that if you wanted
to restate that I would consider that again for a \--

DR. SMURGIAN: I think earlier \--

DR. REVEST: I think it's on the table isn't it?

MALE SPEAKER: (Off microphone).

DR. REVEST: Is that on the table?

MR. KRAFT: Yes.

DR. SMURGIAN: Is that \--

DR. REVEST: I am trying to read it \--

DR. SMURGIAN: That \-- basically this is item two that the electronic
transmission of software to voting machines via networks or wireless
introduces extreme risk and should be approached with extreme caution.
Is that acceptable?

DR. REVEST: Let's proceed with that.

DR. SMURGIAN: Okay. Any other comments? So we are voting on Resolution
15-05 with a couple of modifications. One is under item one, second line
with means of physically distributed read-only and added or write-once
media. The rest of it being the same.

And item number two, the word prohibited at the end of that item was
taken out and replaced with \-- I guess we're taking out is prohibited
and replacing with introduces extreme risk and should be approached with
extreme caution.

DR. REVEST: Should that perhaps be shall instead of should.

MR. KRAFT: No it should be should.

DR. REVEST: It should be should.

DR. SMURGIAN: Item three and four are as originally read. Mr. Green
would you take the role call?

MR. GREEN: 15-05 as amended. Smurgian?

DR. SMURGIAN: Yes.

MR. GREEN: Smurgian votes yes. Davidson? Davidson not replying. Miller?

MS. MILLER: Yes.

MR. GREEN: Miller votes yes. Turnerbuoy?

MS. TURNERBUOY: Yes.

MR. GREEN: Turnerbuoy votes yes. Purcell?

MS. PURCELL: Yes.

MR. GREEN: Purcell votes yes. Harding? Harding has left. Ellekese?

MR. ELLEKESE: Yes.

MR. GREEN: Ellekese votes yes. Caldise?

MS. CALDISE: Abstain.

MR. GREEN: Caldise abstains. Burger?

MR. BURGER: Yes.

MR. GREEN: Burger votes yes. Williams?

DR. WILLIAMS: Yes.

MR. GREEN: Williams votes yes. Kraft?

MR. KRAFT: Yes.

MR. GREEN: Kraft votes yes. Revest?

DR. REVEST: Yes.

MR. GREEN: Revest votes yes. Schutser?

DR. SMURGIAN: Not here.

MR. GREEN: I'm sorry Dr. Schutser has departed. Gannon? Gannon is not
replying. Queseneberry?

MS. QUESENBERRY: Yes.

MR. GREEN: Quesenberry votes yes. That's 10 votes for, four members not
voting, one member abstaining.

DR. SMURGIAN: Resolution 15-05 has been adopted as modified. Dr. Revest
do you want to postpone the others or do you want to go forward.

DR. REVEST: Let's see we have a few minutes. What I would like to do
would be \-- there are two more left. One number 19 on transparent
evaluation. One number 20 on machine readability. The one on machine
readability I think we can pass discussion over with the understanding
that it's already more or less included with the multiple
representations resolution.

Resolution 19, what I would like to propose to do with the agreement of
the chairman would be just to describe briefly what this resolution as
drafted is about. One of the problems we faced in trying to draft it
this way, and then to seek advice briefly about redrafting it, bringing
it back in February because I think it could be much better worded with
some of the styles that I have seen already in some of the other
resolutions here. Do I have your permission to proceed?

DR. SMURGIAN: Yes, please go ahead.

DR. REVEST: This Resolution 19 is attempt to address the issue of
publication of information regarding voting systems, not only the
information that would be in a public information package as earlier
proposed by the CRT committee but also information such as source codes
that's sensitive and maybe contains trade secrets.

And I think we need to have a process by which information containing
trade secrets may be made available to state election directors and
their designees in some controlled manner for their evaluation. To some
extent this is apparently already being done in practice and so this
would be a reflection of current practice.

This resolution was drafted in a somewhat contorted way admittedly as a
trade between more publication versus easier evaluation based on a
perception that the charter of the TGDC here, did not permit us to
approve resolutions that basically said this should be published or not,
or made available in a certain way. That it was beyond our charter.

So given the resolutions that have already been passed, I take it that
that presumption was incorrect and I would be happy to go back and
redraft this amendment more in line with some of the resolutions that
were passed earlier, that talked about making information public in a
more straight forward way.

DR. SMURGIAN: Thank you. Any comments?

DR. WILLIAMS: I would make a suggestion that where you say interested
and qualified parties \-- distributed to interested and qualified
parties, I would add the phrase on a need to know basis. Just because
somebody is interested \-- you know, this is an issue that we discussed
at great length and I think you will find that the consensus of the
people within the election community and I am not speaking as a security
expert now, I am speaking as an elections person.

The people within the elections community are uncomfortable with the
idea of just open source code, that goes everywhere, that anybody might
be interested.

DR. REVEST: Yeah, I am very sensitive to that and I think some
reasonable process needs to be put in place. The kind of thing I had in
mind, if you wanted to comment on that would be the kind of \-- and NIST
could maybe come up with better approaches but one where state election
directors themselves, federal election officials and their designees \--

DR. WILLIAMS: Well as a representative of the elections office of the
state of Georgia I have to tell you that in 18 years I have yet to have
a vendor deny my anything I have asked for. It's real simple, I say if
you want to do business with the state of Georgia, here is what you have
to provide.

DR. REVEST: If this turns out to be a resolution which merely states
what is common practice, it may help in giving people confidence that
the process is more open then they realized.

FEMALE SPEAKER: Yeah, I would just like to suggest that as we kind of
eluded to before that this goes hand and glove with Resolution 28-05,
public available qualification data. And with that work moving forward,
that when that work move forward, this should be wrapped into it and
they should come together as a single \--

DR. REVEST: I think that is good advice. I can talk to Dan about how to
coordinate that.

MALE SPEAKER: (Off microphone).

DR. SMURGIAN: Okay so then we will \-- you will withdraw this for
further consideration?

DR. REVEST: I think that would be best given the context of how \-- my
understanding of what's permissible in terms of these resolutions.

MR. KRAFT: Mr. Chairman may I put in another comment?

DR. SMURGIAN: Yes, Mr. Kraft.

MR. KRAFT: And Ron I think there remains a real good question as to what
do we do to provide interested parties with assurance that we have done
a competent job of evaluating some of this trade secret information. And
I don't know what a good solution is for that.

One of the things that we discussed at one point was perhaps a public
event where while not disclosing all the source code for a system, the
analyst who did the evaluation would perhaps put up sections of the code
showing the kinds of things that he found that supported his opinion and
asked questions about \-- or answered questions about the examination
that he did that I think provides some public assurance if any shows up
that in fact the work was done in a competent manner by a real person,
while not really necessarily laying out their \-- the vendors trade
secrets.

DR. REVEST: Interesting idea.

DR. SMURGIAN: Any other comments? Could you excuse me just for a minute?
All right thank you Dr. Revest. I think this concludes consideration of
some 35 resolutions and I think you all need to be congratulated for a
terrific job.

I would like to thank all the subcommittee chairs, sub committee members
and the staff from NIST to get prepared for this meeting and get so much
done in a very short period of time. I guess we have two options. Let me
present them.

I think I know which way we're going to go but it's snowing out there
first of all. We need to set a prior \-- to a prioritization of these
\-- the work to be done in response to these resolutions. We could
either take a break and then come back or we can simply go ahead, finish
the job or the prioritization and then you can leave as soon as we're
done with that. That will be I guess additional incentive to work
faster.

Does anybody object to simply going on and doing the prioritization or
do we really need a break.

MR. KRAFT: No we need to go home early.

DR. SMURGIAN: Okay. My proposal, see how you feel about this. I don't
think it will be terribly useful to go down the list 1-35 and trying to
you know, put in to an order. Probably a more sort of expeditious way of
doing this prioritization is identify sort of the lower priority, more
future oriented activities that will be done you know, over the next
couple of years perhaps.

Things that really need to be addressed right now and come back with
word processes, you know, for the April session, as our top priority.
And then sort of an in between group that would could do as much as
possible.

DR. SMURGIAN: Yes, Ms. Quesenberry.

MS. QUESENBERRY: If I can we have already addressed this issue somewhat
in our committee as we considered these resolutions. We also tried to
consider the workload we were imposing on ourselves or I was imposing on
Sharon and her staff.

And what we could do quite quickly is just put up a slide that shows
dates \-- you know, which we think are part of April, which we think are
later and people can take a quick look at it. But I think what we might
want to do is go over it now and let everybody think about and then try
to do this online. Would that be possible?

DR. SMURGIAN: Well that's \-- your subcommittee seems to be quite well
prepared but are the other subcommittees in a position to do that? I
know Dr. Schutser had to leave. Do we have \-- I think Mr. Burger and
Mr. Kraft, you were serving on the \--

MR. KRAFT: I was on the CRT.

DR. SMURGIAN: CRT right.

MS. QUESENBERRY: I mean I guess my take would be that I don't \-- within
each committee we have different amounts of resources to bring to bear
and that we probably want to \--

DR. SMURGIAN: So you want to prioritize within each subcommittees work?

MS. QUESENBERRY: And in some cases I think the NIST staff is going to
need to go back and take a serious look at what's happened to these
resolutions because they have been substantially changed. So perhaps if
every committee sort of laid out \-- what did we do, April, August and
November or April, August and beyond, and we could just look at them and
make sure that as a committee we agree on that general prioritization.

MALE SPEAKER: (Off microphone).

DR. SMURGIAN: Okay before we go on I believe Carol are you going to \--
Carol Burket from EAC would like to make a statement.

MS. BURKET: Mr. Chairman, and committee thank you very much for this
opportunity. The vice-chairman and Commissioner Martinez regretted that
they had to leave early but they did ask that if it was appropriate
within the proceedings of this meeting that I speak on their behalf and
on the behalf and on the behalf of the commission in terms of the topics
in which the commission feels a strong sense of urgency to proceed. And
we would certainly focus first and foremost on the security topics.

As you are all aware the 2002 standards are perhaps most efficient in
that area then in any other area with all due respect to the
accessibility issues but this is the area where we believe that the
states need the most immediate guidance. So that would be the statement
of the commissions' priority regarding their work and we offer that for
your consideration. Thank you.

DR. SMURGIAN: Thank you. Would you like to proceed Ms. Quesenberry?

MS. QUESENBERRY: No. I think I was suggesting that since \-- all the
committees might need some time to regroup and perhaps we could do this
by email. If each committee put together their own prioritization and
then we reviewed them.

DR. SMURGIAN: Would there be some benefit at least to sort of see where
each committee is coming from.

DR. WILLIAMS: I don't have a problem finalizing it that way but I think
while we have got this group here, we need to at least make some high
level statements about the process and then we can refine it and
finalize via email and all.

DR. SMURGIAN: Yeah, I agree. I think we don't need to finalize it but at
least everybody ought to see what the various committees are struggling
with, what the workload will be and then given that \-- being aware of
the other requirements then we can finalize it on line. So with that
would you like to share with us MS. Quesenberry?

MS. QUESENBERRY: Do you have the slide?

DR. SMURGIAN: Well \--

MS. QUESENBERRY: It's going to take me a moment to boot my computer
then.

DR. SMURGIAN: I mean do we have the order? We have the items up here for
each \--

MS. QUESENBERRY: I'm sorry I just get \-- why don't we move on to
someone else. I'll reboot my computer and bring it up.

MR. KRAFT: Yeah, I can take a turn.

DR. SMURGIAN: Okay let's hear from Core Requirements and Testing group
\-- subcommittee.

MR. KRAFT: Basically my thoughts are linked to particular resolutions
that passed but at three most critical tasks that have to be performed
before January 1, 2006. We must absolutely have motion forward through
NAVLAP for the accreditation of the ITAs. And the resolutions that would
support that, need to become a high priority.

The second most critical task moving forward, with the work with the
National Software Reference Library. I am happy to say that we have a
conference call set up on that next week and at least a very high level
plan for moving forward with a pilot project on that.

The third most critical thing is taking the prior art and the 2002 FEC
voting systems standards, including the new requirements of the Help
America Vote Act, particularly the accessibility items, and republishing
the revised standard as soon as possible. And I think getting a draft
out in about 30 days would be really good.

So all the resolutions that tie to those three items, I think are
extremely high priority.

DR. SMUGIAN: Dr. Williams?

DR. WILLIAMS: Yeah, I agree with Paul on the lab accreditation issue and
on the software reference library issue. There is one issue that the
commissioners are really getting pelted on and we need to provide some
guidance and that's this issue of voter verifiable paper receipts,
ballots, whatever you want to call them, coming off of DRE voting
machines.

As a result of all of the media attention on this item, and I'll leave
it to individuals to decide for themselves how valid some of that media
attention is, but it has created an error of uncertainty among the
states. And as a result a number of states and jurisdictions have passed
laws and adopted resolutions requiring paper receipts off of their DRE
machines.

The current standard is completely silent on this. We give people no
guidance whatsoever in this are and it's desperately needed. The
specifics are contained in I think, some of the revision of your voter
verification number two that we rejected. But \-- although we didn't put
those down as a requirement, they are definitely an optional feature and
we definitely have states and jurisdictions that want to do that. So
that has got to bubble up to the very top. That's consistent with what
we're talking about.

If you look at the 2002 standard, most of the people on that committee
had a lot of experience in elections. So that 2002 standard is a very
good description of what a functional election system has got to be.
When you are looking at where it's describing the functions of an
election system it's very good. And so what we need to direct to do, is
direct our attention toward the holes in it. We don't need to sit down
and try to redefine what a voting system is. We know what a voting
system is.

But we need to address the security issues and I think the top of that
one is, is the ability to validate the installation of the software to
track the software \-- the genealogy of the software from its source to
its destination and at each stage validate that it hasn't changed. And
the software library is a big step in that but that needs to be
addressed.

The standards for paper receipts need to be addressed and the getting on
with the business of accrediting laboratories and getting that activity
going, needs to be addressed. Those are what I see as the big issues
that need attention quickly.

DR. SMURGIAN: Sounds like Dr. Revest your priorities are set for you.
Before we go there, let me just make sure that \-- see if Mr. Burger has
any additional comments with regard to \--

MR. BURGER: No my thoughts would be very similar and all go back to the
discussion we had yesterday about an analysis showing the pluses,
strengths and weaknesses of various approaches. And I think when you are
talking about paper, I think it is very important to get it down as to
\-- when you are pure electronic, here are your vulnerabilities and
risks, particularly in security but when you go to paper all that
happens is a shift and you have another set primarily in the system
arena of vulnerabilities and risks that need to be mitigated and
addressed.

DR. WILLIAMS: We need some implementation guidelines too because just as
a for instance we are all familiar with Nevada and that system that was
developed out there. And that system will not pass qualification under
the 2002 standards because it does not protect the secrecy of the
ballot.

So here you have got a system that's developed and deployed and in use
in a state that is not going to be qualified. So we have got to get some
guidelines out so that that kind of situation is avoided, vendors know
when they start down this path, have some indication of what will be
acceptable when they reach \--

MALE SPEAKER: (Off microphone).

DR. SMURGIAN: We need to turn some of the microphones off so we can hear
Dr. Revest.

DR. REVEST: It is not just security for some of these but usability. If
I can \--

DR. SMURGIAN: Yes, go ahead.

DR. REVEST: I appreciate the emphasis given by the commissioners on
security and the priorities expressed there for their desirability. In
terms of priorities within security, I think that by in large \-- let me
suggest the order in which the security and transparency resolutions
were ordered for presentation, is a good order.

We started off with voter verifiability. We had COSS early on. We had
set up validation testing early on and so forth. So I think that
ordering is pretty close to what I would recommend. Maybe software
distribution could be moved earlier but aside from that I think this is
good ordering. I will just state that as my sort of first cut at the
ordering. I think having an email discussion about the details of this
as we get into the individual workloads involved, would be fine with me.

DR. SMURGIAN: Ms. Quesenberry?

MS. QUESENBERRY: One of our task items would actually be something that
is not related directly to one of our resolutions but would come out of,
I guess it was the resolution on voter verifiability, which is to look
at issues related to human factors and accessibility surrounding the
whole voter verifiability issues.

That is sort of the place where we intersect and it is in fact the
conflict between assess access and security or usability and security,
is in fact one of the primary issues. And I guess the other one I would
say that's come up in a couple of forms around representation of ballots
is that if we end up with a paper record in any way shape or form that
is intended for a recount, that the usability of that record for
recounting purposes is really important.

I have the sense that often the fight was over whether the printer
should exist at all, and then once somebody made a decision they just
spit out whatever the printer spit out and we could certainly improve on
that and issues and help with that and help raise that bar.

For the other items that we have on our plate, the four that are at the
top of our priorities list are two, which is \-- they are all the early
numbers. Two, which is the basic requirements for accessibility, four,
which are the basic usability and other privacy requirements and five,
which is not the entire part of five but to identify the metrics needed
and a road map for developing those metrics. So five which is the very
large resolution around human performance testing, taking a chuck of
that and sort of moving that to the front and then at each sort of time
period, progressing that forward a little bit.

In lower priority, number three, which is the issues around human
factors and privacy as it relates to the polling place. Number nine,
which is helping to look at other standards as they are developed for
any human factors and privacy issues. And number six, which is the
universal design principles one.

And then there are two, which we hadn't slated until the farthest out
dates that we were looking at all. One of those is eight, which is
guidance on instructions, ballot design and error messages. And number
ten, which is usability of the standards themselves. Since all of those
rely on a certain amount of information gathering and research, those
tasks have to begin and be completed before we can even tackle the
requirements problem.

And I just sort of would like to say one thing about the position of my
committee, which is that as important as security is, accessibility and
usability play into the ability of a system. I mean a system that you
can't read isn't going to be secure because you are going to do things
like open the door and ask for help and so on. We have a very small
group of people working on it, so we are already kind of prioritized
down in resources but we are part of the HAVA law.

And I think that I work under the assumption that election officials are
eager and interested in helping all of their constituents vote
effectively and we would like to be a part of providing the guidance and
requirements that will give them systems that will enable them to do
that.

DR. SMURGIAN: Well thank you very much. If I may let me make a few
closing comments. Resolutions adopted at this planeary meeting instruct
NIST staff on the research and drafting of standard recommendations. The
adopted motions provide essential policy guidance on relevant voting
standard issues.

And resolutions, we have made an attempt to prioritized them and we will
do that off line. By the way all of these resolutions that have been
adopted will be posted on the \-- on our website, vote.nist.gov by the
end of this week. Tomorrow is a holiday so we will try to get that done
by Friday.

The resolutions will be prioritized, trying to tie them together to
address the immediate requirements for development of an issue
recommendations as suggested by EAC also by April of this year.
Additionally, resolutions of a more general nature or lower priority
will be considered at our February 24th and 25th meeting planeary
session. As was proposed some of these resolutions will be reconsidered
and in modified form perhaps.

Provisions of HAVA require the TGDC to submit a first set of
recommendations to the EAC executive director in April of 2005. We just
heard from the executive director so we definitely have our marching
orders.

The forwarding recommendations will vary for each subject matter
depending on the existence of current best practices, specifications or
standards. For example, some guidelines will consist of reviewing
quality, pretest of standards already developed by other organizations.
Recommendations related to such projects may consist of suggestive
standards capable of immediate implementation. NIST staff in cooperation
with the TGDC members will make best efforts to accomplish the critical
tasks most urgently needed by the election community.

HAVA requires election authorities to be in compliance with certain
requirements in January 2006. The TGDC and its subcommittees should give
first priority to projects, which will be useful to the manufacturing,
testing, laboratory and election administration communities in
satisfying these mandates.

I appreciate the participation today and yesterday of all the committee
members in attendance and those on the phone and look forward to our
work ahead. I also would like to thank EAC commissioners and staff for
joining us and sharing their thoughts with the committee. I think that
was very important to hear from them in person.

You know that are staff is working very hard obviously you have been
working hand in hand with them but NIST is committed to get this job
done and do it right the first time. So if there is anything that I can
do personally to make things move along, my contact information is in
your book at the very back. My email is simple, <hratch@nist.gov>. So
that is the best way of contacting me.

I also want to thank the NIST scientists and staff for their efforts to
make this meeting a success. I know they have been working very hard
over the last few weeks and I very much appreciate their efforts. So our
next planeary session of the TGDC will occur here at NIST on February
24th and 25th. And unless there is an objection \-- yes.

MALE SPEAKER: I guess I just want to make it clear representing the
prioritization what we will be getting and when. I know we talked about,
certainly discussed at the high level the issues, but I presume we will
be getting specific resolutions that fall into three bins sort of. The
most immediate that has to be done by April, the ones that can be done a
little later and the more longer term. Is that what we can expect, the
resolutions to actually be partitioned into those three bins?

And then I guess the next question is when will we receive that? The
sooner we get guidance obviously the better off we will be.

DR. SMURGIAN: Well I assume that chairs of the subcommittees will be
working with our staff and among each other to make sure that the
priorities especially since some of the priorities affect other
subcommittees. They are not really working in isolation. So clearly the
sooner that prioritization is done the better it will be for us to get
going.

So with that, I adjourn this meeting of the technical guidelines
development committee. I hope you enjoyed your stay here at NIST and the
fresh air you got yesterday and I guess today. We look forward to seeing
you again in about a month. Thank you very much for coming.

(END OF AUDIOTAPE)

\* \* \* \* \*

[CERTIFICATE OF AGENCY]{.underline}

I, Carol J. Schwartz, President of Carol J. Thomas Stenotype Reporting
Services, Inc., do hereby certify we were authorized to transcribe the
submitted cassette tape, and that thereafter these proceedings were
transcribed under our supervision, and I further certify that the
forgoing transcription contains a full, true and correct transcription
of the cassette furnished, to the best of our ability.

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

CAROL J. SCHWARTZ

PRESIDENT