Un-official Closed Caption Transcript:

3/19/2006 Technical Guidelines Development Meeting

Good morning, everybody, could we take our seats? We\'re ready to begin
this meeting.

Allan Eustis. I welcome you all to this sixth plenary section of the
technical guideline development committee. I welcome you to the George
Mason nation. It is the next state over, but it is close enough to us to
claim credit for their success. A couple of preliminary things I will go
through and I will hand the meeting over to Dr. Semerjian. Our usual
safety slide, we\'re up here on stage or I am. There is an exit
literally in all four corners. If there is an emergency, you will see
the blinking lights and those have been here before know that we have
real practice emergencies here. You will hear a voice, and then just
please proceed to the nearest exit. Out of the back exits can easily
access glass doors outside the building. As far as Jim is concerned and
J.R., we\'ve people down here willing to assist you should there be an
emergency. We will make sure you are taken care of. With that, please
turn off cell phones, pagers and other electronic devices, some of which
don\'t work in here anyhow. Please be aware there is no food allowed in
the auditorium. I have broken the rule in the past. Please wear your
name badge at all times for security reasons clearly. If there is
anybody with hearing issues, our signers are over here stage left.
Please feel free to sit over on that side of the auditorium should you
need their services. They will be here all day long and will continue to
check to make sure people understand that\'s where the signers are. The
web cast and I welcome all the people to our web cast is closed
captioned. It will be available in archive format at the end of this
meeting. My last comment is to TGDC members, they please remember to
identify themselves when they address the Chair, and the rest of the
committee. You have a little button that turns your microphone on and
off. I have actually kept records here of people who remembered to say
their names because I have to go through the minutes of the meeting, and
it is very nice. We have only one person who gets an A for saying his
name, and that\'s Paul Craft. There are a lot of D\'s and F\'s. There
are a few B\'s, Whitney and J.R. Harding and Jim when he is on the phone
identifies him. There were a lot of C minus and D\'s, so I would like to
see that improve, please. With that I turn the meeting over to Dr.
Semerjian.

Good morning, everyone. Welcome. I am the deputy director of the
National Institute of Standards and Technology and the Acting Chairman
of the development committee for today. I hereby call to order the sixth
plenary session of this committee today, Wednesday March 29th, 2006. Let
us now stand and pledge allegiance.

I pledge allegiance to the flag of the United States of America and to
the rep for which it stands, one nation under God, indivisible with
liberty and justice for all.

Thank you. At this time I recognize Mr. Phil Greene as the TGDC
parliamentarian and request he determine if a quorum of the Committee is
present.

Williams. Williams is here. Berger. Berger is here. KARMOL is here.
Craft is here. Galel. Gale is here. Elekes. Elekes is here. Gannon.
Gannon is here. Miller. Miller is here. Harding is here. Purcell.
Purcell is here. Quesenbery is here.

RIVEST is here. SHUTZER is here. Turner Buie is here.

Not at the moment.

Here.

Smerjian is here.

Turner Buie.

Hello. Hello.

We can proceed.

How many votes are necessary to carry an issue?

\-- carry an issue?

Mr. Parliamentarian.

At the present we have fourteen. We would want eight votes to pass a
motion.

Thank you.

Thank you, Mr. Greene. I am pleased to return briefly as Chair of this
committee, Doctor Jeffrey has been invited to appear this morning at a
Senate committee hearing, and he has asked me to fill in for him at this
important TGDC public meeting. This morning I look forward to working
with my former colleagues on the committee. We\'re especially pleased
Mr. Jim ELQUESE is able to participate in person today. He has been a
most valuable contributor to the development work of the TGDC
subcommittee on human factors and privacy. I also welcome Miss Share on
Turner Buie participating via teleconference due to her work load as
director of elections in Kansas City. I also understand congratulations
are in order for J.R. Harding who is engaged to be married in the next
few months. Congratulations, J.R. Finally, let me thank all the members
of the committee for reserving time on their busy schedules to
participate in these proceedings. The initial recommendations for
volunteer voting systems standards delivered by this committee to the
election assistance commission in the nine months mandated by the help
America vote act are a foundation for increasing the nation\'s trust and
confidence in our voting system. In addition, this voting team has
benefited from your willingness to volunteer significant time in
assisting them to complete the preliminary reports for future updates to
the BBSG we will review today. Committee is also pleased today to have
three of our U.S. election assistance commissioners in attendance with
the commission\'s executive director and senior staff. The committee
will shortly receive remarks from EAC commissioner Donnetta Davidson,
Gracia Hillman and Tom Wilkey. I look forward to their comments
regarding the ongoing work of this committee. At this time I will
entertain a motion to adopt the September 29th, 2005, meeting \--

Mr. Chairman this is \-- inaudible.

Yes. March 29th 2006 meeting agenda for the Technical Guidelines
Development Committee. Do I have a second?

Second.

Any comments or discussion?

Yes. Mr. Chairman, I would like to request a slight modification to the
agenda. I think there is several of us that would like to discuss the
general structure of our work given where we are to assure that our
efforts are being focused on where they\'re most needed to improve the
voting system. I think that might be most helpfully done early in the
meeting and revisited at the end of the meeting. If that might be
agreeable, I would make a motion to amend the motion to add a short
discussion at the beginning and at the end looking at the system
organization of our effort.

I will second that.

Any other comments?

Actually I have I guess a question about the intent of the agenda, Mr.
Chair. This is Paul Craft. My grade is sliding already, I am afraid. The
agenda item introduction of resolutions and discussions by the TGDC, am
I to read that to indicate that we should not be introducing other
motions and discussing motions during the body of the meeting until that
point in the day? It seems to hamstring the committee quite a bit.

We have actually discussed that. I think we will take resolutions where
appropriate. I think some resolutions we may want to postpone until the
end because there may be other discussions that may impact.

Then I would move that the 415 agenda item be amended to include
introduction and discussion of any resolutions not discussed earlier in
the day.

That would be fine. We\'ll make the change accordingly, but if there are
indeed resolutions that need to be taken up earlier on, we will do so.

Okay.

Regarding the other Mr. Berger\'s motion, why don\'t we then have the
discussion and presentations by Mark call and John wax so we will hear
at least from these people what the thinking is, and then given that
maybe after the break we can then have a brief discussion? Is that
acceptable? Is that acceptable to everyone else? Then we will have a
brief time period after the break to have a broader discussion. If
that\'s acceptable to everyone, we will proceed as such. Thank you.

Mr. Chairman, point of procedure, do we need to vote on the adoption of
the agenda.

Yes, yes. Could I have another motion to accept the agenda as modified.

So moved.

Do I have a second?

Second.

All in favor?

Aye.

Any opposed? It is passed unanimously. Thank you. At this time I will a
motion to accept the minutes of the September 29 2005 meeting of the
technical guideline development committee.

So moved.

That is in your second tab actually in the book.

Mr. Chair, there was a notice that Mr. Eustis sent out regarding a
correction. Has that been incorporated?

Yes, it has.

Yes, I believe in the version that was on your desk this morning that
change has been made.

Okay. I will second.

Okay. We have a motion on the floor and a second. Any other questions or
comments? Not hearing any, all those in favor of accepting the minutes
of the September 29 meeting of TGDC, all in favor?

Aye.

Any opposed? Thank you. By the way, just for your information, all the
resolutions that have been adopted over the last few meetings of the
TGDC are here for your reference under the third tab labeled as adopted
resolutions from the very first meeting until the last meeting just for
reference. As a brief review for the public in attendance and viewing
the web cast, public law 107-252, the help America vote act, HAVA,
establishes the technical guideline development committee. HAVA charters
the members of this committee to assist the election assistance
commission with the development of volunteer voting system guidelines.
In addition, EAC resolution 2005-1 authors the TGDC to continue its work
beyond the development of initial voting systems standards guidelines.
This committee\'s initial recommendations for the guidelines was sent to
the executive director of the U.S. election assistance commission in
accordance with HAVA\'s nine-month deadline on may 9, 2005. The EAC
issued draft voluntarily voting system guidelines for public comment in
June of 2005. The final volunteer voting systems guidelines, VVSG was
publicly announced and copies were sent to committee members in their
advanced reading material. The guidelines are also posted on the EAC
website, www.EAC.gov. Since the last meeting of the TGDC in September of
2005 this staff in coordination with the three working subcommittees of
the TGDC have continued drafting and editing preliminary reports on
issues pertinent to future volunteer standard development in areas of
human factors and privacy, security and transparency and core
requirements and testing of voting systems. We will discuss these
reports at today\'s preliminary session. Specifically as a committee we
will review, approve and where appropriate provide supplemental
direction to this scientist. This is critical to the development of
recommendations for future volunteer voting system guidelines. The time
required to accomplish the agenda items means that the committee cannot
take public comment at this meeting. However, there will be continue to
be opportunities for the public to comment on relevant issues.
Additional comments and position statements regarding the work of this
committee should be sent to voting aNIST.gov where they would be posted
on the NIST voting website, vote dot NIST.gov. The comments we received
to date have been post and had reviewed by the staff and TGDC committee
members. At this time I neat that the latest revised version of Roberts
rules of order was adopted on July 9, 2004, to govern technical
guideline development committee and subcommittee proceedings. At this
time I invite EAC commissioner Davidson to address the committee. We
look forward to hearing from the EAC members.

Mr. Chairman,.

Yes.

This is J.R. Harding. Before our past colleague and now commissioner
speaks, you mentioned our resolutions in our binder. In the past we\'ve
referenced the resolutions as a kind of an overview or intent of this
group to guide the development of guidelines. Has there been any effort
made by staff to let\'s say quantify or count the integration of these
philosophical statements into those draft guidelines as kind of like a
check and balance in the spirit of our work being integrated into the
VVSG, and if not, I would like to ask that we do that.

Yes, we in fact after Each review of reports, we will refer to the
resolutions that that particular piece of work will be in response to so
to speak. We will have a compile lags between the work being reported
and the resolutions that that particular body of work addresses.

This is Paul Craft. I guess I have the same concern that J.R. voiced.
When they were finalizing the draft on the last version I asked Mr.
Eustis, Allan Eustis for any information on that, and I have been unable
to find a document that has that kind of analysis. I would make a motion
that NIST take it upon itself to do an audit or review and determine if
the paper has to the extent of those prior motions flowing through to
the standards.

I would second that, Mr. Chairman.

I will actually in my comments in my summary after Each review, I will
refer \-- it is in my notes here. I will refer to Each resolution that
this particular piece of work addresses. I believe the speakers will
also have in their presentations, in their drafts, lists of resolutions
that that particular work addresses. Is that sufficient?

Doctor Semerjian, can you hear me?

I think part of the question is the reverse of what you\'re discussing.
Looking at the resolutions and seeing which have been covered, which
have not, we will in the next few weeks be posting on our web page that
exact scenario for you with the break down of all the resolutions and
where we are with respect to Each resolution if that\'s going to be
helpful.

Is that acceptable to the members?

It would be to me so long as it is a public document.

I assume that we will be posting that on our website.

Yes.

I guess if NIST intends to do something of that anyway, shall we go
ahead and call a question? Make it an official resolution of the TGDC
and NIST and address it?

Would you like to make a resolution?

I made a resolution that\'s been seconded.

Is the resolution clear? I believe the resolution is to request NIST to
post on its website a list of \-- would you restate the resolution.

The resolution was for NIST to do an analysis of the extent to which the
TGDC resolutions have flowed through into the most recent version of the
volunteer voting system standards and to publish a report with those
results \--

Published meaning posted on the website.

To me these days it generally means the production of a hard copy
document on official letterhead and posting on the agency\'s website.

Did you have a question? I was wondering if you wanted it to be broader.
You\'re asking specifically flowed through into the current version. I
know in human factors some of is is ongoing work which is a slightly
broader question. We are working on things you will hear about this
afternoon which are not in the current version but are being planned for
future versions.

I would be happy to accept that as an amendment to the motion scbl it
might be something like we ask NIST to report on the status of the
resolutions in regard to work that\'s been donor under way?

That would be acceptable to me.

Any other comments?

Mr. Chairman, I would \-- J.R. Harding. I would take that to mean that
we would continue to track it from here on so it evolves as the work
evolves. Okay?

Is that from the time we request that?

Certainly at least from the same cycle as our meetings.

That\'s what I was going to say, maybe we need to make a practice of
presenting such a report either in a presentation or at least in hard
copy for the information of the committee so that we produce such a list
for Each TGDC meeting. Is that acceptable?

I will make you a deal. I will temporarily withdraw the motion so the
commissioner can go ahead with her presentation. During the break we can
try to put some language around it and represent it after the break.

Thank you. Commissioner Davidson, I apologize for the delay keeping you
here. You have been a member of this committee.

I understand. It is great being here with you today, and I do want to
tell you that the \-- our Chair could not be here today because he is at
the American University, and so he sends his regrets that he couldn\'t
be with you today. As stated we do have our vice Chair in the audience.
We have Ray Martinez and you will be hearing also from Commissioner
Hillman in just a little bit. You have a good representation of us, and
we have staff here, and as you said Tom Wilkey is also here for
presentation. As you just reviewed, and doctor Semerjian you took part
of me speech. I will try not to go into a lot of that. We did have a
busy 2005. We accomplished a great deal. Within the nine months you got
it to the EAC and by December 1st obviously we had \-- 13th we had
standards. We\'re very pleased about that. The help that we received
from NIST we publicly want everybody to know in reviewing all the \--
excuse me, all the 6,000 comments out there, we had lots of support from
the NIST group, and I want to really thank you. As we look forward, we
know we have a lot of work to come. We have the work on future it
rations already under way, you have a lot of resolutions you will be
talking about today, and we have the certification program that\'s top
priority also. I will go into it a little bit coming up. We\'ve come a
long ways in a short time as I said. As a former TGDC member as you just
referred to, I understand how hard you work. I want to thank Each and
every one of you for your loyalty of being here constantly as often as
you can. Also for attending by phone when you can\'t be here. It is
really a rewarding to see how hard all of you work, and we do thank you.
Thanks to Dr. Semerjian for your leadership in the past, and we look
forward in working with Dr. Jeffries in the future. The VVSG addresses
\-- increases complexity our voting systems. The technology and how it
impacts everything obviously security, use ability, which was always
involved with, and accessibility which is also very near and dear to all
of our hearts. Work on the route it ration sincerely ongoing. And we
must scheme up with the technology and the security. As we moved forward
we must keep in mind the wireless. Changes are unreal how it goes
through everything. The V V pack we have to keep addressing. The test
suite is a big one. We\'re anxious for a lot of the test sweeps and more
forms of independent verification. The time frames, we\'ve worked on the
time frames with the NIST and our monthly meetings. We try to have a
meeting every month. Once in awhile it doesn\'t quite work out. The time
frames were shared with everybody. We thought we would be handing out
maybe what we call different versions \-- not really versions, but
modules we\'ll say of the standards that we would be moving forward, and
we feel we after we really looked at it, we felt that we really
couldn\'t accomplish that because when we talk about we would have more
public hearings, I don\'t know on page 6 is where I am at. Okay. More
public hearings, and we really felt like that sometimes we could really
get into a confusion element with all the \-- everybody out there. When
we use \-- when you stop and think about it, if we got in April the V V
pack, there is issues that could take place in the future of that in
this 2006 elections that maybe you want to address after that, so we
really felt like the April time frame of giving that to us, us having
public hearings on it, publicizing it, and we could have another form
come in right away that you would have \-- I don\'t remember which one
it was, the next one, but it could cause some confusion not only with
election people, but the public outside. We could be receiving comments
on more than one at a time. We looked at that and thought, obviously we
think the time frame and moving forward and utilizing that as a draft,
it really would help everybody if we could get them as so that they are
public obviously, you get it, and it is very public, and you put it on
your website, so that the vendors know what\'s moving forward, but at
the same time we\'ve got this 2006 election that\'s coming up that I
think we\'re all going to learn a great deal. Then it can be all
utilized and sent to us at one time. The one thing that I think that is
important to remember is that can be technical amendments that we make
to the 2005. If they are technical, we can actually go through that part
of it and do the technical amendments and the certification portion of
it. That will make it, we feel, more concise and be really open to
everybody and when you present everything that we have at the end
obviously the guidelines will be there for the whole iteration of it to
be changed in 2007. I hope I explained that clearly enough that we
expect the working to forward, but we hope that it will be kind of like
in a draft format so if you see after the election if you need to touch
that again, you have that capability before it is presented to us. In
working that way, we hope that it will be a significant improvement of
over what we have planned originally. The TGDC has been more involved
with our operations than what they had been in the past. We really enjoy
having them. The Chairs we have started inviting them to take part and
be at your standard boards and our boards of advisory meetings, and we
really look forward in working with them. I think that from thee to time
if they want to come in and be part of our monthly meetings we have with
NIST, we have no problem in doing that. Also on the other hand we\'re
becoming more involved and learning as you go along the process, we are
going to be more involved with your communication, your weekly or every
other week meetings that you\'re having over the telephone. That way
we\'re knowing what\'s going on and it is not \-- we\'re more aware and
more prepared to make decisions I think as we move forward in that area.
When we start looking at how we work, we also look at our budgets as
another thing, and as we have to work with NIST to make sure they get
the money they need to support the TGDC and all the efforts that go
behind the scenes that the NIST people are working on, so we need to
work with the Congress and make sure they get their budget. We go in
hand in hand where if they need an increase, NIST, if they need an
increase, it doesn\'t decrease our funding at the EAC. Sometimes they
cut one group short so they can give another group. Obviously we want to
be hand in hand in that proposition as we move forward. The other thing
that I really wanted to go into is one of the things that we need to
start looking at is how do we address the issues of people out there
doing voting systems that we have not the complete standards for like
phone voting. We need to really address some of those issues. They\'re
also utilizing and looking at ATM in the future, so these things I think
that we really need to stand back and say how do we address this and
what moves do we make in the future? Also there is a big one. Congress
is very intent and we\'ve got direction in the law that actually says we
have to better serve our milITAry and over seas voters. We have to take
that seriously. We have to move forward. Getting the certification
program up and running is top priority for us. Not only the
certification for the independent test authorities, but also the taking
over the certification of the voting equipment itself. That\'s very
important to us, and we are going to be part of that process of
pre-assessment so we can learn what they\'re actually doing, the NIST
lab, and so we can move forward and be more knowledgeable in those
areas. We feel it has to be a very trance parent process. We need to be
very open with everything we do. Beyond the BBSG, it is only part of it.
There is also that human factor element we need to look at and training
support has to be addressed. It is hands-on. We have to take care of it.
We have a group right now doing a study on the election management
guidelines that goes side by side with all of the standards and
guidelines really, the guidelines. Our executive director will go more
into a lot of our research programs we\'re doing, and give you some
information there. We\'ve already proven that working together we have
accomplished a great deal. 2006 will be a very important year for all of
us. It will give us a focus and shifting government\'s first voting
system to the certification program, the election reform will always be
ongoing. There is always going to be changes we have to consider. If
2006 elections, if we have issues, they\'re all very \-- the time frames
are very short. By the time that the states and counties are buying
their equipment, the vendors are struggling to meet all of those
deadlines. The shorter the time they have to train judges, then we have
more issues because the judges have been trained properly or if the
equipment hasn\'t been tested properly at the time they received it.
There is issues we think could come up in this election. If it does,
obviously we\'re going to see more in legislation possibly within states
or in Congress. We also have to remember there is a balance in the work
that we\'re doing. The balance of the cost and how usable the equipment
is for the judges and how the costly it is for the elections community.
Obviously we want every state to adopt our volunteer guidelines, and if
we have them so stringent we\'ll see them backing off because they
can\'t afford it or if there is issues. There is a balance there we all
have to remember. I know you understand that. I just wish you all the
very best in working forward and I will now call on my Chairman Hillman
to come up and say a few words to you. Thank you very much. \[APPLAUSE\]

Thank you Commissioner Davidson.

Thank you. Before we begin, let me say there was no could you some
morning. Call is still the Chair of the election assistance commission,
and I am here to thank you in my capacity as last year\'s Chair of the
election assistance commission. Let me begin by saying good morning to
all of you. Dr. Jeffery in absentia, all the members of the committee
and all the NIST staff. The delivery of the 2005 voluntarily voting
assistance guidelines was a major accomplishment. It was indeed a
pleasure for me to have served as Chair during that time. To have
completed one full year of working with the technical guidelines
development committee. It was something that at the beginning of the
election assistance commission we couldn\'t imagine how we were going to
get it done given the great obstacles, and you all were willing to come
to the table even before we knew that we would have sufficient resources
and a budget to complete this work. You were willing to take that risk
with us, and I think it is because of that commitment and conviction
that we were able to prevail, and so again I want to thank you. I also
want to say how important it is that the guidelines were developed as a
result of the different lenses that the committee members brought to the
discussion. It was important to have the scientific and technical input,
but it was also important to have the input of election officials to be
able to bring that perspective to the complexity of this issue and as is
witnessed by the many, many comment that we received, even when we
removed the redundant comments, receiving several hundred comments to
the draft guidelines was just incredible. It let\'s us know how
important this issue is, particularly at a time when at least public
reports remind us of the growing distrust that people have of some of
the newer voting systems, at a time when the technology is developing,
at a time when the federal government is taking on the first time full
responsibility for not only the development of the guidelines, but also
the certification of the equipment. Those three major activities coming
together could create the perfect storm or they could create the perfect
solution, and I believe they will create the perfect solution. Again, I
just want to thank Each and every one of you for everything you\'re
doing, and I look forward to continuing our work together. Thank you.
\[APPLAUSE\]

Thank you Commissioner Hillman. Just personally I would like to say that
the entire EAC, all the commissioners, commissioner Martinez, Hillman,
and commissioner Davidson, GROGORIA, have been great supporters of this
work, great supporters of NIST staff, sometimes people get down as
commissioner Hillman said. Sometimes we have doubts whether this is
going to get done, whether it is going to get done in time. A little
cheering and a little encouragement was very much on target. We
appreciate your continuing encouragement and continuing support of the
work of the TGDC and of NIST staff. We very much appreciate all your
support and your being here today. Thank you very much. As it was
mentioned, this is a team effort and I think the next set of
presentations will reflect that team effort. Mark Skall of our
information technology laboratory will provide a review of NIST\'s
activities since September 2005 but that will also follow up by
presentation by the EAC executive director. You will see progress as
seen from both sides. At this time I call on Mark SKAL to give us a
review of what\'s been accomplished over the last six months or so.
Mark.

Thank you. I am basically going to tell you what\'s been done since the
last TGDC meeting. There has been quite a few activities that NIST and
the TGDC have been engaged with. In October of 2005 we had a thread
analysis work shop for voting systems. November 2005 the VVSG 2007 time
line was approved. As you recall there was a resolution at the last
meeting asking NIST to look at the proposed time line we had developed
to coordinate it with the EAC to ensure that the dates made sense and
modify the time line if appropriate, and after some deliberations with
the EAC, they basically accepted the time line as it was. That happened
in November that the time line was formally accepted by the EAC.

I assume you may know, but I just want to point out you have a copy of
all of these presentation materials inside in the behind the front cover
of your binders. There is a thick package with all the different
presentations. If you want to follow, you have that at your disposal.
Thank you. Sorry for the disruption. In the November to December time
frame as the commissioners have referred to the work we\'ve done with
them to assist the EAC in resolving the comments from the public review
period on the VVSG, this was a fairly intensive effort over about six or
seven weeks, probably six or seven NIST staff involved full time during
that period that did of course take away from the work that we could do
on the next iteration of the VVSG. On the other hand it was extremely
important work, working with our partners at the EAC proved to be a
tremendous experience for us, and I believe for them, and I think we got
the best resolution of the comments we could possibly get from that end.
December 2005 the VVSG was formally adopted by the EAC and the January,
February, March period we were continuing development research on the
next iteration of the VVSG. I would like to say a few words about the
thread analysis work shop held in October of 2005. The goal really was
to arrive at a set of drivers for our requirements, mainly our security
requirements. These are fairly strict requirements we\'re imposing on
states, and we wanted to ensure that these requirements were driven by
real threats. In essence, you can look at the requirements and solutions
to problems and the problems are what we wanted to ensure that we had
documented very precisely so the requirements can mitigate the problems.
We had this thread analysis work shop we believe was very successful. We
got a lot of feedback from people at the conference as to the success of
it, bringing different players together from very aspects on security
and looking at threats from various angles. We have a draft work shop
report available on our website, and we are undergoing more extensive
review to look at the threats in more detail. We also have a follow-up
work shop planned for June. Again, speaking a little about the comments
resolution, I mentioned that we were requested by the EAC to work on
this, and we of course felt this was a very good idea to work jointly.
Again, we analyze comments, the EAC of course made final determinations
as to the resolutions of the comments and the wording that would be
actually incorporated into the standard or into the guideline. Some of
the comments we looked at were deemed to be appropriate for future
iterations of the VVSG, and those were categorized as carry over
comments. They will impact our work on the next iteration of the VVSG. I
would like to talk about the time line for a few minutes. The completion
date when we are targeting our completed next iteration is July of 2007.
Commissioner Davidson referred to this. Let me give me perspective on
this which I believe is the same as Commissioner Davidson. About a year
ago we met with the EAC commissioners and because we all knew the next
iteration would not be available for awhile both the EAC and this were
looking for some way to get the requirements we\'re developing usable
for quickly, and we thought one-way to do this would be to complete
modules, certain modules and swap them is the term we used into the VVSG
2005 so they could take effect immediately. We figured in that way we
could get our requirements used without waiting another year,
year-and-a-half: I think at the time that made sense to us. As time
passed we realized there were clearly problems with this approach. First
of all when we complete a module early, it is very possible some of
those requirements will change as we learn more one we\'re developing
other requirements for the Virginia as a perfect example of V V pat
almost complete now, due next month, we will have a new module
available. However, as we learn more about IV, as we learn more about
human factors, as we continue to develop our standard, clearly some of
the VVPAT requirements will change. That was one issue. Secondly, there
would be many public reviews in parallel. Some of these are due within a
couple of months of Each other. They all have to go through fully
extensive public reviews similar to the VVSG 2005 and having a couple of
public reviews at the same time on similar material I think would be
very confusing to the public. Vendors asking to be certified that would
complicate that issue as well because you would have to be precise what
version with which module being incorporated, one is certified, two, and
we heard vendors were confused by this approach, and they tell us
election officials were confused. We met again with the EAC and decided
on a different strategy. We would still complete the modules, make them
available on our website and vendors could bill to them. They would not
be a part though, officially a part of the VVSG 2005. They will wait
until July 2007 to be incorporated, but giving the vendors a heads up
clearly will help the situation. They can build to these requirements.
There is a possibility some may thing. But they will be in good shape
when we put them on our website. After making this decision we really
looked at what\'s the best way to coordinate these modules with TGDC
meetings and other meetings, and we still have modules we want to
complete that we want endorsement from the TGDC. There are really only
two ways I think to prom he will gate these modules. We could have a
TGDC meeting prior to Each completion schedule date for Each module.
That really seems unwieldy. It would mean many more meetings of the TGDC
we feel are unnecessary. The second approach is not to have a TGDC
meeting every time a module is about to be completed but do a lot of \--
inaudible \-- with the subgroup. The subgroups could not vote but
clearly there would have to be acceptance from the subgroups and wait
until the TGDC meetings to formally endorse, adopt and/or change the
drafts we produce. If you look at this plan, it seemed like the best way
to schedule the uPComing TGDC meetings would be to have the first one
after this in December. I think we heard from everybody we need to wait
until after the elections for the next meeting. Clearly we need one in
July when the final product is due and perhaps one in the middle of
April. That would be our thoughts on uPComing TGDC meetings to account
for this schedule. One

thing I want to mention commissioner Davidson eluded to our FY \'07
funding we are in the President\'s budget to get funding for test suite
development. This is something we haven\'t been funded to do. We are
working full time to producing the standards and guidelines. It is clear
to everybody in the community test suites are an important part of this.
They are not usually official I part of a standard. We have a section in
the VVSG on testing and clearly that would refer to the test suites. We
actually have a field for Each requirement that documents the test
methodology used for that requirement, so we would clearly refer to the
test suites. Test suites as I think you all know is a very, very large
job, requires a lot of resources to do these correctly. One of the
things we will be working on if the funding comes through in 2007 is
three different types of test suites, one to ensure that requirements
are met correctly, another one to look at security open end ended
security testing and a third to do the human factors testing. I want to
speak a few minutes about the NAVLAP conducting the internal
accredITAtion so we can make recommendations to the EAC which will \--
which will be accredited. We received five applications. The first three
applications that we received we have scheduled onsite visits to do
pre-assessments where we speak to the laboratories and we will have a
better sense what shape they\'re in, how long it would take to get them
up to speed for accredITAtion. NAVLAP has asked if the EAC and/or other
parts of NIST information technology laboratory would want to attend and
we feel it is very important to get first hand experience so EAC
commissioner Davidson and I and some others will be going on the first
assessment. There are two more labs, five total, in the queue to be
assessed next. With respect to outrEach, the first iteration of the VVSG
was very constrained by the time limits imposed by HAVA. During the next
iteration we\'re trying to rEach out to many other parts of the election
community. We\'re involved now with the Election Technology Council of
the ITAA representing voting system vendors. We try to coordinate with
ITAA and get their inputs. We want, as well, to get inputs from as many
election officials as possible. We send out questions and receive
answers from various people in the community we think could help us. We
made a lot of presentations at various forums. We\'re coordinating with
the NSF funding accurate group more work shops are planned.

We hope that materials and various agendas will be more easily
accessible from the web. One thing we know is we produce a lot of
material. It is quite a burden to try to read these in a very short time
frame. We\'re getting all material out on the web page as soon as we
possibly can as soon as they\'re done we have them on the web page. You
don\'t have to wait until the meetings or two weeks before. We will have
as much material as possible for you to review early on. These are just
what the new pages look like.

I would like to close with general comments. The VVSG 2007 clearly is a
major undertaking. We are attempting at least at NIST with the support
and coordination of the TGDC to make the VVSG specific, unambiguous and
testable, make it understandable to many audiences, testers, vendors,
election officials, public. that\'s not always very easy to do.
Sometimes vendors need more. We need to do this in a way this is
understandable. We\'re re-examining all previous versions coordinating
with many groups and working with many bodies.

We have an introductory section to VVSG 2007 that is an overview
section. We will really focus on the area to make it understandable to
many, many audiences who are perhaps not as technical as some.
Subsequent drafts in a new format we distributed through drafts for Each
of the requirements, and one thing we\'ve discussed is perhaps expanding
the TGDC meetings. It seems to be a burden to get everything done this
one day. We're thinking of perhaps a day and a half with part of that
time where we could produce some overview material for better
understanding of everything. Lastly, these are just the mentions you
will be hearing today from this supplemental guidance, human factors and
privacy, core requirements and testing and security. any questions
before I allow our colleague Tom Wilkey to speak?

I would suggest that we hear from Tom also. I think your presentations
are sort of complimentary, and then maybe take questions because some of
the questions may be addressed in Tom\'s presentation.

Has there been a date set for the June meeting you referred to?

Which meeting

The work shop

I believe there has.

The last Thursday and Friday of the first week. I think the fifth and
sixth. does someone have a calendar?

June?

The first week of June

Thursday is June 8th. so eighth and ninth. is that what it is?

is it here? Where is it going to be?

George Washington University

June 8th and 9^th^.

If there aren\'t further objections we\'ll hear Tom Wilkey speak and
we\'ll jointly take questions afterwards. At this time I call on Mr.
Wilkey to report to the Committee on the EAC strategy for updating the
voluntary guidelines and on the EAC\'s research projects.

Thank you very much. it is good to be with you today. First let me say
Allan Eustis that having spent more time in meetings with Paul Craft
over the years we always knew when he was in the room. First let me say
that my very good friend and commissioner Ray Martinez likes to point
out every once in awhile that I have been in this business so long that
he was seven-year-olds when I actually started in this business. Over
this career, I have made my share of boo-boo\'s, and I made one last
night when I picked up my notes and picked up the wrong set of notes.
this is going to be what we call the proceed verbal winging it, and I
apologize for that but I think between commissioner Davidson and Mark
Skall they have done an excellent presentation of where both of us are
coming from in this process. first let me add my deep appreciation to
the members of the technical guidelines development committee for their
work they have done thus far. I was privileged to take part in both the
1,990 development of the standards as well as the 2002 iteration of the
standards. I know what an incredible undertaking doing something like
that is. to do what you did in a nine-month period was unheard of, could
never have been done back in the days when we were starting this
process. no one appreciates your efforts and the work that you have done
more than I do, and you are to be congratulated for that. we also are
very, very pleased and are very cognizant of the efforts that the staff
at NIST have made in working with us in the aftermath of your presenting
us with that document, and that is going through the commentary process
and helping us achieve our goals of getting that document out the door
in \-- by the end of the year last year. it was a major effort on the
part of the staff of the EAC as well as NIST. we were very grateful for
their participation. we had a few laughs while we were doing it. When
you get two good New Yorkers like Mark Skall and I in a room, anything
can happen. we are again very pleased with the way we work together. we
have shared and have come to be a real team in this effort, and we
appreciate that very much. as commissioner Davidson mentioned, our top
priority over the commission few months is to get our certification
program up and running. you have heard comments about NAVLAP and the
pre-assessment program. We're looking forward to doing that. both
commissioner Davidson, myself and Brian Hancock of our staff, and we
hope we can continue as we move along through that process. Our own
certification agenda is being developed as we speak. we have excellent
consultants working with us. we expect to give a first draft of the all
of the procedures that we intend to have in place by the first week of
April to our commissioners, hopefully after some discussion and tweaking
and work on that document we will immediately have it all for public
comment and we will keep you updated as members of the TGDC because we
certainly will welcome your comments in the process. after that there
will be some ongoing legal review. Certainly we hope to have that
process up and running as soon as possible. Mark mentioned the issues
with the time line. NIST staff and EAC staff spent a considerable amount
of time last fall reviewing the document. As Mark so eloquently stated
we thought we had come to a great idea and we left the room thinking
this is great, we\'ll be able to get chunks or modules out the door and
get them gutted and get them approved and then as we began to as they
say peel the onion and take a better look at a lot of issues including
our statutory regulatory process we had to go through to get something
approved, a comment period approved, legal research, so on and so forth,
we looked at the involvement and what it would mean not only to the
vendors but to the election community and so we backed up a little and
took another look at it. I think the ideas are in keeping in our
thoughts on this area. You need to know as members of the committee that
our staff meets as often as we can. We try to meet on a monthly basis.
we will continue to look at this. And continue to try to get things out
sooner, up on the website so you can have a greater opportunity to look
at them, and then go through our regulatory process. we think that after
many thought and discussion we have come up with a reasonable way to
make that work given our statutory process that we must follow and give
the re alts of the community at large getting this in a more appropriate
fashion. Mark mentioned the funding we were very pleased, the
commissioners to make the requests to OMB to increase the funding for
NIST, for \'07. we feel that the test suites that are part of the agenda
are critically important. we felt it was necessary for us to try to make
that effort earlier, get it up and running so we did not have to spreads
it over a number of years, thought it would look in terms of the \'08
election coming up. we were very please that had O and B took our
recommendation. it is part of the President\'s budget. we look forward
hopefully to it being viewed favorably in Congress so that that work can
begin immediately and will be the necessary funds to do it. we feel very
strongly those test suites are the hallmark of everything we\'re trying
to do here. That being said, we are hopeful that Congress hears that
word and that everything will move forward and we will be able to
continue that. I would like to talk to you just a few minutes about some
of the other projects not directly related are peripherally related. we
thought that would be of interest to you. we have a number of research
projects issued last fall and coming out over a period of time during
this year, and certainly as I said are not directly related to the work
you\'re doing but certainly in the context of the election process are
part of what you are doing in a certain sense. as many of you know last
year in August we issued our first guidance on statewide voter
registration lists. we in September of last year came out with the first
national election day survey, the results of that survey are up on our
website very interesting statistics on a great number of areas that we
took a look at. we are in the process right now with a research project
that is nearing its completion. with Rutgers University, the Eagle ton
institute on a provisional study on voter ID we are looking at right
now. we are looking based upon the research our first national survey we
did after the 2004 election, we needed to go back and take a look at
that survey document, take a look at the results of that survey, and so
we have convened a meeting of various people who have interest in this
type of data, election officials who are working with that document over
the next couple of weeks and con convenience them in them in our office
and get it out earlier this year so the results of the 2006 election
will be available, our needs, our data we will need will get into the
hands of state and local election officials earlier than we had the
opportunity to do in 2004. \-- where do we go from there? we tested it
against the set of very good standards. We tested it with ITA\'s that
have now gone through an accredITAtion process, a high level
accredITAtion process, but what happens now? what do we do with it? the
issues of acceptance testing, pre-election testing, security
requirements, warehouse requirements, training requirements, everything
that goes into managing and maintaining a voting system. so since 1,990
I have been screaming we need to have these guidelines. so I guess one
of the proudest moments I have had since arriving at the EAC was to be
able to find the necessary dollars to get this project under way, and it
is a project we are doing in conjunction with the national association
of state election directors. we are very grateful. you know, when you go
to do something like that, you try to rEach out to find the very best in
the business that have the experience and level of experience to be able
to do a really good product, and we are grateful that Britt Williams,
one of your colleagues and Connie Schmidt who is the former election
director in Johnson county Kansas came out with one of the first
comprehensive documents ever for managing and maintaining voting systems
to work with us on this project. we are looking forward to getting the
first set of chapters out. We want to get them out as soon as possible.
As you know and I don\'t think I need to tell you, as we go through this
primary seen and the use of new equipment throughout our nation, we see
some bumps in the road. we see things happening. They will be reported
in the press. Some of these things unfortunately could have been
addressed had we been able to get this document out earlier. we will do
all week to get information such as this lessons learned, best
practices, as we move along throughout this primary season, get it up on
our website, and notify our election officials out from what best they
can do so these kinds of problems don\'t happen. I am very excited about
this project as you can tell. I am looking forward to it moving out the
door and into the hands of election officials throughout the country.
One of my other interests has been in the area of design of polling
place materials, ballots, instructional materials, I have a long history
in working with literacy groups for many years. it is a major problem in
our country. It is a problem that we frankly do not like to talk about.
it is the greatest nation in the world but yet our rate of literacy, I
will literacy in this con try is abyss mall. it is awful. we don\'t like
to talk about it as a powerful, well educated, strong nation, but it is
an interest that I think is absolutely necessary for us to look at in
terms of the voting population.

If we look across the board out in our country, we see materials being
developed that are unreadable, hard to understand, hard to comprehend by
many of those who are taking part of our election process. It is
absolutely necessary that we try to do something about that. We have
contracted with a competent organization- Design for Democracy to do a
lot of work in this area of looking at ballot did he, structure and the
flow and design of voter materials so we can do a better job of getting
that information out. There is another area in addition to I will
literacy that is a big concern of mine and many other people who deal
with trying to educate the public, and that is where we are today in our
society. We are overwhelmed with information, stop and think about it.
When you go to your mailbox every day, you look at all the stuff shoved
in the mailbox that people want you to read. We are constantly in a
barrage of the information age. Look as you look across this room people
have computers out, they\'re still \-- they\'ve become part of our daily
lives. Information is thrown at us on a daily basis. We become
accustomed to do reading parts of things or sorting out things we get
that we really want to look at. My, this looks like something I want to
read. It has been designed well and presented well. That\'s what we\'re
trying to do in this whole area of taking a look at what\'s out there
and coming up with some best practices to assist election officials in
getting well organized, well designed information out to the public. We
are also asking them to take a look at the voter registration document
so that we can make that easier to look at and easier to read, and I see
my good friend over at the end of the table there, Alice Miller back
when we were resigning our primary voter registration document in New
York, we took a lot of information from the DC board of elections from
which at the time was one of the most well designed voter registration
forms in the country. In many states including my own continue to use
some of the hard work they did with ill literacy groups tenor fifteen
years ago in helping redesign the form so it was much easier to
understand and read. We\'re going to work hard in that area. One of the
other areas that is of great interest to us because it is absolutely the
hallmark of everything we do in elections, you know, I often said that
election officials work very hard throughout the year and with HAVA
they\'re even working harder. The decisions are harder. New equipment,
new training, new everything that they have been faced with over the
last couple of years, and on election day, we simply turn all of that
hard work over to a group of people that works one or two days a year:
our poll workers and judges out in the field. So it is absolutely
necessary that we try to provide the best resources that we can bring
together to assist local election officials in the area of poll worker
training and poll worker recruitment. We have to do a lot more in the
effort of getting more people out there to work. It is getting very
difficult. As new equipment comes along, a lot of the people that have
been working at the polls for twenty, thirty years say we don\'t want to
deal with this new equipment. We can\'t use it. We\'re confused. We\'re
not going to do it any more. We must look at ways to do better jobs of
recruiting through our local organizations and through all kinds of
activity we can do. We are currently working with a couple of groups
under contract to come out with best practices in the area of poll
worker recruitment and training. We are also working with Cleveland
state university in the area of some work with college poll worker
training. We know that\'s our future. I wish we had a lot more money to
spend on that program. That is where the effort needs to be made. We
need to get our young folks interested in the election process. There is
no better way to do that than to recruit them to work at our polling
places on Election Day. I have physically seen in my travels around the
country where they have utilized this type of a program in the colleges
and in high schools the students really love doing it. It peaks their
interest, and they do a great job. With other research areas we are
currently in the process of coming out with and we will be getting a
status report next month at our meeting is vote count and recounts. I
know you all witnessed through the news the elections held in Washington
state and here in our own area in Virginia for the office of attorney
general where we had very close raises and we had to go through very
difficult recount process, and so we\'re hoping to gather information
best practices, research data, and that all area so we can make that
available to our state and local officials to make their life a little
bit easier. One of the things we look at as we\'re moving down the road
and growing and growing up as a small new agency, is in the area of our
clearing house activities. We are \-- we recognize that congress in
creating the help America vote act made one of our primary
responsibilities a clearing house for everything you need to know about
elections, election data, election information, and it is our goal over
the next several years as we move along and as we become better
acclimated to what is out there that we will be able to provide and I
like to say this to the folks we visit on the hill that we will be able
to be the number one place in America to go for anything you want to
know about elections.

There is a lot of data out there on legal resources and legal litigation
that\'s happened across the country. I can tell you when I was a state
election director trying to get a handle on if you were in litigation
what was going on around the country, was there similar litigation, how
could we put our hands on it? What if we wanted to know about a certain
law in states because our legislature was looking at making a change in
law. We would have to go through this process to try to find litigation
or some piece of statutory information that we could put our hands on
quickly. So we have entered into a contract with Florida State
University to provide us with the beginnings of a legal resource
clearing house where you can do one stop shopping on our website for any
piece of litigation, state election laws, federal election laws, that
type of information that you can get it quickly. Another area that has
blossomed over the last few years is in the area of a public access
portal. That\'s again one-stop shop where you can call on election day,
find out where you\'re registered to vote, get information on whose on
the ballot, where your polling praise is, so on and so forth. We\'re
taking a look through a contract with Publis out of Detroit who has done
a lot of work with the state of Michigan over the years on effective use
of public access portal, what do we need to know, what kind of
recommendations and best practices do we have to recommend to our states
to really make them good and usable, and so we\'re very excited about
that. Those are some of the things we\'re doing. We are continuing to go
look at other areas. We\'re finishing up the \'06 research activities
and I think the next time we\'re together hopefully I will have the
opportunity to go through them with you. I want to again share my deep
thanks and appreciation for the work that you\'re doing. I know the hard
work from first hand experience, the hard work you are doing. I again
want to express my appreciation to the staff at NIST for the good solid
working relationship we have, and I know that it will continue and
because of that relationship this process will be very successful. Thank
you. \[applause\]

Any questions or comments

J.R. Harding?

I am not sure where to go. That was an awful lot of information and I
thank the speakers for the overview. I would like to hone in on
accessibility issues and specifically the talk of new research, perhaps
where are we with the "shoulds" and the "shalls", with what was the
threshold or justifications of moving some of our issues from one to the
other and where might we be in the future on some of that? How might we
deal with the literacy right or the cognitive issue if you might put it
in disability language? Specifically, some of the outrEach and what
might we plan to extricate from the November 2006 activities and what
programs or things we might be able to have in some of the states we
know will do very well versus states that might not do as well, and then
where might the middle line be and after some discussion, Mr. Chairman,
I would like to spruce a\-- introduce a motion regarding out rEach in
education specifically with the disability community, and I would like
to put that on account of everyone\'s radar at the moment and allow the
conversation to develop. Thank you, sir.

Mr. Chair, since Mr. Harding started this, I had an issue I was thinking
about during Mr. Skall's presentation, but I would like to perhaps get
it on the table. They\'re looking at in the future working at test
suites which will cover the three areas \-- which will cover the three
areas. My concern is that as the states throughout the country become
more conscious of the importance of testing and begin efforts to do
their own certification testing, testing is becoming unacceptably
expensive for the vendors and through trickle down it will become
unacceptably expensive for the taxpayers. The expense really is in a
function of the thoroughness of the exams, but the simple fact that the
number of jurisdictions are increasing the amount of testing systems
have to go undergo almost non-routinely and that is getting very
expensive for the nation. I would like to see and it is something there
has been a fair amount of resistance in circles to in the past. I would
like to see the federal test standards as they are developed rEach more
into state specific requirements. I would like to see the functions that
a system has designed into it exercised if they exist, and if there is
no other standard for the performance, then the design standards for
them will dictate. We need to create an environment where states can
rely heavily on the federal testing that\'s done and start limits state
certification to just those state specific issues that for some reason
or another cannot be covered in the federal program.

I would like to support Paul's statement on \-- I have done a
significant amount of state level testing and understand what he is
saying, and one of the things that we really need to work on is this
business of designing the subsequent test for these requirements, and I
don\'t think f exact statistics exist on this. Back prior to this
activity when this was pretty much a NIST voluntary effort we had
something like 35 or 40 states that voluntarily signed up for this
program. We knew for a fact that a lot of those states did no state
level testing. They simply accepted the system as it came from the
ITA\'s.

Do you have particulars of resolution that you want to put forth or this
is just you wanted to make this part of the record?

I wanted to make this part of the record, and as I understand it, you
all are still trying to effectively do your needs assessment for the
testing standards that Mr. Skall is going to be developing. I want to
put it on the record and make staff well aware that at least from my
part that\'s one of the things I will look very closely at

Okay. Mr. Berger

Mr. Chairman, I would like to add my support for this area of
discussion. I think it was Merl King who made the comment once in the
hearing where this is a field we work a week on the issue and worry for
a month what we broke something unintentionally, and I think we\'re well
advice to do worry significantly about unintended consequences. The cost
of testing is one that I have. What we\'re doing in the cost of
implementation is another one. Equally then what we\'re allowing not to
go undone. As we make changes, I very much am interested in knowing what
the cost of the testing is that we\'re requiring, but also get some
feedback on what the cost of implementation is so that we are being as
intentional as possible about directing efforts system wide.

Dr. Schutzer

I would like to support it also. We had a parallel in the banking
industry. We all have to go through certification and testing through
our third party vendors and systems that we use, and the way we had been
approaching it up until a year ago was exactly the way you\'ve been
saddled right now. We all had our unique needs and nuances in the
banking community. We got together with the big four accounting firms,
and we worked together to develop detailed testing criteria that could
be done once, and we actually testing it back with our banks and of
course we did have some unique testing requirements yet to be done

Whether this system completes testing and is deployed in the field, you
have to verify in the field what you have is what was tested. That\'s
not a task either.

(Stand by for change in captioners.)

My impression was that the subcommittees were not expecting a blanket
approval by the TGDC. I think the idea was to put that up there for \--
to discuss \-- to start the discussion, so do speaking, am I right?

I don\'t understand when we\'re going to have these discussions. We get
a four-inch pile of paper. We have a day to go through stuff. We never
really have any chance to discuss it in detail. Sort of reviewed at an
overview level, but we\'ve been \-- we just heard Mr. Berger talk about
the unintended consequences. I know one thing that this staff and the
privacy subcommittee have been concern aud bout is the accessible,
usability, and security which have obviously trade-offs that is to be
made, and it would be bet per we had a way during the course of the
development of these modules to cross communicate some of what we\'re
talking about, so it\'s not just presented as one giant wedge, because
if what we\'ve done is publish something, I don\'t see how we\'re ever
going to go back and seriously revisit it when it has been published as
a working module and then in June we\'ll vote on 10 or 15 of these.

Mark, would you like to comment?

Yeah, the intent was not to imply rubber stamping or blanket
endorsement. The intent was to try to come up with a way that when we
work closely wait subcommittee, and there\'s some sort of meetings of
the minds, that we could at least get this on the website, so they know
this is the direction we\'re going. Clearly there would have to be
caveats by the entire TGDC. We would hope to put it on the website and
get comments, but it doesn\'t seem to happen without a meeting where you
can actually have face to face interaction, but the idea is why not
allow the public and the vendors to see the direction we\'re going in.
This happens with standards all the times. Drafts are made publicly
available, and if vendors choose to implement them, they know there\'s a
risk they may change before they\'re agreed upon. But it seems to me
it\'s in line with sharing information and being as transparent as we
can be to put these on there when we think there\'s a meeting of the
minds. Now, clearly it would be better if the meeting was a broader
meeting of the minds and the TGDC would at least electronically send in
comments that we could vet. On the other side of the picture is the
option to not make it available until the very end, and then there are
issues with that as well. So I understand your concern.

I'm not suggesting that the draft not be made available. I'm suggesting
that the communication materials among the TGDC working groups need to
be more transparent between the groups, and there needs to be a better
communication on the aggregate issues from the technical staff to the
various members, not a single subcommittee.

And I agree with that.

I would like to support that, too. I'll definitely have a comment and
then a recommendation. I think even though we have three separate
groups, they are really very interdependent. There\'s to doubt that
sometimes you bend over backward and accessibility may be sacrificed to
security. You may be sacrificing for security to maintain availability
which as we see in some of the recent can really defeat a lot of the
intent any how. The rest of these different modules, I understand they
are not going to be published as modules, but sometime, even if it\'s
not a face to face meeting, ought to be made where that information
could be available to the whole TGDC, and sometimes a conference call,
where election officials should be goaded to provide input from the
other sides, from just outside of our work to provide your input and
concerns, whatever, and I think the product would be better for it. It
doesn\'t mean that it shouldn\'t also be out on the website and have
vender and public comments and so forth. It certainly should. I would
recommend something in that schedule be devoted to that kind of
interaction.

Mr. Craft?

Yes, I would like to also go down, I guess, another aspect of this while
we\'re on the subject. There is a dearth of vender input in the process
that we went through the last time around. More recently, as the
subcommittees have started trying to deal with, and our committee,
setting a standard for how marginal marks are going to be handled, it
was very, very difficult to get vendor input into this process. I mean,
it\'s my understanding that with all the other industries that NIST
works with in setting standards, the industry that makes the product-
these guys are a very key part of the standard-setting process and
building standards. I really feel that it has been way too difficult for
us to get the vendors involved. None of the subcommittee meetings, to my
knowledge, have had a vendor participating in them. There are brilliant
minds in the vendors. There are people who have been working on some of
the problems that we\'re discussing here for years. Some of them have
done very creative work. There is of course an issue as to how much of
that they\'re willing to show, because some of it is trade secret
information, but I really feel that they need to be brought a little
farther into the circle as we go into our next iteration of how we\'re
going to regulate the industry.

I very much support the comments that Mark and Whitney have made about
open conditions and making the work very visible to all stake holders.
Maybe just consciously having a point in the process where we\'re not
saying this is the direction we\'re going, but she is a direction we\'re
discussing. Being more inviting of input, and I think, Paul, this is
your point. We very much need to know the vendor input and other stake
holder groups input.

We have to break at 10:45 for technical reasons. So this is the last
question.

Mr. Williams:

I'll be brief. The make up of this committee has concerned me from the
get go. I've been involved with all four standards developments in this
arena. The 1990fec, the 2002fec, the IEEE, and now this, and this is the
first one that didn\'t have a balanced mix of members of the election
community & vendors. We\'ve had venders actively participate in all of
our other three efforts as members of the committee. Not just incidental
people that we occasionally ask a question, and the balance of election
experience on this committee is \-- I feel is way out of kilter. Alice,
Helen and I are the only election officials -- we have got three
actually election officials out of the 14. So the makeup of this
committee is specified by law, so there\'s not much we can do about
that.

But there\'s nothing wrong with us forming working subgroups and
actively soliciting their input, not on a what do you think of this
basis, but on a more generic look at this and help us refine it.
There\'s nothing that says that we can\'t do that within the \-- within
the charter.

Okay. We have to take a break, but I think this is a topic that perhaps
can be discussed during the break, and if anybody wants to make a
specific recommendation or resolution, we\'ll take that up. So we will
take a break now. That means that John Wack's presentation will be after
the break, and we\'ll come back at 11:00 to start the next session.
Thank you.

(MORNING BREAK)

Can we all take our seats, please? That includes the TGDC members.

You have to set an example for the rest. Now is the time to put some
resolutions on the floor, right? Allen, maybe you can go out there and
chase them in?

TGDC members, if you could take your seat, we would like to start,
please. This may be an opportunity for resolutions. Okay. I think we\'ll
get going. There is a little change of plan, you know, to respond to the
comments made before, but before we get into a discussion of possible
resolutions, or the need for a fresh look for where we are versus, you
know, where we\'re going, or where the needs are, versus what the plans
are, and appropriate resource allocation, et cetera, I think some of
these questions may be addressed, or at least the discussion should be
held in light of what john Wack will present, so I think we\'ll go ahead
and have John Wack from the information technology laboratory to report
on the development status and supplemental guidance for the VVSG 2007,
and then open up the discussion, open up the floor for the discussion
after that. So unless there are any objections, I propose we proceed as
such. Is that \-- hear nothing objections, john, will you go ahead,
please?

Thank you very much. It is a pleasure and an honor to be up addressing
you again, and Whitney, you mentioned something about feel like a can
air

I a mine shaft, and after the discussion about resolutions, I have to
say I feel the same way a little bit. I'm going to change my
presentation slightly, and I think for the purposes of the discussion
you want to have, maybe it would be best if I focused more on
essentially where are we in the development of the standards and review
some of the structure of the document, and the reason it looks the way
it does, and things of that sort, and then I think that might help you
out. So what I'll too is give you an overview of the volumes, a little
bit of the history. I'll review some material that we went over last
September, and go on to some next steps. Okay. Last September I gave you
a presentation and we talked about VVSG 2007, and I'll go through some
of that material again. We looked at basically iso standards , and
decided it was best if we broke apart the standard, and we actually
separated distinct parts of to standards into separate volumes. So if
you look up there, and it tracks basically as a standalone- tracks to
the overall standards, and it will contain overviews of a variety of
different things that essentially will be very usable to the general
public, as well as, you know, all members of the election official
community. Terminology standard you\'ve seen already parts of that, and
that is the glossary. The product standard really has requirements for
voting systems in there. Standard on data to be provided has
requirements for test labs, and vendors, in terms of documentation. And
then a separate testing standard that with the inclusion of some
additional funding will be filled out with actual test suites, and those
tests will actually be referenced by requirements in the product
standard. Now, what I'm going to do here very quickly is just show you
some of the topics that we are covering. Miss this is not an actual
outline with sections of the document, but what I've done is taken the
outline that we presented to you last September, and I've done a little
bit of color coding of it to show you topics that we have addressed to
some measure, and those items in red, actually, which I'll read out, for
example, auditing assumptions, that\'s basically material that we will
present at some point today. So you can see looking at the introduction
some of the material we\'re going to include in that, and the
introduction will be pretty much our next focus. Some of the discussion
prior to my appearance here on stage has been frankly about making the
material usable for the TGDC as well as other communities, and that\'s
become apparent to us as well, that a bigger part of our job, as we\'ve
developed more material, actually is to put it all together and I think
the next job we have to do is essentially show you on some newer pages
how the document all hangs together, what it\'s starting to look like,
so that, well, essentially what I would like is for a member of let\'s
just say the f hp subcommittee to be able to at a glance take a look at
the document as a whole, and look at other sections, look at the core
requirements areas, or the security areas, and be able to see how it
ties in, and how f hp is being promulgated in those areas as well, and
the same would hold true for the other subcommittees as well. Okay.
Product standard today will also have material on the conformance
clause, generally requirements on Crypto access control. We\'ll present
something on usability. Hardware requirements. One thing I want to point
out is that the product standard is divided into two general area, and
those are general requirements, and then on the next page, requirements
by voting activity, and part of the reason for doing that is to minimize
duplication of requirements. Basically, requirements by voting activity
will likely reference requirements in the more general section, and so
we basically make the document a little easier to follow through, and we
don\'t have to keep repeating requirements, we can just reference them.
I'll just point out we have casting and counting, and some vv pat
material that we\'ll present today as well. Standards on data to be
provided: We\'ve touched on that as well. And don\'t really need to go
into too much detail with that today, because we really aren\'t
presenting material there. Certification test plan, data to be provided
to reference software libraries, areas we have to address. And then the
testing standard. And the testing standard at this point we\'ve got some
material developed for it. It\'s high level material, and I want to
point out that one perspective you might take with the VVSG 2007 is that
there really isn\'t a whole lot of new stuff in there. Basically it\'s
taking what we did for VVSG 2005 and other material and digging down
deeper, being much more accurate and specific in the presentation, but I
think one of the most important things we\'re doing with this standard
is basically making things clear. We will do our best to address the
most important topics in voting and get it done by July of 2007. But
it\'s very important that we provide a document that\'s maintainable,
that can be updated, and that will be very usable to our primary
audiences, and we consider those to be vendors and testers. But at the
same time, it has to be very usable to \-- well, I'll start with the
TGDC and the election community. The testing standard is important.
Basically it\'s going to hopefully include test, and Each requirement
will point to a test, if things work out, and we think that that\'s an
extremely important aspect of the standard. That one of things I've
heard from vendors is they would like to know how requirements are
actually going to be tropical stormed, and I think that it essential as
well for the voting system test labs, so the test labs have common
guidance and common requirements and common language on developing
tests. So right now we have overview material, and we can point to that
with our requirements given the funding and the further development work
we would like to actually have the specific tests in there. Okay. Very
briefly. Who is going to use the VVSG? Well, I've already talked about
vendors, and I've already talked about test labs, but we know that
states are going to be using it, election officials, people doing RFPs,
buying voting systems, researchers will be using it. The general public,
and we\'ve already talked about basically making this standard usable,
very usable not only to the vendors and testers, it for audiences as
well, and it\'s become apparently to us along the way we have been in
situations where we think it\'s important to develop some additional
supplemental guidance on some of the requirements. And I'm not really
talking about best practices so much as information that provides a
context for some of the requirements, and so we plan on adding this
material. In fact, we\'ve already started, but it is not material
that\'s testable, it\'s not material that anybody would have to follow.
They aren\'t actually election official procedures. We will include this
material as part of subsections containing requirements. I'll give you a
couple of examples of some of that material. For example, VVPAT on paper
spools, and you know that if you use a paper spool, there is a problem
in that votes are recorded sequentially on there, and it becomes easier
to determine the order in which voters use a particular machine. Some
contest, some supplemental guidance where there might be a
recommendation to maintain a certain level of security, there will
somebody additional procedures developed along the way. It\'s basically
as of this that a state in using a particular vv pat system may need to
and its own procedures as well. Some of this material, by the way, I
should mention, may find its way into best practices. Some may just be
too specific to certain requirements, and may stay in the VVSG. A couple
of other examples: Essentially things such as notation, using small
fonts on a spool. Or separation of certain types of voting systems with
audio input could be more easily overheard. Distribution of pass words
or security information, if pass words- then cryptographic keys,
certificates, whatever, are used on voting systems, and they have to be
distributed manually, there may be some supplemental guidance there.
Addressing a little bit about the format: Essentially we could do it a
couple of different ways. We may list it somewhat in the requirements
format, so that we can actually reference where we\'re actually getting
the information, the supplemental guidance. We also may just have
informative text in Each section. Okay. What I've tried to do in a
relatively short amount of time, and I guess I apologize that I actually
don\'t have hours to do this, because to digest all of the material that
we have, it would take about that length of time to present it to you. I
just want to wrap it up and say that we will be working more on
introductory material down the road, and we recognize a real need to not
only develop the requirements and do the research, but also to make it
usable to the TGDC as a whole. I think it will get more difficult for
the TGDC, though. I think that, you know, as a result of delving deep
entire the standards, we have more material, and it\'s going to be more
difficult for you. So the more we can talk about working together more
closely and getting this reviewed better, you know, we welcome that. Do
you have any quick questions before we go on?

Just a quick comment, just to illustrate \--

Keep if mind all of the members to identify themselves, please.

Dan Schutzer. If you jump back to page 13, for example, I think that we
do need greater interaction, because some of the things we\'ve been
learning, or some discussions, or even in a case like this where people
have talked about this capability primarily from a privacy and security
point of view, there are other things about that that really ought to be
put in, like, for example, guidelines for how to set up and test is to
insure that the accuracy is up to par- guidelines in terms of procedures
and handles of this in terms of contingencies, how to prevent paper from
being jammed, how to handle it and so forth. I think we need some
thorough review of these things, because we are sometimes identifying a
particular feature of a product from one point of view from one of these
into a total picture that we really could make it better if we could
include that. And of course I would go along with the fact that vender
input in areas like this would be extremely useful.

Mr. Craft?

Yes. John, one thing that I would like to see, and I don\'t know how we
get there, I will take one of the \-- one of the requirements that
we\'ve been kicking around for the last two years, is the ability of a
system to be \-- have its firm wear validated after the firm wear has
been voted. Now, that\'s an issue we\'ve been kicking around for two
years, where we\'ve been telling the vendors that they need to figure
out how to do it, it\'s going to be in the standard one day. That\'s an
issue that really, I think, should be paired off from a draft standard
into a research project, and that\'s an area that I think NIST has some
of the best resources in the world to work on this area. What are the
various kinds of firm wares that these vendors are working with, what
are the real technical issues in being able to validate installed firm
ware, and being able to do it in such a matter that you don\'t
compromise security? And then if NIST could bring back to the TGDC ,
that analysis showing us what specifically has to be done in Each of the
systems that are currently fielded, and the impact, then I think this
Committee could start making informed decisions on those kinds of
issues, but we can\'t \-- we can\'t go down the road of throwing out a
requirement like that, even though it\'s something those of us who have
dealt with the issue would love to see, and how do we get there?

John Wack: Well, if I could respond quickly. That\'s a good question,
how do we get there? We recently started a series of telecons with
vendors. We had one approximately three weeks ago, and we\'re going to
try to do them every month. And we had the major vendors there, and we
initially started off by identifying major issues for the vendors that
they wanted to talk about. That was one of them. So one suggestion was
essentially to start dedicating, you know, basically half a day on
specific subjects, that being one of them. We\'ve talked a little bit
about opportunities where we can have face to face meetings with the
vendor community at large. And maybe we can do that, for example, during
major voting meetings, such as the standards board, or something like
that. That\'s one way we can start building more research into that
area. It is difficult to actually, you know, get the other work done at
the same time, but I think at the part of our charter here, but at least
the start, though, is focusing on those issues with the vendors, and
with our research folks at the same time.

And I'm glad to hear that that kind of rye search is going on, and I
guess I'll follow that up with, is there a way that the NIST staff can
start involving some of us on the TGDC in some of those efforts, because
I would certainly love to sit in on a phone call in and of those
sessions and perhaps advance what we\'re doing.

I actually \-- I think that would be great. We can do that.

Mr. Berger?

John, thank you for your presentation, and I think you were right in
predicting you would address a number of concerns. I have a couple of
questions, one maybe is a bit more of a comment, but on page 5 of your
slides, you talk about their operability. There is a concern, and I
wonder if you all are working under it, we qualify on a number of points
and systems. I'm not sure we\'ve carefully specified what the limits on
replacing cots without additional qualification are. As an example, many
of the systems use PCs. The ITAs test them with a specific PC. I'm not
sure where we give the range of other models or other vendors PCs that
we would be comfortable are replaceable without independent evaluation.
Is that under work anywhere?

Dave Flater, could I point to you? Dave might be able to address that a
little bit better, since he\'s really dealing more with the cots issue.

Dr. Flater: There\'s a pragmatic approach to this, and then there\'s the
hard line testers approach to this. I don\'t know what the pragmatic
approach of ac with respect to how flexible the certifications are \--
is. I can tell you from the hard line tester's perspective, you certify
a particular system. It is a complete system. And any fixes you make to
that system could potentially break it. For example, substituting one
COTs PC for another PC should not break it, but it can, because if there
are race conditions in the system, putting in a faster or slower PC
could trigger those problems. That\'s just one example. So I acknowledge
that there needs to be some flexibility, because we can\'t send it back
every single time something is changed, but speaking as someone with a
lot of testing experience, I can tell you that\'s a very tricky issue
you address.

David, I share your concern, and I think it would be accurate to say
that\'s an area where there\'s a great deal of confusion, more
generally, so \--

Let me bring up a different item. John, you talked about looking at the
VVSG in terms of usability to vendors and testers as the primary
audience. And maybe this is more of a comment, but I'll make it a
question. Have we looked through the VVSG to its work product, the ITA
report, as to its usability but it intended audience, slat state and
local officials, as to how well this supports their efforts to then
state certify equipment, to get to the information they need to perform
their functions in elections?

The answer is we have \-- yeah, we have discussed that, and in the data
to be provided session, in the certification test plan, I \-- first of
a, in looking at this slide, it\'s not complete, it doesn\'t have
everything, but yeah that is an area that we have discussed. I think
again I'm going to \-- well, I don\'t know, Dave, if you want to address
that at all, but we recognize that, yeah, that report has to be
essentially made available and usable and understandable.

Mr. Craft? No.

This is Patrick Gannon. As a follow up to Steven's question that related
to the interoperability under general requirements, I would draw
attention to page 10, where you\'re talking about the test suite
overviews. I don\'t see any occasion there of interoperability testing,
and I would think if the need for interoperability across different
components that could be made by different vendors is one of the major
requirements, then there should be testing that would specifically
address the way to provide interoperability and to provide that, and
specifically focuses on the kinds of data interchange formats that might
be required for exchanging data between different vendor type systems
that is not feasible to just test all of the elements by a single
vendor, but where appropriate to provide test across different vendor
pieces as part of a larger system. Is this something that could be added
to page 9 and 10 to indicate the need for test scripts a round
interoperability testing?

Okay. Thank you. I've noted it.

Regarding the resolutions passed by the committee and the progress made
or work planned that are related to those. However you would like to
phrase it.

Okay. We did a bit of writing by committee. Okay.

Dr. Harding: Before we begin with the motions, I would like to give the
group in the audience some context of why we are doing this and why we
believe it is important, from my perspective. In Portland, I was asked
to represent this group in front of the advisory committee and give them
an overview of who were we and how we were contributing in this process?
Did we fight among ourselves? Specifically, they wanted to know where
were we coming from? I used our resolutions as a Bill of Rights analogy.
That this was at the heart of the issue. This was the expectation and
while many of them were philosophical, they, in fact, created certain
work products. If they wanted to follow the work of this group and see
if, in fact, the standards board and advisory boards and ultimately the
EAC were getting what we thought they should get, they should use the
resolutions as your checks and balances. That is why I raised it in
September and again why I raised it today. So, Paul, I would like to ask
you, if I could, Mr. Chairman, let the motion be read.

Thank you.

Okay. This is difficult from this angle. Shall prepare analysis and
regularly report on \-- that didn\'t work. Regularly report and
regularly prepare a report. Take out \"on.\" that resolutions passed by
the TGDC and the progress of standards development to the specific \--
that did not work at all. \[ laughter \] and the progress of standards
development. And to the specific work products of NIST. After the
initial publication, comma, reports will be provided to the TGDC with
the meeting materials prior to Each meeting and will be included as an
appendix to all NIST and TGDC work products sent to the election
assistance commission. Prior to Each meeting.

Prior to Each meeting.

I think if you just say \"reports will be provided to the TGDC prior to
Each meeting.\"

Okay.

Just to make it more readable.

What kind of time frame? \[ laughter \] well, what you think will be
reasonable particularly with the fast-paced and this is working on in
getting these materials together. A week? Okay.

It is when the rest of \--

When the rest of the materials are due. A week. Okay.

Okay. First of all, is that clear to everybody? Do we need to read it
again? Okay.

Can you read it for the record?

Okay. Could you read it again, Mr. Craft?

Shall prepare analysis and regularly prepare a report regularly passed
by the TGDC and the progress of standards development and \-- okay. And
the specific work product. We need to take that \"to\" out. \"to the
specific work products of NIST.\" it will be provided before Each
meeting and provided to the NIST and TGDC work products sent to the
elections assistance commission.

Do we have a second?

Second.

Any discussion? Comments? Yes.

Thank you, Mr. Chairman. I'm JR Harding . I believe this would also
compliment the work that john was eluding to. Maybe it is the other half
of the equation. It speaks to where are we going and where did the work
originate from?

Any other comments? Yes.

Dave Karmol. A question, I guess to the NIST staff. Does NIST staff
understand what is meant by the resolution? Obviously, they will have to
prepare it. Is this something that can be done in a matrix-type fashion
of a couple of pages? I think the last thing we need is another document
that is 20 or 30 pages long. I guess that is my only concern here. No, I
understand. That is the problem. I don\'t think we need another document
that is 40 or 50 pages long. Does the sponsor believe this is brief?

Frankly, there are a number of us on this board or at least I speak for
myself and j. R. Said his part. There are those on this board who do not
have a clear concept of how our work product from prior meetings has
flowed into the standards. And I just \-- I think that document need to
be created. If there are resolutions that are not flowing into the
standards for some reason, we need to know about that and the report
needs to show that. I'm sure there were things that possibly after
receiving public comment, the EAC took out the final moment for the
document that was published. How do we get it? There is no traceability
right now that I know of from our resolutions to the public standards
and it is very difficult from my perspective to conceptualize where NIST
is and the prior resolutions with that.

We understand the traceability comments. I'm with you on that. In that
spirit, may I suggest the word analysis to me means, volume things. May
I suggest NIST shall prepare a brief report and \-- I will take out the
word \"analysis.\" analysis to me says a lot of studies and this and
that.

You can cut from there to that.

Yeah.

Mr. Chairman, if I may. I would like to speak up. Sharon and her staff
did this for the human factors committee. Went through the final version
that we voted on between there and January 12th and have been continuing
to update us on work they are doing and continuing to review comments
with the EAC. Perhaps there are examples there.

I thought we actually had presented some things along about those that
are not comprehensive.

Back to my comments about the cross reference of the sub committees.
Perhaps some were interested in that.

With the modification, is that acceptable to the original authors of the
resolution?

If I may, I guess I didn\'t hear, were you responding on behalf of the
staff on terms of what was understood?

When I saw the word \"analysis\" that made me worry. It sounded like for
EAC resolution, we would write three pages. I think the idea of a
matrix, these are the resolutions and this is the work product. You
know, maybe a page or heading in the standard that says this is what \--
this is what that material addresses with regard to the resolution.

Just as a friendly amendment here. Because I noticed there are some 40
resolutions and many of these are structural resolutions. How will we
structure the work? I don\'t think we need a report that says we have
three sub committees. We don\'t need a report on all of these. I would
suggest adding the word \"relevant\" resolutions. Some of these \--

I think since that hasn\'t been done up to now, it would be good to
start which of the prior resolutions have now been cleared and
implemented. We can go forward with records.

Some of them may not have follow-up. It may have been in the nature of
the resolution that it was more of a discussion. So, are we comfortable
with this? Anymore discussions or anymore changes? Hearing none, those
all in favor of the resolution? Any opposed?

\[all state aye\]

Let\'s see. Sharon Turner Buie, are you on the phone?

Yes, I am.

Thank you. We hadn\'t \-- I understand you are following the discussion,
but we could not hear you.

Yes, I was \--

Did you capture this resolution?

I did.

Are you in favor.

Yes, I am.

Thank you. The resolution passes unanimously. Okay. The next item we
will take was the discussion we started earlier. I believe Mr.Berger
will lead this discussion.

Okay. Thank you. As I said earlier, I think much of the discussion that
has taken place this morning is in this direction. I think it probably
serves us well to take a look at our work and I might frame it as, are
we being as focused and responsive to the concerns we heard expressed by
Commissioner Davidson and tom wilkey as well as from the NIST staff as
to where we best apply resources today. I'm not sure I have any answers.
In fact, I'm sure I don\'t have final answers. I have a number of areas
where I'm concerned. I would just launch with this observation. In my
career, the things that have caused the greatest problems are those
areas that I wasn\'t working on. So, my question is \-- \[ laughter \]
as we work diligently, and we recognize there is work that needs to be
done in all areas we are engaged in. Are we looking at the things that
we are collectively overlooking that may come back to hurt the elections
system? As an example, some of these have been mentioned this morning.
Is it very clear in the ITA report so that those who get these systems
exactly what system the ITA tested, software and hardware with enough
specificity and detail that they can say they are working with the same
that they are qualified in in a acceptance testing. That would be an
example. As we look to improving the national qualification testing, it
has been mentioned, have we really given guidance on what acceptance
testing should be performed? What pre-election testing should be
performed to ensure the end product of the election is as solid as
accurate as possible? I'm really asking for a resource and focus
discussion. I'm not going to try and give answers, but maybe stop and
see what others might care to contribute.

Any other comments? Dr. Ronald Rivest.

Certainly things like vote by mail are increasing in use. It is not
clear we have done those to address the vote by mail systems. Another
issue that was raised, by Ted Selker and Mike Alvarez, statewide voter
regulations systems. Are we doing enough to set standards there. I'll
circulate this memo to you. We are not looking with much anticipation
for those items.

Actually, there was a resolution that as time permitted, there were
other aspects that we should be looking at besides just what was
encompassed in the scope of the specs. I believe registration was one of
them. It would be in keeping if you track those resolutions, we have
analysis paper that is due us in that area. Among some other things.

Britt Williams: Voter registration is outside our scope. Within our
scope is how the voter registration fits into the voting system.

Any other comments?

I'll wait for the gentleman to stop. Their technical issues \-- they are
technical issues. I'll get back to the heartbeat of the voter.

Whitney Quesenbery.

I have never seen an ITA report. As I understand them now, they
basically say yes or no. Without report if they are available to
designated people. I can see as we move into accessibility standards,
there are a number of them where having enough information about how
those tests were conducted would help any expert hired by the state to
review a certification would be useful. I know we put off a lot of the
testing until later and maybe later is coming.

Dr. Dan Schutzer: there was concern about trying to include more of the
state\'s requirements. When we attempted that same thing in banking, we
could not live with the results. It wasn\'t clear that each bank would
interpret the results the same way. I believe it is the same, but the
state needs the details of the testing information. You will try to
minimize the work of each state by allowing them to rely bulk upon the
national testing.

Mr.berger.

Steve Berger: We need to to ask the question, what are these reports
trying to do? One is so that other experts can look at the tests and
develop an independent judgment on whether that test performed the test
it was intended for. Other purposes for the report are for state
officials to look at the test \-- officials to look at the test so they
are additional concerns, they can add to the \-- do their own testing so
they understand what was and what was not done. I think the issue of the
availability of the reports is a separate one, but important one. Very
important one.

Britt Williams: On this - On the question of the ITA reports. The sticky
wicket there, it is proper priority to the vendors. They contract with
the vendor to do the testing. The reports are proprietary. There should
be a report that says public report. We need from this committee to
specify the content of that public report.

Mr. Craft: We had the same issue. In other words, if I contracted with a
testing authority to investigate a system that was considered private.
We got a look at it in legal, I know the aspects of it. We get the
vendor to agree to release the report to another bank to look at it for
purposes of their auditing. It saves them time and money. They don\'t
have to go through the same test again. You could work out something
similar to that, too.

Britt Williams: That happens now. Any jurisdiction that is considering
buying that system has no problem whatsoever getting the vendor to
release the reports. What I'm talking about is a report that would be
released publicly without the vendors having to approve every single
release of the report.

That is something we can certainly think about and make recommendations.
It seems to me that keeping such a report doesn\'t serve any purposes.
In most cases, the tests are being done for the vendor\'s benefit, but
the election community. To have a two-step process where first they have
proprietary report and you have to release it and an agreed upon format
or content that is expected to be released for public consumption would
cut a lot \--

I guess as an example, the reports will contain a list of anomalies and
how those were resolved. They were not present in the end system. How
they were resolved frequently gets into the actual structure and
internal design of the system. Obviously, a vendor would not want that
in the newspaper. The fact that there was an inn determination and
appropriate testing the system met the standards would be in the public
report.

Certainly, you don\'t want making public the interim reports or
negotiations or whatever. The final result or whatever or however the
vendor responded to the short comings, those may be identified. Are we
comfortable, mark? Getting involved in such a discussion?

Dr. Harding: Yeah, I think we certainly agree with the intent of
everything that is being said. I would like to remind everyone that we
certainly put in the standards that these are available. Where the
rubber meets the road is where, I think if the EAC and certification
imposes this, then the contracts agreements can be essentially swayed if
they say certification shall be granted if the aspects are made publicly
available, that is how we put teeth into what we are working with.

Dr. Semerjian: Am I hearing that this is reading perhaps more a
recommendation to be made to the EAC rather than to NIST?

No, I think it will have to be a reporting standard. All your evaluation
standards have standards for reporting. We will have to address that in
the reporting standards.

Mr. Williams?

Let me see if I can summarize this. We have to laundry list of things
that everybody is nodding their head. If we address these things, we can
have an immediate impact on elections. Instead we are using our
resources to drive the creation of another voting systems standards
document . The voting systems standards we have in place are adequate
right now. What we may be saying is we may need to change our focus a
little bit and do an analysis on what can we focus on-what will have an
immediate benefit on elections and back off on using all of our
resources to continue to refine technical standards.

Mr. Berger?

I think the only fair approach is to prioritize what you are suggesting,
dr. Williams. If we want renewed focus on some areas to bring quick and
effective improvement, we need to equally say we are reducing priority
on other items. I think we also need to ask the question how we might
more effectively bring in wider stake holder input to the process. A lot
of these \-- yeah.

Mr. Chairman?

Yes, go ahead.

John Gale. Secretary of State for Nebraska: I'm the highest election
official on the committee. I have been listening closely to the
discussion and I am fairly new to the committee. What I may say may be
redundant to the other discussions. It seems to me that science, which
is what you are involved in, is driven toward perfection and politics is
the art of the possible practical. If we drive this scientific
perfection, that may accomplish certainty. It may result in equipment
that can not be produced or states cannot afford it. There is a balance
between a standard that we try to set and vendors can live with and
price and states can determine whether or not that equipment is going to
be feasible for their various counties to purchase and to use. For
smaller states, there are 2/3rds of the states that rely heavily on the
standards whether they were the 2002 standards or the 2005 standards,
but these are the voluntary voting system guidelines. These are not
federally mandated guidelines. We are not trying to do a national
mandate that everybody has to comply with whatever we come up with. If
we make this so difficult and impossible and so unwieldy in terms of
cost, every state will have to have its certification process, which
will be different, but more practical for the equipment and vendors to
supply the equipment. A drive on every issue fails the vendors by making
it impossible for them to produce a product to law them a profit and
market. It may be particularly for the smaller 2/3rds states, they
cannot afford the equipment because the standards are way too high for
affordable for their practical use in their states. I'm trying to
balance the discussion in the terms of the art of certainly and the art
of possible in what we are trying to accomplish. I think the vendors
have to know. I remember the discussion in our state about whether we
were going to go with DREs or paper ballots in some form. Then the
discussion came up about the voter verifiable paper audit trail. We knew
it added \$5 million for compliance with the need for handicapped and
visually impaired equipment in each district. That made a difference in
our approach on that issue. I don\'t know \-- I think we have to remain
conscious all the time of the fiscal impact of what we do. I don\'t know
that we can add a fiscal note to each of the additional requirements
that we want to impose, but if we impose so many requirements that the
fiscal \-- requirements that the fiscal element is ignored; we have not
accomplished anything because states won\'t follow those voluntary
standards because they are impossible to follow.

May I ask a clarification?

Whitney Quesenbery.

This is a question, not a statement. Are you talking about the \-- if
the content of the VVSG requirements makes the equipment too expensive
to purchase or the test requirements for the states would be too great a
burden or something else entirely?

Well, I guess what I'm saying is it seems like our standards, since they
are voluntary standards, have to permit some flexibility. Some choices
of vendors in terms of the quality of product. Are they trying to
produce a Chevrolet for Nebraska or Cadillac for New York? What is it? A
minimum standard or maximum standard? If it is minimum, they can produce
a lower-quality piece of equipment that will work well in the great
plain states. In California, they need something more that can be
afforded by those states. If our standard is a standard to try to meet
the needs of New York City or los Angeles, Arthur county, Nebraska is in
a different world in terms of the affordability of that equipment that
meets that standard.

It is past 12:00. We have one other presentation. I think with Mr.
Berger\'s concurrence, I would like to propose that we think about \-- I
think there were a lot of important points made here. Think about those
and at the end of the day, let\'s talk about motions to be put on the
table if there is a general feeling of a resolution to be proposed. We
can do that at that time. I would like to proceed with the
presentations. Last word, Dr. Harding.

I would like professional courtesy to sneak one more in because it is
the appropriate time regarding our Commissioners' comments. The EAC\'s
director and the general heart beat around the table which was, we have
a pretty good document that has evolved significantly in the history of
our voting. The question really is, how is this document going to play
out in our communities and can we take the content of that, identify
what states are going to live up to these expectations in the \'06
round? Could we identify those things for an action item \--
implementation and action items for the next round? I would like to
speak to the disabled community and, if I could, Mr. Chairman, we would
like to introduce a small resolution regarding the TGDC and the EAC with
outreach for the disabled community.

Can we do this at the end of the day?

We could.

Dr. Semerjian: If you don\'t mind, I would like to proceed. We will have
two presentations and we are now going to have one presentation. Is that
right? Before lunch so we don\'t fall too far behind our schedule. I
promise, we will have the opportunity for you to present your
resolution.

Thank you.

At this point, we will have the first of two presentations. I call on
Dr. Alan Goldfine to prevent part of the core requirements and committee
reports. And after \-- we have to break at 12:30. Otherwise, there will
not be lunch let out there. That \-- left out there. \[ laughter \] we
will basically simply have the presentation and then break for lunch and
then when we come back, we will have dr. Flater\'s presentation and the
discussion.

Okay. Thank you. It says part two, but it is part one. What I was
planning to do was a brief overview with the attempt not to get bogged
down into technical details. I'm going to proceed along those lines.
Okay. I'm going to talk very briefly about what we are doing with
respect to a number of areas of requirements within the crt group,
electrical, rf requirements and performance requirements and workmanship
requirements in general. Quality assurance and configuration management.
A brief discussion of future work in this area with the little road map
as to what to expect first. Then, if, in fact, it there is time left at
the end, some discussion time. Okay. In terms of electrical and radio
frequency requirements. We are, in fact, looking at them with the
attempt to update them. These requirements were, in fact, updated for
the VVSG for the comments received during the public review. Most of the
changes were rather minor in terms of values and terminology and so on.
We are looking at it from a slightly broader perspective to try to
reflect the latest available information in these areas to reference
applicable standards, rather than repeating or ex-ending text from the
\-- excerpting standards that were done in the existing standard. To
also clearly separate requirements from testing specifications, again,
two things that got blurred. We are working on the requirements; testing
specs come later as part of the separate document. Finally, to
distinguish in this area between requirements that are, in fact, unique
to voting devices as opposed to requirements on any electrical device.
If we are talking about an fcc requirement on electromagnetic emissions,
that is on all devices. Are there requirements \-- necessary
requirements that are specific to voting devices? That is the area that
we really need to specify and more important, to test to. Or,
performance and workmanship requirements in general. After the
completion of the December VVSG, we then made a rather major effort to
go through the entire collection of public review comments. I heard a
figure of \$6,000. I'm not sure of the number ever stabilized. We
stepped through each requirement, not just the ones that were considered
to be carryover to try to extract to discover and analyze any of them
which would be relevant to our rethinking of the VVSG. So, based on that
analysis, we did make a number of revisions to the VVSG. For example, we
removed the availability requirements. The model that had been used for
this which factored in repair time of voting devices was unrealistic and
not really \-- we felt not helpful in achieving the goal of reliability.
Reliability of equipment is now an even more central requirement defines
solely through the concept of meantime and failure. More on that in a
moment. The point is that revisions were made based on what we extracted
from the public review comments. Many of the requirements that we had
were, in fact, moved to the other part of the CRT report. Cast, count
and report. Okay, quality assurance and configuration management. This
is not a new issue. It was identified by a previous TGDC resolution. The
current text \-- it is not that the current text is poor or totally
inadequate, it provides general goals and good practices. Unfortunately,
it is mostly not specific to voting systems. It is also not very
explicit and not very amenable to verification to testing to
certification and that sort of thing. We are actively involved in
establishing dialogues with relevant parties. I know I had dialogues
with a number of people on the TGDC and several vendors regarding what
is being done in this area? What is the appropriate approach to take?
Okay. Future work. Okay. The first major step is to finalize work
requirements. There are still open questions in the document that is up
on the web. Most of those, though, have default solutions attached to
them. They simmered for a while. We talked to people. We had dialogued.
We asked questions. We reached a point on all of them where it is time
to fish or cut bait. We will resolve them. We will complete the
revision, as I said, of the electrical and rf requirements and create
\-- I'm grouping the first three together, create a single CRT document
via a merger with David Flater\'s documents. A single CRT document to
look at. These are the most immediate work tasks. I think these should
pretty much be complete in a coherent draft form within the next couple
of months. They will be before the next TGDC meeting in the next couple
of months. The products will be out there on the web. We will be
publicizing them as much as possible. After that, we need a lot of
informative text regarding these areas. That remains to be written.
There are a number of places, as I was reviewing the documents, where
there are still notations to the effect that coordination or integration
needs to occur with the other two sub groups with sts and hfp where, we
can proceed to a certain extent, but questions, you know, that need
their input still remain. This is, again, something that will be done in
the immediate months ahead. I mentioned the reliability requirement
issue. I don\'t want to get too deeply into it. The time between voting
systems failing---163 hours of \-- shall be 163 hours of duration. We
are not sure where the number of 163 came from. The feeling that we have
it is probably too small a number, but should the number be increased?
To what? You begin to run into all sorts of affordability issues here as
well. You know, you could test forever, but is that practical from a
cost point of view? Again, this will be the focus of a significant
evaluation and analysis in the months ahead. And, finally, the quality
assurance and configuration management. All sorts of different aspects
of looking to this should a multiple published standards be adopted. The
ISO 9000 standards series. Or the ideas within published standards be
adopted. You are inventing your own standard. These are questions that
we have been asking and trying to come up with a consensus for. One
thing that may become necessary for this particular issue is, you know,
this may begin to sort of veer on to the policies and procedures
question. You know, we may say, what does the EAC \-- how would the EAC
weigh in on this? Beyond all this, as we indicated before, we need to
develop the draft standards on a date to be provided and develop the
draft testing standards as well. Those have been indicated. Longer range
issues. Although, the long-range has become shorter and short as Each
month goes by. Okay. That is a quick overview of what I had to say. Is
there any discussion on this?

Any quick questions?

No questions. Comments.

Mr. Craft.

The one item that stands out in future work is the configuration
management and system validation and I know I harped on this for a
couple of years, but that is the most critical item on this list. I feel
it needs more attention than it has been given. If every election\'s
administrator in this country is not capable of validating or getting to
consulting services that will help them validate their voting system and
proved it a certified system, all of this work is for nothing. There are
five people, I believe, in this room, who can go in with a high-rate of
reliability of the voting system. Three are in my company. As a business
perspective, I'm not disappointed in that, but as public policy, that is
a very bad thing. There are few people outside of this room who have
that expertise. I know we have done quite a bit of work with the
national software reference library and we had done when I was running
the program in Florida, we did quite a bit of work with the vendors on
system validation models. That is something that needs to evolve almost
beyond evolving as a standard. It needs to be developed as a process
jointly pushed forward by the EAC and the \-- EAC and systems. We have
to get together with the states and local jurisdiction can have
confidence that the system they are running is what they think they are
running and what they are legally supposed to be running.

Any other quick questions or comments? Mr. Berger.

On your last point, one question. You brought up the economic issue. Do
you currently have an idea what it costs to test to the VVSG 2005 as you
look to develop the test standards?

Right now, no. Obviously, you know, that is a crucial consideration
which can\'t be overlooked.

A question?

Yes.

Are all of the reliability requirements stated? Or the number of pages
printed or whatever?

For the most part, it centers around that mean time between failure.

The usage rate? Thanks.

JR Harding: Thank you, Mr. Chairman. We have less than a half dozen
people who are capable of certifying a system, we need to figure out how
our guidelines and specifications are expressed or articulated and a
methodology that gives a lay person confidence that whatever is in his
or her county meets this basic guideline. We talk about the election
manufacturers. They are the ones in the middle. Whatever we promulgate
cannot be understood by the average voter, then we have not done a lot
of service. I am troubled if we only have a half dozen people in the
country who can go to some city and bless this machine. If the
characteristics of a certified machine are not clear enough and are not
recognizable enough, then, ladies and gentlemen, we have not done a very
good job. Maybe we need to dumb it down. The average person needs to
know what a certified machine looks like. What the characteristics and
how he or she might be able to validate that in their own observations.

If I may comment.

Yes, Mr. Craft.

Actually, there are about a dozen of us. Which is still not enough. \[
laughter \] the elections community has been beat to death since 2000 by
the activist community with the one question, how do you know? How do
you know? How do you know? What if? How do you know? The answer to how
do you know is release control on system validation. That was the most
important undone task when we started this process two years ago. It is
still not to the point where local officials in the 4,000 elections
jurisdiction have a good answer for that question.

Go ahead.

Thank you, Mr. Chairman. If it is the system validation. The ability to
go in and look at the code running on the system.

Stop. You gave me two words a moment ago.

System validation.

There was another piece to it. Fine. Let\'s define system validation so
the lay person can be able to see that and quantify it. Whether it is
allen or alice, this is the process, put it out there.

That process.

Britt Williams:This situation is not quite as grim as Paul states it. He
is talking about the people who work for the vendors and it\'s election
officials. He is talking about the people available available to do the
work. There are a lot of people capable of doing it, they are vendors or
elected officials or people with a conflict of interest.

Mr. Berger. Last Comment.

I always love to have the last comment. Thank you. I would point out, if
we look at this again as a systems certification performance system.
There is an ISO 720 standard, you need them to do that. That is what
Paul is talking about. It serves us well to think about, have we done
the work to make sure that people know what they need to do at Each
function in the system?

Okay. With that, I would like to close this morning session. We will get
together again at 1:30. You are welcome to join me in dining room a or
b. This is for the TGDC members only and this staff. So, if you would
like to do that, you can get your lunch in the cafeteria. As you go from
here, cafeteria is on your right side. The dining room is on your left
side. The cafeteria and come and join in a private setting to have our
lunch and then we will start again at 1:30. Thank you.

(LUNCH BREAK)