1  
  
  
  
  
        1      TECHNICAL GUIDELINES DEVELOPMENT COMMITTEE (TGDC) MEETING
  
        2                        FRIDAY, JULY 9TH, 2004
  
        3               1225 New York Avenue, N.W., Suite 1100

        4                     WASHINGTON, D.C. 20037-2519  
  
        5  

        6      COMMITTEE MEMBERS:
           
        7           Dr. Arden Bement, Chairman  
                    Hon. Donnetta Davidson  
        8           Alice Miller  
                    Sharon Turner-Buie  
        9           Helen Purcell  
                    James Harding  
       10           James Elekes  
                    Anne Caldas  
       11           H. Stephen Berger  
                    Dr. Brittain Williams  
       12           Paul Craft  
                    Dr. Ronald Rivest  
       13           Dr. Daniel Schutzer (via telephone)  
                    Whitney Quesenbery  
       14
               Others:
       15
                    Allan Eustis, Project Leader Voting 
       16                System Standards  
                    Craig Burkhardt, Chief Counsel for Technology  
       17  
               U.S. ELECTION ASSISTANCE COMMISSION:
       18  
                    DeForest Soaries - Chairman  
       19           Gracia Hillman - Vice-Chair  
                    Ray Martinez  
       20           Paul Degregorio - Federal Officer for TGDC
              
       21      Taken by: LaDonna M. Woods a court reporter

       22                               -  0  -
                                                                      2  
  
  
  
  
        1      (The meeting was called to order at 9:05 o'clock, a.m.)  
  
        2                   CHAIRMAN BEMENT:  I'm Arden Bement, Director  
  
        3      of the National Institute of Standards and Technology, and  
  
        4      Chairman of the Committee of Guidelines and Development  
  
        5      Committee.  
  
        6                   I hereby call to order the first meeting of  
  
        7      this Committee today, July 9th, 2004.  
  
        8                   Public Law 107, the Help America Vote Act,  
  
        9      establishes the Technology and Guidelines Development  
  
       10      Committee.  
  
       11                   HAVA charters the members of this Committee  
  
       12      to assist the Election Assistance Commission in the  
  
       13      development of voluntary voting assistance guidelines.  
  
       14                   As Chairman, I am appointing Allan Eustis,  
  
       15      project director, and Craig Burkhardt, counsel and  
  
       16      parliamentarian.  
  
       17                   Allan currently is Project Leader for NIST's  
  
       18      Voting System Standards efforts.  
  
       19                   Craig is General Counsel for the Department  
  
       20      of Commerce's Technology Administration.  
  
       21                   Both Allan and Craig are here to make sure  
  
       22      the Committee operates smoothly.  They will provide  
                                                                      3  
  
  
  
  
        1      comments later on today.  
  
        2                   At this time, I'd like to call on  
  
        3      Mr. Burkhardt to determine the status of our members, and  
  
        4      also, to establish a quorum.  
  
        5                   MR. BURKHARDT:  Thank you, Dr. Bement.  I'm  
  
        6      Craig Burkhardt, and as Dr. Bement mentioned, I'm the Chief  
  
        7      Counsel for Technology matters at the Department of  
  
        8      Commerce, and I certainly bring you greetings from the  
  
        9      Commerce Secretary, Don Evans, and welcome those of you who  
  
       10      have journeyed to Washington, D.C. to be here.  
  
       11                   I'm on the team, as Dr. Bement will shortly  
  
       12      describe to you, to provide advice and support to the TGDC  
  
       13      and the EAC on a number of the items that you will be  
  
       14      pursuing over the next nine months.  
  
       15                   There are, in addition, lawyers working  
  
       16      directly with the EAC.  And one of the tasks which those  
  
       17      lawyers had to do was to collect from many of you, a  
  
       18      financial disclosure form, which is required by law.  
  
       19                   I will tell you the efforts that have been  
  
       20      made to very quickly distribute those forms, collect them  
  
       21      from you, and distribute and to review them, have been  
  
       22      extraordinary in their timeliness, given the fact that all  
                                                                      4  
  
  
  
  
        1      this has taken place in less than a week, when normally  
  
        2      these things can take up to a month or two.  
  
        3                   I need to announce that there are still yet  
  
        4      four people who the review process has not been completed  
  
        5      for by the lawyers working with the EAC.  Temporarily, they  
  
        6      will not be included on the roll calls for various formal  
  
        7      motions and votes that will be taken.  
  
        8                   I will tell you that, actually, as we speak,  
  
        9      that review process is ongoing.  So during the day, we may  
  
       10      be adding one or more of these four people to the roll call  
  
       11      officially.  
  
       12                   The four people who I need to advise should  
  
       13      restrain yourself from voting and from actively  
  
       14      participating in the conversations today until you are  
  
       15      added to the roll call would be Ms. Turner-Buie,  
  
       16      Dr. Rivest, Dr. Schutzer, and Mr. Gannon.  
  
       17                   The rest of you will be called on roll calls  
  
       18      and will also help make the quorum.  And as the other four  
  
       19      are finished in the review process, if you will, I may have  
  
       20      announcements from time to time during the day.  You are  
  
       21      certainly welcome to stay here, observe, and participate in  
  
       22      that particular manner.  And when we come to the  
                                                                      5  
  
  
  
  
        1      self-introductions part of the agenda, you will be  
  
        2      encouraged to participate in that portion of the agenda.  
  
        3                   In order to conduct this meeting, we need to  
  
        4      have eight voting members who, of course, have finished the  
  
        5      review process.  So I am going to go through, and I'm going  
  
        6      to read a roll call to establish our quorum.  
  
        7                   I might also say that when we take votes on  
  
        8      motions, which are of significant impact, we will take a  
  
        9      roll call, by which Dr. Bement will introduce me, and I  
  
       10      will just rapidly go through the roll call, and ask you to  
  
       11      cast an "aye" or a "nay" vote for formal recording, because  
  
       12      as a federal committee, we have to vote in public.  
  
       13                   A secret ballot may apply in the voting  
  
       14      booth, but it does not apply at the deliberations we'll be  
  
       15      engaging in before this Committee.  
  
       16                   So, for purposes of roll call, I'll say your  
  
       17      last names, and please announce by simply saying "present"  
  
       18      that you are here.  
  
       19                   Dr. Bement?  
  
       20                   CHAIRMAN BEMENT:  Present.  
  
       21                   MR. BURKHARDT:  Ms. Davidson?  
  
       22                   MS. DAVIDSON:  Present.  
                                                                      6  
  
  
  
  
        1                   MR. BURKHARDT:  Ms. Miller?  
  
        2                   MS. MILLER:  Present.  
  
        3                   MR. BURKHARDT:  Ms. Purcell?  
  
        4                   MS. PURCELL:  Present.  
  
        5                   MR. BURKHARDT:  Dr. Hardy?  
  
        6                   DR. HARDY:  Present.  
  
        7                   MR. BURKHARDT:  Mr. Elekes?  
  
        8                   MR. ELEKES:  Present.  
  
        9                   MR. BURKHARDT:  Ms. Caldas?  
  
       10                   MS. CALDAS:  Present.  
  
       11                   MR. BURKHARDT:  Mr. Berger?  
  
       12                   MR. BERGER:  Present.  
  
       13                   MR. BURKHARDT:  Dr. Williams?  
  
       14                   DR. WILLIAMS:  Present.  
  
       15                   MR. BURKHARDT:  Mr. Craft?  Not present yet.  
  
       16                   And Ms. Quesenbery?  
  
       17                   MS. QUESENBERY:  Present.  
  
       18                   MR. BURKHARDT:  Dr. Bement, I would advise  
  
       19      you that a significant quorum exists for the conduct of  
  
       20      business for the Technical Guidelines Development  
  
       21      Committee, and you may wish to make that declaration as  
  
       22      Chairman.  Thank you so much.  
                                                                      7  
  
  
  
  
        1                   CHAIRMAN BEMENT:  Thank you very much.  I so  
  
        2      declare.  
  
        3                   First of all, I would like to thank the  
  
        4      Commission for making available to us these wonderful  
  
        5      facilities and also being present with us this morning as  
  
        6      we go through our first meeting.  
  
        7                   SPEAKER:  Speak louder, please.  
  
        8                   CHAIRMAN BEMENT:  I will try.  One thing I  
  
        9      really need everyone to hear is what we need to do if we  
  
       10      should have an emergency, which we hope will not occur.  
  
       11                   There are three stairwells outside this door  
  
       12      which will provide egress, and there will be people to  
  
       13      assist you, if necessary.  They're fairly easy to find near  
  
       14      the elevators.  Also, the restrooms are just outside that  
  
       15      door to the left.  I think you'll be able to find them  
  
       16      readily.  
  
       17                    As was mentioned, Dr. Schutzer is joining  
  
       18      us by teleconference, and Mr. Craft, and Dr. Rivest are  
  
       19      trying to make it here by plane.  I guess one flight was  
  
       20      cancelled, but they'll join us during the course of our  
  
       21      meeting.  
  
       22                    This Committee's initial set of  
                                                                      8  
  
  
  
  
        1      recommendations for these guidelines are due to the  
  
        2      executive director of the Election Assistance Commission in  
  
        3      April of 2005, in accordance with HAVA's nine-month  
  
        4      deadline.  
  
        5                   In the interim, the 2002, Voting System  
  
        6      Standards adopted by the Federal Election Commission serve  
  
        7      as the first set of voluntary voting system guidelines  
  
        8      under HAVA.  
  
        9                   In accordance with HAVA, the U.S. Election  
  
       10      Assistance Commission and I have appointed 14 individuals  
  
       11      to serve with me on this Committee.  
  
       12                   Those members include:  
  
       13                   Two members of the Standard's Board;.  
  
       14                   Two members of the Board of Advisors;  
  
       15                   Two members of the Architectural and  
  
       16      Transportation Barrier Compliance Board under Section 502  
  
       17      of the Rehabilitation Act of 1973, which forever more will  
  
       18      be referred to as the Access Board;  
  
       19                   A representative of the American National  
  
       20      Standards Institute;  
  
       21                   A Representative of the Institute of  
  
       22      Electrical and Electronics Engineers;  
                                                                      9  
  
  
  
  
        1                   Two representatives of the National  
  
        2      Association of State Election Directors, selected by such  
  
        3      Association, who are not members of the Standards Board or  
  
        4      Board of Advisors, and who are not in the same political  
  
        5      party.  
  
        6                   In addition, the Election Commission has  
  
        7      appointed four individuals with technical and scientific  
  
        8      expertise relating to voting systems and voting equipment.  
  
        9                   A majority of the members of the Technical  
  
       10      Guidelines Development Committee shall constitute a quorum.  
  
       11                   At this time, I would entertain a motion to  
  
       12      adopt that the latest revised version of Roberts Rules of  
  
       13      Order be adopted to govern the Technical Guidelines  
  
       14      Development Committee and subcommittee proceedings.  
  
       15                   SPEAKER:  So moved.  
  
       16                   CHAIRMAN BEMENT:  I have a motion.  Let's  
  
       17      hear a second.  
  
       18                   SPEAKER:  Second.  
  
       19                   CHAIRMAN BEMENT:  All in favor say, aye.  
  
       20                   COMMITTEE COLLECTIVELY:  Aye.  
  
       21                   CHAIRMAN.  BEMENT:  Opposed?  
  
       22                   Our first meeting has an ambitious agenda  
                                                                      10  
  
  
  
  
        1      with a specific outcome that are necessary for us to  
  
        2      accomplish in order to move forward with the Committee's  
  
        3      HAVA responsibilities.  
  
        4                   Specifically, as a Committee, we must agree  
  
        5      on a procedural road map for standards development, as well  
  
        6      as a preliminary work plan.  
  
        7                   In addition, we will also receive briefings  
  
        8      from NIST scientists on HAVA-related issues, and work  
  
        9      currently in process.  
  
       10                   The time required to accomplish the agenda  
  
       11      item means that the Committee cannot take public comment at  
  
       12      this meeting.  However, there will be other opportunities  
  
       13      for the public to comment and provide testimony at future  
  
       14      meetings, as well as electronically.  
  
       15                   At this point, I should make known to those  
  
       16      who may be in the back-up rooms that to provide e-mail  
  
       17      comment, the e-mail address is voting@NIST.gov.  I'll  
  
       18      repeat that again, voting@NIST.gov.  
  
       19                   In fact, public comment that this Committee  
  
       20      receives electronically and in open meetings will be  
  
       21      critical to the standards development process.  
  
       22                   I believe that this Committee must strive  
                                                                      11  
  
  
  
  
        1      for five distinct deliverables to the EAC in the next nine  
  
        2      months.  
  
        3                   These include the following:  
  
        4                   First, a list of publicly noted requirements  
  
        5      for the voluntary voting assistance standards:  
  
        6                   Second, recommendation through standards  
  
        7      that currently exist and changes, if necessary.  
  
        8                   Third, an assessment of best practices that  
  
        9      can be made available to the election community for us in  
  
       10      the 2006 election cycle.  
  
       11                   Four, a recognition and statement, thereof,  
  
       12      of those areas where there are no current standards under  
  
       13      development.  
  
       14                   And, five a prioritized calendar for future  
  
       15      standards development relative to each of the four previous  
  
       16      deliverables.  
  
       17                   I can now announce that Mr. Craft has been  
  
       18      added to the roll call.  Congratulations.  
  
       19                   MR. CRAFT:  Thank you.  
  
       20                   CHAIRMAN BEMENT:  At this time, I would like  
  
       21      to entertain a motion to adopt the agenda for the July 9th  
  
       22      meeting of the TGDC, which was distributed this morning.  
                                                                      12  
  
  
  
  
        1                   MR. ELEKES:  So moved.  
  
        2                   CHAIRMAN BEMENT:  Is there a second?  
  
        3                   MR. HARDING:  Second.  
  
        4                   CHAIRMAN BEMENT:  All in favor say, aye.  
  
        5                   COMMITTEE COLLECTIVELY:  Aye.  
  
        6                   CHAIRMAN BEMENT:  All opposed.  
  
        7                   Before I ask Chairman Soaries and the  
  
        8      Election Assistance Commission to comment, I would like to  
  
        9      find out that the success of our work will be that of a  
  
       10      critical finding from Congress and the Administration in  
  
       11      the next fiscal year and beyond.  
  
       12                   I think at this time, we can be optimistic  
  
       13      that adequate funding money will be available to us in  
  
       14      fiscal year '05, but we won't know until we get it.  
  
       15                   Now, I would like to call on the Election  
  
       16         Assistance Commissioner Soaries, as well as other  
  
       17      members and other Commissioners to address the Committee on  
  
       18      any remarks they may have.  
  
       19                   CHAIRMAN SOARIES:  Thank you, Dr. Bement,  
  
       20      and let me thank all of your for your presence here, which  
  
       21      signals your commitment to participate in this important  
  
       22      Committee.  
                                                                      13  
  
  
  
  
        1                   Let me thank my colleagues on the Election  
  
        2      Assistance Commission for the hard work they've done since  
  
        3      January to advance the cause of implementing the Help  
  
        4      America Vote Act, and the overall mission of election  
  
        5      reform in this country.  
  
        6                   Let me thank members of the public, who are  
  
        7      here.  We would have been so disappointed, we set up chairs  
  
        8      for you, and we lowered the temperature of the room for  
  
        9      you.  So the fact that you're here is quite encouraging,  
  
       10      but moreover, it's encouraging to know that there are  
  
       11      people who are watching our process as we attempt to  
  
       12      improve elections for all Americas.  
  
       13                   Before I give my charge to the Committee,  
  
       14      I'd like to introduce my colleagues on the Commission.  
  
       15                   The Vice-Chair of our Commission is a very  
  
       16      dynamic leader in her own right, who has brought grace and  
  
       17      charm and elegance to what otherwise would be a very boring  
  
       18      and drab set of male Commissioners.  
  
       19                   She has taken on many responsibilities, not  
  
       20      the least of which being the organizing of, and being  
  
       21      liaison to the Standards Board and the Board of Advisers.  
  
       22                   The work of this Committee will ultimately  
                                                                      14  
  
  
  
  
        1      be vetted through those two Boards.  They are defined by  
  
        2      the statute, and she is our designated Federal officer to  
  
        3      those Boards.  And she has brought great energy and insight  
  
        4      to our work, and I'd like to introduce for her brief  
  
        5      comments, Vice-Chairman Gracia Hillman.  
  
        6                   VICE-CHAIR HILLMAN:  Thank you.  I just want  
  
        7      to welcome everybody and say thank you.  And to pick up on  
  
        8      the Chairman's comments, to say that I will be facilitating  
  
        9      the recommendations that come from this Committee to be  
  
       10      reviewed and commented on by the Standards Board and the  
  
       11      Board of Advisors.  And as you heard previously, we do have  
  
       12      members from each of those bodies on this Committee to also  
  
       13      help with the liaison work.  
  
       14                   It is really great to see this.  For six  
  
       15      months we have been trying to figure out how we were going  
  
       16      to put together the resources, and within the time frame,  
  
       17      to see the TDGC in operation.  So, it's really nice to see  
  
       18      that we were able to bring everybody together for the  
  
       19      meeting.  I thank you all for being here.  
  
       20                   CHAIRMAN SOARIES:  You can tell that  
  
       21      Vice-Chair Hillman has been the one among us who has really  
  
       22      specialized in the bureaucracy.  You can assess how  
                                                                      15  
  
  
  
  
        1      bureaucratized a person has become with the ease with which  
  
        2      they use initials to describe organizations.  
  
        3                   We have called the TGDC everything.  But she  
  
        4      said it, it flowed so easily that you can tell she has  
  
        5      really specialized in the bureaucracy.  
  
        6                   The next Commissioner, who I'd like to  
  
        7      introduce, is the young attorney from Texas, who has work  
  
        8      in the Whitehouse, has a wealth of experience with local  
  
        9      election officials, having represented many counties and  
  
       10      their efforts to address the elections and other issues.  
  
       11                   He brings youth and vision and a dynamic to  
  
       12      the Commission that makes us a better Commission.  He has  
  
       13      taken on responsibility for seeing to it that the states  
  
       14      receive their requirements' payments as perhaps the largest  
  
       15      portion of his portfolio.  
  
       16                   All of you know that our embryotic stages  
  
       17      require that the Commissioners not only set policy, but  
  
       18      also do administrative work.  And this Commissioner is  
  
       19      chiefly responsible for our being able to report that the  
  
       20      requirements' payments began to flow to the states within  
  
       21      the statutory time that was prescribed by HAVA; and that  
  
       22      within six months of our having been established as a  
                                                                      16  
  
  
  
  
        1      Commission, that money began being distributed.  
  
        2                   He is a proud father, and as I mentioned, a  
  
        3      very dynamic addition to this Commission.  And I would like  
  
        4      to call on him for his remarks now.  Commissioner Ray  
  
        5      Martinez.  
  
        6                   COMMISSIONER MARTINEZ:  Thank you,  
  
        7      Mr. Chairman.  I appreciate that very kind introduction.  
  
        8              I am delighted to be here this morning.  I join  
  
        9      my colleagues, the Chair and Vice-Chair Hillman in  
  
       10      Welcoming the formation of The Technical Guidelines  
  
       11      Development Committee.  
  
       12                   I think this is going in and out now.  I  
  
       13      think I'm trying to project as loudly as I can.  Hopefully,  
  
       14      everybody can hear me.  
  
       15                   I do want to pick up on the comments that  
  
       16      our Chairman just made; and that is the fact that we have  
  
       17      over the course of the last two months or so, this  
  
       18      Commission distributed close to $1 billion now of payments  
  
       19      to state and local governments -- or to state governments,  
  
       20      I should say, under Title 2 of the Help America Vote Act,  
  
       21      The Requirements Payments.  This is in addition to Title 1,  
  
       22      Early Out Monies, that were distributed prior to the  
                                                                      17  
  
  
  
  
        1      formation of this Committee -- of the EAC, I should say.  
  
        2                   The reason to pick up on those is the simple  
  
        3      fact that we have a lot of money that is Federal money  
  
        4      unprecedented, historical role that the Federal Government  
  
        5      is now playing in the administration of Federal elections.  
  
        6      And we have money going out to states to improve the  
  
        7      Federal Election Administration process, and the states are  
  
        8      waiting for the Election Assistance Commission, and its  
  
        9      very important statutory advisory committees like the TGDC,  
  
       10      to offer guidance and advice on what to do with regard to  
  
       11      the purchase of voting systems and related questions.  
  
       12                   There's no doubt in my mind that the  
  
       13      formation of this committee is a major milestone for the  
  
       14      Election Assistance Commission.  There is much work to be  
  
       15      done, and certainly I'm most supportive of Dr. Bement and  
  
       16      our partnership with NIST, and the very fine folks that  
  
       17      work there as we do the important work.  
  
       18                   It seems like every time we come to a  
  
       19      milestone, I say this is perhaps the most important thing  
  
       20      the EAC will be doing.  I'll be saying that a lot because  
  
       21      we're doing, I think, so many important things, but we're  
  
       22      doing it in a partnership with organizations like NIST,  
                                                                      18  
  
  
  
  
        1      with our state and local election partners, with advocates  
  
        2      for disability rights, advocates for voting rights and  
  
        3      civil rights, with the entire community of people who are  
  
        4      interested in improving the election administration  
  
        5      process.  
  
        6                   So I welcome you all today, and I look  
  
        7      forward to the work of this Committee.  
  
        8                   CHAIRMAN SOARIES:  Thank you, Commissioner.  
  
        9                   I saved the final Commissioner for last, not  
  
       10      because he's the best cook, he's certainly not the best,  
  
       11      but the shortest, I saved him for last because our presence  
  
       12      here today is directly attributable to work he has done  
  
       13      more than any of the four of us.  
  
       14                   As I mentioned, we divided up our  
  
       15      administrative focus into portfolios, and the next  
  
       16      Commissioner took personal responsibility for the  
  
       17      development of the Technical Guidelines Development  
  
       18      Committee, which Commissioner Hillman easily says "TDGC".  
  
       19                   The Election Assistance Commission divided  
  
       20      itself into five focus areas, the administration,  
  
       21      communications, Standards and Advisory Boards, grants and  
  
       22      research, and election resources.  
                                                                      19  
  
  
  
  
        1                   And consistent with the mandate of HAVA, we  
  
        2      perceive that the ultimate resource for voting in this  
  
        3      country is the setting of comprehensive standards that  
  
        4      would be useful for testing, and useful for offering  
  
        5      guidelines to all of the states.  
  
        6                   The Commissioner, who has the most  
  
        7      experience in actually running an election, is Commissioner  
  
        8      Paul Degregorio.  He served as the Director of Elections  
  
        9      for St. Louis County for many years.  He worked with the  
  
       10      International Federation for Elections, and monitored  
  
       11      elections all over the world.  
  
       12                   So he brought to this Commission so much  
  
       13      practical hands-on experience, that consistent with that,  
  
       14      we ask Commissioner Degregorio if he would not take  
  
       15      personal responsibility to oversee the election resources  
  
       16      commitment of our agency.  
  
       17                   He has worked tirelessly to ensure that this  
  
       18      Committee was named, that we work very closely with NIST to  
  
       19      accomplish a meeting as soon as possible, and he is our  
  
       20      designated Federal officer between the Commission as  
  
       21      liaison to this Committee to ensure that there are no gaps,  
  
       22      and to ensure that we have a healthy, productive working  
                                                                      20  
  
  
  
  
        1      relationship.  
  
        2                   All three of us respect Commissioner  
  
        3      Degregorio's expertise.  We bring passion; he brings  
  
        4      experience.  And I'm proud to introduce him, and I thank  
  
        5      him for the work that resulted in our being here today.  
  
        6      Commissioner Degregorio.  
  
        7                   COMMISSIONER DEGREGORIO:  Thank you,  
  
        8      Mr. Chairman.  I'm not a preacher, but I do pray, and I  
  
        9      said a lot of prayers to get us here today.  
  
       10                   Let me just say, the past six months with  
  
       11      EAC has been some of the most challenging but enjoyable six  
  
       12      months of my life, because of my three colleagues here.  
  
       13                   The Chairman articulated very well what  
  
       14      we've all been doing, but you know it's been team work.  
  
       15      Yes, I bring my expertise to the table, but my colleagues  
  
       16      have brought their's to the table, too, and we have worked  
  
       17      as a strong team, a bipartisan team, over the past six  
  
       18      months to do everything we can to implement the Help  
  
       19      America Vote Act.  
  
       20                   This is a historical day for this Committee  
  
       21      to be meeting, and to begin its work, and to begin its work  
  
       22      over nine months.  
                                                                      21  
  
  
  
  
        1                   I just had my first granddaughter about ten  
  
        2      days ago, and I watched my daughter as she went through her  
  
        3      nine months of pregnancy.  And my wife had four daughters,  
  
        4      so I went through that experience.  And now this will be a  
  
        5      nine-month experience, too.  So I know, you know, during  
  
        6      pregnancies there's always ups and downs and many things.  
  
        7                   I know that this is a great team that's been  
  
        8      assembled here.  The 15 people here represent some of the  
  
        9      best in the United States of America, some of the best  
  
       10      professionals, best election officials.  It's an honor for  
  
       11      me to be associated with you, and I hope to be of  
  
       12      assistance to you, to be the liaison with this group, with  
  
       13      the Commission.  
  
       14                   It is my hope and prayer that all of you  
  
       15      work as a team as we have the past six months, and that we  
  
       16      could be your guide in how you proceed to do this work,  
  
       17      because I think this country needs unification, democratic,  
  
       18      republican, and it also needs leadership in this particular  
  
       19      area.  
  
       20                   We recognize that there is not much you can  
  
       21      do to change things between now and November in terms of  
  
       22      election equipment or setting standards, but you will  
                                                                      22  
  
  
  
  
        1      certainly set the stage for the future.  
  
        2                   Commissioner Martinez worked very hard to  
  
        3      get the money to the states, and the states are clamoring  
  
        4      for guidance from you all on what equipment they should  
  
        5      buy, or what standards they should use when they buy this  
  
        6      equipment.  
  
        7                   Vice-Chair Hillman worked very hard over the  
  
        8      past few months to get money for this Committee for next  
  
        9      year.  And we're hopeful that her efforts will be  
  
       10      successful.  And we're confident, actually, that they're  
  
       11      going to be successful, so this Committee can have the  
  
       12      funds, so NIST can have the funds do its work and support  
  
       13      this Committee.  
  
       14                   The Chairman has provided great leadership  
  
       15      over the past six months, but particularly on this issue.  
  
       16      He's made public statements; he's encouraged us as  
  
       17      Commissioners to focus on this issue, and we have.  And  
  
       18      he's provided great leadership.  
  
       19                   I look forward to working with all of you.  
  
       20      I know Dr. Bement will be a great leader of your team; and  
  
       21      anything that I can do for you, please feel free to contact  
  
       22      me.  I'll be sitting through your meetings when you have  
                                                                      23  
  
  
  
  
        1      them, but here available with EAC, if you should need me  
  
        2      for any reason.  Thank you.  
  
        3                   CHAIRMAN SOARIES:  Thank you, Paul.  Let me  
  
        4      now begin with clearly recognizing the fact I'm a Baptist  
  
        5      preacher, and you gave us until 10 after 10 to do our  
  
        6      business.  We will not be with you all day, and we did want  
  
        7      to communicate with you some very specific points so that  
  
        8      you would really understand the significance that we place  
  
        9      on your work, and the quality and relationship that we  
  
       10      pursue.  
  
       11                   I want to begin, though, by thanking the  
  
       12      National Institute of Standards and Technology.  Dr. Bement  
  
       13      and his wonderful team have been marvelous, not only in the  
  
       14      building of this Committee, but in the work of the EAC in  
  
       15      general.  
  
       16                   Even before the EAC was confirmed, you  
  
       17      should know that the National Institute of Standards and  
  
       18      Technology, under Dr. Bement's leadership, began working on  
  
       19      products and on research that would undergird the standard  
  
       20      setting process through public forums, through human  
  
       21      factors' studies.  
  
       22                   NIST has really preceded the EAC in the  
                                                                      24  
  
  
  
  
        1      implementation of the Help America Vote Act.  And I'd like  
  
        2      to thank Dr. Bement publicly and personally for his  
  
        3      commitment to the EAC, to the passion with which he and his  
  
        4      team has pursued election reform.  
  
        5                   Allan Eustis has been an invaluable resource  
  
        6      to us.  Craig Burkhardt has done everything except move  
  
        7      tables in this office.  Susan Zevin, before she left.  It's  
  
        8      just been a marvelous experience.  And then, of course,  
  
        9                   Dr. Bement's capable deputy has stepped in  
  
       10      because Dr. Bement now has added responsibilities due to  
  
       11      recent changes in his professional life.  
  
       12                   There's been just a seamless flow of  
  
       13      communications and coordination between the EAC and NIST,  
  
       14      and we anticipate that that will do nothing but grow.  
  
       15                   Now, let me focus on the Technical  
  
       16      Guidelines Development Committee.  I want to thank all of  
  
       17      you personally for your willingness to serve.  
  
       18                   I don't know all of you as well as I know  
  
       19      some of you, but my assumption about each of you is that  
  
       20      none of you needed more work to do.  I don't think you were  
  
       21      praying and asking God to give you another assignment.  I  
  
       22      want to make it clear that your presence here today is only  
                                                                      25  
  
  
  
  
        1      indicative of the fact that large groups of people have  
  
        2      high respect for you and your work, for many of you have  
  
        3      been designated by various institutions mandated by HAVA.  
  
        4      But it also reflects the great character of this country  
  
        5      because you are volunteers.  
  
        6                   We were able to buy your plane ticket, we  
  
        7      got you a bed at a hotel, and we give you a couple dollars  
  
        8      to eat lunch at McDonalds.  I think it's Subway today.  But  
  
        9      this is a volunteer effort.  
  
       10                   If you look at America from a step away,  
  
       11      you'll discover that America is, in fact, a great country  
  
       12      because people do things just like without remuneration,  
  
       13      without compensation, you'll get no stock options, you'll  
  
       14      make no overtime.  You're giving of your lives, you've  
  
       15      sacrificed time away from your families, you've given up  
  
       16      time away from your jobs to help America vote.  
  
       17                   The fact is when the story of this country  
  
       18      is told, it will have to include the fact that we were not  
  
       19      just a great economic system, a great political system, but  
  
       20      we were a network of people who were willing to do just  
  
       21      what you do, serve on boards, serve on committees, and give  
  
       22      up their own individual resources and intellectual capacity  
                                                                      26  
  
  
  
  
        1      to make this happen.  I just want to thank each of you,  
  
        2      because none of you had to serve.  
  
        3                   This has become perhaps a more high  
  
        4      visibility and high profile mission than it would have been  
  
        5      five years ago.  Had we been meeting here in 1999, we  
  
        6      perhaps, would have had much less pressure on us.  We would  
  
        7      have had less media attention.  Our outcomes would have  
  
        8      been less visible.  
  
        9                   Now, not only do you give of your time, but  
  
       10      you put your name on a list next to a list of names of  
  
       11      people who you know will be criticized because there will  
  
       12      be large groups of people who will not like the outcome of  
  
       13      your work and we know that now.  
  
       14                   I want to thank you on behalf of the four  
  
       15      Commissioners who could not make the critical decisions  
  
       16      that we have to make, if you did not express a willingness  
  
       17      to serve.  Thank you in advance for the hard work and the  
  
       18      hours that you'll invest in this process.  
  
       19                   Our challenge in many ways is your challenge  
  
       20      because we are now in this together.  What we've discovered  
  
       21      since coming to this task is that voting in this country  
  
       22      was something that we really took for granted for a long  
                                                                      27  
  
  
  
  
        1      time.  That we began voting as a country in colonies,  
  
        2      selecting delegates to represent the interests of the  
  
        3      colonies as we discussed the future concept that we now  
  
        4      consider to be America.  And voting through the years has  
  
        5      evolved managed by the states with the growth of the  
  
        6      country.  
  
        7                   It's not as if we waited until we had 250  
  
        8      million people and then decided to vote.  We voted our way  
  
        9      from the 18th Century to the 21st Century.  And that voting  
  
       10      has, in large measure, been something that we assumed to be  
  
       11      normal, but we never looked at it quite the way you've been  
  
       12      asked to look at voting before.  
  
       13                   Because voting has been so normal and so  
  
       14      regular, and it has been an election for something by  
  
       15      somebody somewhere every year since we got started as a  
  
       16      country, many assumptions have grown with voting.  
  
       17                   If you sit where we sit, and now you do sit  
  
       18      where we sit, we've discovered that many of those  
  
       19      assumptions were more mythology than reality.  
  
       20                   One of the myths that has persisted in the  
  
       21      country is that the Federal Government is much more  
  
       22      involved in voting than the Federal Government has been.  
                                                                      28  
  
  
  
  
        1      Another myth that has evolved is that the machines upon  
  
        2      which we vote have gone through the kind of vetting that  
  
        3      our microwave ovens, and our cars, and other utensils have  
  
        4      gone through.  
  
        5                   We are here today to turn that myth into  
  
        6      reality, and to create standards that make us as confident  
  
        7      in the equipment that we use to vote, and the voting  
  
        8      systems that have embraced our democracy as we do all the  
  
        9      other devices that we use in normal life.  
  
       10                   The Help America Vote Act really changed all  
  
       11      of that, and created a historic role for the Federal  
  
       12      Government in the administration of Federal elections.  
  
       13                   Our job as a Commission is to be a national  
  
       14      clearinghouse, providing information to the states.  And  
  
       15      that information has to be guided by the advice that we get  
  
       16      in large measure from persons like yourself.  
  
       17                   We are faced with a challenge exacerbated by  
  
       18      much more media attention given to voting devices than ever  
  
       19      before in human history.  
  
       20                   One of you, with whom I visited recently,  
  
       21      told a group at my hearing that the media has done such a  
  
       22      fine job in one jurisdiction, that there's almost no need  
                                                                      29  
  
  
  
  
        1      to do any public education on the types of voting devices  
  
        2      used because the media has done all of that.  The media has  
  
        3      described with great, great detail what a punch card looks  
  
        4      like, what a legal machine looks like.  And in that sense,  
  
        5      the media has provided a public service.  
  
        6                   The media has also come to some conclusions  
  
        7      and made its decisions without the benefits of real facts,  
  
        8      and without access to real science, that I think has made  
  
        9      our jobs even harder.  
  
       10                   Our challenge over the next nine and a half  
  
       11      months is to pursue the process that has integrity, so that  
  
       12      the products that we produce from this process have equal  
  
       13      integrity.  
  
       14                   I'd like to charge this Committee to be  
  
       15      committed to principles that I know all of you share; but  
  
       16      sometimes it's harder in a group to maintain the commitment  
  
       17      to certain values that you possess as individuals.  I'm  
  
       18      going to charge you as Chair of the EAC, to work hard to  
  
       19      rise above any partisan, geographical, or philosophical  
  
       20      differences by keeping the focus on the mission.  
  
       21                   I want to charge you with keeping open minds  
  
       22      as we have had to, because there are numerous people who  
                                                                      30  
  
  
  
  
        1      have already made up their minds as to what you should do.  
  
        2      They will call you, they will e-mail you, they will lobby  
  
        3      you, they will threaten you, and they will try to convince  
  
        4      you that only they, only they, know exactly what the  
  
        5      outcome of your work should be.  
  
        6                   I want to charge you with having open  
  
        7      exchange.  We've had as a Commission, wonderful experiences  
  
        8      disagreeing without becoming disagreeable.  You will never  
  
        9      read about or hear about any disagreements we've in public,  
  
       10      because all of our disagreements are in private.  
  
       11                   I would urge you to attempt as best you can,  
  
       12      to keep the process on the one hand open and transparent,  
  
       13      but on the other hand, to keep it unified sufficient to  
  
       14      jelling as a team, and fostering a sense of team effort.  
  
       15                   I also want to charge you to do your work in  
  
       16      nine months.  One, because the law requires it; and two,  
  
       17      because we do have a sense of urgency as it relates to this  
  
       18      mission.  It would be better to produce a product in nine  
  
       19      months, and have another product deferred, than to get too  
  
       20      caught in the quagmire of perfection, and not come up with  
  
       21      anything at the end of nine months.  
  
       22                   Dr. Bement will talk in much more detail  
                                                                      31  
  
  
  
  
        1      about these topics, but we are encouraged, again, because  
  
        2      so much work has been done by some very important people,  
  
        3      even prior to today.  
  
        4                   The NIST Human Factors Report is a report  
  
        5      that the four Commissioners embrace enthusiastically.  It  
  
        6      was a report that identified major challenges as it relates  
  
        7      to usability.  It was a report that had objectives that we  
  
        8      think should guide this Committee, at least in the area of  
  
        9      usability.  And it's a report that we would urge you to  
  
       10      take very seriously and consider as you decide on your  
  
       11      priorities and your timetables.  
  
       12                   The IEEE Project 1583 is very important  
  
       13      work.  I'd like to thank that organization for the work  
  
       14      they did as it relates to considering standards and options  
  
       15      for standards, and for establishing really, a precedent for  
  
       16      participatory inclusive standard setting process.  I think  
  
       17      this Committee would do well to embrace the products in the  
  
       18      form that they're in, and to consider what the implication  
  
       19      and requirements.  
  
       20                    We would be remiss if we did not thank and  
  
       21      recognize the National Association of State Election  
  
       22      Directives for the fine work that they did.  The concept of  
                                                                      32  
  
  
  
  
        1      voting system standards did not really emanate from any  
  
        2      vision in the halls of government, at least the Federal  
  
        3      Government.  Rather the notion of voting system standards  
  
        4      really emanated from the passion and the vision of state  
  
        5      election directives.  
  
        6                   We have among us Alice Miller, who is a  
  
        7      former national leader of NASED.  We have two  
  
        8      representatives of NASED at the table, Mr. Craft and Dr.  
  
        9      Williams, and I think we all owe each of you a debt of  
  
       10      thanks.  
  
       11                   Many of you around the table who are not in  
  
       12      NASED contributed to the NASED process.  The National  
  
       13      Association of Secretaries of State and others contributed  
  
       14      from 1975 going forward to a process where volunteers  
  
       15      without compensation, without even reimbursement, traveled  
  
       16      around the country, went to meetings, and created the only  
  
       17      voting system standards to which Dr. Bement referred a  
  
       18      little earlier, that we have today.  
  
       19                   Were it not for NASED, were it not for that  
  
       20      group of people, we would not even have the FEC 2002  
  
       21      standards, which started in 1990, but the work started long  
  
       22      before 1990.  And NIST, in its former life, contributed to  
                                                                      33  
  
  
  
  
        1      that process also.  
  
        2                   We're indebted to the work of NASED, and  
  
        3      therefore, I think we owe it to history, we owe it NASED,  
  
        4      and we owe it frankly to common sense, to really consider  
  
        5      how much of the 2002 Federal Elections Commission standards  
  
        6      can be embraced, and use that as a starting point, and not  
  
        7      throw out the baby with the proverbial bath water, thinking  
  
        8      we have to start from scratch.  
  
        9                   I also want to generally say that we are  
  
       10      ultimately looking for standards that are  
  
       11      performance-based, standards that can measure success as it  
  
       12      relates to voting, standards against which new voting  
  
       13      devices can be measured; and standards that embrace  
  
       14      existing and emerging technologies, recognizing that it  
  
       15      will never be our responsibility to dictate to states what  
  
       16      kind of voting devices they use.  Rather it is our  
  
       17      responsibility to establish standards against which states  
  
       18      can make informal decisions.  
  
       19                   If we were in church, I'd say that's the  
  
       20      Devil.  
  
       21                   MR. EUSTIS:  Could the people that are  
  
       22      around the conference call please push their mute buttons,  
                                                                      34  
  
  
  
  
        1      make sure your mute is on.  Thank you.  
  
        2                   CHAIRMAN SOARIES:  Having said that, let me  
  
        3      share with you our commitment as a Commission.  And I can  
  
        4      speak for all four of us, even though we've never had a  
  
        5      public meeting to vote on what I'm about to say.  That's my  
  
        6      disclaimer.  
  
        7                   We are committed in the first instance to  
  
        8      making sure you have money.  Paul mentioned Commissioner  
  
        9      Hillman's work.  She's taken responsibility for helping us  
  
       10      craft and market our budgets.  You may not know that we  
  
       11      have received support from the Office of Management and  
  
       12      Budget, and the Whitehouse to amend the President's '05  
  
       13      budget, to include $10 million for research for this  
  
       14      undertaking that had not been this individual submission.  
  
       15                   We're going to continue educating members of  
  
       16      Congress.  We are forbidden from lobbying.  We have no  
  
       17      bypass authority, which means we can't go directly to  
  
       18      Congress with funding requests, but we can educate Congress  
  
       19      and brief Congress, and make sure the member understand the  
  
       20      importance of research.  Research without money is like a  
  
       21      car without gas.  And we know that not all things can be  
  
       22      solved with money, but this is one issue that does require  
                                                                      35  
  
  
  
  
        1      money.  
  
        2                   The second commitment we make is the  
  
        3      commitment to communicate on your behalf, and in support of  
  
        4      your work, to the public ensuring that the public knows as  
  
        5      much as the public can know about this process.  It is not  
  
        6      going to help us if you were perceived to be a private  
  
        7      group going off in a clandestine manner to pursue some  
  
        8      secret agenda.  
  
        9                   We have responsibility through our public  
  
       10      appearances, through our access to the media, to  
  
       11      communicate with passion and clarity, the work of the TGDC  
  
       12      to ensure that the country understands that while nine  
  
       13      months may seem like a long time, and may not have much  
  
       14      impact this year, that nine months is an aggressive amount  
  
       15      of time, and that you're not dragging your feet, or taking  
  
       16      this task lightly.  We will do that to the best of our  
  
       17      ability, and the communications link that exists between  
  
       18      the EAC and NIST will, I think, serve you well.  
  
       19                   Finally, we're committed to a partnership.  
  
       20      We're not going to distance ourselves from you, thus  
  
       21      leaving you out on a political limb to die.  Nor are we  
  
       22      going to attempt to micromanage you.  We have a wonderful  
                                                                      36  
  
  
  
  
        1      working partnership with NIST.  We have complete confidence  
  
        2      in the leadership of Dr. Bement, and we believe that this  
  
        3      Committee is enhanced by the unusual degree of competence  
  
        4      and credibility that NIST brings to the enterprise.  
  
        5                   I have nongovernment agencies all of my  
  
        6      adult life, and without fear of contradiction, I can say  
  
        7      this:  The National Institute of Standards and Technology  
  
        8      and Dr. Bement probably have much credibility as any  
  
        9      government agency anywhere has in anything.  And NIST has  
  
       10      done a fine job, has distinguished itself as being capable,  
  
       11      professional, competent, and diligent towards this task.  
  
       12                   We are committed to a partnership with you.  
  
       13      The partnership means we may sometimes disagree, we may  
  
       14      make requests of you that make you uncomfortable, likewise,  
  
       15      it means that we will fly together or die together, and  
  
       16      that's our commitment.  
  
       17                   I do have one favor that I need to ask of  
  
       18      you, and in your binders, you'll find a report that's done  
  
       19      by the Brannon Center for Justice and the Leadership  
  
       20      Conference on Civil Rights.  
  
       21                   As you know, most of your work, if not all  
  
       22      of your work, will focus on the nine-month mission to come  
                                                                      37  
  
  
  
  
        1      back with recommendations, that we will then take to the  
  
        2      Standards Board and Board of Advisors and vet with the  
  
        3      public.  
  
        4                   However, you can appreciate that much of our  
  
        5      work has to include focus on what we can do to make a  
  
        6      positive impact on the November 2004 elections.  We have  
  
        7      not been alone in our concerns about the quality of the  
  
        8      elections' process and the effectiveness of the voting  
  
        9      outcomes.  
  
       10                   This report was prepared at the request of  
  
       11      the Leadership Conference on Civil Rights, which is a group  
  
       12      that we met with early on.  
  
       13                   Dr. Bement and I talked about this.  Our  
  
       14      response to this report has been to say that we would like  
  
       15      to ask the members of this Committee to take a look at it,  
  
       16      and based on your experience, your expertise, give us some  
  
       17      sense of whether or not you think there is any way you  
  
       18      would like us to include this in the kind of guidance or  
  
       19      options that we provide local elected officials.  I say  
  
       20      that because next week, we will begin the process of  
  
       21      releasing a very comprehensive tool kit that has models and  
  
       22      examples of best practices and administrative procedures  
                                                                      38  
  
  
  
  
        1      for this November.  
  
        2                   As we both solicit and receive information  
  
        3      and input from organizations around the country, we need  
  
        4      the fact that you are on our team now to contribute to our  
  
        5      process as we deliberate what our options are.  So, if you  
  
        6      would take time over the next few days and read this  
  
        7      report.  
  
        8                   I'm not asking you to formally include this,  
  
        9      Dr. Bement, in your deliberations, but because you are who  
  
       10      you are, and you know what you know, and you do what you  
  
       11      do, and now you're connected to us joined at the hip, we  
  
       12      value your input, and before we respond to the contents of  
  
       13      this report, that would be great.  
  
       14                   In closing, we had this discussion at the  
  
       15      Standards Board, and I'm sure the Vice-Chair wants me to  
  
       16      say this, so I'll say it.  At the end of this process, what  
  
       17      many people question is this:  Why should you do all this  
  
       18      work to create standards, which at the end of the day are  
  
       19      voluntary to the states?  Why should you fly from across  
  
       20      the country, give up your time and talent, to produce a  
  
       21      document that is not mandatory?  
  
       22                   HAVA uses some very interesting language.  
                                                                      39  
  
  
  
  
        1      It uses standards language; it uses guidelines language;  
  
        2      and it uses recommendation's language.  We are committed as  
  
        3      a Commission because we signed on knowing that we did not  
  
        4      have regulatory authority, and that we could not mandate  
  
        5      standards.  
  
        6                   We all recognize today that every voting  
  
        7      jurisdiction in this country has been waiting for guidance  
  
        8      and waiting for resources to help that jurisdiction conduct  
  
        9      free, fair, and effective elections.  So we are confident  
  
       10      that the voluntary guidelines process can work.  
  
       11                   The fact is that down the road as we  
  
       12      discover that we're wrong, then we'll make a recommendation  
  
       13      to Congress.  
  
       14                   In the meantime, we see our setting  
  
       15      standards and offering them as guidelines as being real  
  
       16      analogist to raising teenagers.  If you're going to raise a  
  
       17      teenager, you know that parents have standards, and then  
  
       18      the kids take them as guidelines.  
  
       19                   So having said that, some of us have been  
  
       20      somewhat successful with that, we've negotiated then.  And  
  
       21      we believe that states and jurisdictions are waiting to  
  
       22      work with us and to receive our guidance.  
                                                                      40  
  
  
  
  
        1                   So having said that and being  
  
        2      uncharacteristically 20 minutes early, I'd like to conclude  
  
        3      by thanking you again, and I hope and pray that this  
  
        4      process will create the America that all of us have assumed  
  
        5      exists, and that can upgrade our democratic practice so  
  
        6      that we can be the beacon of light, and the standard for  
  
        7      voting for the entire world.  Thank you.  
  
        8                   CHAIRMAN BEMENT:  Thank you, Chairman  
  
        9      Soaries, and also other members of the Commission.  You've  
  
       10      not only charged us, but you've supercharged us.  I hope we  
  
       11      can live up to your expectations.  I want to thank you for  
  
       12      working with me and putting together this wonderful  
  
       13      Committee.  And I think with the team work that's already  
  
       14      underway, we're going to make you proud of us.  
  
       15                   I also especially want to thank Commissioner  
  
       16      Degregorio for all the work that he's done with NIST and  
  
       17      with members of our team in getting us to where we are  
  
       18      today.  It's really been a miracle in many respects that we  
  
       19      now have this Committee that is empowered to vote and do  
  
       20      work.  And, in fact, a lot of it went together last  
  
       21      evening.  So thank you very much, Paul.  
  
       22                   I think one or two people have come on the  
                                                                      41  
  
  
  
  
        1      telephone since Dr. Soaries begin his remarks.  I wonder if  
  
        2      they'd take a moment and you could identify yourself, those  
  
        3      who have joined us on teleconference.  
  
        4                   MR. EUSTIS:  Turn your mute back off.  
  
        5                   MR. SCHUTZER:  I am Dan Schutzer.  
  
        6                   CHAIRMAN BEMENT:  Yes, Dan, thank you.  
  
        7      Anyone else?  
  
        8                   MR. EUSTIS:  Turn the mute back on.  
  
        9                   CHAIRMAN BEMENT:  I guess to segue, I have  
  
       10      raised eight teenagers, and I have to say that any  
  
       11      guidelines that they might accept are totally  
  
       12      discretionary.  
  
       13                   Now at this time, I'd like to ask each of  
  
       14      the Committee members in attendance to introduce  
  
       15      themselves, speak briefly about their backgrounds, and also  
  
       16      comment briefly on why they agreed to serve on the  
  
       17      Committee.  
  
       18                   I would also ask that during the course of  
  
       19      the day as we get into a discussion, to assist our  
  
       20      stenographer, our recorder, and to be sure we get a clear  
  
       21      transcript of our deliberations, if you would identify  
  
       22      yourself as you make comments, that would be very great.  
                                                                      42  
  
  
  
  
        1                   Paul, let's start with you.  
  
        2                   MR. CRAFT:  I'm Paul Craft.  I'm Chief of  
  
        3      the Bureau of the Voting Systems Certification in the State  
  
        4      of Florida, and for the last 12 years, had the honor and  
  
        5      privilege of being part of a very small group of  
  
        6      individuals within the National Association of State  
  
        7      Elections, who have been working as volunteers, and with  
  
        8      great passion to attach what we're carrying forward today.  
  
        9                   It is frankly such a thrill and honor to be  
  
       10      here, that I'm rather humbled by the Chairman's gratitude  
  
       11      at our service.  There's really nowhere else I would rather  
  
       12      be.  I think this is probably the most important work in  
  
       13      the nation at this time.  
  
       14                   As many of you who know me know, I actually  
  
       15      would like to push beyond the legal scope of this  
  
       16      Committee.  Florida very strongly believes, and has taken  
  
       17      the position in the last 12 years, that existing standards  
  
       18      should not be voluntary, they should be mandatory.  
  
       19      Obviously, that's outside our scope, but we hope to work  
  
       20      here and lay out good enforcement standards which Congress  
  
       21      can bring at a later date, if they wish.  
  
       22                   CHAIRMAN BEMENT:  Let me interrupt for just  
                                                                      43  
  
  
  
  
        1      a moment.  Chairman Soaries, I did want to respond to your  
  
        2      request to provide comments on the recommendations from the  
  
        3      Brannon Center.  Clearly, this is valuable input for our  
  
        4      deliberations. It would be --  the recommendations here  
  
        5      would be highly valuable in our considering some of the  
  
        6      security aspect of voting.  
  
        7                   I would ask individuals certainly to respond  
  
        8      to the request.  We need to demonstrate that we can operate  
  
        9      realtime, as well as by procedure.  On the other hand, I  
  
       10      would caution you that you would be responding as  
  
       11      individuals, not as members of the TGDC, so this is your  
  
       12      individual input.  Thank you.  
  
       13                   MS. QUESENBERY:  My name is Whitney  
  
       14      Quesenbery, and I guess I'm pretty honored to be in such  
  
       15      experienced company.  I guess I'm one of the voter  
  
       16      volunteers.  I got interested in this in 2000, like so many  
  
       17      people did, a lot of people.  
  
       18                   I am the President of the Usability  
  
       19      Professional Association, a trade association of people who  
  
       20      work with industry and government making the products and  
  
       21      web sites, and the regulations that we often use, easier  
  
       22      for people to use and more effective for people to use.  
                                                                      44  
  
  
  
  
        1                   In 2000, a lot of people got very excited  
  
        2      about the issue of usability in voting, and I was one of  
  
        3      the few people left standing as a volunteer.  So we started  
  
        4      an education process within our organization that contained  
  
        5      some of the other human factors, just to keep members aware  
  
        6      of what was going on.  One thing led to another, and now  
  
        7      we're getting with the IEEE Committee on the  
  
        8      Usability/Accessibility section of that standard.  
  
        9                   I'm really glad to see that we're beginning  
  
       10      to consider people, and not just machines as we think about  
  
       11      voting, because it is ultimately a human process, and not a  
  
       12      systems' process.  
  
       13                   I think of it as more than voting -- there  
  
       14      are more than voters in that system.  There are lots of  
  
       15      people who were working around election officials who are  
  
       16      candidates themselves.  And all of those people need to be  
  
       17      considered as we think about standards.  So, I think that's  
  
       18      all I have to say.  
  
       19                   CHAIRMAN BEMENT:  Thank you, Whitney.  
  
       20                   MR. WILLIAMS:  I'm Britt Williams from the  
  
       21      State of Georgia, and I represent NASED on this Committee.  
  
       22      You've got my bio in the materials that were sent to you,  
                                                                      45  
  
  
  
  
        1      so I won't repeat that.  
  
        2                   I will say that this is the fourth body like  
  
        3      this that I've been involved with to write standards for  
  
        4      voting systems.  I was a consultant for the FDC in the  
  
        5      development of the standards in 1990, again in 2002.  I'm  
  
        6      on a board with the IEEE to develop standards.  Those other  
  
        7      three were non-funded.  It was entirely voluntary.  In  
  
        8      1990, it took us four years to develop a set of standards.  
  
        9      In 2002, it took three years.  
  
       10              The IEEE project has been in business for three  
  
       11      years, and it's not in any danger of closing any time soon.  
  
       12      So, I'm going to be real curious to see what we can do with  
  
       13      funding in nine months.  
  
       14                   MS. DAVIDSON:  My name is Donnetta Davidson,  
  
       15      and I guess you can say elections have been my life, and I  
  
       16      won't tell you how many years.  I started out in the  
  
       17      county.  I probably am one of the very few people that was  
  
       18      sent here that has been a county clerk, the Secretary of  
  
       19      State, and a candidate.  And all of those things are very  
  
       20      important with, you know, what we're doing.  
  
       21              I was part of the birth of NASED, part of the  
  
       22      birth of the original standards.  And, obviously, my goal  
                                                                      46  
  
  
  
  
        1      is to hopefully set standards that will improve the process  
  
        2      as we always like to do, make it a better process of voting  
  
        3      in the United States.  
  
        4                   I think that nearly every state -- I know I  
  
        5      fought in Colorado years ago to put the standards in where  
  
        6      it was a mandate.  They have to be approved by the  
  
        7      independent testing authorities before we can even test  
  
        8      them in our state.  So I think many of the states have done  
  
        9      this, and I think that's a -- if we can't get it where it's  
  
       10      a mandate nationally, that leaves the states to move  
  
       11      forward to do their own mandate.  And I think that's a  
  
       12      great thing that we ought to encourage, Paul.  I agree with  
  
       13      you.  So I look forward to it, and I feel it's a real honor  
  
       14      to be serving with all of you here.  
  
       15                   MR. BERGER:  I'm Steve Berger.  I'm here  
  
       16      representing the IEEE.  It's a great honor to be here, and  
  
       17      it's an honor for the IEEE to be included.  
  
       18                   Professionally, I don't have a long history  
  
       19      in elections.  I certainly haven't served as an election  
  
       20      official.  Most of my career has been in product  
  
       21      development, basically serving the function of advising  
  
       22      management when a new design was ready to be introduced and  
                                                                      47  
  
  
  
  
        1      marketed.  
  
        2              In many ways that's what we're about, helping  
  
        3      voting officials know when equipment was ready for  
  
        4      implementation.  I say sometimes that I spent a lot of  
  
        5      years breaking things and making it look like it was  
  
        6      someone else's fault.  
  
        7                   Along the way, I served on the two Access  
  
        8      Boards of advisory committees, and disability access is a  
  
        9      particular interest of mine, and I'm gratified to see  
  
       10      there's going to be a focus here.  
  
       11                   I'd like to say in terms of the IEEE, and as  
  
       12      much as Whitney said, I view my role as a conduit.  There's  
  
       13      a lot of people in this country, particularly in the  
  
       14      engineering community, that want to contribute.  I would  
  
       15      just offer the invitation that I think it would be entirely  
  
       16      appropriate for this Committee to ask different bodies of  
  
       17      experts to lend a hand, to take specific assignments and  
  
       18      maybe lend their expertise.  
  
       19                   In my experience, a lot of people want to be  
  
       20      involved, and would be glad to serve in that way.  We  
  
       21      certainly view our effort as being part of this process and  
  
       22      under what this Committee does and the EAC more generally.  
                                                                      48  
  
  
  
  
        1                   MS. PURCELL:  Thank you.  Good morning, I'm  
  
        2      Helen Purcell, and I'm also very glad to be with this  
  
        3      Committee.  
  
        4                   My county is Maricopa County, Phoenix,  
  
        5      Arizona, and we have a million three registered voters, and  
  
        6      that requires a tremendous amount of administration putting  
  
        7      together for the election.  I have been an election  
  
        8      official there for 15 years.  
  
        9              I have seen some very good things happen.  But  
  
       10      then as a result of the controversies of the 2000 election,  
  
       11      I think that we have all seen that we either could do a  
  
       12      better job of the job that we do, we can also do a better  
  
       13      job of educating the public as to what we do, and get a  
  
       14      response back from them.  
  
       15                   It is a system that should be very user  
  
       16      friendly.  I don't know that we advertise that we are a  
  
       17      user friendly group of people, but we certainly need to do  
  
       18      that.  And I hope that with the work that we can do here,  
  
       19      we can make some headway in a lot of areas.  
  
       20                   MR. HARDING:  Good morning.  I'm J.R.  
  
       21      Harding from the great state of Florida.  Perhaps that's  
  
       22      one of the reasons why we're here.  
                                                                      49  
  
  
  
  
        1                   I am a member of the Access Board.  My life  
  
        2      is not a voting expert, while I did experience the life of  
  
        3      (inaudible) the need for name recognition.  
  
        4              My perspective, this is perhaps that of the  
  
        5      disabled perspective.  Most of my adult life and part of my  
  
        6      adolescent, I started with trying to run away from whom I  
  
        7      had become.  But as I grew and matured, worked, labored,  
  
        8      and learned, I realized this is part of who I am, and that  
  
        9      through my adult life, I've really become a bit of a  
  
       10      full-time advocate representing from the state of Florida,  
  
       11      nationally and statewide, from transportation to work,  
  
       12      labor, to educational issues.  
  
       13                   I just kind of feel my role in life is  
  
       14      pushing the envelope for independence and self-sufficiency  
  
       15      in our world.  And I think voting is a cornerstone of that  
  
       16      self-sufficiency in terms of expressing and shaping our  
  
       17      view of our America and our cities.  
  
       18                   It's from that advantage point that I'm glad  
  
       19      to be here and offer what little bit I can to ensure that  
  
       20      everyone has the opportunity to vote and reliably.  Thank  
  
       21      you.  
  
       22                    MR. ELEKES:  I'm Jim Elekes.  I should  
                                                                      50  
  
  
  
  
        1      state in full disclaimer I am from New Jersey.  I have  
  
        2      three things in common with Chairman Soaries:  We both  
  
        3      reside in New Jersey; we both reside in Somerset County;  
  
        4      and we are both debonair males.  
  
        5                   I am an associate college professor in  
  
        6      public administration and policy.  My experience in voting  
  
        7      over the years has been to work to ensure that there is  
  
        8      barrier free access to the polling places, as well as  
  
        9      appropriate assistance at polling places, volunteers  
  
       10      trained to assist persons with disabilities in exercising  
  
       11      their right to vote.  
  
       12                   I have also worked as a beta tester for a  
  
       13      number of organizations who produce software speech  
  
       14      services, and braille for persons who are low vision or  
  
       15      blind.  
  
       16                   As a representative for the U.S. Access  
  
       17      Board, I share Mr. Harding's goal as the lynchpin, voting  
  
       18      is a form of expression, and the ability to proactively  
  
       19      participate in one's community.  And whatever contributions  
  
       20      can be made, or input as a tester that I can make to ensure  
  
       21      the standards are capable of meeting the diverse and  
  
       22      specialized needs of persons with disabilities, as well as  
                                                                      51  
  
  
  
  
        1      perhaps being a liaison to the blind and visually impaired  
  
        2      community, I look forward to the challenge of meeting the  
  
        3      goal of this Committee.  
  
        4                   MS. MILLER:  Good morning.  I'm Alice  
  
        5      Miller.  I'm the executive director for the D.C. Board of  
  
        6      Elections and Ethics.  And let me first say I am the mother  
  
        7      of two teenagers, neither of which accept my rules as  
  
        8      guidelines or standards.  
  
        9                   From that, I have been working elections for  
  
       10      about 17, 18 years.  I started off in the legal field on  
  
       11      (inaudible) the staff of the attorney general counsel of  
  
       12      the elections board.  Somehow or another, I ended up as  
  
       13      executive director, and I'm still trying to figure that  
  
       14      out.  
  
       15                   From that, what I've learned is that we've  
  
       16      watched voters by all definitions. I've watched them as a  
  
       17      candidate, watched them as a voter, watched them as a poll  
  
       18      worker learning the process, and from that angle.  
  
       19                   I know that there has to be a comfort level  
  
       20      with what everybody is doing to not only ensure the  
  
       21      person's vote is counted as they have voted, and the poll  
  
       22      workers are comfortable with ensuring that their  
                                                                      52  
  
  
  
  
        1      instructions are where they need to be to ensure the  
  
        2      voter's vote is also accurately done and accounted.  
  
        3                   I'm actually probably one of the least  
  
        4      technical people on this Committee.  I still question how  
  
        5      you all thought it was a very good idea to place me here.  
  
        6      So it's from that perspective that I am indeed honored to  
  
        7      be here, and I do intend to give my all and to contribute  
  
        8      what I can to help with the guidelines development in this  
  
        9      process.  
  
       10                   MS. TURNER-BUIE:  I am Sharon Turner-Buie,  
  
       11      director of elections from Kansas City, Missouri.  May I  
  
       12      start a little bit differently than everyone else by  
  
       13      commending the Commission on your office space.  I know for  
  
       14      a commission to focus on transparency -- you certainly get  
  
       15      that feeling as soon as you get off the elevator, you get  
  
       16      feeling of transparency, which is so important in this  
  
       17      process.  
  
       18                   I am very humbled to be asked to serve on  
  
       19      this Committee.  It's a very intimidating thought because  
  
       20      to be surrounded by engineers and technical experts with  
  
       21      the science background, etcetera, etcetera.  People don't  
  
       22      normally associate that with elections administration, but  
                                                                      53  
  
  
  
  
        1      it certainly is an important part of it, because we need it  
  
        2      for the task that is at hand.  
  
        3                   I would like to say that as an election  
  
        4      administrator, we had problems and concerns prior to  
  
        5      Elections 2000.  What Elections 2000 did, what the HAVA Act  
  
        6      did was give us a voice.  Now we not only have a voice, but  
  
        7      we have an audience.  And with that voice, I hope to be  
  
        8      able to serve as a link between the architects of these  
  
        9      voluntary voting systems standards and guidelines and,  
  
       10      ultimately, the people that use these systems in the  
  
       11      election, certainly the people that work on election day  
  
       12      are crucial to the process and ultimately the voters.  
  
       13                   So it is so important that we remain focused  
  
       14      on the ultimate users as we are surrounded by all of these  
  
       15      engineers, scientists, and technical people so we can keep  
  
       16      the human element close at heart.  
  
       17                   I would like to disagree with one member of  
  
       18      the Committee, and that being Ms. Miller.  She is not the  
  
       19      least technical person here.  I have that distinct honor.  
  
       20      But hopefully we will be able to contribute some necessary  
  
       21      input as we move forward.  
  
       22                   MS. CALDAS:  My name is Anne Caldas, and I  
                                                                      54  
  
  
  
  
        1      represent the American National Standards Institute.  
  
        2      Actually, that was a nice segue into what we do, because  
  
        3      there are many experts in this country who populate  
  
        4      standards writing organizations, including 200 that ANSI  
  
        5      accredited, including IEEE.  
  
        6                   And I agree with Mr. Berger that many will  
  
        7      be willing to assist, and I don't think we need to think  
  
        8      about starting from scratch.  
  
        9                   American National Standards are developed in  
  
       10      an environment that is open-balanced, fair, and  
  
       11      characterized by due process.  ANSI welcomes the  
  
       12      opportunity to participate in this project, and to help the  
  
       13      Election Assistance Commission and the TGDC in the areas  
  
       14      both of standardization and conformity assessment.  
  
       15                   I'm honored to be here.  ANSI is willing to  
  
       16      give whatever resources we can to help implement the Help  
  
       17      America Vote Act.  Thank you very much.  
  
       18                   CHAIRMAN BEMENT:  Well, I'll bring up the  
  
       19      rear.  I'm Arden Bement.  I'm the Director of NIST, and  
  
       20      acting Director of the National Science Foundation.  And  
  
       21      people ask me all the time how can you carry out both those  
  
       22      responsibilities, and it's because there's a hidden secret,  
                                                                      55  
  
  
  
  
        1      and that is that NIST excels in spite of it's director.  It  
  
        2      not only has great breadth but also great depth.  
  
        3                   I think one of the very best decisions I've  
  
        4      ever made was to appoint Dr. Semerjian as deputy director  
  
        5      of NIST.  He has stepped in immediately without a loss of  
  
        6      movement, and has done a superb job in continuing the  
  
        7      day-to-day operations of NIST.  
  
        8                   As far as my enthusiasm in chairing this  
  
        9      Committee, it's both a learning experience as well as a  
  
       10      human experience.  A learning experience because this is  
  
       11      very important as well as a challenging area to be working  
  
       12      in at the moment.  
  
       13                   It's a human experience because I've got  
  
       14      enough time to meet people who have a passion about voting  
  
       15      and want to improve the system, and recognize that voting  
  
       16      is the bedrock of our democracy.  
  
       17                   It's that citizenship and that sense of  
  
       18      civic responsibility that I've tried to live up to in my  
  
       19      career, and I just feel very proud to be associated with  
  
       20      all of you.  
  
       21                   You've heard a lot of great words about  
  
       22      NIST, and I want to call on Allan Eustis in a moment to  
                                                                      56  
  
  
  
  
        1      tell you all the great assistance they're going to provide  
  
        2      when you carry on your impossible assignments over the next  
  
        3      nine months.  
  
        4                   MR. EUSTIS:  Could we ask Dr. Schutzer if  
  
        5      he's still with us, that he give us a little back ground?  
  
        6                   CHAIRMAN BEMENT:  I'm sorry.  Dan, are you  
  
        7      still with us?  
  
        8                   DR. SCHUTZER:  I'm still with you.  I can  
  
        9      only be on for another hour or so.  I apologize for not  
  
       10      being with you.  I'm please to be -- I believe I have no  
  
       11      voting experience except as a voter.  I believe that my  
  
       12      main background and interest was that I worked with banking  
  
       13      and I worked in many of our systems that you assume are  
  
       14      kinds of technologies, including our ATM technology.  
  
       15                   I worked a long time in many different  
  
       16      engineering and standards organizations.  And I've even  
  
       17      worked for the government as a previous career there in the  
  
       18      military defense systems.  And most recently, the last  
  
       19      position I had there, which is on the opposite side of the  
  
       20      coin, I was the technical director for Naval Intelligence.  
  
       21                   So I know a little bit about how to break  
  
       22      and penetrate systems, as well as how to protect them.  I  
                                                                      57  
  
  
  
  
        1      really look forward to what sounds to me like it's going to  
  
        2      be a fascinating and very important function.  
  
        3                   CHAIRMAN BEMENT:  Thank you, Dan.  I now  
  
        4      call on Allan Eustis, and then Craig Burkhardt, and then  
  
        5      Alice McKenna.  
  
        6                   MR. EUSTIS:  Thank you, Dr. Bement.  Good  
  
        7      morning.  I'm Allan Eustis.  I'll be short, because I am.  
  
        8      Along the lines of the short remarks that were made  
  
        9      earlier, I would like to say that I learned and you will  
  
       10      learn with Commissioner Degregorio and myself sharing this  
  
       11      trait in common, that you don't mess with the short people.  
  
       12                   But other than that, I am extremely  
  
       13      gratified that we are well ahead of schedule.  I would  
  
       14      recommend to the Chair that after we finish our remarks,  
  
       15      myself, Mr. Burkhardt, and Ms. McKenna, that we break at  
  
       16      that time, which will probably be about 20 of, a quarter of  
  
       17      11.  And using the discussing, I will extend the break from  
  
       18      30 minutes to 40 minutes.  
  
       19                   For the public in attendance, there is a  
  
       20      cafe downstairs in this building.  And there's a Subway, if  
  
       21      you walk over to -- there's a McDonald's too, I think.  
  
       22      Basically, if you want to go to Subway, you can walk to the  
                                                                      58  
  
  
  
  
        1      corner of 12th, go south on 12th, and it's on the left.  
  
        2      Either buy your food and eat it there, or bring it back.  
  
        3                   For all of the members of the Technical  
  
        4      Guidelines Development Committee who have given us your  
  
        5      names, do not get up from the table when we break because  
  
        6      we need to collect money from you.  The days of the ten  
  
        7      cent lunches when I went to grade school are gone  
  
        8      unfortunately.  If it was, I would pay for it all.  
  
        9                   I thank you all for being here.  I am going  
  
       10      to use a term that Dr. Bement and Commissioner Degregorio  
  
       11      used, this is, this is really a dream team.  I couldn't  
  
       12      think of a better group of individuals with diverse talents  
  
       13      as yours to come together for problems which almost seem  
  
       14      insoluble.  Hopefully we will come up with recommendations.  
  
       15                   Along that line, I do believe that in nine  
  
       16      months you will give birth to recommendations, but that  
  
       17      will be unusual in the fact that we will not have to  
  
       18      involve gynecologists in the process, that I'm aware of.  
  
       19                   So we will do our work; we will make the  
  
       20      Chairman and the Commissioners, I hope, proud of our  
  
       21      product.  I believe we will.  And myself and Craig  
  
       22      Burkhardt are here to assist.  
                                                                      59  
  
  
  
  
        1                   But let me say that the reason that this  
  
        2      meeting is taking place as smoothly as it is is because of  
  
        3      my assistant, Mary Floyd, who you'll all get to know quite  
  
        4      while.  If Mary hadn't been here, we wouldn't be here.  So  
  
        5      thank you, Mary, from the bottom of my heart.  It really is  
  
        6      amazing that we did this in less than three weeks.  
  
        7                   I would like to thank Paul Degregorio, and  
  
        8      say that I've enjoyed working with you, and look forward to  
  
        9      continuing our work relationship.  I've enjoyed working  
  
       10      with all the Commissioners, which goes back to January  
  
       11      actually, before when he came to our symposium.  So I think  
  
       12      we will move forward and we have made great strides, and we  
  
       13      have a lot of very important business to accomplish.  
  
       14                   By the way, for the public that's here, and  
  
       15      for those not in attendance but reading the transcript,  
  
       16      early next week -- and you'll find in your packet a  
  
       17      template for our Technical Guidance Development Committee  
  
       18      web page.  The website is Http://vote@NIST.gov/ and this is  
  
       19      case sensitive, capital TGDC.htm.  On that page will be the  
  
       20      charter, members, public testimony, hearings of  
  
       21      transcripts, comments and position statements that have  
  
       22      been submitted through voting@NIST.go, which is the e-mail  
                                                                      60  
  
  
  
  
        1      address and any other relevant news to be totally  
  
        2      transparent as we move forward.  
  
        3                   This afternoon, we will discuss a little bit  
  
        4      about the subcommittees, the potential subcommittee  
  
        5      structure that may be adopted by this Committee.  I want  
  
        6      you all to know that there will be subject matter experts  
  
        7      available to each Committee from NIST, and some of those  
  
        8      people you will hear from this afternoon.  
  
        9                   And again, I think a lot of where we are  
  
       10      right now is from the starting position.  With a lot of  
  
       11      background knowledge it's due to folks like Dr. Laskowski,  
  
       12      David Alderman, Mark Skall, Lynne Rosenthal, and Jeffery  
  
       13      Horlick, and you'll meet some of the others, perhaps during  
  
       14      the break you'll get the chance to meet them as well if you  
  
       15      haven't.  We are here to assist in any way possible to  
  
       16      officially accomplish the tasks that we need to accomplish.  
  
       17              I appreciate the trust of both Dr. Bement and  
  
       18      Dr. Semerjian in my work.  It's made my life a lot easier,  
  
       19      and I think that we'll continue to move forward and do  
  
       20      quite well.  The final person -- actually they're two  
  
       21      people, Dr. Zevin recently retired, and I think she's met  
  
       22      many of you on the Committee.  It's been a lot of work to  
                                                                      61  
  
  
  
  
        1      lay the foundation for where we are.  
  
        2                   Our general counsel, as appointed by  
  
        3      Dr. Bement, so I call on Craig to make a few brief remarks,  
  
        4      and then keep this moving on.  I thank you for all the help  
  
        5      he has given me.  
  
        6                   MR. BURKHARDT:  Thank you, Allan.  We're now  
  
        7      going to go into a brief presentation on the legal aspects  
  
        8      of your service on this Federal Advisory Committee.  I hope  
  
        9      that this will actually be the first and last presentation  
  
       10      agreed with (inaudible) because what we want to do is we  
  
       11      want to keep a lawyer in on the procedural things, in the  
  
       12      background, which would allow you to move forward very very  
  
       13      repeatedly in technical deliberations, then eventually the  
  
       14      writing of your policy and the proposals for any overall  
  
       15      agenda.  
  
       16                   Essentially, as a Federal Advisory  
  
       17      Committee, you are in the business, if you will, to provide  
  
       18      advice to the Federal Government and senior officials in  
  
       19      the Federal Government.  
  
       20                   Shortly, Alice McKenna, who is an expert on  
  
       21      Federal Advisory Committee laws is going to give you a  
  
       22      presentation on some of the do's and don'ts on how to  
                                                                      62  
  
  
  
  
        1      conduct yourselves as a Committee, and as subcommittees,  
  
        2      which we'll be talking about in a moment.  They're also  
  
        3      personal activities.  There are a lot of Federal Advisory  
  
        4      Committees (inaudible) and generally, speaking function  
  
        5      without any legal difficulties or complications.  
  
        6                   So, we will try to be very client-centered  
  
        7      in managing your activities as opposed to erecting barriers  
  
        8      in your way.  
  
        9                   Keep a couple of things in your mind.  
  
       10      Dr. Bement has agreed to present to you the subcommittee  
  
       11      recommendations, which he has to split this group up into  
  
       12      three-working groups, subcommittees, which are functioned  
  
       13      over the next six months.  
  
       14                   A couple of things to keep in mind as we're  
  
       15      getting together as an overall preview, such as we have  
  
       16      today.  We are regulated by certain rules that require us  
  
       17      to give public notice in the Federal Register seven days in  
  
       18      advance.  And in taking back this meeting, (inaudible)  
  
       19      public availability requirements.  
  
       20                   We're going to be talking about  
  
       21      subcommittees.  The subcommittees will typically be four or  
  
       22      five people.  One thing we need to understand about the  
                                                                      63  
  
  
  
  
        1      subcommittees right from the get-go is they have no  
  
        2      independent authority.  These are essentially going to be  
  
        3      smaller working groups that are going to study specific  
  
        4      subject matter issues and bring proposals back to this  
  
        5      group.  
  
        6                   So this is the only group that actually has  
  
        7      authority to vote and adopt proposals, which would  
  
        8      eventually go back to the Federal Election -- to the  
  
        9      Election Assistance Commission and it's members.  
  
       10                   The subcommittees will operate slightly  
  
       11      differently than this Committee.  When I say slightly  
  
       12      differently, all I mean is that notices, which will be  
  
       13      given in sometimes immediate circumstances.  The intent is  
  
       14      to keep the subcommittee's activities very transparent and  
  
       15      very public to properly serve the vendor community, the  
  
       16      election administration community, as well as the testing  
  
       17      community and the press, because we want to be open.  
  
       18      That's the preference of the Election Assistance Commission  
  
       19      as well as NIST.  We hope that it will be in your  
  
       20      preferences as well.  
  
       21                   A couple of the things that Alice talked  
  
       22      about that you will want to keep in mind is that the  
                                                                      64  
  
  
  
  
        1      subcommittees will have a little flexibility.  So these  
  
        2      will be smaller working groups, but you'll have to be  
  
        3      somewhat more nimble, you'll need to react quickly to  
  
        4      information that comes, and the ability to have meetings by  
  
        5      telephone conference and so forth.  
  
        6              I want to press and the public who was listening  
  
        7      in to know that we will be giving notices by web site.  And  
  
        8      the web site as Mr. Eustis mentioned is vote.NIST.gov, and  
  
        9      that could mean that there will be duplicate postings of  
  
       10      all notices of subcommittees' activities on the Election  
  
       11      Assistance Commission web site, which is www.EAC.gov.  
  
       12                   We will not be unfair to the public at large  
  
       13      or the press.  We will give as advanced notice as we can.  
  
       14      But I do advise those of you who are in the room, and those  
  
       15      who might be listening in who have great interest, to  
  
       16      regularly check in on those web site postings, keep track  
  
       17      of what will be going on over the next six months, which  
  
       18      will essentially be subcommittee hearings, subcommittee  
  
       19      deliberations, and eventually subcommittee formulation of  
  
       20      proposals, which will come back to this overall Committee  
  
       21      probably in about six months.  
  
       22                   As Dr. Bement will mention, we probably  
                                                                      65  
  
  
  
  
        1      won't be getting back together again as an overall  
  
        2      Committee until sometime in January or February of next  
  
        3      year.  
  
        4                   So I will not go into any other details on  
  
        5      the subcommittees because DR. Bement will be presenting a  
  
        6      detailed proposal to you this afternoon.  But those are the  
  
        7      things that I want you to be a aware of as you listen to  
  
        8      Ms. McKenna's presentations on the Committee, which we  
  
        9      comprised, and the subcommittees, which will be comprised.  
  
       10                   Alice McKenna will now come forward.  She is  
  
       11      a senior attorney at the Department of Commerce.  She's an  
  
       12      expert on Federal Advisory Committee activities and the  
  
       13      regulations that govern the activities of those committees.  
  
       14                   Alice?  
  
       15                   MS. MCKENNA:  Thank you, Craig.  Good  
  
       16      morning.  
  
       17                   As Craig said, I'm Alice McKenna from the  
  
       18      Department of Commerce Office of General Counsel.  We are  
  
       19      the parent agency to the parent agency of NIST.  
  
       20                   I have a handout that's going around the  
  
       21      room.  I'll please ask the DFO to take two copies because  
  
       22      one copy is going to need to go in the public file, the  
                                                                      66  
  
  
  
  
        1      other is going to be maintained on this Committee as is  
  
        2      require by the FACA.  
  
        3                   While Craig is also handing out the handout,  
  
        4      I'll tell you that I have served as an election officer,  
  
        5      and I know what it's like to spend 14 hours on your feet,  
  
        6      and then at the end of the 14 hours, have to do a meeting.  
  
        7                   And while NIST is a scientific and computer  
  
        8      and mathematics based organization, I have to tell you that  
  
        9      as an attorney for the Department of Commerce, I am not.  I  
  
       10      want to a large midwestern university that specialized in  
  
       11      electrical engineering and computer science, and I majored  
  
       12      in political science and German literature.  
  
       13                   If I had been any good in math at all, I  
  
       14      would have gone to the electrical engineering department  
  
       15      and had myself a much higher paying job that I have as a  
  
       16      person with two Bachelor's of Arts Degrees from the  
  
       17      University of Illinois.  
  
       18                   You all now have a handout, and I will turn  
  
       19      your attention to that.  I'm going to hit the highlights.  
  
       20      These are some basic bedrock principles that define how  
  
       21      advisory committees work under the Federal Advisory  
  
       22      Committee Act, which is a statute I've come to specialize  
                                                                      67  
  
  
  
  
        1      in in the last 13 years or so at the Commerce Department.  
  
        2                   The main principle of any advisory committee  
  
        3      is it is simply that.  It is advisory only.  Most  
  
        4      committee's, unless otherwise directed by Congress, have no  
  
        5      operational functions.  They don't make and implement  
  
        6      government decisions.  They advise the government and the  
  
        7      government relies on the advice in one way or another.  
  
        8                   If there is any question about whether a  
  
        9      committee is operated in an advisory capacity, or whether  
  
       10      it is operating within the scope of its charter as dictated  
  
       11      by the statute, those questions should be referred to the  
  
       12      designated federal officer, who will then contact the  
  
       13      relevant government officials.  
  
       14                   As I just stated, your advisory duties are  
  
       15      sort of limited by the statute.  The general purpose is to  
  
       16      assist the Executive Director of the Election Assistance  
  
       17      Commission in the development of voluntary voting system  
  
       18      guidelines.  
  
       19                   The Committee is statutorily directed to  
  
       20      provide its first set of recommendations to the Executive  
  
       21      Director not later than nine months.  That work's out to  
  
       22      April 2005.  After that, the Committee shall cause to have  
                                                                      68  
  
  
  
  
        1      published any guidelines on voluntary voting systems that  
  
        2      are adopted by the Commission.  
  
        3                   Your role as members, you're appointed, you  
  
        4      serve at the pleasure of the Commission and the Director of  
  
        5      the National Institute of Standards and technology.  
  
        6                   There is one important don't.  You do not  
  
        7      advise Congress or engage in what we call grassroots  
  
        8      lobbying.  These activities are first, outside the scope of  
  
        9      your charter, and for us, for the government and agencies  
  
       10      involved, may also implement other prohibitions against  
  
       11      grassroots lobbying.  None of this, however, applies to  
  
       12      what you do as private citizens.  I'm talking strictly in  
  
       13      your capacity as Committee members.  
  
       14                   There's two kinds of Committee members under  
  
       15      the ethics statues.  Some of you are appointed as "Special  
  
       16      Government Employees" because you serve as an expert in an  
  
       17      individual capacity.  Others of you have been, however,  
  
       18      appointed in what we call a "representative capacity" to  
  
       19      bring to the table the views of a particular group or  
  
       20      organization.  The Commission -- the Election Assistance  
  
       21      Commission is responsible for identifying who is who.  
  
       22                   The ethics rules, I'm not an ethics'  
                                                                      69  
  
  
  
  
        1      attorney.  I have a short summation of how the ethics rules  
  
        2      apply to the two different groups.  
  
        3                    "Those who are serving in an individual,  
  
        4      expert capacity, are Special Government Employees, and are  
  
        5      subject to the Federal criminal conflict of interest  
  
        6      statutes and rules."  
  
        7                   This is summarized very generally.  "As such  
  
        8      they are prohibited from participating in particular  
  
        9      matters that may have a direct and predictable effect on  
  
       10      their financial interests or on those of a spouse, minor  
  
       11      child or general partner.  They also cannot be registered  
  
       12      as a foreign agent, and I'm sure you all know that.  
  
       13                   It's a requirement -- when somebody works  
  
       14      for a foreign government or a wholly-owned subsidiary of a  
  
       15      foreign government, there's a requirement to register with  
  
       16      the Justice Department and with the government under the  
  
       17      Foreign Agents Registration Statue.  
  
       18                  People who serve in a representative capacity  
  
       19      are not subject to criminal conflict of interest statues;  
  
       20      nevertheless, they are bound by general rules including the  
  
       21      prohibition on use of government affiliation, resources or  
  
       22      information.  
                                                                      70  
  
  
  
  
        1                   Advice on any questions on standards of  
  
        2      conduct matters for both Special Government Employees and  
  
        3      representatives will be provided by the Election Assistance  
  
        4      Commission, and through the offices that support it.  
  
        5                   Meetings of your Committee.  There is a core  
  
        6      requirement in your charter.  The Committee shall not act  
  
        7      in the absence of a quorum or a majority of the members,  
  
        8      not having a conflict of interest in.  If you have a  
  
        9      conflict of interest, you must recuse yourself from  
  
       10      discussion of the matter.  
  
       11                   A quorum is defined as a simple majority of  
  
       12      all people who don't have a conflict.  Half will be enough.  
  
       13                   Committee meetings only occur when called by  
  
       14      a federal officer, usually the DFO. All meetings are open  
  
       15      to the public, and public notice must be given to the  
  
       16      public 15 days in advance, including web site notices.  
  
       17                   While the Federal Advisory Committee Act  
  
       18      provides for limited exceptions to the open vetting  
  
       19      requirement, those have to be decided on advice approved by  
  
       20      the government agency involved.  
  
       21                   Meetings should allow a reasonable  
  
       22      opportunity for the public to make comments.  The public  
                                                                      71  
  
  
  
  
        1      may also file written statements with the Committee at any  
  
        2      time.  
  
        3                   All materials that are made available to the  
  
        4      Committee, my handout prepared for the Committee, handouts  
  
        5      prepared by the Committee, minutes and transcripts of  
  
        6      meetings will be filed in files that are kept by the  
  
        7      government and available to the public at any time upon  
  
        8      request.  This would be like in a reading room format.  
  
        9                   An exception to that would be if it's  
  
       10      material that's exempt under the Freedom of Information  
  
       11      Act.  
  
       12                   Meetings can be in person, via  
  
       13      videoconference or conference call.  The crucial element  
  
       14      for things like video conference, and conference calls, the  
  
       15      public must be afforded access to the deliberations by  
  
       16      having a site where the public can attend, a room where the  
  
       17      public can come in and listen to a speakerphone or  
  
       18      something like that.  
  
       19                   There are some things that don't rise to the  
  
       20      level of a meeting.  One of those is where Committees  
  
       21      members meet and have a conference call to compare  
  
       22      schedules, things like that.  Those are administrative,  
                                                                      72  
  
  
  
  
        1      housekeeping matters, and are not necessarily required to  
  
        2      be public.  
  
        3                    We'll talk briefly about subcommittees that  
  
        4      may be necessary to accomplish the Committee's function.  
  
        5      Subcommittees are a very important function of the  
  
        6      Committee.  
  
        7                   Subcommittees, again, are sort of a pyramid  
  
        8      being the start basically of the pyramid, feeding into the  
  
        9      middle level, being this Committee, and then the top being,  
  
       10      the government agency.  
  
       11                   Subcommittees that meet to advise the  
  
       12      government directly may be in violation if they haven't had  
  
       13      their meetings noticed 15-plus days in advance in the  
  
       14      Federal Register.  
  
       15                    The Committee must deliberate on what the  
  
       16      subcommittee gives them to make sure that it complies with  
  
       17      the FACA prior to advising the government based on the  
  
       18      subcommittee's work.  
  
       19                   Committees or subcommittees that contemplate  
  
       20      using people that are not at the table today, but other  
  
       21      people, we would have to look at that separately.  
  
       22                   Any documents that are provided to the  
                                                                      73  
  
  
  
  
        1      Committee by the subcommittee, also have to go into the  
  
        2      public file.  
  
        3                   Some purely administrative matters, that all  
  
        4      that's standing between you and a break.  Administrative  
  
        5      support for the Committee is provided by the Election  
  
        6      Assistance Commission.  
  
        7                   As NIST representatives, you know, members  
  
        8      are not compensated for their services, but travel,  
  
        9      including per diem in lieu of subsistence, may be paid for  
  
       10      upon request.  
  
       11                   All Government-funded travel must be at the  
  
       12      Government's request and must involve the provision of a  
  
       13      direct service to the Government.  
  
       14                   Election Assistance Commission attorneys  
  
       15      provide legal advice to the Commission, including the DFO,  
  
       16      with the exception of standards of conduct issues.  The  
  
       17      government attorneys do not serve as legal advisors to the  
  
       18      individual Committee members, we only advise the Committee  
  
       19      as a whole.  
  
       20                   If you have any questions about what to do,  
  
       21      or if there is something that comes up, please, direct your  
  
       22      questions to the Committee's Designated Federal Officer.  
                                                                      74  
  
  
  
  
        1      If that person can't answer you on the spot, that  
  
        2      individual is invited to contact Craig or myself to get the  
  
        3      answer, and that DFO will then report back to you.  
  
        4                   At this time do you have questions?  
  
        5                   MR. ELEKES:  You note that the documents  
  
        6      would be placed in a central location.  Should a person  
  
        7      with a disability require alternative format, is there  
  
        8      consideration to provide those documents in alternate  
  
        9      electronic format such as audio or braille?  
  
       10                   MS. MCKENNA:  I'll refer that question to  
  
       11      Allan Eustis.  
  
       12                   MR. EUSTIS:  Yes, we will provide that  
  
       13      through my office.  You can make the request through  
  
       14      e-mail, vote@NIST.gov, or through any of the Commissioners  
  
       15      or myself.  We will provide that in a commonly accepted  
  
       16      format, either disk or document.  We will do that in  
  
       17      whatever is recognized as the most useful format for that,  
  
       18      and we'll take recommendations as well on other formats.  
  
       19                   CHAIRMAN BEMENT:  J. R.  
  
       20                   MR. HARDING:  Thank you, Mr. Chairman, J.R.  
  
       21      Harding.  A follow-up on that.  Perhaps on the bottom of  
  
       22      all of our agenda's, for alternate format needs, I request  
                                                                      75  
  
  
  
  
        1      that a phone number and a contact person would be  
  
        2      identified to make those formal requests.  
  
        3                   MR. EUSTIS:  So noted.  
  
        4                   MR. HARDING:  I've got that language for  
  
        5      you.  
  
        6                   CHAIRMAN BEMENT:  Any other questions by  
  
        7      members?  
  
        8                   While we've earned ourselves a little bit of  
  
        9      slack time, we do have members of the public here today.  
  
       10      I would like to invite them, if they would, to introduce  
  
       11      themselves.  
  
       12                   Let's just start in the front row and just  
  
       13      move to the right and to the back.  
  
       14                   PAUL WOLTE:  Paul Wolte, Sequoia Voting  
  
       15      Systems.  
  
       16                   DAVID:  David from the staff of the Access  
  
       17      Board.  I'm here to provide support to Mr. Harding.  
  
       18                   DR. LASKOWSKI:  Dr. Sharon Laskowski from  
  
       19      NIST.  
  
       20                   MR. ALDERMAN:  David Alderman from NIST.  
  
       21                   MR. SKALL:  Mark Skall from NIST.  
  
       22                   MS. ROSENTHAL:  Lynne Rosenthal from NIST.  
                                                                      76  
  
  
  
  
        1                   MS. CHIN:  Candace Chin.  Senate.  
  
        2                     MS. SCHNEIDER:  Elizabeth Schneider  
  
        3                   MR. AUBLE:  Daniel Auble, Common Cause.  
  
        4                   MR. WACK:  John Wack, NIST.  
  
        5                     MR. WHITE:  Douglas White, NIST.  
  
        6                   MR. KELSEY:  John Kelsey, NIST.  
  
        7                   MR. SMOLKA:  Richard Smolka, Election  
  
        8      Administration Reports.  
  
        9                   CHAIRMAN BEMENT:  Thank you very much.  Well  
  
       10      we have now earned our break.  It's a quarter to 11.  We  
  
       11      will reconvene at 11:30.  
  
       12                   (Recess from 10:45 to 11:30)  
  
       13                   DR. RIVEST:  (...)  In terms of my business  
  
       14      of serving on this Committee, I think that voting is a  
  
       15      great national important civic duty.  It's my duty to help  
  
       16      participate on my Committee.  (Inaudible).  
  
       17                   CHAIRMAN BEMENT:  Thank you, Ron.  I would  
  
       18      like to call your attention to a tab in your book called  
  
       19      "Organization."  And what we're going to be talking about  
  
       20      for the next 30 minutes or so is organizational  
  
       21      recommendations, which is essentially our procedural  
  
       22      roadmap of how we're going to be conducting business.  And  
                                                                      77  
  
  
  
  
        1      I'll go through this piece by piece so we'll have an  
  
        2      opportunity to have discussions as we go.  
  
        3                   The first part, we pretty well covered this  
  
        4      morning all the way through the first paragraph.  
  
        5                   The second paragraph under Committee  
  
        6      structure, there's a recommendation that the business of  
  
        7      the TGDC is to accomplish the three subjects, specific  
  
        8      subcommittees comprised only of TGDC members.  
  
        9                   Each subcommittee will have a member chair.  
  
       10      For technical assistance, each subcommittee will have a  
  
       11      NIST executive, familiar with the subject matter assigned  
  
       12      to it.  I propose that such committees are the Subcommittee  
  
       13      on Security and Transparency; the Subcommittee on Human  
  
       14      Factors and Privacy; and the Subcommittee on Core  
  
       15      Requirements and Testing.  
  
       16                   In the back pocket of your notebooks is a  
  
       17      signup sheet which briefly describes the recommended scope  
  
       18      of each subcommittee.  
  
       19                   MR. HARDING:  Mr. Chairman, do you require a  
  
       20      motion for those three subcommittees?  
  
       21                   CHAIRMAN BEMENT:  We're going to get to that  
  
       22      in a little bit.  So I would ask you, as we go along if  
                                                                      78  
  
  
  
  
        1      you're already familiar with this, go ahead and indicate  
  
        2      your personal choices.  
  
        3                   Now, because of financial disclosure forms  
  
        4      for TGDC members have not been totally completed by this  
  
        5      meeting, it is not appropriate at this time to announce  
  
        6      subcommittee chairs, but that will be done very shortly.  
  
        7      And in selecting the members for the subcommittee, I'll pay  
  
        8      particular attention to your first choices, try to get as  
  
        9      much distribution as we can.  
  
       10                   Any comments or discussions on that part of  
  
       11      our Committee structure?  
  
       12                   All right.  Going on to Committee  
  
       13      operations.  The duties of subcommittees will be to engage  
  
       14      in information gathering and analysis, including but not  
  
       15      limited to conduct that probably carries the analysis of  
  
       16      existing best practices, specifications, and standards.  
  
       17                   As you know, there is a body of standards  
  
       18      already in place.  Subcommittees will prepare proposals to  
  
       19      the entire TGDC, and only those (inaudible) will be  
  
       20      forwarded to the Election Assistance Commission, subject to  
  
       21      prior constraints.  
  
       22                   The subcommittees' meetings can occur in  
                                                                      79  
  
  
  
  
        1      person or by telephone conference.  (Inaudible) subject to  
  
        2      availability of funding essential subcommittee activities  
  
        3      accomplished in concert with the staff.  The National  
  
        4      Institute of Standards and Technology will provide  
  
        5      technical assistance, including first, the conduct of  
  
        6      public hearings to accept oral and written testimony.  
  
        7                   Second, the acceptance of analysis and  
  
        8      written materials by NIST web capabilities.  
  
        9                   Third, the survey and analysis of applicable  
  
       10      best practices, specifications, and standards.  And,  
  
       11      actually, identification of areas requiring writing and  
  
       12      viewing of a revised best practices, specification, and  
  
       13      standards.  And the prioritization projects.  Writing  
  
       14      proposals, best practices, specifications or standards, and  
  
       15      communication of project selection and progress to the  
  
       16      TGDC.  
  
       17                   Subcommittee work product will be shared on  
  
       18      a regular basis will all TDGC members, and subcommittee  
  
       19      activities and communications must be in compliance with  
  
       20      legal requirements as approved by the Chair.  
  
       21                   Now, some of you have broad interests in  
  
       22      almost every activity that the Committee is engaged in and  
                                                                      80  
  
  
  
  
        1      may like to be active on more than one subcommittee, and we  
  
        2      can certainly accommodate that, subject to your  
  
        3      availability.  But recognize that any products from the  
  
        4      subcommittees will be fully vetted and discussed at the  
  
        5      Committee level before it can go forward.  
  
        6                   Also, since we will be making arrangements  
  
        7      for the public to be involved, either by teleconference or  
  
        8      by other means, such as in other languages that monitor  
  
        9      subcommittee activities, if you so wish.  
  
       10                   So I think there will be ample opportunity  
  
       11      to maintain on an ongoing basis, the familiarity with all  
  
       12      activities of the Committee, even though, we have to parse  
  
       13      it into these three subcommittees in order to carry out the  
  
       14      work of the full Committee.  
  
       15                   Having said that, are there any other  
  
       16      comments or questions you might have on the function of  
  
       17      the subcommittees?  If not, at an appropriate time and this  
  
       18      will likely be January/February 2005, the TGDC will hold a  
  
       19      multi-day session during which work product of the  
  
       20      subcommittees will be presented in the form of a series of  
  
       21      resolutions.  
  
       22                   The resolutions will be will bated for  
                                                                      81  
  
  
  
  
        1      potential amendments and adoption.  Adopted resolutions  
  
        2      will thereafter be refereed to this staff for technical  
  
        3      assistance and clean up.  And this staff will return all  
  
        4      adopted resolutions, after providing technical assistance  
  
        5      and clean up, to the TGDC for final review and approval.  
  
        6                   In April of 2005, adopted resolutions and  
  
        7      related material will be presented to the EAC Executive  
  
        8      Director in the form of the statutory mandated first set of  
  
        9      recommendations.  
  
       10                   Any comments or questions on that?  
  
       11                   Work product form and priorities provisions  
  
       12      with HAVA effectively require the TGDC to submit the first  
  
       13      set of recommendations to the EAC Executive Director in  
  
       14      April of 2005.  The form of the recommendations will vary  
  
       15      for each subject matter, depending on the existence of  
  
       16      current best practices, specifications, or standards.  For  
  
       17      example, some products may consist of reviewing quality to  
  
       18      test the standards already developed by other  
  
       19      organizations.  
  
       20                   Recommendations related to such projects  
  
       21      will consist of suggested standards, capable of immediate  
  
       22      implementation.  The subject varies such as human factors,  
                                                                      82  
  
  
  
  
        1      where there are viewing systems specifications or standards  
  
        2      related to voting systems.  Recommendations may consist  
  
        3      primarily of best practice guidelines and protocols for  
  
        4      validation of independent testing and government testing  
  
        5      activities.  
  
        6                   And going on, HAVA requires election  
  
        7      authorities to be in compliance with certain requirements  
  
        8      in January 2006.  I recommend to the TGDC and its  
  
        9      subcommittees, your first priority projects that will be  
  
       10      useful to the manufacturing testing laboratory and election  
  
       11      administration communities.  In other words, that's a  
  
       12      caution that we need to stay focused and prioritized, and  
  
       13      not take a frontal approach to the whole field, but perhaps  
  
       14      approach it by segment.  
  
       15                   If there are any general comments or  
  
       16      additions or corrections to these procedures, we can take  
  
       17      these up now or during the time when we entertain motions.  
  
       18      So let's move on.  
  
       19                   I would like to entertain a motion that  
  
       20      three subcommittees be formed for gathering and analyzing  
  
       21      information --  
  
       22                   MR. HARDING:  So moved.  
                                                                      83  
  
  
  
  
        1                   CHAIRMAN BEMENT:  Let me read the whole  
  
        2      motion.  The subcommittees shall be comprised only of TGDC  
  
        3      members. The subcommittees shall propose resolutions to the  
  
        4      TGDC on best practices, specifications, and standard.  The  
  
        5      subcommittees shall be named:  Security and Transparency,  
  
        6      Human Factors and Privacy, and Core Requirements and  
  
        7      Testing.  
  
        8                   Do you wish to move that?  
  
        9                   MR. HARDING:  Yes, I do.  
  
       10                   CHAIRMAN BEMENT:  We have a motion.  Is  
  
       11      there a second?  
  
       12                   MR. ELEKES:  Second.  
  
       13                   CHAIRMAN BEMENT:  We will take a roll call  
  
       14      on those motions.  If you will read the roll.  
  
       15                   MR. BURKHARDT:  Okay.  We will designate  
  
       16      this as Committee Resolution Number 1.  I will repeat the  
  
       17      motion for purposes of the court reporter to check her  
  
       18      record.  
  
       19                   The motion is as follows:  That three  
  
       20      subcommittees be established to gather and analyze  
  
       21      information.  The subcommittees shall be comprised only of  
  
       22      TGDC members.  The subcommittees shall propose resolutions  
                                                                      84  
  
  
  
  
        1      to the TGDC on best practices, specifications, and  
  
        2      standards.  Subcommittees shall be named:  Security and  
  
        3      Transparency, Human Factors and Privacy, and Core  
  
        4      Requirements and Testing.  
  
        5                   I will now call the roll call.  I will only  
  
        6      be calling the names of people whose financial disclosure  
  
        7      documents have been approved.  
  
        8                   You will answer "aye" or "nay" or state if  
  
        9      you belive you have a conflict of interest.  
  
       10                   Dr. Bement?  
  
       11                   CHAIRMAN BEMENT:  Aye.  
  
       12                   MR. BURKHARDT:  Davidson?  
  
       13                   MS. DAVIDSON:  Aye.  
  
       14                   MR. BURKHARDT:  Miller?  
  
       15                   MS. MILLER:  Aye.  
  
       16                   MR. BURKHARDT:  Purcell?  
  
       17                   MS. PURCELL:  Aye.  
  
       18                   MR. BURKHARDT:  Harding?  
  
       19                   MR. HARDING:  Aye.  
  
       20                   MR. BURKHARDT:  Elekes?  
  
       21                   MR. ELEKES:  Aye.  
  
       22                   MR. BURKHARDT:  Caldas?  
                                                                      85  
  
  
  
  
        1                   MS. CALDAS:  Aye.  
  
        2                   MR. BURKHARDT:  Berger?  
  
        3                   MR. BERGER:  Aye.  
  
        4                   MR. BURKHARDT:  Williams?  
  
        5                   MR. WILLIAMS:  Aye.  
  
        6                   MR. BURKHARDT:  Craft?  
  
        7                   MR. CRAFT:  Aye.  
  
        8                   MR. BURKHARDT:  Quesenbery?  
  
        9                   MS. QUESENBERY:  Aye.  
  
       10                   MR. BURKHARDT:  Mr. Chairman, that's 11 yes  
  
       11      votes, no nay votes.  I believe that you can declare that  
  
       12      as having been adopted.  
  
       13                   CHAIRMAN BEMENT:  I declare that the  
  
       14      resolution has been adopted.  
  
       15                   I would like now to entertain a motion that  
  
       16      the Chair survey the interest of TGDC members, and  
  
       17      thereafter appoint members and chairs of the subcommittee.  
  
       18                   Is there a motion?  
  
       19                   MR. BERGER:  So moved.  
  
       20                   CHAIRMAN:  Is three a second?  
  
       21                   MS. PURCELL:  Second.  
  
       22                   CHAIRMAN BEMENT:  Any discussion on the  
                                                                      86  
  
  
  
  
        1      motions?  If not, please call the roll.  
  
        2                   MR. BURKHARDT:  This will be designated  
  
        3      Committee Resolution Number 2.  I'll repeat for purposes of  
  
        4      the court reporter, the motion at this time:  
  
        5                   That the Chair survey the interests of TGDC  
  
        6      members, and thereafter appoint the members and chairs of  
  
        7      the subcommittees.  
  
        8                   Roll call.  Bement?  
  
        9                   CHAIRMAN BEMENT:  Aye.  
  
       10                   MR. BURKHARDT:  Davidson?  
  
       11                   MS. DAVIDSON:  Aye.  
  
       12                   MR. BURKHARDT:  Miller?  
  
       13                   MS. MILLER:  Aye.  
  
       14                   MR. BURKHARDT:  Purcell?  
  
       15                   MS. PURCELL:  Aye.  
  
       16                   MR. BURKHARDT:  Harding?  
  
       17                   MR. HARDING:  Aye.  
  
       18                   MR. BURKHARDT:  Elekes?  
  
       19                   MR. ELEKES:  Aye.  
  
       20                   MR. BURKHARDT:  Caldas?  
  
       21                   MS. CALDAS:  Aye.  
  
       22                   MR. BURKHARDT:  Berger?  
                                                                      87  
  
  
  
  
        1                   MR. BERGER:  Aye.  
  
        2                   MR. BURKHARDT:  Williams?  
  
        3                   MR. WILLIAMS:  Aye.  
  
        4                   MR. BURKHARDT:  Craft?  
  
        5                   MR. CRAFT:  Aye.  
  
        6                   MR. BURKHARDT:  Quesenbery?  
  
        7                   MS. QUESENBERY:  Aye.  
  
        8                   MR. BURKHARDT:  Mr. Chairman, the voting is  
  
        9      11 ayes, 0 nays.  I believe you can declare Committee  
  
       10      Resolution Number 2 passed.  
  
       11                   CHAIRMAN BEMENT:  I declare that Resolution  
  
       12      Number 2 passed.  
  
       13                   I would like now to entertain the motion  
  
       14      that resolutions prepared by subcommittees be considered by  
  
       15      the TGDC.  Resolutions adopted by the TGDC shall be  
  
       16      referred to NIST for technical assistance and editing.  
  
       17      Upon the return from NIST, the TGDC shall review the  
  
       18      resolutions to confirm they conform to its intent.  
  
       19                   MR. HARDING:  I would move that.  
  
       20                   CHAIRMAN BEMENT:  Okay.  There is a motion.  
  
       21      Is there a second?  
  
       22                   MR. ELEKES:  Second.  
                                                                      88  
  
  
  
  
        1                   CHAIRMAN BEMENT:  Any discussion?  Very  
  
        2      well, please call the roll.  
  
        3                   MR. BURKHARDT:  This will be designated  
  
        4      Committee Resolution Number 3.  I will repeat it for  
  
        5      purposes of the court reporter.  It is designated as  
  
        6      follows:  
  
        7                   The motion is that:  
  
        8                   Resolutions prepared by subcommittees be  
  
        9      considered by the TGDC.  Resolutions adopted by the TGDC  
  
       10      shall be referred to NIST for technical assistance and  
  
       11      editing.  Upon the return from NIST, the TGDC shall review  
  
       12      the resolutions to confirm that they conform to its intent.  
  
       13                   Roll call.  Bement?  
  
       14                   CHAIRMAN BEMENT:  Aye.  
  
       15                   MR. BURKHARDT:  Davidson?  
  
       16                   MS. DAVIDSON:  Aye.  
  
       17                   MR. BURKHARDT:  Miller?  
  
       18                   MS. MILLER:  Aye.  
  
       19                   MR. BURKHARDT:  Purcell?  
  
       20                   MS. PURCELL:  Aye.  
  
       21                   MR. BURKHARDT:  Harding?  
  
       22                   MR. HARDING:  Aye.  
                                                                      89  
  
  
  
  
        1                   MR. BURKHARDT:  Elekes?  
  
        2                   MR. ELEKES:  Aye.  
  
        3                   MR. BURKHARDT:  Caldas?  
  
        4                   MS. CALDAS:  Aye.  
  
        5                   MR. BURKHARDT:  Berger?  
  
        6                   MR. BERGER:  Aye.  
  
        7                   MR. BURKHARDT:  Williams?  
  
        8                   MR. WILLIAMS:  Aye.  
  
        9                   MR. BURKHARDT:  Craft?  
  
       10                   MR. CRAFT:  Aye.  
  
       11                   MR. BURKHARDT:  Quesenbery?  
  
       12                   MS. QUESENBERY:  Aye.  
  
       13                   MR. BURKHARDT:  Mr. Chair, the votes are 11  
  
       14      yes, 0 no.  I recommend that you declare the resolution has  
  
       15      been adopted.  
  
       16                   CHAIRMAN BEMENT:  I declare Resolution  
  
       17      Number 3 is adopted.  
  
       18                   Next, I'd like to entertain the motion that  
  
       19      adopted resolutions and appropriate explanatory materials  
  
       20      comprise the "first set of recommendations" mandated by the  
  
       21      Help America Vote Act.  
  
       22                   MS. DAVIDSON:  So moved.  
                                                                      90  
  
  
  
  
        1                   CHAIRMAN BEMENT:  There's a motion.  
  
        2                   MR. HARDING:  Second.  
  
        3                   CHAIRMAN BEMENT:  Any discussion on this  
  
        4      motion?  If not, call the roll.  
  
        5                   MR. BURKHARDT:  I'm designating this motion  
  
        6      as Committee Resolution Number 4.  
  
        7                   The motion for purposes of confirmation for  
  
        8      the court reporter is as follows:  
  
        9                   Adopted resolutions and appropriate  
  
       10      explanatory materials comprise the "first set of  
  
       11      recommendations" mandated by the Help America Vote Act.  
  
       12                   Roll call.  Bement?  
  
       13                   CHAIRMAN BEMENT:  Aye.  
  
       14                   MR. BURKHARDT:  Davidson?  
  
       15                   MS. DAVIDSON:  Aye.  
  
       16                   MR. BURKHARDT:  Miller?  
  
       17                   MS. MILLER:  Aye.  
  
       18                   MR. BURKHARDT:  Purcell?  
  
       19                   MS. PURCELL:  Aye.  
  
       20                   MR. BURKHARDT:  Harding?  
  
       21                   MR. HARDING:  Aye.  
  
       22                   MR. BURKHARDT:  Elekes?  
                                                                      91  
  
  
  
  
        1                   MR. ELEKES:  Aye.  
  
        2                   MR. BURKHARDT:  Caldas?  
  
        3                   MS. CALDAS:  Aye.  
  
        4                   MR. BURKHARDT:  Berger?  
  
        5                   MR. BERGER:  Aye.  
  
        6                   MR. BURKHARDT:  Williams?  
  
        7                   MR. WILLIAMS:  Aye.  
  
        8                   MR. BURKHARDT:  Craft?  
  
        9                   MR. CRAFT:  Aye.  
  
       10                   MR. BURKHARDT:  Quesenbery?  
  
       11                   MS. QUESENBERY:  Aye.  
  
       12                   MR. BURKHARDT:  Mr. Chair, the votes are 11  
  
       13      yes, 0 no.  I recommend that you declare Resolution Number  
  
       14      4 as adopted.  
  
       15                   CHAIRMAN BEMENT:  I so declare.  
  
       16                   That completes the formal resolutions.  
  
       17      There's time now for discussion of the major issues and  
  
       18      timelines in our road map and anything else that you may  
  
       19      wish to discuss or any other motions that you may want to  
  
       20      put forward.  
  
       21                   According to our agenda, I think the  
  
       22      prioritization time frame in terms of the three most  
                                                                      92  
  
  
  
  
        1      important recommendations is fairly within the next nine  
  
        2      months when we will have our first set of preliminary  
  
        3      recommendations, and that will be a priority effort.  But  
  
        4      then, of course, ongoing, there will be activities toward  
  
        5      that up to and beyond 2006.  There may be some discussion  
  
        6      about that.  
  
        7                   As far as considering all aspects of voting  
  
        8      (inaudible) before we post voting.  As well as the  
  
        9      interplay between people, technology, and process there may  
  
       10      be some discussion on that.  What do we mean by holistic  
  
       11      approach?  
  
       12                   The phase and (inaudible) of our work plan,  
  
       13      which is pretty well laid out in the organization right  
  
       14      now.  If you'd just (inaudible) and then any next steps  
  
       15      that might be of interest to the Committee.  In some  
  
       16      respects I think that (inaudible) of engagement were pretty  
  
       17      well covered this morning as far as the requirements under  
  
       18      the Sunshine Act that we can take up any questions then as  
  
       19      well.  
  
       20                   And following our discussion, we will then  
  
       21      begin our background briefings in this (inaudible).  Is  
  
       22      there anyone who may wish to entertain any of these  
                                                                      93  
  
  
  
  
        1      segments?  
  
        2                   MS. DAVIDSON:  I would just like to offer  
  
        3      since there are going to be subgroups, ANSI does have a  
  
        4      database of existing American National Standards and  
  
        5      standards that are not American National Standards but that  
  
        6      are recognized.  
  
        7                   We can certainly make that data available  
  
        8      and produce specific reports, depending on the area of  
  
        9      interest of the subcommittee, so we can identify standards  
  
       10      that are existing or in process.  And I would like to make  
  
       11      that offer across to all subcommittees, since I will be on  
  
       12      one and there will three.  
  
       13                   CHAIRMAN BEMENT:  Thank you very much.  
  
       14                   MR. HARDING:  Mr. Chairman, J. R. Harding.  
  
       15      In our motion 3 as well as 4, we spoke to specific  
  
       16      timelines, January/February of 2005, April of 2005, in  
  
       17      terms of the deliverable to this group.  Related to the  
  
       18      deliverable perhaps of a work product in a subcommittee,  
  
       19      are there any timelines/expectations or just in that  
  
       20      January/February time frame and again in that April time  
  
       21      frame?  
  
       22                   CHAIRMAN BEMENT:  The only thing that comes  
                                                                      94  
  
  
  
  
        1      to mind J. R. Is that I think it's important that we get as  
  
        2      much up on the website as early as we can in terms of gaps  
  
        3      that we can identify in existing standards which need to be  
  
        4      addressed.  Because normally in the standard development  
  
        5      process and (inaudible) you're talking about what, 12 to 18  
  
        6      months of effort on the flip side.  
  
        7                   MR. HARDING:  Yes.  
  
        8                   CHAIRMAN BEMENT:  So that getting the  
  
        9      standard development organizations attuned to the thinking  
  
       10      of the Committee earlier would be very helpful.  
  
       11                   Clearly in your January meeting, when we  
  
       12      formulize our recommendations, that will also be an  
  
       13      important stage to help us through that process.  
  
       14                    If there are questions that occur to you as  
  
       15      we go along throughout the afternoon, and as we hear the  
  
       16      NIST presentations, we will take time to have a discussion.  
  
       17                   Oh, yeah, Steve.  
  
       18                   MR. BERGER:  I might just offer a thought  
  
       19      for our groups.  I think one of the things we need to be  
  
       20      conscious of is the time required for other parties to  
  
       21      implement this.  One of my particular concerns, given my  
  
       22      background is, the time vendors would need to respond to  
                                                                      95  
  
  
  
  
        1      the direction we might take.  
  
        2                   So in some of the areas that I know we will  
  
        3      being addressing as soon as we could identify the direction  
  
        4      that we will address them, so the vendors could start doing  
  
        5      background and have things like information on usability,  
  
        6      information on reliability of the products and process  
  
        7      along with the speed they will need for ultimate  
  
        8      implementation.  That's exactly what you said that making  
  
        9      sure that we signal that on the web sites so outside  
  
       10      parties can start making their points.  
  
       11                   CHAIRMAN BEMENT:  As we get to the NIST  
  
       12      briefings, which will be pretty much on the procedures for  
  
       13      development standards, any question that you might have  
  
       14      (inaudible) how performance based standards are developed  
  
       15      as compared to process standards, product standards, any  
  
       16      other kind of standard and, basically, we will try and  
  
       17      enter into that discussion.  
  
       18                   So why don't we start with the first  
  
       19      presentation, which will be by Sharon Laskowski, who did a  
  
       20      lot of the yeomen work.  
  
       21                   MR. HARDING:  Mr. Chairman, is this found  
  
       22      within our background tab or is it still --  
                                                                      96  
  
  
  
  
        1                   CHAIRMAN BEMENT:  Yes, it's in the  
  
        2      background.  
  
        3                   Sharon was lead author of NIST HAVA Mandated  
  
        4      Human Factors Report to Congress.  And she's also a  
  
        5      scientist for NIST Information Technology Laboratory.  
  
        6      During her briefing, she'll discuss the findings of her  
  
        7      report, "Improving the Usability and Accessibility of  
  
        8      Voting Systems and Products."  
  
        9                   DR. LASKOWSKI:  Allan, how do you want us --  
  
       10      do you want us to stand here for our presentations or?  
  
       11                   MR. EUSTIS:  Yes, maybe J. R. Doesn't mind  
  
       12      if you put your stuff on the table; is that right?  
  
       13                   DR. LASKOWSKI:  And how much time, now that  
  
       14      we're on a slightly different schedule?  I could try to  
  
       15      keep it to 10 minutes or an hour.  
  
       16                   MR. EUSTIS:  I would say try to keep it to  
  
       17      15 minutes, and then if you can finish earlier than that,  
  
       18      there could be time for some questions from the TGDC.  
  
       19                   DR. LASKOWSKI:  Okay.  I'm going to read the  
  
       20      slides for anyone that might have vision problems and  
  
       21      describe any diagrams.  
  
       22                   Does anyone in the audience need to look at  
                                                                      97  
  
  
  
  
        1      me while I'm speaking?  If so, feel free to move or just  
  
        2      give me a cue so that I can be sure to be looking towards  
  
        3      you.  Okay.  
  
        4                   Let me first just take off on the running  
  
        5      joke of this morning, I have three teenagers, now that my  
  
        6      son just turned 13 two weeks ago, and they view standards  
  
        7      just as guidelines.  But what I found is critical that they  
  
        8      need some notion of how I'm going to evaluate their  
  
        9      performance against those standards and guidelines, and  
  
       10      that's sort of the theme in probably not just my talk, but  
  
       11      in the other NIST talks that you'll hear about.  
  
       12                   I'm talking about improving the usability  
  
       13      and accessibility of voting systems and products based on  
  
       14      what was put together in the Human Factors report.  Okay.  
  
       15                   When we talk about human factors, it's  
  
       16      important to look at all the users in this system.  And  
  
       17      I've got this diagram called "Voting Echo System" that was  
  
       18      develop at a workshop a few weeks ago at the Usability  
  
       19      Professional Association on Voting Systems that had about  
  
       20      14 people, rather international crowd.  That was a day  
  
       21      workshop.  
  
       22                   It was very productive, but one of the first  
                                                                      98  
  
  
  
  
        1      things we did is sit down and say, okay, who do we have to  
  
        2      be concerned about?  I think this approach to who are the  
  
        3      users of the voting system is important not just for the  
  
        4      usability and accessibility, but also when you look at any  
  
        5      aspect of the system, security, elsewhere.  
  
        6                   Any implementation has to be evaluated in  
  
        7      terms of the user of the system, if it's going to work  
  
        8      properly, have people have confidence that it's working.  
  
        9                   In the middle circle, we have voters, of  
  
       10      course, because they're the critical user here.  We also  
  
       11      have poll workers and around the perimeter of that we've  
  
       12      got voter registration workers, challenger judges,  
  
       13      observers, warehouse workers, support staff, election  
  
       14      officials, postal workers, if there's absentee, to be  
  
       15      delivering absentee, etcetera.  
  
       16                   And then around the outside perimeter, we've  
  
       17      got voter advocacy groups, elected officials, legislators,  
  
       18      the press, candidates, vendors.  All those people are part  
  
       19      of a voting system, and we need to keep that in mind as we  
  
       20      design and develop standards that are testable.  
  
       21                   So just to set the stage, the human factors  
  
       22      usability perspective on voting systems with respect to the  
                                                                      99  
  
  
  
  
        1      voters is comprised of the cognitive and physical nature of  
  
        2      the voters, the physical environment, the psychological  
  
        3      environment, the voting product itself.  And usability is  
  
        4      determined by the demands of the system and the voters  
  
        5      ability to perform under those demands.  
  
        6                   Now, do you know how to measure  
  
        7      accessibility and usability?  There are a couple of  
  
        8      definitions here.  
  
        9                   Accessibility is the degree to which the  
  
       10      system is available to and usable by individuals with  
  
       11      disabilities.  
  
       12                   So not just access but usability, can they  
  
       13      actually cast a vote the way they intended if they have a  
  
       14      disability, or a language issue?  
  
       15                   Usability -- this is the standard  
  
       16      definition -- is a measurement of the effectiveness,  
  
       17      efficiency, and satisfaction achieved by a specified set of  
  
       18      users performing specific tasks with a given product.  
  
       19                   Standard metrics, for example, for  
  
       20      effectiveness, efficiency, and satisfaction are things like  
  
       21      that counting errors that causes a vote that is cast not as  
  
       22      intended, or a vote not cast, that is (errors prior to  
                                                                      100  
  
  
  
  
        1      success) for the voting systems -- well, if they do  
  
        2      eventually cast the vote as intended, errors might  
  
        3      contribute to -- prior to success, contribute to taking  
  
        4      more time using the system.  But that is also a critical  
  
        5      aspect because we don't want long lines at the polling  
  
        6      places.  You can measure subjective satisfaction as a  
  
        7      measure of errors as well.  
  
        8                   The process for user-centered design process  
  
        9      goes from design to measurement.  You start with a  
  
       10      user-centered design when a system is being built.  You do  
  
       11      diagnostic usability evaluation to advise the design  
  
       12      process as you iterate and build a bit of pride, and then  
  
       13      at the end, you do a performance test called usability  
  
       14      testing.          Now, what's the state of usability of  
  
       15      U.S. voting systems.  In general, voting systems have not  
  
       16      been measured for usability, nor have they been developed  
  
       17      using a user-centered designed process.  So we really do  
  
       18      not know the degree to which voters cast their vote not as  
  
       19      they intended due to confusion with the user interface.  
  
       20                   There's been a number of observations by  
  
       21      people at either Cal Tech and MIT Voting Technology  
  
       22      Project, Paul Hermanson Digital Government Funded Project  
                                                                      101  
  
  
  
  
        1      on Voting, others such as Doug Jones and many usability  
  
        2      professionals who note that there do appear to be some  
  
        3      potential problems with the designs that we see that are  
  
        4      causing errors.  
  
        5                   Now, when we developed standards through  
  
        6      usability and accessibility, the design standards or how  
  
        7      the product is designed; for example, specified font size  
  
        8      or ballot instructions.  
  
        9                   Performance standards are how the product  
  
       10      actually functions.  So specified functional characters are  
  
       11      no over voting is allowed by the system.  You can test that  
  
       12      usually by demonstration.  There will be users and a lot of  
  
       13      voters to go through the system to test that.  
  
       14         Counting errors, and measuring performance, like, time  
  
       15      to cast the vote, failure in casting vote as intended, that  
  
       16      requires measuring with users against benchmarks.  
  
       17              How do we know if the system passes or fails when  
  
       18      you measure these things?  You need to have realistic  
  
       19      benchmarks that go along with the standards in order to  
  
       20      test the performance standards.  
  
       21              And doing that kind of testing, you also need  
  
       22      sample ballots of different complexities if we're going to  
                                                                      102  
  
  
  
  
        1      follow this ITA process that we see that's currently in  
  
        2      place.  They've got to have some standards ballots to test  
  
        3      with.  And you need well-defined test protocols and user  
  
        4      groups.  
  
        5              So, if we want to measure for this qualification  
  
        6      testing and certification testing that goes on, well, right  
  
        7      now we don't have a protocol for measuring the usability of  
  
        8      voting systems.  We don't need a high degree of usability  
  
        9      because we want low errors. As opposed to if you were using  
  
       10      a word processor, you have some errors that are a little  
  
       11      frustrating; but we expect better from our voting systems.  
  
       12              Following design guidelines don't necessarily  
  
       13      ensure usability.  We can look to the usability engineering  
  
       14      field to provide measurement methods; but again, not  
  
       15      necessarily to the degree we need specifically for voting.  
  
       16              I've got a line graph on the left:  Informal  
  
       17      evaluation, that depends on usability professional  
  
       18      evaluating, they have a good sense of what the design is,  
  
       19      what minimizes errors, probably not necessarily repeatable,  
  
       20      a lot of variation, depending on the expert who is doing  
  
       21      the review.  
  
       22              On the right of the timeline:  Rigorous research  
                                                                      103  
  
  
  
  
        1      and experience, a very complex but reliable kind of human  
  
        2      factors researchers do.  However, that's very costly, so  
  
        3      what we'd like to do is get a conformance testing up to a  
  
        4      point that we've got reproducible reliable results from our  
  
        5      testing, so the ITA's can use their tests, but that is  
  
        6      still feasible, and it's not too costly.  And that's a  
  
        7      research issue.  So if we want conformance standards for  
  
        8      usability, we need to address this problem if we're going  
  
        9      to develop conformance testing on certifying these systems.  
  
       10              What about current voting standards and testing?  
  
       11      Well, the current VSS -- Voting System Standards -- do have  
  
       12      some accessibility standards, but only a usability  
  
       13      appendix.  They're not shallow statements, they're  
  
       14      recommendations because you're not being tested against it  
  
       15      in the ITA process, there's not much incentive to build to  
  
       16      recommendations such as those.  
  
       17              Again, if we really want to test for usability  
  
       18      and accessibility, we need standards that are clear and  
  
       19      testable, and we need good procedures to do that testing.  
  
       20              IEEE P1583 has draft standards, Task Group 3  
  
       21      usability and accessibility has been making some good  
  
       22      progress.  One thing that if you're not involved in a  
                                                                      104  
  
  
  
  
        1      voluntary consensus standards in standard development  
  
        2      organizations, you should be aware that that works best  
  
        3      when you're got a lot of vendors such as Microsoft, large  
  
        4      vendors who can put a lot of capability into this volunteer  
  
        5      effort, other organizations, large organizations with a  
  
        6      good bottom line, so that they have resources to bring to  
  
        7      bear to send teams of people to develop conformance tests  
  
        8      along with the standards.  
  
        9              IEEE has gotten a marvelous set of volunteers,  
  
       10      but they don't have a lot of resources.  Participants don't  
  
       11      have a lot of travel money and, Steve, correct me if I'm  
  
       12      wrong, that's just being realistic.  
  
       13              STEVE:  That's absolutely true.  
  
       14              DR. LASKOWSKI:  If you're going to be serious  
  
       15      about performance based standards, you have to address that  
  
       16      issue that you need resources to develop benchmarks for  
  
       17      testing conformance.  
  
       18              The Human Factors Report.  Our report recommends  
  
       19      an approach that will produce measurable voting system  
  
       20      standards for usability and accessibility.  It doesn't need  
  
       21      a lot of research, but it does need some.  It does need  
  
       22      expertise in conformance test development, some applied  
                                                                      105  
  
  
  
  
        1      research to develop their user testing protocols I've  
  
        2      alluded to.  And because of the sensitivity of voting  
  
        3      systems, we want to make sure we have good neutral third  
  
        4      parties that develop these benchmarks.  
  
        5              So there's no cheap, quick fix here.  But there's  
  
        6      some things we can do in the interim to require some  
  
        7      usability testing so that we could avoid major usability  
  
        8      blunders.  And that's probably a good place to start, but  
  
        9      there is no guarantee.  
  
       10              I guess I have time to quickly go over the 10  
  
       11      recommendations.  
  
       12              I've summarized, and you can read the report for  
  
       13      much more explanation and detailed sentences for each of  
  
       14      the recommendations.  
  
       15              The 10 recommendations are:  
  
       16              Performance-based usability standards.  They're  
  
       17      high enough levels so that they're not what I call  
  
       18      technology agnostic.  We want then independent of the  
  
       19      technology being used, because if you base your standards  
  
       20      on very exact technology, when new technology or approaches  
  
       21      come along, you've got to redo your standards.  So you  
  
       22      don't want them to be very specific.  But if you've got  
                                                                      106  
  
  
  
  
        1      performance-based benchmarks, they are pretty much  
  
        2      technology agnostic.  
  
        3              One thing we also noted is that we need a  
  
        4      complete set of user-related functional requirements.  Over  
  
        5      voting is a functional requirement.  Pulling those out  
  
        6      means usability testing gets easier, because you don't need  
  
        7      to do a lot of usability testing with users, you can  
  
        8      specify the testing to test against functional  
  
        9      requirements.  One thing we've noticed is those things are  
  
       10      kind of haves been intermixed, so it would be helpful to  
  
       11      pull those out.  
  
       12              You want to avoid low-level design  
  
       13      specifications.  Use only product design requirements that  
  
       14      have been validated as necessary for operation of the  
  
       15      system.  
  
       16              We do need to apply research to support the  
  
       17      development of benchmarks for usability and accessibility  
  
       18      standards.  
  
       19              We should review the current requirements form  
  
       20      the Access Board, the current VSS, the draft IEEE standards  
  
       21      for possible adoption, because there are good nuggets in  
  
       22      all of those.  
                                                                      107  
  
  
  
  
        1              Ballot design guidelines.  Butterfly ballot comes  
  
        2      to mind.  There's a lot of good work out there on good  
  
        3      presentation from the America Institute of Graphic Arts,  
  
        4      AIGA, and other groups, and we're pulling those together.  
  
        5      They wouldn't be standards for good conformance testing at  
  
        6      the ITA level because ballots are designed at the state and  
  
        7      precinct level.  But I think those would help election  
  
        8      officials a great deal in producing better ballots and  
  
        9      avoiding bad design.  
  
       10                   And, again, I think that's a matter of  
  
       11      collecting what's currently out there and pulling it  
  
       12      together in a better format.  
  
       13              Also, guidelines for facility and equipment  
  
       14      layout; also guidelines for how to design and usability  
  
       15      tests for the vendors; and usability testing, good  
  
       16      usability based documentation and training materials.  
  
       17              Users beyond the voters here, and all that  
  
       18      supports a better voting system process.  
  
       19              Vendors should incorporate a user-centered design  
  
       20      approach.  There's lots of literature out there, standard,  
  
       21      user-centered approaches in systems engineering.  
  
       22              We need a good set of conformance tests for  
                                                                      108  
  
  
  
  
        1      voting products against the applicable accessibility  
  
        2      requirements.  
  
        3              And, finally, we need valid, reliable,  
  
        4      repeatable, reproducible processes for usability  
  
        5      conformance testing of voting products against the  
  
        6      usability standards described in the first recommendation  
  
        7      with agreed upon usability pass/fail requirements.  That  
  
        8      is:  It passes benchmark, yes or no.  
  
        9              I would say the most critical need is a set of  
  
       10      usability standards that are performance-based with  
  
       11      objective measures and conformance test procedures.  Then  
  
       12      we can certify against usability, and this is the only way  
  
       13      to guarantee high levels of usability.  
  
       14              And just to summarize a very shortened version of  
  
       15      a roadmap here.  In the short term, we can do a lot to  
  
       16      encourage usability and user-centered design.  
  
       17              Some usability testing at various levels.  At the  
  
       18      vendor level, at the state level with actual ballots; for  
  
       19      example, to mitigate possible problems that may occur when  
  
       20      you look at real ballots.  
  
       21              There's a lot that can be done just educationally  
  
       22      to bootstrap people to this way of thinking.  
                                                                      109  
  
  
  
  
        1              Long term:  We should start with using the best  
  
        2      of IEEE and other standards.  Get some good ballot design  
  
        3      guidance.  The parallel, to start developing good user test  
  
        4      procedures, collect user data to define performance  
  
        5      baselines, and develop performance standards and  
  
        6      conformance tests, that would bring us a lot further than  
  
        7      we currently are.  
  
        8                   I guess we have a minute or two for  
  
        9      questions.  
  
       10                   CHAIRMAN BEMENT:  Yes, thank you, Sharon,  
  
       11      very much.  
  
       12                   Are there any questions for Sharon?  
  
       13                   MR. ELEKES:  Yes, Mr. Elekes, Access Board.  
  
       14      As you were going through your list, you said accessibility  
  
       15      and facility, I didn't catch the point.  
  
       16                   DR. LASKOWSKI:  Yes, we do need some  
  
       17      accessibility, both design standards and -- the performance  
  
       18      standards are with respect to usability.  With respect to  
  
       19      facilities, I think we need a -- not these don't go through  
  
       20      the ITA's because facility set-up is done at the polling  
  
       21      place.  
  
       22                   We want to make sure we have good facility  
                                                                      110  
  
  
  
  
        1      layout, including good accessibility written in such a way  
  
        2      the documentation is plan language, and easy to read for  
  
        3      our poll workers so that we make sure that they all have a  
  
        4      good understanding of how to set things up properly.  
  
        5                   MR. ELEKES:  On that point, it may just be a  
  
        6      replication of work, because about a year and a half ago,  
  
        7      the U.S. Department of Justice came out with a check sheet  
  
        8      and a diagram for barrier-free access for polling places.  
  
        9              Now, with the election equipment that might be  
  
       10      built and specified, would those parameters have to be  
  
       11      modified, or is that a consideration for the human factor  
  
       12      to have the equipment fit into the recommendations as  
  
       13      published by the Justice Department.  
  
       14              DR. LASKOWSKI:  Well, there's accessibility  
  
       15      recommendations at the polling place of the set up of  
  
       16      equipment.  There are other considerations; for example, if  
  
       17      there's audio use, are they far enough apart for privacy,  
  
       18      also, on screen glare, it's not accessibility in terms of  
  
       19      the disabled user, but just in general, is your lighting  
  
       20      set up properly, and your polling booth set up so the  
  
       21      screen is easily readable.  So it does depend on the  
  
       22      equipment itself.  And so I think there's a set of  
                                                                      111  
  
  
  
  
        1      guidelines for accessibility and a design kind of  
  
        2      guidelines that would be specific to the technology based  
  
        3      upon the recommendation for equipment.  
  
        4                   MR. HARDING:  Yes, J. R. Harding.  Based on  
  
        5      the earlier comments by our chair, would there have been  
  
        6      room within this discussion for (inaudible) testing  
  
        7      authority.  Does that contribute to that process of  
  
        8      understanding the accessibility or the requirements.  
  
        9                   DR. LASKOWSKI:  I guess I'm not sure I  
  
       10      understand your question.  The ITA's currently don't test  
  
       11      for usability.  I think they're waiting for guidance on  
  
       12      what conformance tests ought to be.  I believe on Task  
  
       13      Group 3, there is at least one person from one of the test  
  
       14      laboratories.  Isn't that right?  
  
       15                   MS. QUESENBERY:  Not active, they're not  
  
       16      active.  There's someone from one of the test labs who's on  
  
       17      the list and has occasionally chimed in, but she hasn't  
  
       18      been reactive in writing.  
  
       19                   DR. LASKOWSKI:  We don't see any immediate  
  
       20      contributions from those parties at this point.  But we  
  
       21      certainly would want to be inclusive here.  
  
       22                   MS. QUESENBERY:  If I could chime in, this  
                                                                      112  
  
  
  
  
        1      is Whitney Quesenbery.  One of the things we've been doing  
  
        2      is looking at other standards. There's an ISO standard  
  
        3      that's being developed for usability testing of walk up and  
  
        4      touch kiosks, and that seems like that work would be  
  
        5      relevant.  They're also trying to define a test procedure.  
  
        6      We're trying to leverage some of that work as well.  
  
        7                   So there are some standards being developed  
  
        8      in other fields that are applicable, although, not directly  
  
        9      applicable to voting.  
  
       10                   MS. TURNER BUIE:  Sharon Turner-Buie.  It  
  
       11      appears that all of the guidelines and standards that are  
  
       12      set up will be developed using an ideal situation as a  
  
       13      benchmark.  And you look at usability/accessibility.  We  
  
       14      will adopt the standards based on an ideal set of poll  
  
       15      situations.  
  
       16                   Will these guidelines also contemplate or  
  
       17      provide standards for those facilities than are less than  
  
       18      ideal, that will be encountered by the elections  
  
       19      administrators?  
  
       20                   DR. LASKOWSKI:  I think there's two levels  
  
       21      of test.  One is the (inaudible) test, which, of course is  
  
       22      in a vacuum in some sense and it's independent of the  
                                                                      113  
  
  
  
  
        1      different kinds of environments that are depending on the  
  
        2      state and precincts.  
  
        3                   If you look at some of the other details or  
  
        4      recommendations that are on the map, one of the  
  
        5      recommendations is also for a for a usability testing  
  
        6      that's done by election officials for their actual ballots,  
  
        7      knowing what their situations are like in the polling place  
  
        8      to get some idea of potential problems.  And I think that's  
  
        9      about -- so not necessarily guidelines, but certainly  
  
       10      guidelines on how to do some usability testing and setup to  
  
       11      avoid problems due to the environment of the polling place  
  
       12      itself.  
  
       13                   Now, of course, if you violate some  
  
       14      temperature range on the equipment, you know, then all bets  
  
       15      are off.  
  
       16                   MR. WILLIAMS:  I'd like to clarify  
  
       17      something.  This is Britt Williams.  When you talk about  
  
       18      the ITA's and the FAC testing and all, we tend to talk at  
  
       19      the federal level.  The FAC standard actually defines three  
  
       20      levels of testing, not only the ITA level, but it defines a  
  
       21      level of testing that should be done at the state level,  
  
       22      and a level of testing that should be done at the local  
                                                                      114  
  
  
  
  
        1      jurisdiction level.  
  
        2                   DR. LASKOSWKI:  Yes, there's also the  
  
        3      certification acceptance testing --  
  
        4                   DR. WILLIAMS:  Yeah, that's what I'm talking  
  
        5      about certification at the state level.  
  
        6                   DR. LASKOWSKI:  Right.  When we're talking  
  
        7      about the standards like the VSS, I've concentrated mainly  
  
        8      to report on the ITA testing that's part of qualification;  
  
        9      but there are some recommendations before that do say you  
  
       10      need to do some of this testing locally as well with  
  
       11      respect to usability.  
  
       12                   DR. WILLIAMS:  And I would expect whatever  
  
       13      standards we develop to make take same approach.  
  
       14                   MR. CRAFT:  Paul Craft.  We have dabbled  
  
       15      recently in usability and accessibility standards for  
  
       16      beginning with the publications of 2001 standard.  We  
  
       17      brought in both a standard set of ballot layout  
  
       18      requirements, and we also have been experimenting actually  
  
       19      using what's a very preliminary rough of information  
  
       20      available standards for audio ballots for accessibility and  
  
       21      developed a procedure on that.  
  
       22                   The biggest problem from our perspective is  
                                                                      115  
  
  
  
  
        1      developing metrics.  We approached it taking those things  
  
        2      that we clearly knew were issues and those things that we  
  
        3      learned about of the test methodology.  It's quite doable.  
  
        4                   DR. LASKOWSKI:  Oh, it is.  I think it is,  
  
        5      and I think that's something for the subcommittee on the  
  
        6      Usability, Human Factors and Privacy to consider in making  
  
        7      progress on those.  
  
        8                   MR. HARDING:  Just for those of us who  
  
        9      aren't quite as familiar with the whole voting application,  
  
       10      could we get one more review of the way you had the three  
  
       11      level of tiers, I guess, ITA's and the national votes,  
  
       12      because I believe the disabled community would really like  
  
       13      to know where that certification or standard process --  
  
       14      where folks can really contribute and participate in their  
  
       15      communities at the local, the state versus federal, and how  
  
       16      might we then ultimately agree to that standard.  
  
       17                   CHAIRMAN BEMENT:  Care to address that,  
  
       18      Britt?  
  
       19                   DR. WILLIAMS:  Sure, be happy to.  This is  
  
       20      Britt Williams.  The testing that's done at the national  
  
       21      level is the testing of the features that are going to  
  
       22      ultimately wind up in the system.  
                                                                      116  
  
  
  
  
        1                   Now, when you get to the state level,  
  
        2      primarily what you're looking at at the state level is  
  
        3      compliance with state requirements, state laws, state  
  
        4      rules, and state regulations that are not addressed at the  
  
        5      national level.  
  
        6                   When something comes to us from the ITA's,  
  
        7      we assume that it's accurate, for instance.  We assume that  
  
        8      it's functional within the functional definitions and the  
  
        9      standards and so forth.  And we test for usability at the  
  
       10      state level.  The ITA's do not do any usability testing  
  
       11      that regards election officials, how difficult it is to run  
  
       12      an election with, that sort of thing.  It's a checklist:  
  
       13      Does it have this feature?  Yes.  Does it have this  
  
       14      feature?  Yes.  
  
       15                   It does not look at how gracefully that  
  
       16      feature is implemented.  We do that at the state level.  We  
  
       17      bring in some local election directors, have them look at  
  
       18      the system, and give us their opinions how easy it would be  
  
       19      or difficult it would be to use the system.  
  
       20                   The federal level does not look at  
  
       21      affordability.  They don't care what it costs.  They don't  
  
       22      even know what it costs.  We very much care what it costs  
                                                                      117  
  
  
  
  
        1      at the state level.  Then when you get down to the local  
  
        2      level, in most states, the local level is the procuring  
  
        3      level.  They're the ones that wrote the procurement policy.  
  
        4                   So for acceptance testing, there generally  
  
        5      are two dimensions. One is you're verifying that what got  
  
        6      delivered at that local level is, in fact, the same thing  
  
        7      that was qualified at the federal level and certified at  
  
        8      the state level.  And then the second thing you're  
  
        9      verifying is that it complies with the procurement  
  
       10      documentation.  So that's generally a description of the  
  
       11      three levels.  
  
       12                   MR. CRAFT:  Paul Craft.  Another perspective  
  
       13      on that is the Florida approach where we really -- we do  
  
       14      not recognize the federal qualification process at all.  We  
  
       15      have our own state level qualification/certification  
  
       16      program.  
  
       17                   We do use some of the work product out of  
  
       18      the federal labs.  But our view of it is the certification  
  
       19      testing evaluates the system to make sure that it meets the  
  
       20      established standard.  And once again in our case, as with  
  
       21      the federal program, our legislature has directed us to  
  
       22      leave the cost and the usability pieces in large part to  
                                                                      118  
  
  
  
  
        1      the local jurisdiction.  
  
        2                   The question of how difficult is it, and  
  
        3      what kind of resources does it take to lay out a ballot and  
  
        4      stage an election, that has been directed to the cost  
  
        5      benefit analysis of the local jurisdictions.  
  
        6                   Our state would level casting the votes and  
  
        7      assuring the system is properly operated according to  
  
        8      documentation can be reliably used and the performance  
  
        9      expected.  And then we look to the county government to do  
  
       10      their acceptance testing and to, in their procurement  
  
       11      process, specify what their standards for the system is,  
  
       12      and then do acceptance testing to verify those standards  
  
       13      are being met.  
  
       14                   Then, of course, the third level of testing  
  
       15      is the logic and accuracy testing which relies on both the  
  
       16      two higher levels of testing, actually verifies that the  
  
       17      programming of the systems in this user's hands has been  
  
       18      properly done.  And then you follow back on that with an  
  
       19      audit process where we can verify the system against our  
  
       20      benchmarks of what we have tested for.  
  
       21                   CHAIRMAN BEMENT:  Do you want to add  
  
       22      anything to that, Sharon?  
                                                                      119  
  
  
  
  
        1                   DR. LASKOWSKI:  I just thank you very much  
  
        2      for giving me this opportunity to talk about the NIST  
  
        3      report.  
  
        4                   CHAIRMAN BEMENT:  Okay.  The next person, Ed  
  
        5      Roback, is probably one of the most overworked division  
  
        6      directors at NIST.  
  
        7                   Ed is responsible for the security division,  
  
        8      which writes the federal standards for information  
  
        9      processing for the entire federal government, and a lot  
  
       10      more, smart cards, and whatever.  He'll provide his  
  
       11      background.  
  
       12                   MR. ROBACK:  Mr. Chairman, members of the  
  
       13      Committee, it's been an honor to be here this morning and  
  
       14      be with you.  
  
       15                   SPEAKER:  Can you speak a little louder?  
  
       16                   MR. ROBACK:  All right, a little louder.  
  
       17      What I thought I'd do is give you a little sense of what we  
  
       18      do out at NIST in the area of computer securities, since we  
  
       19      will obviously be helping to support the activities of the  
  
       20      TGDC.  
  
       21                   First, I'd mention that NIST does, under the  
  
       22      Federal Information Security Management Act of 2002, set  
                                                                      120  
  
  
  
  
        1      the computer standards for the entire federal government.  
  
        2      We're a non-regulatory agency but we use inspectors general  
  
        3      and chief information officers and so forth to implement  
  
        4      security across the entire federal landscape, except for a  
  
        5      very small set of systems that are classified for the  
  
        6      national security system.  
  
        7                   So when we do standards and guidelines, we  
  
        8      range from technical, things like encryption algorithms all  
  
        9      the way up to sort of management standards and guidelines,  
  
       10      the operating systems dealing with questions of risk,  
  
       11      capital planning, procurements, and so forth.  
  
       12                   One of the things we're doing right now is  
  
       13      writing a standard for minimum security requirements for  
  
       14      all federal systems.  In the past, we've done standards  
  
       15      here and there, but Congress has now tasked us to do a  
  
       16      standard that addresses the minimum security requirements  
  
       17      for all federal systems at three levels of sensitivity.  So  
  
       18      we're very busy working on that, and we have a deadline of  
  
       19      December of 2005 to complete that.  
  
       20                   I think it would be a fair bit of work doing  
  
       21      something like that that could translate to work for voting  
  
       22      systems, particularly, when you look at voting systems not  
                                                                      121  
  
  
  
  
        1      just on the tabletop, but an aggregation of systems in  
  
        2      terms of networking and tabulation and so forth.  
  
        3                   Secondly, we also have authority under the  
  
        4      Cyber Security Act where we have authority to run grant  
  
        5      programs that provide support for outside researchers and  
  
        6      so forth.  So in the longer term, if there's a need for  
  
        7      that sort of thing, we do have statutory responsibility to  
  
        8      do that.  
  
        9                     We have an interesting assignment that may  
  
       10      or may not play in your voting equipment, depending upon  
  
       11      the way that the equipment is set up in terms of commercial  
  
       12      off-the-shelf equipment.  NIST has the responsibility to  
  
       13      come up with security configuration advice for IT products  
  
       14      in the marketplace.  So when a company or a particular  
  
       15      Federal agency goes out and procures a certain device or a  
  
       16      certain piece of software, NIST is developing advice as to  
  
       17      how to actually configure that particular product for  
  
       18      securities, again, based on one of these three levels in  
  
       19      terms of sensitivity.  
  
       20                   We will be working very closely with vendors  
  
       21      because, obviously, there's a huge number of products, out  
  
       22      there are unlimited programs.  But that's something that's  
                                                                      122  
  
  
  
  
        1      also in our bailiwick in terms of use of commercial  
  
        2      equipment.  
  
        3                   I have given you a few slides just to remind  
  
        4      you about some of this.  And this is just to remind you of  
  
        5      some of the security issues in the area of voting systems.  
  
        6                   I preface this with just a general comment  
  
        7      about security.  Security is typically filed (inaudible) as  
  
        8      a change.  And it's far easier to have something that's  
  
        9      insecure than something that is secured.  In fact, most  
  
       10      security people would never go out and say something like  
  
       11      this system is secure.  I don't think any of us would ever  
  
       12      want to declare that.  
  
       13                   But it actually doesn't take very much to  
  
       14      find a single little flaw, and that becomes the headline  
  
       15      that the system is broken, so you break one little link in  
  
       16      the chain.  Again, this is just to remind us of the breadth  
  
       17      and scope of security issues.  Obviously, it's going to be  
  
       18      up to the TGDC to consider questions of:  Well, how much  
  
       19      security is enough?  This again gives you sort of a  
  
       20      sampling of the kind of security issues that we heard at  
  
       21      the NIST workshop and, obviously, a concern of many others.  
  
       22                   There are accidental errors and omissions  
                                                                      123  
  
  
  
  
        1      that are typical security problems, and the quality of data  
  
        2      kinds of things, voter manipulation.  If you've ever been  
  
        3      online buying something, you hit that enter key, it's  
  
        4      purchased.  I want to buy it, and you're sort of thinking:  
  
        5      Did I hit that key once or twice?  Did I buy one or two of  
  
        6      them.  Well, obviously, you will see that on the credit  
  
        7      card; but if you hit that twice on the voting machine, you  
  
        8      voted in general, but the nation hopes you're not voting  
  
        9      twice.  
  
       10                   Vote manipulation, obviously, in the  
  
       11      tallying process, adding/deleting votes, is the tabulation  
  
       12      being done correctly?  There are risks to the modification,  
  
       13      whether inadvertent or deliberate, to hardware and  
  
       14      software.  So you're thinking now things down at the chip  
  
       15      level, does the chip have strong integrity in terms of is  
  
       16      it doing what's expected.  Some years ago there was a chip,  
  
       17      a mathematical processor, and was doing very complex  
  
       18      mathematics in terms of (inaudible) for operations, but it  
  
       19      was not actually doing them accurately.  
  
       20                   There are some errors like that in basic IT  
  
       21      infrastructure components.  These are a range of issues you  
  
       22      may have think about, issues of integrity of software.  So  
                                                                      124  
  
  
  
  
        1      that the software that's loaded on a system and actually  
  
        2      being used, is that the same software that's gone from  
  
        3      testing; is it the same software that's being ordered from  
  
        4      the vendors and so forth.  
  
        5                   Audit trails of whatever sort, do you have  
  
        6      integrity of the audit trail.  Modification/prevention of  
  
        7      vote recording, is there privacy.  Adding vote data, adding  
  
        8      duplicate votes, all of these sorts of risks.  
  
        9      Modifications in tallying process, in transit process, not  
  
       10      just when they're actually being tallied, but when they're  
  
       11      being transmitted.  
  
       12                   Preventing access to individual votes, if  
  
       13      that's the policy of vote tallies and so forth.  Denial of  
  
       14      service.  If you found a way to bring down the voting  
  
       15      systems, you obviously can't get the citizen's votes in.  
  
       16                   It's a range of these things.  And,  
  
       17      typically, we talk about having something called the CIA  
  
       18      model.  The CIA model basically refers to the  
  
       19      confidentially, integrity, and availability, for those of  
  
       20      you not in the field.  
  
       21                   So we think about the things that need to be  
  
       22      protected from disclosure, which is confidentiality, and  
                                                                      125  
  
  
  
  
        1      the things that need integrity, which includes hardware and  
  
        2      software, protection from inadvertent or deliberate  
  
        3      modification; and the availability of systems, processes,  
  
        4      and so forth.  
  
        5                   Thinking about some of those risks, I have  
  
        6      given you a table here just to give you a sense that NIST  
  
        7      has already done some work, although, not specific to  
  
        8      voting systems, but to address some of these kinds of  
  
        9      risks.  And that also gives you some of the security  
  
       10      controls that are intended to support.  
  
       11                   So things like access control.  What is  
  
       12      access control?  It has to do with who is allowed to access  
  
       13      what information and what system, for example.  We have  
  
       14      guidelines that address aspects of that.  
  
       15                   Assurance, when I use the word "assurance,"  
  
       16      it is strongly tied to security functionality.  What it has  
  
       17      to do with is a term that us security people use.  
  
       18                   Assurance has to do with what degree of  
  
       19      confidence you have that the system does what it is  
  
       20      intended to do, what the specification says, it is doing  
  
       21      it.  
  
       22                   Security is a little different than some  
                                                                      126  
  
  
  
  
        1      other kinds of functionality.  A typical example I give is  
  
        2      hook up a printer to your PC.  When you hook up a printer,  
  
        3      you turn it on, you send something to print, you can tell  
  
        4      whether it printed or not, you look at it.  When you hook  
  
        5      up some security software, you turn it on, and it sits  
  
        6      there and hums, and you sort of don't know if it's really  
  
        7      filtering all the traffic that I don't want to come to my  
  
        8      system.  Is it keeping out intruders.  Maybe someone came  
  
        9      in and copied some data and got out and you'd never know.  
  
       10                   So security is not always evident or even  
  
       11      obvious, whether or not it's functioning correctly.  So  
  
       12      assurance has to do with an aspect of testing, it is the  
  
       13      degree to which you want confidence that the security is  
  
       14      operating as intended.  And assurance, like security, you  
  
       15      can have as much as you want, depending on how much you're  
  
       16      able to pay.  And there was a comment earlier about who is  
  
       17      responsible for thinking about how much, and who actually  
  
       18      cares how much it costs to do these things.  
  
       19                   I've also gone through some examples here of  
  
       20      integrity, auditing, confidentiality and some of the  
  
       21      others.  And you'll see here the reference to our various  
  
       22      security standards and guidelines.  These are all available  
                                                                      127  
  
  
  
  
        1      on our web pages.  These are hundred and hundreds of pages  
  
        2      of various guidelines that are out there to help people.  
  
        3                   Just to give you a thumbnail of our actual  
  
        4      program at NIST.  We have about 45 folks in the area of  
  
        5      computer security in our division.  We have competence and  
  
        6      strong activities in the area of cryptographic standards  
  
        7      and guidelines.  This has to do with things like encryption  
  
        8      and digital signatures.  We also do a lot of work in the  
  
        9      ares of electronic authentication, and a number of key  
  
       10      government initiatives across the Federal Government space.  
  
       11                   We do a fair bit of -- NIST is a research  
  
       12      institution, and as you expect, a fair bit of research in  
  
       13      the area of computer security, and we raise the  
  
       14      specification element of testing.  
  
       15                   Some of the things we're doing now are in  
  
       16      the area of smart cards.  We will all be carrying around  
  
       17      smart cards one of these things days that have things like  
  
       18      cryptographic keys on them.  We're working in areas that  
  
       19      have to do with wireless security.  We think about the  
  
       20      security problems that might arise with the risks, but if  
  
       21      you start thinking about voting systems and polling places  
  
       22      and the voting has some sort of wireless net, you may be  
                                                                      128  
  
  
  
  
        1      exposed to typical kinds of risks.  
  
        2                   I talked about checklists and benchmarks.  
  
        3                   On the management side, we also address the  
  
        4      management issues, policy development, policy guidance,  
  
        5      personnel and training, education and awayness, some of the  
  
        6      management sides of security as well because, obviously,  
  
        7      systems cannot operate alone.  
  
        8                   We also have two programs that may well be  
  
        9      very useful in conjunction with the National Voluntary  
  
       10      Laboratory Accreditation Program that you're going to be  
  
       11      hearing about; and encourage that we do testing in  
  
       12      conjunction with the government of Canada on cryptographic  
  
       13      module.  So when the Federal agency needs cryptography,  
  
       14      they have to use a module that's gone through testing to  
  
       15      give a higher degree of degree that the algorithm  
  
       16      specifications and security of cryptography are at an  
  
       17      adequate level for Federal use.  
  
       18                   If you're talking about voting systems with  
  
       19      cryptography that may provide functions you need, security  
  
       20      functions you need, this might be something you want to  
  
       21      consider.  
  
       22                   We also have a guideline and a process in  
                                                                      129  
  
  
  
  
        1      place in the Federal Government for something called  
  
        2      "certification and accreditation."  That is, this is a  
  
        3      different use of the word "certification" than used  
  
        4      earlier.  I just want to warn you about that.  
  
        5                   In the Federal space, before a system is  
  
        6      turned on, a management official is supposed to sign their  
  
        7      name on the line saying, "I hereby accept the risk of  
  
        8      operating the system, a technical review has been  
  
        9      accomplished."  
  
       10                   A technical review in this context is called  
  
       11      a "certification."  And the authorization by the management  
  
       12      officials is called an "accreditation."  That might be a  
  
       13      concept that you all in security requirements -- that might  
  
       14      be a particular you may want to think about whether or not  
  
       15      there's any analog to do.  
  
       16                   And, finally, we have a program that we run  
  
       17      with the National Security Agent called the National  
  
       18      Information Assurance Partnership.  Earlier, I talked about  
  
       19      the program and test cryptographic modules. This program  
  
       20      provides for the evaluation of IT products.  And what it  
  
       21      does is allow a vendor to bring in their product to one of  
  
       22      our laboratories, bring in a set of specifications -- the  
                                                                      130  
  
  
  
  
        1      vendor says, I claim my product does these security  
  
        2      features -- take them to a laboratory, bring their product,  
  
        3      they bring their specification and they bring in a degree  
  
        4      to which they want to test it, an assurance level, and then  
  
        5      they can go through testing.  
  
        6                   Now, this testing is a little different from  
  
        7      the testing Sharon talked about, because it's not at the  
  
        8      bit and bytes level.  I think sometimes we talk about  
  
        9      conformance testing, this is a little higher, but  
  
       10      evaluation-type testing.  With everything else, it provides  
  
       11      some useful interesting research.  
  
       12                   There are other NIST security-related  
  
       13      competencies and activities.  I won't go through a lot of  
  
       14      these, but work at NIST is security protocols, network  
  
       15      security, forensics, and biometrics, and you'll be hearing  
  
       16      about some of those as well.  
  
       17                   The last page is my contact information.  
  
       18      That's just a quick thumbnail to give you a sense of what  
  
       19      we do at NIST in the area of security.  
  
       20                   I'll be happy to answer any questions.  
  
       21                   CHAIRMAN BEMENT:  J. R.  
  
       22                   MR. HARDING:  Thank you, Mr. Chairman.  I  
                                                                      131  
  
  
  
  
        1      heard an awful lot of standards in terms of Congressional,  
  
        2      but where do we draw the lines for statistical analysis on  
  
        3      the confidence level or the significant level, you know,  
  
        4      can you conclude something.  Do we have those baselines in  
  
        5      your shop now, or we will we be creating that level of  
  
        6      confidence at 95, 99, 80?  
  
        7                   MR. ROBACK:  I wish it were as quantified as  
  
        8      you are suggesting.  It is not.  What we have done in the  
  
        9      Federal space is said, here are definitions of there levels  
  
       10      of sensitivity based on the risks.  And then we're going to  
  
       11      say that for each of those three levels, here's a minimum  
  
       12      set of controls.  It's up to you, ultimately, as the owner  
  
       13      of the system or the operator to have to make a decision  
  
       14      whether or not you're willing to accept the risk, because  
  
       15      nobody's saying that that set of controls is going to beat  
  
       16      every considerable risk.  
  
       17                   So in the end, people always talk in risk  
  
       18      management, we're always talking security resource  
  
       19      tradeoffs.  That's just the nature of security.  
  
       20                   CHAIRMAN BEMENT:  If I may comment on that  
  
       21      just for a moment.  In dealing with risk management, and  
  
       22      this is where the people, procedures, and technology comes  
                                                                      132  
  
  
  
  
        1      in.  It's a question of where you draw the line between  
  
        2      risk avoidance, risk mitigation, and risk acceptance.  
  
        3                   In some cases, if you have the right  
  
        4      procedures, and the right people training, you can do a  
  
        5      fair amount of mitigation.  It's pretty hard in real life  
  
        6      to do total risk avoidance, certainly by technology alone.  
  
        7      In many cases you can manage risk as well.  
  
        8                   MR. ROBACK:  Just picking up on that, you  
  
        9      can think in terms of do you set security requirements at a  
  
       10      very high level, which are usually called control methods;  
  
       11      that is, the system should be capable of keeping out  
  
       12      intruders, something really really, high like that.  But  
  
       13      that would mean the ultimate in terms of data as a  
  
       14      technical measure or a managerial security control or some  
  
       15      combination thereof to achieve that.  And you can delve all  
  
       16      the way down to saying the requirements are at the bits and  
  
       17      bytes level, if you're going to do encryption, you must use  
  
       18      this.  
  
       19                   MR. CRAFT:  Paul Craft.  As one who's been  
  
       20      battered fairly frequently in the cross over whether or not  
  
       21      we're doing effective standards for security, this is  
  
       22      something somehow I'd very much like to see NIST bring some  
                                                                      133  
  
  
  
  
        1      sanity into this process.  There are a tremendous number of  
  
        2      people out there who are yelling, advocating security,  
  
        3      whose view of security is to put a very high level of  
  
        4      encryption on everything.  
  
        5                   In one of the recent reports, I think the  
  
        6      Ohio report, they demanded the election results be  
  
        7      encrypted for telecommunication from the polling place to  
  
        8      the central count.  
  
        9                   Well, at point where you're  
  
       10      telecommunicating those results, those are now public  
  
       11      documents.  If they've already been published, there is  
  
       12      really no reason to keep the contents of those  
  
       13      transmissions secret.  There is, however, a very high level  
  
       14      of interest in digitally styling those so that you can  
  
       15      authenticate it to validate.  My concern watching it, as  
  
       16      I've seen the vendor community try to adapt to these  
  
       17      criticism as they're putting a lot of procedures, and a lot  
  
       18      of CPU time into high levels security in areas that don't  
  
       19      really need that kind of security.  So, I'm really looking  
  
       20      forward to you all helping bring some sanity.  
  
       21                   MS. DAVIDSON:  Donnetta Davidson.  I think  
  
       22      to tag on to what Paul was talking about.  My question is:  
                                                                      134  
  
  
  
  
        1      Is there anything being done or will they be doing anything  
  
        2      to study what has already been accomplished like the sanity  
  
        3      issues?  
  
        4                   A lot of these, I mean, people really don't  
  
        5      understand the auditing and the process that's been taking  
  
        6      place, and the security that has been put into, you know,  
  
        7      the efforts of making sure that the elections are running  
  
        8      properly to know how we can improve it.  
  
        9                   I mean, I hate to see us spend a lot of time  
  
       10      in writing something and we're already doing it.  So is  
  
       11      there going to be somebody that's really going to go out  
  
       12      right now with the security and with the efforts of  
  
       13      auditing and testing before, and testing after, and how to  
  
       14      secure all the information.  I think that we almost need to  
  
       15      know where the vendor community is at right now to know how  
  
       16      we can improve.  
  
       17                   CHAIRMAN BEMENT:  I think I can refer back  
  
       18      to the charge that Chairman Soaries this morning pointed  
  
       19      out that we don't want to reinvent the wheel.  We want to  
  
       20      start with what is already in place and improve it, if we  
  
       21      can.  
  
       22                   MR. ROBACK:  Obviously, I can't speak to the  
                                                                      135  
  
  
  
  
        1      activities of the subcommittee, but one would naturally  
  
        2      expect that some sort of inventory of current best  
  
        3      practices would be an important part of their kick off.  
  
        4                   DR. RIVEST:  Thanks for the great  
  
        5      presentation.  I had a question at a very high level of  
  
        6      NIST's philosophy about standards in the area of security.  
  
        7      One thinks about the fact that security is sort of a  
  
        8      negative attribute -- in the absence of vulnerability --- I  
  
        9      may be using my terminology wrong.  And also one strives to  
  
       10      be technology independent, so trying to rise above that.  
  
       11                   MR. ROBACK:  There were two or three very  
  
       12      interwoven questions there and issues.  I'll take those  
  
       13      apart.  They're very inciteful questions. The first is NIST  
  
       14      is generally charged with technology neutrality in security  
  
       15      standards development.  However, in some things like  
  
       16      cryptographic standards, in order to have interoperability,  
  
       17      you simply need to be using the same technique.  
  
       18                   So how has NIST dealt with that?  Well, NIST  
  
       19      has improved in some areas neutrality standards, a number  
  
       20      of different techniques, and then it's simply up to the  
  
       21      users.  So that sort of tries to straddle the fence.  
  
       22                   In the area of security, when we put  
                                                                      136  
  
  
  
  
        1      together these minimum requirements for all federal  
  
        2      systems, that will be our first stab really at saying, this  
  
        3      is what we see as a reasonable comprehensive set of  
  
        4      controls.  
  
        5              In the past we've done important work, but it's  
  
        6      been if you do this technique, this is how it should be  
  
        7      done, or this is a way to approach it.  As opposed to say,  
  
        8      that's a technique that must be used, and that if you use  
  
        9      it, you should use it with these other things.  So looking  
  
       10      at what constitutes a comprehensive set for the purpose of  
  
       11      voting systems, given the risks that you all see, and the  
  
       12      risks you're willing to accept is the challenge, of course.  
  
       13      I think that answers your question.  
  
       14                   CHAIRMAN BEMENT:  Anything else?  Thank you  
  
       15      very much.  
  
       16                   MR. ROBACK:  Thank you.  
  
       17                   CHAIRMAN BEMENT:  The next speaker is Mark  
  
       18      Skall, who is serving as Acting Director of the NIST  
  
       19      Information Technology Laboratory.  
  
       20                   MR. SKALL:  My name is Mark Skall.  I'm the  
  
       21      Acting Director of NIST's ITL.  Up until a week ago, I was  
  
       22      actually running the software testing division.  So we have  
                                                                      137  
  
  
  
  
        1      a lot of experience in testing against specifications.  
  
        2                   What I want to do this afternoon is give you  
  
        3      the NIST perspective on how to write quality  
  
        4      specifications.  Many of you, I'm sure, are well-versed in  
  
        5      this area.  
  
        6                   Just a little background on NIST's role in  
  
        7      the information technology industry.  We work industry and  
  
        8      Federal agencies to develop standards and tests to improve  
  
        9      the quality of software and achieve interoperable  
  
       10      solutions.  
  
       11                   We have many years of experience working  
  
       12      with formal standards, organization, like W3C, OASIS, ISO,  
  
       13      IEEE, and HL7.  Our experience dates back at least 30 years  
  
       14      or to the early 70's, and we started out working with  
  
       15      committees such as cobalt standards committees. We spent a  
  
       16      lot of time working on Internet standards such XML.  
  
       17                   What we do with the committees is helped  
  
       18      them develop good testing specifications.  We developed  
  
       19      conformance test suites, tools, and reference  
  
       20      implementations.  And we have in the past done a lot of  
  
       21      work in developing validation and certification testing  
  
       22      programs, but now we, except for the security area,  
                                                                      138  
  
  
  
  
        1      typically we help other people do certifications by helping  
  
        2      them set up procedures.  
  
        3                   The next slide is one of my favorite slides.  
  
        4      It shows the relationship among specifications,  
  
        5      implementation, and conformance testing.  The goal is to  
  
        6      develop trust and confidence in your software.  This is not  
  
        7      an isolated process.  There are may other factors involved.  
  
        8      Clearly we need good specifications with requirements,  
  
        9      which feed the input to the software.  
  
       10                   We need conformance tests which check to  
  
       11      makes sure that the implementation, in fact, conforms to  
  
       12      the requirements.  
  
       13                   If we look at these three parts, merely as  
  
       14      the three legs of a three-legged stool, they are absolutely  
  
       15      necessary if one of the legs fall out, the stool keels  
  
       16      over.  
  
       17                   With that having been said, the  
  
       18      specification is really the most important part.  
  
       19      So good specifications are really the key.  That's  
  
       20      something we preach time and time again.  The goal is to  
  
       21      correct reliable interoperable software and the  
  
       22      requirements for the software are captured in the  
                                                                      139  
  
  
  
  
        1      specifications.  
  
        2                   So if these requirements are wrong, think  
  
        3      about this for a second, everything that ensues from that  
  
        4      requirement, the software, the test, the certification are  
  
        5      all wrong.  So you have to get those requirements right.  
  
        6      So these are some of the things we think about when writing  
  
        7      specifications.  
  
        8                   Specs must use appropriate language to  
  
        9      designate requirements.  So you can have the best  
  
       10      requirements in the world, if the vendor doesn't know their  
  
       11      requirements, then you've lost the battle.  
  
       12                   MR. HARDING:  Mr. Chairman, can I interrupt  
  
       13      for a second?  From your experience in communicating to  
  
       14      vendors, are there more effective ways to ensure the  
  
       15      clarity of those things?  
  
       16                   MR. SKALL:  I think people typically do a  
  
       17      good job, and we'll get into that in the next slide.  So  
  
       18      please ask the question again.  
  
       19                   The specification must be precise,  
  
       20      unambiguous, and testable.  I think you heard that from  
  
       21      Sharon.  You have to tie things down the specification.  
  
       22      You can't speak in generalities.  
                                                                      140  
  
  
  
  
        1                   The spec must contain a conformance clause.  
  
        2      This is something I think we haven't discussed.  This is a  
  
        3      very important concept.  The user must understand what's  
  
        4      expected of him or her.  Those of us who get up at forums  
  
        5      like this and can speak using English, well, understanding  
  
        6      English is not the best way to be precise.  It's in fact  
  
        7      more difficult to use English to write a specification.  
  
        8                   In a perfect world we would use what we call  
  
        9      a formal specification mathematical notation to be more  
  
       10      precise in our requirement.  There are, of course, problems  
  
       11      with these, and there are not many people well-versed in  
  
       12      writing formal specifications.  They are not typically  
  
       13      readable by the public.  We can use semi-formal  
  
       14      specifications, things like XML. Again, there are problems  
  
       15      with these.  
  
       16                   I think for the issues of the voting  
  
       17      community, I use this world (inaudible) tend to be stuck  
  
       18      with the English language.  But to ensure that these things  
  
       19      are read by the public, you probably need to stick with  
  
       20      English, and that presents quite a few challenges.  Again,  
  
       21      English is not the best way to be precise.  
  
       22                   So I want to go through a few of these  
                                                                      141  
  
  
  
  
        1      points one by one.  Specs must use appropriate language.  
  
        2      There are key words that are internationally recognized.  
  
        3      "Must" and "shall" are the two words that are typically  
  
        4      used to indicate mandatory requirements.  They're both  
  
        5      equally acceptable.  Some communities use must and you  
  
        6      shall.  I think those are good languages to say that this  
  
        7      is a mandatory requirement.  
  
        8                   There are other words that are used to  
  
        9      recommend, "you should."  "May" is a weaker word that says  
  
       10      something is permissible to do.  These are all words that  
  
       11      are often used and are useful.  
  
       12                   In our experience, we try to put as few of  
  
       13      those as possible.  When you say they have no bearing on  
  
       14      conformance (inaudible) states you can certainly write  
  
       15      tests for things that say should, and then you can  
  
       16      determine if, in fact, one has this feature, which is an  
  
       17      option, and can make sure that this feature is implemented  
  
       18      correctly.  
  
       19                   Wording like, "good enough," "do the best  
  
       20      you can," are really not appropriate for specifications,  
  
       21      but you do see them in specifications.  
  
       22                   Specifications must be precise.  What I mean  
                                                                      142  
  
  
  
  
        1      by precise is one that can be able to determine if that  
  
        2      requirement has been fulfilled.  So if you're running a  
  
        3      program in the shop, you may have a requirement system  
  
        4      "shall guard against viruses, Trojan horses, and worms."  
  
        5      So you look at us and say, well, that's good key words.  
  
        6      But to say this in a more precise way, you may say, "an  
  
        7      anti-virus program shall be installed on ones system, and  
  
        8      anti-virus software checks and live updates shall be run  
  
        9      daily."  This is a requirement that you can determine  
  
       10      whether it's been met.  
  
       11                   Specifications must be unambiguous.  I think  
  
       12      we've heard this repeated.  This is really just that we've  
  
       13      worked with many many standards' communities over 30 years,  
  
       14      and with a lot of really smart people.  Yet you come up  
  
       15      with ambiguous and contradictory statements.  It's a  
  
       16      difficult process, and many of the specifications are very  
  
       17      very large.  You just have to be really careful to write  
  
       18      things that are not ambiguous.  
  
       19                   This is an example:  "The girl touched the  
  
       20      cat with a feather."  They might have been thinking in his  
  
       21      own mind that the girl was touching the cat, or they might  
  
       22      have been thinking, "the girl touching the cat, which has  
                                                                      143  
  
  
  
  
        1      the feather."  It just really shows how easy it is to write  
  
        2      things that in ones mind, it might seem clear at least two  
  
        3      ways to say it.  
  
        4                   Specifications must be testable.  Is it  
  
        5      better to have a requirement that's not testable, if we  
  
        6      can't figure out how to test it, or is it best just to  
  
        7      eliminate it?  I guess one of the rules of requirements is,  
  
        8      if it can't be tested, can't verify that it has been  
  
        9      satisfied, it's not of any use.  You need to reformulate  
  
       10      that requirement so it can be tested, or remove it from the  
  
       11      standards.  
  
       12                   Testable assertions will be derived from  
  
       13      normative mandatory requirements in the specification.  And  
  
       14      again, unspecified, ambiguous, or imprecise requirements  
  
       15      cannot be tested.  
  
       16                   MR. HARDING:  Mr. Chairman, along that line  
  
       17      about being able to test the requirement, is it our charge  
  
       18      to define that test?  
  
       19                   CHAIRMAN BEMENT:  Yes.  
  
       20                   MR. HARDING:  Thank you.  
  
       21                   MS. QUESENBERY:  I guess in my experience, I  
  
       22      also have life as writer, although, I wouldn't claim to be  
                                                                      144  
  
  
  
  
        1      a major plain language expert.  I have an aspiration to  
  
        2      become one.  But I wonder whether we ought to be including  
  
        3      some requirement that the language of our applications be  
  
        4      reviewed by someone who has expertise in writing of plain  
  
        5      language.  
  
        6                   As ideal as a mathematical language might  
  
        7      be, it's certainly not an appropriate language for a  
  
        8      usability and accessibility section.  And given the public  
  
        9      prominence of this, it's going to be very important that  
  
       10      people can read and understand what we mean very clearly.  
  
       11                   CHAIRMAN BEMENT:  Very good point.  
  
       12                   DR. RIVEST:  It seems like the appropriate  
  
       13      time.  I'd just like to register a --  
  
       14                   MR. BURKHARDT:  Excuse me.  Actually, I just  
  
       15      need to intervene.  Since your financial disclosure  
  
       16      statements haven't been approved, the only thing you can do  
  
       17      at this meeting is ask questions.  You can't express an  
  
       18      opinion.  That's just simply for your own personal  
  
       19      liability reasons.  Sorry to intervene.  
  
       20                   MR. RIVEST:  Okay.  
  
       21                   MR. HARDING:  Rephrase the question.  
  
       22                   CHAIRMAN BEMENT:  Rephrase the question.  
                                                                      145  
  
  
  
  
        1                   DR. RIVEST:  Do you feel that the  
  
        2      testability is appropriate for security requirements where  
  
        3      really the goal such as voter privacy may not be captured  
  
        4      by the specific checklist of items to be tested against,  
  
        5      but rather the ability to withstand (inaudible) you know,  
  
        6      may not be sufficiently captured by that checklist of test  
  
        7      items?  
  
        8                   MR. SKALL:  I think that's a fair statement.  
  
        9      My perspective in regard to the issue has been in  
  
       10      nonsecurity areas, so we're talking about functional  
  
       11      requirements typically.  I think security is a whole  
  
       12      different game.  I'm talking about functional requirements  
  
       13      with a specific shall or must.  There are many gray areas  
  
       14      involved in security.  
  
       15                    If I could just add to that.  In the area  
  
       16      of security, another example would be if you think about  
  
       17      some of the software programs, millions of millions of  
  
       18      them.  One of the security requirements probably would be  
  
       19      along the lines of "there shall be no malicious codes  
  
       20      barrier."  But we don't have right now, any good way to  
  
       21      test that.  Nonetheless, it's probably a good security  
  
       22      requirement that somebody not have inserted a trapdoor or  
                                                                      146  
  
  
  
  
        1      Trojan in there.  
  
        2                   So that would be one example where  
  
        3      unfortunately, the state of the art in terms of testing is  
  
        4      simply not where it should be.  
  
        5                   MR. BERGER:  Steve Berger.  Actually, I'd  
  
        6      like to comment on the difficult issues of repeatability  
  
        7      and uncertainty, should be in your thoughts, especially in  
  
        8      some of these areas whether we should be addressing the  
  
        9      likelihood of a test being repeatable and also the  
  
       10      uncertainty of having to repeat it.  
  
       11                   MR. SKALL:  Absolutely.  I try limiting my  
  
       12      presentation to susceptibility.  There are a whole bunch of  
  
       13      issues in testing and certainly repeatability is a key  
  
       14      requirement that should be there.  Uncertainty is a little  
  
       15      bit more difficult.  We've done some work in trying to  
  
       16      measure statistical uncertainty based on what would be the  
  
       17      probability that, in fact, implementation does, in fact,  
  
       18      conform, assuming it passes a set of conformance tests.  
  
       19      And that's a very difficult problem.  
  
       20                   And, of course, the more comprehensive the  
  
       21      tests are, the more probable it is that implementation does  
  
       22      in fact conform.  It certainly is an issue that needs to be  
                                                                      147  
  
  
  
  
        1      addressed in testing, and certainly is very important.  
  
        2                   MS. QUESENBERY:  I have a question that I  
  
        3      struggled with in writing other standards, which is, how do  
  
        4      you create -- this is probably not a short answer, so I'm  
  
        5      just really putting it on the table.  But how do you handle  
  
        6      creating a standard where it's going to take a number of  
  
        7      requirements together to create the end result you wish,  
  
        8      especially if they cross sections?  
  
        9                   I'm thinking of accessibility, privacy, and  
  
       10      usability, which all need to be considered as a whole in  
  
       11      order.  So you could meet individual requirements and still  
  
       12      not meet the whole.  
  
       13                   How do you make sure that you've written a  
  
       14      standard that does, in fact, achieve the top level goal?  
  
       15                   MR. SKALL:  I did the government system  
  
       16      requirements, and that's something we discussed.  It's a  
  
       17      two-tiered process.  You first have to define individual  
  
       18      requirements, make sure those are met.  And then you have  
  
       19      to look and see what system requirements are requirements  
  
       20      that flow through the intersection on top of all of those.  
  
       21                   You have to think about those things.  And,  
  
       22      again, all you can do is ask to meet everything you have in  
                                                                      148  
  
  
  
  
        1      your specifications requirements.  So if you need to do  
  
        2      other things that the systems will ensure that everything  
  
        3      is done, and you need to think about how to express that.  
  
        4                   MS. QUESENBERY:  It would be great to have  
  
        5      some examples of that.  
  
        6                   CHAIRMAN BEMENT:  If I may for a moment.  
  
        7      I'm a little nervous about the way I answered your  
  
        8      question, J. R., with regard to testing.  They're three  
  
        9      levels of consideration.  First of all, what needs to be  
  
       10      tested; secondly, in what way should it be tested; and  
  
       11      third, what is test to be used?  
  
       12                   MR. HARDING:  Right.  
  
       13                   MR. CRAFT:  I recently had the pleasure of  
  
       14      defending some of our subjective standards and some  
  
       15      findings addressed in our subjective standards in Federal  
  
       16      Court, and fortunately, we prevailed.  
  
       17                   What we had attempted to do -- and I think  
  
       18      this is going to become difficult for this Committee,  
  
       19      particularly in the areas of accessibility and usability --  
  
       20      there will come points where you very clearly know what you  
  
       21      want to enforce, but there's not going to be a good metric  
  
       22      to measure.  
                                                                      149  
  
  
  
  
        1                   And what we had attempted to do, and what we  
  
        2      prevailed with legally was basically establishing a  
  
        3      reasonable manned position on compliance with that  
  
        4      particular standard.  
  
        5                   How comfortable is NIST going to be with  
  
        6      that type of imprecise standards when you get to the point  
  
        7      where there really is no other way to address the issue?  
  
        8                   MR. SKALL:  I think we're here just to give  
  
        9      input to the TGDC, and to tell you what our experiences  
  
       10      are, and what we see.  I think that there are going to be  
  
       11      individual decisions that are going to have to be made,  
  
       12      based on subject matter.  I think we have to make those  
  
       13      decisions one by one.  In general, I think we all agree  
  
       14      that the more precise you can be, the better it is.  If you  
  
       15      can't be precise, I think that's a case by case decision  
  
       16      that has to be made about how to best do it in such a way  
  
       17      to express the intent of what you want to try to ensure  
  
       18      that it is met.  
  
       19                   CHAIRMAN BEMENT:  If I might interject.  
  
       20      What has to distinguish between precision and specificity  
  
       21      are basically two entirely different things.  
  
       22                   MR. BERGER:  Can I just offer a parallel  
                                                                      150  
  
  
  
  
        1      from the FCC where they're trying to deal with  
  
        2      interference.  They've got a clause in all their  
  
        3      requirements that if they find a product that causes  
  
        4      interference, they will shut it down, no matter what the  
  
        5      specifications there are.  
  
        6                   We probably need something like that.  The  
  
        7      authority to automatically have the right to say that  
  
        8      doesn't meet the requirement.  
  
        9                   MS. DAVIDSON:  Are you referring to like a  
  
       10      decertification?  
  
       11                   MR. BERGER:  Yeah, basically it's a safety  
  
       12      net with the Commission.  They're required to assure that  
  
       13      undue interference is a possibility, and there's clearly a  
  
       14      possibility that despite the best efforts of all these  
  
       15      tests, with technical specifications, something slips  
  
       16      through.  
  
       17                   And I think we have some similar things,  
  
       18      some high-level requirements.  Potentially, no matter how  
  
       19      well we do our job, something slips through, a major  
  
       20      security flaw, or something else and the examiner sees it.  
  
       21      And I think in our specifications, there ought to be  
  
       22      something that says, where that's clearly the case, that  
                                                                      151  
  
  
  
  
        1      takes precedent over all the detail underneath.  
  
        2                   CHAIRMAN BEMENT:  You have a followup  
  
        3      question?  
  
        4                   MS. DAVIDSON:  I think that's very good.  
  
        5      Yes.  Donnetta Davidson.  Because I think that once in a  
  
        6      while, we do find a flaw even when we're testing before an  
  
        7      election and, that piece of equipment or, you know,  
  
        8      whatever the problem might be, that is not used and,  
  
        9      obviously, we do not want it used.  
  
       10                   But there's also an issue that at times  
  
       11      states have decertified a certain make of equipment because  
  
       12      it's not meeting the standards that they felt it should.  
  
       13      So that was really why I was asking if you're referring.  I  
  
       14      think that there needs to be any time that they find that  
  
       15      there is issues, definitely they have procedures set forth  
  
       16      that they can decertify or not use the equipment, whatever  
  
       17      words you want to use.  I agree whether it's the equipment  
  
       18      or itself or whatever it might be.  
  
       19                   MR. SKALL:  To be rigorous, in my view, the  
  
       20      best way to do that or, in fact, something is deficient,  
  
       21      you know, and passes the test, that means your tests aren't  
  
       22      comprehensive enough.  What you want to do is go back,  
                                                                      152  
  
  
  
  
        1      enhance the test to, in fact, catch that.  So you want to  
  
        2      keep the test up-to-date as much as possible.  
  
        3                   DR. LASKOWSKI:  Since you're on usability,  
  
        4      the questions that Paul had, quickly, with regard to the  
  
        5      desertification type of thing and usability, so, for  
  
        6      example, you might as a requirement test by demonstration  
  
        7      that this system does not allow over voting, but then in  
  
        8      the course of usability testing, a voter over votes.  
  
        9                   At that point you're supposed to test it,  
  
       10      even though it passed initially, at this course of use, it  
  
       11      didn't pass.  And then, Paul, you referred to accessibility  
  
       12      issues.  The conundrum is that you can design your tests  
  
       13      with certain good coverage of users for usability to show  
  
       14      that different classes of people with disabilities can use  
  
       15      this system effectively and efficiently.  And there's also  
  
       16      going to be some unique sets of individuals that cannot use  
  
       17      the system.  And then you do need some sort of reasonable  
  
       18      clause.  
  
       19                   So I think that maybe an example might be --  
  
       20      although this is a design spec, doors are designed for most  
  
       21      wheelchairs to fit through.  If you have a wheelchair  
  
       22      that's a very nonstandard or very wide design, it's not  
                                                                      153  
  
  
  
  
        1      going to fit through, yet those doors according, to the  
  
        2      accessible specifications are acceptable.  So I think there  
  
        3      is a special case with respect to accessibility that we  
  
        4      have to tread carefully on and think about.  
  
        5                   DR. RIVEST:  Ron Rivest.  Another question.  
  
        6      Could you comment on the viewpoint of testing as providing  
  
        7      information above and beyond either pass or fail with  
  
        8      respect to the standard?  You know, one could derive from  
  
        9      the testing procedure, you know, how long it takes the  
  
       10      average voter to vote, how wide the wheelchair access might  
  
       11      be, things like that that might be useful to the state when  
  
       12      making purchases.  
  
       13                   MR. SKALL:  There are all levels of testing  
  
       14      that one could do.  The absolute minimum is how much the  
  
       15      conformance tests determines what the requirements are.  
  
       16      That is all you need to do to set your conformance test.  
  
       17      Clearly, it helps with as much information as possible and  
  
       18      that's a question of resources, time and tradeoffs.  
  
       19                   So it's something that certainly should be  
  
       20      encouraged. I can't say it's always done, and I'll get into  
  
       21      this later.  Testing is incredibly expensive, and there are  
  
       22      so many spec outs there.  
                                                                      154  
  
  
  
  
        1                   I'll get into some of the tools we've  
  
        2      developed at NIST to try and help.  It's an expensive  
  
        3      process and must be a part of the decision (inaudible).  
  
        4                   MR. WILLIAMS:  Britt Williams.  This is just  
  
        5      kind of a point.  We talk about pass/fail and standards,  
  
        6      but in the ITA process right now, we do not fail a piece of  
  
        7      voting system.  If you submit a voting system for ITA  
  
        8      qualification, it comes out of there one or two ways.  It  
  
        9      either passes or the vendor withdraws.  
  
       10                   If it is found to be deficient, then you go  
  
       11      back to the vendor, find out the deficiency and given them  
  
       12      the opportunity to correct it.  But you never go to the  
  
       13      vendor and say, you're out of here, you failed.  
  
       14                   MR. SKALL:  Thank you.  
  
       15                   MR. HARDING:  J. R. Harding.  Thank you,  
  
       16      Mr. Chairman.  I just want to rap it up a little bit in the  
  
       17      sense to say thank you, because as our colleague pointed  
  
       18      out that no voting location is truly identical.  And the  
  
       19      circumstances to replicate tests will truly be diverse.  
  
       20      And that we will need to remember where the (inaudible) and  
  
       21      the states and counties embrace it.  
  
       22                   MR. BERGER:  Mark, I think it's roughly  
                                                                      155  
  
  
  
  
        1      accurate to say that you paid for every testing in 2002.  
  
        2      You paid a $30,000 plus or minus some thousands.  And  
  
        3      there's some point at which we raise that testing process,  
  
        4      and we start doing bad things to the system.  
  
        5                   And one of the things I'm quite mindful of  
  
        6      is we're dealing with a very narrow industry here, a very  
  
        7      specialize niche.  It's really an ongoing question, but I'm  
  
        8      just wondering what your thoughts might be about how we're  
  
        9      going to do what we all want to do, and yet not in the sum,  
  
       10      end up breaking something worse by just the testing costs,  
  
       11      the costs of compliance?  
  
       12                   MR. SKALL:  The only answer I can give with  
  
       13      this, and you wrestle with this, is that I keep preaching  
  
       14      that there has to be bigger, more efficient, cheaper ways  
  
       15      to do this. (Inaudible)  
  
       16                    I think we as community need to think about  
  
       17      how to develop testing in a more efficient manner, and do  
  
       18      the best we can to make testing more cost productive.  And  
  
       19      part of the decision about tradeoffs, and it's still  
  
       20      expensive, it can be perhaps be a little less expensive and  
  
       21      more comprehensive.  
  
       22                   We need to invest in research to do  
                                                                      156  
  
  
  
  
        1      (inaudible).  In the short term, I think testing is very  
  
        2      very important.  I mean, basically, we have to realize it's  
  
        3      going to cost us, and the best thing to do is we have to  
  
        4      figure out a less costly model.  
  
        5                   MS. DAVIDSON:  That was basically my  
  
        6      question.  I was afraid we might push the States so far  
  
        7      they wouldn't adopt the standards.  As I opened up with  
  
        8      saying, my goal is that every state adopted what we did,  
  
        9      and you touched right on my point.  
  
       10                   CHAIRMAN BEMENT:  Okay.  Maybe we ought to  
  
       11      move forward.  
  
       12                   MR. SKALL:  Actually, I was telling my  
  
       13      colleagues I thought this would be noncontroversial.  
  
       14                   Okay.  I'll try to rap it up.  Conformance  
  
       15      clause is a way to tell the implementers what conformance  
  
       16      should address at a minimum, and who needs to conform.  Any  
  
       17      time more than one type of entity needs to conform, what  
  
       18      must they do to confirm.  It may for conformance purposes,  
  
       19      refer to functional subsets, modules, profiles, levels or  
  
       20      other structures.  But the conformance clause must address  
  
       21      what's expected, so if there are five profiles, one  
  
       22      profile, at least one profile out of five profiles must  
                                                                      157  
  
  
  
  
        1      conform.  
  
        2                   The concept of extensions is very very  
  
        3      important.  Extensions are the inclusion of additional  
  
        4      functionality above and beyond what's required.  Extensions  
  
        5      sometimes are allowed, sometimes are prohibited.  The  
  
        6      specifications should address the concept of extensions,  
  
        7      say whether they're allowed or what constraints are to be  
  
        8      placed.  
  
        9                   A few years ago, we at NIST worked closely  
  
       10      with WC3 to develop guidelines on how specs should be  
  
       11      written, and we had a workshop at NIST.  A few years ago as  
  
       12      a result of this, the WC3 started a new activity for  
  
       13      quality assurance that they're helping to lead.  And we're  
  
       14      developing, among other things, guidelines that have  
  
       15      already specifications.  They initially could be used with  
  
       16      anybody (inaudible) directly in the community.  This is  
  
       17      perhaps one resource we can use to help the voting  
  
       18      community to draft form (inaudible) certainly we can make  
  
       19      it.  
  
       20                   MR. HARDING:  Mark, could you define for the  
  
       21      committee what WC3 is?  
  
       22                   MR. SKALL:  WC3 is worldwide web insertion.  
                                                                      158  
  
  
  
  
        1                   MR. HARDING:  They seem to lead in  
  
        2      technology for accessibility.  
  
        3                   MR. SKALL:  Just summarizing now, I just  
  
        4      wanted to mention a few efforts.  I mentioned the WC3  
  
        5      quality assurance activity.  We are involved in OASIS as  
  
        6      well.  We share the conformance testing activity.  We have  
  
        7      produced a document to help with the conformance tests.  
  
        8                   We do what I call a conformance advisory.  
  
        9      We work with not only standards communities, but other  
  
       10      communities such as the healthcare community, the  
  
       11      Department of Transportation, we work with the airline  
  
       12      industry, to help (inaudible) them how to write conformance  
  
       13      tests and things like that.  
  
       14                   I mentioned before, we've done a lot of work  
  
       15      in testing and developing testing technique.  I mentioned  
  
       16      four or five that are concentrated on different things to  
  
       17      outcome.  We have used this methodology to help generate a  
  
       18      very extensive set of XML tests that are used voluntarily  
  
       19      by just about every vendor.  
  
       20                   Lastly we have a National Software Reference  
  
       21      Library developed for the forensics community?  Doug White  
  
       22      will be speaking about that as soon as I shut up.  
                                                                      159  
  
  
  
  
        1                   The last slide is the contact information.  
  
        2      Do you have anymore questions?  
  
        3                   DR. RIVEST:  Could you comment on the  
  
        4      utility of open source software, the quality of it, and  
  
        5      evaluation of reducing the costs of evaluation?  
  
        6                   MR. SKALL:  I think open sources has some  
  
        7      merits.  Of course, it's a double-edged sword because much  
  
        8      of the open source software, you don't know how to  
  
        9      (inaudible).  I think it's a tool that can be used if you  
  
       10      have to look at again making use of all your resources.  
  
       11      Open source, by it's very nature, sometimes it is not  
  
       12      tested very well, as well, you don't know if it's a tool  
  
       13      that can be used.  
  
       14                   DR. RIVEST:  I was asking about what was  
  
       15      testing of standards of user software which was provided in  
  
       16      open source format, not using open source tools?  
  
       17                   MR. SKALL:  The same issues really apply.  
  
       18      You have to put in the input that you're testing.  The key  
  
       19      issue here is you have to be very careful in writing the  
  
       20      requirements and developing the test.  
  
       21                   DR. RIVEST:  Why does one have open source  
  
       22      -- open setback and open source software.  The argument is  
                                                                      160  
  
  
  
  
        1      often made that one can benefit from the public at large  
  
        2      looking at both the code and spec to see there is no test.  
  
        3                   MR. SKALL:  Is that scientific test  
  
        4      methodology?  
  
        5                   MR. CRAFT:  Paul Craft.  I so have some  
  
        6      comments, and I've tried to avoid just flat out saying the  
  
        7      way things are.  Open source is a real good way to get  
  
        8      comment on your code from other knowledgeable parties.  
  
        9      That's why it is such a popular approach for developing  
  
       10      things, particularly in the academic world.  
  
       11                   As far as evaluating the system, the  
  
       12      evaluators must review and analyze the source of the code,  
  
       13      and then they must witness a compilation of that source  
  
       14      code in a positive compilation environment.  If you don't  
  
       15      have total control of the compilation environment, you're  
  
       16      wasting your time.  If you don't review the source code  
  
       17      before you witness it, you're wasting your time.  And, of  
  
       18      course, if you strictly look at object code produced,  
  
       19      you're doing black box testing, which is pretty much  
  
       20      (inaudible).  
  
       21                   The real value of putting voting systems  
  
       22      source code out there for public comment by people who  
                                                                      161  
  
  
  
  
        1      really have no way of witnessing the compile or getting a  
  
        2      chain of evidence between that code and the stuff you're  
  
        3      going to be voting, it is pretty much a waste of time.  
  
        4                   CHAIRMAN BEMENT:  Thank you for that  
  
        5      feedback.  Anymore questions?  I think we can get you off  
  
        6      the stand.  Thank you.  
  
        7                   We've been at this for a while.  We need to  
  
        8      generate our circulation systems, so we're going to take a  
  
        9      15-minute break.  We will reconvene at 20 minutes of.  But  
  
       10      if you have filled out your preference forms as far as  
  
       11      subcommittee assignment, if you would pass them up, we will  
  
       12      take them now.  
  
       13                   We'll reconvene at 20 of.  
  
       14                   (RECESS 1:20, RECONVENE 1:40)  
  
       15                   CHAIRMAN BEMENT:  The Committee will come to  
  
       16      order.  I'd like to make a general disclaimer so that  
  
       17      everyone understands what NIST is and what our role is.  
  
       18                   NIST is an institution and agency of the  
  
       19      Federal government.  It is not a standards' development  
  
       20      organization.  We develop guidelines, but we don't write  
  
       21      standards. We work with the standard developing community.  
  
       22      There are about 300 standard organizations that we  
                                                                      162  
  
  
  
  
        1      routinely interact with.  We're not a regulatory agency, so  
  
        2      we don't write regulation.  
  
        3                   And so then the question is:  What do we do?  
  
        4      Well, we're the subject matter experts for those who do the  
  
        5      standards writing, and the people who own that process are  
  
        6      the standard developers.  
  
        7                   As far as this committee is concerned, the  
  
        8      products of the Committee are owned by the Committee.  NIST  
  
        9      is there as a service organization to provide useful input  
  
       10      along the way, and no more no less.  
  
       11                   So we're a resource that you can use to your  
  
       12      best advantage.  And I think you'll find that that is going  
  
       13      to be a very useful resource for you.  
  
       14                   The other two points that I wanted to cover:  
  
       15      Everything that has been handed out to you will be  
  
       16      available electronically if you wish to leave the material  
  
       17      behind.  It will be posted on the web site.  I think that  
  
       18      also applies to what Chairman Soaries handed out this  
  
       19      morning, but I would suggest you take that with you so that  
  
       20      you can begin looking at it, because it may take a while  
  
       21      before we can get that on the website, maybe a day or two.  
  
       22                   With that exception, if you wish to leave  
                                                                      163  
  
  
  
  
        1      anything behind, you may do.  
  
        2                   The other thing has to do with the  
  
        3      establishment of the subcommittees.  We would like to do  
  
        4      that today, if we could.  We will do it as soon as the  
  
        5      final clearance is in hand, and that should take another  
  
        6      week, maybe two weeks at the long end.  Those will be  
  
        7      posted on the web site, and you will be notified by  
  
        8      telephone or by mail just as soon as we can get those  
  
        9      organized.  
  
       10                   We will begin working with the Chairs to  
  
       11      begin scheduling the first interactions of the  
  
       12      subcommittees.  
  
       13                   With that, I would like to call on Doug  
  
       14      White.  Doug is project leader for NIST's National Software  
  
       15      Reference Library, and he's going to give us an overview of  
  
       16      how that library operates.  Doug?  
  
       17              MR. WHITE:  Thank you very much, and thank you  
  
       18      for the opportunity.  
  
       19              As an introduction, the National Software  
  
       20      Reference Library is really three things:  
  
       21              One, it's a secured room full of software, over  
  
       22      5000 software applications at this time;  
                                                                      164  
  
  
  
  
        1              Second, it's a database that we keep of all of  
  
        2      the file fingerprints or how you call it, hashes, and  
  
        3      information that uniquely identifies every single file in  
  
        4      every disk and every box on all the software shelves;  
  
        5              And, thirdly, it's a set of CD's that we  
  
        6      distribute to law enforcement and police centers that's  
  
        7      only a subset of the information that's in our database.  
  
        8              How the NSRL is used:  
  
        9              Most commonly it's used to eliminate known files  
  
       10      from a seized computer using automated means.  And there  
  
       11      are several other things you can do with the data.  
  
       12              You can discover an expected file with unexpected  
  
       13      contents.  
  
       14              You can identify origins of files.  You can  
  
       15      actually trace identities of files through operating  
  
       16      systems.  
  
       17              You can look for malicious files such as hacker  
  
       18      tools.  
  
       19              You can identify duplicate files.  
  
       20              And we provide rigorously verified data for  
  
       21      forensic investigations.  
  
       22              Just briefly on how the NSRL project started:  
                                                                      165  
  
  
  
  
        1              Now law enforcement needed software hashes that  
  
        2      could stand up in court, and also could be used in  
  
        3      investigations.  
  
        4              The source needed to be unbiased, and NIST, of  
  
        5      course, is a neutral organization, and we were approached  
  
        6      four years ago at the start of the project.  
  
        7                   The data produced had to be complied from  
  
        8      the highest quality, and NIST has that track record.  
  
        9                    The data needed to be traceable and  
  
       10      repeatable, so we actually can go back, pull any software  
  
       11      application off of our shelves, and we can do mathematics  
  
       12      at any time it's called into question in court, and repeat  
  
       13      the process.  
  
       14                   NIST provides an open rigorous process --  
  
       15      I'll get back to that -- but all of our code naturally is  
  
       16      available, all of our process is peer-reviewed.  
  
       17                   So specifically to dive into three sections,  
  
       18      the software collection is a balance of the most popular  
  
       19      software that law enforcement required and encounters the  
  
       20      most often on machines that they seize, and the balance of  
  
       21      the most desired software, that is the software that is  
  
       22      more frequently pirated, what do people go after to obtain.  
                                                                      166  
  
  
  
  
        1                   We collect software currently in 32  
  
        2      languages.  This software, the data set is used  
  
        3      internationally.  I know of Swiss, German, Dutch, Irish,  
  
        4      Canadian law enforcement groups that actually use our data  
  
        5      set.  
  
        6                   The software is purchased commercially, so  
  
        7      it's exactly what's available to consumers, what we'd find  
  
        8      on anybody's machine.  And it's also donated under a  
  
        9      non-use policy.  Some of the more expensive software,  
  
       10      Oracle, for example, some of the Microsoft software, we get  
  
       11      every single CD and DVD that Microsoft produces to generate  
  
       12      the fingerprints.  
  
       13                   A list of our contents is openly available  
  
       14      on our web site, so anybody can check that.  
  
       15                   As far as what's in the database, again,  
  
       16      it's the information that uniquely identifies every single  
  
       17      file, every CD or floppy in every box on our shelves.  
  
       18                   The database schema, that is the description  
  
       19      of the items we collect, is also available on our web site.  
  
       20      Anybody can look and see all the information we collect.  
  
       21                   And just as a feel or it, I mention some  
  
       22      numbers here:  4200 bytes per application, 750 bytes per  
                                                                      167  
  
  
  
  
        1      file is what we collect.  So our total database size right  
  
        2      now is just over 20 GB, and we've got almost 32 million  
  
        3      files in that database.  
  
        4                   The reference data set, the CD sets, is a  
  
        5      selection of that information from the database.  Not  
  
        6      everybody needs all of the information that we collect, so  
  
        7      we call it down and make up a lot of duplicates.  Instead  
  
        8      of the 32 million files, 20 million files are represented  
  
        9      on these, so that's the smallest set that anybody needs to  
  
       10      go through to do elimination.  This still allows positive  
  
       11      identification of a manufacturer, product, the operating  
  
       12      system, and versions of the files based on that file's  
  
       13      signature.  
  
       14                   The data format, the way that we generate  
  
       15      the files that are the CD's available to the public and  
  
       16      open for developers.  There are several tools, forensic  
  
       17      tools that are built for third-party vendors that can  
  
       18      import this data set.  
  
       19                   It is published quarterly, and it's  
  
       20      available for redistribution.  We'd like to get this out  
  
       21      into the hands of as many law enforcement agencies and  
  
       22      researchers as possible.  
                                                                      168  
  
  
  
  
        1                   Beyond that quarterly deadline, it is  
  
        2      possible to publish critical data out of our regular  
  
        3      schedule.  We were approached back in February by the FBI  
  
        4      and DOD, who asked us if we could generate a specific batch  
  
        5      set of Arabic software, and within that month's time, we  
  
        6      were able to give them a half a million files in the Arabic  
  
        7      language.  We've also done that for certain hacker group  
  
        8      sets also.  
  
        9                   The next page is a diagram of the field use  
  
       10      concept.  And if the computer there, you look at the files  
  
       11      on the disk drive, we feed those into an analysis program,  
  
       12      take our reference data set, also feed that into an  
  
       13      analysis program, what you come out with on the right-hand  
  
       14      side to the lower right are the known files, the files that  
  
       15      we have the signatures of, they're benign operating system  
  
       16      files, office application files.  You know what those are,  
  
       17      you don't need to look at those.  Anything else comes out  
  
       18      in the upper right being unknown, something to be  
  
       19      investigated.  
  
       20                   We can take a hard disk, a large hard disk  
  
       21      with hundreds of thousands of files on it, eliminate up to  
  
       22      90 to 95 percent of those, and then you can concentrate  
                                                                      169  
  
  
  
  
        1      your efforts on the 5 to 10 percent that are left on the  
  
        2      disk that are of interest in the investigation.  
  
        3                   Another particular example, say you are  
  
        4      looking for sensitive facility maps on a computer that has  
  
        5      Windows 2000 running on it.  Just when you install the  
  
        6      Windows 2000, you get almost 6000 images placed on that  
  
        7      machine.  We've got the signatures for those.  We can take  
  
        8      those out of your investigation, so right off the bat, 6000  
  
        9      less images you have to look at in the investigation.  
  
       10                   In particular -- now we're going to get into  
  
       11      a little bit of mathematics and technical things.  
  
       12                    Hashes can be thought of like someone's  
  
       13      fingerprint.  It uniquely identifies the file, based on the  
  
       14      contents of the file.  But like a fingerprint, you can't  
  
       15      reproduce a person from a fingerprint, you can't reproduce  
  
       16      the file from its hash.  
  
       17                   The primary value that we use is the Secure  
  
       18      Hash Algorithm (SHA-1).  That's specified in the Federal  
  
       19      Information Processing Standards that's openly defined.  
  
       20      It's a very rigorous cryptographic algorithm.  The phrase  
  
       21      that we use if someone knew the signature of some voting  
  
       22      software and wanted to try and recreate a hack version of  
                                                                      170  
  
  
  
  
        1      that with the same signature, it is "computationally  
  
        2      infeasible," it's impossible.  It computationally  
  
        3      infeasible to find two different files that would generate  
  
        4      that same signature.  There's not enough computing power in  
  
        5      the world, or time left in the universe for anybody to do  
  
        6      that kind of a trick.  
  
        7                   On the next page you see some hash examples,  
  
        8      and it's just on the right-hand side you can see some long  
  
        9      character strings.  That's a hash; that's a SHA-1.  The  
  
       10      first four lists there are from the files called  
  
       11      notepad.exe off of one single CD, but those four files are  
  
       12      meant to be used on different ship sets of computers. They  
  
       13      have a different number of bytes, they're different sizes,  
  
       14      and they all have different signatures.  
  
       15                   The bottom two that are listed, those came  
  
       16      off of two different CDs.  They both have .exe and have the  
  
       17      same number of bytes, the same size, they have the same  
  
       18      signature, they're exactly the same files based on that.  
  
       19                   We also have done some research with the  
  
       20      National Archives.  We've been using our hashing process  
  
       21      for their non-classified Presidential materials to evaluate  
  
       22      their records.  We have been able to identify the  
                                                                      171  
  
  
  
  
        1      application files on the administration's machines, so  
  
        2      there's a file when we eliminate those, what's left is the  
  
        3      content that was generated by the administration that the  
  
        4      National Archives can focus on.  
  
        5                   We're also able to identify duplicate files  
  
        6      so the National Archives doesn't have to keep 600 copies of  
  
        7      the same memo.  They merely can just keep that one  
  
        8      particular copy.  And back from them, we got access to  
  
        9      older installed software so this helps our data set.  
  
       10                   In particular, from what we see, the ways  
  
       11      that the NSRL can help the needs for voting systems would  
  
       12      be to determine that the software used during elections was  
  
       13      the expected software.  
  
       14                   That the tested certified version is  
  
       15      definitively identifiable.  We know that's the code that's  
  
       16      been certified.  
  
       17                   That the software stays the same during the  
  
       18      distribution, installation, and setup, or use.  
  
       19                   Then we can talk about possible chain of  
  
       20      custody.  
  
       21                   As far as transparency, the NSRL methodology  
  
       22      is in the public domain, and the algorithms that we use are  
                                                                      172  
  
  
  
  
        1      widely known and available for inspection.  And because the  
  
        2      final fingerprints don't allow anyone to know the  
  
        3      proprietary contents of the files, jurisdictions may be  
  
        4      able to share that knowledge with each other.  
  
        5                   As far as what the NSRL, specifically,  
  
        6      technical for supporting EAC.  We can verify that operating  
  
        7      system file contents have not been modified.  If you're  
  
        8      expecting a system that comes with certain things disabled  
  
        9      in the software, we can verify that that is true.  
  
       10                   We can verify that application file contents  
  
       11      have not been modified.  
  
       12                   We can verify that known static sections of  
  
       13      files have not been modified.  So there may be times like  
  
       14      an interface file were you're expecting things to stay  
  
       15      exactly the same, even though parts of the file have  
  
       16      changed.  Or if database files are sometimes modified, we  
  
       17      can check certain sections of those files.  
  
       18                   And just to give you a feel for time.  For  
  
       19      an 866MHz computer, a file with 50MB takes about 5 seconds  
  
       20      to generate a SHA-1 fingerprint, about 4 seconds to  
  
       21      generate an MD5, which is the method more popular, and the  
  
       22      one that's used in the field.  
                                                                      173  
  
  
  
  
        1                   As far as ongoing voting research issues,  
  
        2      with Commissioner Soaries' remarks, and the five points  
  
        3      that he put out there a month ago, we have started working  
  
        4      with software companies to get access to the software.  
  
        5                   There were issues as far as the distribution  
  
        6      versus the installation hashes.  What I mean by that is if  
  
        7      we're given a CD, if we're given that initial software,  
  
        8      that may not be exactly the same as what is installed as  
  
        9      far as the installation process.  There could be  
  
       10      modifications, so we need to research that.  For example if  
  
       11      there is any setup after the hashes are made, how do you  
  
       12      know what changes are valid.  
  
       13                   Is it possible or practical to have  
  
       14      on-location, time-of-certification hashing, and also  
  
       15      research into verification within given time or space or  
  
       16      security constraints?  
  
       17                   Are there any questions?  
  
       18                   MR. WILLIAMS:  I have a comment on this.  We  
  
       19      do this in Georgia.  We use this exact same hashing  
  
       20      technique to develop a software that's installed in the  
  
       21      counties.  
  
       22                   The only suggestion I have, we do not obtain  
                                                                      174  
  
  
  
  
        1      our software from the vendor.  We get it from the ITA. The  
  
        2      ITA qualifies a specific version.  And so to be sure we've  
  
        3      got that qualified version, we get our copy of the software  
  
        4      from the ITA, not from the vendor.  
  
        5                   MR. BERGER:  Thank you for the presentation.  
  
        6      To me it is just a wonderful tool to bring to this arena.  
  
        7                   Let me ask you, I know the Chairman is very  
  
        8      supportive of quick implementation.  Is there any reason  
  
        9      not to encourage that as quickly as possible, any  
  
       10      implementation issues before this could be brought into  
  
       11      this arena?  
  
       12                   MR. WHITE:  I think that would depend on the  
  
       13      requirements that would be set up.  I don't know if I can  
  
       14      make a call on that at this time.  
  
       15                   MS. PURCELL:  Thank you.  Helen Purcell.  A  
  
       16      question about the vendors that have filed with their  
  
       17      software with you.  Have there been some users --  
  
       18                   MR. WHITE:  There were two.  
  
       19                   MS. PURCELL:  Two.  
  
       20                   MR. WHITE:  One that has sent software, just  
  
       21      to explain as my understanding goes.  I haven't talked with  
  
       22      some of the vendors.  We need a non-use agreement, the  
                                                                      175  
  
  
  
  
        1      vendors to sign a nondisclosure agreement, whether it comes  
  
        2      from the ITA or them, and we do this with all software  
  
        3      companies so they know that we're not going to distribute  
  
        4      or make it available.  
  
        5                   VoteHere, Incorporated, which has publically  
  
        6      available software has sent the software in.  The other  
  
        7      companies, one has agreed at this point, and we have worked  
  
        8      out a nondisclosure agreement with them; however, we have  
  
        9      not received the signed nondisclosure agreement back from  
  
       10      the company.  
  
       11                   And while the other vendors have expressed  
  
       12      some interest to the Chairman of the EAC, we have no  
  
       13      agreement from the other major vendors at this point.  So  
  
       14      it's more than just going to the ITA's and getting the  
  
       15      equipment.  We have to look at the software, and we can't  
  
       16      do that until we have an agreement.  I saw this as a good  
  
       17      practice.  
  
       18                   MR. CRAFT:  We currently have -- I'm not  
  
       19      sure about Sequoia -- but with the other two vendors -- and  
  
       20      I can't remember the one you called -- we have verbal  
  
       21      agreement with all the three major vendors to take the  
  
       22      copies of the software that we have a good chain of custody  
                                                                      176  
  
  
  
  
        1      with, haven't breached the certification, and give that to  
  
        2      NIST to work with them on defining both the installed  
  
        3      pictures and the release disk.  
  
        4                   The legal issues basically are holding us  
  
        5      all up right now, and we're going to have to have probably  
  
        6      some sort of a three-way agreement in order to hand that  
  
        7      over.  But also I want to be sure that everyone here  
  
        8      actually understands the importance of this.  This is  
  
        9      probably the most important contribution NIST is going to  
  
       10      make to implementing HAVA nationwide.  
  
       11                   The work that Britt has done in Georgia, and  
  
       12      that we have done in Florida allows our two states to walk  
  
       13      into any county, and in a few minutes prove absolute  
  
       14      certainty that they either have the certified copy of the  
  
       15      software that we certify and test it, or that they don't.  
  
       16                   That's a very expensive thing to do.  I'm  
  
       17      running a bureau of 16 people covering 67 counties.  We're  
  
       18      working 50 and 60 hours a week, everybody in the bureau.  
  
       19                   There has to be an infrastructure that is  
  
       20      going to allow every county in the country to do that  
  
       21      themselves, and do it economically, so it becomes very easy  
  
       22      to do.  And that's what the National Software Reference  
                                                                      177  
  
  
  
  
        1      Library offers through all of this.  
  
        2                   This is a very important part.  Everything  
  
        3      else that we've talked about doing here, if we don't get  
  
        4      this done, will be for nothing.  
  
        5                   CHAIRMAN BEMENT:  Is there a recommendation  
  
        6      to the Election Assistance Commission on this matter?  
  
        7                   MR. CRAFT:  I would like to recommend that  
  
        8      we very aggressively move toward these legal issues and get  
  
        9      a prototype up as soon as possible.  
  
       10                   CHAIRMAN BEMENT:  Can you put that in the  
  
       11      form of motion?  
  
       12                   MR. CRAFT:  I move that we recommend that we  
  
       13      expedite the legal problem solutions, and get this into a  
  
       14      prototype as soon as possible.  
  
       15                   MR. BERGER:  I second it.  
  
       16                   CHAIRMAN BEMENT:  Further discussion?  
  
       17                   MR. BERGER:  I just make a point to my eyes,  
  
       18      this is something quite substantive that can have an impact  
  
       19      on improving this November's election, probably one of the  
  
       20      few things we can do.  I fully support the motion.  
  
       21                   CHAIRMAN BEMENT:  Craig, would you repeat  
  
       22      the motion.  
                                                                      178  
  
  
  
  
        1                   MR. BURKHARDT:  Actually, since this is a  
  
        2      motion from the floor, if the gentleman could renew the  
  
        3      motion and once again state it for the court reporter.  
  
        4                   MR. CRAFT:  I'm Paul Craft.  I'm from the  
  
        5      State of Florida.  The motion is we recommend that the  
  
        6      legal issues be resolved as quickly as possible.  
  
        7                   CHAIRMAN BEMENT:  Make it a full motion so  
  
        8      we know what you mean by legal issues.  
  
        9                   MR. CRAFT:  I move that we recommend that we  
  
       10      expedite solving the legal issues around Florida's deposit  
  
       11      of currently certified software into the National Software  
  
       12      Reference Library as soon as possible so that we can bring  
  
       13      out the prototype of a software licensed process as quickly  
  
       14      as possible.  
  
       15                   CHAIRMAN BEMENT:  Was that satisfactory to  
  
       16      the secondary?  
  
       17                   MR. BERGER:  It's satisfactory, but I think  
  
       18      I'd like to do the following motion.  It would have more of  
  
       19      a national approach.  We would recommend that --  
  
       20                   CHAIRMAN BEMENT:  Would you like to put in a  
  
       21      substitute motion?  
  
       22                   MR. CRAFT:  Well, ours basically has been  
                                                                      179  
  
  
  
  
        1      sets to file through the chain of custody, and I don't know  
  
        2      that Georgia has that from their system.  I don't know what  
  
        3      other sources of files have a good chain of custody, that's  
  
        4      why I mentioned Florida.  
  
        5                   MR. BERGER:  Paul, if I might throw out a  
  
        6      suggestion.  Perhaps if we start the motion with a  
  
        7      recommendation that the EAC as quickly as possibly  
  
        8      implement a software library, and then follow with your  
  
        9      motion since as you've pointed out --  
  
       10                  MS. QUESENBERY:  I think we're just ready to  
  
       11      make a general motion.  
  
       12                   MR. BERGER:  Okay.  
  
       13                   MR. CRAFT:  I'll second yours.  
  
       14                   MR. BURKHARDT:  Actually, if the gentleman  
  
       15      could withdraw his earlier motion.  
  
       16                   MR. CRAFT:  I'll withdraw the earlier  
  
       17      motion.  
  
       18                   MR. BURKHARDT:  And the person who was  
  
       19      second to that withdraw the second.  
  
       20                   MR. BERGER:  I will withdraw the second.  
  
       21                   MR. BURKHARDT:  Okay.  So the earlier motion  
  
       22      made is no longer on the table.  Thank you.  
                                                                      180  
  
  
  
  
        1                   CHAIRMAN BEMENT:  Go ahead with your  
  
        2      motion.  
  
        3                   MR. CRAFT:  Okay.  I move that we recommend  
  
        4      to the EAC that they expedite making the currently  
  
        5      certified software available through the National Software  
  
        6      Reference Library as soon as possible.  
  
        7                   MR. BERGER:  I second it.  
  
        8                   CHAIRMAN BEMENT:  Okay.  You second it.  
  
        9                   Is there any further discussion?  
  
       10                   With no further discussion, please call the  
  
       11      roll.  
  
       12                   MR. BURKHARDT:  Might I ask the court  
  
       13      reporter, did you get the final version properly?  
  
       14                   REPORTER:  (Nodding in the affirmative.)  
  
       15                   MR. BURKHARDT:  Then this will be designated  
  
       16      as Committee Resolution Number 5.  
  
       17                   If you're in favor say, "aye," if you're  
  
       18      opposed say, "nay," if you believe you have a conflict of  
  
       19      interest, please abstain.  
  
       20                   Dr. Bement?  
  
       21                   CHAIRMAN BEMENT:  Aye.  
  
       22                   MR. BURKHARDT:  Davidson?  
                                                                      181  
  
  
  
  
        1                   MS. DAVIDSON:  Aye.  
  
        2                   MR. BURKHARDT:  Miller?  
  
        3                   MS. MILLER:  Aye.  
  
        4                   MR. BURKHARDT:  Purcell?  
  
        5                   MS. PURCELL:  Aye.  
  
        6                   MR. BURKHARDT:  Harding?  He's out of the  
  
        7      room.  
  
        8                   MR. BURKHARDT:  Elekes?  
  
        9                   MR. ELEKES:  Aye.  
  
       10                   MR. BURKHARDT:  Caldas?  
  
       11                   MS. CALDAS:  Aye.  
  
       12                   MR. BURKHARDT:  Berger?  
  
       13                   MR. BERGER:  Aye.  
  
       14                   MR. BURKHARDT:  Williams?  
  
       15                   MR. WILLIAMS:  Aye.  
  
       16                   MR. BURKHARDT:  Craft?  
  
       17                   MR. CRAFT:  Aye.  
  
       18                   MR. BURKHARDT:  Quesenbery?  
  
       19                   MS. QUESENBERY:  Aye.  
  
       20                   MR. BURKHARDT:  Mr. Chairman, the votes are  
  
       21      10 in favor, one absent.  I advise you to declare the  
  
       22      resolution adopted.  
                                                                      182  
  
  
  
  
        1                   CHAIRMAN BEMENT:  I so do declare.  Thank  
  
        2      you very much.  
  
        3                   We'll now go to the last presentation by  
  
        4      David Alderman having to do with NIST Technology Service  
  
        5      Division.  
  
        6                   MR. ALDERMAN:  We've heard a lot about the  
  
        7      technical issues surrounding the voting systems from  
  
        8      security to usability.  Somewhere down that line when all  
  
        9      of those are resolved, we want to have competent testing of  
  
       10      it so we can verify it with governmental standard.  So  
  
       11      that's what our job is.  
  
       12                   I have a brief statement here, and if there  
  
       13      are any questions after that, I'll address those then.  
  
       14                   Under HAVA, NIST is directed to offer  
  
       15      accreditation to laboratories that test voting systems,  
  
       16      hardware and software for conformance to the current Voting  
  
       17      System Standards.  
  
       18                   NIST will carry out the accreditation of  
  
       19      these laboratories through the National Voluntary  
  
       20      Laboratory Accreditation Program, more commonly known as  
  
       21      NVLAP, which is administered by NIST.  
  
       22                   NVLAP was established in 1976, and is  
                                                                      183  
  
  
  
  
        1      recognize both nationally and internationally.  
  
        2                   NVLAP accreditation criteria are codified in  
  
        3      the Code of Federal Regulations.  
  
        4                   Simply stated, laboratory accreditation is  
  
        5      formal recognition that a laboratory is competent to carry  
  
        6      out specific tests.  I think we've talked about testability  
  
        7      -- standards and testability.  
  
        8                   A team of expert technical assessors  
  
        9      conducts a thorough evaluation, using recognized criteria  
  
       10      and procedures of all aspects of laboratory operations that  
  
       11      affect test results.  
  
       12                   General criteria are based on the  
  
       13      international standard ISO/IEC 17025, and those  
  
       14      requirements can be found in the NIST Handbook 150, which  
  
       15      is also on the NVLAP web site.  
  
       16                   Laboratory accreditation bodies use this  
  
       17      standard specifically to assess factors relevant to a  
  
       18      laboratory's ability to produce accurate and consistent  
  
       19      test data, including the technical competency of the staff,  
  
       20      validity and appropriateness of test methods, testing and  
  
       21      quality assurance of tests and calibration data.  
  
       22                   This Laboratory Accreditation Program,  
                                                                      184  
  
  
  
  
        1      (LAP), will specify specific technical criteria that voting  
  
        2      system laboratories must meet, in addition to demonstrating  
  
        3      general technical competence.  
  
        4                   Laboratory accreditation thus provides a  
  
        5      means of evaluating the competence of those testing  
  
        6      laboratories to perform specific types of testing,  
  
        7      measurement and validation.  
  
        8                   The quality system required by ISO/IEC  
  
        9      17025, allows a laboratory to determine whether it is  
  
       10      performing its work correctly, and to the appropriate  
  
       11      standards.  
  
       12                   Laboratories seeking accreditation to test  
  
       13      voting system hardware and software will be required to  
  
       14      meet the NVLAP criteria for accreditation which again  
  
       15      includes:  ISO/IEC 17025, the 2002 Voting System Standards,  
  
       16      and the criteria deemed necessary by the Election  
  
       17      Assistance Commission.  
  
       18                   To ensure continued compliance, all  
  
       19      NVLAP-accredited laboratories undergo an onsite assessment  
  
       20      before initial accreditation, during the first renewal  
  
       21      year, and every two years thereafter.  And you can do  
  
       22      additional onsite assessments for cause, if need be.  
                                                                      185  
  
  
  
  
        1                   Only after the laboratory has met all NVLAP  
  
        2      criteria, will it be accredited and presented to the EAC  
  
        3      for its approval.  The EAC may impose requirements on  
  
        4      laboratories in addition to NVLAP accreditation.  I gave an  
  
        5      example there as a first-party laboratories.  NVLAP can  
  
        6      accredit first-party laboratories, that is a vendor's  
  
        7      laboratories.  You may not want to do that in this case.  
  
        8      It's just an example.  
  
        9                   On June 23rd, NIST published a Federal  
  
       10      Register Notice announcing that any laboratory wishing to  
  
       11      conduct testing under HAVA should contact NVLAP for further  
  
       12      information.  That Federal Register Notice is in your  
  
       13      notebooks and the handouts.  
  
       14                   NVLAP will conduct a public workshop with  
  
       15      interested laboratories on August 17th to review its  
  
       16      accreditation, as well as receive comments and feedback  
  
       17      from the participating laboratories and other interested  
  
       18      parties.  
  
       19                   Of course, all of you are more than welcomed  
  
       20      to attend and participate.  
  
       21                   As part of the development process, the LAP  
  
       22      development process, NVLAP will have to finalize specific  
                                                                      186  
  
  
  
  
        1      technical criteria for testing laboratories and make the  
  
        2      necessary logistical arrangements to begin the actual  
  
        3      assessment of the laboratories.  
  
        4                   NVLAP must identify, contract, and train  
  
        5      technical expert assessors to ISO 17025.  
  
        6                   Laboratories must complete NVLAP  
  
        7      applications and pay applicable fees; rigorous onsite  
  
        8      assessments must be conducted; and laboratories undergoing  
  
        9      assessment must resolve any identified nonconformities  
  
       10      before accreditation can be granted.  
  
       11                   More detailed information regarding the  
  
       12      accreditation process can found on the NVLAP website at  
  
       13      www.nist.gov/nvlap.  
  
       14                   While NVLAP has over 25 years experience  
  
       15      accrediting laboratories, each LAP, Laboratory  
  
       16      Accreditation Program, is unique, and NIST will be seeking  
  
       17      advice and guidance from the EAC, the TGDC, the  
  
       18      laboratories and the community.  
  
       19                   It is anticipated that the laboratories will  
  
       20      be able to formally apply to NVLAP and initiate the  
  
       21      assessment process in early 2005.  
  
       22                   Both NIST and NVLAP look forward to working  
                                                                      187  
  
  
  
  
        1      with all of you on the development of this laboratory  
  
        2      accreditation program.  
  
        3                    Questions?  
  
        4                    MS. PURCELL:  Are there any laboratories  
  
        5      now certified?  
  
        6                   MR. ALDERMAN:  No, no, we're just developing  
  
        7      the program at this time.  We have received interest from  
  
        8      the Federal Register Notice.  We have received -- from the  
  
        9      original ITA's, we received other interest from other  
  
       10      laboratories, too.  
  
       11                   CHAIRMAN BEMENT:  Is there another workshop  
  
       12      that's planned on this, or did you already cover that?  
  
       13                   MR. ALDERMAN:  Yes, August 17th is the  
  
       14      workshop.  
  
       15                   MS. QUESENBERY:  Just a quick question.  
  
       16      Would a laboratory that's certified to test voting systems  
  
       17      have to be able to perform all the testing standards?  
  
       18                   MR. ALDERMAN:  That's a good question.  As  
  
       19      Ed mentioned earlier, there are NVLAP accredited  
  
       20      laboratories that do cryptographic module testing, and  
  
       21      those labs are capable of doing that testing now.  
  
       22                   In the current VSS or voluntary Voting  
                                                                      188  
  
  
  
  
        1      Systems Standards, ITA is responsible for an enormous  
  
        2      amount, an enormous amount of responsibilities from testing  
  
        3      mechanical to software, firmware, and assembly testing  
  
        4      also.  So it still needs to be determine -- one of things  
  
        5      at the workshop is we have to determine the scope of what a  
  
        6      laboratory has to meet.  And then the ITA is really going  
  
        7      to lose as it stands right now anyway, is kind of  
  
        8      coordinates bodies.  There are certainly people on this  
  
        9      committee that are very familiar with the ITA process.  
  
       10                   MR. CRAFT:  We're going to keep going back  
  
       11      in circles.  That's the same thing as we said earlier.  
  
       12                   As we said earlier, these standards are  
  
       13      voluntary.  The prior and current standards have been  
  
       14      voluntary.  What has been absent and will probably remain  
  
       15      absent in the foreseeable future is a central authority to  
  
       16      pull in the work product of various laboratories and  
  
       17      determine that a system end-to-end can be certified.  
  
       18                   In Florida we do that, and that's one reason  
  
       19      that we do our own program.  NASED has frankly tried to do  
  
       20      that on a voluntary basis.  It's on a number of heads, a  
  
       21      number of problems.  As you say, the biggest problems is  
  
       22      that these labs are taking on a tremendous responsibility,  
                                                                      189  
  
  
  
  
        1      all of them.  
  
        2                   I think that something I would like to see  
  
        3      possibly come out of the EAC is a recommendation of  
  
        4      Congress is perhaps setting up a federal government office  
  
        5      that would be responsible for pulling the products  
  
        6      together, then you would get the labs out of the role of  
  
        7      having this tremendous responsibility for overall project.  
  
        8                   MR. ALDERMAN:  There's really two steps that  
  
        9      the ITA are doing.  One is the testing, which accredited  
  
       10      for them.  And then there's another step which goes beyond  
  
       11      that which is certification, which is the review and  
  
       12      someone's stamp of approval on that.  
  
       13                   The accreditation will be done by NVLAP.  
  
       14      They will accredit the lab and make sure they're competent  
  
       15      to perform those tests.  
  
       16                   Now, there's a step after that.  They've  
  
       17      said they've met the requirements for the voting system  
  
       18      standards.  This product needs to go -- the law calls for  
  
       19      EAC approval.  
  
       20                   I think what will need to be worked out  
  
       21      through some people in this room and others; will be what  
  
       22      form; what procedure is this certification process; how  
                                                                      190  
  
  
  
  
        1      rigorous; what do we want out of it?  
  
        2                   So I think it's kind of a two-step process  
  
        3                   DR. WILLIAMS:  I can tell you how it works  
  
        4      now.  
  
        5                   MR. ALDERMAN:  Excuse me.  
  
        6                   DR. WILLIAMS:  I can tell you how it works  
  
        7      now.  
  
        8                   MR. ALDERMAN:  Okay.  
  
        9                   MR. WILLIAMS:  This has been under the NASED  
  
       10      voting system.  And in the NASED voting system board,  
  
       11      there's a subcommittee called the technical subcommittee.  
  
       12      When the ITA finishes its evaluation of a voting system,  
  
       13      they send their final reports to this technical  
  
       14      subcommittee, and the technical subcommittee approves or  
  
       15      disapproves the reports.  
  
       16                   MR. ALDERMAN:  Okay.  I think what needs to  
  
       17      be looked at is whether it's at the subcommittee or  
  
       18      whatever entity looks at that, what needs to be looked at,  
  
       19      how in-depth are --  
  
       20                   MR. WILLIAMS:  This is a question that needs  
  
       21      to be worked out as we develop the standards, because this  
  
       22      is dependant on the standards that you develop, obviously.  
                                                                      191  
  
  
  
  
        1      To say, number 1, how are you going to test for those  
  
        2      standards; and number 2, who's going to evaluate the  
  
        3      results of those tests and say, yep, it's official.  
  
        4                   This is a realtime function.  This is not  
  
        5      something that you can convene a committee and discuss, or  
  
        6      hold public hearings on.  This is something that generally  
  
        7      happens within a week.  
  
        8                   MR. ALDERMAN:  And do you just look at the  
  
        9      test results or do you look at all the data?  
  
       10                   MR. WILLIAM:  Yeah.  To what extent are you  
  
       11      --  
  
       12                   MR. ALDERMAN:  I think that process really  
  
       13      has to be hammered out.  
  
       14                   CHAIRMAN BEMENT:  Thank you, both of you.  
  
       15      Anything else?  Thank you very much.  
  
       16                   Well, this has been a very productive  
  
       17      meeting.  We have a lot to be satisfied about.  
  
       18                   I have appreciated working with you today,  
  
       19      and I look forward to our work ahead.  
  
       20                   You will find my contact information, as you  
  
       21      know, in your notebooks.  It's also on the website.  And  
  
       22      you also note that our next session will be sometime  
                                                                      192  
  
  
  
  
        1      shortly after the first of the year, and that will be  
  
        2      published well in advance.  
  
        3                   We'll include all information including  
  
        4      public testimony hearings and so forth as they occur, on  
  
        5      the web site.  So there will be an ongoing archive of  
  
        6      information that will be available to everyone on the  
  
        7      committee.  
  
        8                   And again to repeat our website address,  
  
        9      it's http://vote.nist.gov.  What could be more simple than  
  
       10      that?  
  
       11                   DR. WILLIAMS:  Could I ask a question?  
  
       12                   CHAIRMAN BEMENT:  Yes.  
  
       13                   DR. WILLIAMS:  Sometimes you say "vote," and  
  
       14      sometimes you say "voting".  
  
       15                   MR. EUSTIS:  "Voting" is the e-mail address;  
  
       16      "vote" --  
  
       17                   MR. WILLIAM:  Is the web site?  
  
       18                   MR. EUSTIS:  An URL for the website.  
  
       19                   CHAIRMAN BEMENT:  Thank you for pointing  
  
       20      that out.  
  
       21                   One person has been with us this afternoon,  
  
       22      and I'm sure he would have been with us this morning if he  
                                                                      193  
  
  
  
  
        1      could be.  I'd like to acknowledge Tom Wolte.  And, Tom,  
  
        2      could you introduce yourself to the group.  
  
        3                   MR. WOLTE:  I'm Tom Wolte.  I am the former  
  
        4      executive director of the New York State Board of  
  
        5      Elections.  I've had the privilege of being Chair of the  
  
        6      NASED ITA Board since 1996.  I have two of my best  
  
        7      colleagues here, who have served us, Britt as Chairman of  
  
        8      the Technical Subcommittee, as he just mentioned.  
  
        9                   I want to just quickly say that while we're  
  
       10      giving up this program in the near future, we are turning  
  
       11      it over to very very capable hands that we've been working  
  
       12      with, Dr. Bement, the folks at NIST, for over a year and a  
  
       13      half on this transition, and we'll continue to do that over  
  
       14      the next several months.  
  
       15                   I pledge on behalf of the Committee as well  
  
       16      as NASED, our full cooperation in making this go smoothly  
  
       17      and congratulate you on your first meeting today.  
  
       18                   CHAIRMAN BEMENT:  Thank you very much, Tom.  
  
       19      Tom has been a real resource to us, as well as the whole  
  
       20      community.  
  
       21                   Any final comments?  
  
       22                   MR. BURKHARDT:  Yes, this is Phil Green, and  
                                                                      194  
  
  
  
  
        1      he is one of the senior lawyers at the Technology  
  
        2      Administration at the Department of Commerce, who happens  
  
        3      to be a specialist in Internet law, among other things.  I  
  
        4      just wanted to introduce him to you because he will be one  
  
        5      of the people that will be assisting me in staffing out the  
  
        6      subcommittee meetings and activities along with  
  
        7      Mr. Eustis.  He wasn't here this morning only for the  
  
        8      reason he was still on vacation.  He decided to come here  
  
        9      early just to make sure he was able to make it.  
  
       10                   CHAIRMAN BEMENT:  Thank you.  Helen?  
  
       11                   MS. PURCELL:  Mr. Chairman, if I could,  
  
       12      please.  We are having a primary election in Arizona the  
  
       13      7th of September, and I would certainly invite any members  
  
       14      of the Committee and on the Commission, if you'd like to  
  
       15      come and view our election, you can do that.  As I said, we  
  
       16      have a 1.3 million registered voters in Maricopa County.  
  
       17      You are invited to observe any portion of that, it could be  
  
       18      the week before the election, or any of the activities on  
  
       19      election day or on the 7 or 8 afterwards when we tabulate  
  
       20      the ballots.  
  
       21                   CHAIRMAN BEMENT:  It occurs to me that we  
  
       22      ought to routinely put on our website all the pending  
                                                                      195  
  
  
  
  
        1      elections, at least the national elections.  Yours is a  
  
        2      national election?  
  
        3                   MS. PURCELL:  It's the primary.  
  
        4                   CHAIRMAN BEMENT:  Primary election.  We'll  
  
        5      try to keep a running list of all upcoming elections for  
  
        6      general reference, again, so people can attend.  
  
        7                   MS. QUESENBERY:  Was that September 7th?  
  
        8                   MS. PURCELL:  Yes.  
  
        9                   DR. WILLIAMS:  The Georgia primary is July  
  
       10      20th.  
  
       11                   MR. CRAFT:  Florida is August the 30th, I  
  
       12      believe.  The last Tuesday in August.  
  
       13                   CHAIRMAN BEMENT:  Are you getting all of  
  
       14      that, Allan?  
  
       15                   MR. EUSTIS:  As much as I can.  
  
       16                   CHAIRMAN BEMENT:  Anything else?  Well, I  
  
       17      would like to now turn the meeting over to our Federal  
  
       18      officer to say anything he wants to say about this meeting.  
  
       19                   MR. DEGREGORIO:  It's been a pleasure for me  
  
       20      to sit through your proceedings today.  And I think you're  
  
       21      off to a great start.  I know that Dr. Bement and his able  
  
       22      staff have put a lot of work into this.  You certainly saw  
                                                                      196  
  
  
  
  
        1      the presentation this afternoon.  He has some experts at  
  
        2      NIST, and we have enjoyed working with them since the EAC  
  
        3      came into existence in January.  
  
        4                   I know that this Committee will rely on  
  
        5      their expertise over the next nine months to come up with  
  
        6      the Voluntary Voting System guidelines.  
  
        7                   I am the designated Federal officer, and I'm  
  
        8      the keeper of the books along with relying on your staff,  
  
        9      particularly Allan, to try to keep me legal here.  
  
       10                   I want to introduce to you my special  
  
       11      assistant, Dan Murphy, who is here.  And Dan is going to be  
  
       12      keeping the books for me here on this Committee.  Dan just  
  
       13      started with me yesterday, in fact, so this is really his  
  
       14      first full day here.  
  
       15                   Dan has great experience.  He was my  
  
       16      executive assistant when I was the executive vice-president  
  
       17      over at IFES, and he was the executive assistant to the  
  
       18      president there, and just came back to Washington after two  
  
       19      years in L.A. as an assistant to the county clerk there.  
  
       20      So I stole him from the county, and he's now here at the  
  
       21      EAC.  
  
       22                   If you need to reach me for any reason or  
                                                                      197  
  
  
  
  
        1      have information given to me, please contact me and Dan  
  
        2      will be able to facilitate that for me, if I'm not  
  
        3      available.  
  
        4                   It's been a pleasure, and Dr. Bement, I  
  
        5      commend you for your agenda today, and your hard work in  
  
        6      pulling this all together.  
  
        7                   CHAIRMAN BEMENT:  Thank you.  
  
        8                   Incidentally, I also want to acknowledge the  
  
        9      NIST staff and the presenters that were here this  
  
       10      afternoon, thank them for their presentation.  
  
       11                   My experience over the years is that the  
  
       12      Chairman gets high grades if he starts the meeting on time  
  
       13      and ends it on time.  If he ends it before time, what does  
  
       14      he get.  
  
       15                   So with that, I declare the first meeting of  
  
       16      the TGDC adjourned.  
  
       17                   (The above meeting adjourned at approximately 

       18                   2:45 o'clock, p.m.)  
  
       19                *         *         *         *         *
  
       20  
  
       21  
  
       22  
  
  
                                                                      198  
  
  
  
  
        1

        2                   CERTIFICATE OF COURT REPORTER

        3
  
        4              I, LaDonna M. Woods, and a court reporter in and  
  
        5      for the Commonwealth of Virginia, before whom the  
  
        6      foregoing meeting was taken, do hereby certify that the  

        7      meeting was taken by me in Shorthand at the time and  
  
        8      place mentioned in the caption hereof and thereafter  
  
        9      transcribed by me; that said meeting is a true record;
  
       10

       11
  
       12  
  
       13                                   _________________________  
                                               LaDonna M. Woods  
       14                                   Court Reporter in and for  
                                            COMMONWEALTH OF VIRGINIA  
       15

       16

       17

       18

       19

       20
         
       21
  
       22  
