MR. CRAFT: What are the ground rules? Am I allowed to ask myself
questions?

MR. SCHUTZER: Only if you can answer them.

MR. CRAFT: Okay. I\'m Paul Craft, the Chief, Bureau of Voting Systems
Certification for the State of Florida. My background is in tax
auditing, financial auditing, performance auditing. I\'m both a CPA and
a certified information systems auditor. I also did some forensic
auditing back before that became popular when my employers were saying,
\"That\'s not real auditing. You should be ashamed of that.\"

In 1991, I was in the Inspector General\'s office of the Florida
Department of Revenue doing internal audits and forensic audits,
tracking embezzlement through computer systems, decided that life was
getting to stressful and I saw this interesting looking job at the
Department of State for an EDP auditor auditing voting systems. I
thought, \"How stressful could that be, one plus one?\"

Actually, until the year 2001, it was not terribly stressful. To date, I
think we have examined in the mid 200s of different voting systems. We,
you know, are, I guess, one of the groups that\'s done a lot of work in
that nationally.

I have got to say I thank you all for coming. Britt Williams, my
predecessor, Judith Light, Steve Freedom, Doug Jones, Roy Solomon and me
have been preaching this sermon for a lot of years.

Up until 2000, we couldn\'t get anyone to come down the aisle. In 2000,
the first people up the aisle wanted a new preacher. So, it\'s kind of
rewarding that the expertise that all of us has now has been brought to
the table and is growing in this process.

I\'m not going to say a lot of what I was going to say because the three
panelists that have preceded me have handled that fairly nicely.

But for the first two questions, describing what you feel the test
should consist of for accepting a voting machine and describing what you
feel the test should consist of for certifying a system, it\'s not the
test and it\'s not the machine. It is a series of tests and a batch of
work product that, as you go through time, relies on each other.

To put that in perspective first, you have to get the idea of what a
voting system is, which I touched on earlier today. It is everything
within a boundary that goes into the system, the application code, the
hardware, the environment that is specified for the system, the
supplies, the training.

You have to certify a voting system as a unit operating system that
requires someone that is competent in administering a unit system. You
are not running a certified system unless you have that person. If the
ballot stock is specified at 110 pounds and you are running 90 pounds,
you are not running a certified system. So, they have to be defined and
they have to be tested, and they have to be used within that
configuration.

That said, all of the testing, first off, needs to be public. It needs
to objective, well documented, repeatable, reviewable, and it has to
yield results supporting a conclusion. That is true for all of the
different layers of testing.

Use of the word certification as to testing is something that we try to
avoid. The FCC standards, I think it still needs to be avoided.
Certification, certification is the decision and the expression of that
decision by some authority, usually a government entity, that a system
does meet the applicable standards and can be used. Certification
generally will take into effect not only the federal standards, but also
the applicable state election codes for the state in which it is going
to be used.

Qualification testing, which is generally referred to as certification
testing, should be a series of tests that are designed to make sure that
the system definition actually meets the standard. It has to be
carefully planned, it has to, as all the testing does, have a link from
the objectives and standards, to a methodology to be used and results.
Best way to do that, I think, is a structured test matrix, which I think
everybody in the business is familiar with.

Qualification testing is the first piece. The objective of qualification
testing overall is to make sure that the system is accurate, reliable,
dependable, and that it meets the global standards.

Beyond certification testing, you have acceptance testing. The
acceptance testing is when the buyer of a system makes sure that in fact
what they bought is what they intended to buy, and also, the
configuration of what they are buying is going to meet their needs.

If you\'re in Miami/Dade, Florida, and you want your election results in
by 10 p.m. on an election with a 70 percent turnout, your acceptance
testing should verify that all your devices meet that requirement and
you use that in both the evaluation of the RFP responses and also in
verifying those before you take delivery of the system.

Building on the work product of both the qualification testing and the
acceptance testing is system validation. System validation is that
process of making sure periodically that you maintain your chain of
custody for the thing that you originally accepted, and make sure that
all of the characteristics of that other system are still there based
upon the fact that system hasn\'t changed.

That has to be done in some mix with your chain of custody. I think it
could ideally be done before each election cycle. I think that may be
something that, as we come up with easier methodologies for validation
testing, could become more frequent than it is.

After that, there is pre-election testing. Pre-election testing is that
process where you verify that all your equipment is in good shape, it\'s
operating as it should be. It\'s relying on the validation testing as
far as system integrity.

You are looking at the state of the equipment, the charge of batteries,
the extent to which the definition of the election be defined on the
system and has been properly programmed. It\'s a combination of
measuring the state of the system and also measuring the quality of the
job when preparing for the election.

Finally, there is the often-bantered-about L&A testing. It disturbed me,
in the debates over the years, we hear so many election administrators
say, \"Oh, our system is fine because we do an L&A test.\" An L&A test
is not really a test. A public L&A test is an exhibition for the public
proving that in fact that, within reasonable doubt, all the work and all
the testing that\'s gone on previously has been properly done.

In Florida code, the L&A test, the public L&A test, includes a
presentation of all the test records for all equipment. Basically it\'s
designed to provide that assurance to the public that the job has been
properly done.

If you omit any of those steps, if you neglect any of those, then
you\'re basically just betting that you have some system integrity but
you can\'t actually be sure of it. I think that\'s the standard that we
need get to.

As to using risk assessment, risk assessment has to be the overall
process wrapping around all of that testing. I think within our
standards, we need go to more of a combination of a risk assessment
model and a system throughout life cycle model where as issues come up,
we evaluate the risk. We bring that risk back in. We look for mitigation
to it, and we evaluate the mitigation and we see our systems evolved
along with those standards.

MR. HARDING: Question, Mr. Chairman?

MR. SCHUTZER: Sure.

MR. HARDING: J. R. Harding. Thank you, Paul. As everyone knows, our
state has some issues, but so do other states. Since then, it\'s my
understanding our state has written a pretty good voting law; is that
correct?

MR. CRAFT: I think so.

MR. HARDING: Within that law, is testing specifically described?

MR. CRAFT: No. Within that law and within the Florida election codes
since 1989, there was created the bureau that I run and a charge from
the legislature that we will develop, maintain and enforce voting system
standards. Basically the legislature probably wisely decided that trying
to deal with these kind of technical issues through the legislative
process would be too cumbersome. We received incredibly intense
scrutiny.

MR. HARDING: Okay. I guess you being the greater expert than I on this,
what within the Florida law might be beneficial now to the nation?
Because as I understand it, Florida and Texas, at least from the
accessible perspective, is really a leader in the field now.

MR. CRAFT: I think, once again, our approach to kind of the wholistic
definition of the system, our approach to functional standards, and I
don\'t know if you reviewed the audio ballot statements, but they are
very, very specific as to function, wide open as to methodology for
achieving that function, which was intended to basically allow the law
to evolve.

Additionally, I think the biggest thing that we and the State of Georgia
\-- and I\'m not sure what Texas is doing with it \-- is system
validation. I have currently four examiners in my office who can be
deployed to a county tomorrow morning, and within a couple hours, can go
through and validate the entire system and prove that it is or is not
one of our certified systems.

There are a lot of states that can\'t do that. California had some
problem with that and got some newspaper flak fairly recently. We have
been able to do that for some ten years. I think that\'s probably one of
the most important things that we as a committee need to see get
nationally.

MR. HARDING: Right. I guess that\'s where I was leading. Okay. So, how
do we, as a committee or as public, you know, pursue that objectively at
a national level?

MR. CRAFT: I think through the standards that are in process.

MR. HARDING: Okay. And setting it up as a system as a whole and one of
the components within it?

MR. CRAFT: Yes, and then accessibility just becomes one of a wide range
of characteristics.

MR. HARDING: But clearly defined as one of those things that you must
check and must be able to verify?

MR. CRAFT: Right.

MR. HARDING: Okay.

MR. SCHUTZER: Any other questions? Thank you, are we breaking.