VVSG Tutorial Narration[^1]

Core Requirements Part 5\
*\[Slide 1\]*

\[NARRATOR:\] This is the Training Module for Part 1, Chapter 7 of the
Next Voluntary Voting System Guidelines. The presentation on Chapter 7,
Core Requirements by Voting Activity, is presented by Dr. David Flater
of NIST's Information Technology Laboratory. The presentation includes
questions and answers from the members of the Election Assistance
Commission's Board of Advisors and Standards Boards.

> \[MR. FLATER:\] Good morning, everybody. I am David

Flater. I work with the Core Requirements and Testing

Subcommittee.

*\[Slide 2\]*

What I'm going to do is rationalize the fact that I'm going to breeze
past a lot of material as follows.

Most of Chapter 7, Registration by Voting Activity, does not contain any
huge changes. There were huge changes in the general requirements and
the benchmarks that we talked about earlier. Most of this, however, is
reorganization and rehashing of requirements that you've heard before
from the previous additions of the spec. *\[Slide 3\]*

Furthermore, in Chapter 8, most of the material is informative in
nature, or it is stuff that can be picked up later, so I'm going to skip
that. I am, however, going to hit on a few spots that I think have
caused confusion in the past, or where there are major changes. *\[Slide
4\]*

Okay, a couple of terms. This has been a point of confusion. We have two
different terms defined. One is ballot configuration, and one is ballot
style, and they are both used in requirements. The difference is this. A
ballot configuration is a set of contests in which some group of voters
are entitled to vote. A ballot style is a concrete presentation of a
particular ballot configuration which may nail down certain details like
the language that's used, the ordering of contests, etc. *\[Slide 5\]*

We actually do need both of these terms. Most functional requirements on
what the voting system has to do have nothing to do with the ballot
style. They have to do with the configuration. You have a certain set of
contests that abstractly includes a certain set of choices. You know
that regardless of what language those choices are written and
regardless of what order they were rotated into, a vote for this choice
is a vote for this choice. Most requirements are written at that level.
However, there are exceptions. There are functional requirements on the
system's capability to produce and tabulate different ballot styles. The
EMS has to give you the ability to lay out ballots, associate ballot
styles with political parties, etc. So that's why we have both terms.\
*\[Slide 6\]*

The question that came up earlier was about the misfeed rate, and if a
misfeed occurs, what happens then. The requirements are in Part 1,
Section 7.7.4, titled Misfed Ballots. The equipment has to give you the
ability to clear a misfeed, and very importantly, it must indicate the
status of a misfed ballot. Under no circumstance do you have a ballot
stuck in the machine, and you don't know if it's been added to the tally
yet or not. That's completely banned. Now, with regards to optical scan
accuracy, I mentioned early about how we don't simply have perfect marks
and no marks. We have a lot of other marks.*\
\[Slide 7\]*

And this image gives examples of the different categories. If the voter
perfectly followed the vendor's instructions, you have an ideal mark. If
the voter made a diligent effort to follow the vendors instructions,
sort of mostly filled in the oval, for example, you have reliably
detectable mark, although it's not ideal.

There's another mark specified which I'm calling the standard mark. This
was in response to various issues and questions that were raised back in
the VVSG '05 review period. If you make a nice thick horizontal line
with a number-two pencil, that should be reliably detectable in any
voting system. Beyond that, you get into the category of marginal marks,
which is where lots of interesting behaviors come up. This is where if
you are presented by a ballot containing a mark like this in the
precinct, you have the opportunity to give it back to the voter and say
please clarify your intent. This is the sort of mark that you'd want to
reject. Now below this threshold, we have hesitation marks where
somebody put a dot in the oval but then changed their mind, didn't vote.
They have extraneous marks, which is where someone just didn't put
anything in the oval at all but scribbled in the margin, and then you
have no mark at all.*\
\[Slide 8\]*

So the requirements for optical scanners say 'shall count' reliably
detectable marks which includes the ideal marks, the ones where the
voter made diligence, and also the standard mark of this, a horizontal
line made with a number-two pencil. In the precinct, you shall reject
marginal marks, and you shall ignore extraneous marks and non-marks. But
now there's a 'should' in there, about 'should ignore' hesitation marks
and imperfections, and there is an interesting rationale for why this is
a 'should.' In the real world, you know that you have to calibrate this
equipment to do a good job, knowing that not everyone is going to use
the prescribed marking device. We could make this a 'shall,' and vendors
could produce equipment, could calibrate the equipment they have to meet
all these 'shall' requirements at the same time. If you make the
assumption that everyone will always use the specified marking device,
which is a funny way of saying the right pencil or the right pen,
whatever the vendor specified, but as a pragmatic decision, you know, if
we did that, took all the slack out of the system, then you'd have a
system that performed poorly when given a range of different utensils.
So pragmatically the 'should' is there to deliberately leave some slack
there for calibration to give you the best performance pragmatically.

*\[Slide 9\]*

Jim?

\[QUESTIONER:\] Yes. The extraneous marks, if someone is a little
compulsive, and they filled in the oval either perfectly or almost, and
then they draw a circle and bring a line over to the candidate's name.
That kind of extraneous mark, where does that set of circumstances fall
in this standard?\
*\[Slide 10\]*

\[MR. FLATER:\] Okay, Jim asked about the case where someone has filled
in the oval but then has also drawn around the oval and drawn a line
over the candidate's name. The extraneous parts would be irrelevant in
terms of the system's behavior. That should be detected as a reliably
detectably mark if, in fact, they have diligently filled in the oval.
The extraneous verbiage here is trying to say that if it's not inside
the oval, if it's not in the prescribed marking area, it is irrelevant,
and the only exceptions to this would be if there's a state law that
prescribes a certain behavior with respect to extraneous marks, but that
is outside the scope of this product standard.*\
\[Slide 11\]*

And we have a question.

\[QUESTIONER:\] I was curious about the marginal mark where you say that
it shall reject the marginal mark. Is that saying that it shall reject
it back to the voter to give them another chance to fix that correction?
Or does it say anything about when you override such a ballot? Is it
counting that vote or not counting it?

\[MR. FLATER:\] Okay, the question was in the case of rejections of
marginal marks and what happens if you override it. The override
behavior, I believe, has ended up in the human factors section. By
definition, in the range of marks that the vendor has declared to be
marginal, by definition you are not going to get reproducible results
necessarily. It will be as reproducible as possible, but this is why we
reject these marks because this is in the nebulous zone. We want to get
a clear mark if we can. The equipment within the marginal zone is not
going to be tested for accuracy, so we expect there to be due diligence
given to making it as repeatable as possible. By definition, we're
within the calibration zone here, so I mean this again is a piece of
slack that must be left if the devices are going to be usable in the
pragmatic level. That's also why we want to reject these if we can. We
want you to either give us a clear yes or clear non-mark, don't give us
this thing in the middle.\
*\[Slide 12\]\
* \[QUESTIONER:\] And I was just trying to look toward an absentee that
came in if, in fact, it was being overridden. And what I am saying to
you is that it might be counted depending on how it went through the
next time, it might read a little bit more.

\[MR. FLATER:\] If the voter says oh, I don't want to change the ballot,
just go ahead and override it, I mean they've been informed of their
decision hopefully. You know, this means we don't know how it's going to
be counted.

\[QUESTIONER:\] If it comes by an absentee ballot, then the voter is not
there.

\[MR. FLATER:\] Oh, in the case of absentee, this is not a precinct
count situation anymore. You know, in the central count situation,
you're not rejecting these. You're making a best effort at counting
them. Yes.

\[QUESTIONER:\] What those might turn into would be overvotes at that
point. That's what we would find that the marginal mark may well yield
an overvote.

\[MR. FLATER:\] The comment was that a marginal mark could then cause an
overvote. That is possible.

*\[Slide 13\]*

\[QUESTIONER:\] Okay, on detecting this and it's specifically the
requirement with the line 1 millimeter thick. I think it's 7.7.5-D. You
know I went with the punch card before optical scan years ago. And every
time we got a recount with the punch card, the chad would fall out a
little bit more each time. The more you used them, the more
imperfections you had. Similarly with the optical scan with pencils, the
more they are read through there, the more the lead starts to come off,
and the higher the rate of those marginal marks not being read. So a
line might read fine the first time, but if you run it through the
reader a couple of times as recounts, it won't. So are you going to be
testing that that particular standard holds through multiple counting
approaches?

\[MR. FLATER:\] The question was, given that apparently with pencil
lead, there is an issue that if you recount it multiple times, it starts
to rub off, or smudge, or what have you, and you can lose the
reproducibility of the recount, and are we going to test that effect?
The answer is no, that wasn't on the agenda. The choice of a number-two
pencil here was after considerable debate already about the merits of
talking about specific pencil versus ink versus whatever, and I think
the dominant concern there was, well, pencil should work on everything
at least once, whereas ink won't necessarily work on everything, and
this is a standard mark. We can specify that every equipment, every
optical scanner known to mankind, old and new, any technology should be
able to read this mark that's made in pencil.

\[QUESTIONER:\] I think that is why I was bringing this up, because you
are also saying you want these results to be reproducible, and they may
not be.

\[MR. FLATER:\] The comment was, she brought it up because I was talking
about reproducibility. When we start talking about the relative merits
of different marking devices, it can become a religious and possibly
political argument. So we are at 4:31. I am going to take one more
question, and then we are going to talk about SI.

*\[Slide 14\]*

\[QUESTIONER:\] What you are really saying is that you want it to
reject. At that point, it comes up to the election official to handle
it. And at that point, they are going to have live humans looking at it.
So you have basically done the right thing. You are going after saying
it should be these and not these. Those then become rejected ballots
that live humans have to look at. And if you are going to reprocess
them, then you are going to have to do it as remade ballots.

\[MR. FLATER:\] Yeah, the comment was the handling of marginal marks,
which is a separable issue from the reproducibility of reading pencil
lead. But yeah, ideally marginal marks will be rejected and dealt with.
*\[Slide 15\]*

Last question-

\[QUESTIONER:\] Mine is a real quick one. On your slide right here, I'm
having a hard time understanding

ideal mark, reliable mark, standard. Those all have the parenthetical of
non-marginal. And then you say marginal mark and below you are using the
same parenthetical- not marginal, not marginal, for hesitation,
extraneous. What is the significance on not marginal on this slide?

\[MR. FLATER:\] The significance of not marginal is that we expect the
tabulator to count this mark without rejecting it, or to not count the
non-mark as the case may be. Everything above marginal, we expect to be
counted without rejection. Everything below marginal mark, we expect to
be counted as a non-vote, if you will, same as an empty oval without
rejection. Marginal mark is the boundary case where we expect there to
be rejection and some kind of response.*\
\[Slide 16\]*

\[NARRATOR:\] Additional explanatory presentations on the Voluntary
Voting System Guidelines can be accessed from the Web site:
vote.nist.gov.

[^1]: ^\*^ Certain commercial entities, equipment, or materials may be
    identified in this presentation in order to describe an experimental
    procedure or concept adequately. Such identification is not intended
    to imply recommendation or endorsement by the National Institute of
    Standards and Technology, nor is it intended to imply that the
    entities, materials, or equipment are necessarily the best available
    for the purpose.