Maintenance of the VVSG

Alan Goldfine

*Notes:*

*a) This document uses the \"peer review board\" concept currently being
considered by the EAC.*

*b) This document could be an appendix in the VVSG.*

# Introduction

This document addresses TGDC resolution 31-05. The resolution states,
\"All specifications contain ambiguities that are discovered during
testing of implementations. Similarly, all specifications contain
requirements that can be subject to multiple, equally defensible
interpretations. The TGDC requests that NIST draft a strategy for
maintenance of the VSS, which would address the issuance of
interpretations of the VSS, the resolution of disputes, and the
continuous improvement and revision of the VSS.\"

This document proposes maintenance procedures for the revised VSS, now
called the Voluntary Voting Systems Guidelines (VVSG). The procedures
include the handling of:

1.  requests for interpretation of the VVSG

2.  errors discovered in the VVSG

3.  future revision of the VVSG.

The EAC has the overall responsibility for managing interpretation and
revision of the VVSG. In practice, the EAC would delegate the technical
portion of this work to an appropriate peer review board (PRB). The PRB
would be an impartial body of experts who report to and function on
behalf of the EAC, and who would be responsible for resolving queries
and disputes related to the VVSG itself as well as to the VVSG testing
process. Additional activities that may be under the auspices of the PRB
include:

-   maintaining liaison with EAC accredited testing authorities (ATAs)
    and appropriate standards bodies,

-   participating in the assessment of testing laboratories seeking EAC
    accreditation status,

-   recommending changes to the VVSG conformance test suite and the ATA
    conformance test methods,

-   serving as technical advisor to the EAC.

The following sections propose explicit procedures for VVSG maintenance.

# Interpretation of the VVSG

Requests for interpretation of the VVSG may come from one of several
interested parties:

-   a voting system vendor, who might need clarification of a
    requirement in the VVSG during the development of an implementation,
    or who might disagree with an interpretation of the VVSG contained
    in a certification test report received from an ATA,

-   an ATA, who might encounter an ambiguity in the VVSG during its
    testing of an implementation,

-   a state or local election official.

The interpretation procedure is as follows:

1.  The interested party submits a request for interpretation to the
    EAC. The request should include the following information:

    -   Date of request,

    -   Complete contact information of the requestor,

    -   Identification of the VVSG text, including section and page
        numbers, for which the interpretation is being requested,

    -   The specific question regarding the VVSG text

    -   A suggested interpretation (optional).

2.  The EAC refers the request to the PRB.

3.  The PRB performs a preliminary analysis of the request, and either
    rejects the request and sends an explanation of the rejection to the
    EAC, or provides the EAC with an estimate of the time the PRB will
    take to resolve the issue.

4.  The EAC notifies the requestor of the rejection or the estimated
    resolution time, as appropriate.

5.  If the PRB will be providing an interpretation, it conducts a full
    analysis, and submits its recommended interpretation to the EAC.

6.  The EAC approves the interpretation, and forwards the interpretation
    to the requestor.

7.  The EAC/PRB enters the interpretation into a publicly available
    register of official interpretations of the VVSG.

It is possible that the interpretation will lead to an error report to
correct or clarify the VVSG text that caused the interpretation question
in the first place. See Resolution of Errors below.

# Resolution of Errors in the VVSG

An error in the VVSG may be discovered by any of the interested parties
listed in the interpretation section above, by the peer review board
itself during its resolution of an ambiguity or other examination of the
VVSG, or by a member of the general public. The error resolution
procedure is as follows:

1.  The interested party submits an error report to the EAC. The report
    should include the following information:

    -   Date of submission,

    -   Complete contact information of the submitter,

    -   Identification of the error in the VVSG text, including section
        and page numbers,

    -   A suggested change to the VVSG that would correct the error
        (optional).

2.  The EAC refers the error report to the PRB.

3.  The PRB performs a preliminary analysis of the error report, and
    either determines that the VVSG is not in error and sends an
    explanation to the EAC, or provides the EAC with an estimate of the
    time the PRB will take to correct the error.

4.  The EAC notifies the error report submitter of the rejection or the
    estimated resolution time, as appropriate.

5.  If the PRB will be addressing the error, it conducts a full
    analysis, develops a proposed correction for the error, and posts a
    description of the correction on a publicly available web site for
    comment. This description includes a discussion of any implications
    that the proposed correction would have regarding conformance to the
    VVSG.

6.  Upon conclusion of the comment period, the PRB, based on the
    comments it received, submits a possibly revised correction to the
    EAC.

7.  The EAC publishes the proposed correction in the Federal Register.

8.  Based on Federal Register comments, the EAC either approves or
    disapproves the proposed correction.

9.  If the correction is approved, the EAC/PRB enters the description of
    the correction into a publicly available register of changes to the
    VVSG.

10. A revised VVSG is made publicly available.

If the error report had been inspired by a request for interpretation of
the VVSG, it is possible that the final approved correction to the VVSG
might not agree totally with the earlier interpretation. More generally,
a previously certified voting system might no longer be considered
conformant to the VVSG after the correction to the VVSG. In such a
situation, the EAC would determine, on a case-by-case basis, whether to
retest for certification the affected system(s). This determination
would be in accordance with the EAC\'s certification/decertification
policies, and with due consideration of the impact of the
non-conformities.

# Strategy for Continuous Improvement and Revision of the VVSG

# Minor Revisions of the VVSG

The peer review board may, from time to time, propose limited changes to
the VVSG. These changes may be suggested by interpretation questions
that have revealed ambiguities, by advancements in technology, or for
other reasons. The minor revision procedure is as follows:

1.  The PRB proposes, to the EAC, the change in the VVSG.

2.  Upon approval by the EAC, the PRB drafts a complete description of
    the proposed change and posts the proposal on its website for public
    comment. This proposal should include the following information:

    -   Date of proposal,

    -   Closing date for comments,

    -   A complete description of the proposed change, including
        rationale,

    -   A discussion of any implications that the proposed change would
        have regarding conformance to the VVSG.

3.  Upon conclusion of the comment period, the PRB, based on the
    received comments, either withdraws the change proposal, or revises
    and submits it to the EAC.

4.  The EAC publishes the proposed change in the Federal Register.

5.  Based on Federal Register comments, the EAC either approves or
    disapproves the proposed change.

6.  If the change is approved, the EAC/PRB enters the change proposal
    into a publicly available register of changes to the VVSG.

7.  A revised VVSG is made publicly available.

It is possible that a previously certified voting system might no longer
be considered conformant to the VVSG after the change to the VVSG. In
such a situation, the EAC would determine, on a case-by-case basis,
whether to retest for certification the affected system(s). This
determination would be in accordance with the EAC\'s
certification/decertification policies, and with due consideration of
the impact of the non-conformities.

# Major Revisions of the VVSG

The EAC may decide that advances in technology or changes in
requirements for voting systems require a major rewrite of the VVSG. The
EAC would then establish appropriate procedures for developing and
promulgating a comprehensive revision of the VVSG. These procedures
would include appropriate transition rules for handling previously
certified voting systems.

The EAC may wish to establish a revision review cycle, at the end of
which the VVSG is reviewed to determine if a revision needs to occur or
if the document is still relevant. Organizations such as ISO often set 5
years as the length of the cycle, after which a standard is reviewed if
it hasn\'t been revised before then.