Voting System Standards

Volume I:

Performance Standards

April, 2002

Federal Election Commission

United States of America

Acknowledgm nts

Voting Systems Standards, Volumes I and II are the long awaited update to the Federal

Election Commission's (FEC's) 1990 publication the Performance and Test Standards

for Punchcard, Marksense, and Direct Recording Electronic Voting Systems. The update

was executed under contract to the FEC by American Management Systems (AMS).

Although a host of AMS staff members contributed to this document, the FEC is

especially indebted to AMS Senior Principal James Ward and AMS Project Manager

Dennis Berg for their energy, patience and genuine dedication to understanding not only

the technical requirements of the Standards, but also the intricacies of the U.S. election

process and the unique role played by federal, State and local governments in this arena.

The program to test and certify voting systems using the Voting Systems Standards was

initiated in 1994 by the National Association of State Election Directors (NASED).

Members of NASED's Voting Systems Board contributed continuously and significantly

to the update process. Special thanks go to Thomas Wilkey, Executive Director of the

New York State Board of Elections and Chairman of NASED's Voting Systems Board.

Mr. Wilkey not only served the FEC and AMS as a twenty-four hour-a-day resource on

technical issues, but also provided invaluable experience and calm leadership in steering

the project through sometimes stormy procedural and political waters.

Several members of the Technical Subcommittee of NASED's Voting Systems Board

contributed specific language and technical expertise to the update. The FEC would like

to recognize the following Subcommittee members for their time and effort:

Brit Williams, Professor Emeritus, Computer Science & Information

Systems, Kennesaw State College

Steve Freeman, Software Consultant, League City, Texas

Jay W. Nispel, Senior Principal Engineer, Computer Sciences Corporation

Paul Craft, Computer Audit Analyst, State of Florida

David Elliott, Assistant Director of Elections, Washington

Other Voting system Board members contributing to the process included:

Denise Lamb, Director, New Mexico state Bureau of Elections and ViceChair of the NASED Voting Systems Board

Sandy Steinbach, Director of Elections, Iowa

Donetta Davidson, Colorado Secretary of State
Connie Schmidt, Commissioner, Johnson County Election Commission

Donna Royson, Director of Election Services, South Carolina State

Election Commission

Robert Naegle, President, Granite Creek Technologies

Yvonne Smith, Assistant Executive Director, Illinois state Board of

Elections (Retired)

The FEC would also like to recognize the sizable contributions of Doug Lewis, Executive

Director of the Election Center, and Secretariat to the NASED Voting Systems Board, for

his substantive input as well as for his tireless logistical efforts during numerous meetings

and conferences throughout the country.

The 2002 update of the Voting Systems Standards benefited immeasurably from the

contributions and practical experience of the certified Independent Test Authorities

(ITA's) whose task it is to utilize the Standards in real-world testing scenarios.

Representatives of the ITA's include:

Jim Dearman, Wyle Laboratories

Shawn Southworth & Jennifer Price, CIBER

Gail Audette & Carolyn Coggins, Systest Labs

While literally hundreds of other people contributed to this effort, the FEC would also

like to recognize the specific material contributions of Stephen Berger, Chairman, and the

members of the IEEE Voting Equipment Standards Project 1583; Roy Saltman,

Consultant; David Capozzi and David Baquis of the U.S. Access Board; Janet Blizzard

and Jonathan Hahn of the Disability Rights Section of the U.S. Department of Justice;

and John O'Hara, Ph.D., Brookhaven National Laboratory.

Finally, the Federal Election Commission is grateful to those members of the academic

community, the voting system vendor community, and the community of public interest

groups for providing comments, highlighting discrepancies, and ultimately, creating a

dynamic and useful final product.

ii
In Memory of:

James F. Hendrix

Executive Director, South Carolina State Election Commission

Member, NASED Voting Systems Board

For a career of dedicated service to the election community and the voters of the

State of South Carolina

iii

Volume I — Performance Standards

Table of Contents

1 INTRODUCTION 1-7

1.1 Objectives and Usage of the Voting System Standards 1-7

1.2 Development History for Initial Standards 1-8

1.3 Update of the Standards 1-9

1.4 Accessibility for Individuals with Disabilities 1-9

1.5 Definitions 1-10

1.5.1 Voting System 1-10

1.5.2 Paper-Based Voting System 1-11

1.5.3 Direct Record Electronic (DRE) Voting System 1-12

1.5.4 Public Network Direct Record Electronic (DRE) Voting System 1-12

1.5.5 Precinct Count Voting System 1-12

1.5.6 Central Count Voting System 1-13

1.6 Application of the Standards and Test Specifications 1-13

1.6.1 Qualification Tests 1-14

1.6.2 Certification Tests 1-15

1.6.3 Acceptance Tests 1-16

1.7 Outline of Contents 1-16

2 FUNCTIONAL CAPABILITIES 2-19

2.1 Scope 2-19

2.2 Overall System Capabilities 2-20

2.2.1 Security 2-20

2.2.2 Accuracy 2-21

2.2.3 Error Recovery 2-22

2.2.4 Integrity 2-22

2.2.5 System Audit 2-23

2.2.6 Election Management System 2-27

2.2.7 Accessibility 2-28

2.2.8 Vote Tabulating Program 2-32

2.2.9 Ballot Counter 2-33

2.2.10 Telecommunications 2-33

2.2.11 Data Retention 2-34

2.3 Pre-voting Functions 2-35

11
Volume I — Performance Standards

2.3.1 Ballot Preparation 2-35

2.3.2 Election Programming 2-38

2.3.3 Ballot and Program Installation and Control 2-38

2.3.4 Readiness Testing 2-39

2.3.5 Verification at the Polling Place 2-40

2.3.6 Verification at the Central Location 2-41

2.4 Voting Functions 2-41

2.4.1 Opening the Polls 2-41

2.4.2 Activating the Ballot (DRE Systems) 2-43

2.4.3 Casting a Ballot 2-43

2.5 Post-Voting Functions 2-46

2.5.1 Closing the Polling Place (Precinct Count) 2-46

2.5.2 Consolidating Vote Data 2-46

2.5.3 Producing Reports 2-47

2.5.4 Broadcasting Results 2-48

2.6 Maintenance, Transportation, and Storage 2-48

3 HARDWARE STANDARDS 3-49

3.1 Scope 3-49

3.1.1 Hardware Sources 3-50

3.1.2 Organization of this Section 3-50

3.2 Performance Requirements 3-50

3.2.1 Accuracy Requirements 3-51

3.2.2 Environmental Requirements 3-52

3.2.3 Election Management System (EMS) Requirements 3-57

3.2.4 Vote Recording Requirements 3-57

3.2.5 Paper-based Conversion Requirements 3-62

3.2.6 Processing Requirements 3-64

3.2.7 Reporting Requirements 3-66

3.2.8 Vote Data Management Requirements 3-67

3.3 Physical Characteristics 3-68

3.3.1 Size 3-68

3.3.2 Weight 3-68

3.3.3 Transport and Storage of Precinct Systems 3-68

3.4 Design, Construction, and Maintenance Characteristics 3-69

3.4.1 Materials, Processes, and Parts 3-69

3.4.2 Durability 3-69

3.4.3 Reliability 3-69

3.4.4 Maintainability 3-70

3.4.5 Availability 3-71

3.4.6 Product Marking 3-72

3.4.7 Workmanship 3-73

3.4.8 Safety 3-73

1.2
Volume I — Performance Standards

3.4.9 Human Engineering—Controls and Displays 3-73

4 SOFTWARE STANDARDS 4-75

4.1 Scope 4-75

4.1.1 Software Sources 4-76

4.1.2 Location and Control of Software and Hardware on Which it Operates4-76

4.1.3 Exclusions 4-77

4.2 Software Design and Coding Standards 4-77

4.2.1 Selection of Programming Languages 4-78

4.2.2 Software Integrity 4-78

4.2.3 Software Modularity and Programming 4-78

4.2.4 Control Constructs 4-80

4.2.5 Naming Conventions 4-80

4.2.6 Coding Conventions 4-81

4.2.7 Comment Conventions 4-81

4.3 Data and Document Retention 4-82

4.4 Audit Record Data 4-82

4.4.1 Pre-election Audit Records 4-82

4.4.2 System Readiness Audit Records 4-83

4.4.3 In-Process Audit Records 4-84

4.4.4 Vote Tally Data 4-85

4.5 Vote Secrecy (DRE Systems) 4-85

5 TELECOMMUNICATIONS 5-87

5.1 Scope 5-87

5.1.1 Types of Components 5-88

5.1.2 Telecommunications Operations and Providers 5-89

5.1.3 Data Transmissions 5-90

5.2 Design, Construction, and Maintenance Requirements 5-90

5.2.1 Accuracy 5-91

5.2.2 Durability 5-91

5.2.3 Reliability 5-91

5.2.4 Maintainability 5-91

5.2.5 Availability 5-91

5.2.6 Integrity 5-91

5.2.7 Confirmation 5-92

6 SECURITY STANDARDS 6-93

6.1 Scope 6-93

6.1.1 System Components and Sources 6-94

6.1.2 Location and Control of Software and Hardware on Which it Operates6-94

1.3
Volume I — Performance Standards

6.1.3 Elements of Security Outside Vendor Control 6-95

6.1.4 Organization of this Section 6-95

6.2 Access Control 6-96

6.2.1 Access Control Policy 6-96

6.2.2 Access Control Measures 6-97

6.3 Physical Security Measures 6-98

6.3.1 Polling Place Security 6-98

6.3.2 Central Count Location Security 6-98

6.4 Software Security 6-98

6.4.1 Software and Firmware Installation 6-99

6.4.2 Protection Against Malicious Software 6-99

6.5 Telecommunications and Data Transmission 6-99

6.5.1 Access Control 6-100

6.5.2 Data Integrity 6-100

6.5.3 Data Interception Prevention 6-100

6.5.4 Protection Against External Threats 6-100

6.5.5 Shared Operating Environment 6-102

6.5.6 Access to Incomplete Election Returns and Interactive Queries 6-103

6.6 Security for Transmission of Official Data Over Public Communications

Networks 6-103

6.6.1 General Security Requirements for Systems Transmitting Data Over

Public Networks 6-103

6.6.2 Voting Process Security for Casting Individual Ballots over a Public

Telecommunications Network 6-104

7 QUALITY ASSURANCE 7-107

7.1 Scope 7-107

7.2 General Requirements 7-107

7.3 Components from Third Parties 7-108

7.4 Responsibility for Tests 7-108

7.5 Parts & Materials Special Tests and Examinations 7-108

7.6 Quality Conformance Inspections 7-109

7.7 Documentation 7-109

8 CONFIGURATION MANAGEMENT 8-111

8.1 Scope 8-111

8.1.1 Configuration Management Requirements 8-111

1-4
Volume I — Performance Standards

8.1.2 Organization of Configuration Management Standards 8-112

8.1.3 Application of Configuration Management Requirements 8-112

8.2 Configuration Management Policy 8-113

8.3 Configuration Identification 8-113

8.3.1 Structuring and Naming Configuration Items 8-113

8.3.2 Versioning Conventions 8-113

8.4 Baseline, Promotion, and Demotion Procedures 8-114

8.5 Configuration Control Procedures 8-114

8.6 Release Process 8-114

8.7 Configuration Audits 8-115

8.7.1 Physical Configuration Audit 8-115

8.7.2 Functional Configuration Audit 8-116

8.8 Configuration Management Resources 8-116

9 OVERVIEW OF QUALIFICATION TESTS 9-119

9.1 Scope 9-119

9.2 Documentation Submitted by Vendor 9-120

9.3 Voting Equipment Submitted by Vendor 9-121

9.4 Testing Scope 9-121

9.4.1 Test Categories 9-122

9.4.2 Sequence of Tests and Audits 9-126

9.5 Test Applicability 9-126

9.5.1 General Applicability 9-126

9.5.2 Modifications to Qualified Systems 9-127

9.6 Qualification Test Process 9-128

9.6.1 Pre-test Activities 9-129

9.6.2 Qualification Testing 9-130

9.6.3 Qualification Report Issuance and Post-test Activities 9-133

9.6.4 Resolution of Testing Issues 9-134

10 APPENDIX A - GLOSSARY 10-135

11 APPENDIX B - APPLICABLE DOCUMENTS 11-143

11.1 Documents Incorporated in the Standards 11-143

1.5
Volume I — Performance Standards

11.2 Standards Development Documents 11-144

11.3 Guidance Documents 11-146

12 APPENDIX C – USABILITY 12-149

12.1 Scope 12-149

12.2 General Principles 12-150

12.3 Overall Design and Layout of the Voter Workspace 12-151

12.4 Ballot Legibility and Understandability 12-152

12.5 Information Grouping 12-154

12.6 Voting Input Fields 12-155

12.7 Navigation and Manipulation of Ballots 12-156

12.8 Preventing and Minimizing Voter Errors 12-157

12.9 Help and System Failure 12-158

12.10 Voter Familiarization and Training 12-159

1-6
Volume I — Performance Standards

1 Introduction

1.1 Objectives and Usage of the Voting

System Standards

State and local officials today are confronted with increasingly complex voting system

technology and an increased risk of voting system failure. Responding to calls for

assistance from the states, the United States Congress authorized the Federal Election

Commission (FEC) to develop voluntary national voting systems standards for

computer-based systems. The resulting FEC Voting System Standards ("the

Standards") seek to aid state and local election officials in ensuring that new voting

systems are designed to function accurately and reliably, thus ensuring the system's

integrity. States are free to adopt the Standards in whole or in part. States may also

choose to enact stricter performance requirements for systems used in their

jurisdictions.

The Standards specify minimum functional requirements, performance characteristics,

documentation requirements, and test evaluation criteria. For the most part, the

Standards address what a voting system should reliably do, not how system

components should be configured to meet these requirements. It is not the intent of the

Standards to impede the design and development of new, innovative equipment by

vendors. Furthermore, the Standards balance risk and cost by requiring voting systems

to have essential , but not excessive, capabilities.

The Standards are not intended to define appropriate election administration practices.

However, the total integrity of the election process can only be ensured if

implementation of the Standards is coupled with effective election administration

practices.

The Standards are intended for use by multiple audiences to support their respective

roles in the development, testing, and acquisition of voting systems:

♦ Authorities responsible for the analysis and testing of such systems in support

of qualification and/or certification of systems for purchase within a

designated jurisdiction;

1.7
Volume I — Performance Standards

♦ State and local agencies evaluating voting systems to be procured within their

jurisdictions; and

♦ Designers and manufacturers of voting systems.

1.2 Development History for Initial Standards

Much of the groundwork for the Standards' development was laid by a national study

conducted in 1975 by the National Bureau of Standards, now known as the National

Institute of Standards and Technology (NIST). This study was requested by the FEC's

Office of Election Administrator's predecessor, the Office of Federal Elections of the

General Accounting Office. The report, "Effective Use of Computing Technology in

Vote-Tallying," made a number of recommendations bearing directly on the Standards

project. After analyzing computer-related election problems encountered in the past,

the report concluded that one of the basic causes for these difficulties was the lack of

appropriate technical skill at the state and local level for developing or implementing

sophisticated and complex standards against which voting system hardware and

software could be tested.

Following the release of this report, Congress mandated that the FEC, with the

cooperation and assistance of the National Bureau of Standards, study and report on

the feasibility of developing "voluntary engineering and procedural performance

standards for voting systems used in the United States." (2 U.S.C. §431 Note) The

resulting 1983 study cited a substantial number of technical and managerial problems

that affected the integrity of the vote counting process. It also asserted the need for a

federal agency to develop national performance standards that could be used as a tool

by state and local election officials in the testing, certification, and procurement of

computer-based voting systems. In 1984, Congress approved initial funding for the

Standards.

The FEC held a series of public hearings in developing the initial Standards. State and

local election officials, election system vendors, technical consultants, and others

reviewed drafts of the proposed criteria. The FEC considered their many comments

and made appropriate revisions. Before final issuance, the FEC publicly announced the

availability of the latest draft of the Standards in the Federal Register and requested

that all interested parties submit final comments. The FEC meticulously reviewed all

responses to the notice and incorporated corrections and suitable suggestions.

Ultimately, the final product was the result of considerable deliberation, close

consultation with election officials, and careful consideration of comments from all

interested parties.

In January 1990, the FEC issued the performance standards and testing procedures for

punchcard, marksense, and direct recording electronic (DRE) voting systems. The

Standards did not cover paper ballot and mechanical lever systems because paper

ballots are sufficiently self-explanatory not to require technical standards and

mechanical lever systems are no longer manufactured or sold in the United States. The

FEC also did not incorporate requirements for mainframe computer hardware because

1 8
Volume I — Performance Standards

it was reasonable to assume that sufficient engineering and performance criteria

already governed the operation of mainframe computers. However, vote tally software

installed on mainframes is covered by the Standards.

1.3 Update of the Standards

Today, over two-thirds of the States have adopted the Standards in whole or in part. As

a result, the voting systems marketed today are dramatically improved. Election

officials are better assured that the voting systems they procure will work accurately

and reliably. Voting system failures are declining, and now primarily involve preStandard equipment, untested equipment configurations, or the mismanagement of

tested equipment. Overall, systems integrity and the election processes have improved

markedly.

However, advances in voting technology, legislative changes, and the proliferation of

electronic voting systems make an update of the Standards necessary. The industry has

been marked by widespread integration of personal computer technology and nonmainframe servers into DRE voting systems.

In addition, voting systems need to be responsive to the Americans with Disabilities

Act (ADA) of 1990 and guidelines developed to assist in implementing the ADA.

1.4 Accessibility for Individuals with

Disabilities

Voters and election officials who use voting systems represent a broad spectrum of the

population, and include individuals with disabilities who may have difficulty using

traditional voting systems. In developing accessibility provisions for the Standards,

the FEC requested assistance from the Access Board, the federal agency in the

forefront of promulgating accessibility provisions. The Access Board submitted

technical standards designed to meet the diverse needs of voters with a broad range of

disabilities. The FEC has adopted the entirety of the Access Board's recommendations

and incorporated them into the Standards. These recommendations comprise the bulk

of the accessibility provisions found in Section 2.2.7. Implementing these provisions,

however, will not entirely eliminate the need to accommodate the needs of some

disabled voters by human interface.

The FEC anticipates that during the lifetime of this version of the Standards increased

obligations will be placed upon election officials at every jurisdictional level to

provide voting equipment tailored to meet the needs of voters with disabilities. To

facilitate jurisdictions in meeting accessibility needs, the Standards mandate that most

voting systems incorporate some accessible voting capabilities. The Standards also

1.9
Volume I — Performance Standards

mandate that systems incorporating a DRE component meet specific technological

requirements. To do so, it is anticipated that a vendor will have to either configure all

of the system's voting stations to meet the accessibility specifications or will have to

design a unique station that conforms to the accessibility requirements and is part of

the overall voting system configuration.

Under no circumstances should compliance with requirements for accessibility be

viewed as mutually exclusive from compliance with any other provision of the

Standards. If a voting system contains a machine uniquely designed to meet the

accessibility requirements, such a machine will be tested for compliance with the

accessibility requirements, as well as for compliance with all of the DRE standards, in

order to ensure that an accessible machine does not unintentionally abrogate the

mandates of the Standards.

1.5 Definitions

The Standards contain terms describing function, design, documentation, and testing

attributes of equipment and computer programs. Unless otherwise specified, the

intended sense of technical terms is that which is commonly used by the information

technology industry. In some cases terminology is specific to elections or voting

systems, and a glossary of those terms is contained in Appendix A. Non-technical

terms not listed in Appendix A shall be interpreted according to their standard

dictionary definitions.

Additionally, the following terms are defined below:

♦ Voting system;

♦ Paper-based voting system;

♦ Direct record electronic (DRE) voting system;

♦ Public network direct record electronic (DRE) voting systems;

♦ Precinct count voting system; and

♦ Central count voting system.

1.5.1 Voting System

A voting system is a combination of mechanical, electromechanical, or electronic

equipment. It includes the software required to program, control, and support the

equipment that is used to define ballots; to cast and count votes; to report and/or

display election results; and to maintain and produce all audit trail information. A

1.10
Volume I — Performance Standards

voting system may also include the transmission of results over telecommunication

networks.

Additionally, a voting system includes the associated documentation used to operate

the system, maintain the system, identify system components and their versions, test

the system during its development and maintenance, maintain records of system errors

and defects, and determine specific changes made after system qualification. By

definition, this includes all documentation required in Section 9.4.

Traditionally, a voting system has been defined by the mechanism the system uses to

cast votes and further categorized by the location where the system tabulates ballots.

However, the Standards recognize that as the industry develops unique solutions to

various challenges and as voting systems become more responsive to the needs of

election officials and voters, the rigid dichotomies between voting system types may

be blurred. Innovations that use a fluid understanding of system types can greatly

improve the voting system industry, but only if controls are in place to monitor and

control integrity through the proper evaluation of the system brought for qualification.

As such, vendors that submit a system that integrates components from more than one

traditional system type or a system that includes components not addressed in this

Standard shall submit the results of all beta tests of the new system. Vendors also shall

submit a proposed test plan to the appropriate independent test authority recognized by

the National Association of State Election Directors (NASED) to conduct national

qualification testing of voting systems. The Standards permit vendors to produce or

utilize interoperable components of a voting system that are tested within the full

voting system configuration.

1.5.2 Paper-Based Voting System

A Paper-Based Voting System, (referred to in the initial Standards as a Punchcard and

Marksense [P&M] Voting System) records votes, counts votes, and produces a

tabulation of the vote count from votes cast on paper cards or sheets. A punchcard

voting system allows a voter to record votes by means of holes punched in designated

voting response locations. A marksense voting system allows a voter to record votes by

means of marks made by the voter directly on the ballot, usually in voting response

locations. Additionally, a paper based system may record votes using other approaches

whereby the voter's selections are indicated by marks made on a paper ballot by an

electronic input device, as long as such an input device does not independently record,

store, or tabulate the voters selections.
Volume I — Performance Standards

1.5.3 Direct Record Electronic (DRE) Voting System

A Direct Record Electronic (DRE) Voting System records votes by means of a ballot

display provided with mechanical or electro-optical components that can be activated

by the voter; that processes data by means of a computer program; and that records

voting data and ballot images in memory components. It produces a tabulation of the

voting data stored in a removable memory component and as printed copy. The system

may also provide a means for transmitting individual ballots or vote totals to a central

location for consolidating and reporting results from precincts at the central location.

1.5.4 Public Network Direct Record Electronic (DRE)

Voting System

A Public Network Direct Record Electronic (DRE) Voting System is an election

system that uses electronic ballots and transmits vote data from the polling place to

another location over a public network as defined in Section 5.1.2. Vote data may be

transmitted as individual ballots as they are cast, periodically as batches of ballots

throughout the election day, or as one batch at the close of voting. For purposes of the

Standards, Public Network DRE Voting Systems are considered a form of DRE Voting

System and are subject to the standards applicable to DRE Voting Systems. However,

because transmitting vote data over public networks relies on equipment beyond the

control of the election authority, the system is subject to additional threats to system

integrity and availability. Therefore, additional requirements discussed in Section 5

and 6 apply.

The use of public networks for transmitting vote data must provide the same level of

integrity as other forms of voting systems, and must be accomplished in a manner that

precludes three risks to the election process: automated casting of fraudulent votes,

automated manipulation of vote counts, and disruption of the voting process such that

the system is unavailable to voters during the time period authorized for system use.

1.5.5 Precinct Count Voting System

A Precinct Count Voting System is a voting system that tabulates ballots at the polling

place. These systems typically tabulate ballots as they are cast, and print the results

after the close of polling. For DREs, and for some paper-based systems, these systems

provide electronic storage of the vote count and may transmit results to a central

location over public telecommunication networks.

1.12
Volume I — Performance Standards

1.5.6 Central Count Voting System

A Central Count Voting System is a voting system that tabulates ballots from multiple

precincts at a central location. Voted ballots are typically placed into secure storage at

the polling place. Stored ballots are transported or transmitted to a central counting

place. The systems produce a printed report of the vote count, and may produce a

report stored on electronic media.

1.6 Application of the Standards and Test

Specifications

The Standards apply to all system hardware, software, telecommunications, and

documentation intended for use to:

♦ Prepare the voting system for use in an election;

♦ Produce the appropriate ballot formats;

♦ Test that the voting system and ballot materials have been properly prepared

and are ready for use;

♦ Record and count votes;

♦ Consolidate and report results;

♦ Display results on-site or remotely; and

♦ Maintain and produce all audit trail information.

In general, the Standards define functional requirements and performance

characteristics that can be assessed by a series of defined tests . Standards are

mandatory requirements and are designated by use of the term "shall."

Some voting systems use one or more readily available commercial off-the-shelf

(COTS) devices (such as card readers, printers, or personal computers) or software

products (such as operating systems, programming language compilers, or database

management systems). COTS devices and software are exempted from certain portions

of the qualification testing process as defined herein, as long as such products are not

modified for use in a voting system.

Generally, voting systems are subject to the following three testing phases prior to

being purchased or leased:

♦ Qualification tests;

♦ State certification tests; and

1.13
Volume I — Performance Standards

♦ State and/or local acceptance tests.

1.6.1 Qualification Tests

Qualification tests validate that a voting system meets the requirements of the

Standards and performs according to the vendor's specifications for the system. Such

tests encompass the examination of software; the inspection and evaluation of system

documentation; tests of hardware under conditions simulating the intended storage,

operation, transportation, and maintenance environments; operational tests to validate

system performance and function under normal and abnormal conditions; and

examination of the vendor's system development, testing, quality assurance, and

configuration management practices. Qualification tests address individual system

components or elements, as well as the integrated system as a whole.

Since 1994, qualification tests for voting systems have been performed by Independent

Test Authorities (ITAs) certified by the National Association of State Election

Directors (NASED). NASED has certified an ITA for either the full scope of

qualification testing or a distinct subset of the total scope of testing. To date, ITAs

have been certified only for distinct subsets of testing. Upon the successful completion

of testing by an ITA, the ITA issues a Qualification Test Report to the vendor and

NASED. The qualification test report remains valid for as long as the voting system

remains unchanged.

Upon receipt of test reports that address the full scope of testing, NASED issues a

Qualification Number that indicates the system has been tested by certified ITAs for

compliance with the Standards and qualifies for the certification process of states that

have adopted the Standards. The Qualification Number applies to the system as a

whole, and does not apply to individual system components or untested configurations.

After a system has completed qualification testing, further examination of a system is

required if modifications are made to hardware, software, or telecommunications,

including the installation of software on different hardware. Vendors request review of

modifications by the appropriate ITA based on the nature and scope of changes made

and the scope of the ITA's role in NASED qualification. The ITA will determine the

extent to which the modified system should be resubmitted for qualification testing and

the extent of testing to be conducted.

Generally, a voting system remains qualified under the standards against which it was

tested, as long as no modifications not approved by an ITA are made to the system.

However, if a new threat to a particular voting system is discovered, it is the

prerogative of NASED to determine which qualified voting systems are vulnerable,

whether those systems need to be retested, and the specific tests to be conducted. In

addition, when new standards supersede the standards under which the system was

qualified, it is the prerogative of NASED to determine when systems that were

qualified under the earlier standards will lose their qualification, unless they are tested

to meet current standards.

1.14
Volume I — Performance Standards

Among other things, qualification testing complements and evaluates the vendor's

developmental testing and beta testing. The ITA is expected to evaluate the

completeness of the vendor's developmental test program, including the sufficiency of

vendor tests conducted to demonstrate compliance with the Standards as well as the

system's performance specifications. The ITA undertakes sample testing of the

vendor's test modules and also designs independent system-level tests to supplement

and check those designed by the vendor. Although some of the qualification tests are

based on those prescribed in the Military Standards, in most cases the test conditions

are less stringent, reflecting commercial, rather than military, practice.

1.6.2 Certification Tests

Certification tests are performed by individual states, with or without the assistance of

outside consultants, to:

♦ Confirm that the voting system presented is the same as the one qualified

through the Standards;

♦ Test for the proper implementation of state-specific requirements;

♦ Establish a baseline for future evaluations or tests of the system, such as

acceptance testing or state review after modifications have been made; and

♦ Define acceptance tests.

Precise certification test scripts are not included in the Standards, as they must be

defined by the state, with its laws, election practices, and needs in mind. However, it is

recommended that they not duplicate qualification tests, but instead focus on

functional tests and qualitative assessment to ensure that the system operates in a

manner that is acceptable under state law. If a voting system is modified after state

certification, it is recommended that States reevaluate the system to determine if

further certification testing is warranted.

Certification tests performed by individual states typically rely on information

contained in documentation provided by the vendor for system design, installation,

operations, required facilities and supplies, personnel support and other aspects of the

voting system. States and jurisdictions may define information and documentation

requirements additional to those defined in the Standards. By design, the Standards,

and qualification testing of voting systems for compliance with the Standards, do not

address these additional requirements. However, qualification testing addresses all

capabilities of a voting system stated by the vendor in the system documentation

submitted to an ITA, including additional capabilities that are not required by the

Standards.

1.15
Volume I — Performance Standards

1.6.3 Acceptance Tests

Acceptance tests are performed at the state or local jurisdiction level upon system

delivery by the vendor to:

♦ Confirm that the system delivered is the specific system qualified by NASED

and, when applicable, certified by the state;

♦ Evaluate the degree to which delivered units conform to both the system

characteristics specified in the procurement documentation, and those

demonstrated in the qualification and certification tests; and

♦ Establish a baseline for any future required audits of the system.

Some of the operational tests conducted during qualification may be repeated during

acceptance testing.

1.7 Outline of Contents

The organization of the Standards has been simplified to facilitate its use. Volume I,

Voting System Performance Standards, is intended for use by the broadest audience,

including voting system developers, equipment manufacturers and suppliers,

independent test authorities, local agencies that purchase and deploy voting systems,

state organizations that certify a system prior to procurement by a local jurisdiction,

and public interest organizations that have an interest in voting systems and voting

systems standards.

♦ Section 2 describes the functional capabilities required of voting systems.

♦ Sections 3 through 6 describe specific performance standards for election

system hardware, software, telecommunications and security, respectively.

♦ Sections 7 and 8 describe practices for quality assurance and configuration

management, respectively, to be used by vendors, and required information

about vendor practices that will be reviewed in concert with system

qualification and certification test processes and system purchase decisions.

♦ Section 9 provides an overview of the test and measurement process used by

test authorities for qualification and re-qualification of voting systems.

♦ Appendix A provides a glossary of important terms used in Volume I.

♦ Appendix B lists the publications that were used for guidance in the

preparation of the Standards. These publications contain information that is

useful in interpreting and complying with the requirements of the Standards.

♦ Appendix C addresses issues of usability of voting systems, commonly

referred to as "human factors." This appendix does not represent mandates that

116
Volume I — Performance Standards

voting systems will be tested against, but rather contain recommendations and

best practices on usability issues designed to provide vendors and election

officials with guidance on designing and procuring systems that are easy and

intuitive to use by voters.

Volume II, Voting System Qualification Testing Standards describes the standards for

the technical information submitted by the vendor to support testing; the development

of test plans by the ITA for initial system testing and testing of system modifications;

the conduct of system qualification tests by the ITA; and the test reports generated by

the ITA. This volume complements the content of Volume I and it is intended

primarily for use by ITAs, state organizations that certify a system, and vendors.

1.17

Volume I — Performance Standards

2 Functional Capabilities

2.1 Scope

This section contains standards detailing the functional capabilities required of a

voting system. This section sets out precisely what it is that a voting system is required

to do. In addition, this section sets forth the minimum actions a voting system must be

able to perform to be eligible for qualification.

For organizational purposes, functional capabilities are categorized by the phase of

election activity in which they are required:

♦ Overall Capabilities: These functional capabilities apply throughout the

election process. They include security, accuracy, integrity, system

auditability, election management system, vote tabulation, ballot counters,

telecommunications, and data retention.

♦ Pre-voting Capabilities: These functional capabilities are used to prepare the

voting system for voting. They include ballot preparation, the preparation of

election-specific software (including firmware), the production of ballots or

ballot pages, the installation of ballots and ballot counting software (including

firmware), and system and equipment tests.

♦ Voting Capabilities: These functional capabilities include all operations

conducted at the polling place by voters and officials including the generation

of status messages.

♦ Post-voting Capabilities: These functional capabilities apply after all votes have

been cast. They include closing the polling place; obtaining reports by voting

machine, polling place, and precinct; obtaining consolidated reports; and

obtaining reports of audit trails.

♦ Maintenance, Transportation and Storage Capabilities: These capabilities are

necessary to maintain, transport, and store voting system equipment.

219
Volume I — Performance Standards

In recognition of the diversity of voting systems, the Standards apply specific

requirements to specific technologies. Some of the Standards apply only if the system

incorporates certain optional functions (for example, voting systems employing

telecommunications to transmit voting data). For each functional capability, common

standards are specified. Where necessary, common standards are followed by

standards applicable to specific technologies (i.e., paper-based or DRE) or intended

use (i.e., central or precinct count).

2.2 Overall System Capabilities

This section defines required functional capabilities that are system-wide in nature

and not unique to pre-voting, voting, and post-voting operations. All voting systems

shall provide the following functional capabilities:

♦ Security;

♦ Accuracy;

♦ Error recovery;

♦ Integrity;

♦ System auditability;

♦ Election management system;

♦ Accessibility:

♦ Vote tabulating;

♦ Ballot counters; and

♦ Data Retention.

Voting systems may also include telecommunications components. Technical

standards for these capabilities are described in Sections 3 through 6 of the Standards.

2.2.1 Security

System security is achieved through a combination of technical capabilities and sound

administrative practices. To ensure security, all systems shall:

a. Provide security access controls that limit or detect access to critical system

components to guard against loss of system integrity, availability,

confidentiality, and accountability.

2.20
Volume I — Performance Standards

b. Provide system functions that are executable only in the intended manner and

order, and only under the intended conditions.

c. Use the system's control logic to prevent a system function from executing if

any preconditions to the function have not been met.

d. Provide safeguards to protect against tampering during system repair, or

interventions in system operations, in response to system failure.

e. Provide security provisions that are compatible with the procedures and

administrative tasks involved in equipment preparation, testing, and operation.

f. If access to a system function is to be restricted or controlled, the system shall

incorporate a means of implementing this capability.

g. Provide documentation of mandatory administrative procedures for effective

system security.

2.2.2 Accuracy

Memory hardware, such as semiconductor devices and magnetic storage media, must

be accurate. The design of equipment in all voting systems shall provide for the highest

possible levels of protection against mechanical, thermal, and electromagnetic stresses

that impact system accuracy. Section 3 provides additional information on

susceptibility requirements.

2.2.2.1 Common Standards

To ensure vote accuracy, all systems shall:

a. Record the election contests, candidates, and issues exactly as defined by

election officials;

b. Record the appropriate options for casting and recording votes;

c. Record each vote precisely as indicated by the voter and be able to produce an

accurate report of all votes cast;

d. Include control logic and data processing methods incorporating parity and

check-sums (or equivalent error detection and correction methods) to

demonstrate that the system has been designed for accuracy; and

e. Provide software that monitors the overall quality of data read-write and

transfer quality status, checking the number and types of errors that occur in

any of the relevant operations on data and how they were corrected.

2.21
Volume I — Performance Standards

2.2.2.2 DRE System Standards

As an additional means of ensuring accuracy in DRE systems, voting devices shall

record and retain redundant copies of the original ballot image. A ballot image is an

electronic record of all votes cast by the voter, including undervotes.

2.2.3 Error Recovery

To recover from a non-catastrophic failure of a device, or from any error or

malfunction that is within the operator's ability to correct, the system shall provide the

following capabilities:

a. Restoration of the device to the operating condition existing immediately prior

to the error or failure, without loss or corruption of voting data previously

stored in the device;

b. Resumption of normal operation following the correction of a failure in a

memory component, or in a data processing component, including the central

processing unit; and

c. Recovery from any other external condition that causes equipment to become

inoperable, provided that catastrophic electrical or mechanical damage due to

external phenomena has not occurred.

2.2.4 Integrity

Integrity measures ensure the physical stability and function of the vote recording and

counting processes.

2.2.4.1 Common Standards

To ensure system integrity, all systems shall:

a. Protect, by a means compatible with these Standards, against a single point of

failure that would prevent further voting at the polling place;

b. Protect against the interruption of electronic power;

c. Protect against generated or induced electromagnetic radiation;

d. Protect against ambient temperature and humidity fluctuations;

e. Protect against the failure of any data input or storage device;

2-22
Volume I — Performance Standards

f. Protect against any attempt at improper data entry or retrieval;

g. Record and report the date and time of normal and abnormal events;

h. Maintain a permanent record of all original audit data that cannot be modified

or overridden but may be augmented by designated authorized officials in

order to adjust for errors or omissions (e.g. during the canvassing process.)

i. Detect and record every event, including the occurrence of an error condition

that the system cannot overcome, and time-dependent or programmed events

that occur without the intervention of the voter or a polling place operator; and

Include built-in measurement, self-test, and diagnostic software and hardware

for detecting and reporting the system's status and degree of operability.

2.2.4.2 DRE Systems Standards

In addition to the common standards, DRE systems shall:

a. Maintain a record of each ballot cast using a process and storage location that

differs from the main vote detection, interpretation, processing, and reporting

path; and

b. Provide a capability to retrieve ballot images in a form readable by humans.

2.2.5 System Audit

This section describes the context and purpose of voting system audits and sets forth

specific functional requirements. Additional technical audit requirements are set forth

in Section 4.

2.2.5.1 System Audit Purpose and Context

Election audit trails provide the supporting documentation for verifying the correctness

of reported election results. They present a concrete, indestructible archival record of

all system activity related to the vote tally, and are essential for public confidence in

the accuracy of the tally, for recounts, and for evidence in the event of criminal or civil

litigation.

The following audit trail requirements are based on the premise that system-generated

creation and maintenance of audit records reduces the chance of error associated with

manually generated audit records. Because most audit capability is automatic, the

system operator has less information to track and record, and is less likely to make

mistakes or omissions.

2.23
Volume I — Performance Standards

The sections that follow present operational requirements critical to acceptable

performance and reconstruction of an election. Requirements for the content of audit

records are described in Section 4 of the Standards.

The requirements for all system types, both precinct and central count, are described in

generic language. Because the actual implementation of specific characteristics may

vary from system to system, it is the responsibility of the vendor to describe each

system's characteristics in sufficient detail that ITAs and system users can evaluate the

adequacy of the system's audit trail. This description shall be incorporated in the

System Operating Manual, which is part of the Technical Data Package (TDP).

Documentation of items such as paper ballots delivered and collected, administrative

procedures for system security, and maintenance performed on voting equipment are

also part of the election audit trail, but are not covered in these technical standards.

Future volumes of the Standards will address these and other system operations

practices. In the interim, useful guidance is provided by the Innovations in Election

Administration #10, Ballot Security and Accountability, available from the FEC's

Office of Election Administration.

2.2.5.2 Operational Requirements

Audit records shall be prepared for all phases of elections operations performed using

devices controlled by the jurisdiction or its contractors. These records rely upon

automated audit data acquisition and machine-generated reports, with manual input of

some information. These records shall address the ballot preparation and election

definition phase, system readiness tests, and voting and ballot-counting operations.

The software shall activate the logging and reporting of audit data as described in the

following sections.

2.2.5.2.1 Time, Sequence, and Preservation of Audit Records

The timing and sequence of audit record entries is as important as the data contained in

the record. All voting systems shall meet the following requirements for time,

sequence and preservation of audit records:

a. Except where noted, systems shall provide the capability to create and

maintain a real-time audit record. This capability records and provides the

operator or precinct official with continuous updates on machine status. This

information allows effective operator identification of an error condition

requiring intervention, and contributes to the reconstruction of election-related

events necessary for recounts or litigation.

b. All systems shall include a real-time clock as part of the system's hardware.

The system shall maintain an absolute record of the time and date or a record

relative to some event whose time and data are known and recorded.

c. All audit record entries shall include the time-and-date stamp.

2.24
Volume I — Performance Standards

d. The audit record shall be active whenever the system is in an operating mode.

This record shall be available at all times, though it need not be continually

visible.

e. The generation of audit record entries shall not be terminated or altered by

program control, or by the intervention of any person. The physical security

and integrity of the record shall be maintained at all times.

f. Once the system has been activated for any function, the system shall preserve

the contents of the audit record during any interruption of power to the system

until processing and data reporting have been completed.

g. The system shall be capable of printing a copy of the audit record. A separate

printer is not required for the audit record, and the record may be produced on

the standard system printer if all the following conditions are met:

1) The generation of audit trail records does not interfere with the production

of output reports;

2) The entries can be identified so as to facilitate their recognition,

segregation, and retention; and

3) The audit record entries are kept physically secure.

2.2.5.2.2 Error Messages

All voting systems shall meet the following requirements for error messages:

a. The system shall generate, store, and report to the user all error messages as

they occur;

b. All error messages requiring intervention by an operator or precinct official

shall be displayed or printed unambiguously in easily understood language

text, or by means of other suitable visual indicators;

c. When the system uses numerical error codes for trained technician

maintenance or repair, the text corresponding to the code shall be selfcontained, or affixed inside the unit device. This is intended to reduce

inappropriate reactions to error conditions, and to allow for ready and effective

problem correction;

d. All error messages for which correction impacts vote recording or vote

processing shall be written in a manner that is understandable to an election

official who possesses training on system use and operation, but does not

possess technical training on system servicing and repair;

e. The message cue for all systems shall clearly state the action to be performed

in the event that voter or operator response is required;

f. System design shall ensure that erroneous responses will not lead to

irreversible error; and

2.25
Volume I — Performance Standards

g. Nested error conditions shall be corrected in a controlled sequence such that

system status shall be restored to the initial state existing before the first error

occurred.

2.2.5.2.3 Status Messages

The Standards provide latitude in software design so that vendors can consider various

user processing and reporting needs. The jurisdiction may require some status and

information messages to be displayed and reported in real-time. Messages that do not

require operator intervention may be stored in memory to be recovered after ballot

processing has been completed.

The system shall display and report critical status messages using unambiguous

indicators or English language text. The system need not display non-critical status

messages at the time of occurrence. Systems may display non-critical status messages

(i.e., those that do not require operator intervention) by means of numerical codes for

subsequent interpretation and reporting as unambiguous text.

Systems shall provide a capability for the status messages to become part of the realtime audit record. The system shall provide a capability for a jurisdiction to designate

critical status messages.

2.2.5.3 COTS General Purpose Computer System Requirements

Further requirements must be applied to COTS operating systems to ensure

completeness and integrity of audit data for election software. These operating systems

are capable of executing multiple application programs simultaneously. These systems

include both servers and workstations (or "PCs"), including the many varieties of

UNIX and Linux, and those offered by Microsoft and Apple. Election software

running on these COTS systems is vulnerable to unintended effects from other user

sessions, applications, and utilities, executing on the same platform at the same time as

the election software.

"Simultaneous processes" of concern include unauthorized network connections,

unplanned user logins, and unintended execution or termination of operating system

processes. An unauthorized network connection or unplanned user login can host

unintended processes and user actions, such as the termination of operating system

audit, the termination of election software processes, or the deletion of election

software audit and logging data. The execution of an operating system process could

be a full system scan at a time when that process would adversely affect the election

software processes. Operating system processes improperly terminated could be

system audit or malicious code detection.

To counter these vulnerabilities, three operating system protections are required on all

such systems on which election software is hosted. First, authentication shall be

configured on the local terminal (display screen and keyboard) and on all external

2.26
Volume I – Performance Standards

connection devices ("network cards" and "ports"). This ensures that only authorized

and identified users affect the system while election software is running.

Second, operating system audit shall be enabled for all session openings and closings,

for all connection openings and closings, for all process executions and terminations,

and for the alteration or deletion of any memory or file object. This ensures the

accuracy and completeness of election data stored on the system. It also ensures the

existence of an audit record of any person or process altering or deleting system data

or election data.

Third, the system shall be configured to execute only intended and necessary processes

during the execution of election software. The system shall also be configured to halt

election software processes upon the termination of any critical system process (such

as system audit) during the execution of election software.

2.2.6 Election Management System

The Election Management System (EMS) is used to prepare ballots and programs for

use in casting and counting votes, and to consolidate, report, and display election

results. An EMS shall generate and maintain a database, or one or more interactive

databases, that enables election officials or their designees to perform the following

functions:

a. Define political subdivision boundaries and multiple election districts as

indicated in the system documentation;

b. Identify contests, candidates, and issues

c. Define ballot formats and appropriate voting options;

d. Generate ballots and election-specific programs for vote recording and vote

counting equipment;

e. Install ballots and election-specific programs;

f. Test that ballots and programs have been properly prepared and installed;

g. Accumulate vote totals at multiple reporting levels as indicated in the system

documentation;

h. Generate the post-voting reports required by Section 2.5; and

i Process and produce audit reports of the data indicated in Section 4.5.

2-27
Volume I — Performance Standards

2.2.7 Accessibility

The Standards provide requirements for voting systems to meet the accessibility needs

of a broad range of voters with disabilities. To do so, it is anticipated that a vendor

will have to either configure all of the system's voting stations to meet the accessibility

specifications or will have to design a unique station that conforms to the accessibility

requirements and is part of the overall voting system configuration. Efforts to meet the

accessibility requirements shall not violate the privacy, secrecy, and integrity demands

of the Standards.

2.2.7.1 Common Standards

To facilitate accessibility, all voting systems shall be capable of meeting the following

conditions, as illustrated in Figures 2-1 through 2-4 :

a. Where clear floor space only allows forward approach to an object, the maximum

high forward reach allowed shall be 48inches. The minimum low forward reach is

15 inches.

b. Where forward reach is over an obstruction with knee space below, the maximum

level forward reach is 25 inches. When the obstruction is less than 20 inches deep,

the maximum high forward reach is 48 inches. When the obstruction projects 20

to 25 inches, the maximum high forward reach is 44 inches.

c. The position of any operable control is determined with respect to a vertical plane

that is 48 inches in length, centered on the operable control, and at the maximum

protrusion of the product within the 48-inch length;

d. Where any operable control is 10 inches or less behind the reference plane, have a

height that is between 15 inches and 54 inches above the floor;

e. Where any operable control is more than 10 inches and not more than 24 inches

behind the reference plane, have a height between 15 inches and 46 inches above

the floor; and

f. Have operable controls that are not more than 24 inches behind the reference

plane.

2-28
Volume I — Performance Standards

Figure 2-1

NOTE x shall be 4 25 in (635 mm): z shza be x, When x < 20 in (510 mm), then y shall be 48 kr, t1220 mm) tiniurn.

When x e 20 1:13 25 in (510 to 635 mm), then y stein be 44 in ( I I20 mm) mezaznurn

Figure 2-2
Vertical Plane Relative to the Operable Control

Figure 2-3

2.2.7.2 DRE Standards

DRE voting systems shall provide, as part of their configuration, the capability to

provide access to voters with a broad range of disabilities. This capability shall:

a. Not require, the voter to bring their own assistive technology to a polling

place;

b. Provide audio information and stimulus that:
Volume I — Performance Standards

1) Communicates to the voter the complete content of the ballot;

2) Provides instruction to the voter in operation of the voting device;

3) Provides instruction so that the voter has the same vote capabilities and

options as those provided by the system to individuals who are not using

audio technology;

4) For a system that supports write-in voting, enables the voter to review the

voter's write-in input, edit that input, and confirm that the edits meet the

voter's intent;

5) Enables the voter to request repetition of any information provided by the

system;

6) Supports the use of headphones provided by the system that may be

discarded after each use;

7) Provides the audio signal through an industry standard connector for

private listening using a 1/8 inch stereo headphone jack to allow individual

voters to supply personal headsets; and

8) Provides a volume control with an adjustable amplification up to a

maximum of 105 dB that automatically resets to the default for each voter;

c. Provide, in conformance with FCC Part 68, a wireless coupling for assistive

devices used by people who are hard of hearing when a system utilizes a

telephone style handset to provide audio information;

d. Meet the requirements of ANSI C63.19-2001 Category 4 to avoid

electromagnetic interference with assistive hearing devices;

e. For electronic image displays, permit the voter to:

1) Adjust the contrast settings;

2) Adjust color settings, when color is used; and

3) Adjust the size of the text so that the height of capital letters varies over a

range of 3 to 6.3 millimeters;

f. For a device with touchscreen or contact-sensitive controls, provide an input

method using mechanically operated controls or keys that shall:

1) Be tactilely discernible without activating the controls or keys;

2) Be operatable with one hand and not require tight grasping, pinching, or

twisting of the wrist;

231
Volume I — Performance Standards

3) Require a force less than 5 lbs (22.2 N) to operate; and

4) Provide no key repeat function;

g. For a system that requires a response by a voter in a specific period of time,

alert the voter before this time period has expired and allow the voter

additional time to indicate that more time is needed;

h. For a system that provides sound cues as a method to alert the voter about a

certain condition, such as the occurrence of an error, or a confirmation, the

tone shall be accompanied by a visual cue for users who cannot hear the audio

prompt; and

i. Provide a secondary means of voter identification or authentication when the

primary means of doing so uses biometric measures that require a voter to

possess particular biological characteristics.

2.2.8 Vote Tabulating Program

Each voting system shall have a vote tabulation program that will meet specific

functional requirements.

2.2.8.1 Functions

The vote tabulating program software resident in each voting device, vote count server,

or other devices shall include all software modules required to:

a. Monitor system status and generate machine-level audit reports;

b. Accommodate device control functions performed by polling place officials

and maintenance personnel;

c. Register and accumulate votes; and

d. Accommodate variations in ballot counting logic.

2.2.8.2 Voting Variations

There are significant variations among the election laws of the 50 states with respect to

permissible ballot contents, voting options, and the associated ballot counting logic.

The TDP accompanying the system shall specifically identify which of the following

items can and cannot be supported by the system, as well as how the system can

implement the items supported:

232
Volume I — Performance Standards

a. Closed primaries;

b. Open primaries;

c. Partisan offices;

d. Non-partisan offices;

e. Write-in voting;

f. Primary presidential delegation nominations;

g. Ballot rotation;

h. Straight party voting;

i. Cross-party endorsement;

j. Split precincts;

k. Vote for N of M;

1. Recall issues, with options;

m. Cumulative voting;

n. Ranked order voting; and

o. Provisional or challenged ballots.

2.2.9 Ballot Counter

For all voting systems, each device that tabulates ballots shall provide a counter that:

a. Can be set to zero before any ballots are submitted for tally;

b. Records the number of ballots cast during a particular test cycle or election;

c. Increases the count only by the input of a ballot;

d. Prevents or disables the resetting of the counter by any person other than

authorized persons at authorized points; and

e. Is visible to designated election officials.

2.2.10 Telecommunications

For all voting systems that use telecommunications for the transmission of data during

pre-voting, voting or post-voting activities, capabilities shall be provided that ensure

data are transmitted with no alteration or unauthorized disclosure during transmission.

Such transmissions shall not violate the privacy, secrecy, and integrity demands of the

233
Volume I — Performance Standards

Standards. Section 5 of the Standards describes telecommunications standards that

apply to, at a minimum, the following types of data transmissions:

♦ Voter Authentication: Coded information that confirms the identity of a voter

for security purposes for a system that transmit votes individually over a public

network;

♦ Ballot Definition: Information that describes to a voting machine the content

and appearance of the ballots to be used in an election;

♦ Vote Transmission to Central Site: For systems that transmit votes individually

over a public network, the transmission of a single vote to the county (or

contractor) for consolidation with other county vote data;

♦ Vote Count: Information representing the tabulation of votes at any one of

several levels: polling place, precinct, or central count; and

♦ List of Voters: A listing of the individual voters who have cast ballots in a

specific election.

2.2.11 Data Retention

United States Code Title 42, Sections 1974 through 1974e, states that election

administrators shall preserve for 22 months "all records and paper that came into

(their) possession relating to an application, registration, payment of poll tax, or other

act requisite to voting." This retention requirement applies to systems that will be used

at anytime for voting of candidates for Federal offices (e.g., Member of Congress,

United States Senator, and/or Presidential Elector). Therefore, all systems shall

provide for maintaining the integrity of voting and audit data during an election and for

a period of at least 22 months thereafter.

Because the purpose of this law is to assist the Federal government in discharging its

law enforcement responsibilities in connection with civil rights and elections crimes,

its scope must be interpreted in keeping with that objective. The appropriate state or

local authority must preserve all records that may be relevant to the detection and

prosecution of federal civil rights or election crimes for the 22-month federal retention

period, if the records were generated in connection with an election that was held in

whole or in part to select federal candidates. It is important to note that Section 1974

does not require that election officials generate any specific type or classification of

election record. However, if a record is generated, Section 1974 comes into force and

the appropriate authority must retain the records for 22 months.

For 22-month document retention, the general rule is that all printed copy records

produced by the election database and ballot processing systems shall be so labeled

and archived. Regardless of system type, all audit trail information spelled out in

subsection 4.5 of the Standards shall be retained in its original format, whether that be

real-time logs generated by the system, or manual logs maintained by election

personnel. The election audit trail includes not only in-process logs of election-night

234
Volume I — Performance Standards

(and subsequent processing of absentee or provisional ballots), but also time logs of

baseline ballot definition formats, and system readiness and testing results.

In many voting systems, the source of election-specific data (and ballot formats) is a

database or file. In precinct count systems, this data is used to program each machine,

establish ballot layout, and generate tallying files. It is not necessary to retain this

information on electronic media if there is an official, authenticatable printed copy of

all final database information. However, it is recommended that the state or local

jurisdiction also retain electronic records of the aggregate data for each device so that

reconstruction of an election is possible without data re-entry. The same requirement

and recommendation applies to vote results generated by each precinct device or

system.

2.3 Pre-voting Functions

This section defines capabilities required to support functions performed prior to the

opening of polls. All voting systems shall provide capabilities to support:

♦ Ballot preparation;

♦ Election programming;

♦ Ballot and program installation and control;

♦ Readiness testing;

♦ Verification at the polling place; and

♦ Verification at the central counting place.

The standards also include requirements to ensure compatible interfaces with the ballot

definition process and the reporting of election results.

2.3.1 Ballot Preparation

Ballot preparation is the process of using election databases to define the specific

contests, questions, and related instructions to be contained in ballots and to produce

all permissible ballot layouts. Ballot preparation requirements include:

♦ General capabilities for ballot preparation;

♦ Ballot formatting; and

♦ Ballot production.

235
Volume I — Performance Standards

2.3.1.1 General Capabilities

All systems shall provide the general capabilities for ballot preparation.

2.3.1.1.1 Common Standards

All systems shall be capable of:

a. Enabling the automatic formatting of ballots in accordance with the

requirements for offices, candidates, and measures qualified to be placed on

the ballot for each political subdivision and election district;

b. Collecting and maintaining the following data:

1) Offices and their associated labels and instructions;

2) Candidate names and their associated labels; and

3) Issues or measures and their associated text;

c. Supporting the maximum number of potentially active voting positions as

indicated in the system documentation;

d. For a primary election, generating ballots that segregate the choices in partisan

races by party affiliation;

e. Generating ballots that contain identifying codes or marks uniquely associated

with each format; and

f. Ensuring that vote response fields, selection buttons, or switches properly

align with the specific candidate names and/or issues printed on the ballot

display, ballot card or sheet, or separate ballot pages.

2.3.1.1.2 Paper-Based System Standards

In addition to the common standards, paper-based systems shall meet the following

standards applicable to the technology used:

a. Enable voters to make selections by punching a hole or by making a mark in

areas designated for this purpose upon each ballot card or sheet;

b. For punchcard systems, ensure that the vote response fields can be properly

aligned with punching devices used to record votes; and

c. For marksense systems, ensure that the timing marks align properly with the

vote response fields.

2-36
Volume I — Performance Standards

2.3.1.2 Ballot Formatting

Ballot formatting is the process by which election officials or their designees use

election databases and vendor system software to define the specific contests and

related instructions contained on the ballot and present them in a layout permitted by

state law. All systems shall provide a capability for:

a. Creation of newly defined elections;

b. Rapid and error-free definition of elections and their associated ballot layouts;

c. Uniform allocation of space and fonts used for each office, candidate, and

contest such that the voter perceives no active voting position to be preferred

to any other;

d. Simultaneous display of the maximum number of choices for a single contest

as indicated by the vendor in the system documentation;

e. Retention of previously defined formats for an election;

f. Prevention of unauthorized modification of any ballot formats; and

g. Modification by authorized persons of a previously defined ballot format for

use in a subsequent election.

2.3.1.3 Ballot Production

Ballot production is the process of converting ballot formats to a media ready for use

in the physical ballot production or electronic presentation.

2.3.1.3.1 Common Standards

The voting system shall provide a means of printing or otherwise generating a ballot

display that can be installed in all system voting devices for which it is intended. All

systems shall provide a capability to ensure:

a. The electronic display or printed document on which the user views the ballot

is capable of rendering an image of the ballot in any of the languages required

by The Voting Rights Act of 1965, as amended;

b. The electronic display or printed document on which the user views the ballot

does not show any advertising or commercial logos of any kind, whether

public service, commercial, or political, unless specifically provided for in

State law. Electronic displays shall not provide connection to such material

through hyperlink; and

c. The ballot conforms to vendor specifications for type of paper stock, weight,

size, shape, size and location of punch or mark field used to record votes,

2.37
Volume I — Performance Standards

folding, bleed through, and ink for printing if paper ballot documents or paper

displays are part of the system.

2.3.1.3.2 Paper-Based System Standards

In addition to the common standards, vendor documentation for marksense systems

shall include specifications for ballot materials to ensure that vote selections are read

from only a single ballot at a time, without detection of marks from multiple ballots

concurrently (e.g., reading of bleed-through from other ballots).

2.3.2 Election Programming

Election programming is the process by which election officials or their designees use

election databases and vendor system software to logically define the voter choices

associated with the contents of the ballots. All systems shall provide for the:

a. Logical definition of the ballot, including the definition of the number of

allowable choices for each office and contest;

b. Logical definition of political and administrative subdivisions, where the list

of candidates or contests varies between polling places;

c. Exclusion of any contest on the ballot in which the voter is prohibited from

casting a ballot because of place of residence, or other such administrative or

geographical criteria;

d. Ability to select from a range of voting options to conform to the laws of the

jurisdiction in which the system will be used; and

e. Generation of all required master and distributed copies of the voting program,

in conformance with the definition of the ballots for each voting device and

polling place, and for each tabulating device.

2.3.3 Ballot and Program Installation and Control

All systems shall provide a means of installing ballots and programs on each piece of

polling place or central count equipment in accordance with the ballot requirements of

the election and the requirements of the jurisdiction in which the equipment will be

used.

All systems shall include the following at the time of ballot and program installation:

a. A detailed work plan or other documentation providing a schedule and steps

for the software and ballot installation, which includes a table outlining the

key dates, events and deliverables;

2 38
Volume I — Performance Standards

b. A capability for automatically verifying that the software has been properly

selected and installed in the equipment or in a programmable memory devices

and for indicating errors; and

c. A capability for automatically validating that software correctly matches the

ballot formats that it is intended to process, for detecting errors, and for

immediately notifying an election official of detected errors.

2.3.4 Readiness Testing

Election personnel conduct equipment and system readiness tests prior to the start of

an election to ensure that the voting system functions properly, to confirm that system

equipment has been properly integrated, and to obtain equipment status reports.

2.3.4.1 Common Standards

All systems shall provide the capabilities to:

a. Verify that voting machines or vote recording and data processing equipment,

precinct count equipment, and central count equipment are properly prepared

for an election, and collect data that verifies equipment readiness;

b. Obtain status and data reports from each set of equipment;

c. Verify the correct installation and interface of all system equipment;

d. Verify that hardware and software function correctly;

e. Generate consolidated data reports at the polling place and higher

jurisdictional levels; and

f. Segregating test data from actual voting data, either procedurally or by

hardware/software features.

Resident test software, external devices, and special purpose test software connected to

or installed in voting devices to simulate operator and voter functions may be used for

these tests provided that the following standards are met:

a. These elements shall be capable of being tested separately, and shall be proven

to be reliable verification tools prior to their use; and

b. These elements shall be incapable of altering or introducing any residual effect

on the intended operation of the voting device during any succeeding test and

operational phase.

2.39
Volume I — Performance Standards

2.3.4.2 Paper-Based Systems

Paper-based systems shall:

a. Support conversion testing that uses all potential ballot positions as active

positions; and

b. Support conversion testing of ballots with active position density for systems

without pre-designated ballot positions.

2.3.5 Verification at the Polling Place

Election officials perform verification at the polling place to ensure that all voting

systems and equipment function properly before and during an election. All systems

shall provide a formal record of the following, in any media, upon verification of the

authenticity of the command source:

a. The election's identification data;

b. The identification of all equipment units;

c. The identification of the polling place;

d. The identification of all ballot formats;

e. The contents of each active candidate register by office and of each active

measure register at all storage locations (showing that they contain only zeros);

f. A list of all ballot fields that can be used to invoke special voting options; and

g. Other information needed to confirm the readiness of the equipment, and to

accommodate administrative reporting requirements.

To prepare voting devices to accept voted ballots, all voting systems shall provide the

capability to test each device prior to opening to verify that each is operating correctly.

At a minimum, the tests shall include:

a. Confirmation that there are no hardware or software failures; and

b. Confirm that the device is ready to be activated for accepting votes.

If a precinct count system includes equipment for the consolidation of polling place

data at one or more central counting places, it shall have means to verify the correct

extraction of voting data from transportable memory devices, or to verify the

transmission of secure data over secure communication links.

2-40
Volume I — Performance Standards

2.3.6 Verification at the Central Location

Election officials perform verification at the central location to ensure that vote

counting and vote consolidation equipment and software function properly before and

after an election. Upon verification of the authenticity of the command source, any

system used in a central count environment shall provide a printed record of the

following :

a. The election's identification data;

b. The contents of each active candidate register by office and of each active

measure register at all storage locations (showing that they contain all zeros);

and

c. Other information needed to ensure the readiness of the equipment and to

accommodate administrative reporting requirements.

2.4 Voting Functions

All systems shall support:

♦ Opening the polls; and

♦ Casting a ballot.

Additionally, all DRE systems shall support:

♦ Activating the ballot.

♦ Augmenting the election counter; and

♦ Augmenting the life-cycle counter.

2.4.1 Opening the Polls

The capabilities required for opening the polls are specific to individual voting system

technologies. At a minimum, the systems shall provide the functional capabilities

indicated below.

2.4.1.1 Opening the Polling Place (Precinct Count Systems)

To allow voting devices to be activated for voting, the system shall provide:
Volume I — Performance Standards

a. An internal test or diagnostic capability to verify that all of the polling place

tests specified in Section 2.3.5 have been successfully completed; and

b. Automatic disabling any device that has not been tested until it has been

tested.

2.4.1.2 Paper-Based System Standards

The standards for opening the polling place for paper-based systems consist of

common standards and additional standards that apply to precinct count paper-based

systems.

2.4.1.2.1 All Paper-Based Systems

To facilitate opening the polls, all paper-based systems shall include:

a. A means of verifying that ballot punching or marking devices are properly

prepared and ready to use;

b. A voting booth or similar facility, in which the voter may punch or mark the

ballot in privacy; and

c. Secure receptacles for holding voted ballots.

2.4.1.2.2 Precinct Count Paper-Based Systems

In addition to the above requirements, all paper-based precinct count equipment shall

include a means of:

a. Activating the ballot counting device;

b. Verifying that the device has been correctly activated and is functioning

properly; and

c. Identifying device failure and corrective action needed.

2.4.1.3 DRE System Standards

To facilitate opening the polls, all DRE systems shall include:

a. A security seal, a password, or a data code recognition capability to prevent

the inadvertent or unauthorized actuation of the poll-opening function;

b. A means of enforcing the execution of steps in the proper sequence if more

than one step is required;

c. A means of verifying the system has been activated correctly; and

2-42
Volume I — Performance Standards

d. A means of identifying system failure and any corrective action needed.

2.4.2 Activating the Ballot (DRE Systems)

To activate the ballot, all DRE systems shall:

a. Enable election officials to control the content of the ballot presented to the

voter, whether presented in printed form or electronic display, such that each

voter is permitted to record votes only in contests in which that voter is

authorized to vote;

b. Allow each eligible voter to cast a ballot;

c. Prevent a voter from voting on a ballot to which he or she is not entitled; and

d. Prevent a voter from casting more than one ballot in the same election.

e. Activate the casting of a ballot in a general election;

f. Enable the selection of the ballot that is appropriate to the party affiliation

declared by the voter in a primary election;

g. Activate all portions of the ballot upon which the voter is entitled to vote; and

h. Disable all portions of the ballot upon which the voter is not entitled to vote.

2.4.3 Casting a Ballot

Some required capabilities for casting a ballot are common to all systems. Others are

specific to individual voting technologies or intended use. Systems must provide

additional functional capabilities that enable accessibility to disabled voters as defined

in Section 2.2.7 of the Standards.

2.4.3.1 Common Standards

To facilitate casting a ballot, all systems shall:

a. Provide text that is at least 3 millimeters high and provide the capability to

adjust or magnify the text to an apparent size of 6.3 millimeters;

b. Protect the secrecy of the vote such that the system cannot reveal any

information about how a particular voter voted, except as otherwise required

by individual State law;

2.43
Volume I — Performance Standards

c. Record the selection and non-selection of individual vote choices for each

contest and ballot measure;

d. Record the voter's selection of candidates whose names do not appear on the

ballot, if permitted under State law, and record as many write-in votes as the

number of candidates the voter is allowed to select;

e. In the event of a failure of the main power supply external to the voting

system, provide the capability for any voter who is voting at the time to

complete casting a ballot, allow for the graceful shutdown of the voting system

without loss or degradation of the voting and audit data, and allow voters to

resume voting once the voting system has reverted to back-up power; and

f. Provide the capability for voters to continue casting ballots in the event of a

failure of a telecommunications connection within the polling place or

between the polling place and any other location.

2.4.3.2 Paper-Based Systems Standards

The standards for casting a ballot for paper-based systems consist of common

standards and additional standards that apply to precinct count paper-based systems.

2.4.3.2.1 All Paper-Based Systems

All paper-based systems shall:

a. Allow the voter to easily identify the voting field that is associated with each

candidate or ballot measure response;

b. Allow the voter to punch or mark the ballot to register a vote;

c. Allow either the voter or the appropriate election official to place the voted

ballot into the ballot counting device (for precinct count systems) or into a

secure receptacle (for central count systems); and

d. Protect the secrecy of the vote throughout the process.

2.4.3.2.2 Precinct Count Paper-Based Systems

In addition to the above requirements, all paper-based precinct count systems shall:

a. Provide feedback to the voter that identifies specific contests or ballot issues

for which an overvote or undervote is detected;

b. Allow the voter, at the voter's choice, to vote a new ballot or submit the ballot

`as is' without correction; and

c. Allow an authorized election official to turn off the capabilities defined in 'a'

and 'b' above.

244
Volume I — Performance Standards

2.4.3.3 DRE Systems Standards

In addition to the above common requirements, DRE systems shall:

a. Prohibit the voter from accessing or viewing any information on the display

screen that has not been authorized by election officials and preprogrammed

into the voting system (i.e., no potential for display of external information or

linking to other information sources);

b. Enable the voter to easily identify the selection button or switch, or the active

area of the ballot display that is associated with each candidate or ballot

measure response;

c. Allow the voter to select his or her preferences on the ballot in any legal

number and combination;

d. Indicate that a selection has been made or canceled;

e. Indicate to the voter when no selection, or an insufficient number of selections,

has been made in a contest;

f. Prevent the voter from overvoting;

g. Notify the voter when the selection of candidates and measures is completed;

h. Allow the voter, before the ballot is cast, to review his or her choices and, if

the voter desires, to delete or change his or her choices before the ballot is

cast;

For electronic image displays, prompt the voter to confirm the voter's choices

before casting his or her ballot, signifying to the voter that casting the ballot is

irrevocable and directing the voter to confirm the voter's intention to cast the

ballot;

Notify the voter after the vote has been stored successfully that the ballot has

been cast;

k. Notify the voter that the ballot has not been cast successfully if it is not stored

successfully, including storage of the ballot image, and provide clear

instruction as to the steps the voter should take to cast his or her ballot should

this event occur;

1. Provide sufficient computational performance to provide responses back to

each voter entry in no more than three seconds;

m. Ensure that the votes stored accurately represent the actual votes cast;

n. Prevent modification of the voter's vote after the ballot is cast;

o. Provide a capability to retrieve ballot images in a form readable by humans (in

accordance with the requirements of Section 2.2.2.2 and 2.2.4.2);

P. Increment the proper ballot position registers or counters;

245
Volume I — Performance Standards

q. Protect the secrecy of the vote throughout the voting process;

r. Prohibit access to voted ballots until after the close of polls;

s. Provide the ability for election officials to submit test ballots for use in

verifying the end-to-end integrity of the system; and

t. Isolate test ballots such that they are accounted for accurately in vote counts

and are not reflect in official vote counts for specific candidates or measures.

2.5 Post-Voting Functions

All systems shall provide capabilities to accumulate and report results for the

jurisdiction and to generate audit trails. In addition, precinct count systems must

provide a means to close the polling place including generating appropriate reports. If

the system provides the capability to broadcast results, additional standards apply.

2.5.1 Closing the Polling Place (Precinct Count)

These standards for closing the polling place are specific to precinct count systems.

The system shall provide the means for:

a. Preventing the further casting of ballots once the polling place has closed;

b. Providing an internal test that verifies that the prescribed closing procedure

has been followed, and that the device status is normal;

c. Incorporating a visible indication of system status;

d. Producing a diagnostic test record that verifies the sequence of events, and

indicates that the extraction of voting data has been activated; and

e. Precluding the unauthorized reopening of the polls once the poll closing has

been completed for that election.

2.5.2 Consolidating Vote Data

All systems shall provide a means to consolidate vote data from all polling places, and

optionally from other sources such as absentee ballots, provisional ballots, and voted

ballots requiring human review (e.g., write-in votes).

246
Volume I — Performance Standards

2.5.3 Producing Reports

All systems shall be able to create reports summarizing the data on multiple levels.

2.5.3.1 Common Standards

All systems shall provide capabilities to:

a. Support geographic reporting, which requires the reporting of all results for

each contest at the precinct level and additional jurisdictional levels;

b. Produce a printed report of the number of ballots counted by each tabulator;

c. Produce a printed report for each tabulator of the results of each contest that

includes the votes cast for each selection, the count of undervotes, and the

count of overvotes;

d. Produce a consolidated printed report of the results for each contest of all

votes cast (including the count of ballots from other sources supported by the

system as specified by the vendor) that includes the votes cast for each

selection, the count of undervotes, and the count of overvotes;

e. Be capable of producing a consolidated printed report of the combination of

overvotes for any contest that is selected by an authorized official (e.g.; the

number of overvotes in a given contest combining candidate A and candidate

B, combining candidate A and candidate C, etc.);

f. Produce all system audit information required in Section 4.5 in the form of

printed reports, or in electronic memory for printing centrally; and

g. Prevent data from being altered or destroyed by report generation, or by the

transmission of results over telecommunications lines.

2.5.3.2 Precinct Count Systems

In addition to the common reporting requirements, all precinct count voting systems

shall:

a. Prevent the printing of reports and the unauthorized extraction of data prior to

the official close of the polling place;

b. Provide a means to extract information from a transportable programmable

memory device or data storage medium for vote consolidation;

c. Consolidate the data contained in each unit into a single report for the polling

place when more than one voting machine or precinct tabulator is used; and

2.47
Volume I — Performance Standards

d. Prevent data in transportable memory from being altered or destroyed by

report generation, or by the transmission of results over telecommunications

lines.

2.5.4 Broadcasting Results

Some voting systems offer the capability to make unofficial results available to

external organizations such as the news media, political party officials, and others.

Although this capability is not required, systems that make unofficial results available

shall:

a. Provide only aggregated results, and not data from individual ballots;

b. Provide no access path from unofficial electronic reports or files to the storage

devices for official data; and

c. Clearly indicate on each report or file that the results it contains are unofficial.

2.6 Maintenance, Transportation, and Storage

All systems shall be designed and manufactured to facilitate preventive and corrective

maintenance, conforming to the hardware standards described in Section 3.

All vote casting and tally equipment designated for storage between elections shall:

a. Function without degradation in capabilities after transit to and from the place

of use, as demonstrated by meeting the performance standards described in

Section 3; and

b. Function without degradation in capabilities after storage between elections, as

demonstrated by meeting the performance standards described in Section 3.

248
Volume I — Performance Standards

3 Hardware Standards

3.1 Scope

This section contains the requirements for the machines and manufactured devices that

are part of a voting system. It specifies minimum values for certain performance

characteristics; physical characteristics; and design, construction, and maintenance

characteristics for the hardware and selected related components of all voting systems,

such as:

♦ Ballot printers;

♦ Ballot cards and sheets;

♦ Ballot displays;

♦ Voting devices, including punching and marking devices and DRE recording

devices;

♦ Voting booths and enclosures;

♦ Ballot boxes and ballot transfer boxes;

♦ Ballot readers;

♦ Computers used to prepare ballots, program elections, consolidate and report

votes, and perform other elections management activities;

♦ Electronic ballot recorders;

♦ Electronic precinct vote control units;

♦ Removable electronic data storage media;

♦ Servers; and

♦ Printers.

349
Volume I — Performance Standards

This section applies to the combination of software and hardware to accomplish

specific performance and system control requirements. Standards that are specific to

software alone are provided in Section 4 of the Standards.

3.1.1 Hardware Sources

The requirements of this section apply generally to all hardware used in voting

systems, including:

a. Hardware provided by the voting system vendor and its suppliers;

b. Hardware furnished by an external provider (for example, providers of

commercial off-the-shelf (COTS) machines and devices) where the hardware

may be used in any way during voting system operation; and

c. Hardware provided by the voting jurisdiction.

3.1.2 Organization of this Section

The standards presented in this section are organized as follows:

♦ Performance Requirements: These requirements address the combined

operational capabilities of the voting system's hardware and software across a

broad range of parameters;

♦ Physical Requirements: These requirements address the size, weight and

transportability of the voting system; and

♦ Design, Construction, and Maintenance Requirements: These requirements

address the reliability and durability of materials, product marking, quality of

system workmanship, safety, and other attributes to ensure smooth system

operation in the voting environment.

3.2 Performance Requirements

The performance requirements address a broad range of parameters, encompassing:

a. Accuracy requirements, where requirements are specified for distinct

processing functions of paper-based and DRE systems;

b. Environmental requirements, where no distinction is made between

requirements for paper-based and DRE systems, but requirements for precinct

and central count are described;

3-50
Volume I — Performance Standards

c. Vote data management requirements, where no differentiation is made

between requirements for paper-based and DRE systems;

d. Vote recording requirements, where separate and distinct requirements are

delineated for paper-based and DRE systems;

e. Conversion requirements, which apply only to paper-based systems;

f. Processing requirements, where separate and distinct requirements are

delineated for paper-based and DRE systems; and

g. Reporting requirements, where no distinction is made between requirements

for paper-based and DRE systems, but where differences between precinct and

central count systems are readily apparent based on differences of their

reporting.

The performance requirements include such attributes as ballot reading and handling

requirements; system accuracy; memory stability; and the ability to withstand specified

environmental conditions. These characteristics also encompass system-wide

requirements for shelter, electrical supply, and compatibility with data networks.

Performance requirements for voting systems represent the combined operational

capability of both system hardware and software. Accuracy, as measured by data error

rate, and operational failure are treated as distinct attributes in performance testing. All

systems shall meet the performance requirements under operating conditions and after

storage under non-operating conditions.

3.2.1 Accuracy Requirements

Voting system accuracy addresses the accuracy of data for each of the individual ballot

positions that could be selected by a voter, including the positions that are not selected.

For a voting system, accuracy is defined as the ability of the system to capture, record,

store, consolidate and report the specific selections and absence of selections, made by

the voter for each ballot position without error. Required accuracy is defined in terms

of an error rate that for testing purposes represents the maximum number of errors

allowed while processing a specified volume of data. This rate is set at a sufficiently

stringent level such that the likelihood of voting system errors affecting the outcome of

an election is exceptionally remote even in the closest of elections.

The error rate is defined using a convention that recognizes differences in how vote

data is processed by different types of voting systems. Paper-based and DRE systems

have different processing steps. Some differences also exist between precinct count

and central count systems. Therefore, the acceptable error rate applies separately and

distinctly to each of the following functions:

a. For all paper-based systems:

351
Volume I — Performance Standards

1) Scanning ballot positions on paper ballots to detect selections for

individual candidates and contests;

2) Conversion of selections detected on paper ballots into digital data;

b. For all DRE systems:

1) Recording the voter selections of candidates and contests into voting data

storage; and

2) Independently from voting data storage, recording voter selections of

candidates and contests into ballot image storage.

c. For precinct-count systems (paper-based and DRE):

Consolidation of vote selection data from multiple precinct-based systems to

generate jurisdiction-wide vote counts, including storage and reporting of the

consolidated vote data; and

d. For central-count systems (paper-based and DRE):

Consolidation of vote selection data from multiple counting devices to

generate jurisdiction-wide vote counts, including storage and reporting of the

consolidated vote data.

For testing purposes, the acceptable error rate is defined using two parameters: the

desired error rate to be achieved, and the maximum error rate that should be accepted

by the test process.

For each processing function indicated above, the system shall achieve a target error

rate of no more than one in 10,000,000 ballot positions, with a maximum acceptable

error rate in the test process of one in 500,000 ballot positions.

3.2.2 Environmental Requirements

The environmental requirements for voting systems include shelter, space, furnishings

and fixtures, supplied energy, environmental control , and external telecommunications

services. Environmental conditions applicable to the design and operation of voting

systems consist of the following categories:

♦ Natural environment, including temperature, humidity, and atmospheric

pressure;

♦ Induced environment, including proper and improper operation and handling

of the system and its components during the election processes;

♦ Transportation and storage; and

♦ Electromagnetic signal environment, including exposure to and generation of

radio frequency energy.

352
Volume I — Performance Standards

All voting systems shall be designed to withstand the environmental conditions

contained in the appropriate test procedures of the Standards. These procedures will be

applied to all devices for casting, scanning and counting ballots, except those that

constitute COTS devices that have not been modified in any manner to support their

use as part of a voting system and that have a documented record of performance under

conditions defined in the Standards.

The TDP supplied by the vendor shall include a statement of all requirements and

restrictions regarding environmental protection, electrical service, recommended

auxiliary power, telecommunications service, and any other facility or resource

required for the proper installation and operation of the system.

3.2.2.1 Shelter Requirements

All precinct count systems shall be designed for storage and operation in any enclosed

facility ordinarily used as a warehouse or polling place, with prominent instructions as

to any special storage requirements.

3.2.2.2 Space Requirements

There is no restriction on space allowed for the installation of voting systems, except

that the arrangement of these systems shall not impede performance of their duties by

polling place officials, the orderly flow of voters through the polling place, or the

ability for the voter to vote in private.

3.2.2.3 Furnishings and Fixtures

Any furnishings or fixtures provided as a part of voting systems, and any components

provided by the vendor that are not a part of the system but that are used to support its

storage, transportation, or operation, shall comply with the design and safety

requirements of Subsection 3.4.8.

3.2.2.4 Electrical Supply

Components of voting systems that require an electrical supply shall meet the

following standards:

a. Precinct count systems shall operate with the electrical supply ordinarily

found in polling places (120vac/60hz./1);

3-53
Volume I — Performance Standards

b. Central count systems shall operate with the electrical supply ordinarily

found in central tabulation facilities or computer room facilities

(120vac/60hz/l, 208vac/60hz/3, or 240vac/60hz/2); and

c. All systems shall also be capable of operating for a period of at least 2

hours on backup power, such that no voting data is lost or corrupted, nor

normal operations interrupted. When backup power is exhausted the

system shall retain the contents of all memories intact.

The backup power capability is not required to provide lighting of the voting area.

3.2.2.5 Electrical Power Disturbance

Vote scanning and counting equipment for paper-based systems, and all DRE

equipment, shall be able to withstand, without disruption of normal operation or loss of

data:

a. Surges of 30% dip @10 ms;

b. Surges of 60% dip @100 ms & 1 sec

c. Surges of >95% interrupt @5 sec;

d. Surges of +15% line variations of nominal line voltage; and

e. Electric power increases of 7.5% and reductions of 12.5% of nominal

specified power supply for a period of up to four hours at each power level.

3.2.2.6 Electrical Fast Transient

Vote scanning and counting equipment for paper-based systems, and all DRE

equipment, shall be able to withstand, without disruption of normal operation or loss of

data, electrical fast transients of:

a. 2 kV AC & DC external power lines;

b. +1 kV all external wires >3m no control; and

c. +2 kV all external wires control.

3.2.2.7 Lightning Surge

Vote scanning and counting equipment for paper-based systems, and all DRE

equipment, shall be able to withstand, without disruption of normal operation or loss of

data, surges of:

a. +2 kV AC line to line;

3-54
Volume — Performance Standards

b. +2 kV AC line to earth;

c. +.5 kV DC line to line >10m;

d. +.5 kV DC line to earth >10m; and

e. +1 kV 1/0 sig/control >30m.

3.2.2.8 Electrostatic Disruption

Vote scanning and counting equipment for paper-based systems, and all DRE

equipment, shall be able to withstand ±15 kV air discharge and ±8 kV contact

discharge without damage or loss of data. The equipment may reset or have momentary

interruption so long as normal operation is resumed without human intervention or loss

of data. Loss of data means votes that have been completed and confirmed to the voter.

3.2.2.9 Electromagnetic Radiation

Vote scanning and counting equipment for paper-based systems, and all DRE

equipment, shall comply with the Rules and Regulations of the Federal

Communications Commission, Part 15, Class B requirements for both radiated and

conducted emissions.

3.2.2.10 Electromagnetic Susceptibility

Vote scanning and counting equipment for paper-based systems, and all DRE

equipment, shall be able to withstand an electromagnetic field of 10 V/m modulated by

a 1 kHz 80% AM modulation over the frequency range of 80 MHz to 1000 MHz,

without disruption of normal operation or loss of data.

3.2.2.11 Conducted RF Immunity

Vote scanning and counting equipment for paper-based systems, and all DRE

equipment, shall be able to withstand, without disruption of normal operation or loss of

data, conducted RF energy of:

a. 10V AC & DC power; and

b. 10V, 20 sig/control >3m.

3.55
Volume I — Performance Standards

3.2.2.12 Magnetic Fields Immunity

Vote scanning and counting equipment for paper-based systems, and all DRE

equipment, shall be able to withstand, without disruption of normal operation or loss of

data, AC magnetic fields of 30 A/m at 60 Hz.

3.2.2.13 Environmental Control - Operating Environment

Equipment used for election management activities or vote counting (including both

precinct and central count systems) shall be capable of operation in temperatures

ranging from 50 to 95 degrees Fahrenheit.

3.2.2.14 Environmental Control - Transit and Storage

Equipment used for vote casting, or for counting votes in a precinct count system, shall

meet specific minimum performance standards that simulate exposure to physical

shock and vibration associated with handling and transportation by surface and air

common carriers, and to temperature conditions associated with delivery and storage in

an uncontrolled warehouse environment.

a. High and low storage temperatures ranging from -4 to +140 degrees

Fahrenheit, equivalent to MIL-STD-810D, Methods 501.2 and 502.2,

Procedure I-Storage;

b. Bench handling equivalent to the procedure of MIL-STD-810D, Method 516.3,

Procedure VI;

c. Vibration equivalent to the procedure of MIL-STD-810D, Method 514.3,

Category 1- Basic Transportation, Common Carrier; and

d. Uncontrolled humidity equivalent to the procedure of MIL-STD-810D,

Method 507.2, Procedure I-Natural Hot-Humid.

3.2.2.15 Data Network Requirements

Voting systems may use a local or remote data network. If such a network is used, then

all components of the network shall comply with the telecommunications requirements

described in Section 5 of the Standards and the Security requirements described in

Section 6.

3-56
Volume I — Performance Standards

3.2.3 Election Management System (EMS) Requirements

The EMS requirements address electronic hardware and software used to conduct the

pre-voting functions defined in Section 2 with regard to ballot preparation, election

programming, ballot and program installation, readiness testing, verification at the

polling place, and verification at the central location.

3.2.3.1 Recording Requirements

Voting systems shall accurately record all election management data entered by the

user, including election officials or their designees. For recording accuracy, all systems

shall:

a. Record every entry made by the user;

b. Add permissible voter selections correctly to the memory components of the

device;

c. Verify the correctness of detection of the user selections and the addition of

the selections correctly to memory;

d. Add various forms of data entered directly by the election official or designee,

such as text, line art, logos, and images;

e. Verify the correctness of detection of data entered directly by the user and the

addition of the selections correctly to memory;

f. Preserve the integrity of election management data stored in memory against

corruption by stray electromagnetic emissions, and internally generated

spurious electrical signals; and

g. Log corrected data errors by the system.

3.2.3.2 Memory Stability

Electronic system memory devices, used to retain election management data, shall

have demonstrated error-free data retention for a period of 22 months.

3.2.4 Vote Recording Requirements

The vote recording requirements address the enclosure, equipment, and supplies used

by voters to vote.

3-57
Volume I - Performance Standards

3.2.4.1 Common Standards

All systems shall provide voting booths or enclosures for poll site use. Such booths or

enclosures may be integral to the voting system or supplied as components of the

voting system, and shall:

a. Be integral to, or makes provision for, the installation of, the voting device;

b. Ensure by its structure stability against movement or overturning during entry,

occupancy, and exit by the voter;

c. Provide privacy for the voter, and be designed in such a way as to prevent

observation of the ballot by any person other than the voter; and

d. Be capable of meeting the accessibility requirements of Section 2.2.7.1.

3.2.4.2 Paper-Based Recording Standards

The paper-based recording requirements govern:

♦ Ballot cards or sheets, and pages or assemblies of pages containing ballot field

identification data;

♦ Punching devices;

♦ Marking devices;

♦ Frames or fixtures to hold the ballot while it is being punched;

♦ Compartments or booths where voters record selections; and

♦ Secure containers for the collection of voted ballots.

3.2.4.2.1 Paper Ballot Standards

Paper ballots used by paper-based voting systems shall meet the following standards:

a. Punches or marks that identify the unique ballot format, in accordance with

Section 2.3.1.1.1.c., shall be outside the area in which votes are recorded, so as

to minimize the likelihood that these punches or marks will be mistaken for

vote responses and the likelihood that recorded votes will obliterate these

punches or marks;

b. If printed or punched alignment marks are used to locate the vote response

fields on the ballot, these marks shall be outside the area in which votes are

recorded, so as to minimize the likelihood that these marks will be mistaken

for vote responses and the likelihood that recorded votes will obliterate these

marks; and

3-58
Volume I — Performance Standards

c. The TDP shall specify the required paper stock, size, shape, opacity, color,

watermarks, field layout, orientation, size and style of printing, size and

location of punch or mark fields used for vote response fields and to identify

unique ballot formats, placement of alignment marks, ink for printing, and

folding and bleed-through limitations for preparation of ballots that are

compatible with the system.

3.2.4.2.2 Punching Devices

Punching devices used by voting systems shall:

a. Be suitable for the type of ballot card specified;

b. Facilitate the clear and accurate recording of each vote intended by the voter;

c. Be designed to avoid excessive damage to vote recorder components; and

d. Incorporate features to ensure that the chad (debris) is completely removed,

without damage to other parts of the ballot card.

3.2.4.2.3 Marking Devices

The TDP shall specify marking devices (such as pens or pencils) that, if used to make

the prescribed form of mark, produce readable marked ballots such that the system

meets the performance requirements for accuracy specified previously. These

specifications shall identify:

a. Specific characteristics of marking devices that affect readability of marked

ballots;

b. Performance capabilities with regard to each characteristic; and

c. For marking devices manufactured by multiple external sources, a listing of

sources and model numbers that are compatible with the system.

3.2.4.2.4 Frames or Fixtures for Punchcard Ballots

The frame or fixture for punchcards shall:

a. Hold the ballot card securely in its proper location and orientation for voting;

b. When contests are not printed directly on the ballot card or sheet, incorporate

an assembly of ballot label pages that identify the offices and issues

corresponding to the proper ballot format for the polling place where it is used

and that are aligned with the voting fields assigned to them; and

c. Incorporate a template to preclude perforation of the card except in the

specified voting fields; a mask to allow punches only in fields designated by

the format of the ballot; and a backing plate for the capture and removal of

3.59
Volume I — Performance Standards

chad. This requirement may be satisfied by equipment of a different design as

long it achieves the same result as the Standards with regard to:

1) Positioning the card;

2) Association of ballot label information with corresponding punch fields;

3) Enabling of only those voting fields that correspond to the format of the

ballot; and

4) Punching the fields and the positive removal of chad.

3.2.4.2.5 Frames or Fixtures for Printed Ballots

A frame or fixture for printed ballot cards is optional. However, if such a device is

provided, it shall:

a. Be of any size and shape consistent with its intended use;

b. Position the card properly;

c. Hold the ballot card securely in its proper location and orientation for voting;

and

d. Comply with the requirements for design and construction contained in

Section 3.4.

3.2.4.2.6 Ballot Boxes and Ballot Transfer Boxes

Ballot boxes and ballot transfer boxes, which serve as secure containers for the storage

and transportation of voted ballots, shall:

a. Be of any size, shape, and weight commensurate with their intended use;

b. Incorporate locks or seals, the specifications of which are described in the

system documentation;

c. Provide specific points where ballots are inserted, with all other points on the

box constructed in a manner that prevents ballot insertion; and

d. For precinct count systems, contain separate compartments for the segregation

of unread ballots, ballots containing write-in votes, or any irregularities that

may require special handling or processing. In lieu of compartments, the

conversion processing may mark such ballots with an identifying spot or stripe

to facilitate manual segregation.

3.2.4.3 DRE Systems Recording Requirements

The DRE systems recording requirements address the detection and recording of votes,

including the logic and data processing functions required to determine the validity of

3.60
Volume I — Performance Standards

voter selections, to accept and record valid selections, and to reject invalid selections.

The requirements also address the physical environment in which ballots are cast.

3.2.4.3.1 Activity Indicator

DRE systems shall include an audible or visible activity indicator providing the status

of each voting device. This indicator shall:

a. Indicate whether the device has been activated for voting; and

b. Indicate whether the device is in use.

3.2.4.3.2 DRE System Vote Recording

To ensure vote recording accuracy and integrity while protecting the anonymity of the

voter, all DRE systems shall:

a. Contain all mechanical, electromechanical, and electronic components; software;

and controls required to detect and record the activation of selections made by the

voter in the process of voting and casting a ballot;

b. Incorporate redundant memories to detect and allow correction of errors caused by

the failure of any of the individual memories;

c. Provide at least two processes that record the voter's selections that:

1) To the extent possible, are isolated from each other;

2) Designate one process and associated storage location as the main vote

detection, interpretation, processing and reporting path; and

Use a different process to store ballot images, for which the method of recording may

include any appropriate encoding or data compression procedure consistent with the

regeneration of an unequivocal record of the ballot as cast by the voter.

d. Provide a capability to retrieve ballot images in a form readable by humans; and

e. Ensure that all processing and storage protects the anonymity of the voter.

3.2.4.3.3 Recording Accuracy

DRE systems shall meet the following requirements for recording accurately each vote

and ballot cast:

a. Detect every selection made by the voter;

b. Correctly add permissible selections to the memory components of the device;

c. Verify the correctness of the detection of the voter selections and the addition

of the selections to memory;

3.61
Volume I — Performance Standards

d. Achieve an error rate not to exceed the requirement indicated in Section 3.2.1;

e. Preserve the integrity of voting data and ballot images (for DRE machines)

stored in memory for the official vote count and audit trail purposes against

corruption by stray electromagnetic emissions, and internally generated

spurious electrical signals; and

f. Maintain a log of corrected data.

3.2.4.3.4 Recording Reliability

Recording reliability refers to the ability of the DRE system to record votes accurately

at its maximum rated processing volume for a specified period of time. The DRE

system shall record votes reliably in accordance with the requirements of Section

3.4.3.

3.2.5 Paper-based Conversion Requirements

The paper-based conversion requirements address the ability of the system to read the

ballot card and to translate its pattern of punches or marks into electronic signals for

later processing. These capabilities may be built into the voting system in an integrated

fashion, or may be provided by one or more components that are not unique to the

system, such as a general-purpose data processing card reader or read head suitably

interfaced to the system. These requirements address two major functions: ballot

handling and ballot reading.

3.2.5.1 Ballot Handling

Ballot handling consists of a ballot card's acceptance, movement through the read

station, and transfer into a collection station or receptacle.

3.2.5.1.1 Capacity (Central Count)

The capacity to convert the punches or marks on individual ballots into signals is

uniquely important to central count systems. The capacity for a central count system

shall be documented by the vendor. This documentation shall include the capacity for

individual components that impact the overall capacity.

3.2.5.1.2 Exception Handling (Central Count)

This requirement refers to the handling of ballots for a central count system when they

are unreadable or when some condition is detected requiring that the cards be

3.62
Volume I — Performance Standards

segregated from normally processed ballots for human review. In response to an

unreadable ballot or a write-in vote all central count paper-based systems shall:

a. Outstack the ballot, or

b. Stop the ballot reader and display a message prompting the election official or

designee to remove the ballot, or

c. Mark the ballot with an identifying mark to facilitate its later identification.

Additionally, the system shall provide a capability that can be activated by an

authorized election official to identify ballots containing overvotes, blank ballots, and

ballots containing undervotes in a designated race. If enabled, these capabilities shall

perform one of the above actions in response to the indicated condition.

3.2.5.1.3 Exception Handling (Precinct Count)

This requirement refers to the handling of ballots for a precinct count system when

they are unreadable or when some condition is detected requiring that the cards be

segregated from normally processed ballots for human review. All paper based

precinct count systems shall:

a. In response to an unreadable or blank ballot, return the ballot and provide a

message prompting the voter to examine the ballot;

b. In response to a ballot with a write-in vote, segregate the ballot or mark the ballot

with an identifying mark to facilitate its later identification;

c. In response to a ballot with an overvote the system shall:

1) Provide a capability to identify an overvoted ballot;

2) Return the ballot;

3) Provide an indication prompting the voter to examine the ballot;

4) Allow the voter to submit the ballot with the overvote; and

5) Provide a means for an authorized election official to deactivate this

capability entirely and by contest; and

d. In response to a ballot with an undervote the system shall:

1) Provide a capability to identify an undervoted ballot;

2) Return the ballot;

3) Provide an indication prompting the voter to examine the ballot;

4) Allow the voter to submit the ballot with the undervote; and

5) Provide a means for an authorized election official to deactivate this

capability.

3.63
Volume I — Performance Standards

3.2.5.1.4 Multiple Feed Prevention

Multiple feed refers to the situation arising when a ballot reader attempts to read more

than one ballot at a time. The requirements govern the ability of a ballot reader to

prevent multiple feed or to detect and provide an alarm indicating multiple feed.

a. If multiple feed is detected, the card reader shall halt in a manner that permits

the operator to remove the unread cards causing the error, and reinsert them in

the card input hopper.

b. The frequency of multiple feeds with ballots intended for use with the system

shall not exceed 1 in 10,000.

3.2.5.2 Ballot Reading Accuracy

This paper-based system requirement governs the conversion of the physical ballot into

electronic data. Reading accuracy for ballot conversion refers to the ability to:

♦ Recognize vote punches or marks, or the absence thereof, for each possible

selection on the ballot;

♦ Discriminate between valid punches or marks and extraneous perforations,

smudges, and folds; and

♦ Convert the vote punches or marks, or the absence thereof, for each possible

selection on the ballot into digital signals.

To ensure accuracy, paper-based systems shall:

a. Detect punches or marks that conform to vendor specifications with an error

rate not exceeding the requirement indicated in Section 3.2.1;

b. Ignore, and not record, extraneous perforations, smudges, and folds; and

c. Reject ballots that meet all vendor specifications at a rate not to exceed 2

percent.

3.2.6 Processing Requirements

Processing requirements apply to the hardware and software required to accumulate

voting data for all candidates and measures within voting machines and polling places,

and to consolidate the voting data at a central level or multiple levels. These

requirements also address the generation and maintenance of audit records, the

detection and disabling of improper use or operation of the system, and the monitoring

of overall system status. Separate and distinct requirements for paper-based and DRE

voting systems are presented below.

3.64
Volume I – Performance Standards

3.2.6.1 Paper-Based System Processing Requirements

The paper-based processing requirements address all mechanical devices,

electromechanical devices, electronic devices, and software required to perform the

logical and numerical functions of interpreting the electronic image of the voted ballot,

and assigning votes to the proper memory registers.

3.2.6.1.1 Processing Accuracy

Processing accuracy refers to the ability of the system to receive electronic signals

produced by punches for punchcard systems and vote marks and timing information

for marksense systems; perform logical and numerical operations upon these data; and

reproduce the contents of memory when required, without error. Specific requirements

are detailed below:

a. Processing accuracy shall be measured by vote selection error rate, the ratio of

uncorrected vote selection errors to the total number of ballot positions that

could be recorded across all ballots when the system is operated at its nominal

or design rate of processing;

b. The vote selection error rate shall include data that denotes ballot style or

precinct as well as data denoting a vote in a specific contest or ballot

proposition;

c. The vote selection error rate shall include all errors from any source; and

d. The vote selection error rate shall not exceed the requirement indicated in

Section 3.2.1.

3.2.6.1.2 Memory Stability

Paper-based system memory devices, used to retain control programs and data, shall

have demonstrated error-free data retention for a period of 22 months, under the

environmental conditions for operation and non-operation (i.e. storage).

3.2.6.2 DRE System Processing Requirements

The DRE system processing requirements address all mechanical devices,

electromechanical devices, electronic devices, and software required to process voting

data after the polling places are closed.

3.2.6.2.1 Processing Speed

DRE voting systems shall meet the following requirements for processing speed:

3-65
Volume I – Performance Standards

a. Operate at a speed sufficient to respond to any operator and voter input

without perceptible delay (no more than three seconds); and

b. If the consolidation of polling place data is done locally, perform this

consolidation in a time not to exceed five minutes for each device in the

polling place.

3.2.6.2.2 Processing Accuracy

Processing accuracy is defined as the ability of the system to process voting data stored

in DRE voting devices, or in removable memory modules installed in such devices.

Processing includes all operations to consolidate voting data after the polling places

have been closed. DRE voting systems shall:

a. Produce reports that are completely consistent, with no discrepancy among

reports of voting device data produced at any level; and

b. Produce consolidated reports containing absentee, provisional, or other voting

data that are similarly error-free. Any discrepancy, regardless of source, is

resolvable to a procedural error, to the failure of a non-memory device, or to

an external cause.

3.2.6.2.3 Memory Stability

DRE system memory devices used to retain control programs and data shall have

demonstrated error-free data retention for a period of 22 months. Error-free retention

may be achieved by the use of redundant memory elements, provided that the

capability for conflict resolution or correction among elements is included.

3.2.7 Reporting Requirements

The reporting requirements govern all mechanical, electromechanical, and electronic

devices required for voting systems to print audit record entries and results of the

tabulation. These requirements also address data storage media for transportation of

data to other sites.

3.2.7.1 Removable Storage Media

In voting systems that use storage media that can be removed from the system and

transported to another location for readout and report generation, these media shall use

devices with demonstrated error-free retention for a period of 22 months under the

environmental conditions for operation and non-operation contained in Section 3.2.2.

Examples of removable storage media include: programmable read-only memory

(PROM), random access memory (RAM) with battery backup, magnetic media, or

optical media.

3-66
Volume I — Performance Standards

3.2.7.2 Printers

All printers used to produce reports of the vote count shall be capable of producing:

a. Alphanumeric headers;

b. Election, office and issue labels; and

c. Alphanumeric entries generated as part of the audit record.

3.2.8 Vote Data Management Requirements

The vote data management requirements for all systems address capabilities that

manage, process, and report voting data after the data has been consolidated at the

polling place or other intermediate levels. These capabilities allow the system to:

a. Consolidate voting data from polling place data memory or transfer devices;

b. Report polling place summaries; and

c. Process absentee ballots, data entered manually, and administrative ballot

definition data.

The requirements address all hardware and software required to generate output

reports in the various formats required by the using jurisdiction.

3.2.8.1 Data File Management

All voting systems shall provide the capability to:

a. Integrate voting data files with ballot definition files;

b. Verify file compatibility; and

c. Edit and update files as required.

3.2.8.2 Data Report Generation

All voting systems shall include report generators for producing output reports at the

device, polling place, and summary level, with provisions for administrative and

judicial subdivisions as required by the using jurisdiction.

3.67
Volume I — Performance Standards

3.3 Physical Characteristics

This section covers physical characteristics of all voting systems and components that

affect their general utility and suitability for election operations.

3.3.1 Size

There is no numerical limitation on the size of any voting system equipment, but the

size of each device should be compatible with its intended use and the location at

which the equipment is to be used.

3.3.2 Weight

There is no numerical limitation on the weight of any voting system equipment, but the

weight of each device should be compatible with its intended use and the location at

which the equipment is to be used.

3.3.3 Transport and Storage of Precinct Systems

All precinct systems shall:

a. Provide a means to safely and easily handle, transport, and install polling place

equipment, such as wheels or a handle or handles; and

b. Be capable of using, or be provided with, a protective enclosure rendering the

equipment capable of withstanding:

1) Impact, shock and vibration loads accompanying surface and air

transportation; and

2) Stacking loads accompanying storage.

3.68
Volume I — Performance Standards

3.4 Design, Construction, and Maintenance

Characteristics

This section covers voting system materials, construction workmanship, and specific

design characteristics important to the successful operation and efficient maintenance

of the system.

3.4.1 Materials, Processes, and Parts

The approach to system design is unrestricted, and may incorporate any form or variant

of technology capable of meeting the voting systems requirements and standards.

Precinct count systems shall be designed in accordance with best commercial practice

for microcomputers, process controllers, and their peripheral components. Central

count voting systems and equipment used in a central tabulating environment shall be

designed in accordance with best commercial and industrial practice.

All voting systems shall:

a. Be designed and constructed so that the frequency of equipment malfunctions

and maintenance requirements are reduced to the lowest level consistent with

cost constraints;

b. Include, as part of the accompanying TDP, an approved parts list; and

c. Exclude parts or components not included in the approved parts list.

3.4.2 Durability

All voting systems shall be designed to withstand normal use without deterioration and

without excessive maintenance cost for a period of ten years.

3.4.3 Reliability

The reliability of voting system devices shall be measured as mean time between

Failure (MTBF) for the system submitted for testing. MBTF is defined as the value of

the ratio of operating time to the number of failures which have occurred in the

specified time interval. A typical system operations scenario consist of approximately

45 hours of equipment operation, consisting of 30 hours of equipment set-up and

369
Volume I — Performance Standards

readiness testing and 15 hours of elections operations. For the purpose of

demonstrating compliance with this requirement, a failure is defined as any event

which results in either the:

a. Loss of one or more functions; or

b. Degradation of performance such that the device is unable to perform its

intended function for longer than 10 seconds.

The MTBF demonstrated during qualification testing shall be at least 163 hours.

3.4.4 Maintainability

Maintainability represents the ease with which maintenance actions can be performed

based on the design characteristics of equipment and software and the processes the

vendor and election officials have in place for preventing failures and for reacting to

failures. Maintainability includes the ability of equipment and software to self-diagnose

problems and make non-technical election workers aware of a problem.

Maintainability addresses all scheduled and unscheduled events, which are performed

to:

• Determine the operational status of the system or a component;

• Adjust, align, tune, or service components;

• Repair or replace a component having a specified operating life or replacement

interval;

• Repair or replace a component that exhibits an undesirable predetermined

physical condition or performance degradation;

• Repair or replace a component that has failed; and

• Verify the restoration of a component, or the system, to operational status.

Maintainability shall be determined based on the presence of specific physical

attributes that aid system maintenance activities, and the ease with which system

maintenance tasks can be performed by the ITA. Although a more quantitative basis

for assessing maintainability, such as the mean to repair the system is desirable, the

qualification of a system is conducted before it is approved for sale and thus before a

broader base of maintenance experience can be obtained.

3.4.4.1 Physical Attributes

The following physical attributes will be examined to assess reliability:

a. Presence of labels and the identification of test points;

b. Provision of built-in test and diagnostic circuitry or physical indicators of

condition;

3-70
Volume I – Performance Standards

c. Presence of labels and alarms related to failures; and

d. Presence of features that allow non-technicians to perform routine

maintenance tasks (such as update of the system database).

3.4.4.2 Additional Attributes

The following additional attributes will be considered to assess system maintainability.

a. Ease of detecting that equipment has failed by a non-technician;

b. Ease of diagnosing problems by a trained technician;

c. Low false alarm rates (i.e., indications of problems that do not exist);

d. Ease of access to components for replacement;

e. Ease with which adjustment and alignment can be performed;

f. Ease with which database updates can be performed by a non-technician; and

g. Adjust, align, tune, or service components.

3.4.5 Availability

The availability of a voting system is defined as the probability that the equipment

(and supporting software) needed to perform designated voting functions will respond

to operational commands and accomplish the function. The voting system shall meet

the availability standard for each of the following voting functions:

a. For all paper-based systems:

1) Recording voter selections (such as by ballot marking or punch); and

2) Scanning the punches or marks on paper ballots and converting them into

digital data;

b. For all DRE systems, recording and storing the voter's ballot selections.

c. For precinct-count systems (paper-based and DRE), consolidation of vote selection

data from multiple precinct-based systems to generate jurisdiction-wide vote

counts, including storage and reporting of the consolidated vote data; and

d. For central-count systems (paper-based and DRE), consolidation of vote selection

data from multiple counting devices to generate jurisdiction-wide vote counts,

including storage and reporting of the consolidated vote data.

System availability is measured as the ratio of the time during which the system is

operational a (up time) to the total time period of operation (up time plus down time).

3-71
Volume I — Performance Standards

Inherent availability (Ai) is a the fraction of time a system is functional, based upon

Mean Time Between Failure (MTBF) and Mean Time to Repair (MTTR), that is:

Ai = (MTBF)/(MTBF + MTTR)

Mean Time to Repair (MTTR) is the average time required to perform a corrective

maintenance task during periods of system operation. Corrective maintenance task

time is active repair time, plus the time attributable to other factors that could lead to

logistic or administrative delays, such as travel notification of qualified maintenance

personnel and travel time for such personnel to arrive at the appropriate site.

Corrective maintenance may consist of substitution of the complete device or one of its

components, as in the case of precinct count and some central count systems, or it may

consist of on-site repair.

The voting system shall achieve at least ninety nine percent availability during normal

operation for the functions indicated above. This standard encompasses for each

function the combination of all devices and components that support the function,

including their MTTR and MTBF attribute.

Vendors shall specify the typical system configuration that is to be used to assess

availability, and any assumptions made with regard to any parameters that impact the

MTTR. These factors shall include at a minimum:

a. Recommended number and locations of spare devices or components to be

kept on hand for repair purposes during periods of system operation;

b. Recommended number and locations of qualified maintenance personnel who

need to be available to support repair calls during system operation; and

c. Organizational affiliation (i.e., jurisdiction, vendor) of qualified maintenance

personnel.

3.4.6 Product Marking

All voting systems shall:

a. Identify all devices by means of a permanently affixed nameplate or label

containing the name of the manufacturer or vendor, the name of the device, its

part or model number, its revision letter, its serial number, and if applicable,

its power requirements;

b. Display on each device a separate data plate containing a schedule for and list

of operations required to service or to perform preventive maintenance; and

c. Display advisory caution and warning instructions to ensure safe operation of

the equipment and to avoid exposure to hazardous electrical voltages and

moving parts at all locations where operation or exposure may occur.

372
Volume I — Performance Standards

3.4.7 Workmanship

To help ensure proper workmanship, all manufacturers of voting systems shall:

a. Adopt and adhere to practices and procedures to ensure that their products are

free from damage or defect that could make them unsatisfactory for their

intended purpose; and

b. Ensure that components provided by external suppliers are free from damage

or defect that could make them unsatisfactory for their intended purpose.

3.4.8 Safety

All voting systems shall meet the following requirements for safety:

a. All voting systems and their components shall be designed so as to eliminate

hazards to personnel, or to the equipment itself;

b. Defects in design and construction that can result in personal injury or

equipment damage must be detected and corrected before voting systems and

components are placed into service; and

c. Equipment design for personnel safety shall be equal to or better than the

appropriate requirements of the Occupational Safety and Health Act (OSHA),

as identified in Title 29, part 1910, of the Code of Federal Regulations.

3.4.9 Human Engineering—Controls and Displays

All voting systems and components shall be designed and constructed so as to simplify

and facilitate the functions required, and to eliminate the likelihood of erroneous

stimuli and responses on the part of the voter or operator. Other specific requirements

for controls and displays are described below. In addition, specific functional

requirements for system use by voters with disabilities are described in Section 2.2.7

of the Standards. Appendix C provides additional advisory guidance on the

application of human engineering principles to the interface between the voter and the

voting system.

All voting systems shall meet the following requirements for controls and displays:

a. In all systems, controls used by the voter or equipment operator shall be

conveniently located, shall use designs that are consistent with their functions,

and shall be clearly labeled. Instruction plates shall be provided, if they are

necessary to avoid ambiguity or incorrect actuation;

3.73
Volume I — Performance Standards

b. Information or data displays shall be large enough to be readable by voters and

operators with no disabilities and by voters with disabilities consistent with the

requirements defined is Section 2.2.7 of the Standards;

c. Status displays shall meet the same requirements as data displays, and they

shall also follow conventional industrial practice with respect to color:

1) Green, blue, or white displays shall be used for indications of normal

status;

2) Amber indicators shall be used to indicate warnings or marginal status;

and

3) Red indicators shall be used to indicate error conditions or equipment

states that may result in damage, or in hazards to personnel; and unless the

equipment is designed to halt under conditions of incipient damage or

hazard, an audible alarm shall also be provided.

d. Color coding shall be selected so as to assure correct perception by voters and

operators with color blindness; and shall not be used as the only means of

conveying information, indicating an action, prompting a response, or

distinguishing a visual element (see Appendix B for suggested references); and

e. The system's display shall not use flashing or blinking text objects, or other

elements having a flash or blink frequency, greater than 2 Hz and lower than

55 Hz.

3.74
Volume I — Performance Standards

4 Software Standards

4.1 Scope

This section describes essential design and performance characteristics of the software

used in voting systems, addressing both system-level software, such as operating

systems, and voting system application software, including firmware. The

requirements of this section are intended to ensure that voting system software is

reliable, robust, testable, and maintainable. The standards in this section also support

system accuracy, logical correctness, privacy, security and integrity.

The general requirements of this section apply to software used to support the entire

range of voting system activities described in Section 2. More specific requirements

are defined for ballot counting, vote processing, creating an audit trail, and generating

output reports and files. Although this section emphasizes software, the standards

described also influence hardware design considerations.

This section recognizes that there is no best way to design software. Many

programming languages are available for which modern programming practices are

applicable, such as the use of rigorous program and data structures, data typing, and

naming conventions. Other programming languages exist for which such practices are

not easily applied.

The Standards are intended to guide the design of software written in any of the

programming languages commonly used for mainframe, mini-computer, and

microprocessor systems. They are not intended to preclude the use of other languages

or environments, such as those that exhibit "declarative" structure, "object-oriented"

languages, "functional" programming languages, or any other combination of language

and implementation that provides appropriate levels of performance, testability,

reliability, and security. The vendor makes specific software selections. However, the

use of widely recognized and proven software design methods will facilitate the

analysis and testing of voting system software in the qualification process.

4-75
Volume I — Performance Standards

4.1.1 Software Sources

The requirements of this section apply generally to all software used in voting systems,

including:

♦ Software provided by the voting system vendor and its component suppliers;

♦ Software furnished by an external provider (for example, providers of COTS

operating systems and web browsers) where the software may be used in any

way during voting system operation; and

♦ Software developed by the voting jurisdiction.

Compliance with the requirements of the software standards is assessed by several

formal tests, including code examination. Unmodified software is not subject to code

examination; however, source code generated by a package and embedded in software

modules for compilation or interpretation shall be provided in human readable form to

the ITA. The ITA may inspect source code units to determine testing requirements or

to verify that the code is unmodified and that the default configuration options have

not been changed.

Configuration of software, both operating systems and applications, is critical to

proper system functioning. Correct test design and sufficient test execution must

account for the intended and proper configuration of all system components.

Therefore, the vendors shall submit to the ITA, in the TDP, a record of all user

selections made during software installation. The vendor shall also submit a record of

all configuration changes made to the software following its installation. The ITA shall

confirm the propriety and correctness of these user selections and configuration

changes.

4.1.2 Location and Control of Software and Hardware on

Which it Operates

The requirements of this section apply to all software used in any manner to support

any voting-related activities, regardless of the ownership of the software or the

ownership and location of the hardware on which the software is installed or operates.

These requirements apply to:

♦ Software that operates on voting devices and vote counting devices installed at

polling places under the control of the voting jurisdiction;

♦ Software that operates on ballot printers, vote counting devices, and other

hardware typically installed at central or precinct locations (including

contractor facilities); and

♦ Election management software.

4.76
Volume I — Performance Standards

However, some requirements apply only in specific situations indicated in this section.

In addition to the requirements of this section, all software used in any manner to

support any voting-related activities shall meet the requirements for security described

in Section 6 of the Standards.

4.1.3 Exclusions

Some voting systems use equipment, such as personal computers, that may be used for

other purposes and have resident on the equipment general purpose software such as

operating systems, programming language compilers, database management systems,

and Web browsers. Such software is governed by the Standards unless:

♦ The software provides no support of voting system capabilities;

♦ The software is removable, disconnectable, or switchable such that it cannot

function while voting system functions are enabled; and

♦ Procedures are provided that confirm that the software has been removed,

disconnected, or switched.

4.2 Software Design and Coding Standards

The software used by voting systems is selected by the vendor and not prescribed by

the Standards. This section provides standards for voting system software with regard

to:

♦ Selection of programming languages;

♦ Software integrity;

♦ Software modularity and programming;

♦ Control constructs;

♦ Naming conventions;

♦ Coding conventions; and

♦ Comment conventions.

4.77
Volume I — Performance Standards

4.2.1 Selection of Programming Languages

Software associated with the logical and numerical operations of vote data shall use a

high-level programming language, such as: Pascal, Visual Basic, Java, C and C++. The

requirement for the use of high-level language for logical operations does not preclude

the use of assembly language for hardware-related segments, such as device controllers

and handler programs. Also, operating system software may be designed in assembly

language.

4.2.2 Software Integrity

Self-modifying, dynamically loaded, or interpreted code is prohibited, except under the

security provisions outlined in section 6.4.e. This prohibition is to ensure that the

software tested and approved during the qualification process remains unchanged and

retains its integrity. External modification of code during execution shall be prohibited.

Where the development environment (programming language and development tools)

includes the following features, the software shall provide controls to prevent

accidental or deliberate attempts to replace executable code:

♦ Unbounded arrays or strings (includes buffers used to move data);

♦ Pointer variables; and

♦ Dynamic memory allocation and management.

4.2.3 Software Modularity and Programming

Voting system application software, including COTS software, shall be designed in a

modular fashion. However, COTS software is not required to be inspected for

compliance with this requirement.. For the purpose of this requirement', "modules"

may be compiled or interpreted independently. Modules may also be nested. The

modularity rules described here apply to the component sub modules of a library. The

principle concept is that the module contains all the elements to compile or interpret

successfully and has limited access to data in other modules. The design concept is

simple replacement with another module whose interfaces match the original module.

A module is designed in accordance with the following rules:

1 Some software languages and development environments use a different definition of module

but this principle still applies.

4.78
Volume I — Performance Standards

a. Each module shall have a specific function that can be tested and verified

independently of the remainder of the code. In practice, some additional

modules (such as library modules) may be needed to compile the module under

test, but the modular construction allows the supporting modules to be

replaced by special test versions that support test objectives;

b. Each module shall be uniquely and mnemonically named, using names that

differ by more than a single character. In addition to the unique name, the

modules shall include a set of header comments identifying the module's

purpose, design, conditions, and version history, followed by the operational

code. Headers are optional for modules of fewer than ten executable lines

where the subject module is embedded in a larger module that has a header

containing the header information. Library modules shall also have a header

comment describing the purpose of the library and version information;

c. All required resources, such as data accessed by the module, should either be

contained within the module or explicitly identified as input or output to the

module. Within the constraints of the programming language, such resources

shall be placed at the lowest level where shared access is needed. If that

shared access level is across multiple modules, the definitions should be

defined in a single file (called header files in some languages, such as C)

where any changes can be applied once and the change automatically applies

to all modules upon compilation or activation;

d. A module is small enough to be easy to follow and understand. Program logic

visible on a single page is easy to follow and correct. Volume II, Section 5

provides testing guidelines for the ITA to identify large modules subject to

review under this requirement;

e. Each module shall have a single entry point, and a single exit point, for normal

process flow. For library modules or languages such as the object-oriented

languages, the entry point is to the individual contained module or method

invoked. The single exit point is the point where control is returned. At that

point, the data that is expected as output must be appropriately set. The

exception for the exit point is where a problem is so severe that execution

cannot be resumed. In this case, the design must explicitly protect all

recorded votes and audit log information and must implement formal

exception handlers provided by the language; and

f. Process flow within the modules shall be restricted to combinations of the

control structures defined in Volume II, Section 5. These structures support

the modular concept, especially the single entry/exit rule above. They apply to

any language feature where program control passes from one activity to the

next, such as control scripts, object methods, or sets of executable statements,

even though the language itself is not procedural.

4.79
Volume I — Performance Standards

4.2.4 Control Constructs

Voting system software shall use the control constructs identified in Volume II,

Section 5:

a. Acceptable constructs are Sequence, If-Then-Else, Do-While, Do-Until, Case,

and the General loop (including the special case for loop);

b. If the programming language used does not provide these control constructs,

the vendor shall provide them (that is, comparable control structure logic). The

constructs shall be used consistently throughout the code. No other constructs

shall be used to control program logic and execution;

c. While some programming languages do not create programs as linear

processes, stepping from an initial condition, through changes, to a conclusion,

the program components nonetheless contain procedures (such as "methods"

in object-oriented languages). Even in these programming languages, the

procedures must execute through these control constructs (or their equivalents,

as defined and provided by the vendor); and

d. Operator intervention or logic that evaluates received or stored data shall not

re-direct program control within a program routine. Program control may be

re-directed within a routine by calling subroutines, procedures, and functions,

and by interrupt service routines and exception handlers (due to abnormal

error conditions). Do-While (False) constructs and intentional exceptions

(used as GoTos) are prohibited.

4.2.5 Naming Conventions

Voting system software shall use the following naming conventions:

a. Object, function, procedure, and variable names shall be chosen so as to

enhance the readability and intelligibility of the program. Insofar as possible,

names shall be selected so that their parts of speech represent their use, such as

nouns to represent objects, verbs to represent functions, etc.;

b. Names used in code and in documentation shall be consistent;

c. Names shall be unique within an application. Names shall differ by more than

a single character. All single-character names are forbidden except those for

variables used as loop indexes. In large systems where subsystems tend to be

developed independently, duplicate names may be used where the scope of the

name is unique within the application. Names should always be unique where

modules are shared; and

4.80

iF
Volume I — Performance Standards

d. Language keywords shall not be used as names of objects, functions,

procedures, variables, or in any manner not consistent with the design of the

language.

4.2.6 Coding Conventions

Voting system software shall adhere to basic coding conventions. The coding

conventions used shall meet one of the following conditions:

a. The vendors shall identify the published, reviewed, and industry-accepted

coding conventions used and the ITAs shall test for compliance; or

b. The ITAs shall evaluate the code using the coding convention requirements

specified in Volume II, Section 5.

These standards reference conventions that protect the integrity and security of the

code, which may be language-specific, and language-independent conventions that

significantly contribute to readability and maintainability. Specific style conventions

that support economical testing are not binding unless adopted by the vendor.

4.2.7 Comment Conventions

Voting system software shall use the following comment conventions:

a. All modules shall contain headers. For small modules of 10 lines or less, the

header may be limited to identification of unit and revision information. Other

header information should be included in the small unit headers if not clear from

the actual lines of code. Header comments shall provide the following information:

1) The purpose of the unit and how it works;

2) Other units called and the calling sequence;

3) A description of input parameters and outputs;

4) File references by name and method of access (read, write, modify ,

append, etc.);

5) Global variables used; and

6) dDate of creation and a revision record;

b. Descriptive comments shall be provided to identify objects and data types. All

variables shall have comments at the point of declaration clearly explaining

their use. Where multiple variables that share the same meaning are required,

the variables may share the same comment;

4-81
Volume I – Performance Standards

c. In-line comments shall be provided to facilitate interpretation of functional

operations, tests, and branching;

d. Assembly code shall contain descriptive and informative commentssuch that

its executable lines can be clearly understood; and

e. All comments shall be formatted in a uniform manner that makes it easy to

distinguish them from executable code.

4.3 Data and Document Retention

All systems shall:

a. Maintain the integrity of voting and audit data during an election, and for at

least 22 months thereafter, a time sufficient in which to resolve most contested

elections and support other activities related to the reconstruction and

investigation of a contested election; and

b. Protect against the failure of any data input or storage device at a location

controlled by the jurisdiction or its contractors, and against any attempt at

improper data entry or retrieval.

4.4 Audit Record Data

Audit trails are essential to ensure the integrity of a voting system. Operational

requirements for audit trails are described in Section 2.2.5.2 of the Standards. Audit

record data are generated by these procedures. The audit record data in the following

subsections are essential to the complete recording of election operations and reporting

of the vote tally. This list of audit records may not reflect the design constructs of

some systems. Therefore, vendors shall supplement it with information relevant to the

operation of their specific systems.

4.4.1 Pre-election Audit Records

During election definition and ballot preparation„ the system shall audit the

preparation of the baseline ballot formats and modifications to them, a description of

these modifications, and corresponding dates. The log shall include:

a. The allowable number of selections for an office or issue;

b. The combinations of voting patterns permitted or required by the jurisdiction;

4-82
Volume I — Performance Standards

c. The inclusion or exclusion of offices or issues as the result of multiple districting

within the polling place;

d. Any other characteristics that may be peculiar to the jurisdiction, the election, or

the polling place's location;

e. Manual data maintained by election personnel;

f. Samples of all final ballot formats; and

g. Ballot preparation edit listings.

4.4.2 System Readiness Audit Records

The following minimum requirements apply to system readiness audit records:

a. Prior to the start of ballot counting, a system process shall verify hardware and

software status and generate a readiness audit record. This record shall include

the identification of the software release, the identification of the election to be

processed, and the results of software and hardware diagnostic tests;

b. In the case of systems used at the polling place, the record shall include the

polling place's identification;

c. The ballot interpretation logic shall test and record the correct installation of

ballot formats on voting devices;

d. The software shall check and record the status of all data paths and memory

locations to be used in vote recording to protect against contamination of

voting data;

e. Upon the conclusion of the tests, the software shall provide evidence in the

audit record that the test data have been expunged;

f. If required and provided, the ballot reader and arithmetic-logic unit shall be

evaluated for accuracy, and the system shall record the results. It shall allow

the processing, or simulated processing, of sufficient test ballots to provide a

statistical estimate of processing accuracy; and

g. For systems that use a public network, provide a report of test ballots that

includes:

1) Number of ballots sent;

2) When each ballot was sent;

3) Machine from which each ballot was sent; and

4) Specific votes or selections contained in the ballot.

4.83
Volume I — Performance Standards

4.4.3 In-Process Audit Records

In-process audit records document system operations during diagnostic routines and

the casting and tallying of ballots. At a minimum, the in-process audit records shall

contain:

a. Machine generated error and exception messages to demonstrate successful

recovery. Examples include, but are not necessarily limited to:

1) The source and disposition of system interrupts resulting in entry into

exception handling routines;

2) All messages generated by exception handlers;

3) The identification code and number of occurrences for each hardware and

software error or failure;

4) Notification of system login or access errors, file access errors, and

physical violations of security as they occur, and a summary record of

these events after processing;

5) Other exception events such as power failures, failure of critical hardware

components, data transmission errors, or other type of operating anomaly;

b. Critical system status messages other than informational messages displayed

by the system during the course of normal operations. These items include, but

are not limited to:

1) Diagnostic and status messages upon startup;

2) The "zero totals" check conducted before opening the polling place or

counting a precinct centrally;

3) For paper-based systems, the initiation or termination of card reader and

communications equipment operation; and

4) For DRE machines at controlled voting locations, the event (and time, if

available) of activating and casting each ballot (i.e., each voter's

transaction as an event). This data can be compared with the public

counter for reconciliation purposes;

c. Non-critical status messages that are generated by the machine's data quality

monitor or by software and hardware condition monitors; and

d. System generated log of all normal process activity and system events that

require operator intervention, so that each operator access can be monitored

and access sequence can be constructed.

4.84
Volume I — Performance Standards

4.4.4 Vote Tally Data

In addition to the audit requirements described above, other election-related data is

essential for reporting results to interested parties, the press, and the voting public, and

is vital to verifying an accurate count.

Voting systems shall meet these reporting requirements by providing software capable

of obtaining data concerning various aspects of vote counting and producing reports of

them on a printer. At a minimum, vote tally data shall include:

a. Number of ballots cast, using each ballot configuration, by tabulator, by

precinct, and by political subdivision;

b. Candidate and measure vote totals for each contest, by tabulator;

c. The number of ballots read within each precinct and for additional

jurisdictional levels, by configuration, including separate totals for each party

in primary elections;

d. Separate accumulation of overvotes and undervotes for each contest, by

tabulator, precinct and for additional jurisdictional levels (no overvotes would

be indicated for DRE voting devices); and

e. For paper-based systems only, the total number of ballots both processed and

unprocessable; and if there are multiple card ballots, the total number of cards

read.

For systems that produce an electronic file containing vote tally data, the contents of

the file shall include the same minimum data cited above for printed vote tally reports.

4.5 Vote Secrecy (DRE Systems)

All DRE systems shall ensure vote secrecy by:

a. Immediately after the voter chooses to cast his or her ballot, record the voter's

selections in the memory to be used for vote counting and audit data (including

ballot images), and erase the selections from the display, memory, and all

other storage, including all forms of temporary storage; and

b. Immediately after the voter chooses to cancel his or her ballot, erase the

selections from the display and all other storage, including buffers and other

temporary storage.

4.85

Volume I — Performance Standards

5 Telecommunications

5.1 Scope

This section contains the performance, design, and maintenance characteristics of the

telecommunications components of voting systems and the acceptable levels of

performance against these characteristics. For the purpose of the Standards,

telecommunications is defined as the capability to transmit and receive data

electronically using hardware and software components over distances both within and

external to a polling place.

The requirements in this section represent acceptable levels of combined

telecommunications hardware and software function and performance for the

transmission of data that is used to operate the system and report election results.

Where applicable, this section specifies minimum values for critical performance and

functional attributes involving telecommunications hardware and software

components.

This section does not apply to other means of moving data, such as the physical

transport of data recorded on paper-based media, or the transport of physical devices,

such as memory cards, that store data in electronic form.

Voting systems may include network hardware and software to transfer data among

systems. Major network components are local area networks (LANs), wide area

networks (WANs), workstations (desktop computers), servers, data, and applications.

Workstations include voting stations, precinct tabulation systems, and voting

supervisory terminals. Servers include systems that provide registration forms and

ballots and accumulate and process voter registrations and cast ballots.

Desirable network characteristics include simplicity, flexibility (especially in routing,

to maintain good response times) and maintainability (including availability, provided

primarily through redundancy of resources and connections, particularly of

connections to public infrastructure).

5-87
Volume I — Performance Standards

A wide area network (WAN) public telecommunications component consists of the

hardware and software to transport information, over shared, public (i.e., commercial

or governmental) circuitry, or among private systems. For voting systems, the

telecommunications boundaries are defined as the transport circuitry, on one side of

which exists the public telecommunications infrastructure, outside the control of

voting system supervisors. On the other side of the transport circuitry are the local area

network (LAN) resources, workstations, servers, data and applications controlled by

voting system supervisors.

Local area network (LAN) components consist of the hardware and software

infrastructure used to transport information between users in a local environment,

typically a building or group of buildings. Typically a LAN connects workstations,

perhaps with a local server.

An application may be a single program or a group of programs that work together to

provide a function to an end user, who may be a voter or an election administrator.

Voter programs may include voter registration, balloting, and status checking.

Administrator programs may include ballot preparation, registration for preparation,

registration approval, ballot vetting, ballot processing, and election processing.

This Section is intended to compliment the network security requirements found in

Volume I Section 6, which include requirements for voter and administrator access,

availability of network service, data confidentiality, and data integrity. Most

importantly, security services will restrict access to local election system components

from public resources, and these services will also restrict access to voting system data

while it is in transit across public resources. (This is corollary to voting supervisors

controlling local election systems and not assuming control over public resources.)

5.1.1 Types of Components

This section addresses telecommunications hardware and software across a broad

range of technologies including, but not limited to:

♦ Dial-up communications technologies:

• Standard landline;

• Wireless;

• Microwave;

• Very Small Aperture Terminal (VSAT);

• Integrated Services Digital Network (ISDN); and

• Digital Subscriber Line (DSL);

♦ High-speed telecommunications lines (public and private):

5.88
Volume I — Performance Standards

• FT-1, T-1, T-3;

• Frame Relay; and

• Private line;

♦ Cabling technologies:

• Universal Twisted Pair (UTP) cable (CAT 5 or higher);

• Ethernet hub/switch; and

• Wireless connections (Radio Frequency (RF) and Infrared);

♦ Communications routers;

♦ Modems, whether internal and external to personal computers, computer

servers, and other voting system components (whether installed at the polling

place or central count location);

♦ Modem drivers, dial-up networking software;

♦ Channel service units (CSU)/Data service units (DSU) (whether installed at

the polling place or central count location); and

♦ Dial-up networking applications software.

5.1.2 Telecommunications Operations and Providers

This section applies to voting-related transmissions over public networks, such as

those provided by regional telephone companies and long distance carriers. This

section also applies to private networks regardless of whether the network is owned

and operated by the election jurisdiction.

For systems that transmit official data over public networks, this Section applies to

telecommunications components installed and operated at settings supervised by

election officials, such as polling places or central offices. These standards apply to:

♦ Components acquired by the jurisdiction for the purpose of voting, including

components installed at the poll site or a central office (including central site

facilities operated by vendors or contractors); and

♦ Components acquired by others (such as school systems, libraries, military

installations and other public organizations) that are used at settings

supervised by election officials, including minimum configuration components

required by the vendor but that the vendor permits to be acquired from third

party sources not under the vendor's control (e.g., router or modem card

manufacturer or supplier)

5.89
Volume I — Performance Standards

5.1.3 Data Transmissions

These requirements apply to the use of telecommunications to transmit data for the

preparation of the system for an election, the execution of an election, and the

preservation of the system data and audit trails during and following an election. While

this section does not assume a specific model of voting system operations and does not

assume a specific model for the use of telecommunications to support such operations,

it does address the following types of data, where applicable:

♦ Voter Authentication: Coded information that confirms the identity of a voter

for security purposes for a system that transmits votes individually over a

public network;

♦ Ballot Definition: Information that describes to a voting machine the content

and appearance of the ballots to be used in an election;

♦ Vote Transmission: For systems that transmit votes individually over a public

network, the transmission of a single vote within a network at a polling place

and to the county (or contractor) for consolidation with other county vote data;

♦ Vote Count: Information representing the tabulation of votes at any level

within the control of the jurisdiction, such as the polling place, precinct, or

central count; and

♦ List of Voters: A listing of the individual voters who have cast ballots in a

specific election.

Additional data transmissions used to operate a voting system in the conduct of an

election, but not explicitly listed above, are also subject to the standards of this section.

For systems that transmit data using public networks, this section applies to

telecommunications hardware and software for transmissions within and among all

combinations of senders and receivers indicated below:

♦ Polling places;

♦ Precinct count facilities; and

♦ Central count facilities (whether operated by the jurisdiction or a contractor).

5.2 Design, Construction, and Maintenance

Requirements

Design, construction, and maintenance requirements for telecommunications represent

the operational capability of both system hardware and software. These capabilities

shall be considered basic to all data transmissions.

5.90
Volume I — Performance Standards

5.2.1 Accuracy

The telecommunications components of all voting systems shall meet the accuracy

requirements of Section 3.2.1.

5.2.2 Durability

The telecommunications components of all voting systems shall meet the durability

requirements of Section 3.4.2.

5.2.3 Reliability

The telecommunications components of all voting systems shall meet the reliability

requirements of Section 3.4.3.

5.2.4 Maintainability

The telecommunications components of all voting systems shall meet the

maintainability requirements of Section 3.4.4.

5.2.5 Availability

The telecommunications components of all voting systems shall meet the availability

requirements of Section 3.4.5.

5.2.6 integrity

For WANs using public telecommunications, boundary definition and implementation

shall meet the following requirements.

a. Outside service providers and subscribers of such providers shall not be given

direct access or control of any resource inside the boundary;

5.91
Volume I — Performance Standards

b. Voting system administrators shall not require any type of control of resources

outside this boundary. Typically, an end point of a telecommunications circuit

will be a subscriber termination on a Digital Service Unit/Customer Service

Unit (DSU/CSU) (though the precise technology may vary, being such things

as cable modems or routers). Regardless of the technology used, the boundary

point must ensure that everything on one side is locally configured and

controlled while everything on the other side is controlled by an outside

service provider; and

c. The system shall be designed and configured such that it is not vulnerable to a

single point of failure in the connection to the public network causing total

loss of voting capabilities at any polling place.

5.2.7 Confirmation

Confirmation occurs when the system notifies the user of the successful or

unsuccessful completion of the data transmission, where successful completion is

defined as accurate receipt of the transmitted data. To provide confirmation, the

telecommunications components of a voting system shall:

a. Notify the user of the successful or unsuccessful completion of the data

transmission; and

b. In the event of unsuccessful transmission, notify the user of the action to be

taken.

5.92
Volume I — Performance Standards

6 Security Standards

6.1 Scope

This section describes essential security capabilities for a voting system, encompassing

the system's hardware, software, communications, and documentation. The Standards

recognize that no predefined set of security standards will address and defeat all

conceivable or theoretical threats. However, the Standards articulate requirements to

achieve acceptable levels of integrity, reliability, and inviolability. Ultimately, the

objectives of the security standards for voting systems are:

♦ To establish and maintain controls that can ensure that accidents, inadvertent

mistakes, and errors are minimized;

♦ To protect the system from intentional manipulation and fraud, and from

malicious mischief;

♦ To identify fraudulent or erroneous changes to the system; and

♦ To protect secrecy in the voting process.

The Standards are intended to address a broad range of risks to the integrity of a voting

system. While it is not possible to identify all potential risks, the Standards identify

several types of risk that must be addressed by a voting system. These include:

♦ Unauthorized changes to system capabilities for:

• Defining ballot formats;

• Casting and recording votes;

• Calculating vote totals consistent with defined ballot formats; and

• Reporting vote totals;

♦ Alteration of voting system audit trails;

♦ Changing, or preventing the recording of, a vote;

6.93
Volume I — Performance Standards

♦ Introducing data for a vote not cast by a registered voter;

♦ Changing calculated vote totals;

♦ Preventing access to vote data, including individual votes and vote totals, to

unauthorized individuals; and

♦ Preventing access to voter identification data and data for votes cast by the

voter such that an individual can determine the content of specific votes cast

by the voter.

This section describes specific capabilities that vendors shall integrate into a voting

system in order to address the risks listed above.

6.1.1 System Components and Sources

The requirements of this section apply to the broad range of hardware, software,

communications components, and documentation that comprises a voting system.

These requirements apply to components:

♦ Provided by the voting system vendor and the vendor's suppliers;

♦ Furnished by an external provider (for example providers of personal

computers and commercial off-the-shelf (COTS) operating systems) where the

components are capable of being used during voting system operation; and

♦ Developed by a voting jurisdiction.

6.1.2 Location and Control of Software and Hardware on

Which it Operates

The requirements of this section apply to all software used in any manner to support

any voting-related activity, regardless of the ownership of the software or the

ownership and location of the hardware on which the software is installed or operated.

These requirements apply to software that operates on:

♦ Voting devices and vote counting devices installed at polling places under the

control or authority of the voting jurisdiction; and

♦ Ballot printers, vote counting devices, and other hardware typically installed at

central or precinct locations (including contractor facilities).

However, some requirements are applicable only in circumstances specified by this

section.

6.94
Volume I — Performance Standards

6.1.3 Elements of Security Outside Vendor Control

The requirements of this section apply to the capabilities of a voting system provided

by the vendor. The Standards recognizes that effective security requires safeguards

beyond those provided by the vendor. Effective security demands diligent security

practices by the purchasing jurisdiction and the jurisdictions representatives. These

practices include:

♦ Administrative and management controls for the voting system and election

management, including access controls;

♦ Internal security procedures;

♦ Adherence to, and enforcement of, operational procedures (e.g., effective

password management);

♦ Security of physical facilities; and

♦ Organizational responsibilities and personnel screening.

Because specific standards for these elements are not under the direct control of the

vendor, they will be addressed in forthcoming Operational Guidelines that address best

practices for jurisdictions conducting elections and managing the operation of voting

systems.

6.1.4 Organization of this Section

The standards presented in this section are organized as follows:

♦ Access Control: These standards addresses procedures and system

capabilities that limit or detect access to critical system components in order to

guard against loss of system integrity, availability, confidentiality, and

accountability.

♦ Equipment and Data Security: These standards address physical security

measures and procedures that prevent disruption of the voting process at the

poll site and corruption of voting data.

♦ Software Security: These standards address the installation of software,

including firmware, in the voting system and the protection against malicious

software.

♦ Telecommunication and Data Transmission: These standards address

security for the electronic transmission of data between system components or

locations over both private and public networks

6.95
Volume I — Performance Standards

♦ Security for Transmission of Official Data Over Public Communications

Networks: These standards address security for systems that communicate

individual votes or vote totals over public communications networks.

It should be noted that computer-generated audit controls facilitate system security and

are an integral part of software capability. These audit requirements are presented in

Section 4.

6.2 Access Control

Access controls are procedures and system capabilities that detect or limit access to

system components in order to guard against loss of system integrity, availability,

confidentiality, and accountability. Access controls provide reasonable assurance that

system resources such as data files, application programs, and computer-related

facilities and equipment are protected against unauthorized operation, modification,

disclosure, loss, or impairment. Unauthorized operations include modification of

compiled or interpreted code, run-time alteration of flow control logic or of data, and

abstraction of raw or processed voting data in any form other than a standard output

report by an authorized operator.

Access controls may include physical controls, such as keeping computers in locked

rooms to limit physical access, and technical controls, such as security software

programs designed to prevent or detect unauthorized access to sensitive files. The

access controls contained in this section of the Standards are limited to those controls

required of system vendors. Access controls required of jurisdictions will be addressed

in future documents detailing operational guidelines for jurisdictions.

6.2.1 Access Control Policy

The vendor shall specify the general features and capabilities of the access control

policy recommended to provide effective voting system security.

6.2.1.1 General Access Control Policy

Although the jurisdiction in which the voting system is operated is responsible for

determining the access policies applying to each election, the vendor shall provide a

description of recommended policies for:

a. Software access controls;

b. Hardware access controls;

6.96
Volume I — Performance Standards

c. Communications;

d. Effective password management;

e. Protection abilities of a particular operating system;

f. General characteristics of supervisory access privileges;

g. Segregation of duties; and

h. Any additional relevant characteristics.

6.2.1.2 Individual Access Privileges

Voting system vendors shall:

a. Identify each person to whom access is granted, and the specific functions and

data to which each person holds authorized access;

b. Specify whether an individual's authorization is limited to a specific time, time

interval, or phase of the voting or counting operations; and

c. Permit the voter to cast a ballot expeditiously, but preclude voter access to all

other aspects of the vote-counting processes.

6.2.2 Access Control Measures

Vendors shall provide a detailed description of all system access control measures

designed to permit authorized access to the system and prevent unauthorized access.

Examples of such measures include:

a. Use of data and user authorization;

b. Program unit ownership and other regional boundaries;

c. One-end or two-end port protection devices;

d. Security kernels;

e. Computer-generated password keys;

f. Special protocols;

g. Message encryption; and

h. Controlled access security.

Vendors also shall define and provide a detailed description of the methods used to

prevent unauthorized access to the access control capabilities of the system itself.

6.97
Volume I — Performance Standards

6.3 Physical Security Measures

A voting system's sensitivity to disruption or corruption of data depends, in part, on

the physical location of equipment and data media, and on the establishment of secure

telecommunications among various locations. Most often, the disruption of voting and

vote counting results from a physical violation of one or more areas of the system

thought to be protected. Therefore, security procedures shall address physical threats

and the corresponding means to defeat them.

6.3.1 Polling Place Security

For polling place operations, vendors shall develop and provide detailed

documentation of measures to anticipate and counteract vandalism, civil disobedience,

and similar occurrences. The measures shall:

a. Allow the immediate detection of tampering with vote casting devices and

precinct ballot counters; and

b. Control physical access to a telecommunications link if such a link is used.

6.3.2 Central Count Location Security

Vendors shall develop and document in detail the measures to be taken in a central

counting environment. These measures shall include physical and procedural controls

related to the:

a. Handling of ballot boxes;

b. Preparing of ballots for counting;

c. Counting operations; and

d. Reporting data.

6.4 Software Security

Voting systems shall meet specific security requirements for the installation of

software and for protection against malicious software.

6.98
Volume I — Performance Standards

6.4.1 Software and Firmware Installation

The system shall meet the following requirements for installation of software,

including hardware with embedded firmware:

a. If software is resident in the system as firmware, the vendor shall require and

state in the system documentation that every device is to be retested to validate

each ROM prior to the start of elections operations;

b. To prevent alteration of executable code, no software shall be permanently

installed or resident in the system unless the system documentation states that

the jurisdiction must provide a secure physical and procedural environment for

the storage, handling, preparation, and transportation of the system hardware;

c. The system bootstrap, monitor, and device-controller software may be resident

permanently as firmware, provided that this firmware has been shown to be

inaccessible to activation or control by any means other than by the authorized

initiation and execution of the vote-counting program, and its associated

exception handlers;

d. The election-specific programming may be installed and resident as firmware,

provided that such firmware is installed on a component (such as computer

chip) other than the component on which the operating system resides; and

e. After initiation of election day testing, no source code or compilers or

assemblers shall be resident or accessible.

6.4.2 Protection Against Malicious Software

Voting systems shall deploy protection against the many forms of threats to which they

may be exposed such as file and macro viruses, worms, Trojan horses, and logic

bombs. Vendors shall develop and document the procedures to be followed to ensure

that such protection is maintained in a current status.

6.5 Telecommunications and Data

Transmission

There are four areas that must be addressed by telecommunications and data

transmission security capabilities:

♦ Access control for telecommunications capabilities;

♦ Data integrity;

6.99
Volume I — Performance Standards

♦ Detection and prevention of data interception; and

♦ Protection against external threats to which commercial products used by a

voting system may be susceptible.

6.5.1 Access Control

Voting systems that use telecommunications to communicate between system

components and locations are subject to the same security requirements governing

access to any other system hardware, software, and data function.

6.5.2 Data Integrity

Voting systems that use electrical or optical transmission of data shall ensure the

receipt of valid vote records is verified at the receiving station. This should include

standard transmission error detection and correction methods such as checksums or

message digest hashes. Verification of correct transmission shall occur at the voting

system application level and ensure that the correct data is recorded on all relevant

components consolidated within the polling place prior to the voter completing casting

of his or her ballot.

6.5.3 Data Interception Prevention

Voting systems that use telecommunications as defined in Section 5 to communicate

between system components and locations before the poll site is officially closed shall:

a. Implement an encryption standard currently documented and validated for use

by an agency of the U.S. Federal Government; and

b. Provide a means to detect the presence of an intrusive process, such as an

Intrusion Detection System.

6.5.4 Protection Against External Threats

Voting systems that use public telecommunications networks shall implement

protections against external threats to which commercial products used in the system

may be susceptible.

6-100
Volume I — Performance Standards

6.5.4.1 Identification of COTS Products

Voting systems that use public telecommunications networks shall provide system

documentation that clearly identifies all COTS hardware and software products and

communications services used in the development and/or operation of the voting

system, including:

a. Operating systems;

b. Communications routers;

c. Modem drivers; and

d. Dial-up networking software.

Such documentation shall identify the name, vendor, and version used for each such

component.

6.5.4.2 Use of Protective Software

Voting systems that use public telecommunications networks shall use protective

software at the receiving-end of all communications paths to:

a. Detect the presence of a threat in a transmission;

b. Remove the threat from infected files/data;

c. Prevent against storage of the threat anywhere on the receiving device;

d. Provide the capability to confirm that no threats are stored in system memory

and in connected storage media; and

e. Provide data to the system audit log indicating the detection of a threat and the

processing performed.

Vendors shall use multiple forms of protective software as needed to provide

capabilities for the full range of products used by the voting system.

6.5.4.3 Monitoring and Responding to External Threats

Voting systems that use public telecommunications networks may become vulnerable,

by virtue of their system components, to external threats to the accuracy and integrity

of vote recording, vote counting, and vote consolidation and reporting processes.

Therefore, vendors of such systems shall document how they plan to monitor and

respond to known threats to which their voting systems are vulnerable. This

documentation shall provide a detailed description, including scheduling information,

of the procedures the vendor will use to:

6101
Volume I — Performance Standards

a. Monitor threats, such as through the review of assessments, advisories, and

alerts for COTS components issued by the Computer Emergency Response

Team (CERT), for which a current listing can be found at http://www.cert.org,

the National Infrastructure Protection Center (NIPC), for which a current

listing can be found at http://www.nipc.gov/warnings/warnings.htm, and the

Federal Computer Incident Response Capability (FedCIRC), for which

additional information can be found at http://www.fedcirc.gov/;

b. Evaluate the threats and, if any, proposed responses;

c. Develop responsive updates to the system and/or corrective procedures;

d. Submit the proposed response to the ITAs and appropriate states for approval,

identifying the exact changes and whether or not they are temporary or

permanent;

e. After implementation of the proposed response is approved by the state, assist

clients, either directly or through detailed written procedures, how to update

their systems and/or to implement the corrective procedures no later than one

month before an election; and

f. Address threats emerging too late to correct the system at least one month

before the election, including:

1) Providing prompt, emergency notification to the ITAs and the affected

states and user jurisdictions;

2) Assisting client jurisdictions directly, or advising them through detailed

written procedures, to disable the public telecommunications mode of the

system; and

3) After the election, modifying the system to address the threat; submitting

the modified system to an ITA and appropriate state certification authority

for approval, and assisting client jurisdictions directly, or advising them

through detailed written procedures, to update their systems and/or to

implement the corrective procedures after approval.

6.5.5 Shared Operating Environment

Ballot recording and vote counting can be performed in either a dedicated or nondedicated environment. If ballot recording and vote counting operations are performed

in an environment that is shared with other data processing functions, both hardware

and software features shall be present to protect the integrity of vote counting and of

vote data. Systems that use a shared operating environment shall:

a. Use security procedures and logging records to control access to system

functions;

b. Partition or compartmentalize voting system functions from other concurrent

functions at least logically, and preferably physically as well;

6-102
Volume I — Performance Standards

c. Controlled system access by means of passwords, and restriction of account

access to necessary functions only; and

d. Have capabilities in place to control the flow of information, precluding data

leakage through shared system resources.

6.5.6 Access to Incomplete Election Returns and Interactive

Queries

If the voting system provides access to incomplete election returns and interactive

inquiries before the completion of the official count, the system shall:

a. For equipment that operates in a central counting environment, be designed to

provide external access to incomplete election returns only if that access for

these purposes is authorized by the statutes and regulations of the using

agency. This requirement applies as well to polling place equipment that

contains a removable memory module, or that may be removed in its entirety

to a central place for the consolidation of polling place returns.

b. Use voting system software and its security environment designed such that

data accessible to interactive queries resides in an external file, or database,

that is created and maintained by the elections software under the restrictions

applying to any other output report, namely, that:

1) The output file or database has no provision for write-access back to the

system.

2) Persons whose only authorized access is to the file or database are denied

write-access, both to the file or database, and to the system.

6.6 Security for Transmission of Official Data

Over Public Communications Networks

DRE systems that transmit data over public telecommunications networks face security

risks that are not present in other DRE systems. This section describes standards

applicable to DRE systems that use public telecommunications networks.

6.6.1 General Security Requirements for Systems

Transmitting Data Over Public Networks

All systems that transmit data over public telecommunications networks shall:

6-1 03
Volume I – Performance Standards

a. Preserve the secrecy of a voter's ballot choices, and prevent anyone from

violating ballot privacy;

b. Employ digital signature for all communications between the vote server and

other devices that communicate with the server over the network; and

c. Require that at least two authorized election officials activate any critical

operation regarding the processing of ballots transmitted over a public

communications network takes place, i.e. the passwords or cryptographic keys

of at least two employees are required to perform processing of votes.

6.6.2 Voting Process Security for Casting Individual Ballots

over a Public Telecommunications Network

Systems designed for transmission of telecommunications over public networks shall

meet security standards that address the security risks attendant with the casting of

ballots from poll sites controlled by election officials using voting devices configured

and installed by election officials and/or their vendor or contractor, and using inperson authentication of individual voters.

6.6.2.1 Documentation of Mandatory Security Activities

Vendors of systems that cast individual ballots over a public telecommunications

network shall provide detailed descriptions of:

a. All activities mandatory to ensuring effective system security to be performed

in setting up the system for operation, including testing of security before an

election; and

b. All activities that should be prohibited during system setup and during the time

frame for voting operations, including both the hours when polls are open and

when polls are closed.

6.6.2.2 Capabilities to Operate During Interruption of

Telecommunications Capabilities

These systems shall provide the following capabilities to provide resistance to

interruptions of telecommunications service that prevent voting devices at the poll site

from communicating with external components via telecommunications:

a. Detect the occurrence of a telecommunications interruption at the poll site and

switch to an alternative mode of operation that is not dependent on the

connection between poll site voting devices and external system components;

6-104
Volume I — Performance Standards

b. Provide an alternate mode of operation that includes the functionality of a

conventional DRE machine without losing any single vote;

c. Create and preserve an audit trail of every vote cast during the period of

interrupted communication and system operation in conventional DRE system

mode;

d. Upon reestablishment of communications, transmit and process votes

accumulated while operating in conventional DRE system mode with all

security safeguards in effect; and

e. Ensure that all safeguards related to voter identification and authentication are

not affected by the procedures employed by the system to counteract potential

interruptions of telecommunications capabilities.

6.105

Volume I — Performance Standards

7 Quality Assurance

7.1 Scope

Quality Assurance provides continuous confirmation that a voting system conforms

with the Standards and to the requirements of state and local jurisdictions. Quality

Assurance is a vendor function with associated practices that is initiated prior to

system development and continues throughout the maintenance life cycle of the voting

system. Quality Assurance focuses on building quality into a system and reducing

dependence on system tests at the end of the life cycle to detect deficiencies, thus

helping ensure the system:

♦ Meets stated requirements and objectives;

♦ Adheres to established standards and conventions;

♦ Functions consistent with related components and meets dependencies for use

within the jurisdiction; and

♦ Reflects all changes approved during its initial development, internal testing,

qualification, and, if applicable, additional certification processes.

7.2 General Requirements

The voting system vendor is responsible for designing and implementing a quality

assurance program to ensure that the design, workmanship, and performance

requirements of this standard are achieved in all delivered systems and components.

At a minimum, this program shall:

a. Include procedures for specifying, procuring, inspecting, accepting, and

controlling parts and raw materials of the requisite quality;

b. Require the documentation of the hardware and software development process;

c. Identify and enforce all requirements for:

7-107
Volume I — Performance Standards

1) In-process inspection and testing that the manufacturer deems necessary to

ensure proper fabrication and assembly of hardware, and

2) Installation and operation of software (including firmware).

d. Include plans and procedures for post-production environmental screening and

acceptance test; and

e. Include a procedure for maintaining all data and records required to document and

verify the quality inspections and tests.

7.3 Components from Third Parties

A vendors who does not manufacture all the components of its voting system, but

instead procures components as standard commercial items for assembly and

integration into a voting system, should verify that the supplier vendors follow

documented quality assurance procedures that are at least as stringent as those used

internally by the voting system vendor.

7.4 Responsibility for Tests

The manufacturer or vendor shall be responsible for:

a. Performing all quality assurance tests;

b. Acquiring and documenting test data; and

c. Providing test reports for review by the ITA, and to the purchaser upon

request.

7.5 Parts & Materials Special Tests and

Examinations

In order to ensure that voting system parts and materials function properly, vendors

shall:

a. Select parts and materials to be used in voting systems and components

according to their suitability for the intended application. Suitability may be

determined by similarity of this application to existing standard practice, or by

means of special tests;

7108
Volume I — Performance Standards

b. Design special tests, if needed, to evaluate the part or material under

conditions accurately simulating the actual operating environment; and

c. Maintain the resulting test data as part of the quality assurance program

documentation.

7.6 Quality Conformance Inspections

The vendor performs conformance inspections to ensure the overall quality of the

voting system and components delivered to the ITA for testing and to the jurisdiction

for implementation. To meet the conformance inspection requirements the vendor or

manufacturer shall:

a. Inspect and test each voting system or component to verify that it meets all

inspection and test requirements for the system; and

b. Deliver a record of tests, or a certificate of satisfactory completion, with each

system or component.

7.7 Documentation

Vendors are required to produce documentation to support the development and formal

testing of voting systems. To meet documentation requirements, vendors shall provide

complete product documentation with each voting systems or components, as

described Volume II, Section 2 for the TDP. This documentation shall:

a. Be sufficient to serve the needs of the ITA, voters, election officials, and

maintenance technicians;

b. Be prepared and published in accordance with standard industrial practice for

information technology and electronic and mechanical equipment; and

c. Consist, at a minimum, of the following:

1) System overview;

2) System functionality description;

3) System hardware specification;

4) Software design and specifications;

5) System security specification;

6) System test and verification specification;

7) System operations procedures;

7.109
Volume I — Performance Standards

8) System maintenance procedures;

9) Personnel deployment and training requirements;

10) Configuration management plan;

11) Quality assurance program; and

12) System Change Notes.

7110
Volume I — Performance Standards

8 Configuration Management

8.1 Scope

This section contains specific requirements for configuration management of voting

systems. For the purpose of the Standards, configuration management is defined as a

set of activities and associated practices that ensures full knowledge and control of the

components of a system, starting with its initial development and progressing through

its ongoing maintenance and enhancement. This section describes activities in terms of

their purposes and outcomes. It does not describe specific procedures or steps to be

employed to accomplish them. Specific steps and procedures are left to the vendor to

select.

Vendors are required to submit these procedures to the Independent Test Authority

(ITA) as part of the Technical Data Package (TDP) for system qualifications described

in Volume II, Voting Systems Qualification Testing Standards, for review against the

requirements of this section. Additionally, state or local election legislation,

regulations, or contractual agreements may require the vendor to conform to additional

standards for configuration management or to adopt specific required procedures.

Further, authorized election officials or their representatives reserve the right to inspect

vendor facilities and operations to determine conformance with the vendor's reported

procedures and with any additional requirements.

8.1.1 Configuration Management Requirements

Configuration management addresses a broad set of record keeping, audit, and

reporting activities that contribute to full knowledge and control of a system and its

components. These activities include:

♦ Identifying discrete system components;

8.111
Volume I — Performance Standards

♦ Creating records of a formal baseline and later versions of components;

♦ Controlling changes made to the system and its components;

♦ Releasing new versions of the system to ITAs;

♦ Releasing new versions of the system to customers;

♦ Auditing the system, including its documentation, against configuration

management records;

♦ Controlling interfaces to other systems; and

♦ Identifying tools used to build and maintain the system.

8.1.2 Organization of Configuration Management Standards

The standards for configuration management presented in this section include:

♦ Application of configuration management requirements;

♦ Configuration management policy;

♦ Configuration identification;

♦ Baseline, promotion, and demotion procedures;

♦ Configuration control procedures;

♦ Release process;

♦ Configuration audits; and

♦ Configuration management resources.

8.1.3 Application of Configuration Management

Requirements

Requirements for configuration management apply regardless of the specific

technologies employed to all voting systems subject to the Standards. These system

components include:

a. Software components;

b. Hardware components;

c. Communications components;

d. Documentation;

8-112
Volume I — Performance Standards

e. Identification and naming and conventions (including changes to these

conventions) for software programs and data files;

f. Development and testing artifacts such as test data and scripts; and

g. File archiving and data repositories.

8.2 Configuration Management Policy

The vendor shall describe its policies for configuration management in the TDP. This

description shall address the following elements:

a. Scope and nature of configuration management program activities; and

b. Breadth of application of the vendor's policies and practices to the voting

system (i.e., extent to which policies and practices apply to the total system,

and extent to which policies and practices of suppliers apply to particular

components, subsystems, or other defined system elements.

8.3 Configuration Identification

Configuration identification is the process of identifying, naming, and acquiring

configuration items. Configuration identification encompasses all system components.

8.3.1 Structuring and Naming Configuration Items

The vendor shall describe the procedures and conventions used to:

a. Classify configuration items into categories and subcategories;

b. Uniquely number or otherwise identify configuration items; and

c. Name configuration items;

8.3.2 Versioning Conventions

When a system component is used to identify higher-level system elements, a vendor

shall describe the conventions used to:

8113
Volume I — Performance Standards

a. Identify the specific versions of individual configuration items and sets of

items that are used by the vendor to identify higher level system elements such

as subsystems;

b. Uniquely number or otherwise identify versions; and

c. Name versions.

8.4 Baseline, Promotion, and Demotion

Procedures

The vendor shall establish formal procedures and conventions for establishing and

providing a complete description of the procedures and related conventions used to:

a. Establish a particular instance of a component as the starting baseline;

b. Promote subsequent instances of a component to baseline status as

development progresses through to completion of the initial completed version

released to the ITAs for qualification testing; and

c. Promote subsequent instances of a component to baseline status as the

component is maintained throughout its life cycle until system retirement (i.e.,

the system is no longer sold or maintained by the vendor).

8.5 Configuration Control Procedures

Configuration control is the process of approving and implementing changes to a

configuration item to prevent unauthorized additions, changes, or deletions. The

vendor shall establish such procedures and related conventions, providing a complete

description of those procedures used to:

a. Develop and maintain internally developed items;

b. Acquire and maintain third-party items;

c. Resolve internally identified defects for items regardless of their origin; and

d. Resolve externally identified and reported defects (i.e., by customers and ITAs).

8.6 Release Process

The release process is the means by which the vendor installs, transfers, or migrates

the system to the ITAs and, eventually, to its customers. The vendor shall establish

8-114
Volume I — Performance Standards

such procedures and related conventions, providing a complete description of those

used to:

a. Perform a first release of the system to an ITA;

b. Perform a subsequent maintenance or upgrade release of the system, or a

particular components, to an ITA;

c. Perform the initial delivery and installation of the system to a customer,

including confirmation that the installed version of the system matches exactly

the qualified system version; and

d. Perform a subsequent maintenance or upgrade release of the system, or a

particular component, to a customer, including confirmation that the installed

version of the system matches exactly the qualified system version.

8.7 Configuration Audits

The Standards require two types of configuration audits: Physical Configuration

Audits (PCA) and Functional Configuration Audits (FCA).

8.7.1 Physical Configuration Audit

The PCA is conducted by the ITA to compare the voting system components submitted

for qualification to the vendor's technical documentation. For the PCA, a vendor shall

provide:

a. Identification of all items that are to be a part of the software release;

b. Specification of compiler (or choice of compilers) to be used to generate

executable programs;

c. Identification of all hardware that interfaces with the software;

d. Configuration baseline data for all hardware that is unique to the system;

e. Copies of all software documentation intended for distribution to users,

including program listings, specifications, operations manual, voter manual,

and maintenance manual;

f. User acceptance test procedures and acceptance criteria; and

g. Identification of any changes between the physical configuration of the system

submitted for the PCA and that submitted for the FCA, with a certification that

any differences do not degrade the functional characteristics; and

8-115
Volume I – Performance Standards

h. Complete descriptions of its procedures and related conventions used to

support this audit by:

1) Establishing a configuration baseline of the software and hardware to be

tested; and

2) Confirming whether the system documentation matches the

corresponding system components.

8.7.2 Functional Configuration Audit

The FCA is conducted by the ITA to verify that the system performs all the functions

described in the system documentation. The vendor shall:

a. Completely describe its procedures and related conventions used to support this

audit for all system components;

b. Provide the following information to support this audit:

1) Copies of all procedures used for module or unit testing, integration

testing, and system testing;

2) Copies of all test cases generated for each module and integration test,

and sample ballot formats or other test cases used for system tests; and

3) Records of all tests performed by the procedures listed above, including

error corrections and retests.

In addition to such audits performed by ITAs during the system qualification process,

elements of this audit may also be performed by state election organizations during the

system certification process, and individual jurisdictions during system acceptance

testing.

8.8 Configuration Management Resources

Often, configuration management activities are performed with the aid of automated

tools. Assuring that such tools are available throughout the system life cycle, including

if the vendor is acquired by or merged with another organization, is critical to effective

configuration management. Vendors may choose the specific tools they use to perform

the record keeping, audit, and reporting activities of the configuration management

standards. The resources documentation standard provided below focus on assuring

that procedures are in place to record information about the tools to help ensure that

they, and the data they contain, can be transferred effectively and promptly to a third

party should the need arise. Within this context, a vendor is required to develop and

provide a complete description of the procedures and related practices for maintaining

information about:

8-116
Volume I — Performance Standards

a. Specific tools used, current version, and operating environment;

b. Physical location of the tools, including designation of computer directories

and files; and

c. Procedures and training materials for using the tools.

8.117

Volume I — Performance Standards

9 Overview of Qualification Tests

9.1 Scope

This section provides an overview of the testing process for qualification testing of

voting systems. Qualification testing is the process by which a voting system is shown

to comply with the requirements of the Standards and the requirements of its own

design and performance specifications.

Qualification testing encompasses the examination of software; tests of hardware

under conditions simulating the intended storage, operation, transportation, and

maintenance environments; the inspection and evaluation of system documentation;

and operational tests to validate system performance and function under normal and

abnormal conditions. The testing also evaluates the completeness of the vendor's

developmental test program, including the sufficiency of vendor tests conducted to

demonstrate compliance with stated system design and performance specifications, and

the vendor's documented quality assurance and configuration management practices.

The tests address individual system components or elements, as well as the integrated

system as a whole. Since 1994, qualification tests for voting systems have been

performed by Independent Test Authorities (ITAs) certified by the National

Association of State Election Directors (NASED). NASED has certified an ITA for

either the full scope of qualification testing or a distinct subset of the total scope of

testing. The test process described in this section may be conducted by one or more

ITAs, depending on the nature of tests to be conducted and the expertise of the

certified ITAs.

Qualification testing is distinct from all other forms of testing, including

developmental testing by the vendor, certification testing by a state election

organization, and system acceptance testing by a purchasing jurisdiction:

♦ Qualification testing follows the vendor's developmental testing;

♦ Qualification testing provides an assurance to state election officials and local

jurisdictions of the conformance of a voting system to the Standards as input to

state certification of a voting system and acceptance testing by a purchasing

jurisdiction; and

9 119
Volume I — Performance Standards

♦ Qualification testing may precede state certification testing, or may be

conducted in parallel as established by the certification program of individual

states.

Generally a voting system remains qualified under the standards against which it was

tested, as long as all modifications made to the system are evaluated and passed by a

certified ITA. The qualification test report remains valid for as long as the voting

system remains unchanged from the last tested configuration. However, if a new threat

to a particular voting system is discovered, it is the prerogative of NASED to

determine which qualified voting systems are vulnerable, whether those systems need

to be retested, and the specific tests to be conducted. In addition, when new standards

supersede the standards under which the system was qualified, it is the prerogative of

NASED to determine when systems that were qualified under the earlier standards will

lose their qualification, unless they are tested to meet current standards.

The remainder of this section describes the documentation and equipment required to

be submitted by the vendor, the scope of qualification testing, the applicability to

voting system components, and the flow of the test process.

9.2 Documentation Submitted by Vendor

The vendor shall submit to the ITA documentation necessary for the identification of

the full system configuration submitted for evaluation and for the development of an

appropriate test plan by the ITA for system qualification testing.

One element of the documentation is the Technical Data Package (TDP). The TDP

contains information that that defines the voting system design, method of operation,

and related resources. It provides a system overview and documents the system's

functionality, hardware, software, security, test and verification specifications,

operations procedures, maintenance procedures, and personnel deployment and

training requirements. It also documents the vendor's configuration management plan

and quality assurance program. If the system was previously qualified, the TDP also

includes the system change notes.

This documentation is used by the ITA in constructing the qualification testing plan

and is particularly important in constructing plans for the re-testing of systems that

have been qualified previously. Re-testing of systems submitted by vendors that

consistently adhere to particularly strong and well documented quality assurance and

configuration management practices will generally be more efficient than for systems

developed and maintained using less rigorous or less well documented practices.

Volume II provides a detailed description of the documentation required for the

vendor's quality assurance and configuration management practices used for the

system submitted for qualification testing.

9120
Volume I — Performance Standards

9.3 Voting Equipment Submitted by Vendor

Vendors may seek to market a complete voting system or an interoperable component

of a voting system. Nevertheless, vendors shall submit for testing the specific system

configuration that is to be offered to jurisdictions or that comprises the component to

be marketed plus the other components with which the vendor recommends that

component be used. The system submitted for testing shall meet the following

requirements:

a. The hardware submitted for qualification testing shall be equivalent, in form

and function, to the actual production versions of the hardware units or the

COTS hardware specified for use in the TDP;

b. The software submitted for qualification testing shall be the exact software

that will be used in production units;

c. Engineering or developmental prototypes are not acceptable, unless the vendor

can show that the equipment to be tested is equivalent to standard production

units in both performance and construction; and

d. Benchmark directory listings shall be submitted for all software/firmware

elements (and associated documentation) included in the vendor's release as

they would normally be installed upon setup and installation.

9.4 Testing Scope

The qualification test process is intended to discover vulnerabilities that, should they

appear in actual election use, could result in failure to complete election operations in

a satisfactory manner.

Five types of focuses guide the overall qualification testing process:

♦ Operational accuracy in the recording and processing of voting data, as

measured by target error rate, for which the maximum acceptable error rate is

no more than one in ten million ballot positions, with a maximum acceptable

error rate in the test process of one in 500,000 ballot positions (while it would

be desirable that there be an error rate of zero, if this had to be proven by a

test, the test itself would take an infinity of time);

♦ Operational failures or the number of unrecoverable failures under conditions

simulating the intended storage, operation, transportation, and maintenance

environments for voting systems, using an actual time-based period of

processing test ballots;

♦ System performance and function under normal and abnormal conditions; and

9-121
Volume I — Performance Standards

♦ Completeness and accuracy of the system documentation and configuration

management records to enable purchasing jurisdictions to effectively install,

test, and operate the system.

Qualification testing complements and evaluates the vendor's developmental testing,

including any beta testing. The ITA evaluates the completeness of the vendor's

developmental test program, including the sufficiency of vendor tests conducted to

demonstrate compliance with the Standards as well as the system's performance

specifications. The ITA undertakes sample testing of the vendor's test modules and

also designs independent system-level tests to supplement and check those designed by

the vendor. Although some of the qualification tests are based on those prescribed in

the Military Standards, in most cases the test conditions are less stringent, reflecting

commercial, rather than military, practice. The ITA may use automated software

testing tools to assist in this process if they are available for the software under

examination.

The procedure for disposition of system deficiencies discovered during qualification

testing is described in Volume II of the Standards. This procedure recognizes that

some but not necessarily all operational malfunctions (apart from software logic

defects) may result in rejection. Basically, any defect that results in or may result in the

loss or corruption of voting data, whether through failure of system hardware,

software, or communication, through procedural deficiency, or through deficiencies in

security and audit provisions, shall be cause for rejection. Otherwise, malfunctions that

result from failure to comply fully with other requirements of this standard will not in

every case warrant rejection. Specific failure definition and scoring criteria are also

contained in Volume II.

9.4.1 Test Categories

The qualification test procedure is presented in several parts:

♦ Functionality testing;

♦ Hardware testing;

♦ Software evaluation;

♦ System-level integration tests, including audits; and

♦ Examination of documented vendor practices for quality assurance and for

configuration management.

In practice, there may be concurrent indications of hardware and software function, or

failure to function, during certain examinations and tests. Operating tests of hardware

partially exercise the software as well and therefore supplement software qualification.

Security tests exercise hardware, software and communications capabilities.

Documentation review conducted during software qualification supplements the

review undertaken for system-level testing.

9.122
Volume I — Performance Standards

The qualification test procedures are presented in these categories because test

authorities frequently focus separately on each. The following subsections provide

information that test authorities need to conduct testing.

Not all systems being tested are required to complete all categories of testing. For

example, if a previously-qualified system has had hardware modifications, the system

may be subject only to non-operating environmental stress testing of the modified

component, and a partial system-level test. If a system consisting of general purpose

COTS hardware or one that was previously qualified has had modifications to its

software, the system is subject only to software qualification and system-level tests,

not hardware testing. However, in all cases the system documentation and

configuration management records will be examined to confirm that they completely

and accurately reflect the components and component versions that comprise the

voting system.

9.4.1.1 Focus of Functionality Tests

Functionality testing is performed to confirm the functional capabilities of a voting

system submitted for qualification. The ITA designs and performs procedures to test a

voting system against the requirements outlined in Section 2. In order to best

compliment the diversity of the voting systems industry, this part of the qualification

testing process is not rigidly defined. Although there are basic functionality testing

requirements, additions or variations in testing are appropriate depending on the

system's use of specific technologies and configurations, the system capabilities, and

the outcomes of previous testing.

9.4.1.2 Focus of Hardware Tests

Hardware testing begins with non-operating tests that require the use of an

environmental test facility. These are followed by operating tests that are performed

partly in an environmental facility and partly in a standard test laboratory or shop

environment.

The non-operating tests are intended to evaluate the ability of the system hardware to

withstand exposure to the various environmental conditions incidental to voting system

storage, maintenance, and transportation. The procedures are based on test methods

contained in Military Standards (MIL-STD) 810D, modified where appropriate, and

include such tests as: bench handling, vibration, low and high temperature, and

humidity.

The operating tests involve running the system for an extended period of time under

varying temperatures and voltages. This period of operation ensures with confidence

that the hardware meets or exceeds the minimum requirements for reliability, data

reading, and processing accuracy contained in Section 3. The procedure emphasizes

equipment operability and data accuracy; it is not an exhaustive evaluation of all

9.123
Volume I — Performance Standards

system functions. Moreover, the severity of the test conditions, in most cases, has been

reduced from that specified in the Military Standards to reflect commercial and

industrial, rather than military and aerospace, practice.

9.4.1.3 Focus of Software Evaluation

The software qualification tests encompass a number of interrelated examinations,

involving assessment of application source code for its compliance with the

requirements spelled out in Volume I, Section 4. Essentially, the ITA will look at

programming completeness, consistency, correctness, modifiability, structuredness and

traceability, along with its modularity and construction. The code inspection will be

followed by a series of functional tests to verify the proper performance of all system

functions controlled by the software.

The ITA may inspect COTS generated software source code in the preparation of test

plans and to provide some minimal scanning or sampling to check for embedded code

or unauthorized changes. Otherwise, the COTS source code is not subject to the full

code review and testing. For purposes of code analysis, the COTS units shall be treated

as unexpanded macros.

9.4.1.4 Focus of System-Level Integration Tests

The functionality, hardware, and software qualification tests supplement a fuller

evaluation performed by the system-level integration tests. System-level tests focus on

these aspects jointly, throughout the full range of system operations. They include tests

of fully integrated system components, internal and external system interfaces,

usability and accessibility, and security. During this process election management

functions, ballot-counting logic, and system capacity are exercised. The process also

includes the Physical Configuration Audit (PCA) and the Functional Configuration

Audit (FCA).

The ITA tests the interface of all system modules and subsystems with each other

against the vendor's specifications. Some, but not all, systems use telecommunications

capabilities as defined in Section 5. For those systems that do use such capabilities,

components that are located at the poll site or separate vote counting site are tested for

effective interface, accurate vote transmission, failure detection, and failure recovery.

For voting systems that use telecommunications lines or networks that are not under

the control of the vendor (e.g., public telephone networks), the ITA tests the interface

of vendor-supplied components with these external components for effective interface,

vote transmission, failure detection, and failure recovery.

The security tests focus on the ability of the system to detect, prevent, log, and recover

from a broad range of security risks as identified in Section 6. The range of risks tested

is determined by the design of the system and potential exposure to risk. Regardless of

system design and risk profile, all systems are tested for effective access control and

physical data security. For systems that use public telecommunications networks, to

9 124
Volume I — Performance Standards

transmit election management data or official election results (such as ballots or

tabulated results), security tests are conducted to ensure that the system provides the

necessary identity-proofing, confidentiality, and integrity of transmitted data. The tests

determine if the system is capable of detecting, logging, preventing, and recovering

from types of attacks known at the time the system is submitted for qualification. The

ITA may meet these testing requirements by confirming the proper implementation of

proven commercial security software.

The interface between the voting system and its users, both voters and election

officials, is a key element of effective system operation and confidence in the system.

At this time, general standards for the usability of voting systems by the average voter

and election officials have not been defined, but are to be addressed in the next update

of the Standards. However, standards for usability by individual voters with disabilities

have been defined in Section 2.7 based on Section 508 of the Rehabilitation Act

Amendments of 1998. Voting systems are tested to ensure that an accessible voting

station is included in the system configuration and that its design and operation

conforms with these standards.

The Physical Configuration Audit (PCA) compares the voting system components

submitted for qualification to the vendor's technical documentation and confirms that

the documentation submitted meets the requirements of the Standards. As part of the

PCA, the ITA also witnesses the buildof the executable system to ensure that the

qualified executable release is built from the tested components.

The Functional Configuration Audit (FCA) is an exhaustive verification of every

system function and combination of functions cited in the vendors' documentation.

Through use, the FCA verifies the accuracy and completeness of the system's TDP.

The various options of software counting logic that are claimed in the vendor's

documentation shall be tested during the system-level FCA. Generic test ballots or test

entry data for DRE systems, representing particular sequences of ballot-counting

events, will test the counting logic during this audit.

9.4.1.5 Focus of Vendor Documentation Examination

The ITA reviews the documentation submitted by the vendor to evaluate the extent to

which it conforms to the requirements outlined in Sections 7 and 8 for vendor

configuration and quality assurance practices. The ITA also evaluates the conformance

of other documentation and information provided by the vendor with the vendor's

documented practices for quality assurance and configuration management.

The Standards do not require on-site examination of the vendor's quality assurance and

configuration management practices during the system development process. However,

the ITA conducts several activities while at the vendor site to witness the system build

that enable assessment of the vendor's quality assurance and configuration

management practices and conformance with them. These include surveys, interviews

with individuals at all levels of the development team, and examination of selected

internal work products such as system change requests and problem tracking logs.

9125
Volume I — Performance Standards

9.4.2 Sequence of Tests and Audits

There is no required sequence for performing the system qualification tests and audits.

For a new system, not previously qualified, a test using the generic test ballot decks

might be performed before undertaking any of the more lengthy and expensive tests or

documentation review. The ITA or vendor may, however, schedule the PCA, FCA, or

other tests in any convenient order, provided that the prerequisite conditions for each

test have been met before it is initiated.

9.5 Test Applicability

Qualification tests are conducted for new systems seeking initial qualification as well

as for systems that are modified after qualification.

9.5.1 General Applicability

Voting system hardware, software, communications and documentation are examined

and tested to determine suitability for elections use. Examination and testing addresses

the broad range of system functionality and components, including system

functionality for pre-voting, voting, and post-voting functions described in Section 2.

All products custom designed for election use shall be tested in accordance with the

applicable procedures contained in this section. COTS hardware, system software and

communications components with proven performance in commercial applications

other than elections, however, are exempted from certain portions of the test as long as

such products are not modified for use in a voting system. Compatibility of these

products all other components of the voting system shall be determined through

functional tests integrating these products with the remainder of the system.

9.5.1.1 Hardware

Specifically, the hardware test requirements shall apply in full to all equipment used in

a voting system with the exception of the following:

a. Commercially available models of general purpose information technology

equipment that have been designed to an ANSI or IEEE standard, have a

documented history of successful performance for relevant requirements of the

standards, and have demonstrated compatibility with the voting system

components with which they interface;

9.126
Volume I — Performance Standards

b. Production models of special purpose information technology equipment that

have a documented history of successful performance under conditions

equivalent to election use for relevant requirements of the standards and that

have demonstrated compatibility with the voting system components with

which they interface; and

c. Any ancillary devices that do not perform ballot definition, election database

maintenance, ballot reading, ballot data processing, or the production of an

official output report; and that do not interact with these system functions

(e.g.; modems used to broadcast results to the press, printers used to generate

unofficial reports, or CRTs used to monitor the vote counting process).

This equipment shall be subject to functional and operating tests performed during

software evaluation and system-level testing. However, it need not undergo hardware

non-operating tests. If the system is composed entirely of off-the-shelf hardware, then

the system also shall not be subject to the 48-hour environmental chamber segment of

the hardware operating tests.

9.5.1.2 Software

Software qualification is applicable to the following:

a. Application programs that control and carry out ballot processing, commencing with the definition of a ballot, and including processing of the ballot

image (either from physical ballots or electronically activated images), and

ending with the system's access to memory for the generation of output

reports;

b. Specialized compilers and specialized operating systems associated with ballot

processing; and

c. Standard compilers and operating systems that have been modified for use in

the vote counting process.

Specialized software for ballot preparation, election programming, vote recording, vote

tabulation, vote consolidation and reporting, and audit trail production shall be

subjected to code inspection. Functional testing of all these programs during software

evaluation and system-level testing shall exercise any specially tailored software offline from the ballot counting process (e.g.; software for preparing ballots and

broadcasting results).

9.5.2 Modifications to Qualified Systems

Changes introduced after the system has completed qualification under these Standards

or earlier versions of the national Voting System Standards will necessitate further

review.

9.127
Volume I — Performance Standards

9.5.2.1 General Requirements for Modifications

The ITA will determine tests necessary for to qualify the modified system based on a

review of the nature and scope of changes, and other submitted information including

the system documentation, vendor test documentation, configuration management

records, and quality assurance information. Based on this review, the ITA may:

a. Determine that a review of all change documentation against the baseline

materials is sufficient for recommendation for qualification; or

b. Determine that all changes must be retested against the previously qualified

version (this will include review of changes to source code, review of all

updates to the TDP, and a performance of system-level and functional tests);

or

c. Determine that the scope of the changes is substantial and will require a

complete retest of the hardware, software, and/or telecommunications.

9.5.2.2 Basis for Limited Testing Determinations

The ITA may determine that a modified system will be subject only to limited

qualification testing if the vendor demonstrates that the change does not affect

demonstrated compliance with these Standards for:

a. Performance of voting system functions;

b. Voting system security and privacy;

c. Overall flow of system control; and

d. The manner in which ballots are defined and interpreted, or voting data are

processed.

Limited qualification testing is intended to facilitate the correction of defects, the

incorporation of improvements, the enhancement of portability and flexibility, and the

integration of vote-counting software with other systems and election software.

9.6 Qualification Test Process

The qualification test process may be performed by one or more ITAs that together

perform the full scope of tests required by the Standards. Where multiple ITAs are

involved, testing shall be conducted first for the voting system hardware, firmware,

and related documentation; then for the system software and communications; and

finally for the integrated system as a whole. Voting system hardware and firmware

testing may be performed by one ITA independently of the other testing performed by

9128
Volume I — Performance Standards

other ITAs. Testing may be coordinated across ITAs so that hardware/firmware tested

by one ITA can be used in the overall system tests performed by another ITA.

Whether one or more ITAs are used, the testing generally consists of three phases:

♦ Pre-test Activities;

♦ Qualification Testing; and

♦ Qualification Report Issuance and Post-test Activities.

9.6.1 Pre-test Activities

Pre-test activities include the request for initiation of testing and the pre-test

preparation.

9.6.1.1 Initiation of Testing

Qualification testing shall be conducted at the request of the vendor, consistent with

the provision of the Standards. The vendor shall:

a. Request the performance of qualification testing from among the certified

ITAs,

b. Enter into formal agreement with the ITAs for the performance of testing, and

c. Prepare and submit materials required for testing consistent with the

requirements of the Standards.

Qualification testing shall be conducted for the initial version of a voting system as

well as for all subsequent changes to the system prior to release for sale or for

installation. As described in Section 9.5.2, the nature and scope of testing for system

changes or new versions shall be determined by the ITA based on the nature and scope

of the modifications to the system and on the quality of system documentation and

configuration management records submitted by the vendor.

9.6.1.2 Pre-test Preparation

Pre-test preparation encompasses the following activities:

a. The vendor shall prepare and submit a complete TDP to the ITA. The TDP

should consist of the items listed in Section 9.2 and specified in greater detail

in Standards Volume II;

9-129
Volume I — Performance Standards

b. The ITA shall perform an initial review of the TDP for completeness and

clarity and request additional information as required;

c. The vendor shall provide additional information, if requested by the ITA;

d. The vendor and ITA shall enter into an agreement for the testing to be

performed by the ITA in exchange for payment by the vendor; and

e. The vendor shall deliver to the ITA all hardware and software needed to

perform testing.

9.6.2 Qualification Testing

Qualification testing encompasses the preparation of a test plan, the establishment of

the appropriate test conditions, the use of appropriate test fixtures, the witness of the

system build and installation, the maintenance of qualification test data, and the

evaluation of the data resulting from tests and examinations.

9.6.2.1 Qualification Test Plan

The ITA shall prepare a Qualification Test Plan to define all tests and procedures

required to demonstrate compliance with Standards, including:

a. Verifying or checking equipment operational status by means of manufacturer

operating procedures;

b. Establishing the test environment or the special environment required to

perform the test;

c. Initiating and completing operating modes or conditions necessary to evaluate

the specific performance characteristic under test;

d. Measuring and recording the value or range of values for the characteristic to

be tested, demonstrating expected performance levels;

e. Verifying, as above, that the equipment is still in normal condition and status

after all required measurements have been obtained;

f. Confirming that documentation submitted by the vendor corresponds to the

actual configuration and operation of the system; and

g. Confirming that documented vendor practices for quality assurance and

configuration management comply with the Standards.

A recommended outline for the test plan and the details of required testing are

contained in Standards Volume II.

9130
Volume I — Performance Standards

9.6.2.2 Qualification Test Conditions

The ITA may perform Qualification tests in any facility capable of supporting the test

environment. The following practices shall be employed:

a. Preparations for testing, arrangement of equipment, verification of equipment

status, and the execution of procedures shall be witnessed by at least one

independent, qualified observer, who shall certify that all test and data

acquisition requirements have been satisfied;

b. When a test is to be performed at "standard" or "ambient" conditions, this

requirement shall refer to a nominal laboratory or office environment, with a

temperature in the range of 68 to 75 degrees Fahrenheit, and prevailing

atmospheric pressure and relative humidity; and

c. Otherwise, all tests shall be performed at the required temperature and

electrical supply voltage, regulated within the following tolerances:

1) Temperature +/- 4 degrees F

2) Electrical supply voltage +/- 2 vac.

9.6.2.3 Qualification Test Fixtures

ITAs may use test fixtures or ancillary devices to facilitate qualification testing. These

fixtures and devices may include arrangements for automating the operation of voting

devices and the acquisition of test data:

a. For systems that use a light source as a means of detecting voter selections, the

generation of a suitable optical signal by an external device is acceptable. For

systems that rely on the physical activation of a switch, a mechanical fixture

with suitable motion generators is acceptable;

b. ITAs may use a simulation device, and appropriate software, to speed up the

process of testing and eliminate human error in casting test ballots, provided

that the simulation covers all voting data detection and control paths that are

used in casting an actual ballot. In the event that only partial simulation is

achieved, then an independent method and test procedure shall be used to

validate the proper operation of those portions of the system not tested by the

simulator; and

c. If the vendor provides a means of simulating the casting of ballots, the

simulation device is subject to the same performance, reliability, and quality

requirements that apply to the voting device itself.

9-131
Volume I — Performance Standards

9.6.2.4 Witness of System Build and Installation

Although most testing is conducted at facilities operated by the ITA, a key element of

voting system testing shall be conducted at the vendor site. The ITA responsible for

testing voting system software, telecommunications, and integrated system operation

(i.e., system wide testing) shall witness the final system build, encompassing hardware,

software and communications, and the version of associated records and

documentation. The system elements witnessed, including their specific versions, shall

become the specific system version that is recommended for qualification.

9.6.2.5 Qualification Test Data Requirements

The following qualification test data practices shall be employed:

a. A test log of the procedure shall be maintained. This log shall identify the

system and equipment by model and serial number;

b. Test environment conditions shall be noted; and

c. All operating steps, the identity and quantity of simulated ballots, annotations

of output reports, the elapsed time for each procedure step, and observations of

equipment performance and, in the case of non-operating hardware tests, the

condition of the equipment shall be recorded.

9.6.2.6 Qualification Test Practices

The ITA shall conduct the examinations and tests defined in the Test Plan such that all

applicable tests identified in Standards Volume II are executed to determine

compliance with the requirements in Sections 2-8 of the Standards. The ITA shall

evaluate data resulting from examinations and tests, employing the following practices:

a. If any malfunction or data error is detected that would be classified as a

relevant failure using the criteria in Volume II, its occurrence, and the duration

of operating time preceding it, shall be recorded for inclusion in the analysis of

data obtained from the test, and the test shall be interrupted;

b. If a malfunction is due to a defect in software, then the test shall be terminated

and system returned to the vendor for correction;

c. If the malfunction is other than a software defect, and if corrective action is

taken to restore the equipment to a fully operational condition within 8 hours,

then the test may be resumed at the point of suspension;

d. If the test is suspended for an extended period of time, the ITA shall maintain

a record of the procedures that have been satisfactorily completed. When

testing is resumed at a later date, repetition of the successfully completed

9-132
Volume I — Performance Standards

procedures may be waived, provided that no design or manufacturing change

has been made that would invalidate the earlier test results;

e. Any and all failures that occurred as a result of a deficiency shall be classified

as purged, and test results shall be evaluated as though the failure or failures

had not occurred, if the:

1) Vendor submits a design, manufacturing, or packaging change notice to

correct the deficiency, together with test data to verify the adequacy of the

change;

2) Examiner of the equipment agrees that the proposed change will correct

the deficiency; and

3) Vendor certifies that the change will be incorporated into all existing and

future production units; and

f. If corrective action cannot be successfully taken as defined above, then the

test shall be terminated, and the equipment shall be rejected.

9.6.3 Qualification Report Issuance and Post-test Activities

Qualification report issuance and post-test activities encompass the activities described

below:

a. The ITA may issue interim reports to the vendor, informing the vendor of the

testing status, findings to date, and other information. Such reports do not

constitute official test reports for voting system qualification;

b. The ITA shall prepare a Qualification Test Report that confirms the voting has

passed the testing conducted by the ITA. The ITA shall include in the

Qualification Test Report the date testing was completed, the specific system

version addressed by the report, the version numbers of all system elements

separately identified with a version number by the vendor, and the scope of

tests conducted. A recommended outline for the test report is contained in

Volume II;

c. Where a system is tested by multiple ITAs, each ITA shall prepare a

Qualification Test Report;

d. The ITA shall deliver the Qualification Test Report to the vendor and to

NASED;

e. NASED shall issue a single Qualification Number for the system to the vendor

and to the ITAs. The issuance of a Qualification Number indicates that the

system has been tested by certified ITAs for compliance with the national test

standards and qualifies for the certification process of states that have adopted

the national standards;

9.133
Volume I — Performance Standards

f. This number applies to the system as a whole only for the configuration and

versions of the system elements tested by the ITAs and identified in the

Qualification Test Reports. The Qualification Number does not apply to

individual system components or untested configurations; and

g. The Qualification Number is intended for use by the states and their

jurisdictions to support state and jurisdiction processes concerning voting

systems. States and their jurisdictions shall request ITA Qualification Test

Reports based on the Qualification Number as part of their voting system

certification and procurement processes systems that rely on the Standards.

9.6.4 Resolution of Testing Issues

The NASED Voting Systems Board (the Board) is responsible for resolving questions

about the application of the Standards in the testing of voting systems. The Secretariat

for the Board will relay its decisions to the NASED certified ITAs and voting system

vendors. The Federal Election Commission will monitor these decisions in order to

determine which of them, if any, should be reflected in a subsequent version of the

standards.

9.134
Volume I — Performance Standards

10 Appendix A - Glossary

Absentee Ballot

Acceptance Test

Ballot Configuration

Ballot Format

A ballots cast by a voter unable to vote in person at his or her polling

place on election day.

The examination of a voting system and its components by the purchasing

election authority (usually in a simulated-use environment) to validate

performance of delivered units in accordance with procurement

requirements, and to validate that the delivered system is, in fact, the

certified or qualified system purchased. Testing to validate performance

may be less broad than that involved with qualification testing and

successful performance for multiple units (precinct count systems) may

be inferred from a sample test.

The combination of contests, ballot measures, or both that is unique to a

particular political subdivision, precinct, or portion of a precinct (for split

precincts) in a particular election. Typically, in primary elections, there

are separate ballot configurations for each participating political party and

for nonpartisan races and ballot issues. Depending on state law and

practice, contests for federal, state, and local office may be presented in

separate ballot configurations or combined into a single ballot

configuration.

A counter in a voting device that counts the ballots cast in a single

election or election test. Previously known as public counter.

The software logic that defines the combinations of voter choices that are

valid and invalid on a given ballot and that determines how the vote

choices are totaled in a given election. States differ from each other in the

way they define valid and invalid votes and in their vote counting

procedures. For example, voters in some States are permitted to both

select the straight party option and vote "by exception" for candidates

from a different political party. Voters in other States that choose the

straight party option and any candidates from a different party for some

contests will be considered to have overvoted in those contests.

One of any number of specific ballot configurations issued to the

appropriate precinct. At minimum, ballot formats differ from one another

in content. They may also differ in size of type, in language used, or in

method of presentation (e.g.; visual or audio). Also referred to as "ballot

style."

Ballot Counter

Ballot Counting Logic

10135
Volume I — Performance Standards

Ballot Image

Ballot Pr paration

An electronically produced record of all votes cast by a single voter. (Also

referred to as "ballot set").

The process of using election databases to select the specific contests and

questions to be contained in a ballot format and related instructions;

preparing election specific software containing these selections; producing

all possible ballot formats (or styles); and validating the correctness of

ballot materials and software containing these selections for an upcoming

election.

Ball t Production The process of converting the ballot format to a media ready for use in the

physical ballot production or electronic presentation.

Ballot Rotation The process of varying the order of the candidate names within a given

contest to reduce the impact of voter bias towards the candidate(s) listed

first. States that require ballot rotation may do so for primary elections,

general elections, or both. States may rotate the names according to a

number of different formulas including by political subdivision, by election

district, by precinct, or by ballot displays or voting machines.

Ballot Set See "Ballot Image."

Ballot Scanner A device used to read the data from a marksense ballot

Ballot Style One of any number of specific ballot configurations issued to the

appropriate precinct. At minimum, ballot styles differ from one another in

content. They may also differ in size of type, in language used, or in

method of presentation (e.g.; visual or audio). Also referred to as "ballot

format."

A product configuration that has been formally submitted for review

against the the Standards, which thereafter serves as the basis for further

development; and can be changed and offered to jurisdictions only

through formal change control and requalification procedures (and/or

recertification procedures where applicable). (Patterned after IEEE Std.

610.12-1990)

The record that reflects the total votes cast for the candidate. This record

is augmented as each ballot is cast on a DRE or as digital signals from the

conversion of voted paper ballots are logically interpreted and recorded.

A compilation of election returns and validation of the outcome that form

the basis of the official results.

A total loss of function or functions, such as the loss or unrecoverable

corruption of voting data, or the failure of an on-board battery for volatile

memory.

The state examination, and possibly testing, of a voting system to

determine its compliance with state laws, regulations, and rules and any

other state requirements for voting systems.

A ballot provided to individuals whose eligibility to vote has been

challenged. Once voted, such ballots are not included in the tabulation

until after the voter's eligibility is confirmed.

Baseline

Candidate Register

Canvass

Catastrophic System Failure

C rtification Testing

Challeng d Ballot

10-136
Volume I — Performance Standards

Closed Primary

Commercial Off-the-Shelf

(COTS)

Component

Configuration Identification

Configuration Item

Configuration Management

Configuration Status

Accounting

COTS

Count

Cross-party Endorsement

A primary election in which voters receive a ballot listing only those

candidates running for office in the political party with which the voters

are affiliated, along with nonpartisan offices and ballot issues presented at

the same election. Usually, unaffiliated voters are permitted to vote only

on nonpartisan offices and ballot issues. In some cases, one or more

political parties within a state may allow unaffiliated voters to choose to

vote in their party's primary.

Commercial, readily-available hardware devices (such as card readers,

printers, or personal computers) or software products (such as operating

systems, programming language compilers, or database management

systems). These devices and software are exempted from certain portions

of the qualification testing process so long as such products are not

modified in any manner for use in the voting system.

Individual elements or items that collectively comprise a device. Examples

include circuit boards, internal modems, processors, disk drives,

computer memory.

An element of configuration management, consisting of selecting the

configuration items for a system and recording their functional and

physical characteristics in technical documentation. (Patterned after IEEE

Std. 610.12.1990)

An aggregation of hardware, software, or both that is designated for

configuration management and treated as a single entity in the

configuration management process. (Patterned after IEEE Std. 610.12-

1990)

A discipline applying technical and administrative direction and

surveillance to: identify and document functional and physical

characteristics of a configuration item, control changes to these

characteristics, record and report change processing and implementation

status, and verify compliance with specified requirements. (Patterned

after IEEE Std. 610.12-1990)

An element of configuration management, consisting of the recording and

reporting of information needed to manage a configuration effectively.

This information includes a listing of the approved configuration

identification, the status of proposed changes to the configuration, and

the implementation status of approved changes. (Patterned after IEEE

Std. 610.12.1990)

See "Commercial Off-the-Shelf."

The process of totaling votes.

The endorsement of a single candidate or slate of candidates by more

than one political party. The candidate or slate appears on the ballot

representing each endorsing political party. State requirements vary for

how votes are recorded when a voter selects the same candidate or slate

more than once. Also referred to as "cross filing."

10-137
Volume I — Performance Standards

Cumulative Voting

Data Accuracy

Data Integrity

Election Coding

Election Databases

Election District

Election Management System

Election Programming

Functional Configuration Audit

(FCA)

A practice where voters are permitted to cast as many votes as there are

seats to be filled. Voters are not limited to giving only one vote to a

candidate. Instead, they can put multiple votes on one or more

candidates. (For additional information, access the Center for Voting and

Democracy's web site at http://www.fairvote.org/contents.htm#irv.)

The system's ability to process voting data absent internal errors

generated by the system. It is distinguished from data integrity, which

encompasses errors introduced by an outside source.

The invulnerability of the system to accidental intervention or deliberate,

fraudulent manipulation that would result in errors in the processing of

data. It is distinguished from data accuracy that encompasses internal,

system-generated errors.

A functional unit that performs its assigned tasks as an integrated whole.

• A voting system that records votes by means of a ballot display provided

with mechanical or electro-optical components that can be actuated by

the voter; that processes the data by means of a computer program; and

that records voting data and ballot images in internal and/or external

memory components. It produces a tabulation of the voting data stored in

a removable memory component and in printed copy.

See "Election Programming."

A data file or set of files that contains geographic information about

political subdivisions and boundaries; all contests and questions to be

included in an election; and the candidates for each contest.

A contiguous geographic area represented by a public official who is

elected by voters residing within the district boundaries. The district may

cover an entire state or political subdivision, may be a portion of the state

or political subdivision, or may include portions of more than one political

subdivision.

A set of processing functions and databases within a Voting System that

define, develop and maintain election databases; perform election

definition and setup functions; format ballots; count votes; consolidate

and report results; and maintain audit trails.

The process by which election officials or their designees use voting

system software to logically define the ballot for a specific election. Also

referred to as "election coding."

An acronym for the Federal Election Commission.

Computer programs (software) stored in read-only memory (ROM) devices

embedded in the system and not capable of being altered during system

operation. For purposes of applying the Standards, firmware is

considered a form of software.

An exhaustive verification of every system function and combination of

functions cited in the vendors' documentation. Through use, the FCA

verifies the accuracy and completeness of the system's Voter Manual,

Operations Procedures, Maintenance Procedures, and Diagnostic Testing

Procedures.

Device

Direct Record Electronic (DRE)

Voting System

FEC

Firmware

10.138
Volume I — Performance Standards

Functional T st A test performed to verify or validate the accomplishment of a function or

a series of functions.

G n ral Election An election in which voters, regardless of party affiliation, are permitted to

select persons to fill public office and vote on ballot issues. Where the

public office may be filled by a candidate affiliated with a political party,

voters choose among the nominees of political parties and, as permitted

by state law, unaffiliated candidates.

ITA An acronym for Independent Test Authority.

A condition signifying that, for a given input, a computer program will

satisfy the program specification (produce the required output).

A system by which votes are recorded by means of marks made in voting

response fields designated on one or both faces of a ballot card or series

of cards. Marksense systems use a ballot scanner to read the ballots.

The record that reflects the total votes cast for and against a specific

ballot issue. This record is augmented as each ballot is cast on a DRE or

as digital signals from the conversion of voted paper ballots are logically

interpreted and recorded.

An elected office for which candidates run independent of political party

affiliation.

Memory in which information can be stored indefinitely with no power

applied. ROMs and EPROMs are examples of nonvolatile memory.

A primary election in which voters, regardless of political affiliation, may

choose in which party's primary they will vote. Some states require

voters to publicly declare their choice of party ballot at the polling place,

after which the poll worker provides or activates the appropriate ballot.

Other states allow the voters to make their choice of party ballot within

the privacy of the voting booth. Voters also are permitted to vote on

nonpartisan offices and ballot issues that are presented at the same

election.

The generally prohibited practice of voting for more than the allotted

number of candidates for the office being contested.

A voting system referred to in the 1990 Standards as a Punchcard and

Marksense (P&M) Voting System that records votes, counts votes, and

produces a tabulation of the vote count, using one or more ballot cards.

An elected office for which candidates run as representatives of a political

party.

An inspection that compares the voting system components submitted for

qualification to the vendor's technical documentation and confirms that

the documentation submitted meets the requirements of the Standards.

As part of the PCA, the ITA also witnesses the building of the executable

system to ensure that the qualified executable release is built from the

tested components.

Logical Correctness

Marksense Voting System

Measure Register

Non-partisan Office

Nonvolatile Memory

Open Primary

Overvotes

Paper-Based Voting System

Partisan Office

Physical Configuration Audit

(PCA)

10.139
Volume I — Performance Standards

Political Subdivision Any unit of government, such as counties and cities but often excepting

school districts, having authority to hold elections for public offices or on

ballot issues.

Polling Location The physical address of a polling place.

Polling Place The area within the polling location where voters cast ballots.

Precinct An administrative division representing a contiguous geographic area in

which voters cast ballots at the same polling place. Voters casting

absentee ballots may also be combined into one or more administrative

absentee precincts for purposes of tabulating and reporting votes.

Generally, voters in a polling place precinct are eligible to vote in a

general election using the same ballot format. In some jurisdictions,

however, the ballot formats may be different due to split precincts or

required ballot rotations within the precinct.

Primary Election In most cases, an election held to determine which candidate will

represent a political party in the general election. During presidential

election years, voters in primary elections may also select delegates to

presidential nominating conventions. Some states have an "open

primary", while others have a "closed primary". Sometimes elections for

nonpartisan offices and ballot issues are held during primary elections.

Primary Pr sidential Delegation A primary election in which voters choose the delegates to the

Nominations Presidential nominating conventions allotted to their state by the national

party committees.

Provisional Ballot A ballot provided to individuals who claim they are eligible to vote but

whose eligibility cannot be confirmed when they present themselves to

vote. Once voted, such ballots are not included in the tabulation until

after the voter's eligibility is confirmed.

Public Network Direct Record A form of DRE voting system that uses electronic ballots and transmits

Electronic (DRE) Voting System official vote data from the polling place to another location (such as a

central count facility) over a public network beyond the control of the

election authority. These networks include public telephone lines and the

Internet.

Punchcard Voting System

Qualification Number

Qualificati n Test Report

A voting system where votes are recorded by means of punches made in

voting response fields designated on one or both faces of a ballot card or

series of cards.

A number issued by NASED to a system that has been tested by certified

Independent Test Authorities for compliance with the qualification test

standards. The issuance of a Qualification Number indicates that the

system qualifies for certification process of states that have adopted the

Standards.

A report of results of independent testing of a voting system by an

Independent Test Authority indicating the date testing was completed, the

specific system version tested, and the scope of tests conducted

10-140
Volume I – Performance Standards

Qualification Testing

Ranked Order Voting

Recall Issues (with Options)

Recertification

Runoff Election

Split Precinct

The examination and testing of a computerized voting system by an

Independent Test Authority using qualification test standards to

determine if the system complies with the qualification performance and

test standards and with its own specifications. This process occurs prior

to state certification.

A practice that allows voters to rank candidates in a contest in order of

choice: 1, 2, 3 and so on. It takes a majority to win. If anyone receives a

majority of the first choice votes, that candidate wins that election. If not,

the last place candidate is deleted, and all ballots are counted again, but

this time each ballot cast for the deleted candidate counts for the next

choice candidate listed on the ballot. The process of eliminating the last

place candidate and recounting the ballots continues until one candidate

receives a majority of the vote. The practice is also known as instant

runoff voting, preferences or preferential voting, or choice voting. (For

additional information, access the Center for Voting and Democracy's web

site at http://www.fairvote.org/contents.htm#irv.)

The process that allows voters to remove their elected representatives

from office prior to the expiration of their terms of office. Often, the recall

involves not only the question of whether a particular officer should be

removed from office, but also the question of naming a successor in the

event that there is an affirmative vote for the recall. There are no

provisions for the recall of federal office holders.

The state examination, and possibly the retesting, of a voting system that

was modified subsequent to receiving state certification. The object of

this process is to determine if the modification still permits the system to

function properly in accordance with state requirements.

An election to select a winner following a primary, or sometimes a general

election, in which no candidate in the contest received the required

minimum percentage of the votes cast. The two candidates receiving the

most votes for the race in question proceed to the runoff election.

A split precinct is a precinct containing more than one ballot format in

order to accommodate a contiguous geographic area served by the

precinct that contains more than one election district.

A mechanism by which voters are permitted to cast a vote indicating the

selection of all candidates on the ballot for a single political party.

Software that aids in the development or maintenance of other software,

for example compilers, loaders and other utilities. (Patterned after IEEE

Std. 610.12-1990)

See "Count."

The practice of voting for less than the total number of election contests

listed on the ballot, or of voting for less than the number of positions to

be filled for a single office. (i.e. A person would undervote if a contest

required the selection of 3 out of a given number of candidates, and the

voter chose only two candidates).

Straight Party Voting

Support Software

Tabulation

Undervotes

10-141
Volume I – Performance Standards

Validation

V rification

Vote for N of M

Voter Registration System

Write-in Voting

The process of evaluating a system or component during or at the end of

the development process to determine whether it satisfies specified

requirements. (Patterned after IEEE Std. 610.12-1990)

The process of evaluating a system or component to determine whether

the products of a given development phase satisfy the conditions (such as

specifications) imposed at the start of that phase. (Patterned after IEEE

Std. 610.12-1990)

A ballot choice in which voters are required to vote for a limited number

of candidates for a single office from a larger field of candidates. For

example, in an election for six open city council seats, voters may be told

that they can vote for six out of twelve candidates actually listed on the

ballot.

A set of processing functions and data storage that maintains records of

eligible voters. This system generally is not considered a part of a Voting

System subject to the Standards.

Specific response fields on a ballot where the voter indicates the selection

of a candidate or ballot proposition.

A location within the polling place where voters may record their votes. A

voting station includes the voting booth or enclosure and the voterecording device.

A means to cast a vote for an individual not listed on the ballot. Voters

may do this by using a marking device to physically write their choice on

the ballot or they may use a keypad, touchscreen or other electronic

means to indicate their choice.

Voting Position

Voting Station

10-142
Volume I — Performance Standards

11 Appendix B - Applicable

Documents

11.1 Documents Incorporated in the

Standards

The following publications have been incorporated into the Standards. When specific

provisions from these publications have been incorporated, specific references are

made in the body of the Standards.

Federal Regulations Code of Federal Regulations, Title 20, Part 1910, Occupational Safety and Health Act

Code of Federal Regulations, Title 36, Part 1194, Architectural and Transportation

Barriers Compliance Board, Electronic and Information Technology Standards - Final

Rule

Code of Federal Regulations, Title 47, Parts 15 and 18, Rules and Regulations of the

Federal Communications Commission

Code of Federal Regulations, Title 47, Part 15, "Radio Frequency Devices", Subpart J,

"Computing Devices", Rules and Regulations of the Federal Communications

Commission

American National

Standards Institute

(ANSI)

ANSI C63.4 Methods of Measurement of Radio-Noise Emissions from LowVoltage Electrical and Electronic Equipment in the Range of 9Khz

to 40 GHz

ANSI C63.19 American National Standard for Methods of Measurement of

Compatibility between Wireless Communication Devices and

Hearing Aids

11-143
Volume I - Performance Standards

Electromagnetic Compatibility (EMC) Part 4: Testing and

Measurement Techniques. Section 2 Electrostatic Discharge

Immunity Test (Basic EMC publication).

Electromagnetic Compatibility (EMC) Part 4: Testing and

Measurement Techniques. Section 3 Radiated Radio-Frequency

Electromagnetic Field Immunity Test.

Electromagnetic Compatibility (EMC) Part 4: Testing and

Measurement Techniques. Section 4 Electrical Fast Transient/Burst

Immunity Test.

Electromagnetic Compatibility (EMC) Part 4: Testing and

Measurement Techniques. Section 5 Surge Immunity Test.

Electromagnetic Compatibility (EMC) Part 4: Testing and

Measurement Techniques. Section 6 Immunity to Conducted

Disturbances Induced by Radio-Frequency Fields.

Electromagnetic Compatibility (EMC) Part 4: Testing and

Measurement Techniques. Section 8 Power-Frequency Magnetic

Field Immunity Test. (Basic EMC publication).

Electromagnetic Compatibility (EMC) Part 4: Testing and

Measurement Techniques. Section 11. Voltage Dips, Short

Interruptions and Voltage Variations Immunity Tests.

Electromagnetic compatibility (EMC) Part 5-7: Installation and

mitigation guidelines—Degrees of protection provided by enclosures

against electromagnetic disturbances

Security Requirements for Cryptographic Modules

Secure Hash Standard

Standard Security Label for Information Transfer

Entity Authentication Using Public Key Cryptography

Advanced Encryption Standard (AES) (Expected to become official

December 2001)

Software Development and Documentation Standard, 1989

Environmental Test Methods and Engineering Guidelines, 19 July

1983

International IEC 61000-4-2 (1995-

Electrotechnical 01)

Commission (IEC)

IEC 61000-4-3 (1996)

IEC 61000-4-4 (1995-

1)

IEC 61000-4-5 (1995-

2)

IEC 61000-4-6 (1996-

04)

IEC 61000-4-8 (1993-

06)

IEC 61000-4-11 (1994-

06)

IEC 61000-5-7 Ed. 1.0

b:2001

National Institute of

Standards and

Technology

FIPS 140-1

FIPS 180-1

FIPS 188

FIPS 196

FIPS (number TBD)

Military Standards MIL-STD-498

MIL-STD-810D (2)

11.2 Standards Development Documents

The following publications have been used for guidance in the revision of the

Standards.

11-144
Volume I - Performance Standards

ANSI/ISO/IEC TR

9294.1990

ISO/IEC TR 13335-

4:2000

ISO/IEC TR 13335-

3:1998

ISO/IEC TR 13335-

2:1997

ISO/IEC TR 13335-

1:1996

ISO 10007:1995

ISO 10005-1995

ANSI/ISO/ASQC

QS9000-3-1997

American National

Standards Institute

(ANSI)

International

Organization for

Standardization (ISO)

International Electrotechnical Commission

(IEC)

Electronic Industries

Alliance Standards

Information Technology Guidelines for the Management of Software

Documentation

Information technology—Guidelines for the management of IT

Security—Part 4: Selection of safeguards

Information technology—Guidelines for the management of IT

Security—Part 3 Techniques for the management of IT security

Information technology—Guidelines for the management of IT

Security—Part 2: Managing and planning IT security

Information technology—Guidelines for the management of IT

Security—Part 1: Concepts and models for IT security

Quality Mgmt. Guidelines for Configuration Management

Quality Mgmt. Guidelines for Quality Plans

QM and QA standards Part 3: Guidelines for the application of

ANSI/ISO/ASQC Q9000-1994 to the Development, Supply,

Installation, and Maintenance of Computer Software

MB2, MB5, MB9 Maintainability Bulletins

EIA 157 Quality Bulletin

EIA QB2-QB5 Quality Bulletins

EIA RB9 Failure Mode and Effect Analysis, Revision 71

EIA SEB1—SEB4 Safety Engineering Bulletins

RS-232-C Interface Between Data Terminal Equipment and Data

Communications Equipment Employing Serial Binary Data

Interchange

RS-366-A Interface Between Data Terminal Equipment and Automatic Calling

Equipment for Data Communication

RS-404 Standard for Start-Stop Signal Quality Between Data Terminal

Equipment and Non-synchronous Data Communication Equipment

National Institute of

Standards and

Technology

Institute of Electrical

and Electronics

Engineers

NISTIR 4909

610.12-1990

730-1998

828-1998

829-1998

830-1998

Software Quality Assurance: Documentation and Reviews

IEEE Standard Glossary of Software Engineering Terminology

IEEE Standard for Software Quality Assurance Plans

IEEE Standard for Software Configuration Management Plans

IEEE Standard for Software Test Documentation

IEEE Recommended Practice for Software Requirements

Specifications

11-145
Volume I - Performance Standards

Military Standards MIL-STD-498 Software Development and Documentation, 27 May 1998

11.3 Guidance Documents

The following publications contain information that is useful in understanding and

complying with the Standards.

ANSI/ISO/IEC TR

10176.1998

ANSI/ISO/IEC

6592.2000

ANSI/ISO/ASQC

Q9000-3-1997

ANSI/ISO/ASQC

Q9000-1-1994

ANSI/ISO/ASQC

Q10007-1995

Information Technology Guidelines for the Preparation of

Programming Language Standards

Information Technology Guidelines for the Documentation of

Computer Based Application Systems

Quality management and quality assurance standards Part 3:

Guidelines for the application of ANSI/IAO/ASQC Q9001-1994 to

the Development, supply, installation and maintenance of computer

software

Quality Management and Quality Assurance Standards—Guidelines

for Selection and Use

Quality Management Guidelines for Configuration Management

American National

Standards Institute

(ANSI)

International

Organization for

Standardization (ISO)

International Electrotechnical Commission

(IEC)

National Institute of

Standards and

Technology

Institute of Electrical

and Electronics

Engineers

Military Standards

FIPS 102

FIPS 112

FIPS 113

488-1987

796-1983

750.1-1995

1008-1987

1016-1998

1012-1998

MIL-HDBK-454

MIL-HDBK-470

MIL-STD-882

MIL-STD-1472

MIL-STD-973

Guideline for Computer Security Certification and Accreditation

Password Usage (3)

Computer Data Authentication

IEEE Standard Digital Interface for Programmable Instrumentation

IEEE Standard Microcomputer System Bus IEEE/ANSI Software

Engineering Standards

IEEE Guide for Software Quality Assurance Planning

IEEE Standard for Software Unit Testing

IEEE Recommended Practice for Software Design Descriptions

IEEE Guide for Software Verification and Validation Plans

Standard General Requirements for Electronic Equipment

Maintainability Program for Systems & Equipment

Systems Safety Program Requirements

Human Engineering Design Criteria for Military Systems, Equipment

and Facilities

Configuration Management, 30 September 2000

11-146
Volume I - Performance Standards

Other References Designing for the Color-Challenged: A Challenge, by Thomas G.

Wolfmaier (March 1999);

htto://www.sandia.qov/itqinewsletter/mar99/accessibility color challe

nqed.html; 

Effective Color Contrast: Designing for People with Partial Sight and

Color Deficiencies, by Aries Arditi, Ph.D;

http://www.liqhthouse.orq/color contrast.htm 

11147
-
Volume I — Performance Standards

12 Appendix C Usability

12.1 Scope

This appendix addresses the design of the voting system to meet the needs of the

voters, that is, to develop the interfaces between the voter and the system that are easy

to use and that minimize voter errors due to poor interface design. Depending on the

voting technology employed, the main elements of this interface are:

♦ Information displays, e.g., presentations of contests, candidates, propositions,

and instructions

♦ Vote input fields, e.g., the location where the voter indicates his or her

selection; and

♦ Navigation aids, e.g., the way that voters "move" from one part of the system

to another.

The most effective interfaces are almost transparent to the voter. They enable the

voter to devote his or her complete attention to the task at hand - voting for the

candidates and propositions of their choice. A good voter-voting system interface

guides the voter to appropriate behavior. It should be obvious to the voter what he or

she should do, and importantly, what seems obvious to the voter should be correct. To

the extent that the design confuses the voter or causes the voter to stop and think, for

example "where on this ballot do I place my vote" or "how do I change my vote,"

attention is directed away from the voter's main task and to the interface. At best, this

can lead to voter frustration. The voter must shift attention away from voting to

figuring out how to use the voting system. At worst, it can lead to errors such as

failing to vote for a contest, improperly indicating the vote so it is not counted, or

voting for more than the required number of candidates.

Designing effective and usable interfaces between the voter and the voting system

involves a number of activities. First, voter task requirements should be identified.

The requirements reflect the fact that the way that voters interact with the system is

different depending on the voting system technology. For example, the way a voter

casts a vote and navigates through a ballot will be quite different for a paper ballot

12-149
Volume I – Performance Standards

when compared to a computer-based voting system. These tasks need to be carefully

analyzed and addressed in the design.

Second, human factors design guidelines should be used to guide the interface design.

These guidelines have evolved from scientific research on the human performance

aspects of system design and from many years of application in the design of systems.

Their application can help to ensure that the design of voting systems is consistent

with and compatible with the physical and cognitive characteristics of the voting

public.

Third, usability tests and evaluations should be conducted to ensure the voting system

has achieved its design goals. In part, these tests can help verify that the design

features recommended in this Appendix are successfully implemented in the final

voting system design. Usability tests are based on the feedback and performance of

samples of voters and can help identify aspect of the design that may be unclear to

voters. Results from the tests and evaluations can be used to correct any design

deficiencies before the system are actually used for voting.

While all three activities are important, this appendix mainly addresses the second

activity discussed above. It provides guidance on the design of usable voter-voting

system interfaces based on human factors principles.

12.2 General Principles

The equipment used by voters to cast ballots should meet the following general

principles:

♦ The design should support voter tasks by providing alerts, information,

instructions, and controls when and where they are needed;

♦ The design should ensure compatibility with human physiological and

cognitive characteristics and limitations, including: visual and auditory

perception, information processing and memory, anthropometry and

biomechanics;

♦ The design should ensure voter safety. The potential for hazards such as sharp

edges, falling objects, pinch points, and electrical shock should be anticipated

and eliminated as much as possible from the design;

♦ Design conventions should be established to provide consistency and

standardization of the voter's interface with the system;

♦ The system should be the simplest design needed to meet its intended function;

♦ The design should provide guidance to the voter through the balloting process;

12-150
Volume I — Performance Standards

♦ The design should minimize voter inputs, e.g., don't add unnecessary steps,

minimize need to turn pages, and to navigate displays;

♦ The design should minimize attention shifts and interruptions, e.g., all

necessary information to cast a vote for a single race should in one place

without the need to turn pages or page to other screens; and

♦ Provisions should be made to accommodate the unique demands of all voters.

Additional criteria for accessibility that are mandated by the Standards is

discussed in 2.2.7.

12.3 Overall Design and Layout of the Voter

Workspace

The workspace is the booth, workstation, or other location provided by the election

district where the voter goes to use the paper, mechanical, or electronic systems

provided for voting.

The ballot and supporting system elements should be properly located so the displays

are directly within the voter's visual field and comfortably with the voter's reach. This

may require making the voter-system interface adjustable so voters can adjust the

interface for their unique demands. Alternatively, different workspaces can be

provided where equipment is positioned to accommodate voters of different heights,

voters who are in wheel chairs, and voters that have to sit.

All displays and controls should be located to avoid parallax. Parallax refers to the

apparent change in the relative positions of objects depending on the position of the

viewer. Error will be minimized if the distance between the displays and controls is

small, and if the ballot is located so that it can be viewed "straight-on," i.e., with the

observer's line of sight perpendicular to the plane of the ballot.

The ambient lighting provided should be consistent with the balloting technology used.

More lighting should be provided for paper ballots than for electronic ballots. When

VDU are used, ensure that lighting does not produce glare or reflections on screens.

Where both VDU and paper must be used, task lighting for reading paper should be

provided.

When ballots extend to more than one page (paper or electronic) the same general

organization layout should be used for all pages, i.e. location of page identifiers, page

numbers, items to be voted on, navigation aids, etc.

12.151
Volume I — Performance Standards

12.4 Ballot Legibility and Understandability

In order to facilitate usability, voting system designers should play close attention to

design elements that affect the voter's ability to clearly read and easily understand the

information provided. The following guidance addresses these design features:

a. The font size and style used should ensure that written material can be

easily and unambiguously read. Special provisions may be needed for

visually-impaired voters:

1) Text (except labels) should be presented using upper and lower case

characters. Reading text is easier and faster when capitalization is used

conventionally to start sentences and to indicate proper nouns and

acronyms;

2) A clearly legible font should be utilized. Fonts should have true

ascenders and descenders, uniform stroke width, and uniform aspect ratio.

Preference should be given to simple styles. Script and other highly

stylized fonts should be avoided;

3) For a given font, it should be possible to clearly distinguish between

the following characters: X and K, T and Y, I and L, I and 1, 0 and Q, 0

and 0, S and 5, and U and V;

4) Character size should be large enough to easily read the text from the

normal sitting and standing position without squinting or leaning forward

for a person with normal corrected vision (guidance for character size is

provided in Table C-1;

Table C-1 Approximate Point Sizes For Different Viewing

Distances

Viewing Distance Minimum Preferred

25 9 12

30 10 14

36 12 17

42 14 20

48 16 22

Note: Point sizes refer to the size of letters when printed. When viewed

on monitors, it is not exactly the same.

b. Instructions should be concise. Instructions should be designed to

communicate information clearly and unambiguously so that they can

be easily understood and interpreted without error:

1) Instructions should be available in the voter's preferred language (as

required by the Voting Rights Act of 1965);

12152
Volume I — Performance Standards

2) Instructions should be written as short sentences with short, simple

words;

3) Instruction steps should be written in active voice as positive

commands (focusing on what to do, not what not to do);

4) Punctuation should conform to standard usage of the language used;

5) Words, phrases, and names used in instructions should be used

consistently within and among instructions and all other the voting system

components;

6) Abbreviations and acronyms should be avoided or, if necessary,

limited to those well known to the voters;

7) If instructions include number lists, Arabic numerals should be used.

Numbers that are spelled out should be consistently spelled under the same

conditions;

8) The instructions should specify any conditions that must be met before

an action can be undertaken. Information about preconditions should be

located so that voters read the information before acting. Information

given in other locations may be overlooked, or require additional actions

to retrieve it, which may be distracting and time consuming. Further, if

conditions are implied, voters may easily miss or misinterpret them;

9) Applicable cautions or warnings should be displayed when the

relevant instructions are in view of the voter. Displaying warnings and

cautions at the same time as their associated instructions will help ensure

that voters read the information. Information provided elsewhere may be

overlooked, or may require retrieval by distracting and time-consuming

actions;

10) Cautions or warnings should be uniquely presented, so that they are

easily distinguished from each other and from other display elements; and

11) All supplementary information (such as explanatory figures) required

for a procedure step should be shown concurrently with the step;

c. Graphics should be simple and have an obvious meaning that is

consistent with population stereotypes (unless well known graphics

are used, the meaning of graphics should be tested in advance to

ensure the they communicate the intended message). Voter

understanding of graphics can be enhanced when the graphics are

accompanied by instructional labels. For example, if an arrow is used

to indicate where to vote it may be more clearly understood if the text

"To register your vote, click here";

12153
Volume I – Performance Standards

d. If the information is communicated by means of visual coding, such as

by color or shape, the following principles should be followed:

1) A limited number of codes should be used;

2) The meaning of code levels should be clearly presented to the voter;

3) Voters should be able to easily discriminate between the levels of the

code, e.g., the different colors; and

4) If the information being coded differs in importance, the code levels

should be mapped for salience, e.g., most salient display characteristics

should direct voter's attention to the most important information;

e. Decorative features with no information content should be minimized

since they can create distractions; and

f. All information (e.g., contest labels, candidate names, instructions,

graphics, and coding) should have good contrast against the

background.

12.5 Information Grouping

Proper use of information is facilitated by the application of grouping principles.

When information is presented in a display, people have a tendency to group elements

in the display based on how they are presented. It is far preferable to intentionally

design the voting system display for proper grouping than to leave it to chance. The

guidance in this section addresses these design considerations.

a. Information on the ballot should be grouped. A group should include

the following:

1) Candidates for a given office;

2) The office for which a group of candidates are running; and

3) Vote response fields;

b. Any applicable instructions pertinent to the specific vote, such as an

indication of the number of candidates to vote for;

c. Information on the ballot should reflect principles of grouping:

12-154
Volume I — Performance Standards

1) A group should be visually distinct, e.g., examples of techniques that can

be used to visually set apart a group include boarders and demarcations,

background color, and textures;

2) The office for which a group of candidates are running should be

prominently labeled; and

3) The names should be grouped so they appear together (not on separate

areas of the display or separate pages); and

4) There should be clear separations between groups. The separations

between the groups should be greater than the separation between the

items of information within a group.

12.6 Voting Input Fields

The design of the voting input field is as important as the presentation of information

itself. The design should make it clear where and how to vote and the system should

provide feedback that the vote was accepted by the system. The guidance in this

section addresses these design features.

a. Ballot should clearly indicate the action voters must take to cast a vote

and where the action must be made in order to vote for specific

candidates;

b. There should be a consistent relationship between names of the

candidates and where to cast a vote. For example, if the response field

where voters indicate their selection is located to the right of a

candidates name, it should always be located to the right of all

candidates names and never to the left or some alternative position.

The reason for this is that people are active information processors and

will abstract rules about the relationships between information

elements in the display. The rules then guide their subsequent

behavior. If the design in inconsistent, applying the rule leads to error.

Consistency, therefore, will help establish voter expectancy with

balloting systems and minimize errors;

c. Ballot should clearly indicate how many candidates are to be voted

for;

d. The design should support the ability of the voter to remain in visual

contact with the current options when in the act of casting their vote.

That is, the design should minimize as much as possible, the occlusion

of the current item being voted for by the voting action, such as when

12.155
Volume I — Performance Standards

the movement of the voter's hand to cast a vote blocks information on

the display;

e. Feedback on the voter's selection should be provided. It should be

clearly obvious to voters what they voted for. In paper ballots, this is

supported by clear grouping principles. In electronic systems, an

informative feedback message should be provided;

f. Voters should be able to review all their votes prior to final

submission. While this is easily to implement in electronic systems it

can be more difficult with some paper ballots, like punch cards. In

such cases, where possible, it is desirable to provide voters with easy

access to a punch card reader or similar device to check that their

votes were cast as intended;

g. Voters should be able to modify their votes at any time before

finalizing their voting session;

h. In electronic and computer-based systems, fields where voters have to

enter identifying information, if any, should be clearly labeled and the

place where the information is to go should be clearly visible;

In computer-based systems, the cursor should be automatically

positioned in the first data entry field and when the voter hits the

"enter/return" key, the cursor should automatically move to the next

data entry field;

j. Voters should be able to correct the information if mistakes are made;

and

k. In electronic and computer-based systems, voters should not have to

input identifying information more that once, e.g., if voters input their

names at the beginning of the voting session, they should not have to

repeat the input on subsequent pages.

12.7 Navigation and Manipulation of Ballots

As noted earlier, navigation and manipulation of ballots can be a distracting task that

shifts the voter's attention away from the voting task and, therefore, can increase the

probability of error. Therefore, careful attention has to be paid to the design of these

aspects of the voting system. The guidance provided in this section is intended to

minimize the demands of these activities.

a. Voters should be able to control the pace and sequence of their use of

the ballot. Voters should be able to freely move back and forth;

12156
Volume I — Performance Standards

b. The means by which voters navigate through the system should be

simple and not require complex or complicated actions (e.g., clicking

on a "Next Page" button rather than scrolling);

c. The display should provide orientation and landmark features to

support the voter in determining where they are in the ballot;

d. Navigation features should be provided that are distinct and should be

clearly separated from voting response fields;

e. Any cursors should be visually distinct and should not move beyond

the boundaries of the screen (become invisible);

f. The input device (such as a mouse) and cursor response to voter

movements should be as precise as needed to reliably enter a vote; and

g. The system should provide feedback to user inputs in less than a

second, but if processing takes longer, feedback should be provided

that the system is processing the voter's input.

12.8 Preventing and Minimizing Voter Errors

During the design of voting systems, it is important to anticipate the types of errors

voters may make so that features can be deigned to minimize voter errors and to

provide the means for voters to realize their errors and correct them. The guidance in

this section addresses these design considerations.

a. The system should provide clear and explicit instructions on what

procedures the user should follow throughout the voting process.

b. The system should check user inputs for acceptability, e.g., check for

inputs that seem to be in error (such as putting a Arabic number in a

name field) and alert the voter when such a situation exists;

c. When feasible, interlocks should prevent voters from voting for more

candidates than is permitted or from providing other types of

unacceptable voter inputs. When this occurs, voters should be alerted

as to what is incorrect;

d. The system should inform voters of items on the ballot that they have

not voted for. This should be done before the voter leaves the system.

Voters should be given the opportunity to compete their vote if they

choose to or they should be able to exit without voting for those they

omitted; and

12157
Volume I — Performance Standards

e. A means for correcting a vote response should be readily available.

For non-paper based systems, this should be built into the design of

the system. For paper-based system, procedures for undoing votes

should be available and voters should be explicitly told in advance

what they are and that information should be posted close to where

they will use the ballot.

12.9 Help and System Failure

The availability of useful help features can support voting system usability. Similarly,

the voter should be alerted to any system failures that may impact the proper recording

of votes or personal safety. The guidance in this section addresses these design

considerations.

a. Help should be available to support users with specific questions;

b. The system should provide voters with information on what to do if

the instructions provided are not understood;

c. Acceptable voter behavior should be clearly identified (e.g., whether

the voter can leave the booth, open a curtain, remove a ballot, etc.);

d. System messages should be informative and in "plain English" and

should not contain technical or jargon terms;

e. Alarms should be provided to alert voters to system failures. The

alarms should be accompanied by instructions informing voters of the

actions to take; and

1. Status and alarm displays should follow conventional practice with

respect to color:

2. green, blue, or white displays shall be used for indications of normal

status;

3. Amber indicators shall be used to indicate warnings or marginal status;

4. Red indicators shall be used to indicate error conditions or equipment

states that may result in damage, or in hazards to personnel; and

5. unless the equipment is designed to halt under conditions of incipient

damage or hazard, an audible alarm shall also be provided.

12-158
Volume I — Performance Standards

12.10 Voter Familiarization and Training

Successful use of any voting system is supported by the availability of mean for voters

to become familiar with voting system operation. Being able to use and become

familiar with the system prior to voting will minimize confusion and errors. The

guidance in this section addresses these design considerations.

a. Voters should have access to sample ballots and all instructions before they

have to vote;

b. Voters should have an opportunity to practice before they vote, especially if

using electronic systems. On-line support can be provided, e.g., provide webbased access to all ballot information; and

c. Voters should have access to knowledgeable personnel to resolve any

questions.

12-159