EVS 6200 Certification Test Plan 

Report No. ESS-7005-CTP-01 

Page 1 of 79 

Template Rev 2015-03 

Certification Test Plan 

Document Number: ESS-7005-CTP-01 

Prepared for: 

Vendor Name Election Systems and Software 

(ES&S) 

Vendor System EVS 6.2.0.0 

EAC Application No. EVS6200 

Vendor Address 11208 John Galt Boulevard 

Omaha, Nebraska 68137

Prepared by: 

4720 Independence St. 

Wheat Ridge, CO 80033 

303-422-1566 

www.SLICompliance.com

Accredited by the Election 

Assistance Commission (EAC) for 

Selected Voting System Test 

Methods or Services 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 2 of 79 

Template Rev 2015-03 

Managing technology risk 

Copyright  2017 by SLI ComplianceSM, a Division of Gaming Laboratories International, LLC 

Revision History 

Date Version Author Revision Summary 

July 24th, 2017 1.0 M. Santos Initial Draft 

Disclaimer 

The information reported herein must not be used by the client to claim product 

certification, approval, or endorsement by NVLAP, NIST, or any agency of the Federal 

Government. 

Trademarks 

• SLI is a registered trademark of SLI Compliance. 

• All products and company names are used for identification purposes only and may be 

trademarks of their respective owners. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 3 of 79 

Template Rev 2015-03 

Managing technology risk 

TABLE OF CONTENTS 

CERTIFICATION TEST PLAN ...................................................................................................................... 1

1 INTRODUCTION .................................................................................................................................... 5

1.1 CERTIFICATION TEST PLAN ATTACHMENTS .......................................................................................... 5

1.2 REFERENCES ..................................................................................................................................... 5

1.3 TERMS AND ABBREVIATIONS................................................................................................................ 6

1.4 PROJECT OVERVIEW ........................................................................................................................... 8

1.5 PURPOSE ........................................................................................................................................... 8

1.6 TESTING RESPONSIBILITIES ................................................................................................................. 8

1.6.1 Project Schedule ...................................................................................................................... 8

1.7 SCOPE OF TESTING .......................................................................................................................... 11

1.8 SCOPE OF THE ES&S EVS 6.2.0.0 VOTING SYSTEM .......................................................................... 12

1.8.1 System Overview ................................................................................................................... 12

1.8.2 Block Diagram ....................................................................................................................... 15

1.8.3 System Limits ........................................................................................................................ 21

1.8.4 Supported Languages ........................................................................................................... 23

1.8.5 Supported Functionality ......................................................................................................... 23

1.8.6 Excluded Requirements......................................................................................................... 36

1.8.7 Additional Functionality/Requirements .................................................................................. 37

1.9 CHANGE CONTROL AND CONFIGURATION MANAGEMENT..................................................................... 37

2 PRE-CERTIFICATION TESTING AND ISSUES ................................................................................. 39

2.1 EVALUATION OF PRIOR VSTL TESTING ............................................................................................... 39

2.2 EVALUATION OF PRIOR NON-VSTL TESTING ....................................................................................... 39

2.3 KNOWN VULNERABILITIES.................................................................................................................. 39

3 PHYISCAL CONFIGURATION AUDIT ............................................................................................... 40

3.1 TDP DOCUMENTATION ANALYSIS & REVIEW ...................................................................................... 40

3.1.1 Document Review .................................................................................................................. 40

3.2 TDP SOURCE CODE ANALYSIS & REVIEW .......................................................................................... 41

3.2.1 Source Code Review ............................................................................................................. 41

3.3 QA & CM PROCESS REVIEW ............................................................................................................. 41

3.4 COMPLIANCE/TRUSTED BUILD ........................................................................................................... 42

4 MATERIALS REQUIRED FOR TESTING ........................................................................................... 43

4.1 SOFTWARE/FIRMWARE ...................................................................................................................... 43

4.1.1 Manufacturer Software/Firmware .......................................................................................... 44

4.1.2 COTS Software/Firmware...................................................................................................... 45

4.1.3 Additional Supporting Test Software ..................................................................................... 45

4.2 EQUIPMENT ...................................................................................................................................... 46

4.2.1 ES&S EVS 6.2.0.0 Equipment ............................................................................................... 46

4.2.2 COTS Equipment ................................................................................................................... 46

4.3 TEST MATERIALS .............................................................................................................................. 48

4.4 DELIVERABLE MATERIALS.................................................................................................................. 48

5 TEST SPECIFICATIONS ..................................................................................................................... 50

5.1 REQUIREMENTS ................................................................................................................................ 50

5.2 HARDWARE CONFIGURATION AND DESIGN ......................................................................................... 52

5.3 ES&S EVS 6.2.0.0 VOTING SYSTEM REVIEW .................................................................................... 53

5.3.1 Define..................................................................................................................................... 53

5.3.2 Design .................................................................................................................................... 54
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 4 of 79 

Template Rev 2015-03 

Managing technology risk 

5.3.3 Deliver .................................................................................................................................... 55

5.3.4 ExpressVote Universal Voting System, HW1.0 ..................................................................... 56

5.3.5 ExpressVote Universal Voting System, HW2.1 ..................................................................... 56

5.3.6 DS200 .................................................................................................................................... 57

5.3.7 DS450 .................................................................................................................................... 58

5.3.8 DS850 .................................................................................................................................... 58

5.3.9 Electionware Reporting Module ............................................................................................. 58

5.4 TESTING STRATEGIES ....................................................................................................................... 59

5.4.1 How each Polling Place Device will be tested ....................................................................... 59

5.4.2 How each Application will be tested ...................................................................................... 59

5.4.3 How different System Level Configurations will be tested ..................................................... 60

5.5 SOFTWARE SYSTEM FUNCTIONS ....................................................................................................... 61

5.5.1 Election Definition Creation ................................................................................................... 61

5.5.2 Election Media Creation......................................................................................................... 61

5.5.3 Pre-voting Aspects ................................................................................................................. 61

5.5.4 Voting Aspects – Polling Place .............................................................................................. 61

5.5.5 Voting Aspects – Central Count ............................................................................................ 61

5.5.6 Post Voting Aspects ............................................................................................................... 62

5.5.7 Error Messaging and Recovery ............................................................................................. 62

5.5.8 Auditing .................................................................................................................................. 62

5.5.9 Security .................................................................................................................................. 62

5.6 TEST SUITE DESIGN .......................................................................................................................... 62

5.6.1 Hardware Qualitative Examination Design ............................................................................ 62

5.6.2 Hardware Environmental Test Design ................................................................................... 63

5.6.3 Software Module Test Design and Data ................................................................................ 63

5.6.4 Software Functional Test Design and Data ........................................................................... 63

5.6.5 System-level Test Design ...................................................................................................... 64

5.6.6 Software Functional Test Case Design ................................................................................. 65

5.6.7 System-level Test Suite Design ............................................................................................. 67

5.7 STANDARD VSTL TEST METHODS AND UNCERTAINTY OF TEST DATA MEASUREMENT ......................... 73

5.8 EAC INTERPRETATIONS .................................................................................................................... 73

5.9 SECURITY FUNCTIONS ...................................................................................................................... 75

5.9.1 Security Test .......................................................................................................................... 75

6 TEST DATA ......................................................................................................................................... 76

6.1 DATA RECORDING ............................................................................................................................ 76

6.2 TEST DATA CRITERIA ........................................................................................................................ 76

6.3 TEST DATA REDUCTION .................................................................................................................... 77

7 TEST PROCEDURE AND CONDITIONS ........................................................................................... 77

7.1 FACILITY REQUIREMENTS .................................................................................................................. 77

7.2 TEST SETUP ..................................................................................................................................... 78

7.3 TEST SEQUENCE .............................................................................................................................. 78

8 TEST OPERATIONS PROCEDURES ................................................................................................. 78

9 APPROVAL SIGNATURES ................................................................................................................. 79
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 5 of 79 

Template Rev 2015-03 

Managing technology risk 

1 INTRODUCTION 

This Certification Test Plan outlines the test approach SLI Compliance (SLI) will 

follow when performing Certification Testing on the ES&S EVS 6.2.0.0 voting 

system against the 2005 Voluntary Voting System Guidelines (VVSG). The 

purpose of this document is to provide a clear understanding of the work SLI will 

conduct and a detailed plan outlining the test effort. 

When the testing is complete, SLI will submit a Certification Test Report that details 

all test results and findings from the Certification Test effort, as well as a 

recommendation to the EAC. 

1.1 Certification Test Plan Attachments 

The following attachments apply to this Certification Test Plan: 

1. Attachment A - Elec_NTS_Longmont_A2LA_0214-43_022818 

2. Attachment B - ENV_NTS_Longmont_A2LA_0214-44_022818 

3. Attachment C - ESS EV6000 Electrical Hardware Test Plan v1.0 

4. Attachment D - ESS EV6000 Environmental Hardware Test Plan v1.0 

1.2 References 

The following key documents were used in preparing this test plan. 

1. Election Assistance Commission Voluntary Voting System Guidelines (EAC 

VVSG), 2005 Version 1.0 Volumes I and II. 

2. NIST Handbook 150: 2016. 

3. NIST Handbook 150-22: 2008. 

4. EAC Voting System Testing and Certification Program Manual, United States 

Election Assistance Commission, v 2.0, May 2015 

5. SLI VSTL Quality System Manual, 2.3, prepared by SLI, dated January 20, 

2017. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 6 of 79 

Template Rev 2015-03 

Managing technology risk 

1.3 Terms and Abbreviations 

The following terms and abbreviations will be used throughout this document: 

Table 1 – Terms and Abbreviations 

Term Abbreviation Description 

American Association 

for Laboratory 

Accreditation 

A2LA A nonprofit, non-governmental, public service,

membership society whose mission is to provide 

comprehensive services in laboratory accreditation 

and laboratory-related training. 

Ballot Marking 

Device 

BMD An accessible computer-based voting system that

produces a marked ballot (usually paper) that is the 

result of voter interaction with visual or audio 

prompts. 

Central Count 

Scanner 

CCS High Speed Optical Scanner is a mark sense-based 

ballot and vote counting device typically located at a 

central count facility and is operated by an 

automated multi-sheet feeding capability. 

Compact Flash card CF This is a type of flash memory card in a 

standardized enclosure often used in voting 

systems to store ballot and/or vote results data. 

Commercial Off the 

Shelf 

COTS Term used to designate computer software, 

hardware or accessories that are ready-made and 

available for sale, lease, or license to the general 

public 

Direct Recording 

Electronic 

DRE Voting systems that, using Touch Screen or other 

user interfaces, directly record the voter’s selections 

in each race or contest on the ballot in electronic

form. 

Election Assistance 

Commission 

EAC An independent, bipartisan commission created by 

the Help America Vote Act (HAVA) of 2002 that 

operates the federal government's voting system 

certification program. 

Election 

Management System 

EMS Typically a database management system used to 

enter jurisdiction information (district, precincts, 

languages, etc.) as well as election specific 

information (races, candidates, voter groups 

(parties), etc.). In addition, the EMS is also used to 

layout the ballots, download the election data to the 

voting devices, upload the results and produce the 

final results reports. 

Electromagnetic 

Compatibility 

EMC The goal of EMC is to validate the correct 

functioning of different equipment in the same 

environment and the avoidance of any interference 

effects between them. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 7 of 79 

Template Rev 2015-03 

Managing technology risk 

Term Abbreviation Description 

Functional 

Configuration Audit 

FCA The testing activities associated with the functional 

testing of the system. 

National Institute of 

Standards and 

Technology 

NIST A non-regulatory federal agency within the U.S. 

Dept. of Commerce. Its mission is to promote U.S. 

innovation and industrial competitiveness by 

advancing measurement science, standards, and 

technology in ways that enhance economic security 

and improve our quality of life. 

National Voluntary 

Laboratory 

Accreditation 

Program 

NVLAP A division of NIST that provides third-party 

accreditation to testing and calibration laboratories. 

Physical 

Configuration Audit 

PCA Confirms that the documentation submitted 

meets the national certification requirements. 

Includes Trusted Build activities.

Precinct Count 

Scanner 

PCS A precinct-count optical scanner is a mark sensebased ballot and vote counting device located at a 

precinct and is typically operated by scanning one 

ballot at a time. 

Request For 

Information 

RFI A means used by testing laboratories and 

manufacturers to request that the EAC provide an 

interpretation of a technical issue related to testing 

of voting systems. 

Technical Data 

Package 

TDP The data package supplied by the vendor, which 

includes Functional Requirements, Specifications, 

End-user documentation, Procedures, System 

Overview, Configuration Management Plan, Quality 

Assurance Program, and manuals for each of the 

required hardware, software, firmware components 

of a voting system. 

Voluntary Voting 

System Guidelines 

VVSG A set of specifications and requirements against 

which voting systems can be tested to determine if 

the systems provide all of the basic functionality,

accessibility and security capabilities required for 

EAC certification. 

Voting System Test 

Lab 

VSTL An independent testing organization accredited by 

NVLAP and the EAC to conduct voting system 

testing for EAC certification. 

Voting Test Specialist VTS An SLI employee within the Compliance division 

who has been qualified to perform EAC voting 

system certification testing. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 8 of 79 

Template Rev 2015-03 

Managing technology risk 

1.4 Project Overview 

This test plan outlines the approach SLI will implement to perform testing of the 

ES&S EVS 6.2.0.0 voting system against the requirements of the federal 2005 

Voluntary Voting System Guidelines (VVSG), Volumes 1 and 2. This project is for 

an initial certification effort. 

1.5 Purpose 

The purpose of this EAC Test Plan is to create clear and precise documentation of 

the processes that SLI, as an EAC VSTL, will implement throughout the course of 

voting system verification testing. 

This Test Plan: 

• Defines the overall test approach. 

• Identifies required voting system hardware and software to be tested, and in 

what configurations. 

• Identifies required hardware, software, and tools to be used to support the 

testing efforts. 

• Defines the types of tests to be performed and against what declared 

functionality. 

• Defines the types of election and vote data required for effective testing. 

• Defines the types of voting system threats and vulnerabilities against which 

the voting system will be tested. 

• Serves as a foundation for the development of ES&S EVS 6.2.0.0 test 

suites 

• Defines the process for recording and reporting test results. 

1.6 Testing Responsibilities 

The following schedule describes the high-level tasks and assigned personnel titles 

that will be involved in the Certification Test effort of the ES&S EVS 6.2.0.0 voting 

system. 

1.6.1 Project Schedule 

The project schedule consists of the components listed in the following 

subsections: 

1.6.1.1 Owner Assignments 

• System Analysis and Review will be conducted by Source Code Review, 

Security and Voting Test Specialists, with oversight by the Test Manager 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 9 of 79 

Template Rev 2015-03 

Managing technology risk 

• Source code review will be conducted by Source Code Review 

Specialists, with oversight by the Test Manager 

• Documentation review will be conducted by Security and Voting Test 

Specialists, with oversight by the Test Manager 

• Test Module Development and Validation will be conducted by Security 

and Voting Test Specialists (VTS), with oversight provided by the Test 

Manager 

• Test Suite Development and Validation will be conducted by Security 

and Voting Test Specialists, utilizing SLI’s formal Test Methods, with 

oversight provided by the Test Manager 

• Formal Test Execution will be conducted by Security and Voting Test 

Specialists, with oversight by the Test Manager 

• 3rd Party testing will be conducted by the subcontracting 3rd Party 

Hardware Laboratories, with oversight by the Hardware Specialist 

1.6.1.2 Test Module Development and Validation 

Test Modules will be developed to provide repeatable detailed test steps. The 

Modules are designed for use in any suite that employs their functionality. The 

modules will be validated prior to Formal Test Execution to ensure accurate testing 

of the voting system. Test modules are validated by internal review and execution. 

The test modules and the test suites that are written by SLI contribute directly to 

the “Exhaustive verification of every system function and combination of functions 

cited in the vendor’s documentation”, as defined by the VVSG’s definition of the 

Functional Configuration Audit (FCA). Under the 2005 VVSG, this portion of the 

certification is considered to be part of the FCA. 

1.6.1.3 Test Suite Development 

Test Suites will be developed to group and focus testing around key areas of the 

voting system. Each Test Suite will contain multiple test modules providing clear 

and traceable test scripts and information. Various system configurations will be 

identified within the suites. Under the 2005 VVSG, this portion of the certification is 

considered to be part of the FCA. 

 

1.6.1.4 Formal Test Execution 

Formal Test Execution of the validated Test Suites will be conducted against the 

declared voting system, utilizing Compliance/Trusted Build software and firmware 

components, in order to test to determine whether or not the system is compliant 

with the VVSG requirements. Under the 2005 VVSG, this portion of the certification 

is considered to be part of the FCA. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 10 of 79 

Template Rev 2015-03 

Managing technology risk 

1.6.1.5 3rd Party Hardware Testing 

Hardware testing will be conducted by 3rd Party certified hardware test laboratories 

to verify the voting system devices are in compliance with the VVSG hardware 

requirements. 

Other Labs Performing Non-Core Hardware Testing 

SLI Compliance is responsible for all core voting system tests as identified in NIST 

NVLAP Handbook 150-22 (2005). The labs listed below will perform non-core 

hardware testing for this certification test campaign. 

Table 2 – Labs Performing Hardware Testing 

Laboratory Address Test(s) Date(s) 

NTS – EMI / 

EMC 

1736 Vista View Drive 

Longmont, CO 80504 

EMC / EMI Tests: 

Radiated Emissions, 

Conducted Emissions, 

ESD, Electromagnetic 

Susceptibility, Electrical 

Fast Transient, Lightning 

Surge, Conducted RF 

Immunity, Magnetic 

Fields Immunity, 

Electrical Power 

Disturbance 

7/7/2017 – 

8/7/2017 

NTS – 

Environmental/

Dynamic 

1601 Dry Creek Drive 

Suite 200 

Longmont, CO 80503 

MIL-STD-810D Tests: 

Bench Handling, 

Vibration, Low 

Temperature, High 

Temperature, Humidity, 

Temperature/Power 

Variation, and Reliability 

7/7/2017 – 

8/24/2017 

1.6.1.6 EAC & Manufacturer Dependencies 

The Test Plan will require EAC approval prior to finalization. 

ES&S will be required to provide all source code, documentation, equipment and 

supporting materials identified as part of the voting system. 

The source code must have all discrepancies resolved, be able to be built 

successfully, be installed, as well as successfully complete operational status 

checks prior to Formal Test Execution. 

In addition, ES&S is required to provide training on the voting system and support 

throughout the life of the project. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 11 of 79 

Template Rev 2015-03 

Managing technology risk 

1.7 Scope of Testing 

SLI will provide testing on the ES&S EVS 6.2.0.0 voting system based on the 

guidelines established for voting system verification testing as defined by the 

EAC’s 2005 VVSG. This effort will include all required levels of software, firmware, 

system and hardware testing required to demonstrate that the ES&S EVS 6.2.0.0 

voting system is tested against the requirements of the 2005 VVSG. 

SLI’s major task categories for voting system verification testing, as defined by 

SLI’s NVLAP audited and approved Quality System Manual, as well as the 2005 

VVSG, include: 

• Physical Configuration Audit (PCA) 

o Iterative documentation review and assessment 

o Iterative source code review 

o Trusted Builds/Compliance Builds. Identify the Trusted Build process 

to establish the system version and components being tested and 

ensure that the qualified executable release is built from the reviewed 

components. 

o Software and Hardware Configuration Audit. Verification of software 

and hardware physical and functional configurations. 

Note that Documentation and Source Code Review are denoted as 

“Iterative”. When either Documentation or Source Code is reviewed, 

and items that do not meet the VVSG requirements are found, 

discrepancy reports are generated and sent to the manufacturer. 

Next, the manufacturer is allowed to resolve the issue and re-submit 

the item in question. SLI then re-reviews to verify that the 

requirement is met. (SLI also validates that nothing else has been 

modified; if something else was modified, it is also re-reviewed.) 

• Functional Configuration Audit (FCA) 

o Review of other relevant state verification testing or risk analysis 

results. 

o Review of prior hardware testing results. 

o Testing of voting system components and fully integrated systems to 

validate functionality, logic processing, system performance, security, 

and system level integration. This testing includes the formal test 

execution, utilizing customized iterations of SLI’s formal Test 

Methods for the ES&S EVS 6.2.0.0 voting system. Regression testing 

will be done when issues are found in testing and source code is 

modified, re-reviewed and re-built. 

o Accuracy testing and validation of a voting system’s ability to 

accurately read and tally a large number of ballot marking positions 

within the requirements allowed by the 2005 VVSG. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 12 of 79 

Template Rev 2015-03 

Managing technology risk 

o Security testing and validation that a voting system is tested against 

all security related requirements as well as assessing the 

effectiveness of a voting system’s security controls. 

o Verification of every system function cited in the vendor’s 

documentation. 

• Management of ES&S EVS 6.2.0.0 supplied deliverables, SLI’s test 

artifacts, and software, firmware, hardware and system test configurations. 

• Generation of detailed and repeatable test modules based on SLI’s formal 

Test Methods, to ensure the voting system is tested against all applicable 

requirements of the 2005 VVSG and associated ES&S EVS 6.2.0.0 specific 

requirements. 

• Generation of test data required for all test modules. 

• Traceability of test modules to the requirements of the 2005 VVSG. 

• Reporting of all test results. 

SLI will develop and submit to the EAC a final test report for the ES&S EVS 6.2.0.0

voting system. 

1.8 Scope of the ES&S EVS 6.2.0.0 Voting System 

1.8.1 System Overview 

This section provides a description of the scope of ES&S EVS 6.2.0.0 voting 

system and components: 

• The ES&S EVS 6.2.0.0 voting system represents a set of software applications 

for pre-voting, voting and post-voting election project activities for jurisdictions 

of various sizes and political division complexities. ES&S EVS 6.2.0.0 functions 

include: 

o Defining the political divisions of a jurisdiction and organizing the 

election with its hierarchical structure, attributes and associations. 

o Defining the election events with their attributes such as the election 

name, date and type, as well as contests, candidates, referendum 

questions, voting locations and their attributes. 

o Preparing and producing ballot for polling place and absentee voting. 

o Preparing media for precinct voting devices and central count 

devices. 

o Configuring and programming the DS200 digital scanners. 

o Configuring and programming the ExpressVote HW1.0 and

ExpressVote HW2.1 Universal Voting System devices. 

o Producing the election definition and auditing reports. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 13 of 79 

Template Rev 2015-03 

Managing technology risk 

o Providing administrative management functions for user, database, 

networking and system management. 

o Import or manual data entry of the Cast Vote Records from DS200, 

DS450, and DS850. Preview and validation of the election results. 

o Producing election results tally according to voting variations and 

election system rules. 

o Producing a variety of reports of the election results in the desired 

format. 

o Publishing of the official election results. 

o Auditing of election results including ballot images and log files. 

The ES&S EVS 6.2.0.0 voting system is composed of software applications, 

central count location devices and polling place devices with accompanying 

firmware, and COTS that are supported by the voting system. 

• Electionware is an end-to-end election management software application 

that provides election data creation, ballot formation, equipment 

configuration, result consolidation, adjudication and report creation.

o Prior to the election, the Electionware interface handles all tasks that 

define the image of a paper ballot of an election including: contests, 

parties and options. Then, Electionware converts all database 

information into ballot definition parameters for ExpressVote HW1.0

and ExpressVote HW2.1 Universal Voting Systems, as well as, 

DS200, DS450, and DS850 tabulators through the Removable 

Media Service by programming media for each device. 

o Post-election, Electionware gathers and combines results from all 

tabulator types and generates reports. Ballot images for adjudication 

are also able to be collected and reviewed. 

• ExpressVote Hardware 1.0 is a polling place Universal Voting System that 

provides digital vote capture through a touch screen or accessibility 

interface using a detachable ADA keypad or ADA support peripheral such 

as sip-and-puff device or two-position switch. When the voter selections are 

complete a vote summary screen requires voters to confirm or revise their 

selections prior to printing. Once printed, the voter-verifiable paper record is 

ready to be scanned on any ES&S tabulator. 

• ExpressVote Hardware 2.1 is a polling place Universal Voting System and 

precinct tabulator that provides digital vote capture through a touch screen 

or accessibility interface using a detachable ADA keypad or ADA support 

peripheral such as sip-and-puff device or two-position switch. 

When the voter selections are complete a vote summary screen requires 

voters to confirm or revise their selections prior to printing. Once printed, the 

voter-verifiable paper record is digitally scanned on a compatible ES&S
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 14 of 79 

Template Rev 2015-03 

Managing technology risk 

tabulator. When mounted on a rolling kiosk the device can tabulate printed 

vote summary cards. 

• DS200 is a polling place scanner and tabulator that scans completed printed 

ballots and/or vote summary cards. After the voter marks a paper ballot 

either manually or digitally, their ballot is inserted into the unit and tabulated 

to a USB memory device and internal vote counter. Once the ballot is 

tabulated, the ballot is dropped into an integrated ballot box. An integrated 

printer prints zero reports, log reports, and polling place totals upon the 

official closing of the polls. 

• DS450 is a central scanner and tabulator that simultaneously scans the front 

and back of a paper ballot and/or vote summary card in any of four 

orientations. The DS450 sorts tabulated ballots into discrete output bins 

without interrupting scanning, based on pre-defined sort criteria. Optionally, 

this device may be configured to transmit tabulation results to the results 

server through a closed network connection rather than using physically 

transported media. 

• DS850 is a high-speed digital processing scanner and central tabulator that 

scans and automatically sorts paper ballots and/or vote summary cards in 

all four orientations. Optionally, this device may be configured to transmit 

tabulation results to the results server through a closed network connection 

rather than using physically transported media. 

• Event Log Service (ELS) monitors user’s interactions with the Election 

Management System. 

• Removable Media Service (RMS) is a utility which reads information from 

attached USB devices. 

 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 15 of 79 

Template Rev 2015-03 

Managing technology risk 

1.8.2 Block Diagram 

Election Management Diagram 

Election Data Management Diagram Description 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 16 of 79 

Template Rev 2015-03 

Managing technology risk 

Ballot Formatting and Printing Diagram 

Ballot Formatting and Printing Diagram Description 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 17 of 79 

Template Rev 2015-03 

Managing technology risk 

Voting Equipment Configuration Diagram 

Voting Equipment Configuration Diagram Description 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 18 of 79 

Template Rev 2015-03 

Managing technology risk 

Voting and Tabulation Diagram 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 19 of 79 

Template Rev 2015-03 

Managing technology risk 

Voting and Tabulation Diagram Description 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 20 of 79 

Template Rev 2015-03 

Managing technology risk 

Results Consolidation, Adjudication and Reporting Diagram 

Results Consolidation, Adjudication and Reporting Diagram Description 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 21 of 79 

Template Rev 2015-03 

Managing technology risk 

1.8.3 System Limits 

This section details various limits within the ES&S EVS 6.2.0.0 voting system. 

Limits will be reviewed and verified for accuracy. 

• Max precinct allowed in an election: 9,900 

• Max candidate allowed per election: 30,000 

• Max contests allowed in an election: 30,000 

• Max contests allowed per ballot style: 500 or number of positions on ballot 

• Max candidate (ballot choices) allowed per contest: 230 

• Max number of parties allowed in General Election: 75 

• Max number of parties allowed in Primary Election: 30 

• Max ‘vote for’ per contest: 500 

• Max ballot styles: 15,000 

• Max district types/groups: 25 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 22 of 79 

Template Rev 2015-03 

Managing technology risk 

• Max district of a given type: 250 

• Max reporting groups: 14 

• Electionware 

o Electionware Export Ballot Images function is limited to: 250 districts 

per export 

• ExpressVote, Hardware 2.1 

o Secure Card Container Capacity: 300 vote summary cards 

• DS200 

o Early voting station does not support precinct level results reporting 

o Storage limitation for write-in ballot images is 3,600 images 

o Write-in image review requires min 1 GB of onboard RAM. 

o Plastic Ballot Box Capacity 

 Standard compartment: 1,250 ballots 

 Removable tote bin: 1,500 ballots 

 Auxiliary compartment: 100 ballots 

o Metal Ballot Box Capacity 

 Main compartment: 1,500 ballots 

 Write-in compartment: 1,500 ballots 

 Auxiliary compartment: 125 ballots 

• DS450 

o Approx. 5 million ballot images and related data of memory 

o Tray/Bin Capacity 

 Input tray 480 

 Output bin 480 

 Outstack bins 150 

• DS850 

o Approx. 5 million ballot images and related data of memory 

o Tray/Bin Capacity 

 Input tray 480 

 Output bin 480 

 Outstack bin 150 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 23 of 79 

Template Rev 2015-03 

Managing technology risk 

• Number of characters allowed on a single Write-In selection: 30 

1.8.4 Supported Languages 

The ES&S EVS 6.2.0.0 voting system has declared support for English, Spanish, 

Chinese, Korean, Japanese, Hindi, Bengali, Vietnamese, Tagalog, Creole, 

Russian, and French languages for this certification project. 

1.8.5 Supported Functionality 

The following standard VVSG functionality and manufacturer extensions are 

included in the voting system. 

1.8.5.1 Voting Variations 

The ES&S EVS 6.2.0.0 supports the following voting variations: 

• Closed primaries 

• Open primaries 

• Partisan offices 

• Non-partisan offices 

• Write-in voting 

• Ballot rotation 

• Straight party voting 

• Cross-party endorsement 

• Split precincts 

• Vote for N of M 

• Recall issues with options 

• Ranked Choice Voting (ballot formatting and CVR export only) 

• Provisional or challenged ballots 

1.8.5.2 Electionware - Pre-voting Functionality 

Electionware consists of four pre-voting software groups: Manage, Define, Design 

and Deliver, all of which allow users to prepare an election by defining election 

data and ballot styles that can be transferred to portable flash media for purposes 

of voting and vote capture. 

The Manage group allows the election official to manage users and jurisdictions. 

The Setup module adds and edits users and jurisdictions and sets the strength of 

election codes. 

The Define group is used to input critical election data. The Define group allows 

users to: create, edit and manage elections; enter election based information into 

database like precincts, districts, contests, candidates, and other pertinent 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 24 of 79 

Template Rev 2015-03 

Managing technology risk 

information; and stores graphics for ES&S equipment and for managing system 

translations and audio. 

The Design group designs ballots and audio elements for an election. The Design 

group controls: paper ballot design for use with ES&S equipment, electronic ballot 

design to support multiple screen layout sizes, format ballot display options as they 

will appear, manage ADA audio settings, and validate data before the election files 

are generated. 

The Deliver group configures equipment, packages election data and prints ballots 

on demand. This group provides specific access to allow users to: set parameters 

and access codes for election equipment, create election media files, view and 

print BOD (Ballot on Demand) ballots by precinct. 

Importing and Managing Election Data

Electionware supports importation of and the creation and configuration of an 

election as follows: 

• Stores all of a jurisdiction’s precinct, office, and candidate information. 

• Once an election database is complete, it can be recalled and edited. 

• The System Event Log tracks all user and equipment actions throughout the 

election process. 

Formatting Ballots 

Electionware includes a functional subsystem providing all activities related to 

defining the image of a paper ballot and populating the ballot with contest, 

candidate and referendum information from the database in the Paper Ballot 

module, which prepares an election by defining ballot styles. 

Proofing Election Data 

Electionware provides the following proofing capabilities: 

• Provides methods to validate ballot data that has been entered. 

• Users can: 

• View ballot style previews through an embedded viewer. 

• Print ballot style previews. 

• Save ballot styles to image files. 

• Proof recorded audio strings. 

• Electionware allows users to proof translations for supported languages. 

• Electionware supports the production of various reports that allow users to 

proof-read and validate ballot data entry. 

Ballot Content – Supported Information 

Electionware supports the following types of data: 

• Ballot header information, includes the election type, ballot label, jurisdiction 

name, election title, election date. 

• Standard and customized ballot instructions. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 25 of 79 

Template Rev 2015-03 

Managing technology risk 

• Election types. 

• Party information and graphics. 

• Contest types and associated contest options, including contest-specific 

instruction text and/or proposition text. 

• Rotation Indices. 

Electionware is capable of formatting ballots with the following information: 

• Unique ballot identifying information. 

• Page numbering. 

• Static text strings. 

• Ballot instructions with custom content. 

• Ballot images. 

• Multi-language data (including audio, text, and image files as appropriate). 

• Election data per contest with fully detailed options/candidates and images 

according to the contest type. Contests include candidates and propositions. 

• Voting logic per contest entry to set the voting and tabulation method per 

contest. 

• Write-in options. 

Ballot Standards 

Electionware employs and supports the ballot standards as follows: 

• Supports the following paper sizes: 

• 8.5” x 11” 

• 8.5” x 14” 

• 8.5” x 17” 

• 8.5” x 19” 

• Supports ballot layouts in portrait and landscape orientation. 

• Supports duplex ballot layouts. 

• Supports the inclusion of ballot stubs on paper ballots. 

• Compatible with the production of ballots on standard, commercially 

available white paper stock. 

Ballot Layout Features 

Electionware supports ballot layouts as follows: 

• Supports the layout of ballots in paper-based and electronic formats. 

Previewing Ballot Layouts 

Electionware supports previewing ballot layouts as follows: 

• Allows users to view on-screen graphic previews of exact ballot layouts. 

• Allows users to specify which ballot types are previewed, according to a 

variety of user-defined attributes, including precinct, style, type, language, 

and party. 

• Supports users to proof-read and validate ballot layouts. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 26 of 79 

Template Rev 2015-03 

Managing technology risk 

• Supports the production of audit logs for transparency. 

Configuring Ballot Handling Equipment 

Configuration of ballot handling equipment includes all tasks required to convert 

Electionware database information into ballot definition parameters for tabulation 

and Universal Voting System equipment. Within this functional subsystem, election 

management software users program the ballot counting rules for a specific 

election – the election definition – to the memory devices used to program 

tabulation equipment. 

Generating Election Media 

Electionware employs and supports generation of election media as follows: 

• Allows users to generate portable flash media that includes election data 

and ballot style information that the user has accepted. 

Ballot Production 

Electionware employs and supports ballot production as follows:

• Supports ballot production in a variety of formats, including paper ballot 

printing, generation of electronic ballots, and production of ballot printing 

files in PDF format, for use by third-party commercial printers. 

1.8.5.3 DS200 – Device Functionality 

DS200 provides paper ballot scanning capabilities. When scanned, the system 

validates the ballot according to voter marks and the associated election contests. 

If issues are located, the system may provide a warning or flag the scanned ballot 

for review. The scanned voter selections are stored to a USB memory device. The 

USB memory device is removable from the system for transport to a central 

election location where vote totals are consolidated for reporting. 

General 

DS200 devices support the following components: 

• Scanning mechanism for all ballot sizes and vote summary cards 

• Paper ballot transport path 

• Plastic Ballot Box Capacity 

• Metal Ballot Box Capacity 

• Collapsible Ballot Box Capacity 

• USB Memory Capacity 

• Optional system health status in a QR two dimensional barcode that can be 

scanned to pass the data back to a central administration site in preparation 

for opening the polls. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 27 of 79 

Template Rev 2015-03 

Managing technology risk 

Scanning 

DS200 Devices: 

• Include a bi-directional scanner. 

• Accommodate duplex scanning. 

• Are capable of capturing and storing the ballot image on removable media. 

• Are capable of capturing and storing the digital image of a write-in vote. 

• Enlist a multi-sheet scanning functionality. 

• Scan paper ballots and/or vote summary cards in all portrait orientations 

(face up, face down, header first, footer first). 

• Mechanism that allows counted ballots to be stamped and can be 

configured to stamp all ballots or ballots meeting criteria selected by the 

election administrator such as ballots containing over votes or write-ins. 

• Maintains a real-time audit log printable from each terminal’s integrated 

printer and stored on the election media. 

Configurable Settings 

DS200 Devices are configurable for: 

• Unconditional acceptance where the DS200 accepts and tabulates results 

for all ballots. 

• Unconditional rejection where the DS200 automatically rejects ballots that 

contain the specified condition. 

• Query the voter for revision where the DS200 displays a screen message 

that describes the condition and prompts the voter to either review and 

revise the ballot or cast the ballot as it is. 

• Optional results backup media capability. 

• Capturing ballot images that can be viewed in Electionware. 

Supported Ballot Types 

DS200 Devices: 

• Accept vote summary cards of all lengths 

• Accept ballots of all lengths 

• Maintain public ballot counters that reflect the total number of ballots cast or 

marked for the current election configuration and the total number of sheets 

processed. 

Scanning Device Vote process 

DS200 Devices: 

• Display voter instruction messages for all valid operations. 

• If configured to unconditionally reject an exception ballot, the system returns 

the ballot to the voter and displays a message describing the exception 

condition. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 28 of 79 

Template Rev 2015-03 

Managing technology risk 

• If configured to unconditionally accept, the inserted ballot will be tabulated 

as marked, which may result in the inclusion of undervotes and/or overvotes 

in system totals. 

• If configured to query the voter prior to acceptance, the DS200 displays a 

message describing the exception condition, indicates the mismarked ballot, 

and prompts the voter to revise the ballot. 

Scanning Device Error Handling 

DS200 Devices: 

• When applicable, display all error messages as they occur on the LCD 

display. 

• Provide error messages to assist operators to determine the correct action. 

• Recognize multiple sheet detection where more than one sheet of paper is 

recognized and returns an error message to the user display indicating the 

error. 

• Are designed to detect and record critical events in the Audit log and critical 

events report. 

Security 

DS200 uses digital signatures, encryption, and security access codes to safeguard 

data. 

1.8.5.4 DS450 – Device Functionality 

DS450 is a high-throughput central scanning component that provides paper ballot 

scanning functionality and conversion of voter selection marks to electronic Cast 

Vote Records (CVR). Once the CVR’s are stored they can be transferred into 

Electionware software for vote tabulation, adjudication and reporting of election 

results. 

General 

The DS450 central scanner: 

• Is capable of operating in local network or standalone configurations. 

• Loads a set of election definition files via an ES&S Election flash drive. 

• Captures an image of the ballot to create a vote image record that is used to 

aggregate totals whenever the user selects to export results to the ES&S

Results flash drive. 

• Transports paper ballot to output bins. 

• Has USB Memory Capacity. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 29 of 79 

Template Rev 2015-03 

Managing technology risk 

Scanning 

The DS450 device scans ballots, designed by Electionware in the Paper Ballot 

module, through the paper transport mechanism. The DS450 is able to: 

• Scan bi-directionally. 

• Accommodate duplex scanning. 

• Analyze each possible oval location. 

• Use software to determine user markings at each possible active response 

location and store these values along with the Election Definition to 

determine the voter’s choices. 

• Scan multiple precincts using the precinct selection option. 

• Tabulate ballots at a central scanning location. 

• Capture and store ballot images on removable media. 

• Capture and store the digital image of a write-in vote. 

• Support Multi-Sheet scanning functionality. 

• Scan paper ballots and/or vote summary cards in all portrait orientations 

(face up, face down, header first, footer first). 

• Provide system audit log records all error conditions and corrective actions. 

Configurable Settings 

DS450 devices are configurable: 

• To scan vote summary cards printed by the ExpressVote HW1.0 and 

ExpressVote HW2.1. 

• To use the ballot tracking option to visually show the user which ballots are 

not processed and which ballots can remain in the output bin. 

Supported Ballot Types 

DS450 devices: 

• Accept vote summary cards of all lengths. 

• Accept ballots of all lengths. 

• Maintain public ballot counters that reflect the total number of ballots cast or 

marked for the current election configuration and the total number of sheets 

processed. 

Scanning Device Operator process 

DS450 devices: 

• Allow the operator to preview/print the election Zero Report once the 

election definition is loaded. 

• Allow the operator to save and export the results and ballot images, after 

each batch is scanned, continue scanning additional batches, change sort 

options, delete the current unsaved batch, or generate election reports. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 30 of 79 

Template Rev 2015-03 

Managing technology risk 

Scanning Device Error Handling 

DS450 devices check: 

• If any problems occur during the scanning process or if the ballot is not 

identified in the Election Definition, the ballot may be diverted to an 

exception bin for further analysis. 

• If the precinct on the ballot does not match the selected precinct for that 

stack of ballots, that ballot will be out stacked. 

• If a jam occurs and stops processing ballots and identifies which ballots 

were counted and which need to be returned to the input tray for scanning. 

• If exceptions occur, the Master Control Program handles exceptions by 

logging the event through the Audit ID, which contains the error. The error is 

logged using the built-in operating system logging functionality. 

1.8.5.5 DS850 – Device Functionality 

DS850 is a high speed central scanning component that provides paper ballot 

scanning functionality and conversion of voter selection marks to electronic Cast 

Vote Records (CVRs). Once the CVRs are stored they can be transferred into 

Electionware software for vote tabulation, adjudication and reporting of election 

results. 

General 

The DS850: 

• Is capable of operating in local network or stand-alone configurations. 

• Loads a set of election definition files via an ES&S Election flash drive. 

• Captures an image of the ballot to create a vote image record that is used to 

aggregate totals whenever the user selects to export results to the ES&S

Results flash drive. 

• Transports paper ballot to output bins. 

• Has USB Memory Capacity. 

Scanning 

The DS850 device scans ballots and/or vote summary cards, designed by 

Electionware in the Paper Ballot module, through the paper transport mechanism. 

The DS850 is able to: 

• Scan both the front and back sides of the ballot.

• Analyze each possible oval location. 

• Use software to determine user markings at each possible active response 

location and store these values along with the Election Definition to 

determine the voter’s choices. 

• Scan multiple precincts using the precinct selection option. 

• Tabulate ballots at a central scanning location. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 31 of 79 

Template Rev 2015-03 

Managing technology risk 

• Capture and store ballot images on removable media. 

• Capture and store the digital image of a write-in vote. 

• Support Multi-Sheet scanning functionality. 

• Scan paper ballots and/or vote summary cards in all portrait orientations 

(face up, face down, header first, footer first). 

• Provide system audit log records of all error conditions and corrective 

actions. 

Configurable Settings 

DS850 devices are configurable: 

• To scan vote summary cards printed by the ExpressVote HW1.0 and 

ExpressVote HW2.1. 

Supported Ballot Types 

DS850 devices: 

• Accept vote summary cards of all lengths. 

• Accept ballots of all lengths. 

• Maintain public ballot counters that reflect the total number of ballots cast or 

marked for the current election configuration and the total number of sheets 

processed. 

Scanning Device Operator process 

DS850 devices: 

• Allow the operator to preview/print the election Zero Report once the 

election definition is loaded. 

• Allow the operator to save and export the results and ballot images, after 

each batch is scanned, continue scanning additional batches, change sort 

options, delete the current unsaved batch, or generate election reports. 

Scanning Device Error Handling 

DS850 devices check: 

• If any problems occur during the scanning process or if the ballot is not 

identified in the Election Definition, the ballot may be diverted to an 

exception bin for further analysis. 

• If the precinct on the ballot does not match the selected precinct for that 

stack of ballots, that ballot will be out stacked. 

• If a jam occurs and stops processing ballots and identifies which ballots 

were counted and which need to be returned to the input tray for scanning. 

• If exceptions occur, the Master Control Program handles exceptions by 

logging the event through the Audit ID, which contains the error. The error is 

logged using the built-in operating system logging functionality. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 32 of 79 

Template Rev 2015-03 

Managing technology risk 

1.8.5.6 ExpressVote Hardware 1.0 – Device Functionality 

ExpressVote HW1.0 is a vote capture device designed for all voters, with 

independent voter-verifiable paper record that is digitally scanned for tabulation on 

a compatible ES&S tabulator. This system combines paper-based voting with 

touch screen technology. 

Multiple Ballot Styles 

If an election has more than one ballot style: 

• The initial ExpressVote HW1.0 system prompts the user to insert an 

activation card, and then select the correct ballot style and precinct, in order 

to display the proper ballot for the voter. 

• If an activation barcode has been printed on the card by a poll worker, when 

the voter inserts the card, the ExpressVote HW1.0 scans the barcode and 

immediately displays the proper ballot for the voter. 

• The ExpressVote HW1.0 accepts both ES&S created activation barcodes 

and authorized activation barcodes from other voter identification vendors. 

Accessibility Support 

The ExpressVote HW1.0 is designed for all voters, along with the ballot display 

and audio prompts, the device supports: 

• A detachable keypad (DKB), which is used with audio prompts to vote the 

ballot. 

• Two-position switch devices such as a sip-and-puff and rocker switch. 

Configurations 

The ExpressVote HW1.0 has four configurations: ADA table, tabletop, voting 

booth, or rolling kiosk. 

• The rolling kiosk is easily positioned for seated or standing voters and 

can optionally provide QR code scanning, report printing, and a secure 

card container. 

• The ExpressVote HW1.0 supports judge’s initial and review boxes. 

Telecommunications Abilities 

• No telecommunication capabilities for transfer of data from the polling 

site. 

Security 

ExpressVote HW1.0 uses digital signatures, encryption, and security access 

codes to safeguard data. 

Audio Interface 

With audio election data to be loaded and headphones connected to the 

ExpressVote HW1.0, the unit’s operating system and loaded firmware provide a 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 33 of 79 

Template Rev 2015-03 

Managing technology risk 

platform for playing audio files from the ES&S Election flash drive. This audio 

interface complements the GUI screens by helping the voter navigate the ballot 

contests, select their desired candidates, print their selections on the activation 

card, and either retrieve or automatically cast (AutoCAST) their marked vote 

summary card. 

1.8.5.7 ExpressVote Hardware 2.1 – Device Functionality 

ExpressVote HW2.1 is a vote capture device designed for all voters, with 

independent voter-verifiable paper record that can be configured to digitally scan 

marked paper vote summary cards for tabulation. This system combines paperbased voting with touch screen technology. 

Multiple Ballot Styles 

If an election has more than one ballot style: 

• The initial ExpressVote HW2.1 system prompts the user to insert an 

activation card, and then select the correct ballot style and precinct, in order 

to display the proper ballot for the voter. 

• If an activation barcode has been printed on the card by a poll worker, when 

the voter inserts the card, the ExpressVote HW2.1 scans the barcode and 

immediately displays the proper ballot for the voter. 

• The ExpressVote HW2.1 accepts both ES&S created activation barcodes 

and authorized activation barcodes from other voter identification vendors. 

Accessibility Support 

The ExpressVote HW2.1 is designed for all voters, along with the ballot display 

and audio prompts, the device supports: 

• A detachable keypad (DKB), which is used with audio prompts to vote the 

ballot. 

• Two-position switch devices such as a sip-and-puff and rocker switch. 

Configurations 

The ExpressVote HW2.1 has four configurations: ADA table, tabletop, voting 

booth, or rolling kiosk. 

• The rolling kiosk can be positioned for seated or standing voters and 

can optionally provide QR code scanning, reporting printing, and a 

secure card container. 

• The ExpressVote HW2.1 supports judge’s initial and review boxes. 

• Optional rear-eject feature; when disabled, the printed card is returned 

to the voter for insertion into a tabulator or secure ballot box. When 

enabled, the voter makes their selections, they are given the option to 

either receive their printed card or have it AutoCAST. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 34 of 79 

Template Rev 2015-03 

Managing technology risk 

Telecommunications Abilities 

• No telecommunication capabilities for transfer of data from the polling 

site. 

Security 

ExpressVote HW2.1 uses digital signatures, encryption, and security access 

codes to safeguard data. 

Audio Interface 

With audio election data to be loaded and headphones connected to the 

ExpressVote HW2.1, the unit’s operating system and loaded firmware provide a 

platform for playing audio files from the ES&S Election flash drive. This audio 

interface complements the GUI screens by helping the voter navigate the ballot 

contests, select their desired candidates, print their selections on the activation 

card, and either retrieve or automatically cast (AutoCAST) their marked vote 

summary card. 

Tabulation 

In addition to retaining all feature functionality as a ballot marker, the ExpressVote

HW2.1 can also be optionally configured to tabulate printed vote summary cards. 

The ExpressVote HW2.1 as a tabulator uses a Master Media device for Poll Open 

and Poll Close functions. The tabulated vote summary card is deposited into a 

removable, secure container attached to the rolling kiosk. 

1.8.5.8 Post Voting Functionality 

The Results group in Electionware contains the post voting capability to import, 

manage, and report election results data from the tabulation media. This group 

includes all of the tools used for loading results, machine logs, cast vote records, 

and ballot images; creates the results reports; ballot review and adjudication; and 

write-in review and management. 

After the election, the Reporting module can display or export detailed status 

information about individual flash drives, and produce machine log reports. It can 

clear loaded results files, in order to delete test data or to start the loading process 

over. The Reporting module is also used to filter, view and export ballot data, print 

ballot scan images and facsimiles, and review ballot images, write-ins and cast 

vote records. 

General 

The Electionware Reporting module: 

• Is capable of tabulating and reporting results for any election 

• Can display or export detailed status information about individual status 

information about individual flash drives and produce machine log reports. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 35 of 79 

Template Rev 2015-03 

Managing technology risk 

• Only accepts elections that have been created in Electionware and USB 

media created by Electionware

• Encrypts the key exchange process between the EMS and the tabulators to 

ensure confidentiality. 

• Supports 14 reporting groups that are used to categorize different groups of 

ballot styles for statistical reporting purposes. 

• Is capable of being installed on a “standalone” PC that is not networked to 

other PCs. 

• If configured in a standalone configuration, the component is capable of 

reading an election definition from portable media that is inserted into the 

tabulation PC. 

• Is capable of being installed on a networked workstation that is connected to 

other Electionware workstations. 

• Digitally signs every cast vote record and digitally signs the package of cast 

vote records captured by the tabulators. 

• Is capable of receiving tabulation results from a network for Central Count 

Tabulators. 

• Is capable of supporting multiple elections at the same time. 

• Allows the user to manage different elections. 

• Supports the following user functions in appropriate sections of the user 

interface: 

 • Tabulation database management 

 • Reading and tabulating portable election media 

 • Election progress dashboard 

 • Managing write-in votes 

 • Reporting (including standard reports, and customized reports) 

 • Audits and recounts 

• Allows users to import data from external sources to facilitate tabulation 

database configuration. 

Results Loading 

Electionware: 

• Will automatically load results from every connected poll media flash drive 

when activated, and from the network until stopped by the user. 

• Checks each flash drive as it is read to verify it is both an approved ES&S

flash drive and that it contains matching signatures and hashes. 

• Authenticates the flash drive, and then proceeds to decrypt the files on the 

flash drive. 

• Verifies that the decrypted files have not already been processed, based on 

the polling location and flash drive serial numbers. 

• Bases its results and reporting totals on the tabulated results in the Poll 

Place Collection Files. 

• The CVR’s and Ballot Images are used for ballot review and auditing 

tabulated results. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 36 of 79 

Template Rev 2015-03 

Managing technology risk 

Ballot Review 

Electionware provides the ability for on-screen ballot review of write-in votes and 

unreadable marks. Electionware allows an election official to: 

• Review images of each write-in or unreadable mark in a contest to be 

assigned and tallied for reporting purposes. 

• Filter reviewable fields by contest type or condition 

• Bulk assign write-ins to candidates 

Reporting 

The election results workflow in Electionware is used to generate paper and 

electronic tabulated results reports and exports. Electionware:

• Offers methods to protect voter privacy in instances where reported results 

are based on small numbers of ballots cast. 

• Provides user access level reports like: system administrator only reports, 

administrator reports, media creator reports, system user only reports. 

• Provides several reports regardless of the user access level. 

• Allows users to generate reports to various formats for review and usage in 

third party components. 

1.8.6 Excluded Requirements 

In this section VVSG requirements are identified that do not pertain to the declared 

system being certified. For this certification project ES&S has elected to not 

support their ExpressTouch (DRE) implementation, as such, the table below 

enumerates the requirements that will not be subject to verification for this project. 

DRE Related 

Requirements Not 

Under Test 

1.5.2.2 

1.5.2.3 

2.1.2.f 

2.1.4.k,l 

2.3.1.3 

2.3.2 

2.3.3.3 

3.1.2.f,g 

4.1.1.b 

4.1.4.3 

4.1.6.2 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 37 of 79 

Template Rev 2015-03 

Managing technology risk 

4.3.5.b 

5.4.3.b.iv 

5.5 

6.2.5 

7.8 

7.9 

3.2.1.c 

Vol. 2 , 4.7.4.c 

1.8.7 Additional Functionality/Requirements 

As per 2005 VVSG, volume 2 section 3.2.2, review of the ES&S EVS 6.2.0.0

Technical Requirements Documentation (TRDs) resulted in SLI’s determination 

that some functionality/requirements that are considered beyond the scope of the 

VVSG, but since they are in the declared system, will require review and 

verification. 

1.9 Change Control and Configuration Management 

The SLI project team follows a standard Change Control and Configuration 

Management (CM) process. This specifies the methods used by SLI to ensure 

changes are managed and controlled effectively and efficiently and defines the 

process for receipt, check-in, storage and disposition of hardware, source code, 

TDP and non-TDP Documents. 

It specifies the processes and procedures used by SLI to identify, monitor, and 

control versions of all of the stated EAC project deliverables, ensuring that the 

items are constantly and reliably managed throughout the entire duration of the 

engagement to ensure changes are managed and will not result in invalidated or 

wasted testing efforts. 

A summary of these tasks includes: 

• Receive TDP code: 

o Delivery department will perform a code check-in on the server. 

Code is placed in the specific project’s TDP folder with the delivery 

date and documentation of what was delivered. 

o Delivery department notifies Test Manager (TM) of delivery via email. 

o TM will notify the Source Code Review (SCR) team, via email, the 

code is ready for review. 

o TM will review the documentation of what was delivered to determine 

possible testing impacts. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 38 of 79 

Template Rev 2015-03 

Managing technology risk 

o Source Code Reviewer updates status of delivered code as received, 

what code to compare it to (if this is not new code), where code has 

been placed, and who will do the review of the code. 

o The code is checked out by the SCR who is responsible for the code 

review. 

o When the code review is complete, the TM is notified. 

o A peer review is done on the review and all written discrepancies are 

verified and an official Discrepancy Report is sent. 

o If the Source Code Review (SCR) Team determines the code does 

not meet the requirements of the 2005 VVSG, the SCR team will 

review code impacts with the TM. 

o TM determines when code is ready for a Trusted Build. When it is, 

the TM sends approval for a Trusted Build to the Trusted Build team. 

• Receive TDP documents: 

o Delivery department will perform a check-in of the documents on the 

server. 

o Documents are placed in the specific project’s TDP folder with the 

delivery date. 

o Delivery department notifies TM of delivery via email. 

o TM assigns VTS personnel to perform the documentation review. 

o VTS personnel perform a PCA on the pertinent documentation. 

o A peer review is done on the VTS personnel’s review, all written 

discrepancies are verified and an official Discrepancy Report is sent. 

• Receive Hardware 

o The HW Specialist or designee checks in equipment that is delivered 

by ES&S.

o Operational Status Checks are performed on all delivered hardware. 

o When the equipment comes back from the hardware lab testing, the 

HW Specialist checks in the equipment and verifies it’s the same 

hardware that was sent to the lab. 

o The equipment is stored in a secure room with controlled access. 

• Compliance/Trusted Build: 

o The TM assigns a Trusted Build team member to review the 

Compliance/Trusted Build procedures and perform the build. 

o Trusted Build team member follows build procedures exactly as 

written. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 39 of 79 

Template Rev 2015-03 

Managing technology risk 

o Trusted Build team member pulls the finalized source code from the 

specific folder where it was checked in. 

o Once build is complete the Trusted Build team member provides the 

completed build to the SLI Delivery Team to be checked in. 

o Trusted Build team member notifies the TM that the

Compliance/Trusted Build is completed. 

o TM assigns appropriate VTS(s) to load the resulting 

applications/installs and begin testing. 

o The VTS team performs appropriated installations, and begins 

testing. 

2 PRE-CERTIFICATION TESTING AND ISSUES 

This section addresses any pre-certification testing that has been performed on the 

system under review. For this certification project, no previous testing has been 

performed on the ES&S EVS 6.2.0.0 voting system, as it is a newly developed and 

as of yet un-fielded system. 

2.1 Evaluation of prior VSTL testing 

No prior VSTL testing is pertinent to the ES&S EVS 6.2.0.0 voting system, as it is a 

system undergoing an Initial Certification process. 

2.2 Evaluation of prior non-VSTL testing 

No prior state or non-VSTL lab testing is pertinent to the ES&S EVS 6.2.0.0 voting 

system, as it is a system undergoing an Initial Certification process. Review of 

ES&S internal testing is performed during the FCA review. 

2.3 Known Vulnerabilities 

ES&S EVS 6.2.0.0 voting system does not contain a DRE precinct voting device, 

nor does it support public transmissions. 

Within the declared system, the public facing components are the DS200, 

ExpressVote HW1.0, and ExpressVote HW2.1. The DS200 is a precinct ballot 

scanning and tabulation device that processes ballots and/or vote summary cards 

at a polling place. ExpressVote HW1.0 is a Universal Voting System that produces 

an independent voter verifiable paper record. ExpressVote HW2.1 is a Universal 

Voting System that produces an independent voter verifiable paper record and can 

be configured as a precinct tabulator for producing and processing vote summary 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 40 of 79 

Template Rev 2015-03 

Managing technology risk 

cards at a polling place. The DS450 and DS850 devices are central ballot scanning 

and tabulation devices, which are implemented in a secure environment. 

Review of the “Known Vulnerabilities” database, maintained by SLI, has provided 

49 known vulnerabilities to previous ES&S systems, which are already accounted 

for in SLI’s Testing. 

3 PHYISCAL CONFIGURATION AUDIT

3.1 TDP Documentation Analysis & Review 

SLI will complete an assessment of the deliveries of the Technical Data Package, 

including Functional Requirements, Specifications, End-user documentation, 

Procedures, System Overview, Configuration Management Plan, Quality 

Assurance Program, and manuals for each of the required hardware, software and 

firmware components of the ES&S EVS 6.2.0.0 voting system, in order to ensure 

that the documentation corresponds to the configuration and operation of the 

system. Any subsequent re-deliveries of the TDP items will be solely the result of 

fixes to discrepancies identified in the remaining FCA or PCA activities. 

3.1.1 Document Review 

SLI conducts a PCA review of all vendor traced documents submitted for review in 

the delivery of the ES&S EVS 6.2.0.0 TDP. These include the following areas: 

• System configuration overview 

• System functionality description 

• System hardware specifications 

• Software design and specifications 

• System test and verification specifications 

• System security specifications 

• User/system operations procedures 

• System maintenance procedures 

• Personnel deployment and training requirements 

• Configuration management plan 

• Quality assurance program 

Each document included in the delivery of the voting system TDP is reviewed for 

compliance to the pertinent 2005 VVSG, Volume 2, Sections 2.2 through 2.13 and 

Volume 2, Section 6.6 requirements that are applicable to that document. Results 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 41 of 79 

Template Rev 2015-03 

Managing technology risk 

and associated discrepancies of the documentation TDP review will be detailed in 

the final test report. 

3.2 TDP Source Code Analysis & Review 

3.2.1 Source Code Review 

The certification campaign for the ES&S EVS 6.2.0.0 voting system includes 

software and firmware that have been created as proprietary to ES&S as well as 

review of any COTS products. SLI has conducted a source code review of all 

proprietary source code, and modified COTS products, submitted in the delivery of 

the voting system TDP for compliance to the VVSG, version 2005, Volume 2, 

Section 6.6. The coding languages involved in the vendor’s applications include: 

• C 

• C++ 

• C# 

• Java 

• VB 

• VB.net 

Source Code Review Tools utilized by SLI include: 

• LocMetrics Line Counter: a commercial application used to determine the 

counts of executable and comment lines; 

• Module Finder: an SLI proprietary application used to parse module names 

from C/C++ and VB code and populate the identified module names into 

the review documents; 

• ExamDiff Pro: a commercial application used to compare revised code to 

previously reviewed code; 

• Understand: a commercial application to perform automated review of 

source code. 

Any subsequent re-reviews of source code will be the result of fixes to 

discrepancies identified in the FCA activities. 

COTS Operating systems and software used in the voting system will be verified 

as authentic and unmodified, otherwise reviewed as is applicable. 

3.3 QA & CM Process Review 

The review processes employed are designed to verify that the manufacturer not 

only has written processes and procedures in both the Quality Assurance and 

Configuration Management arenas, but that those processes and procedures are 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 42 of 79 

Template Rev 2015-03 

Managing technology risk 

actually implemented within the software development life cycle that is used to 

produce the ES&S EVS 6.2.0.0 version that is submitted for certification. 

The QA portion of the review focuses on the testing performed by ES&S. The 

scope of the testing performed is reviewed in order to verify that ES&S has verified 

that all VVSG requirements are met. SLI reviews the test case design documents 

and data as provided by ES&S. In evaluating each module, with respect to flow 

control parameters and data on both entry and exit, SLI assesses discrepancies 

between the Software Specifications and the test case design. Surveys 

addressing Quality Assurance aspects as implemented by ES&S will be sent to 

ES&S QA personnel. Additionally, interviews of key Quality Assurance staff may 

be conducted to verify that the QA processes and procedures are known, 

understood and implemented by the appropriate personnel within the organization. 

The interviews will be conducted if issues are seen with either project artifacts or 

survey responses. 

The CM portion of the review focuses on the organizations understanding and 

implementation of the declared configuration management processes, procedures 

and policies. Initial deliverables will be reviewed against all pertinent CM processes 

employed by ES&S. Any and all subsequent deliveries will also be reviewed to 

determine that appropriate processes are employed. Surveys addressing 

Configuration Management aspects as implemented by ES&S will be sent to ES&S

CM personnel. Additionally, interviews of pertinent staff, with regard to 

configuration management, may be conducted to verify that processes, procedures 

and policies are known, understood and implemented within the organization. The 

interviews will be conducted if issues are seen with either project artifacts or survey 

responses. 

3.4 Compliance/Trusted Build 

A Compliance Build will be conducted prior to SLI’s formal test execution and will 

be completed on site at SLI’s facility or a secure lab at the vendor’s facility 

approved by SLI. The Trusted Build will be conducted prior to SLI’s final test 

execution and will be completed on site at SLI’s facility or a secure lab at the 

vendor’s facility approved by SLI. SLI will use its approved standard lab procedure 

that details the processes for controlling, managing, and conducting the Trusted 

Build. This process includes the following: 

• Preparation for the Compliance/Trusted Build - Obtaining and reviewing 

ES&S’s procedure for constructing the build platform, verifying the target build 

platform, and acquiring and verifying the necessary materials. 

• Execution of the Compliance/Trusted Build – SLI will perform the 

Compliance/Trusted Build by using the step-by-step build procedure, as 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 43 of 79 

Template Rev 2015-03 

Managing technology risk 

provided by ES&S to create a pristine build environment. SLI records and 

ascertains the following items throughout the build process: 

o Build environment images at various key points 

o Build environment and file hashes at various key points 

o Build environment hardware characteristics 

o Build results from code compilation and file hashes 

o Final software install files and file hashes 

• Deliverables to Testing – Upon completion of the Trusted Build, certain items 

are sent to the SLI test group. The final result will be media containing the 

following: 

o Final software install files

o Hash values to validate install files 

• Final Record Keeping and Archiving Procedures – At the conclusion of the 

Trusted Build process, SLI completes all final record keeping and archiving 

procedures at SLI’s facility. This record keeping includes any unique identifiers, 

results of the build with version numbers and dates and descriptions of all 

hashes and images in the repository. 

4 MATERIALS REQUIRED FOR TESTING 

Any materials that are used in an election cycle must be provided to SLI to facilitate 

testing of the voting system. This section outlines these materials that are required. 

4.1 Software/Firmware 

All software and firmware that is to be used by the declared voting system, whether 

directly or indirectly, in a production environment, must be validated during the 

certification process. 

The following software/firmware is required for the execution of hardware, 

software, telecommunications, and security tests. This includes all supporting 

software such as operating systems, compilers, assemblers, application software, 

firmware, any applications used for burning of media, transmission of data or 

creation/management of databases. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 44 of 79 

Template Rev 2015-03 

Managing technology risk 

4.1.1 Manufacturer Software/Firmware 

The ES&S EVS 6.2.0.0 voting system consists of the following software and 

firmware components: 

• Electionware (Client and/or Server) Election database creation, media 

programming and tally/reporting software 

• DS450 Central Tabulator firmware, Central Count scanner and tabulator 

• DS850 Central Tabulator firmware, Central Count scanner and tabulator 

• DS200 Precinct Tabulator firmware, Precinct scanner and tabulator 

• ExpressVote HW1.0 UVS firmware, Precinct Universal Voting System 

• ExpressVote HW2.1 UVS firmware, Precinct Universal Voting System 

and/or tabulator 

• ExpressVote HW1.0 Previewer ballot preview firmware 

• ExpressVote HW2.1 Previewer ballot preview firmware 

• Event Log Service (ELS) software service monitoring user’s interactions 

with the Election Management System 

• Removable Media Service (RMS) software service supporting election 

media programming 

Note: Versions for each will be available after execution of the Trusted Build, and 

will be listed in the Test Report. 

Table 3 – ES&S EVS 6.2.0.0 Software/Firmware 

Application Version 

Electionware – Client/Server 5.0.0.0 

Event Log Service 1.6.0.0 

Removable Media Service 1.5.0.0 

ExpressVote HW2.1 Previewer 2.4.0.0 

ExpressVote HW1.0 Previewer 1.5.0.0 

ExpressLink 1.4.0.0 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 45 of 79 

Template Rev 2015-03 

Managing technology risk 

4.1.2 COTS Software/Firmware 

This section details the COTS software and firmware utilized within the ES&S EVS 

6.2.0.0 voting system. 

Table 3 – COTS Software/Firmware 

Manufacturer Application Version 

Microsoft Corporation Window 7 Professional SP-1 (64-bit) 

Microsoft Corporation Windows Server 2008 R2, SP-1 (64-bit) 

Microsoft Corporation WSUS Microsoft Windows Offline 

Update Utility 

10.9.2 

Symantec Symantec Endpoint Protection 14.0.0 (64-bit) 

Symantec 

Symantec Endpoint Protection 

Intelligent Updater (File-Based 

Protection) 

20170523-003-

v5i64.exe 

Symantec 

Symantec Endpoint Protection 

Intelligent Updater (Network-Based 

Protection) 

20170522-011-

IPS_IU_SEP.exe 

Symantec 

Symantec Endpoint Protection 

Intelligent Updater (Behavior-Based 

Protection) 

20170516-001-

SONAR_IU_SEP.exe 

Cerberus Cerberus FTP Server – Enterprise 8.0.12 (64-bit) 

Adobe Adobe Acrobat Standard XI 

Microsoft Corporation Visual C++ Redistributable 

vc_redist.x64.exe (64-

bit) 

vc_redist.x86.exe (86-

bit) 

4.1.3 Additional Supporting Test Software 

This section outlines any and all test specific software that will be used in the 

certification campaign. 

Table 3 – Additional Supporting Test Software 

Manufacturer Application Version 

Riverbed WireShark 1.10.6 

 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 46 of 79 

Template Rev 2015-03 

Managing technology risk 

4.2 Equipment 

The following equipment is required for the execution of the hardware, software, 

telecommunications, and security tests. This includes system hardware, general 

purpose data processing and communications equipment, and any test 

instrumentation required. 

4.2.1 ES&S EVS 6.2.0.0 Equipment 

The following manufacturer equipment will be used in testing: 

Table 4 – ES&S EVS 6.2.0.0 Equipment 

Hardware Model 

ExpressVote Activation Card Printer -- 

ExpressVote Universal Voting System HW1.0 

ExpressVote Universal Voting System HW2.1 

DS200 v1.2 

DS200 v1.3 

DS450 v1.0 

DS850 v1.0 

ExpressVote Rolling Kiosk 98-00049 

ExpressVote Voting Booth 87001 

ExpressVote ADA Table 87031 

DS200 Collapsible Ballot Box N/A 

DS200 Plastic Ballot Box 57521 

DS200 Metal Ballot Box N/A 

DS200 Tote Bin 00074 

DS450 Cart 3002 

DS850 Cart 6823 

4.2.2 COTS Equipment 

The following COTS equipment will be used in testing: 

Table 5 – COTS Equipment 

Manufacturer Hardware Model Operating System 

Innodisk USB EDC H 2SE DEEUH 1-

01GI72AC1SB 

N/A 

Symbol Scanner (External) DS9208 N/A 

Zebra Technologies Scanner (Integrated) DS457-SR20009 N/A 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 47 of 79 

Template Rev 2015-03 

Managing technology risk 

Manufacturer Hardware Model Operating System 

OKI Audit Printer Microline 420 N/A 

Dell Report Printer S2810dn N/A 

OKI Report Printer B431DN 

B431D 

N/A 

Tripp Lite Spike Cube SPIKECUBE N/A 

APC Backup power supply 

(Uninterruptible Power 

Supply) 

Back-UPS Pro 

1500 

Back-UPS RS 

1500 

N/A 

Various (EMS 

Networked or 

Standalone 

configuration) 

• Processor: Dual Core 

• RAM: 4 GB, 8 GB 

recommended 

• Hard Disk: 150 GB 

• Keyboard 

• Mouse 

• Monitor: 1280x800 

resolution 

• CD/DVD reader: 16x 

min 

• 2 USB ports: 2.0 min 

• Report Printer: 

w/printer control 

language driver 

N/A Windows 7 

Professional, SP-1 

(64-bit) 

Various (EMS 

Networked server 

configuration) 

• Processor: Dual Core 

or Quad Core 

• RAM: 4 GB, 8 GB 

recommended 

• Hard Disk: 150 GB or 

320 GB 

• Keyboard 

• Mouse 

• Monitor: 1280x800 

resolution 

• CD/DVD reader: 16x 

min 

• 2 USB ports: 2.0 min 

• Report Printer: 

Network printer 

w/printer control 

language driver 

• Ethernet Port 

• Back up power 

supply: 865 Watts / 

Windows Server 

2008 R2, SP-1 (64-

bit) 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 48 of 79 

Template Rev 2015-03 

Managing technology risk 

Manufacturer Hardware Model Operating System 

1500 VA output 

capacity 

• Network Switch: 1 GB 

throughput 

Delkin USB Flash Drive: 

512 MB, 1 GB, 2 GB, 4 

GB, 8 GB, 16 GB 

N/A N/A 

AVID Headphones 86002 N/A 

Seiko Instruments Thermal Printer LTPD-347B N/A 

NCR / Nashua Paper Roll 2320 N/A 

Delkin Compact Flash Memory 

Card 

N/A N/A 

Delkin Compact Flash Memory 

Card Reader/Writer 

6381 N/A 

4.3 Test Materials 

The following test materials are required for the performance of testing including, 

as applicable, test ballot layout and generation materials, test ballot sheets, test 

ballot cards and control cards, standard and optional output data report formats, 

and any other materials used in testing. 

• Ballots and blank ballot grade paper 

• Ballot pens 

• Compact Flash memory cards 

• USB flash drives 

• Printer paper rolls 

4.4 Deliverable Materials 

The following are documents and materials to be delivered as a part of the ES&S 

EVS 6.2.0.0 voting system. 

• System Overview 

• System Functionality Description 

• DS200 Operator’s Guide 

• DS450 Operator’s Guide 

• DS850 Operator’s Guide 

• EVS Event Log Service User’s Guide 

• Electionware Vol. I: Administrator Guide 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 49 of 79 

Template Rev 2015-03 

Managing technology risk 

• Electionware Vol. II: Define User Guide 

• Electionware Vol. III: Design User Guide 

• Electionware Vol. IV: Deliver User Guide 

• Electionware Vol. V: Results User Guide 

• Electionware Vol. VI: Appendices 

• ExpressVote Operator’s Guide (HW1.0) 

• ExpressVote Operator’s Guide (HW2.1) 

• Requirements of the 2005 VVSG Trace to Vendor Testing 

• Requirements of the 2005 VVSG Trace to Technical Data Package 

• DS200 Hardware Specification 

• DS450 Hardware Specification 

• DS850 Hardware Specification 

• ExpressVote Hardware Specification HW1.0 

• ExpressVote Hardware Specification HW2.1 

• System Development Program 

• Coding Standards 

• License Agreements for Procured Software 

• DS200 – Software Design Specification 

• DS450 – Software Design Specification 

• DS850 – Software Design Specification 

• Electionware – Software Design Specification 

• Event Log Service – Software Design Specification 

• ExpressVote (HW1.0) Software Design Specification 

• ExpressVote (HW2.1) Software Design Specification 

• Electionware PostSQL Table and Field Descriptions 

• Electionware XML Diagrams 

• System Test Plan 

• Usability Test Report: DS200 Precinct-Based Scanner and Tabulator 

• Usability Test Report: ExpressVote Universal Voting System (HW1.0) 

• Usability Test Report: ExpressVote Universal Voting System (HW2.1) 

• EMS Client Workstation Secure Setup & Configuration Guide 

• EMS Server Secure Setup & Configuration Guide 

• Standalone EMS Workstation Secure Setup & Configuration Guide 

• Voting System Security Specification 

• Security Script Description 

• Verification Procedure: DS200 Precinct Scanner and Tabulator 

• Verification Procedure: DS450 High-Throughput Scanner & Tabulator 

• Verification Procedure: DS850 High-Speed Scanner & Tabulator 

• Verification Procedure: Election Management System Workstation and 

Server 

• Verification Procedure: ExpressVote Hardware 1.0 

• Verification Procedure: ExpressVote Hardware 2.1 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 50 of 79 

Template Rev 2015-03 

Managing technology risk 

• Verification Procedure: Verification PC Setup 

• Validation File List: DS200 

• Validation File List: DS450 

• Validation File List: DS850 

• Validation File List: Event Log Service 

• Validation File List: ExpressVote HW1.0 

• Validation File List: ExpressVote HW2.1 

• Validation File List: ExpressVote HW1.0 Previewer 

• Validation File List: ExpressVote HW2.1 Previewer 

• Validation File List: Electionware 

• Validation File List: Removable Media Service 

• DS200 Maintenance Manual 

• DS450 Maintenance Manual 

• DS850 Maintenance Manual 

• ExpressVote Maintenance Manual (HW1.0) 

• ExpressVote Maintenance Manual (HW2.1) 

• Personnel Deployment and Training Program 

• Configuration Management Program 

• Technical Documentation Program 

• Manufacturing Quality Assurance Program 

• Software Quality Assurance Program 

• Ballot Production Guide for EVS 

• Conformity Statement: 2005 VVSG 

• COTS Production Implementation Plan 

• EAC Application Requirements Trace 

5 TEST SPECIFICATIONS 

The following are the specifications for testing to be conducted on ES&S EVS 

6.2.0.0 voting system. The specifications contain details on the focus of testing, 

configuration(s), and the functions to be tested. 

5.1 Requirements 

The ES&S EVS 6.2.0.0 will be tested to the approved 2005 VVSG requirements. 

All requirements within the VVSG are verified against the ES&S EVS 6.2.0.0 voting 

system unless noted otherwise in the “Excluded Requirements” section below. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 51 of 79 

Template Rev 2015-03 

Managing technology risk 

ES&S has not declared a DRE for the ES&S EVS 6.2.0.0 voting system. As such, 

VVSG 1.0 DRE specific requirements are not scheduled to be evaluated, as listed 

in the “Excluded Requirements” section. 

To evaluate the system test requirements, each section of the EAC 2005 VVSG 

shall be analyzed to determine the applicable tests, EAC 2005 VVSG Volume I 

sections, as well as the accompanying strategy for evaluation, are described 

below: 

• Section 2: Functional Requirements – The requirements in this section 

shall be tested during the FCA, utilizing the SLI test suites specially 

designed for the ES&S EVS 6.2.0.0 voting system. The data input during 

these tests shall be the predefined election definitions. 

• Section 3: Usability and Accessibility – The requirements in this section 

shall be tested during the Usability Test, utilizing SLI test suites developed 

to cover the applicable requirements of this section. 

• Section 4: Hardware Requirements – The requirements in this section 

shall be tested and/or overseen by SLI personnel, per Sections 4.4.2 and 

6.3.1. 

• Section 5: Software Requirements – The requirements in this section 

shall be tested during source code review, TDP review, and FCA. A 

combination of review and functional testing shall be performed to verify 

these requirements are met. The functional testing will occur if the source 

code review reveals potential areas of issue within the code that are not 

explicitly covered by the VVSG source code requirements. The functional 

test team will work with the source code review team to determine what 

testing will need to be done in order to verify no issue is resident within the 

inspected code. 

• Section 6: Telecommunication – The requirements in this section shall be 

tested during source code review, FCA, and Security Tests, utilizing SLI test 

suites custom designed for the ES&S EVS 6.2.0.0 voting system. 

• Section 7: Security Requirements – The requirements in this section shall 

be tested during source code review, FCA, and Security Tests, utilizing SLI 

test suites custom designed for the ES&S EVS 6.2.0.0 voting system. 

• Section 8: Quality Assurance (QA) Requirements – The requirements in 

this section shall be tested throughout the test campaign via various 

methods. The TDP review shall be performed on ES&S QA documentation 

to determine compliance to EAC 2005 VVSG requirements and the 

requirements stated in the ES&S QA Program document. All source code 

shall be checked to ensure that proper QA documentation has been 

completed. All equipment received for initial testing and follow-up testing 

shall be checked against ES&S documentation to ensure their QA process 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 52 of 79 

Template Rev 2015-03 

Managing technology risk 

is being followed. SLI personnel will complete the requirements of EAC 

2005 VVSG Vol. 2, Section 7, Quality Assurance Testing and Section 

1.3.1.5, Focus of Vendor Documentation that requires SLI personnel to 

examine documents or conduct an external evaluation utilizing equipment, 

documents and support information provided by ES&S during the test 

campaign. SLI may also choose to interview ES&S’s QA staff for further 

evaluation. 

• Section 9: Configuration Management (CM) Requirements – The 

requirements in this section shall be tested throughout the test campaign. 

The TDP review shall be performed on the ES&S configuration 

management documentation to determine EAC 2005 VVSG compliance and 

to further determine whether ES&S is following its documented CM 

requirements. Any discrepancies shall be formally reported to ES&S and the 

EAC. SLI personnel may conduct an audit of the ES&S CM Program at the 

ES&S facility, if deemed necessary. 

5.2 Hardware Configuration and Design 

The ES&S EVS 6.2.0.0 as declared in the application for certification submitted to 

the EAC, consists of: 

• An Electionware workstation with minimum requirements of 150 GB hard 

disk, 8 GB of recommended memory and Dual Core processing speed. The 

networked workstation with minimum requirements of 150 GB hard disk, 8 

GB of recommended memory and Dual Core processing speed includes a 

server with minimum requirements of 320 GB hard disk, 4 GB of memory 

and Quad Core processing speed. 

• At the precinct level, DS200 tabulators and ExpressVote HW2.1 Universal 

Voting Systems are employed. 

• The central count location employs scanners, in combination with 

workstations that utilizes the DS450 and DS850 software and hardware for 

tabulation of ballots. 

• The consolidation, tally and reporting location employs a workstation with 

Electionware software, with minimum requirements of 150 GB hard disk, 8 

GB of recommended memory and Dual Core processing speed, as well as 

a printer. The networked workstation with minimum requirements of 150 GB 

hard disk, 8 GB of recommended memory and Dual Core processing speed 

includes a server with minimum requirements of 320 GB hard disk, 4 GB of 

memory and Quad Core processing speed, as well as either a direct 

connect or a network printer. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 53 of 79 

Template Rev 2015-03 

Managing technology risk 

5.3 ES&S EVS 6.2.0.0 Voting System Review 

This section describes the workings of the ES&S EVS 6.2.0.0 voting system and 

will assist the reader with understanding the flow of the voting system. 

5.3.1 Define 

The Define group is used to input critical data for your election. The Define 

group modules are Home, Capture, and Element Library. 

5.3.1.1 Home Module 

The Home module is where election management, template management, and 

equipment selection options take place. New elections are created within the Home 

module as well. Use the Create New Election feature to create a new election, give 

it a name, and set some basic parameters. A county must be selected to create an 

election. Enter the election information. This includes data about the jurisdiction 

and jurisdiction ID (such as county number), this particular election (election type), 

any alternate names/IDs, and the rules profile (state-specific rules that apply to this 

election). 

Existing elections created in Electionware will appear under the Home module and 

can be reopened from here for editing and management. Additional functionality 

includes backing up, restoring, and deleting elections. 

5.3.1.2 Capture Module 

The Capture module is where the election data is entered. Items such as 

languages, audio files, precincts, districts, contests, candidates, and ballot sets are 

in this module. 

The Capture Status screen opens when the Capture module is opened. This 

screen is split into two sections. 

Election Properties displays general information about the current election, 

including the template, jurisdiction information, election date, description, type, 

management of languages, audio, parties, precincts, districts, contests, 

candidates, polling place and ballot styles. 

Capture Status displays the current status and last time/date stamp for captured 

election information, including Election Data Last Imported and Ballot Styles Last 

Generated. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 54 of 79 

Template Rev 2015-03 

Managing technology risk 

5.3.1.3 Elements Module 

The Element Library module is where electronic elements (such as system 

translations, audio and graphics, and equipment screen definitions) are managed 

and customized. These features include both system files (instructional text and 

audio displayed/played for the user) and screen graphics. 

The Element Library module enables you to manage files and support data for 

ADA accessible voting equipment. These functions include the management of 

system audio files and translations. System audio and translations are used to 

instruct voters on audio and on-screen ballot navigation. 

The Export ExpressVote System Translations Script feature is used to export 

system prompt translations for editing. The ExpressVote can support voting in 

multiple languages. The Element Library module is used to export system 

translations in the form of a spreadsheet for editing. 

The Import ExpressVote System Translations feature is used to import the system 

translations from the spreadsheet you exported previously and incorporate them 

into your election. 

The Export ExpressVote System Audio Script feature is used to export the 

instructional system script for audio recording purposes only, for voters using the 

ExpressVote accessible equipment. 

The Import ExpressVote System Audio Recordings feature is used to import 

recorded instructional audio file sets for the ExpressVote accessible equipment. 

The Import Election Header Graphic feature imports election headers for the ES&S 

equipment. 

5.3.2 Design 

The Design group is for designing ballot and audio elements for an election. The 

Design group modules are Paper Ballot and Accessible Ballot. 

5.3.2.1 Paper Ballot Module 

The Paper Ballot module incorporates the data in your election database (created 

in the Capture module) with finished ballot layouts. Paper Ballot can produce four 

distinct ballot layout types, depending on the needs of your jurisdiction: Column 

Portrait, Grid-Landscape, Grid-Portrait, Paper Flexible Grid. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 55 of 79 

Template Rev 2015-03 

Managing technology risk 

5.3.2.2 Accessible Ballot Module 

The Accessible Ballot module is used to format and manage display options and 

ADA audio settings, and validate data before generating election files for ES&S 

accessible voting equipment. 

The Export Ballot Audio Scripts feature is used to export the ballot audio script for 

the current election, for audio recording purposes only, for voters using accessible 

equipment. 

Validate Data is used to verify the status of the election and system translations 

and audio for accessible equipment. 

5.3.3 Deliver 

The Deliver group is used to configure equipment, package election data and print 

ballots on demand. The Deliver group modules are Configure, Package, and Print. 

5.3.3.1 Configure Module 

In the Configure module, you can manage election equipment and security settings 

and generate election files for your ES&S digital ballot scanning and vote 

tabulation equipment, and voter-assistive devices. 

Election Properties displays general information about the current election, 

including the template, jurisdiction information, election date, description, and type. 

The DS/ExpressVote Equipment Security feature is for setting up the security 

codes for your digital scanner and ExpressVote equipment. When an election is 

loaded onto election equipment, the user must enter the correct code on the device 

to access the control screens. 

DS200 Settings is used to set up poll place options, configure ballot images and 

handling, and select report options for the DS200. 

Use ExpressVote Settings to configure vote session, audio, card printing/handling, 

and terminal properties, poll place options, and reporting options, for the 

ExpressVote. 

5.3.3.2 Package Module 

The Package module is for creating Qualification, Election, and Reporting media 

(election files saved on an ES&S portable flash drive or compact flash card). 

Qualification Media contains the election qualification code (EQC), election keys, 

and security codes required by the DS200, ExpressVote equipment, and Central 

Count tabulator. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 56 of 79 

Template Rev 2015-03 

Managing technology risk 

The DS200 election media contains the election definition required by the DS200, 

including election-related images, and folders for result collections. Election 

Qualification Media must be created before creating DS200 Election Media. 

The Central Count election media device is a flash drive containing the election 

definition required by the Central Count tabulator, including election related 

images, and folders for result collections. 

The ExpressVote election media device is a flash drive containing the election 

definition required by the ExpressVote, including election-related images, and 

folders for result collections. 

5.3.4 ExpressVote Universal Voting System, HW1.0 

The ExpressVote® Universal Voting System, HW1.0, is a vote capture and 

marking device designed for all voters, including non-native English speakers, nonEnglish speakers, and those who need special assistance. 

ExpressVote HW1.0 uses a touch screen, an optional external barcode scanner, 

and/or assistive technology to capture a voter’s contest choices. The system 

prevents overvotes and can alert voters about undervotes. 

The ExpressVote’s integrated printer marks a voter’s contest selections on a 

verifiable hardcopy vote summary card in human-readable format. The vote 

summary card enables a voter to confirm that the ExpressVote HW1.0 captured 

all votes correctly, alleviating potential concerns about system accuracy and 

integrity. Marked vote summary cards can be tabulated using the ES&S DS200, 

DS450, DS850 tabulators or the ExpressVote HW2.1 if it is configured as a 

tabulator. 

Voters with visual impairments can use headphones to hear audio instructions and 

can make selections using an audio-tactile keypad with Braille legends. Voters with 

other physical requirements can make selections using a rocker switch device or a 

sip-and-puff device. Voters can use zoom and contrast display settings to make 

on-screen text more readable. Jurisdictions can include multiple language support 

for non-English speakers. 

5.3.5 ExpressVote Universal Voting System, HW2.1 

The ExpressVote® Universal Voting System, HW2.1, is a vote capture and 

marking device designed for all voters, including non-native English speakers, nonEnglish speakers, and those who need special assistance. 

The election definition controls whether the ExpressVote HW2.1 functions as a 

tabulator or as a vote summary card marker only. If you use the ExpressVote as a 

marker, marked vote summary cards can be tabulated using the ES&S DS200, 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 57 of 79 

Template Rev 2015-03 

Managing technology risk 

DS450, or DS850 tabulators. If you use the ExpressVote HW2.1 as a tabulator, 

the system saves a digital image of the marked vote summary card after a voter 

verifies all contest choices. The system tabulates all captured votes after the 

polling location closes. 

ExpressVote HW2.1 uses a touch screen, an optional external barcode scanner, 

and/or assistive technology to capture a voter’s contest choices. The system 

prevents overvotes and can alert voters about undervotes. 

The ExpressVote’s integrated printer marks a voter’s contest selections on a 

verifiable hardcopy vote summary card in human-readable format. The vote 

summary card enables a voter to confirm that the ExpressVote HW2.1 captured 

all votes correctly, alleviating potential concerns about system accuracy and 

integrity. Marked vote summary cards can be tabulated using the ES&S DS200, 

DS450, or DS850 tabulators or the ExpressVote HW2.1 if it is configured as a 

tabulator. 

Voters with visual impairments can use headphones to hear audio instructions and 

can make selections using an audio-tactile keypad with Braille legends. Voters with 

other physical requirements can make selections using a rocker switch device or a 

sip-and-puff device. Voters can use zoom and contrast display settings to make 

on-screen text more readable. Jurisdictions can include multiple language support 

for non-English speakers. 

5.3.6 DS200 

The DS200 is a precinct-based ballot scanner and vote tabulator that is part of a 

jurisdiction-wide election system. Voters insert their ballots directly into the DS200

at the polling place. The DS200 scans ballots, tabulates votes and feeds inserted 

ballots into an attached, secure ballot box. 

The DS200 reads marks on both one- and two-sided ballots. The DS200 can 

discriminate between valid marks (pen) within the ballot target locations and 

extraneous marks, perforations, smudges, and folds.

The DS200 can generate paper reports produced from the unit’s internal thermal 

printer. The DS200 stores election results on a removable USB flash drive that can 

be used to transfer scanner results to Electionware after the polls are closed. 

The DS200 operates in three separate modes, each of which provides access to 

different types of functions. The administration mode provides access to diagnostic 

and testing functions to calibrate and test the scanner. 

In the polls open mode, the DS200 actively scans ballots and tabulates results. 

In the polls closed mode, the DS200 prints poll reports and prepares voting results 

for transfer to election headquarters. Election officials or poll workers close the 

polls at the assigned time. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 58 of 79 

Template Rev 2015-03 

Managing technology risk 

5.3.7 DS450 

The DS450 is a high-throughput central count scanner and tabulator with ballot 

sorting capabilities. The DS450 is designed to process a wide variety of ballot 

types and lengths, including folded ballots and printed ExpressVote vote summary 

cards. 

The DS450 reads marks on both one- and two-sided ballots. The DS450 can 

discriminate between valid marks (pen marks within the ballot target locations) and 

extraneous marks, perforations, smudges, and folds.

Voter accumulation records are saved on the machine’s hard drive and can be 

transferred from the hard drive to a USB flash drive. 

5.3.8 DS850 

The DS850 is a high-speed central-count scanner and tabulator with ballot sorting 

capabilities. The DS850 is designed to process a wide variety of ballot types and 

lengths, including folded ballots and printed ExpressVote vote summary cards. 

The DS850 reads marks on both one- and two-sided ballots. The DS850 can 

discriminate between valid marks (pen marks within the ballot target locations) and 

extraneous marks, perforations, smudges, and folds.

Voter accumulation records are saved on the machine’s hard drive and can be 

transferred from the hard drive to a USB flash drive. 

5.3.9 Electionware Reporting Module 

After the election, the Reporting module is used to import tabulated results, 

machine logs, cast vote records, and ballot images by reading the election media 

from the ExpressVote HW1.0, ExpressVote HW2.1, DS200, DS450 and DS850, 

media; review, export, and report election results and media device-related data; 

and review/adjudicate ballot images. 

The Reporting module can display or export detailed status information about 

individual flash drives, and produce machine log reports. It can also clear loaded 

results files, in order to delete test data or to start the loading process over. 

The Ballot Review provides the adjudicator for listing scanned ballot images based 

on configurable filter criteria and adjudicating ballot exceptions 

In the Reporting module, use the Reports menu to generate precinct status, 

machine log file, media load, Central Count networked results, double vote, writeins, and manual entry status reports. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 59 of 79 

Template Rev 2015-03 

Managing technology risk 

5.4 Testing Strategies 

This section looks at the voting system and the individual components in order to 

determine testing strategies at the component level as well as the system level. 

5.4.1 How each Polling Place Device will be tested 

• How the ExpressVote HW1.0 will be tested 

o ExpressVote HW1.0 will be tested first as an individual component in

order to verify that all declared functionality is present and working as 

documented, with Electionware produced media and data, then as 

an integrated piece of the voting system where it will accept user 

input instructions, prior to producing marked vote summary cards that 

mirror user intent. 

• How the ExpressVote HW2.1 will be tested 

o ExpressVote HW2.1 will be tested first as an individual component in

order to verify that all declared functionality is present and working as 

documented, with Electionware produced media and data, then as 

an integrated piece of the voting system where it will accept user 

input instructions, prior to producing marked vote summary cards that 

mirror user intent.

• How DS200 will be tested 

o DS200 will be tested first as an individual component in order to 

verify that all declared functionality is present and working as 

documented, then as an integrated piece of the voting system where 

it will input Electionware produced media and data, then accept user 

filled out ballots as well as vote summary cards, prior to producing all 

defined output medias.

5.4.2 How each Application will be tested 

• How Electionware will be tested 

o Electionware will be tested first as an individual component in order 

to verify that all declared functionality is present and working as 

documented, then as an integrated piece of the voting system where 

it will output Electionware produced media and data, which will feed 

into ExpressVote HW1.0, ExpressVote HW2.1, DS200, DS450, 

DS850 and Electionware Reporting.

• How the DS450 and DS850 will be tested 

o DS450 and DS850 will be tested first as individual components in 

order to verify that all declared functionality is present and working as 

documented, then as an integrated piece of the voting system where 

it will input Electionware produced media and data, then accept user 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 60 of 79 

Template Rev 2015-03 

Managing technology risk 

filled out ballots as well as vote summary cards, prior to producing all 

defined output medias.

• How Electionware Reporting will be tested 

o Electionware Reporting will be tested first as an individual 

component in order to verify that all declared functionality is present 

and working as documented, then as an integrated piece of the 

voting system where it will input Electionware produced media and 

data, then accept media data from ExpressVote HW1.0, 

ExpressVote HW2.1, DS200, DS450 and DS850, prior to producing 

all defined output medias.

5.4.3 How different System Level Configurations will be tested 

As noted earlier, Electionware, DS450, DS850 and Electionware Reporting are 

each capable of being run as standalone instantiations or networked with a central 

database. When run as a standalone implementation, given these possible 

configurations, the following configurations will be exercised: 

• Electionware will be tested in standalone mode 

• Electionware Reporting will be tested in standalone mode 

• DS450 will be tested in standalone mode 

• DS850 will be tested in standalone mode 

• Electionware will be tested in a networked mode 

• DS450 will be tested in a networked mode with a central repository 

• DS450 will be tested in a networked mode with Electionware 

• DS450 will be tested in a networked mode with other DS450 units 

• DS850 will be tested in a networked mode with a central repository 

• DS850 will be tested in a networked mode with Electionware 

• DS850 will be tested in a networked mode with other DS850 units 

• Electionware Reporting will be tested in a networked mode 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 61 of 79 

Template Rev 2015-03 

Managing technology risk 

5.5 Software System Functions 

The ES&S EVS 6.2.0.0 voting system operations documentation has been 

reviewed in conjunction with the Supported Functionality Declaration provided by 

ES&S. Based on this review, the applicable system functions have been identified 

for testing. The following key areas of voting system functionality will be evaluated 

during test suite design. 

5.5.1 Election Definition Creation 

The Election Definition focus will target creation of contests, candidates, 

propositions, ballot formatting and instruction. All aspects of creating regional 

districts, jurisdictional parameters, grouping and displaying of associated election 

data will be analyzed and tested. In addition, the ability to support baseline election 

types, various other election types, voting variations and languages will be verified. 

5.5.2 Election Media Creation 

This area focuses on the creation and handling of media for the purposes of 

installing election data onto voting devices, as well as the creation of physical ballot 

layouts and creation of any/all media used to hold/transfer election data. 

5.5.3 Pre-voting Aspects 

Pre-voting aspects include pre-election preparatory, diagnostic, and election 

verification functions of a voting system. The focus will include device preparation, 

all required pre-voting tasks, and verification of manufacturer recommended prevoting tasks. 

5.5.4 Voting Aspects – Polling Place 

Polling place aspects include all required and additional supported voting functions, 

including HAVA compliant requirements. This area will focus on all aspects of 

election functions and capabilities at the polling place from opening of the polls 

through closing the polls. 

5.5.5 Voting Aspects – Central Count 

The focus of the central count functions is primarily the usage of a scanners to 

scan large quantities of absentee ballots and passing each image to Electionware 

Reporting for interpretation of the voter’s markings on the ballot and/or vote 

summary card. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 62 of 79 

Template Rev 2015-03 

Managing technology risk 

5.5.6 Post Voting Aspects 

This area will focus on all required election post-voting functions. This includes any 

additional supported election functions performed after closing the polls, device 

auditing, transmission, and reporting aspects of the voting system. 

5.5.7 Error Messaging and Recovery 

This area will focus on the system’s ability to generate appropriate error messaging 

within each system component and the system’s ability to recover from error 

conditions in order to proceed with all election functions. 

5.5.8 Auditing 

This area will focus on device and system level auditing capabilities and will verify 

at a minimum the required audit functionally. This includes audit trail capability 

throughout the life cycle of the voting system and audit log content requirements. 

5.5.9 Security 

Overall system and device level logical and physical security aspects will be tested. 

Physical security will focus on the areas of integrity (ballot box doors, locks and 

seals) and detection (compromised ballot box doors, locks or seals). Logical 

security will focus on the areas of access controls, accountability, confidentiality, 

and integrity. These logical security areas will be applied to the OS, Database, 

Network and Application entities used by the EMS, Universal Voting Systems and 

tabulators used by the voting system under test. 

5.6 Test Suite Design 

5.6.1 Hardware Qualitative Examination Design 

SLI will review all reports submitted by the manufacturer of previous testing 

conducted by acceptable hardware test labs, on the equipment contained in the 

ES&S EVS 6.2.0.0 voting system. The results will be compared against the 

guidelines of the VVSG 2005, Section 2 to identify any additional testing required. 

In addition, SLI will create the following test suites to focus on Maintainability, 

Accessibility, and Usability of the voting system: 

• Accessibility - Accessibility requirements for a voting system generally 

include both objective and observable requirements. In combination, the two 

types of requirements verify that the voting system components are 

accessible to as many eligible voters as possible, including those that have 

a type of challenge that creates a need for assistance of some type. The 

voting systems should be self-contained such that the individual voter is 

able to cast their vote without assistance from another party. Accessibility 

calls for the voting system to take into account vision, varying degrees of 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 63 of 79 

Template Rev 2015-03 

Managing technology risk 

vision, dexterity, mobility, aural issues, and speech and language 

proficiency. 

• Maintainability - Maintainability encompasses a range of maintenance 

actions that examine all scheduled and unscheduled events in place for 

preventing failures on all hardware devices. Testing verifies the ease with 

which maintenance actions can be performed based on the design 

characteristics of the equipment and software. Non-technical election 

workers are able to be made aware of the problem through the equipment 

and software’s ability to correctly self-diagnose problems. 

• Usability – Usability is defined as a measure of the effectiveness, 

efficiency, and satisfaction achieved by a specified set of users with a given 

product in the performance of specified tasks. In the context of voting, the 

primary user is the voter, the product is the voting system, and the task is 

the correct recording of the voter ballot selections. Testing is conducted to 

ensure voters are able to negotiate the process effectively, efficiently and 

comfortably according to the requirements dictated.

5.6.2 Hardware Environmental Test Design 

Hardware environmental testing is performed to verify conformance to Vol. 1, 

Section 4 of the VVSG 2005. Testing will be accomplished through a combination 

of testing performed by SLI and testing performed by subcontractor labs as listed 

in “Table 2 – Labs Performing Hardware Testing”. Specific hardware test plans 

from the subcontractor labs are included in Attachments B & C. 

5.6.3 Software Module Test Design and Data 

Incorporating the manufacturer’s software specifications, SLI will validate that all 

software/firmware components of the system adhere to expected flow control 

parameters and specifications for data input and output. 

5.6.4 Software Functional Test Design and Data 

SLI will prepare functional test modules using the operator/user procedures 

contained within ES&S’s TDP. Functionality provided by ES&S EVS 6.2.0.0 voting 

system is exercised in order to verify that each functional component performs as 

expected. Accept/reject criteria are based on requirements of the VVSG and the 

system specification documents provided within the TDP. As many of the individual 

functional components rely on preceding functionality within the system, SLI 

incorporates system level suites that employ modules that exercise the individual 

functional components of the system. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 64 of 79 

Template Rev 2015-03 

Managing technology risk 

5.6.5 System-level Test Design 

Testing of the voting system involves exercising the specific functions of each 

component of a voting system as well as the entire voting system. Testing will 

focus on the functionality of an election management system, the polling place 

devices, and devices required for communications and data loading and will then 

focus on functionality of the integrated voting system. 

There are various types of testing. Table 8 - Types of Testing provides the 

descriptions of these kinds of tests and their associated benefit. 

Table 8 - Types of Testing 

Type of Testing Description Benefit 

Nominal Conditions Testing all nominal functional 

capabilities of all components of the 

voting system. 

Nominal conditions testing ensures that 

the voting system is tested against all 

elements. 

Injection Attack This testing is primarily for verifying the 

security of databases. An injection 

attack consists of insertion or 

"injection" of a SQL query via the input 

data from the client to the application. 

A successful SQL injection exploit can 

read sensitive data from the database, 

modify database data 

(Insert/Update/Delete), execute 

administration operations on the 

database, recover the content of a 

given file existing on the DBMS file 

system and, in some cases, issue 

commands to the operating system. 

Injection Attack testing hardens the 

database against unauthorized access 

and manipulation of data. 

Data Driven This is testing that is based on the 

data that is used throughout the voting 

system. For example, if ‘x’ is input in a 

field, it would branch to a different part 

of the application than if ‘y’ was 

entered. The goal is to ensure that all 

branches get tested. 

The tests verify that each unique data 

element can be accessed and used 

according to the voting system’s user 

documentation. These test cases are 

based on the election data or vote data 

used and not the functional flow or work 

flow of the voting system. 

Usability The purpose of UI Testing is to test all 

of the screen and data elements that 

exist on each and every screen. SLI 

will verify responses to input, text 

syntax, error message content, and 

audit message input. 

These tests verify every action will work 

that a user can perform on a screen. 

These tests will also verify that any 

screen or data element will not take the 

user by surprise. 

Error Messaging and 

Recovery 

Exercise system’s ability to recover 

from hardware, software, and data 

errors. 

Ensures that the system is able to 

successfully recover should there be a 

system or data error. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 65 of 79 

Template Rev 2015-03 

Managing technology risk 

End-to-End This is system testing in a true enduser environment following all preelection day, election day, and postelection day voting rules and 

processes. 

This is used to demonstrate that a system 

can be used to perform its job following 

the exact set of processes and steps that 

would be used by the target customer or 

end-user. 

Regression Testing that validates that existing 

functionality is unchanged with the 

introduction of new functionality and 

correction of defects. 

This is used to verify that issues are 

corrected, and that no know issues are 

been introduced or uncovered. 

Formal Execution Final verification of the voting system 

in a true end-user environment, 

following all pre-election day, election 

day, and post-election day voting rules 

and processes. 

Testing “for the record” that includes all 

aspects of the voting system, verifying 

that all VVSG requirements are met. 

Volume Test Testing the voting system’s response 

to conditions that range from 

processing more than the expected 

number of ballots/voters per precinct to 

processing more than the expected 

number of precincts to any other 

similar volume conditions. 

Determine if there are limits to the voting 

system’s ability to operate under 

conditions that tend to overload the 

system’s capacity to process, store, and 

report data. 

Stress Tests Testing the voting system’s responses 

to transient overload conditions by 

subjecting polling place devices to 

ballot processing at high volume rates. 

Evaluates the voting system and 

software’s response to hardwaregenerated interrupts and wait states. 

Accessibility Test Exercises system capabilities of

features for voters with disabilities. 

Validates that the voting system is tested 

against all applicable ADA and HAVA 

requirements for voters with disabilities. 

Performance Tests Tests accuracy, processing rate, ballot 

format, handling capability and other 

performance attributes specified by 

ES&S. 

Performance testing ensures that the 

voting system meets all performance 

elements. 

5.6.6 Software Functional Test Case Design 

As described in the “Testing of the System” section above, each individual 

component of the voting system will be reviewed independently, such that all 

functionality present in the component is verified to work as documented, and that 

all functionality is appropriately documented. For the components that are able to 

be networked, this will be reviewed as well. This approach corresponds to the 

following functional testing to be done. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 66 of 79 

Template Rev 2015-03 

Managing technology risk 

5.6.6.1 Electionware – Standalone workstation 

All functionality present in Electionware, including those covered in the “System 

Review” and “Supported Functionality” sections above, is verified to work as 

documented, and that all functionality is appropriately documented. 

This test covers Electionware. 

5.6.6.2 Electionware – Client/Server configuration 

Testing of the client/server configuration, for Electionware, will be the focus of this 

testing, such that reliability of data consistency is verified. 

This test covers Electionware, in a server and multi-workstation configuration. 

5.6.6.3 DS450 and DS850 – Standalone workstation 

All functionality present in DS450/DS850, including that covered in “System 

Review” and “Supported Functionality” sections above, is verified to work as 

documented, and that all functionality is appropriately documented. 

This test covers DS450/DS850. 

5.6.6.4 DS450 and DS850 – Client/Server configuration 

Testing of the client/server configuration, for DS450/DS850, will be the focus of this 

testing, such that reliability of data consistency is verified. 

This test covers DS450/DS850, in a server and multi-workstation configuration. 

5.6.6.5 Electionware Reporting – Standalone workstation 

All functionality present in Electionware Reporting, including that covered in 

“System Review” and “Supported Functionality” sections above, is verified to work 

as documented, and that all functionality is appropriately documented. 

This test covers Electionware Reporting. 

5.6.6.6 Reporting – Client/Server configuration 

Testing of the client/server configuration, for Electionware Reporting, will be the 

focus of this testing, such that reliability of data consistency is verified. 

This test covers Electionware Reporting, in a server and multi-workstation 

configuration. 

5.6.6.7 DS200 

All functionality present in DS200, including that covered in the “System Review” 

and “Supported Functionality sections above, is verified to work as documented, 

and that all functionality is appropriately documented. 

This test covers DS200. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 67 of 79 

Template Rev 2015-03 

Managing technology risk 

5.6.6.8 ExpressVote HW1.0 

All functionality present in ExpressVote HW1.0, including that covered in the 

“System Review” and “Supported Functionality sections above, is verified to work 

as documented, and that all functionality is appropriately documented. 

This test covers ExpressVote HW1.0.

5.6.6.9 ExpressVote HW2.1 

All functionality present in ExpressVote HW2.1, including that covered in the 

“System Review” and “Supported Functionality sections above, is verified to work 

as documented, and that all functionality is appropriately documented. 

This test covers ExpressVote HW2.1. 

5.6.7 System-level Test Suite Design 

System level test suites will include the following: 

• Election Validations - Election suites are created to replicate each type of 

election that can be implemented by the jurisdiction. Within the election 

types, pertinent voting variations that are applicable to that type of election 

will be validated and verified. Each suite will have a particular focus in order 

to test the voting system’s implementation of a given requirement or set of 

requirements. Each test suite below is explained in terms of the components 

of the voting system that will be addressed. 

o GenVariation1

Additional definition is added, with a focus on validating N of M 

voting, Partisan offices, Non-Partisan Offices, Ranked Order Voting, 

Straight Party Voting, Ballot Rotations, Ballot Formatting, precincts 

and districts, and Tally and Reporting functionality. 

This test covers Electionware, ExpressVote HW1.0, ExpressVote 

HW2.1, DS200, DS450 and DS850 as well as Electionware 

Reporting. Vote counts will be accumulated from DS450/DS850, 

ExpressVote HW2.1 and DS200. 

o GenVariation2

Additional definition is added, with a focus on validating Recalls, 

Cross Party Endorsement, and Write-Ins. 

This test covers Electionware, ExpressVote HW1.0, ExpressVote 

HW2.1, DS200, DS450 and DS850 as well as Electionware 

Reporting. Vote counts will be accumulated from DS450/DS850, 

ExpressVote HW2.1 and DS200. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 68 of 79 

Template Rev 2015-03 

Managing technology risk 

o GenVariation3

Additional definition is added, with a focus on validating HAVA 

concerns, as well as items such as pre-vote capabilities. 

This test covers Electionware, ExpressVote HW1.0, ExpressVote 

HW2.1, DS200, DS450 and DS850 as well as Electionware 

Reporting. Vote counts will be accumulated from DS450/DS850, 

ExpressVote HW2.1 and DS200. 

o PriOpen

This suite modifies the election definition to conform to an Open 

Primary election with a focus on validating presidential delegation 

nominations. 

This test covers Electionware, ExpressVote HW1.0, ExpressVote 

HW2.1, DS200, DS450 and DS850 as well as Electionware 

Reporting. Vote counts will be accumulated from DS450/DS850, 

ExpressVote HW2.1 and DS200. 

o PriClosed

This suite builds on the PriOpen Test, taking the election definition 

and modifying it to conform to a Closed Primary election. 

This test covers Electionware, ExpressVote HW1.0, ExpressVote 

HW2.1, DS200, DS450 and DS850 as well as Electionware 

Reporting. Vote counts will be accumulated from DS450/DS850, 

ExpressVote HW2.1 and DS200. 

o PriOpen

This suite builds on the PriOpen Test, taking the election definition 

and modifying it to conform to a Open Primary election. 

This test covers Electionware, ExpressVote HW1.0, ExpressVote 

HW2.1, DS200, DS450/DS850 as well as Electionware Reporting. 

Vote counts will be accumulated from DS450/DS850, ExpressVote 

HW2.1 and DS200. 

• Error Recovery - The VSTL will focus on Error Messaging and Recovery in 

key areas of the system identified from researching previous testing and 

voting system documentation to help identify potential failure points. Voting 

systems can be subject to various conditions, and when the system exceeds 

limitations, errors are typically found. The VSTL leverages its election 

experience and voting system knowledge to test the Error Messaging and 

Recovery of a voting system that has incurred errors caused by stressing 

the system. Testing of Error messaging will focus on the appropriate error 

messages being generated in response to a specific error and content of the 

message. The testing of the voting system Error Recovery capability is 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 69 of 79 

Template Rev 2015-03 

Managing technology risk 

incorporated into Stress testing in order to leverage the necessary range of 

performance impacts needed to generate system errors and force recovery. 

This test covers Electionware, ExpressVote HW1.0, ExpressVote HW2.1, 

DS200, DS450/DS850 as well as Electionware Reporting. 

• Audit Content Validations - Audit records are used to track what system 

functions have been executed, what data has been modified, as well as by 

who and when. Additionally, audit record data content can be a key factor in 

identifying system anomalies and provide assistance in troubleshooting 

system errors. In tandem with the System Audit Validation, analysis of a 

voting system is performed to determine strategic points of the system that 

require auditing, along with the content needed to accurately depict the 

machinations of the system for the given situation. Tests are incorporated 

either into Election Validation suites or specific Audit Validation suites, as 

needed, such that all related requirements are explicitly validated. 

This test covers Electionware, ExpressVote HW1.0, ExpressVote HW2.1, 

DS200, DS450/DS850 as well as Electionware Reporting. 

• System Audit – Election audit trails provide the supporting documentation 

for verifying the accuracy of reported election results. They present a 

concrete, indestructible archival record of all system activity related to the 

vote tally, and are essential for public confidence in the accuracy of the tally, 

for recounts, and for evidence in the event of criminal or civil litigation. This 

testing focuses on validating system’s ability to provide audit capability 

throughout the entire voting system, including availability, generation, 

integrity, and accuracy of the system’s audit capability to ensure it meets the 

necessary requirements. Negative testing will be utilized to force the system 

into conditions that will trigger errors and verify the voting system captures 

those conditions. 

This test covers Electionware, ExpressVote HW1.0, ExpressVote HW2.1, 

DS200, DS450/DS850 as well as Electionware Reporting. 

• Security Validations - Test suites are created to validate and verify various 

aspects of system security at both the component level and the system 

level, as applicable. Each suite will have a particular focus to it, such that 

when executed successfully, a given requirement, or set of requirements, 

will be determined to have been successfully implemented by the voting 

system. Topics of Security review will include: 

 Access Control 

Access control testing verifies procedures and system capabilities 

that detect or limit access to system components in order to guard 

against loss of system integrity, availability, confidentiality, and 

accountability. This testing verifies that system resources such as 

data files, application programs and computer-related facilities 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 70 of 79 

Template Rev 2015-03 

Managing technology risk 

and equipment are protected against unauthorized operation, 

modification, disclosure, loss or impairment. Unauthorized 

operations include modification of compiled or interpreted code, 

run-time alteration of flow control logic or of data, and abstraction 

of raw or processed voting data in any form other than a standard 

output report by an authorized operator. 

This test covers Electionware, ExpressVote HW1.0, 

ExpressVote HW2.1, DS200, DS450/DS850 as well as 

Electionware Reporting. 

 Software Security 

Software security testing will be conducted to verify the installation 

procedures and ongoing foreign software detection mitigation 

abilities of the voting system in order to protect against the 

modification of the software and/or the insertion of malicious 

software during the installation and during ongoing operations. 

This test covers Electionware, ExpressVote HW1.0, 

ExpressVote HW2.1, DS200, DS450/DS850 as well as 

Electionware Reporting. 

 Physical Security Measures. 

Physical security testing verifies monitoring and control of the 

environment of the work place and computing facilities. It also 

verifies monitor and control access to and from such facilities. For 

example: doors, locks, cameras, barricades, fencing, cable locks, 

etc., can be utilized for implementation of physical security. 

Separating the network and work place into functional areas are 

also physical controls. Some portions of physical security are 

functional while other portions are procedural. Functional portions 

will be tested as appropriate while procedural portions will be 

verified to be documented as called out by the VVSG. 

This test covers Electionware, ExpressVote HW1.0, 

ExpressVote HW2.1, DS200, DS450/DS850 as well as 

Electionware Reporting. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 71 of 79 

Template Rev 2015-03 

Managing technology risk 

• Language – Testing is conducted to ensure the voting system is capable of 

presenting the ballot, ballot selections, review screens and instructions in 

the required languages. The system’s ability to handle the prescribed 

foreign languages that have been declared to be supported, English, 

Spanish, Chinese, Korean, Japanese, Hindi, Bengali, Vietnamese, Tagalog, 

Creole, Russian, and French will be validated. 

This test covers Electionware, ExpressVote HW1.0, ExpressVote HW2.1, 

DS200, DS450/DS850 as well as Electionware Reporting. Vote counts will 

be accumulated from DS450/DS850, ExpressVote HW2.1 and DS200. 

• Data Retention/HW Integrity – Testing integrity requirements ensure the 

physical stability and function of the vote recording and counting processes, 

such that the system is not prone to a single point of failure that would 

prevent voting at a polling place. Testing will also verify prevention of failure 

of data input or storage, in terms of data retention, as well as confirming that 

appropriate audit records are maintained without modification. 

This test covers Electionware, ExpressVote HW1.0ExpressVote HW2.1, 

DS200, DS450/DS850 as well as Electionware Reporting. Vote counts will 

be accumulated from DS450/DS850, ExpressVote HW2.1 and DS200. 

• Accuracy – Testing the ability of the system to capture, record, store, 

consolidate and report the specific selections and absence of selections, 

made by the voter for each ballot position without error. Required accuracy 

is defined in terms of an error rate that for testing purposes represents the 

maximum number of errors allowed while processing a specified volume of 

data. 

Accuracy testing is conducted at both the device level and the system level. 

Each device is subjected to scrutiny that will verify that the requirements for 

accuracy are met. Additionally, the system will be reviewed and exercised to 

validate that the accumulation, tallying and reporting mechanisms at the 

system level are able to accurately perform their functions. 

There are three potential stages to the accuracy test: 

o Go/no go, where after approximately 26,000 ballot positions, if any 

errors are found the test fails, 

o If no errors are found the test continues to the 1.55 million ballot 

position mark, where if 1 error is found the test may progress to 

approximately the 3.1 million ballot position mark where if no more 

errors are encountered, the test passes. 

o If more than 1 error is found at the 1.55 million ballot position mark 

the test fails. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 72 of 79 

Template Rev 2015-03 

Managing technology risk 

This test covers ExpressVote HW1.0, ExpressVote HW2.1, DS200, 

DS450 and DS850 as well as Electionware Reporting. Vote counts will be 

accumulated from DS450/DS850, ExpressVote HW2.1 and DS200. 

• Mark Sensitivity - The purpose of Ballot Mark Sensitivity testing is to 

determine that the system under test is able to accurately determine when a 

mark has been made within a ballot marking position. For this test, various 

marks are made within the ballot marking positions, using EVS supported 

colors of ink (black), as well as non-supported, in order to determine how 

the system handles unexpected colors. 

Marks include fully filled boxes, left and right oriented slashes, “X” markings, 

check marks, horizontal single line marks, and circles of various sizes. 

Marks also include vertical lines within the marking position that fill 

approximately ten percent of the designated space. Small dots down to 

approximately five percent of the ballot marking position are also included. 

This test covers, DS200, DS450/DS850. Vote counts will be accumulated 

from DS450/DS850, ExpressVote HW2.1 and DS200. 

• Volume – Testing a system’s response when subjected to large volumes of 

data, “more than the expected”, as called out in the standards. Volume 

testing is typically considered a type of non-functional testing. However, as a 

voting system’s primary function is to accumulate, tally, and pass a volume 

of data (votes) the VSTL approaches volume testing as a functional test. 

Experience has shown that large amounts of data can slow a system, or 

even cause failures and loss of data due to architectural limitations. Utilizing 

the VSTL’s experience with voting systems the testing will focus on not only 

passing large amounts of data but how the system operates and handles 

the data in key areas of functionality within the voting system. 

This test covers ExpressVote HW1.0, ExpressVote HW2.1, DS200, 

DS450 and DS850 as well as Electionware Reporting. Vote counts will be 

accumulated from DS450/DS850, ExpressVote HW2.1 and DS200. 

• Stress - Testing a system’s “response to transient overload conditions.” 

Experience has shown that when passing a dataset through a system that 

eclipses the system architectural limitations, failures can occur and result in 

the loss of critical data. Utilizing the VSTL’s experience with voting systems, 

the testing will focus on the system’s ability to operate after the limitations 

have been exceeded and if failures occur, how the data is maintained or 

recovered in key areas of functionality within the voting system. 

This test covers ExpressVote HW1.0, ExpressVote HW2.1, DS200, 

DS450/DS850 as well as Electionware Reporting. Vote counts will be 

accumulated from DS450/DS850, ExpressVote HW2.1 and DS200. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 73 of 79 

Template Rev 2015-03 

Managing technology risk 

5.7 Standard VSTL Test Methods and Uncertainty of Test 

Data Measurement 

This test campaign utilizes Standard VSTL test methods and election specific type 

test data only. 

5.8 EAC Interpretations 

The test engagement described in this Certification Test Plan utilizes only standard 

VSTL test methods that conform to the EAC Testing and Certification Program 

Manual and the identified voting system standard. Additional EAC interpretations 

that affect the test plan and test methodology are as follows: 

• EAC Decision on RFI 2007-01 

• EAC Decision on RFI 2007-03 

• EAC Decision on RFI 2007-04 

• EAC Decision on RFI 2007-05 

• EAC Decision on RFI 2007-06 

• EAC Decision on RFI 2008-01 

• EAC Decision on RFI 2008-02 

• EAC Decision on RFI 2008-03 

• EAC Decision on RFI 2008-04 

• EAC Decision on RFI 2008-07 

• EAC Decision on RFI 2008-08 

• EAC Decision on RFI 2008-09 

• EAC Decision on RFI 2008-10 

• EAC Decision on RFI 2008-12 

• EAC Decision on RFI 2009-02 

• EAC Decision on RFI 2009-03 

• EAC Decision on RFI 2009-04 

• EAC Decision on RFI 2009-05 

• EAC Decision on RFI 2009-06 

• EAC Decision on RFI 2010-01 

• EAC Decision on RFI 2010-02 

• EAC Decision on RFI 2010-03 

• EAC Decision on RFI 2010-04 

• EAC Decision on RFI 2010-06 

• EAC Decision on RFI 2010-07 

• EAC Decision on RFI 2010-08 

• EAC Decision on RFI 2012-01 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 74 of 79 

Template Rev 2015-03 

Managing technology risk 

• EAC Decision on RFI 2012-03 

• EAC Decision on RFI 2012-04 

• EAC Decision on RFI 2012-06 

• EAC Decision on RFI 2013-01 

• EAC Decision on RFI 2013-02 

• EAC Decision on RFI 2013-03

• EAC Decision on RFI 2013-04

• EAC Decision on RFI 2015-05

RFI’s not implemented in this test campaign include: 

• EAC Decision on RFI 2009-01, VVPAT accessibility 

• EAC Decision on RFI 2010-05, modifications of a certified system 

• EAC Decision on RFI 2012-02, transmission of official/unofficial results 

• EAC Decision on RFI 2012-05, public telecommunications and cryptography 

EAC notices of clarification affecting the certification project are implemented as 

noted below. 

• NOC 2007-005 VSTL responsibilities and Third Party Lab Oversight 

• NOC 2008-003 EAC Conformance Testing Requirements 

• NOC 2009-001 Test Plan NOC 

• NOC 2009-002 Testing Independence 

• NOC 2009-004 Test Report 

• NOC 2012-02 Clarification of System ID Tool Functionality FINAL 4.19.12 

• NOC 2013-01 Discrepancy Listing in Test Report-FINAL-10 17 13 

• NOC 2015-01 Test Readiness Review (TRR) applies to both new certs and 

modifications 

• NOC 2016-01 Test Readiness Review 

• NOC 2016-02 Trusted Build 

• NOC 2016-03 Test Case Upload 

EAC notices of clarification, noted below, not affecting the certification project are 

not implemented: 

• NOC 2007-001 Timely Submission of Application for Certification 

• NOC 2007-002 VSTL work outside of voting system testing 

• NOC 2007-003 State and Federal Testing 

• NOC 2007-004 Manufacturing facilities 

• NOC 2008-001 Use of Previous Non-core HW Testing 

• NOC 2008-002 EAC Mark of Certification 

• NOC 2009-003 de minimis change TDP 

• NOC 2009-005 Modification Test Plan 

• NOC 2011-01 de minimis data change final 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 75 of 79 

Template Rev 2015-03 

Managing technology risk 

• NOC 2012-01 COTS Computer Equivalency for de minimis Change 

• NOC 2013-02 Detailed Description of Changes for Modifications 

• NOC 2014-01 Technology Testing Agreement 

5.9 Security Functions 

5.9.1 Security Test 

The Security Test Suites are SLI’s tests for verifying that a voting system will 

correspond to requirements in VVSG Volume 1, Section 7. It incorporates systems 

security provisions, unauthorized access, deletion or modification of data, audit trail 

data, and modification or elimination of security mechanisms. The vendor 

documentation will be reviewed to ensure sufficient detail is present to operate the 

voting system in a secured manner. Where the vendor statements assert the 

voting system is secured via mechanisms and seals, procedures will test the 

presence and effectiveness of such controls. 

In its security testing SLI identifies the specific threats that are tested for and the 

associated risk if a flaw or exception is identified in a voting system. SLI maintains 

a “Threat Matrix Database” that is updated minimally once a year as well as 

whenever an issue is uncovered. The tests used by SLI are designed to ensure 

that the voting system meets or exceeds the requirements in the VVSG. Any 

instance where an anomaly or possible security flaw is identified, the potential risk 

is reported and evaluated. 

Security testing includes testing each individual component of the system and the 

system as a whole. As such, Electionware, ExpressVote HW1.0, ExpressVote 

HW2.1, DS200, DS450/DS850 as well as Electionware Reporting will be 

subjected to review, as will the system as a whole and its interactions between 

components. 

 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 76 of 79 

Template Rev 2015-03 

Managing technology risk 

6 TEST DATA 

Test data for the ES&S EVS 6.2.0.0 voting system is compiled such that all 

functionality declared will be tested to determine conformance to the standards.

6.1 Data Recording 

SLI has evaluated the system functionality, as described by manufacturer technical 

documentation, as well as requirements as listed in the EAC 2005 VVSG, and 

made determinations as to expected results of all data inputs into the ES&S EVS 

6.2.0.0 voting system. This includes: 

• Election type 

• Precincts of all types 

• Districts 

• Offices 

• Contests 

• Candidates 

• Parties 

• Devices used 

• Voting variations employed 

• Issues/Referendums 

• Votes cast for each candidate/issue/referendum 

• Vote consolidation data from one device/level to the next 

The data is contained in one master data record, including each input and each 

expected output. This data is incorporated into the appropriate test suite, 

populating test modules with exact expected data for the function being tested. 

Testing information is recorded in the test suites, as well as in test notebooks, 

which are utilized according to SLI’s standard lab procedure SLP-VC-30 - Test 

Notebooks. 

6.2 Test Data Criteria 

SLI has evaluated the system functionality as described by manufacturer technical 

documentation, as well as requirements as listed in the EAC 2005 VVSG, and 

made determinations as to expected output of all data inputs into the ES&S EVS 

6.2.0.0 voting system. A data matrix will be recorded into one master data record 

that couples data inputs to their expected output, as determined above. The 

system’s execution shall be measured against the expected results. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 77 of 79 

Template Rev 2015-03 

Managing technology risk 

6.3 Test Data Reduction 

SLI processes the test data by manually recording input data into each pertinent 

module within the Test Suites as well as the exact output that is generated, e.g., 

the vote counts when the data is consolidated. 

7 TEST PROCEDURE AND CONDITIONS 

This section describes the test conditions and procedures for execution of test 

suites. If a particular sequence is mandatory for the execution of suites, a rationale 

will be given. Additionally, this section is used to describe procedures for setting up 

equipment that will be utilized in the execution of the test suites. 

7.1 Facility Requirements 

Testing will be performed on site at SLI in Colorado. 

Four secure labs are available with appropriate power supply and space to 

accommodate the various configurations defined within this test plan. 

Temperature/humidity gauges will be employed in order to determine the 

appropriate conditions exist during testing. 

Unless otherwise specified herein, all remaining tests, including system level 

functional testing, shall be performed at standard ambient conditions: 

• Temperature: 25°C ± 10°C (77°F ± 18°F) 

• Relative Humidity: 20 to 90% 

• Atmospheric Pressure: Local Site Pressure 

All TDP and test documentation is stored on site at SLI’s facility in a secure project 

directory on SLI’s secure Voting server. 

Environmental hardware testing for hardware components of the ES&S EVS 

6.2.0.0 voting system will be performed at either NVLAP or A2LA accredited testing 

laboratories or at laboratories audited by SLI to NVLAP Handbook 150-22 

requirements. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 78 of 79 

Template Rev 2015-03 

Managing technology risk 

7.2 Test Setup 

Configurations of ES&S EVS 6.2.0.0 will be deployed that conform to each specific 

test suite’s needs. Some configurations will consist of standalone implementations, 

while other configurations will utilize networked implementations of various 

applications, such as Electionware, DS450 and DS850. In all instances, ES&S

EVS 6.2.0.0 documentation will be followed in the setup of the configurations. 

Successful completion of operational status checks will indicate that the system is 

ready for test execution. 

7.3 Test Sequence 

While there is no required sequence for performing voting system certification 

testing and audits, there are prerequisite tasks for some testing. Any needed 

prerequisites are contained within the suite for that test. 

8 TEST OPERATIONS PROCEDURES 

An inventory has been performed to verify the voting equipment received contains 

hardware and software elements as defined in the TDP prior to commencement of 

testing. 

Throughout the testing effort, test suites and modules will be marked as follows: 

• Accept – Test is accepted as successful.

• Reject – Test is rejected as unsuccessful. 

• NT – Not Testable is used for test modules that cannot be completed. For 

example, if failure of one test module precludes attempting subsequent test 

modules, the latter will be marked as NT. 

Test results Reject and NT will include comments by the VTS explaining the 

reason for the result. 

Issues encountered during review and testing will be documented in the 

Discrepancy Report as well as the EAC VRT system. Issues that do not conform to 

the requirements of the VVSG, version 2005 will be marked as Documentation 

Discrepancies or Functional Discrepancies (a discrepancy occurs when the 

software does not meet defined software requirements or specifications). SLI 

employs a system of checks such that any issue uncovered during testing is first 

designated as an “anomaly”. The anomaly is then reviewed and the cause is 

determined as to be either caused by a flaw in the test or in the voting system 

itself. If the issue is determined to be a flaw in the test, the test will be re-written, 

re-validated and then formally re-run. If the issue is determined to be a flaw in the 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.0 

Document No. ESS-7005-CTP-01

Page 79 of 79 

Template Rev 2015-03 

Managing technology risk 

voting system, then a discrepancy is opened against the system. While test suites 

and test modules undergo a validation phase prior to formal execution, last minute 

code changes can possibly change the behavior from what the test module defines 

as expected, which if this is the case, the review process employed during the 

anomaly phase will reveal, thus reducing the chance of a false positive in terms of 

an unfounded discrepancy being written against the voting system. 

Issues that are encountered during testing or documentation review, but are not 

addressed by the applicable standard will be added to the Discrepancy Report and 

noted as Informational. The vendor has the option whether to address 

Informational issues. 

9 APPROVAL SIGNATURES 

SLI: 

End of Certification Test Plan 

Traci Mapps 

VSTL Director 

July 24th, 2017