EVS 6200 Certification Test Plan v1.1 

Report No. ESS-7005-CTP-01 

Page 1 of 84 

Template Rev 2015-03 

Certification Test Plan 

Document Number: ESS-7005-CTP-01 

Prepared for: 

Vendor Name Election Systems and Software 

(ES&S) 

Vendor System EVS 6.2.0.0 

EAC Application No. EVS6200 

Vendor Address 11208 John Galt Boulevard 

Omaha, Nebraska 68137

Prepared by: 

4720 Independence St. 

Wheat Ridge, CO 80033 

303-422-1566 

www.SLICompliance.com

Accredited by the Election 

Assistance Commission (EAC) for 

Selected Voting System Test 

Methods or Services 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 2 of 84 

Template Rev 2015-03 

Managing technology risk 

Copyright  2017 by SLI ComplianceSM, a Division of Gaming Laboratories International, LLC 

Revision History 

Date Version Author Revision Summary 

July 24th, 2017 1.0 M. Santos Initial Draft 

1.1 M. Santos Updates for EAC comments 

Disclaimer 

The information reported herein must not be used by the client to claim product 

certification, approval, or endorsement by NVLAP, NIST, or any agency of the Federal 

Government. 

Trademarks 

• SLI is a registered trademark of SLI Compliance. 

• All products and company names are used for identification purposes only and may be 

trademarks of their respective owners. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 3 of 84 

Template Rev 2015-03 

Managing technology risk 

TABLE OF CONTENTS 

CERTIFICATION TEST PLAN ...................................................................................................................... 1

1 INTRODUCTION .................................................................................................................................... 5

1.1 CERTIFICATION TEST PLAN ATTACHMENTS .......................................................................................... 5

1.2 REFERENCES ..................................................................................................................................... 5

1.3 TERMS AND ABBREVIATIONS................................................................................................................ 6

1.4 PROJECT OVERVIEW ........................................................................................................................... 8

1.5 PURPOSE ........................................................................................................................................... 8

1.6 TESTING RESPONSIBILITIES ................................................................................................................. 9

1.6.1 Project Schedule ...................................................................................................................... 9

1.7 SCOPE OF TESTING .......................................................................................................................... 11

1.8 SCOPE OF THE ES&S EVS 6.2.0.0 VOTING SYSTEM .......................................................................... 13

1.8.1 System Overview ................................................................................................................... 13

1.8.2 Block Diagram ....................................................................................................................... 15

1.8.3 System Limits ........................................................................................................................ 21

1.8.4 Supported Languages ........................................................................................................... 23

1.8.5 Supported Functionality ......................................................................................................... 23

1.8.6 Excluded Requirements......................................................................................................... 37

1.8.7 Additional Functionality/Requirements .................................................................................. 38

1.9 CHANGE CONTROL AND CONFIGURATION MANAGEMENT..................................................................... 42

2 PRE-CERTIFICATION TESTING AND ISSUES ................................................................................. 44

2.1 EVALUATION OF PRIOR VSTL TESTING ............................................................................................... 44

2.2 EVALUATION OF PRIOR NON-VSTL TESTING ....................................................................................... 44

2.3 KNOWN VULNERABILITIES.................................................................................................................. 44

3 PHYISCAL CONFIGURATION AUDIT ............................................................................................... 44

3.1 TDP DOCUMENTATION ANALYSIS & REVIEW ...................................................................................... 44

3.1.1 Document Review .................................................................................................................. 44

3.2 TDP SOURCE CODE ANALYSIS & REVIEW .......................................................................................... 45

3.2.1 Source Code Review ............................................................................................................. 45

3.3 QA & CM PROCESS REVIEW ............................................................................................................. 46

3.4 COMPLIANCE/TRUSTED BUILD ........................................................................................................... 47

4 MATERIALS REQUIRED FOR TESTING ........................................................................................... 48

4.1 SOFTWARE/FIRMWARE ...................................................................................................................... 48

4.1.1 Manufacturer Software/Firmware .......................................................................................... 48

4.1.2 COTS Software/Firmware...................................................................................................... 49

4.1.3 Additional Supporting Test Software ..................................................................................... 50

4.2 EQUIPMENT ...................................................................................................................................... 50

4.2.1 ES&S EVS 6.2.0.0 Equipment ............................................................................................... 50

4.2.2 COTS Equipment ................................................................................................................... 51

4.3 TEST MATERIALS .............................................................................................................................. 52

4.4 DELIVERABLE MATERIALS.................................................................................................................. 53

5 TEST SPECIFICATIONS ..................................................................................................................... 55

5.1 REQUIREMENTS ................................................................................................................................ 55

5.2 HARDWARE CONFIGURATION AND DESIGN ......................................................................................... 56

5.3 ES&S EVS 6.2.0.0 VOTING SYSTEM REVIEW .................................................................................... 57

5.3.1 Define..................................................................................................................................... 57

5.3.2 Design .................................................................................................................................... 58
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 4 of 84 

Template Rev 2015-03 

Managing technology risk 

5.3.3 Deliver .................................................................................................................................... 59

5.3.4 ExpressVote Universal Voting System, HW1.0 ..................................................................... 60

5.3.5 ExpressVote Universal Voting System, HW2.1 ..................................................................... 61

5.3.6 DS200 .................................................................................................................................... 61

5.3.7 DS450 .................................................................................................................................... 62

5.3.8 DS850 .................................................................................................................................... 62

5.3.9 Electionware Reporting Module ............................................................................................. 62

5.4 TESTING STRATEGIES ....................................................................................................................... 63

5.4.1 How each Polling Place Device will be tested ....................................................................... 63

5.4.2 How each Application will be tested ...................................................................................... 64

5.4.3 How different System Level Configurations will be tested ..................................................... 64

5.5 SOFTWARE SYSTEM FUNCTIONS ....................................................................................................... 65

5.5.1 Election Definition Creation ................................................................................................... 65

5.5.2 Pre-voting Aspects ................................................................................................................. 65

5.5.3 Voting Aspects – Polling Place .............................................................................................. 65

5.5.4 Voting Aspects – Central Count ............................................................................................ 65

5.5.5 Post Voting Aspects ............................................................................................................... 66

5.5.6 Error Messaging and Recovery ............................................................................................. 66

5.5.7 Auditing .................................................................................................................................. 66

5.5.8 Security .................................................................................................................................. 66

5.6 TEST SUITE DESIGN .......................................................................................................................... 66

5.6.1 Hardware Qualitative Examination Design ............................................................................ 66

5.6.2 Hardware Environmental Test Design ................................................................................... 67

5.6.3 Software Module Test Design and Data ................................................................................ 67

5.6.4 Software Functional Test Design and Data ........................................................................... 67

5.6.5 System-level Test Design ...................................................................................................... 68

5.6.6 Software Functional Test Case Design ................................................................................. 69

5.6.7 System-level Test Suite Design ............................................................................................. 71

5.7 STANDARD VSTL TEST METHODS AND UNCERTAINTY OF TEST DATA MEASUREMENT ......................... 77

5.8 EAC INTERPRETATIONS .................................................................................................................... 77

5.9 SECURITY FUNCTIONS ...................................................................................................................... 79

5.9.1 Security Test .......................................................................................................................... 79

6 TEST DATA ......................................................................................................................................... 80

6.1 DATA RECORDING ............................................................................................................................ 80

6.2 TEST DATA CRITERIA ........................................................................................................................ 80

6.3 TEST DATA REDUCTION .................................................................................................................... 81

7 TEST PROCEDURE AND CONDITIONS ........................................................................................... 81

7.1 FACILITY REQUIREMENTS .................................................................................................................. 81

7.2 TEST SETUP ..................................................................................................................................... 82

7.3 TEST SEQUENCE .............................................................................................................................. 82

8 TEST OPERATIONS PROCEDURES ................................................................................................. 82

9 APPROVAL SIGNATURES ................................................................................................................. 83

10 APPENDIX A – ANCILLARY PRODUCTS ......................................................................................... 84
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 5 of 84 

Template Rev 2015-03 

Managing technology risk 

1 INTRODUCTION 

This Certification Test Plan outlines the test approach SLI Compliance (SLI) will 

follow when performing Certification Testing on the ES&S EVS 6.2.0.0 voting 

system against the 2005 Voluntary Voting System Guidelines (VVSG). The 

purpose of this document is to provide a clear understanding of the work SLI will 

conduct and a detailed plan outlining the test effort. 

When the testing is complete, SLI will submit a Certification Test Report that details 

all test results and findings from the Certification Test effort, as well as a 

recommendation to the EAC. 

1.1 Certification Test Plan Attachments 

The following attachments apply to this Certification Test Plan: 

1. Attachment A - Elec_NTS_Longmont_A2LA_0214-43_022818 

2. Attachment B - ENV_NTS_Longmont_A2LA_0214-44_022818 

3. Attachment C - ESS EV6000 Electrical Hardware Test Plan v1.0 

4. Attachment D - ESS EV6000 Environmental Hardware Test Plan v1.0 

1.2 References 

The following key documents were used in preparing this test plan. 

1. Election Assistance Commission Voluntary Voting System Guidelines (EAC 

VVSG), 2005 Version 1.0 Volumes I and II. 

2. NIST Handbook 150: 2016. 

3. NIST Handbook 150-22: 2008. 

4. EAC Voting System Testing and Certification Program Manual, United States 

Election Assistance Commission, v 2.0, May 2015 

5. SLI VSTL Quality System Manual, 2.3, prepared by SLI, dated January 20, 

2017. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 6 of 84 

Template Rev 2015-03 

Managing technology risk 

1.3 Terms and Abbreviations 

The following terms and abbreviations will be used throughout this document: 

Table 1 – Terms and Abbreviations 

Term Abbreviation Description 

American Association 

for Laboratory 

Accreditation 

A2LA A nonprofit, non-governmental, public service,

membership society whose mission is to provide 

comprehensive services in laboratory accreditation 

and laboratory-related training. 

Ballot Marking 

Device 

BMD An accessible computer-based voting system that

produces a marked ballot (usually paper) that is the 

result of voter interaction with visual or audio 

prompts. 

Cast Vote Record CVR Permanent record of all votes produced by a single 

voter whether in electronic, paper or other form. 

Also referred to as ballot image when used to refer

to electronic ballots. 

Central Count 

Scanner 

CCS High Speed Optical Scanner is a mark sense-based 

ballot and vote counting device typically located at a 

central count facility and is operated by an 

automated multi-sheet feeding capability. 

Compact Flash card CF This is a type of flash memory card in a 

standardized enclosure often used in voting 

systems to store ballot and/or vote results data. 

Commercial Off the 

Shelf 

COTS Term used to designate computer software, 

hardware or accessories that are ready-made and 

available for sale, lease, or license to the general 

public 

Direct Recording 

Electronic 

DRE Voting systems that, using Touch Screen or other 

user interfaces, directly record the voter’s selections 

in each race or contest on the ballot in electronic

form. 

Election Assistance 

Commission 

EAC An independent, bipartisan commission created by 

the Help America Vote Act (HAVA) of 2002 that 

operates the federal government's voting system 

certification program. 

Election 

Management System 

EMS Typically a database management system used to 

enter jurisdiction information (district, precincts, 

languages, etc.) as well as election specific 

information (races, candidates, voter groups 

(parties), etc.). In addition, the EMS is also used to 

layout the ballots, download the election data to the 

voting devices, upload the results and produce the 

final results reports. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 7 of 84 

Template Rev 2015-03 

Managing technology risk 

Term Abbreviation Description 

Electromagnetic 

Compatibility 

EMC The goal of EMC is to validate the correct 

functioning of different equipment in the same 

environment and the avoidance of any interference 

effects between them. 

Functional 

Configuration Audit 

FCA The testing activities associated with the functional 

testing of the system. 

National Institute of 

Standards and 

Technology 

NIST A non-regulatory federal agency within the U.S. 

Dept. of Commerce. Its mission is to promote U.S. 

innovation and industrial competitiveness by 

advancing measurement science, standards, and 

technology in ways that enhance economic security 

and improve our quality of life. 

National Voluntary 

Laboratory 

Accreditation 

Program 

NVLAP A division of NIST that provides third-party 

accreditation to testing and calibration laboratories. 

Physical 

Configuration Audit 

PCA Confirms that the documentation submitted 

meets the national certification requirements. 

Includes Trusted Build activities.

Precinct Count 

Scanner 

PCS A precinct-count optical scanner is a mark sensebased ballot and vote counting device located at a 

precinct and is typically operated by scanning one 

ballot at a time. 

Ranked Choice Vote RCV Practice that allows voters to rank candidates in a

contest in order of choice: 1, 2, 3 and so on. A 

candidate receiving a majority of the first choice 

votes wins that election. If no candidate receives a 

majority, the last place candidate is deleted, and all 

ballots are counted again, with each ballot cast for 

the deleted candidate applied to the next choice 

candidate listed on the ballot. 

Request For 

Information 

RFI A means used by testing laboratories and 

manufacturers to request that the EAC provide an 

interpretation of a technical issue related to testing 

of voting systems. 

Technical Data 

Package 

TDP The data package supplied by the vendor, which 

includes Functional Requirements, Specifications, 

End-user documentation, Procedures, System 

Overview, Configuration Management Plan, Quality 

Assurance Program, and manuals for each of the 

required hardware, software, firmware components 

of a voting system. 

Universal Voting 

System 

UVS A device designed for all voters 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 8 of 84 

Template Rev 2015-03 

Managing technology risk 

Term Abbreviation Description 

Voluntary Voting 

System Guidelines 

VVSG A set of specifications and requirements against 

which voting systems can be tested to determine if 

the systems provide all of the basic functionality,

accessibility and security capabilities required for 

EAC certification. 

Voting System Test 

Lab 

VSTL An independent testing organization accredited by 

NVLAP and the EAC to conduct voting system 

testing for EAC certification. 

Voting Test Specialist VTS An SLI employee within the Compliance division 

who has been qualified to perform EAC voting 

system certification testing. 

1.4 Project Overview 

This test plan outlines the approach SLI will implement to perform testing of the 

ES&S EVS 6.2.0.0 voting system against the requirements of the federal 2005 

Voluntary Voting System Guidelines (VVSG), Volumes 1 and 2. This project is for a 

full new-system certification effort. 

1.5 Purpose 

The purpose of this EAC Test Plan is to create clear and precise documentation of 

the processes that SLI, as an EAC VSTL, will implement throughout the course of 

voting system verification testing. 

This Test Plan: 

• Defines the overall test approach. 

• Identifies required voting system hardware and software to be tested, and in 

what configurations. 

• Identifies required hardware, software, and tools to be used to support the 

testing efforts. 

• Defines the types of tests to be performed and against what declared 

functionality. 

• Defines the types of election and vote data required for effective testing. 

• Defines the types of voting system threats and vulnerabilities against which 

the voting system will be tested. 

• Serves as a foundation for the development of ES&S EVS 6.2.0.0 test 

suites 

• Defines the process for recording and reporting test results. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 9 of 84 

Template Rev 2015-03 

Managing technology risk 

1.6 Testing Responsibilities 

The following schedule describes the high-level tasks and assigned personnel titles 

that will be involved in the Certification Test effort of the ES&S EVS 6.2.0.0 voting 

system. 

1.6.1 Project Schedule 

The project schedule consists of the components listed in the following 

subsections: 

1.6.1.1 Owner Assignments 

• System Analysis and Review will be conducted by Source Code Review, 

Security and Voting Test Specialists, with oversight by the Test Manager 

• Source code review will be conducted by Source Code Review 

Specialists, with oversight by the Test Manager 

• Documentation review will be conducted by Security and Voting Test 

Specialists (VTS), with oversight by the Test Manager 

• Test Module Development and Validation will be conducted by Security 

and VTS personnel, with oversight provided by the Test Manager 

• Test Suite Development and Validation will be conducted by Security 

and Voting Test Specialists, utilizing SLI’s formal Test Methods, with 

oversight provided by the Test Manager 

• Formal Test Execution will be conducted by Security and Voting Test 

Specialists, with oversight by the Test Manager 

• 3rd Party testing will be conducted by the subcontracting 3rd Party 

Hardware Laboratories, with oversight by the Hardware Specialist 

1.6.1.2 Test Module Development and Validation 

Test Modules will be developed to provide repeatable detailed test steps. The 

Modules are designed for use in any suite that employs their functionality. The 

modules will be validated prior to Formal Test Execution to ensure accurate testing 

of the voting system. Test modules are validated by internal review and execution. 

The test modules and the test suites that are written by SLI contribute directly to 

the “Exhaustive verification of every system function and combination of functions 

cited in the vendor’s documentation”, as defined by the VVSG’s definition of the 

Functional Configuration Audit (FCA). 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 10 of 84 

Template Rev 2015-03 

Managing technology risk 

1.6.1.3 Test Suite Development 

Test Suites will be developed to group and focus testing around key areas of the 

voting system. Each Test Suite will contain multiple test modules providing clear 

and traceable test scripts and information. Various system configurations will be 

identified within the suites. Under the 2005 VVSG, this portion of the certification is 

considered to be part of the FCA. 

 Formal Test Execution 

Formal Test Execution of the validated Test Suites will be conducted against the 

declared voting system, utilizing Compliance/Trusted Build software and firmware 

components, in order to test to determine whether or not the system is compliant 

with the VVSG requirements. Under the 2005 VVSG, this portion of the certification 

is considered to be part of the FCA. 

1.6.1.4 3rd Party Hardware Testing 

Hardware testing will be conducted by 3rd Party certified hardware test laboratories 

to verify the voting system devices are in compliance with the VVSG hardware 

requirements. 

Other Labs Performing Non-Core Hardware Testing 

SLI Compliance is responsible for all core voting system tests as identified in NIST 

NVLAP Handbook 150-22 (2008). The labs listed below will perform non-core 

hardware testing for this certification test campaign. 

Table 2 – Labs Performing Hardware Testing 

Laboratory Address Test(s) Date(s) 

NTS – EMI / 

EMC 

1736 Vista View Drive 

Longmont, CO 80504 

EMC / EMI Tests: 

Radiated Emissions, 

Conducted Emissions, 

ESD, Electromagnetic 

Susceptibility, Electrical 

Fast Transient, Lightning 

Surge, Conducted RF 

Immunity, Magnetic 

Fields Immunity, 

Electrical Power 

Disturbance 

7/7/2017 – 

8/7/2017 

NTS – 

Environmental/

Dynamic 

1601 Dry Creek Drive 

Suite 200 

Longmont, CO 80503 

MIL-STD-810D Tests: 

Bench Handling, 

Vibration, Low 

Temperature, High 

Temperature, Humidity, 

Temperature/Power 

Variation, and Reliability 

7/7/2017 – 

8/24/2017 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 11 of 84 

Template Rev 2015-03 

Managing technology risk 

1.6.1.5 EAC & Manufacturer Dependencies 

The Test Plan will require EAC approval prior to finalization. 

ES&S is required to provide all source code, documentation, equipment and 

supporting materials identified as part of the voting system. 

The source code must have all discrepancies resolved, be able to be built 

successfully, be installed, as well as successfully complete operational status 

checks prior to Formal Test Execution. 

In addition, ES&S is required to provide training on the voting system and support 

throughout the life of the project. 

1.7 Scope of Testing 

SLI will provide testing on the ES&S EVS 6.2.0.0 voting system based on the 

guidelines established for voting system verification testing as defined by the 

EAC’s 2005 VVSG. This effort will include all required levels of software, firmware, 

system and hardware testing required to demonstrate that the ES&S EVS 6.2.0.0 

voting system is tested against the requirements of the 2005 VVSG. 

SLI’s major task categories for voting system verification testing, as defined by 

SLI’s NVLAP audited and approved Quality System Manual, as well as the 2005 

VVSG, include: 

• Physical Configuration Audit (PCA) 

o Iterative documentation review and assessment 

o Iterative source code review 

o Trusted Builds/Compliance Builds. Identify the Trusted Build process 

to establish the system version and components being tested and 

ensure that the qualified executable release is built from the reviewed 

components. 

o Software and Hardware Configuration Audit. Verification of software 

and hardware physical and functional configurations. 

Note that Documentation and Source Code Review are denoted as 

“Iterative”. When either Documentation or Source Code is reviewed, 

and items that do not meet the VVSG requirements are found, 

discrepancy reports are generated and sent to the manufacturer. 

Next, the manufacturer is allowed to resolve the issue and re-submit 

the item in question. SLI then re-reviews to verify that the 

requirement is met. (SLI also validates that nothing else has been 

modified; if something else was modified, it is also re-reviewed.) 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 12 of 84 

Template Rev 2015-03 

Managing technology risk 

• Functional Configuration Audit (FCA) 

o Review of other relevant state verification testing or risk analysis 

results. 

o Review of prior hardware testing results. 

o Testing of voting system components and the fully integrated 

systems to validate functionality, logic processing, system 

performance, security, and system level integration. This testing 

includes the formal test execution, utilizing customized iterations of 

SLI’s formal Test Methods for the ES&S EVS 6.2.0.0 voting system. 

Regression testing will be done when issues are found in testing and 

source code is modified, re-reviewed and re-built. 

o Accuracy testing and validation of a voting system's scanners 

ability to accurately read ballots and tally a large number of 

ballot marking positions within the requirements allowed by the 

2005 VVSG. 

o Security testing and validation that a voting system is tested against 

all security related requirements as well as assessing the 

effectiveness of a voting system’s security controls. 

o Verification of every system function cited in the vendor’s 

documentation. 

• Management of ES&S EVS 6.2.0.0 supplied deliverables, SLI’s test 

artifacts, and software, firmware, hardware and system test configurations. 

• Generation of detailed and repeatable test modules based on SLI’s formal 

Test Methods, to ensure the voting system is tested against all applicable 

requirements of the 2005 VVSG and associated ES&S EVS 6.2.0.0 specific 

requirements. 

• Generation of test data required for all test modules. 

• Traceability of test modules to the requirements of the 2005 VVSG. 

• Reporting of all test results. 

SLI will develop and submit to the EAC a final test report for the ES&S EVS 6.2.0.0

voting system. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 13 of 84 

Template Rev 2015-03 

Managing technology risk 

1.8 Scope of the ES&S EVS 6.2.0.0 Voting System 

1.8.1 System Overview 

This section provides a description of the scope of ES&S EVS 6.2.0.0 voting 

system and components: 

The ES&S EVS 6.2.0.0 voting system is composed of software applications, 

central count location devices and polling place devices with accompanying 

firmware, and COTS hardware and software. 

In addition to the voting system components, this documentation references 

ancillary products that are not required to be part of the certified system; however, 

these components were used to facilitate testing and demonstrate system 

interoperability. 

 

• Electionware is an end-to-end election management software application 

that provides election definition creation, ballot formation, equipment 

configuration, result consolidation, adjudication and report creation.

o Prior to the election, the Electionware interface handles all tasks that 

define the image of a paper ballot of an election including: contests, 

parties and options. Then, Electionware converts all database 

information into ballot definition parameters for ExpressVote HW1.0

and ExpressVote HW2.1 Universal Voting Systems, as well as, 

DS200, DS450, and DS850 tabulators through the Removable 

Media Service by programming media for each device. 

o Post-election, Electionware manages results consolidation, Election 

Night reporting and ballot/write-in adjudication based on collected 

ballot images.. 

• ExpressVote Hardware 1.0 is a polling place universal voting device that 

provides digital vote capture through a touch screen or accessibility 

interface using a detachable ADA keypad or ADA support peripheral such 

as sip-and-puff device or two-position switch. When the voter selections are 

complete a vote summary screen requires voters to confirm or revise their 

selections prior to printing. Once confirmed, vote selections are printed in 

both human and machine readable formats on the vote summary card. Vote 

selections are not stored in memory after the vote summary card is ejected. 

A separate ES&S tabulator may process the printed vote summary card. 

• ExpressVote Hardware 2.1 is a polling place device which in addition to 

retaining all feature functionality of the ExpressVote HW1.0, the 

ExpressVote HW2.1 may be configured as a tabulator and provides the 

added, optional capability of tabulating printed vote summary cards. When 

configured as a tabulator, the ExpressVote uses a master media device for 

Poll Open and Poll Close functions. Once confirmed by the voter, vote 

selections are printed in both human and machine readable formats on the 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 14 of 84 

Template Rev 2015-03 

Managing technology risk 

vote summary card. If the ExpressVote HW2.1 is programmed as a 

tabulator and in the rolling kiosk configuration, the printed vote summary 

card is internally scanned and tabulated to a USB memory device. 

Otherwise, a separate ES&S tabulator may process the vote summary card. 

• DS200 is a polling place scanner and tabulator that simultaneously scans 

the front and back of a paper ballot and/or vote summary card in any of four 

orientations. After the voter marks a paper ballot either manually or digitally, 

their ballot is inserted into the unit and tabulated to a USB memory device 

and internal vote counter. Once the ballot is tabulated, the ballot is dropped 

into an integrated ballot box. An integrated printer prints zero reports prior to 

opening of pools, as well as log reports and polling place totals upon the 

official closing of the polls. 

• DS450 is a central scanner and tabulator that simultaneously scans the front 

and back of a paper ballot and/or vote summary card in any of four 

orientations. The DS450 sorts tabulated ballots into discrete output bins 

without interrupting scanning, based on pre-defined sort criteria, such as the 

ballot being unreadable, having write-ins, overvotes, undevotes, blank 

ballots or invalid id. Optionally, this device may be configured to transmit 

tabulation results to the results server through a closed network connection 

rather than using physically transported media. 

• DS850 is a central scanner and tabulator that simultaneously scans the front 

and back of a paper ballot and/or vote summary card in any of four 

orientations. The DS850 sorts tabulated ballots into discrete output bins 

without interrupting scanning, based on pre-defined sort criteria. Optionally, 

this device may be configured to transmit tabulation results to the results 

server through a closed network connection rather than using physically 

transported media. 

• Event Log Service (ELS) monitors user’s interactions with the Election 

Management System, and Events that happen when a connection to the 

database is not available are logged to the Windows Operating System log 

through the Event Log Service 

• Removable Media Service (RMS) is a utility that runs in the background of 

the Windows operating system. RMS reads specific information from any 

attached USB devices so that ES&S applications such as ElectionWare can 

use that information for media validation purposes. The service also 

provides functionality similar to the Safely Remove Hardware feature on the 

Windows system tray, allowing applications to eject media devices without 

user input. 

 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 15 of 84 

Template Rev 2015-03 

Managing technology risk 

1.8.2 Block Diagram 

Election Management Diagram 

Election Data Management Diagram Description 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 16 of 84 

Template Rev 2015-03 

Managing technology risk 

Ballot Formatting and Printing Diagram 

Ballot Formatting and Printing Diagram Description 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 17 of 84 

Template Rev 2015-03 

Managing technology risk 

Voting Equipment Configuration Diagram 

Voting Equipment Configuration Diagram Description 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 18 of 84 

Template Rev 2015-03 

Managing technology risk 

Voting and Tabulation Diagram 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 19 of 84 

Template Rev 2015-03 

Managing technology risk 

Voting and Tabulation Diagram Description 

Note: ExpressPass is an ancillary system product, which is not part of the EAC certified 

configuration, that has a voter’s selection summary with scannable QR code that the voter can 

create with their vote selections, and either print or save in an electronic format on their mobile 

device. See “Appendix A – Ancillary Products” for additional information.
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 20 of 84 

Template Rev 2015-03 

Managing technology risk 

Results Consolidation, Adjudication and Reporting Diagram 

Results Consolidation, Adjudication and Reporting Diagram Description 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 21 of 84 

Template Rev 2015-03 

Managing technology risk 

 

1.8.3 System Limits 

This section details various limits within the ES&S EVS 6.2.0.0 voting system. All 

Limits will be reviewed and verified for accuracy. 

• Max precinct allowed in an election: 9,900 

• Max candidate allowed per election: 30,000 

• Max contests allowed in an election: 30,000 

• Max contests allowed per ballot style: 500 or number of positions on ballot 

• Max candidate (ballot choices) allowed per contest: 230 

• Max number of parties allowed in General Election: 75 

• Max number of parties allowed in Primary Election: 30 

• Max ‘vote for’ per contest: 500 

• Max ballot styles: 15,000 

• Max district types/groups: 25 

• Max district of a given type: 250 

• Max reporting groups: 14 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 22 of 84 

Template Rev 2015-03 

Managing technology risk 

• Electionware 

o Electionware Export Ballot Images function is limited to: 250 districts 

per export 

o Special Characters that have limitations will be reviewed 

• ExpressVote, Hardware 2.1 

o Secure Card Container Capacity: 300 vote summary cards 

• DS200 

o Early voting station does not support precinct level results reporting 

o Storage limitation for write-in ballot images is 3,600 images 

o Write-in image review requires min 1 GB of onboard RAM. 

o Plastic Ballot Box Capacity 

 Standard compartment: 1,250 ballots 

 Removable tote bin: 1,500 ballots 

 Auxiliary compartment: 100 ballots 

o Metal Ballot Box Capacity 

 Main compartment: 1,500 ballots 

 Write-in compartment: 1,500 ballots 

 Auxiliary compartment: 125 ballots 

• DS450 

o Approx. 5 million ballot images and related data of memory 

o Tray/Bin Capacity 

 Input tray 480 

 Output bin 480 

 Outstack bins 150 

• DS850 

o Approx. 5 million ballot images and related data of memory 

o Tray/Bin Capacity 

 Input tray 480 

 Output bin 480 

 Outstack bin 150 

• Number of characters allowed on a single Write-In selection: 30 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 23 of 84 

Template Rev 2015-03 

Managing technology risk 

1.8.4 Supported Languages 

The ES&S EVS 6.2.0.0 voting system has declared support for English, Spanish, 

Chinese, Korean, Japanese, Hindi, Bengali, Vietnamese, Tagalog, Creole, 

Russian, and French languages for this certification project. 

1.8.5 Supported Functionality 

The following standard VVSG functionality and manufacturer extensions are 

included in the voting system. 

1.8.5.1 Voting Variations 

The ES&S EVS 6.2.0.0 supports the following voting variations: 

• Closed primaries 

• Open primaries 

• Partisan offices 

• Non-partisan offices 

• Write-in voting 

• Ballot rotation 

• Straight party voting 

• Cross-party endorsement 

• Split precincts 

• Vote for N of M 

• Recall issues with options 

• Provisional or challenged ballots 

Note that in ES&S EVS 6.2.0.0, ranked order voting contests are supported in the 

ballot layout and the system offers a results report that can be utilized by a 

jurisdiction to resolve ranked order voting contests. The logic for determining 

results for straight party is not supported. 

1.8.5.2 Electionware - Pre-voting Functionality 

Electionware consists of four pre-voting software groups: Manage, Define, Design 

and Deliver, all of which allow users to prepare an election by defining election 

data and ballot styles that can be transferred to portable flash media for purposes 

of voting and vote capture. 

The Manage group allows the election official to manage users and jurisdictions. 

The Setup module adds and edits users and jurisdictions and sets the strength of 

election codes (the election code, which is used for accessing devices to load 

election definitions, strength feature is for determining the length and complexity of 

the election security code for ES&S equipment). 

The Define group is used to input critical election data. The Define group allows 

users to: create, edit and manage elections; enter election based information into 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 24 of 84 

Template Rev 2015-03 

Managing technology risk 

database like precincts, districts, contests, candidates, and other pertinent 

information; and stores graphics for ES&S equipment and for managing system 

translations and audio. 

The Design group designs ballots and audio elements for an election. The Design 

group controls: paper ballot design for use with ES&S equipment, electronic ballot 

design to support multiple screen layout sizes, format ballot display options as they 

will appear, manage ADA audio settings, and validate data before the election files 

are generated. 

The Deliver group configures equipment, packages election data and prints ballots 

on demand. This group provides specific access to allow users to: set parameters 

and access codes for election equipment, create election media files, view and 

print BOD (Ballot on Demand) ballots by precinct. 

Importing and Managing Election Data

Electionware supports importation of and the creation and configuration of an 

election as follows: 

• Stores all of a jurisdiction’s precinct, office, and candidate information. 

• Once an election database is complete, it can be recalled and edited. 

• The System Event Log tracks all user and equipment actions throughout the 

election process. 

Formatting Ballots 

Electionware includes a functional subsystem providing all activities related to 

defining the image of a paper ballot and populating the ballot with contest, 

candidate and referendum information from the database in the Paper Ballot 

module, which prepares an election by defining ballot styles. 

Proofing Election Data 

Electionware provides the following proofing capabilities: 

• Provides methods to validate ballot data that has been entered. 

• Users can: 

• View ballot style previews through an embedded viewer. 

• Print ballot style previews. 

• Save ballot styles to image files. 

• Proof recorded audio strings. 

• Electionware allows users to proof translations for supported languages. 

• Electionware supports the production of various reports that allow users to 

proof-read and validate ballot data entry. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 25 of 84 

Template Rev 2015-03 

Managing technology risk 

Ballot Content – Supported Information 

Electionware supports the following types of data: 

• Ballot header information, includes the election type, ballot label, jurisdiction 

name, election title, election date. 

• Standard and customized ballot instructions. 

• Election types. 

• Party information and graphics. 

• Contest types and associated contest options, including contest-specific 

instruction text and/or proposition text. 

• Rotation Indices. 

Electionware is capable of formatting ballots with the following information: 

• Unique ballot identifying information (to force the creation of unique ballot 

styles for the selected poll place). 

• Page numbering. 

• Static text strings. 

• Ballot instructions with custom content. 

• Ballot images. 

• Multi-language data (including audio, text, and image files as appropriate). 

• Election data per contest with fully detailed options/candidates and images 

according to the contest type. Contests include candidates and propositions. 

• Voting logic per contest entry to set the voting and tabulation method per 

contest. 

• Write-in options. 

Ballot Standards 

Electionware employs the ballot standards, which are supported by EVS 6.2.0.0, 

as follows: 

• Supports the following paper sizes: 

• 8.5” x 11” 

• 8.5” x 14” 

• 8.5” x 17” 

• 8.5” x 19” 

• Supports ballot layouts in portrait and landscape orientation. 

• Supports duplex ballot layouts. 

• Supports the inclusion of ballot stubs on paper ballots. 

• Compatible with the production of ballots on standard, commercially 

available white paper stock. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 26 of 84 

Template Rev 2015-03 

Managing technology risk 

Electionware employs and supports the activation card paper standards as 

follows: 

• Supports the following activation card paper sizes: 

 • 4.26” x 11” 

 • 4.26” x 14” 

 • 4.26” x 17” 

 • 4.26” x 19” 

• Compatible with specially manufactured thermal paper stock 

• Supports single-sided printing 

• Requires a die-cut corner 

Ballot Layout Features 

Electionware supports ballot layouts as follows: 

• Supports the layout of ballots in paper-based and electronic formats. 

Previewing Ballot Layouts 

Electionware supports previewing ballot layouts as follows: 

• Allows users to view on-screen graphic previews of exact ballot layouts. 

• Allows users to specify which ballot types are previewed, according to a 

variety of user-defined attributes, including precinct, style, type, language, 

and party. 

• Supports users to proof-read and validate ballot layouts. 

• Supports the production of audit logs for transparency. 

Configuring Ballot Handling Equipment 

Configuration of ballot handling equipment includes all tasks required to convert 

Electionware database information into ballot definition parameters for tabulation 

and Universal Voting System equipment. Within this functional subsystem, election 

management software users program the ballot counting rules for a specific 

election – the election definition – to the memory devices used to program 

tabulation equipment. 

Generating Election Media 

Electionware employs and supports generation of election media as follows: 

• Allows users to generate portable flash media that includes election data 

and ballot style information that the user has accepted. 

Ballot Production 

Electionware employs and supports ballot production as follows:

• Supports ballot production in a variety of formats, including generation of 

electronic ballots, and production of ballot printing files in PDF format 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 27 of 84 

Template Rev 2015-03 

Managing technology risk 

1.8.5.3 DS200 – Device Functionality 

DS200 provides paper ballot and vote summary card ballot scanning capabilities. 

When scanned, the system validates the ballot according to voter marks and the 

associated election contests. If issues are located, the system may provide a 

warning or flag the scanned ballot for review. Some examples include overvotes, 

undervotes, if so configured, damaged ballot, and a ballot not configured for the 

device or election. The scanned voter selections are stored to a USB memory 

device. The USB memory device is removable from the system for transport to a 

central election location where vote totals are consolidated for reporting. 

General 

DS200 devices support the following components: 

• Scanning mechanism for all specified/documented ballot sizes and vote 

summary cards lengths 

• Paper ballot transport path 

• Plastic Ballot Box Capacity 

• Metal Ballot Box Capacity 

• Collapsible Ballot Box Capacity 

• USB Memory Capacity 

Scanning 

DS200 Devices: 

• Include a bi-directional scanner. 

• Accommodate duplex scanning. 

• Are capable of capturing and storing the ballot image on removable media. 

• Are capable of capturing and storing the digital image of a write-in vote. 

• Enlist a multi-sheet scanning functionality. 

• Scan paper ballots and/or vote summary cards in all portrait orientations 

(face up, face down, header first, footer first). 

• Maintains a real-time audit log printable from each terminal’s integrated 

printer and stored on the election media. 

Configurable Settings 

DS200 Devices are configurable for: 

• Unconditional acceptance where the DS200 accepts and tabulates results 

for all ballots. 

• Unconditional rejection where the DS200 automatically rejects ballots that 

contain the specified condition. 

• Query the voter for revision where the DS200 displays a screen message 

that describes the condition and prompts the voter to either review and 

revise the ballot or cast the ballot as it is. 

• Results backup media capability. 

• Capturing ballot images that can be viewed in Electionware. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 28 of 84 

Template Rev 2015-03 

Managing technology risk 

• Mechanism that allows counted ballots to be stamped. It can be 

configured to stamp all ballots or just ballots meeting criteria such as 

ballots containing over votes or write-ins.

• Configurable system health status in a QR two dimensional barcode that 

can be scanned, via smarphone app, to pass the data back to a central 

administration site in preparation for opening the polls.

Supported Ballot Types 

DS200 Devices: 

• Accept vote summary cards of all specified/documented lengths 

• Accept ballots of all specified/documented lengths 

• Maintain public ballot counters that reflect the total number of ballots cast or 

marked for the current election configuration and the total number of sheets 

processed. 

Scanning Device Vote process 

DS200 Devices: 

• Display voter instruction messages for all valid operations. 

• If configured to unconditionally reject an exception ballot, the system returns 

the ballot to the voter and displays a message describing the exception 

condition. 

• If configured to unconditionally accept, the inserted ballot will be tabulated 

as marked, which may result in the inclusion of undervotes and/or overvotes 

in system totals. 

• If configured to query the voter prior to acceptance, the DS200 displays a 

message describing the exception condition, indicates the exception 

condition, and prompts the voter to revise the ballot, which may result in the 

inclusion of undervotes and/or overvotes in system totals. 

Scanning Device Error Handling 

DS200 Devices: 

• When applicable, display all error messages as they occur on the LCD 

display. 

• Provide error messages to assist operators to determine the correct action. 

• Recognize multiple sheet detection where more than one sheet of paper is 

recognized and returns an error message to the user display indicating the 

error. 

• Are designed to detect and record critical events in the Audit log and critical 

events report. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 29 of 84 

Template Rev 2015-03 

Managing technology risk 

Security 

DS200 uses digital signatures, encryption, and security access codes to safeguard 

data. 

1.8.5.4 DS450 – Device Functionality 

DS450 is a high-throughput central scanning component that provides paper ballot 

and vote summary card scanning functionality and conversion of voter selection 

marks to electronic Cast Vote Records (CVR). Once the CVR’s are stored they can 

be transferred into Electionware software for vote tabulation, adjudication and 

reporting of election results. 

General 

The DS450 central scanner: 

• Is capable of operating in local network or standalone configurations. 

• Loads a set of election definition files via an ES&S Election flash drive. 

• Captures an image of the ballot to create a vote image record that is used to 

aggregate totals whenever the user selects to export results to the ES&S

Results flash drive. 

• Transports paper ballot to output bins. 

• Has USB Memory Capacity. 

Scanning 

The DS450 device scans ballots, designed by Electionware in the Paper Ballot 

module, through the paper transport mechanism. The DS450 is able to: 

• Scan bi-directionally. 

• Accommodate duplex scanning. 

• Analyze each possible oval location. 

• Use software to determine user markings at each possible active response 

location and store these values along with the Election Definition to 

determine the voter’s choices. 

• Scan multiple precincts using the precinct selection option. 

• Tabulate ballots at a central scanning location. 

• Capture and store ballot images on removable media. 

• Capture and store the digital image of a write-in vote. 

• Support Multi-Sheet scanning functionality. 

• Scan paper ballots and/or vote summary cards in all portrait orientations 

(face up, face down, header first, footer first). 

• Provide system audit log records of all error conditions and corrective 

actions. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 30 of 84 

Template Rev 2015-03 

Managing technology risk 

Configurable Settings 

DS450 devices are configurable: 

• To scan vote summary cards printed by the ExpressVote HW1.0 and 

ExpressVote HW2.1. 

• To use the ballot tracking option to visually show the user which ballots are 

not processed and which ballots can remain in the output bin. 

Supported Ballot Types 

DS450 devices: 

• Accept vote summary cards of all specified/documented lengths. 

• Accept ballots of all specified/documented lengths. 

• Maintain public ballot counters that reflect the total number of ballots cast or 

marked for the current election configuration and the total number of sheets 

processed. 

Scanning Device Operator process 

DS450 devices: 

• Allow the operator to preview/print the election Zero Report once the 

election definition is loaded. 

• Allow the operator to save and export the results and ballot images, after 

each batch is scanned, continue scanning additional batches, change sort 

options, delete the current unsaved batch, or generate election reports. 

Scanning Device Error Handling 

DS450 devices check: 

• If any problems occur during the scanning process or if the ballot is not 

identified in the Election Definition, the ballot may be diverted to an 

exception bin for further analysis. 

• If the precinct on the ballot does not match the selected precinct for that 

stack of ballots, that ballot will be out stacked. 

• If a jam occurs and stops processing ballots and identifies which ballots 

were counted and which need to be returned to the input tray for scanning. 

If exceptions occur, the Master Control Program, which maintains the lowlevel hardware lists and provides an application program interface to the 

information, handles exceptions by logging the event through the Audit ID, 

an internal system code which contains the error. The error is logged using 

the built-in operating system logging functionality. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 31 of 84 

Template Rev 2015-03 

Managing technology risk 

1.8.5.5 DS850 – Device Functionality 

DS850 is a high speed central scanning component that provides paper ballot 

scanning functionality and conversion of voter selection marks to electronic Cast 

Vote Records (CVRs). Once the CVRs are stored they can be transferred into 

Electionware software for vote tabulation, adjudication and reporting of election 

results. 

General 

The DS850: 

• Is capable of operating in local network or stand-alone configurations. 

• Loads a set of election definition files via an ES&S Election flash drive. 

• Captures an image of the ballot to create a vote image record that is used to 

aggregate totals whenever the user selects to export results to the ES&S

Results flash drive. 

• Transports paper ballot to output bins. 

• Has USB Memory Capacity. 

Scanning 

The DS850 device scans ballots and/or vote summary cards, designed by 

Electionware in the Paper Ballot module, through the paper transport mechanism. 

The DS850 is able to: 

• Scan bi-directionally. 

• Accommodate duplex scanning. 

• Analyze each possible oval location. 

• Use software to determine user markings at each possible active response 

location and store these values along with the Election Definition to 

determine the voter’s choices. 

• Scan multiple precincts using the precinct selection option. 

• Tabulate ballots at a central scanning location. 

• Capture and store ballot images on removable media. 

• Capture and store the digital image of a write-in vote. 

• Support Multi-Sheet scanning functionality. 

• Scan paper ballots and/or vote summary cards in all portrait orientations 

(face up, face down, header first, footer first). 

• Provide system audit log records of all error conditions and corrective 

actions. 

Configurable Settings 

DS850 devices are configurable: 

• To scan vote summary cards printed by the ExpressVote HW1.0 and 

ExpressVote HW2.1. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 32 of 84 

Template Rev 2015-03 

Managing technology risk 

Supported Ballot Types 

DS850 devices: 

• Accept vote summary cards of all specified/documented lengths. 

• Accept ballots of all specified/documented lengths. 

• Maintain public ballot counters that reflect the total number of ballots cast or 

marked for the current election configuration and the total number of sheets 

processed. 

Scanning Device Operator process 

DS850 devices: 

• Allow the operator to preview/print the election Zero Report once the 

election definition is loaded. 

• Allow the operator to save and export the results and ballot images, after 

each batch is scanned, continue scanning additional batches, change sort 

options, delete the current unsaved batch, or generate election reports. 

Scanning Device Error Handling 

DS850 devices check: 

• If any problems occur during the scanning process or if the ballot is not 

identified in the Election Definition, the ballot may be diverted to an 

exception bin for further analysis. 

• If the precinct on the ballot does not match the selected precinct for that 

stack of ballots, that ballot will be out stacked. 

• If a jam occurs and stops processing ballots and identifies which ballots 

were counted and which need to be returned to the input tray for scanning. 

• If exceptions occur, the Master Control Program handles exceptions by 

logging the event through the Audit ID, which contains the error. The error is 

logged using the built-in operating system logging functionality. 

1.8.5.6 ExpressVote Hardware 1.0 – Device Functionality 

ExpressVote HW1.0 is a vote capture device designed for all voters, with 

independent voter-verifiable paper record that is digitally scanned for tabulation on 

a compatible ES&S tabulator. This system combines paper-based voting with 

touch screen technology. 

Multiple Ballot Styles 

If an election has more than one ballot style: 

• Initially, the ExpressVote HW1.0 system prompts the poll worker to insert 

an activation card, and then select the correct ballot style and precinct, in 

order to display the proper ballot for the voter. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 33 of 84 

Template Rev 2015-03 

Managing technology risk 

• If an activation barcode has been printed on the card, then the voter checks 

in, the voter will insert the card, into the ExpressVote HW1.0, the 

barcode is scanned and immediately displays the proper ballot for the 

voter is displayed. 

• The ExpressVote HW1.0 accepts both ES&S created activation barcodes 

and authorized activation barcodes from other voter identification vendors. 

Accessibility Support 

The ExpressVote HW1.0 is designed for all voters, along with the ballot display 

and audio prompts, the device supports: 

• A detachable keypad (DKB), which is used with audio prompts to vote the 

ballot. 

• Two-position switch devices such as a sip-and-puff and rocker switch. 

Configurations 

The ExpressVote HW1.0 has four configurations: ADA table, tabletop, voting 

booth, or rolling kiosk. 

• The rolling kiosk is easily positioned for seated or standing voters and 

can optionally provide QR code scanning, report printing, and a secure 

card container. The QR code enables the voter to scan a printed QR 

code used for either activation barcodes, and or sample ballot 

selections, from an ancillary product the voter made before arriving at 

the polling place. The voter can review and/or change any of the sample 

selections via the ExpressVote terminal. 

• The ExpressVote HW1.0 supports judge’s initial and review boxes. 

Telecommunications Abilities 

• No telecommunication capabilities for transfer of data from the polling 

site. 

Security 

ExpressVote HW1.0 uses digital signatures, encryption, and security access 

codes to safeguard data. 

Audio Interface 

With audio election data to be loaded and headphones connected to the 

ExpressVote HW1.0, the unit’s operating system and loaded firmware provide a 

platform for playing audio files from the ES&S Election flash drive. This audio 

interface complements the GUI screens by helping the voter navigate the ballot 

contests, select their desired candidates, print their selections on the activation 

card, and either retrieve or automatically cast (AutoCAST) their marked vote 

summary card. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 34 of 84 

Template Rev 2015-03 

Managing technology risk 

1.8.5.7 ExpressVote Hardware 2.1 – Device Functionality 

ExpressVote HW2.1 is a vote capture device designed for all voters, with the 

capability to print an independent voter-verifiable paper record (VVPR) to be 

tabulated in a scanner. ExpressVote HW 2.1 also has the capability to be used as 

a scanner to tabulate the VVPR. This system combines paper-based voting with 

touch screen technology. 

Multiple Ballot Styles 

If an election has more than one ballot style: 

• Initially, the ExpressVote HW2.1 system prompts the poll worker to insert 

an activation card, and then select the correct ballot style and precinct, in 

order to display the proper ballot for the voter. 

• If an activation barcode has been printed on the card, then the voter checks 

in, the voter will inserts the card, into the ExpressVote HW1.0, the 

barcode is scanned and immediately displays the proper ballot for the 

voter is displayed. 

• The ExpressVote HW2.1 accepts both ES&S created activation barcodes 

and authorized activation barcodes from other voter identification vendors. 

Accessibility Support 

The ExpressVote HW2.1 is designed for all voters, along with the ballot display 

and audio prompts, the device supports: 

• A detachable keypad (DKB), which is used with audio prompts to vote the 

ballot. 

• Two-position switch devices such as a sip-and-puff and rocker switch. 

Configurations 

The ExpressVote HW2.1 has four configurations: ADA table, tabletop, voting 

booth, or rolling kiosk. 

• The rolling kiosk is easily positioned for seated or standing voters and 

can optionally provide QR code scanning, report printing, and a secure 

card container. Enables the voter to scan a printed QR code containing 

sample selections the voter made before arriving at the polling place. 

The voter can review and/or change any of the sample selections via 

the ExpressVote terminal. 

• The ExpressVote HW2.1 supports judge’s initial and review boxes. 

• Rear-eject feature (Marker Only); when disabled, the printed card is 

returned to the voter for insertion into a tabulator or secure ballot box. 

When enabled, the voter makes their selections, they are given the 

option to either receive their printed card or have it AutoCAST. 

AutoCAST is an option provided to the voter that allows them to 

automatically cast their vote summary card, if they don’t wish to have it 

returned to them. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 35 of 84 

Template Rev 2015-03 

Managing technology risk 

Telecommunications Abilities 

• No telecommunication capabilities for transfer of data from the polling 

site. 

Security 

ExpressVote HW2.1 uses digital signatures, encryption, and security access 

codes to safeguard data. 

Audio Interface 

With audio election data to be loaded and headphones connected to the 

ExpressVote HW2.1, the unit’s operating system and loaded firmware provide a 

platform for playing audio files from the ES&S Election flash drive. This audio 

interface complements the GUI screens by helping the voter navigate the ballot 

contests, select their desired candidates, print their selections on the activation 

card, and either retrieve or automatically cast (AutoCAST) their marked vote 

summary card. 

Tabulation 

In addition to retaining all feature functionality as a ballot marker, the ExpressVote

HW2.1 can also be configured to tabulate printed vote summary cards, from itself 

or other ExpressVote HW2.1 and ExpressVote HW1.0 devices. The 

ExpressVote HW2.1 as a tabulator uses a Master Media device for Poll Open and 

Poll Close functions. The tabulated vote summary card is deposited into a 

removable, secure container attached to the rolling kiosk. The Master Media is a 

flash drive that is used to open each terminal for voting. This procedure is referred 

to as opening the poll. The master media tracks the terminals it is used to open 

and is used to generate the consolidated Zero Totals report for those terminals. 

1.8.5.8 Post Voting Functionality 

The Results group in Electionware contains the post voting capability to import, 

manage, and report election results data from the tabulation media. This group 

includes all of the tools used for loading results, machine logs, cast vote records, 

and ballot images; creates the results reports; ballot review and adjudication; and 

write-in review and management. 

After the election, the Reporting module can display or export detailed status 

information about individual flash drives, and produce machine log reports. It can 

clear loaded results files, in order to delete test data or to start the loading process 

over. The Reporting module is also used to filter, view and export ballot data, print 

ballot scan images and facsimiles, and review ballot images, write-ins and cast 

vote records. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 36 of 84 

Template Rev 2015-03 

Managing technology risk 

General 

The Electionware Reporting module: 

• Is capable of tabulating and reporting results for any EVS 6.2.0.0 election 

• Can display or export detailed status information for individual flash drives, 

and produce machine log reports. 

• Only accepts elections that have been created in Electionware and USB 

media created by Electionware

• Encrypts the key exchange process between the EMS and the tabulators to 

ensure confidentiality. 

• Supports 14 reporting groups that are used to categorize different groups of 

ballot styles for statistical reporting purposes. 

• Is capable of being installed on a “standalone” PC that is not networked to 

other PCs. 

o If configured in a standalone configuration, the component is capable 

of reading an election definition from portable media that is inserted 

into the tabulation PC. 

• Is capable of being installed on a networked workstation that is connected to 

other Electionware workstations. 

o A networked Central Count Tabulator is capable of transmitting 

tabulation results, CVRs, ballot images and the Audit Log. 

• Digitally signs every cast vote record and digitally signs the package of cast 

vote records captured by the tabulators. 

• Is capable of supporting multiple elections at the same time. 

• Allows the user to manage different elections. 

• Supports the following user functions in appropriate sections of the user 

interface: 

 • Tabulation database management 

 • Reading and tabulating portable election media 

 • Election progress dashboard 

 • Managing write-in votes 

 • Reporting (including standard reports, and customized reports) 

 • Audits and recounts 

• Allows users to import data from external sources, such as importing voter 

registration data and customer determined values into the alternate ID 1 

field, to facilitate tabulation database configuration. 

Results Loading 

Electionware: 

• Will automatically load results from every connected poll media flash drive 

when activated. 

• Will automatically load results from the network until stopped by the user. 

• Checks each flash drive as it is read to verify it is both an approved ES&S

flash drive and that it contains matching signatures and hashes. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 37 of 84 

Template Rev 2015-03 

Managing technology risk 

• Authenticates the flash drive, and then proceeds to decrypt the files on the 

flash drive. 

• Verifies that the decrypted files have not already been processed, based on 

the polling location and flash drive serial numbers. 

• Bases its results and reporting totals on the tabulated results in the Poll 

Place Collection Files. 

• The CVR’s and Ballot Images may be used for ballot review and auditing 

tabulated results. 

Ballot Review 

Electionware provides the ability for on-screen ballot review of write-in votes and 

unreadable marks. Electionware allows an election official to: 

• Review images of each write-in or unreadable mark in a contest to be 

assigned and tallied for reporting purposes. 

• Filter reviewable fields by contest type or condition 

• Bulk assign write-ins to candidates, where user can edit multiple instances 

of a single contest that have similar write-in names. 

Reporting 

The election results workflow in Electionware is used to generate paper and 

electronic tabulated results reports and exports. Electionware:

• Offers methods to protect voter privacy in instances where reported results 

are based on small numbers of ballots cast. 

• Provides user access level reports like: system administrator only reports, 

administrator reports, media creator reports, system user only reports. 

• Provides several reports regardless of the user access level. 

• Allows users to generate reports to various formats for review. 

1.8.6 Excluded Requirements 

In this section VVSG requirements are identified that do not pertain to the declared 

system being certified. Cumulative Voting, Ranked Choice Voting, DRE, 

biometrics, shared environments, public telecommunications, nor wireless 

telecommunications capabilities are not included in this system configuration. The 

table below enumerates the requirements that will not be subject to verification for 

this project. 

Requirements Not 

Under Test 

Requirement Title 

1.5.2.2 Direct-Recording Electronic Voting System 

1.5.2.3 Public Network Direct-Recording Electronic Voting 

System 

2.1.2.f Accuracy, DRE systems 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 38 of 84 

Template Rev 2015-03 

Managing technology risk 

2.1.4.k,l Integrity, DRE systems 

2.1.7.2.m Cumulative Voting 

2.1.7.2.n Ranked Choice Voting 

2.3.1.3 Voting Capabilities, DRE System Requirements 

2.3.2 Activating the Ballot 

2.3.3.3 Casting a Ballot, DRE System Requirements 

3.1.2.f,g Functional Capabilities, DRE 

3.2.1.c General (biometric measures) 

4.1.1.b Accuracy Requirements, DRE 

4.1.4.3 DRE System Recording Requirements 

4.1.6.2 DRE System Processing Requirements 

4.3.5.b Availability, DRE 

5.4.3.b.iv In-process Audit Records, DRE 

5.5 Vote Secrecy on DRE Systems 

3.2.1.c General, Biometrics 

Vol. 2 , 4.7.4.c Availability, DRE 

6.2.6 Integrity (public telecommunications) 

7.5.2 Protection Against External Threats 

7.5.3 Monitoring and Responding to External Threats 

7.5.4 Shared Operating Environment 

7.6 Use of Public Communications Networks 

7.7 Wireless Communications 

7.8 Independent Verification Systems 

7.9 Voter Verifiable Paper Audit Trail Requirements 

1.8.7 Additional Functionality/Requirements 

As per 2005 VVSG, volume 2 section 3.2.2, review of the ES&S EVS 6.2.0.0

System Functionality Description document resulted in SLI’s determination that 

some functionality/requirements that are considered beyond the scope of the 

VVSG, but since they are in the declared system, will require review and 

verification. 

VVSG related 

requirement (if 

applicable) 

Feature 

EMS 

2.1.2.b Electionware can configure the DS200 to accept or reject ballots based 

on whether or not Review Box and/or Judges Initials Boxes are required. 

2.1.7.2 Electionware supports both NY and PA style cross-endorsements 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 39 of 84 

Template Rev 2015-03 

Managing technology risk 

through the assigning of additional parties to a candidate in the 

candidate dialog. 

2.3.3.1.d Electionware can configure the DS200 for ballot stamping write-ins.

None Electionware can configure the DS200 to display system health 

status in a QR two dimensional barcode that can be scanned to 

pass the data back to a central administration site in preparation 

for opening the polls. A poll worker can take a picture of the code 

with their smartphone and use the smartphone-based scanning 

application to pass the data to a central administration site. 

2.3.1.2.c Electionware can configure the DS200 that a steel ballot box can be 

equipped to optionally divert ballots into two compartments based 

on ballot criteria such as blank ballots and overvotes. 

2.3.3.1.d Electionware can configure the DS200 to print a report containing an 

image or “snippet” of the write-in entries from voters’ ballots or vote 

summary cards. 

Electionware can configure the DS200, DS450 and DS850 so that 

ballot images captured devices, can be viewed in Electionware. 

Electionware can configure the DS200, DS450 and DS850 so that 

ballots with Write Ins can be configured to be filtered from other 

ballots in Electionware. 

Electionware can configure the DS200, such that ballots containing 

write-ins can also be reviewed by displaying their images onscreen.

Electionware can configure ExpressVote’s to allow a voter can take 

a barcode (printed or mobile device) and load their vote selections 

to an ExpressVote session.

The Electionware Reporting module can be used to export 

decrypted ballot images for FOIA purposes. 

DS200 

2.1.2.b The DS200 may be configured to accept or reject ballots based on 

whether or not Review Box and/or Judges Initials Boxes are required. 

2.1.7.2 The DS200 may be configured for cross party endorsement where a 

candidate may be associated with up to four different parties. 

Selection of any party associated with the candidate will result in 

that candidate receiving a vote. 

2.3.3.1.d The DS200 may be configured for ballot stamping of ballots with 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 40 of 84 

Template Rev 2015-03 

Managing technology risk 

write-ins.

None The DS200 is able to display system health status in a QR two 

dimensional barcode that can be scanned to pass the data back to 

a central administration site in preparation for opening the polls. A 

poll worker can take a picture of the code with their smartphone 

and use the smartphone-based scanning application to pass the 

data to a central administration site. 

2.3.1.2.c The DS200 can be configured such that when paired to a steel 

ballot box that is equipped to optionally divert ballots into two 

compartments based on ballot criteria such as blank ballots and 

overvotes, will do so. 

2.3.3.1.d DS200 can print a report containing an image or “snippet” of the 

write-in entries from voters’ ballots or vote summary cards. Images 

of ballot pages that contain write-ins can also be reviewed on 

screen after the polls have closed.

Ballot images captured by the DS200 can be viewed in 

Electionware. 

The DS200 can be configured such that ballots containing write-ins 

can also be reviewed by displaying their images on-screen.

EV HW1.0 

2.1.7.2 ExpressVote HW1.0 can be configured to support cross party 

endorsement where a candidate may be associated with up to four 

different parties. Selection of any party associated with the 

candidate will result in that candidate receiving a vote. 

ExpressVote HW1.0 can be configured to allow a voter to be able 

to take a barcode (printed or mobile device) to their precinct and 

load their vote selections to an ExpressVote session using a 

barcode scanning device attached to the ExpressVote or 

integrated into the ExpressVote rolling kiosk. The selections the 

voter made will be presented to the voter on the ExpressVote 

summary screen. The voter will then be able to proceed with the 

ExpressVote session as normal (making/ changing/confirming vote 

selections, printing selections, etc.).

EV HW2.1 

2.1.7.2 ExpressVote HW2.1 can be configured to support cross party 

endorsement where a candidate may be associated with up to four 

different parties. Selection of any party associated with the 

candidate will result in that candidate receiving a vote. 

ExpressVote HW2.1 can be configured to allow a voter to be able 

to take a barcode (printed or mobile device) to their precinct and 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 41 of 84 

Template Rev 2015-03 

Managing technology risk 

load their vote selections to an ExpressVote session using a 

barcode scanning device attached to the ExpressVote or 

integrated into the ExpressVote rolling kiosk. The selections the 

voter made will be presented to the voter on the ExpressVote 

summary screen. The voter will then be able to proceed with the 

ExpressVote session as normal (making/ changing/confirming vote 

selections, printing selections, etc.).

2.4.2 The ExpressVote Tabulator allows for the poll-level consolidation 

and reporting of votes cast. 

DS450 

2.3.3.1.d DS450 tabulators can be set to out stack or sort ballots for which 

the Review Box is marked or the Judges Initials Boxes are not 

properly completed. 

Election officials have the option to change these settings after the 

proper access code is provided and as the laws of the particular 

jurisdiction allow.

2.1.7.2 DS450 supports cross party endorsement where a candidate may 

be associated with up to four different parties. Selection of any 

party associated with the candidate will result in that candidate 

receiving a vote. 

Ballot images captured by the DS450 can be viewed in 

Electionware. 

The DS450 can be configured to physically sort ballots containing 

write-ins and allow an option for these ballots to be counted or 

uncounted. 

DS850 

2.3.3.1.d DS850 tabulators can be set to outstack or sort ballots for which 

the Review Box is marked or the Judges Initials Boxes are not 

properly completed. 

Election officials have the option to change these settings after the 

proper access code is provided and as the laws of the particular 

jurisdiction allow.

2.1.7.2 The DS850 supports cross party endorsement where a candidate 

may be associated with up to four different parties. Selection of 

any party associated with the candidate will result in that candidate 

receiving a vote. 

Ballot images captured by the DS850 can be viewed in 

Electionware. 

The DS850 can be configured to physically sort ballots containing 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 42 of 84 

Template Rev 2015-03 

Managing technology risk 

write-ins and allow an option for these ballots to be counted or 

uncounted. 

1.9 Change Control and Configuration Management 

The SLI project team follows a standard Change Control and Configuration 

Management (CM) process. This specifies the methods used by SLI to ensure 

changes are managed and controlled effectively and efficiently and defines the 

process for receipt, check-in, storage and disposition of hardware, source code, 

TDP and non-TDP Documents. 

It specifies the processes and procedures used by SLI to identify, monitor, and 

control versions of all of the stated EAC project deliverables, ensuring that the 

items are constantly and reliably managed throughout the entire duration of the 

engagement to ensure changes are managed and will not result in invalidated or 

wasted testing efforts. 

A summary of these tasks includes: 

• Receive TDP source code: 

o Delivery department will perform a code check-in on the server. 

Code is placed in the specific project’s TDP folder with the delivery 

date and documentation of what was delivered. 

o Delivery department notifies Test Manager (TM) of delivery via email. 

o TM will notify the Source Code Review (SCR) team, via email, the 

code is ready for review. 

o TM will review the documentation of what was delivered to determine 

possible testing impacts. 

o Source Code Reviewer updates status of delivered code as received, 

what code to compare it to (if this is not new code), where code has 

been placed, and who will do the review of the code. 

o The code is checked out by the SCR who is responsible for the code 

review. 

o When the code review is complete, the TM is notified. 

o A peer review is done on the review and all discrepancies are verified 

and an official Discrepancy Report is sent to ES&S. 

o If the Source Code Review (SCR) Team determines the code does 

not meet the requirements of the 2005 VVSG, the SCR team will 

review code impacts with the TM. 

o TM determines when code is ready for a Trusted Build. When it is, 

the TM sends approval for a Trusted Build to the Trusted Build team. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 43 of 84 

Template Rev 2015-03 

Managing technology risk 

• Receive TDP documents: 

o Delivery department will perform a check-in of the documents on the 

server. 

o Documents are placed in the specific project’s TDP folder with the 

delivery date. 

o Delivery department notifies TM of delivery via email. 

o TM assigns VTS personnel to perform the documentation review. 

o VTS personnel perform a PCA on the pertinent documentation. 

o A peer review is done on the VTS personnel’s review, all 

discrepancies are verified and an official Discrepancy Report is sent 

to ES&S. 

• Receive Hardware 

o The HW Specialist or designee checks in equipment that is delivered 

by ES&S.

o Operational Status Checks are performed on all delivered hardware. 

o When the equipment comes back from the hardware lab testing, the 

HW Specialist checks in the equipment and verifies it’s the same 

hardware that was sent to the lab. 

o The equipment is stored in a secure room with controlled access. 

• Compliance/Trusted Build: 

o The TM assigns a Trusted Build team member to review the 

Compliance/Trusted Build procedures and perform the build. 

o Trusted Build team member follows build procedures exactly as 

written. 

o Trusted Build team member pulls the finalized source code from the 

specific folder where it was checked in. 

o Once build is complete the Trusted Build team member provides the 

completed build to the SLI Delivery Team to be checked in. 

o Trusted Build team member notifies the TM that the

Compliance/Trusted Build is completed. 

o TM assigns appropriate VTS(s) to load the resulting 

applications/installs and begin testing. 

o The VTS team performs appropriated installations, and begins 

testing. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 44 of 84 

Template Rev 2015-03 

Managing technology risk 

2 PRE-CERTIFICATION TESTING AND ISSUES 

This section addresses any pre-certification testing that has been performed on the 

system under review. For this certification project, no previous testing has been 

performed on the ES&S EVS 6.2.0.0 voting system, as it is a newly developed and 

as of yet un-fielded system. 

2.1 Evaluation of prior VSTL testing 

No prior VSTL testing is pertinent to the ES&S EVS 6.2.0.0 voting system, as it is a 

system undergoing an Initial Certification process. 

2.2 Evaluation of prior non-VSTL testing 

No prior state or non-VSTL lab testing is pertinent to the ES&S EVS 6.2.0.0 voting 

system, as it is a system undergoing an Initial Certification process. Review of 

ES&S internal testing is performed during the FCA review. 

2.3 Known Vulnerabilities 

Review of the “Known Vulnerabilities” database, maintained by SLI, has provided 

49 known vulnerabilities to previous ES&S systems, which are already accounted 

for in SLI’s Testing. 

3 PHYISCAL CONFIGURATION AUDIT

3.1 TDP Documentation Analysis & Review 

SLI will complete an assessment of the deliveries of the Technical Data Package, 

including Functional Requirements, Specifications, End-user documentation, 

Procedures, System Overview, Configuration Management Plan, Quality 

Assurance Program, and manuals for each of the required hardware, software and 

firmware components of the ES&S EVS 6.2.0.0 voting system, in order to ensure 

that the documentation corresponds to the configuration and operation of the 

system. Any subsequent re-deliveries of the TDP items will be solely the result of 

fixes to discrepancies identified in the remaining FCA or PCA activities. 

3.1.1 Document Review 

SLI conducts a PCA review of all vendor traced documents submitted for review in 

the delivery of the ES&S EVS 6.2.0.0 TDP. These include the following areas: 

• System configuration overview 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 45 of 84 

Template Rev 2015-03 

Managing technology risk 

• System functionality description 

• System hardware specifications 

• Software design and specifications 

• System test and verification specifications 

• System security specifications 

• User/system operations procedures 

• System maintenance procedures 

• Personnel deployment and training requirements 

• Configuration management plan 

• Quality assurance program 

Each document included in the delivery of the voting system TDP is reviewed for 

compliance to the pertinent 2005 VVSG, Volume 2, Sections 2.2 through 2.13 and 

Volume 2, Section 6.6 requirements that are applicable to that document. Results 

and associated discrepancies of the documentation TDP review will be detailed in 

the final test report. 

3.2 TDP Source Code Analysis & Review 

3.2.1 Source Code Review 

SLI has conducted a source code review of all proprietary source code, and 

modified COTS products, submitted in the delivery of the voting system TDP for 

compliance to the VVSG, version 2005, Volume 2, Section 6.6. The coding 

languages involved in the vendor’s applications include: 

• C 

• C++ 

• C# 

• Java 

• VB 

• VB.net 

Source Code Review Tools utilized by SLI include: 

• LocMetrics Line Counter: a commercial application used to determine the 

counts of executable and comment lines; 

• Module Finder: an SLI proprietary application used to parse module names 

from C/C++ and VB code and populate the identified module names into 

the review documents; 

• ExamDiff Pro: a commercial application used to compare revised code to 

previously reviewed code; 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 46 of 84 

Template Rev 2015-03 

Managing technology risk 

• Understand: a commercial application to perform automated review of 

source code. 

Any subsequent re-reviews of source code will be the result of fixes to 

discrepancies identified in the FCA activities. 

COTS Operating systems and software used in the voting system will be verified 

as authentic and unmodified, otherwise reviewed as is applicable. 

3.3 QA & CM Process Review 

The review processes employed are designed to verify that the manufacturer not 

only has written processes and procedures in both the Quality Assurance and 

Configuration Management arenas, but that those processes and procedures are 

actually implemented within the software development life cycle that is used to 

produce the ES&S EVS 6.2.0.0 version that is submitted for certification. 

The QA portion of the review focuses on the review of the ES&S documented QA 

processes and policies, as well as the actual QA implementation and testing 

performed by ES&S. The scope of the testing performed is reviewed in order to 

verify that ES&S has verified that all VVSG requirements are met. SLI reviews the 

test case design documents and data as provided by ES&S. In evaluating each 

module, with respect to flow control parameters and data on both entry and exit, 

SLI assesses discrepancies between the Software Specifications and the test case 

design. Surveys addressing Quality Assurance aspects as implemented by ES&S

will be sent to ES&S QA personnel. Additionally, interviews of key Quality 

Assurance staff may be conducted to verify that the QA processes and procedures 

are known, understood and implemented by the appropriate personnel within the 

ES&S organization. The interviews will be conducted if issues are seen with either 

project artifacts or survey responses. 

The CM portion of the review focuses on the review of the ES&S documented CM 

processes and policies, as well as the ES&S organization’s understanding and 

implementation of the declared configuration management processes, procedures 

and policies. Initial deliverables will be reviewed against all pertinent CM processes 

employed by ES&S. Any and all subsequent deliveries will also be reviewed to 

determine that appropriate processes are employed. Surveys addressing 

Configuration Management aspects as implemented by ES&S will be sent to ES&S

CM personnel. Additionally, interviews of pertinent staff, with regard to 

configuration management, may be conducted to verify that processes, procedures 

and policies are known, understood and implemented within the ES&S

organization. The interviews will be conducted if issues are seen with either project 

artifacts or survey responses. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 47 of 84 

Template Rev 2015-03 

Managing technology risk 

3.4 Compliance/Trusted Build 

A Compliance Build will be conducted prior to SLI’s formal test execution and will 

be completed on site at SLI’s facility or a secure lab at the vendor’s facility 

approved by SLI. The Trusted Build will be conducted prior to SLI’s final test 

execution and will be completed on site at SLI’s facility or a secure lab at the 

vendor’s facility approved by SLI. SLI will use its approved standard lab procedure 

that details the processes for controlling, managing, and conducting the Trusted 

Build. This process includes the following: 

• Preparation for the Compliance/Trusted Build - Obtaining and reviewing 

ES&S’s procedure for constructing the build platform, verifying the target build 

platform, and acquiring and verifying the necessary materials. 

• Execution of the Compliance/Trusted Build – SLI will perform the 

Compliance/Trusted Build by using the step-by-step build procedure, as 

provided by ES&S to create a pristine build environment. SLI records and 

ascertains the following items throughout the build process: 

o Build environment images at various key points 

o Build environment and file hashes at various key points 

o Build environment hardware characteristics 

o Build results from code compilation and file hashes 

o Final software install files and file hashes 

• Deliverables to Testing – Upon completion of the Trusted Build, certain items 

are sent to the SLI test group. The final result will be media containing the 

following: 

o Final software install files

o Hash values to validate install files 

• Final Record Keeping and Archiving Procedures – At the conclusion of the 

Trusted Build process, SLI completes all final record keeping and archiving 

procedures at SLI’s facility. This record keeping includes any unique identifiers, 

results of the build with version numbers and dates and descriptions of all 

hashes and images in the repository. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 48 of 84 

Template Rev 2015-03 

Managing technology risk 

4 MATERIALS REQUIRED FOR TESTING 

Any materials that are used in an election cycle must be provided to SLI to facilitate 

testing of the voting system. This section outlines these materials that are required. 

4.1 Software/Firmware 

All software and firmware that is to be used by the declared voting system, whether 

directly or indirectly, in a production environment, must be validated during the 

certification process. 

The following software/firmware is required for the execution of hardware, 

software, telecommunications, and security tests. This includes all supporting 

software such as operating systems, compilers, assemblers, application software, 

firmware, any applications used for burning of media, transmission of data or 

creation/management of databases. 

4.1.1 Manufacturer Software/Firmware 

The ES&S EVS 6.2.0.0 voting system consists of the following software and 

firmware components: 

• Electionware Election database creation, media programming and 

tally/reporting software 

• DS450 Central Tabulator firmware, Central Count scanner and tabulator 

• DS850 Central Tabulator firmware, Central Count scanner and tabulator 

• DS200 Precinct Tabulator firmware, Precinct scanner and tabulator 

• ExpressVote HW1.0 UVS firmware, Precinct Universal Voting System 

• ExpressVote HW2.1 UVS firmware, Precinct Universal Voting System 

and/or tabulator 

• ExpressVote HW1.0 Previewer ballot preview firmware 

• ExpressVote HW2.1 Previewer ballot preview firmware 

• Event Log Service (ELS) software service monitoring user’s interactions 

with the Election Management System 

• Removable Media Service (RMS) software service supporting election 

media programming 

Note: Versions for each will be available after execution of the Trusted Build, and 

will be listed in the Test Report. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 49 of 84 

Template Rev 2015-03 

Managing technology risk 

Table 3 – ES&S EVS 6.2.0.0 Software/Firmware 

Application Version 

Electionware – Client/Server 5.0.0.0 

Event Log Service 1.6.0.0 

Removable Media Service 1.5.0.0 

ExpressVote HW2.1 Previewer 2.4.0.0 

ExpressVote HW1.0 Previewer 1.5.0.0 

DS200 2.17.0 

DS850 3.1.0.0 

DS450 3.1.0.0 

4.1.2 COTS Software/Firmware 

This section details the COTS software and firmware utilized within the ES&S EVS 

6.2.0.0 voting system. 

Table 4 – COTS Software/Firmware 

Manufacturer Application Version 

Microsoft Corporation Window 7 Professional SP-1 (64-bit) 

Microsoft Corporation Windows Server 2008 R2, SP-1 (64-bit) 

Microsoft Corporation WSUS Microsoft Windows Offline 

Update Utility 

10.9.2 

Symantec Symantec Endpoint Protection 14.0.0 (64-bit) 

Symantec 

Symantec Endpoint Protection 

Intelligent Updater (File-Based 

Protection) 

20170523-003-

v5i64.exe 

Symantec 

Symantec Endpoint Protection 

Intelligent Updater (Network-Based 

Protection) 

20170522-011-

IPS_IU_SEP.exe 

Symantec 

Symantec Endpoint Protection 

Intelligent Updater (Behavior-Based 

Protection) 

20170516-001-

SONAR_IU_SEP.exe 

Cerberus Cerberus FTP Server – Enterprise 8.0.12 (64-bit) 

Adobe Adobe Acrobat Standard XI 

Microsoft Corporation Visual C++ Redistributable 

vc_redist.x64.exe (64-

bit) 

vc_redist.x86.exe (86-

bit) 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 50 of 84 

Template Rev 2015-03 

Managing technology risk 

4.1.3 Additional Supporting Test Software 

This section outlines any and all test specific software that will be used by SLI in 

the certification campaign. 

Table 5 – Additional Supporting Test Software 

Manufacturer Application Version 

Riverbed WireShark 1.10.6 

LocMetrics LocMetrics Line Counter 2007 

SLI Compliance Module Finder 1.0 

Prestosoft ExamDiff Pro 3.0 

Scitools Understand 4.0 

4.2 Equipment 

The following equipment is required for the execution of the hardware, software, 

telecommunications, and security tests. This includes system hardware, general 

purpose data processing and communications equipment, and any test 

instrumentation required. 

4.2.1 ES&S EVS 6.2.0.0 Equipment 

The following manufacturer equipment will be used in testing: 

Table 6 – ES&S EVS 6.2.0.0 Equipment 

Hardware Model 

ExpressVote Universal Voting System HW1.0 

ExpressVote Universal Voting System HW2.1 

DS200 v1.2 

DS200 v1.3 

DS450 v1.0 

DS850 v1.0 

ExpressVote Rolling Kiosk 98-00049 

ExpressVote Voting Booth 87001 

ExpressVote ADA Table 87031 

DS200 Collapsible Ballot Box N/A 

DS200 Plastic Ballot Box 57521 

DS200 Metal Ballot Box N/A 

DS200 Tote Bin 00074 

DS450 Cart 3002 

DS850 Cart 6823 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 51 of 84 

Template Rev 2015-03 

Managing technology risk 

4.2.2 COTS Equipment 

The following COTS equipment will be used in testing: 

Table 6 – COTS Equipment 

Manufacturer Hardware Model Operating System 

Innodisk USB EDC H 2SE (1GB) DEEUH 1-

01GI72AC1SB (for 

EV HW1.0) 

N/A 

Innodisk USB EDC H 2SE (16GB) DEUH1-

16GI72AC1SB 

(for EV HW1=2.1) 

N/A 

Delkin Devices USB Embedded 2.0 

Module (16GB) 

MY16MGFSYRA000-D 

N/A 

Symbol Scanner (External) DS9208 N/A 

Zebra Technologies Scanner (Integrated) DS457-SR20009 N/A 

OKI Audit Printer Microline 420 N/A 

Dell Report Printer S2810dn N/A 

OKI Report Printer B431DN 

B431D 

N/A 

Tripp Lite Spike Cube SPIKECUBE N/A 

APC Backup power supply 

(Uninterruptible Power 

Supply) 

Back-UPS Pro 

1500 

Back-UPS RS 

1500 

N/A 

Various (EMS 

Networked or 

Standalone 

configuration) 

• Processor: Dual Core 

• RAM: 4 GB, 8 GB 

recommended 

• Hard Disk: 150 GB 

• Keyboard 

• Mouse 

• Monitor: 1280x800 

resolution 

• CD/DVD reader: 16x 

min 

• 2 USB ports: 2.0 min 

• Report Printer: 

w/printer control 

language driver 

N/A Windows 7 

Professional, SP-1 

(64-bit) 

Various (EMS 

Networked server 

configuration) 

• Processor: Dual Core 

or Quad Core 

• RAM: 4 GB, 8 GB 

Windows Server 

2008 R2, SP-1 (64-

bit) 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 52 of 84 

Template Rev 2015-03 

Managing technology risk 

Manufacturer Hardware Model Operating System 

recommended 

• Hard Disk: 150 GB or 

320 GB 

• Keyboard 

• Mouse 

• Monitor: 1280x800 

resolution 

• CD/DVD reader: 16x 

min 

• 2 USB ports: 2.0 min 

• Report Printer: 

Network printer 

w/printer control 

language driver 

• Ethernet Port 

• Back up power 

supply: 865 Watts / 

1500 VA output 

capacity 

• Network Switch: 1 GB 

throughput 

Delkin USB Flash Drive: 

512 MB, 1 GB, 2 GB, 4 

GB, 8 GB, 16 GB 

N/A N/A 

AVID Headphones 86002 N/A 

Seiko Instruments Thermal Printer LTPD-347B N/A 

NCR / Nashua Paper Roll 2320 N/A 

Delkin Compact Flash Memory 

Card: 1 GB max 

N/A N/A 

Delkin Compact Flash Memory 

Card Reader/Writer 

6381 N/A 

4.3 Test Materials 

The following test materials are required for the performance of testing including, 

as applicable, test ballot layout and generation materials, test ballot sheets, test 

ballot cards and control cards, standard and optional output data report formats, 

and any other materials used in testing. 

• Ballots and blank ballot grade paper 

• Ballot pens 

• Compact Flash memory cards 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 53 of 84 

Template Rev 2015-03 

Managing technology risk 

• USB flash drives 

• Printer paper rolls 

4.4 Deliverable Materials 

The following are documents and materials to be delivered as a part of the ES&S 

EVS 6.2.0.0 voting system. 

• System Overview 

• System Functionality Description 

• DS200 Operator’s Guide 

• DS450 Operator’s Guide 

• DS850 Operator’s Guide 

• EVS Event Log Service User’s Guide 

• Electionware Vol. I: Administrator Guide 

• Electionware Vol. II: Define User Guide 

• Electionware Vol. III: Design User Guide 

• Electionware Vol. IV: Deliver User Guide 

• Electionware Vol. V: Results User Guide 

• Electionware Vol. VI: Appendices 

• ExpressVote Operator’s Guide (HW1.0) 

• ExpressVote Operator’s Guide (HW2.1) 

• Requirements of the 2005 VVSG Trace to Vendor Testing 

• Requirements of the 2005 VVSG Trace to Technical Data Package 

• DS200 Hardware Specification 

• DS450 Hardware Specification 

• DS850 Hardware Specification 

• ExpressVote Hardware Specification HW1.0 

• ExpressVote Hardware Specification HW2.1 

• System Development Program 

• Coding Standards 

• License Agreements for Procured Software 

• DS200 – Software Design Specification 

• DS450 – Software Design Specification 

• DS850 – Software Design Specification 

• Electionware – Software Design Specification 

• Event Log Service – Software Design Specification 

• ExpressVote (HW1.0) Software Design Specification 

• ExpressVote (HW2.1) Software Design Specification 

• Electionware PostSQL Table and Field Descriptions 

• Electionware XML Diagrams 

• System Test Plan 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 54 of 84 

Template Rev 2015-03 

Managing technology risk 

• Usability Test Report: DS200 Precinct-Based Scanner and Tabulator 

• Usability Test Report: ExpressVote Universal Voting System (HW1.0) 

• Usability Test Report: ExpressVote Universal Voting System (HW2.1) 

• EMS Client Workstation Secure Setup & Configuration Guide 

• EMS Server Secure Setup & Configuration Guide 

• Standalone EMS Workstation Secure Setup & Configuration Guide 

• Voting System Security Specification 

• Security Script Description 

• Verification Procedure: DS200 Precinct Scanner and Tabulator 

• Verification Procedure: DS450 High-Throughput Scanner & Tabulator 

• Verification Procedure: DS850 High-Speed Scanner & Tabulator 

• Verification Procedure: Election Management System Workstation and 

Server 

• Verification Procedure: ExpressVote Hardware 1.0 

• Verification Procedure: ExpressVote Hardware 2.1 

• Verification Procedure: Verification PC Setup 

• Validation File List: DS200 

• Validation File List: DS450 

• Validation File List: DS850 

• Validation File List: Event Log Service 

• Validation File List: ExpressVote HW1.0 

• Validation File List: ExpressVote HW2.1 

• Validation File List: ExpressVote HW1.0 Previewer 

• Validation File List: ExpressVote HW2.1 Previewer 

• Validation File List: Electionware 

• Validation File List: Removable Media Service 

• DS200 Maintenance Manual 

• DS450 Maintenance Manual 

• DS850 Maintenance Manual 

• ExpressVote Maintenance Manual (HW1.0) 

• ExpressVote Maintenance Manual (HW2.1) 

• Personnel Deployment and Training Program 

• Configuration Management Program 

• Technical Documentation Program 

• Manufacturing Quality Assurance Program 

• Software Quality Assurance Program 

• Ballot Production Guide for EVS 

• Conformity Statement: 2005 VVSG 

• COTS Production Implementation Plan 

• EAC Application Requirements Trace 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 55 of 84 

Template Rev 2015-03 

Managing technology risk 

5 TEST SPECIFICATIONS 

The following are the specifications for testing to be conducted on ES&S EVS 

6.2.0.0 voting system. The specifications contain details on the focus of testing, 

configuration(s), and the functions to be tested. 

5.1 Requirements 

The ES&S EVS 6.2.0.0 will be tested to the approved 2005 VVSG requirements. 

All requirements within the VVSG are verified against the ES&S EVS 6.2.0.0 voting 

system unless noted otherwise in the “Excluded Requirements”, section 1.8.6. 

ES&S has not declared a DRE for the ES&S EVS 6.2.0.0 voting system. As such, 

VVSG 1.0 DRE specific requirements are not scheduled to be evaluated, as listed 

in the “Excluded Requirements” section. 

To evaluate the system test requirements, each section of the EAC 2005 VVSG 

shall be analyzed to determine the applicable tests, EAC 2005 VVSG Volume I 

sections, as well as the accompanying strategy for evaluation, are described 

below: 

• Section 2: Functional Requirements – The requirements in this section 

shall be tested during the FCA, utilizing the SLI test suites specially 

designed for the ES&S EVS 6.2.0.0 voting system. The data input during 

these tests shall be the predefined election definitions. 

• Section 3: Usability and Accessibility – The requirements in this section 

shall be tested during the Usability Test, utilizing SLI test suites developed 

to cover the applicable requirements of this section. 

• Section 4: Hardware Requirements – The requirements in this section 

shall be tested and/or overseen by SLI personnel. 

• Section 5: Software Requirements – The requirements in this section 

shall be tested during source code review, TDP review, and FCA. A 

combination of review and functional testing shall be performed to verify 

these requirements are met. The functional testing will occur if the source 

code review reveals potential areas of issue within the code that are not 

explicitly covered by the VVSG source code requirements. The functional 

test team will work with the source code review team to determine what 

testing will need to be done in order to verify no issue is resident within the 

inspected code. 

• Section 6: Telecommunication – The requirements in this section shall be 

tested during source code review, FCA, and Security Tests, utilizing SLI test 

suites custom designed for the ES&S EVS 6.2.0.0 voting system. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 56 of 84 

Template Rev 2015-03 

Managing technology risk 

• Section 7: Security Requirements – The requirements in this section shall 

be tested during source code review, FCA, and Security Tests, utilizing SLI 

test suites custom designed for the ES&S EVS 6.2.0.0 voting system. 

• Section 8: Quality Assurance (QA) Requirements – The requirements in 

this section shall be tested throughout the test campaign via various 

methods. The TDP review shall be performed on ES&S QA documentation 

to determine compliance to EAC 2005 VVSG requirements and the 

requirements stated in the ES&S QA Program document. All source code 

shall be checked to ensure that proper QA documentation has been 

completed. All equipment received for initial testing and follow-up testing 

shall be checked against ES&S documentation to ensure their QA process 

is being followed. SLI personnel will complete the requirements of EAC 

2005 VVSG Vol. 2, Section 7, Quality Assurance Testing and Section 

1.3.1.5, Focus of Vendor Documentation that requires SLI personnel to 

examine documents or conduct an external evaluation utilizing equipment, 

documents and support information provided by ES&S during the test 

campaign. SLI may also choose to interview ES&S’s QA staff for further 

evaluation. 

• Section 9: Configuration Management (CM) Requirements – The 

requirements in this section shall be tested throughout the test campaign. 

The TDP review shall be performed on the ES&S configuration 

management documentation to determine EAC 2005 VVSG compliance and 

to further determine whether ES&S is following its documented CM 

requirements. Any discrepancies shall be formally reported to ES&S and the 

EAC. SLI personnel may conduct an audit of the ES&S CM Program at the 

ES&S facility, if deemed necessary. 

5.2 Hardware Configuration and Design 

The ES&S EVS 6.2.0.0 as declared in the application for certification submitted to 

the EAC, consists of: 

• An Electionware standalone workstation with minimum requirements of 

150 GB hard disk, 8 GB of recommended memory and Dual Core 

processing speed. 

• For network configuration, a networked workstation with minimum 

requirements of 150 GB hard disk, 8 GB of recommended memory and 

Dual Core processing speed. Additionally the network configuration 

includes a server with minimum requirements of 320 GB hard disk, 4 GB of 

memory and Quad Core processing speed. 

• At the precinct level, DS200 tabulators and ExpressVote HW2.1 universal 

voting devices are employed. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 57 of 84 

Template Rev 2015-03 

Managing technology risk 

• The central count location employs scanners, in combination with 

workstations that utilizes the DS450 and DS850 software and hardware for 

tabulation of ballots. 

• The consolidation, tally and reporting process employs the workstation with 

either a direct connect or a network printer. 

5.3 ES&S EVS 6.2.0.0 Voting System Review 

This section describes the workings of the ES&S EVS 6.2.0.0 voting system and 

will assist the reader with understanding the flow of the voting system. 

5.3.1 Define 

The Define group is used to input critical data for your election. The Define 

group modules are Home, Capture, and Element Library. 

5.3.1.1 Home Module 

The Home module is where election management, template management, and 

equipment selection options take place. New elections are created within the Home 

module as well. Use the Create New Election feature to create a new election, give 

it a name, and set some basic parameters. A county must be selected to create an 

election. Enter the election information. This includes data about the jurisdiction 

and jurisdiction ID (such as county number), the election type (i.e. general, 

primary), any alternate names/IDs, and the rules profile (state-specific rules 

that apply to this election). 

Existing elections created in Electionware will appear under the Home module and 

can be reopened from here for editing and management. Additional functionality 

includes backing up, restoring, and deleting elections. 

5.3.1.2 Capture Module 

The Capture module is where the election data is entered. Items such as 

languages, audio files, precincts, districts, contests, candidates, and ballot sets are 

in this module. 

The Capture Status screen opens when the Capture module is opened. This 

screen is split into two sections. 

Election Properties displays general information about the current election, 

including the template, jurisdiction information, election date, description, type, 

management of languages, audio, parties, precincts, districts, contests, 

candidates, polling place and ballot styles. 

Capture Status displays the current status and last time/date stamp for captured 

election information, including Election Data Last Imported and Ballot Styles Last 

Generated. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 58 of 84 

Template Rev 2015-03 

Managing technology risk 

5.3.1.3 Element Library Module 

The Element Library module is where electronic elements (such as system 

translations, audio and graphics, and equipment screen definitions) are managed 

and customized. These features include both system files (instructional text and 

audio displayed/played for the user) and screen graphics. 

The Element Library module enables you to manage files and support data for 

ADA accessible voting equipment. These functions include the management of 

system audio files and translations. System audio and translations are used to 

instruct voters on audio and on-screen ballot navigation. 

The Export ExpressVote System Translations Script feature is used to export 

system prompt translations for editing. The ExpressVote can support voting in 

multiple languages. The Element Library module is used to export system 

translations in the form of a spreadsheet for editing. 

The Import ExpressVote System Translations feature is used to import the system 

translations from the spreadsheet you exported previously and incorporate them 

into your election. 

The Export ExpressVote System Audio Script feature is used to export the 

instructional system script for audio recording purposes only, for voters using the 

ExpressVote accessible equipment. 

The Import ExpressVote System Audio Recordings feature is used to import 

recorded instructional audio file sets for the ExpressVote accessible equipment. 

The Import Election Header Graphic feature imports election headers for the ES&S 

equipment. 

5.3.2 Design 

The Design group is for designing ballot and audio elements for an election. The 

Design group modules are Paper Ballot and Accessible Ballot. 

5.3.2.1 Paper Ballot Module 

The Paper Ballot module incorporates the data in your election database (created 

in the Capture module) with finished ballot layouts. Paper Ballot can produce four 

distinct ballot layout types, depending on the needs of your jurisdiction: Column 

Portrait, Grid-Landscape, Grid-Portrait, Paper Flexible Grid. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 59 of 84 

Template Rev 2015-03 

Managing technology risk 

5.3.2.2 Accessible Ballot Module 

The Accessible Ballot module is used to format and manage display options and 

ADA audio settings, and validate data before generating election files for ES&S 

accessible voting equipment. 

The Export Ballot Audio Scripts feature is used to export the ballot audio script for 

the current election, for audio recording purposes only, for voters using accessible 

equipment. 

Validate Data is used to verify the status of the election and system translations 

and audio for accessible equipment. 

The Touch Screen Ballot module is used to format machine ballot display options 

that support multiple touchscreen layout sizes. 

5.3.3 Deliver 

The Deliver group is used to configure equipment, package election data and print 

ballots on demand. The Deliver group modules are Configure, Package, and Print. 

5.3.3.1 Configure Module 

In the Configure module, you can manage election equipment and security settings 

and generate election files for your ES&S EVS 6.2.0.0 voting system components.. 

Election Properties displays general information about the current election, 

including the template, jurisdiction information, election date, description, and type. 

The DS/ExpressVote Equipment Security feature is for setting up the security 

codes for your digital scanner and ExpressVote equipment. When an election is 

loaded onto election equipment, the user must enter the correct code on the device 

to access the control screens. 

DS200 Settings is used to set up poll place options, configure ballot images and 

handling, and select report options for the DS200. 

Use ExpressVote Settings to configure vote session, audio, card printing/handling, 

and terminal properties, poll place options, and reporting options, for the 

ExpressVote. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 60 of 84 

Template Rev 2015-03 

Managing technology risk 

5.3.3.2 Package Module 

The Package module is for creating Qualification, Election, and Reporting media 

(election files saved on an ES&S portable flash drive or compact flash card). 

Qualification Media contains the election qualification code (EQC), election keys, 

and security codes required by the DS200, ExpressVote equipment, and Central 

Count tabulator. 

The DS200 election media contains the election definition required by the DS200, 

including election-related images, and folders for result collections. Election 

Qualification Media must be created before creating DS200 Election Media. 

The Central Count election media device is a flash drive containing the election 

definition required by the Central Count tabulator, including election related 

images, and folders for result collections. 

The ExpressVote election media device is a flash drive containing the election 

definition required by the ExpressVote, including election-related images, and 

folders for result collections. 

5.3.3.3 Print Module 

Use the Print module to print extra ballots on demand (BOD) for individual polling 

places. 

5.3.4 ExpressVote Universal Voting System, HW1.0 

The ExpressVote Universal Voting System, HW1.0, is a vote capture and 

marking device designed for all voters, including non-native English speakers, nonEnglish speakers, and those who need special assistance. 

ExpressVote HW1.0 uses a touch screen, an optional external barcode scanner, 

and/or assistive technology to capture a voter’s contest choices. The system 

prevents overvotes and can alert voters about undervotes. 

The ExpressVote integrated printer marks a voter’s contest selections on a 

verifiable hardcopy vote summary card in human-readable format. The vote 

summary card enables a voter to confirm that the ExpressVote HW1.0 captured 

all votes correctly, alleviating potential concerns about system accuracy and 

integrity. Marked vote summary cards can be tabulated using the ES&S DS200, 

DS450, DS850 tabulators or the ExpressVote HW2.1 if it is configured as a 

tabulator. 

Voters with visual impairments can use headphones to hear audio instructions and 

can make selections using an audio-tactile keypad with Braille legends. Voters with 

other physical requirements can make selections using a rocker switch device or a 

sip-and-puff device. Voters can use zoom and contrast display settings to make 

on-screen text more readable. Jurisdictions can include multiple language support 

for non-English speakers. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 61 of 84 

Template Rev 2015-03 

Managing technology risk 

5.3.5 ExpressVote Universal Voting System, HW2.1 

The ExpressVote Universal Voting System, HW2.1, is a vote capture and 

marking device designed for all voters, including non-native English speakers, nonEnglish speakers, and those who need special assistance. 

The election definition controls whether the ExpressVote HW2.1 functions as a 

tabulator or as a vote summary card marker only. If you use the ExpressVote as a 

marker, marked vote summary cards can be tabulated using the ES&S DS200, 

DS450, or DS850 tabulators. If you use the ExpressVote HW2.1 as a tabulator, 

once the voter has printed their vote summary card and scans it into the tabulator, 

ExpressVote HW2.1 saves a cast vote record of the vote summary card, as well 

as a digital image of the marked vote summary card, if configured to do so within 

the election. Note that the ExpressVote HW2.1 can tabulate vote summary cards 

from other ExpressVote HW2.1’s as well as from ExpressVote HW1.0 devices. 

The system tabulates all captured votes after the polling location closes. 

ExpressVote HW2.1 uses a touch screen, an optional external barcode scanner, 

and/or assistive technology to capture a voter’s contest choices. The system 

prevents overvotes and can alert voters about undervotes. 

The ExpressVote integrated printer marks a voter’s contest selections on a vote 

summary card in human-readable format. The vote summary card enables a voter 

to confirm that the ExpressVote HW2.1 captured all votes correctly. 

Voters with visual impairments can use headphones to hear audio instructions and 

can make selections using an audio-tactile keypad with Braille legends. Voters with 

other physical requirements can make selections using a rocker switch device or a 

sip-and-puff device. Voters can use zoom and contrast display settings to make 

on-screen text more readable. Jurisdictions can include multiple language support 

for non-English speakers. 

5.3.6 DS200 

The DS200 is a precinct-based ballot scanner and vote. Voters insert their ballots 

directly into the DS200 at the polling place. The DS200 scans ballots, tabulates 

votes and feeds inserted ballots into an attached, secure ballot box. 

The DS200 reads marks on both one- and two-sided ballots. The DS200 can 

discriminate between valid marks (pen) within the ballot target locations and 

extraneous marks, perforations, smudges, and folds.

The DS200 can generate paper reports produced from the unit’s internal thermal 

printer. The DS200 stores election results on a removable USB flash drive that is 

used to transfer scanner results to Electionware after the polls are closed. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 62 of 84 

Template Rev 2015-03 

Managing technology risk 

The DS200 operates in three separate modes, each of which provides access to 

different types of functions. The administration mode provides access to diagnostic 

and testing functions to calibrate and test the scanner. 

In the polls open mode, the DS200 actively scans ballots and tabulates results. 

In the polls closed mode, the DS200 prints poll reports and prepares voting results 

for transfer to election headquarters. Election officials or poll workers close the 

polls at the assigned time. 

5.3.7 DS450 

The DS450 is a high-throughput central count scanner and tabulator with ballot 

sorting capabilities. The DS450 is designed to process a wide variety of ballot 

types and lengths, including folded ballots and printed ExpressVote vote summary 

cards. 

The DS450 reads marks on both one- and two-sided ballots. The DS450 can 

discriminate between valid marks (pen marks within the ballot target locations) and 

extraneous marks, perforations, smudges, and folds.

Cast vote records, ballot images, as well as accumulated results, are saved on the 

machine’s hard drive and can be transferred from the hard drive to a USB flash 

drive, as well as can be configured to export files to a network server after each 

batch is scanned and saved. 

5.3.8 DS850 

The DS850 is a high-speed central-count scanner and tabulator with ballot sorting 

capabilities. The DS850 is designed to process a wide variety of ballot types and 

lengths, including folded ballots and printed ExpressVote vote summary cards. 

The DS850 reads marks on both one- and two-sided ballots. The DS850 can 

discriminate between valid marks (pen marks within the ballot target locations) and 

extraneous marks, perforations, smudges, and folds.

Cast vote records, ballot images, as well as accumulated results, are saved on the 

machine’s hard drive and can be transferred from the hard drive to a USB flash 

drive, as well as can be configured to export files to a network server after each 

batch is scanned and saved. 

5.3.9 Electionware Reporting Module 

After the election, the Reporting module is used to import tabulated results, 

machine logs, cast vote records, and ballot images by reading the election media 

from the ExpressVote HW1.0, ExpressVote HW2.1, DS200, DS450 and DS850, 

media; review, export, and report election results and media device-related data; 

and review/adjudicate ballot images. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 63 of 84 

Template Rev 2015-03 

Managing technology risk 

The Reporting module can display or export detailed status information about 

individual flash drives, and produce machine log reports. It can also clear loaded 

results files, in order to delete test data or to start the loading process over. 

The Ballot Review provides the adjudicator for listing scanned ballot images based 

on configurable filter criteria and adjudicating ballot exceptions. 

The module can be used to generate precinct status, machine log file, media load, 

Central Count networked results, double vote, write-ins, and manual entry status 

reports. 

5.4 Testing Strategies 

This section looks at the voting system and the individual components in order to 

determine testing strategies at the component level as well as the system level. 

5.4.1 How each Polling Place Device will be tested 

• How the ExpressVote HW1.0 will be tested 

o ExpressVote HW1.0 will be tested first as an individual component in

order to verify that all declared functionality is present and working as 

documented, including functionality such as buttons, configurable 

settings, input fields, inputs, etc, with Electionware produced media 

and data, then as an integrated piece of the voting system where it 

will accept user input instructions, prior to producing marked vote 

summary cards that mirror user intent. 

• How the ExpressVote HW2.1 will be tested 

o ExpressVote HW2.1 will be tested first as an individual component in

order to verify that all declared functionality is present and working as 

documented, including functionality such as buttons, configurable 

settings, input fields, inputs, etc, with Electionware produced media 

and data, then as an integrated piece of the voting system where it 

will accept user input instructions, prior to producing marked vote 

summary cards that mirror user intent.

• How DS200 will be tested 

o DS200 will be tested first as an individual component in order to 

verify that all declared functionality is present and working as 

documented, including functionality such as buttons, configurable 

settings, input fields, inputs, etc, then as an integrated piece of the 

voting system where it will input Electionware produced media and 

data, then accept user filled out ballots as well as vote summary 

cards, prior to producing all defined output data on the assigned USB 

memory device.
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 64 of 84 

Template Rev 2015-03 

Managing technology risk 

5.4.2 How each Application will be tested 

• How Electionware will be tested 

o Electionware will be tested first as an individual component in order 

to verify that all declared functionality is present and working as 

documented, including functionality such as buttons, configurable 

settings, input fields, inputs, etc, then as an integrated piece of the 

voting system where it will output Electionware produced media and 

data, which will feed into ExpressVote HW1.0, ExpressVote HW2.1, 

DS200, DS450, DS850 and Electionware Reporting.

• How the DS450 and DS850 will be tested 

o DS450 and DS850 will be tested first as individual components in 

order to verify that all declared functionality is present and working as 

documented, including functionality such as buttons, configurable 

settings, input fields, inputs, etc, then as an integrated piece of the 

voting system where it will input Electionware produced media and 

data, then accept user filled out ballots as well as vote summary 

cards, prior to producing all defined output medias.

• How Electionware Reporting will be tested 

o Electionware Reporting will be tested first as an individual 

component in order to verify that all declared functionality is present 

and working as documented, including functionality such as buttons, 

configurable settings, input fields, inputs, etc, then as an integrated 

piece of the voting system where it will input Electionware produced 

media and data, then accept media data from ExpressVote HW1.0, 

ExpressVote HW2.1, DS200, DS450 and DS850, prior to producing 

all defined reports and exports.

5.4.3 How different System Level Configurations will be tested 

As noted earlier, Electionware, DS450, DS850 and Electionware Reporting are 

each capable of being run as standalone instantiations or networked with a central 

database. When run as a standalone implementation, given these possible 

configurations, the following configurations will be exercised: 

• Electionware will be tested in standalone mode 

• Electionware Reporting will be tested in standalone mode 

• DS450 will be tested in standalone mode 

• DS850 will be tested in standalone mode 

• Electionware will be tested in a networked mode 

• DS450 will be tested in a networked mode with a central repository 

• DS450 will be tested in a networked mode with Electionware 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 65 of 84 

Template Rev 2015-03 

Managing technology risk 

• DS450 will be tested in a networked mode with other DS450 units 

• DS850 will be tested in a networked mode with a central repository 

• DS850 will be tested in a networked mode with Electionware 

• DS850 will be tested in a networked mode with other DS850 units 

• Electionware Reporting will be tested in a networked mode 

5.5 Software System Functions 

The ES&S EVS 6.2.0.0 voting system operations documentation has been 

reviewed in conjunction with the Supported Functionality Declaration provided by 

ES&S. Based on this review, the applicable system functions have been identified 

for testing. The following key areas of voting system functionality will be evaluated 

during test suite design. 

5.5.1 Election Definition Creation 

The Election Definition focus will target creation of contests, candidates, 

propositions, ballot formatting and instruction. All aspects of creating regional 

districts, jurisdictional parameters, grouping and displaying of associated election 

data will be analyzed and tested. In addition, the ability to support baseline election 

types, various other election types, voting variations and languages will be verified. 

5.5.2 Pre-voting Aspects 

Pre-voting aspects include pre-election preparatory, diagnostic, and election 

verification functions of a voting system. The focus will include device preparation, 

all required pre-voting tasks including the creation and handling of media for the 

purposes of installing election data onto voting devices, as well as the creation of 

physical ballot layouts and creation of any/all media used to hold/transfer election 

data, and verification of manufacturer recommended pre-voting tasks. 

5.5.3 Voting Aspects – Polling Place 

Polling place aspects include all required and additional supported voting functions, 

including HAVA compliant requirements. This area will focus on all aspects of 

election functions and capabilities at the polling place from opening of the polls 

through closing the polls. 

5.5.4 Voting Aspects – Central Count 

The focus of the central count functions is primarily the usage of scanners to scan 

large quantities of absentee ballots and passing each image to Electionware 

Reporting for interpretation of the voter’s markings on the ballot and/or vote 

summary card. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 66 of 84 

Template Rev 2015-03 

Managing technology risk 

5.5.5 Post Voting Aspects 

This area will focus on all required election post-voting functions. This includes any 

additional supported election functions performed after closing the polls, device 

auditing, transmission, and reporting aspects of the voting system. 

5.5.6 Error Messaging and Recovery 

This area will focus on the system’s ability to generate appropriate error messaging 

within each system component and the system’s ability to recover from error 

conditions in order to proceed with all election functions. 

5.5.7 Auditing 

This area will focus on device and system level auditing capabilities and will verify 

at a minimum the required audit functionally. This includes audit trail capability 

throughout the life cycle of the voting system and audit log content requirements. 

5.5.8 Security 

Overall system and device level logical and physical security aspects will be tested. 

Physical security will focus on the areas of integrity (ballot box doors, locks and 

seals) and detection (compromised ballot box doors, locks or seals). Logical 

security will focus on the areas of access controls, accountability, confidentiality, 

and integrity. These logical security areas will be applied to the OS, Database, 

Network and Application entities used by the EMS, universal voting devices and 

tabulators used by the voting system under test. 

5.6 Test Suite Design 

5.6.1 Hardware Qualitative Examination Design 

SLI will review all reports submitted by the manufacturer of previous testing 

conducted by acceptable hardware test labs, on the equipment contained in the 

ES&S EVS 6.2.0.0 voting system. The results will be compared against the 

guidelines of the VVSG 2005, Section 4 to identify any additional testing required. 

In addition, SLI will create the following test suites to focus on Maintainability, 

Accessibility, and Usability of the voting system: 

• Accessibility - Accessibility requirements for a voting system generally 

include both objective and observable requirements. In combination, the two 

types of requirements verify that the voting system components are 

accessible to as many eligible voters as possible, including those that have 

a type of challenge that creates a need for assistance of some type. The 

voting systems should be self-contained such that the individual voter is 

able to cast their vote without assistance from another party. Accessibility 

calls for the voting system to take into account vision, varying degrees of 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 67 of 84 

Template Rev 2015-03 

Managing technology risk 

vision, dexterity, mobility, aural issues, and speech and language 

proficiency. 

• Maintainability - Maintainability encompasses a range of maintenance 

actions that examine all scheduled and unscheduled events in place for 

preventing failures on all hardware devices. Testing verifies the ease with 

which maintenance actions can be performed based on the design 

characteristics of the equipment and software. Non-technical election 

workers are able to be made aware of the problem through the equipment 

and software’s ability to correctly self-diagnose problems. 

• Usability – Usability is defined as a measure of the effectiveness, 

efficiency, and satisfaction achieved by a specified set of users with a given 

product in the performance of specified tasks. In the context of voting, the 

primary user is the voter, the product is the voting system, and the task is 

the correct recording of the voter ballot selections. Testing is conducted to 

ensure voters are able to negotiate the process effectively, efficiently and 

comfortably according to the requirements dictated.

5.6.2 Hardware Environmental Test Design 

Hardware environmental testing is performed to verify conformance to Vol. 1, 

Section 4 of the VVSG 2005. Testing will be accomplished through a combination 

of testing performed by SLI and testing performed by subcontractor labs as listed 

in “Table 2 – Labs Performing Hardware Testing”. Specific hardware test plans 

from the subcontractor labs are included in Attachments B & C. 

5.6.3 Software Module Test Design and Data 

Incorporating the manufacturer’s software specifications, SLI will validate that all 

software/firmware components of the system adhere to expected flow control 

parameters and specifications for data input and output. 

5.6.4 Software Functional Test Design and Data 

SLI will prepare functional test modules using the operator/user procedures 

contained within ES&S’s TDP. Functionality provided by ES&S EVS 6.2.0.0 voting 

system is exercised in order to verify that each functional component performs as 

expected. Accept/reject criteria are based on requirements of the VVSG and the 

system specification documents provided within the TDP. As many of the individual 

functional components rely on preceding functionality within the system, SLI 

incorporates system level suites that employ modules that exercise the individual 

functional components of the system. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 68 of 84 

Template Rev 2015-03 

Managing technology risk 

5.6.5 System-level Test Design 

Testing of the voting system involves exercising the specific functions of each 

component of a voting system as well as the entire voting system. Testing will 

focus on the functionality of an election management system, the polling place 

devices, and devices required for communications and data loading, then will focus 

on functionality of the integrated voting system. 

There are various types of testing. Table 8 - Types of Testing provides the 

descriptions of these kinds of tests and their associated benefit. 

Table 8 - Types of Testing 

Type of Testing Description Benefit 

Nominal Conditions Testing all nominal functional 

capabilities of all components of the 

voting system. 

Nominal conditions testing ensures that 

the voting system is tested against all 

elements. 

Injection Attack This testing is primarily for verifying the 

security of databases. An injection 

attack consists of insertion or 

"injection" of a SQL query via the input 

data from the client to the application. 

A successful SQL injection exploit can 

read sensitive data from the database, 

modify database data 

(Insert/Update/Delete), execute 

administration operations on the 

database, recover the content of a 

given file existing on the DBMS file 

system and, in some cases, issue 

commands to the operating system. 

Injection Attack testing hardens the 

database against unauthorized access 

and manipulation of data. 

Data Driven This is testing that is based on the 

data that is used throughout the voting 

system. For example, if ‘x’ is input in a 

field, it would branch to a different part 

of the application than if ‘y’ was 

entered. The goal is to ensure that all 

branches get tested. 

The tests verify that each unique data 

element can be accessed and used 

according to the voting system’s user 

documentation. These test cases are 

based on the election data or vote data 

used and not the functional flow or work 

flow of the voting system. 

Usability The purpose of UI Testing is to test all 

of the screen and data elements that 

exist on each and every screen. SLI 

will verify responses to input, text 

syntax, error message content, and 

audit message input. 

These tests verify every action will work 

that a user can perform on a screen. 

These tests will also verify that any 

screen or data element will not take the 

user by surprise. 

Error Messaging and 

Recovery 

Exercise system’s ability to recover 

from hardware, software, and data 

errors. 

Ensures that the system is able to 

successfully recover should there be a 

system or data error. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 69 of 84 

Template Rev 2015-03 

Managing technology risk 

End-to-End This is system testing in a true enduser environment following all preelection day, election day, and postelection day voting rules and 

processes. 

This is used to demonstrate that a system 

can be used to perform its job following 

the exact set of processes and steps that 

would be used by the target customer or 

end-user. 

Regression Testing that validates that existing 

functionality is unchanged with the 

introduction of new functionality and 

correction of defects. 

This is used to verify that issues are 

corrected, and that no know issues are 

been introduced or uncovered. 

Formal Execution Final verification of the voting system 

in a true end-user environment, 

following all pre-election day, election 

day, and post-election day voting rules 

and processes. 

Testing “for the record” that includes all 

aspects of the voting system, verifying 

that all VVSG requirements are met. 

Volume Test Testing the voting system’s response 

to conditions that range from 

processing more than the expected 

number of ballots/voters per precinct to 

processing more than the expected 

number of precincts to any other 

similar volume conditions. 

Determine if there are limits to the voting 

system’s ability to operate under 

conditions that tend to overload the 

system’s capacity to process, store, and 

report data. 

Stress Tests Testing the voting system’s responses 

to transient overload conditions by 

subjecting polling place devices to 

ballot processing at high volume rates. 

Evaluates the voting system and 

software’s response to hardwaregenerated interrupts and wait states. 

Accessibility Test Exercises system capabilities of

features for voters with disabilities. 

Validates that the voting system is tested 

against all applicable ADA and HAVA 

requirements for voters with disabilities. 

Performance Tests Tests accuracy, processing rate, ballot 

format, handling capability and other 

performance attributes specified by 

ES&S. 

Performance testing ensures that the 

voting system meets all performance 

elements. 

5.6.6 Software Functional Test Case Design 

As described in the “Testing of the System” section above, each individual 

component of the voting system will be reviewed independently, such that all 

functionality present in the component is verified to work as documented, and that 

all functionality is appropriately documented. For the components that are able to 

be networked, this will be reviewed as well. This approach corresponds to the 

following functional testing to be done. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 70 of 84 

Template Rev 2015-03 

Managing technology risk 

5.6.6.1 Electionware – Standalone workstation 

All functionality present in Electionware, including those covered in the “System 

Review” and “Supported Functionality” sections above, is verified to work as 

documented, and that all functionality is appropriately documented. 

This test covers Electionware. 

5.6.6.2 Electionware – Client/Server configuration 

Testing of the client/server configuration, for Electionware, will be the focus of this 

testing, such that reliability of data consistency is verified. 

This test covers Electionware, in a server and multi-workstation configuration. 

5.6.6.3 DS450 and DS850 – Standalone workstation 

All functionality present in DS450 and DS850, including that covered in “System 

Review” and “Supported Functionality” sections above, is verified to work as 

documented, and that all functionality is appropriately documented. 

This test covers DS450 and DS850. 

5.6.6.4 DS450 and DS850 – Client/Server configuration 

Testing of the client/server configuration, for DS450 and DS850, will be the focus 

of this testing, such that reliability of data consistency is verified. 

This test covers DS450 and DS850, in a server and multi-workstation 

configuration. 

5.6.6.5 Electionware Reporting – Standalone workstation 

All functionality present in Electionware Reporting, including that covered in 

“System Review” and “Supported Functionality” sections above, is verified to work 

as documented, and that all functionality is appropriately documented. 

This test covers Electionware Reporting. 

5.6.6.6 Reporting – Client/Server configuration 

Testing of the client/server configuration, for Electionware Reporting, will be the 

focus of this testing, such that reliability of data consistency is verified. 

This test covers Electionware Reporting, in a server and multi-workstation 

configuration. 

5.6.6.7 DS200 

All functionality present in DS200, including that covered in the “System Review” 

and “Supported Functionality sections above, is verified to work as documented, 

and that all functionality is appropriately documented. 

This test covers DS200. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 71 of 84 

Template Rev 2015-03 

Managing technology risk 

5.6.6.8 ExpressVote HW1.0 

All functionality present in ExpressVote HW1.0, including that covered in the 

“System Review” and “Supported Functionality sections above, is verified to work 

as documented, and that all functionality is appropriately documented. 

This test covers ExpressVote HW1.0.

5.6.6.9 ExpressVote HW2.1 

All functionality present in ExpressVote HW2.1, including that covered in the 

“System Review” and “Supported Functionality sections above, is verified to work 

as documented, and that all functionality is appropriately documented. 

This test covers ExpressVote HW2.1. 

5.6.7 System-level Test Suite Design 

System level test suites will include the following: 

• Election Validations - Election suites are created to replicate each type of 

election that can be implemented by the jurisdiction. Within the election 

types, pertinent voting variations that are applicable to that type of election 

will be validated and verified. Each suite will have a particular focus in order 

to test the voting system’s implementation of a given requirement or set of 

requirements. Each test suite below is explained in terms of the components 

of the voting system that will be addressed. 

o GenVariation1

Additional definition is added, with a focus on validating N of M 

voting, Partisan offices, Non-Partisan Offices, Straight Party Voting, 

Ballot Rotations, Ballot Formatting, precincts and districts, and Tally 

and Reporting functionality. 

This test covers Electionware, ExpressVote HW1.0, ExpressVote 

HW2.1, DS200, DS450 and DS850 as well as Electionware 

Reporting. Vote counts will be accumulated from DS450 and 

DS850, ExpressVote HW2.1 and DS200. 

o GenVariation2

Additional definition is added, with a focus on validating Recalls, 

Cross Party Endorsement, and Write-Ins. 

This test covers Electionware, ExpressVote HW1.0, ExpressVote 

HW2.1, DS200, DS450 and DS850 as well as Electionware 

Reporting. Vote counts will be accumulated from DS450 and DS850 

ExpressVote HW2.1 and DS200. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 72 of 84 

Template Rev 2015-03 

Managing technology risk 

o GenVariation3

Additional definition is added, with a focus on validating HAVA 

concerns (for example ATI usage, paddles, sip and puff, audio, visual 

adjustments), as well as items such as pre-vote capabilities. 

This test covers Electionware, ExpressVote HW1.0, ExpressVote 

HW2.1, DS200, DS450 and DS850 as well as Electionware 

Reporting. Vote counts will be accumulated from DS450 and 

DS850, ExpressVote HW2.1 and DS200. 

o PriClosed

This suite builds on the PriOpen Test, taking the election definition 

and modifying it to conform to a Closed Primary election. 

This test covers Electionware, ExpressVote HW1.0, ExpressVote 

HW2.1, DS200, DS450 and DS850 as well as Electionware 

Reporting. Vote counts will be accumulated from DS450 and 

DS850, ExpressVote HW2.1 and DS200. 

o PriOpen

This suite builds on the PriClosed Test, taking the election definition 

and modifying it to conform to an Open Primary election. 

This test covers Electionware, ExpressVote HW1.0, ExpressVote 

HW2.1, DS200, DS450 and DS850 as well as Electionware 

Reporting. Vote counts will be accumulated from DS450 and 

DS850, ExpressVote HW2.1 and DS200. 

• Error Recovery - The VSTL will focus on Error Messaging and Recovery in 

key areas of the system identified from researching previous testing and 

voting system documentation to help identify potential failure points. Voting 

systems can be subject to various conditions, and when the system exceeds 

limitations, errors are typically found. The VSTL leverages its election 

experience and voting system knowledge to test the Error Messaging and 

Recovery of a voting system that has incurred errors caused by stressing 

the system. Testing of Error messaging will focus on the appropriate error 

messages being generated in response to a specific error and content of the 

message. The testing of the voting system Error Recovery capability is 

incorporated into Stress testing in order to leverage the necessary range of 

performance impacts needed to generate system errors and force recovery. 

This test covers Electionware, ExpressVote HW1.0, ExpressVote HW2.1, 

DS200, DS450 and DS850 as well as Electionware Reporting. 

• Audit Content Validations - Audit records are used to track what system 

functions have been executed, what data has been modified, as well as by 

who and when. Additionally, audit record data content can be a key factor in 

identifying system anomalies and provide assistance in troubleshooting 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 73 of 84 

Template Rev 2015-03 

Managing technology risk 

system errors. In tandem with the System Audit Validation, analysis of a 

voting system is performed to determine strategic points of the system that 

require auditing, along with the content needed to accurately depict the 

machinations of the system for the given situation. Tests are incorporated 

either into Election Validation suites or specific Audit Validation suites, as 

needed, such that all related requirements are explicitly validated. 

This test covers Electionware, ExpressVote HW1.0, ExpressVote HW2.1, 

DS200, DS450 and DS850 as well as Electionware Reporting. 

• System Audit – Election audit trails provide the supporting documentation 

for verifying the accuracy of reported election results. They present a 

concrete, indestructible archival record of all system activity related to the 

vote tally, and are essential for public confidence in the accuracy of the tally, 

for recounts, and for evidence in the event of criminal or civil litigation. This 

testing focuses on validating system’s ability to provide audit capability 

throughout the entire voting system, including availability, generation, 

integrity, and accuracy of the system’s audit capability to ensure it meets the 

necessary requirements. Negative testing will be utilized to force the system 

into conditions that will trigger errors and verify the voting system captures 

those conditions. 

This test covers Electionware, ExpressVote HW1.0, ExpressVote HW2.1, 

DS200, DS450 and DS850 as well as Electionware Reporting. 

• Security Validations - Test suites are created to validate and verify various 

aspects of system security at both the component level and the system 

level, as applicable. Each suite will have a particular focus to it, such that 

when executed successfully, a given requirement, or set of requirements, 

will be determined to have been successfully implemented by the voting 

system. Topics of Security review will include: 

 Access Control 

Access control testing verifies procedures and system capabilities 

that detect or limit access to system components in order to guard 

against loss of system integrity, availability, confidentiality, and 

accountability. This testing verifies that system resources such as 

data files, application programs and computer-related facilities 

and equipment are protected against unauthorized operation, 

modification, disclosure, loss or impairment. Unauthorized 

operations include modification of compiled or interpreted code, 

run-time alteration of flow control logic or of data, and abstraction 

of raw or processed voting data in any form other than a standard 

output report by an authorized operator. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 74 of 84 

Template Rev 2015-03 

Managing technology risk 

This test covers Electionware, ExpressVote HW1.0, 

ExpressVote HW2.1, DS200, DS450 and DS850 as well as 

Electionware Reporting. 

 Software Security 

Software security testing will be conducted to verify the installation 

procedures and ongoing foreign software detection mitigation 

abilities of the voting system in order to protect against the 

modification of the software and/or the insertion of malicious 

software during the installation and during ongoing operations. 

This test covers Electionware, ExpressVote HW1.0, 

ExpressVote HW2.1, DS200, DS450 and DS850 as well as 

Electionware Reporting. 

 Physical Security Measures. 

Physical security testing verifies monitoring and control of the 

environment of the work place and computing facilities. It also 

verifies monitor and control access to and from such facilities. For 

example: doors, locks, cameras, barricades, fencing, cable locks, 

etc., can be utilized for implementation of physical security. 

Separating the network and work place into functional areas are 

also physical controls. Some portions of physical security are 

functional while other portions are procedural. Functional portions 

will be tested as appropriate while procedural portions will be 

verified to be documented as called out by the VVSG. 

This test covers Electionware, ExpressVote HW1.0, 

ExpressVote HW2.1, DS200, DS450 and DS850 as well as 

Electionware Reporting. 

 Telecommunications and Data Transmission 

Telecommunications and Data Transmission testing verifies that 

voting systems which use electrical or optical transmission of data 

shall ensure the receipt of valid vote records is verified at the 

receiving station, such that data integrity remains intact. This 

covers preparation of the system for an election, the execution of 

an election, and the preservation of the system data and audit 

trails during and following an election. 

This test covers Electionware, DS450 and DS850 as well as 

Electionware Reporting. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 75 of 84 

Template Rev 2015-03 

Managing technology risk 

• Language – Testing is conducted to ensure the voting system is capable of 

presenting the ballot, ballot selections, review screens and instructions in 

the required languages. The system’s ability to handle the prescribed 

foreign languages that have been declared to be supported, English, 

Spanish, Chinese, Korean, Japanese, Hindi, Bengali, Vietnamese, Tagalog, 

Creole, Russian, and French will be validated. 

This test covers Electionware, ExpressVote HW1.0, ExpressVote HW2.1, 

DS200, DS450 and DS850 as well as Electionware Reporting. Vote 

counts will be accumulated from DS450 and DS850, ExpressVote HW2.1

and DS200. 

• Data Retention/HW Integrity – Testing integrity requirements ensure the 

physical stability and function of the vote recording and counting processes, 

such that the system is not prone to a single point of failure that would 

prevent voting at a polling place. Testing will also verify prevention of failure 

of data input or storage, in terms of data retention, as well as confirming that 

appropriate audit records are maintained without modification. ES&S will 

also provide an attestation that all data is able to be maintained for a 

minimum of 22 months. 

This test covers Electionware, ExpressVote HW1.0ExpressVote HW2.1, 

DS200, DS450 and DS850 as well as Electionware Reporting. Vote 

counts will be accumulated from DS450 and DS850, ExpressVote HW2.1

and DS200. 

• Accuracy – Testing the ability of the system to capture, record, store, 

consolidate and report the specific selections and absence of selections, 

made by the voter for each ballot position without error. Required accuracy 

is defined in terms of an error rate that for testing purposes represents the 

maximum number of errors allowed while processing a specified volume of 

data. 

Accuracy testing is conducted at both the device level and the system level. 

Each device is subjected to scrutiny that will verify that the requirements for 

accuracy are met. Additionally, the system will be reviewed and exercised to 

validate that the accumulation, tallying and reporting mechanisms at the 

system level are able to accurately perform their functions. 

The ExpressVote HW1.0 and ExpressVote HW2.1 will both be utilized to 

mark and print vote summary cards, which will then be scanned into an 

ExpressVote HW2.1 tabulator, as well as DS200, DS450 and DS850

devices. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 76 of 84 

Template Rev 2015-03 

Managing technology risk 

There are three potential stages to the accuracy test: 

o Go/no go, where after approximately 26,000 ballot positions, if any 

errors are found the test fails, 

o If no errors are found the test continues to the 1.55 million ballot 

position mark, where if 1 error is found the test may progress to 

approximately the 3.1 million ballot position mark where if no more 

errors are encountered, the test passes. 

o If more than 1 error is found at the 1.55 million ballot position mark 

the test fails. 

This test covers ExpressVote HW1.0, ExpressVote HW2.1, DS200, 

DS450 and DS850 as well as Electionware Reporting. Vote counts will be 

accumulated from DS450 and DS850, ExpressVote HW2.1 and DS200. 

• Mark Sensitivity - The purpose of Ballot Mark Sensitivity testing is to 

determine that the system under test is able to accurately determine when a 

mark has been made within a ballot marking position. For this test, various 

marks are made within the ballot marking positions, using EVS supported 

colors of ink (black), as well as non-supported, in order to determine how 

the system handles unexpected colors. 

Marks include fully filled boxes, left and right oriented slashes, “X” markings, 

check marks, horizontal single line marks, and circles of various sizes. 

Marks also include vertical lines within the marking position that fill 

approximately ten percent of the designated space. Small dots down to 

approximately five percent of the ballot marking position are also included. 

This test covers, DS200, DS450, DS850, as well as the adjudication 

algorithm within the Electionware Results application. Vote counts will be 

accumulated from DS450 and DS850, ExpressVote HW2.1 and DS200. 

• Volume – Testing a system’s response when subjected to large volumes of 

data, “more than the expected”, as called out in the standards. Volume 

testing is typically considered a type of non-functional testing. However, as a 

voting system’s primary function is to accumulate, tally, and pass a volume 

of data (votes) the VSTL approaches volume testing as a functional test. 

Experience has shown that large amounts of data can slow a system, or 

even cause failures and loss of data due to architectural limitations. Utilizing 

the VSTL’s experience with voting systems the testing will focus on not only 

passing large amounts of data but how the system operates and handles 

the data in key areas of functionality within the voting system. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 77 of 84 

Template Rev 2015-03 

Managing technology risk 

This test covers ExpressVote HW1.0, ExpressVote HW2.1, DS200, 

DS450 and DS850 as well as Electionware Reporting. Vote counts will be 

accumulated from DS450 and DS850, ExpressVote HW2.1 and DS200. 

• Stress - Testing a system’s “response to transient overload conditions.” 

Experience has shown that when passing a dataset through a system that 

eclipses the system architectural limitations, failures can occur and result in 

the loss of critical data. Utilizing the VSTL’s experience with voting systems, 

the testing will focus on the system’s ability to operate after the limitations 

have been exceeded and if failures occur, how the data is maintained or 

recovered in key areas of functionality within the voting system. 

This test covers ExpressVote HW1.0, ExpressVote HW2.1, DS200, 

DS450 and DS850 as well as Electionware Reporting. Vote counts will be 

accumulated from DS450 and DS850, ExpressVote HW2.1 and DS200. 

5.7 Standard VSTL Test Methods and Uncertainty of Test 

Data Measurement 

This test campaign utilizes Standard VSTL test methods and election specific type 

test data only. 

5.8 EAC Interpretations 

The test engagement described in this Certification Test Plan utilizes only standard 

VSTL test methods that conform to the EAC Testing and Certification Program 

Manual and the identified voting system standard. Additional EAC interpretations 

that affect the test plan and test methodology are as follows: 

• EAC Decision on RFI 2007-01 

• EAC Decision on RFI 2007-03 

• EAC Decision on RFI 2007-04 

• EAC Decision on RFI 2007-05 

• EAC Decision on RFI 2007-06 

• EAC Decision on RFI 2008-01 

• EAC Decision on RFI 2008-02 

• EAC Decision on RFI 2008-03 

• EAC Decision on RFI 2008-04 

• EAC Decision on RFI 2008-07 

• EAC Decision on RFI 2008-08 

• EAC Decision on RFI 2008-09 

• EAC Decision on RFI 2008-10 

• EAC Decision on RFI 2008-12 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 78 of 84 

Template Rev 2015-03 

Managing technology risk 

• EAC Decision on RFI 2009-02 

• EAC Decision on RFI 2009-03 

• EAC Decision on RFI 2009-04 

• EAC Decision on RFI 2009-05 

• EAC Decision on RFI 2009-06 

• EAC Decision on RFI 2010-01 

• EAC Decision on RFI 2010-02 

• EAC Decision on RFI 2010-03 

• EAC Decision on RFI 2010-04 

• EAC Decision on RFI 2010-06 

• EAC Decision on RFI 2010-07 

• EAC Decision on RFI 2010-08 

• EAC Decision on RFI 2012-01 

• EAC Decision on RFI 2012-03 

• EAC Decision on RFI 2012-04 

• EAC Decision on RFI 2012-06 

• EAC Decision on RFI 2013-01 

• EAC Decision on RFI 2013-02 

• EAC Decision on RFI 2013-03

• EAC Decision on RFI 2013-04

• EAC Decision on RFI 2015-05

RFI’s not implemented in this test campaign include: 

• EAC Decision on RFI 2009-01, VVPAT accessibility 

• EAC Decision on RFI 2010-05, modifications of a certified system 

• EAC Decision on RFI 2012-02, transmission of official/unofficial results 

• EAC Decision on RFI 2012-05, public telecommunications and cryptography 

EAC notices of clarification affecting the certification project are implemented as 

noted below. 

• NOC 2007-005 VSTL responsibilities and Third Party Lab Oversight 

• NOC 2008-003 EAC Conformance Testing Requirements 

• NOC 2009-001 Test Plan NOC 

• NOC 2009-002 Testing Independence 

• NOC 2009-004 Test Report 

• NOC 2012-02 Clarification of System ID Tool Functionality FINAL 4.19.12 

• NOC 2013-01 Discrepancy Listing in Test Report-FINAL-10 17 13 

• NOC 2015-01 Test Readiness Review (TRR) applies to both new certs and 

modifications 

• NOC 2016-01 Test Readiness Review 

• NOC 2016-02 Trusted Build 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 79 of 84 

Template Rev 2015-03 

Managing technology risk 

• NOC 2016-03 Test Case Upload 

EAC notices of clarification, noted below, not affecting the certification project are 

not implemented: 

• NOC 2007-001 Timely Submission of Application for Certification 

• NOC 2007-002 VSTL work outside of voting system testing 

• NOC 2007-003 State and Federal Testing 

• NOC 2007-004 Manufacturing facilities 

• NOC 2008-001 Use of Previous Non-core HW Testing 

• NOC 2008-002 EAC Mark of Certification 

• NOC 2009-003 de minimis change TDP 

• NOC 2009-005 Modification Test Plan 

• NOC 2011-01 de minimis data change final 

• NOC 2012-01 COTS Computer Equivalency for de minimis Change 

• NOC 2013-02 Detailed Description of Changes for Modifications 

• NOC 2014-01 Technology Testing Agreement 

5.9 Security Functions 

5.9.1 Security Test 

The Security Test Suites are SLI’s tests for verifying that a voting system will 

correspond to requirements in VVSG Volume 1, Section 7. It incorporates systems 

security provisions, unauthorized access, deletion or modification of data, audit trail 

data, and modification or elimination of security mechanisms. The vendor 

documentation will be reviewed to ensure sufficient detail is present to operate the 

voting system in a secured manner. Where the vendor statements assert the 

voting system is secured via mechanisms and seals, procedures will test the 

presence and effectiveness of such controls. 

In its security testing SLI identifies the specific threats that are tested for and the 

associated risk if a flaw or exception is identified in a voting system. SLI maintains 

a “Known Vulnerabilities Database” that is updated minimally once a year as well 

as whenever an issue is uncovered. The tests used by SLI are designed to ensure 

that the voting system meets or exceeds the requirements in the VVSG. Any 

instance where an anomaly or possible security flaw is identified, the potential risk 

is reported and evaluated. 

Security testing includes testing each individual component of the system and the 

system as a whole. As such, Electionware, ExpressVote HW1.0, ExpressVote 

HW2.1, DS200, DS450 and DS850 as well as Electionware Reporting will be 

subjected to review, as will the system as a whole and its interactions between 

components. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 80 of 84 

Template Rev 2015-03 

Managing technology risk 

6 TEST DATA 

Test data for the ES&S EVS 6.2.0.0 voting system is compiled such that all 

functionality declared will be tested to determine conformance to the standards.

6.1 Data Recording 

SLI has evaluated the system functionality, as described by manufacturer technical 

documentation, as well as requirements as listed in the EAC 2005 VVSG, and 

made determinations as to expected results of all data inputs into the ES&S EVS 

6.2.0.0 voting system. This includes: 

• Election type 

• Precincts of all types 

• Districts 

• Offices 

• Contests 

• Candidates 

• Parties 

• Devices used 

• Voting variations employed 

• Issues/Referendums 

• Votes cast for each candidate/issue/referendum 

• Vote consolidation data from one device/level to the next 

The data is contained in one master data record, including each input and each 

expected output. This data is incorporated into the appropriate test suite, 

populating test modules with exact expected data for the function being tested. 

Testing information is recorded in the test suites, as well as in test notebooks, 

which are utilized according to SLI’s standard lab procedure SLP-VC-30 - Test 

Notebooks. 

6.2 Test Data Criteria 

SLI has evaluated the system functionality as described by manufacturer technical 

documentation, as well as requirements as listed in the EAC 2005 VVSG, and 

made determinations as to expected output of all data inputs into the ES&S EVS 

6.2.0.0 voting system. A data matrix will be recorded into one master data record 

that couples data inputs to their expected output, as determined above. The 

system’s execution shall be measured against the expected results. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 81 of 84 

Template Rev 2015-03 

Managing technology risk 

6.3 Test Data Reduction 

SLI processes the test data by manually recording input data into each pertinent 

module within the Test Suites as well as the exact output that is generated, e.g., 

the vote counts when the data is consolidated. 

7 TEST PROCEDURE AND CONDITIONS 

This section describes the test conditions and procedures for execution of test 

suites. If a particular sequence is mandatory for the execution of suites, a rationale 

will be given. Additionally, this section is used to describe procedures for setting up 

equipment that will be utilized in the execution of the test suites. 

7.1 Facility Requirements 

Testing will be performed on site at SLI in Colorado. 

Four secure labs are available with appropriate power supply and space to 

accommodate the various configurations defined within this test plan. 

Temperature/humidity gauges will be employed in order to determine the 

appropriate conditions exist during testing. 

Unless otherwise specified herein, all remaining tests, including system level 

functional testing, shall be performed at standard ambient conditions: 

• Temperature: 25°C ± 10°C (77°F ± 18°F) 

• Relative Humidity: 20 to 90% 

• Atmospheric Pressure: Local Site Pressure 

All TDP and test documentation is stored on site at SLI’s facility in a secure project 

directory on SLI’s secure Voting server. 

Environmental hardware testing for hardware components of the ES&S EVS 

6.2.0.0 voting system will be performed at either NVLAP or A2LA accredited testing 

laboratories or at laboratories audited by SLI to NVLAP Handbook 150-22 

requirements. 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 82 of 84 

Template Rev 2015-03 

Managing technology risk 

7.2 Test Setup 

Configurations of ES&S EVS 6.2.0.0 will be deployed that conform to each specific 

test suite’s needs. Some configurations will consist of standalone implementations, 

while other configurations will utilize networked implementations of various 

applications, such as Electionware, DS450 and DS850. In all instances, ES&S

EVS 6.2.0.0 documentation will be followed in the setup of the configurations. 

Successful completion of operational status checks will indicate that the system is 

ready for test execution. 

7.3 Test Sequence 

While there is no required sequence for performing voting system certification 

testing and audits, there are prerequisite tasks for some testing. Any needed 

prerequisites are contained within the suite for that test. 

8 TEST OPERATIONS PROCEDURES 

An inventory has been performed to verify the voting equipment received contains 

hardware and software elements as defined in the TDP prior to commencement of 

testing. 

Throughout the testing effort, test suites and modules will be marked as follows: 

• Accept – Test is accepted as successful.

• Reject – Test is rejected as unsuccessful. 

• NT – Not Testable is used for test modules that cannot be completed. For 

example, if failure of one test module precludes attempting subsequent test 

modules, the latter will be marked as NT. 

Test results Reject and NT will include comments by the VTS explaining the 

reason for the result. 

Issues encountered during review and testing will be documented in the 

Discrepancy Report as well as reported to the EAC in an authorized manner. 

Issues that do not conform to the requirements of the VVSG, version 2005 will be 

marked as Documentation Discrepancies or Functional Discrepancies (a 

discrepancy occurs when the software does not meet defined software 

requirements or specifications). SLI employs a system of checks such that any 

issue uncovered during testing is first designated as an “anomaly”. The anomaly is 

then reviewed and the cause is determined as to be either caused by a flaw in the 

test or in the voting system itself. If the issue is determined to be a flaw in the test, 

the test will be re-written, re-validated and then formally re-run. If the issue is 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 83 of 84 

Template Rev 2015-03 

Managing technology risk 

determined to be a flaw in the voting system, then a discrepancy is opened against 

the system. While test suites and test modules undergo a validation phase prior to 

formal execution, last minute code changes can possibly change the behavior from 

what the test module defines as expected, which if this is the case, the review 

process employed during the anomaly phase will reveal, thus reducing the chance 

of a false positive in terms of an unfounded discrepancy being written against the 

voting system. 

Issues that are encountered during testing or documentation review, but are not 

addressed by the applicable standard will be added to the Discrepancy Report and 

noted as Informational. The vendor has the option whether to address 

Informational issues. 

9 APPROVAL SIGNATURES 

SLI: 

 

Traci Mapps 

VSTL Director 

September 8h, 2017 
ES&S 

EVS 6.2.0.0 

EAC Certification Test Plan 

EAC Certification Test Plan V1.1 

Document No. ESS-7005-CTP-01

Page 84 of 84 

Template Rev 2015-03 

Managing technology risk 

10 Appendix A – Ancillary Products 

Ancillary systems represent products and utilities that are not part of the EAC 

certified configuration. 

Ancillary systems include: 

• Ballot Production 

o Ballot on Demand (BOD), is a process that allows you to print ad hoc ballots. 

o Balotar, is a product that allows you to print ad hoc ballots. 

• intElect, (Ballot Online) – ExpressPass 

o Ballot Online is a system that allows a user to access online and make sample 

ballot selections on any device connected to the Internet. When finished, the output 

from this system is the ExpressPass – a selection summary with scannable QR 

code that the user can either print or save in an electronic format on their mobile 

device.

• Electronic Pollbook 

o ExpressPoll-5000 electronic pollbook stores registered voter information for 

precincts, districts, or entire jurisdictions.

• ExpressLink System 

o ExpressLink, is a standalone application that interfaces with voter registration 

(electronic Pollbook) systems and the ExpressVote Activation Card Printer to print 

the ballot activation code on an ExpressVote activation card. 

o ExpressVote Activation Card Printer, a thermal, on demand printer used to 

print the ballot activation code on the ExpressVote activation card. 

End of Certification Test Plan