ES&S EVS 6000 2005 VVSG 

ENVIR. HW Test Plan v1.0.doc 

Document Duplication Prohibited Page 1 of 17 

Election Systems & Software 

EVS6000 

Environmental Test Plan for compliance with 

2005 Voluntary Voting System Guidelines 

(VVSG) 

Prepared by 

4720 Independence St. 

Wheat Ridge, CO 80033 

303-422-1566 

www.SLICompliance.com

Accredited by the Election Assistance 

Commission (EAC) for Selected Voting 

System Test Methods or Services 
ES&S EVS 6000 2005 VVSG 

ENVIR. HW Test Plan v1.0.doc 

Document Duplication Prohibited Page 2 of 17 

Copyright  2017 by SLI ComplianceSM

. 

All products and company names are used for identification purposes only and may be trademarks of 

their respective owners. 

Revision History 

Date Release Author Revision Summary 

June 27, 2017 1.0 Darrick Forester Initial Release 

Disclaimer 

The information reported herein must not be used by the client to claim product certification, approval, 

or endorsement by NVLAP, NIST, or any agency of the Federal Government. 

Trademarks 

• SLI is a registered trademark of SLI Compliance, a Division of Gaming Laboratories 

International, LLC 

• All products and company names are used for identification purposes only and may be 

trademarks of their respective owners 
ES&S EVS 6000 2005 VVSG 

ENVIR. HW Test Plan v1.0.doc 

Document Duplication Prohibited Page 3 of 17 

TABLE OF CONTENTS 

1. INTRODUCTION .......................................................................................................................................................... 4

1.1 OVERVIEW ................................................................................................................................................................ 4

1.2 QUALIFICATIONS ....................................................................................................................................................... 4

1.3 CLIENT ...................................................................................................................................................................... 4

1.4 COMPANY INFORMATION .......................................................................................................................................... 4

1.5 TEST FACILITY LOCATION......................................................................................................................................... 4

1.6 LONGMONT, CO 80504 ............................................................................................................................................. 4

1.7 REFERENCE DOCUMENTS .......................................................................................................................................... 4

2 TESTING SUMMARY .................................................................................................................................................. 5

3 PRODUCT DESCRIPTION.......................................................................................................................................... 6

3.1 UNIT UNDER TEST .................................................................................................................................................... 6

3.2 PRODUCT INFORMATION ........................................................................................................................................... 6

3.3 SUPPORT EQUIPMENT (SE) ........................................................................................................................................ 6

3.4 POWER SUPPLIES ....................................................................................................................................................... 7

3.5 ACCESSORIES ............................................................................................................................................................ 7

3.6 SOFTWARE ................................................................................................................................................................ 7

3.7 ENGINEERING CHANGES ........................................................................................................................................... 7

4 TEST REQUIREMENTS .............................................................................................................................................. 8

4.1 TEST PROCEDURES .................................................................................................................................................... 8

4.1.1 Operating Environment .................................................................................................................................... 8

4.1.2 Environmental Control - Transit and Storage .................................................................................................. 8

4.2 DESIGN, CONSTRUCTION AND MAINTENANCE .......................................................................................................... 8

4.2.1 Physical Attributes............................................................................................................................................ 8

4.2.2 Additional Attributes ........................................................................................................................................ 8

4.3 NON-OPERATING ENVIRONMENTAL TESTS ............................................................................................................... 9

4.3.1 Data Retention Requirements ........................................................................................................................... 9

4.3.2 Operational Status Check ................................................................................................................................. 9

4.3.3 Space Requirements .......................................................................................................................................... 9

4.3.4 Bench Handling Test ........................................................................................................................................ 9

4.3.5 Vibration Test ................................................................................................................................................. 10

4.3.6 Low Temperature Test .................................................................................................................................... 11

4.3.7 High Temperature Test ................................................................................................................................... 11

4.3.8 Humidity Test ................................................................................................................................................. 12

4.4 OPERATING ENVIRONMENTAL TESTS ...................................................................................................................... 12

4.4.1 Simulated Operation Diagnostic .................................................................................................................... 13

4.4.2 Integrity .......................................................................................................................................................... 13

4.4.3 Temperature and Power Variation Tests ........................................................................................................ 13

4.4.4 Reliability Test ................................................................................................................................................ 15

5 HANDLING HARDWARE ANOMALIES AND INCIDENTS ............................................................................... 17

5.1 HARDWARE TEST ANOMALIES ................................................................................................................................ 17

5.2 HARDWARE INCIDENT PROCESS .............................................................................................................................. 17
ES&S EVS 6000 2005 VVSG 

ENVIR. HW Test Plan v1.0.doc 

Document Duplication Prohibited Page 4 of 17 

1. Introduction 

1.1 Overview 

This test plan covers the environmental test requirements and methods for the ES&S 

EVS6000, hereafter known as the Unit Under Test (UUT), to the requirements as stated in 

Election Assistance Commission 2005 Voluntary Voting System Guidelines (VVSG). 

1.2 Qualifications 

The UUT supplied by ES&S is representative of product produced in their volume 

manufacturing process. 

1.3 Client 

Election Systems & Software 

11208 John Galt Blvd 

Omaha, NE 68137

1.4 Company Information 

This document shall not be reproduced outside of SLI Compliance without written consent. 

This document must be reproduced in whole unless written consent has been attained from 

SLI Compliance. 

1.5 Test Facility Location 

NTS Environmental Lab 

1601 Dry Creek Road, Ste 2000 

1.6 Longmont, CO 80504 

1.7 Reference Documents 

• Election Assistance Commission: 2005 Voluntary Voting System Guidelines (EAC 

VVSG), 2005, Version 1.0, Volumes I and II. 

• NIST Handbook 150-22. 2008. 

• EAC Notice of Clarification 07-05: Voting System Test Laboratory (VSTL) responsibilities 

in the management and oversight of third party testing. 

• EAC Decision on Request for Interpretation 2007-05 (COTS). 

• EAC Decision on Request for Interpretation 2008-01 FINAL (temp and power variation 

tests). 

• EAC Decision on Request for Interpretation 2009-06 (Temperature Power Variation) 

FINAL.041610. 

• SLI Standard Lab Procedure SLP-VC-20: Engineering Change Evaluation and 

Reporting. 

• SLI Standard Lab Procedure SLP-VC-23: Hardware Test Management. 

• SLI Standard Lab Procedure SLP-VC-24: Subcontractor Laboratory Management. 
ES&S EVS 6000 2005 VVSG 

ENVIR. HW Test Plan v1.0.doc 

Document Duplication Prohibited Page 5 of 17 

2 Testing Summary 

The following Table 1 shows the tests to be performed on the UUT. 

Test VVSG 

Reference Test Specification 

Non-Operating Environmental Tests

Bench Handling Test V1, 4.1.7.1, 

V2, 4.6.2 

MIL-STD-810D, Method 516.3, Procedure VI; See note below. 

Vibration Test V1, 4.1.7.1, 

V2, 4.6.3 

MIL-STD-810D, Method 514.3, Category 1- Basic 

Transportation, Common Carrier; See note below. 

Low Temperature Test V1, 4.1.7.1, 

V2, 4.6.4 

MIL-STD-810D, Methods 502.2, Procedure I-Storage; See note 

below. 

High Temperature Test V1, 4.1.7.1, 

V2, 4.6.5 

MIL-STD-810D, Methods 501.2, Procedure I-Storage 

See note below. 

Humidity Test V1, 4.1.7.1, 

V2, 4.6.6 

MIL-STD-810D, Method 507.2, Procedure I-Natural 

Hot-Humid; See note below. 

Operating Environmental Tests

Temp / Power Variation V1, 4.1.7.1, 

V2, 4.7.1 

This test is similar to the low temperature and high temperature tests 

of MIL-STD-810-D, Method 502.2 and Method 501.2. See RFI 2009-

06; See note below. 

Reliability Assessment V1, 4.3.3.4, 

V2, 4.7.2 

See V1 4.3.3.4 for additional information. 

Integrity V1, 2.1.4 (d) Protect against ambient temperature and humidity fluctuations. 

Table 1 Test Summary 

Note: V1 4.1.7.1 Removable Storage Media; In voting systems that use storage media that can be 

removed from the system and transported to another location for readout and report generation, these 

media shall use devices with demonstrated error-free retention for a period of 22 months under the 

environmental conditions for operation and non-operation contained in Subsection 4.1.2. Examples of 

removable storage media include: programmable read-only memory (PROM), random access 

memory (RAM) with battery backup, magnetic media or optical media. 
ES&S EVS 6000 2005 VVSG 

ENVIR. HW Test Plan v1.0.doc 

Document Duplication Prohibited Page 6 of 17 

3 Product Description 

3.1 Unit Under Test 

Part No. Serial No. Description Qty 

DS200 v1.2 Precinct Tabulator on metal ballot box 3 

DS200 v1.3 Precinct Tabulator on plastic ballot box 3 

DS450 Central Tabulator 3 

DS850 Central Tabulator 3 

ExpressVote v1.0 Precinct Ballot Marker Device 3 

ExpressVote v2.1 Precinct Tabulator 3 

ExpressVote XL Precinct Tabulator 3 

ExpressTouch DRE 3 

3.2 Product Information 

Description Dimensions Weight 

DS200 v1.2 25” L x 22”W x 41” H (DS200/Metal Ballot Box) 121 lbs 

DS200 v1.3 27” L x 24”W x 45” H (DS200/Plastic Ballot Box) 97 lbs 

DS450 

45”L x 20”W x 32”H 

26”L x 48”W x 62”H unit, cart, accessories 

137 lbs 

415 lbs unit, cart, accessories 

DS850 

20.5”L x 41”W x 37”H 

26”L x 48”W x 60”H unit, cart, accessories 

200 lbs 

500 lbs unit, cart, accessories 

ExpressVote v1.0 

Tabletop: 12”L x 20”W x 17”H 

ExpressVote/Booth: 32” W x 21.5”W x 58”H 

Tabletop: 22 lbs 

ExpressVote/Booth: 45 lbs 

ExpressVote v2.1 

Tabletop: 12”L x 20”W x 17”H 

ExpressVote/Kiosk: 29.5”L x 23.75”W x 70.5”H 

Tabletop: 22 lbs 

ExpressVote/Kiosk: 76 lbs 

ExpressVote XL 45.5”W x 25”D x 73”H 296 lbs 

ExpressTouch 19.5”L x 19.5”W x 53.5”H 21 lbs 

3.3 Support Equipment (SE) 

Support Equipment (SE) 

Name Model No. Serial No. Description

Dell S2810 Laser Printer Dell S2810 DS450 report printer 

OKI B431d Laser Printer OKI B431d DS850 report printer 

OKI 420 Dot Matrix Printer OKI 420 Dot Matrix DS450 and DS850 report printer 

UPS APC Pro 1500 DS450 and DS850 UPS 

Report Printer ExpressTouch report printer 

Universal Voting Console 

(UVC) 

 Detachable audio-tactile keypad and 

ADA support peripheral for use 

w/ExpressVote XL and 

ExpressTouch 

Bar Code Scanner Optional external barcode scanner 

provides the ability to read a 

barcode that automatically preloads 

voters’ contest choices via the Ballot 

Online application or automatically 

selects the voter’s correct ballot 
ES&S EVS 6000 2005 VVSG 

ENVIR. HW Test Plan v1.0.doc 

Document Duplication Prohibited Page 7 of 17 

style. 

3.4 Power Supplies 

Manufacturer Model Input Output and Type 

 

3.5 Accessories 

Type Model Function 

USB Media 

Activation Cards w/Ballot Style ExpressVote Cards, Supervisor 

Test Ballots 

Headphone 

Rocker Switch Two-position switch assistive technology that can 

assist voter’s who are unable to use the 

ExpressVote touch screen or the audio tactile 

keypad. 

3.6 Software 

Type Version Description 

DS200 v1.2 Test software 

DS200 v1.3 Test software 

DS450 Test software 

DS850 Test software 

ExpressVote v1.0 Test software 

ExpressVote v2.1 Test software 

ExpressVote XL Test software 

ExpressTouch Test software 

3.7 Engineering Changes 

Engineering Change (EC)# Description 

N/A 
ES&S EVS 6000 2005 VVSG 

ENVIR. HW Test Plan v1.0.doc 

Document Duplication Prohibited Page 8 of 17 

4 Test Requirements 

4.1 Test Procedures 

4.1.1 Operating Environment 

Equipment used for election management activities or vote counting (including both precinct and 

central count systems) shall be capable of operation in temperatures ranging from 50 to 95 

degrees Fahrenheit. 

4.1.2 Environmental Control - Transit and Storage 

Equipment used for vote casting or for counting votes in a precinct count system, shall meet 

these specific minimum performance standards that simulate exposure to physical shock and 

vibration associated with handling and transportation by surface and air common carriers, and 

to temperature conditions associated with delivery and storage in an uncontrolled warehouse 

environment: 

• High and low storage temperatures ranging from -4 to +140 degrees Fahrenheit, 

equivalent to MIL-STD-810D, Methods 501.2 and 502.2, Procedure I-Storage; 

• Bench handling equivalent to the procedure of MIL-STD-810D, Method 516.3, Procedure 

VI; 

• Vibration equivalent to the procedure of MIL-STD-810D, Method 514.3, Category 1- Basic 

Transportation, Common Carrier; and 

• Uncontrolled humidity equivalent to the procedure of MIL-STD-810D, Method 507.2, 

Procedure I-Natural Hot-Humid. 

4.2 Design, Construction and Maintenance 

This covers voting system materials, construction workmanship, and specific design 

characteristics important to the successful operation and efficient maintenance of the voting 

system. 

4.2.1 Physical Attributes 

The following physical attributes will be examined to assess reliability: 

a. Presence of labels and the identification of test points 

b. Provision of built-in test and diagnostic circuitry or physical indicators of condition 

c. Presence of labels and alarms related to failures 

d. Presence of features that allow non-technicians to perform routine maintenance tasks 

(such as update of the system database) 

4.2.2 Additional Attributes 

The following additional attributes will be considered to assess system maintainability: 

a. Ease of detecting that equipment has failed by a non-technician 

b. Ease of diagnosing problems by a trained technician 

c. Low false alarm rates (i.e., indications of problems that do not exist) 

d. Ease of access to components for replacement 

e. Ease with which adjustment and alignment can be performed 

f. Ease with which database updates can be performed by a non-technician 
ES&S EVS 6000 2005 VVSG 

ENVIR. HW Test Plan v1.0.doc 

Document Duplication Prohibited Page 9 of 17 

g. Adjust, align, tune or service components 

4.3 Non-Operating Environmental Tests 

4.3.1 Data Retention Requirements 

Electronic memory storage devices are required to retain their data for at least 22 months to 

meet the United States Code Title 42 requirements and various sections of the VVSG. The data 

retention capability of the devices will be verified by engineering analysis including a review of 

the manufacturer's specifications to ensure that it exceeds the requirement. In the absence of 

other information, such as field failures, the results of this analysis will be used to determine 

compliance with the 22 month retention requirement.

4.3.2 Operational Status Check 

When all tests, inspections, repairs, and adjustments have been completed, normal operation 

shall be verified by conducting an operational status check. 

During this process, all equipment shall be operated in a manner and under environmental 

conditions that simulate election use to verify the functional status of the system. Prior to the 

conduct of each of the environmental hardware non-operating tests, a supplemental test shall be 

made to determine that the operational state of the equipment is within acceptable performance 

limits. 

The following procedures shall be followed to verify the equipment status: 

Step 1: Arrange the system for normal operation. 

Step 2: Turn on power, and allow the system to reach recommended operating temperature. 

Step 3: Perform any servicing, and make any adjustments necessary, to achieve operational 

status. 

Step 4: Operate the equipment in all modes, demonstrating all functions and features that would 

be used during election operations. 

Step 5: Verify that all system functions have been correctly executed. 

4.3.2.1 Failure Criteria 

Upon completion of each non-operating test, the system hardware shall be subject to 

functional testing to verify continued operability. If any portion of the voting machine or precinct 

counter hardware fails to remain fully functional, the testing will be suspended until the failure 

is identified and corrected by the manufacturer. The system will then be subject to a retest. 

4.3.3 Space Requirements 

There is no restriction on space allowed for the installation of voting systems, except that the 

arrangement of these systems shall not impede performance of their duties by polling place 

officials, the orderly flow of voters through the polling place, or the ability for the voter to vote in 

private. 

4.3.4 Bench Handling Test 

The bench handling test simulates stresses faced during maintenance and repair of voting 

machines and ballot counters. 
ES&S EVS 6000 2005 VVSG 

ENVIR. HW Test Plan v1.0.doc 

Document Duplication Prohibited Page 10 of 17 

4.3.4.1 Applicability 

All systems and components, regardless of type, shall meet the requirements of this test. This 

test is equivalent to the procedure of MIL-STD-810D, Method 516.3, and Procedure VI. 

4.3.4.2 Procedure 

Step 1: Place each piece of equipment on a level floor or table, as for normal operation or 

servicing. 

Step 2: Make provision, if necessary, to restrain lateral movement of the equipment or its 

supports at one edge of the device. Vertical rotation about that edge shall not be restrained. 

Step 3: Using that edge as a pivot, raise the opposite edge to an angle of 45 degrees, to a 

height of four inches above the surface, or until the point of balance has been reached, 

whichever occurs first. 

Step 4: Release the elevated edge so that it may drop to the test surface without restraint. 

Step 5: Repeat steps 3 and 4 for a total of six events. 

Step 6: Repeat steps 2, 3, and 4 for the other base edges, for a total of 24 drops for each 

device. 

4.3.5 Vibration Test 

The vibration test simulates stresses faced during transport of voting machines and ballot 

counters between storage locations and polling places. 

4.3.5.1 Applicability 

All systems and components, regardless of type, shall meet the requirements of this test. This 

test is equivalent to the procedure of MIL-STD-810D, Method 514.3, Category 1- Basic 

Transportation, and Common Carrier. 

4.3.5.2 Procedure 

Step 1: Install the test item in its transit or combination case as prepared for transport. 

Step 2: Attach instrumentation as required to measure the applied excitation. 

Step 3: Mount the equipment on a vibration table with the axis of excitation along the vertical 

axis of the equipment. 

Step 4: Apply excitation as shown in MIL-STD-810D, Method 514.3-1, “Basic transportation, 

common carrier, vertical axis”, with low frequency excitation cutoff at 10 Hz, for a period of 30 

minutes. 

Step 5: Repeat steps 2 and 3 for the transverse and longitudinal axes of the equipment with the 

excitation profiles shown in Figures 514.3-2 and 514.3-3, respectively. (Note: The total 

excitation period equals 90 minutes, with 30 minutes excitation along each axis.) 

Step 6: Remove the test item from its transit or combination case and verify its continued 

operability. 
ES&S EVS 6000 2005 VVSG 

ENVIR. HW Test Plan v1.0.doc 

Document Duplication Prohibited Page 11 of 17 

4.3.6 Low Temperature Test 

The low temperature test simulates stresses faced during storage of voting machines and ballot 

counters. 

4.3.6.1 Applicability 

All systems and components, regardless of type, shall meet the requirements of this test. This 

test is equivalent to the procedure of MIL-STD-810D, Method 502.2, and Procedure I-Storage. 

The minimum temperature shall be -4 degrees F. 

4.3.6.2 Procedure 

Step 1: Arrange the equipment as for storage. Install it in the test chamber. 

Step 2: Lower the internal temperature of the chamber at any convenient rate, but not so rapidly 

as to cause condensation in the chamber, and in any case no more rapidly than 10 degrees F 

per minute, until an internal temperature of -4 degrees F has been reached. 

Step 3: Allow the chamber temperature to stabilize. Maintain this temperature for a period of 4 

hours after stabilization. 

Step 4: Allow the internal temperature of the chamber to return to standard laboratory 

conditions, at a rate not exceeding 10 degrees F per minute. 

Step 5: Allow the internal temperature of the equipment to stabilize at laboratory conditions 

before removing it from the chamber. 

Step 6: Remove the equipment from the chamber and from its containers, and inspect the 

equipment for evidence of damage. 

Step 7: Verify continued operability of the equipment. 

4.3.7 High Temperature Test 

The high temperature test simulates stresses faced during storage of voting machines and 

ballot counters. 

4.3.7.1 Applicability 

All systems and components, regardless of type, shall meet the requirements of this test. This 

test is equivalent to the procedure of MIL-STD-810D, Method 501.2, and Procedure I-Storage. 

The maximum temperature shall be 140 degrees F. 

4.3.7.2 Procedure 

Step 1: Arrange the equipment as for storage. Install it in the test chamber. 

Step 2: Raise the internal temperature of the chamber at any convenient rate, but in any case 

no more rapidly than 10 degrees F per minute, until an internal temperature of 140 degrees F 

has been reached. 

Step 3: Allow the chamber temperature to stabilize. Maintain this temperature for a period of 4 

hours after stabilization. 
ES&S EVS 6000 2005 VVSG 

ENVIR. HW Test Plan v1.0.doc 

Document Duplication Prohibited Page 12 of 17 

Step 4: Allow the internal temperature of the chamber to return to standard laboratory 

conditions, at a rate not exceeding 10 degrees F per minute. 

Step 5: Allow the internal temperature of the equipment to stabilize at laboratory conditions 

before removing it from the chamber. 

Step 6: Remove the equipment from the chamber and from its containers, and inspect the 

equipment for evidence of damage. 

Step 7: Verify continued operability of the equipment. 

4.3.8 Humidity Test 

The humidity test simulates stresses faced during storage of voting machines and ballot 

counters. 

4.3.8.1 Applicability 

All systems and components regardless of type shall meet the requirements of this test. This 

test is similar to the procedure of MIL-STD-810D, Method 507.2, Procedure I-Natural HotHumid. It is intended to evaluate the ability of the equipment to survive exposure to an 

uncontrolled temperature and humidity environment during storage. This test lasts for ten days. 

4.3.8.2 Procedure 

Step 1: Arrange the equipment as for storage. Install it in the test chamber. 

Step 2 Adjust the chamber conditions to those given in MIL-STD-810D Table 507.2-I, for the 

time 0000 of the Hot Humid cycle (Cycle 1). 

Step 3: Perform a 24-hour cycle with the time and temperature-humidity values specified in 

Figure 507.2-1, Cycle 1. 

Step 4: Repeat Step 2 until 5, 24-hour cycles have been completed. 

Step 5: Continue with the test commencing with the conditions specified for time = 0000 hours. 

Step 6: At any convenient time in the interval between time = 120 hours and time = 124 hours, 

place the equipment in an operational configuration, and perform a complete operational status 

check as defined in Subsection 4.6.1.5. 

Step 7: If the equipment satisfactorily completes the status check, continue with the sixth 24-

hour cycle. 

Step 8: Perform 4 additional 24-hour cycles, terminating the test at time = 240 hours. 

Step 9: Remove the equipment from the test chamber and inspect it for any evidence of 

damage. 

Step 10: Verify continued operability of the equipment. 

4.4 Operating Environmental Tests 

This section addresses a range of tests for all voting system equipment, including equipment for 

both precinct count and central count systems. 
ES&S EVS 6000 2005 VVSG 

ENVIR. HW Test Plan v1.0.doc 

Document Duplication Prohibited Page 13 of 17 

4.4.1 Simulated Operation Diagnostic 

A diagnostic test routine is performed to exercise and diagnose failures from internal 

subsystems in the UUT. The test performs various operations including writing and reading to 

storage devices and printing to internal printers. The diagnostic can be looped continuously and 

will halt if an error is detected while performing an operation. 

For DRE equipment, each loop may be very short. For paper ballot scanning devices, a 

recirculation ballot operation is typically used where the ballot is read, reversed to the input 

position and then read again. The test will loop continuously until manually interrupted. 

4.4.2 Integrity 

The UUT is subject to integrity verification based on the provisions of Volume I, Section 2.1.4 (d) 

to ensure system integrity, all system shall protect against ambient temperature and humidity 

fluctuations. The ambient temperature fluctuations are verified during the temperature and power 

variation test. The MIL-STD-810D, Method 507.2, Procedure I-Natural Hot-Humid is considered 

to be industry standard and is selected to verify humidity fluctuations.

4.4.3 Temperature and Power Variation Tests 

This test is similar to the low temperature and high temperature tests of MIL-STD-810-D, 

Method 502.2 and Method 501.2, with test conditions that correspond to the requirements of the 

performance standards. This procedure tests system operation under various environmental 

conditions for at least 163 hours. During 48 hours of this operating time, the device shall be in a 

test chamber. For the remaining hours, the equipment shall be operated at room temperature. 

The system shall be powered for the entire period of this test; the power may be disconnected 

only if necessary for removal of the system from the test chamber. 

Operation shall consist of ballot-counting cycles, which vary with system type. An output report 

need not be generated after each counting cycle. The interval between reports, however, should 

be no more than 4 hours to keep to a practical minimum the time between the occurrence of a 

failure or data error and its detection. 

Test Ballots per Counting Cycle 

• Precinct count systems 100 ballots/hour 

• Central count systems 300 ballots/hour 

The recommended pattern of votes is one chosen to facilitate visual recognition of the reported 

totals; this pattern shall exercise all possible voting locations. System features such as data 

quality tests, error logging, and audit reports shall be enabled during the test. Each operating 

cycle shall consist of processing the number of ballots indicated above. 

Other functionality of the ES&S EVS 6000 will be verified as well, such as any available data 

quality tests, error logging as well as audit log reports. 

The criteria reviewed for this requirement will include: 

• The vote data reports generated every 4th hour on devices and Count. 

• Error reports generated 

• Audit log reports generated 

• Any anomalies will be researched in order to determine if it is explainable or is a test 

failure. 

Each operating cycle shall consist of processing the number of ballots indicated above.
ES&S EVS 6000 2005 VVSG 

ENVIR. HW Test Plan v1.0.doc 

Document Duplication Prohibited Page 14 of 17 

Step 1: Arrange the equipment in the test chamber. Connect as required and provide for power, 

control, and data service through enclosure wall. 

Step 2: Set the supply voltage at 117 voltage alternating current. 

Step 3: Power the equipment, and perform an operational status check as in Section 4.6.1.5. 

Step 4: Set the chamber temperature to 50 degrees F, observing precautions against thermal 

shock and condensation. 

Step 5: Begin 24 hour cycle. 

Step 6: At T=4 hrs, lower the supply voltage to 105 vac. 

Step 7: At T=8 hrs, raise the supply voltage to 129 vac. 

Step 8: At T=11:30 hrs, return the supply voltage to 117 vac and return the chamber 

temperature to lab ambient, observing precautions against thermal shock and condensation. 

Step 9: At T=12:00 hrs, raise the chamber temperature to 95 degrees Fahrenheit. 

Step 10: Repeat Steps 5 through 8, with temperature at 95 degrees Fahrenheit, complete at 

T=24 hrs. 

Step 11: Set the chamber temperature at 50 degrees Fahrenheit as in Step 4. 

Step 12: Repeat the 24 hour cycle as in Steps 5-10, complete at T=48 hrs. 

Step 13: After completing the second 24 hour cycle, disconnect power from the system and 

remove it from the chamber if needed. 

Step 14: Reconnect the system as in Step 2, and continue testing for the remaining period of 

operating time. 

Note: Requires 24-hr continuous coverage / support from the hardware test lab for the duration 

of Temperature / Power Variation Test. Support from the hardware test lab includes monitoring 

and setting voltage meter when required base off test plan requirements. 

4.4.3.1 Test Approach 

Test configuration: 

• 3 - DS200 v1.2, duration = 64 hours (48 + 16 hrs) for each unit under test 

• 3 - DS200 v1.3, duration = 64 hours (48 + 16 hrs) for each unit under test 

• 3 - ExpressVote v1.0, duration = 64 hours (48 + 16 hrs) for each unit under test 

• 3 - ExpressVote v2.1, duration = 64 hours (48 + 16 hrs) for each unit under test 

• 3 - ExpressVote XL, duration = 64 hours (48 + 16 hrs) for each unit under test 

• 3 - ExpressTouch, duration = 64 hours (48 + 16 hrs) for each unit under test

• 3 - DS450, duration = 64 hours (48 + 16 hrs) for each unit under test 

• 3 - DS850, duration = 64 hours (48 + 16 hrs) for each unit under test 

Per test ballots counting cycle requirement, test will be conducted as such: 

• DS200 v1.2 / v1.3 – scan 100 pre-marked ballots every hour. 

• ExpressVote v1.0 - ??? 

• ExpressVote v2.1 - ??? 

• ExpressVote XL - ??? 

• ExpressTouch - ?? 
ES&S EVS 6000 2005 VVSG 

ENVIR. HW Test Plan v1.0.doc 

Document Duplication Prohibited Page 15 of 17 

Central Count 

The Central Count systems consist scanners and COTS workstations. 

Test duration is for 64 continuous hours. Only the first 48 hours is required to be in 

temperature/power variation phase of the test. 

Test Configuration = 64 hours (minimum total hours for the statistical model) 

 3 - DS450 mid-range scanner and tabulator 

3 - DS850 high-speed scanner and tabulator 

Per test ballots counting cycle requirement, test will be conducted as such: 

• 300 pre-marked ballots will be scanned every hour. Then same 300 ballots with be use 

in each of the scanner workstations. After each hour ballots will be retired. 

The test will be conducted to run in 4 hour cycles, such that each cycle is concluded with the 

generation of a report that details the vote data cast during that period. 

Reports generated will be: 

4.4.4 Reliability Test 

The accredited test lab shall test for reliability based on the provisions of Volume I, Section 4 

for the acceptable Mean Time Between Failure (MBTF). The MBTF shall be measured during 

the conduct of other system performance tests specified in this section, and shall be at least 

163 hours. Appendix C of VVSG Vol. II provides further details of the calculation for this testing 

period. 

The “cause for failure” is only limited by the functions being performed by the scanner while in 

use, as partially denoted by the criteria “Loss of one or more functions“, while this list attempts 

to cover all potential points of failure, if an issue occurs outside of this list will still be reviewed. 

For this test, the criteria will be defined as any function observed to have failed from its 

intended purpose, during the conduction of the test. E.g., for the scanner, scanning ballots 

(physically moving them in and out of the scanner), the screen remains on and active, 

counters are active, printer continues to function.

A failure is defined as any event which results in either: 

a. USB ports not writing media 

b. AutoBallot (USB Bar Code Scanner) port not functioning 

c. Thermal printer not printing 

d. Tablet display does not function 

e. Static Audio not playing 

f. Device does not power on or off properly 

g. Device reports do not print 

h. Loss of ATI functionality 

i. Media fails 

j. Media data is corrupted 

k. Error logging is incorrect 

l. Audit logging is incorrect 

m. Battery becomes damaged 
ES&S EVS 6000 2005 VVSG 

ENVIR. HW Test Plan v1.0.doc 

Document Duplication Prohibited Page 16 of 17 

n. Loss of Poll Worker functionality 

o. Loss of LED functionality 

p. Operational Status check registers a failure Access 

q. Any “Devices Tests” fail post testing 

r. Any other unexpected action that deviates from device documentation 

1. Loss of one or more functions. 

2. Degradation of performance such that the device is unable to perform its intended function 

for longer than 10 seconds; will look for degradation of performance of the device, as 

opposed to actual functionality failure of the first failure criteria.

3. Criteria C: COTS and support equipment may have temporary loss of function or 

degradation of performance, the correction of which requires operator intervention or 

system reset.
ES&S EVS 6000 2005 VVSG 

ENVIR. HW Test Plan v1.0.doc 

Document Duplication Prohibited Page 17 of 17 

5 Handling Hardware Anomalies and Incidents 

5.1 Hardware Test Anomalies 

An anomaly with the subcontractor’s test equipment or a procedural misstep can cause a 

test to fail. For any suspected test equipment issue or procedural error, analysis will be 

performed and the decision whether to continue testing based on the severity of the 

anomaly will be appropriately tracked. The subcontractor test lab will issue a corrective 

action to address any test equipment and/or procedure errors. This is part of the hardware 

test subcontractor’s quality system process that allows the hardware test lab to train all 

personnel, repair/calibrate equipment, and prevent any recurrence. 

5.2 Hardware Incident Process 

For every test failure of any voting system component at the hardware test lab, the lab 

completes a data sheet (per their laboratory procedures and templates) and immediately 

informs the SLI Hardware Specialist. This can be communicated in the daily status update, 

with the data sheet attached. 

• Failure Analysis: Once a failure has occurred, the SLI Hardware Specialist will be 

involved with the subcontractor test lab(s) to identify the hardware discrepancy in the 

device. The results of the analysis will be documented and tracked in the discrepancy 

reporting tool used for the test campaign. The analysis will focus on the failure, what 

caused the failure, the severity (minor or major), and possible impacts to other testing. 

• Mitigation: The SLI Hardware Specialist monitors any work done by the manufacturer, 

with the full understanding of what is occurring and why. 

 The Manufacturer will document what work is done and the SLI Hardware Specialist 

will sign off on or can stop the work at any time. 

 The Hardware Specialist will determine the number of "minor" fixes the 

manufacturer can incorporate without a re-start of the test. 

 Any modification to the equipment is followed up with the related manufacturer 

EC(s). All related ECs must be entered into the hardware test report and the 

certification test report. 

When issues are identified during hardware environmental testing, they result in 

discrepancies. Discrepancies are tracked in the discrepancy reporting tool.