ES&S Unity 3.2.0.0 

DS200 and Ballot Box 

Voting System Test Plan 

Performed for 

Systest Labs 

By 

4888 Pearl East Circle, #110 

Boulder, Colorado 80301 

303.444.7480 

www.percept.com

Revision History: 

Version Date Comments Contributor 

1.0 4/9/07 Initial Release M. Wolf 
ES&S DS200 Test Plan.doc Percept Technology Page 2 of 11 

Table of Contents 

1.0 OVERVIEW.................................................................................................................................................................... 3

1.1 TESTING PROCESS OVERVIEW ....................................................................................................................................... 3

2.0 DEFINITIONS ................................................................................................................................................................ 3

2.1 FAILURES ...................................................................................................................................................................... 3

3.0 PRODUCT ASSESSMENT PHASE ............................................................................................................................. 3

3.1 HARDWARE CLASSIFICATION ........................................................................................................................................ 4

3.2 DOCUMENTATION REVIEW............................................................................................................................................ 4

3.3 OPERATIONAL STATUS CHECK...................................................................................................................................... 4

3.4 TESTING PHASE ENTRANCE CRITERIA........................................................................................................................... 5

3.5 TEST PLAN GENERATION............................................................................................................................................... 5

4.0 TESTING PHASE........................................................................................................................................................... 7

4.1 HARDWARE REQUIREMENTS ......................................................................................................................................... 7

4.2 TEST PROCEDURES ........................................................................................................................................................ 7

4.2.1 Non-Operating Tests ................................................................................................................................................ 7

4.2.1.1 Maintainability (VVSG II 4.7.2) ........................................................................................................................................7

4.2.1.2 Safety (VVSG I 4.3.8)........................................................................................................................................................7

4.2.1.3 Transit Drop .......................................................................................................................................................................8

4.2.2 Non-Operating Environmental Tests........................................................................................................................ 8

4.2.2.1 Bench Handling Test (VVSG II 4.6.2) ...............................................................................................................................8

4.2.2.2 Vibration Test (VVSG II 4.6.3)..........................................................................................................................................8

4.2.2.3 Low Temperature Test (VVSG II 4.6.4).............................................................................................................................8

4.2.2.4 High Temperature Test (VVSG II 4.6.5)............................................................................................................................8

4.2.2.5 Humidity Test (VVSG II 4.6.6)..........................................................................................................................................8

4.2.3 Operating Environmental Test ................................................................................................................................. 9

4.2.3.1 Accessibility and Human Engineering Evaluation .............................................................................................................9

4.2.3.2 Temperature and Power Variation Tests (VVSG II 4.7.1)..................................................................................................9

4.2.3.3 Reliability (VVSG II 4.7.3) ................................................................................................................................................9

4.2.3.4 Data Accuracy (VVSG II 4.7.1.1) ......................................................................................................................................9

4.2.4 Electrical Tests......................................................................................................................................................... 9

4.2.4.1 Power Disturbance (VVSG II 4.8)......................................................................................................................................9

4.2.4.2 Electromagnetic Radiation (VVSG II 4.8)........................................................................................................................10

4.2.4.3 Electrostatic Disruption (VVSG II 4.8) ............................................................................................................................10

4.2.4.4 Electromagnetic Susceptibility (VVSG II 4.8) .................................................................................................................10

4.2.4.5 Electrical Fast Transient (VVSG II 4.8) ...........................................................................................................................10

4.2.4.6 Lightning Surge (VVSG II 4.8)........................................................................................................................................10

4.2.4.7 Conducted RF Immunity (VVSG II 4.8) ..........................................................................................................................11

4.2.4.8 Magnetic Fields Immunity (VVSG II 4.8)........................................................................................................................11
ES&S DS200 Test Plan.doc Percept Technology Page 3 of 11 

1.0 Overview 

This test plan describes the hardware evaluations performed in accordance with the Voluntary 

Voting System Guidelines (VVSG) 2005. While some sections are quoted in this test plan, please 

see the VVSG itself for full details. 

1.1 Testing Process Overview 

The following steps shall be followed for all submitted products: 

• Systest and Percept Technology Labs shall inventory and document the receipt of all 

vendor materials. 

• Testing, as defined in the test plan, will not proceed until formal authorization has been 

given. Deficiencies encountered during the Testing Phase will be documented and 

tracked. 

• Upon project completion, the deliverable shall be a test report outlining in detail all of the 

testing performed per the test plan and a written recommendation of approval or rejection 

to Systest Labs. 

2.0 Definitions 

2.1 Failures 

The following definitions apply to errors or failures (deficiencies) that occur during the testing: 

1. Critical deficiency 

• Unrecoverable data error that would affect the loss or corruption of voting data. 

• Unrecoverable functional error that causes the system to be inoperable. 

• Mechanical failure 

• Safety issue 

2. Non-critical deficiency 

• Performance degradation 

• Specification deviation 

• Command completed, but failed to meet a specification 

The nature of any deficiency encountered during the testing is described in detail sufficient to 

support the classification as critical or non-critical. The classification is based on consideration of 

the probable effect the deficiency will have on reliable, safe, accurate and efficient system 

operation during all phases of election use. 

All critical deficiencies must be resolved before a recommendation of approval can be made for 

the system. An unresolved non-critical deficiency shall not necessarily be cause for a rejection 

recommendation. Deficiencies of this type may include failure to fully achieve the levels of 

performance specified in Volume I, Sections 3 and 4 of the Standards. 

Note: The resolution of a deficiency may, in some cases, require re-executing already completed 

tests resulting in additional charges. 

3.0 Product Assessment Phase 

The following items shall be taken into consideration during the Assessment Phase to ensure that 

the product is ready for test and determine the specific tests that will be performed during the 

Testing Phase. Although the Assessment Phase, typically, is completed prior to the generation of 

the test plan it is included in this document since it is crucial to successful execution of the test 

plan. 
ES&S DS200 Test Plan.doc Percept Technology Page 4 of 11 

3.1 Hardware Classification 

Hardware shall be identified and classified in one of the following categories: 

• Unmodified COTS (commercial off-the-shelf): Equipment that is commercially available, 

which can be independently purchased based on the vendor’s system specification, with 

assembly performed by the jurisdiction. Vendors shall provide the manufacturers 

specifications and evidence that the equipment has been tested to the equivalent of the 

VVSG. 

• Modified COTS: Commercially available (COTS) components that are assembled or 

packaged in a case or modified by the vendor. 

• Non-COTS: The vendor manufactures the voting system. 

Note: Non-COTS and modified COTS components are subject to all environmental testing. 

Unmodified COTS hardware is exempt provided the specifications and evidence of testing 

meets the requirements of the VVSG. 

3.2 Documentation Review 

The vendor shall provide hardware specifications and documentation, as part of the Technical 

Data Package (TDP), needed to assess, operate and maintain the voting system. These 

documents include, but are not limited to: 

• All manuals and product specifications required for constructing, operating and 

maintaining the system; 

• Set-up/operation instructions for all support equipment, including software and hardware. 

• Previous qualification/test reports on COTS and Modified COTS equipment. 

• Previous qualification/test reports from other testing organizations. 

• Maintenance procedures. 

The supplied documentation will be reviewed to ensure that it is adequate for operating the 

equipment and that testing operations can be performed. 

3.3 Operational Status Check 

When all tests, inspections, repairs, and adjustments have been completed, normal operation 

shall be verified by conducting an operational status check. During this process, all equipment 

shall be operated in a manner and environmental conditions that simulate election use to verify 

the functional status of the system. Prior to and following the conduct of each of the 

environmental hardware non-operating tests, a test shall be performed to determine that the 

operational state of the equipment is within acceptable performance limits. 

The following procedures shall be followed to verify the equipment status: 

Step 1: Arrange the system for normal operation. 

Step 2: Turn on power, and allow the system to reach recommended operating temperature. 

Step 3: Perform any servicing, and make any adjustments necessary, to achieve operational 

status. 

Step 4: Operate the equipment in all modes, demonstrating all functions and features that would 

be used during election operations. 

Step 5: Verify that all system functions have been correctly executed. 
ES&S DS200 Test Plan.doc Percept Technology Page 5 of 11 

3.4 Testing Phase Entrance Criteria 

The following are deliverables from the Assessment Phase and trigger entry into the Testing 

Phase: 

• Test Plan agreed upon by all involved parties 

• Estimated testing time and budget 

• Fully operational system 

• Key contacts identified for all involved parties (email, phone and fax numbers) 

3.5 Test Plan Generation 

The following checklist is the final step in the assessment phase of the product evaluation. Once 

this checklist is complete, the product will progress to the testing phase. 
ES&S DS200 Test Plan.doc Percept Technology Page 6 of 11 

Applicable Item VVSG 

Reference Comments 

Assessment 

 EUT inventory N/A See the PCA Traveler. 

 Hardware classification N/A No COTS equipment is involved in this test. 

 Documentation Review N/A Complete. 

 Operational Status Check N/A Complete. 

Non-Operating Tests

 Maintainability Test V2 4.7.2 

 Safety Evaluation V1 4.3.8 

Transit Drop N/A Florida Voting System Standards, 

Form DS-DE 101, Eff. 1-12-05 

Non-Operating Environmental Tests

 Bench Handling Test V2 4.6.2 

 Vibration Test V2 4.6.3 

 Low Temperature Test V2 4.6.4 

 High Temperature Test V2 4.6.5 

 Humidity Test V2 4.6.6 

Operating Environmental Tests

Accessibility and Human Engineering 

Evaluation 

V1 Appendix B,

Mil Std 1472 

Temperature/Power Variation and Reliability V2 4.7.1, 

V2 4.7.3 

Data Accuracy V2 4.7.1.1 This test is the responsibility of Systest 

Labs. However, it begins with the 

Temperature/ Power Variation test which 

lasts 48 hours. 

Electrical Tests

 Power Disturbance V2 4.8 

 Electromagnetic Radiation V2 4.8 

 Electrostatic Disruption V2 4.8 

 Electromagnetic Susceptibility V2 4.8 

 Electrical Fast Transient V2 4.8 

 Lightning Surge V2 4.8 

 Conducted RF Immunity V2 4.8 

 Magnetic Fields Immunity V2 4.8 
ES&S DS200 Test Plan.doc Percept Technology Page 7 of 11 

4.0 Testing Phase 

Prior to the commencement of testing, the Exit Criteria referenced above shall be met and all 

involved parties shall have agreed on a schedule. 

4.1 Hardware Requirements 

Refer to Section 4 Volume 1 of the VVSG. This section contains the requirements for the 

machines and manufactured devices that are part of a voting system. It specifies minimum values 

for certain performance characteristics, physical characteristics; and design, construction, and 

maintenance characteristics for the hardware and selected related components of all voting 

systems. These requirements are reflected in the following test procedures. 

4.2 Test Procedures 

4.2.1 Non-Operating Tests 

4.2.1.1 Maintainability (VVSG II 4.7.2) 

Maintainability represents the ease with which maintenance actions can be performed based on 

the design characteristics of equipment and software and the processes the vendor and election 

officials have in place for preventing failures and for reacting to failures. Maintainability includes 

the ability of equipment and software to self-diagnose problems and make non-technical 

election workers aware of a problem. Maintainability addresses all scheduled and unscheduled 

events, which are performed to: 

• Determine the operational status of the system or a component; 

• Adjust, align, tune, or service components; 

• Repair or replace a component having a specified operating life or replacement 

interval; 

• Repair or replace a component that exhibits an undesirable predetermined physical 

condition or performance degradation; 

• Repair or replace a component that has failed; and 

• Verify the restoration of a component, or the system, to operational status. 

Maintainability shall be determined based on the presence of specific physical attributes that aid 

system maintenance activities, and the ease with which system maintenance tasks can be 

performed by the ITA. Although a more quantitative basis for assessing maintainability, such as 

the mean to repair the system is desirable, the qualification of a system is conducted before it is 

approved for sale and thus before a broader base of maintenance experience can be obtained. 

Should significant impediments or difficulties be encountered that are not remedied by the 

vendor, Percept will include such findings in the qualification test results of the qualification test 

report. 

4.2.1.2 Safety (VVSG I 4.3.8) 

All voting systems shall meet the following requirements for safety: 

a. All voting systems and their components shall be designed to eliminate hazards to personnel 

or to the equipment itself. 

b. Defects in design and construction that can result in personal injury or equipment damage 

must be detected and corrected before voting systems and components are placed into service. 
ES&S DS200 Test Plan.doc Percept Technology Page 8 of 11 

c. Equipment design for personnel safety shall be equal to or better than the appropriate 

requirements of the Occupational Safety and Health Act, Code of Federal Regulations, Title 29, 

Part 1910. 

4.2.1.3 Transit Drop 

This test is equivalent to the Transit Drop Test for equipment weighing between 100 and 

1000 pounds (corner drop), MIL STD 810D, Method 516.3, Procedure IV, except that the 

drop height specified in Table 516.3-II therein is reduced to 12 inches. Drops shall be made 

from a quick-release hook or drop tester. 

4.2.2 Non-Operating Environmental Tests 

4.2.2.1 Bench Handling Test (VVSG II 4.6.2) 

The bench-handling test simulates stresses faced during maintenance and repair of voting 

machines and ballot counters. 

4.2.2.1.1 Applicability 

All systems and components, regardless of type, shall meet the requirements of this test. This 

test is equivalent to the procedure of MIL-STD-810D, Method 516.3, Procedure VI. 

4.2.2.2 Vibration Test (VVSG II 4.6.3) 

The vibration test simulates stresses faced during transport of voting machines and ballot 

counters between storage locations and polling places. 

4.2.2.2.1 Applicability 

All systems and components, regardless of type, shall meet the requirements of this test. This 

test is equivalent to the procedure of MIL-STD-810D, Method 514.3, Category 1- Basic 

Transportation, Common Carrier. 

4.2.2.3 Low Temperature Test (VVSG II 4.6.4) 

The low temperature test simulates stresses faced during storage of voting machines and ballot 

counters. 

4.2.2.3.1 Applicability 

All systems and components, regardless of type, shall meet the requirements of this test. This 

test is equivalent to the procedure of MIL-STD-810D, Method 502.2, Procedure I-Storage. The 

minimum temperature shall be -4 degrees F. 

4.2.2.4 High Temperature Test (VVSG II 4.6.5) 

The high temperature test simulates stresses faced during storage of voting machines and ballot 

counters. 

4.2.2.4.1 Applicability 

All systems and components, regardless of type, shall meet the requirements of this test. This 

test is equivalent to the procedure of MIL-STD-810D, Method 501.2, Procedure I-Storage. The 

maximum temperature shall be 140 degrees F. 

4.2.2.5 Humidity Test (VVSG II 4.6.6) 

The humidity test simulates stresses faced during storage of voting machines and ballot counters. 
ES&S DS200 Test Plan.doc Percept Technology Page 9 of 11 

4.2.2.5.1 Applicability 

All systems and components regardless of type shall meet the requirements of this test. This test 

is similar to the procedure of MIL-STD-810D, Method 507.2, Procedure I-Natural Hot-Humid. It is 

intended to evaluate the ability of the equipment to survive exposure to an uncontrolled 

temperature and humidity environment during storage. This test lasts for ten days. 

4.2.3 Operating Environmental Test 

4.2.3.1 Accessibility and Human Engineering Evaluation 

Refer to MIL-STD-1472 Human Engineering Design Criteria for Military Systems, Equipment 

and Facilities. 

4.2.3.2 Temperature and Power Variation Tests (VVSG II 4.7.1) 

This test is similar to the low temperature and high temperature tests of MIL-STD810D, Method 

502.2 and Method 501.2, with test conditions that correspond to the requirements of the 

performance standards. This procedure tests system operation under various environmental 

conditions for at least 163 hours. During 48 hours of this operating time, the device shall be in a 

test chamber. For the remaining hours, the equipment shall be operated at room temperature. 

The system shall be powered for the entire period of this test; the power may be disconnected 

only if necessary for removal of the system from the test chamber. 

Operation shall consist of ballot-counting cycles, which vary with system type. An output report 

need not be generated after each counting cycle; the interval between reports, however, should 

be no more than 4 hours to keep to a practical minimum the time between the occurrence of a 

failure or data error and its detection. 

 Test Ballots per Counting Cycle

Precinct count systems 100 ballots/hour 

Central count systems 300 ballots/hour 

The recommended pattern of votes is one chosen to facilitate visual recognition of the reported 

totals; this pattern shall exercise all possible voting locations. System features such as data 

quality tests, error logging, and audit reports shall be enabled during the test. 

4.2.3.3 Reliability (VVSG II 4.7.3) 

The MBTF shall be measured during the conduct of other system performance tests specified in 

this section, and shall be at least 163 hours. 

4.2.3.4 Data Accuracy (VVSG II 4.7.1.1) 

This test is run in conjunction with Temperature and Power Variation Tests (VVSG II 4.7.1). 

Refer to the VVSG for details. 

4.2.4 Electrical Tests 

4.2.4.1 Power Disturbance (VVSG II 4.8) 

The test for power disturbance disruption shall be conducted in compliance with the test 

specified in IEC 61000-4-11 (1994-06). 

Vote scanning and counting equipment for paper-based systems, and all DRE equipment, shall 

be able to withstand, without disruption of normal operation or loss of data: 

a. Surges of 30% dip @10 ms; 
ES&S DS200 Test Plan.doc Percept Technology Page 10 of 11 

b. Surges of 60% dip @100 ms & 1 sec 

c. Surges of >95% interrupt @5 sec; 

d. Surges of +15% line variations of nominal line voltage; and 

e. Electric power increases of 7.5% and reductions of 12.5% of nominal specified 

power supply for a period of up to four hours at each power level. 

4.2.4.2 Electromagnetic Radiation (VVSG II 4.8) 

Vote scanning and counting equipment for paper-based systems, and all DRE equipment, shall 

comply with the Rules and Regulations of the Federal Communications Commission, Part 15, 

Class B requirements for both radiated and conducted emissions. 

4.2.4.3 Electrostatic Disruption (VVSG II 4.8) 

The test for electrostatic disruption shall be conducted in compliance with the test specified in 

IEC 61000-4-2 (1995-01). 

Vote scanning and counting equipment for paper-based systems, and all DRE equipment, shall 

be able to withstand ±15 kV air discharge and ±8 kV contact discharge without damage or loss 

of data. The equipment may reset or have momentary interruption so long as normal operation 

is resumed without human intervention or loss of data. Loss of data means votes that have been 

completed and confirmed to the voter. 

4.2.4.4 Electromagnetic Susceptibility (VVSG II 4.8) 

The test for electromagnetic susceptibility shall be conducted in compliance with the test 

specified in IEC 61000-4-3 (1996). 

Vote scanning and counting equipment for paper-based systems, and all DRE equipment, shall 

be able to withstand, without disruption of normal operation or loss of data, conducted RF 

energy of: 

a. 10V AC & DC power; and 

b. 10V, 20 sig/control >3m. 

4.2.4.5 Electrical Fast Transient (VVSG II 4.8) 

The test for electrical fast transient protection shall be conducted in compliance with the test 

specified in IEC 61000-4-4 (1995-01). 

Vote scanning and counting equipment for paper-based systems, and all DRE equipment, shall 

be able to withstand, without disruption of normal operation or loss of data, electrical fast 

transients of: 

c. 2 kV AC & DC external power lines; 

d. +1 kV all external wires >3m no control; and 

e. +2 kV all external wires control. 

4.2.4.6 Lightning Surge (VVSG II 4.8) 

The test for lightning surge protection shall be conducted in compliance with the test specified in 

IEC 61000-4-5 (1995-02). 

Vote scanning and counting equipment for paper-based systems, and all DRE equipment, shall 

be able to withstand, without disruption of normal operation or loss of data, surges of: 

a. +2 kV AC line to line; 
ES&S DS200 Test Plan.doc Percept Technology Page 11 of 11 

b. +2 kV AC line to earth; 

c. +.5 kV DC line to line >10m; 

d. +.5 kV DC line to earth >10m; and 

e. +1 kV I/O sig/control >30m. 

4.2.4.7 Conducted RF Immunity (VVSG II 4.8) 

The test for conducted RF immunity shall be conducted in compliance with the test specified in 

IEC 61000-4-6 (1996-04). 

Vote scanning and counting equipment for paper-based systems, and all DRE equipment, shall 

be able to withstand, without disruption of normal operation or loss of data, conducted RF 

energy of: 

a. 10V AC & DC power; and 

b. 10V, 20 sig/control >3m. 

4.2.4.8 Magnetic Fields Immunity (VVSG II 4.8) 

The test for AC magnetic fields RF immunity shall be conducted in compliance with the test 

specified in IEC 61000-4-8 (1993-06). 

Vote scanning and counting equipment for paper-based systems, and all DRE equipment, shall 

be able to withstand, without disruption of normal operation or loss of data, AC magnetic fields 

of 30 A/m at 60 H