3131 South Vaughn Way, Suite 650, Aurora, Colorado, 80014

Form-D: VSTL Test Report template

Election Systems & Software 

Unity 3.2.0.0 Voting System

VSTL Certification Test Report

Prepared for 

Election Systems & Software

11208 John Galt Blvd.

Omaha, NE 68137

EAC Application # ESS0701

Version 1.0

(V)2009-30Jun-001(A)

This report has been submitted to the EAC for review and is pending their acceptance. No 

certification number has been issued. When iBeta receives notification that the report is accepted, 

a revised version of the report will be issued. The Certification number will appear here, in page 

headers and in Appendix F. Any other revisions will be noted in the version history.

Trace to Standards

NIST Handbook 150-22

Section 5.5, 5.10.1 through 5.10.3, 5.10.5, 5.10.6

VVS

Vol. # Section(s) #

1 1.6.1

1 2, 3, 4, 5, & 6

1 9.6.3

2 2, 3, 4, 5, & 6

2 7.4 & 7.5.

2 Appendix B

Test Results in this report apply to the voting system configuration tested. Testing of voting systems that have been 

modified may or may not produce the same test results. This report shall not be reproduced, except in full.

iBeta Quality Assurance is accredited for Voting System Testing:

EAC Lab Code: 0702 - Effective thru 2/28/2009 - Renewal Pending Determination of 2009 EAC Audit
VSTL Certification # pending

(V)2009-30Jun-001(A) ii

Version History

Ver # Description of Change Author Approved by Date

v.1.0 Initial Release Gail Audette 

Carolyn Coggins

Jennifer Garcia 

Sridevi Jakileti

Kelly Swift

Kevin Wilson

Carolyn 

Coggins

6/30/09
VSTL Certification # pending

- iii - (V)2009-30Jun-001(A)

TABLE OF CONTENTS

1 INTRODUCTION ............................................................................................................................................ 6

1.1 UNITY 3.2.0.0 PHYSICAL & FUNCTIONAL CONFIGURATION AUDIT SCOPE .................................................... 6

1.2 UNITY 3.2.0.0 EXCLUSIONS: ..................................................................................................................... 7

1.2.1 Unity v.4.0.0.0 Scope Excluded from Unity 3.2.0.0 ........................................................................... 7

1.2.2 Unity 3.2.0.0 Other Exclusions .......................................................................................................... 8

1.3 INTERNAL DOCUMENTATION...................................................................................................................... 8

Table 1 Internal Documents........................................................................................................................................ 8

1.4 EXTERNAL DOCUMENTATION..................................................................................................................... 9

Table 2 External Documents ...................................................................................................................................... 9

1.5 TECHNICAL DATA PACKAGE DOCUMENTS ................................................................................................ 13

1.6 TEST REPORT CONTENTS....................................................................................................................... 13

1.6.1 VSTL Program Manual Format Trace ............................................................................................. 13

Table 3 Trace of the Test Report to the VSTL Program Manual............................................................................... 13

2 CERTIFICATION TEST BACKGROUND .................................................................................................... 15

2.1 TERMS AND DEFINITIONS ........................................................................................................................ 15

Table 4 Terms and Definitions.................................................................................................................................. 15

2.2 PHYSICAL CONFIGURATION AUDIT........................................................................................................... 17

2.2.1 PCA TDP Source Code Review ...................................................................................................... 17

2.2.2 PCA TDP Document Review........................................................................................................... 18

2.2.3 PCA System Configuration Review................................................................................................. 18

2.2.4 Witness Build and Installation.......................................................................................................... 18

2.3 FUNCTIONAL CONFIGURATION AUDIT....................................................................................................... 18

2.3.1 FCA Test Documentation Review ................................................................................................... 18

2.3.2 FCA Functional, Accessibility, Maintainability, Accuracy & Reliability Tests .................................. 18

2.3.3 FCA Volume, Stress and Error Recovery Tests.............................................................................. 19

2.3.4 FCA Security Tests.......................................................................................................................... 19

2.3.5 FCA Hardware Environmental Tests ............................................................................................... 19

2.3.6 FCA Telephony and Cryptographic Review and Tests ................................................................... 19

3 VOTING SYSTEM IDENTIFICATION .......................................................................................................... 20

3.1 SUBMITTED VOTING SYSTEM IDENTIFICATION .......................................................................................... 20

Table 5 Voting System Name and Version............................................................................................................... 20

Table 6 Voting System Polling Place and Central Count Hardware ......................................................................... 20

Table 7 Voting System EMS Software...................................................................................................................... 22

3.2 VOTING SYSTEM TEST ENVIRONMENT ..................................................................................................... 22

Table 8 Voting System Hardware............................................................................................................................. 22

Table 9 Voting System Software .............................................................................................................................. 25

Table 10 Voting System Technical Data Package Documents................................................................................. 26

Table 11 Other Software, Hardware and Materials................................................................................................... 32

4 VOTING SYSTEM OVERVIEW.................................................................................................................... 34

4.1 ELECTION MANAGEMENT SYSTEM- PRE VOTING CAPABILITIES ................................................................. 34

4.1.1 Election Data Manager (EDM)......................................................................................................... 34

4.1.2 ES&S Ballot Image Manager (ESSIM) ............................................................................................ 34

4.1.3 Hardware Programming Manager (HPM)........................................................................................ 35

4.1.4 AutoMARK Information Management System (AIMS) .................................................................... 35

4.1.5 Audit Manager (AM) ........................................................................................................................ 35

4.2 POLLING PLACE- VOTING CAPABILITIES................................................................................................... 36

4.2.1 intlElect DS200 Precinct Ballot Scanner ......................................................................................... 36

4.2.2 AutoMARK Voter Assist Terminal (VAT) ......................................................................................... 36

4.3 CENTRAL COUNT SCANNER- POST VOTING CAPABILITIES ........................................................................ 37

4.3.1 Model 650 Central Ballot Scanner (M650) ...................................................................................... 38

4.4 ELECTION MANAGEMENT SYSTEM- POST VOTING CAPABILITIES ............................................................... 38

4.4.1 Election Reporting Manager............................................................................................................ 38

5 CERTIFICATION REVIEW AND TEST RESULTS...................................................................................... 40
VSTL Certification # pending

(V)2009-30Jun-001(A) iv

5.1 PCA SOURCE CODE REVIEW.................................................................................................................. 40

5.1.1 3% Source Code Review Assessment for Reuse Results .............................................................. 40

5.1.2 Unity 3.2.0.0 Source Code Review Results .................................................................................... 40

5.2 PCA TDP DOCUMENT REVIEW............................................................................................................... 41

5.2.1 Unity 3.2.0.0 TDP PCA Document Review Results ........................................................................ 41

5.3 FCA FUNCTIONAL SYSTEM LEVEL, ACCESSIBILITY, MAINTAINABILITY, ACCURACY & RELIABILITY TESTING AND 

REUSE OF TESTING ............................................................................................................................................. 41

5.3.1 Evaluation for Reuse of the SysTest Labs Testing: Functional, Accessibility, Maintainability, 

Accuracy & Reliability Test Results .............................................................................................................. 41

5.3.2 Regression Functional & System Level Test Results...................................................................... 42

5.4 FCA VOLUME, STRESS, AND ERROR RECOVERY TESTING........................................................................ 42

5.4.1 Volume, Stress and Error Recovery Test Results........................................................................... 43

5.5 FCA SECURITY REVIEW AND TESTING..................................................................................................... 43

5.5.1 FCA Security Review and Test Results .......................................................................................... 43

5.6 FCA HARDWARE ENVIRONMENTAL TESTING & REUSE ............................................................................. 44

5.6.1 FCA Hardware Environmental Tests ............................................................................................... 44

5.7 FCA TELEPHONY AND CRYPTOGRAPHIC REVIEW AND TESTING ................................................................ 44

5.7.1 FCA Telephony and Cryptographic Review and Tests ................................................................... 44

6 OPINIONS & RECOMMENDATIONS.......................................................................................................... 45

7 APPENDICES: TEST OPERATION, FINDINGS & DATA ANALYSIS....................................................... 46

7.1 APPENDIX A: CERTIFICATION TEST REQUIREMENTS................................................................................. 46

7.2 APPENDIX B: PCA SOURCE CODE REVIEW ............................................................................................. 90

7.2.1 SysTest Source Code Review Results............................................................................................ 90

7.2.2 3% Source Code Review Results.................................................................................................... 90

7.2.3 Unity 3.2.0.0 Source Code Review Results .................................................................................... 90

7.3 APPENDIX C: PCA TDP DOCUMENTATION REVIEW.................................................................................. 92

7.3.1 PCA TDP Document Review........................................................................................................... 92

7.3.2 Technical Data Package Configuration & Quality Assurance Practices ......................................... 92

7.4 APPENDIX D: FCA TESTING.................................................................................................................... 93

7.4.1 FCA Functional and System Level Accessibility, Maintainability, Accuracy & Reliability Testing... 93

7.4.1.1 Reuse Functional System Level, Accessibility, Maintainability, Accuracy & Reliability Test Results ..... 93

7.4.1.2 Regression System Level Test Results ................................................................................................. 93

7.4.2 FCA Volume (Volume Stress, Performance and Error Recovery) Testing ..................................... 99

7.4.2.1 Volume 1 Test Results .......................................................................................................................... 99

7.4.2.2 Volume 2 Test Results ........................................................................................................................ 103

7.4.2.3 Volume 3 Test Results ........................................................................................................................ 107

7.4.2.4 Volume 4 Test Results ........................................................................................................................ 108

7.4.2.5 Volume 5 Test Results ........................................................................................................................ 109

7.4.2.6 Volume 6 Test Results ........................................................................................................................ 110

7.4.2.7 Volume 7 Test Results ........................................................................................................................ 112

7.4.2.8 Volume 8 Test Results ........................................................................................................................ 116

7.4.2.9 Volume 9 Test Results ........................................................................................................................ 118

7.4.2.10 Volume 10 Test Results ...................................................................................................................... 121

7.4.3 FCA Security Review and Testing................................................................................................. 124

7.4.4 Telephony and Cryptographic Review and Testing....................................................................... 132

7.4.5 FCA Reuse Environmental Testing ............................................................................................... 134

7.4.5.1 DS200 Environmental Hardware Test Reports &Tested Configuration Matrix .................................... 134

7.4.5.2 M-650 Environmental Hardware Test Reports & Tested Configuration Matrix .................................... 135

7.4.5.3 VAT A-100 Environmental Hardware Test Reports & Tested Configuration Matrix............................. 135

7.4.5.4 VAT A-200 Environmental Hardware Test Reports & Tested Configuration Matrix ............................. 136

7.5 APPENDIX E- DISCREPANCY REPORT.................................................................................................... 137

7.6 APPENDIX F: WARRANT OF ACCEPTING CHANGE CONTROL RESPONSIBILITY........................................... 221

7.7 APPENDIX G: TRUSTED BUILD & VALIDATION TOOLS UNITY 3.2.0.0 VOTING SYSTEM ............................... 222

7.7.1 Witness of the Trusted Build of AIMSCRYPT v. 1.0.0.2................................................................ 222

7.7.2 Witness of the Trusted Build AutoMARK PrinterEngineBoard1.7 (PEB) ...................................... 223

7.7.3 Witness of the Trusted Build AutoMARK VAT1.3.2906 &AIMS1.3.157 ........................................ 226

7.7.4 Witness of the Trusted Build DS200 Ancillary Devices (PowermanagementMsp430 1.2.0.1 

&Scanner C8051 2.11.0.1)......................................................................................................................... 229
VSTL Certification # pending

(V)2009-30Jun-001(A) v

7.7.5 Witness of the Trusted Build DS200 v.1.3.10.0............................................................................. 231

7.7.6 Witness of the Trusted Build DS200 TOS v.1.0.1.0 ...................................................................... 232

7.7.7 Witness of the Trusted Build Model 650 v.2.2.2.0......................................................................... 234

7.7.8 Witness of the Trusted Build Unity 3.2.0.0 .................................................................................... 236

7.7.9 Witness of the Trusted Build Unity 3.2.0.0 (Supplemental Build A) - ERM v.7.5.4.0 .................... 240

7.7.10 ES&S Unity 3.2.0.0 System Identification Tools ....................................................................... 243

7.8 APPENDIX H: AMENDED TEST PLAN ...................................................................................................... 244

7.9 APPENDIX I: STATE TEST REPORTS....................................................................................................... 245

7.10 APPENDIX J ES&S UNITY 3.2.0.0 IMPLEMENTATION STATEMENT............................................................ 246

7.11 APPENDIX K – EAC CERTIFICATION NUMBER & VOTING SYSTEM CONFIGURATION.................................. 247
VSTL Certification # pending

Page 6 of 247 (V)2009-30Jun-001(A)

1 Introduction

This report is submitted to the Election Assistance Commission (EAC) by iBeta Quality Assurance summarizing the 

federal voting system certification testing of the Election System & Software (ES&S) Unity 3.2.0.0 voting system to 

the Voting System Standards 2002 (VSS 2002).

The ES&S Unity 3.2.0.0 voting system has been submitted to iBeta for testing to support ES&S‟ application # 

ESS0701 to the US Election Assistance Commission (EAC) for certification to the VSS 2002. This is an initial EAC 

certification. 

The Unity 3.2.0.0 is a paper-based voting system that includes the:

Election management system election (EMS) preparation software: Election Data Manager, ES&S Ballot 

Image Manager, Hardware Programming Manager, AutoMARK Information Management System

EMS audit software: Audit Manager

Pre-vote hardware: Ballot-on-Demand COTs printer

Polling place optical scanner hardware and firmware: Model DS200

Polling place ballot marker hardware and firmware: AutoMARK Voter Assist Terminal A100, AutoMARK 

Voter Assist Terminal Model A200

Central count hardware and firmware: Model 650

Central count EMS software: Election Reporting Manager

The purpose of this document is to provide an overview of the certification testing and findings. The complete list of 

the systems names, major subsystems, version numbers and any interfacing devices is contained detailed in 

section 3 Voting System Identification. Additional details of the design, structure, and processing capabilities are 

identified in the section 4 Voting System Overview.

Application #ESS0701 originally identified SysTest Labs (SysTest) as the VSTL. Due to the suspension of SysTest 

in the middle of various Unity certification efforts, ES&S was authorized by the EAC to transfer their application for 

certification of the Unity 3.2.0.0 to iBeta. Unity 3.2.0.0 is a subset of paper ballot voting systems contained in the 

Unity v.4.0.0.0 voting system. At the time of the suspension the Unity v.4.0.0.0 test plan was approved by the EAC 

and a substantial amount of relevant testing had been successfully completed. ES&S petitioned the EAC to assess 

the testing performed by SysTest for consideration of reuse. The EAC issued a letter to ES&S, 2-3-2009 Letter to 

ESS Reuse of Testing Final, in which they outlined the conditions for the assessment of reuse process. This 

process is outlined in the as run test plan which is contained in the attached Appendix H Amended Test Plan.

In the letter EAC approval to reuse portions of SysTest's testing of ES&S Unity 3.2.0.0 the EAC authorized the 

reuse of the functional, accessibility, maintainability, accuracy, and reliability testing conducted for Unity 3.2.0.0 

base upon the EAC technical reviewer's audit of all test plans, test methods, test cases, and test results related to 

the scope of the Unity 3.2.0.0 test campaign. This included a review of a document created by SysTest that 

summarized all related testing conducted for the scope of the Unity 3.2.0.0 with the test results. The EAC 

concluded:

All functional, accessibility, maintainability, accuracy, and reliability testing outlined in the approved 

SysTest Unity 4.0 test plan is approved for reuse in the Unity 3.2.0.0 test campaign.

As part of the remaining testing the EAC is tasking iBeta with testing and verifying that the Unity 3.2.0.0 

system is in compliance with EAC RFI 2008-07 “'0‟ count to start the election”. This testing should be 

reflected in the test plan being developed by iBeta for the Unity 3.2.0.0 system.

iBeta is also tasked with testing and resolving the discrepancies listed by SysTest under the following 

tests: GEN 02 – Straight Party, GEN 03 – Add Languages, and PR101 – Pick-a-Party tests.

In a subsequent conversation with the EAC this last bullet was clarified to the open functional discrepancies 

identified in Table 5 of the as run test plan.

1.1 Unity 3.2.0.0 Physical & Functional Configuration Audit Scope

This certification test effort included a Physical Configuration Audit (PCA) and Functional Configuration Audit (FCA) 

of the Unity 3.2.0.0 voting system. Due to the petition for reuse these tasks were performed by either SysTest or 

iBeta Quality Assurance. Assessment of the SysTest test results was performed by either iBeta or the EAC 

Technical Reviewers as stipulated in 2-3-2009 Letter to ESS Reuse of Testing Final. 

The Physical Configuration Audit (PCA) performed by iBeta for Unity 3.2.0.0 incorporated a:

PCA Document Review Assessment for reuse of the SysTest's review of the Unity v.4.0.0.0 Technical 

Data Package (TDP); 

3% PCA Source Code Review Assessment for reuse of the SysTest source code review of the Unity 

v.4.0.0.0 source code review;

Witnessed Build performed by SysTest from the source code they reviewed;
VSTL Certification # pending

Page 7 of 247 (V)2009-30Jun-001(A)

Trusted Build performed by iBeta from the SysTest and iBeta reviewed source code; and

Examination of the voting system configuration submitted to iBeta. 

A Functional Configuration Audit (FCA) of the Unity 3.2.0.0 included an EAC review of the Unity v.4.0.0.0 testing 

performed by SysTest to:

The requirements of VSS 2002;

The Unity v.4.0.0.0 specifications of the ES&S TDP; and

The voting system requirements of section 301 of the Help American Vote Act (HAVA).

For the balance of the FCA iBeta identified the scope of the Unity 3.2.0.0 volume, stress, error recovery, and 

security requirements of the VSS 2002, in addition to a sampling of the requirements necessary to conduct a single 

end-to-end system level functional regression test. iBeta: 

Developed a test plan; 

Customize test cases; 

Managed the system configurations; 

Executed tests, and 

Analyzed the test results for the iBeta executed tests.

Certification testing performed by iBeta complied with the requirements of VSS 2002, Volume 2 Test Standards. 

The iBeta test record included test executions and reviews performed by iBeta. These test executions and reviews 

included the record of requirements that were satisfactorily and unsatisfactorily completed, deficiencies noted, 

reports to ES&S, software and manufacturing resolutions, validations of resolutions and documentation of 

incorporation of resolutions into the voting system. Test records for work performed by SysTest were retained by 

them. Materials were provided to the EAC and iBeta for the assessment of reuse.

iBeta Quality Assurance, a limited liability company, is located in Aurora, Colorado. The company is a full service 

software testing laboratory providing Quality Assurance and Software Testing for the business and interactive 

entertainment communities. iBeta Quality Assurance accreditations for the testing of voting systems to the federal 

standards include 

National Voluntary Lab Accreditation Program (NVLAP) Voting System Test Lab (VSTL)

Election Assistance Commission Voting Systems Test Lab (VSTL)

Testing was conducted at iBeta Quality Assurance in Aurora, Colorado, SysTest Labs in Denver, Colorado and 

various SysTest subcontractor non-core hardware environmental test labs. Non-core hardware environmental 

testing is outside SysTest's test accreditation scope as a VSTL. SysTest's methods for validating the qualifications 

of the subcontractor laboratories was provided to the EAC and considered in their decision to permit reuse of the 

non-core environmental testing. SysTest conducted the non-core safety and hardware environmental assessments 

and testing with the following subcontractors:

Compliance Technology Services 1820 Skyway Drive Unit J, Longmont, Colorado 80504

Components Reliability & Safety 1955 West 153rd Place, Broomfield, CO 80020

Criterion Technology 1350 Tolland Road, P.O. Box 489, Rollinsville, CO 80474

Nebraska Center for Excellence in Electronics (NCEE) 4740 Discovery Dr., Lincoln, NE 68521

Percept Technology Labs 4735 Walnut St. #E, Boulder, CO 80301

Sun Advanced Product Testing (APT) 1601 Dry Creek Drive Suite 2000, Longmont, CO 80503

Wyle Laboratories, 7800 Highway 20 West, Huntsville, AL, 35806

1.2 Unity 3.2.0.0 Exclusions:

The following identifies the exclusions of the Unity 3.2.0.0 voting system. As identified in the VSS2002 vol.1 section 

4.1.2, software is excluded if it:

Provides no support of voting system capabilities;

Cannot function while voting system functionality is enabled; and

Procedures are provided that confirm software has been removed, disconnected or switched. 

1.2.1 Unity v.4.0.0.0 Scope Excluded from Unity 3.2.0.0

The Unity 4.0.0.0 items identified as exclusions are not contained in the Unity 3.2.0.0 system submitted for 

Certification under EAC Application # ESS0701.

Hardware including related software/firmware and peripherals: Automated Bar Code Reader (ABCR), 

iVotronic DRE Precinct Tabulator, Model 100 Precinct Ballot Counter, the DS200 modem kit, and the 

M650 configured with a network card;

EMS Software: Data Acquisition Manager and iVotronic Ballot Image Manager; and

System functionality and maintenance: DRE, VVPAT

Network functionality: Network data transmission for remote transmission of votes or consolidated results

Language accessibility other than English and Spanish.
VSTL Certification # pending

Page 8 of 247 (V)2009-30Jun-001(A)

In an email dated October 15, 2008 the EAC granted permission for ES&S to reuse the Unity v.4.0.0.0 TDP if the 

documents bore a disclaimer outlining the uncertified functionality that was not part of the Unity 3.2.0.0 certification. 

As such the review of the document content related to the uncertified Unity v.4.0.0.0 functionality was excluded 

from this review.

In receiving the source code, documents and test artifacts from SysTest, iBeta determined if the material was in or 

out of the Unity 3.2.0.0 test scope. Items determined to be out of scope were stored without further examination. 

No out of scope hardware was received. 

1.2.2 Unity 3.2.0.0 Other Exclusions

The following functions are excluded from Unity 3.2.0.0 voting system and are not tested in this certification effort. 

Provisional ballots: The handling of provisional ballots is procedural. There is no provisional ballot 

functionality.

Transmission via Public Telecommunications: There is no transmission via public telecommunications. 

The DS200 modem is removed from this certification.

Use of Wireless Communications : There is no use of wireless communications

Shared Operating Environment: Unity 3.2.0.0 does not share an environment with other data processing 

functions.

Enhanced AutoCast: This AutoMARK functionality requires both PEB v.1.70 and Auto MARK FW v.1.4. 

That version of AutoMARK firmware is not supported in Unity 3.2.0.0.

The hardware environment for the election management system is limited to stand along PCs. The 

election preparation, central count tally and reporting functions do not communicated via a Local Area 

Network or other network connection in Unity 3.2.0.0.

There is no provision for the broadcasting of results.

Any activities in these areas are limited to documentation that the functions are not applicable to this voting system.

1.3 Internal Documentation

The documents identified below are iBeta internal documents used in certification testing 

Table 1 Internal Documents

Version # Title Abbreviation Date Author (Org.)

v.07 Voting Certification Master 

Services AgreementElection Systems & Software

MSA contract 11/15/08 iBeta Quality Assurance

Rev 02 Statement of Work No. 02 

Commencement Phase: 

Assessment for Reuse and 

Reporting

SOW 2-02 iBeta Quality Assurance

Rev 01 Statement of Work No. 03 

Maximum Reuse Project 

Estimate

SOW 3-01 iBeta Quality Assurance

iBeta VSTL Procedures

v.2.1 Voting Deliverable Receipt 

Procedure

9/19/08 iBeta Quality Assurance

v.1.0 PCA Source Code Review 

Procedure

11/13/06 iBeta Quality Assurance

v.4.0 C and C++ Review Criteria 11/17/08 iBeta Quality Assurance

v.1.0 Z80 Assembler Review 

Criteria

10/19/07 iBeta Quality Assurance

v.2.0 SQL Server Review Criteria 6/19/07 iBeta Quality Assurance

v.0.1 COBOL Review Criteria 12/4/08 iBeta Quality Assurance

v.2.0 Visual Basic Review Criteria 6/19/2007 iBeta Quality Assurance

v.1.0 Witness Build Procedure 4/18/08/ iBeta Quality Assurance

v.2.0 Trusted Build Procedure 1/23/09 iBeta Quality Assurance

v.4.0 Test Case Preparation 

and Execution Procedure

5/23/08 iBeta Quality Assurance

v.4.0 VSTL Test Planning 

Procedure

5/23/08 iBeta Quality Assurance

v.4.0 VSTL Certification Report 

Procedure

4/24/09 iBeta Quality Assurance
VSTL Certification # pending

Page 9 of 247 (V)2009-30Jun-001(A)

Version # Title Abbreviation Date Author (Org.)

iBeta Project Documents

ESS Unity 3.2 Code and 

Equipment Receipt

6/24/09 iBeta Quality Assurance

E001 through E039

Equipment Photos

Equipment 

Images

various iBeta Quality Assurance

ESS Source Code Review 

results documents for the 

applications

Various dates

iBeta letter to EAC on ES&S 

Unity 3.2.0.0 source code 

review

3% Source 

Code Review 

Assessment

1/16/09 iBeta Quality Assurance

iBeta recommendation on 

reuse of SysTest PCA 

Document Review in ES&S 

Unity 3.2.0.0 testing

PCA Document 

Review 

Assessment

1/14/09 iBeta Quality Assurance

Election Systems & Software 

Unity 3.2.0.0 Voting System 

VSTL Certification Test Plan

PCA Configuration- Unity 3.2 6/26/09 iBeta Quality Assurance

Reuse Environmental Test 

Case -Unity 3.2

2/15/09 iBeta Quality Assurance

Reuse Characteristics Test 

Case -Unity 3.2

2/15/09 iBeta Quality Assurance

FCA Volume 1 5/11/09 iBeta Quality Assurance

FCA Volume 2 6/29/09 iBeta Quality Assurance

FCA Volume 3 6/3/09 iBeta Quality Assurance

FCA Volume 4 4/16/09 iBeta Quality Assurance

FCA Volume 5 6/8/09 iBeta Quality Assurance

FCA Volume 6 6/17/09 iBeta Quality Assurance

FCA Volume 7 6/17/09 iBeta Quality Assurance

FCA Volume 8 6/17/09 iBeta Quality Assurance

FCA Volume 9 6/29/09 iBeta Quality Assurance

FCA Volume 10 5/12/09 iBeta Quality Assurance

FCA Security Review Unity 

3.2

6/23/09 iBeta Quality Assurance

FCA Security Test - Unity 

3.2 Windows Configuration 

Test steps 

6/3/09 iBeta Quality Assurance

Regression System Level 

TC

6/17/09 iBeta Quality Assurance

ES&S Unity 3.2.0.0 EAC 

Matrix

3/6/09 iBeta Quality Assurance

A200 Configuration 

Inspection Results

4/1/09 iBeta Quality Assurance

Trace to Test Methods 3/20/09 iBeta Quality Assurance

1.4 External Documentation

The documents identified below are external resources used to in certification testing.

Table 2 External Documents

Version 

#

Title Abbreviation Date Author (Org.)

Test Plan Approval Letter 4/7/09 EAC

Help America Vote Act HAVA 10/29/02 107th

Congress

NIST 

Hdbk 150 

2006 Ed.

NVLAP Voting System Testing NIST 150 February 

2006

NVLAP

NIST 

Hdbk 

150-22

NVLAP Voting System Testing NIST 150-22 December 

2005

NVLAP
VSTL Certification # pending

Page 10 of 247 (V)2009-30Jun-001(A)

Version 

#

Title Abbreviation Date Author (Org.)

Federal Election Commission Voting System 

Standards

VSS April 2002 Federal 

Election 

Commission

Voting System Testing and Certification Manual 1/1/07 EAC

v.1.0 Voting System Test Laboratory Program Manual VSTL 

Program 

Manual

July 2008 EAC

Form 

v.5.2

EAC Test Matrix Template EAC

EAC Decision on Request for Interpretation 2007-02, 

2002 VSS, Vol. 1, Section 4.2.5 (single character)

Interpretation 

2007-02

5/14/07 EAC

EAC Decision on Request for Interpretation 2007-05,

2005 VVSG Vol. 1 Section 4.2.1 (Testing Focus and 

Applicability)

Interpretation 

2007-05

11/6/07 EAC

EAC Decision on Request for Interpretation 2008-01,

2002 VSS Vol. II, 2005 VVSG Vol. II, Section 4.7.1 & 

Appendix C

Interpretation 

2008-01

2/6/08 EAC

EAC Decision on Request for Interpretation 2008-02,

Battery Backup for Optical Scan Voting machines

Interpretation 

2008-02

2/19/08 EAC

EAC Decision on Request for Interpretation 2008-04,

2002 VSS Vol. I, Section 2.3.1.3.1a; 2005 VVSG Vol. 

II, Section 2.2.1.3a Ballot Production

Interpretation 

2008-04

5/19/08 EAC

EAC Decision on Request for Interpretation 2008-05, 

2002 VSS Vol. I, Section 3.4.2, 2005 VVSG Vol. I, 

Section 4.3.2, Durability

Interpretation 

2008-05

5/19/08 EAC

EAC Decision on Request for Interpretation 2008-06, 

2002 VSS Vol. I, Sections 3.2.2.4c, 3.2.2.5

2005 VVSG Vol. I, V. 1.0, Sections 4.1.2.4c (Electrical 

Supply), 4.1.2.5 (Electrical Power Disturbance)

Interpretation 

2008-06

8/29/2008 EAC

EAC Decision on Request for Interpretation 2008-07; 

2002 VSS Vol. I, Sections, 2.3.4, 2.3.5, 2.3.6, 2.4.1, 

4.4.3, 9.4; 2002 VSS Vol. II, Sections, 3.3.1, 3.3.2; 

2005 VVSG Vol. I, Sections, 2.2.4, 2.2.5, 2.2.6, 2.3.1, 

5.4.3; 2005 VVSG Vol. II, Sections, 1.3, 3.3.1, 3.3.2

Interpretation 

2008-07

8/27/08 EAC

EAC Decision on Request for Interpretation 2008-08

2002 VSS Vol. I, Glossary

2005 VVSG Vol. I, Glossary

Interpretation 

2008-08

8/1/2008 EAC

EAC Decision on Request for Interpretation 2008-09 

(Safety Testing) 

2002 VSS Vol. I, Section, 3.4.8; 2005 VVSG Vol. I, 

Section 4.3.8

Interpretation 

2008-09

8/25/2008 EAC

EAC Decision on Request for Interpretation 2008-10 

(Electrical Fast Transient)

2005 VVSG Vol. I, Sect. 4.1.2.6; 2005 VVSG Vol. II, 

Section 4.8

Interpretation 

2008-10

8/28/2008 9/7/07

Notice of Clarification NOC 07-001: Timely 

Submission of Certification Application

NOC 07-01 7/17/07 EAC

Notice of Clarification NOC 07-002: VSTL Work with 

Manufacturers Outside of Voting System Certification 

Engagements

NOC 07-02 7/24/07 EAC

Notice of Clarification 07-05: Voting System Test 

Laboratory (VSTL) responsibilities in the management 

and oversight of third party testing.

NOC 07-05 9/7/07 EAC

Notice of Clarification NOC 08-001: Validity of Prior 

Non-Core Hardware Environ-mental and EMC Testing

NOC 08-01 3/26/08 EAC

Notice of Clarification

NOC 08-003: Clarification of EAC Conformance 

Testing Requirements for VSTLs

NOC 08-03 7/30/08 EAC

Unity 3.2.0.0 EAC Correspondence

2002 VSS Supported Functionality Declaration Unity 

3.2.0.0

10/29/08 ES&S

Unity 3.2.0.0 Implementation Statement 10/29/08 ES&S
VSTL Certification # pending

Page 11 of 247 (V)2009-30Jun-001(A)

Version 

#

Title Abbreviation Date Author (Org.)

Unity 3.2.0.0 Modules No date ES&S

ES&S letter to EAC requesting permission to change 

VSTL Unity

10/31/08 ES&S

EAC letter to ES&S granting their request to replace 

SysTest Labs

11/18/08 EAC

SysTest iBeta Notice Ltr 11_21_08 11/21/08 ES&S

EAC letter to iBeta QA Director on reuse of testing 2/3/09 EAC

EAC approval to reuse portions of SysTest's testing 

of ES&S Unity 3.2.0.0

2/12/09 EAC

EAC Approval letter of ES&S Unity 3.2 Test 

Plan version 2.0

4/7/09 EAC

Unity 3.2 Discrepancy 135 Request for Interpretation 5/26/09 ES&S

Formal Response Discrepancy 135 6/5/09 EAC

Letter- EAC Decision on Request for Interpretation 

2008-05 Voting Equipment Durability

9/23/08 ES&S

Unity v.4.0.0.0 Reuse Correspondence

Email: Reuse of Previous Testing for Unity 3.2.0.0 11/21/08 EAC

2-3-2009 Letter to ESS Reuse of Testing Final 2/3/09 EAC

Unity v.4.0.0.0 Test Documents

Rev.10.0 ES&S Unity 4.0 Certification Test Plan Document 

Number 07-V-ESS-035-CTP-01

12/9/08 SysTest Labs

Rev.0.2 Voting System Test Summary Report, Test Report for 

testing through 10/22/08 for ES&S Unity 4.0 Voting 

System, Report Number 01-V-ESS-035-CTP-01

12/19/08 SysTest Labs

Unity 4.0 Disc Rpt 10-28-08 10/28/08 SysTest Labs

v.1.16 Retest Matrix v1.16 11/24/08 ES&S

Test Report No.- 080521-1251A

EMC Qualification Test Report ES&S AUTOMARK, 

VAT A200

6/11/08 Criterion 

Technology

v.1.3 AutoMARK Voter Assist Terminal Test Report 6/19/05 Percept Technology Labs

Test Report No.- 041223-857

EMC Qualification Test Report AutoMARK Technical 

Systems, LLC VAT

1/31/05 Criterion 

Technology

Test Report No. - 04-00542

Testing Services Report AutoMARK VAT SN:002

1/14/05 APT

Test Report No. 48489-08 

Hardware Qualification Report of the ES&S M650 

Central Ballot Counter Firmware Release 2.0.1.0

1/7/05

Rev. 1 Test Report No.- ATS-0501-R01-Rev.1

Electrical Safety Testing to UL 60950-1 (Replaces 

#ATS-0501-R01, dated 4/30, 2005)

4/10/06 AutoMARK 

Technical 

Systems

v.1.4 Operational Status Check Test Case (ATS VAT) 1/11/2005 SysTest Labs

Test Report No.- 080327-1225

EMC Qualification Test Report AutoMARK, VAT A100

4/21/08 Criterion 

Technology

Test Report No.- 070730-1165

EMC Qualification Test Report AutoMARK Technical 

Systems, LLC. Ballot Marking Device, VAT A300

8/9/07 Criterion 

Technology

v.1.0 AutoMARK Voter Assist Terminal 1.1 Test Report 1/4/06 Percept Technology Labs

Rev. 2 VAT Accuracy Test Case Status Report SysTest Labs

Test Report No.- 070730-1165

DS200 Scanner EMC Test Report

7/31/07 NCEE

Test Report No.- R071107-30-01B

DS200 Scanner EMC Test Report (Amended with 

Original)

5/27/08 NCEE

Test Report No.- 070314-1134A

EMC Qualification Test Report ES&S DS200 Ballot 

Scanner with Optional 76246 Ballot Box

5/15/07 Criterion 

Technology

Test Report No.- 080521-1244

EMC Qualification Test Report ES&S Precinct Count 

6/18/08 Criterion 

Technology
VSTL Certification # pending

Page 12 of 247 (V)2009-30Jun-001(A)

Version 

#

Title Abbreviation Date Author (Org.)

Ballot Scanner, DS200

Test Report No.- 07-00231Testing Services Report 

DS200 Scanner and Ballot Box (Temp and Humidity)

4/16/07 APT

Test Report No.- 07-00207Testing Services Report 

DS200 Scanner and Ballot Box (Vibration)

4/25/07 APT

v.1.0 DS200 Op Stat Check v1.0 11/21/08 SysTest Labs

v.1.0 ES&S Unity 3.2.0.0 DS200 and Ballot Box and Voting 

System Test Report

5/1/07 Percept Technology Labs

v.1.0 DS200 with Optional Ballot Box ESD Test Report 4/25/07 Percept Technology Labs

Test Report No.- ESS-0802-R04

Summary Test Report Physical Stability Testing to UL 

60950-1

2/ 12/08 Components 

Reliability & 

Safety, Inc.

Test Report No.- 07-1001-A

Product Safety Testing and Evaluation for Ballot 

Reader Model number DS200 with or w/o ballot box

4/27/07 Components 

Reliability & 

Safety, Inc.

DS200 Accuracy Test Summary 4/21/08 SysTest Labs

Test Report No.- 0806-R05

Electrical Safety Testing to UL 60950-1:2007

7/28/08 Compliance 

Integrity 

Services

Test Report No.- R071107-30-02

EMC Test Report (M650)

7/31/07 NCEE

Unity 4.0 Certification Test Plan Rev 6.0 Attachment E 

Test Case Matrix 10071228

Test Report No.- 08-00654

Testing Services Report (M650)

5/2/08 APT

v.1.1 M650 with Attached Printers Test Report 3/ 7/08 SysTest Labs

v.1.3 M650 with Epson Printer Test Plan 7/31/07 SysTest Labs

v.1.1 DS200 Scanner EMC Test Plan 7/30/07 SysTest Labs

Rev.01 Certification Test Plan ESS HW Test Matrix 2/1/08 SysTest Labs

Rev03 Rev03_Model650_TDP06202007 SysTest Labs

Rev05 Rev05_AuditManager_TDP07312007 SysTest Labs

Rev05 Rev05.DAM_TDP09262007_ESS SysTest Labs

Rev09 Rev09.HPM_TDP09122007_ESS SysTest Labs

Rev02 Rev.02_CF_Utility_TDP05072007 SysTest Labs

Rev03 Rev03.ERM_TDP08082007_ESS SysTest Labs

Rev03 Rev03.EDM_BallotDataManager_TDP08012007_ESS SysTest Labs

Rev03 Rev03.DS200_TDP09072007_ESS SysTest Labs

Rev02 Rev02.ESSZIP_TDP07062007 SysTest Labs

Rev02 Rev.02_GetAuditData_TDP04022007 SysTest Labs

Rev02 Rev.02_MPRBOOT_TDP05162007 SysTest Labs

Rev02 Rev.02_SHELL_TDP05072007 SysTest Labs

Rev03 Rev.03_CB_EAGL_TDP05312007 SysTest Labs

Rev03 Rev.03_MAKEIBIN_08072007_ESS SysTest Labs

Rev04 Rev.04_ESSEAGL_TDP07202007_ESS SysTest Labs

Rev04 Rev.04_REGUTIL_TDP5312007 SysTest Labs

Engineering Change Evaluation & Reviews for the 

DS200 ECOs 690 to 693 & 702 to 706 (multiple 

documents)

Various 

dates

SysTest Labs

Non-conforming Work & Corrective Action Request 

SN008 ( for VAT A100 ECO #0025)

1/18/05 Percept 

Technology 

Lab

Engineering Change Evaluation & Review for the VAT 

A200 References 200-206,208, 2 10-247, 256-278, 

324-346.

Various 

dates

SysTest Labs

A Engineering Specification -Model PW-080A2-1Y24AP 

(G) -(DS200 -ferrite molded power supply)

2/3/09 Wall 

Industries
VSTL Certification # pending

Page 13 of 247 (V)2009-30Jun-001(A)

1.5 Technical Data Package Documents

The Technical Data Package Documents submitted for this certification test effort are listed in Section 3 System 

Identification.

1.6 Test Report Contents

The contents of this Test Report include: 

Section 1: The Introduction- identifies the scope of certification testing.

Section 2 The Certification Test Background identifies the process for the Physical and Functional 

Configuration Audits.

Section 3 The Voting System Identification identifies the system configuration including hardware, software 

and the Technical Data Package documentation.

Section 4 The Voting System Overview identifies the overall design and functionality of voting system.

Section 5 The Certification Review and Test Results are the methods and results of the testing effort.

Section 6 The Opinions & Recommendations of the acceptability of the voting system.

Test Operations, Findings and Data Analysis are in the appendices. 

Appendix A: Certification Test Requirement (conformance to the applicable standard).

Appendix B: Source Code Reviews 

Appendix C: TDP Document Reviews

Appendix D: Test Results (Functional, Environmental, Accuracy etc.)

Appendix E- Discrepancy Report: Discrepancy Report

Appendix F: Warrant of Accepting Change Control Responsibility 

Appendix G: Trusted Build and Validation Tools

Appendix H: Test Plan'

Appendix I: State Test Reports

Appendix J:ES&S and voting system Implementation Statement

Appendix K: EAC Certification Number

1.6.1 VSTL Program Manual Format Trace

Appendix B of the Voting System Test Laboratory Program Manual v.1.0 identifies content in a specific format. The 

format of this report follows the recommended outline stipulated in the VSS 2002 vol. 2 Appendix B. As these 

documents indentify placement of information in different locations a trace is being provided to clarify the location of 

the specified content in this report.

Table 3 Trace of the Test Report to the VSTL Program Manual 

EAC VSTL Program Manual Appendix B Test Report - VSS 2002 vol. 2 Appendix B

1. System Identification and Overview 1.

3.

4

Introduction 

Voting System Identification

Voting System Overview

2 Certification Test Background 2. Certification Test Background

2.1 Revision History 2 Certification Test Background

2.2 Implementation Statement 2

7.9

Certification Test Background

Implementation Statement 

3

3.1

Test Findings and Recommendations

Summary Finding and Recommendation

5

6

Certification Review and Test Results

Opinions & Recommendations

3.2 Reasons for Recommendation of Rejection N/A Not applicable; no recommendation of rejection

3.3 Anomalies (may also be identified as 

discrepancies, issues or defects )

Section 5 & 

Appendix E

Appendices:

A

B

D

E

Provides a general description of how anomalies 

were encountered and reported during testing.

Appendix A traces the VVSG requirements to the 

specific anomalies.

Addendum to Appendix B contains software 

related source code discrepancy detail.

Appendix D Tables: "Issues Opened" traces the 

specific anomalies to the relevant test.

Appendix E, Unity 3.2.0.0 Discrepancy Report, 

provides the discrepancy number, date, tester, 

location, description, and VSS requirement 
VSTL Certification # pending

Page 14 of 247 (V)2009-30Jun-001(A)

EAC VSTL Program Manual Appendix B Test Report - VSS 2002 vol. 2 Appendix B

information about anomalies encountered during 

document reviews and testing.

3.4 Correction of Deficiencies Section 5 & 

Appendix E

Appendices:

A

B

D

E

Provides a general description of how deficiency 

corrections were confirmed.

Appendix A traces the VVSG requirements to the 

specific closed anomalies.

Addendum to Appendix B reflects pass criteria for

all reviewed source code.

Appendix D Tables: "Issues Closed" traces the 

specific anomaly resolutions to the test 

Appendix E, Unity 3.2.0.0 Discrepancy Report, 

provides the vendor responses and resolution 

validations for anomalies encountered during 

document reviews and testing.

Appendix 

A

Additional Findings Appendices:

A

D

Appendix A: Certification Test Requirements 

contains "should" and "not applicable" 

requirements. Comments provide rationale and 

references to relevant EAC Interpretations or 

Notices of Clarification.

Appendix D: Supported Voting Variations of the 

VSS 2002 Section 2.2.8.2 identifies 

"unsupported" optional functionality.

Appendix 

B

Warrant of Accepting Change Control 

Responsibility

Appendix 

F

Warrant of Accepting Change Control 

Responsibility

Appendix 

C

Witness Build Appendix 

G

Trusted Build and Validation Tools

documents the Witness of the Trusted Build

Appendix 

D

Test Plan Appendix 

H

Test Plan

Appendix 

C

State Test Reports Appendix I State Test Reports

Appendix J Implementation Statement

Appendix K EAC Certification Number
VSTL Certification # pending

Page 15 of 247 (V)2009-30Jun-001(A)

2 Certification Test Background

Earlier versions of products in this effort completed qualification testing under the defunct NASED program. These 

earlier version are in use, as permitted under the laws of the various states. Under the EAC program, all systems 

submitted must be fully tested as a new system. As such the Unity 3.2.0.0 Certification Test effort is an initial 

certification to the VSS 2002.

Following the circumstances outlined in section 1, the scope of the ES&S Unity 3.2.0.0 certification test effort 

resulted in a unique set of pre-certification test activities. The purpose of these activities was to assist the EAC in 

determining what certification testing and reviews performed by SysTest could be reused. Responsibility for these 

activities was designated to either iBeta or the EAC. These activities are outlined in section 1. Assessment and 

determination of the reuse of the Functional, Usability, Accessibility, Maintainability, Accuracy and Reliability testing 

was provided by the EAC. Results of the activities performed by iBeta are provided below.

Following the determination of reuse and issuance of instructions by the EAC, iBeta conducted a review of the test 

documentation provided by ES&S and SysTest to assess the scope of testing for conformance to the 2002 VSS for 

the Volume, Stress, Error Recovery, Telecommunication and Security requirements. 

As part of the EAC Certification application ES&S submitted an implementation statement for Unity 3.2.0.0. A copy 

of this statement is contained in section 7.10. Certification testing of the ES&S Unity 3.2.0.0 included a Physical 

Configuration Audit and a Functional Configuration Audit. Daily status reports were sent to ES&S Unity 3.2.0.0 

certification management staff and iBeta project test staff. These reports included project activity status, issues, 

and other relevant information. Weekly status calls were held with the EAC, EAC Reviewers and ES&S. Upon 

request, iBeta provided the EAC with information to clarify the testing and the test process and weekly status 

reports. 

2.1 Terms and Definitions

The Terms and Definitions identified below are used in this test report.

Table 4 Terms and Definitions

Term Abbreviation Definition

Absentee Ballot A paper ballot cast outside of an early voting center or 

election day polling place

Adobe Acrobat Standard v.8 & v.9 COTS software used in ESSIM for creation of Portable 

Document Format (PDF) ballot files. 

Audit Manager AM A Unity election management system audit logging 

software application for the Election Data Manager and 

Ballot Image Manager

Ballot Control - Accepts HPM option that instructs the DS200 to accept and 

tabulate overvoted, blank, primary crossovers or ballots 

with unreadable marks without alerting the voter. 

Ballot Control- Query HPM option that instructs the DS200 to return and 

query the voter when encountering an overvoted, 

blank, primary crossovers or ballots with unreadable 

marks. Voter has the option to request a new ballot or 

instruct the system to accept the ballot as is.

Ballot Control - Reject HPM option that instructs the DS200 to automatically 

reject crossover, overvoted or blank ballots. Ballots will 

not be accepted.

Ballot Marking Device BMD A device that marks a paper ballot for a voter

Ballot On Demand BOD An optional operating mode in ESSIM that is used to 

print a small quantity of election quality ES&S paper 

ballots on a COTS OKI 9600 HDN color laser printer.

Certified Information System Security 

Profession

CISSP A certification for information system security 

practitioners, indicating successful completion of the 

CISSP examination administered by the International 

Information Systems Security Certification Consortium

Central counter A type of voting system that records and reports paper 

ballots at the central count

Double Spit and Wipe Functionality on the VAT to support older ES&S optical 

scanners outside the scope of Unity 3.2.0.0

Early voting mode - A mode on the DS200 that permits ballots to be cast 
VSTL Certification # pending

Page 16 of 247 (V)2009-30Jun-001(A)

Term Abbreviation Definition

prior to election day. A flag is set in HPM to include all 

precincts for the election. The poll-worker can select a 

voter's precinct and ballot style when used in Early 

Voting or an Absentee configuration.

Election Data Manager EDM A Unity election management system software 

application to define and store jurisdiction election data 

Election Systems and Software ES&S Manufacturer of the Unity Voting System

Election management system EMS The ballot preparation and central count portions of a 

voting system.

Election Reporting Manager ERM A Unity central count software application to compile 

and report election results from Unity voting devices

Enhanced AutoCast Functionality for automatically dropping AutoMARK 

ballots into a ballot box. This functionality requires PEB 

FW v.1.70 and Auto MARK FW v.1.4. That version of 

AutoMARK firmware is not supported in Unity 3.2.0.0

Escrow Agency EAC identified repository that retains the file signature 

of the trusted build

ES&S AutoMARK Information 

Management System

AIMS A windows-based election management system 

software application to define election parameters for 

the VAT, including functionality to import election 

definition files produced by the Unity EMS and create

VAT flash memory cards

ES&S Ballot Image Manager ESSIM A Unity election management system desktop 

publishing tool to layout and format paper ballots

Executable Lines of Code eLOC Lines of code that execute functionality. Comments 

and blank lines are excluded from counts of executable 

lines of code.

Flash Memory Card FMC Portable memory that contains the election definition to 

display the ballot content on a VAT.

Full or New Code Review First time submission submitted for certification review 

or previously certified code with changes to the code so 

significant that a full review is warranted.

Graphical User Interface GUI A method of interaction with a computer which uses 

pictorial buttons (icons) and command lists controlled 

by a mouse

Hardware Programming Manager HPM A Unity election management system software 

application to import, format, and convert an election 

file and create election definitions for ballot scanning 

equipment

Help America Vote Act HAVA Legislation enacted in 2002 which includes creation of 

the EAC, federal voting standards and accreditation of 

test labs

intElect DS200 DS200 A Unity Voting System precinct count optical scanner 

paper ballot tabulator including a 12-inch touch screen 

display providing clear voter feedback and poll worker 

messaging.

Model 650 M650 A Unity Voting System central count high-speed optical 

scanner paper ballot tabulator The M650 prints results 

reports to an external printer and saves results to a zip 

disk.

National Standard Reference Library NSRL Part of NIST that provides software escrow.

National Voluntary Laboratory 

Accreditation Program

NVLAP Part of NIST that provides third-party accreditation to 

testing and calibration laboratories.

Open Primary Pick a Party (Party 

Preference)

Ballot contains all contests that the voter is eligible to 

vote for in addition to any nonpartisan contests. Voter 

only votes the partisan contests for one party but 

chooses which party in the privacy of the voting booth 

by only voting for candidates from the desired party. 

Pick a Party is where a party selection contest appears 

before the partisan section of the ballot. If the voter 

chooses a party from the party selection contest, votes 

for candidates that represent any other party are 

ignored so that the voter cannot spoil the ballot.
VSTL Certification # pending

Page 17 of 247 (V)2009-30Jun-001(A)

Term Abbreviation Definition

Precinct counter A type of voting system that records paper or electronic 

ballots at the polling place

Printer Engine Board version PEB v. The version of the firmware on the Printer Engine 

Board identifies support or non-support of Enhanced 

AutoCast and Double Spit & Wipe (v.1.70 supports) 

Single Board Computer version SBC v. Version of the Single Board Computer identifying board 

connections and chips

Trusted Build A compile and build of the source code reviewed by 

iBeta into executable code. Construction of the build 

platform and compile is performed by iBeta following 

the documented instructions of the manufacturer. A 

manufacturer's representative is present to witness the 

build. 

Technical Data Package TDP The documentation and code relating to the voting 

system, submitted by the manufacturer for review.

Universal Power Supply UPS Uninterrupted power supply

U.S. Election Assistance Commission EAC U.S. agency established by the Help America Vote Act 

of 2002 to administer Federal elections.

Voluntary Voting System Guidelines VVSG Federal voting system test standards created by the 

EAC. Eventually these will replace the VSS.

Voting System Standards VSS Federal voting system test standards, predecessor of 

the VVSG.

Voting System Test Lab VSTL Lab accredited by the EAC to perform certification 

testing of voting systems.

Voting Variations Significant variations among state election laws 

incorporating permissible ballot content, voting options 

and associated ballot counting logic 

Voter Assist Terminal VAT A ballot marking device to assist multilingual voters and 

voters with visual, aural or dexterity disabilities to vote a 

paper ballots in a private manner

Unity x.x.x.x A voting system produced by ES&S configured with 

various election software applications, DREs, optical 

scanners and ballot marking devices. The 

configuration varies for each version of Unity. 

Witness Build for Unity 3.2.0.0 The Unity 4.0.0.0 Trusted Build performed by SysTest 

Labs. iBeta shall initiate testing with this build. 

Following iBeta's performance of the Trusted Build a 

regression test will be run. 

2.2 Physical Configuration Audit

The Physical Configuration Audit (PCA) deals with the physical elements of the voting system, including the source 

code, documentation and system configuration reviews. Validation of COTs software and hardware, execution of a 

Trusted Build with the reviewed source code and installation of the executable are part of the PCA 

2.2.1 PCA TDP Source Code Review

The PCA TDP Source Code Review of ES&S Unity 3.2.0.0 was performed by SysTest Labs. ES&S petitioned for 

reuse of this review. In order to assist in making a determination of reuse the EAC instructed iBeta to audit 3% of 

the ES&S Unity 3.2.0.0 source code for assessment and recommendation of reuse of the applicable Unity v.4.0.0.0-

PCA Source Code Review conducted by SysTest. In assessing this sample iBeta reviewed the sampling to Vol. 1 

Sect. 4.2 and Vol. 2 Sect. 5 of the VSS2002. iBeta focused the review by selecting source code files and functions 

that process vote data, audit logs, and reporting. The results were recorded on Source Code Review sheets (Excel 

spreadsheets). All issues were identified in the review and logged on a Discrepancy Report. Following a peer 

review issued were identified as follows: 

Green: Non-logic issues - recommend for reuse per EAC instruction letter;

Yellow: Potential logic issues- attach issues to the recommendation letter to the EAC for their 

consideration in determination of reuse; and

Red: Confirmed logic issues - recommend 100% review to the EAC.

Any source code updates or changes received from ES&S during the test effort were compared to the code 

transferred from SysTest. All identified changes were 100% reviewed by iBeta. 
VSTL Certification # pending

Page 18 of 247 (V)2009-30Jun-001(A)

2.2.2 PCA TDP Document Review

The PCA TDP Document Review of ES&S Unity 4.0.0.0 for conformance to Vol. 2 Sect. 2 of the VSS 2002 was 

performed by SysTest Labs. ES&S petitioned the EAC for reuse. The EAC instructed iBeta to perform an 

assessment and provide a recommendation for reuse of the TDP in Unity 3.2.0.0. iBeta sampled the ES&S Unity 

3.2.0.0 documents. The sample selection included the documents identified in the SysTest issued discrepancies 

and documents needed to complete the Unity 3.2.0.0 trusted builds, a sample 3% source code review, test 

planning and test execution. Criteria for the review included confirmation that the Unity 3.2.0.0 documents 

addressed any SysTest identified document discrepancies within the scope of the Unity 3.2.0.0 test effort and the 

content provided sufficient information in order to complete the test tasks list above. Issues, which were identified 

as all or partially relevant to the Unity 3.2.0.0 scope, were transferred to iBeta's Unity 3.2.0.0 Discrepancy Report. 

Issues or parts of issues, outside this scope were excluded. Scope assessment was recorded in a review 

disposition document. iBeta confirmed the issues were valid and traced to an appropriate 2002 VSS requirement. 

iBeta reviewed the SysTest description history from the original SysTest discrepancy report and the Unity 3.2.0.0 

documents submitted by ES&S to validate resolution of the issue. 

The review of documents necessary to complete Unity 3.2.0.0 trusted builds, sample code review; test planning and 

test execution was incorporated into these tasks and recorded in the daily status. Missing content or discrepancies 

were reported in iBeta's Unity 3.2.0.0 Discrepancy Report. The Discrepancy Report was forwarded to ES&S for 

correction.

2.2.3 PCA System Configuration Review

The PCA System Configuration Review of ES&S Unity 3.2.0.0 was performed to verify the hardware and software 

configuration is consistent with the technical data package (VSS Vol. 1 Sect. 8.7.1). Reviewed results were 

recorded on PCA System Configuration Review sheets (Excel spreadsheets).

2.2.4 Witness Build and Installation

The Witness Build and Installation of the executable code (“trusted build”) for the ES&S Unity 3.2.0.0 was 

performed using the source code reviewed by SysTest and iBeta, per VSS Vol. 1 Sect. 9.6.2. Observation of the 

build was documented in the Witness of the Final Build and Code Comparison Template (Word Document).

2.3 Functional Configuration Audit

The Functional Configuration Audit was an examination of the functional aspects of the voting system. This 

included review of the ES&S Unity 3.2.0.0 submitted test documentation and execution of all VSS 2002 required 

tests.

2.3.1 FCA Test Documentation Review

iBeta initiated an assessment to identify and separate Unity v.4.0.0.0 hardware and software excluded from Unity 

3.2.0.0, SysTest test results petitioned for reuse by ES&S, and items in scope of additional testing required in the 

Unity 3.2.0.0 certification test effort. Unity 4.0.0.0 hardware and software that was excluded from Unity 3.2.0.0 was 

identified as out of scope. Open discrepancies from the SysTest testing related to Unity 3.2.0.0 system functionality 

and system changes submitted during the Unity v.4.0.0.0 test effort were identified as in scope. A comparison of 

the versions submitted in the SysTest report and those identified discrepancies for Unity 3.2.0.0 was conducted to 

confirm if the versions being submitted for Unity 3.2.0.0 matched the versions that were tested in the Unity v.4.0.0.0 

certification. If the software version number of the submitted system changes was equal to or less than the version 

identified in the SysTest report it was excluded due to the EAC acceptance of the SysTest results. If the open 

functional discrepancy was equal to the version or greater than the identified in the report it was included in the 

iBeta testing of Unity 3.2.0.0

2.3.2 FCA Functional, Accessibility, Maintainability, Accuracy & Reliability 

Tests

Functional, Accessibility, Maintainability, Accuracy and Reliability testing was performed by SysTest Labs. ES&S 

petitioned the EAC for reuse. The EAC Technical Reviewers reviewed the Functional, Accessibility, Maintainability, 

Accuracy, and Reliability test summary reports provided by SysTest on the DS-200, M650, AutoMARK VATs, 

Ballot-on-Demand printer, and Unity EMS software. The EAC approved the reuse of this testing to the requirements 

of the requirements in Vol.1 Sect. 2, 3 and 4.4 (excluding the out of scope DRE specific requirements), in 

accordance with Vol. 2 Sect. 6.

iBeta conducted a single regression end-to-end mock election to demonstrate the integrated functionality and 

processes of the ES&S Unity 3.2.0.0 for a sampling of Vol. 1 Sect. 2, 3 and 4.4, in accordance with Vol. 2 Sect. 6. 

Additionally the Volume suite of tests generally incorporated end-to-end mock elections. Issues that remained open 

from the SysTest testing were incorporated into the regression test, volume test or separate security tests. Any 
VSTL Certification # pending

Page 19 of 247 (V)2009-30Jun-001(A)

issues encountered during testing were identified in the test record and logged on a Discrepancy Report, after 

completion of peer review. . ES&S has resolved all discrepancies of the VSS2002. All submitted fixes were 

validated and recorded in the Discrepancy Report.

2.3.3 FCA Volume, Stress and Error Recovery Tests

iBeta reviewed the ES&S System Limitations Unity 3.2.0.0 document to identify relevant application and system 

limits. Based upon the system and application limits identified in this document iBeta defined and conducted a set 

of ten test cases with a single or multiple scenarios. These test cases incorporated end-to-end mock elections to 

demonstrate the ability to the system to operate at the declared limits. Additional scenarios were incorporated into 

the test cases to demonstrate the systems ability to provide an appropriate response to an overloading conditions 

exceeding the limits and recover without losing vote data. Any issues encountered during testing were identified in 

the test record and logged on a Discrepancy Report, after completion of peer review. ES&S resolved 

discrepancies of the VSS2002 or requested interpretation from the EAC if they disagreed with iBeta's findings. All 

submitted fixes were validated and recorded in the Discrepancy Report, as were any interpretations from the EAC.

2.3.4 FCA Security Tests

iBeta performed a security review of the ES&S security documentation addressing Vol. 1 Sect. 2.2.1, 2.2.3, 2.2.5 

and 6 and Vol. 2 Sect. 6.4. Based upon this review security specific tests were identified. These tests incorporated 

source code and document reviews. Functionality to meet the requirements incorporated secrecy, integrity, system 

audit, error recovery or access to the voting system. The review was either conducted or peer reviewed by an iBeta 

CISSP staff member. The tests or reviews to validate the security of Unity 3.2.0.0 were recorded in the FCA 

Security Review. ES&S resolved discrepancies of the VSS2002 or requested interpretation from the EAC if they 

disagreed with iBeta's findings. All submitted fixes were validated and recorded in the Discrepancy Report, as were 

any interpretations from the EAC.

2.3.5 FCA Hardware Environmental Tests

The SysTest's subcontractors listed in section 1 performed hardware testing of the Unity v.4.0.0.0 voting system to 

the requirements of Vol.1 Sect. 3 in accordance with Vol.2 Sect. 4. The review, analysis, testing and test results are 

contained in the test reports and engineering change assessments listed in Appendix D - FCA Reuse 

Environmental Testing Unity v.4.0.0.0 Test Documents. The EAC issued their approval for reuse of the results of 

the SysTest Environmental Hardware testing in 2-3-2009 Letter to ESS Reuse of Testing Final. In order to ensure 

that iBeta had all documentation of the Environmental Hardware test assessment and results for the Unity 3.2.0.0 

voting system, iBeta reviewed the reports to confirm they included documentation that the Unity 3.2.0.0 submitted 

hardware passed the required tests and that any failures resulting in engineering changes were documented. This 

work was performed as part of the Pre-Certification Test Activities.

At the time of initial issuance of this report hardware testing for ECO 829 and 830 is pending. If testing of these 

changes is completed prior to issuance of an Initial Decision by the EAC, this test report shall be updated to 

incorporate these test results.

2.3.6 FCA Telephony and Cryptographic Review and Tests

An examination of the Unity 3.2.0.0voting system was conducted to confirm that it does not contain wireless 

technology or use of the public networks. The results of this review were recorded in the FCA Telephony and 

Cryptographic Test Case. As a result of this review it was determined that the voting system is exempted from the 

Telephony and Cryptographic requirements of VSS Vol.1 Sect. 5 & 6. It should be noted that connection of the 

election management system PCs or laptops to a network (LAN or WAN) is outside the system configuration 

submitted for certification.
VSTL Certification # pending

Page 20 of 247 (V)2009-30Jun-001(A)

3 Voting System Identification

The System Identification stipulates the ES&S Unity 3.2.0.0 submitted for certification and the hardware, software 

and the Technical Data Package documentation used in testing

3.1 Submitted Voting System Identification

Table 5 Voting System Name and Version

Voting System Name Version

Election Systems and Software Unity 3.2.0.0

This Voting System includes the following:

Table 6 Voting System Polling Place and Central Count Hardware

Hardware OS or Firmware & Version Description 

Ballot Prep & Central Count

PC, 2.2 GHz Pentium 4 minimum, 256 

MB memory minimum, 20 GB hard drive 

minimum, CD-ROM, USB port, Flash 

memory card drive, Blank formatted 

compact Flash Memory Card (FMC) 

Windows XP Professional AIMS Personal computer for preparation 

of the election definition and writing 

media for installation on the VAT

PC, 1 GHz or faster processor, 512 MB 

RAM, 40 GB hard drive, 48x CD-ROM or 

DVD drive, 3.5-inch drive, Super VGA 

(800x600) or higher resolution video 

adapter and monitor, appropriate drivers

Windows XP Professional, SP 2 & 

SP 3

Unity Personal computer for creating the 

election definition, ballot layouts, writing 

media for installation, consolidation of 

election results and reporting 

SanDisk Imagemate CF 

Models #SDDR-91 & #SDDR-92

N/A COTS: Flash card reader/writer 

programming the VAT

Iomega Zip Drive N/A COTS: Zip Disk reader/writer for 

programming the M650 and reading 

election results

HP LaserJet 4050N Inkjet Printer N/A COTS: Printer for AM, EDM, ESSIM, 

HPM, ERM reports

Paper Ballots 

Length: 11", 14", 17" & 19"

Ovals/Inch: 3 & 4 

N/A ES&S: Ballot types supported for VAT, 

DS200, & M650 

Ballot Marking Device

ES&S AutoMARK Voter Assist Terminal

Model A100-00, HW Rev. 1.0

VAT FW: 1.3.2906, OS WinCE 

5.0.1400, Printer Engine Board 

(PEB) 1.65, Single Board 

Computer (SBC) 1.0

Accessible paper ballot marking device

with assistive technology audio output 

devices and voter inputs

Original HW: multiple cable connector 

and printed circuit boards are mounted in 

the lower portion of the VAT

ES&S AutoMARK Voter Assist Terminal

Model A200-00, HW Rev. 1.1

VAT FW: 1.3.2906, OS WinCE 

5.0.1400, Printer Engine Board 

(PEB) 1.65, Single Board 

Computer (SBC) 2.0

Accessible paper ballot marking device 

with assistive technology audio output 

devices and voter inputs

HW Change: Consolidate PCB, relocate 

PCB and cables to upper portion for 

easier maintenance

ES&S AutoMARK Voter Assist Terminal

Model A200-00, HW Rev. 1.3.1 

VAT FW: 1.3.2906

OS WinCE 5.00.19

Printer Engine Board (PEB) 1.65,

Single Board Computer (SBC) 2.5

Accessible paper ballot marking device 

with assistive technology audio output 

devices and voter inputs

HW Change: LCD replacement, ROHS 

board components, change CPU and 

Flash Chips on the SBC board FW, Win 

CE OS Bootloader for P30 flash, OS 

update to support DST and Hash check 

(Note: Hash check is not supported in 

this version of the VAT FW)

ES&S AutoMARK Voter Assist Terminal

Model A200-00, HW Rev. 1.3.1 

VAT FW: 1.3.2906

OS WinCE 5.00.19,

Printer Engine Board (PEB) 1.70,

Accessible paper ballot marking device 

with assistive technology audio output 

devices and voter inputs
VSTL Certification # pending

Page 21 of 247 (V)2009-30Jun-001(A)

Hardware OS or Firmware & Version Description 

Single Board Computer (SBC) 2.5 HW Change: PEB FW to support 

Enhanced AutoCast and Double Spit & 

Wipe (Note: Enhanced Auto Cast is not 

supported in this version of the VAT 

FW.)

VAT Kit:

(2) Compact Flash Keys

(2) AutoMARK Unit Keys

(1) Compact Flash card

(1) Printer Cartridge

User Documentation Manual CD 

(1) Headphones w/industry standard 

connector

(1) AC Power Cord

(1) @ Short/long privacy sleeves

A standard equipment and instruction kit 

provided with the VAT for security, 

power, audio and printer outputs

Foot Paddle N/A Assistive technology switch device used 

with the VAT for voter inputs

SanDisk Compact Flash Cards (256MB) N/A COTS: Storage media for the VAT

ES&S Ballot Marker Pens N/A ES&S specified pens to hand mark 

DS200 ballots

Precinct Optical Scan Equipment

intElect DS200, HW Rev. 1.2.0 DS200 FW: 1.3.10.0

(Power Management v. 1.2.0.1 

Scanner v. 2.11.0.1)

Precinct count optical scanner

intElect DS200, HW Rev. 1.2.1 DS200 FW: 1.3.10.0

(Power Management v. 1.2.0.1 

Scanner v. 2.11.0.1)

Precinct count optical scanner.

HW change: Mylar spacing tabs to 

eliminate paper jams and a changed 

battery pack resistor value R109 from 1 

M ohms to 100 k ohms

DS200 Plastic Ballot Box without a 

diverter

N/A Precinct plastic ballot box with a single 

chamber 

DS200 Steel Ballot Box without a diverter N/A Precinct steel ballot box with a single 

chamber

DS200 Steel Ballot Box with a diverter N/A Precinct steel ballot box, with diverter to 

segregate ballots into multiple chambers 

as programmed in the EMS

ES&S USB Flash (thumb) drives 

(512MB, 2 GB, 4GB, 8GB)

N/A Storage media with election definition 

and results totals for the DS200

Thermal paper rolls Paper COTS: Paper for the DS200 reports

Central Count Optical Scan Equipment 

Model 650 (M650

HW Rev. 1.1

Oval options: Right or Left

Optical light options: Green or Red

M650 FW: 2.2.2.0 A central count optical scanner with 

either a red or green optical light which

reads a right or left ballot oval

HW v.1.1 contains a new AMP board, 

option for networking, additional ferrites 

and green sensors. 

Model 650 (M650)

HW Rev. 1.2

Oval options: Right or Left

Optical light options: Green or Red

M650 FW: 2.2.2.0 A central count optical scanner with 

either a red or green optical light which 

reads a right or left ballot oval

HW v.1.2 contains a new PMS board, 

printer and additional ferrites. 

Okidata 520 Microline 9pin Printers N/A COTS: M650 report & audit log printer

Epson LQ-590 Printers N/A COTS: M650 report & audit log printer

Belkin Universal Power Supply N/A COTS: M650 back up power supply

Zip disk (100MB) N/A COTS: M650 election definition and 

results storage media

Martin Yale 400 Ballot Jogger N/A COTS: M650 Ballot stacking device 

9 1/2 X 11 - 1 ply Computer form N/A COTS: M650 Report & log paper

Other Equipment

Okidata 9600 HDN Ballot On Demand 

(BOD) printer (used with ESSIM)

N/A COTS: Color laser printer for onsite 

printing of ballots
VSTL Certification # pending

Page 22 of 247 (V)2009-30Jun-001(A)

The following is required software for Unity 3.2.0.0. Section 3.2.2 of the ES&S System Overview, Unity v.3.2.0.0, Rev 4.0; 

identifies optional COTS software. Table 9 reflects the optional software that was part of the tested configuration.

Table 7 Voting System EMS Software

Software Applications Version EMS Function Description 

Unity Ballot Prep & Central Count

Audit Manager (AM) 7.5.2.0 Election management system audit logging software 

application for the Election Data Manager and Ballot Image 

Manager

Election Data Manager (EDM) 7.8.1.0 Election management system software application to define 

and store jurisdiction election data

ES&S Ballot Image Manager (ESSIM) 7.7.1.0 Election management system desktop publishing tool to 

layout and format paper ballots

Hardware Programming Manager (HPM) 5.7.1.0 Election management system software application to import, 

format, and convert an election file and create election 

definitions for ballot scanning equipment

Election Reporting Manager (ERM) 7.5.4.0 Central count software application to compile and report 

election results from Unity voting devices

Adobe Acrobat Standard v.9 COTS creation and display of pdf files in ESSIM and BOD

Adobe Type Manager Light None specified COTS creation and display of pdf files in ESSIM and BOD

Adobe Type Basics None specified COTS creation and display of pdf files in ESSIM and BOD

RM/Cobol v.11.01 COTS Run time system, used in HPM and ERM

Commercially available and standard 

virus detection software.

COTS anit-virus software 

Norton AntiVirus 2009 for Windows XP Pro (Recommended)

AIMS 

ES&S AutoMARK Information 

Management System (AIMS) 1.3.157

1.3.157 Software to program the election database required by the 

AutoMARK VAT.

Microsoft Excel v.5 or greater COTS: Used to export text that requires translations

ScanDisk Drivers N/A COTS: Write election database to the compact flash memory

Microsoft.NET Framework None specified COTS: graphical user interface for the VAT previewer

Commercially available and standard 

virus detection software.

COTS anit-virus software 

Norton AntiVirus 2009 for Windows XP Pro (Recommended)

3.2 Voting System Test Environment

The Voting System Test Environment identifies the specific hardware and software that was used in the test 

environment. The Test Methods in Appendix D identify the specific ES&S Unity 3.2.0.0 voting system software and 

firmware used in each test.

Table 8 Voting System Hardware

Hardware OS or Version Manufacturer Description (include functional purpose 

and condition of the equipment)

Ballot Prep & Central Count

(2) Dell Optiplex GX270 computer 

desktops with monitor, keyboard &

mouse (AIMS)

SN: DNC2F51, GBGCT51

Windows XP 

Professional 

Version 2002 SP 2 

& SP 3

Dell Pre-Vote: COTS PCs for AIMS

Condition: Good

(1) Dell Latitude 600 Laptop, Model 

#PP05L

Intel Pentium Processor 1400 MHz 

587 MHz 1.00 GB Ram

SN: CN0G512486434501261

Windows XP 

Professional 

Version 2002 SP 2 

& SP 3

Dell Pre-Vote & Post-Vote: COTS laptop for the 

Unity election management system

Condition: Good

(1) Dell GX260 computer desktop 

with monitor, keyboard & mouse

SN: Tower: 7D0WL21,

Keyboard:CN07N242388422C82Q06

Monitor:CN09M55664180-2BC-0A4S

Windows XP 

Professional 

Version 2002 SP 2

& SP 3

Dell Pre-Vote & Post-Vote: COTS Unity PC for 

the Unity election management system

Condition: Good

(1) Iomega Zip Drive

SN: 1GBS2250K7

Z250USBPCMBP Iomega Pre-Vote & Post-Vote: COTS Zip Disk 

reader/writer for programming the M650 and 

reading election results

Condition: Good

SanDisk Reader/Writer

(1) Model SDDR-91 (no SN)

(3)Model SDDR- 92

N/A SanDisk Pre-Vote: COTS SanDisk reader/writer for 

programming the VAT media in AIMS

Condition: Good
VSTL Certification # pending

Page 23 of 247 (V)2009-30Jun-001(A)

Hardware OS or Version Manufacturer Description (include functional purpose 

and condition of the equipment)

SN: 0343331, 018543 & (1) w/o a SN

(1) HP LaserJet Inkjet Printer

SN:600004

4050N Hewlett 

Packard

Pre-Vote & Post-Vote: COTS Printer for AM, 

EDM, ESSIM, HPM, ERM reports

Condition: Good

Paper Ballots 

Length: 14", 19", Ovals/Inch: 3 & 4 

N/A ES&S Vote: ES&S: Ballots for testing

Test consumables

Ballot Marking Device

(1) AutoMARK Voter Assist Terminal 

A100-00

HW Rev. 1.0

SN: AM0106430376

FW Rev. 1.3.2906

OS WinCE 

5.0.1400, PEB 

1.65, SBC 1.0

ES&S Vote: A ballot marking device to assist multilingual voters and voters with visual, aural or 

dexterity disabilities to vote a paper ballots in 

a private manner

Condition: Good

(1) AUTOMARK VOTER ASSIST 

TERMINAL A200-00

HW Rev. 1.1

SN: AM0206443384

FW Rev. 1.3.2906, 

OS WinCE 

5.0.1400, PEB 

1.65, SBC 2.0

ES&S Vote: Accessible paper ballot marking device 

with assistive technology audio output 

devices and voter inputs

HW Change: Consolidate PCB, relocate 

PCB & cables to upper portion for easier 

maintenance

Condition: Good

(1) AutoMARK Voter Assist Terminal 

A200-00

HW Rev. 1.3.1

SN: AM0208470767

FW Rev. 1.3.2906, 

OS WinCE 5.00.19, 

PEB 1.65, SBC 2.5

ES&S Vote: Accessible paper ballot marking device 

with assistive technology audio output 

devices and voter inputs

HW Change: LCD replacement, ROHS 

board components, change CPU & Flash 

Chips on the SBC board FW, Win CE OS 

Bootloader for P30 flash, OS update to 

support DST & Hash check (Note: Hash 

check is not supported in this version of VAT

FW)

Condition: Good

ECO 329, 330, 354, 759 inclusion verified by 

iBeta inspection

(1) AutoMARK Voter Assist Terminal 

A200-00, HW Rev. 1.3.1

SN: AM0208470815

FW Rev. 1.3.2906, 

OS WinCE 5.00.19, 

PEB 1.70, SBC 2.5

ES&S Vote: Accessible paper ballot marking device 

with assistive technology audio output 

devices and voter inputs

HW Change: PEB FW to support Enhanced 

AutoCast and Double Spit & Wipe (Note: 

Enhanced Auto Cast is not supported in this 

version of the VAT FW.)

Condition: Good

ECO 329, 330, 354 inclusion verified by 

iBeta inspection

(1) AutoMARK Voter Assist Terminal 

A200-00

HW Rev. 1.3.0

SN: AM0206462702

OS WinCE 5.00.17, 

PEB 1.70, SBC 2.0

ES&S Note: This configuration was not submitted 

as part of Unity 3.2.0.0; however it was used

in environmental test report #080521-1251A 

on 6/11/08

Condition: Good

ECO's 329, 330, 759, & 761verified by iBeta 

inspection

(1) AutoMARK Voter Assist Terminal 

A300-00

HW Rev. 1.4.0

SN: AM0307420125 

FW: 1.4.2970, 

WinCE OS 5.00.17, 

PEB 1.70, SBC 2.0

ATS for 

Premier

This unit was inspected to confirm inclusion 

of ECO 354 per test report 070730-1165.

Condition: Good

VAT Kit:

(2) Compact Flash Keys

(2) AutoMARK Unit Keys

(1) Compact Flash card

(1) Printer Cartridge

User Documentation Manual CD 

(1) Headphones w/industry standard 

N/A ES&S Vote: A standard equipment and instruction 

kit provided with each VAT for security, 

power, audio and printer outputs

Most items are COTS
VSTL Certification # pending

Page 24 of 247 (V)2009-30Jun-001(A)

Hardware OS or Version Manufacturer Description (include functional purpose 

and condition of the equipment)

connector

(1) AC Power Cord

(1) @ Short/long privacy sleeves

Foot Paddle (no identification) N/A Unidentified Vote: Assistive technology switch device 

used with the VAT for voter inputs

Condition: Good

Multiple Compact Flash Cards 

(256MB)

N/A SanDisk Vote: COTS Storage media for the VAT

Condition: Good

ES&S Ballot Marker Pens N/A ES&S Vote: ES&S specified pens to hand mark 

DS200 ballots, Condition: Good

Precinct Optical Scan Equipment

(2) intElect DS200

HW Rev. 1.2.0

SN: ES0107360007, ES0107370002

FW: 1.3.10.0

Power Mgmt v. 

1.2.0.1

Scanner v. 2.11.0.1

ES&S Vote: A Unity Voting System precinct count 

optical scanner paper ballot tabulator 

including a 12-inch touch screen display 

providing clear voter feedback and poll 

worker messaging, Condition: Good

iBeta observed removal of the modem cards.

(1) intElect DS200

HW Rev. 1.2.1

SN: ES0107370025

FW: 1.3.10.0

Power Mgmt v. 

1.2.0.1

Scanner v. 2.11.0.1

ES&S Vote: A Unity Voting System precinct count 

optical scanner paper ballot tabulator 

including a 12-inch touch screen display 

providing clear voter feedback and poll 

worker messaging

v.1.2.1 change: Mylar spacing tabs to 

eliminate paper jams and a changed battery 

pack resistor value R109 from 1 M ohms to 

100 k ohms

Condition: Good

iBeta observed removal of the modem cards.

(1) intElect DS200, HW Rev. 1.2.1

SN: ES0107380927

N/A ES&S Pending testing for ECO829 and ECO830 

by Criterion.

Condition: Good

(1) DS200 Plastic Ballot Box without 

a diverter

N/A ES&S Vote: Precinct plastic ballot box with a single 

chamber

Condition: Good

(1) DS200 Steel Ballot Box without a 

diverter

N/A ES&S Vote: Precinct steel ballot box with a single 

chamber

Condition: Good

(1) DS200 Steel Ballot Box with a 

diverter

N/A ES&S Vote: Precinct steel ballot box, with diverter 

to segregate ballots into multiple chambers 

as programmed in the EMS

Condition: Good

Multiple ES&S USB Flash (thumb) 

drives (512MB, 2 GB, 4GB, 8GB)

N/A Belkin & 

SanDisk for 

ES&S 

Pre-Vote & Vote: Storage media with 

election definition and results totals for the 

DS200

Condition: Good

Multiple Thermal paper rolls N/A NCR Vote: COTS: Paper for the DS200 reports. 

Condition: Good

Central Count Optical Scan 

Equipment

(2) Model 650 (M650)

HW Rev. 1.2

SN: 7003 (Optical Red/ Left Oval)

SN:102 7011 (Optical Green/Left 

Oval)

v.2.2.2.0 ES&S Post-Vote: A Unity Voting System central 

count high-speed optical scanner paper 

ballot tabulator The M650 prints results 

reports to an external printer and saves 

results to a zip disk.

Condition: Good

iBeta verified no network card was installed

(1) Model 650 (M650)

HW Rev. 1.1

SN: 2406 8013 (Optical Green/Right 

Oval)

v.2.2.2.0 ES&S Post-Vote: A Unity Voting System central 

count high-speed optical scanner paper 

ballot tabulator The M650 prints results 

reports to an external printer and saves 

results to a zip disk

Condition: Good
VSTL Certification # pending

Page 25 of 247 (V)2009-30Jun-001(A)

Hardware OS or Version Manufacturer Description (include functional purpose 

and condition of the equipment)

iBeta verified no network card was installed

(4) Okidata 520 Microcline 9pin 

Printers

SN: 204A2005641, 407D4011099, 

407D4010960, 407D4010894

520 OkiData Post-Vote: COTS M650 results report & audit 

log printer

Condition: Good

(2) Epson LQ-590 Printers

SN: FSQY093447, FSQY094255

LQ-590 Epson Post-Vote: COTS M650 results report & audit 

log printer. Condition: Good

(3) Belkin Universal Power Supply

SN: 20V06516249WE, 

20V06516248WE, 20V06516228WE

N/A Belkin Post-Vote: COTS: M650 back up power 

supply. Condition: Good

Zip disk (100MB) N/A Iomega Pre-Vote & Post-Vote: COTS Storage media 

with election definition and results totals for 

M650. Condition: Good

Ballot Jogger 400 Martin Yale Post-Vote: COTS Device to shake ballots 

into a neat stack for placement in the M650 

ballot feeding tray. Condition: Good

9 1/2 X 11 - 1 ply Computer form

paper

N/A None 

identified

Post-Vote: COTS Report and Audit log 

printing for the M650. Condition: Good

Other Equipment

None

Table 9 Voting System Software

Software Version Manufacturer Description (Identify COTS)

Ballot Prep & Central Count

Audit Manager (AM) 7.5.2.0 ES&S Election management system audit logging 

software application for the Election Data 

Manager and Ballot Image Manager

Election Data Manager (EDM) 7.8.1.0 ES&S Election management system software 

application to define and store jurisdiction 

election data

Ballot Image Manager (ESSIM) with 

Ballot On Demand (BOD)

7.7.1.0 ES&S Election management system desktop 

publishing tool to layout and format paper 

ballots

Hardware Programming Manager 

(HPM)

5.7.1.0 ES&S Election management system software 

application to import, format, and convert an 

election file and create election definitions for 

ballot scanning equipment

Election Reporting Manager (ERM) 7.5.4.0 ES&S Central count software application to compile 

and report election results from Unity voting 

devices

AutoMARK Information Management 

System (AIMS)

1.3.157 ES&S Software to program the election database 

required by the AutoMARK VAT.

Adobe Acrobat Standard 8 & 9 Adobe 

Systems

COTS supports creation and display of pdf files 

in ESSIM & BOD

Adobe Type Manager 4.1 Adobe 

Systems

COTS supports creation and display of pdf files 

in ESSIM & BOD

Excel (Office Professional 2003) 2003 Microsoft COTS supports creation and display of pdf files 

in ESSIM & BOD

RM/Cobol v11.01 Runtime System 11.01 RM/Cobol COTS supports ERM and HPM

Other COTS

02Micro Smartcard Driver 2.26 02Micro 

Electronics

COTS driver for the Unity PC

Access 2002 Runtime 10.0.2627.01 Microsoft COTS supports runtime environment on the 

AIMS PC 

ATI Software Uninstall Utility 6.14.10.1014 ATI COTS: utility application for the Unity PC

ATI Control Panel 6.14.10.5.73 ATI COTS: utility application for the Unity PC

ATI Display Driver 8.20 ATI COTS: utility application for the Unity PC

Broadcom Gigibit Integrated 

Controller

9.02.06 Broadcom COTS driver for the Unity PC

C-Major Audio 42xx SigmaTel COTS driver update software utility
VSTL Certification # pending

Page 26 of 247 (V)2009-30Jun-001(A)

Software Version Manufacturer Description (Identify COTS)

Conexant D480 MDC v.92 Modem 92 Unknown COTS: modem drivers (modem was only 

removed from the DS200)

Digital audio drivers No version Soundmax COTS: sound drivers for Unity & AIMS PC

Omnidrive USB Professional No version Omnidrive COTS Unity PC USB reader driver

PCM v2.01 2.0.1 CSM GmbH 

Filderstadt

COTS driver for Unity PC

PRO Network Adapters & Drivers No version Intel COTS: drivers for the Unity & AIMS PC

SanDisk TransferMate No Ver. SanDisk COTS: drivers to write compact flash AIMS PC 

SQL Server Desktop Engine 

(AIMS_SQLS)

8.00.761 Microsoft COTS AIMS PC 

SQLXML 3.0 service 

pack 3

Microsoft COTS: XML support for Unity & AIMS PC

Windows XP XP service 

pack 3

Microsoft COTS: OS for Unity & AIMS PCs & laptop

Windows Internet Explorer 7 Microsoft COTS: AIMS & Unity PC 

ES&S does not want Internet Explorer to 

be run on the election System PCs. 

However, Internet Explorer must be 

resident on the PC to contain the latest 

security updates.

Access 2002 Runtime 10.0.2627.01 Microsoft COTS supports runtime environment on the 

AIMS PC 

.NET Framework 1.1 Microsoft COTS AIMS PC 

Norton AntiVirus 11.0.2 Symantec 

Corporation

COTS: AIMS and Unity PC

Polling Place

VAT 1.3.2906 ES&S Firmware for the Voter Assist Terminal

DS200 1.3.10.0 ES&S Firmware for the intlElect DS200 scanner

Central Count Optical Scan 

Equipment

Model 650 v.2.2.2.0 ES&S Firmware for the Model 650 central scanner

Table 10 Voting System Technical Data Package Documents

Title Version Date Author

System Security Test Cases 5.0 06/16/09 ES&S

System Security Test Procedure 4.0 06/16/09 ES&S

Printed Wiring Assembly, AUTOMARK 2 drawing no. 620118-6000 A None ES&S

BOM 621118-6001A None No date Applied Data 

Systems

BOM 621118-6002 VA1 None No date Applied Data 

Systems

Automark-2 Schematic Rev A None ES&S

Election Systems & Software, Inc. Indented Bill of Material None 05/15/08 ES&S

Adobe Installation Reference Guide None 05/28/08 ES&S

AutoMARKTM INFORMATION MANAGEMENT SYSTEM (AIMS) 

TECHNICAL DATA PACKAGE

None None ES&S

AutoMARKTM Information Management System TECHNICAL DATA 

PACKAGE TABLE OF CONTENTS

None 06/16/09 ES&S

AIMS Requirements Trace Matrix None 06/16/09 ES&S

AutoMARK Information Management System AIMS Release Notes 10.0 06/16/09 ES&S

AutoMARK Information Management System (AIMS) System Overview 5.0 06/16/09 ES&S

AutoMARK Information Management System (AIMS) System 

Functionality

5.0 06/16/09 ES&S

AIMS Hardware Specifications 4.0 06/16/09 ES&S

Compact Flash Memory Card Design Specifications 4.0 06/16/09 ES&S

AutoMARK Information Management System (AIMS) Programming 

Specifications Details

3.0 06/16/09 ES&S

AutoMARK Information Management System (AIMS) Software Design 

Specifications

5.0 06/16/09 ES&S

AutoMARK Information Management System Election Official‟s Guide 16.0 06/12/09 ES&S
VSTL Certification # pending

Page 27 of 247 (V)2009-30Jun-001(A)

Title Version Date Author

AutoMARK Information Management System (AIMS) Operations 

Procedures

5.0 06/16/09 ES&S

AutoMARK Information Management System (AIMS) System Security 

Specifications

5.0 06/16/09 ES&S

AIMS Quality Assurance Policy & Procedures 5.0 06/16/09 ES&S

AIMS Quality Assurance Test Cases 6.0 06/16/09 ES&S

AIMS Quality Assurance Test Procedures 4.0 06/16/09 ES&S

AIMS Configuration Management Plan 4.0 06/16/09 ES&S

AIMS System Change Notes 25.0 06/16/09 ES&S

Audit Manager 7.5.0.0 Relational Model None None ES&S

Setting the Date and Time on an AutoMARK None 05/13/08 ES&S

AutoMARK Component Storage and Handling Procedure 4.0 06/16/09 ES&S

ATS Configuration Management Policy 4.0 06/16/09 ES&S

Corrective Action Control Log 1.0 None ES&S

Design Review Attendance Sheet 1.0 None ES&S

Design Review Minutes 1.0 None ES&S

AutoMARK Design Review Policy 4.0 06/16/09 ES&S

ATS Document Change Order 1.0 None ES&S

AutoMARK Document Change & Issue Procedure 5.0 06/16/09 ES&S

Document Change Pending Re-Release 1.0 None ES&S

AutoMARK Document Control Policy 4.0 06/16/09 ES&S

ATS Employee Training Procedure 4.0 06/16/09 ES&S

Engineering Change Order/Change Request Form 1.0 None ES&S

AutoMARK Engineering Change Request/Change Order Process 5.0 06/16/09 ES&S

AutoMARK Engineering Development Policy 4.0 06/16/09 ES&S

ATS Purchasing Procedure 4.0 06/16/09 ES&S

AutoMARK Quality Assurance Policy 4.0 06/16/09 ES&S

AutoMARK Quality System Audit Process 4.0 06/16/09 ES&S

AutoMARK Receiving Procedure 4.0 06/16/09 ES&S

AutoMARK Software and Hardware Release Process 9.0 06/16/09 ES&S

System Bug Report Form 1.0 None ES&S

AutoMARK System Report (Bug Reporting) Procedure 4.0 06/16/09 ES&S

Audit Manager Checklist-Election Day Training Manual None 08/2007 ES&S

Build Procedure Audit Manager 3.1 03/24/09 ES&S

ATS Quality System Master Audit Schedule 1.0 09/02/08 ES&S

Ballot Image Processing Specifications 5.0 06/16/09 ES&S

AutoMARKTM Ballot Scanning and Printing Specification 4.0 06/16/09 ES&S

AutoMARK Configuration Management Plan (AQS) -13-5020-000-F 5.0 06/16/09 ES&S

AutoMARK Driver API Specification 4.0 06/16/09 ES&S

Automark Environmental Test Cases 6.0 06/16/09 ES&S

AutoMARK Environmental Test Plan 6.0 06/16/09 ES&S

AutoMARK Environmental Test Procedures 6.0 06/16/09 ES&S

AutoMARK Graphical User Interface Design Specifications 4.0 06/16/09 ES&S

Initial Software Installation Procedure 5.0 06/16/09 ES&S

ES&S AutoMARK Jurisdiction Guide 9.0 05/07/09 ES&S

AutoMARK Operating Software (AMOS) Design Specifications 4.0 06/16/09 ES&S

AutoMARK Operations and Diagnostic Log Specifications 5.0 06/16/09 ES&S

Operations and Diagnostic Log Test Cases 5.0 06/16/09 ES&S

Operations & Diagnostic Log Test Procedures 5.0 06/16/09 ES&S

Personnel Deployment and Training Requirements 5.0 06/16/09 ES&S

ES&S AutoMARK Poll Worker's Guide 10.0 05/07/09 ES&S

AutoMARK Programming Specifications Details 6.0 06/16/09 ES&S

ATS Quality System Procedures (QSP) Master List 1.0 09/02/08 ES&S

AutoMARK Rapid Application Development Methodology (RAD) 5.0 06/16/09 ES&S

AutoMARK 3010 VAT Release Notes 13.0 06/16/09 ES&S

AutoMARKTM Requirements Trace Matrix 3.0 06/16/09 ES&S

AutoMARK Software Design Specifications 5.0 06/16/09 ES&S

AutoMARK Software Development Environment Specifications 5.0 06/16/09 ES&S

AutoMARK Software Diagnostics Specification 5.0 06/16/09 ES&S

Software Standards Specification 5.0 06/16/09 ES&S

AutoMARK Software Quality Assurance Test Plan 5.0 06/16/09 ES&S
VSTL Certification # pending

Page 28 of 247 (V)2009-30Jun-001(A)

Title Version Date Author

Software Quality Assurance Test Cases 7.0 06/16/09 ES&S

Software Quality Assurance Test Procedures 5.0 06/16/09 ES&S

AutoMARKTM System Change Notes 91.0 06/16/09 ES&S

AutoMARK System Functionality 6.0 06/16/09 ES&S

ES&S System Installation and Maintenance Guide 10.0 06/16/09 ES&S

AutoMARK System Introduction 4.0 06/16/09 ES&S

System Level Test Cases 6.0 06/16/09 ES&S

AutoMARK System Level Test Plan 6.0 06/16/09 ES&S

AutoMARK System Level Test Procedures 5.0 06/16/09 ES&S

System Security Test Procedure 4.0 06/16/09 ES&S

AutoMARK System Overview 5.0 06/16/09 ES&S

AutoMARKTM TECHNICAL DATA PACKAGE TABLE OF CONTENTS None 09/02/08 ES&S

ES&S AutoMARK Voter's Guide 9.0 06/16/09 ES&S

AUTOMARKTM EMBEDDED DATABASE INTERFACE SPECIFICATION 6.0 06/16/09 ES&S

AutoMARK VAT – Printer Engine Board Firmware Compilation 

Instructions

3.0 06/16/09 ES&S

No Title (VAT Configuration) None No Date ES&S

AutoMARK System Hardware Specification 4.0 06/16/09 ES&S

AutoMARK VAT Firmware and Hardware Installation Instructions 3.0 06/04/09 ES&S

AutoMARK VAT Software and Firmware Compilation Instructions 13.0 06/16/09 ES&S

ES&S Ballot Production Handbook None 07/17/07 ES&S

Ballot Data File Specification Unity Version 4.0.0.0 1.0 04/30/07 ES&S

ES&S Ballot On Demand Printer Setup and Printing Procedures Version 

Release 7.7.0.0 Okidata part number 58273508

None 08/22/08 ES&S

Ballot Set Collection File Specification Unity Version 4.0.0.0 1.0 04/30/07 ES&S

Automark Technical Systems Integration & Testing Bug Report 1.0 None ES&S

Development Practices and Coding Standards Election Systems and 

Software Version Number 2.3.0.0

2.3 07/11/08 ES&S

DS 200 Part list None 05/12/08 ES&S

DS200 Election Day Checklist Version Number 1.3.7.0 None 05/09/08 ES&S

ES&S DS200 Power Management Board Validation None 08/01/08 ES&S

ES&S DS200 Scanner Board Dump Compare Hardware Version 1.2.1.0 

Firmware Version 2.0.0.0

None 09/26/08 ES&S

DS200 Test Cases Unity 4.0 Version 1.3.7.0 None 06/13/08 ES&S

Engineering Specification -Model PW-080A2-1Y24AP (G) -(DS-200 -

ferrite molded power supply)

A 02/03/09 Wall Industries

Engineering Change of Order documentation None None ES&S

Election Data Manager (EDM) Checklist-Election Day Training Manual None 08/2007 ES&S

Election Data Manager Test Case Specifications Software Version 

7.8.0.0

None 10/27/08 ES&S

Election Data Manager 7.8.0.0 Election Tables Relational Model None None ES&S

EDMXML File Specification None 06/15/07 ES&S

EL80 File Specification None None ES&S

Election Reporting Manager Pre-Election Day Training Manual Version 

Number 7.5.x

None 06/22/08 ES&S

Election Reporting Manager / ERM Product Test Cases Unity 4.0 

Version 7.5.2.0

None 10/23/08 ES&S

ESS Hardware Acceptance Checklists None None ES&S

ES&S License Agreements Software Development None 06/10/05 ES&S

ESS Sample Deliverable Timeline None None ES&S

ES&S Software/Firmware Acceptance 1.0 02/25/08 ES&S

ESSCRYPT Functional Specification Version 1.8.1.0 None 11/16/07 ES&S

ESSDECPT Functional Specification Version 1.8.1.0 None 11/16/07 ES&S

ESS Hardware Revision History None 11/02/07 ES&S

ESS Image Manager (ESSIM) Checklist-Election Day Training Manual None 08/2007 ES&S

ESS Image Manager Test Case Specification Software Version 7.7.0.0 

Test Case 2.0

None 10/17/08 ES&S

ESSXML File Specification None 04/30/07 ES&S

Hardware Revision Description 1.0 08/27/07 ES&S

Hardware Programming Manager (HPM) Checklist-Election Day Training 

Manual

None 08/2007 ES&S
VSTL Certification # pending

Page 29 of 247 (V)2009-30Jun-001(A)

Title Version Date Author

Hardware Programming Manager Test Case 1.0 Unity Version 4.0 None 06/06/08 ES&S

Letter- EAC Decision on Request for Interpretation 2008-05 Voting 

Equipment Durability

None 09/23/08 ES&S

Interface (IFC) File Specification None None ES&S

ISO Certification Pivot None None ES&S

Ricoh Electronics Quality Manual 4.0 07/06/06 ES&S

Jurisdiction Security Procedures Version 1.0.0.1 None 05/09/08 ES&S

Language Data File Specification None 04/30/07 ES&S

Setting the Date and Time on a Model 100 Scanner None 05/13/08 ES&S

Setting the Date and Time on a Model 650 Scanner None 05/13/08 ES&S

Model 650 Output File Specification None None ES&S

Setting the Machine ID on a Model 650 Scanner None 05/13/08 ES&S

Model 650 Test Case Specification Firmware Version 2.2.1.0 Hardware 

Version 1.1 Test Case 1.0

None 10/17/08 ES&S

OmniDrive USB/USB2 Installation Guide 1.0 05/20/08 ES&S

Open Source & 3rd Party Code Management Procedure None 01/03/06 ES&S

Election Results Export (EXP) Election Day Checklist None 02/29/08 ES&S

Model 650 Election Day Checklist Version Number 2.2.1.0 1.0 02/29/08 ES&S

Model 650 Pre-Election Day Checklist Version Number 2.2.1.0 1.0 02/29/08 ES&S

Model 650 Handout A: Setting the Date & Time None 02/29/08 ES&S

Product Release Request None None ES&S

Quality Assurance Manual K 03/17/05 ES&S

QMI Management Systems Registration Certificate of Registration None None ES&S

QMI Certificate of Registration None None ES&S

RM/COBOL® Installation Guide 1.1 05/20/08 ES&S

ES&S Software Validation Phase I Create ES&S Preliminary Definition 

File

1.1 04/10/08 ES&S

ES&S Software Validation Phase II-Create ES&S Package Definition 

File-Using the ES&S Software Validation Utility

1.2 04/10/08 ES&S

ES&S Software Validation Phase III-ES&S Software Validation 

Procedure-Using the ES&S Software Validation Utility

1.1 04/10/08 ES&S

Unity 3.2.0.0 Source Code Repository & Version None No date ES&S

ES&S Configuration Management Plan 4.0 05/21/09 ES&S

System Change Notes 1.0 11/25/08 ES&S

Election Systems and Software System Overview Unity 3.2.0.0 6.0 06/23/09 ES&S

2002 VSS Supported Functionality Declaration Unity 3.2.0.0 None 10/29/08 ES&S

Election Systems & Software System Limitations Unity v. 3.2.0.0 6.0 05/08/09 ES&S

ES&S TDP Organization and Abstract Rev 1.0 No Date ES&S

Requirements of the 2005 VVSG Trace to Vendor Testing and Technical 

Data Package

None 06/17/09 ES&S

Election Systems & Software ES&S Quality Assurance Program 

Manufacturing Unity v. 3.2.0.0

3.0 06/16/09 ES&S

Election Systems & Software ES&S Quality Assurance Program 

Software and Firmware Unity v. 3.2.0.0

3.0 06/16/09 ES&S

Audit Manager 7.5.0.0 Relational Model None None ES&S

Election Data Manager 7.8.0.0 Election Tables Relational Model None None ES&S

Election Systems & Software ES&S Software Design Specifications 

Audit Manager Unity v. 3.2.0.0

2.0 06/15/09 ES&S

Election Systems & Software ES&S Software Design Specifications 

DS200 Unity v. 3.2.0.0

2.0 06/15/09 ES&S

Election Systems & Software ES&S Software Design Specifications 

Election Data Manager (EDM) Unity v. 3.2.0.0

3.0 06/15/09 ES&S

Election Data Manager 7.8.0.0 County Tables Relational Model None None ES&S

ES&S Software Design Specifications ERM Unity v. 3.2.0.0 3.0 06/19/09 ES&S

ES&S Software Design and Specification ES&S Ballot Image Manager 

(ESSIM) Unity v. 3.2.0.0

3.0 06/16/09 ES&S

ES&S Software Design and Specification Hardware Programming 

Manager (HPM) Unity v. 3.2.0.0

2.0 06/15/09 ES&S

ES&S Software Design Specifications Model 650 Unity v. 3.2.0.0 2.0 04/05/09 ES&S

ES&S System Functionality Description Model 650 Unity v. 3.2.0.0 2.0 04/03/09 ES&S

Election Systems & Software ES&S System Functionality Description 2.0 06/15/09 ES&S
VSTL Certification # pending

Page 30 of 247 (V)2009-30Jun-001(A)

Title Version Date Author

Audit Manager Unity v. 3.2.0.0

Election Systems & Software ES&S System Functionality Description 

DS200 Unity v. 3.2.0.0

2.0 06/15/09 ES&S

ES&S System Functionality Description EDM Unity v. 3.2.0.0 3.0 04/03/09 ES&S

ES&S System Functionality Description ERM Unity v.3.2.0.0 5.0 06/23/09 ES&S

ES&S System Functionality Description ES&S Ballot Image Manager 

(ESSIM) Unity v. 3.2.0.0

2.0 04/09/09 ES&S

ES&S System Functionality Description Hardware Programming 

Manager (HPM) Unity v. 3.2.0.0

2.1 04/09/09 ES&S

ES&S System Hardware Specification Model 650 1.0 11/1742008 ES&S

ES&S System Hardware Specification DS200 Unity v. 3.2.0.0 2.0 04/06/09 ES&S

Election Systems & Software ES&S System Hardware Specification 

Model 650 Unity v. 3.2.0.0

2.0 06/15/09 ES&S

ES&S DS200 System Maintenance Manual Hardware Version 1.2.0 

Firmware Version 1.3.10.0

None 06/23/09 ES&S

ES&S Model 650 System Maintenance Manual None 01/30/09 ES&S

ES&S Audit Manager System Operations Procedure Version Release 

7.5.2.0

None 05/26/09 ES&S

ES&S DS200 System Operations Procedures Hardware Version 1.2.1 

Firmware Version 1.3.10.0

None 06/19/09 ES&S

ES&S Election Data Manager System Operating Procedures Release 

Version 7.8.1.0

None 05/26/09 ES&S

ES&S Election Reporting Manager Release Version 7.5.4.0 None 06/24/09 ES&S

ES&S Image Manager System Operations Procedure Version Release 

7.7.1.0

None 04/03/09 ES&S

ES&S Hardware Programming Manager System Operations Procedure 

Version Release 5.7.1.0

Unity 

v.3.2.0.0

05/11/09 ES&S

ES&S Model 650 System Operation Procedures Release Firmware 

Version 2.2.2.0 Hardware Version 1.1 and 1.2

None 05/29/2009 ES&S

ES&S System Security Specification Version Release 4.0.0.0 None 03/27/09 ES&S

Hardening Procedures for the Election Management System PC None 06/24/09 ES&S

Election Systems & Software Unity System Test Plan Version 3.2.0.0 2.0 06/16/09 ES&S

Audit Manager Test Case Specifications Software Version 7.5.0.0 Test 

Case 1.0

None 02/23/09 ES&S

ES&S Personnel Deployment and Training Recommendations 1.0 11/21/08 ES&S

DS200 Election Day Checklist Version Number 1.3.7.0 None 05/09/08 ES&S

DS200 Pre-Election Day Checklist Version Number 1.3.7.0 None 11/20/08 ES&S

Election Results Export (EXP) Election Day Checklist Version Number 

4.0.0.0

None 02/29/08 ES&S

Model 650 Election Day Checklist Version Number 2.2.1.0 None 02/29/08 ES&S

Model 650 Pre-Election Day Checklist Version Number 2.2.1.0 None 02/29/08 ES&S

M650 Setting Date Time None 02/20/09 ES&S

DS200 Precinct Ballot Scanner Election Day Training Manual Version 

Number 1.3.10.0

None 06/22/09 ES&S

DS200 Precinct Ballot Scanner Pre-Election Day Training Manual 

Version Number 1.3.10.0

None 06/22/09 ES&S

Election Data Manager Training Manual None 02/20/09 ES&S

ESSIM Training Manual Version Number 7.7.x None 02/20/09 ES&S

Election Results Export (EXP) Election Day Checklist Version Number 

4.0.0.0

None 02/29/08 ES&S

Hardware Program Manager Training Manual None 02/20/09 ES&S

Model 650 Election Day Checklist Version Number 2.2.1.0 None 02/29/08 ES&S

Model 650 Pre-Election Day Checklist Version Number 2.2.1.0 None 02/29/08 ES&S

U3200_TRN00_M650_SettingDateTime None 02/20/09 ES&S

Election Systems & Software ES&S Personnel Deployment and Training 

Recommendations Unity v. 3.2.0.0

2.0 06/16/09 ES&S

Installation Guide Windows XP On Dell Optiplex GX520 1.2 05/21/08 ES&S

Verify DS200 Operating System Using Open SSL None 09/19/08 ES&S

EAC Application Documents

VSTL Source Code Status Report None None ES&S

AIMS Requirements Trace Matrix Unity 

v.3.2.0.0

9/2/08 ES&S
VSTL Certification # pending

Page 31 of 247 (V)2009-30Jun-001(A)

Title Version Date Author

AutoMARK Requirements Trace Matrix Unity 

v.3.2.0.0

9/2/08 ES&S

Requirements of the 2005 VVSG Trace to Vendor Testing and Technical 

Data Package

Unity 

v.3.2.0.0

10/29/08 ES&S

2002 VSS Supported Functionality Declaration Unity 3.2.0.0 Unity 

v.3.2.0.0

10/29/08 ES&S

Unity 3.2.0.0 Implementation Statement Unity 

v.3.2.0.0

10/29/08 ES&S

Unity 3.2.0.0 Unity 

v.3.2.0.0

No date ES&S

Unity 3.2.0.0 Application Unity 

v.3.2.0.0

10/29/09 ES&S

Build and Installation

AIMSCRYPT Build Environment Compile-Install Guide 1.0 5/23/09 ES&S

AutoMARK VAT Printer Engin Board Firmware Compilation Instructions 2 06/09/09 ES&S

AutoMARK VAT Firmware and Hardware Installation Instructions version 

1.3.157

3 06/01/09 ES&S

AutoMARK VAT Software and Firmware Compilation Instructions 16.0 05/20/09 ES&S

Build Procedure ESS Linux 7.2 Beyond Linux From Scratch (BLFS) 3.1 04/22/09 ES&S

Build Procedures DS200 Firmware 1.2 06/25/09 ES&S

Build Procedure Detail DS200 Firmware 3.6 06/25/09 ES&S

Pre-Build Task List DS200 Firmware Unity 3.2.0.0 1.1 06/25//09 ES&S

Build Procedure DS200 TOS 1.0 04/30/09 ES&S

Build Procedure ESS Linux 6.2 Target Operating System (TOS) 3.1 04/22/09 ES&S

Pre-Build Task List DS200 TOS Unity 3.2.0.0 1.0 05/13/09 ES&S

Build Procedure ESS Linux 6.2 Linux From Scratch (LFS) 2.1 04/20/09 ES&S

Install Procedure Cypress EZ-USB 1.0 04/13/09 ES&S

Build Procedure DS200 Ancillary Devices Unity 3.2.0.0 1.0 04/30/09 ES&S

Pre-build Task LIst DS200 Ancillary Devices Unity 3.2.0.0 1.0 05/13/09 ES&S

Install Prodecure IAR Embedded Workbench 1.0 04/13/09 ES&S

Install Procedure Keil uVision3 Development Tools .1.0 4/13/09 ES&S

Installation Guild Microsoft Windows XP with Service Pack 3 on Dell 

Optiplex GX520

1.0 03/31/09 ES&S

Pre-build Task LIst Model 650 Unity 3.2.0.0 1.0 05/13/09 ES&S

Build Procedure Model 650 1.0 05/12/09 ES&S

Installation Guide QNX Software Systems, Ltd. QNX 4.25 Product Suite 

May 2001 1.0 05/12/09

ES&S

Installation Guild Avocet Systems Inc. ADX-Z180 Version 5.2 1.0 04/30/09 ES&S

Installation Guild Sequiter Software CodeBase 6.5 Release 3 2.2.1 04/21/09 ES&S

Installation Guide Business Objects Crystal Reprots 9 Developer, Full 

Product with Hot Fixes

2.2.2 04/29/09 ES&S

Installation Guild Install Shield Express, 2.12 2.21 11/30/08 ES&S

Installation Guide InstallShield Professional, 7.01 2.2 03/23/09 ES&S

Installation Guide CSM GmbH PC=Card SDK v.2.20 1.0 04/30/09 ES&S

Installation Guild Liant RM/COBOL Developer 11.01 Liant RM/COBOL 

WOW Extensions 11.01

3.3.2 04/29/09 ES&S

Election Systems & Software Build Procedure Unity Software 

Applications Unity 3.2.0.0

1.5 06/16/09 ES&S

Election Systems & Software Build Procedure Unity Software 

Applications Unity 3.2.0.0 (Supplimental Build A)

1.0 06/25/09 ES&S

Pre-build Task List Unity Software Applications Unity 3.2.0.0 1.0 05/13/09 ES&S

Pre-build Task List Unity Software Applications (Supplemental Build A) 

Unity 3.2.0.0

1.0 06/19/09 ES&S

Installation Guide Young Dynamic Software vbAdvance 3.1 1.3.1 04/30/09 ES&S

Installation Guide Microsoft Visual Studio 6.0, Enterprise Edition with 

Service Pack 5

2.0.1 04/30/09 ES&S

Installation Guide Microsoft Visual Studio 2005 Professional Edition 2.0 04/21/09 ES&S

Installation Guide Microsoft Windows XP On Corsair PC 1.0 04/22/09 ES&S

Engineering Change Orders

ECO 802 wREV ID Unity 

v.3.2.0.0

12/05/08 ES&S

ECR037120408 Unity 12/04/08 ES&S
VSTL Certification # pending

Page 32 of 247 (V)2009-30Jun-001(A)

Title Version Date Author

v.3.2.0.0

ECO 803 wRev ID - cable Unity 

v.3.2.0.0

12/15/08 ES&S

ECR033120108 Unity 

v.3.2.0.0

12/01/09 ES&S

ECO 825 Unity 

v.3.2.0.0

04/13/09 ES&S

AG64L64T8SHC4S Unity 

v.3.2.0.0

03/25/09 ATP Electronics

ECO 826 DRAM 2nd source Unity 

v.3.2.0.0

04/13/09 ES&S

TS64MLD64V4F_2430_S Unity 

v.3.2.0.0

no date ATP Eletronics

ECO 827 - UPS Unity 

v.3.2.0.0

04/29/09 ES&S

ECO 829 ground (inclusion is pending environmental testing) Unity 

v.3.2.0.0 

05/06/09 ES&S

ECO 830 -USB circuit board (inclusion is pending environmental testing) Unity 

v.3.2.0.0

5/6/09 ES&S

ECO 831 version ID Unity 

v.3.2.0.0

05/06/09 ES&S

APC spec Back-UPS RS 1500VA None no date APC

Belkin Spec UPS change None no date Belkin

Table 11 Other Software, Hardware and Materials

Material Material Description Use in the Voting System

Other 

Paper ballots COTS printed ballots:

14 inch 216 positions 

14 inch 288 positions

19 inch 408 positions 

Paper ballots to record votes

Ballot Pens ES&S ballot marking Pen Marking votes on paper ballots

DS200 Media USB (512 MB, 2 GB, 4 GB, & 8 GB) DS200 media storing the election data and 

the election results

Paper rolls COTS Thermal paper, internal 

thermal printer

DS200 reports printer (open and close 

polls reports)

VAT Media 256 MB CompactFlash VAT media storing election data

Ink cartridges Black ink VAT ink to mark ballots and print reports

Paper (9 1/2 X 11 - 1 ply Computer 

form) 

COTS paper, Oki Microline 520 and 

Epson printers

M650 audit log and results printer paper

M650 Media Iomega Zip Disk (100 MB) M650 election data and transfer disk.

Norton AntiVirus Viruses and malicious software

protection

Unity and AIMS PC antivirus protection 

software

Multiple desktop and laptop PCs A variety of PCs running Microsoft 

operating systems

Supplied by iBeta: Preparation, 

management and recording of test plans, 

test cases, reviews and results

Repository servers Separate servers for storage of test 

documents and source code, 

running industry standards 

operating systems, security and 

back up utilities

Supplied by iBeta: Documents are 

maintained on a secure network server. 

Source code is maintained on a separate 

data disk on a restricted server 

Microsoft Office 2003 & 2007 Excel and Word software and 

document templates

Supplied by iBeta: The software used to 

create and record test plans, test cases, 

reviews and results

SharePoint 2003 TDP and test documentation 

repository

Supplied by iBeta: TDP and test 

documentation repository and 

configuration management tool 

Other standard business application 

software

Internet browsers, PDF viewers 

email

Supplied by iBeta: Industry standard tools 

to support testing, business and project 

implementation

Visual Studio 2003 v.7.1.3808 Build and source code review Supplied by iBeta: View source code 
VSTL Certification # pending

Page 33 of 247 (V)2009-30Jun-001(A)

Material Material Description Use in the Voting System

(Microsoft) Integrated Development 

Environment

review 

RSM v.6.92 

(M Squared Technologies)

C, C++, Java & C# static analysis 

tool

Supplied by iBeta: identify line counts and 

cyclomatic complexity

Beyond Compare 2 v.2.4.3 (Scooter 

Software)

Comparison utility Supplied by iBeta: used to compare 

file/folder differences

WinDiff 5.1 (Microsoft) Comparison utility Supplied by iBeta: used to compare 

file/folder differences

Hash.exe v.7.08.10.07.12 (Maresware) Hash creation utility Supplied by iBeta: used to generate hash 

signatures for Trusted Builds

Automation Anywhere 4.0 Automation tool Supplied by iBeta: used to execute 

Volume 3.

Nessus 4.0.0.987 Security tool Supplied by iBeta: Tenable Network 

Security. Used on the Unity PC for 

security testing

Trusted Build Software

This is listed in Appendix G: Trusted 

Build & Validation Tools Unity 3.2.0.0 

voting system
VSTL Certification # pending

Page 34 of 247 (V)2009-30Jun-001(A)

4 Voting System Overview

The ES&S Unity 3.2.0.0 is a paper-based voting system that includes the election management software 

applications: Election Data Manager, ES&S Ballot Image Manager (ESSIM), Hardware Programming Manager 

(HPM), AutoMARK Information Management System (AIMS), the audit software, : Audit Manager, and the Election 

Reporting Manger (ERM) central counts reporting software. . Paper ballots can be printed by Ballot-on-Demand 

COTs printer in addition to providing ballots to commercials printers for printing. The voting system includes the 

DS200 precinct optical scanner hardware and firmware, the AutoMARK Voter Assist Terminal A100, AutoMARK 

Voter Assist Terminal Model A200 precinct ballot marker hardware and firmware and the Model 650 central count 

hardware and firmware.

4.1 Election Management System- Pre Voting Capabilities

The Unity 3.2.0.0 election management system pre-vote functions are performed by five software applications that 

are installed on two COTS PCs.

4.1.1 Election Data Manager (EDM)

The Election Data Manager functionality includes: 

Definition of election databases for the M650 and DS200 paper ballot scanners and VAT paper ballot 

marker;

Creation and edit of closed, open and pick-a-party primaries and general elections with office, candidate 

election, and absentee preferences;

Set up of early, Election Day and absentee voting;

Creation and edit of new elections from existing files;

Creation and edit of ballot sets, rotations, groupings and straight party;

Creation and edit of parties, candidates, referendum, recall questions, and write-in targets;

Creation, edit and assignment of precincts and polling places;

Creation, edit and generation of ballot styles;

Merging preferences;

Use of the Import Wizard to import lists of parties, language, precincts, county, district typed, district 

names, district relations, office headings, office relations, candidates, and polling places;

Addition and edit of language files;

Select and generate statistical counters;

Display, print and export of EDM reports, including: Master Precinct Report, District Names, District 

Relations, District Relations by Precinct, Master Office, Party, Office Headings, Precincts this Election, 

Offices this Election, Office Relations, Candidates this Election -in party order sorted by last name, Offices 

and Candidates this election Ballot Styles in Ballot Style Order, Ballot Styles in Precinct Order, Ballot 

Galley Report (Precinct/Office/Candidate), Candidate Rotations by Office, and Standard Rotation;

Generates the interface file(.iff) and ballot set collection file (.bsc) to create the ballot data file (.bdf); and

Back up of election files.

4.1.2 ES&S Ballot Image Manager (ESSIM)

The ES&S Image Manage (ESSIM) is a desktop publishing tool to design and publish Election Systems and 

Software (ES&S) paper ballots for the Unity 3.2.0.0 DS200 precinct scanner and the M650 central count scanner. 

ESSIM is used to:

Import the ballot data file(.bdf) from EDM;

Create and edit ballot formats for ES&S ballot services or a printer to print official ballots;

Create and edit style sheets for ballot elements corresponding to EDM election data (offices, parties, 

candidates, etc.);

Create and edit text frames to place instructional text on a ballot;

Create and edit graphic frames to place images on a ballot;

Create and edit production frames to place variable information (precinct or style identifiers) on a 

ballot;

Reuse previously created ballot formats;

Use layouts created with the program to print extra Election Day ballots with Ballot on Demand;

Reads and convert the information contained in an EDM election database into finished ballot layouts; 

Generation of the interface file (ifc);
VSTL Certification # pending

Page 35 of 247 (V)2009-30Jun-001(A)

Generate Ballot Validation and Ballot Style Reports to validate election data properly fits the ballot and 

is properly positioned; and

Package elections for back up, transfer to other computers or send to ES&S as requested for election 

support. 

4.1.3 Hardware Programming Manager (HPM)

Hardware Programming Manager (HPM) is used to convert the election file for use with Election Reporting Manager 

(ERM) and for create election parameters and loading them to the memory device for the DS200 or M650. 

Create and edit the election shell for importation of the interface file (.ifc);

Create and edit access control for HPM and the DS200;

Set and edit jurisdiction tabulator controls for selection of equipment and tape/report printing 

sequence and "vote for" information;

Set and edit election specification tabulator controls for handling of blank, cross-voted, write-ins; 

unreadable marks, absentee ballots and report printing;

Set and edit certification tabulator controls for text to appear on reports and tapes;

Write elections to zip disk and USB memory storage devices for the M650 and DS200;and 

Update the election for use by AutoMARK Information Management System (AIMS) and the Election 

Reporting Manager (ERM).

HPM may also be used for coding an election, if necessary. HPM permits importation, formatting, and conversion 

of the election file, definition of districts, election contests and candidates, election definitions for ballot scanning 

equipment.

4.1.4 AutoMARK Information Management System (AIMS)

The AutoMARK Information Management System includes the AIMS application software installed on a COTS PC. 

It was originally developed by Automark Technical Services, LLC (ATS) and acquired by ES&S. ES&S has 

assumed responsibility for the product. In many instances the TDP documents referenced ATS instead of ES&S. . 

The AIMS application manages information required by the AutoMARK Voter Assist Terminal (VAT) for an election, 

including:

Importation of HPM election files and a corresponding printed optical scan ballot;

Optional manual entry of election data;

Edit of stored election multilingual text information for proper pronunciation of synthesized speech 

messages;

Storing of recorded multilingual voice messages in WAV format;

Writing of the election database to a compact flash memory card (FMC) in order to provide ballot

content information to the VAT;

Review ballot set-up and preview on-screen ballot display;

Performs no ballot counting or vote counting/reporting functions; 

Logging of changes to the election database in the AIMS audit log; and

Backing up (archive) of the election database.

4.1.5 Audit Manager (AM)

EDM and Image Manager use Audit Manager to store detailed logs of the actions performed in both programs. 

Audit Manager:

Prints or displays audit listings;

Listings include date and time stamps;

Listings can be exported;

Displays logs in cascade, vertical and horizontal views; and

Archives logs.
VSTL Certification # pending

Page 36 of 247 (V)2009-30Jun-001(A)

4.2 Polling Place- Voting Capabilities

The Unity 3.2.0.0 polling place voting functions are performed by the intlElect DS200 Precinct Scanner and the 

AutoMARK Voter Assist Terminal.

4.2.1 intlElect DS200 Precinct Ballot Scanner

The intlElect DS200 is a jurisdiction-wide election tabulation system. DS200 scanners

Process single or dual-sided paper ballots for up to ten Election Day precincts and 1639 Early Voting 

precincts;

Permit programming of separate election groups for the procedural processing and storage of provisional 

ballots separately from Election Day totals for inclusion, after determination of voter validity;

Support Early Voting; 

Permit opening, closing and reopening of the polls;

Automatically prints a Zero report when the polls open;

Can be configured to automatically print one or more reports (Status, Race Results, Certification or Audit 

Log)

Have a public counter that displays the number of ballots cast;

Store paper ballots in attached ballot storage bins (key locked ballot boxes);

Do not store any ballot data; all ballot data, election totals and optional ballot images are stored on an 

external USB flash drive which can be transported to a central count location; 

Prevents access to the USB election flash drive via a key locked compartment;

Print reports including: Election Startup, Poll Closing, Diagnostic, Initial State, Audit Log, Zero and 

Certification;

Audit logging and reporting;

Prohibit transmitting of results by removal of the modem. (Transmission of results via the public telephone 

network is excluded from testing in Unity 3.2.0.0); and

Operates on standard or two hour back-up battery power.

4.2.2 AutoMARK Voter Assist Terminal (VAT)

The AutoMARK VAT is an automated voter assistive paper ballot marking device. It was originally developed by 

Automark Technical Services, LLC (ATS) and acquired by ES&S. ES&S has assumed responsibility for the 

product. In many instances the TDP documents referenced ATS instead of ES&S. Four configurations of the VAT 
VSTL Certification # pending

Page 37 of 247 (V)2009-30Jun-001(A)

were submit for certification in Unity 3.2.0.0. A description of the four configurations and their differences is found in 

Section 3.1 Submitted Voting System Identification.

The VAT device assists voters with visual, language and manual dexterity challenges. It only displays ballot for 

marking on paper. Temporary memory only retains votes until the printing operation is complete. VSS 

requirements for a DRE are applicable for ballot display and voter selection functions, only. The VAT is exempt 

from vote storage and reporting function requirements. The VAT:

Incorporates a touch screen monitor, tactile input buttons, connections for assistive input devices, 

audio output and a ballot marking printer;

Password protects the System Maintenance Menu for setting date/time and loading firmware;

Permits installation of an AIMS election database on a compact flash memory card; 

Provides a test mode for performance of set-up, reporting and maintenance functions;

Provides audio, printing, screen and button readiness tests and verification of the ballot definition;

Draws a preprinted blank ballot from the voter input tray and scans a preprinted bar code on the ballot 

to determine the form of ballot inserted;

Presents the voter with the options to make a language selection for either an audio or visual ballot;

Presents the voter with controls to adjust the display contrast/size, volume, speed for synthesized 

speech and repeat audio output;

Present the ballot as a series of menu-driven voting choices on a color screen;

Permits vote selection inputs via a touch screen or assistive switch-based devices (foot paddles);

Accumulates the voter's choices in an internal memory until the voter has completed the selection 

process;

Provides a summary of the voter‟s choices for review and confirmation;

Marks and prints the paper ballot following voter confirmation of the summary;

Accommodates insertion of the ballot in any orientation;

Prints single and double-sided ballots; 

Returns the ballot to the voter after printing is completed;

Clears its internal memory so that the paper ballot is the only lasting record of voter selections;

Prevents access to the compact flash memory card via a key locked compartment;

Operates on standard or back-up battery power; and

Provides a date/time stamped audit log of ballot marking operations that can be viewed or printed.

4.3 Central Count Scanner- Post Voting Capabilities

The central count scanner functions are performed by the Model 650 Central Ballot Scanner.
VSTL Certification # pending

Page 38 of 247 (V)2009-30Jun-001(A)

4.3.1 Model 650 Central Ballot Scanner (M650)

The Model 650 Central Ballot Scanner is a high-speed, computerized, paper ballot, optical mark reader. The M650 

scanner:

Options include a left and right ballot oval read and red and green light optical read;

Loads and tests election definitions and readiness for Election Day tasks;

Checks the pre-printed codes along the ballot edge to determine each ballot's precinct, split and type;

Reads voter choices for candidates or issues in a fraction of a second for each ballot.

Checks for ballot irregularities, stops and provides the operator instructions for handling the ballot;

Tabulates votes in each race and tracks the race count and total ballots by precinct;

Permits separate scanning for Election Day and Absentee ballots;

Permits sorting of over-voted, blank and write-in ballots;

Tracks absentee results by a user determined method defined in the election definition;

Clearing of vote counts to permit rescanning for accidental user counting errors;

Generates printed reports on-demand to provide up-to-the-minute totals by precinct, city, or by county;

Provides report options for inclusion of over and under-votes, totals per race, ranking of candidate by votes 

received, certification messages and write-ins;

Provides a time/date stamped audit log of scanner activities on a separate printer.

Saves election results to a zip disk in order to make a permanent record of the election, transfer to ERM or 

to use as backup data;

Permits adding of vote totals from a zip disk into the scanner; and

Network card can be removed to disable networking capabilities (Networking is excluded from testing in 

Unity 3.2.0.0.

4.4 Election Management System- Post Voting Capabilities

The post vote consolidation and reporting functions are performed by the Election Reporting Manager.

4.4.1 Election Reporting Manager

Election Reporting Manager (ERM) is ES&S‟s election results reporting program. ERM is designed to display 

updated election totals on a monitor as election data is consolidated. ERM:

Supports configuration of uses and user permissions to limit access to specific functions; 

Warns the user when votes are present in the election database at startup;

Supports creation of the Results Database for an election;'

Supports creation and definition of Groups;

Only supports importation of election results from the M650 on zip disks and DS200 on USB memory drives 

in Unity 3.2.0.0;

Supports manual entry of hand counted election results;

Supports close out of Precincts with no ballots cast; 
VSTL Certification # pending

Page 39 of 247 (V)2009-30Jun-001(A)

Generates paper and electronic reports: including ; Precinct, Precinct Group Detail, Election Summary, 

election Summary with Group Detail, Canvas (Numbered Key, Statistics, Numbered Key Districts only, 

Numbered Key Districts Turnout only, Name heading, District Totals, Block Style, Jurisdiction, Local Office, 

Precincts Counted Precincts Completed Listing, Precincts Process Listing), and System Log;

Supports temporary suppression and subsequent release of precinct results for unique circumstances that 

may require investigation prior to release of results to prevent counting errors or vote total altering problems;

Supports merging of election results from multiple ERM with files transferred on 3.5 inch disk or other 

appropriate memory devices. It does not support transmission of results via the public telephone or a local 

area network in Unity 3.2.0.0 (no testing was performed for network transmission of results);

Support creation of state specific transfer and web files;

Was not submitted for use with the iVotronic or M100 scanner in Unity 3.2.0.0 (no testing was performed 

with these tabulators).
VSTL Certification # pending

Page 40 of 247 (V)2009-30Jun-001(A)

5 Certification Review and Test Results

The results and evaluations of the PCA and FCA reviews tests are identified below. Detailed data regarding the 

Acceptance/Rejection criteria, reviews and tests are found in the appendices.

Appendix A identifies all certification test requirements traced to specific Test Cases

Appendix B identified the PCA Source Code Review Acceptance/Rejection criteria

Appendix C identifies the PCA TDP Document Review Acceptance/Rejection criteria

Appendix D identifies all FCA Testing Acceptance/Rejection criteria

Appendix E identifies the PCA and FCA Discrepancies reported during review and testing

5.1 PCA Source Code Review

The source code review was performed by SysTest in the Unity 4.0.0.0 test effort. Documentation of that review is 

addressed in the Summary Report of the testing performed by SysTest. iBeta performed a 3% Source Code 

Review and provided a recommendation to the EAC regarding reuse of the SysTest code review for Unity 3.2.0.0. 

Any subsequent changes to source code submitted by ES&S for Unity 3.2.0.0 were review by iBeta. 

During receipt and check in of the Unity 3.2.0.0 source code delivered by ES&S, iBeta observed if the source code 

version control was consistent with the ES&S configuration management practices.

5.1.1 3% Source Code Review Assessment for Reuse Results

The 3% source code review was conducted using iBeta's standard PCA Source Code Review Procedure. The 

detailed process for this review is found in Appendix H Amended Test Plan section 2.1.3.1 Documentation of the 

3% Source Code Review Process. This detail includes information about the language specific interpretations VSS

2002 (Vol. 1 Sect. 4.2.2 through 4.2.7, 6.2 & 6.4.2; and Vol. 2 Sect. 2.4.5.d & 5.4.2), use of analysis tools, sampling 

selection, management of code, and peer review.

Potential logic issues, flagged as yellow, needed an EAC decision. There were no confirmed logic issues, which 

otherwise would have been flagged red. These were submitted to the EAC as individual discrepancy spreadsheets 

provided as separate confidential compressed files delivered on CD.

A total of 330 discrepancies were identified. The majority, 307 or 93%, were categorized as non-logic issues

(Green- comments, headers, formatting, and style only, recommendation of reuse). The remaining 23 

discrepancies were categorized as Yellow EAC Decision. These discrepancies with ES&S responses were 

forwarded to the EAC with the iBeta recommendation (see Appendix B). There were no confirmed logic issues 

(Red- recommendation of 100% review).

ES&S provided justification for non-compliance or their disagreement with the iBeta interpretation of the VSS 2002 

requirements. As iBeta had established a precedent in testing for other clients and the iBeta interpretations must 

be applied consistently to all manufacturers under test with iBeta. However, iBeta acknowledged that in some 

instances other interpretations are possible and the EAC Reviewers could determine if these alternative 

interpretations were acceptable.

In order to provide a recommendation, iBeta evaluated the results of the 3% source code review. Whereas the 

results would be recommended for acceptance if only non-significant discrepancies were found (i.e. less critical 

requirement or interpretations inconsistent with documented industry accepted practices), there were discrepancies 

written that potentially impact the source code. Thus iBeta initiated two additional analyses:

iBeta confirmed that the results of the 3% source code review were consistent with the previous results 

(not identical but consistent). This confirmation was reached by reviewing the types of discrepancies 

generated by SysTest in the 100% review against those generated by iBeta.

iBeta reviewed the severity of the discrepancies identified and assessed that the number of discrepancies 

potentially impacting the source code is considered very low versus the overall number of discrepancies 

consistent with a 100% review. The severity of the discrepancies and the manufacturer responses further

indicate that the majority of the potential logic discrepancies would be resolved without source code 

modifications. 

Based on the limited or perhaps non-impact on the source code as a result of these discrepancies, iBeta 

recommended reuse of the results of the SysTest source code review. In EAC letter to iBeta QA Director on reuse 

of testing the EAC approved the reuse of the source code review conducted by SysTest Labs. The data supporting 

this review are found in Appendix B.

5.1.2 Unity 3.2.0.0 Source Code Review Results

The code submitted for reuse was a baseline that remained essentially unchanged during the iBeta portion of 

certification testing. ES&S did submit a minor update to Audit Manager which was reviewed and incorporated into a 
VSTL Certification # pending

Page 41 of 247 (V)2009-30Jun-001(A)

witnessed build prior to iBeta initiating testing. ES&S' configuration management policy does not permit them to 

perform a new build with the same version information. Hence the source code had to be updated to reflect a new 

build version for the Trusted Builds performed by iBeta. ES&S resubmitted code with the updated versions. iBeta 

performed a differences check and confirmed if review of the code was required. If any changes requiring code 

review were found, the code was reviewed. There were no discrepancies in any of the code delivered for the initial 

Trusted Build. Functional discrepancies 121 and 151 required delivery of software fixes in ERM and the DS200 

Firmware. A limited number of header comment discrepancies were identified in ERM and forwarded to ES&S for 

resolution.

The data supporting this review are found in Appendix B. No instances of inconsistency in the version control of 

code delivered by ES&S was noted. 

5.2 PCA TDP Document Review

The PCA Technical Data Package Document Review was performed by SysTest in the Unity 4.0.0.0 test effort. 

Documentation of that review is contained in the attached document Summary Report of the testing performed by 

SysTest. As part of ES&S' petitioned to reuse the Unity 4.0.0.0 TDP without removing the products that were out of 

scope of Unity 3.2.0.0. In permitting ES&S to do this, the EAC required a disclaimer be inserted into the individual 

documents clearly identifying the out of scope product content and that the out of scope content and products were 

not certified in Unity 3.2.0.0.

As instructed by the EAC iBeta performed an audit to assess and recommend if any additional review was required. 

The audit of the ES&S Unity 3.2.0.0 Technical Data Package (TDP) was in accordance with the EAC instructions. 

iBeta sampled the ES&S Unity 3.2.0.0 documents. The sample selection included the documents identified in the 

SysTest Labs issued discrepancies and documents needed to complete the Unity 3.2.0.0 trusted builds, a sample 

3% source code review, test planning and test execution. Criteria for the review included confirmation that the Unity 

3.2.0.0 documents addressed any document discrepancies within the scope of the Unity 3.2.0.0 test effort and the 

content provided sufficient information in order to complete the test tasks list above. 

During receipt and check in of the Unity 3.2.0.0 TDP iBeta observed if document version control was consistent with 

the ES&S configuration management practices.

5.2.1 Unity 3.2.0.0 TDP PCA Document Review Results

iBeta identified the SysTest PCA Document Review of Unity 4.0.0.0 adequate for reuse. This was reported to the 

EAC Links to the applicable recommendation and reuse approval are contained in Appendix C. Any errors or 

nonconformities observed in this review and any subsequent use of the documentation was summarized in 

Appendix E- Discrepancy Report. Documentation of corrections and verification of corrections are contained in 

each summary. Any instance of inconsistency in the version control of the delivered documents was reported as an 

informational issue. 

5.3 FCA Functional System Level, Accessibility, Maintainability, 

Accuracy & Reliability Testing and Reuse of Testing

The Functional System Level, Accessibility, Maintainability, Accuracy and Reliability test was performed by SysTest 

in the Unity 4.0.0.0 test effort. Documentation of that testing is contained in the attached document Summary 

Report of the testing performed by SysTest.

iBeta performed a single regression end-to-end mock election to demonstrate the integrated functionality and 

processes of the ES&S Unity 3.2.0.0. Additional functional system level test coverage was provided in the Volume 

suite of tests. Testing was conducted on the system configuration identified in Section 3. The application was 

tested for the pre-vote and post vote election management system of the Unity 3.2.0.0 voting system and the 

voting/counting functionality of the AutoMARK VAT, DS200 and M650 optical scanners. 

5.3.1 Evaluation for Reuse of the SysTest Labs Testing: Functional, 

Accessibility, Maintainability, Accuracy & Reliability Test Results

Section 5.3.1 is provided by the EAC and is also contained in the approved test plan Appendix D EAC Reuse of Testing 

Review Process

Due to the suspension of accreditation of a VSTL this project was moved from that VSTL to iBeta as requested by 

ES&S and approved by the EAC. This very unusual circumstance required that a transition plan be developed for 

the orderly transition of the project. A number of factors impacted the development of this transition plan. 
VSTL Certification # pending

Page 42 of 247 (V)2009-30Jun-001(A)

The overriding consideration had to be that the quality of the evaluation meets the EAC‟s standards for excellence 

and that any decision to certify the system be clearly based on rigorous and thorough testing. If other legitimate 

concerns could also be met then every attempt was made to do so. Among those considerations was the timely 

evaluation of the system, avoiding duplicative testing that provided little real value and supporting the needs of 

election officials for improvements and upgrades.

In developing a transition plan a number of factors were taken into consideration:

1. The quality of testing already performed was evaluated. In some cases iBeta was directed to review or 

audit that testing. Another factor was the probability that testing to be performed by iBeta would identify 

any system issues that may have been missed in prior testing. In some cases iBeta was directed to 

modify the testing it would do to provide additional checks and redundancy in areas of particular concern.

2. Prior versions of this system are in wide use. In addition individual states and other organizations have 

conducted their own, independent evaluation of either this exact system or very similar prior versions. This 

provides a significant body of information from both experience in actual elections and testing performed 

for other purposes.

All these sources of information were used in developing the transition plan. A risk assessment was made and a 

transition plan approved. This plan allowed for reuse of some testing, reuse of some testing after an audit and 

recommendation by iBeta, and requirements for further testing or correlated testing by iBeta. The results of this 

evaluation were communicated to ES&S and iBeta in several E-Mails and letters between November 2008 and 

letters dated February 3, 2009 and February 12, 2009. In those communications the following was approved:

1. All hardware testing was approved for reuse.

2. The technical data package review was approved after an audit of that review and recommendation for 

reuse by iBeta.

3. The source code review was approved after a 3% audit and recommendation for reuse by iBeta.

4. The EAC Technical Reviewers reviewed the Functional, Accessibility, Maintainability, Accuracy, and 

Reliability test summary reports provided by SysTest on the DS-200, M650, AutoMARK VATs, Ballot-onDemand printer, and Unity EMS software. The EAC approved the reuse of this testing.

5. The Volume, Stress, Error Recovery and Security test methods and testing had not yet been completed. 

Accordingly iBeta was to perform this testing on the Unity 3.2.0.0 system.

6. A new test plan for the Unity 3.2.0.0 system was prepared by iBeta using applicable areas from the Unity 

v.4.0.0.0 test plan.

5.3.2 Regression Functional & System Level Test Results

iBeta validated the testing performed by SysTest Labs by confirmed that Unity 3.2.0.0 met a sampling of the VSS 

2002 requirements outlined in Appendix A and the test case.. Any functionality that did not meet the requirements 

of the VSS 2002 was reported to ES&S. Fixes were submitted and the failed tests were rerun. 

Appendix D Section 7.4.1.2 Regression System Level Test Results details specific information on the Functional 

and System Integration Testing. Failures, errors, nonconformities and anomalies observed during testing by iBeta 

are summarized in Appendix E- Discrepancy Report. Any relevant Unity 4.0.0.0 issues that remained open at the 

time of transfer were incorporated into Appendix E- Discrepancy Report. Documentation of corrections and 

verification of corrections are contained in each issue summary. 

5.4 FCA Volume, Stress, and Error Recovery Testing

The Volume Stress and Error Recovery Tests were performed by iBeta. ES&S provided documentation of their 

system limitations and information regarding the largest jurisdiction for each limit. While the capacity varied for 

each limit, iBeta observed that for each limit, the system limit capacity ranged from 115% to 474% of ES&S' largest 

jurisdiction.

iBeta performed 10 test cases with maximum and overloaded volume scenarios to test the various system limit 

conditions identified in the System Limitations Unity 3.2.0.0 document. In discussions with the EAC it was agreed 

that the Volume test scenarios would incorporate validation that the system could perform to the identified system 

limit. Stress and error recovery conditions would validate that appropriate responses were encountered for 

overloaded conditions. Appropriate responses were to handle the overload, generate an error, or if the system 

halts processing without generating an error, the system recovers without any loss of data.

Due to the substantial excess capacity of the limits to the ES&S' real world customer base, during the test process 

additional limits or limiting factors were identified. These limits or limiting factors were previously unidentified 

because they were substantially beyond the capacity needed by ES&S' customer base. As these were identified 
VSTL Certification # pending

Page 43 of 247 (V)2009-30Jun-001(A)

test cases were modified to incorporate the newly identified conditions. While this did not change the scope of the 

overall Volume, Stress and Error Recovery testing it did involve reworking of several test cases and test scenarios. 

(Example: Testing of the 2900 precinct limit was moved from Volume 1 Test Case to Volume 9 Test Case.). These 

modifications were identified in the individual test cases.

Additionally error recovery was addressed in the source code review of the requirement v.1: 4.2.3.e which specifies 

a single exit point and SysTest Labs' power recovery test results which validated recovery from power or system 

failure without loss of vote data and the minimum two hour back-up power.

Testing by iBeta was conducted on the system configuration identified in Section 3. The application was tested for 

the system limits, overload conditions and error recovery of the Unity 3.2.0.0 voting system and the voting/counting 

functionality of the AutoMARK VAT, DS200 and M650 optical scanners.

5.4.1 Volume, Stress and Error Recovery Test Results

iBeta confirmed that Unity 3.2.0.0 could process the maximum system limits identified in the System Limitations 

Unity 3.2.0.0 document and appropriate responses were encountered for overloaded and error conditions.

Appendix D FCA Volume (Volume Stress, Performance and Error Recovery) Testing details specific information on 

the Volume, Stress, and Error Recovery Testing. Failures, errors, nonconformities and anomalies observed during 

testing by iBeta are summarized in Appendix E- Discrepancy Report. Documentation of corrections and verification 

of corrections are contained in each issue summary. Discrepancy 144 (misread of a ballot mark) was referred to 

the EAC for interpretation. Their response is provided as part of the issue and validation.

5.5 FCA Security Review and Testing

iBeta‟s security specialist, a Certified Information System Security Profession, supervised execution of a security 

analysis of the threat model for the applicable Unity 3.2.0.0 Security TDP documents to the requirements of the 

VSS 2002 Vol.1 Sect. 2.2.1 a to -g, 2.2.2.1 d & e, 2.2.3 a-c, 2.2.4.1 e, f, g, i & j, 2.2.4.2, 2.2.5.3 , 4.5, and section 6. 

First the analysis identified VSS 2002 security requirements that were currently addressed in the standard testing, 

source code and document reviews. The analysis next identified any unique voting system specific tests, source 

code and document reviews that were needed. The tests, source code or documents reviews were traced to the 

VSS 2002 requirement in the FCA Security Review and Testing table. The results of the standard tests and 

reviews were recorded in the applicable FCA Functional and System Integration Testing, PCA Source Code Review 

or the PCA Document Review. The unique tests and reviews were documented in the FCA Security Review and 

Testing table. This documentation included the steps, acceptance and rejection criteria, and results. Appendix D

contains the FCA Security Review and Testing table and the specific Test Methodology. In the initial document

review gaps were noted in the content of the Security documentation. These were identified as discrepancies 55 

though 84 (see Appendix E- Discrepancy Report). Responses provided to many of these discrepancies were used 

to complete the test criteria for the Security Review and Testing.

In order to comply with the security test requirements identified in Vol.2 Sect. 6.4 of the VSS 2002. iBeta 

approached security testing of the VSS 2002 by first creating test scenarios which discounted the exposure to risk 

and excluded physical security procedures. However, in establishing acceptance and rejection criteria, iBeta 

assessed the potential exposure to risk and included physical security procedures as an acceptable security 

control, per the requirements of Vol. 1 Sect. 7.3 of the VSS 2002. To assess if an access control was effective 

iBeta considered the degree to which one or more of the following security controls was present: physical security

procedures, password protection, detection in an audit, technical expertise required, obfuscation of sensitive 

material, and encryption of sensitive material. In determining potential exposure to risk the security specialist 

considered access from the user and if the exposure was from a trusted user or non-trusted user. Systems were 

accepted as meeting the security requirements of the VSS 2002 if the security controls present were deemed 

effective to address the identified risk.

Testing was conducted on the system configuration identified in Section 3. The unique security tests and reviews 

were grouped over together by the individual products of the Unity 3.2.0.0 voting system (M650, DS200, VAT, 

AIMS, and the Unity applications, EDM, ESSIM, HPM, and ERM).

5.5.1 FCA Security Review and Test Results

Testing, source code and document reviews of the Unity 3.2.0.0 voting system found the system met the applicable 

VSS 2002 security requirements identified in the security analysis. Appendices A, B, C and Appendix D Security 

Review and Testing Method provide specific information on the FCA Security Review and Testing. Failures, errors, 

nonconformities and anomalies observed in review and testing are summarized in Appendix E- Discrepancy Report. 

Documentation of corrections and verification of corrections are contained in each summary. Discrepancies 121, 
VSTL Certification # pending

Page 44 of 247 (V)2009-30Jun-001(A)

126, 132, and 151were referred to the EAC for interpretation. The EAC required resolution of 121 and 151. They 

accepted 126 and 132. Their responses are provided as part of the issue and validation.

5.6 FCA Hardware Environmental Testing & Reuse

SysTest Lab's and their subcontractors completed hardware environmental testing for the Unity 4.0.0.0 voting 

system. The hardware of the Unity 3.2.0.0 is a subset of the tested hardware. ES&S petitioned the EAC for reuse 

of this testing in the Unity 3.2.0.0 certification test effort. SysTest documented that testing is addressed in the 

Summary Report of the testing performed by SysTest. In the February 3, 2009 EAC letter to iBeta QA Director on 

reuse of testing the EAC authorized the reuse of the hardware testing conducted by SysTest's sub-contractors. In 

order to ensure that these test results provided sufficient documentation of the Environmental Hardware test 

assessment and results iBeta reviewed the reports to confirm any failures resulting in engineering changes were 

documented and the reports document that all hardware submitted under Unity 3.2.0.0 passed.

The hardware environmental testing was conducted on the system configuration identified in Section 3. 

5.6.1 FCA Hardware Environmental Tests

SysTest reports the hardware tested in the Unity 4.0.0.0 passed the environmental tests. The result of the iBeta 

review generated requests for clarification or additional documentation from SysTest Labs. These requests were 

documented in issues 1, 2, 3, 6, 7, 8, and 9 of iBeta's Unity 3.2.0.0 Discrepancy Report (see Appendix EDiscrepancy Report). Responses to all issues were accepted. Issues 6 and 7 were accepted by iBeta but were 

referred to the EAC for determination of sufficient documentation for test result reuse. These issues are traced to 

the Test Report and Tested Configuration Matrixes in Appendix D.

5.7 FCA Telephony and Cryptographic Review and Testing

As noted in section 1.2 the Unity 3.2.0.0 Voting System does not use the public networks or any other network. As 

such it is exempt from the Telecommunications requirements of Vol. 1 Sect. 5 & 6. The Telephony and 

Cryptographic Review and Testing were conducted to confirm the absence of network functionality. 

5.7.1 FCA Telephony and Cryptographic Review and Tests

iBeta confirmed that the Unity 3.2.0.0 voting system election management hardware and installation procedures 

reflect the prohibition of connection of the certified system to a public or any other network. Polling place equipment 

was inspected to confirm modem hardware was not present. The M650 was inspected to confirm that network cards 

were not present. Appendix D details specific information on the Telephony and Cryptographic Review and Testing. 

Failures, errors, nonconformities and anomalies observed in testing are summarized in Appendix E- Discrepancy 

Report. Documentation of corrections and verification of corrections are contained in each summary.
VSTL Certification # pending

Page 45 of 247 (V)2009-30Jun-001(A)

6 Opinions & Recommendations

In addition to the portions SysTest Labs testing of Unity 4.0.0.0 accepted for reuse by the EAC, iBeta Quality 

Assurance has completed the testing of ES&S Unity 3.2.0.0 voting system. Testing prescribed by the iBeta test 

plan or amended test plan was performed as identified. Documentation of any divergence from the test plan was 

included in the amended as run test plan (see Appendix H). All identified anomalies or failures were reported and 

resolved. Questions with regard to iBeta's interpretation of the test standards were referred to the EAC. All 

relevant EAC interpretations were documented in iBeta's verification of resolution. The information provided in this 

report is an accurate representation of the ES&S Unity 3.2.0.0 voting system Certification test effort.

Based upon the findings identified in Section 5 it is our opinion that the acceptance requirements of the Federal 

Election Commission Voting System Standards April 2002 have been met as demonstrated in testing and EAC 

interpretations of the presented test results for the hardware, software and user documentation of the system 

configuration submitted for certification testing.

iBeta Quality Assurance recommends that the Election Assistance Commission certifies the ES&S Unity 3.2.0.0

voting system.

See Appendix K for information regarding the EAC Certification number.

Carolyn E. Coggins

QA Director – Voting

iBeta Quality Assurance

Note: The system configuration of the Unity 3.2.0.0 voting system submitted for certification by ES&S explicitly 

excluded connection to any network (public or private). Modification of the hardware or operating system 

configuration to permit connection to a network invalidates the recommendations of this test report.
VSTL Certification # pending

Page 46 of 247 (V)2009-30Jun-001(A)

7 APPENDICES: TEST OPERATION, FINDINGS & DATA 

ANALYSIS

The Voting System Test Laboratory Program Manual v.1.0 Appendix B identifies content in specific appendices. In 

order to ensure that this content and content required by VSS 2002 Volume 2 Appendix B a trace is provided in 

section 1.4 to clarify the location of this specified content

7.1 Appendix A: Certification Test Requirements

Appendix A identifies the test results to the Certification Test Requirement of the VSS 2002. Requirements marked: 

Accept: met the VSS 2002 requirement

Reject: did not meet the VSS 2002 requirement

NA: the requirement is not applicable to the voting system type submitted for Certification Testing 

Pending: VSS 2002 requirements that cannot be completed by the VSTL until after Certification

Out of Scope: VSS 2002 requirements which are performed by entities other than the VSTL 

Requirements marked Reject, NA, Pending or Out of Scope shall include an explanatory note. (Example: If a voting 

system is only a Central Count Scanner, the requirement is marked “NA” and a comment indicates “Not a DRE.")

Optional requirements which apply to the voting system type but are not supported by the ES&S Unity 3.2.0.0 

voting system are not marked "NA". Instead they are marked “Accept”, with an explanatory comment. The reason

for this is to provide a positive identification that iBeta reviewed the voting system for all applicable requirements, 

including this optional functionality and confirmed non-support. (Example: If a voting system does not have a 

VVPAT. The requirements are marked “Accept” and a comment indicates “DRE does not have a VVPAT”.)

The test case trace corresponds to the Test Methods identified in the Appendix H- Amended Test Plan and 

Appendix D- FCA Testing.

F= SysTest PCA Document Review, Source Code Review, Functional System Level, Accessibility, 

Maintainability, Accuracy & Reliability Reuse;

R = Regression sampling of system functionality;

V#= Volume 1 through 10 for testing of identified system limits;

S = Security;

T= Telephony & Cryptographic; and

E= SysTest subcontractor Environmental

Many functional requirements were exercised in multiple test cases. These instances are noted in the Function 

Exercised column. The actually test case or test method might not contain a requirement trace in these instances. 

Issues identified during testing are cross-referenced to the Appendix E- Discrepancy Report. 

EAC Decisions on Requests for Interpretation which were applicable to the voting system submitted for certification 

testing are noted in the comments

Maufacturer Voting System & Version Scope Prior EAC Certification

ES&S Unity 3.2.0.0 Voting System Full VSS 2002 Initial EAC Certification

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

2.2 Overall System Capabilities 

2.2.1 Security

System security is achieved through a 

combination of technical capabilities and sound 

administrative practices. Te ensure security all 

systems shall:

a. Provide security access controls that limits limit 

or detect access to critical system components 

to guard against loss of system integrity, 

availability, confidentiality, and accountability. 

Accept S R #61, 62, 63, 69, 76,

121, 130,131,126, 

132 138, 139, 150-

Closed

b. Provide system functions that are executable 

only in the intended manner and order, and only 

under the intended conditions.

Accept S, R #135 - Closed

c. Use the system's control logic to prevent a 

system function from executing, if any

preconditions to the function have not been met.

Accept S, R

d. Provide safeguards to protect against tampering Accept S #126 Closed
VSTL Certification # pending

Page 47 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

during system repair, or interventions in system 

operations, in response to system failure.

e. Provide security provisions that are compatible 

with the procedures and administrative tasks 

involved in equipment preparation, testing, and 

operation.

Accept S R #55, 83 Closed

f. If access to a system function is to be restricted 

or controlled the system shall incorporate the 

means of implementing this capability.

Accept S R #118, 119 - Closed

g. Provide documentation of mandatory 

administrative procedures for effective system 

security.

Accept S #36, 56, 76 Closed

2.2.2 Accuracy 

To ensure vote accuracy, all systems shall:

2.2.2.1 Common Standards to Ensure Vote Accuracy

To ensure vote accuracy, all systems shall:

a. Records the election contests, candidates, and 

issues exactly as defined by election officials.

Accept F, R V1-10

b. Records the appropriate options for casting and 

recording votes.

Accept F, R V1-10

c. Records each vote precisely as indicated by the 

voter and have the ability to produce an 

accurate report of all votes cast.

Accept F, R V1-10 RFI 2007-06

#19, 28, 144 -Closed

d. Control logic and data processing methods 

incorporation parity and check sums (or 

equivalent error detection and correction 

methods) to demonstrate the system has been 

designed for accuracy.

Accept S #57, 80, 127, 13, 

132- Closed

e. The software monitors the overall quality of data 

read-write and transfer quality status, checks the 

number and types of errors that occur in any of 

the relevant operations on data and how they 

were corrected.

Accept S #58, 80 - Closed

2.2.2.2 DRE System Standards

In additional DRE systems shall:

As an additional means of ensuring accuracy in 

DRE systems, voting devices record and retain 

redundant copies of the original ballot image. A 

ballot image electronic record of all votes cast 

by the voter, including undervotes.

NA RFI 2007-06

No DRE

2.2.3 Error Recovery

To recover from a non-catastrophic failure of a 

device, or from any error or malfunction that is 

within the operator's ability to correct, the 

system shall provide the following capabilities:

a. Restoration of the device to the operating 

condition existing immediately prior to an error 

or failure, without loss or corruption of voting 

data previously stored in the device

Accept S, V1-10, 

R, F

b. Resumption of normal operation following the 

correction of a failure in a memory component, 

or in a data processing component, including the 

central processing unit

Accept S, R, F

c. Recovery from any other external condition that 

causes equipment to become inoperable, 

provided that catastrophic electrical or 

mechanical damage due to external phenomena 

has not occurred.

Accept S, R, F #140 - Closed

2.2.4 Integrity

Integrity measures ensure the physical stability 

and function of the vote recording and counting 

processes. To ensure system integrity, all 
VSTL Certification # pending

Page 48 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

systems shall:

2.2.4.1 Common Standards

To ensure system integrity, all systems shall:

a. Protect against a single point of failure that 

would prevent further voting at the polling place. 

Accept F R

b. Protects against the interruption of electronic 

power.

Accept F, V- S

c. Protects against electromagnetic radiation. Accept E

d. Protects against the ambient temperature and 

humidity fluctuations.

Accept E

e. Protects against failure of any data input or 

storage device. 

Accept S, V4 F, R #74 - Closed

f. Protects against any attempt at improper data 

entry or retrieval

Accept S F, R

g. Records and reports of any normal or abnormal 

events.

Accept S #121,150, 151-

Closed

h. Maintains a permanent record of original audit 

data that cannot be bypassed or turned off.

Accept S F, R

i. Detect and record every event, including the 

occurrence of an error condition that the system 

cannot overcome, and time-dependent or 

programmed events that occur without the 

intervention of the voter or a polling place 

operator

Accept R S, F #64- Closed

j. Include built-in measurement, self-test, and 

diagnostic software and hardware for detecting 

and reporting the system's status and degree of 

operability

Accept S #65- Closed

2.2.4.2 DRE Systems Standards

In addition to the common requirements, DRE 

systems shall:

a. Maintain a record of each ballot cast using a 

process and storage location that differs from 

the main vote detection, interpretation, 

processing, and reporting path

NA No DRE

b. Provide a capability to retrieve ballot images in a 

form readable by humans

NA No DRE

2.2.5 System Audit

See the requirement for context of these 

requirements.

RFI 2008-12

2.2.5.2 Operational Requirements

Audit records shall be prepared for all phases of 

election operations performed using devices 

controlled by the jurisdiction or its contractors. 

These records rely upon automated audit data 

acquisition and machine-generated reports, with 

manual input of some information. These 

records shall address the ballot preparation and 

election definition phase, system readiness

tests, and voting and ballot-counting operations. 

The software shall activate the logging and 

reporting of audit data as described below.

Accept S, R, F V1, 2, 6 -

10

2.2.5.2.

1

Time, Sequence, and Preservation of Audit 

Records

The timing and sequence of audit record entries 

is as important as the data contained in the

record. All voting systems shall meet the 

requirements for time, sequence and 

preservation of audit records outlined below.

a. Except where noted, systems shall provide the 

capability to create and maintain a real-time 

audit record. This capability records and 

Accept S, R, F V1, 2, 6 -

10

#93, 99, 105 -

Closed
VSTL Certification # pending

Page 49 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

provides the operator or precinct official with 

continuous updates on machine status. This 

information allows effective operator 

identification of an error condition requiring 

intervention, and contributes to the 

reconstruction of election-related events 

necessary for recounts or litigation.

b. All systems shall include a real-time clock as 

part of the system‟s hardware. The system shall 

maintain an absolute record of the time and date 

or a record relative to some event whose time 

and data are known and recorded.

Accept S, R, F V1, 2, 6 -

10

c. All audit record entries shall include the timeand-date stamp.

Accept S, R, F V1, 2, 6 -

10

d. The audit record shall be active whenever the 

system is in an operating mode. This record 

shall be available at all times, though it need not 

be continually visible.

Accept S, R, F V1, 2, 6 -

10

e. The generation of audit record entries shall not 

be terminated or altered by program control, or 

by the intervention of any person. The physical 

security and integrity of the record shall be 

maintained at all times.

Accept S, R, F V1, 2, 6 -

10

#126 - closed

f. Once the system has been activated for any 

function, the system shall preserve the contents 

of the audit record during any interruption of 

power to the system until processing and data 

reporting have been completed.

Accept S, R, F V1, 2, 6 -

10

g.

1)

2)

3)

The system shall be capable of printing a copy 

of the audit record. A separate printer is not 

required for the audit record, and the record may 

be produced on

the standard system printer if all the following 

conditions are met:

• The generation of audit trail records does not 

interfere with the production

of output reports

• The entries can be identified so as to facilitate 

their recognition,

segregation, and retention

• The audit record entries are kept physically 

secure

Accept S, R, F V1, 2, 6 -

10

2.2.5.2.

2

Error messages

All voting systems shall meet the requirements 

for error messages below.

a. The voting system shall generate, store, and 

report to the user all error messages as they 

occur. 

Accept S, R, F V1, 2, 6 -

10

#151 - Closed

b. All error messages requiring intervention by an 

operator or precinct official shall be displayed or 

printed clearly in easily understood language 

text, or by means of other suitable visual 

indicators.

Accept S, R, F V1, 2, 6 -

10

92, 129 - Closed

c. When the voting system uses numerical error 

codes for trained technician maintenance or 

repair, the text corresponding to the code shall 

be self-contained or affixed inside the voting 

machine. This is intended to reduce 

inappropriate reactions to error conditions, and 

to allow for ready and effective problem 

correction.

Accept S, R, F V1, 2, 6 -

10

d. All error messages for which correction impacts Accept S, R, F V1, 2, 6 -
VSTL Certification # pending

Page 50 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

vote recording or vote processing shall be 

written in a manner that is understandable to an 

election official who possesses training on 

system use and operation, but does not possess 

technical training on system servicing and 

repair.

10

e. The message cue for all voting systems shall 

clearly state the action to be performed in the 

event that voter or operator response is 

required. 

Accept S, R, F V1, 2, 6 -

10

#98, 111 - Closed

f. Voting system design shall ensure that 

erroneous responses will not lead to irreversible 

error. 

Accept S, R, F V1, 2, 6 -

10

g. Nested error conditions are corrected in a 

controlled sequence such that voting system 

status shall be restored to the initial state 

existing before the first error occurred.

Accept S, R, F V1, 2, 6 -

10

2.2.5.2.

3

Status Messages

The Standards/Guidelines provide latitude in 

software design so that vendors can consider 

various user processing and reporting needs. 

The jurisdiction may require some status and 

information messages to be displayed and 

reported in real-time. Messages that do not 

require operator intervention may be stored in 

memory to be recovered after ballot processing 

has been completed.

The voting system shall display and report 

critical status messages using clear indicators or 

English language text. The voting system need 

not display non-critical status messages at the 

time of occurrence. Voting systems may display 

non-critical status messages (i.e., those that do 

not require operator intervention) by means of 

numerical codes for subsequent interpretation 

and reporting as unambiguous text.

Accept S, R, F V1, 2, 6 -

10

Voting systems shall provide a capability for the 

status messages to become part of the real-time 

audit record.

Accept S, R, F V1, 2, 6 -

10

The voting system shall provide a capability for 

a jurisdiction to designate critical status 

messages.

Accept S, R, F V1, 2, 6 -

10

2.2.5.3 COTS General Purpose Computer System 

Requirements

See the standards for the context these 

requirements. Three operating system 

protections are required on all such systems on 

which election software is hosted. 

RFI 2008-03

RFI 2008-12

Authentication shall be configured on the local 

terminal (display screen and keyboard) and on 

all external connection devices (“network cards” 

and “ports”). This ensures that only authorized 

and identified users affect the system while 

election software is running.

Accept S

Operating system audit shall be enabled for all 

session openings and closings, for all 

connection openings and closings, for all 

process executions and terminations, and for 

the alteration or deletion of any memory or file 

object. This ensures the accuracy and 

completeness of election data stored on the 

system. It also ensures the existence of an audit 

Accept S #123, 124,138, 139 -

Closed
VSTL Certification # pending

Page 51 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

record of any person or process altering or 

deleting system data or election data.

The system shall be configured to execute only 

intended and necessary processes during the 

execution of election software. The system shall 

also be configured to halt election software 

processes upon the termination of any critical 

system process (such as system audit) during 

the execution of election software.

Accept S #66, 78, 122, 125 -

Closed

#146 - Open 

Informational only

2.2.6 Election Management System

The Election Management System (EMS) is 

used to prepare ballots and programs for use in 

casting and counting votes, and to consolidate, 

report, and display election results. An EMS 

shall generate and maintain a database, or one 

or more interactive databases, that enables 

election officials or their designees to perform 

the following functions:

Accept F, R V1, 2, 6 -

10

a. Define of the political subdivision boundaries 

and multiple election districts, as indicated in the 

system documentation.

Accept F, R V2, 6

b. Identify of contests, candidates, and issues. Accept F, R

c. Define of ballot formats and appropriate voting 

options.

Accept F, R V1, 2, 6 -

10

d. Generate ballots and election-specific programs 

for vote recording and vote counting equipment.

Accept F, R V1, 2, 6 -

10

#104 - Closed

e. Install ballots and election-specific programs. Accept F, R V1, 2, 6 -

10

f. Test that ballots and programs have been 

properly prepared and installed.

Accept F, R V1, 2, 6 -

10

g. Accumulate vote totals at multiple reporting 

levels as indicated in the system documentation.

Accept F, R V1, 2, 6 -

10

h. Generate of post-voting reports per Section 2.5. Accept F, R V1, 2, 6 -

10

i. Process and produce audit reports of the data 

indicated in Section 4.5.

Accept F, R V1, 2, 6 -

10

2.2.7 Accessibility

2.2.7.1 Common Standards 

See the standard for diagrams. The voting 

system meets the following conditions: 

a. Where clear floor space only allows forward 

approach to an object, the maximum high 

forward reach allowed shall be 48inches. The 

minimum low forward reach is 15 inches.

Accept F

b. Where forward reach is over an obstruction with 

knee space below, the maximum level forward 

reach is 25 inches. When the obstruction is less 

than 20 inches deep, the maximum high forward 

reach is 48 inches. When the obstruction 

projects 20 to 25 inches, the maximum high 

forward reach is 44 inches.

Accept F

c. The position of any operable control is 

determined with respect to a vertical plane that 

is 48 inches in length, centered on the operable 

control, and at the maximum protrusion of the 

product within the 48-inch length.

Accept F

d. Where any operable control is 10 inches or less 

behind the reference plane, have a height that is 

between 15 inches and 54 inches above the 

floor.

Accept F

e. Where any operable control is more than 10 

inches and not more than 24 inches behind the 

Accept F
VSTL Certification # pending

Page 52 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

reference plane, have a height between 15 

inches and 46 inches above the floor.

f. Have operable controls that are not more than 

24 inches behind the reference plane.

Accept F

2.2.7.2 DRE Standards for Accessibility

DRE voting systems shall provide, as part of 

their configuration, the capability to provide 

access to voters with a broad range of 

disabilities. This capability shall:

a. Not require the voter to bring their own assistive 

technology to a polling place.

Accept F R VAT - Ballot marking 

only

b Provide Audio information and stimulus that:

b.1. Communicates to the voter the complete content 

of the ballot.

Accept F R VAT - Ballot marking 

only

b.2. Provides instruction to the voter in operation of 

the voting device.

Accept F R VAT - Ballot marking 

only

b.3. Provides instruction so that the voter has the 

same vote capabilities and options as those 

provided by the system to individuals who are 

not using audio technology

Accept F R VAT - Ballot marking 

only

b.4. For a system that supports write-in voting, 

enables the voter to review the voter‟s write-in 

input, edit that input, and confirm that the edits 

meet the voter‟s intent.

Accept F R VAT - Ballot marking 

only

b.5. Enables the voter to request repetition of any 

system provided information.

Accept F R VAT - Ballot marking 

only

b.6. Supports the use of headphones provided by 

the system that may be discarded after each 

use

Accept F VAT - Ballot marking 

only

b.7. Provides the audio signal through an industry 

standard connector for private listening using a 

1/8 inch stereo headphone jack to allow 

individual voters to supply personal headsets

Accept F R VAT - Ballot marking 

only

b.8. Provides a volume control with an adjustable 

amplification up to a maximum of 105 dB that 

automatically resets to the default for each voter

Accept F VAT - Ballot marking 

only

c. Provide, in conformance with FCC Part 68, a 

wireless coupling for assistive devices used by 

people who are hard of hearing when a system 

utilizes a telephone style handset to provide 

audio information

Accept F VAT - Ballot marking 

only

d. Meet the requirements of ANSI C63.19-2001 

Category 4 to avoid electromagnetic

interference with assistive hearing devices

Accept F VAT - Ballot marking 

only

e. For Electronic Image Displays, permit the voter 

to:

e.1. Adjust contrast settings Accept F R VAT - Ballot marking 

only

e.2. Adjust color settings, when color is used Accept F R VAT - Ballot marking 

only

e.3. Adjust the size of the text so that the height of 

capital letters varies over a range of 3 to 6.3 

millimeters

Accept F R VAT - Ballot marking 

only

f. For a devise with touch screen or contactsensitive controls, provide an input method 

using mechanically operated controls or keys 

that shall:

f.1. Be tactilely discernible without activating the 

controls or keys.

Accept F VAT - Ballot marking 

only

f.2. Be operable with one hand and not require tight 

grasping, pinching, or twisting of the wrist.

Accept F VAT - Ballot marking 

only

f.3. Require a force less than 5 lbs (22.2 N) to Accept F VAT - Ballot marking 
VSTL Certification # pending

Page 53 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

operate. only

f.4. Provide no key repeat function. Accept F VAT - Ballot marking 

only

g. For a system that requires a response by a voter 

in a specific period of time, alert the voter before 

this time period has expired and allow the voter 

additional time to indicate that more time is 

needed

Accept F VAT - Ballot marking 

only

h. For a system that provides sound cues as a 

method to alert the voter about a certain 

condition, such as the occurrence of an error, or 

a confirmation, the tone shall be accompanied 

by a visual cue for users who cannot hear the 

audio prompt

Accept F VAT - Ballot marking 

only

i. Provide a secondary means of voter 

identification or authentication when the primary 

means of doing so uses biometric measures that 

require a voter to possess particular biological 

characteristics

Accept F VAT has no biometric 

measures

2.2.8 Vote Tabulating Program

2.2.8.1 Functions 

The vote tabulating program software resident in 

each voting machine, vote count server, or other 

devices shall include all software modules 

required to:

a. Monitor of system status and generating 

machine-level audit reports

Accept F, R V1, 2, 6 -

10

b. Accommodate device control functions 

performed by polling place officials and 

maintenance personnel

Accept F, R V1, 2, 6 -

10

c. Register and accumulating votes Accept F, R V1, 2, 6 -

10

d. Accommodate variations in ballot counting logic Accept F, R V1, 2, 6 -

10

2.2.8.2 Voting Variation 

The Technical Data Package accompanying the 

system shall specifically identify which of the 

following items can and cannot be supported by 

the voting system, as well as how the voting 

system can implement the items support.

a. Documented support or non-support of closed 

primaries.

Accept F V10

b. Documented support or non-support of open 

primaries.

Accept F R, V9

c. Documented support or non-support of partisan 

offices.

Accept F R, V7 &10

d. Documented support or non-support of nonpartisan offices.

Accept F R, V1-6, 

8, & 9

e. Documented support or non-support of write-in 

voting.

Accept F R

f. Documented support or non-support of primary 

president.

Accept F

g. Documented support or non-support of ballot 

rotation.

Accept F

h. Documented support or non-support of straight 

party voting.

Accept F

i. Documented support or non-support of crossparty endorsement

Accept F

j. Documented support or non-support of split 

precincts.

Accept F R

k. Documented support or non-support of vote for 

N of M.

Accept F R, V1 to 

10
VSTL Certification # pending

Page 54 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

l. Documented support or non-support of recall 

issues, with options.

Accept F

m. Documented support or non-support of 

cumulative voting.

Accept F

n. Documented support or non-support of ranked 

over voting.

Accept F

o. Documented support or non-support of 

provisional or challenged ballots.

Accept F #27 - Closed

2.2.9 Ballot Counter 

For all voting systems, each device that 

tabulates ballots shall provide a counter that:.

a. Can be set to zero before any ballots are 

submitted for tally

Accept F, R

b. Records the number of ballots cast during a 

particular test cycle or election

Accept F, R V1, 2, 6-

10

#137 - Closed

c. Increases the count only by the input of a ballot Accept F, R V1, 2, 6 -

10

d. Prevents or disables the resetting of the counter 

by any person other than authorized persons at 

authorized points

Accept F R

e. Is visible to designated election officials Accept F, R V1, 2, 6 -

10

2.2.10 Telecommunications

For all voting systems that use 

telecommunications for the transmission of data 

during pre-voting, voting or post-voting activities 

shall include capabilities to ensure data are 

transmitted with no alternation or unauthorized 

disclosure during transmission for:

Voter Authentication: Coded information that 

confirms the identity of a voter for security 

purposes for a system that transmit votes 

individually over a public network 

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

Ballot Definition: Information that describes to 

voting equipment the content and appearance of 

the ballots to be used in an election

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

Vote Transmission to Central Site: For voting 

systems that transmit votes individually over a 

public network, the transmission of a single vote 

to the county (or contractor) for consolidation 

with other county vote data

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

Vote Count: Information representing the 

tabulation of votes at any one of several levels: 

polling place, precinct, or central count

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

List of Voters: A listing of the individual voters 

who have cast ballots in a specific election

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

2.2.11 Data Retention

See standard/guideline for context.

All voting systems shall provide for maintaining 

the integrity of voting and audit data during an 

election and for a period of at least 22 months 

thereafter.

Accept TDP Attestation from ESS

2.3 Pre-voting Functions #50 Closed

2.3.1 Ballot Preparation

2.3.1.1 General Capabilities

All systems shall provide the general capability 

for ballot preparation, ballot formatting and ballot 

production. All systems shall be capable of:

Accept F, R V1, 2, 6 -

10

2.3.1.1.

1

Common Standards

All systems shall be capable of: 

a. Enable the automatic formatting of ballots in Accept F, R V1, 2, 6 -
VSTL Certification # pending

Page 55 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

accordance with the requirements for offices, 

candidates, and measures qualified to be placed 

on the ballot for each political subdivision and 

election district.

10

b.

1)

2)

3)

Collecting and maintaining the following data: 

Offices with labels/instructions

Candidate names with labels

Issues or measures with their text

Accept F, R

c. Supporting the maximum number of potentially 

active voting positions as indicated in the 

system documentation.

Accept F, V8

d. For a primary election, generating ballots that 

segregate the choices in partisan races by party 

affiliation

Accept F, R V7 & 9

e. Generating ballots that contain identifying 

codes or marks uniquely associated with each 

format.

Accept F, R V1, 2, 6 -

10

f. Ensuring voter response fields, selection 

buttons, or switches properly align with the 

specific candidate names and/or issues printed 

on the ballot display, ballot card or sheet, or 

separate ballot pages.

Accept F, R V1, 2, 6 -

10

2.3.1.1.

2

Paper-Based System Standards

Paper-based voting systems shall also meet the 

following requirements applicable to the 

technology used.

a. Enable voters to make selections by punching a 

hole or by making a mark in areas designated 

for this purpose upon each ballot card or sheet.

Accept F, R V1, 2, 6 -

10

b. For punchcard systems ensure that the vote 

response fields can be properly aligned with 

punching devices used to record votes.

NA Not a punchcard 

system

c. For marksense systems, the timing marks align 

properly with the vote response fields.

Accept F, R V1, 2, 6 -

10

2.3.1.2 Ballot Formatting 

All voting systems shall provide a capability for: 

a. Creation of newly defined elections Accept F, R V1, 2, 6 -

10

b. Rapid and error-free definition of elections and 

their associated ballot layouts

Accept #136 - Closed

c. Uniform allocation of space and fonts used for 

each office, candidate, and contest such that the 

voter perceives no active voting position to be 

preferred to any other.

Accept F, R V1, 2, 6 -

10

d. Simultaneous display of the maximum number 

of choices for a single contest as indicated by 

the vendor in the system documentation

Accept F V8

e. Retention of previously defined formats for an 

election

Accept F, R V1, 2, 6 -

10

f. Prevention of unauthorized modification of any 

ballot formats

Accept F, R

g. Modification by authorized persons of a 

previously defined ballot format for use in a 

subsequent election

Accept F, V3 & 4 S

2.3.1.3 Ballot Production

The voting system shall provide a means of 

printing or other wise generating a ballot display 

that can be installed in all system voting devices 

for which it is intended: All Systems shall 

provide a capability to ensure. 

2.3.1.3.

1

Common Standards

The voting system shall provide a means of 
VSTL Certification # pending

Page 56 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

printing or other wise generating a ballot display 

that can be installed in all system voting devices 

for which it is intended: All systems shall provide 

a capability to ensure. 

a. The electronic display or printed document on 

which the user views the ballot is capable of 

rendering an image of the ballot in any of the 

languages required by The Voting Rights Act of 

1965, as amended

Accept F R RFI 2008-04

b. The electronic display or printed document 

where the user views the ballot does not show 

any advertising or commercial logos of any kind, 

whether public service, commercial, or political, 

unless specifically provided for in State law. 

Electronic displays do not provide connection 

through hyperlink.

Accept F R #49- Closed

c. The ballot conforms to vendor specifications for 

type of paper stock, weight, size, shape, size 

and location of punch or mark field used to 

record votes, folding, bleed through, and ink for 

printing if paper ballot documents or paper 

displays are part of the system

Accept F, R V1, 2, 6 -

10

2.3.1.3.

2

Paper-based System Standards

Vendor documentation for marksense systems 

shall include specifications for ballot materials to 

ensure that vote selections are read from only a 

single ballot at a time, without detection of 

marks from multiple ballots concurrently (e.g., 

reading of bleed-through from other ballots)

Accept F R

2.3.2 Election Programming 

Process by which election officials or their 

designees use election databases and vendor 

system software to logically define the voter 

choices associated with the contents of the 

ballots. All systems shall provide for: 

a. Logical definition of the ballot, including the 

definition of the number of allowable choices for 

each office and contest

Accept F, R V1, 2, 6 -

10

b. Logical definition of political and administrative 

subdivisions, where the list of candidates or 

contests varies between polling places

Accept F, R V1, 2, 6 -

10

c. Exclusion of any contest on the ballot in which 

the voter is prohibited from casting a ballot 

because of place of residence, or other such 

administrative or geographical criteria

Accept F, R V1, 2, 7, 9

d. Ability to select from a range of voting options to 

conform to the laws of the jurisdiction in which 

the system will be used

Accept F, R V1, 2, 6 -

10

e. Generation of all required master and distributed 

copies of the voting program, in conformance 

with the definition of the ballots for each voting 

device and polling place, and for each tabulating 

device

Accept F, R V1, 2, 6 -

10

2.3.3 Ballot and Program Installation and Control

All systems shall include the following at the 

time of ballot an program installation:

All systems provide a means of installing ballots 

and programs on each piece of polling place or 

central count equipment according to the ballot 

requirements of the election and the jurisdiction. 

Accept F, R V1, 2, 6 -

10

a. A detailed work plan or other documentation Accept F
VSTL Certification # pending

Page 57 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

providing a schedule and steps for the software 

and ballot installation, including a table outlining 

the key dates, events and deliverables.

b. A capability for automatically verifying that the 

software has been properly selected and 

installed in the equipment or in programmable 

memory devices and for indicating errors. 

Accept F,S R

c. A capability for automatically validating that 

software correctly matches the ballot formats 

that it is intended to process, for detecting 

errors, and for immediately notifying an election 

official of detected errors. 

Accept F, S R

2.3.4 Readiness Testing

Election personnel conduct voting equipment 

and voting system readiness tests prior to the 

start of an election to ensure that the voting 

system functions properly, to confirm that voting 

equipment has been properly integrated, and to 

obtain equipment status reports. All voting 

systems shall provide the capabilities to

RFI 2008-07

#147 -Closed

2.3.4.1 Common Standards

All voting systems shall provide the capabilities 

to:

a. Verify the voting machines or vote recording and 

data processing equipment, precinct count 

equipment, and central count equipment are 

properly prepared for an election, and collect 

data that verifies equipment readiness

Accept F, S R #32 - Closed

b. Obtains status and data reports from each set of 

equipment

Accept F, R

c. Verify the correct installation and interface of all 

system equipment

Accept F, R

d. Verify that hardware and software function 

correctly

Accept F, R #33, 87- Closed

e. Generate consolidated data reports at the 

polling place and higher jurisdictional levels

Accept F, R V6, 9

f. Segregate test data from actual voting data, 

either procedurally or by hardware/software 

features

Accept F, R

Resident test software, external devices, and 

special purpose test software connected to or 

installed in voting devices to simulate operator 

and voter functions used for these tests meeting 

the following standards: 

a. These elements are capable of being tested 

separately, and are proven to be reliable 

verification tools prior to their use

Accept F

b. These elements are incapable of altering or 

introducing any residual effect on the intended 

operation of the voting device during any 

succeeding test and operational phase.

Accept F R

2.3.4.2 Paper-Based Systems

Paper-based systems shall:

a. Supports conversion testing that uses all 

potential ballot positions as active positions

Accept F V 8 & 9

b. Supports conversion testing of ballots with 

active position density for systems without predesignated ballot positions

Accept F

2.3.5 Verification at the Polling Place

All systems shall provide a formal record of the 

following, in any media, upon verification of the 

authenticity of the command source:

RFI 2008-07
VSTL Certification # pending

Page 58 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

a. The election's identification data; Accept F, R V1, 2, 6 -

10

b. The identification of all equipment units; Accept F, R V1, 2, 6 -

10

c. The identification of the polling place; Accept F, R V1, 2, 6 -

10

d. The identification of all ballot formats; Accept F, R V1, 2, 6 -

10

e. The contents of each active candidate register 

by office and of each active measure register at 

all storage locations (showing that they contain 

only zeros);

Accept F, R, S V1, 2, 6 -

10

f. A list of all ballot fields that can be used to 

invoke special voting options

Accept F

g. Other information needed to confirm the 

readiness of the equipment, and to 

accommodate administrative reporting 

requirements

Accept F, R V1, 2, 6 -

10

To prepare voting devices to accept voted 

ballots, all voting systems shall provide the 

capability to test each device prior to opening to 

verify that each is operating correctly. At a 

minimum the tests shall include.

Accept F, R V1, 2, 6 -

10

a. Confirmation that there are no hardware or 

software failures.

Accept F, R V1, 2, 6 -

10

b. Confirmation that the device is ready to be 

activated for accepting votes.

Accept F, R V1, 2, 6 -

10

If a precinct count system includes equipment 

for the consolidation of polling place data at one 

or more central counting locations, it shall have 

means to verify the correct extraction of voting 

data from transportable memory devices, or to 

verify the transmission of secure data over 

secure communication links.

Accept F, R S, V1, 2, 6 

-10

Telecommunications 

is disabled in Unity 

3.2.0.0

2.3.6 Verification at Central Location

Election officials perform verification at the 

central location to ensure that vote counting and 

vote consolidation equipment and software 

function properly before and after an election. 

Upon verification of the authenticity of the 

command source, any system used in a central 

count environment shall provide a printed record 

of the following:

RFI 2008-07

a. The election's identification data Accept F, R V1, 2, 6 -

10

b. The contents of each active candidate register 

by office and of each active measure register at 

all storage locations (showing that they contain 

only zeros);

Accept F, R S V1, 2, 6 -

10

S - per v.2: 3.3.1

c. Other information needed to confirm the 

readiness of the equipment, and to 

accommodate administrative reporting 

requirements.

Accept F, R V1, 2, 6 -

10

2.4 Voting Functions

2.4.1. Opening the Polls

At a minimum , the systems shall provide the 

functional capabilities indicated below.

RFI 2008-07

2.4.1.1 Opening the polling Place (Precinct Count 

Systems) 

To allow voting devices to be activated for 

voting, the system shall provide:

a. An internal test or diagnostic capability to verify Accept F, R,S V1, 2, 6 - S - per v.2: 3.3.1
VSTL Certification # pending

Page 59 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

that all of the polling place tests specified in 

2.3.5 have been successfully completed.

10

b. Automatic disabling any device that has not 

been tested until it has been tested.

Accept F, R,S V1, 2, 6 -

10

S - per v.2: 3.3.1

2.4.1.2 Paper-Based System Standards

2.4.1.2.

1

All Paper-Based systems

To facilitate opening the polls, all paper-based 

systems shall include: 

a. A means of verifying ballot punching or marking 

devices are prepared and ready to used;

Accept F, R V1, 2, 6 -

10

No ballot punching

b. A voting booth or similar facility, in which the 

voter may punch or mark the ballot in privacy

Accept F No ballot punching

c. Secure receptacles for holding voted ballots. 

Ballot boxes.

Accept F, R, S DS200

2.4.1.2.

2

Precinct Count Paper-Based Systems

In addition to the above requirements, all paperbased precinct count equipment shall include a 

means of: 

a. Activating the ballot counting device. Accept F, R V1, 2, 6 -

10

b. Verifying that the device has been correctly 

activated and is functioning properly

Accept F, R V1, 2, 6 -

10

c. Identifying device failure and corrective action 

needed.

Accept F, R V1, 2, 6 -

10

2.4.1.3 DRE System Standards

To facilitate opening the polls, all DRE systems 

shall include:

a. A security seal, a password, or a data code 

recognition capability to prevent the inadvertent 

or unauthorized actuation of the poll-opening 

function

Accept F, R, S VAT doesn't open 

polls; switched to 

election marking 

mode

b. A means of enforcing the execution of steps in 

the proper sequence if more than one step is 

required

Accept F

c. A means of verifying the system has been 

activated correctly

Accept F, R

d. A means of identifying system failure and any 

corrective action needed

Accept F

2.4.2 Activating the Ballot (DRE Systems)

To activate the ballot, all DRE systems shall:

a. Enable election officials to control the content of 

the ballot presented to the voter, whether 

presented in printed form or electronic display, 

such that each voter is permitted to record votes 

only in contests in which that voter is authorized 

to vote

Accept F, R V1, 2, 6, 

10

VAT ballot marking 

functionality

b. Allow each eligible voter to cast a ballot Accept F, R

c. Prevent a voter from voting on a ballot to which 

he or she is not entitled

Accept F, R

d. Prevent a voter from casting more than one 

ballot in the same election

Accept F, R Blank paper ballot

required

e. Activate the casting of a ballot in a general 

election

Accept F V1,2, 6, 

10

f. Enable the selection of the ballot that is 

appropriate to the party affiliation declared by 

the voter in a primary election

Accept F, R Appropriate blank 

paper ballot required

g. Activate all parts of the ballot upon which the 

voter is entitled to vote

Accept F,R Some controls in 

addition to the paper 

ballot

h. Disable of all parts of the ballot upon which the 

voter is not entitled to vote

Accept F,R Some controls in 

addition to the paper 

ballot
VSTL Certification # pending

Page 60 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

2.4.3 Casting a Ballot

2.4.3.1 Common Standards

To facilitate casting a ballot, all systems shall:

a. Provide test that is at least 3 millimeters high 

and provide the capability to adjust or magnify 

the text to an apparent size of 6.3 millimeters

Accept F

b. Protect the secrecy of the vote such that the 

system cannot reveal any information about how 

a particular voter voted, except as otherwise 

required by individual State law

Accept F, R

c. Record the selection and non-selection 

(undervote) of individual vote choices for each 

contest and ballot measure

Accept F, R V1, 2, 6, 

10

d. Record the voter‟s selection of candidates 

whose names do not appear on the ballot, if 

permitted under State law, and record as many 

write-in votes as the number of candidates the 

voter is allowed to select

Accept F, R

e. In the event of a failure of the main power 

supply external to the voting system, provide the 

capability for any voter who is voting at the time 

to complete casting a ballot, allow for the 

successful shutdown of the voting system 

without loss or degradation of the voting and 

audit data, and allow voters to resume voting 

once the voting system has reverted to back-up 

power 

Accept F, V5 S

f. Provide the capability for voters to continue cast 

ballots in the event of a failure of a 

telecommunications connection within the 

polling place or between the polling place and 

any other location

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

2.4.3.2 paper-based System Standards

2.4.3.2.

1

All Paper-Based Systems

All paper-based systems shall: 

a. Allow the voter to easily identify the voting field 

that is associated with each candidate or ballot 

measure response

Accept F, R V1, 2, 6 -

10

b. Allow the voter to mark the ballot to register a 

vote

Accept F, R V1, 2, 6 -

10

c. Allow either the voter or the appropriate election 

official is able to place the voted ballot into the 

ballot counting device (precinct count systems) 

or a secure receptacle (central count systems)

Accept F, R V1, 2, 6 -

10

d. Protect the secrecy of the vote throughout the 

process

Accept F, R V1, 2, 6 -

10

2.4.3.2.

2

Precinct Count Paper-Based Systems

In addition to the above requirements, all paperbased precinct count equipment shall include a 

means of: 

a. Provide feedback to the voter identifies specific 

contests or ballot issues for which an overvote 

or undervote is detected

Accept F, R

b. Allow the voter, at the voter‟s choice, to vote a 

new ballot or submit the ballot „as is‟ without 

correction

Accept F, R

c. Allow an authorized election official to turn off 

the capabilities defined in the two prior 

provisions. 

Accept F

2.4.3.3 DRE Systems Standards

a. Prohibit the voter from accessing or viewing any 

information on the display screen that has not 

Accept F,S VAT ballot marking
VSTL Certification # pending

Page 61 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

been authorized by election officials and 

preprogrammed into the voting system (i.e., no 

potential for display of external information or 

linking to other information sources)

b. Enable the voter to easily identify the selection 

button or switch, or the active area of the ballot 

display that is associated with each candidate or 

ballot measure response

Accept F, R VAT ballot marking

c. Allow the voter to select his or her preferences 

on the ballot in any legal number and 

combination

Accept F, R V1, 2, 6, 

10

VAT ballot marking

d. Indicate that a selection has been made or 

canceled

Accept F, R V1, 2, 6, 

10

VAT ballot marking

e. Indicate to the voter when no selection, or an 

insufficient number of selections, has been 

made in a contest (e.g. undervotes)

Accept F, R VAT ballot marking

f. Prevent the voter from overvoting Accept F, R VAT ballot marking

g. Notify the voter when the selection of candidates 

and measures is completed

Accept F, R V1, 2, 6, 

10

VAT ballot marking

h. Allowing the voter, before the ballot is cast, to 

review his or her choices and, if the voter 

desires, to delete or change his or her choices 

before the ballot is cast

Accept F, R V1, 2, 6, 

10

VAT ballot marking

i. For electronic image displays, prompt the voter 

to confirm the voter's choices before casting his 

or her ballot, signifying to the voter that casting 

the ballot is irrevocable and directing the voter to 

confirm the voter‟s intention to cast the ballot

Accept F, R V1, 2, 6, 

10

VAT ballot marking: 

printing is irrevocable 

but not casting of the 

ballot

j. Notify the voter after the vote has been stored 

successfully that the ballot has been cast

NA No DRE

k Notify the voter that the ballot has not been cast 

successfully if it is not stored successfully, 

including storage of the ballot image, and 

provide clear instruction as to the steps the voter 

should take to cast his or her ballot should this 

event occur

NA No DRE

l. Provides sufficient computational performance 

to provide responses back to each voter entry in 

no more than three seconds

Accept F VAT ballot marking; 

printing exceeds 3 

seconds

m. The votes stored accurately represent the actual 

votes cast

Accept F, R V1, 2, 6, 

10

Storage is ballot 

printing

n. Preventing modification of the voter‟s vote after 

the ballot is cast

Accept S Paper ballot handling 

documentation

o. Provides a capability to retrieve ballot images in 

a form readable by humans (in accordance with 

the requirements of Section 2.2.2.2 and 2.2.4.2)

NA No DRE

p. Incrementing the proper ballot position registers 

or counters

Accept F, R Counts successful 

prints, not votes cast

q. Protecting the secrecy of the vote throughout 

the voting process

Accept F, R

r. Prohibiting access to voted ballots until after the 

close of polls

NA No DRE

s. Provides the ability for election officials to submit 

test ballots for use in verifying the end-to-end 

integrity of the system

Accept F, R

t. Isolating test ballots such that they are 

accounted for accurately in vote counts and are 

not reflect in official vote counts for specific 

candidates or measures

Accept F, R VAT has a separate 

test mode; isolating 

ballot is procedural

2.5 Post-Voting Functions

2.5.1 Closing the Polling Place (Precinct Count)

These standards for closing the polls are 
VSTL Certification # pending

Page 62 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

specific to precinct count systems. The system 

shall provide the means for:

a. Preventing the further casting of ballots once the 

polls has closed

Accept F, R VAT doesn't close, 

switched to Off 

b. Provides an internal test that verifies that the 

prescribed closing procedure has been followed, 

and that the device status is normal

Accept F, R

c. Incorporating a visible indication of system 

status

Accept F, R

d. Producing a diagnostic test record that verifies 

the sequence of events, and indicates that the 

extraction of voting data has been activated

Accept F, R

e. Precluding the unauthorized reopening of the 

polls once the poll closing has been completed 

for that election

Accept F, R DS200 reopened with 

authorization

2.5.2 Consolidating Vote Data

All systems provide a means to consolidate and 

report vote data from all polling places, and 

optionally from other sources such as absentee 

ballots, provisional ballots, and voted ballots 

requiring human review (e.g., write-in votes).

Accept F, R V1,2,6 -10 #27- Closed

2.5.3 Producing Reports

All systems shall be able to create reports 

summarizing the data on multiple levels.

Accept F, R V1,2,6 -10 #48 - Closed

2.5.3.1 Common Standards

All systems shall provide capabilities to:

a. Support of geographic reporting, which requires 

the reporting of all results for each contest at the 

precinct level and additional jurisdictional levels

Accept F, R V1,2,6 -10

b. Produce a printed report of the number of ballots 

counted by each tabulator

Accept F, R V1,2,6 -10 #101 - Closed

c. Produce a printed report for each tabulator of 

the results of each contest that includes the 

votes cast for each selection, the count of 

undervotes, and the count of overvotes

Accept F, R V1,2,6 -10 RFI 2007-06

d. Produce a consolidated printed report of the 

results for each contest of all votes cast 

(including the count of ballots from other 

sources supported by the system as specified 

by the vendor) that includes the votes cast for 

each selection, the count of undervotes, and the 

count of overvotes

Accept F, R V1,2,6 -10 RFI 2007-06

e. Be capable of producing a consolidated printed 

report of the combination of overvotes for any 

contest that is selected by an authorized official 

(e.g.; the number of overvotes in a given contest 

combining candidate A and candidate B, 

combining candidate A and candidate C, etc.)

Accept F, R V1,2,6 -10

f. Produce all system audit information required in 

Section 4.4 in the form of printed reports, or in 

electronic memory for printing centrally

Accept F, R V1,2,6 -10

g. Prevent data from being altered or destroyed by 

report generation, or by the transmission of 

results over telecommunications lines

Accept F, R V1,2,6 -10 Telecommunications 

is disabled in Unity 

3.2.0.0

2.5.3.2 Precinct Count Systems

In addition, all precinct count voting systems 

shall:

a. Prevent the printing of reports and the 

unauthorized extraction of data prior to the 

official close of the polling place

Accept F, R V1,2,6 -10

b. Provide a means to extract information from a 

transportable programmable memory device or 

Accept F, R V1,2,6 -10
VSTL Certification # pending

Page 63 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

data storage medium for vote consolidation

c. Consolidate the data contained in each unit into 

a single report for the polling place when more 

than one voting machine or precinct tabulator is 

used

Accept F, R V1,2,6 -10

d. Prevent data in transportable memory from 

being altered or destroyed by report generation, 

or by the transmission of results over 

telecommunications lines

Accept F, R V1,2,6 -10 Telecommunications 

is disabled in Unity 

3.2.0.0

2.5.4 Broadcasting Results

a. Provide only aggregated results, and not data 

from individual ballots

Accept F

b. Provide no access path from unofficial electronic 

reports or files to the storage devices for official 

data

Accept F

c. Clearly indicate on each report or file that the 

results it contains are unofficial

Accept F

2.6 Maintenance, Transpiration and Storage

All systems shall be designed and manufactured 

to facilitate preventive and corrective 

maintenance, conforming to the hardware 

standards described in Section 3. All vote 

casting and tally equipment designated for 

storage between elections shall:

a. Function without degradation in capabilities 

after transit to and from the place of use, as 

demonstrated by meeting the performance 

standards described in Section 3

b. Function without degradation in capabilities 

after storage between elections, as 

demonstrated by meeting the performance 

standards described in Section 3.

(See Section 3.2)

The vendor shall conduct summative usability 

tests on the voting system using individuals 

representative of the general population. The 

vendor shall document the testing performed 

and report the test results using the Common 

Industry Format. This documentation shall be 

included in the Technical Data Package 

submitted to the EAC for national certification.

Accept F RFI 2007-03

3 Hardware Standards

3.2 Performance Requirements

Performance requirements address a broad 

range of parameters (see below)

3.2.1 Accuracy Requirements 

Voting system accuracy addresses the accuracy 

of data for each of the individual ballot positions 

that could be selected by a voter, including the 

positions that are not selected. For a voting 

system, accuracy is defined as the ability of the 

system to capture, record, store, consolidate 

and report the specific selections and absence 

of selections, made by the voter for each ballot 

position without error. Required accuracy is 

defined in terms of an error rate that for testing 

purposes represents the maximum number of 

errors allowed while processing a specified 

volume of data.

RFI 2007-06

a.

1)

2)

For all paper-based voting systems:

Scanning ballot positions on paper ballots to 

detect selections for individual candidates and 

Accept F, R V1, 2, 6-

10
VSTL Certification # pending

Page 64 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

contests Conversion of selections detected on 

paper ballots into digital data

b.

1)

2)

For all DRE voting systems:

Recording the voter selections of candidates 

and contests into voting data storage

Recording voter selections of candidates and 

contests into ballot image storage independently 

from voting data storage

NA No DRE

c.

1)

For precinct-count voting systems (paper-based 

and DRE):

Consolidation of vote selection data from 

multiple precinct-based voting machines to 

generate jurisdiction-wide vote counts, including 

storage and reporting of the consolidated vote 

data

Accept F, R V1, 7, 9

d.

1)

For central-count voting systems (paper-based 

and DRE):

Consolidation of vote selection data from 

multiple counting devices to generate 

jurisdiction-wide vote counts, including storage 

and reporting of the consolidated vote data

Accept F, R V1,2,4,6-

10

For testing purposes, the acceptable error rate 

is defined using two parameters: the desired 

error rate to be achieved, and the maximum 

error rate that should be accepted by the test 

process. For each processing function indicated 

above, the voting system shall achieve a target 

error rate of no more than one in 10,000,000 

ballot positions, with a maximum acceptable 

error rate in the test process of one in 500,000 

ballot positions.

Accept F, V9 #144 Closed

3.2.2 Environmental Requirements

All voting systems shall be designed to 

withstand the environmental conditions 

contained in the appropriate test procedures of 

the Standards/Guidelines. These procedures will 

be applied to all devices for casting, scanning 

and counting ballots, except those that 

constitute COTS devices that have not been 

modified in any manner to support their use as 

part of a voting system and that have a 

documented record of performance under

conditions defined in the Standards/Guidelines.

The Technical Data Package supplied by the 

vendor shall include a statement of all 

requirements and restrictions regarding 

environmental protection, electrical service, 

recommended auxiliary power, 

telecommunications service, and any other 

facility or resource required for the proper 

installation and operation of the system.

Accept E

3.2.2.1 Shelter Requirements

Precinct count systems are designed for storage 

and operation in any enclosed facility ordinarily 

used as a warehouse or polling place, with 

prominent instructions as to any special storage 

requirements

Accept F

3.2.2.2 Space Requirements

The arrangement of the voting system does not 

impede performance of their duties by polling 

place officials, the orderly flow of voters through 

the polling place, or the ability for the voter to 

Accept F
VSTL Certification # pending

Page 65 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

vote in private

3.2.2.3 Furnishings and Fixtures

Any furnishings or fixtures provided as a part of 

voting systems, and any components provided 

by the vendor that are not a part of the system 

but that are used to support its storage, 

transportation, or operation, comply with the 

design and safety requirements of Subsection 

3.4.8.

Accept F, E

3.2.2.4 Electrical Supply

Components of voting systems that require an 

electrical supply shall meet the following 

standards: 

a. Precinct count systems operate with the 

electrical supply ordinarily found in polling 

places (Nominal 120 Vac/60Hz/1 phase)

Accept E

b. For components of voting systems that require 

an electrical supply, central count systems 

operate with the electrical supply ordinarily 

found in central tabulation facilities or computer 

room facilities (120vac/60hz/1, 208vac/60hz/3, 

or 240vac/60hz/2);

Accept E

c. All voting machines shall also be capable of 

operating for a period of at least 2 hours on 

backup power, such that no voting data is lost or 

corrupted nor normal operations interrupted. 

When backup power is exhausted the voting 

machine shall retain the contents of all 

memories intact. The backup power capability 

is not required to provide lighting of the voting 

area.

Accept E RFI 2008-02 

RFI 2008-06

3.2.2.5 Electrical Power Disturbance

Vote scanning and counting equipment for 

paper-based systems, and all DRE equipment, 

shall be able to withstand, without disruption of 

normal operation or loss of data:

RFI 2008-02

RFI 2008-06

a. Surges of 30% dip @10 ms; Accept E

b. Surges of 60% dip @100 ms & 1 sec Accept E

c. Surges of >95% interrupt @5Sec; Accept E

d. Surges of + or - 15% line variations of nominal 

line voltage

Accept E

e. Electric power increases of 7.5% and reductions 

of 12.5% of nominal specified power supply for a 

period of up to four hours at each power level.

Accept E

3.2.2.6 Electrical Fast Transient

Vote scanning and counting equipment for 

paper-based systems, and all DRE equipment, 

shall be able to withstand, without disruption of 

normal operation or loss of data, electrical fast 

transients of:

RFI 2008-10

a. 2 kV AC & DC External Power lines Accept E

b. + or - 1 kV all external wires > 3 m no control Accept E

c. + or - 2 kV all external wires control. Accept E

3.2.2.7 Lighting Surge

Vote scanning and counting equipment for 

paper-based systems, and all DRE equipment, 

shall be able to withstand, without disruption of 

normal operation or loss of data, surges of:

a. + or - 2 kV AC line to line Accept E

b. + or - 2 kV AC line to earth Accept E

c. + or – 0.5 kV DC line to line >10m Accept E

d. + or – 0.5 kV DC line to earth >10m Accept E
VSTL Certification # pending

Page 66 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

e. + or - 1 kV I/O sig/control >30m Accept E

3.2.2.8 Electrostatic Disruption

The vote scanning and counting equipment for 

paper-based systems, and all DRE equipment, 

is able to withstand ±15 kV air discharge and ±8 

kV contact discharge without damage or loss of 

data. The equipment may reset or have 

momentary interruption so long as normal 

operation is resumed without human 

intervention or loss of data. Loss of data means 

votes that have been completed and confirmed 

to the voter.

Accept E

3.2.2.9 Electromagnetic Radiation

Vote scanning and counting equipment for 

paper-based systems, and all DRE equipment, 

complies with the Rules and Regulations of the 

Federal Communications Commission, Part 15, 

Class B requirements for both radiated and 

conducted emissions

Accept E

3.2.2.10 Electromagnetic Susceptibility

Vote scanning and counting equipment for 

paper-based systems, and all DRE equipment, 

is able to withstand an electromagnetic field of 

10 V/m modulated by a 1 kHz 80% AM 

modulation over the frequency range of 80 MHz 

to 1000 MHz, without disruption of normal 

operation or loss of data

Accept E

3.2.2.11 Conducted RF Immunity

Vote scanning and counting equipment for 

paper-based systems, and all DRE equipment, 

shall be able to withstand, without disruption of 

normal operation or loss of data, conducted RF 

energy of:

a. 10V AC & DC power Accept E

b. 10V, 20 sig/control >3m. Accept E

3.2.2.12 Magnetic Fields Immunity

Vote scanning and counting equipment for 

paper-based systems, and all DRE equipment, 

shall be able to withstand, without disruption of 

normal operation or loss of data, AC magnetic 

fields of 30 A/m at 60 Hz

Accept E

3.2.2.13 Environmental Control – Operating Environment

Equipment used for election management 

activities or vote counting (including both 

precinct and central count systems) shall be 

capable of operation in temperatures ranging 

from 50 to 95 degrees Fahrenheit.

Accept E

3.2.2.14 Environmental Control – Transit and Storage

Equipment used for vote casting or for counting 

votes in a precinct count system, shall meet 

these specific minimum performance standards 

that simulate exposure to physical shock and 

vibration associated with handling and 

transportation by surface and air common 

carriers, and to temperature conditions 

associated with delivery and storage in an 

uncontrolled warehouse environment:

a. High and low storage temperatures ranging from 

-4 to +140 degrees Fahrenheit, equivalent to 

MIL-STD-810D, Methods 501.2 and 502.2, 

Procedure I-Storage;

Accept E

b. Bench handling equivalent to the procedure of Accept E
VSTL Certification # pending

Page 67 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

MIL-STD-810D, Method 516.3, Procedure VI;

c. Vibration equivalent to the procedure of MILSTD-810D, Method 514.3, Category 1- Basic 

Transportation, Common Carrier

Accept E

d. Uncontrolled humidity equivalent to the 

procedure of MIL-STD-810D, Method 507.2, 

Procedure I-Natural Hot-Humid.

Accept E

3.2.2.15 Data Network Requirements

Voting systems may use a local or remote data 

network. If such a network is used, then all 

components of the network shall comply with the 

telecommunications requirements described in 

Section 5 and the Security requirements 

described in Section 6.

Accept S, T Network functionality 

is disabled in the 

submitted voting 

system

3.2.3 Election Management System (EMS) 

Requirements

The Election Management System (EMS) 

requirements address electronic hardware and 

software used to conduct the pre-voting 

functions defined in Section 2 with regard to 

ballot preparation, election programming, ballot 

and program installation, readiness testing, 

verification at the polling place, and verification 

at the central location.

3.2.3.1 Recording Requirements

Voting systems shall accurately record all 

election management data entered by the user, 

including election officials or their designees.

a. Record every entry made by the user; Accept F, R V1-10

b. Add permissible voter selections correctly to the 

memory components of the device;

Accept F, R V1-10

c. Verify the correctness of detection of the user 

selections and the addition of the selections 

correctly to memory

Accept F, R V1-10

d. Add various forms of data entered directly by the 

election official or designee, such as text, line 

art, logos, and images

Accept F R, V1-10

e. Verify the correctness of detection of data 

entered directly by the user and the addition of 

the selections correctly to memory

Accept F, R V1-10

f. Preserve the integrity of election management 

data stored in memory against corruption by 

stray electromagnetic emissions, and internally 

generated spurious electrical signals

Accept E

g. Log corrected data errors by the system. Accept F, R V1, 2, 6-

10

3.2.3.2 Memory Stability

Memory devices used to retain election 

management data shall have demonstrated 

error-free data retention for a period of 22 

months.

Accept TDP Attestation from ESS

3.2.4 Vote Recording Requirements 

3.2.4.1 Common Standards

All voting systems shall provide voting booths or 

enclosures for poll site use. Such booths or 

enclosures may be integral to the voting system 

or supplied as components of the voting system, 

and shall:

a. Be integral to, or make provisions for installation 

of the voting device;

Accept F

b. Ensure by its structure stability against 

movement or overturning during entry, 

Accept F
VSTL Certification # pending

Page 68 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

occupancy, and exit by the voter

c. Provide privacy for the voter, and be designed in 

such a way as to prevent observation of the 

ballot by any person other than the voter

Accept F

d. Be capable of meeting the accessibility 

requirements of Subsection 3.2

Accept F

3.2.4.2 Paper-based Recording Standards 

The paper-based recording requirements 

govern:

• Ballot cards or sheets, and pages or 

assemblies of pages containing ballot field 

identification data

• Punching devices 

• Marking devices

• Frames or fixtures to hold the ballot while it is 

being punched

• Compartments or booths where voters record 

selections

• Secure containers for the collection of voted 

ballots

3.2.4.2.

1

Paper Ballot Standards 

Paper ballots used by paper-based voting 

systems shall meet the following standards: 

a. Paper ballots used by paper-based voting 

systems shall meet the following standards: 

Punches or marks that identify the unique ballot 

format, in accordance with Section 2.3.1.1.1.c., 

shall be outside the area in which votes are 

recorded, so as to minimize the likelihood that 

these punches or marks will be mistaken for 

vote responses and the likelihood that recorded 

votes will obliterate these punches or marks

Accept F, R V1, 2, 6-

10

No ballot punches

b. If printed or punched alignment marks are used 

to locate the vote response fields on the ballot, 

these marks shall be outside the area in which 

votes are recorded, so as to minimize the 

likelihood that these marks will be mistaken for 

vote responses and the likelihood that recorded 

votes will obliterate these marks

Accept F, R V1, 2, 6-

10

No ballot punches

c. The TDP shall specify the required paper stock,

size, shape, opacity, color, watermarks, field 

layout, orientation, size and style of printing, size 

and location of punch or mark fields used for 

vote response fields and to identify unique ballot 

formats, placement of alignment marks, ink for 

printing, and folding and bleed-through 

limitations for preparation of ballots that are 

compatible with the system.

Accept F R, V1, 2, 

4, 6-10

3.2.4.2.

2

Punching Devices

Punching devices used by voting systems shall: 

a. Be suitable for the type of ballot card specified; NA Not a punch card 

system

b. Facilitate the clear and accurate recording of 

each vote intended by the voter;

NA Not a punch card 

system

c. Be designed to avoid excessive damage to vote 

recorder components

NA Not a punch card 

system

d. Incorporate features to ensure that chad (debris) 

is removed, without damage to other parts of the 

ballot card.

NA Not a punch card 

system

3.2.4.2.

3

Marking Devices 

The Technical Data Package shall specify 

marking devices (such as pens or pencils) that, 
VSTL Certification # pending

Page 69 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

if used to make the prescribed form of mark, 

produce readable marked ballots such that the 

system meets the performance requirements for 

accuracy specified previously. These 

specifications shall identify:

a. Specific characteristics of marking devices that 

affect readability of marked ballots

Accept F

b. Performance capabilities with regard to each 

characteristic

Accept F

c. For marking devices manufactured by multiple 

external sources, a listing of sources and model 

numbers that are compatible with the system.

Accept F

3.2.4.1.

4

Frames or Fixtures for Punchcard Ballots 

A frame or fixture for punchcard ballot shall:

a. Hold the ballot card securely in the proper 

location and orientation for voting:

NA Not a punch card 

system

b. When contests not directly printed on the ballot 

card or sheet, incorporate an assembly of ballot 

label pages that identify offices and issues 

corresponding to the proper ballot format for the 

polling place where it is used and are aligned 

with the voting fields assigned to them

NA Not a punch card 

system

c. Incorporate a template to preclude perforation of 

the card except in the specified voting fields; a 

mask to allow punches only in fields designated 

by the format of the ballot; and a backing plate 

for the capture and removal of chad. The 

requirement may be satisfied by equipment of a 

different design as long it achieves the same 

result as the Standard with regard to:

NA Not a punch card 

system

1) Positioning the card; NA Not a punch card 

system

2) Association of ballot label information with 

corresponding punch fields;

NA Not a punch card 

system

3) Enable only those voting fields that correspond 

to the format of the ballot; and

NA Not a punch card 

system

4) Punching the fields and the positive removal of 

chad.

NA Not a punch card 

system

3.2.4.2.

5

Frames or Fixtures for Printed Ballots 

A frame or fixture for printed ballot cards is 

optional. If such a device is provided, it shall:

a. A frame or fixture for printed ballot cards is 

optional. If such a device is provided, it shall:

Accept F

a. Be of any size and shape consistent with its 

intended use;

Accept F

b. Position the card properly; Accept F

c. Hold the ballot card securely in its proper 

location and orientation for voting

Accept F

d. Comply with the design and construction 

requirements in Subsection 4.3.

Accept F

3.2.4.2.

6

Ballot Boxes and Ballot Transfer Boxes

Ballot boxes and ballot transfer boxes which 

serve as secure containers for the storage and 

transportation of voted ballots, shall:

Ballot boxes and ballot transfer boxes which 

serve as secure containers for the storage and 

transportation of voted ballots, shall:

Accept F

a. Be of any size, shape, and weight 

commensurate with their intended use

Accept F R

b. Incorporate locks or seals, and specifications in 

the system documentation

Accept F, S DS200 v.1:2.2.1

c. Provide specific points where ballots are Accept F
VSTL Certification # pending

Page 70 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

inserted, with all other points on the box 

constructed in a manner that prevents ballot 

insertion

d. For precinct count systems, contain separate 

compartments for segregating unread ballots, 

ballots with write-in votes, or irregularities that 

may require special handling or processing. In 

lieu of compartments, conversion processing 

may mark such ballots with an identifying spot or 

stripe to facilitate manual segregation

Accept F

3.2.4.3 DRE Systems Recording Requirements

3.2.4.3.

1

Activity Indicator

DRE systems shall include an audible or visible 

activity indicator providing the status of each 

voting device. This indicator shall:

a. Indicate whether the device has been activated 

for voting

Accept F, R VAT prompts to insert 

a ballot

b. Indicate whether the device is in use. Accept F, R

3.2.4.3.

2

DRE System Vote Recording

To ensure vote recording accuracy and integrity 

while protecting the anonymity of the voter, all 

DRE systems shall: 

a. Contain all mechanical, electromechanical, and 

electronic components; software; and controls 

required to detect and record the activation of 

selections made by the voter in the process of 

voting and casting a ballot

Accept F, R V1, 2, 6, 

10

b. Incorporate redundant memories to detect and 

allow correction of errors caused by the failure 

of any of the individual memories

NA No DRE

c.

1)

2)

Provide at least two processes that record the 

voter‟s selections that:

• To the extent possible, are isolated from each 

other

• Designate one process and associated storage 

location as the main vote detection, 

interpretation, processing and reporting path

NA No DRE

Use a different process to store ballot images, 

for which the method of recording may include 

any appropriate encoding or data compression 

procedure consistent with the regeneration of an 

unequivocal record of the ballot as cast by the 

voter.

NA No DRE

d. Provide a capability to retrieve ballot images in a 

form readable by humans.

NA No DRE

e. Ensure that all processing and storage protects 

the anonymity of the voter.

Accept F S

3.2.4.3.

3

Recording Accuracy

DRE systems meet the following requirements 

for recording accurately each vote and ballot 

cast:'

a. Detect every selection made by the voter Accept F, R V1, 2, 6, 

10

b. Correctly add permissible selections to the 

memory components of the device

Accept F, R V1, 2, 6, 

10

Temporary memory 

prior to VAT printing

c. Verify the correctness of the detection of the 

voter selections and the addition of the 

selections to memory

Accept F, R V1, 2, 6, 

10

d. Achieve an error rate not to exceed the 

requirement indicated in Section 3.2.1

Accept F VAT paper ballot 

marking

e. Preserve the integrity of voting data and ballot 

images (for DRE machines) stored in memory 

NA No DRE
VSTL Certification # pending

Page 71 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

for the official vote count and audit trail purposes 

against corruption by stray electromagnetic 

emissions, and internally generated spurious 

electrical signals

f. Maintain a log of corrected data Accept F, R V1, 2, 6, 

10

3.2.4.3.

4

Recording Reliability

Recording reliability refers to the ability of the 

DRE system to record votes accurately at its 

maximum rated processing volume for a 

specified period of time. The DRE system shall 

record votes reliably in accordance with the 

requirements of Subsection 3.4.3.

Accept F VAT paper ballot 

marking

3.2.5 Paper-based Conversion Requirements

3.2.5.1 Ballot Handling

Ballot handling consists of a ballot card‟s 

acceptance, movement through the read station 

and transfer into a collection station or 

receptacle.

Accept F, R V1, 2, 6, 

7, 9, 10

3.2.5.1.

1

Capacity (Central Count)

The capacity to convert the marks on individual 

ballots into signals is uniquely important to 

central count systems. The capacity for a central 

count system shall be documented by the 

vendor. This documentation shall include 

capacity for individual components that impact 

the overall capacity.

Accept F, R V1, 2, 4, 

8, 10

3.2.5.1.

2

Exception Handling (Central Count)

This requirement refers to the handling of ballots 

when they are unreadable or some condition is 

detected requiring that the cards be segregated 

from normally processed ballots for human 

review. In response to an unreadable ballot or a 

write-in vote all central count paper-based 

systems shall central count paper-based 

systems shall:

a.

b.

c.

Outstack the ballot, or

Stop the ballot reader and display a message 

prompting the election official or

designee to remove the ballot, or

Mark the ballot with an identifying mark to 

facilitate its later identification.

Accept F, R V1, 2, 6-

10

Additionally, the system shall a capability that 

can be activated by an authorized election 

official to identify ballots containing overvotes, 

blank ballots, and ballots containing undervotes 

in a designated race. If enabled, these 

capabilities shall perform one of the above 

actions in response to the indicated condition

Accept F, R V1, 2, 6-

10

3.2.5.1.

3

Exception Handling (Precinct Count)

This requirement refers to the handling of ballots 

for precinct count system when they are 

unreadable or when some condition is detected 

requiring that the cards be segregated from 

normally processed ballots for human review. All 

paper based precinct count systems shall:

a. In response to an unreadable or blank ballot, 

return the ballot and provide a

message prompting the voter to examine the 

ballot

Accept F, R V1, 2, 6-

10

b. In response to a ballot with a write-in vote, Accept F, R
VSTL Certification # pending

Page 72 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

segregate the ballot or mark the ballot with an 

identifying mark to facilitate its later identification

c.

1)

2)

3)

4)

5)

In response to a ballot with an overvote the 

system shall:

• Provide a capability to identify an overvoted 

ballot

• Return the ballot

• Provide an indication prompting the voter to 

examine the ballot

• Allow the voter to correct the ballot

• Provide a means for an authorized election 

official to deactivate this capability entirely and 

by contest

Accept F, R

d.

1)

2)

3)

4)

5)

In response to a ballot with an undervote, the 

system shall:

• Provide a capability to identify an undervoted 

ballot

• Return the ballot

• Provide an indication prompting the voter to 

examine the ballot

• Allow the voter to submit the ballot with the 

undervote

• Provide a means for an authorized election 

official to deactivate this capability

Accept F, R V1, 2, 6-

10

3.2.5.1.

4

Multiple Feed Prevention

Multiple feed refers to the situation arising when 

a ballot reader attempts to read more than one 

ballot at a time. The requirements govern the 

ability of a ballot reader to prevent multiple feed 

or to detect and provide an alarm indicating 

multiple feed.

a. If multiple feed is detected, the card reader shall 

halt in a manner that permits the operator to 

remove the unread cards causing the error, and 

reinsert them in the card input hopper

Accept F R

b. The frequency of multiple feeds with ballots 

intended for use with the system shall not 

exceed 1 in 10,000

Accept F

3.2.5.2 Ballot Reading Accuracy

This paper-based system requirement governs 

the conversion of the physical ballot into 

electronic data. Reading accuracy for ballot 

conversion refers to the ability to:

♦ Recognize vote punches or marks, or the 

absence thereof, for each possible selection on 

the ballot 

♦ Discriminate between valid punches or marks 

and extraneous perforations, smudges, and 

folds 

♦ Convert the vote punches or marks, or the 

absence thereof, for each possible selection on 

the ballot into digital signals

To ensure accuracy, paper-based systems shall:

a. Detect punches or marks that conform to vendor 

specifications with an error rate not exceeding 

the requirement indicated in Section 3.2.1

Accept F, R

V1,2,4, 

6-10

#144 Closed

b. Ignore, and not record, extraneous perforations, 

smudges, and folds;

Accept F, R

c. Reject ballots that meet all vendor specifications 

at a rate not to exceed 2 percent.

Accept F, R, 

V1,2,4,6-

10

1 incidence @ DS200 

& M650 prompted for 

maintenance at iBeta

3.2.6 Tabulation Processing Requirements
VSTL Certification # pending

Page 73 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

3.2.6.1 Paper-based Processing Requirements 

3.2.6.1.

1

Processing Accuracy 

Processing accuracy refers to the ability of the 

system to receive electronic signals produced by 

punches for punchcard systems and vote marks 

and timing information for marksense systems; 

perform logical and numerical operations upon 

these data; and reproduce the contents of 

memory when required, without error. Specific 

requirements are detailed below:

a. Processing accuracy shall be measured by vote 

selection error rate, the ratio of uncorrected vote 

selection errors to the total number of ballot 

positions that could be recorded across all 

ballots when the system is operated at its 

nominal or design rate of processing

Accept See 

3.2.6.1.1

d

There is no pass/fail 

criteria in this 

requirement It is a 

definition of 

processing accuracy

b. The vote selection error rate shall include data 

that denotes ballot style or precinct as well as 

data denoting a vote in a specific contest or 

ballot proposition

Accept F, R V1,2,4,6-

10

c The vote selection error rate shall include all 

errors from any source

Accept F, R V1,2,4,6-

10

d. The vote selection error rate shall not exceed 

the requirement indicated in Subsection 4.1.1

Accept F, R

V1,2,4, 

6-10

#144 - Closed

V1,2,6,7,9,10 -DS200

3.2.6.2 DRE System Processing Requirements 

The DRE voting systems processing 

requirements address all mechanical devices, 

electromechanical devices, electronic devices, 

and software required to process voting data 

after the polls are closed.

3.2.6.2.

1

Processing Speed

DRE voting systems shall meet the following 

requirements for processing speed:

a. Operate at a speed sufficient to respond to any 

operator and voter input without perceptible 

delay (no more than three seconds)

Accept F VAT ballot marking; 

printing exceeds 3 

seconds

b. if the consolidation of polling place data is done 

locally, perform this consolidation in a time not 

to exceed five minutes for each device in the 

polling place

NA No DRE

3.2.6.2.

2

Processing Accuracy

Processing accuracy is defined as the ability of 

the system to process voting data stored in DRE 

voting devices or in removable memory modules 

installed in such devices. Processing includes all 

operations to consolidate voting data after the 

polls have been closed. DRE voting systems 

shall:

a. Produce reports that are completely consistent, 

with no discrepancy among reports of voting 

device data produced at any level

Accept F, R V1,2,4,6-

10

b. Produce consolidated reports containing 

absentee, provisional or other voting data that 

are similarly error-free. Any discrepancy, 

regardless of source, is resolvable to a 

procedural error, to the failure of a non-memory 

device or to an external cause

Accept F, R V1,2,4,6-

10

3.2.6.2.

3

Memory Stability

DRE system memory devices used to retain 

control programs and data shall have 

NA No DRE
VSTL Certification # pending

Page 74 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

demonstrated error-free data retention for a 

period of 22 months. Error-free retention may be 

achieved by the use of redundant memory 

elements, provided that the capability for conflict 

resolution or correction among elements is 

included.

3.2.7 Reporting Requirements 

3.2.7.1 Removable Storage Memory

All storage media that can be removed from the 

voting system and transported to another 

location for readout and report generation, these 

media shall use devices with demonstrated 

error-free retention for a period of 22 months 

under the environmental conditions for operation 

and non-operation contained in Section 3.2.2. 

Examples of removable storage media include: 

programmable read-only memory (PROM), 

random access memory (RAM) with battery 

backup, magnetic media or optical media.

Accept TDP 

Review

Attestation from ESS

3.2.7.2 Printers

All printers used to produce reports of the vote 

count shall be capable of producing:

a. Alphanumeric headers Accept F, R V1,2,4,6-

10

b. Election, office and issue labels Accept F, R V1,2,4,6-

10

c. Alphanumeric entries generated as part of the 

audit record.

Accept F, R V1,2,4,6-

10

3.2.8 Vote Data Management Requirements 

The vote data management requirements for all 

systems address capabilities that manage, 

process, and report voting data after the data 

has been consolidated at the polling place or 

other jurisdictional levels. These capabilities 

allow the system to:

a. Consolidate voting data from polling place data 

memory or transfer devices 

Accept F, R V1,2,4,6-

10

b. Report polling place summaries; and Accept F, R V1,2,4,6-

10

c. Process absentee ballots, data entered 

manually, and administrative ballot definition 

data.

Accept F, R V1,2,4,6-

10

The requirements address all hardware and 

software required to generate output reports in 

the various formats required by the using 

jurisdiction.

Accept F, R V1,2,4,6-

10

3.2.8.1 Data File Management

All voting systems shall provide the capability to:

a. Integrate voting data files with ballot definition 

files

Accept F, R V1,2,4,6-

10

b. Verify file compatibility. Accept F, R V1,2,4,6-

10

c. Edit and update files as required. Accept F, R V1,2,4,6-

10

3.3 Physical Characteristics

3.3.1 Size

There is no numerical limitation on the size of 

any voting equipment, but the size of each 

voting machine should be compatible with its

intended use and the location at which the 

equipment is to be used.

Accept F RFI 2007-05

3.3.2 Weight
VSTL Certification # pending

Page 75 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

There is no numerical limitation on the weight of 

any voting equipment, but the weight of each 

voting machine should be compatible with its 

intended use and the location at which the 

equipment is to be used.

Accept F

3.3.3 Transport and Storage of Precinct Systems

All precinct voting systems shall:

a. Provide a means to safely and easily handle, 

transport, and install voting equipment, such as 

wheels or a handle or handles

Accept F No handling issues 

noted by iBeta

b.

1)

2)

Be capable of using, or be provided with, a 

protective enclosure rendering the equipment 

capable of withstanding:

Impact, shock and vibration loads associated 

with surface and air transportation

Stacking loads associated with storage 

Accept F

3.3.4 Control Constructs Source 

Code 

Review 

RFI 2007-02

3.3.5 Naming Conventions Source 

Code 

Review 

3.4 Design, Construction, and Maintenance 

Characteristics

3.4.1 Materials Process and Parts

The approach to system design is unrestricted, 

and may incorporate any form or variant of 

technology capable of meeting the voting 

systems requirements and standards. Precinct 

count systems shall be designed in accordance 

with best commercial practice for 

microcomputers, process controllers, and their 

peripheral components. Central count voting 

systems and equipment used in a central 

tabulating environment shall be designed in 

accordance with best commercial and industrial 

practice. All voting systems shall:

a. Be designed and constructed so that the 

frequency of equipment malfunctions and 

maintenance requirements are reduced to the 

lowest level consistent with cost constraints. 

Accept F R

b. Include, as part of the accompanying TDP, an 

approved parts list

Accept F

c. Exclude parts or components not included in the 

approved parts list.

Accept F

3.4.2 Durability

All voting systems shall be designed to 

withstand normal use without deterioration and 

without excessive maintenance cost for a period 

of ten years.

Accept F, TDP 

Review

RFI 2008-05

Attestation from 

ES&S

3.4.3 Reliability

The reliability of voting system devices shall be 

measured as Mean Time Between Failure 

(MTBF) for the system submitted for testing. 

MBTF is defined as the value of the ratio of 

operating time to the number of failures which 

have occurred in the specified time interval. A 

typical system operations scenario consists of 

approximately 45 hours of equipment operation, 

consisting of 30 hours of equipment set-up and 

readiness testing and 15 hours of elections 

operations. For the purpose of demonstrating 

Accept E
VSTL Certification # pending

Page 76 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

compliance with this requirement, a failure is 

defined as any event which results in either the:

a. Loss of one or more functions

b. Degradation of performance such that the 

device is unable to perform its intended function 

for longer than 10 seconds

The MTBF demonstrated during certification 

testing shall be at least 163 hours.

3.4.4 Maintainability

See the requirement for detail about 

Maintainability

Accept F

3.4.4.1 Physical Attributes

The following physical attributes will be 

examined to assess reliability:

a. Presence of labels and the identification of test 

points

Accept F R

b. Provision of built-in test and diagnostic circuitry 

or physical indicators of condition

Accept F

c. Presence of labels and alarms related to failures Accept F

d. Presence of features that allow non-technicians 

to perform routine maintenance tasks (such as 

update of the system database)

Accept F

3.4.4.2 Additional Attributes

The following additional attributes will be 

examined to assess maintainability:

a. Ease of detecting that equipment has failed by a 

non-technician

Accept F R

b. Ease of diagnosing problems by a trained 

technician

Accept F

c. Low false alarm rates (i.e., indications of 

problems that do not exist)

Accept F

d. Ease of access to components for replacement Accept F

e. Ease with which adjustment and alignment can 

be performed

Accept F

f. Ease with which database updates can be 

performed by a non-technician

Accept F

g. Adjust, align, tune or service components Accept F

3.4.5 AvailabilityThe availability of a voting system is defined as 

the probability that the equipment (and 

supporting software) needed to perform 

designated voting functions will respond to

operational commands and accomplish the

function. The voting system shall meet the 

availability standard for each of the following 

voting functions:

a. For all paper-based voting systems: Accept F, E

1 Recording voter selections (such as by ballot 

marking or punch)

Accept F, E

2 Scanning the punches or marks on paper ballots 

and converting them into digital data

Accept F, E

b. For all DRE systems, recording and storing 

voter ballot selections

Accept F, E

c. For precinct count systems (paper-based and 

DRE), consolidation of vote selection data from 

multiple precinct based systems to generate 

jurisdiction-wide vote counts, including storage 

and reporting of the consolidated vote data

Accept F, E

d. For central-count systems (paper-based and 

DRE), consolidation of vote selection data from 

multiple counting devices to generate 

jurisdiction-wide vote counts, including storage 

Accept F, E
VSTL Certification # pending

Page 77 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

and reporting of the consolidated vote data 

System availability is measured as the ratio of 

the time during which the system is operational 

(up time) to the total time period of operation (up 

time plus down time). Inherent availability (Ai) is 

the fraction of time a system is functional, based 

upon Mean Time Between Failure (MTBF) and 

Mean Time To Repair (MTTR), that is: Ai = 

(MTBF)/(MTBF + MTTR) MTTR is the average 

time required to perform a corrective 

maintenance task during periods of system 

operation. Corrective maintenance task time is 

active repair time, plus the time attributable to 

other factors that could lead to logistic or 

administrative delays, such as travel notification 

of qualified maintenance personnel and travel 

time for such personnel to arrive at the 

appropriate site. Corrective maintenance may 

consist of substitution of the complete device or 

one of its components, as in the case of precinct 

count and some central count systems, or it may 

consist of on-site repair. 

The voting system shall achieve at least 99 

percent availability during normal operation for 

the functions indicated above. This standard 

encompasses for each function the combination 

of all devices and components that support the 

function, including their MTTR and MTBF 

attributes.

Accept F, E

Vendors shall specify the typical system 

configuration that is to be used to assess 

availability, and any assumptions made with 

regard to any parameters that impact the MTTR. 

These factors shall include at a minimum:

Accept F

a. Recommended number and locations of spare 

devices or components to be kept on hand for 

repair purposes during periods of system 

operation

Accept F

b. Recommended number and locations of 

qualified maintenance personnel who need to be 

available to support repair calls during system 

operation. Organizational affiliation (i.e., 

jurisdiction, vendor) of qualified maintenance 

personnel

Accept F

c. Organizational affiliation (i.e., jurisdiction, 

vendor) of qualified maintenance personnel

Accept F

3.4.6 Product Marking

All voting systems shall:

a. Identify all devices with a permanently affixed 

nameplate or label containing the name of the 

manufacturer or vendor, the name of the device, 

its part or model number, its revision letter, its 

serial number, and if applicable, its power 

requirements

Accept F

b. Display on each device a separate data plate 

containing a schedule for and list of operations 

required to service or to perform preventive 

maintenance

Accept F

c. Display advisory caution and warning 

instructions to ensure safe operation of the 

equipment and to avoid exposure to hazardous 

electrical voltages and moving parts at all 

Accept F
VSTL Certification # pending

Page 78 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

locations where operation or exposure may 

occur

3.4.7 Workmanship

To help ensure proper workmanship, all 

manufacturers of voting systems shall:

a. Adopt and adhere to practices and procedures 

to ensure their products are free from damage 

or defect that could make them unsatisfactory 

for their intended purpose

Accept F

b. Ensure components provided by external 

suppliers are free from damage or defect that 

could make them unsatisfactory for their 

intended purpose.

Accept F

3.4.8 Safety

All voting systems shall meet the following 

requirements for safety:

RFI 2008-09

a. All voting system and their components shall be 

designed to eliminate hazards to personnel or 

the equipment itself.

Accept E

b. Defects in design and construction that can 

result in personal injury or equipment damage 

must be detected and corrected before voting 

systems and components are placed into 

service.

Accept E

c. Equipment design for personnel safety is equal 

to or better than the appropriate requirements of 

the Occupational Safety and Health Act, Code of 

Federal Regulations, as identified in Title 29, 

part 1910

Accept E

3.4.9 Human Engineering- Controls and Displays

All voting systems and components shall be 

designed and constructed so as to simplify and 

facilitate the functions required , and to eliminate 

the likelihood of erroneous stimuli and 

responses on the part of the voter or operator. 

All voting systems shall meet the following 

requirements for controls and displays:

a. In all systems, controls used by the voter or 

equipment operator shall be conveniently 

located, shall use designs consistent with their 

functions, and shall be clearly labeled. 

Instruction plates are provided, if necessary to 

avoid ambiguity or incorrect actuation.

Accept F R

b. Information or data displays are large enough to 

be readable by voters and operators with no 

disabilities and by voters with disabilities 

consistent with the requirements defined is 

Section 2.2.7 of the Standards.

Accept F

c. Status displays meet the same requirements as 

data displays, and they shall also follow 

conventional industrial practice with respect to 

color:

Accept F

1 Green, blue, or white displays shall be used for 

indications of normal status;

Accept F

2 Amber indicators shall be used to indicate 

warnings or marginal status; and

Accept F

3 Red indicators shall be used to indicate error 

conditions or equipment states that may result in 

damage or hazard to personnel; and unless the 

equipment is designed to halt under conditions 

of incipient damage or hazard, an audible alarm 

is also be provided.

Accept F
VSTL Certification # pending

Page 79 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

d. Color coding shall be selected so as to assure 

correct perception by voters and operators with 

color blindness; and shall not bet used as the 

only means of conveying information, indicating 

an action, prompting a response, or 

distinguishing a visual element (see Appendix C 

for suggested references).

Accept F

e. The system‟s display does not use flashing or 

blinking text objects, or other elements having a 

flash or blink frequency, greater than 2 Hz and 

lower than 55 Hz

Accept F

4 Software Standards

4.1.1 Software Sources RFI 2008-03

4.3 Data and Document Retention

All systems shall:

a. Maintain the integrity of voting and audit data 

during an election, and for at least 22 months 

thereafter, a time sufficient to resolve most 

contested elections and support other activities 

related to the reconstruction and investigation of 

a contested election

Accept TDP 

Review

Attestation from ESS

b. Protect against the failure of any data input or 

storage device at a location controlled by the 

jurisdiction or its contractors, and against any 

attempt at improper data entry or retrieval

Accept S, V4 #132 Closed

4.4 Audit Record Data

Audit trails are essential to ensure the integrity 

of a voting system. Operational requirements for 

audit trails are described in Subsection 2.2.5.2 

of the Standards. Audit record data are 

generated by these procedures. The audit 

record data in the following subsections are 

essential to the complete recording of election 

operations and reporting of the vote tally. This 

list of audit records may not reflect the design 

constructs of some systems. Therefore, vendors 

shall supplement it with information relevant to 

the operation of their specific systems.

Accept F, S Document review

4.4.1 Pre-election Audit Records

During election definition and ballot preparation, 

the system shall audit the preparation of the 

baseline ballot formats and modifications to 

them, a description of these modifications, and 

corresponding dates. The log shall include:

Accept F,R V1,2,4,6-

10

a. The allowable number of selections for an office 

or issue;

Accept F, R

b. The combinations of voting patterns permitted or 

required by the jurisdiction

Accept F, R

c. The inclusion or exclusion of offices or issues as 

the result of multiple districting within the polling 

place

Accept F, R

d. Any other characteristics that may be peculiar to 

the jurisdiction, the election, or the polling 

place's location

Accept F, R

e. Manual data maintained by election personnel Accept F, R

f. Samples of all final ballot formats Accept F, R

g. Ballot preparation edits listings. Accept F, R

4.4.2 System Readiness Audit Records

The following minimum requirements apply to 

system readiness audit records:

a. Prior to the start of ballot counting, a system 

process shall verify hardware and software 

Accept F, R V1,2,4,6-

10
VSTL Certification # pending

Page 80 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

status and generate a readiness audit record. 

This record shall include the identification of the 

software release, the identification of the 

election to be processed, and the results of 

software and hardware diagnostic tests

b. In the case of systems used at the polling place, 

the record shall include polling place 

identification

Accept F, R V9

c. The ballot interpretation logic shall test and 

record the correct installation of ballot formats 

on voting devices

Accept F, R V2, 7, 9

d. The software shall check and record the status 

of all data paths and memory locations to be 

used in vote recording to protect against 

contamination of voting data 

Accept F, R

e. Upon the conclusion of the tests, the software 

shall provide evidence in the audit record that 

the test data have been expunged

Accept F, R

f. If required and provided, the ballot reader and 

arithmetic-logic unit shall be evaluated for 

accuracy, and the system shall record the 

results. It shall allow the processing or simulated 

processing of sufficient test ballots to provide a 

statistical estimate of processing accuracy

Accept F

g.

1)

2)

3)

4)

For systems that use a public network, provide a 

report of test ballots that includes:

Number of ballots sent

When each ballot was sent

Machine from which each ballot was sent

specific votes or selections contained in the 

ballot

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

4.4.3 In-Process Audit Records

In-process audit records document system 

operations during diagnostic routines and the 

casting and tallying of ballots. At a minimum, the 

in-process audit records shall contain:

RFI 2008-07

a. Machine generated error and exception 

messages to demonstrate successful recovery. 

Examples include, but are not necessarily 

limited to:

Accept V1-10 

Code 

review 

v.1:4.2.3

e

F, R

1) The source and disposition of system interrupts 

resulting in entry into exception handling 

routines

Accept V1-10. F, 

R

2) All messages generated by exception handlers Accept V1-10, F, 

R

3) The identification code and number of 

occurrences for each hardware and software 

error or failure

Accept F, R

4) Notification of system login or access errors, file 

access errors, and physical violations of security 

as they occur, and a summary record of these 

events after processing

Accept S F, R #138, 139

5) Other exception events such as power failures, 

failure of critical hardware components, data 

transmission errors or other types of operating 

anomalies

Accept S

b. Critical system status messages other than 

informational messages displayed by the system 

during the course of normal operations. These 

items include, but are not limited to:

Accept F, R, S v.2: 3.3.1

1) Diagnostic and status messages upon startup Accept F, R
VSTL Certification # pending

Page 81 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

2) The “zero totals” check conducted before 

opening the polling place or counting a precinct 

centrally

Accept F, R, S v.2: 3.3.1

3) For paper-based systems, the initiation or 

termination of card reader and communications 

equipment operation

Accept F, R

4) For DRE machines at controlled voting 

locations, the event (and time, if available) of 

activating and casting each ballot (i.e., each 

voter's transaction as an event). This data can 

be compared with the public counter for 

reconciliation purposes

Accept F R VAT ballot printing

c. Non-critical status messages that are generated 

by the machine's data quality monitor or by 

software and hardware condition monitors

Accept F

d. System generated log of all normal process 

activity and system events that require operator 

intervention, so that each operator access can 

be monitored and access sequence can be 

constructed

Accept F, R, S v.2: 3.3.1

4.4.4 Vote Tally Data

In addition to the audit requirements described 

above, other election-related data is essential 

for reporting results to interested parties, the 

press, and the voting public, and is vital to 

verifying an accurate count. Voting systems 

shall meet these reporting requirements by 

providing software capable of obtaining data 

concerning various aspects of vote counting and 

producing printed reports. At a minimum, vote 

tally data shall include:

a. Number of ballots cast, using each ballot 

configuration, by tabulator, by precinct, and by 

political subdivision

Accept F, R V1,2,4,6-

10

#34, 35- Closed

b. Candidate and measure vote totals for each 

contest, by tabulator

Accept F, R V1,2,4,6-

10

#35 - Closed

c. The number of ballots read within each precinct 

and for additional jurisdictional levels, by 

configuration, including separate totals for each 

party in primary elections

Accept F, R V1,2,4,6-

10

d. Separate accumulation of overvotes and 

undervotes for each contest, by tabulator, 

precinct and for additional jurisdictional levels 

(no overvotes would be indicated for DRE voting 

devices)

Accept F, R

e. For paper-based systems only, the total number 

of ballots both able to be processed and unable 

to be processed; and if there are multiple card 

ballots, the total number of cards read

Accept F, R

For systems that produce an electronic file 

containing vote tally data, the contents of the file 

shall include the same minimum data cited 

above for printed vote tally reports.

Accept F, R V1,2,4,6-

10

4.5 Voter Secrecy on DRE Systems

All DRE systems shall ensure vote secrecy by:

a. Immediately after the voter chooses to cast his 

or her ballot, record the voter‟s selections in the 

memory to be used for vote counting and audit 

data (including ballot images), and erase the 

selections from the display, memory, and all 

other storage, including all forms of temporary 

storage

Accept S Post printing on the 

VAT
VSTL Certification # pending

Page 82 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

b. Immediately after the voter chooses to cancel 

his or her ballot, erase the selections from the 

display and all other storage, including buffers 

and other temporary storage

Accept S Pre-printing on the 

VAT

5 Telecommunications 

5.2 Design, Construction, and Maintenance 

Requirement

Design, construction, and maintenance 

requirements for telecommunications represent 

the operational capability of both system 

hardware and software. These capabilities shall 

be considered basic to all data transmissions.

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

5.2.1 Accuracy

The telecommunications components of all 

voting systems shall meet the accuracy 

requirements of 3.4.1.

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

5.2.2 Durability

The telecommunications components of all 

voting systems shall meet the Durability 

requirements of 3.4.2.

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

5.2.3 Reliability

The telecommunications components of all 

voting systems shall meet the Reliability 

requirements of 3.4.3.

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

5.2.4 Maintainability

The telecommunications components of all 

voting systems shall meet the maintainability 

requirements of 3.4.4.

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

5.2.5 Availability

The telecommunications components of all 

voting systems shall meet the availability 

requirements of 3.4.5.

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

5.2.6 Integrity

For WANs using public telecommunications, 

boundary definition and implementation shall 

meet the requirements below.

a. Outside service providers and subscribers of 

such providers shall not be given direct access 

or control of any resource inside the boundary.

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

b. Voting system administrators shall not require 

any type of control of resources outside this 

boundary. Typically, an end point of a 

telecommunications circuit will be a subscriber 

termination on a Digital Service Unit/Customer 

Service Unit although the specific technology 

configuration may vary. Regardless of the 

technology used, the boundary point must 

ensure that everything on the voting system side 

is locally configured and controlled by the 

election jurisdiction while everything on the 

public network side is controlled by an outside 

service provider.

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

c. The system shall be designed and configured 

such that it is not vulnerable to a single point of 

failure in the connection to the public network 

which could cause total loss of voting 

capabilities at any polling place.

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

5.2.7 Confirmation

Confirmation occurs when the system notifies 

the user of the successful or unsuccessful 

completion of the data transmission, where 

successful completion is defined as accurate 
VSTL Certification # pending

Page 83 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

receipt of the transmitted data. To provide 

confirmation, the telecommunications 

components of a voting system shall 

d. Notify the user of the successful or unsuccessful 

completion of the data transmission; and 

Accept S, T No network transmission; see 2.2.2.1 

d & e

e. In the event of unsuccessful transmission, notify 

the user of the action to be taken.

Accept S, T No network transmission; see 2.2.2.1 

d & e

6 Security Standards

6.2 Access Controls 

6.2.1 Access Control Policy 

6.2.1.1 General Access Control Policy RFI 2008-03

Although the jurisdiction in which the voting 

system is operated is responsible for 

determining the access policies for each 

election, the vendor shall provide a description 

of recommended policies for:

Accept S- Doc 

Review

Tested w/ 

2.2.1

#67 -Closed

a. Software access controls; Accept S- Doc 

Review

Tested w/ 

2.2.1

b. Hardware access controls; Accept S- Doc 

Review

Tested w/ 

2.2.1

c. Communications; Accept S- Doc 

Review

Tested w/ 

2.2.1

Networking is 

disabled

d. Effective password management; Accept S- Doc 

Review

Tested w/ 

2.2.1

#59 - Closed

e. Protection abilities of a particular operating 

system;

Accept S- Doc 

Review

Tested w/ 

2.2.1

f. General characteristics of supervisory access 

privileges;

Accept S- Doc 

Review

Tested w/ 

2.2.1

g. Segregation of duties; and Accept S- Doc 

Review

Tested w/

2.2.1

h. Any additional relevant characteristics. Accept S- Doc 

Review

Tested w/ 

2.2.1

6.2.1.2 Individual Access Privileges

Voting system vendors shall:

a. Identify each person to whom access is granted, 

and the specific functions and data to which 

each person holds authorized access

Accept S- Doc 

Review

Tested w/ 

2.2.1

#60, 68 - Closed

b. Specify whether an individual‟s authorization is 

limited to a specific time, time interval or phase 

of the voting or counting operations

Accept S- Doc 

Review

Tested w/ 

2.2.1

#60, 75 - Closed

c. Permit the voter to cast a ballot expeditiously, 

but preclude voter access to all aspects of the 

vote counting processes

Accept S- Doc 

Review

Tested w/ 

2.2.1

6.2.2 Access Control Measures

Vendors shall provide a detailed description of 

all system access control measures designed to 

permit authorized access to the system and 

prevent unauthorized access, such as:

a. Use of data and user authorization Accept S- Doc & 

Review

Tested w/ 

2.2.1

#70 - Closed

b. Program unit ownership and other regional 

boundaries

Accept S- Doc 

Review

c. One-end or two-end port protection devices Accept S- Doc 

Review

d. Security kernels Accept S- Doc 

Review

e. Computer-generated password keys Accept S- Doc & 

Code 

Review

#148 - Closed

f. Special protocols Accept S- Doc #79 - Closed
VSTL Certification # pending

Page 84 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

Review

g. Message encryption and Accept S- Doc & 

Code 

Review

#71, 72, 81, 82, 149 

Closed

h. Controlled access security. Accept S- Doc 

Review

Tested w/ 

2.2.1

Vendors also shall define and provide a detailed 

description of the methods used to prevent 

unauthorized access to the access control 

capabilities of the system itself.

Accept S- Doc 

Review

#84 - Closed

6.3 Physical Security Measures

A voting system‟s sensitivity to disruption or 

corruption of data depends, in part, on the 

physical location of equipment and data media, 

and on the establishment of secure 

telecommunications among various locations. 

Most often, the disruption of voting and vote 

counting results from a physical violation of one 

or more areas of the system thought to be 

protected. Therefore, security procedures shall 

address physical threats and the corresponding 

means to defeat them.

Accept S- Doc 

Review

6.3.1 Polling Place Security

For polling place operations, vendors shall 

develop and provide detailed documentation of 

measures anticipate and counteract vandalism, 

civil disobedience, and similar occurrences. The 

measures shall.

a. Allow the immediate detection of tampering with 

vote casting devices and precinct ballot 

counters. 

Accept S- Doc 

Review

b. Control physical access to a 

telecommunications link if such a link is used

Accept S- Doc 

Review

6.3.2 Central Count Location Security

a.

b.

c.

d.

Vendors shall develop and document in detailed 

measures to be taken in a central counting 

environment. These measures shall include 

physical and procedural controls related to the

Handling of ballot boxes

Preparing of ballots for counting

Counting operations and

Reporting data

Accept S- Doc 

Review

6.4 Software Security 

6.4.1 Software and Firmware Installation

The system shall meet the following 

requirements for installation of software, 

including hardware with embedded firmware.

a. If software is resident in the system as firmware, 

the vendor shall require and state in the system 

documentation that every device is to be 

retested to validate each ROM prior to the start 

of elections operations.

Accept S- Doc 

Review

b. To prevent alteration of executable code, no 

software shall be permanently installed or 

resident in the voting system unless the system 

documentation states that the jurisdiction must 

provide a secure physical and procedural 

environment for the storage, handling, 

preparation, and transportation of the system 

hardware.

Accept S

c. The voting system bootstrap, monitor, and 

device-controller software may be resident 

Accept S
VSTL Certification # pending

Page 85 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

permanently as firmware, provided that this 

firmware has been shown to be inaccessible to 

activation or control by any means other than by 

the authorized initiation and execution of the 

vote counting program, and its associated 

exception handlers.

d. The election-specific programming may be 

installed and resident as firmware, provided that 

such firmware is installed on a component (such 

as a computer chip) other than the component 

on which the operating system resides.

Accept S

e. After initiation of election day testing, no 

source code or compilers or assemblers 

shall be resident or accessible. 

Accept S #77 - Closed

6.4.2 Protection Against Malicious Software

Voting systems shall deploy protection against 

the many forms of threats to which they may be 

exposed such as file and macro viruses, worms, 

Trojan horses, and logic bombs

Vendors shall develop and document the 

procedures to be followed to ensure that such 

protection is maintained in a current status.

Accept S

6.5 Telecommunications and Data Transmission 

6.5.1 Access Controls

Voting systems that use telecommunications to 

communicate between system components and 

locations are subject to the same security 

requirements governing access to any other 

system hardware, software, and data function.

Accept S, T #135 - Closed

Telecommunications 

is disabled in Unity 

3.2.0.0

6.5.2 Data Integrity

Voting systems that use electrical or optical 

transmission of data shall ensure the receipt of 

valid vote records is verified at the receiving 

station. This should include standard 

transmission error detection and correction 

methods such as checksums or message digest 

hashes. Verification of correct transmission shall 

occur at the voting system application level and 

ensure that the correct data is recorded on all 

relevant components consolidated within the 

polling place prior to the voter completing 

casting of his or her ballot.

Accept S, T No transmission 

within the polls prior 

to voter casting their 

ballot

6.5.3 Data Interception Prevention

Voting systems that use telecommunications to 

communicate between system

components and locations before the polling 

place is officially closed shall:

a. Implement an encryption standard currently 

documented and validated for use by an agency 

of the U.S. Federal Government and

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

b. Provide a means to detect the presence of an 

intrusive process, such as an Intrusion Detection 

System.

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

6.5.4 Protection Against External Threats

Voting systems that use public 

telecommunications networks shall implement 

protections against external threats to which 

commercial products used in the system may be 

susceptible.

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

6.5.4.1 Identification of COTS Products

Voting systems that use public 

telecommunications networks shall provide 

Accept S, T Telecommunications 

is disabled in Unity 
VSTL Certification # pending

Page 86 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

a.

b.

c.

d.

system documentation that clearly identifies all 

COTS hardware and software products and 

communications services used in the 

development and/or operation of the voting 

system, including 

operating systems, 

communications routers,

modem drivers and 

dial-up networking software.

3.2.0.0

Such documentation shall identify the name, 

vendor, and version used for each

such component.

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

6.5.4.2 Use of Protective Software

Voting systems that use public 

telecommunications networks shall use 

protective software at the receiving-end of all 

communications paths to:

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

a. Detect the presence of a threat in a transmission Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

b. Remove the threat from infected files/data Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

c. Prevent against storage of the threat anywhere 

on the receiving device

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

d. Provide the capability to confirm that no threats 

are stored in system memory and in connected 

storage media

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

e. Provide data to the system audit log indicating 

the detection of a threat and the processing 

performed

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

Vendors shall use multiple forms of protective 

software as needed to provide capabilities for 

the full range of products used by the voting 

system.

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

6.5.4.3 Monitoring and Responding to External Threats

Voting system that use public 

telecommunications networks may become 

vulnerable, by virtue of their system 

components, to external threats to the accuracy 

and integrity of vote recording, vote counting, 

and vote consolidation and reporting processes. 

Therefore, vendors of such systems shall 

document how they plan to monitor and respond 

to known threats to which their voting systems 

are vulnerable. This documentation shall provide 

a detailed description, including scheduling 

information, of the procedures the vendor will 

use to:

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

a. Monitor threats, such as through the review of 

assessments, advisories, and alerts for COTS 

components issued by the Computer 

Emergency Response Team (CERT), for which 

a current listing can be found at 

http://www.cert.org, the National Infrastructure 

Protection Center (NIPC), and the Federal 

Computer Incident Response Capability 

(FedCIRC), for which additional information can 

be found at www.uscert.gov

Accept S, T Telecommunications 

is disabled in Unity

3.2.0.0

b. Evaluate the threats and, if any, proposed 

responses

Accept S, T Telecommunications 

is disabled in Unity 
VSTL Certification # pending

Page 87 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

3.2.0.0

c. Develop responsive updates to the system 

and/or corrective procedures

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

d. Submit the proposed response to the test labs 

and appropriate states for approval, identifying 

the exact changes and whether or not they are 

temporary or permanent

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

e. After implementation of the proposed response 

is approved by the state, assist clients, either 

directly or through detailed written procedures, 

how to update their systems and/or to 

implement the corrective procedures within the 

timeframe established by the state

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

f. Address threats emerging too late to correct the 

system by:

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

1 Providing prompt, emergency notification to the 

accredited test labs and the affected states and 

user jurisdictions

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

2 Assisting client jurisdictions directly or advising 

them through detailed written procedures to 

disable the public telecommunications mode of 

the system

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

3 Modifying the system after the election to 

address the threat, submitting the modified 

system to an accredited test lab and the EAC or 

state certification authority for approval, and 

assisting client jurisdictions directly or advising 

them through detailed written procedures, to 

update their systems and/or to implement the 

corrective procedures after approval

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

6.5.5 Shared Operating Environment

Ballot recording and vote counting can be 

performed in either a dedicated or nondedicated environment. If ballot recording and 

vote counting operations are performed in an 

environment that is shared with other data 

processing functions, both hardware and 

software features shall be present to protect the 

integrity of vote counting and of vote data.

Systems that use a shared operating 

environment shall:

a. Use security procedures and logging records to 

control access to system functions

Accept S Network disabled in 

Unity 3.2.0.0

b. Partition or compartmentalize voting system 

functions from other concurrent functions at 

least logically, and preferably physically as well

Accept S Network disabled in 

Unity 3.2.0.0

c. Control system access by means of passwords, 

and restrict account access to necessary 

functions only

Accept S Network disabled in 

Unity 3.2.0.0

d. Have capabilities in place to control the flow of 

information, precluding data leakage through 

shared system resources

Accept S Network disabled in 

Unity 3.2.0.0

6.5.6 Access to Incomplete Election Returns and 

Interactive Queries 

If the voting system provides access to 

incomplete election returns and interactive 

inquiries before the completion of the official 

count, the system shall:

a. Be designed to provide external access to 

incomplete election returns (for equipment that 

Accept S No access to 

incomplete returns
VSTL Certification # pending

Page 88 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

operates in a central counting environment), 

only if that access for these purposes is 

authorized by the statutes and regulations of the 

using agency. This requirement applies as well 

to polling place equipment that contains a 

removable memory module or that may be 

removed in its entirety to a central place for the 

consolidation of polling place returns

b. Design voting system software and its security 

environment such that data accessible to 

interactive queries resides in an external file or 

database created and maintained by the 

elections software under the restrictions 

applying to any other output report:

Accept S No external access

1 The output file or database has no provision for 

write-access back to the system.

Accept S No write back 

provision

2 Persons whose only authorized access is to the 

file or database are denied write-access, both to 

the file or database, and to the system.

Accept S No external access

6.6 Security for Transmission of Official Data 

Over Public Communications Networks

6.6.1 General Security Requirements for Systems 

Transmitting Data Over Public Networks

All systems that transmit data over public 

telecommunications networks shall:

a. Preserve the secrecy of voter ballot selections 

and prevent anyone from violating ballot privacy

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

b. Employ digital signatures for all communications 

between the vote server and other devices that 

communicate with the server over the network

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

c. Require that at least two authorized election 

officials activate any critical operation regarding 

the processing of ballots transmitted over a 

public communications network, i.e. the 

passwords or cryptographic keys of at least two 

employees are required to perform processing 

of vote

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

6.6.2 Voting Process Security for Casting 

Individual Ballots over a Public 

Telecommunications Network

Systems designed for transmission of 

telecommunications over public networks shall 

meet security standards that address the 

security risks attendant with the casting of 

ballots from polling places controlled by election 

officials using voting devices configured and 

installed by election officials and/or their vendor 

or contractor, and using in-person authentication 

of individual voters.

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

6.6.2.1 Documentation of Mandatory Security Activities

Vendors of voting systems that cast individual 

ballots over a public telecommunications 

network shall provide detailed descriptions of:

a. All activities mandatory to ensuring effective 

voting system security to be performed in setting 

up the system for operation, including testing of 

security before an election

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

b. All activities that should be prohibited during 

voting equipment setup and during the timeframe for voting operations, including both the 

hours when polls are open and when polls are 

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0
VSTL Certification # pending

Page 89 of 247 (V)2009-30Jun-001(A)

VSS Certification Test Requirements: VSS 2002 Test 

Result

Test 

Case

Function

Exercised

Comments

closed

6.6.2.2 Capabilities to Operate During Interruption of 

Telecommunications Capabilities

These systems shall provide the following 

capabilities to provide resistance to interruptions 

of telecommunications service that prevent 

voting devices at the polling place from 

communicating with external components via 

telecommunications:

a. Detect the occurrence of a telecommunications 

interruption at the polling place and switch to an 

alternative mode of operation that is not 

dependent on the connection between polling 

place voting devices and external system 

components

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

b. Provide an alternate mode of operation that 

includes the functionality of a conventional 

electronic voting system without losing any 

single vote

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

c. Create and preserve an audit trail of every vote 

cast during the period of interrupted 

communication and system operation in 

conventional electronic voting system mode

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

d. Upon reestablishment of communications, 

transmit and process votes accumulated while 

operating in conventional electronic voting 

system mode with all security safeguards in 

effect

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0

e. Ensure that all safeguards related to voter 

identification and authentication are not affected 

by the procedures employed by the system to 

counteract potential interruptions of 

telecommunications capabilities

Accept S, T Telecommunications 

is disabled in Unity 

3.2.0.0
VSTL Certification # pending

Page 90 of 247 (V)2009-30Jun-001(A)

7.2 Appendix B: PCA Source Code Review

PCA Source Code Review was performed by SysTest. ES&S petitioned the EAC for reuse. The terms of the reuse 

are identified in the EAC February 3, 2009 letter: EAC letter to iBeta QA Director on reuse of testing.

7.2.1 SysTest Source Code Review Results

The results of the SysTest source code review are addressed in the Summary Report of the testing performed by 

SysTest.

7.2.2 3% Source Code Review Results

The iBeta recommendation on reuse of SysTest PCA Document Review in ES&S Unity 3.2.0.0 testing is listed in 

the Voting System Certification Correspondence on the EAC web site. 

The applications and the source code reviewed by iBeta in the 3% sample and the potential logic issues provided to 

the EAC for consideration in their determination of reuse are found in Appendix H - Amended Test Plan section 

2.1.3 PCA Source Code Review

7.2.3 Unity 3.2.0.0 Source Code Review Results

Listed below is the version of code received from SysTest Labs and the final code version that was used in the 

Trusted Builds performed by iBeta. (See Appendix G: Trusted Build & Validation Tools Unity 3.2.0.0 voting system). 

The only discrepancy identified in any updated code was in ERM. Ten routines had the description comments but 

did not contain the other required infomation for routines over 10 lines of code. ES&S provided code with the 

required information. 

Product Source Code Language 3% Source 

CodeReview

Trusted Build Discrepancies

AIMS VB 1.003.057 1.003.157 None

AutoMark AIMSCrypt C / C++ 1.0.0.1 1.0.0.2a None

Information AIMSESS DLL C 1.0.1.0 1.0.1.0 None

Management AutoMarkEncoder C/C++ 1.0.105 1.0.105 None

System SQL Server SQL 1.3.054 1.3.054 None

VAT C++ 1.3.2816 1.3.2906 None

VAT VB.NET 1.3.2816 1.3.2906 None

AMCode C++ 1.0.0.1 None

Makebin C 1.1.0.0 None

Automark Service C++ 1.0.4 1.0.4 None

Automark Startup C++ 1.0.7 1.0.7 None

DiagnosticLogger C++ 1.0.105 1.0.105 None

NonVolatileLibrary C++ 1.0.116 1.0.116 None

AutoMark OperationLogger C++ 1.0.135 1.0.135 None

Voter PrinterEngineBoard C 1.65.1.3 65.1.3 None

Assist PrinterEngineBoard 1.70 C 1.70.0.1 1.70.0.1 None

Terminal Scanner Assembly 1.48 1.48 None

SecurityLibrary C/C++ 1.2.4. 1.2.4 None

SwitchinterfaceBoard C 1.43.2 1.8.37 None

UltrasonicSheetDetector C 8.0.1 8.01 None

Getmarks C++ 1.4.9 1.4.9 None

AutomarkDataHelperLibrary C++ 1.0.119 1.0.121 None

ScannerPrinterLibrary C++ 1.8.37 1.8.37 None

Automark VB Net 1.3.2816 1.3.2906 None

Automark32 File 1.3.2816 1.3.2906 None

AutomarkData C# 1.3.2816 1.3.2906 None

Audit Manager VB 7.5.0.0g 7.5.2.0a None

EDM C++ 7.8.0.0j 7.8.1.0a None

ESSXML.DLL C++ 2.1.0.0b 2.1.1.0a None

Unity MFC Shared Source C++ 1.1.0.0a 1.1.1.0a None

ESSIM C++ 7.7.0.0f 7.7.1.0a None

HPM COBOL 5.7.0.0f 5.7.1.0b None

HPMDLL C++ 1.0.0.0a 1.1.0.0a None

ERM COBOL 7.5.2.0c 7.5.4.0b 10 Header 

Comments 

v.1: 4.2.3.b
VSTL Certification # pending

Page 91 of 247 (V)2009-30Jun-001(A)

Product Source Code Language 3% Source 

CodeReview

Trusted Build Discrepancies

4.2.7 (a, a.1-a.6)

ERMDLL C++ 1.0.0.0a 1.1.0.0a None

MAKEIBIN.EXE C++ 9.2.0.0t 9.2.1.0a None

UNDRVOTE.EXE C++ 9.2.1.0b 9.2.2.0a None

 VIOWIN.EXE C/C++ 9.2.0.0b 9.2.1.0a None

 EVENTS.EXE C/C++ 9.2.0.0h 9.2.1.0a None

VIODIALOG.EXE C/C++ 9.2.1.0c 9.2.2.0a None

 IMAGES.EXE C/C++ 9.2.0.0f 9.2.1.0a None

CF_Utility.EXE VB 9.2.0.0i 9.2.1.0a None

GetAuditData.EXE VB 9.2.0.0b 9.2.1.0a None

ESSPEB.DLL C++ 1.0.1.0.c 1.0.2.0a None

CB_PEB.DLL C++ 1.0.1.0b 1.0.2.0a None

Shared CRCDLL.DLL C++ 1.4.1.0b 1.4.2.0a None

Utilities ESSM100.DLL C/C++ 1.7.1.0c 1.7.2.0b None

ESSPCMIO.DLL C++ 1.1.0.0a 1.1.1.0a None

CB_M100.DLL C++ 1.4.0.0a 1.4.1.0a None

ESSEAGL.DLL C++ 1.3.1.0e 1.3.2.0a None

CB_EAGL.DLL C++ 1.3.1.0c 1.3.2.0a None

CB_RAND.DLL C++ 1.1.0.0a 1.1.1.0a None

MYDLL.DLL C 1.1.0.0a 1.1.0.1a None

MPRBOOT.HEX Assembler 2.6.1.0b 2.6.2.0a None

ESSCRYPT.DLL C/C++ 1.9.0.0a 1.9.3.0a None

ESSDECPT.EXE C++ 1.9.0.0a 1.9.3.0b None

ESSCRYPT1.DLL C++ 1.1.0.0b 1.1.1.0a None

ElectionPackager C++ 1.0.0.0e 1.1.0.0a None

ESSZIP C++ 2.0.0.0f 2.1.0.0a None

PCCARD30.EXE C++ 3.5.0.0h 3.5.1.0a None

PBMtoBMP C++ 1.1.0.0c 1.1.0.1a None

WIN650 C 2.2.1.0.4 2.2.2.0.1 None

INIT650.EXE C/C++ 2.2.1.0.4 2.2.1.0.4 None

SERVE650.EXE (Newserve650) C++ 2.2.1.0.4 2.2.1.0.4 None

M650 CB_650.DLL C 1.2.0.0a 1.2.1.0a None

REGUTIL.DLL C++ 1.1.0.0d 1.1.0.1a None

SHELLSETUP.EXE C++ 1.1.0.0a 1.1.0.1a None

 SHELL.EXE C++ 1.1.0.0b 1.1.0.1a None

EXITWIN.EXE VB 1.1.0.0a 1.1.1.0a None

DS200 C/C++ 1.3.7.0g 1.3.10.0a None

DS200 Power_Management_MSP430 C 1.2.0.0a 1.2.01a None

Scanner_C8051 C 2.11.0.0a 2.11.0.1a None
VSTL Certification # pending

Page 92 of 247 (V)2009-30Jun-001(A)

7.3 Appendix C: PCA TDP Documentation Review

PCA TDP Document review was performed by SysTest. ES&S petitioned the EAC for reuse. The terms of the 

reuse are identified in the EAC February 3, 2009 letter: EAC letter to iBeta QA Director on reuse of testing.

7.3.1 PCA TDP Document Review

The results of the SysTest PCA TDP Document review are addressed in the Summary Report of the testing 

performed by SysTest.

The iBeta recommendation on reuse of SysTest PCA Document Review in ES&S Unity 3.2.0.0 testing is listed in 

the Voting System Certification Correspondence on the EAC web site. Any document issues that were applicable 

to the Unity 3.2.0.0 voting system were transferred to the iBeta certification test effort. These are identified in noted 

in the Unity 3.2.0.0 Discrepancy Report (see Appendix E- Discrepancy Report) and include the original SysTest 

discrepancy reference. 

7.3.2 Technical Data Package Configuration & Quality Assurance Practices

During the certification testing iBeta tracked all materials provided by ES&S. Any instances where the delivered 

materials did not conform to ES&S' identified Configuration and Quality Assurance practices were noted in the 

Unity 3.2.0.0 Discrepancy Report (see Appendix E- Discrepancy Report) as Informational issues. 
VSTL Certification # pending

Page 93 of 247 (V)2009-30Jun-001(A)

7.4 Appendix D: FCA Testing

The system configuration identified in section 3 identifies the final configuration of the Unity 3.2.0.0 voting system. 

The initial baseline configuration is identified in the Appendix H Amended Test Plan. Testing was conducted using 

the Trusted Builds provided by SysTest. ES&S submitted any code changes for issues encountered during testing. 

At that time iBeta completed a Trusted Build and conducted final regression testing. All changes to the system 

configuration were identified and recorded in the PCA Configuration document. A cross reference to the applicable 

configuration in the PCA Configuration document was inserted in the test record each time a test case was 

executed.

7.4.1 FCA Functional and System Level Accessibility, Maintainability, Accuracy 

& Reliability Testing 

Testing was performed by SysTest and petitioned for reuse by ES&S. The terms for this reuse were identified by 

the EAC in their February 12, 2009 letter, EAC approval to reuse portions of SysTest's testing of ES&S Unity 

3.2.0.0.

7.4.1.1 Reuse Functional System Level, Accessibility, Maintainability, Accuracy & 

Reliability Test Results

The test results and test configuration of the testing by SysTest are contained in the Summary Report of testing 

performed by SysTest. See Appendix H Amended Test Plan Tables 17, 25, & 26for the VSS 200 requirements 

associated with this testing. 

7.4.1.2 Regression System Level Test Results

Testing was conducted on the system configuration cross referenced in the test method. Specific software and

firmware builds for each test execution are recorded in the PCA Configuration as identified in the individual test 

case document.

Date Test Result Issues Opened Issues Closed Notes

4/15/09 Reject #129

5/11/09 Accept #129 1 of the 4 VATs encountered a maintenance required 

alert during test execution. . The test was continued 

without this unit. After completion of the test ES&S 

personnel performed on-site maintenance under 

observation of iBeta staff. This unit is included in the 

post Trusted Build test execution. 

6/11/09 to 

6/15/09

Accept #160 Informational #143 Test with the Trusted Build for all Unity 3.2.0.0 hardware 

and software

Only functional issues are identified. While documentation discrepancies may be encountered in testing they do not 

result in the rejection of a functional test. All issues are documented in Appendix E- Discrepancy Report.

Method Detail Regression Test Method

Test Case Name Regression System Level Test Case 

Scope - identifies the type of test A regression system level test incorporating validations of a substantial portion of the VSS 

2002 required and vendor identified functionality for the Unity 3.2.0.0 voting system. 

Pre-vote: Create a Pick-a-Party Primary election; prepare election media and paper ballots in 

EDM, ESSIM and HPM; import into AIMS . 

Vote: Vote Election Day hand & machine marked paper ballots (VAT:A100 & A200); precinct 

scanning (DS200)

Post Vote: Write election results (DS200); scan absentee hand marked and VAT marked 

ballots (M650 central scanner); consolidate absentee & Election Day votes into ERM for 

tallying and reporting.

Testing includes validation of measurable performance including accuracy, processing rate, 

and ballot format handling capability, incorporating: testing 

- ENH14322 (zero totals in ERM- RFI-2008-07) 

- Discrepancy #35 (SysTest 475 ERM Number-Key District report BUG13966,)

- Discrepancy 30 (SysTest 429 Election description, Vote for phrase when only 1, Vote for 

phrase) Out of scope of Unity 3.2.0.0 because it only contains 2 groups. Issue requires a 

minimum of 3 groups to test. It is being moved to Unity 4.0.0.0 discrepancy #2.

- Discrepancy #48 (SysTest 556 AM archive functionality)

Functional aspects include error recovery, security, and usability of the hardware, software 

procedures in the pre-vote, voting, and post-voting operations with voter accessibility and 

multilingual ballots included. 

Test Objective Validation of the ability to: 

- Accurately and securely create paper English and Spanish visual and audio ballots for a 
VSTL Certification # pending

Page 94 of 247 (V)2009-30Jun-001(A)

Method Detail Regression Test Method

pick-a-party primary election;

- Create and install election specific media for the VAT and DS200 and M650;

- Independently and securely vote audio and visual ballots with mobility and non-mobility 

restrictions; 

- Count and report the results; and

Validate identified enhancements and discrepancies.

Test Variables: 

Voting Variations

(as supported by the voting system)

In Scope for Unity 3.2.0.0:

Open Pick-a-Party Primary comprising:

14 inch ballot (3 ovals per inch, 36 oval positions per column, 216 total positions)

- An 14 inch combined paper ballot containing Dem, Rep & Non-Partisan selections, with 

ovals on the right side

- 1 Polling Place

- 2 Ballot Styles comprising: 

- 3 Precincts (1000, 2000, 3000) splits (3000-01, 3000-02)

- 2 Partisan, 1 Non-Partisan, 1 Referendum Contests & a Party Selection 

Election Day voting (VAT & DS200)

Absentee Voting (M650)

Write-in votes

Vote for N of M

Overvotes

Undervotes

Blank Ballots

Assistive Devices (AT paddles, tactilely discernible keypad, Audio\Visual ballots)

Multi-lingual Audio & Visual Ballots (English & Spanish) 

- Two audio preparation methods: conversion of election text file from Unity to synthesized 

speech in AIMS (Eloquence COTS SW) & WAV audio files recorded in AIMS 

- AIMS Overvote and Undervote alerts selected for VAT. (Overvotes prevented) 

- DS200 Ballot Control Options (HPM): Query: Overvotes, Crossover, and Blank ballots; 

Reject: Unreadable marks; Accept: undervote.

- Diverter Installed to divert Blank and Write-in ballots in ballot box

A description of the voting system type and 

the operational environment

The Unity 3.2.0.0 voting system includes: 

EMS Ballot Preparation SW: Audit Manager (AM), Election Data Manager (EDM), ES&S 

Image Manager (ESSIM), Hardware Programming Manager (HPM), AutoMARK Information 

Management System (AIMS)

Ballot Marking Device: Voter Assist Terminal (VAT), Models A100 & A200

Precinct Count scanner/tabulator: intElect DS200 (DS200)

Central Count scanner/tabulator: Model 650 (M650)

Central Count Tally system: Election Reporting Manager (ERM)

VSS 2002 vol. 1 2.2.1 thru 2.2.9, 2.3.1.1 thru 2.5.3.2 , (DRE requirements applicable to VAT excluding vote 

storage) 3.2.4.2.5, 3.2.4.2.6, 3.2.5.1.3 a thru d.4, 3.2.6.1.1, 3.2.8 thru 3.2.8.2

HAVA a thru c2

RFI: 2007-04, 2007-06, 2008-04, 2008-07, 2008-12

VSS 2002 vol. 2 6.2, 6.2.1, 6.2.2, 6.3, 6.4, 6.4.1 , 6.5, 6.6, 6.7

RFI: 2007-06, 2008-07, 2008-12

Hardware, Software voting system 

configuration and test location

EMS Software: 

AM v. 7.5.2.0

EDM v. 7.8.1.0

ESSIM v. 7.7.1.0

AIMS v. 1.3.157

HPM v. 5.7.1.0

ERM v. 7.5.3.0

Hardware/Firmware specific to this test case:

VAT Model s including A100 & A200's 

Precinct count: DS200: HW: 1.2.0; FW: 1.3.7.0, SN: ES0107360007

Central count: M650: Green (Right) HW Rev. 1.1, FW: 2.2.1.0 SN: 2406 8013

Test Location: iBeta, 3131 S. Vaughn Way, Aurora, CO 80014

Pre-requisites and preparation for execution 

of the test case. 

Prior to execution of testing, the following prerequisites must be completed:

- Record the testers & date

- Perform and install witness/trusted build of software/firmware components utilizing ES&S 

documentation

- System has been installed and set up as identified in the user manuals

- Gather any necessary materials or manuals (A microphone, PC soundcard and speakers 

are available/installed to record audio, white and blue blank ballot stock paper)

- Ensure customization of the test case template is complete

Getting Started Checks Check the voting system to: 
VSTL Certification # pending

Page 95 of 247 (V)2009-30Jun-001(A)

Method Detail Regression Test Method

- Verify the test environment and system configuration is documented in the PCA 

Configuration matches the configuration of the system used in the 48 hr. temp & power 

variation test and vendor described configuration.

- Validate installation of the witnessed build

- Testers understand that no change shall occur to the test environment without 

documentation in the test record and the authorization of the project manager.

-During testing an operational readiness test will be performed. 

Documentation of Test Data & Test Results Test Data:

- Record all programmed & observed election, ballot & vote data fields and field contents on 

the corresponding tabs to provide a method to repeat the test

- Preserve all tabs for each instance the test is run.

Test Results: 

- Enter Accept/Reject on the Test Steps

- In Comments enter any deviations, discrepancies, or notable observations

- Log discrepancies on the Discrepancy Report and insert the number in the Comments

Pre-vote:

Ballot Preparation procedures verifications

Ballot Prep: Verify

- Spanish/English, visual/audio ballots (contests, candidates , propositions and associated 

offices/labels) can be accurately/securely defined with multiple ballot styles, precincts and 

splits.

- Ballots contain partisan races segregated by party and non-partisan races (Dem, Rep, NonPartisan)

- Ballots contain identifying marks (ballot style, precincts/splits)

- Volume test elections and ballot styles are retained and can be accessed

- Ballot & VAT: ovals properly align with candidate names/issues so voters can clearly mark 

selections; spacing and font size is consistent so there is no preferential voting position

- VAT: maximum choices for a single contest are displayed on one page

- The election can be accurately/securely imported from Unity 3.2.0.0 into AIMS. 

(Prerequisite: define and print ballot in Unity 3.2.0.0, before importing into AIMS.)

- The AIMS database can be modified, as required, to support the election definition required 

for VAT operation; and using AIMS Preview function confirm data was imported correctly and 

ballots are set up correctly.

- Election media can be accurately/securely programmed in HPM and AIMS for installation in 

all voting & tabulating devices. (VAT, DS200, M650)

- AM, EDM, ESSIM, HPM, ERM, VAT, M650 & DS200 Application & hardware readiness 

tests performed according to VSS requirements. 

Validate Discrepancy 30 (Election description, Vote for phrase when only 1, Vote for phrase) 

Installation of Election

VAT: Setup & install election; perform maintenance checks: 1. ink cartridge. 2. battery charge 

3. Install Flash Memory Card. 4. Test VAT operations (Jurisdiction Guide Ch. 5) 5. Set Admin 

password 6. Calibrate 7. Set 'Maint' password (Jurisdiction Guide Ch. 6) to confirm there are 

no hardware/software failures 

DS200: Setup & install election; perform readiness tests

M650: Setup & install election; set Date & Time; and perform readiness tests

Pre-vote:

Ballot Preparation Security

Ballot Prep: 

-Security access controls limit or detect access to critical systems and the loss of system 

integrity, availability, confidentiality & accountability, including

AM: A user ID/password control access to EDM & ESSIM; confirm access is permitted and 

denied without proper credentials

HPM: An administrator password; access the DS200 Admin menu on the DS200 Scanner 

Options screen; and a password to reopen polls 

ERM: An administrator password; prevent access to "Suspension Menu"; and confirm access 

is denied.

DS200: A password is required to access Admin menu; a separate password is required to 

reopen polls

M650: Back door is locked

AIMS: Password required to start AIMS

VAT: Admin password controls the functions on the System Maint menu (password set on 

each VAT)

-Functions are only executable in the intended manner, order & under intended conditions

-Prevents execution of functions if preconditions weren't met

-Implemented restrictions on controlled functions

- Documentation of mandatory administrative procedures.

COTS 

-Authentication is configured on the local terminal & external connection devices,

-Operating systems are enabled for all session & connection openings, & closings, all process 

executions & terminations & for the alteration or detection of any memory or file object

-Configure the system to only execute intended & needed processes during the execution 

election software. Processes are halted until termination of critical system processes (such as 

audit).
VSTL Certification # pending

Page 96 of 247 (V)2009-30Jun-001(A)

Method Detail Regression Test Method

Readiness Testing and Poll Verification Readiness Testing: Verification that: 

VAT: Proper election has been installed: all buttons, printers and screen function correctly; 

matching version is displayed ; and a ballot can be marked in test mode . 

- Review audit logs to confirm readiness for VAT

DS200: Readiness testing automatically incorporated into Opening the Polls; Election name, 

equipment identification, polling place & ballot format and matching version is displayed or 

printed on initial state report and/or zero count report; confirmation that there are no 

hardware/software failures ; and device is ready to be activated to accept votes. Perform 

readiness testing according to VSS requirements

- Obtain status, data reports, audit logs and other artifacts to confirm readiness for DS200

- Attempt to open polls with test totals. Verify a visual screen warning is provided if memory 

locations (including data on media) contains votes, and the reports/audit log contain a timestamp record of the status of the votes/results memory and disk storage locations. If a unit or 

system contains a non-zero counter, a warning message is provided, along with corrective 

actions to resolve the votes. The unit is disabled until type of resolution is selected. 

- Verify test data has been cleared

Pre- vote:

Opening the Polls Verification

Precinct Count: 

Internal testing:

- DS200 select 'Open Polls". Zero report will automatically print, an internal test will be 

performed and results will display. If test is unsuccessful, DS200 will automatically shut down; 

If successful will display "Please Insert Your Ballot" message

- Insert election FMC. VAT will boot up when key switch is turned to 'On' flashing displays of 

the boot procedure will appear on the screen. If the self-test fails the VAT will shutdown. If 

successful the VAT will give the "Please Insert Your Ballot" message. (Insert a blank CF card 

to ensure VAT will NOT boot up)

Paper based: Verify VAT & DS200 are ready for use:

- VAT & DS200 display "Please Insert Your Ballot" message. 

- Any failures provide a message for resolution 

- VAT holds the ballot securely

- DS200 does not contain a frame or fixture for ballot marking

- DS200 is attached to a custom DS200 ballot box; with locks and separate compartments; 

slots prevent unauthorized ballot insertion. Write-ins will be marked with a red circle to indicate 

review is necessary 

- VAT security seals are checked: compact flash compartment, top cover & ink compartment

Voting: 

Ballot Activation and Casting Verifications

VAT & DS200

Protects secrecy of ballot/vote

- Voter can make selections based on ballot programming & indicate selection, cancellation, & 

non-selection (undervotes)

- Gives feedback & an opportunity to correct or accept, before the ballot is counted

VAT BMD

- Control of ballot (single ballot cast per vote session) and content of ballot is restricted to the 

eligible voter

- Correct ballot is presented (language, audio/visual, precinct/split)

- Party affiliation content is controlled/activated via the "Party Preference" 

- Touching an area outside the identified selection box does not mark the ballot or display 

external information

- Provides all displays, instructions, messages, alerts and status in multilingual audio & visual 

displays

- Voters are able to edit and review write-ins. # of write-ins match Vote For.

- Audio voting provides repeat functionality & volume control 

- Voter is allowed to mark the ballot, in any combination, or return it without marking (blank)

- Overvote and Undervote provides alerts, with overvotes prevented

- Summary screen is provided to signify end of candidate/measures and provides instructions to 

review/change selections prior to ballot marking

- Verify alert of selection's complete, ballot is being marked, and to take completed ballot to 

tabulator 

DS200

- Alert successful/unsuccessful storage of cast ballot; provide review & instruction to resolve 

unsuccessful casting (Query: Overvotes, Crossover, and Blank ballots; Reject: Unreadable 

marks; Accept: undervote.

- Diverter Installed to divert Blank and Write-in ballots in ballot box

- Increments the ballot counter for successfully cast ballots

- Print Precinct and Status reports to compare to vote data to verify actual votes cast is correct 

& undervotes/overvotes are counted separately

- Access to voted ballot is prevented until after polls close (locked ballot box)

Voting: 

Voting System Integrity, System Audit, 

Errors & Status Indicators

The system audit provides a time stamped, always available, report of normal/abnormal 

events that can't be turned off when the system is in operating mode. 

- Maintain accurate and complete audit records; verify at various points (After poll open; vote 
VSTL Certification # pending

Page 97 of 247 (V)2009-30Jun-001(A)

Method Detail Regression Test Method

query, reject & accept: any abnormal event encountered in testing; poll close)

- Self-tests and diagnostic messages for the hardware will be verified at polll open/close points 

in the test case

Status messages are part of the real time audit record. 

- Critical status messages requiring operator intervention shall use clear indicators or text

Error messages are:

- Generated, stored & reported as they occur

- Errors requiring intervention by the voter or poll worker clearly display issues & action 

instructions in easily understood text language or with indicators

- The text for any numeric codes is contained in the error or affixed to the inside of the voting 

system

- Incorrect responses will not lead to irreversible errors. 

- Nested conditions are corrected in the sequence to restore the system to the state before the 

error occurred

Post-vote:

Closing the Polls

VAT:

- Turn VAT to 'Off' position & remove FMC to prevent further marking of ballots; verify a voting 

session cannot be activated.

- Review the audit logs (only available report ) to verify entries are in the proper sequence for 

operational tests, switching from test to vote modes, ballot printing, audit report access during 

voting , including complete & accurate error and status messages 

DS200:

- Attempt to print reports while polls are open; verify this is prohibited. 

- Press 'Close Poll' button, a results report will print preventing further casting of ballots 

(attempt to scan a ballot without reopening the polls)

- Visibly displays the status "Polls Closed" 

- Internally tests and verifies that the closing procedures have been followed and the device 

status is normal by preventing report printing or processing vote totals unless polls were 

properly closed. 

- Confirm polls cannot be reopened without password

- Review the audit log to verify test records exists that verify entries for the proper sequence 

for operational tests, poll open; vote query, reject & accept: any abnormal event encountered 

in testing; poll close, including complete & accurate error and status messages

- Print Status report, Race Results report, Certification report, Precinct Report Summary, Poll 

Report Summary and Audit Log report once polls are closed. Ensure undervote & overvote is 

counted. 

- Validate data from USB is extractable by transmitting results into ERM

Reopen the polls testing: 

- Reopen of polls, enter an incorrect and then a correct password

- Alert to resume voting or clear votes: select 'resume voting', do not clear votes

- Status message "Please insert your ballot" is displayed

-Cast a vote and close the polls. 

- Check audit for proper sequence for operational tests, poll open, vote accept, poll close, 

reopen, password entry

- Verify correct vote totals. 

Post-vote:

Central Count

Readiness Test:

- Obtain status, data reports, audit logs and other artifacts to confirm readiness

- Verify test data has been cleared

M650: Readiness: Proper election is installed; all buttons, printers and screen function 

correctly; verify election name, equipment identification, polling place, ballot format and 

matching versions is printed on Machine Readiness and/or Zero count reports; confirmation 

that there are no hardware/software failures; and device is ready to be activated to accept 

votes. Perform readiness testing according to VSS requirements 

- Attempt to start the M650 with test totals. Verify a visual screen warning is provided if 

memory locations (including data on media) contains votes, and the reports/audit log contain a 

time-stamp record of the status of the votes/results memory and disk storage locations. If a 

unit or system contains a non-zero counter, a warning message is provided, along with 

corrective actions to resolve the votes. The unit is disabled until type of resolution is selected. 

ERM: Readiness: confirm proper election is installed 

- Attempt to read in vote totals with test totals present. Verify a visual screen warning is 

provided if memory locations contain votes, and the reports/audit log contain a time-stamped 

record of the status of the votes/results in the memory locations. If this is not provided, a 

corrective action message is provided along with a message indicating the attempt to read in 

vote totals, while there are totals present. 

Vote Consolidation for M650:

- Votes match predicted votes (absentee) 

- Geographic reports of votes; each contest by precinct & other jurisdictional levels. Reports 

include: 

Zero, Grand Totals (long format), Precincts Processed, Totals by Precinct (long format) 

Machine Readiness, Audit log. Ensure audit logs are accurate & complete and contain error 
VSTL Certification # pending

Page 98 of 247 (V)2009-30Jun-001(A)

Method Detail Regression Test Method

and status messages.

Vote Consolidation for ERM:

Consolidated reported votes match predicted votes from polling places, & optionally other 

sources (absentee) 

- Geographic reports of votes; each contest by precinct & other jurisdictional levels. Reports 

include: 

- Zero - Validate ERM Enhancement: RFI2008-07/ ENH14322 to ensure ERM is zeroed out 

before processing election results.

- EL30A - Prec Report–Group Detail individual precincts & contest results. 

- EL45- Election Summary - total number of votes for each candidate/question & % of total 

vote for y each candidate/question

- EL111 - Name Heading Canvass - statistics of total number of precincts counted, total 

number of votes cast for each candidate and % of total vote received by each candidate

- EL50D - DS200 Precincts Processed Listing - DS200 machine IDs imported from the USB 

flash drive into ERM

- Audit log. 

- Retrieve ballot images from the DS200

- Verify data from M650, DS200 is prevented from being altered or destroyed by report 

generation, or extraction from media

- Verify DS200 SN is displayed in ERM, once the USB flash drive is read into ERM 

Post-vote:

Security

The central count: (See Security Test for detail)

During execution confirm:

- Security access controls limit or detect access to critical systems& the loss of system 

integrity, availability, confidentiality and accountability

- Functions are only executable in the intended manner, order & under the intended conditions

- Prevented execution of functions if preconditions were not met

- Implemented restrictions on controlled functions

- Provided documentation of mandatory administrative procedures.

COTS systems 

-Authentication is configured on the local terminal and external connection devices,

-Operating systems are enabled for all session and connection openings, and closings, all 

process executions and terminations and for the alteration or detection of any memory or file 

object

- Configure the system to only execute the intended and necessary processes during the 

execution of the election software. Election software process are halted until the termination 

of any critical system process, such as system audit.

Post-vote:

System Audit

The system audit provides a central count time stamped always available, report of normal 

and abnormal events that cannot be turned off when the system is in operating mode. Status 

message are part of the real time audit record. 

Audit Messages to be validated: 

AM: Archive functionality

EDM: Precinct set up

ESSIM: 2 ballot styles created

HPM: media is created for M650 & DS200

VAT: date/time set

DS200 & M650: Election id

ERM: DS200 SN is recorded

AIMS: IUImport - Performed full Unity election import

DS200, M650 & ERM: Message of vote totals present, Corrective action messages to resolve 

residual vote totals

Status/Error messages to be validated:

AM: 1. Cannot delete „Admin‟ user!

EDM: 1. Minimum password length is 6 characters. 2. District Type Name can not be blank

ESSIM: 1. Please Enter a Style Sheet Name

HPM: 1. Admin password is required

VAT: 1. System Maintenance (requires password), 2. The Flash Card has been removed. 

Turn OFF the machine and insert a valid Flash Card.

AIMS: Missing Translations

DS200: 1. Blank Ballot Rejected, 2. More than one party has votes. Votes In Party Contests 

Will Be Ignored, 3. Ballot Jammed, 4. 119 – MULTIPLE BALLOTS DETECTED/Please Reinsert One Ballot After Beeps 

M650: 1. Back Door Open, 2. Ballot BACKWARDS or UPSIDE-DOWN!

ERM: 1. ####-Not a valid precinct, 2. Canvass Left Edge Heading exceeds the maximum 

length of 20 for 1UP format report.

DS200, M650 & ERM: Warning message of vote totals present, Corrective action messages 

to resolve residual vote totals

Validate AM archive functionality as identified in discrepancy #48. (Data from the day selected 

does not archive.)
VSTL Certification # pending

Page 99 of 247 (V)2009-30Jun-001(A)

Method Detail Regression Test Method

Expected Results are observed Review the test result against the expected result: 

• Accept: the expected result is observed

• Reject: the expected result of the test case is not observed

• Not Testable (NT): rejection of a previous test step prevents execution of this step, or tested 

in another TC.

• Not Applicable (NA): not applicable to test scope

Record observations and all input/outputs 

for each election;

All inputs, outputs, observations, deviations and any other information impacting the integrity 

of the test results will be recorded in the test case. 

- Any failure against the requirements of the EAC guidelines will mean the failure of the 

system and shall be reported as such. 

- Failures will be reported to the vendor as Defect Issues in the Discrepancy Report. 

- The vendor shall have the opportunity to cure all discrepancies prior to issuance of the 

Certification Report.

- If cures are submitted the applicable test will be rerun. Complete information about the rerun 

test will be preserved in the test case. The cure and results of the retest will be noted in the -

Discrepancy Report and submitted as an appendix of the Certification Report.

- Operations which do not fail the requirements but could be deemed defects or inconsistent 

with standard software practices or election practices will be logged as Informational Issues 

on the Discrepancy Report. It is the vendor's option to address these issues. Open items will 

be identified in the report. 

7.4.2 FCA Volume (Volume Stress, Performance and Error Recovery) Testing

During test case development, test setup or preparation of test data, iBeta encountered limits that were either not 

identified or only identified at the application level in the System Limitation Unity 3.2.0.0 document. This resulted in 

iBeta revising the test method identified in the approved test plan. Document discrepancies were written against 

the limits document. Volume Test Cases were appropriately modified to address testing to the applicable system 

limit. In some instances this resulted in halting a test because the original test parameters were no longer valid. In 

these instances we recorded the halted test execution and updated the test case. Due to the unusual 

configurations required to generate testing of the systems limits, the optimal test parameters were occasionally 

miscalculated by the tester. If during test setup or preparation of test data iBeta discovered a tester miscalculation 

in the election parameters which required reconfiguration of the test parameters, test execution was halted and the 

miscalculation was clarified in the test case. In these instances we recorded the halted test execution and updated 

the calculations. All halted test executions were recorded and preserved. Test execution number was incremented 

and the changes to the test parameters were recorded on the incremented test execution tab. As the original test 

was not valid it is documented in the Test Results as "Execution halted, parameters modified". See Volume 1 Test 

Results for an example of this process.

7.4.2.1 Volume 1 Test Results

Testing was conducted on the system configuration identified in the PCA Configuration and Test Case. Specific 

software and firmware builds for each test execution are recorded in the PCA Configuration as identified in the 

individual test case document.

Date Test Result Issues Opened Issues Closed Notes

3/13/09 Halted - modified 

test parameters

#85 -System Limits

document

In set-up we identified that 1639 is the limit on ballot 

styles for paper. Reduced test to 1639 ballot styles and 

precincts. Testing of 2900 precinct on the DS200 is 

covered in volume 9. 

3/18/09 Halted - modified 

test parameters

Miscalculation of optimal District Types, Names, 

Relations, Master Office, Office Relations and 

Candidates parameters.

4/7/09 Accept

Only functional issues are identified. While documentation discrepancies may be encountered in testing they do not 

result in the rejection of a functional test. All issues are documented in Appendix E- Discrepancy Report.

Method Detail Volume 1 Test Method

Test Case Name Volume 1 - Maximum Precincts and Ballot Styles for paper 

Scope - identifies the type of test The scope of this test 1639 precincts,1639 ballot styles:

Scenario 1) The maximum allowed number of precincts with the maximum number ballot styles 

allowed for paper based systems.

To verify that errors are generated in scenarios 2:

Scenario 2) Exceeding the maximum number of Precincts and the maximum number of ballot 
VSTL Certification # pending

Page 100 of 247 (V)2009-30Jun-001(A)

Method Detail Volume 1 Test Method

styles.

In set-up we identified that 1639 is the limit on ballot styles for paper. Reduced test to 1639 ballot 

styles and precincts. Testing of 2900 precinct on the DS200 is covered in volume 9

Test Objective The objective is to validate the ability to process, store and report data using the allowed 

maximum number of precincts and ballot styles within an election. To validate that the system 

generates errors during EMS ballot preparation (ballot preparation including: EDM, ESSIM & 

HPM) when exceeding the maximum numbers of precincts and ballot styles. Validating the 

processing, storing and reporting shall occur without system degradation. If there are system 

errors that cause the system to crash the system shall recover without any loss of data.

Test Variables: General election

Local offices

Vote for 1

4 Ovals per Inch ballot - (14 inch ballot, 48 ovals positions per Column, 6 columns per ballot, 288 

total oval positions)

4 candidates per contest 

Election Day (DS200 and VAT)

Voting (M650) one tabulator

Scenario 1) 1639 precincts with 1639 ballot styles (Maximum precincts/Maximum ballot styles)

- Contests 1 - 290 in Polling Places 1 -29 (10 precincts to a polling place, 3 contest to a precinct) 

total of 290 ballot styles 

- No contest/Precincts assigned to Polling Places 30 -290

- Contests 291 - 1639 in Polling Places 291- 1639 (1 precinct to a polling Place, 3 contest to a 

polling place) 1348 ballot styles

- Contest 1639 in Polling Place 1639 with Precincts 1639 (3 contest in the precinct, and all polling 

places) 1 ballot style

TOTALS

1639 Ballot Styles

1639 Precincts

1378 Polling Places

Scenario 2) 1640 Precincts w/ 1640 ballot styles(over the Maximum precincts/Maximum ballot 

styles)

Add a new contest 1640 to a new Precinct1640

TOTALS

1640 Ballot Styles

1640 Precincts

1640 contest

1640 Polling Places

A description of the voting system type and 

the operational environment

The Unity 3.2.0.0 EMS Ballot Preparation includes:

VSS 2002 vol. 1 The Unity 3.2.0.0 EMS Ballot Preparation includes:

Audit Manger (AM ), Election Data Manger (EDM), (ESSIM), hardware Program Manger (HPM), 

AutoMARK Information (AIMS)

2 @ Unity 3.2.0.0 marking device: Voter Terminal(VAT)

2 @ Unity 3.2.0.0 precinct count includes: DS200

Unity 3.2.0.0 central count tabulator: Model 650 (M650)

Unity 3.2.0.0 central count tally: Election Reporting Manager (ERM)

VSS 2002 vol. 2 6.2.3 Volume (maximum number of ballot styles)

A4.3.5 Volume (maximum and exceeding more than the maximum number of precincts)

A4.3.5 Volume/Stress (Processing, storing and reporting data when overloading the number of 

precincts and ballot styles)

A4.3.5 Performance/Recovery (Ballot format handling capability-graceful shut down and recovery 

without loss of data)

A4.3.5 Performance/Recovery (Processing rates-graceful shut down and recovery without loss of 

data)

Hardware, Software voting system 

configuration and test location

The Unity 3.2 Voting System consist of the following:

Audit Manger (AM ), Election Data Manger (EDM), (ESSIM), Hardware Program Manger (HPM), 

DS200, Model 650 (M650), Election Reporting Manager (ERM), AutoMARK Information (AIMS), 

Voter Terminal(VAT)

All testing was perform by iBeta LLC located at 3131 S. Vaughn Way, Aurora, CO 80014.

Pre-requisites and preparation for execution 

of the test case. 

Complete the prerequisites: 

Test Method Validation: Technical review conducted by C. Coggins; Approved 3/4/09 for 

validation of test method as defined in ISO/IEC 17025 clause 5.4.5. -

Condition of approval - iBeta validated the successful use of the Import Wizard to import large 

amounts of data into EDM. As configuration of the imported file can impact the success of the 

data importation, the import file structure must be validated as a prerequisite of all applicable test 

cases.

Import Wizard method tested and validated: 3/18/09
VSTL Certification # pending

Page 101 of 247 (V)2009-30Jun-001(A)

Method Detail Volume 1 Test Method

- Document in the test case the percentage that the system limit exceeds the customer maximum. 

(System Limit * 100) /Customer Maximum =% System Limit ) 

- Record the testers & date

- System has been set up as identified in the user manual

- Gather any necessary materials or manuals. 

- Ensure customization of the test case template is complete

- Order ballots

- Set Election Date: 11/03/2009

- 8 Excel spreadsheets saved as "Tab Delimited". Tab Delimited documents containing election 

creating information was imported into EDM using the Import Wizard option.

Spreadsheet 1 - Precinct 1639

Spreadsheet 2 - District Types 4

Spreadsheet 3 - Districts Names 47

Spreadsheet 4 - District Relations 4920

Spreadsheet 5 - Master Office 46

Spreadsheet 6 - Office Relations 46

Spreadsheet 7 - Candidates 184

Spreadsheet 8 - Polls1639 (actual number of Polls is 1378)

Spreadsheet 9 - Poll Relations1639

Getting Started Checks Check the voting system to: 

- Verify the test environment and system configuration is documented in the PCA Configuration 

and matches the system used in the 48 hr. temp & power variation test and vendor described 

configuration. 

- Validate installation of the witnessed build

- Testers understand that no change shall occur to the test environment without documentation in 

the test record and the authorization of the project manager.

-During testing an operational readiness test was performed.

Documentation of Test Data & Test 

Results

Test Data:

- Record all programmed & observed election, ballot & vote data fields and field contents on the 

corresponding tabs to provide a method to repeat the test

- Preserve all tabs for each instance the test is run.

Test Results: 

- Enter Accept/Reject on the Test Steps

- In Comments enter any deviations, discrepancies, or notable observations

- Log discrepancies on the Discrepancy Report and insert the discrepancy number in the 

Comments field of Test Step.

Volume: Paper-based voting systems

Processing

Ballot Prep:

Scenario 1)

- 4 candidates per contest

- 1639 Ballot Styles

- 1639 Precincts

- 1639 contest

- 1639 Polling Places

-An election database was accurately/securely defined & formatted using the Import Wizard.

- Set up election by Style

-Ballots (candidates & propositions) were accurately defined & generated.

- Check EDM reports for election set up

- Election media was installed 

- There were no system errors that caused the EMS ballot preparation applications to crash.

Scenario 2)

- 4 candidates per contest

- 1640 Ballot Styles

- 1640Precincts

- 1640 contest

- 1640 Polling Places

Test execution of Scenario 2 stop at this point with errors generated prior to the creation of 

election media in ballot preparation)

- Check audit logs for critical status messages. Test stops unless system does not error and 

creates media) 

- If EDM does not error during the "Ballot Sets Merge" then the EDM reports must be reviewed to 

verify 1640 precincts and 1640 ballot styles have been created and assigned to Early Voting 

Polling Places. Continue to ESSIM and HPM. The system should display a critical status 

message prior to exiting the HPM.

- 1640 Precincts in an election

-1640 ballot styles in an election

- There were no system errors that caused the EMS ballot preparation applications to crash.

Volume: System response to processing more than the expected number of precincts and maximum 

number of ballot styles.

Maximum capacity is successfully processed without errors. 

System's capacity to process, store, and report data.
VSTL Certification # pending

Page 102 of 247 (V)2009-30Jun-001(A)

Method Detail Volume 1 Test Method

- When importing the allowed precincts and/or ballot styles into the EDM using the Import Wizard 

no errors were generated

Stress System responses to overloading conditions. Exceeding the maximum allow number precincts 

and ballot styles by sequence.

Performance No system degradation (Ballot format handling capability and Processing rates):

-When importing large amount of data into the EDM using the Import Wizard.

-When installing an election with 1639 precincts and 1639 ballot styles onto each device (DS200, 

M650, and VAT)

-When installing an election over the maximum number of precincts and ballot styles onto each 

device 

- The system will not slow down as more and more data is added

Error Recovery Voting system gracefully shuts down (no crash) and recovers from errors caused by overloading 

the number of precincts and ballots styles. 

- Ballot format handling capabilities and processing capabilities-graceful shut down and recover 

without loss of data

- Critical Status Messages

Readiness Testing and Poll Verification Voting system was ready for the election: 

- The election was correctly installed (Election ID, polling place name, precincts)

- Test data (run 10 different precincts to validate the system was ready) was segregated from 

voting data, with no residual effect'

Test confirmation that there are:

- No hardware/software failures 

- The device was ready to be activated to accept votes (No Identification of any failures & 

corrective action)

Pre- vote: Opening the Polls Verification Precinct Count/ Paper based:

- Zero count report 

Voting: Ballot Activation and Casting 

Verifications

Scenario 1) 

Protects secrecy of ballot/vote

- Mark ballots using the VAT

- The DS200 Election Day

- Vote an 8% sample of the 1639 precincts 

- Vote 21 precincts each with a different ballot style 

- Each precinct will contain 3 contest with 4 candidates

Scenario 2) Errors should prevent the tests from reaching this point. If the test does get to this 

point: 

- Load election

- No system failures that cause the DS200 and VAT to crash

- There were no errors that cause the DS200 and VAT to crash

Voting: Voting System Integrity, System 

Audit, Errors & Status Indicators

The system audit provides a time stamped, always available, report of normal/abnormal events 

found within the 10% sampling tested. 

Error messages: 

- Are generated, stored & reported as they occur

- Errors requiring intervention by the voter or poll worker clearly display issues & action 

instructions in easily understood text language or with indicators

- Incorrect responses will not lead to irreversible errors. 

Post-vote: Closing the Polls Once the polls were closed the voting system

- Printed reports of ballots counted by tabulator

- Reported that votes match predicted votes from the tabulator with votes and undervotes.

Post-vote: Central Count Paper Based: When loading results mix the input of results such that media was read out of 

precinct order and where possible mix the reading of DS200 and M650 results. Record the order 

at test execution.

Scenario 1) 

The central count voting system includes: 

- Election identification

- M650 was used for absentee ballots

- Zero count report (to verify no votes were on the M650 prior to starting absentee voting)

- There were no system errors that cause the M650 to shut down or crash.

-M650s scan the ballots generated by the VAT with different precincts/ballots styles within the 

deck.

Reports include: 

- Printed reports of ballots counted by tabulator, with votes and undervotes

 - Printer Summary Report (containing all precincts)

 - View (save to disk) Precinct by Precinct Reports but do not print

Scenario 2) Errors should prevent the test from reaching this point. If the test does get to this 

point:

- Load election

- No system failures that cause the M650 or in the EMS ERM application to crash

- There were no system errors that cause the M650 or in the EMS ERM application to crash
VSTL Certification # pending

Page 103 of 247 (V)2009-30Jun-001(A)

Method Detail Volume 1 Test Method

Expected Results are observed Review the test result against the expected result: 

• Accept: the expected result is observed

• Reject: the expected result of the test case is not observed

• Not Testable (NT): rejection of a previous test step prevents execution of this step, or tested in 

another TC.

• Not Applicable (NA): not applicable to test scope

Record observations and all input/outputs 

for each election;

All inputs, outputs, observations, deviations and any other information impacting the integrity of 

the test results will be recorded in the test case. 

- Any failure against the requirements of the EAC guidelines will mean the failure of the system. 

and shall be reported as such. 

- Failures will be reported to the vendor as Defect Issues in the Discrepancy Report. 

- The vendor shall have the opportunity to cure all discrepancies prior to issuance of the 

Certification Report.

- If cures are submitted the applicable test will be rerun. Complete information about the rerun test 

will be preserved in the test case. The cure and results of the retest will be noted in the -

Discrepancy Report and submitted as an appendix of the Certification Report.

- Operations which do not fail the requirements but could be deemed defects or inconsistent with 

standard software practices or election practices will be logged as Informational Issues on the 

Discrepancy Report. It is the vendor's option to address these issues. Open items will be 

identified in the report. 

7.4.2.2 Volume 2 Test Results

Testing was conducted on the system configuration identified in the PCA Configuration and Test Case. Specific 

software and firmware builds for each test execution are recorded in the PCA Configuration as identified in the 

individual test case document. 

Date Test Result Issues Opened Issues Closed Notes

3/19/09 Halted - modified 

test parameters

Tester miscalculation of districting parameters for the 

test data

4/8/09 Halted -

modified test 

parameters

Tester miscalculation of districting parameters for the 

test data

4/17/09 Accept Scenarios 1 through 3

4/17/09 Reject #116 Scenario 4

6/6/09 Accept #116 Discrepancy response

Only functional issues are identified. While documentation discrepancies may be encountered in testing they do not 

result in the rejection of a functional test. All issues are documented in Appendix E- Discrepancy Report.

Method Detail Volume 2 Test Method

Test Case Name Volume 2 - Maximum Ballot Styles in a Single Precinct

Scope - identifies the type of test The scope is to test: 

Scenario 1) The maximum allowed number of 40 ballot styles on the DS200 within a single 

precinct. 

To verify that errors are generated in scenario 2: 

Scenario 2) Exceeding the maximum allowed number of 40 ballot styles on the DS200 within a 

single precinct.

Scenario 3) The maximum allowed number of 100 ballot styles on the M650 within a single 

precinct

To verify that errors are generated in scenario 4:

Scenario 4) Exceeding the maximum allowed: number of ballot styles within a single precinct

Test Objective The objective is to validate the ability to process, store and report data using the allowed 

maximum number of ballot styles within a single precinct. To validate that the system generates 

errors during EMS ballot preparation (ballot preparation including: EDM, ESSIM & HPM) when 

exceeding the maximum numbers of ballot styles within a single precinct. Validating the 

processing, storing and reporting shall occur without system degradation. If there are system 

errors that cause the system to crash the system shall recover without any loss of data.

Test Variables: General election for each scenario

-An election database can be accurately/securely defined & formatted using the Import Wizard, 

containing:

1 Precinct with 40 splits

Vote for 1 & Vote for N of M

Oval Positions Left

Certified Write-Ins

5 contest for each ballot style

4 candidates for each contest

Check EDM reports for election set up
VSTL Certification # pending

Page 104 of 247 (V)2009-30Jun-001(A)

Method Detail Volume 2 Test Method

Election day (DS200)

40 Ballot Styles on the DS200 Maximum ballot styles

Election set up for the DS200 & VAT devices

Non-partisan offices

one page ballot

4 Ovals per Inch ballot - (19 inch ballot, 68 ovals positions per Column, 6 columns per ballot, 408 

total oval positions)

Election Day Voting (DS200 and VAT)

Absentee Voting (M650)

4 candidates for each contest

4 Ovals per Inch ballot - (19 inch ballot, 68 ovals positions per Column, 6 columns per ballot, 408 

total oval positions)

Scenario 1) 1 precinct with 40 Ballot Styles on the DS200 & the VAT (DS200 Maximum ballot 

styles)

Scenario 2) 1 precinct with 41 Ballot Styles on the DS200 & the VAT (Over the DS200 Maximum 

ballot styles)

Scenario 3) 1 Precinct with 100 ballot styles on the M650 & the VAT (M650 Maximum ballot 

styles per precinct)

Scenario 4) 1 Precinct with 101 ballot styles on the M650 & the VAT (M650 Maximum ballot 

styles per precinct) (Over the maximum ballot styles)

A description of the voting system type and 

the operational environment

• Same as Volume 1 - Maximum Precincts and Ballot Styles except:

- 1 platform of each

VSS 2002 vol. 1 • Same as Volume 1 - Maximum Precincts and Ballot Styles 

VSS 2002 vol. 2 6.2.3 Volume (maximum number of ballot styles/precincts)

A4.3.5 Volume/Stress (Processing, storing and reporting data when overloading the number of 

ballot styles/precincts)

A4.3.5 Performance/Recovery (Ballot format handling capability-graceful shut down (no crash) 

and recovery without loss of data) if the number of ballot styles/precincts is exceeded

A4.3.5 Performance/Recovery (Processing rates-graceful shut down and recovery without loss of 

data)

Hardware, Software voting system 

configuration and test location

• Same as Volume 1 - Maximum Precincts and Ballot Styles 

Pre-requisites and preparation for execution 

of the test case. 

Complete the prerequisites: 

Test Method Validation: Technical review conducted by C. Coggins; Approved 1/26/09 For 

validation of test method as defined in ISO/IEC 17025 clause 5.4.5

Condition of approval - iBeta validated the successful use of the Import Wizard to import large 

amounts of data into EDM. As configuration of the imported file can impact the success of the 

data importation, the import file structure must be validated as a prerequisite of all applicable test 

cases.

Import Wizard method tested and validated: 1/23/09.

- Document in the test case the percentage that the system limit exceeds the customer maximum. 

(System Limit * 100) /Customer Maximum =% System Limit)

- 7 Excel spreadsheets saved as "Tab Delimited". Tab Delimited documents containing election 

creating information will be imported into EDM using the Import Wizard option for Scenario 1 

Election Day Voting (DS200 and VAT)

1 precinct with 40 Ballot Styles on the DS200 & the VAT (DS200 Maximum ballot styles)

- Spreadsheet 1 - Precinct 1 Splits 40 

- Spreadsheet 2 - District Type 3

- Spreadsheet 3 - Districts Names 23

- Spreadsheet 4 - District Relations 80

- Spreadsheet 5 - Master Office 82 

- Spreadsheet 6 - Office Relations 82

- Spreadsheet 7 - Candidates 328 

Scenario 2.

- Spreadsheet 1 - Precinct 1 Splits 41

- Spreadsheet 2 - District Types 3

- Spreadsheet 3 - Districts Names 24

- Spreadsheet 4 - District Relations 81

- Spreadsheet 5 - Master Office 83

- Spreadsheet 6 - Office Relations 83

- Spreadsheet 7 - Candidates 332

- 7 Excel spreadsheets saved as "Tab Delimited". Tab Delimited documents containing election 

creating information will be imported into EDM using the Import Wizard option for Scenario 3

- Spreadsheet 1 - 3 precincts, Precinct 1 40 splits, Precinct 2 & 3 30 splits (100 total)

- Spreadsheet 2 - District Types 4

- Spreadsheet 3 - Districts Names 29
VSTL Certification # pending

Page 105 of 247 (V)2009-30Jun-001(A)

Method Detail Volume 2 Test Method

- Spreadsheet 4 - District Relations300

- Spreadsheet 5 - Master Office 116

- Spreadsheet 6 - Office Relations 116

- Spreadsheet 7 - Candidates 464

Scenario 4

- Spreadsheet 1 - 3 precincts, Precinct 1 40 splits, Precinct 2 & 3 31 splits (101 total)

- Spreadsheet 2 - District Types 4

- Spreadsheet 3 - Districts Names 30

- Spreadsheet 4 - District Relations302

- Spreadsheet 5 - Master Office 117

- Spreadsheet 6 - Office Relations 117

- Spreadsheet 7 - Candidates 468

Getting Started Checks Check the voting system to: 

- Same as Volume 1 - Maximum Precincts and Ballot Styles

Documentation of Test Data & Test 

Results

Test Data:

- Same as Volume 1 - Maximum Precincts and Ballot Styles 

Volume: Paper-based voting systems

Processing

Ballot Prep: 

Scenario 1 & 3 maximum limits:

-An election database can be accurately/securely defined & formatted using the Import Wizard. 

and containing

1 Precinct

Vote for 1 (contest 1 & 2) & Vote for N of M (contest 3,4, & 5)

19 inch ballot

5 contest for each ballot style

4 candidates for each contest

- Check EDM reports for election set up

Scenario 1) -Election day (DS200)

-40 Ballot Styles on the (DS200 Maximum ballot styles)

-Election set up for the DS200 & VAT devices

Senario3) -Absentee voting (M650)

-100 Ballot Styles on the (M650 Maximum ballot styles)

-Election set up for the M650 & VAT devices

- If there are any system errors that cause the EMS ballot preparation applications to crash then 

verify the applications recover without any loss of data.

Scenario 2 & 4 Exceeding limits: 

Test execution of Scenario 2 & 4 stop at this point with errors generated prior to the creation of 

election media in ballot preparation

- Check audit logs for critical status messages. Test stops unless system does not error and 

creates media

- If EDM does not error during the "Ballot Sets Merge" then the EDM reports must be reviewed to 

verify the DS200 has 41 ballot styles and the M650 has 101 ballot styles have been created and 

assigned to Election Day Polling Places. Continue to ESSIM and HPM. The system should 

display a critical status message prior to exiting the HPM.

Scenario2) -Election day (DS200)

Same as Scenario 1 except:

-41 Ballot Styles on the DS200

Scenario 4) -Absentee voting (M650)

Same as Scenario 3 except: 

-101 Ballot Styles on the 

-Election set up for the M650 & VAT devices 

- If there are any system errors that cause the EMS ballot preparation applications to crash then 

verify the applications recover without any loss of data.

Volume: Same as Volume 1 - Maximum Precincts and Ballot Styles; except

- the system responds to processing more than the expected number of ballot styles in a single 

precinct

Stress System provides a response to an overloading condition: Exceeding the maximum allow number 

of ballot styles in a single precinct.

Performance There is no system degradation (ballot format handling capability and processing rates):

-When importing large amount of data into the EDM using the Import Wizard.

-When installing an election with 1 precinct and over the maximum number of ballot styles for a 

give device 

- The system will not slow down as more and more data is added

Error Recovery Same as Volume 1 - Maximum Precincts and Ballot Styles; except - the errors are caused by 

overloading the number ballots styles per precinct. 

Readiness Testing and Poll Verification Same as Volume 1 - Maximum Precincts and Ballot Styles; except

- The device is ready to be activated to accept votes with the maximum ballot styles per a single 
VSTL Certification # pending

Page 106 of 247 (V)2009-30Jun-001(A)

Method Detail Volume 2 Test Method

precinct (No Identification of any failures & corrective action)

Pre- vote: Opening the Polls Verification Precinct Count/ Paper based:

- Zero count report (verify no votes are on the DS200 prior to starting Election Day voting)

Voting: Ballot Activation and Casting 

Verifications

Protects secrecy of ballot/vote

Scenario 1) 

- 20 ballots will be test (a 50% sample of 40 ballot styles) 

- VAT -Generate the ballots for 20 different ballot styles within the deck.

- DS200- scans the ballots generated by the VAT with different ballot styles within the deck.

- Ballot styles 1 through 10, 20 and 40 will be voted

- The DS200 In Election Day mode with a single precinct and 40 ballot styles will not error. If 

there are any system errors that cause the DS200 to shut down then the DS200 shall recover 

without any loss of data.

- The VAT with a single precinct and 40 ballot styles will not error. If there are any system errors 

that cause the VAT to shut down then the VAT shall recover without any loss of data.

Scenario 2) Errors should prevent the test from reaching this point. If the test does get to this 

point: DS200 and VAT

- Load election

- No system failures that cause the DS200 and VAT to crash

- If there are any system errors that cause the DS200 and VAT to crash then the DS200 and VAT 

shall recover without any loss of data.

Voting: Voting System Integrity, System 

Audit, Errors & Status Indicators

Same as Volume 1 - Maximum Precincts and Ballot Styles; except -report of normal/abnormal 

events is found within the 50% sample.

Post-vote: Closing the Polls Once the polls are closed the voting system

- Printed reports of ballots counted on the DS200

- Reported votes match predicted votes from tabulator with votes and undervotes

- DS200 Prints a single precinct totals report totaling all ballot styles within the precinct (Election 

Day voting ends)

Post-vote: Central Count Paper Based: 

Scenario 2) 

- Election identification

- Zero count report (to verify no votes are on the M650 prior to starting absentee voting) 

- 20 ballots will be test (a 20% sample of 100 ballot styles) 

- VAT -Generate the ballots for 20 different ballot styles within the deck.

- M650- scans the ballots generated by the VAT with different ballot styles within the deck.

- Ballot styles 1 through 10, 20 and 40 will be voted

- The M650 is used for Absentee ballots with a single precinct and 100 ballot styles will not error 

will not error. If there are any system errors that cause the M650 to shut down then the M650 

shall recover without any loss of data.

Scenario 1 & 3)

Vote Consolidation:

ERM consolidated reports match the predicted votes from the polling places 

Reports include:

- Printed reports of ballots counted by tabulator, with votes and undervotes

 - Print Summary Report (containing all a single precinct)

 - View and Print Precinct by Precinct Reports

Scenario 4) Errors should prevent the test from reaching this point. If the test does get to this 

point: M650

- Load election

- No system failures that cause the M650 to crash

- If there are any system errors that cause the M650 to crash then the M650 shall recover without 

any loss of data.

Scenario 2& 4) Errors should prevent the test from reaching this point. If the test does get to this 

point: ERM

- Load election

- No system failures that cause the ERM application to crash

- If there are any system errors that cause the ERM to crash then the ERM application shall 

recover without any loss of data.

Expected Results are observed Review the test result against the expected result: 

Same as Volume 1 - Maximum Precincts and Ballot Styles 

Record observations and all input/outputs 

for each election;

All inputs, outputs, observations, deviations and any other information impacting the integrity of 

the test results will be recorded in the test case. 

Same as Volume 1 - Maximum Precincts and Ballot Styles 
VSTL Certification # pending

Page 107 of 247 (V)2009-30Jun-001(A)

7.4.2.3 Volume 3 Test Results

Testing was conducted on the system configuration identified in the PCA Configuration and Test Case. Specific 

software and firmware builds for each test execution are recorded in the PCA Configuration as identified in the 

individual test case document. 

Date Test Result Issues Opened Issues Closed Notes

5/22/09 Accept Test executed without issues or halting

Method Detail Volume 3 Test Method

Test Case Name Volume 3 - Audit Manager database test

Scope - identifies the type of test The scope of the test was to confirm that 2GB JET database can record and store audit inputs 

generated in the Election Data Manger for a period of 72 consecutive hours (150% of the ES&S 

predicted maximum).

Test Objective The objective was to validate that the Audit Manager capacity can record and retain data inputs 

(150%) of the ES&S predicted maximum time of use in an election. (48 hours estimated 

maximum run for 72 consecutive hours). Throughout the 72 hours of testing the application 

should not have any system crashes, loss of data and/or loss of degradation. If there are system 

errors that cause the system to crash the system shall recover without any loss of data. 

Test Variables: General election -

• Same as Volume 1 - Maximum Precincts Limitations and ballot styles for paper except:

- only using Scenario 1

A description of the voting system type and 

the operational environment

The Unity 3.2.0.0 EMS Ballot Preparation includes:

Audit Manger (AM ) and Election Data Manger (EDM)

VSS 2002 vol. 1 2.1.5.1b Audit/Error message

2.2.5.2.3 Status message

5.4.1 Audit/description of modifications with time stamp

2.2.3 Error Recovery

VSS 2002 vol. 2 A4.3.5 Volume (Processing, storing and reporting data when overloading the systems capacity)

A4.3.5 Performance/Recovery (Ballot format handling capability-graceful shut down and recovery 

without loss of data)

A4.3.5 Performance/Recovery (Processing rates-system does not slow down as more data was 

being added, no loss of data, and no system crashes)

Stress - overloading conditions over a consecutive period of 72 hours.

Hardware, Software voting system 

configuration and test location

The Unity 3.2 Voting System consists of the following: 

Audit Manger (AM) and Election Data Manger (EDM) 

All testing was performed by iBeta LLC located at 3131 S. Vaughn Way, Aurora, CO 80014.

Pre-requisites and preparation for execution 

of the test case. 

Complete the prerequisite

• Same as Volume 1 - Maximum Precincts Limitations and ballot styles for paper 

Test Method Validation: Technical review conducted by C. Coggins; Approved 2-15-09 for 

validation of test method as defined in ISO/IEC 17025 clause 5.4.5. 

• Same as Volume 1 - Maximum Precincts Limitations and ballot styles for paper except - only 

using Scenario 1

Getting Started Checks Check the voting system to :

• Same as Volume 1 - Maximum Precincts Limitations and ballot styles for paper 

Documentation of Test Data & Test 

Results

Test Data:

• Same as Volume 1 - Maximum Precincts Limitations and ballot styles for paper 

Volume: Paper-based voting systems

Processing

Ballot Prep: 

Using an automation tool the EDM and AM application ran for 90.5 hours consecutively importing 

election data. (189% of ES&S predicted time of use in an election)

- Automation Anywhere

- EDM Import Wizard options

- Same spreadsheets as Volume 1 - Maximum Precincts Limitations and ballot styles for paper 

Volume: System responses when attempting to overload the systems capacity:

- Successfully processed without errors.

- Processed, stored, and reported data.

Stress System responded when attempting to overload conditions within 90.5 hours. 

Performance No noticeable system degradation (Processing rates):

-during the 90.5 consecutive hours of operation and accessing the Audit Manager logs.

Error Recovery The Audit Manager application should not error or crash within the 90.5 consecutive hours. 

- If the application does error the system shall provide a clear description of the problem. 

- If there are any system errors that cause the Audit Manager application to crash then the 

application shall recover without any loss of data.

Readiness Testing and Poll Verification Not Applicable (Audit Manager is not located at the polls)

Pre- vote: Opening the Polls Verification Not Applicable (Audit Manager is not located at the polls)

Voting: Ballot Activation and Casting 

Verifications

Not Applicable (Audit Manager is not located at the polls)

Voting: Voting System Integrity, System 

Audit, Errors & Status Indicators

Not Applicable (Audit Manager is not located at the polls)
VSTL Certification # pending

Page 108 of 247 (V)2009-30Jun-001(A)

Method Detail Volume 3 Test Method

Post-vote: Closing the Polls Not Applicable (Audit Manager is not located at the polls)

Post-vote: Central Count Not Applicable (Audit Manager is not located at the Central Count)

Expected Results are observed Review the test result against the expected result: 

• Same as Volume 1 - Maximum Precincts Limitations and ballot styles for paper 

Record observations and all input/outputs 

for each election;

All inputs, outputs, observations, deviations and any other information impacting the integrity of 

the test results will be recorded in the test case.

• Same as Volume 1 - Maximum Precincts Limitations and ballot styles for paper

7.4.2.4 Volume 4 Test Results

Testing was conducted on the system configuration identified in the PCA Configuration and Test Case. Specific 

software and firmware builds for each test execution are recorded in the PCA Configuration as identified in the

individual test case document.

Date Test Result Issues Opened Issues Closed Notes

4/13/09 Accept None None Confirmed appropriate errors generated

Method Detail Volume 4 Test Method

Test Case Name Volume 4 - Storage Error Generation

Scope - identifies the type of test The Test Scope is to test: The M650 and DS200 component media generate an error messages 

when capacity was reached

Test Objective The objective is to validate error messages are generated when media capacity has been 

reached.

Test Variables: Same as Volume 10 - Maximum ballot limitations except: 512MB USB (491 free space) for the 

DS200 with over 488MB of storage used; 100MB for the M650 with over 85MB of storage used.

A description of the voting system type and 

the operational environment

The Unity 3.2.0.0 precinct count includes: DS200

The Unity 3.2.0.0 central count tabulator: Model 650 (M650)

VSS 2002 vol. 1 2.2.5.2.2 System Audit Error Messages

2.2.5.2.3 System Audit Status Messages

VSS 2002 vol. 2 A4.3.5 Performance/Recovery (Processing rates-graceful shut down "no system crash" and 

recovery without loss of data)

A4.3.5 Stress (system response to overloading data on hardware media)

Hardware, Software voting system 

configuration and test location

The Unity 3.2 Voting System consist of the following: DS200, Model 650 (M650) 

Testing perform by iBeta LLC located at 3131 S. Vaughn Way, Aurora, CO 80014.

Pre-requisites and preparation for execution 

of the test case. 

Complete the prerequisites; Test Method Validation: Technical review conducted by C. Coggins; 

Approved 2/23/09. for validation of test method as defined in ISO/IEC 17025 clause 5.4.5. -

Condition of approval - iBeta validates component media was populated to near capacity prior to 

test execution.

Getting Started Checks Check the voting system to : • Same as Volume 1 - Maximum Precincts Limitations and ballot 

styles for paper 

Documentation of Test Data & Test 

Results

Test Data: - Same as Volume 1 - Maximum Precincts Limitations and ballot styles for paper 

Volume: Paper-based voting systems

Processing

Same as Volume 10 - Maximum ballot limitations

Volume: Not Applicable (only testing for error generation of full media on hardware)

Stress Not Applicable (only testing for error generation of full media on hardware)

Performance No system degradation (Ballot Processing rate): - On the M650 and DS200 with a large amount 

of data filling up the media storage the system will not be observed to slow down during testing

Error Recovery The systems should not error or crash. - If the application does error the system shall provide a 

clear description of the problem.

Readiness Testing and Poll Verification Not Applicable (only testing for error generation of full media on hardware)

Pre- vote: Opening the Polls Verification Pre-Vote:-Same as Volume 1 - Maximum Precincts Limitations and ballot styles for paper 

Voting: Ballot Activation and Casting 

Verifications

DS200 Only- Election Day Voting - in Polling Place 1 Precincts/Ballot Style 1.

- Using media that was near capacity scan the marked ballots from Volume 10 ballots until the 

error "Full memory" is generated.

- error message must advise the official how to handle the error.

- There were no system errors that cause the DS200 to crash.

Voting: Voting System Integrity, System 

Audit, Errors & Status Indicators

The system audit provides a time stamped, report of normal/abnormal events found within the 

tested. Error messages are: 

- Are generated, stored & reported as they occur

- Errors requiring intervention by the poll worker clearly display issues & action instructions in 

easily understood text language or with indicators

- Incorrect responses will not lead to irreversible errors. 

Post-vote: Closing the Polls Not Applicable (only testing for error recovery of full media on hardware)
VSTL Certification # pending

Page 109 of 247 (V)2009-30Jun-001(A)

Method Detail Volume 4 Test Method

Post-vote: Central Count M650 Paper Based: 

- Zero count report (Absentee) 

- using media that was near capacity scan the marked ballots from Volume 10 ballots until an 

error "Full memory" generated.

- If there are any system errors that cause the M650 to crash then the M650 shall recover without 

any loss of data.

Expected Results are observed Review the test result against the expected result: 

• Same as Volume 1 - Maximum Precincts Limitations and ballot styles for paper 

Record observations and all input/outputs 

for each election;

All inputs, outputs, observations, deviations and any other information impacting the integrity of 

the test results will be recorded in the test case. 

• Same as Volume 1 - Maximum Precincts Limitations and ballot styles for paper 

7.4.2.5 Volume 5 Test Results

Testing was conducted on the system configuration identified in the PCA Configurations and Volume Test Cases 1 

though 4 and 6 though 10.

Date Test 

Result

Issues 

Opened

Issues 

Closed

Notes

6/8/09 Accept

Method Detail Volume 5 Test Method

Test Case Name Volume 5 - Electrical Supply Recovery 

Scope - identifies the type of test Recovery tests verify the ability of the system to recover from hardware and data errors. Power 

recovery was tested by SysTest in the Electrical Supply Test Case. ES&S has petitioned the EAC 

for reuse of the applicable components in scope for Unity 3.2.0.0 from the SysTest Labs testing of 

the Unity v.4.0.0.0 certification test effort. Determination of reuse is based upon the EAC review 

of SysTest Labs Electrical Supply test results. 

iBeta incorporates verification of audit logging of error recovery in the Volume test cases.

Test Objective The objective of the test case is to verify the ability of the system to recover from electrical supply 

and audit logging of errors. 

Test Variables: The test variables for the SysTest Labs' Electric Supply test case is contained in Rev. 10 of the 

EAC approved Unity v.4.0.0.0 Test Plan and the associated test case.

The test variables for the iBeta Volume Test Methods are identified in Volume Tests through 4 

and 6 through 10

A description of the voting system type and 

the operational environment

The voting system type and operational environment for SysTest Labs' usability, accessibility and 

maintainability testing is identified in Rev. 10 of the EAC approved Unity v.4.0.0.0 Test Plan

VSS 2002 vol. 1 2.2.5.2.2 Audit/Error messages

2.2.3.2.3 Audit/Status messages

2.2.3 Error Recovery

VSS 2002 vol. 2 A4.3.5 Stress (high volume with interrupts and overloading the systems)

A4.3.5 Recovery (system recovers from software and hardware errors without loss of data)

Hardware, Software voting system 

configuration and test location

The hardware, software voting system configuration and location of testing for SysTest Labs' 

Electrical Supply testing is identified in Rev. 10 of the EAC approved Unity v.4.0.0.0 Test Plan

iBeta - Identified in Volume Tests 1 through 4 and 6 through 10

Pre-requisites and preparation for execution 

of the test case. 

Complete the prerequisites:

- Document in the test case the percentage that the system limit exceeds the customer 

maximum. (System Limit * 100) /Customer Maximum =% System Limit) 

Test Method Validation: Technical review conducted by C. Coggins; Approved 2/4/09 for 

validation of test method as defined in ISO/IEC 17025 clause 5.4.5. 

Determination by the EAC allowing the reuse of SysTest Labs Electrical Supply test. 

iBeta Volume Tests 1 through 4 and 6 through 10 test cases have been executed and passed

Getting Started Checks Identified in Volume Tests 1 through 4 and 6 through 10

Documentation of Test Data & Test 

Results

Identified in Volume Tests 1 through 4 and 6 through 10

Volume: Paper-based voting systems

Processing

Not applicable to Electrical Supply Recovery

Volume: Not applicable to Electrical Supply Recovery

Stress EAC to review the SysTest Labs test results and verifies:

Software responds to power interrupts

iBeta to review the Volume test results and verifies the system responds to interrupts. 

Performance EAC to review the SysTest Labs Cases and verifies: 

Voting system is able to recover gracefully from errors or crashes caused by power failures 

without loss of data
VSTL Certification # pending

Page 110 of 247 (V)2009-30Jun-001(A)

Method Detail Volume 5 Test Method

iBeta to review the Volume test results and verifies the system recovers from errors or crashes 

without loss of data

Error Recovery EAC to review the SysTest Labs Cases and verifies: 

Voting system is able to recover from errors or crashes caused by power failures.

iBeta to review the Volume test results and verifies the system recovers from errors or crashes

Readiness Testing and Poll Verification Not applicable to Electrical Supply Recovery

Pre- vote: Opening the Polls Verification Not applicable to Electrical Supply Recovery

Voting: Ballot Activation and Casting 

Verifications

Not applicable to Electrical Supply Recovery

Voting: Voting System Integrity, System 

Audit, Errors & Status Indicators

Not applicable to Electrical Supply Recovery

Post-vote: Closing the Polls Not applicable to Electrical Supply Recovery

Post-vote: Central Count Not applicable to Electrical Supply Recovery

Expected Results are observed Review the test result against the expected result: 

Same as Volume 1 - Maximum Precincts and Ballot Styles 

Record observations and all input/outputs 

for each election;

All inputs, outputs, observations, deviations and any other information impacting the integrity of 

the test results will be recorded in the test case. 

Same as Volume 1 - Maximum Precincts and Ballot Styles 

7.4.2.6 Volume 6 Test Results

Testing was conducted on the system configuration identified in the PCA Configuration and Test Case. Specific 

software and firmware builds for each test execution are recorded in the PCA Configuration as identified in the 

individual test case document. 

Date Test Result Issues Opened Issues Closed Notes

3/23/09 Accept Test parameters were adjusted prior to test execution. 

These changes are identified in the as run Test Plan

Only functional issues are identified. While documentation discrepancies may be encountered in testing they do not 

result in the rejection of a functional test. All issues are documented in Appendix E- Discrepancy Report.

Method Detail Volume 6 Test Method

Test Case Name Volume 6 - Maximum number precincts and Maximum number of candidates per polling place.

Scope - identifies the type of test The scope is to:

Scenario 1) Test the maximum allowed: number of precincts and maximum number of 

candidates per polling place.

To verify that errors are generated when:

Scenario 2) Exceeding the HPM maximum allowed: number of precincts in a single polling place

Test Objective The objective is to validate the ability to process, store and report data to the maximum and 

exceeding the maximum allowed number of precincts in a single polling place. To validate that the 

system generates errors during EMS ballot preparation (ballot preparation including: EDM, 

ESSIM & HPM) when exceeding maximum the allowed number of precincts in a single polling 

place. Validating the processing, storing and reporting shall occur without system degradation. If 

there are system errors then the system shall recover without any loss of data.

Test Variables: General election

Scenario 1)

- DS200 set up for Early Voting

- 19 inch ballot (4 Ovals per inch)

- 1900 precincts (early voting)

- 8 ballot styles

- 8 Non Partisan contest

- Precincts 1 - 6 with each will a single contest containing 175 candidates per contest (ballot style 

1-6)

- Precincts 7-800 with 75 candidates in a single contest (ballot style 7)

- Precincts 801 - 1900 with 75 candidates in a single contest (ballot style 8)

- Vote for 1

- 1 Statistical Counters (Precincts Counted)

- 1 Polling Place

Scenario 2) Same as scenario 1 except:

- 9 ballot styles

- 9 Non-Partisan contest 

- Precincts 1901 with 2 candidates in a single new contest (1 new ballot style, 1 new precincts , 1 

new contest, same polling place as in Scenario 1)

A description of the voting system type and 

the operational environment

• Same as Volume 1 - Maximum Precincts Limitations and ballot styles for paper 
VSTL Certification # pending

Page 111 of 247 (V)2009-30Jun-001(A)

Method Detail Volume 6 Test Method

VSS 2002 vol. 1 • Same as Volume 1 - Maximum Precincts Limitations and ballot styles for paper 

VSS 2002 vol. 2 A4.3.5 Volume (maximum and exceeding more than the maximum number of precincts in a 

Polling Place)

A4.3.5 Volume/Stress (Processing, storing and reporting data when overloading the number of 

precincts in a Polling Place)

A4.3.5 Performance/Recovery (Ballot format handling capability-graceful shut down and recovery 

without loss of data)

A4.3.5 Performance/Recovery (Processing rates-graceful shut down and recovery without loss of 

data)

Hardware, Software voting system 

configuration and test location

The Unity 3.2 Voting System consist of the following:

Audit Manger (AM ), Election Data Manger (EDM), ES&S Image Manager (ESSIM), hardware 

Program Manger (HPM), DS200, Election Reporting Manager (ERM), AutoMARK Information 

(AIMS), Voter Terminal(VAT)

All testing will be perform by iBeta located at 3131 S. Vaughn Way, Aurora, CO 80014.

Pre-requisites and preparation for execution 

of the test case. 

Complete the prerequisites:

Test Method Validation: Technical review conducted by C. Coggins; Approved 1/27/09 for 

validation of test method as defined in ISO/IEC 17025 clause 5.4.5. 

Condition of approval - iBeta validated the successful use of the Import Wizard to import large

amounts of data into EDM. As configuration of the imported file can impact the success of the 

data importation, the import file structure must be validated as a prerequisite of all applicable test 

cases.

- Document in the test case the percentage that the system limit exceeds the customer maximum. 

(System Limit * 100) /Customer Maximum =% System Limit ) 

Import Wizard method tested and validated:

- 7 Excel spreadsheets saved as "Tab Delimited". Tab Delimited documents containing election 

creating information will be imported into EDM using the Import Wizard option.

- Spreadsheet 1 - Precinct 1900

- Spreadsheet 2 - District Types 8

- Spreadsheet 3 - Districts Names 8

- Spreadsheet 4 - District Relations 8

- Spreadsheet 5 - Master Office 8 

- Spreadsheet 6 - Office Relations 8

- Spreadsheet 7 - candidates 1200

Getting Started Checks Check the voting system to :

Same as Volume 1 - Maximum Precincts Limitations and ballot styles for paper 

Documentation of Test Data & Test 

Results

Test Data:

Same as Volume 1 - Maximum Precincts Limitations and ballot styles for paper 

Volume: Paper-based voting systems

Processing

Ballot Prep:

-An election database can be accurately/securely defined & formatted using the Import Wizard.

-Ballots (candidates) can be accurately defined & generated.

Scenario 1) Election can be created and installed with 1900 Precincts in a single Polling Place.

No error occurs

- If there are any system errors that cause the EMS ballot preparation applications to crash then 

verify the applications recover without any loss of data.

- Review the EDM reports to verify election set up. 

Scenario 2)

Same as scenario 1 except over the maximum allowed number of Precincts in a single Polling 

Place (1901)

Test execution of Scenario 2 is expected to stop at this point with errors generated in the ballot 

preparation prior to the creation of election media 

- Check audit logs for critical status messages. Test stops unless system does not error and 

creates media) 

- If EDM does not error during the "Ballot Sets Merge" then the EDM reports must be reviewed to 

verify 1901 precincts have been created and assigned to a single early voting Polling Place. 

Continue to ESSIM and HPM. The system should display a critical status message prior to 

exiting the HPM.

- If there are any system errors that cause the EMS ballot preparation applications to crash then 

verify the applications recover without any loss of data.

Volume: Systems capacity to process, store, and report data. 

- When importing over the allowed amount of data into the EDM using the Import Wizard

Stress System responses to overloading conditions. Exceeding the maximum allowed number of Early 

Voting precincts in a single Polling Place.

Performance There is no system degradation (Ballot format handling capability and Processing rates):

- When importing large amount of data into the EDM using the Import Wizard.

- The system does not slow down throughout the testing

Error Recovery Same as Volume 1 - Maximum Precincts Limitations and ballot styles for paper 
VSTL Certification # pending

Page 112 of 247 (V)2009-30Jun-001(A)

Method Detail Volume 6 Test Method

Readiness Testing and Poll Verification Voting system is ready for the election: 

Same as Volume 1 - Maximum Precincts Limitations and ballot styles for paper except:

- Run 1 precincts to validate the system is ready; confirm the test data is segregated from voting 

data, with no residual effect. Verify totals and audit logs.

Pre- vote: Opening the Polls Verification Precinct Count/ Paper based:

Same as Volume 1 - Maximum Precincts Limitations and ballot styles for paper 

Voting: Ballot Activation and Casting 

Verifications

Scenario 1) Election Day Voting - The VAT & DS200 are in Polling Place 1 with Precincts 1-

1900.

- Voting using 95 different precincts (5% of 1900 precincts), 2 ballots per precinct for a total of 190 

ballots (10% sample voted). 

- Mark ballot using the VAT

- Scan using the DS200

- No errors are expected.

- If there are any system errors that cause the DS200 & the VAT to crash then verify the DS200 

and the VAT recover without any loss of data.

- Verify the counter (number of voters) on the DS200 and the VAT match the expect results.

Scenario 2) Errors should prevent the test from reaching this point. If the test does get to this 

point:

- Load election

- No system failures that cause the DS200 and/or the VAT to crash

- If there are any system errors that cause the DS200 and the VAT to crash then the DS200 and 

the VAT shall recover without any loss of data.

Voting: Voting System Integrity, System 

Audit, Errors & Status Indicators

Same as Volume 1 - Maximum Precincts Limitations and ballot styles for paper 

Post-vote: Closing the Polls Once the polls are closed the voting system

- Printed reports of ballots counted by tabulator

- Reported votes match predicted votes from tabulator with votes and undervotes.

- In Polling Place 1 the DS200 prints precincts 1 - 1900 totals (early voting ends)

Post-vote: Central Count Paper Based: When loading results mix the input of results such that media is read out of precinct 

order and where possible mix the reading of DS200 results. Record the order at test execution. 

Scenario 1) 

The central count voting system M650 Not Applicable in this test case

Vote Consolidation: 

- ERM consolidated reports match the predicted votes.

- Verify no data was lost within the audit logs or results

Reports include: 

- Printed reports of ballots counted by tabulator, with votes and undervotes

 - Printer Summary Report 

 - View and Print Precinct by Precinct Reports

Scenario 2) 

Errors should prevent the test from reaching this point. If the test does get to this point:

- Load election

- If there are any system errors that cause the EMS ERM application to crash then the ERM 

application shall recover without any loss of data.

Expected Results are observed Review the test result against the expected result: 

Same as Volume 1 - Maximum Precincts Limitations and ballot styles for paper 

Record observations and all input/outputs 

for each election;

All inputs, outputs, observations, deviations and any other information impacting the integrity of 

the test results will be recorded in the test case. 

Same as Volume 1 - Maximum Precincts Limitations and ballot styles for paper 

7.4.2.7 Volume 7 Test Results

Testing was conducted on the system configuration identified in the PCA Configuration and Test Case. Specific 

software and firmware builds for each test execution are recorded in the PCA Configuration as identified in the 

individual test case document. 

Date Test Result Issues Opened Issues Closed Notes

3/16/09 Halted modified 

test parameters

Test data provide by ES&S was for a General Election 

and not a Primary as stated in the Test Case

3/20/09 Halted modified 

test parameters

ES&S' internal testing found a lower contest limit for 

Primary Election. No issue was logged as new 

documentation was immediately provided. The test 

case was modified to now test the limits for both a 

General and an Open Primary 

5/2/09 Accept #30, 32, 33

Only functional issues are identified. While documentation discrepancies may be encountered in testing they do not 

result in the rejection of a functional test. All issues are documented in Appendix E- Discrepancy Report.
VSTL Certification # pending

Page 113 of 247 (V)2009-30Jun-001(A)

Method Detail Volume 7 Test Method

Test Case Name Volume 7 - Maximum ballot limitations

Scope - identifies the type of test The scope is to test:

Scenario 1) For all election types except an Open Primary the maximum allowed : number of 

contests in a ballot style; number of candidates in a contest; number of parties; number of " VOTE 

FOR" in a contest; and number of candidate counters in a precinct

Scenario 3) For an Open Primary the maximum allowed: number of contests in a ballot style; 

number of candidates in a contest; number of parties; number of "VOTE FOR" in a contest; and 

number of candidate counters in a precinct

To verify that errors are generated or that the system continues without system failure when 

exceeding the maximum limits in scenarios 2 and 4:

Scenario 2 & 4) The maximum allowed number of candidates in a contest, number of parties, 

number of " VOTE FOR" in a contest, while exceeding the maximum: number of contests in a 

single ballot style. The maximum allowed number of contests in a ballot style and candidate 

counters in a precinct while exceeding the maximum: candidates in a contest; "VOTE FOR" in a 

contest. The maximum allowed number of contests in a ballot style, candidates in a contest, 

number of "VOTE FOR" in a contest, and number of candidate counters in a precinct while 

exceeding the maximum: number of parties. The maximum allowed number of contests in a 

ballot style, candidates in a contest, number of parties, number of "VOTE FOR" in a contest, while 

exceeding the maximum: number of candidate counters in a precinct. 

- Discrepancy 30 (SysTest 429 Election description, Ballot Name/Full path to ballot definition file )

- Discrepancy 32(SysTest 453 orientation ballot errors)

- Discrepancy 33(SysTest 454 internal rollers)

Test Objective The objective is to validate the ability to process, store and report data to the maximum and 

exceed the maximum allowed number of contest in a ballot style, maximum number of candidates 

in a contest, maximum number of parties, maximum number of "VOTE FOR" in a contest, and the 

maximum number of candidate counters in a precinct. To validate that the system generates 

errors during EMS ballot preparation (ballot preparation including: EDM, ESSIM & HPM) when 

exceeding maximum allowed limits. Validating the processing, storing and reporting shall occur 

without system degradation. If there are system errors that cause the system to crash the system 

shall recover without any loss of data.

Test Variables: Scenario 1) General Election

2 Precincts (Precinct 1/ballot style 1& Precinct 2/ballot style 2)

- 2 Statistical Counter (Precincts counted, Ballots counted)

1 Polling Place

19 inch ballot (4 ovals per inch, 68 oval positions per column, 408 total positions) 

Precinct 1/ballot style 1

- 1 Partisan contest:

- 18 parties (max allowed in an election)

- Vote for 1

- 3 candidates per party

- 1 Non-Partisan contest:

- vote for 90 (max allowed in a contest)

- 175 candidates (max allowed in a contest)

Precinct 2/ballot style 2 - 200 Non-Partisan contest (max number of contest allowed with a 19 

inch ballot)

 - vote for 1

 - 200 candidates (1 candidate per contest)

Counters:

200 candidates

200 undervotes

200 overvotes

400 Statistical Counter 

1000 total counters in a precinct

Scenario 2) Same as scenario 1 except :

- Precinct 2/ballot style 2: 201 contest and 201 candidates (exceeding contest in a single ballot 

style)

- Precinct 1/ballot style 1Non-Partisan contest: 176 candidates, Vote For 91(exceeding 

candidates and VOTE FOR in a contest)

- Precinct 1/ballot style 1 Partisan contest: 19 parties

- Precinct 2/ballot style 2: 3 Statistical Counters (exceeding candidate counters in a precinct)

Counters:

201 candidates

201 undervotes

201 overvotes

401 Statistical Counter 

1004 total counters in a precinct 

Scenario 3)
VSTL Certification # pending

Page 114 of 247 (V)2009-30Jun-001(A)

Method Detail Volume 7 Test Method

Same as scenario 1 except:

-Open Primary Election

- Precinct 2/ballot style 2 - 70 Non-Partisan contest (max number of contest allowed with a 19 

inch ballot for an Open Primary Election)

Scenario 4)

Same as scenario 1 except:

-Open Primary Election

- Precinct 2/ballot style 2 - 71 Non-Partisan contest (max number of contest allowed with a 19 

inch ballot for an Open Primary Election)

A description of the voting system type and 

the operational environment

Same as Volume 1 - Maximum Precincts and Ballot Styles 

VSS 2002 vol. 1 Same as Volume 1 - Maximum Precincts and Ballot Styles 

VSS 2002 vol. 2 6.2.3 Volume (maximum number Parties, Vote for, Statistical Counters, candidates in a single 

contest, and contests )

A4.3.5 Volume (maximum and exceeding more than the maximum number of Parties, Vote for, 

Statistical Counters, candidates in a single contest, and contests)

A4.3.5 Volume/Stress (Processing, storing and reporting data when overloading the number of 

Parties, Vote for, Statistical Counters, candidates in a single contest, and contests)

A4.3.5 Performance/Recovery (Ballot format handling capability-graceful shut down (no crash) 

and recovery without loss of data)

A4.3.5 Performance/Recovery (Processing rates- shut down (no crash)and a graceful recovery 

without loss of data)

Hardware, Software voting system 

configuration and test location

Same as Volume 1 - Maximum Precincts and Ballot Styles 

Pre-requisites and preparation for execution 

of the test case. 

Complete the prerequisites: 

Test Method Validation: Technical review conducted by C. Coggins; Approved 1/27/09 for 

validation of test method as defined in ISO/IEC 17025 clause 5.4.5. -

- Document in the test case the percentage that the system limit exceeds the customer maximum. 

(System Limit * 100) /Customer Maximum =% System Limit)

- 7 Excel spreadsheets saved as "Tab Delimited". Tab Delimited documents containing election 

creating information will be imported into EDM using the Import Wizard option.

Scenario 1)

- Spreadsheet 1 - Parties 18

- Spreadsheet 2 - Precinct 2

- Spreadsheet 3 - Districts Names 3

- Spreadsheet 4 - District Relations 3

- Spreadsheet 5 - Master Office 202

- Spreadsheet 6 - Office Relations 202

- Spreadsheet 7 - Candidates 429

Scenario 3)

- Spreadsheet 1 - Parties 18

- Spreadsheet 2 - Precinct 2

- Spreadsheet 3 - Districts Names 3

- Spreadsheet 4 - District Relations 3

- Spreadsheet 5 - Master Office 89

- Spreadsheet 6 - Office Relations 89

- Spreadsheet 7 - Candidates 299

Getting Started Checks Check the voting system to :

Same as Volume 1 - Maximum Precincts and Ballot Styles 

Documentation of Test Data & Test 

Results

Test Data:

Same as Volume 1 - Maximum Precincts and Ballot Styles 

Volume: Paper-based voting systems

Processing

Ballot Prep:

-An election database can be accurately/securely defined & formatted using the Import Wizard.

- Discrepancy 30 (SysTest 429 Election description, Ballot Name/Full path to ballot definition file) 

using the default file name. 

-Ballots (candidates & propositions) can be accurately defined & generated.

Scenario 1 General Election and Scenario 3 Open Primary Election) Election media can be 

installed with the maximum allowed number of contests in a ballot style, maximum number of 

candidates in a contest, maximum number of parties, maximum number of " VOTE FOR" in a 

contest, and the maximum number of candidate counters in a precinct without error.

- If there are any system errors that cause the EMS ballot preparation applications to crash then 

verify the applications recover without any loss of data.

Scenarios 2 and 4)

Test execution of Scenario's expected to stop at this point with errors generated prior to the 

creation of election media in ballot preparation)

- Check audit logs for critical status messages. Test stops unless system does not error and 

creates media) 

- If EDM does not error during the "Ballot Sets Merge" then the EDM reports must be reviewed to 

verify each of Scenarios listed below have been created exceeding the ballot limits. Continue to 

ESSIM and HPM. The system should display a critical status message prior to exiting the HPM.
VSTL Certification # pending

Page 115 of 247 (V)2009-30Jun-001(A)

Method Detail Volume 7 Test Method

-If HPM does not error continue to voting the election and tally at the central count.

- If there are any system errors that cause the EMS ballot preparation applications to crash then 

verify the applications recover without any loss of data.

Scenario 2)

- Precinct 2/ballot style 2 has 201 contest and 201 candidates 

- Precinct 1/ballot style 1 has 176 candidates, Vote For 91

- Precinct 1/ballot style 1 has 19 parties

- Precinct 2/ballot style 2 has 3 Statistical Counters

Scenario 4)

Same as Scenario 2 except:

- Precinct 2/ballot style 2 has 71 contest and 71 candidates 

Volume: Maximum capacity is successfully processed without errors, as identified in the Test Variables

Systems capacity to process, store, and report data. 

- When importing over the allowed amount of data into the EDM using the Import Wizard

Stress System provides a response to overloading conditions. Exceeding/overloading the maximum 

allow number of ballot limits identified in the scope.

Performance There is no system degradation (Ballot format handling capability and Processing rates):

- When importing large amount of data into the EDM using the Import Wizard.

- The system does not slow down throughout the testing

Error Recovery Same as Volume 1 - Maximum Precincts and Ballot Styles 

Readiness Testing and Poll Verification Voting system is ready for the election: 

Same as Volume 1 - Maximum Precincts and Ballot Styles except:

- Run 1 precinct to validate the system is ready; confirm the test data is segregated from voting 

data, with no residual effect. Verify totals and audit logs.

Pre- vote: Opening the Polls Verification Precinct Count/ Paper based:

Same as Volume 1 - Maximum Precincts and Ballot Styles 

Voting: Ballot Activation and Casting 

Verifications

- Discrepancy 32(SysTest 453 orientation ballot errors) no orientation ballot errors while scanning 

the ballots

- Discrepancy 33(SysTest 454 internal rollers) internal rollers do not stop while scanning ballots

Scenario1) Election Day Voting - The VAT & DS200 are in Polling Place 1 Precincts 1 - 2.

- Mark 10% of 20 ballots per ballot style using the VAT and scan all of the ballots on the DS200)

- scanning in each of the 4 orientation.

- No errors are expected.

- If there are any system errors that cause the DS200 & the VAT to crash then verify the DS200 

and the VAT recover without any loss of data.

- Verify the counter on the DS200 and the VAT match the expect results.

Scenario 3) Same as Scenario 1 except:

- Scan 12 ballots for ballot style 2.

Scenario 2 and 4) Errors should prevent the test from reaching this point. If the test does get to 

this point:

- Load election(s) 

- No system failures that cause the DS200 and/or the VAT to crash

- If there are any system errors that cause the DS200 and the VAT to crash then the DS200 and 

the VAT shall recover without any loss of data.

Voting: Voting System Integrity, System 

Audit, Errors & Status Indicators

The system audit provides a time stamped, always available, report of normal/abnormal events 

found within the test. 

- Same as Volume 2 - Maximum Ballot Styles in a Single Precinct except: 

Post-vote: Closing the Polls Once the polls are closed the voting system:

Same as Volume 2 - Maximum Ballot Styles in a Single Precinct except:

- In Polling Place 1 the DS200 prints precincts 1 & 2 totals 

Post-vote: Central Count Paper Based: When loading results mix the input of results such that media is read out of precinct 

order and where possible mix the reading of DS200 and M650 results. Record the order at test 

execution.

Scenario 1) The central count voting system includes:

- Election identification

- Zero count report (to verify no votes are on the M650 prior to starting voting)

- 20 ballots per ballot style 

- No errors are expected.

- If there are any system errors that cause the M650 to crash then the M650 shall recover without 

any loss of data.

- Verify the counter on the DS200 and the VAT match the expect results.

Vote Consolidation: 

- ERM consolidated reports match the predicted votes.

- Verify no data was lost within the audit logs or results

Reports include:

- Printed reports of ballots counted by tabulator, with votes and undervotes

 - Printer Summary Report 

 - View and Print Precinct by Precinct Reports

Scenario 2 and 4) 
VSTL Certification # pending

Page 116 of 247 (V)2009-30Jun-001(A)

Method Detail Volume 7 Test Method

Errors should prevent the test from reaching this point. If the test does get to this point:

- Load election

- No system failures that cause the M650 or in the EMS ERM application to crash

- If there are any system errors that cause the M650 or in the EMS ERM application to crash then 

the M650 or in the EMS ERM application shall recover without any loss of data.

Expected Results are observed Review the test result against the expected result: 

Same as Volume 1 - Maximum Precincts and Ballot Styles 

Record observations and all input/outputs 

for each election;

All inputs, outputs, observations, deviations and any other information impacting the integrity of 

the test results will be recorded in the test case. 

Same as Volume 1 - Maximum Precincts and Ballot Styles 

7.4.2.8 Volume 8 Test Results

Testing was conducted on the system configuration identified in the PCA Configuration and Test Case. Specific 

software and firmware builds for each test execution are recorded in the PCA Configuration as identified in the 

individual test case document. 

Date Test Result Issues Opened Issues Closed Notes

3/20/09 Fail - v8s1 #104 Scenario 1

3/23/09 Accept v8s2 Scenario 2

5/14/09 Accepted v8s1 #104 Scenario 1Acceptance is based upon acceptance that 

this discrepancy is out of scope of Unity 3.2.0.0. Issue 

104 will be transferred to Unity 4.0 as discrepancy #1.

Only functional issues are identified. While documentation discrepancies may be encountered in testing they do not 

result in the rejection of a functional test. All issues are documented in Appendix E- Discrepancy Report.

Method Detail Volume 8 Test Method

Test Case Name Volume 8 - M650 maximum number of candidates/counter in an election.

Scope - identifies the type of test The scope is to test: 

Scenario 1) The M650 maximum allowed: number of candidates/counter within an election. 

To verify that errors are generated scenario 2: 

Scenario 2) Exceeding the M650 maximum: allowed number of candidates/counter within an 

election. 

Test Objective The objective is to validate the ability to process, store and report data using the maximum and 

exceeding the maximum allowed number of candidates/counter. To validate that the system 

generates errors during EMS ballot preparation (ballot preparation including: EDM, ESSIM & 

HPM) when exceeding the M650 maximum allowed number of candidates/counter. Validating the 

processing, storing and reporting shall occur without system degradation. If there are system 

errors that cause the system to crash the system shall recover without any loss of data.

Test Variables: General election

M650 set to Absentee 

10 Precincts on 1 M650

Each Precinct contains 75 contest

19 inch ballot (4 ovals per inch, 68 oval positions per column, 408 total positions)

General election

Absentee 

Scenario 1)

- 750 contest

- 3 candidates per contest

- 0 Statistical Counters 

Counters: 

2250 candidates (750 contest, 3 candidates no Write-ins)

750 undervotes

750 overvotes

Total counters = 3750

Scenario 2) Same as scenario 1 except:

- 751 contests 

Counters: 

2253 candidates (751 contest, 3 candidates no Write-ins)

751 undervotes

751 overvotes

Total counters = 3755

A description of the voting system type and 

the operational environment

The Unity 3.2.0.0 EMS Ballot Preparation includes:

Audit Manger (AM ), Election Data Manger (EDM), (ESSIM), hardware Program Manger (HPM)
VSTL Certification # pending

Page 117 of 247 (V)2009-30Jun-001(A)

Method Detail Volume 8 Test Method

The Unity 3.2.0.0 central count tabulator:

Model 650 (M650)

The Unity 3.2.0.0 central count tally

Election Reporting Manager (ERM)

VSS 2002 vol. 1 • Same as Volume 1 - Maximum Precincts and Ballot Styles 

VSS 2002 vol. 2 6.2.3 Volume (maximum number of M650 Candidate Counters)

A4.3.5 Volume (maximum and exceeding more than the maximum number of M650 Candidate 

Counters)

A4.3.5 Volume/Stress (Processing, storing and reporting data when overloading the number of 

M650 Candidate Counters)

A4.3.5 Performance/Recovery (Ballot format handling capability-graceful shut down (no crash) 

and recovery without loss of data)

A4.3.5 Performance/Recovery (Processing rates- shut down (no crash) and a graceful recovery 

without loss of data)

Hardware, Software voting system 

configuration and test location

The Unity 3.2 Voting System consists of the following: 

Audit Manger (AM), Election Data Manger (EDM), (ESSIM), hardware Program Manger (HPM), 

Model 650 (M650), Election Reporting Manager (ERM), 

All testing will be performing by iBeta LLC located at 3131 S. Vaughn Way, Aurora, CO 80014.

Pre-requisites and preparation for execution 

of the test case. 

Complete the prerequisites:

Test Method Validation: Technical review conducted by C. Coggins; Approved with the 

incorporation of review comments on 1/22/09 (validation of test method as defined in ISO/IEC 

17025 clause 5.4.5) 

- Document in the test case the percentage that the system limit exceeds the customer maximum. 

(System Limit * 100) /Customer Maximum =% System Limit) 

Condition of approval - iBeta validates the successful use of the Import Wizard to import large 

amounts of data into EDM. As configuration of the imported file can impact the success of the 

data importation, the import file structure must be validated as a prerequisite of all applicable test 

cases.

Import Wizard method tested and validated on 1/21/2009 by Stephanie Eaton.

- 7 Excel spreadsheets saved as "Tab Delimited". Tab Delimited documents containing election 

creating information will be imported into EDM using the Import Wizard option.

- Spreadsheet 1 - Precinct 10

- Spreadsheet 2 - District Type 10

- Spreadsheet 3 - Districts Names10

- Spreadsheet 4 - District Relations 10

- Spreadsheet 5 - Master Office 750

- Spreadsheet 6 - Office Relations 750

- Spreadsheet 7 - Candidates 2250

Getting Started Checks Check the voting system to :

Same as Volume 1 - Maximum Precincts and Ballot Styles 

Documentation of Test Data & Test 

Results

Check the voting system to :

Same as Volume 1 - Maximum Precincts and Ballot Styles 

Volume: Paper-based voting systems

Processing

Ballot Prep: General election

Scenario 1) 10 Precincts , each Precinct contains 75 contest

-An election database can be accurately being defined & formatted using the Import Wizard.

-Ballots (candidates & propositions) can be accurately defined & generated.

-19 inch ballot

-0 Statistical Counters 

- Create media for the M650 only - all precincts assigned to 1 M650

The election can be created with 3800 candidate counters with in an election. 

- If there are any system errors that cause the EMS ballot preparation applications to crash then 

verify the applications recover without any loss of data. 

Scenario 2)

Test execution of Scenario 2 stops at this point with errors generated prior to the creation of 

election media in ballot preparation)

- Check audit logs for critical status messages. Test stops unless system does not error and 

creates media) 

- If EDM does not error during the "Ballot Sets Merge" then the EDM reports must be reviewed to 

verify the election is set up. Continue to ESSIM and HPM. The system should display a critical 

status message prior to exiting the HPM.

- If there are any system errors that cause the EMS ballot preparation applications to crash then 

verify the applications recover without any loss of data.

Scenario 2)

Same as Scenario 1 except:

- 751 contests and 2253 candidates

Volume: Maximum capacity is successfully processed without errors. 
VSTL Certification # pending

Page 118 of 247 (V)2009-30Jun-001(A)

Method Detail Volume 8 Test Method

Systems capacity to process, store, and report data.

- When installing an election on the M650 containing over the allowed candidate counters, errors 

are generated.

Stress System provides a response to overloading conditions. Exceeding/overloading the maximum 

allow number of Candidate Counters in the M650.

Performance No system degradation (Ballot format handling capability and Processing rates) is observed: 

- When importing large amount of data into the EDM using the Import Wizard.

-When importing 3750 candidate counters 

-When importing 3755 candidate counters

- The system will not slow down throughout the testing

Error Recovery Precinct Count/ Paper based:

Same as Volume 1 - Maximum Precincts and Ballot Styles 

Readiness Testing and Poll Verification See below - Post Vote: Central Count

Pre- vote: Opening the Polls Verification Not Applicable (M650 is not located at the polls)

Voting: Ballot Activation and Casting 

Verifications

Not Applicable (M650 is not located at the polls)

Voting: Voting System Integrity, System 

Audit, Errors & Status Indicators

Not Applicable (M650 is not located at the polls)

Post-vote: Closing the Polls Not Applicable (M650 is not located at the polls)

Post-vote: Central Count Paper Based: 

Scenario 1) 

- Load election with 3750 Candidate Counters

- Hand mark and scan ballots through the M650

- Verify the counter on the M650 match the expect results.

- If there are any system errors that cause the M650 to shut down (crash) then the M650 shall 

recover without any loss of data.

Vote Consolidation: 

- ERM consolidated reports match the predicted votes.

- Verify no data was lost within the audit logs or results

Reports include:

- Printed reports of ballots counted by tabulator, with votes and undervotes

 - Printer Summary Report 

 - View and Print Precinct by Precinct Reports for Precincts 1 - 10

: Scenario 2) 

Errors should prevent the test from reaching this point. If the test does get to this point:

- Load election

- No system failures that cause the M650 or in the EMS ERM application to crash

- If there are any system errors that cause the M650 or in the EMS ERM application to crash then 

the M650 or in the EMS ERM application shall recover without any loss of data.

Expected Results are observed Review the test result against the expected result: 

Same as Volume 1 - Maximum Precincts and Ballot Styles 

Record observations and all input/outputs 

for each election;

All inputs, outputs, observations, deviations and any other information impacting the integrity of 

the test results will be recorded in the test case. 

Same as Volume 1 - Maximum Precincts and Ballot Styles 

7.4.2.9 Volume 9 Test Results

Testing was conducted on the system configuration identified in the PCA Configuration and Test Case. Specific 

software and firmware builds for each test execution are recorded in the PCA Configuration as identified in the 

individual test case document. 

Date Test Result Issues Opened Issues Closed Notes

4/24/09 Reject v9s1 144, 145 Scenario 1Handled volume without issues; Misread in a 

vote in a single contest

5/12/09 Accept v9s2 Scenario 2

6/19/09 Accept #144 Acceptance of Data Accuracy & state testing 

6/26/09 Accept #121, 145, 151 Regression for 145 absence of image, 121 -DS200 

audit log, 151 ERM log an error for loading a corrupted 

file

Only functional issues are identified. While documentation discrepancies may be encountered in testing they do not 

result in the rejection of a functional test. All issues are documented in Appendix E- Discrepancy Report.

Method Detail Volume 9 Test Method

Test Case Name Volume 9 - ERM maximum number of candidates/counter in an election.

Scope - identifies the type of test The scope is to test: 

Scenario 1) The ERM maximum allowed: number of ERM candidates/counter within an election, 
VSTL Certification # pending

Page 119 of 247 (V)2009-30Jun-001(A)

Method Detail Volume 9 Test Method

maximum allowed number of Precincts and the maximum number of Precincts in a single Polling 

Place in Election Day mode.

To verify that errors are generated in scenario 2:

Scenario 2) Exceeding the maximum number of ERM candidates/counter within an election, 

maximum allowed number of Precincts and the maximum number of Precincts in a single Polling 

Place in Election Day mode.

Test Objective The objective is to validate the ability to process, store and report data using the maximum and 

exceeding the maximum allowed number of Precincts, ERM candidates/counter and Election Day 

Precincts within a single Polling Place. To validate that the system generates errors during EMS 

ballot preparation (ballot preparation including: EDM, ESSIM & HPM) when exceeding the 

Precincts, ERM maximum allowed number of candidates/counter and Election Day Precincts 

within a single Polling Place. Validating the processing, storing and reporting shall occur without 

system degradation. If there are system errors that cause the system to crash the system shall 

recover without any loss of data.

Test Variables: General election -

Election Day

10 precincts to a polling place (max limit on polling places for election day) 

19 inch ballot (4 ovals per inch, 68 oval positions per column, 408 total positions)

290 polling places

Scenario 1)

- 16 Districts Types 

- 129 District Names

- 2900 Precincts (Volume 1"Precincts" spreadsheet) 

- 3670 contest (added 170 contest)

- 3 candidates (removed W/Is)

-1 Statistical Counters (added 1 counter)

Scenario 1 counters: 

-9990 candidates (new limit identified; removed 3500 W/Is, then removed 510 candidates)

-3670 (undervote counters) (added 170)

-3670 (overvote counters) (added 170) 

Spreadsheet 1 - Precincts 2900

Spreadsheet 2 - District Types 16 (including countywide)

Spreadsheet 3 - District Names 129 

Spreadsheet 4 - District Relations 5930

Spreadsheet 5 - Master Office 3670

Spreadsheet 6 - Office Relations 3670

Spreadsheet 7 - Candidates 9990

Spreadsheet 8 - Master Polling Place 290

Spreadsheet 9 - Poll Relations 290

Scenario 2) Same as scenario 1 except:

- 3671 contests 

-9993 candidates 

-3671 (undervote counters)

-3671 (overvote counters)

Polling Place 290 has 11 Precincts (instead of 10) 

A description of the voting system type and 

the operational environment

The Unity 3.2.0.0 EMS Ballot Preparation includes:

Audit Manger (AM ), Election Data Manger (EDM), (ESSIM), Hardware Program Manger (HPM), 

AutoMARK Information (AIMS)

The Unity 3.2.0.0 marking device: 1 @ Voter Terminal(VAT)

The Unity 3.2.0.0 precinct count includes: 2 @ DS200

The Unity 3.2.0.0 central count tally: Election Reporting Manager (ERM)

VSS 2002 vol. 1 Same as Volume 1 - Maximum Precincts and Ballot Styles 

VSS 2002 vol. 2 6.2.3 Volume (maximum number of ERM Candidate Counters)

A4.3.5 Volume (maximum and exceeding more than the maximum number of ERM Candidate 

Counters)

A4.3.5 Volume/Stress (Processing, storing and reporting data when overloading the number of 

ERM Candidate Counters)

A4.3.5 Recovery (EMS capabilities to gracefully shut down (no crash) and recovery without loss 

of data)

A4.3.5 Performance/Recovery (Processing rates- ballot formatting handling capabilities (no 

crash)and a graceful recovery without loss of data)

Hardware, Software voting system 

configuration and test location

The Unity 3.2 Voting System consist of the following:

Audit Manger (AM ), Election Data Manger (EDM), (ESSIM), hardware Program Manger (HPM), 

DS200, Election Reporting Manager (ERM), AutoMARK Information (AIMS), Voter Terminal(VAT)

All testing was perform by iBeta located at 3131 S. Vaughn Way, Aurora, CO 80014.

Pre-requisites and preparation for execution 

of the test case. 

Complete the prerequisites:

Test Method Validation: Technical review conducted by C. Coggins; Approved 2/4/09 ( validation 

of test method as defined in ISO/IEC 17025 clause 5.4.5) Reconfiguration of data import 
VSTL Certification # pending

Page 120 of 247 (V)2009-30Jun-001(A)

Method Detail Volume 9 Test Method

reviewed by J Garcia 4/28/09

- Document in the test case the percentage that the system limit exceeds the customer maximum. 

(System Limit * 100) /Customer Maximum =% System Limit) 

- 9 Excel spreadsheets saved as "Tab Delimited". Tab Delimited documents containing election 

creating information was imported into EDM using the Import Wizard option.

Spreadsheet 1 - Precincts 2900

Spreadsheet 2 - District Types 16 (including countywide)

Spreadsheet 3 - District Names 129 

Spreadsheet 4 - District Relations 5930

Spreadsheet 5 - Master Office 3670

Spreadsheet 6 - Office Relations 3670

Spreadsheet 7 - Candidates 9990

Spreadsheet 8 - Master Polling Place 290

Spreadsheet 9 - Poll Relations 290

Getting Started Checks Check the voting system to :

Same as Volume 1 - Maximum Precincts and Ballot Styles 

Documentation of Test Data & Test 

Results

Test Data:

Same as Volume 1 - Maximum Precincts and Ballot Styles 

Volume: Paper-based voting systems

Processing

Ballot Prep: Scenario 1) 

- General election

-An election database was accurately defined & formatted using the Import Wizard.

-Ballots (candidates & propositions) were accurately defined & generated.

-19 inch ballot

-290 Polling Places 

-10 precincts to a Polling Place 

- 1 Statistical Counters 

- A total of 129 District Names within 16 District Types creating 1639 ballot styles.

District Type / District Name:

P1 / P1, P2 / P2, P3 / P3, P4 / P4, P5 / P5, P6 / P6, P7 / P7, P8 / P8, P9 / P9, P10 / P10, BA / BA 

1-20, BB / BB 1 - 22, BC / BC 1 - 4, CA / CA 1 - 35, CB / CB 1 - 37, County wide / countywide.

- Each unique ballot style was configured by incorporating 3670 different contests. Each ballot 

contains a minimum of 38 contests to a maximum of 78 contests. The election will contain 9990 

unique candidates with either 2 or 3 candidates to a single contest. 

- The election was created with 21000 candidate counters, 2900 precincts, and 10 precincts to a 

single polling place.

- Check EDM reports for election set up.

- If there are any system errors that cause the EMS ballot preparation applications to crash then 

verify the applications recover without any loss of data

Scenario 2) 

Test execution of Scenario 2 did not stop at this point with errors generated prior to the creation of 

election media in ballot preparation.

Test continued because system did not error. Media was created. 

- EDM did not error during the "Ballot Sets Merge"; the EDM reports were reviewed to verify 

Scenario 2 had 3671 contest, 2901 precincts and 11 Precincts assigned to a single early voting 

Polling Place. Continued to ESSIM and HPM. The system did not display a critical status 

message prior to exiting the HPM.

- There were no system errors that caused the EMS ballot preparation applications to crash.

Scenario 2) Same as Scenario 1 except:

- 3671 contests

- 9993 candidates

- 2901 Precincts

- 11 Precincts in Polling Place 290

Volume: Maximum capacity was successfully processed without errors. 

Systems capacity to process, store, and report data.

- When importing over the allowed candidate counters into the ERM errors are generated.

Stress System provides a response to overloading conditions. Exceeding the maximum allow number of 

Candidate Counters in the ERM.

Performance No system degradation (Ballot format handling capability and Processing rates) was observed: 

- When importing a large amount of data into the EDM using the Import Wizard.

- When importing 21000 candidate counters (9990 candidates, 3670 contests) 

- When importing 21006 candidate counters (9993 candidates, 3671 contests) 

- The system did not slow down throughout the testing

Error Recovery Same as Volume 1 - Maximum Precincts and Ballot Styles 

Readiness Testing and Poll Verification Voting system was ready for the election: 

Same as Volume 1 - Maximum Precincts and Ballot Styles 

Pre- vote: Opening the Polls Verification Precinct Count/ Paper based:

Same as Volume 1 - Maximum Precincts and Ballot Styles 

Voting: Ballot Activation and Casting 

Verifications

Scenario 1) The DS200 was programmed for Election Day Voting.

-Of the 16 District Types vote District Types./District Names: 

P1, BA / BA 1-20, BB / BB 1 - 22, BC / BC 1 - 4, CA / CA 1 - 35, 
VSTL Certification # pending

Page 121 of 247 (V)2009-30Jun-001(A)

Method Detail Volume 9 Test Method

CB / CB 1 - 37, County wide / countywide. A total of 290 ballots were tested in 29 Polling Places, 

Polling Places 1 (Precincts 1 - 10), 38 thru 50 (Precincts 371 - 500), 145 thru 158 (Precincts 1441 

- 1580), 290 (Precincts 2891 - 2900), 10 ballots per Polling Place- one ballot per Precinct.

- Polling Place 290 with Precincts2891 - 2900 were marked by the VAT and then scanned into the 

DS200.

- If there are any system errors that cause the DS200 or the VAT to shut down (crash) then the 

DS200and the VAT shall recover without any loss of data.

Scenario 2) No errors prevented the test from reaching this point: 

- The election(s) loaded

- No system failures that caused the DS200 and/or the VAT to crash

Voting: Voting System Integrity, System 

Audit, Errors & Status Indicators

The system audit provides a time stamped, always available, report of normal/abnormal events 

found within the test. 

- Same as Volume 2 - Maximum Ballot Styles in a Single Precinct

Post-vote: Closing the Polls Once the polls are closed the voting system

- Prints reports of ballots counted by tabulator

- Reported votes match the predicted votes from tabulator with votes and undervotes.

- Polling Places will print the precincts totals (Election Day voting ends)

Post-vote: Central Count Vote Consolidation: 

Scenario 1) 

- M650 Not Applicable (M650 limit was 3800 and was tested in Volume 8) 

- ERM does not crash with 21000 candidate counters and 10 precincts within an Election.

- ERM consolidated reports match the predicted votes.

Vote Consolidation: 

ERM consolidated reports match the predicted votes from the polling places 

Reports include: 

- Printed reports of ballots counted by tabulator, with votes and undervotes

 - Printer Summary Report 

If there are any system errors that cause the ERM application to crash then the ERM application 

shall recover without any loss of data.

Scenario 2) No errors prevented the test from reaching this point:

- The election loaded into ERM

- No system failures caused the EMS ERM application to crash

Expected Results are observed Review the test result against the expected result: 

Same as Volume 1 - Maximum Precincts and Ballot Styles 

Record observations and all input/outputs 

for each election;

All inputs, outputs, observations, deviations and any other information impacting the integrity of 

the test results will be recorded in the test case. 

Same as Volume 1 - Maximum Precincts and Ballot Styles 

7.4.2.10 Volume 10 Test Results

Testing was conducted on the system configuration identified in the PCA Configuration and Test Case. Specific 

software and firmware builds for each test execution are recorded in the PCA Configuration as identified in the 

individual test case document.

Date Test Result Issues Opened Issues Closed Notes

3/18/09 Reject 99 Requesting to print > 5000 precincts generates an error, 

shut down ERM and does not write to the audit log

5/13/09 Accept 99 Accepted because operation is consistent with Vol.1: 

4.2.3.e "the design must explicitly protect all recorded 

votes and audit log information and must implement 

formal exception handlers provided by the language". 

RMCobol Error is using the handler provided by the 

language and it protected vote and audit log data.

Only functional issues are identified. While documentation discrepancies may be encountered in testing they do not 

result in the rejection of a functional test. No issues are documented in Appendix E- Discrepancy Report.

Method Detail Volume 10 Test Method

Test Case Name Volume 10 - maximum number of Ballot Styles in an election.

Scope - identifies the type of test The scope is to test: 

Scenario 1) The HPM maximum allowed number of Ballot Styles within an election. 

To verify that errors are generated when: 

Scenario 2) Exceeding the HPM maximum allowed number of Ballot Style within an election. 
VSTL Certification # pending

Page 122 of 247 (V)2009-30Jun-001(A)

Method Detail Volume 10 Test Method

Test Objective The objective is to validate the ability to process, store and report data using the maximum and 

exceeding the maximum number of Ballot Styles allowed in an Election. To validate that the

system generates errors during EMS ballot preparation (ballot preparation including: EDM, 

ESSIM & HPM) when exceeding the maximum allowed number of Ballot Styles within an election. 

Validating the processing, storing and reporting shall occur without system degradation. If there 

are system errors that cause the system to crash the system shall recover without any loss of 

data.

Test Variables: Primary Election - Closed by Precinct Style

Election Day

100 Polling Places

10 Precincts to a polling Place

19 inch ballot (4 ovals per inch, 68 oval positions per column, 408 total positions)

5 Parties

Scenario 1) 

- 1000 Precincts 

- 10 contest

- 2 contest per precinct 

- 2 district types each with 5 district names per 200 precincts

- 100 candidates (10 contest *5 party*2 )

- 2 district types each with 5 district names per 200 precincts

- 5005 ballot styles (5 Parties each with a separate style*1000 precincts)

Scenario 2) Exceed the HPM maximum number of ballot styles

- 1001 Precincts 

- 11 contest

- 1 contest to a precinct

- 5 candidates (1 per contest by party)

- 5005 ballot styles (5 Parties each with a separate style)

Wizard option

Spreadsheet 1 - Precinct 1001

Spreadsheet 2 - Districts Names

Spreadsheet 3 - District Relations

Spreadsheet 4 - Master Office primary 11 

Spreadsheet 5 - Office Relations primary 11

Spreadsheet 6 - Candidates w/party

A description of the voting system type and 

the operational environment

Same as Volume 1 - Maximum Precincts and Ballot Styles 

VSS 2002 vol. 1 Same as Volume 1 - Maximum Precincts and Ballot Styles 

VSS 2002 vol. 2 6.2.3 Volume (maximum number of ballot styles in an election)

A4.3.5 Volume/Stress (Processing, storing and reporting data when overloading the number of 

ballot styles in an election)

A4.3.5 Performance/Recovery (Ballot format handling capability-graceful shut down and recovery 

without loss of data) if the number of ballot styles/precincts was exceeded

A4.3.5 Performance/Recovery (Processing rates-graceful shut down and recovery without loss of 

data)

Hardware, Software voting system 

configuration and test location

Same as Volume 1 - Maximum Precincts and Ballot Styles 

Pre-requisites and preparation for execution 

of the test case. 

Complete the prerequisites:

Test Method Validation: Technical review conducted by C. Coggins; Approved 2/4/09 ( 

validation of test method as defined in ISO/IEC 17025 clause 5.4.5) 

Condition of approval - iBeta validated the successful use of the Import Wizard to import large 

amounts of data into EDM. As configuration of the imported file can impact the success of the 

data importation, the import file structure must be validated as a prerequisite of all applicable test 

cases.

Import Wizard method tested and validated: 2/2/09.

- Document in the test case the percentage that the system limit exceeds the customer maximum. 

(System Limit * 100) /Customer Maximum =% System Limit) 

- Excel spreadsheets saved as "Tab Delimited". Tab Delimited documents containing election 

creating information were imported into EDM using the Import Wizard option.

Manually input District Type as District Type A and B

Wizard option

Spreadsheet 1 - Precinct 1000 

Spreadsheet 2 - District Types 2 plus countywide

Spreadsheet 3 - Districts Names 10

Spreadsheet 4 - District Relations

Spreadsheet 5 - Party

Spreadsheet 6 - Master Office primary 10

Spreadsheet 7 - Office Relations primary 

Spreadsheet 8 - Candidates w/party 100
VSTL Certification # pending

Page 123 of 247 (V)2009-30Jun-001(A)

Method Detail Volume 10 Test Method

Spreadsheet 9 - Polling Places 100

Spreadsheet 10 - Poll Relations

Getting Started Checks Check the voting system to :

Same as Volume 1 - Maximum Precincts and Ballot Styles 

Documentation of Test Data & Test 

Results

Test Data:

Same as Volume 1 - Maximum Precincts and Ballot Styles 

Volume: Paper-based voting systems

Processing

Ballot Prep: 

Primary Election - Closed by Precinct Style

Election Day

100 Polling Places

10 Precincts to a polling Place

5 Parties

Scenario 1) 

- 1000 Precincts 

- 10 contest

- 2 contest per precinct 

- 2 district types each with 5 district names per 200 precincts

- 100 candidates (10 contest *5 party*2 )

- 2 district types each with 5 district names per 200 precincts

- 5005 ballot styles (5 Parties each with a separate style*1000 precincts)

-An election database was accurately defined & formatted using the Import Wizard.

-Ballots (candidates & propositions) were accurately defined & generated.

-19 inch ballot

- Check EDM reports for election set up.

- If there are any system errors that cause the EMS ballot preparation applications to crash then 

verify the applications recover without any loss of data.

Scenario 2) 

Test execution of Scenario 2 did not stop at this point. No errors were generated prior to the 

creation of election media in ballot preparation.

- Test continued because EDM did not error during the "Ballot Sets Merge". EDM reports were 

reviewed to verify Scenario 2 has been set up correctly. Continued to ESSIM and HPM. 

- The system did not error and cause the EMS ballot preparation applications to crash

Scenario 2) Same as scenario 1 except for:

- 1001 Precincts 

- 11 contest

- 1 contest to a precinct

- 5 candidates (1 per contest by party)

- 5005 ballot styles (5 Parties each with a separate style)

Wizard option

Spreadsheet 1 - Precinct 1001

Spreadsheet 2 - Districts Names

Spreadsheet 3 - District Relations

Spreadsheet 4 - Master Office primary 11 

Spreadsheet 5 - Office Relations primary 11

Spreadsheet 6 - Candidates w/party

Volume: Maximum capacity was successfully processed without errors. 

HPMs maximum number of ballot styles.

Systems capacity to process, store, and report data: 

- HPM successfully processed maximum plus one. No errors were generated. 

Stress System successfully processed the overloading conditions of exceeding the maximum allow 

number of 5000 ballot styles by one.

Performance No system degradation (Ballot format handling capability and Processing rates) was observed: 

- When importing large amount of data into the EDM using the Import Wizard.

- When installing an election with over the maximum number of ballot styles for an election. 

- The system did not slow down as more and more data was added

Error Recovery Same as Volume 1 - Maximum Precincts and Ballot Styles 

Readiness Testing and Poll Verification Voting system was ready for the election: 

Same as Volume 1 - Maximum Precincts and Ballot Styles 

Pre- vote: Opening the Polls Verification Precinct Count/ Paper based:

Same as Volume 1 - Maximum Precincts and Ballot Styles

Voting: Ballot Activation and Casting 

Verifications

Scenario 1) The DS200 was programmed for Election Day Voting.

- All Polling Places were activated, with Polling Place 1, 21, 41, 61, 81 voted. 

- The VAT and DS200 were in Polling Place 1: Precincts 1 - 200.

- Each ballot was marked by the VAT and then scanned into the DS200.

- Each precinct contained 2 contests with 2 candidates.

- A total of 100 ballots were tested in Precincts 1-1000. Precincts Voted: 1-5, 201 - 205, 401 -

405, 601 - 605, 801- 805. Voting Contests 1 - 10.
VSTL Certification # pending

Page 124 of 247 (V)2009-30Jun-001(A)

Method Detail Volume 10 Test Method

- S1: 20 ballots per polling place for a total of 5 Polling Places 

- There were no system errors that cause the DS200 or the VAT to shut down (crash)

Scenario 2) No errors prevented the test from reaching this point

- The election(s) loaded

- Scenario 2: A total of 110 ballots were tested in Precincts 1-1001. Precincts Voted: 1-5, 201 -

205, 401 - 405, 601 - 605, 801- 805, 1001-1005. Voting Contests 1 - 11.

- 20 ballots per polling place for a total of 5 Polling Places, 10 ballots for Poll Place 6

- No system failures caused the DS200 and/or the VAT to crash.

Voting: Voting System Integrity, System 

Audit, Errors & Status Indicators

The system audit provided a time stamped, always available, report of normal/abnormal events 

found within the test. 

- Same as Same as Volume 1 - Maximum Precincts Limitations and ballot styles for paper 

Post-vote: Closing the Polls Once the polls were closed the voting system

- Printed reports of ballots counted by tabulator

- Reported votes match the predicted votes from the tabulator with votes and undervotes.

- In Polling Place 1 the DS200 Prints precincts 1 - 10 totals (Election Day voting ends)

- In Polling Place 2 - precincts 11 -100 no reports were run (all voting was executed using 

Precincts 1 - 10) 

Post-vote: Central Count Paper Based: When loading results the input of results were mixed such that media was read out 

of precinct order and the reading of DS200 and M650 results were mixed. (Testing of the 

reordering of media was based upon an EAC report )

Scenario 1) 

- Election identification

- Zero count report (to verify no votes are on the M650 prior to starting voting) 

- 100 ballots were tested

- VAT -Generated the ballots for 10 different ballot styles within the deck.

- Precincts Voted on VAT: 1-5, 201 - 205, 401 - 405, 601 - 605, 801- 805.

 (S2 - also included 1001 - 1005)

- The M650 with a 1000 precinct and 5000 ballot styles did not error.

Vote Consolidation: 

ERM consolidated reports matched the votes from the polling places and the predicted results 

from the test case

Reports included: 

- Printed reports of ballots counted by tabulator, with votes and undervotes

 - Printing of a Summary Report (containing all precincts)

 - Viewed and printed Precinct by Precinct Reports

- There were no errors that cause the M650 or the ERM application to crash.

No errors prevented the test from reaching this point. 

- The election loaded without error. 

Expected Results are observed Review the test result against the expected result: 

Same as Volume 1 - Maximum Precincts Limitations and ballot styles for paper 

Record observations and all input/outputs 

for each election;

All inputs, outputs, observations, deviations and any other information impacting the integrity of 

the test results will be recorded in the test case. 

Same as Volume 1 - Maximum Precincts Limitations and ballot styles for paper 

7.4.3 FCA Security Review and Testing

During the initial Security Document Review conducted 1/28/09 through 2/3/09 test criteria was identified. Security 

testing or reviews (source code or document) were broken down into unique security tests/reviews or test/reviews

already addressed in standard testing and document or source code reviews. Unique security tests, source code, 

or document reviews are identified in 7.4.3. Test results for the standard tests, source code, or document reviews 

are identified in those sections. Security Testing was performed conducted on the system configuration identified in 

the PCA Configuration and section 3.

Date Test 

Result

Issues 

Opened

Issues 

Closed

Notes

1/28/09 -

2/3/09

Reject 55 

through 

84

Unity 3.2.0.0 Security Document Review to the requirements of v.1: 2.2.1 a to -g, 

2.2.2.1 d & e, 2.2.3 a-c, 2.2.4.1 e, f, g, i & j, 2.2.4.2, 2.2.5.3 , 4.5, and

section 6

3/11/09 -

3/13/09

Accept 55, 56, 

57, 58, 59

VAT Security Document Review verified

Security provisions compatible with the procedure and administrative tasks involved 

in equipment preparation testing and operation;

Mandatory administrative procedures for security;

Documentation for restoring VAT in case of error or failure;

Documentation for read/write and data transfer quality validation;

Documentation for effective password management;

3/12/09 - Accept 61, 62, 63, EDM Security Document Review verified: 
VSTL Certification # pending

Page 125 of 247 (V)2009-30Jun-001(A)

Date Test 

Result

Issues 

Opened

Issues 

Closed

Notes

4/16/09 64, 65, 66, 

67, 68, 69,

70, 71, 72, 

73, 75, 79,

83, 84

Documentation of access controls and prevention of unauthorized access

Removal of detail that is not applicable to the current configuration

Addresses detection/recording of events, self test and diagnostic software

Detail for jurisdiction security planning, procedures and administrative tasks is 

provided

Login security and physical protection of ballots is mandatory

A description of encryption is provided

Non-use of telephony, network and LAN is documented

Security protocols are documented

Incorporation of Window access controls is documented

3/13/09 Accept 60, 81, DS200 Security Document Review verified

Documentation of individual access to the specific tabulator functions

System submitted for certification does not transmit results via network

3/13/09 Accept 70 ESSIM Security Document Review verified: 

Login security is mandatory

3/13/09 -

4//14/09

Accept 70, 74, 77 HPM Security Document Review verified: 

Audit log activity is documented 

Access control policies are documented

Login security is mandatory

No compilers are present after Election Day testing

3/13/09 -

5/15/09

Accept 76, 77, 80 ERM Security Document Review verified: 

Mandatory administrative procedures are documented 

No compilers are present after Election Day testing

Error detection and correction methods are performed by RMCOBOL

4/2/09 Reject 122, 

123, 124

Unity & AIMS – Windows Configuration Test Steps

No audit logging of successful or unsuccessful logins

No audit logging in Audit Manager or Windows of copying to an external file

Non-administrative user cannot install executable software

4/14/09 

–

4/29/09

Accept 29, 61, 73, 

78, 82, 

M650 Security Testing verified: 

Election definition cannot be changed or overwritten after installation;

Locks cannot be by passed without breaking

Zero totals clears M650 but not totals saved to disk; audit log records zeroing of 

totals;

Can reload zeroed totals with totals saved to disk;

Results cannot be saved to a FAT32 zip disk;

Error is generated and logged if disk is removed during vote processing;

A restart after a system interrupt generates the readiness report with an log notation

and no vote data is lost;

Invalid ballots are rejected (different election)

Copies of ballots are rejected

Installed firmware matches readiness report;

Audit logging cannot be stopped or changed; and

Installed virus was ignored; · All events are detected and recorded in the log;

When prior votes exist at the start of an election a message prompts the user to 

keep the exiting election definition or load the new election from the zip disk. 

Pressing stop keeps the existing election definition and displays "Totals present in 

database, Zero totals or press start to continue". Pressing start wipes the database: 

loads the new election definition and logs clearing the database counter;

M650 Unique Source Coder Review verified: 

Audit log is written to zip disk

No computer generated password key used to encrypt the database or work space 

file;

Memory is zeroed out prior to election counting. If the memory is not zeroed a 

warning displays and it is recorded in the audit log.

M650 Security Document Review, verified documentation of: 

Security kernels;

Unauthorized access to access control capabilities;

Security procedures shall address physical threats and the corresponding means to 

defeat them;

Measures for physical and procedural control;

Every device is to be retested to validate the ROM.

No software shall be permanently installed or resident in the system unless the 

system documentation states that the jurisdiction must provide physical security at 

all times; device-controller software is resident permanently as firmware, it has been 

shown to be inaccessible to activation or control by means other than the authorized 

initiation and execution of the vote-counting program, and its associated exception 

handlers;

Election-specific programming (if resident as firmware) resides on a component 
VSTL Certification # pending

Page 126 of 247 (V)2009-30Jun-001(A)

Date Test 

Result

Issues 

Opened

Issues 

Closed

Notes

other than the operating system;

The zero report is part of the official audit record

4/28/09-

4/29/09

Reject 131, 132 M650 Security Testing:

Loaded an externally modified election definition on the M650 - Modified the 

election definition by replacing 0007 to 0999 in .PRF file and replacing 0010>9999 

(ballot counter -Totals) in RPT file, The M650 accepted the election definition

The zip disk with election results was externally modified. The modifications did not 

generate errors and modified results load into ERM:

o CRC or checksum errors were not generated-On the zip media the .EC file 

PRECINCT was replaced with "pppppppp", modifying results loaded to ERM 

without generating CRC or checksum validation errors; 

o In the .log file some bytes were replaced with zeros, this modified results 

loaded to ERM, No validation errors occurred.

o In the .pr file replaced 0001 with 999999, this modified results loaded to ERM, 

No validation error occurred.

M650 Unique Source Code Review 

There's no CRC validation for loading election definition (.RPT, .PRE, .OFC)

4/21/09 -

4/24/09

Accept DS200 Security Testing verified;

There is no access to administration menu when polls open;

There are 3 locks to prevent unauthorized access. Bypassing these locks without 

breaking is not possible;

Update firmware option is located under admin menu. To access admin menu 

requires a password;

Administrator password can be changed in HPM, password is not hardcoded in 

DS200;

Passwords (admin and reopen polls passwords) are retrieved from the election 

definition;

The polls open option is not available in the administrative menu when the election 

definition is not installed;

When polls are open for voting the Zero Totals report confirms the vote count is 

zero;

An invalid ballot (from different election) is rejected with an error "ballot too long”. 

There is a entry in the audit log; 

Attempting to remove the USB flash drive during ballot scanning the DS200 halts 

operating and resumes normal operation after reinsertion. An entry in the audit log 

reports the system halt;

DS200 does not respond to insertion of a ballot before the polls are open;

Scanned ballots are retained after a power discharge;

Invalid and corrupted election definitions are not loaded on the DS200;

Audit records can‟t be deleted nor recording stopped

Audit log entries include the date and time;

Results cannot be accessed or printed when the polls are open 

Removing the report paper halts operation (critical operation)

A virus inserted on the DS200 was ignored;

Voter‟s have no access to vote counting/reporting;

If votes are on the system when attempting to open polls, a warning "Election test 

counts exists on media, clear test counts to continue” is generated

Poles cannot open until the counter is cleared;

Clearing the counter is written to the audit log; 

DS200 Security Testing verified in Regression TC;

System performs diagnostic readiness test;

A password is required to access the Admin menu

DS200 Security Document Review verified: 

Documentation of proper and mandatory administrative procedures security in the 

polls & central count;

Access control measures;

Permanent bootstrap is inaccessible to unauthorized activation;

Physical protections of firmware

DS200 Unique Security Source Code Review verified:

Election definition on the USB drive validates the CRC of the election media header, 

election definition, counter block, system log and audit log sections

No computer generated passwords;

Previously loaded election definition and counter blocks are cleared from memory;

If counts exist displays "Election test counts exists on media clear test counts to 

continue".

Logs "Counters Cleared" when counters cleared;

Zero totals report type appears in the source code;

Counter & CRC update with each cast vote;

4/23/09 Reject 121 DS200 Security Testing
VSTL Certification # pending

Page 127 of 247 (V)2009-30Jun-001(A)

Date Test 

Result

Issues 

Opened

Issues 

Closed

Notes

The audit log does not record a successful or failed administrative login 

4/23/09 

to 5/3/09

Accept EDM Security Testing verified

Windows configuration was BIOS protected

Recovery from unexpected interruption and resume normal operation without loss of 

data'

Detect and record events, including error conditions

Audit logging of normal and abnormal events

EDM will not operate if a critical system is (audit logging) is not available

No public telephony or networking is being used

4/23/09 

to 5/3/09

Reject 123, 124, 

130

EDM Security Testing 

No audit logging in Audit Manager or Windows of copying to an external file

No audit logging of successful or unsuccessful logins

4/23/09 

to 5/3/09

Accept 146 (informationa

l)

ESSIM Security Testing verified

A valid user password is required

Attempts to modify ballot layout files (.ais) are detected

If a critical process is lost (system audit) error message are generated that prevent 

processing (#146)

ESSIM Security Document Review verified

Software, hardware access controls , segregation of duties and roles, supervisory 

privileges and password management are documented

No public telephony or use of a network

4/30/09 

to 5/1/09

Accept AIMS – Security Testing verified: 

No connection to a network;

Password protection of the database and system administration;

Entry of a blank password is not accepted; 

Failed & successful logins are recorded in the audit log;

An election audit log cannot be deleted;

An election can be backed up with data; deleted and restored from the backup;

Recovery from an abnormal shut down without loss of data;

VAT Security Testing verified 

Failed & successful logins are recorded in the audit log;

Audit log records: Abnormal shut down &recovery; export & data validation; 

Ballots in various orientations can be marked; generation is recorded in the audit 

log;

Inserted a torn (top of the ballot) ballot, VAT machine accepted that ballot, marked 

correctly There is an entry in the log.

Externally modified or internally adding a subdirectory in election data folder 

generates the error “Files are tampered with or wrong access code”;

Insertion of a non-valid election ballot generates “Ballot has not been recognized” 

error and audit log entry; 

Access to the compact flash card is prevented by a lock on the VAT machine;

Inserting a pre-marked in the VAT displays the summary view of the marked 

selections;

Ballot selections can be printed after a voting session interrupted by an abnormal 

shut down (shut down occurs post selection and prior to printing);

After printing votes on a ballot the record is erased;

Test mode displays the VAT software version;

VAT Security Testing verified in Regression TC:

Pre-vote, vote and post vote access controls;

Monitoring and audit logging of VAT errors; 

AIMS & VAT Unique Security Source Code Review verified:

Securing the data with a hash code resented to the AutoMark on the compact flash 

card; 

VAT is validating the hash code upon insertion.

After voting, viewing the summary, exit of the voting session and ejection of the 

ballot (without printing) the record is erased from the memory;

After printing votes on a ballot the record is erased;

No computer generated passwords effecting AIMS or the election process; 

AIMS & VAT Security Document Review verified: 

Supervisory access privileges & segregation of duties;

4/23/09-

5/6/09

Accept HPM Security Testing verified: 

System recovers from an abnormal shut down without loss of data;

Error warning if media is not available during writing; recovers to prior state before 

error;

Operation is halted when audit is inoperable;

Invalid logins are recorded in the Windows Security Event Log;

No telecommunication functionality;

4/23/09- Reject #126 HPM Security Testing: 
VSTL Certification # pending

Page 128 of 247 (V)2009-30Jun-001(A)

Date Test 

Result

Issues 

Opened

Issues 

Closed

Notes

5/6/09 Successful and unsuccessful logins are not recorded in the audit log

Audit log can be modified

5/1/09 to 

5/8/09

Reject #154, 158 Unity– Windows Configuration Test Steps: 

Procedures detail user rights

Procedures detail all required materials 

5/6/09 Accept ERM Security Testing verified: 

System recovers from an abnormal shut down without loss of data;

Audit log identifies users importing election results

No networking is available; turned off in BIOS

4/28/09 Reject 138,139, 

140, 148, 

149, 150, 

151

ERM Security Testing: 

Successful and unsuccessful logins are not recorded in the audit log

Administrator modifications of access privileges is not logged

Modification of the election results file was not logged

Unsuccessful DS200 imports are not recorded in the audit log

Encryption key generation did not reflect industry best practice (NIST)

Algorithm code, which is not being called in the application, is present without selftest verification.

5/26/09 Accept #131 M650 Security Document Review & Testing verified: 

The readiness report included check election options (election definition files are 

included with time stamp). With this it is possible to detect the modification

5/19/09 

to 

5/30/09

Accept Trusted Builds for all Unity 3.2.0.0 applications (Appendix G)

No source code, compilers or assemblers are resident or accessible

6/2/09 to 

6/6/09

Accept #123, 

124, 130

Unity– Windows Configuration Test Steps: 

Windows events log successful and unsuccessful login to Unity applications

Windows events log file access

6/11/09 Reject #159 AIMS– Windows Configuration: 

Procedures detail all needed steps

6/16/09 Accept #154, 

158, 159

Unity & AIMS– Windows Configuration Discrepancies: 

Procedures detail user rights

Procedures detail all needed materials

Procedures detail all needed steps

6/17/09 Accept #122, 150 Unity– Windows Configuration Discrepancies: 

Non-administrative users cannot install executable software

Administrator modifications of access privileges is not logged

6/18/09 Accept #138, 139 Unity– Windows Configuration Discrepancies: 

Modification of the election results file was logged

6/18/09 Accept #149 Unity– 3.2.0.0 Scope Discrepancies: 

Code is in place for the iVotronic which is not supported in Unity 3.2.0.0; moved to 

Unity 4.0.0.0 Discrepancy Report

6/19/09 Accept #148 ERM Security Testing: 

Windows performance of password management; meets industry best practices 

(NIST)

6/22/09 Accept #132, 140 ERM Security Testing: 

Successfully recover from an external error not generated by electrical or 

mechanical conditional

Physical security measures are documented

6/22/09 Accept #126 Unity– Windows Configuration Test Steps: 

Windows events log file access

6/23/09 Incorporate Discrepancies #121 and 151 into Volume 9 testing

All issues are documented in Appendix E- Discrepancy Report.

Method Detail Security Test Method

Test Case Name Security

Scope - identifies the type of test Security testing crosses into several areas of voting system testing and thus must be tested at the 

integrated system level. The Regression System Level test is customized for the specific voting 

system to test the security elements incorporated into the pre-vote, voting and post voting 

functions. Further examination is performed in Telephony and Cryptographic Tests. A review of 

the security documentation addresses Access Controls, Physical Security and Software Security. 

Test Objective The objective of security testing is to minimize the risk of accidents, inadvertent mistakes and 

errors; protect from intentional manipulation, fraud or malicious mischief;

Test Variables: In the Regression elections validate the security of the pre-vote, voting, and post voting functions 

of the voting system by test incorporating overflow conditions, boundaries, password 

configurations, negative testing, and inputs to exercise errors and status messages, protection of 

the secrecy in the voting process and identification of fraudulent or erroneous changes. Including: 
VSTL Certification # pending

Page 129 of 247 (V)2009-30Jun-001(A)

Method Detail Security Test Method

Unauthorized changes to system capabilities for: 

- Defining ballot formats,

- Casting and recording votes, 

- Calculating vote totals consistent with defined ballot formats,

- Reporting vote totals,

- Alteration of voting system audit trails,

- Changing or preventing the recording of a vote,

- Introducing data not cast by an authorized voter,

- Changing calculated vote totals, 

- Preventing access to vote data, including individual votes and vote totals, to unauthorized 

individuals, and

- Preventing access to voter identification data and data for votes cast by the voter such that an 

individual can determine the content of specific votes cast by the voter.

A description of the voting system type and 

the operational environment

The voting system types and operational environments 

Election Data Manager (EDM) to create the election data used for all ballot layout and tabulation 

for all equipment used in the election.

 -Super VGA (800x600) or higher

 -Keyboard and Mouse

 -512 MB RAM

 -48x CD-ROM or DVD drive

 -40-GB hard drive

 -PC with 1-GHz or faster processor

ESSIM to format the ballots by using election database

 --partner printer

 -24x CD-ROM

 -Windows XP Professional with Service Pack 2(SP2)

 -40-GB hard drive

 -Laser Printer(recommends Okidata C9600HDN)

 -512 MB RAM 

 -PC with 1-GHz or faster processor

HPM import IFC to import the ballot interface (.ifc) file ,containing all contest, candidate, precinct, 

rotation, polling place, and ballot style information, from the Election Data Manager(EDM) and 

Image Manager ballot (ESSIM)

 -SanDisk Compact Flash Card Reader/Writer

 -CD-ROM or DVD drive

 -Keyboard and Mouse

 -3.5-inch disk drive

 - 40-GB hard drive

 -PCL capable Laser Printer

 -PC Card Manager(optional)

 -Windows XP Professional

 -PC with 1-GHz or faster processor

Audit Manager(AM) functions are Administer username and login for Unity modules and 

Administer audit log information

 -Pentium 266MHz

 -32 Meg RAM

 -3.5 Inch Floppy Disk Drive

 -24X CD Drive

 -printer(optional)

Hardware Programming Manager (HPM) creates election definition for DS200

-DS200 scan paper ballot precinct tabulator

 -12-inch touch screen

 -thermal printer(internal)

 -USB flash drive(compact flash card)

 -external DC power

 -120-volt AC outlet,

 -internal memory(DRAM)

HPM creates election definition for M650(central count tabulator)

 -External Zip drive(FAT16 ZIP disk)

 -External Printer

 -internal memory

 -three-prong electrical outlet

 -128 MB solid-state hard drive

 -133 MHZ CPU

VAT(Voter Assist Terminal (Ballot marking device) is used to mark the ballot selections of voters 

who are visually impaired, have a disability, or who are more comfortable using an alternative 

language) and AIMS(Database)

 -Printed Circuit Boards

 -Single Board Computer

 -Compact Flash Memory Cartridge

 -Liquid Crystal Display

 -Touch Panel
VSTL Certification # pending

Page 130 of 247 (V)2009-30Jun-001(A)

Method Detail Security Test Method

 -Audio Subsystem

 -Switch Interface Board

 -Keypad For Visually Impaired

 -Audible Feedback

 -AT Dual-Switch Access Port

 -Printer Engine Board

 -Operating System – Microsoft Windows XP, SP1

 -MS Access, version XP

 -SQL Server (MSDE), version 2000, SP3

VSS 2002 vol. 1 2.2.1, 2.2.4 thru 2.2.5.2.3, 6.2 thru 6.4

VSS 2002 vol. 2 6.4 thru 6.4.2

Hardware, Software voting system 

configuration and test location

Same as Regression System Level test case

Pre-requisites and preparation for execution 

of the test case. 

Test Method Validation: Technical review conducted by K Wilson; Approved 2/20/09 for 

validation of test method as defined in ISO/IEC 17025 clause 5.4.5. -

Same as Regression System Level test case

Getting Started Checks Same as Regression System Level test case

Prior to testing Verify the following through Document Review:

-DS200 and M650 Indentify procedural requirements for the usage of locks to prevent 

unauthorized access

-DS200 provide adequate procedural requirements for polling place security.

-DS200 procedures relating to the preparation and configuration of the tabulation.

-DS200 and M650 procedures to identifying electronic media type.

-DS200 and M650 maintenance of a secured location for storing the

electronic media and voting machines

-Manual identifies all required access control security measures.

-M650 procedures for ballot security

-Procedures for administration security (database security)

-Operations manual indentifies specific instructions during a failure to input or storage devices.

-During witness and trusted build procedures verify source code, compilers or assemblers are not 

resident.

Documentation of Test Data & Test 

Results

Same as Regression System Level test case

Record the results of the security testing, document & source code reviews in the applicable 

Security Review tab

Enter Accept/Reject against each review requirement.

Log discrepancies on the appropriate Discrepancy Report

Pre-vote:

Ballot Preparation procedures verifications

Same as Regression System Level test case

Pre-vote:

Ballot Preparation Security

Same as Regression System Level test case

-Attempt to modify the ballot layout files. 

--Power can be interrupted & restored without loss of election data.

-- Attempt to halt the Audit Mgr before starting ESSIM. If it is not running, rename the file. Verify 

that ESSIM will not start. Restart Audit Mgr or if Audit Mgr (AuditManager.exe) was renamed, 

rename it back to the original name. Reboot and verify that ESSIM will run.

--Attempt to modify the audit log.

--Audit logs contain entries for failed attempts, normal & abnormal events.

--Verify Computer-generated password keys are unpredictable and random (v1:6.2.2.e)

--Verify that removing one of the RAID drives on the EDM system does not result in catastrophic 

data loss. System is operational without drive or system recovers when an empty drive is 

restored.

--Unplug the system (EDM) during a save operation. Verify that the system is capable of 

resuming operation when power is restored or a backup copy restored.

--Ghost the system prior to this test. For each of EDM, ESSIM, HPM and ERM, connect an iBeta 

computer to the network connected to the Audit Manager computer. Turn on Remote Access in 

the DUT computer. Access the audit manager database file as administrator and rename the file. 

Verify that the program halts further processing of election preparation, tabulation or reporting as 

necessary. (As an alternative, turn off the Audit Manager service and/or monitoring service or use 

task manager to kill the Audit Manager process and/or monitoring service/process).

--Attempt to access the database (EDM) and modify ballot information

--Default passwords are changeable after initial login

--Verified detailed information of encryption messages.

--Attempt to load the software with unauthorized user on AIMS

--Attempt to access AIMS database with invalid or blank password.

--Verify AIMS not networked or does not telecommunicate with any other system

Readiness Testing and Poll Verification Same as Regression System Level test case

Before installing the election definition in tabulators, perform the following test 

--Attempt to bypass the locks 

--Attempt to access Administration mode with invalid password and blank password

-Attempt to access administration Menu screen, when election definition is not installed.
VSTL Certification # pending

Page 131 of 247 (V)2009-30Jun-001(A)

Method Detail Security Test Method

--Attempt to install the firmware or software with unauthorized user.

--Attempt to load wrong election definition.

--Attempt to modify the election definition.

--Verify the firmware versions 

--Verify there is no public network to install election definition.

--Attempt to install virus or malicious software via compact flash card or zip disk 

--Audit logs contain entries for failed attempts, normal & abnormal events.

--Minimal password strength constraints are imposed by the vendor or settable by the jurisdiction

--Verify physically there is no modem or Ethernet card.

After installing the election definition in tabulators, perform the following steps

--Verify polls can not be opened after election data is installed into the system, validate this by 

attempting to open polls before election definition installed

--Attempt to modify the audit log with admin password.

--Attempt to change the election definition and overwrite the election definition after election 

definition is installed

--Attempt to insert the ballot prior to opening the polls. No votes can be recorded prior to opening 

the polls

--Attempt to insert invalid zip disk (FAT 32) or invalid compact flash card to verify only valid 

memory packs are accepted by tabulators.

--Verify the zero totals report, to check vote count is "0" when the scanner is turned on.

--Audit logs contain entries for failed attempts, normal & abnormal events.

Pre- vote:

Opening the Polls Verification

Same as Regression System Level test case

Opening the polls, perform the following

--System access controls are implemented for opening the polls; for the identified entity confirm 

access and use to only the permitted functions and data

--Attempt to access administration menu when the polls are open to verify voter does not have 

the ability to count votes

--Verify the locks

--Verify the zero total report when opening the polls for voting zero report lists the date and time 

that the polls open followed

by the vote count for all of your contests that is "0" and blank signature lines for poll worker 

certification

--Verify the right version of firmware is installed on ballot marking device.

--Verify VAT does not telecommunicate with any other system. 

--Opening the polls communication errors are reported to the user & require corrective action to 

continue operation

Voting: 

Ballot Activation and Casting Verifications

Ballot casting, perform the test

--Attempt to insert a blank, invalid ballot, torn ballots and multiple ballots

--Attempt to stop the system or event log to verify election process halts

--Attempt to remove the zip disk or USB flash drive in the middle of the process, verify that normal 

operation can be resumed

--Power can be interrupted and restored without loss of election data, validate this by pulling the 

power during ballot installation, verify that when power is restored; and recovery is possible. Audit 

log record (time/date) of power interruption and restore.

-- Attempt to Zero the totals on a scanner in middle of the processing, verify there is a possibility 

to reload the scanner with totals saved to disk.

--Attempt to remove the USB flash drive during ballot scan to verify normal process resumes after 

reinserting it.

--Attempt to remove the zip disk prior to saving election count data to check no loss of votes.

--View audit log to verify all attempts are recorded(success and fail)

--Attempt to remove the compact flash card from VAT to check normal process resumes after 

reinserting it.

--Vote errors & communication errors are displayed with action to resolve

Voting: 

Voting System Integrity, System Audit, 

Errors & Status Indicators

--Attempt to access the vote counts when the polls are open

--Attempt to open admin menu with invalid password.

--Attempt to feed in ballots that are torn, ripped, not of standard, incorrect data, incorrect precinct. 

Verify that only valid ballots of the correct election and precinct are accepted, all others are 

rejected.

--Voting continues after a power interruption and restore, verify this by attempting to interrupt 

power and then restore.

--Attempt to print results, when polls are open. Verify that the polls must be closed prior to viewing 

a results report.

--Attempt to save results on FAT32 format zip disk in M650.

--view audit log to verify all error messages are recorded.

Post-vote:

Closing the Polls

Same as Regression System Level testing

Central count Post vote

-- Verify Zero totals report having vote count as "0"

--Attempt to modify the results on zip disk.
VSTL Certification # pending

Page 132 of 247 (V)2009-30Jun-001(A)

Method Detail Security Test Method

--Verify there is no public network or LAN to transfer election results.

--Errors are displayed with action to resolve

--Audit logs contain entries for failed attempts, normal & abnormal events.

Post-vote:

Central Count

Close polls, perform following test

--Verify authorized reopening of poll, once the poll closing has been completed for that election.

--Attempt to modify the election results on memory pack, verify the election results cannot modify 

due to CRC written by DS200

--Verify there is no modem to transfer results to ERM.

--Precinct counts cannot be printed or viewed prior to the close of the polls

--Audit logs contain entries for failed attempts, normal & abnormal events.

Document Review

-- Verify there is no access to public network, no external access to incomplete returns, and no 

communication between locations and components before the polls close. 

--Verify environment do not share with non-election data processing functions.

Post-vote:

Security

Attempt to change the vote totals on memory packs before loading into ERM

--Audit logs contain entries for failed attempts, normal & abnormal events.

--verification of Authentication is required to access the ERM

----Errors are displayed with action to resolve

---Power can be interrupted & restored without loss of election data.

-A technical administrator, attempt to modify vote total counts for a race in an election.

-Attempt to modify vote counts after all vote counts are in.

--Attempt to modify the audit log

Post-vote:

System Audit

During system audit, verify the following validation

--Review of Audit logs to verify all login success and failed attempts are recorded

--Verify the Zero total reports

--Compare vote totals on memory pack with printed vote totals are the same.

Additional Security Source code review 

- Verify by source code review that user-generated passwords are not used directly as keys to an 

encryption algorithm.

-- Verify by source code review that encryption algorithms utilized in documentation match the 

actual encryption utilized and that any known vulnerabilities are mitigated (in so far as encryption 

is utilized in the system).

--Verify AIMS database is password protected.

--Verify through the source code review, hash code is generated by AIMS for the data on the flash 

card and upon insertion of flash card VAT check the hash code against the database to ensure 

that data has not been modified.

-- Verify the temporary memory is wiped out after a vote prints on the VAT

Expected Results are observed See System Level and Telephony and Cryptographic Test Cases.

Security Review Criteria:

- Accept meets the guideline

- Reject does not meet the guideline

- NA the guideline does not apply

Record observations and all input/outputs 

for each election;

All inputs, outputs, observations, deviations and any other information impacting the integrity of 

the test results will be recorded in the System Level Security Test Case. 

A separate statement will be prepared addressing the results of from the security perspective. It 

will provide the results of the testing and review required in vol. 1 section 6.

7.4.4 Telephony and Cryptographic Review and Testing

Testing was conducted on the system configuration identified in the PCA Configuration and Test Case.

Date Test Result Issues Opened Issues Closed Notes

3/20/09 Accept Removal of modem and network card observed. 

5/8/09 to 

6/3/09

See Security Test results were recorded in Security 

Only functional issues are identified. While documentation discrepancies may be encountered in testing they do not 

result in the rejection of a functional test. All issues are documented in Appendix E- Discrepancy Report.

Method Detail Telephony & CryptographicTest Method

Test Case Name Telephony and Cryptographic

Scope - identifies the type of test Unity 3.2.0.0 was not loading or transmitting election data via telecommunications or network

Test Objective The objective of the telephony and cryptographic testing is to confirm that Unity 3.2.0.0 is not 

loading or transmitting election data via telecommunications or network

Test Variables: Configured as the Regression System Level testing the DS200 does not contain a modem 

and M650 does not contain a network card for loading or transmitting election data via 
VSTL Certification # pending

Page 133 of 247 (V)2009-30Jun-001(A)

Method Detail Telephony & CryptographicTest Method

telecommunications or network

A description of the voting system type and 

the operational environment

In the Regression System Level and Security testing vote results from the DS200‟s and 

M650's was handled externally (via USB and zip disk) by the Unity Election Reporting 

Manager (ERM).

- No election definition (from HPM) was loaded.

- No results transmission via network or telecommunications.

VSS 2002 vol. 1 5.1 thru 5.2.7, 6.5.3, 6.6.1

VSS 2002 vol. 2 6.4.2

Hardware, Software voting system 

configuration and test location

see Security 

Pre-requisites and preparation for execution 

of the test case. 

see Security 

Getting Started Checks see Security 

Documentation of Test Data & Test Results see Security 

Pre-vote:

Ballot Preparation procedures verifications

see Security 

Pre-vote:

Ballot Preparation Security

see Security 

Readiness Testing and Poll Verification Security testing verifies that there was no network to install the election definition. 

Pre- vote:

Opening the Polls Verification

In Security testing verify the Unity 3.2.0.0 was not loading or transmitting election data via 

telecommunications or a LAN network.

Voting: 

Ballot Activation and Casting Verifications

In Security testing verify the Unity 3.2.0.0 was not loading or transmitting election data via 

telecommunications or a network consolidated within the polling place prior to the voter 

casting a ballot

Voting: 

Voting System Integrity, System Audit, 

Errors & Status Indicators

N/A

Post-vote:

Closing the Polls

In Security testing verify that the DS200 has no modem to transmit data

Post-vote:

Central Count

In Security testing verify that central count has no public network to transmit data. 

Post-vote:

Security

N/A

Post-vote:

System Audit

N/A

Additional Security 

Expected Results are observed see Security 

Record observations and all input/outputs 

for each election;

see Security 
VSTL Certification # pending

Page 134 of 247 (V)2009-30Jun-001(A)

7.4.5 FCA Reuse Environmental Testing

Testing was performed by SysTest and their subcontractors. ES&S petitioned the EAC for reuse. The terms of the reuse are identified in the EAC February 3, 2009 

letter: EAC letter to iBeta QA Director on reuse of testing .

The results of the hardware environmental testing by SysTest and their subcontractors are contained in the Summary Report of the testing performed by SysTest. iBeta 

reviewed the submitted reports. The following sections trace the test requirement to the applicable test report(s) (number) and the specific hardware (alpha) for each of 

the Unity 3.2.0.0 voting devices. In the review iBeta noted several identified information that required greater detail or clarification. These are listed in Appendix EDiscrepancy Report with a trace (#X) referenced next to the individual report name.

7.4.5.1 DS200 Environmental Hardware Test Reports &Tested Configuration Matrix

1) DS200 EMS Test Report 070214-134A 5/15/07 (Criterion - #3)

2) DS200 ENV Temp Humid Report 5/15/07 (APT)

3) DS200 ENV VIB Report 07-00207 5/15/07 (APT)

4) Percept Hardware Test Report 1.0 (#2 & 3)

5) ESS DS200 Product Safety Test Report Rev E-2 (Components)

6) DS200with Optional Ballot Box ESD Test Report 1.0 (Percept - #1) 

7) DS200EMC Report R071107-30-01 (NCEE #3)

8) DS200EMC Report R071107-30-01B (NCEE #3)

DS200 Hardware MIL STD 810D EMC OSHA

Tested Configuration

516.3 Bench 

Handling

514.3 Cat 1Vibration502 Low Temp501 High Temp507-2 Humidity501 & 502 Temp & 

Power Var & 163 hr Reliability

Electromag RadPart 15 Class BPower Disturb 61000-4-11ESD61000-4-2Electromag Susct 61000-4-3Elec Fast Trans

61000-4-4

Lightening Surge

61000-4-5

RF Immunity 61000-4-6Magnetic Fields Immunity 61000-4-8Safety Title 29,Part 1910

Configurations tested w/ ballot box:

A: DS200 SN0002, AC Adapter 

SN72573415, Ballot box SN2007

B: DS200 SN0004, AC Adapter 

SN72573407, Ballot box SN3016

C: DS200 SN0003, AC Adapter 

SN72573407, Ballot box SN3016

D: DS200 SN0010, AC Adapter 

SN72632719, Ballot box SN3016

E: DS200 SN0011, AC Adapter 

SN72573413, Ballot box SN2804

H: DS200 SN0001, AC Adapter 

SN72573407 or not specifically 

identified, Ballot box SN2804 

Configurations tested w/o ballot box: 

F: DS200 SN0003, AC Adapter 

SN72632720

G: DS200 SN0004, AC Adapter 

SN72573407

4 C 3 & 

4 C

4 C 4 C 4 C 2 & 4

D, E, 

F, & G

8 I

1 & 4

H

7 I

1 & 4

H

8 I

6 & 4

H

7 I

1 & 4

H

7 I

1 & 4

H

1 & 4

H

7 I

1 & 4 

H

7 I

1 & 4 

H

4 & 5

A
VSTL Certification # pending

Page 135 of 247 (V)2009-30Jun-001(A)

DS200 Hardware MIL STD 810D EMC OSHA

Tested Configuration

516.3 Bench 

Handling

514.3 Cat 1Vibration502 Low Temp501 High Temp507-2 Humidity501 & 502 Temp & 

Power Var & 163 hr Reliability

Electromag RadPart 15 Class BPower Disturb 61000-4-11ESD61000-4-2Electromag Susct 61000-4-3Elec Fast Trans

61000-4-4

Lightening Surge

61000-4-5

RF Immunity 61000-4-6Magnetic Fields Immunity 61000-4-8Safety Title 29,Part 1910

I: DS200 SN S/N11027011 AC 

Adapter not identified

7.4.5.2 M-650 Environmental Hardware Test Reports & Tested Configuration Matrix 

Central count scanner is exempt from non-operating environmental tests

1) NCEE EMC Test Report No. R071107-30-02A

2) Certificate of Compliance UL 60950-1 (2nd Ed.) No. ESS-0806-R05-COC 

3) Testing Services Report M650 Job No. 08-00654 (APT #6)

4) Voting System Test Summary Report, Test Report for testing through 10/22/08 for ES&S Unity 4.0 Voting System, Report Number 01-V-ESS-035-CTP-01 rev.0.2

M-650 Hardware MIL STD 810D D EMC OSHA

Tested Configuration

516.3 Bench 

Handling

514.3 Cat 1Vibration502 Low Temp501 High Temp507-2 Humidity501 & 502 Temp & 

Power Var & 163 hr Reliability

Electromag RadPart 15 Class BPower Disturb 61000-4-11ESD61000-4-2Electromag Susct 61000-4-3Elec Fast Trans

61000-4-4

Lightening Surge

61000-4-5

RF Immunity 61000-4-6Magnetic Fields Immunity 61000-4-8Safety Title 29, Part 1910

Configurations: 

A: M-650 1102 7011 Accessories: 

2 @ Epson LQ-590 Dot Matrix 

Printers S/N: FSQY094255, 

FSQY093497, 1 @ Belkin 

F6C1500-TW-RK, Battery Backup 

S/N: 20V06516248WE

B: M-650 S/N 11027011 & 7003

C: M-650 S/N 2406 8013

ExemptExemptExemptExemptExempt

3 & 4

B

1 A 1 A 1 A 1 A 1 A 1 A 1 A 1 A 2 C

7.4.5.3 VAT A-100 Environmental Hardware Test Reports & Tested Configuration Matrix

1) AutoMARK EMC Test Report1/31/05 (Criterion)

2) Electrical Safety Testing to UL 60950-1 (Report No. ATS-0501-R01-Rev.1 4/10/06; replaces R01 4/30/05)

3) VAT A100 EMC report 080327-1225 Criterion – Report issued for Premier

4) ES&S AutoMARK VAT A200 (Report No. 080521-1251A 6/11/08) (#8)

5) AutoMARK Voter Assist Terminal Test Report rev.1.3 (Percept - #7)
VSTL Certification # pending

Page 136 of 247 (V)2009-30Jun-001(A)

6) Testing Services Report AutoMARK Voter Assist Terminal S/N: 002 Job No. 04-00542 (APT 1/12/05 Vibration & Bench)

VAT A-100 MIL STD 810D EMC OSHA

Tested Configuration

516.3 Bench 

Handling

514.3 Cat 1Vibration502 Low Temp501 High Temp507-2 Humidity501 & 502 Temp & 

Power Var

& 163 hr 

Reliability

Electromag RadPart 15 Class BPower Disturb 61000-4-11ESD61000-4-2Electromag Susct 61000-4-3Elec Fast Trans

61000-4-4

Lightening Surge

61000-4-5

RF Immunity 61000-4-6Magnetic Fields Immunity 61000-4-8Safety Title 29, Part 1910

Configurations: 

A: A100 – S/N 005

B: A100 – S/N AM0205420004

C: A100 – S/N AM0105521108 (HW 

submitted by Premier)

D: A100 – S/N 002

E: A200 – S/N AM0206462702

F: A100 – S/N 008

G: A100 – S/N 005, 007, 008, 

DV3.5-2, & DV3.5-3

6 D 6 D 5 G 5 G 5 F 5 G 1 A

4 E

1 A 1 A

3 C

4 E

1 A 1 A

4 E

1 A 1 A 1 A 2 B

7.4.5.4 VAT A-200 Environmental Hardware Test Reports & Tested Configuration Matrix 

1) AutoMARK EMC Test Report1/31/05 (Criterion)

2) Electrical Safety Testing to UL 60950-1 (Report No. ATS-0501-R01-Rev.1 4/10/06; replaces R01 4/30/05)

3) VAT A300 EMC report 070730-1165 (Criterion - #9)

4) VAT Accuracy Test Case Rev.02 (no date or organization identified)

5) AutoMARK Voter Assist Terminal Test Report rev.1.3 (Percept 5/19/05)

6) Testing Services Report AutoMARK Voter Assist Terminal S/N: 002 Job No. 04-00542 (APT 1/12/05 Vibration & Bench)

VAT A-200 MIL STD 8810D EMC OSHA

Tested Configuration

516.3 Bench 

Handling

514.3 Cat 1Vibration502 Low Temp501 High Temp507-2 Humidity501 & 502 Temp & 

Power Var & 163 hr Reliability

Electromag RadPart 15 Class BPower Disturb 61000-4-11ESD61000-4-2Electromag Susct 61000-4-3Elec Fast Trans

61000-4-4

Lightening Surge

61000-4-5

RF Immunity 61000-4-6Magnetic Fields Immunity 61000-4-8Safety Title 29, Part 1910

VAT A100 Configurations: 

A: A100 – S/N 005

B: A100 – S/N AM0205420004

D: A100 – S/N 002

F: A100 – S/N 008 

G: A100 – S/N 005, 007, 008, 

DV3.5-2, or DV3.5-3

VAT A300 Configurations:

C: A300 – S/N AM0307420125

6 D 6 D 5 G 5 G 5 F 5 G 3 C 1 A 3 C 3 C 1 A 1 A 1 A 1 A 2 B
VSTL Certification # pending

Page 137 of 247 (V)2009-30Jun-001(A)

7.5 Appendix E- Discrepancy Report

This report incorporates open discrepancies in the scope of Unity 3.2.0.0 that were transferred from SysTest Labs at the time of their suspension. The SysTest 

discrepancy number is reflected in the description. Issues encountered by iBeta during testing are contained in the report. The handling of discrepancies closed 

by SysTest should be identified in the Summary Report of the testing performed by SysTest.

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

1 12/5/08 C 

Cog

gins

Document 

Defect

Closed Unity 4.0 

Discrepancy 

Report 

10/28/08 

(SysTest)

DS-200 with 

Optional 

Ballot Box 

ESD Test 

Report 1.0 

(Percept)

Potentially reusable Unity 4.0 hardware 

test results do not document validation 

of the ES&S' resolution of an ESD 

failure

On page 2 of the ESD report a failure 

and mitigation is identified, however 

the failure and validation resolution is 

not documented in the Discrepancy 

Report or the sub-contractor report. 

There is no documentation that an 

ES&S associated engineering change 

was issued to address the 

"Modifications Required: The poll close 

button failed at +15kV in stand alone 

mode. Copper tape on backside of 

switch cover was applied to pass at 

+15kV. The previous VSTL did not 

provide detail that evidences their 

validation that an engineering change 

was initiated by ES&S as a result of the 

mitigation performed by the 

subcontractor lab in ESD testing. 

v.1: 9.6.2.6.e The ITA shall evaluate 

data resulting from examinations 

and tests employing the following 

practices: Any and all failures that 

occurred as a result of a deficiency 

shall be classified as purged, and 

test results shall be evaluated ...if 

the 1) vendor submits a design, 

manufacturing ... change notice... 2) 

examiner of the equipment agrees 

that the proposed change will 

correct the deficiency; and 3) vendor 

certifies that the change will be 

incorporated...

EAC NOC 07-005 it is the lead 

VSTL‘s responsibility to properly test 

the voting system and accurately 

report those tests to the EAC.

ES&S referred this issue to 

SysTest; SysTest responded 

1/8/09:

The failure and validation 

resolution is documented on page 

4 and 19 of the sub-contractor 

report. ES&S submitted ECO 693 

to address the "Modifications 

Required" and Systest' hardware 

subcontractor Percept completed 

the Engineering Change 

Evaluation & Review form. Systest 

will provide both documents to 

iBeta.

Accepted, 1/13/09 

KS 

Verified doc Optional 

Ballot Box ESD, v. 

1.0, 4/25/07; pg. 4

shows the failure, 

and resolution 

retested and passing. 

Pg. 19 is a photo 

showing the part with 

the copper tape. 

ECO693 reflected 

the identified 

changes. 

2 12/5/08 C 

Cog

gins

Document 

Defect

Closed Unity 4.0 

Discrepancy 

Report 

10/28/08 

(SysTest)

Percept 

Hardware 

Test Report 

1.0 (DS-200 

5/1/07)

Potentially reusable Unity 4.0 hardware 

test results contain no description of 

two test failures and the validation of 

their resolution by the VSTL.

On page 29 of the sub-contractor 

(Percept) report two failures (CAR001_DS200-Radiated Emissions, CAR002_DS200-Radiated Immunity) and 

mitigation with 4 ECOs 690 to 693 are 

identified. Neither the sub-contractor 

report nor the Discrepancy Report 

provide a description of how, what, 

when and where the failures occurred 

or who, how, when and where the 

mitigations were performed that 

resulted in the ECO. There is no 

identification of the validation of the 

resolution. 

1/14/09 KS

v.1: 9.6.2.6 The ITA shall evaluate 

data resulting from examinations 

and tests employing the following 

practices: 

a: If any malfunction ... is detected 

that would be classified as a 

relevant failure using the criteria in 

Vol.2, its occurrence ... shall be 

recorded for inclusion in the analysis 

of data obtained from the test...

e: Any and all failures that occurred 

as a result of a deficiency shall be 

classified as purged, and test results 

shall be evaluated ...if the 2) 

examiner of the equipment agrees 

that the proposed change will 

correct the deficiency

EAC NOC 07-005 it is the lead 

VSTL‘s responsibility to properly test 

the voting system and accurately 

report those tests to the EAC.

ES&S referred this issue to 

SysTest; SysTest responded 

1/8/09: 

EMC test report "DS200 EMC Test 

Report 070314-1134A.pdf" 

Appendix A page 80 of 84 issued 

by Criterion and Percept CAR001_DS200-Radiated Emissions, 

and CAR-002_DS200-Radiated 

Immunity provide a description of 

modifications. Systest will provide 

these documents to iBeta.

Reject 1/14/09 KS 

ECOs are not 

provided

Accepted 2/6/09 

CEC 

ECO 692 and COTS 

power supply 

specification were 

provided 

documenting the 

mitigation changes.
VSTL Certification # pending

Page 138 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

- Accepted: Verified that "DS200 EMC 

Test Report 070314-1134A.pdf" 

Section 6.5 Appx. A, pg. 80 describes 

4 modifications made to the DS200 

and these modifications match CAR001 & CAR-002

- Rejected: The ECOs 690 to 693 

were not provided. (Note: ECO693 

was provided for discrepancy #1. It 

does not match the description in these 

submitted CARs.)

3 12/5/08 C 

Cog

gins

Document 

Defect

Closed ES&S Retest 

Matrix v.1.16 -

DS-200 

testing 

(SysTest)

DS-200 EMC 

Report 

R071107-30-

01 (NCEE 

original)

DS-200 EMC 

Report 

R071107-30-

01B (NCEE 

amended)

DS200 EMS 

Test Report 

070214-134A 

5/15/07 

(Criterion)

Percept 

Hardware 

Test Report 

1.0 (DS-200 

5/1/07)

Potentially reusable Unity 4.0 hardware 

test results do not contain an 

assessment of the scope of testing.

The HW test matrix lists three EMC 

reports from two labs for the DS-200. 

Testing performed at Criterion in March 

2007 included a ballot box. Testing a 

few months later at NCEE excluded the 

ballot box, Power Disturbance and 

Lightening Surge. An original and 

amended report was issued by NCEE. 

The HW test matrix indicates that the 

ESD & FCC Part 15B applicable test 

results are in the amended NCEE 

report. Four additional tests run by 

NCEE are traced to the original NCEE 

report. All reports identify the DS-200 

as passing. No report or test plan 

provides an assessment addressing 

the NCEE testing or why:

1) The EMC testing needed to be 

repeated by NCEE for six tests when 

the Percept and Criterion report 

indicate the system passed. 

2) Power Disturbance and Lightening 

Surge weren't repeated.

3) Only ESD and FCC Part 15B results 

use the amended NCEE report when 

updates were made to all tests.

4) The NCEE testing excluded the 

ballot box.

v.1: 9.6.1.1 As described in 9.5.2, 

the nature and scope of testing for 

system changes or new versions 

shall be determined by the ITA 

based upon the nature and scope of 

the modifications to the system and 

on the quality of system 

documentation and configuration 

management records submitted by 

the vendor. 

ES&S referred this issue to 

SysTest; SysTest responded 

1/8/09:

ES&S changed components on 

PMB, USB, PEB, ASB, and PSB to 

be RoHS compliant as detailed in 

ECOs 702-706. These changes 

have no impact on the power

supply, therefore Power 

Disturbance, and Lightening Surge 

tests weren't repeated. Note both 

original and amended NCEE 

reports are identical except the 

amended report now reference the 

correct FEC document (see sec. 

1.3 Reason for Amendments pg 3 

of 43 for details in the amended 

report). Also the changes have no 

impact on ballot box, therefore the 

NCEE testing excluded the ballot 

box. Systest will provide these 

documents to iBeta.

Accept 1/14/09 KS

Verified that ES&S 

ECO's 702-706 

addressing the 

changes to DS200 

for Restriction of 

Hazardous 

Substances (Lead) 

were provide. In 

addition the 

corresponding 

SysTest ECO 

assessment and the 

comments submitted 

with these 

documents address 

the SysTest rationale 

for testing. 

4 12/9/08 J 

Garc

ia

Document 

Defect

Closed Unity 4.0 Test 

Plan rev. 9.1 

Attachments

The appendix identified in the rev.9.1 

of the Test Plan were not provided in 

the package from SysTest. 

The EAC has instructed that testing of 

Unity 3.2 shall incorporate system 

limitation testing per the approved 

Unity 4.0 Test Plan. The appendices 

v.1: 8.7.2.b.1 The FCA s conducted 

by the ITA to verify that the system 

performs all the functions described 

in the system documentation. The 

vendor shall: provide the following 

information to support his audit: 

copies of all procedures used for ... 

integration testing and system 

Accept 1/14/09 KS

The EAC provided a 

chain of evidence 

copy - Unity 4.0 

T.P.v.6 Attachments 

A -H
VSTL Certification # pending

Page 139 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

referenced in the Section 1.1 were not 

provided with the Test Plan.

testing

5 12/9/08 J 

Garc

ia

Document 

Defect

Closed Unity 4.0 Test 

Plan rev. 9.1 

spreadsheet 

of system 

limitations

A spreadsheet containing information 

regarding the testing of system 

limitations for the approved EAC Unity 

4.0 Test Plan was not provided. 

The EAC has instructed that testing of 

Unity 3.2 shall incorporate system 

limitation testing per the approved 

Unity 4.0 Test Plan. "The attached 

spreadsheet" that provides a matrix of 

limitation is identified in section 

4.3.10.2 but was not provided with the 

Test Plan.

v.1: 8.7.2.b.3 The FCA s conducted 

by the ITA to verify that the system 

performs all the functions described 

in the system documentation. The 

vendor shall: provide the following 

information to support his audit: 

records of all tests performed ... 

including error corrections and 

retests

ES&S referred this issue to 

SysTest; SysTest responded 

1/8/09:

Systest will provide a spreadsheet 

containing information regarding 

the testing of system limitations to 

iBeta.

Accepted: 1/14/09 

KS

Verified the 

limitations 

spreadsheet was 

received

6 12/9/08 C 

Cog

gins

Document 

Defect

Closed ES&S Retest 

Matrix v.1.16 -

DS-200 

testing 

(SysTest) 

APT Labs 

Testing 

Services 

Report M650 

Job no.08-

00654 

(5/2/08)

The Temperature, Power Variation 

and Reliability report does not identify 

whether the M650 passed or failed.

The matrix indicates the APT report 

contains the results of M650 Testing 

for Temperature, Power Variations and 

Reliability. Section 5.1 indicates that 

the operational tests are performed by 

SysTest and they will determine the 

pass/fail of the test. No SysTest report 

identifying the pass/fail report has been 

provided. 

v.2: B.5 The test report shall be 

organized so as to facilitate the 

presentation of conclusions ...a 

summary of test results ... 

ES&S referred this issue to 

SysTest; SysTest responded 

1/8/09:

The APT policy is not to state the 

results of testing in their test report 

as they do not perform operational 

status check. Systest performed 

the operational status check prior 

to and after each test so they left it 

up to Systest to state whether a 

product passed or failed. Systest 

stated that the product passed in 

their Environmental Test Case 

Summary. A copy of 

Environmental Test Case 

Summary will be provided to iBeta.

Accepted: 1/15/09 

KS -

Verified the SysTest

Test Summary 

Report references 

SUN APT lab as 

performing 

environmental testing 

and "All tested 

equipment 

successfully passed 

each of the 

environmental tests 

to which the 

equipment

was subjected." 

Defer to EAC for 

determination of 

reuse.

7 12/11/08 C 

Cog

gins

Document 

Defect

Closed AutoMARK 

Voter 

Assisted 

Terminal Test 

Report rev.1.3 

(Percept 

5/19/05 )

Potentially reusable Unity 4.0 hardware 

test results (A100) contain no 

description of the engineering changes 

initiated during testing. 

Section 2.1 of the sub-contractor report 

identifies S/N-008 returned for a 

calibration error; it does not identify if it 

was associated with the test failure 

identified in section 3.4.1 & 3.4.1.1.1. 

The VAT failure identifies mechanical 

changes but does not identify the 

engineering change. As neither the 

original ITA report nor supporting 

documentation of the failure was 

submitted it could not be validated if 

the discrepancy and resolution was 

v.1: 9.6.2.6 The ITA shall evaluate 

data resulting from examinations 

and tests employing the following 

practices: 

a: If any malfunction ... is detected 

that would be classified as a 

relevant failure using the criteria in 

Vol.2, its occurrence ... shall be 

recorded for inclusion in the analysis 

of data obtained from the test...

e: Any and all failures that occurred 

as a result of a deficiency shall be 

classified as purged, and test results 

shall be evaluated ...if the 2) 

examiner of the equipment agrees 

that the proposed change will 

correct the deficiency

ES&S referred this issue to 

SysTest; SysTest responded 

1/8/09:

Per Humidity Test Nonconforming 

Work and Corrective Action 

Request S/N-008 returned for a 

calibration error was not 

associated with the test failure 

identified in section 3.4.1 & 

3.4.1.1.1. S/N:-008 was 

associated with 120 hrs humidity 

test Sec. 3.3.5 of the test report. 

Auto mark submitted ECO 0025 to 

address mechanical change. 

Systest will provide these 

documents to iBeta.

Accepted: 1/14/09 

KS

Verified that 

ECN025 matches the 

failure identified in 

sections 3.4.1 & 

3.4.1.1.1. 

CAR SN-008 

identifies "humidity 

test was restarted 

after installing a new 

touch screen panel 

with adequate 

clearance for the 

wires". The CAR 

identifies how the 

system was restored 
VSTL Certification # pending

Page 140 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

documented in the test record. EAC NOC 07-005 it is the lead 

VSTL‘s responsibility to properly test 

the voting system and accurately 

report those tests to the EAC.

but does not clearly 

identify the reason 

for the failure. It is 

unclear if "clearance 

for the wires" was an 

Engineering Change 

or replacement of a 

failed part. iBeta 

accepts the response 

but refers these 

findings to the EAC 

for determination of 

reuse. 

8 12/19/08 C 

Cog

gins

Document 

Defect

Closed ES&S 

AutoMARK 

VAT A200 

(Report No. 

080521-

1215R 

6/11/08) 

Potentially reusable Unity 4.0 hardware 

test results for the AutoMARK VAT 

A100 do not contain an assessment of 

the changes in the VAT models that 

permit the use of A100 and A200 

reports.

An EMC report for the A200 was 

submitted with the A100 reports. 

Reuse of prior hardware environmental 

testing is permitted by the EAC if an 

ESD test is performed. A 2008 ESD 

for the A200 was submitted to support 

reuse of the 2005 A100 testing. There 

is no assessment of the hardware that 

identifies the impact on testing of the 

changes between the A100 and A200 

so that the A200 ESD testing is 

sufficient to support reuse of the A100 

2005 reports.

The A200 report indicates that Electric 

Fast Transit was repeated but there is 

no assessment identifying why this test 

was required but the other tests were 

not required. 

1/15/09 KS

Accept: Verified that Phase 2 Change 

Summary.pdf and submitted SysTest 

ECO 200-206, 208, 210-247, 256-278 

assessments identify changes 

between A100 & A200 . Confirmed 

that all required testing identified in 

these assessments was performed in 

AutoMARK VAT1.1 EMC Test Report 

051214-995R.pdf; Document 5K50-30 

vs. 5K50-20 Differential items_G.pdf 

reviewed for changes between A200 & 

v.1: 9.5.2.1 The ITA will determine 

the test necessary for to qualify the 

modified system based on a review 

of the nature and scope of 

changes...

EAC Voting System Test and 

Certification Program Manual v.1.0

2.10.5.2 Use of valid prior testing is 

authorized only when:

2.10.5.2.1. The discrete software or 

hardware component previously 

tested is demonstrably identical to 

that presently offered for testing. 

VSTLs must examine the 

components to ensure no change 

has taken place consistent with all

documentation. When valid prior 

testing is used, the system 

presented must be subject to 

regression testing, functional testing 

and system integration testing;

2.10.5.2.2. The voting system 

standards applicable to the prior and 

current testing are identical;

2.10.5.2.3. The test methods used 

are substantially identical to current 

test methods approved by the EAC; 

and

2.10.5.2.4. The adoption and use of 

valid prior testing is noted in the test 

plan and test report. 

ES&S referred this issue to 

SysTest; SysTest responded 

1/8/09:

Phase 2 Change Summary.pdf 

document describes the 

differences between the model 

A100 and A200. 5K50-30 vs. 

5K50-20 Differential items_G.pdf 

document describes the 

differences between the model 

A200 and A300. Please note there 

are no hardware differences 

between the model A200 and 

A300. AutoMARK Voter Assist 

Terminal Test Report rev 1.3.pdf is 

the test report for model A200. 

Systest will provide these 

documents to iBeta.

Reject: 1/15/09 KS 

Phase 2 Change 

Summary.pdf 

references ECO324-

346 which were not 

provided. 

Accept: 1/15/09 KS 

Verified that Phase 2 

Change 

Summary.pdf and 

submitted SysTest 

ECO 200-206, 208, 

210-247, 256-278 

assessments identify 

changes between 

A100 & A200 . 

Confirmed that all 

required testing 

identified in these

assessments was 

performed in 

AutoMARK VAT1.1 

EMC Test Report 

051214-995R.pdf; 

Document 5K50-30 

vs. 5K50-20 

Differential 

items_G.pdf 

reviewed for changes 

between A200 & 

A300. 

Accept: 2/6/09 CEC 

Verified receipt of the 

ESO324 - 346.
VSTL Certification # pending

Page 141 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

A300. 

Reject: The Phase 2 Change 

Summary.pdf identifies ECO324-346. 

SysTest did not provide these 

assessments 

9 12/19/08 C 

Cog

gins

Document 

Defect

Closed VAT A300 

EMC report 

070730-1165 

Criterion 

Potentially reusable Unity 4.0 hardware 

test results for the AutoMARK VAT 

A200 do not contain an assessment of 

the changes that permits use of the 

A300 reports.

An EMC report for the A300 was 

submitted for the A200 report. There 

is no assessment of scope that 

identifies the differences between the 

A200 and A300. 

v.1: 9.5.2.1 The ITA will determine 

the test necessary for to qualify the 

modified system based on a review 

of the nature and scope of 

changes...

ES&S referred this issue to 

SysTest; SysTest responded 

1/8/09:

Premier Election Systems is listed 

as the client in the test report but 

the model number that was tested 

is VAT A100 which is common to 

both companies. Both Al Backlund 

and Darrick Forester believe that 

there was discussion of joint 

testing between ES&S and 

Premier but Systest was not 

involved in it. 

 

Accept 1/14/09 KS

Accepted based 

upon the response in 

discrepancy #8 that 

there are no 

differences between 

the A200 and A300. 

10 12/19/08 S 

jakile

ti

Document 

Defect

Closed Win650 Build 

EnvironmentCompileInstall Guide 

v.1.0

Procedures and validations are 

incomplete (TBD).

Section 2: CREATE BUILD 

ENVIRONMENT is missing the 

procedure and validation list. 

Section 4: BUILD SOURCE CODE is 

missing "validation" to validate the 

compile is successful

This information is listed as "TBD"

v.1: 8.5.a :The vendor shall establish 

such procedures and related 

conventions, providing a complete 

description of those procedures 

used to:

a. Develop and maintain internally 

developed items;

EAC Voting System Testing and 

Certification Program Manual, v.1.0 

Sec 5.5.1. Demonstrate that the 

software was built as described in 

the Technical Data Package.

20090512 - MDN -- Completed 

build documentation to be staged 

to the FTP server on May 13.

Accepted SJ 6/4/09

Procedure was 

verified in the M650 

trusted build.

11 12/23/08 S 

eato

n

Document 

Defect

Closed NCEE Labs 

EMC Test 

Report 

(R071107-30-

02) (page 4) 

2.1.5 Serial 

Number

The report does not correctly identify 

the equipment tested. 

The report identifies the serial number 

for the M650 Central Ballot Counter as 

ES0107360007. This number does not 

match the format of the M650 serial 

number. It appears to be a serial 

number of an intElect DS200.

v.2: 4.6.1.1 Equipment 

identification... shall be recorded.

Accepted C Coggins 

1/21/09 Verified 

revised NCEE Labs 

EMC Test Report 

(R071107-30-02A) 

identifies M650 

S/N11027011

12 12/30/08 K 

Swift

Document 

Defect

Closed M650 Sys 

Maint Man, v. 

2.2.0.0, No 

rev., May 17, 

2007

DS200 Sys 

Maint Man, 

From SysTest Unity 4.0 Testing -

Disc: 6

In the cited documents, no concept of 

operation is presented.

V.2: 2.9.1 a, b, e, f, g, h

The vendor shall describe the 

structure and function of the 

equipment (and related software) for 

election preparation, programming, 

vote recording, tabulation, and 

reporting in sufficient detail to 

provide an overview of the system 

10.17.08 - RDG

M650 SMM v.2.2.1.0_10.17.2008

Added information about data 

output to the end of the Concept 

of Operation section in Chapter 1: 

Introduction. 

10.17.08 - RDG

Accepted: K Swift, 

12/30/08

Reviewed the 

SysTest description 

history and verified 

all other 

requirements have 

been previously 
VSTL Certification # pending

Page 142 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

HW v. 1.0.0.0, 

No Rev. Jun 

8, 2007

for maintenance, and for 

identification of faulty hardware or 

software. The description shall 

include a concept of operations that 

fully describes such items as:

a. The electrical and mechanical 

functions of the equipment;

b. How the processes of ballot 

handling and reading are performed 

(paper-based systems);

e. How data are handled in the 

processor and memory units;

f. How data output is initiated and 

controlled;

g. How power is converted or 

conditioned;

h. How test and diagnostic 

information is acquired and used."

DS200 SMM v.1.3.7.0_10.17.2008

Added additional information at the 

end of the DS200 Concept of 

Operations and Electrical 

Information sections in Chapter 1: 

Introduction.

validated by SysTest 

as applicable to Unity 

3.2.0.0.

Verified that the 

DS200 SMM, dated 

10/17/08, Ch. 1, 

addresses 2.9.1.b -

how ballot handling 

and reading is 

performed; and 

2.9.1.f - how data 

output is controlled.

Verified M650 SMM, 

dated 10/17/08, Ch. 

1, addresses 2.9.1.f -

how data output is 

initiated and 

controlled; and 

2.9.1.g - how data 

output is controlled. 

13 12/30/08 K 

Swift

Document 

Defect

Closed Unity System 

Test Plan, v. 

3.2.0.0, 

11/21/08

From SysTest Unity 4.0 Testing -

Disc: 23

Cited documents do not provide test 

and verification specifications for:

Development test specifications.

V.2: 2.7.1 a, b, c, d, e

The vendor shall describe the plans, 

procedures, and data used during 

software development and system 

integration to verify system logic 

correctness, data quality, and 

security. 

a. Test identification and design, 

including:

1) Test structure;

2) Test sequence or progression; 

and

3) Test conditions;

a. Standard test procedures, 

including any assumptions or 

constraints;

b. Special purpose test procedures 

including any assumptions or 

constraints;

c. Test data; including the data 

source, whether it is real or 

simulated, and how test data is 

controlled; 

d. Expected test results; and

e. Criteria for evaluating test results.

5.15.2008 CHE - Added a 

statement addressing 

Development Test specifications to 

Unity Test Plan Test Strategy 

Overview, Section 2.1.1.

Accepted: K Swift, 

12/30/08

Reviewed the 

SysTest description 

history and verified 

Unity System Test 

Plan, version 

3.2.0.0, dated 

11/21/08 meets all 

the applicable 

requirements.

14 12/30/08 K 

Swift

Document 

Defect

Closed Unity System 

Test Plan, v. 

3.2.0.0, 

11/21/08

From SysTest Unity 4.0 Testing -

Disc: 24

Cited documents do not provide test 

V.2: 2.7.2 a, b, c, d, e, f, g, h

The vendor shall provide 

specifications for verification and 

validation of overall software 

10/17/08 T.Omel

Added a significant amount of 

detail to Section 2.1.1. If the VSTL 

feels this does not address this 

Accepted: K Swift, 

12/30/08

Reviewed the 

SysTest description 
VSTL Certification # pending

Page 143 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

and verification specifications for: 

National Certification test 

specifications.

performance. 

a. Control and data input/output;

b. Acceptance criteria;

c. Processing accuracy;

d. Data quality assessment and 

maintenance;

e. Ballot interpretation logic;

f. Exception handling;

g. Security; and

h. Production of audit trails and 

statistical data.

discrepancy, more specific 

information regarding the nature of 

the deficiency is required.

history and verified 

Unity System Test 

Plan, version 

3.2.0.0, dated 

11/21/08 meets all 

the applicable 

requirements of the 

VSS

15 12/30/08 K 

Swift

Document 

Defect

Closed ESSIM Sys 

Oper Proc, v. 

7.7.0.0, 

06/08/07

M650 Sys 

Oper Proc, 

05/19/08

HPM Sys 

Oper Proc, 

05/19/08

From SysTest Unity 4.0 Testing -

Disc: 26

Documentation of operating 

procedures are incomplete.

V.2: 2.8.5 a, b, c, f, g, h 

The vendor shall provide 

documentation of system operating 

procedures that meets the following 

requirements: 

a. Provides procedures required to 

initiate, control, and verify proper 

system operation;

b. Procedures that clearly enable the 

operator to assess the correct flow 

of system functions (as evidenced 

by system-generated status and 

information messages);

c. Procedures that clearly enable the 

operator to intervene in system 

operations to recover from an 

abnormal system state; 

f. Procedures and off-line operator 

duties (if any) if they relate to the 

initiation or termination of system 

operations, to the assessment of 

system status, or to the 

development of an audit trail;

g. Supports successful ballot and 

program installation and control by 

election officials, provides a detailed 

work plan or other form of 

documentation providing a schedule 

and steps for the software and ballot 

installation, which includes a table 

outlining the key dates, events and 

deliverables; and

h. Supports diagnostic testing, 

specifies diagnostic tests that may 

be employed to identify problems in 

the system, verifies the correction of 

maintenance problems; and isolates 

and diagnoses faults from various 

systems states. 

TMT - 10.17.2008 - ESSIM SOP -

Refer to Chapter 1: Overview -

"General Timeline for Election 

Preparation" heading for 

requirement G.

10.17.2008 - TMT - HPM SOP -

Refer to Chapter 1: Overview, 

section General Timeline for 

Election Preparation for the 

timeline. 

10.17.2008 - SS - M650 SOP -

Sample timeline added to Chapter 

1 of all files. 

Accepted: K Swift, 

12/30/08

Reviewed the 

SysTest description 

history and verified 

all other 

requirements have 

been previously 

validated by SysTest 

as applicable to Unity 

3.2.0.0.

Verified in HPM 

SOP, dated

10/17/08; M650 

SOP, dated 10/17/08 

the addition of the 

General Timeline in 

Ch. 1 to satisfy 

2.8.5.g.

16 12/30/08 K Docu- Closed AM Sys Oper From SysTest Unity 4.0 Testing - v.2: 2.8.6 a, b TMT - 10.17.2008 - Audit Manager Accepted: K Swift, 
VSTL Certification # pending

Page 144 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

Swift ment 

Defect

Proc, V 

7.5.0.0, 

06/08/07

DS200 Sys 

Oper Proc, 

Hardware V 

1.1.0.0, 

06/15/07

EDM Sys 

Oper Proc, V 

7.8.0.0, 

05/18/07

ERM Sys 

Oper Proc, V 

7.4.0.0, 

06/14/07

ESSIM Sys 

Oper Proc, V 

7.7.0.0, 

06/08/07

HPM Sys 

Oper Proc, V 

5.6.0.0

M650 Sys 

Oper Proc, V 

2.2.0.0, 

05/11/07

Disc: 27

System operating procedures are 

incomplete.

The cited documentation does not 

define the procedures required to 

support system acquisition and 

readiness testing and does not provide 

information on system maintenance, 

correction of defects, and incorporating 

hardware and new software releases. 

(EDM, ERM, ESSIM, HPM, M650)

The cited documentation does not 

define the procedures required to 

support system acquisition and 

readiness testing, or hardware 

upgrades. (DS200)

The cited documentation does not 

provide procedures for providing 

technical support, correction of defects, 

and for incorporating hardware 

upgrades and new software releases. 

(AM)

The vendor shall provide 

documentation of system operating 

procedures: 

a. Defines the procedures required 

to support system acquisition, 

installation, and readiness testing. 

These procedures may be provided 

by reference, if they are contained 

either in the system hardware 

specifications, or in other vendor 

documentation;

b. Describes procedures for 

providing technical support, system 

maintenance and correction of 

defects, and for incorporating 

hardware upgrades and new 

software releases.

SOP - Chapter 1: Introduction, 

Added a new heading, Operations 

Support Frequently Asked 

Questions, providing

information about the following.

• how the system is purchased

• how the system is installed

• setup of the system

• how a user can verify the system

• the training needed

• the checklist to be followed

TMT - 10.17.2008 - EDM SOP -

Chapter 1: Introduction, section 

Operations Support Frequently 

Asked Questions. Added a new 

heading, Operations Support 

Frequently Asked Questions, 

providing information about the 

following: how the system is 

purchased, how the system is 

installed, setup of the system, 

how a user can verify the system, 

the training needed, the checklist 

to be followed.

10.17.2008 SF -- Election 

Reporting Manager System Ops 

Procedures Chapter 1: Overview

Added a new heading, Operations 

Support Frequently Asked 

Questions, providing information 

about the following.

how the system is purchased

how the system is installed

setup of the system

how a user can verify the system

the training needed

the checklist to be followed

TMT - 10.17.2008 - ESSIM SOP -

Chapter 1: Overview, Added a new 

heading, Operations Support 

Frequently Asked Questions, 

providing information about the 

following: how the system is 

purchased, how the system is 

installed, setup of the system, how 

a user can verify the system the 

training needed, the checklist to be 

followed.

12/30/08

Reviewed the 

SysTest description 

history and verified 

all other 

requirements have 

been previously 

validated by SysTest 

as applicable to Unity 

3.2.0.0.

Verified the addition 

of "Operations 

Support Frequently 

Asked Questions in 

Ch. 1 for: M650 SOP, 

dated 10/17/08; AM 

SOP, dated 

10/17/08;DS200 

SOP, dated 

10/17/08; EDM SOP, 

dated 10/17/08; ERM 

SOP, dated 

10/17/08; ESSIM 

SOP, dated 

10/17/08; HPM SOP, 

dated 10/17/08 to 

meet 2.8.6.a

ESSIM SOP and 

HPM SOP, dated 

10/17/08 have been 

addressed to meet 

requirement 2.8.6.b 

(correction of 

defects) by listing the 

Error Messages 

chapters; New 

software releases 

has been addressed 

with the addition of 

the Operations 

Support Frequently 

Asked Questions 

section. System 

Maintenance is Not 

Applicable as these 

are software 

applications.
VSTL Certification # pending

Page 145 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

TMT - 10.17.2008 - M650 SOP -

Chapter 1: Introduction. Added a 

new heading, Operations Support 

Frequently Asked Questions, 

providing

information about the following.

• how the system is purchased

• how the system is installed

• setup of the system

• how a user can verify the system

• the training needed

• the checklist to be followed

17 12/30/08 K 

Swift

Document 

Defect

Closed QA Program 

SW & FW 

Verification, v. 

1.2.0.0, May 

10, 2007

QA Program 

Manufacturing

, v. 1.1.0.0, 

June 15, 2007

From SysTest Unity 4.0 Testing -

Disc: 43

Cited documents do not address 

quality conformance inspections or 

specify that the test date must be part 

of the test record.

V.2: 2.12.3 b

The vendor shall provide a 

description of its practices for quality 

conformance inspections that meet 

the requirements of Volume 1, 

Subsection 8.6. For each test 

performed, the record of tests 

provided shall include:

b. Test date

9/17/08 T.Omel QAP SW/FW

Item #1 - Included a passage in 

Sect.12 of the QAP SW/FW 

document, referencing that the 

practices for QC inspections can 

be found in detail in the individual 

product test cases, as well as the 

Unity Test Plan document.

10/17/08 MDN QAP 

MANUFACTURING

A description of quality 

conformance procedures appears 

in Section 7 for the QAP for 

manufacturing and Section 12 of 

the QAP for system software and 

firmware. Updated both of these 

sections for clarity. Flowcharts 

reflect the procedure for incoming 

parts and device QC. Physical 

conformance testing for HW items 

is executed by a contracted 

Nationally Recognized Test 

Laboratories who sends test 

results directly to the Voting 

System Test Laboratory (VSTL). 

Functional test results are 

executed according to procedures 

outlined in the Unity test plan and 

system test cases. Reference to 

both documents added to indicate 

this. These items are provided 

with the TDP.

Accepted: K Swift, 

12/30/08

Reviewed the 

SysTest description 

history and verified 

QAP Manufacturing 

Unity v. 3.2.0.0, 

dated 12/1/08, 

Section 7 has been 

updated to address

test date. Flowchart 2 

addresses inspecting 

parts. Flowchart 14 

addresses the QA 

tests performed.

Verified QAP SW/FW 

Unity v. 3.2.0.0, 

dated 12/1/08, 

Section 12 updated 

to reflect QC 

inspections are 

detailed in test cases 

and/or Test Plan. 

Test cases submitted 

provide the detailed 

steps that comprise 

quality conformance 

inspections.

18 12/30/08 K 

Swift

Document 

Defect

Closed Creating a 

Shortcut for 

Ballot on 

Demand 

(BOD) 

From SysTest Unity 4.0 Testing -

Disc: 284

Installation procedures have not all 

been submitted in TDP.

v.2: 1.3 e

Overall, qualification testing focuses 

on:

e. The documentation developed 

and maintained by the vendor to 

SS 10/17/08 -- Drivers for the 

listed SOP's are derived from the 

Operating System. ES&S does 

not develop drivers for USB 

devices. 

K Swift, 12/30/08

Reviewed the 

SysTest description 

history and verified 

all other 
VSTL Certification # pending

Page 146 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

8/29/07

Installing 

HPM (HPM) 

8/28/07

Installing 

Election 

Reporting 

Manager 

(ERM)

8/28/07

Installing 

EDM (EDM) 

8/28/07

Installing ESS 

Image 

Manger 

(ESSIM) 

8/28/07

Windows XP 

on a Dell 

Optiplex 

GX520 PC

Windows XP 

On Dell 

Optiplex 

GX520 

Document v. 

1.2, dated 

5/21/08

The listed documents 1) need to be 

included in the TDP, and 2) need to 

state what drivers and/or COTS 

software that must be installed on the 

system. 

support system development, 

testing, installation, maintenance 

and operation, as addressed by the 

TDP described in Volume II, Section 

2.

MDN 10/20/2008 WINXP Optiplex 

INSTALL GUIDES -- The Optiplex 

install guides were authored for 

two separate purposes. The file 

stored under \13_Other VSTL 

Reports \06_Build 

Environment\Install Guide, titled 

"WinXP-DellOptiplexGX520_INST" 

is a build install guide that 

assumes no network connectivity 

is allowed. Renamed the 

document file stored under 

13_Other VSTL Reports 

\15_Install 

Instructions\U4000_COTS Install 

Instructions from "IG, WinXP - Dell 

Optiplex GX520 PC" to "UIG, 

WinXP - Dell Optiplex GX520 PC." 

To explicitly designate this guide 

as an end user install guide. 

requirements have 

been previously 

validated by SysTest 

as applicable to Unity 

3.2.0.0.

Verified the Install 

procedures are 

included in the SOP 

documents. 

Confirmed receipt in 

the Unity 3.2.0.0 TDP 

of the renamed 

document "UIG, 

WinXP - Dell 

Optiplex GX520 PC", 

and document 

WinXPDellOptiplexGX520_I

NST

19 12/30/08 K 

Swift

Functional 

Defect

Closed ERM 7.4.0.0 From SysTest Unity 4.0 Testing -

Disc: 297

The District Canvass report has two 

different pages of totals. 

The first page of totals lists "Voting 

Precinct Totals through Sheriff Totals" 

down the left had column, but 

Governor/Lt. Gov totals, as the only 

contest on the top of the page; 

however, none of the totals match the 

tapes. The next page of totals that lists 

the 7 precincts, show accurate totals. 

We are unable to determine why the 

totals are different or what this page is 

V1: 2.2.2.1.c

To ensure vote accuracy, all 

systems shall:

c. Record each vote precisely as 

indicated by the voter and be able to 

produce an accurate report of all 

votes cast.

12/12 TJO: This report is designed 

for use in only one state. Statespecific functionality is not being 

tested by the VSTL.

Note: VSTL created an accurate 

.DST file, which is detailed in the 

ERM Operator's Guide. This 

corrected any issues. ES&S will 

update the System Limitations 

document and Operator's Guide.

07.07.2008 - MDN - Limitations 

Document Item 15 under Election 

Reporting Manager documents 

referenced limitation. Included in 

Accepted: K Swift, 

12/30/08 -

Reviewed the 

SysTest description 

history and verified a 

.DST file was created 

correctly and 

validated. 

Verified Config Mgmt 

Plan Unity v. 3.2.0.0, 

dated 11/21/08, has 

been updated to 

include System 

Limitations as a 

document the end
VSTL Certification # pending

Page 147 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

reporting. 

12/13/07, D. Valdez - As the VSTL was 

not aware the District Canvass report 

was state specific (applicable to CA 

only) the DST file was created and 

report was re-generated and all totals 

were verified to be accurate. Then the 

VSTL was notified the report was not 

supported. The discrepancy will 

remain open until vendor 

documentation can be updated, 

submitted, and verified.

July 08 documentation 

submission.

Sue - 09.29.08 - ES&S supplies 

the System Limitations Document 

to it's customers as part of the end 

user documents. Added System 

limitations documentation to the 

list of documents supplied to end 

users in the Configuration 

Management Plan (Section 7.1.5).

10.17.2008 SF -- Election 

Reporting Manager System 

Operations Procedures Chapter 

34: Canvass

Added a warning under the Names 

and District Totals heading that 

failure to create a .dst file before 

attempting to generate the report 

may result in inaccurate results 

and inconsistent formatting.

user will receive. 

Verified ERM SOP, 

dated 10/17/08, Ch. 

28: Canvass, the 

addition of the 

Warning message.

20 12/30/08 K 

Swift

Document 

Defect

Closed Configuration 

Management 

Plan, V. 

1.1.0.0, 

6/15/07

From SysTest Unity 4.0 Testing -

Disc: 317

A review of the TDP does not indicate 

that copies of the procedures used for 

module or unit testing have been 

submitted. 

v.2: 2.11.6 b

The vendor shall provide a 

description of the procedures and 

related conventions for the two 

audits required by Volume I, Section 

8.7.2.b.3 of the Standards. These 

requirements pertain to:

b. Functional configuration audit that 

verifies the system performs all the 

functions described in the system 

documentation.

Sue - 09.29.08 - ES&S submitted 

elections and data for SysTest to 

review on 9/12/08. ES&S awaiting 

SysTest review.

MDN - 10.24.2008- 1) Procedures 

for system testing are included in 

the Unity test plan and test cases 

for individual products. Added a 

cross reference to these 

documents to CM Plan Section 

7.2.1. 2)After demotion by QA, 

construction for re-work is reinitialized and follows the same 

process as initial construction. 

The developer peer review 

described in CM Plan section 7.2.1 

occurs for both bugs and 

enhancements after the 

"Construction Complete" phase of 

both workflows. This occurs for 

both initial work and re-work (after 

demotion by QA). See Sections 

9.3.1 and 9.3.2.

Accepted: K Swift, 

12/30/08

Reviewed the 

SysTest description 

history and verified 

1) Config Mgmt Plan 

Unity v. 3.2.0.0, 

dated 11/21/08, 

Section 7.2.1 has 

been updated with 

information regarding 

Unit/Module Testing, 

Test Procedures, 

Test Cases, and Test 

Records. The 

procedures for 

integration and 

system testing have 

been submitted in the 

TDP.

2) Verified flowcharts 

in section 9.3.1 and 

9.3.2 contain 

construction reviews. 

21 12/30/08 K 

Swift

Document 

Defect

Closed Configuration 

Management 

Plan, V. 

1.1.0.0, 

6/15/07

From SysTest Unity 4.0 Testing -

Disc: 318

In the test cases submitted, the 

following have failed and do not show 

v.2: 2.11.6 b

The vendor shall provide a 

description of the procedures and 

related conventions for the two 

audits required by Volume I, Section 

10.24.2008 TJO-- Regarding iVIM, 

Audit Manager and ESSIM; 

incomplete information for each of 

these test cases was an accidental 

omission from the original 

Accepted: K Swift, 

12/30/08

Reviewed the 

SysTest description 

history and verified 
VSTL Certification # pending

Page 148 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

them to be retested: 

AM - 1.0

EDM - 2.6.5

ERM and HPM - None of the test 

cases have been executed

ESSIM - 1) Edit menu, 2) Load 

template, 3) Configuring Advanced 

Ballot Option

8.7.2.b.1 of the Standards. These 

requirements pertain to:

b. Functional configuration audit that 

verifies the system performs all the 

functions described in the system 

documentation.

documents. The missing 

information has been added to 

each test case. The cases in 

question have not been re-run and 

the listed dates are still accurate.

Regarding EDM; the functionality 

described in test case 2.6.5 is not 

supported by EDM in Unity 4.0.0.0 

(automated candidate name 

formatting through the merge 

preferences window). Removed 

the test case from the document 

and documented this issue as a 

system limitation.

all other 

requirements have 

been previously 

validated by SysTest 

as applicable to Unity 

3.2.0.0.

iBeta has verified 

that the Config Mgmt 

Plan Unity v. 3.2.0.0, 

dated 11/21/08, 

contains adequate 

content to prove out 

regression testing is 

performed by the 

vendor. Reviewed 

AM TC Spec, dated 

8/26/08, ESSIM TC 

Spec, dated 

10/17/08, the test 

cases identified in 

the description have 

passed. 

Verified Unity 3.2.0.0 

System Limits, dated 

12/9/08 has been 

updated with 

automated candidate 

name formatting 

through the merge 

preferences window 

is not supported.

22 12/30/08 K 

Swift

Document 

Defect

Closed QA Program, 

SW & FW 

Verification, v. 

1.2.0.0, Aug 

21, 2007

QA Program, 

Manufacturing

, Ver. 1.1.0.0, 

Aug 21, 2007

From SysTest Unity 4.0 Testing -

Disc: 339

The requirement is not being met for 

software/firmware/hardware in general, 

as detailed by item #1, and a specific 

case is cited in item #2.

1) The documents do not contain a 

procedure for maintaining data and 

records to verify the quality inspections 

and tests. 

2) In the QA Manufacturing document, 

Flow chart 3, Auditing Process, details 

an audit of the ECO Process, however 

at the bottom of the page, it is stated 

that it is "Currently not practiced". 

v.2: 2.12.3 a, b, c, d

The vendor shall provide a 

description of its practices for quality 

conformance inspections that meet 

the requirements of Volume I, 

Section 7.6 of the Standards. For 

each test performed, the record of 

tests provided shall include:

a. Test location;

b. Test date;

c. individual who conducted the test; 

and

d. Test outcomes.

10.17.2008 MDN -- Updated QAP 

for SW/FW submitted with current 

revision. Procedures for document 

archiving and records retention 

appear under Section 2.5 and 

2.2.2.

Accepted: K Swift, 

12/30/08

Reviewed the 

SysTest description 

history and verified 

all other 

requirements have 

been previously 

validated by SysTest 

as applicable to Unity 

3.2.0.0.

Verified QAP SW/FW 

dated 12/1/08, 

Sections 2.2.2 and 

2.5 have been 

updated to address 

archiving and records 

retentions.

23 12/30/08 K 

Swift

Document Closed DS200 Test 

Case Spec, 

From SysTest Unity 4.0 Testing - Disc: 

348

v.2: 2.12.3 a, b, c, d

The vendor shall provide a 

10/16/08 T.Omel

2. M650 --

Accepted: K Swift, 

12/30/08
VSTL Certification # pending

Page 149 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

Defect FW v. 1.2.0.0, 

HW v. 1.2, 

Test v. 1.0, 

Aug 20, 2007

EDM Test

Case Specs, 

SW v. 

7.8.0.0j, Test 

Case 1.0, Aug 

20, 2007

QA Program 

Test Cases, 

Unity 4.0, v. 

7.4.0.0, June 

6, 2007 (file 

name: ERM 

TC v. 

7.4.0.0_6.06.

2007.pdf)

ESS Image 

Manager Test 

Case Spec 

SW v. 

7.7.0.0f, Test 

Case 1.0, Aug 

20, 2007

HPM Unity v. 

4.0, Product 

v. 5.6.0.0, 

Test Case v. 

1.0, June 15, 

2007

M650, Test 

Case Spec, 

FW v. 2.2.1.0, 

HW v. 1.1, 

Test Case 

1.0, Aug 20, 

2007

The test cases are not complete as 

follows: 

DS200 TC v.1.2_8.20.2007

- TC 7.1 states "Test applies: N", and 

there is no date filled out, however the 

test case was executed, as the 

Pass/Fail criteria has been completed.

EDM TC v.7.8.0.0j_8.20.2007

- TC 3.1 does not have all the 

outcomes for each step filled out. 

There is a test step that is "N/A", with 

no explanation why it is not applicable. 

(This is present in other test cases as 

well.)

- TC 3.8 contains a test step that 

contains no Expected Result or 

Pass/Fail criteria. There are test steps 

that are "N/A", with no explanation why 

it is not applicable.

- TC 4.1 states "Test applies: N", there 

is no date or tester filled out, however 

the test case was executed, as the 

Pass/Fail criteria has been completed.

- TC 4.8 states "Test applies: N", 

however name, date, location and 

Pass/Fail criteria has been completed. 

- Some of the test cases do not list the 

build tested or are against a previous 

version of the most recent version (j).

- There are highlighted and question 

mark items that don't appear to have 

been resolved.

- There are test cases that have the 

Expected Results and Actual Results 

listed as "SAME". It is unclear what 

that specifies.

- TC 6.1 does not have all the 

outcomes for each step filled out.

ERM TC v. 7.4.0.0_6.06.2007 - This 

document does not meet any of the 

requirement.

ESSIM TC v.7.7.0.0f_8.20.07

- TC 'Edit Menu' states "Test applies: 

N", however name, date and location 

has been completed. Pass/Fail 

indicates a bug.

- TC 'Candidates Style Sheets' does 

not have all the outcomes for each step 

filled out.

HPM TC v.5.6.0.0_6.15.2007

- The Introduction contains the wrong 

description of its practices for quality 

conformance inspections that meet 

the requirements of Volume I, 

Section 7.6 of the Standards. For 

each test performed, the record of 

tests provided shall include:

a. Test location;

b. Test date;

c. individual who conducted the test; 

and

d. Test outcomes.

- Many test cases (e.g. 4.11) have 

sections that do not include the 

"Actual Results" and "Pass/Fail" 

categories. These sections are 

simply helpful addenda that aide 

the tester in setting up the test in 

other areas of the system.

- Corrected an empty field on Page 

169.

- Several test cases in Section 10 

of the document do not have 

complete information. This is 

because Section 10 covers 

Specific Customer Special 

Handling Scenarios. These 

scenarios usually address 

situations for individual customers 

or states. ES&S does not intend to 

include these tests in our federal 

certification effort. 

3. ESSIM --

- Completed missing information 

on Pages 7, 8, 30, and 84.

Reviewed the 

SysTest description 

history and verified 

all other 

requirements have 

been previously 

validated by SysTest 

as applicable to Unity 

3.2.0.0.

Verified M650 TC, 

dated 10/17/08, test 

cases 4.6, 4.11, page 

169, and Section 10 

test cases have been 

updated as 

applicable.

Verified ESSIM TC, 

dated 10/17/08; all 

test cases have 

passed and 

information is 

complete.
VSTL Certification # pending

Page 150 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

HPM version.

- This document does not meet any of 

the requirement.

- TC 'Question Style Sheets' does not 

have all the outcomes for each step 

filled out.

- TC 'Creating Precinct Headers' does 

not have all the outcomes for each step 

filled out.

Model 650, Test Case Specification, 

Firmware Version 2.2.1.0, HW v. 1.1, 

Test Case 1.0, August 20, 2007

4.6 & 4.11: Not all Steps contain 

Pass/Fail criteria

24 12/30/08 K 

Swift

InformationalClosed Unity 4.0.0.0 

Test Plan Aug 

20, 2007

From SysTest Unity 4.0 Testing -

Disc: 355

1) It is unclear from the documentation 

how to interpret the column heading 

"No Test" in section 2.2.2.1.

2) Section 2.2.2.2, indicates that 

testing was run against the firmware 

version 1.1.0.0 for the DS200. This 

certification is for firmware version 

1.2.0.0.

3) The functionality section of the 

document, 2.3, does not address AM 

or ESSIM or testing the full suite of 

hardware components for this 

certification.

10/14/08 T.Omel

In Section 2.4.3 of the Unity Test 

Plan, replaced all references to 

OPTIM with ESSIM. 

Accepted: K Swift, 

12/30/08

Reviewed the 

SysTest description 

history and verified 

all other 

requirements have 

been previously 

validated by SysTest 

as applicable to Unity 

3.2.0.0.

Unity TP v. 3.2.0.0, 

11/21/08; verified all 

references to OPTIM 

have been corrected.

25 12/30/08 K 

Swift

Document 

Defect

Closed Unity 4.0.0.0 

Test Plan Aug 

20, 2007

From SysTest Unity 4.0 Testing -

Disc: 359

1) Section 2.5.5 does not include the 

DS200 in the test cases, so it is not 

possible to assess the verification and 

validation of overall performance. 

2) Requirements to determine 

Processing Accuracy has not been 

addressed.

V.2:2.7.2 c 

The vendor shall provide 

specifications for verification and 

validation of overall software 

performance. 

c. Processing accuracy;

9/11/2008 -- T. Omel

-Please provide clarification on the 

inability to "assess the verification 

and validation of overall 

performance." The discrepancy 

cites that the DS200 is not in 

Section 2.5.5 -- is the test lab 

requesting further information on 

how to assess overall performance 

of the DS200 with relation to the 

C1LONE test election? or 

information to assess the overall 

performance of the entire Unity 

system? 

Accepted: K Swift, 

12/30/08

Reviewed the 

SysTest description 

history and verified 

all other 

requirements have 

been previously 

validated by SysTest 

as applicable to Unity 

3.2.0.0.

Verified in Unity TP 

v. 3.2.0.0, 11/21/08, 

Section 2.3.1 and 

2.3.3 contain a note 

that processing 

accuracy is validated 

by scanning ballots 

and vote 

consolidation.
VSTL Certification # pending

Page 151 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

26 12/30/08 K 

Swift

Document 

Defect

Closed Unity 4.0.0.0 

Test Plan Aug 

20, 2007

From SysTest Unity 4.0 Testing -

Disc: 361

Production of audit trails and statistical 

data has not been addressed.

V.2:2.7.2 h

The vendor shall provide 

specifications for verification and 

validation of overall software 

performance. 

h. Production of audit trails and 

statistical data.

9/11/2008 --T.Omel

Added the following to 1.2.2:

To ensure that correct audit and 

statistical records are kept as 

actions are performed during the 

election process. Testers are to 

regularly keep external logs of 

their activities and compare these 

to the various audit logs that are 

maintained by the Unity software 

and tabulators.

Accepted: K Swift, 

12/30/08

Reviewed the 

SysTest description 

history and verified 

all other 

requirements have 

been previously 

validated by SysTest 

as applicable to Unity 

3.2.0.0.

Verified in Unity TP 

v. 3.2.0.0, 11/21/08, 

Section 2 has been 

updated per vendor 

comment.

27 12/30/08 K 

Swift

Document 

Defect

Closed ERM System 

Functionality 

Description v. 

7.4.0.0 Nov 

15, 2007

HPM System 

Functionality 

Description v. 

5.6.0.0

M650 System 

Ops 

Procedures 

FW v. 2.2.1.0, 

HW v. 1.1 

Nov 16, 2007

ERM System 

Ops 

Procedures v. 

7.4.0.0 Nov 

15, 2007

DS200 

System Ops 

Procedures 

HW v. 1.2.0.0, 

FW v. 1.2.0.0 

Nov 8, 2007

From SysTest Unity 4.0 Testing -

Disc: 372

1) The ERM and HPM SFD documents 

states that "provisional ballots are

handled by each individual tabulator", 

however, the SFD documents for the 

DS200 and M650, state that it is 

handled procedurally outside the scope 

of each tabulator. This is inconsistent. 

2) The various tabulator SOP 

documents do not indicate how the 

scanners track or report on paper 

provisional ballots. 

3) The ERM SOP document, does not 

detail the reporting/totals of provisional 

ballot totals regardless of whether they 

are paper or DRE.

4) The VSTL can not locate in the TDP 

how paper provisional ballots are 

handled in the election definition or for 

reporting.

V1: 2.2.8.2 o

The TDP accompanying the system 

shall specifically identify which of the 

following items can and cannot be 

supported by the system, as well as 

how the system can implement the 

items supported:

o. Provisional or challenged ballots.

V1: 2.5.2

All systems shall provide a means to 

consolidate vote data from all polling 

places, and optionally from other 

sources such as provisional ballots

10.17.2008 SF -- Election 

Reporting Manager SOP Chapter 

1: Overview

Added information under the 

Provisional Voting heading to 

clarify that provisional votes are 

tabulated according to jurisdiction 

regulations.

TMT - 10.17.2008 - DS200 SOP 

Chapter 1: Introduction. Added 

information under the Provisional 

Voting heading to clarify that 

provisional votes are tabulated 

according to jurisdiction 

regulations.

TMT - 10.17.2008 - M650 SOP -

Chapter 1: Introduction. Added 

information under the Provisional 

Voting heading to clarify that 

provisional votes are tabulated 

according to jurisdiction 

regulations.

10.20.2008 - MDN ERM SFD--

Updated ERM SFD Section 2.1.7.2 

to indicate support for provisional 

and added an expanded 

description of the provisional 

voting process. No special 

functionality is included in ERM for 

handling provisional votes. 

Segregating and handling 

provisional votes is an entirely 

Accepted: K Swift, 

12/30/08

Reviewed the 

SysTest description 

history and verified 

all other 

requirements have 

been previously 

validated by SysTest 

as applicable to Unity 

3.2.0.0, or iBeta has 

determined they 

have been met.

ERM SOP, DS200 

SOP, M650 SOP, 

dated 10/17/08, 

verified provisional 

ballot handling is 

clarified in Ch. 1.

ERM SFD, dated 

11/20/08, Section 

1.1.7.2 has been 

updated to clarify the 

handling of 

provisional ballots.

M650 and DS200 

SFD, dated 11/20/08, 

still state that 

provisional ballots 

are handled outside 

the scope of the 

tabulator, but 
VSTL Certification # pending

Page 152 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

administrative process and there is 

no difference in how provisional 

votes are processed in ERM 

compared to election day votes. 

Provisional ballots are identified 

during voting and segregated from 

approved ballots prior to being 

counted on an ES&S vote 

tabulator. All provisional ballots 

are then considered for 

approval/rejection by a resolution 

or review board according to the 

jurisdiction's election rules. 

Approved ballots are then counted 

normally on an ES&S ballot 

tabulator and read as standard 

election day results by ERM in 

exactly the same manner as all 

other votes. Rejected ballots are 

never tabulated. The ERM system 

includes no special functionality for 

handling provisional voting and 

does not recognize provisional 

balloting vs. early voting or 

election day ballots. See the 

ERM SOP for a description of how 

paper ballot results are passed to 

ERM. 

because of the 

clarification in ERM, 

this is a non-issue.

28 12/30/08 K 

Swift

Functional 

Defect

Closed M650 v. 

2.2.1.0

From SysTest Unity 4.0 Testing - Disc: 

411

Some valid ballots were rejected due to 

a simple manipulation of the election 

definition data in one file. 

We obtained the M650 zip disk with a 

set of valid election definition files, than 

manipulated the data in the *.PRF file 

using a Hex editor. We changed the 

last value within the file from "0007" to 

a value of "0999", and then resaved 

the data back to the zip disk. We then 

inserted the zip disk with the changed 

file of the set of election definition files 

into the M650, powered up the M650 

and loaded the election definitions to 

the M650.

During 3 test runs, the number of valid 

ballots which registered as good ballots 

when read by the M650 after loading 

the manipulated data set election 

V1: 2.2.2.1.c

To ensure vote accuracy, all 

systems shall:

c. Record each vote precisely as 

indicated by the voter and be able to 

produce an accurate report of all 

votes cast.

Sue - 10.02.08 - Awaiting SysTest 

to rerun security test based on 

earlier vendor comments. 

Sue - 04.03.09 - Stress, Volume 

and Regression ballots are printed 

with a security mark. These 

elections could be used to test the 

referenced functions. 

Accept 5/5/2009 SJ: 

We can not 

reproduce this test 

by modifying the 

.PRF file. The test 

results are not 

related to 

modification of this 

.PRF file. With or 

without modification 

in the election 

definition loaded on 

the M650 ,while 

scanning ballots ,if 

the ballot is folded or 

torn ,M650 is not 

able to count it and 

displays the error 

"Invalid Ballot Style 

or Type Code

Mismatch." 

Scanned 7 ballots 

with manipulation in 
VSTL Certification # pending

Page 153 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

definition files was different.

Known: Total ballots were 28; 

Registered as Good ballots, when read 

through M650 were 25; Registered as 

Bad ballots with errors, when read 

through M650 were 3.

1st test run: 

Total ballots: 28 ballots; Ballots 

registered as Good ballots, when read 

through M650 were: 23; Registered as

Bad ballots with errors, when read 

through M650 were: 5

Errors that came up for the bad ballots 

were either: 1) "Could not Read Code 

Channel!" or 2) "Invalid Ballot Style", 

"Type Code Mismatch"

2nd test run:

Total ballots: 28 ballots; Ballots 

registered as Good ballots, when read 

through M650 were: 24; Registered as 

Bad ballots with errors, when read 

through M650 were: 4 

Errors that came up for the bad ballots 

were: "Could not Read Code Channel!" 

(this error only for all bad ballots)

3rd test run:

Total ballots: 28 ballots; Ballots 

registered as Good ballots, when read 

through M650 were: 22; Registered as 

Bad ballots with errors, when read 

through M650 were: 6

Errors that came up for bad ballots 

were either: 1) 'Invalid Ballot Style", 

"Type Code Mismatch" or 2) "Could not 

Read Code Channel" 

The errors that were generated when 

the ballots were rejected were not 

consistent. The outcome is that the 

tabulation of valid ballots was affected, 

affecting the accuracy of the voting.

election 

definition(.PRF file 

0007 to 0999) , 

performed 3 runs, 21 

ballots are counted 

with out error.

29 12/30/08 K 

Swift

Functional 

Defect

Closed M650 v. 

2.2.1.0

From SysTest Unity 4.0 Testing - Disc: 

427

Fake ballots copied from un-voted 

original ballots on a laser color copier 

printer (XEROX WORKCENTRE 7665) 

were accepted by the M650.

V1: 6.1 

This section describes essential 

security capabilities for a voting 

system, encompassing the system‘s 

hardware, software, 

communications, and 

Sue - 10.02.08 - ES&S supplied 

SysTest with 25 blank ballots with 

the security mark for review on 

09.23.08.

Sue - 04.03.09 - Stress, Volume 

Accepted: 5/8/2009 

SJ: Tried photo copy 

of the ballot in black 

and white from 

cannon model 

Imagescanner 5070 
VSTL Certification # pending

Page 154 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

We obtained three un-voted original 

ballots from PRI02 election and copied 

several variations of the original unvoted ballots. This was due, to not 

being able to copy the entire original 

ballots as they were, the copy of the 

original ballots had the very top 

sections of the markings or the very 

bottom sections of the markings 

chopped off by the laser copier 

machine due to not being able to copy 

the whole original ballot. There was 

one copy made from an original unvoted ballot that we did successfully 

capture the whole ballot on the copier 

but this was only a copy of the front 

and not the back side. We also printed 

some in color and some in black and 

white only.

Variations copied (with some portions 

of the ballot chopped off.) and scanned 

on the M650 are:

Variation 1: This was printed in black 

and white. We copied the front only, 

the back side of the copy was blank, 

top section markings were fine but the 

bottom section markings were 

chopped. M650 rejected this ballot.

Variation 2: This was printed in black 

and white. We copied the front only, 

the back side of the copy was blank, 

top section markings were chopped but 

the bottom section markings were fully 

copied. M650 rejected this ballot.

Variation 3: This was printed in black 

and white. We copied the front and the 

back to create a double sided copy of 

the original ballots. The top section 

had been chopped off a little in the 

front and the back side, but the bottom 

section was fully copied on the front, as 

well as the back side of the copy. M650 

accepted this ballot.

Variation 4: This was printed in black 

and white. We copied the front and the 

documentation. The Standards 

recognize that no predefined set of 

security standards will address and 

defeat all conceivable or theoretical 

threats. However, the Standards 

articulate requirements to achieve 

acceptable levels of integrity, 

reliability, and inviolability.

Ultimately, the objectives of the 

security standards for voting 

systems are:

 ̈ To protect the system from 

intentional manipulation and fraud, 

and from malicious mischief; 

 ̈ To identify fraudulent or erroneous 

changes to the system; 

and Regression ballots are printed 

with a security mark. These 

elections could be used to test the 

referenced functions. 

to reproduce gray 

scale.

1. One side copied 

ballot, rejected by 

M650 with an error 

invalid ballot style.

2. Copied two sides 

,Rejected by M650 

with an error "Invalid 

row count, Row 28"
VSTL Certification # pending

Page 155 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

back to create a double sided copy of 

the original ballots. The top section 

was fully copied on the front, as well as 

the back side of the copy, but the 

bottom section on the front and the 

back side of the copy was chopped off 

a little. M650 accepted this ballot.

Variation 5: This was printed in color. 

We copied the front and the back to 

create a double sided copy of original 

ballots. The top section had been 

chopped off a little in the front and the 

back side, but the bottom section was 

fully copied on the front, as well as the 

back side of the copy. M650 accepted 

this ballot.

Variation 6: This was printed in color. 

We copied the front and the back to 

create a double sided copy of original 

ballots. The top section was fully 

copied on the front, as well as the back 

side of the copy, but the bottom section 

on the front and the back side of the 

copy was chopped off a little. M650 

accepted this ballot.

After we copied them, we filled in the 

fake ballots using a BIC pen with blank 

ink (BIC Pen was a round stic, medium 

point, 1.0 mm, ballpoint, white barrel, 

black ink). These BIC pens are 

abundant and are easily found in most 

stores.

We loaded the election definition files 

to the M650 from a zip disk, zeroed out 

the totals and ran the fake ballots 

through. Out of 14 fake ballots, we 

registered 7 ballots as good. Those 

that passed were the double sided 

ones that were copies of the original 

ballots of the front and the back. One 

copy that was copied only of the front 

passed also. Fake ballots that are 

intentional and accepted by the M650 

is fraud and can cause issues in the 

accuracy of the votes.

30 12/30/08 K 

Swift

Document Closed Software 

Design & 

From SysTest Unity 4.0 Testing -

Disc: 429

v.2: 2.5.9.2.b

For each interface identified in the 

Sue - 10.02.08 - ES&S provided 

SysTest with 25 blank ballots with 

Accept - K Swift -

3/12/09 
VSTL Certification # pending

Page 156 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

Defect Spec, EDM, v. 

7.8.0.0, Mar 

4, 2008

Quite a few of the field limitations 

provided in the SDS are inaccurate 

(Example: Election description, Full 

path to ballot definition file, Vote for 

phrase when only 1, Vote for phrase). 

The limitation states 80; however, 

when tested the fields allowed fewer 

than 80 or greater than 80.

Pg. 71 shows a "Merge with" field; 

however, the "Merge with" field does 

not exist.

The "Add to type and Add to 

sequence" field allowed alpha and 

numeric; however the "Type" states 

numeric only.

Reject 12/30 08 K Swift

Response does not clearly address the 

issue

system overview, the vendor shall 

provide information that describes:

b. Characteristics of individual data 

elements that the interfacing 

entity(ies) will provide, store, send, 

access, receive, etc., such as:

1) Names/identifiers;

2) Data type (alphanumeric, integer, 

etc.);

3) Size and format (such as length 

and punctuation of a character 

string);

7) Priority, timing, frequency, 

volume, sequencing, and other 

constraints, such as whether the 

data element may be updated and 

whether business rules apply;

security marks for review on 

09.23.08.

CJ - 02.16.09 - 1) Regarding the 

field limitations: These are not 

issues as the application's table 

entries reflect the length of the 

container (variable) within the 

application. The following text was 

added to the Notes section of 

Section 9.2.: "Type and 

Size/Format entries in the tables 

below reflect the internal 

programming element types, 

lengths, with internal validation. 

These may not necessarily reflect 

initial user input. When dialogs are 

re-displayed, all data is redisplayed with appropriate length 

validation applied."

CJ - 02.16.09 - 2) The "Merge 

Width" field reference was 

removed in an earlier release of 

this document. Note: This 

document version contains a new 

Page 1 graphic and introduction, 

causing your earlier page 

references to become inaccurate. 

To locate your referenced section, 

search for "Merge Preferences 

Dialog".

CJ - 02.16.09 - 3) Add To Type 

and Add to Sequence (and Add to

Split) data entry fields - These 

fields are validated after exiting (as 

are other fields). When exiting a 

validated field or ultimately, a 

dialog, a pop-up dialog will be 

displayed if validation fails. In the 

case for these fields after entering 

alphanumeric data, the pop-up 

dialog will display "Please enter an 

integer".

Verified in EDM SDS 

dated 2/23/09, the 

removal of the 

'Merge Width' field. 

The Election 

description, Vote for 

phrase when only 1, 

Vote for phrase, and 

Add to type and Add 

to sequence fields 

will be tested in the 

Regression Test 

Case. Full path to 

ballot definition file 

and The Election 

description will be 

tested in Volume 7 

Test Case.

K Swift, 12/30/08 -

Reject -

31 12/30/08 K 

Swift

Document 

Defect

Closed System 

Security 

Spec, v. 

4.0.0.0, Feb 

29, 2008

From SysTest Unity 4.0 Testing - Disc: 

435

The System Security Specification 

does not provide sufficient guidance for 

the steps necessary to implement the 

security settings specified in Chapter 

12 Baseline for Windows XP SP2, 

which are required to secure the 

Windows installation in a network 

v.2: 2.6.4

The vendor shall provide a detailed 

description of the system capabilities 

and mandatory procedures for 

purchasing jurisdictions to ensure 

secure software (including firmware) 

installation to meet the specific 

requirements of Volume I, Section 

6.4 of the Standards. This 

10.27.2008 PMZ -- This item was 

discussed in a weekly phone 

conference during August. The 

ITA was to review the discussion 

and respond. To date no 

response has been given.

ES&S maintains its belief that it 

already provides the necessary 

information for the secure 

kgw 2/18/2009 

Rejected

4/12009 Accepted K 

Wilson Closed per 

delivery of Hardening 

Procedures for the 

Election 

Management System 
VSTL Certification # pending

Page 157 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

environment. 

The Ch. 12 security settings appear to 

be following a recommended practice 

by the operating system provider, 

Microsoft, which recommend industry 

best practices. Either similar 

documentation should be provided for 

the steps necessary to implement the 

security settings of Ch 12, or reference 

should be made to industry standard 

documents to ensure secure 

configuration of Windows XP SP2. 

2/18/09 Rejected Reviewer concurs w/ 

SysTest. These settings appear to 

span Local Security Policy, Log File 

Properties, Registry settings, System 

Properties. While many of these 

settings can be found online, the 

lifetime of this product may long 

exceed the supported lifetime of the 

OS and Microsoft has been known to 

remove support information from its 

website leaving even the experienced 

system administrator unable to recall or 

find the appropriate setting. 

Furthermore, smaller jurisdictions may 

not have the available expertise to 

make these settings, leaving systems 

under their care open to security 

vulnerabilities. We will be unable to 

commence Security Testing on 

systems that cannot be hardened 

according to the vendor 

documentation. Furthermore in regards 

to the specific settings noted: In the 

case of Data Execution Prevention, the 

term "enabled" is ambiguous since it is 

always enabled, it is just a matter of 

degree (unless your path and my path 

differ). 

information shall address software 

installation for all system 

components.

configuration by a client's trained 

system administration staff; we 

specifically require that trained 

system administration staff must 

perform the configuration.

PC dated 3/27/2009. 

32 12/30/08 K 

Swift

Functional 

Defect

Closed DS200 –

Election Day 

Checklist 

Rev.1.3.7.0 , 

05/09/2008

Model DS200 

From SysTest Unity 4.0 Testing -

Disc: 453

Operational Checklist Test:

While performing the DS200 

Acceptance Testing Criteria on the 

DS200, when the step listed below was 

performed;

V1: 2.3.4.1.a

All systems shall provide the 

capabilities to:

a. Verify that voting machines or 

vote recording and data processing 

equipment, precinct count 

Sue - 10.02.08 - The DS200 had a 

cracked image sensor (CIS). An 

ES&S technician replaced the 

sensor and the DS200 functioned 

correctly. SysTest logged the 

steps the ES&S technician took.

Accept JG 05/21/09 

Did not encounter 

these error during 

execution of testing
VSTL Certification # pending

Page 158 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

– SN: 

ES010737000

2, FW Ver. 

1.2.0.0

―Ensure the test ballots scan in all four 

orientations.‖ When the test ballot was 

scanned in the third orientation, one of 

the three ballot errors occurred. The 

ballot errors that appeared are listed 

below;

Ballot Error: Invalid CC Sequence 

(107) Please see Election Official

Ballot Error: Invalid CC Spilt (109)

Please see Election Official

Ballot Error: Mis-Matched Marks (100)

Missed orientation marks, turn over & 

try again

After referring to the SOP document for 

the DS200, the vendor was notified, 

per the documentation. 

equipment, and central count 

equipment are properly prepared for 

election, and collect data that 

verifies equipment readiness.

33 12/30/08 K 

Swift

Functional 

Defect

Closed Model DS200 

– SN: 

ES010737000

2, FW Ver. 

1.2.0.0

From SysTest Unity 4.0 Testing -

Disc: 454

4.0 Heavy Test 1 While performing the 

4.0 Heavy Test 1 on the DS200 and 

during the scanning of ballots, after 30 

ballots had been scanned, the internal 

rollers of the DS200 stopped 

functioning and the DS200 stopped 

scanning.

T. Fuller 06/5/08 – After a technician 

from the vendor company performed 

repairs on the DS200, a Readiness 

Test was performed, and the DS200 

passed. However, once testing started 

again, the internal rollers of the DS200 

stopped functioning and the DS200 

stopped scanning after 18 ballots had 

been scanned.

V1: 2.3.4.1.d

All systems shall provide the 

capabilities to:

d. Verify that hardware and software 

function correctly.

Sue - 10.02.08 - When the ES&S 

technician replaced the CIS, he 

accidentally pinched a cable when 

putting the DS200 cover back on. 

He reopened the machine and 

removed the pinched cable. 

SysTest logged the steps the 

ES&S technician took.

Accept JG 05/21/09 

Did not encounter 

these error during 

execution of testing

34 12/30/08 K 

Swift

Functional 

Defect

Closed ERM V. 

7.5.0.0

From SysTest Unity 4.0 Testing -

Disc: 473

It was discovered that the ifc file was 

not copied to \ELECDATA directory 

and not uploaded to the HPM. (This 

was for an IVO only election).

The HPM SOP dated 5/19/08 states 

"After you copy the ballot interface file 

(.IFC) to the \ELECDATA directory on 

the hard disk, you are ready to create 

an empty election shell. When you 

V1: 4.4.4.a, b

Voting systems shall meet these 

reporting requirements by providing 

software capable of obtaining data 

concerning various aspects of vote 

counting and producing reports of 

them on a printer. At a minimum, 

vote tally data shall include:

a. Number of ballots cast, using 

each ballot configuration, by 

tabulator, by precinct, and by 

political subdivision;

10.17.2008 SF -- This issue is 

addressed in the HPM SOP as the 

resolution for the problem is an 

upstream change in HPM 

procedures rather than a change 

to HPM. 

HPM SOP Chapter 7: Create New

Added information to a note on 

page 37 clarifying that if your 

election is iVotronic only, you 

should copy the .IFC file to the 

Accepted: K Swift, 

12/30/08

Reviewed the 

SysTest description. 

Verified HPM SOP, 

dated 10/17/08; Ch. 

7: Create New, note 

has been updated to 

include that the .IFC 

file must be copied to 

the \Elecdata 

directory for an IVO 
VSTL Certification # pending

Page 159 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

import the .IFC file, the empty shell will 

then be filled with the election 

information from the .IFC file. 1. From 

the Election menu, click Create New. 

2. Type the name (up to eight 

characters) of the new election." 

The EDM SOP dated 5/19/08 does not 

reference the copying of the ifc file to 

the \ELECDATA directory.

b. Candidate and measure vote 

totals for each contest, by tabulator;

ELECDATA directory for import in 

HPM.

only election, not 

imported.

35 12/30/08 K 

Swift

Functional 

Defect

Closed ERM v. 

7.5.0.0 

40HTEST1 

TC

From SysTest Unity 4.0 Testing -

Disc: 475

Numbered Key - Districts report is 

showing two M650 groups and the iVo 

PEB group does not appear; however, 

the PEB totals match the totals 

appearing alongside the second M650 

group totals (it appears that the 'label' 

is incorrect and should read 'IVO 

PEB"). I then went into add/change 

groups and switched the location of the 

M650 group and the M100 group, 

regenerated the report and we now 

show two M100 groups and again the 

iVo totals appeared under the second 

M100 group. It seems the report is 

mimicking the name in group three into 

group four, but applying the correct

totals. Copies of the report and screen 

shot of the add/change groups faxed to 

vendor.

V1: 4.4.4.a, b

Voting systems shall meet these 

reporting requirements by providing 

software capable of obtaining data 

concerning various aspects of vote 

counting and producing reports of 

them on a printer. At a minimum, 

vote tally data shall include:

a. Number of ballots cast, using 

each ballot configuration, by 

tabulator, by precinct, and by 

political subdivision;

b. Candidate and measure vote 

totals for each contest, by tabulator;

Sue - 10.02.08 - This issue will be 

resolved in the latest software drop 

of ERM 7.5.2.0.

Sue - 04.03.09 - Please review 

using updated software.

Sue - 05.19.09 - The scope of 

Unity 3.2.0.0 does not contain the 

number (3) of groups to test this. 

Accepted C Coggins 

5/19/09 Out of scope 

of Unity 3.2.0.0 

because it only 

contains 2 groups. 

Issue is being moved 

to Unity 4.0.0.0 

discrepancy #2. 

36 12/30/08 K 

Swift

Document 

Defect

Closed ES&S QA 

Program 

Manufacturing 

v. 1.1.0.0 Jul 

14, 2008

ES&S QA 

Program SW 

& FW v. 

1.3.0.0 Jul 14, 

2008

From SysTest Unity 4.0 Testing -

Disc: 479

Both document contain a 

Documentation section that details 

what TDP documents will be sent to a 

purchaser of the system. However, the 

System Security Specification 

document shows that it is not 

submitted to a purchaser, and this is a 

requirement.

V1: 2.2.1.g

System security is achieved through 

a combination of technical 

capabilities and sound 

administrative practices. To ensure 

security, all systems shall:

g. Provide documentation of 

mandatory administrative 

procedures for effective system 

security.

10.20.2008 MDN -- Updated QAP 

for SW/FW submitted with current 

release. Section 13 updated as 

specified above.

Accepted: K Swift, 

12/30/08

Reviewed the 

SysTest description 

history and verified 

other requirements 

have been previously 

validated by SysTest.

Verified QAP 

SW/FW, dated 

12/1/08; Section 13 

has been updated to 

include the SSS is 

part of the 

documentation 

package submitted to 

the end user.

37 12/30/08 K 

Swift

Document 

Defect

Closed ES&S QA 

Program SW 

& FW v. 

1.3.0.0 Jul 14, 

From SysTest Unity 4.0 Testing -

Disc: 480

Section 5.3 of the document lists 

"ES&S Ballot Image Manager (BIM)". 

v.2: 2.12.4

The vendor shall provide a 

description of its practices for 

documentation of the system and 

10.20.2008 MDN -- Updated QAP 

for SW/FW submitted with current 

release. Document updated as 

specified on 8/25.

Accepted: K Swift, 

12/30/08

Reviewed the 

SysTest description 
VSTL Certification # pending

Page 160 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

2008 The System Overview documentation 

lists the acronym to be 'ESSIM' for this 

product. 

system development process that 

meet the requirements of Volume I, 

Section 7.7.a of the Standards.

history and verified 

QAP SW/FW, dated 

12/1/08; Section 5.3 

has been updated 

with correct acronym.

38 12/30/08 K 

Swift

Document 

Defect

Closed ERM System 

Ops 

Procedures v. 

7.5.0.0 Jul 8, 

2008

From SysTest Unity 4.0 Testing -

Disc: 492

There is no information in the 

document that explains how to validate 

the reports created in ERM. This is 

especially applicable to the District 

Canvass Report, EL116.

VVSG Vol. 2: 2.8.5.a

The vendor shall provide 

documentation of system operating 

features that meets the following 

requirements:

A detailed description of all input, 

output, control, and display features 

accessible to the operator or voter

RDG- ERM SOP v. 

7.5.0.0_9.19.2008 - Added note to 

Chapter 3: Election Day Tasks 

under the Produce Election 

Reports heading stating that you 

can verify reports by comparing 

the ERM reports with the reports 

printed from the voting machines. 

Also added the note in Chapter 32: 

Precinct, Chapter 33: Election and 

Chapter 34: Canvass. 

Accepted: K Swift, 

12/30/08

Reviewed the 

SysTest description 

and verified ERM 

SOP, dated 

10/17/08; the 

addition of 'Notes' 

stating to compare 

ERM reports to 

tabulator reports.

39 12/30/08 K 

Swift

Document 

Defect

Closed ES&S CM 

Plan Unity v. 

4.0.0.0 Jul 13, 

2008

From SysTest Unity 4.0 Testing -

Disc: 493

The CM Plan addresses the file name 

structure, however it does not 

adequately address the versioning 

practices used in the title pages of the 

documents submitted in a TDP. Some 

documents will contain the 'Unity 

4.0.0.0' versioning, while others will 

inconsistently list the hardware and/or 

software versions. The QA documents 

submitted in the TDP, contain 

versioning that is not explained.

v.2: 2.11.2.a, b

The vendor shall provide a 

description of the procedures and 

naming conventions used to address 

the specific requirements of Volume 

I, Section 8.4. These requirements 

pertain to:

a. Classifying configuration items 

into categories and subcategories;

b. Uniquely numbering or otherwise 

identifying configuration items; 

MDN- CMPlan_10.24.2008 -

Updated Section 3.2.2.2 to include 

descriptions of versioning and 

naming conventions for all TDP 

documents. 

Accepted: K Swift, 

12/30/08

Verified in Unity 

3.2.0.0 CMP, dated 

11/21/08; Section 

3.2.2.2 has been 

updated to list the 

various versions that 

could be contained 

on documents 

submitted in the 

TDP.

40 12/30/08 K 

Swift

Document 

Defect

Closed HPM System 

Functionality 

Description v. 

5.6.0.0 Nov 

15, 2007

From SysTest Unity 4.0 Testing -

Disc: 495

The document states in the table in 

section 2.1.7.2, "Primary presidential 

delegation nominations", "This is 

supported by each individual tabulator". 

However, the DS200, M100 and M650 

SFD documents all state "This 

requirement exceeds the scope of the 

DS200/M100/M650.", and the IVO SFD 

document states "The iVotronic 

supports these nominations through a 

―local‖ contest type." 

It appears the documents are 

contradictory to the statement made in 

the HPM SFD. The IVO SFD appears 

to support it as a "Local contest", 

however a) 'Local Contest' is not 

defined, and b) it is not clear why the 

IVO would support it, but not the 

DS200, M100 and M650.

v.2: 2.3.b

The vendor shall provide a listing of 

the system‘s functional processing 

capabilities, encompassing 

capabilities required by the 

Standards and any additional 

capabilities provided by the system. 

This listing shall provide a simple 

description of each capability...

b. Additional capabilities shall be 

clearly indicated. They may be 

presented using the same structure 

as that used for required capabilities 

(i.e., overall system capabilities, prevoting functions, voting functions, 

post-voting functions), or may be 

presented in another format of the 

vendor‘s choosing

10.20.08 - MDN

Primary Presidential Delegation 

Nominations are not supported by 

the Unity 4000 system. Updated 

all SFDs to indicate that this voting 

variation is not supported.

Accepted: K Swift, 

12/30/08

Verified in all SFD 

documents submitted 

for Unity 3.2.0.0; that 

the Primary 

Presidential 

Delegation 

Nominations are not 

supported.
VSTL Certification # pending

Page 161 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

It is the VSTL's understanding that the 

tabulator media is created in HPM, 

therefore, that would be where the 

tabulation rules/parameters would be 

burned for each tabulator.

41 12/30/08 K 

Swift

Document 

Defect

Closed ES&S M650 

System Maint 

Manual FW v. 

2.2.1.0 HW v. 

1.1 & 1.2 Aug 

22, 2008

From SysTest Unity 4.0 Testing -

Disc: 496

In Chapter 1 pg 2 under "Electrical 

Information" of the referenced 

document states the M100 and not the 

M650.

v.2: 2.9.1.a

The description shall include a 

concept of operations that fully 

describes such items as:

a. The electrical and mechanical 

functions of the equipment

10.17.08 - RDG

M650 SMM v.2.2.1.0_10.17.2008

Changed references from M100 to 

M650.

Accepted: K Swift, 

12/30/08

Verified in M650 

SMM, dated 

10/17/08, the correct 

component 

referenced.

42 12/30/08 K 

Swift

Document 

Defect

Closed ES&S CM 

Plan Unity v. 

4.0.0.0 Aug 

22, 2008

From SysTest Unity 4.0 Testing -

Disc: 497

The cited document does not contain 

the System Maintenance Manuals as 

part of the submitted documents that 

must be provided to purchasers. 

v.2: 2.9

The system maintenance 

procedures shall provide information 

in sufficient detail to support election 

workers, information systems 

personnel, or maintenance 

personnel in the adjustment or 

removal and replacement of 

components or modules in the field. 

Technical documentation needed 

solely to support the repair of 

defective components or modules 

ordinarily done by the manufacturer 

or software developer is not 

required.

MDN - 10.02.2008 - Added system 

maintenance manuals and system 

limitations documentation to 

section 7.1.5. (see discrepancy 

#297 for limitations requirement).

Accepted, K Swift, 

12/30/08

Verified the Unity 

3.2.0.0 CMP, dated 

11/21/08 contains 

the SMM, and 

System Limitations 

documents in the list 

of documents to be 

given to a 

purchaser/end user

43 12/31/08 K 

Swift

Document 

Defect

Closed System 

Overview 

ES&S v. 

4.0.0.0 Aug 

22, 2008

From SysTest Unity 4.0 Testing -

Disc: 549

On Page 18 of the document, a 

reference is made to the Electronic 

Pollbook in the 'Note' section. However 

the Electronic Pollbook is not part of 

this certification.

v.2: 2.1.1

The content of the Technical Data 

Package (TDP) is intended to 

provide clear, complete descriptions 

of the following information about 

the system:

Overall system design, including 

subsystems, modules and the 

interfaces among them

MDN - 10.02.2008 - System 

Overview - removed reference to 

the ES&S Electronic PollBook from 

Section 1.2.4. 

Accepted: K Swift, 

12/31/08

Verified Unity 3.2.0.0 

Sys Overview, dated 

11/20/08; does not 

contain any 

reference to 

Pollbook.

44 12/31/08 K 

Swift

Document 

Defect

Closed System 

Overview 

ES&S v. 

4.0.0.0 Aug 

22,2008

From SysTest Unity 4.0 Testing -

Disc: 550

Section 2.2.4 Usability states "Please 

see the usability test reports included 

with the TDP for ES&S system usability 

statistics." However, we can not locate 

these reports in the TDP.

v.2: 2.2.2.b

The vendor shall provide system 

performance information including:

Quality attributes such as reliability, 

maintainability, availability, usability, 

and portability

MDN - 10.02.2008 - Section 2.2.4. 

Revised section to provide an 

overview of ES&S' approach to 

promoting usability in design. 

Usability testing has not been 

performed. While ES&S has 

requested a review of 

documentation to 2005 standards, 

the submitted system is to be 

tested to the 2002 standard, which 

does not require usability testing.

Accepted: K Swift, 

12/31/08

Confirmed Unity 

3.2.0.0 has been 

submitted to the EAC 

as a certification 

against the 2002 

VSS standards

45 12/31/08 K 

Swift

Document 

Defect

Closed Software 

Design & 

Spec ES&S 

From SysTest Unity 4.0 Testing -

Disc: 553

The "Election Data Manager Data" flow 

v.2: 2.5.3.d4

The vendor shall provide an 

overview of the software that 

CJ - 02/16/09 I believe this graphic 

attempted to show where the 

generated files were destined for. 

Accepted - KSwift -

3/12/09 

Verified ESSIM SDS, 
VSTL Certification # pending

Page 162 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

Ballot Image 

Manager 

(ESSIM) v. 

7.7.0.0 Apr 

16, 2008

Software 

Design & 

Specs ERM v. 

7.5.0.0 May 

15, 2008

chart on page 5 of the referenced 

document shows the IFC File coming 

from EDM to HPM. However, 

"Overview of Unity and Context 

Boundary Definitions" flow chart in the 

referenced ERM SDS document shows 

no IFC file coming out of EDM, but 

ESSIM. The documents are conflicting 

as to where the IFC file originates.

includes the following items:

d4 Identification of interfaces with 

other items that provide data to, or 

receive data from, the item

In the case of the IFC file it is 

ultimately destined for the HPM 

but does pass through ESSIM first. 

This graphic was updated to show 

the file relationship (for ESSIM as 

well as other ES&S applications) 

and standardized on graphic 

element usage.

MDN - 02/16/09 The diagram 

included with the ERM SDS is 

correct and requires no change. 

2/23/09; Section 3.1, 

explains the 

origination and 

migration through the 

Unity suite of the IFC 

file. EDM SDS, 

2/23/09; flowchart on 

in Section 3.1, also 

supports this.

46 12/31/08 K 

Swift

Document 

Defect

Closed ES&S EDM 

System Ops 

Procedures v. 

7.8.0.0 Aug 

22, 2008

From SysTest Unity 4.0 Testing -

Disc: 554

Chapter 15 of the referenced document 

contains a table called "Language 

Candidate Table" This table has "???" 

under the comments section for the 

Language Party Field. It is unclear 

what is "???".

Rejected - Kswift - 3/21/09

Reviewed EDM SOP, dated 1/30/09; 

document has not been updated per 

ES&S response.

v.2: 2.8.4.a

The vendor shall provide 

documentation of system operating 

features that meets the following 

requirements:

a. Provides a detailed description of 

all input, output, control, and display 

features accessible to the operator 

or voter;

1.30.2009 TMT- Chapter 15, 

Updated the Language Candidates 

Table to include the Language 

Party description. 

4.3.2009 TMT- Submitted 

documentation now reflects the 

change detailed above. 

Rejected - Kswift -

3/21/09

Doc has not been 

updated.

Accepted C Coggins 

4/1659 

Verified EDM SOP 

v.7.8.1.0 4/3/09 

"Language 

Candidates Table" 

comments reflect a 

"Language Party" 

description

47 12/31/08 K 

Swift

Document 

Defect

Closed ES&S EDM 

System Ops 

Procedures v. 

7.8.0.0 

8/22/08

From SysTest Unity 4.0 Testing -

Disc: 555

Chapter 15 of the referenced document 

contains a table called "Poll Relations 

Table". Under the IFC column, it 

shows N/A for the Poll Location ID 

field. However, throughout the other 

tables in the document, it states that 

the IFC has a maximum of 4 

characters for the Poll Location ID field. 

It is unclear if the .IFC file gathers the 

Poll Location ID from the same point or 

if the information is gathered based on 

menu selection.

Rejected - Kswift - 3/21/09

Reviewed EDM SOP, dated 1/30/09; 

document has not been updated per 

ES&S response.

v.2: 2.8.4.a

The vendor shall provide 

documentation of system operating 

features that meets the following 

requirements:

a. Provides a detailed description of 

all input, output, control, and display 

features accessible to the operator 

or voter;

1.30.2009 TMT- Chapter 15, 

Updated the Poll Relations Table 

to edit the maximum characters 

from N/A to 4 in the Poll Location 

ID box. 

4.3.2009 TMT- Submitted 

documentation now reflects the 

change detailed above.

Rejected - Kswift -

3/21/09

Doc has not been 

updated.

Accepted C Coggins 

4/15/09 

Verified EDM SOP 

v.7.8.1.0 4/3/09 "Poll 

Relations Table" 

reflects 4 in the "Poll 

Location ID"

48 12/31/08 K 

Swift

Functional 

Defect

Closed AM v. 7.5.0.0

Test Case: 

Readiness TC 

IA Rev00 

TE01

From SysTest Unity 4.0 Testing -

Disc: 556

The archive function in AM doesn't 

appear to work correctly.

Steps Taken while executing the "Audit 

Manager 7.5.0.0 Audit Manager 

V1: 2.5.3

All systems shall be able to create 

reports summarizing the vote data 

on multiple levels. 

2.16.2009 MDN -- Item will be 

addressed with updated Audit 

Manager software.

04.06.2009 SUE -- Trusted build 

has been performed and updated 

Accept K Swift 

4/28/09 

Verified in AM 

7.5.2.0, the AM 

Archive functionality 
VSTL Certification # pending

Page 163 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

Checklist Election Day Training Manual 

Unity Version 4.0 Release Date: 

August 2007: 

From the Audit Manager application 

'Audit' menu

Select 'Archive Log' button; Input range 

of dates to archive From 10/10/08 To 

10/10/08; Select 'Archive' button 

In Save As window input file name 

'10.10.08.archive' Select 'Save' button 

Audit manager window states '0 

records archived' Select 'OK' button 

Reproduced the steps, and found a 

text file called '10.10.08.archive.txt' 

was created empty. 

Reviewed AM SOP v. 7.5.0.0, August 

22, 2008, chapter 3, page 13, section 

'Archive the Audit Log' , item # 3, there 

was a 'Note' which stated that 

"Archiving selected records will 

permanently remove them from the 

Audit Manager interface and place 

them in the text file you create". Reviewed the Audit log by selecting 'View 

Log' from the Audit menu, and files 

were not removed.

Executed above steps using Input 

range of dates From 10/8/08 To 

10/10/08. This time it did archive the 

10/10/08 records and removed them 

from the log, but it did not archive or 

remove the 10/8/08 records. 

10/14/2008 - Troubleshooting Reexecuted the above steps to archive a 

longer range of dates: From 10/02/08 

To 10/14/08. 

(5 different log entry dates exist in the 

audit log during this range of dates). 

The 10/02/08 entries were not archived 

or removed from the audit log. Audit 

log entries after 10/02/08 through 

10/14/08 were successfully archived to 

the designated file '10.2.08.archive' 

and removed from the audit log.

software delivered. works as 

documented.

49 12/31/08 K 

Swift

Document 

Defect

Closed ES&S Image 

Manager 

System Ops 

Procedures, 

Ver. 7.7.0.0, 

Jul 8, 2008

From SysTest Unity 4.0 Testing -

Disc: 557

Ch. 35 does not specify that the 

placing of a graphic on a ballot in 

ESSIM is a state specific functionality. 

The current verbiage leads the user to 

believe a graphic can be placed on any 

V1: 2.3.1.3.1.b

The voting system shall provide a 

means of printing or otherwise 

generating a ballot display that can 

be installed in all voting equipment 

for which it is intended. All voting 

systems shall provide the 

Accepted, K Swift, 

12/31/08

Excessive 

interpretation; 

function is consistent 

with the requirement
VSTL Certification # pending

Page 164 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

ballot which is not in compliance with 

the VVSG requirement.

capabilities below:

b. The electronic display or printed 

document on which the user views 

the ballot does not show any 

advertising or commercial logos of 

any kind, whether public service, 

commercial, or political, unless 

specifically provided for in State law. 

Electronic displays shall not provide 

connection to such material through 

hyperlink.

50 1/6/09 K 

Swift

Document 

Defect

Closed Unity 3.2.0.0 

System 

Overview, 

11/20/08

Unity 3.2.0.0 

System 

Limitation, 

Dec 1, 2008 

Unity 3.2 language scope is not 

identified.

The System Overview and System 

Limitations do not state that the Unity 

3.2 certification is limited to English 

and Spanish only.

v.2: 2.3

The vendor shall provide a listing of 

the system‘s functional processing 

capabilities, encompassing 

capabilities required by the 

Standards and any additional 

capabilities provided by the system. 

2.16.2009 MDN -- Updated 

system limitations document to 

specify that only Spanish and 

English are supported in the 

U3200 system. System 

Limitations and the Supported 

Functionality Declaration are 

packaged as appendices to the 

System Overview.

Accepted - KSwift - 

3/12/09 Verified Unity 

3.2.0.0 System 

Limits, dated 

2/11/2009; the 

addition of support 

for English and 

Spanish only.

51 1/8/09 K 

Swift

Document 

Defect

Closed CM Plan 

Unity v. 

3.2.0.0Nov 

21, 2008 

Section 7.1.5

QA Program 

Manufacturing 

Unity v. 

3.2.0.0 Dec 1, 

2008 Sect. 8

QA Program 

SW & FW 

Unity v. 

3.2.0.0 Dec 1, 

2008 Sect 13

Lists of documents provided to end 

users are inconsistent.

Page 56, of the Unity 3.2.0.0 CMP, 

identifies the following documents 

delivery to a purchaser/end user: Install 

Procedures, SOP's, SMM's, Training 

Procedures, SSS, and System 

Limitations;

This does not match the list in the QAP 

Manufacturing document (page 25 -

27), or the QAP SW/FW document 

(page 41 - 43): SOP's, Training 

Procedures, SSS, and System Change 

Notes. 

V1: 7.7.a

Vendors are required to produce 

documentation ...

a. Be sufficient to serve the needs of 

the VSTL, voters, election officials, 

and maintenance technicians;

2.16.2009 MDN -- The lists 

included in the Quality Assurance 

Plans affirm that ES&S generated 

documentation meets the 

requirements for each document 

category defined under VVSG 

2005 V.1, S.8.7. These categories 

may encompass multiple 

document types. For example, 

installation instructions relate to 

requirements for System 

Operations Procedures. The list in 

the CM plan responds to VVSG 

2005 V.1, S. 9.7.1.e by providing a

list of specific , end-user 

deliverables. RESOLUTION: 

Copied the user documentation 

deliverables section from the CM 

Plan to the QA documents and 

updated the QA lists to indicate 

that Maintenance guides are 

provided to the end user, which 

was not indicated in the previous 

revision. Added system change 

notes (when applicable) to the CM 

Plan doc deliverables list.

Accepted - KSwift - 

3/12/09 Verified 

resubmitted 

documents as 

identified, dated 

2/23/09; are 

consistent with each 

other in specifying 

the documents 

provided to end 

users.

52 1/9/09 C 

Cog

gins

Document 

Defect

Closed Unity 3.2.0.0 

System 

Overview, 

11/20/08

Typographical error in the DS-200 HW 

version number

The System Overview identifies the 

v.1: 8.7.1.d For the PCA, a vendor 

shall provide configuration baseline 

data for all hardware that is unique 

to the system

2.16.2009 MDN -- Updated the 

DS200 hardware revision number 

to 1.2.1.

Reject - KSwift - 

3/12/09 

Accept C. Coggins 
VSTL Certification # pending

Page 165 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

DS-200 hardware version as 1.3. The 

submitted hardware versions are 1.2 

and 1.2.1.

3/12/09 Kswift - Reject

DS200 hardware version 1.2 has not 

been included. 

Accept - KSwift

Verified System Overview 2/23/09; 

updated to include v. 1.2.1 hardware

4.3.2009 MDN -- Updated the file 

to include both versions of DS200 

hardware submitted for testing.

4/15/09

Verified Unity 3.2.0.0 

System Overview 

4/3/09 section 3.1.2 

reflected both DS200 

HW versions

53 1/9/09 C 

Cog

gins

Document 

Defect

Closed Unity 3.2.0.0 

System 

Limitation, 

Dec 1, 2008 

VAT and AIMS are not identified in the 

system limitations. 

The System Limitations document 

does not explicitly identify all Unity 3.2. 

entities. The VAT and AIMS are 

neither included nor identified as 

excluded.

v.2: 2.2.2.a The vendor shall provide 

system performance information that 

includes descriptions of: the 

performance characteristics of each 

operating modes and function in 

terms of ... maximum volume 

(maximum number of voting 

positions and maximum number of 

ballot styles supported)

2.16.2009 MDN -- Added AIMS 

and AutoMARK system limitations 

to the referenced document. 

Accept - KSwift - 

3/12/09 

Verified System 

Limits, 2/11/09; has 

been updated to 

include an AIMS 

limitation.

54 1/26/09 C 

Cog

gins

Document 

Defect

Closed AutoMARK 

System 

Hardware 

Spec Rev 3 

AQS-13-

5000-001-F

The VAT schematics for the SBC was 

not delivered

On page 59, it identifies the SBC 

schematic is delivered as an external 

document. This document was not 

delivered. (Test purpose is to review 

the engineering change in SBC v.2.0 

and v.2.5)

v.2: 2. 4.2 a Paragraphs and 

diagrams shall be provided that 

describe: ... parts used in the 

system, their assembly, and the 

configuration control measures to 

ensure compliance with the system 

specification

2.17.2009 MDN -- Scanner board 

schematics appear under the 

following path: ...14_AMVATAIMS_1.3\VAT 

TDP\SECTION03\Schematics as 

an attachment to the AMVAT 

Hardware specification . The file 

named 'SBC_640117-4000C2AGP.pdf' is the referenced 

schematic.

Accept C Coggins 

3/3/09 The SBC 

schematic 

documentation was 

delivered 

55 2/3/09 Sjaki

leti

Document 

Defect

Closed Security 

review-VAT

AutoMARK 

System 

Functionality 

rev.4.0

VAT: There is a reference to the 

"AutoMARK System Security 

Specifications rev.4.0 Section 5". But 

there is no section 5.

v.1: 2.2.1e: Provide security 

provisions that are compatible with 

the procedure and administrative 

tasks involved in equipment 

preparation, testing, and operation.

2.17.2009 MDN -- Updated the 

flagged cross-reference in the 

System Functionality Description 

to map to access control 

guidelines located under Section 

2.B of the System Security 

Specification.

Accepted 

SJ:3/11/09.Updated 

documentation refers 

to the correct section. 

Updated document is 

'AutoMARK System 

Functionality v5.0"

56 2/3/09 Sjaki

leti

Document 

Defect

Closed Security 

review-VAT

AutoMARK 

System 

Functionality 

rev.4.0

VAT: No documentation for mandatory 

administrative procedures.

v.1: 2.2.1g: Provide documentation 

of mandatory administrative 

procedures for effective system 

security.

2.17.2009 MDN -- Updated 

content under Section 3.B.1 to 

specify that election administrators 

are required to develop and 

implement an effective voting 

system security program. Updated 

the AutoMARK System Security 

Specification (multiple locations 

within document) to emphasize 

that the system features and 

procedure recommendations 

included in that document were 

developed to be incorporated into 

a mandatory voting system 

security program.

Accepted SJ:3/11/09. 

verified there are 

now administrative 

procedures in 

"AutoMARK System 

Functionality v5.0" 

and "AutoMARK 

System Security 

Specifications v5.0"
VSTL Certification # pending

Page 166 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

57 2/3/09 Sjaki

leti

Document 

Defect

Closed Security 

review-VAT

AutoMARK 

System 

Functionality 

rev.4.0

VAT: Documentation for restoring the 

device in case error or failure is 

incomplete 

v.1: 2.2.2.1d: Include control logic 

and data processing methods 

incorporating parity and check-sums 

(or equivalent error detection and 

correction methods) to demonstrate

that the system has been designed 

for accuracy, and

2.17.2009 MDN -- Updated the 

cross-reference under Section 

3.B.3 to map to Section 3 of the 

AutoMARK 3010 Poll Workers 

Guide AQS-13-5061-002-R, which 

contains AMVAT error detection 

and correction methods including a 

list of system error messages and 

corresponding procedures for error 

recovery.

Accepted SJ 3/11/09: 

Updated the 

documentation by 

cross referencing to 

the error detection 

and correction 

methods including a 

list of system error 

messages and 

corresponding 

procedures for error 

recovery in 

"'AutoMARK System 

Functionality v5.0"

58 2/3/09 Sjaki

leti

Document 

Defect

Closed Security 

review-VAT

AutoMARK 

System 

Functionality 

rev.4.0

VAT: Documentation for data readwrite and transfer quality status is 

incomplete

v.1: 2.2.2.1e: Provide software that 

monitors the overall quality of data 

read-write and transfer quality 

status, checking the number and 

types of errors that occur in any of 

the relevant operations on data and 

how they were corrected.

2.17.2009 MDN -- Added crossreferences under Section 3.B.2 

that map to sections 2.C and 

Section 3 of the AutoMARK 3010 

Ballot Image Processing 

Specification AQS-13-5002-003-S. 

These sections include a 

description of the ES&S 

AutoMARK‘s procedure for 

accurately converting paper ballot 

contents to audio and display 

formats, Cross-referenced Section 

3 of the AutoMARK 3010 Poll 

Workers Guide AQS-13-5061-002-

R for a description of data readwrite error conditions and 

procedures for correcting those 

errors and Section 3 of 

AutoMARK 3010 Programming 

Specifications Details AQS-13-

5001-011-R for detailed 

descriptions of the system 

software that performs data quality 

assessment

Accepted SJ 3/11/09: 

Updated the 

documentation by 

cross referencing the 

section of data readwrite and transfer 

quality status in 

"'AutoMARK System 

Functionality v5.0"

59 2/3/09 Sjaki

leti

Document 

Defect

Closed Security 

review-VAT

AIMS System 

Security Spec 

v.3.0

VAT and AIMS: No documentation is 

provided for effective password 

management

v.1: 6.2.1.1d: Effective password 

management;

2.17.2009 MDN -- Updated 

Section 2.D.1 with ES&S 

recommended Windows OS 

password settings for any 

mandatory election system 

security program.

Accepted SJ 

3/11/2009: Updated 

the documentation 

for effective 

password 

management in 

"AutoMARK 

Information 

Management System 

(AIMS) System 

Security 
VSTL Certification # pending

Page 167 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

Specifications v4.0"

60 2/3/09 Sjaki

leti

Document 

Defect

Closed Security 

review-DS200

ES&S System 

Security Spec 

Ver. Rel. 

4.0.0.0 dated 

7/8/08

The access control information 

provided in the security specification is 

too general. It does not identify user 

roles, functions and data or any time 

limits associated with specific election 

operations. 

v.1: 6.2.1.2a: Identify each person to 

whom access is granted, and the 

specific functions and data to which 

each person holds authorized 

access

v.1: 6.2.1.2b: Specify whether an 

individual‘s authorization is limited to 

a specific time, time interval or 

phase of the voting or counting 

operations

2.13.2009 PMZ -- Revised pg 11 

item "a" and pg 12 "Software 

Access Controls"

Accepted 3/13/2009 

SJ: Updated the 

documentation for 

Individual's 

authorization , 

access to the specific 

functions on 

tabulators in "ES&S 

System Security 

Specification Ver. 

Rel. 4.0.0.0 

2/20/2009"

61 2/3/09 Kwil

son

Document 

Defect

Closed Security 

review-EDM

ES&S System 

Security Spec 

Ver. Rel 

4.0.0.0 dated 

7/8/08

Pg 78 describes using the Windows XP 

audit log but does not describe how. 

References to documentation on the 

Internet may not survive the product 

lifetime. As the specific enablements 

are left to the jurisdiction's 

interpretation of the Microsoft 

documentation this is insufficient to 

meet the requirement

v.1: 2.2.1a: Provide security access 

controls that limit or detect access to 

critical system components to guard 

against loss of system integrity, 

availability, confidentiality, and 

accountability.

2.13.2009 PMZ -- Section does not 

apply to current system and was 

deleted.

Accepted 3/12/2009 

SJ: 

Updated the 

documentation by 

deleting NTFS 

logging and NOS 

Security sections in 

"ES&S System 

Security Specification 

Ver. Rel. 4.0.0.0 

date 02/20/09".This 

section does not 

apply to the current 

system

62 2/3/09 Kwil

son

Document 

Defect

Closed Security 

review-EDM

ES&S System 

Security Spec 

V. Rel 4.0.0.0 

dated 7/8/08

On pg 79 and earlier mentions IIS and 

Web Server logs. The security 

specification does not identify how IIS 

is used in Unity v.3.2.0.0

v.1: 2.2.1a: Provide security access 

controls that limit or detect access to 

critical system components to guard 

against loss of system integrity, 

availability, confidentiality, and 

accountability.

2.13.2009 PMZ -- Section does not 

apply to current system and was 

deleted.

Accepted SJ 

3/12/2009: Updated 

the documentation by 

deleting the IIS 

logging features in 

the "ES&S System 

Security Specification 

Ver. Rel. 4.0.0.0 

date 02/20/09".This 

section does not 

apply to the current 

system.

63 2/3/09 Kwil

son

Document 

Defect

Closed Security 

review-EDM

ES&S System 

Security Spec 

Ver Rel 

4.0.0.0 dated 

7/8/08

Pg 81 references the database 

folders. The security specification 

does not identify which specific 

database folder(s) need to be secured.

v.1: 2.2.1a: Provide security access 

controls that limit or detect access to 

critical system components to guard 

against loss of system integrity, 

availability, confidentiality, and 

accountability.

2.13.2009 PMZ -- Section does not 

apply to current system and was 

deleted.

Accepted SJ 

3/12/2009: Updated 

the documentation by 

deleting Windows 

database lockdown

procedures in ES&S 

System Security 

Specification Ver. 

Rel. 4.0.0.0 date 

02/20/09". This 

section does not 

apply to 3.2.0.0.
VSTL Certification # pending

Page 168 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

64 2/3/09 Kwil

son

Document 

Defect

Closed Security 

review-EDM 

ES&S System 

Functionality 

Description 

EDM v.1.0

The functional description states that 

the requirement "all systems shall 

detect and record every event including 

..." is not applicable, without providing 

justification.

v.1: 2.2.4.1i Detect and record 

every event, including the 

occurrence of an error condition that 

the system cannot overcome, and 

time-dependent or programmed 

events that occur without the 

intervention of the voter or a polling 

place operator; and

02.16.09 CJ - Updated row "I" to 

read "Included in System" and 

reference audit events referenced 

in Section 1.1.5.

Accepted 3/12/2009 

SJ: Updated the 

documentation by 

referencing the 

appropriate sections 

of detection and 

recording all events , 

including error 

condition sin 

"Election Systems & 

Software ES&S 

System Functionality 

Description EDM 

Unity v. 3.2.0.0" Rev 

2.0

65 2/3/09 Kwil

son

Document 

Defect

Closed Security 

review-EDM 

ES&S System 

Functionality 

Description 

EDM v.1.0

The functional description states that 

the requirement "all systems shall 

include built in measurement, self test 

and diagnostic software" is not 

applicable, without providing 

justification.

v.1: 2.2.4.1j Include built-in 

measurement, self-test, and 

diagnostic software and hardware 

for detecting and reporting the 

system's status and degree of 

operability.

02.16.09 CJ -- Row "j" of this 

table is related to "I" and also 

updated.

Accepted 3/12/2009 

SJ: Updated the 

documentation by 

describing the Audit 

Manager application 

and EDM Error and 

Warning events in 

"Election Systems & 

Software ES&S 

System Functionality 

Description EDM 

Unity 3.2.0.0" 

Revv2.0"

66 2/3/09 Kwil

son

Document 

Defect

Closed Security 

review-EDM

ES&S System 

Functionality 

Description 

EDM v.1.0

The functional description states that 

the VSS v.1: 2.2.5.3 is not applicable. 

As EDM, ESSIM, HPM, and ERM 

operate on COTS operating systems 

capable of executing multiple 

application programs simultaneously, 

this requirement is applicable.

3/12/09 Reject SJ the response does 

not completely address the 

discrepancy, system functionality 

descriptions of ESSIM,HPM and ERM 

tare missing

v.1: 2.2.5.3 COTS General Purpose 

Computer System Requirements

...the system shall be configured to 

execute only intended and 

necessary processes during the 

execution of election software. 

02.16.09 CJ- Updated referenced 

section with a table as 

incorporated in other EDM SFD 

TDP releases.

04.09.09 MDN - Added more 

detailed responses to this 

requirement to Section 1.1.5.3 of 

the ESSIM, HPM and ERM SFD 

documents.

Accepted SJ 

4/14/2009: Updated 

documentation to 

configure the system 

to execute only in the 

intended manner 

during the execution 

of election software 

in Unity 3.2.0.0 

System Functionality 

Description for 

ESSIM, HPM & 

ERM (dtd 4/9/09)

Rejected - SJ -

3/12/09

67 2/3/09 Kwil

son

Document 

Defect

Closed Security 

review-EDM

ES&S System 

Security Spec 

Ver Rel 

4.0.0.0 dated 

7/8/08

The boundaries of the system 

submitted for EAC certification are not 

clearly defined. The security 

specification does not if the Unity 

v.3.2.0.0 system includes such security 

vulnerabilities as LAN, WAN, 

telecommunications (modems etc.) and 

v.1: 6.2.1: General Access Control 

Policy

Although the jurisdiction in which the 

voting system is operated is 

responsible for determining the 

access policies for each election, the

vendor shall provide a description of 

2.13.2009 PMZ -- The boundary of 

the system submitted includes a 

private, election specific LAN to 

which the EMS PCs are 

connected. No other networking is 

used in the system. 

Changes have been made to the 

Accepted 3/12/2009 

SJ: Updated the 

documentation by 

identifying that the 

boundaries of the 

system include a 

private election 
VSTL Certification # pending

Page 169 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

the impact of physical security to 

mitigate these vulnerabilities. Modems 

appear as optional equipment in 

System Overview. While other sections 

of the TDP state which devices and 

programs are included in the Unity 

3.2.0.0 certification, they do not 

specifically provide detailed 

descriptions sufficient to assess 

potential vulnerabilities. Example: It is 

not clearly identified if the 

EDM/ESSIM/HPM and ERM 

computers are networked or not 

networked. The appearance of network 

nomenclature throughout the document 

suggest that these computers are 

networked.

recommended policies for: manual pg 13 "Firewall, 

Networking, and IDS"

specific LAN to which 

only EMS PCS is 

connected, and no 

other networking is 

available, in "ES&S 

System Security 

Specification Ver. 

Rel. 4.0.0.0 date 

02/20/09"

68 2/3/09 Kwil

son

Document 

Defect

Closed Security 

review-EDM

ES&S System 

Security Spec 

Ver Rel 

4.0.0.0 dated 

7/8/08

The information provided in security 

specification does not provide 

sufficient detail for the jurisdiction to 

create a security plan and policy. 

Specific information about the 

capabilities and vulnerabilities of the 

Unity v.3.2.0.0 is required to complete 

these plans and policies. Example 

EDM has 2 roles, administrator and 

user. Under Segregation of Duties the 

security specification suggests at least 

3 roles: System Administrator, Election 

Administrator, Election Definition 

Workers. It is unclear if there is a role 

assigned for Election Reporting. It is 

unclear how a system with 3 or more 

roles fits into the EDM example of 2 

roles? Information appears to be 

missing for software access controls 

and segregation of duties.

3/13/09 Reject SJ The information 

added in the documentation does not 

address the whole discrepancy. The 

information provided in the security 

specification does not provide sufficient 

detail for the jurisdiction to create a 

security plan and policy, including: 

- Audit Manager permitting the 

jurisdiction to establish separate 

credentials for EDM and ESSIM not for 

ERM and HPM

-How to limit access to HPM and ERM.

v.1: 6.2.1.2: Identify each person to 

whom access is granted, and the 

specific functions and data to which 

each person holds authorized 

access

2.13.2009 PMZ -- see item 60, 

above

20090324 PMZ -- Additions to the 

System Security Specification and 

the inclusion of the System 

Hardening document clarify ES&S 

recommended procedures for 

assigning role based access. 

System Hardening Procedures 

provides protocols for setting 

access controls using Windows 

functionality.

Accepted 4/16/2009 

SJ: Updated the 

documentation 

hardening 

procedures for 

setting access 

controls using 

windows functionality 

in ES&S System 

Security Specification 

and Hardening 

Procedures for the 

Election 

Management System 

PC v3.2.0.0 (dtd 

3/27/09)

Rejected 3/13/2009 

SJ

69 2/3/09 Kwil Docu- Closed Security Lesson 2 Recommending NOT to v.1: 2.2.1a: Provide security access RJS - 2/20/09 EDM - version 8.2.x Accepted SJ 3/12/09: 
VSTL Certification # pending

Page 170 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

son ment 

Defect

review-EDM 

EDM Training 

Manual 

Version 

Number 

4.0.0.0 v.1.

change hard-coded initial passwords 

does not meet the requirement for 

access control. Furthermore the NOTE 

in Lesson 2 is in direct contradiction to 

the Lesson 3 recommendation.

controls that limit or detect access to 

critical system components to guard 

against loss of system integrity, 

availability, confidentiality, and 

accountability.

Changed password 

recommendations p. 8, 9 and 89 to 

comply with requirement 

Updated 

documentation AM 

password changes 

should be carefully 

safeguarded and 

keep tracked 

."Election Data 

Manager Training 

Manual Version 

Number 8.2.x"

70 2/3/09 Kwil

son

Document 

Defect

Closed Security 

review-EDM

EDM Training 

Manual V. # 

4.0.0.0 v.1.

ES&S System 

Functionality 

Description 

HPM v.1.0

ESSIM 

System Ops 

Procedures 

Ver Rel 

7.7.0.0 

10/17/08

Lesson 2.3 indicates that login security 

is optional. As mandatory procedures 

cannot be optional, iBeta must execute 

security tests in the least secure mode 

of acceptable operation of the system. 

Similar optional login security appears 

in the HPM functional description 

(1.1.1.a., f, and g) and the ESSIM 

operations procedure.

v.1: 6.2.2a: Vendors shall provide a 

detailed description of all system 

access control measures designed 

to permit authorized access to the 

system and prevent unauthorized 

access. Examples of such 

measures include: Use of data and 

user authorization

RJS - 2/20/09 EDM - version 

8.2.x Changed password 

recommendations p. 8, 9 and 89 to 

comply with requirement 

HPM version 5.6. x Changed 

password recommendations p. 36 

to comply with requirement 

ESSIM version 7.7.x Changed 

password recommendations p.8 to 

comply with requirement 

2.18.2009 -- Beth BingerDunaway

Added the following statement to 

1.1.1.a, f and g, HPM SFD:

"Federal guidelines require 

Election Administrators to 

implement a mandatory election 

security program."

1.30.2009 TMT - Chapter 6 ESSIM 

SOP: Start ESSIM. Edited the text 

under the main heading to state 

that security access controls are 

mandatory. Added verbiage to the 

Warning (Election Security 

Caution) also stating and clarifying 

as such.

Accepted 3/13/2009 

SJ: Updated in 

EDM_TM and 

HPM_SFD and 

ESSIM_SOP 

documentation with 

login security is 

mandatory.

"Election Data 

Manager Training 

Manual Version 

Number 8.2.x".

"Election Systems & 

Software ES&S 

System Functionality 

Description 

Hardware 

Programming 

Manager (HPM) 

Unity v. 3.2.0.0 v2.0"

"ES&S Image 

Manager System 

Operations 

Procedures Ver. Rel. 

7.7.0.0 date 

01/30/2009"

71 2/3/09 Kwil

son

Document 

Defect

Closed Security 

review-EDM

ES&S System 

Security Spec 

Ver Rel 

4.0.0.0 dated 

7/8/08

The security specification does not 

provide a detailed description of 

encryption. Section 6.2.2.e describes 

a computer-generated key but does not 

identify the purpose. Types of 

encryption being used as well as the 

specific instances of "data in transit" 

that are encrypted are not detailed. 

v.1: 6.2.2g: Vendors shall provide a 

detailed description of all system 

access control measures designed 

to permit authorized access to the 

system and prevent unauthorized 

access. Examples of such 

measures include: Message 

encryption 

2.13.2009 PMZ -- Changes were 

made to the document to clarify 

that transmission is not used in the 

system and how information 

moved between the precinct 

devices and the EMS PC (via 

removable memory) is protected.

Accepted 3/12/2009 

SJ: Updated 

documentation 

reflecting that M650 

and DS200 

exchanges 

information only with 

the EMS PC and 

uses removable 

media, no electronic 

transmission of 
VSTL Certification # pending

Page 171 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

results in "ES&S 

System Security 

Specification Ver. 

Rel. 4.0.0.0 date 

02/20/09"

72 2/3/09 Kwil

son

Document 

Defect

Closed Security 

review-EDM

ES&S System 

Security Spec 

Ver Rel 

4.0.0.0 dated 

7/8/08

Pg 77 describes NTFS/EFS encryption 

but does not describe how to 

implement it. Nor does it make any 

statements recommending, mandating, 

not recommending or not mandating its 

use.

v.1: 6.2.2g: Vendors shall provide a 

detailed description of all system 

access control measures designed 

to permit authorized access to the 

system and prevent unauthorized 

access. Examples of such 

measures include: Message 

encryption 

2.13.2009 PMZ -- section does not 

apply to current system and was 

deleted.

Accepted 3/13/2009 

SJ: Updated 

documentation by 

deleting NTFS/EFS 

encryption 

section(this section 

does not apply to 

current system in 

"ES&S System 

Security Specification 

Ver. Rel. 4.0.0.0 

date 02/20/09"

73 2/3/09 Kwil

son

Document 

Defect

Closed Security 

review-EDM

ES&S System 

Security Spec 

Ver Rel 

4.0.0.0 dated 

7/8/08

Jurisdiction 

Security 

Procedures v. 

1.0.0.1

Pg 13 Firewall states that "ES&S 

recommends that a jurisdiction use a 

firewall." Additionally the security 

procedure (6.7.4) references the 

possibility of a VPN within the 

boundary of the EAC certified system. 

A Unity v.3.2.0.0 disclaimer indicates 

that the system does not contain 

network data transmission, indicating 

that the system is not subject to the 

telecommunication requirements of the 

VSS. If it is the case that the Unity 

v.3.2.0.0 can be exposed to an 

external network which is potentially 

exposed to the public 

telecommunications system then it is 

subject to the telecommunications 

requirements and malicious attack 

requirements of the VSS regardless of 

the disclaimer. The documentation is 

insufficient to assess the vulnerability 

of the recommended network 

environment. 

v.1: 7.7.a This documentation shall : 

be sufficient to serve the needs of 

the ITA , voters, election officials 

and maintenance technicians.

2.13.2009 PMZ --Updated the file 

to indicate that the system uses 

only a local, private LAN. It does 

not use any data transmission nor 

does it allow any external 

connections. 

Accepted 3/13/2009 

SJ: Updated 

documentation 

referring system uses 

only a local private 

LAN, not using any 

data transmission 

and not allowing any 

external connections 

in "ES&S System 

Security Specification 

Ver. Rel. 4.0.0.0 

date 02/20/09"

74 2/3/09 Kwil

son

Document 

Defect

Closed Security 

review- HPM

ES&S System 

Functionality 

Description 

HPM v.1.0

The functional description states this 

requirement is not applicable. HPM 

utilizes an audit log, the existence of 

and read/write capabilities which are 

required to continue processing. 

v.1: 2.2.4.1e: Include built-in 

measurement, self-test, and 

diagnostic software and hardware 

for detecting and reporting the 

system's status and degree of 

operability.

02.18.2009 -- Beth BingerDunaway

Added the following statement to 

1.1.4.j.

"Federal guidelines require 

Election Administrators to 

implement a mandatory election 

security program."

Accepted 3/13/2009 

SJ: Updated the 

documentation 

stating that HPM 

System Audit Log 

contains a log of all 

activity within the 

application in 

"Election Systems & 

Software ES&S 

System Functionality 
VSTL Certification # pending

Page 172 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

Description 

Hardware 

Programming 

Manager (HPM) 

Unity v. 3.2.0.0 v2.0"

75 2/3/09 Kwil

son

Document 

Defect

Closed Security 

review-EDM

System 

Security Spec 

v4.0.0.0 

7/8/2008

Pre-election security considerations 

(pg 14) use the word "Ideally" and 

"should" which is not mandatory in 

reference to physical protection of 

paper or electronic ballots.

v.1: 6.2.1.2.b Specify whether an 

individual‘s authorization is limited to 

a specific time, time interval or 

phase of the voting or counting 

operations

2.13.2009 PMZ -- Added verbiage 

to indicate to the jurisdiction that 

any lessening of security 

recommendations is a weakening 

of the jurisdiction's mandatory 

security program. Strengthened 

language throughout the document 

to indicate that jurisdiction 

adaption security procedures are 

mandatory.

Accepted 3/13/209 

SJ: Updated the 

documentation for 

pre-election security 

considerations 

mandatory in "ES&S 

System Security 

Specification Ver. 

Rel. 4.0.0.0 date 

02/20/09"

76 2/4/09 Kwil

son

Document 

Defect

Closed Security 

review-ERM

ES&S System 

Functionality 

Description 

ERM v.1

ES&S ERM 

System Ops 

Procedures 

Ver Rel 

7.5.2.0 

10/17/08

The ERM functional description states 

that security "can be achieved through 

the standard use of Windows network 

security to limit access." This statement 

contains no mandatory verbs. 

Combined with non-mandatory 

programmatic access control (ERM 

operations procedure) as well as nonmandatory OS access control, Unity 

v.3.2.0.0 does not meet VSS 

requirements for system integrity, 

confidentiality or accountability

v.1: 2.2.1a & g: To ensure security 

all systems shall: 

a) Provide security access controls 

that limit or detect access to critical 

system components to guard 

against loss of system integrity, 

availability, confidentiality, and 

accountability.

g) Provide documentation of 

mandatory administrative 

procedures for effective system 

security. 

02.18.2009 -- Beth BingerDunaway

Updated 1.1.1.a to include the 

following statement:

"Further security of the installed 

programs as well as external 

access to the files is achieved 

through the standard use of 

Windows network security to limit 

access. Federal guidelines require 

Election Administrators to 

implement a mandatory election 

security program."

Accepted 3/13/2009 

SJ: Updated the 

documentation with 

mandatory 

administrative 

procedures in "ES&S 

System Security 

Specification Ver. 

Rel. 4.0.0.0 date 

02/20/09"

77 2/4/09 Sjaki

leti

Document 

Defect

Closed Security 

review-EDM

ES&S System 

Security Spec 

Ver Rel 

4.0.0.0 dated 

7/8/08

ES&S HPM 

System Ops 

Procedures 

Ver Rel 

5.7.0.0 

10/17/08

ES&S ERM 

System Ops 

Procedures 

Ver Rel 

7.5.2.0 

10/17/08

Pg 29 of the security specification 

states that no compilers are present 

after election day testing. The HPM 

and ERM operations procedures 

describe installation of "RM/Cobol 

11.01" which appears to be a compiler.

Rejected 3/5/09 SJ The response from 

ES&S clarifies the issue but does not 

address the lack of clarity in the 

documents.

v.1: 6.4.1e: After initiation of election 

day testing, no source code or 

compilers or assemblers shall be 

resident or accessible.

2.13.2009 PMZ -- The RM/COBOL 

run-time install installs the run-time 

executable library. It does not 

install a compiler.

3-31-09 DJZ - Added note 

clarifying RM/COBOL does not 

install a compiler. 

04.01.2009 BB -- Added note 

clarifying that the install if 

RM/Cobol is not installing a 

compiler.

Accepted SJ: 

4/14/2009: Updated 

documentation with 

RM/COBOL install 

process, clarified 

installs only the runtime library not the 

compiler in "ES&S 

ERM System 

Operations 

Procedures Version 

Release 7.5.3.0" 

4/3/09 and "ES&S 

HPM System 

Operations 

Procedures Ver. Rel. 

5.7.0.0" 4/3/09

Reject 3/5//09 SJ 

78 2/3/09 Sjaki Docu- Closed Security The functional description does not v.1: 2.2.5.3 To counter these 4.3.2009 MDN -- Updated Section Accepted SJ 
VSTL Certification # pending

Page 173 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

leti ment 

Defect

review-M650

ES&S System 

Functionality 

Description 

Model 650 

v.1.0

provide the documentation of the 

correct configuration of the COTS 

Operating System .

vulnerabilities three operating 

system protections are required 

...Third, the system shall be 

configured to execute only intended 

and necessary processes during the 

execution of election software. 

1.1.5.2 to indicate that the system 

executes only intended and 

necessary procedures during an 

election.

4/14/2009: Updated 

M650 configuration 

documentation in 

ES&S System 

Functionality 

Description Model 

650 Unity v. 

3.2.0.0"v2.0

79 2/3/09 Kwil

son

Document 

Defect

Closed Security 

review-EDM

ES&S System 

Security 

Specification 

Ver Rel 

4.0.0.0

The EDM documentation does not 

provide a detailed description of 

special protocols. 

v.1: 6.2.2f: Vendors shall provide a 

detailed description of all system 

access control measures designed 

to permit authorized access to the 

system and prevent unauthorized

access. Examples of such 

measures include: Special protocols

4.3.2009 MDN -- Added a security 

protocol section to Part 2, Chapter 

5 of the Election Data Manager 

System Operations Procedures 

manual listing critical procedures 

that must be implemented to limit 

access controls to authorized 

users.

Accepted SJ 

4/14/2009: Verified 

documentation for 

security protocols for 

EDM setup 

preventing the 

unauthorized access 

is in ES&S EDM 

System Operations 

Procedures Version 

Release 7.8.1.0 

4/3/09

80 2/3/09 Kwil

son

Document 

Defect

Closed Security 

review-EDM

ES&S System 

Functionality 

Description 

EDM v1.0

Correction: 

ES&S System 

Functionality 

Description 

ERM Unity v. 

3.2.0.0 v1.0

There are no references to Cobol 

documentation to support the claim 

that EDC is performed by Cobol.

4/14/09 SJ Rejected: This discrepancy 

cited the wrong application. It should 

have identified ERM 

v.1: 2.2.2.1d: Include control logic 

and data processing methods 

incorporating parity and check-sums 

(or equivalent error detection and 

correction methods) to demonstrate 

that the system has been designed 

for accuracy, and

v.1: 2.2.2.1e: Provide software that 

monitors the overall quality of data 

read-write and transfer quality 

status, checking the number and 

types of errors that occur in any of

the relevant operations on data and 

how they were corrected.

04.03.09 CJ - Cobol is not used in 

the development of the EDM 

application. It is not referenced in 

either the related SFD or SDS 

documents. For further information 

please refer to the System 

Overview Unity v.3.2.0.0 and the 

ES&S Software Design 

Specifications Election Data 

Manager (EDM) Unity v. 3.2.0.0 

documents.

05.05.2009 - Beth BingerDunaway Updated d and e on 

page 7 to include the following 

information:

Handled by PC operating system 

and Cobol. All data I/O is handled 

thru the Cobol runtime and reports 

back any unsuccessful data I/O. 

Please see the following sections 

of this document for additional 

information regarding RMCOBOL 

errors:

Data Reference Errors (page 67)

Input/Output Errors (page 76)

File Manager Detected Error 

Codes (page 97)

System Initialization Messages 

(page101)

5/15/09 KS - Accept

Verified ERM SFD, 

dated 5/11/09, Rev3 

contains updated 

information for d & e 

on pg. 7, and 

additional sections 

for RMCobol Errors 

starting on pg 68-

106. 

Peer review KW 

5/19/2009 -- OK

Rejected SJ 

4/14/2009

81 2/4/09 Sjaki Docu- Closed Security There is no detailed documentation for v.1: 6.2.2g: Vendors shall provide a 04.03.09 MDN - Access controls Accepted SJ 
VSTL Certification # pending

Page 174 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

leti ment 

Defect

review-DS200

ES&S System 

Security 

Specification 

Ver Rel 

4.0.0.0

the DS200 Encryption process in the 

security specification. 

detailed description of all system 

access control measures designed 

to permit authorized access to the 

system and prevent unauthorized 

access. Examples of such 

measures include: Message 

encryption 

are described in System 

Operations Procedures, the ES&S 

System Security Specification and 

System Hardening Documentation.

4/14/2009 : Updated 

the documentation 

stating that unity 

3.2.0.0 does not use 

transmission. As 

such encryption 

does not apply in 

ES&S System 

Security Specification 

Ver. Rel. 4.0.0.0 

3/27/09

82 2/4/09 Sjaki

leti

Document 

Defect

Closed Security 

review-M650

ES&S System 

Security 

Specification 

Ver. Rel. 

4.0.0.0

There is no detailed documentation for 

the M650 Encryption process in the 

security specification. 

v.1: 6.2.2g: Vendors shall provide a 

detailed description of all system 

access control measures designed 

to permit authorized access to the 

system and prevent unauthorized 

access. Examples of such 

measures include: Message 

encryption 

04.03.09 MDN - Access controls 

are described in System 

Operations Procedures, the ES&S 

System Security Specification and 

System Hardening Documentation.

Accepted SJ 

4/14/2009 : Updated 

the documentation 

stating that unity 

3.2.0.0 does not use 

transmission. As 

such encryption 

does not apply in 

ES&S System 

Security Specification 

Ver Rel. 4.0.0.0 

3/27/09

83 2/3/09 Kwil

son

Document 

Defect

Closed Security 

review-EDM

ES&S System 

Functionality 

Description 

EDM v.1.0

The EDM functional description states 

that this requirement is not applicable . 

No justification is provided for this 

statement. 

v.1: 2.2.1e: Provide security 

provisions that are compatible with 

the procedure and administrative 

tasks involved in equipment 

preparation, testing, and operation.

04.03.09 CJ - The N/A entry to this 

item was originally construed as 

only relating to 'equipment' but 

VVSG V1. 2.1 states "System 

Wide" in nature, pertaining to all 

hardware or software. This will 

affect the following change to 

Section 1.1.1. for this item.

"EDM internal security requires 

log-in passwords to be defined by 

election administrators to limit 

access to the application and data. 

Externally, EDM physical security 

protocols are incorporated by 

election administrators at the 

system and central office 

procedure(s) level."

Accepted SJ 

4/14/2009: Updated 

documentation with 

the procedures and 

administrative tasks 

involved in 

equipment 

preparation,, testing 

and operation in 

ES&S System 

Functionality 

Description EDM 

Unity v. 3.2.0.0 

4/3/09

84 2/3/09 Kwil

son

Document 

Defect

Closed Security 

review-EDM

ES&S System 

Security 

Specification 

Version 

Release 

4.0.0.0

The reviewer could not locate a 

detailed description of the methods 

used to prevent unauthorized access to 

the EDM access control capabilities of 

the system. 

v.1: 6.2.2: Vendors also shall define 

and provide a detailed description of 

the methods used to prevent 

unauthorized access to the access 

control capabilities of the system 

itself.

20090324 PMZ -- It is now 

explicitly stated in the System 

Security manual that all login 

Access Control is done through 

the Windows user credentials and 

that the older features in the 

individual modules of Unity are not 

to be relied on to provide any level 

of access control. See Chapter 3, 

"Access Control 

Recommendations," Item a.

Accepted SJ 

4/14/2009: Updated 

documentation 

providing the 

procedures for 

preventing 

unauthorized access 

to the system in 

ES&S System 

Security Specification 

Version Release 
VSTL Certification # pending

Page 175 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

4.0.0.0 3/27/09

85 2/18/09 J 

Garc

ia

Document 

Defect

Closed System 

Limitations 

Unity v.3.2.0.0 

2/11/09

The table in Section 1 Unity System 

Limits does not correctly reflect all 

applicable application limits. 

The table does not reflect the following 

application limits that limit the entire 

system.:

- HPM limit of 2200 precincts per 

polling place

- HPM limit of 1200 candidates per 

polling place

- ERM limit of 1900 for all precincts 

included in a poll

- M650 limit of 3750 maximum 

candidate/counters allowed per 

election

- HPM limit of 1639 ballot styles 

allowed for an election coded by style

V1: 7.7.a

Vendors are required to produce 

documentation ...

a. Be sufficient to serve the needs of 

the VSTL, voters, election officials, 

and maintenance technicians;

Accepted,3/20/09 JG

Verified doc System 

Limitations Unity 

v.3.2.0.0 Feb. 23, 

2009

Section 1 Unity 

System Limits 

correctly reflect the 

HPM limit of 2200 

precincts per polling 

place, HPM limit of 

1200 candidates per 

polling place, ERM 

limit of 1900 for all 

precincts included in 

a poll, and the M650 

limit of 3750 

maximum 

candidate/counters 

allowed per election

86 2/27/09 K. 

Swift

InformationalClosed AutoMARK 

VAT Version 

labels , VAT 

hardware and 

ESS 

AutoMARK 

Models and 

Hardware 

Revisions 

document

VAT configuration OS label, HW 

revision labels, installed configuration 

and configuration document are 

inconsistent. 

OS label and installed software version 

match but the document version does 

not match. OS Label & Display 

Software Version show OS WinCE 

5.0.1400; ESS AutoMARK Models and 

Hardware Revisions document says it 

should be WinCE 5.00.14

SN: AM0106430376 & AM0206443384 

OS label and document match but the 

installed software version does not 

match: OS Label & doc : 5.00.19, 

Display SW Version screen displays 

Wince 5.0.0 SN: AM0208470815 & 

SN: AM0208470767 

Document and installed HW Revision 

do not match the HW Rev label (on 

back): HW Rev label displays 1.3.1.0, 

Display SW Version screen displays 

HW Rev 1.3.1

SN: AM0208470767

5/21/09 KS Partial Reject

Item 1 above; SN: 376 & 384; Display 

200900408 SUE -- This item will 

be resolved with VAT version 

1.3.2905.

5.26.09 slm - ES&S will provide 

the label for the VAT that was 

incorrectly marked. ES&S also 

provided the VAT hardware sheet 

listing all of the four different 

hardware configurations.

Accepted 5/27/09 

KS -Verified a new 

document has been 

submitted with the 

correct OS for SN: 

376 & 384. A new 

label with the correct 

hardware rev has 

been applied to SN: 

767

5/21/09 KS -

Verified itsm 2 for 

SN: 815 & 767, the 

correct OS WinCE 

version (5.00.19) is 

displayed in Software 

versions
VSTL Certification # pending

Page 176 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

Software Versions still displays the 

Platform as "WinCE 5.0.1400". The 

ESS AutoMARK Models and Hardware 

Revisions document states the version 

should display as "WinCE 5.00.14".

Item 3 above; SN: 767; the hardware 

label on the back of the VAT still reads 

as "1.3.1.0", it should read as 1.3.1.

Verified itsm 2 for SN: 815 & 767, the 

correct OS WinCE version (5.00.19) is 

displayed in Software versions

87 3/4/09 C 

Cog

gins

Functional 

Defect

Closed DS-200 

System 

Launch

The 3 DS-200's received from SysTest 

Labs failed to boot up. 

12/18/08 When attempting to boot up 

the DS-200 the following error was 

displayed: 

"Symlink/lib/modules/2.6.16.27/source(i

node #40637) is invalid. Failure File 

system errors were encountered that 

could not be fixed automatically. This 

system cannot continue to boot and will 

therefore be halted until those errors 

are fixed manually by the system 

administrator. After you press enter 

the system will be halted and power 

off. "

v.1: 2.3.4.1d All systems shall 

provide the capabilities to: verify the 

software and hardware function 

correctly

Accepted C Coggins 

3/4/09 A file system 

error that performed 

a check was 

incorrectly set to 6 

months. ES&S 

provided a script file 

to change the setting 

of Max amount to 

equal 1 and remove 

W-TEMP. iBeta 

reviewed the script 

and restored the 

corrupted compact 

flash using the build 

provided by SysTest 

Labs and ran the 

script. System 

launch was verified. 

88 3/16/09 Sjaki

leti

Document 

Defect

Closed ESSIM 

System 

Operations 

Procedures 

Ver. Rel. 

7.7.0.0 

8/22/08

Chapter 4: Install ES&S Image 

Manager showing the system 

requirements ESSIM requires "Adobe 

Type Manager Light" and " Adobe 

Type Basics".

The requirements for ESSIM COTS 

software has been updated to Adobe 

Acrobat Standard V.9 and to remove 

the requirement for the other Adobe 

software. This is not reflected in the 

document. 

4/15/09 update: ES&S PM verbally 

informed the iBeta PM that the 

requirement for the other Adobe 

software was being retained at the 

request of the ES&S election support 

staff. 

v.1: 8.5.a :The vendor shall establish 

such procedures and related 

conventions, providing a complete 

description of those procedures 

used to:

a. Develop and maintain internally 

developed items;

TMT - 4.3.09 - Chapter 4: Install 

ES&S Image Manager, System 

Requirements. Deleted the bullet 

points regarding Adobe Manager 

Light and Adobe Type Basics. 

Edited the "Adobe Acrobat 

Standard v. 8" to now reflect 

"Adobe Acrobat Standard v. 9". 

Also removed the 3.5 inch disk 

from this requirement.

Accept C. Coggins 

4/15/09 

Verified in ESSIM 

SOP Ver. Rel. 

7.7.0.0 4/3/09 

reflects the update to 

Acrobat Standard v.9 

with the additional 

Adobe software.

89 3/16/09 K. 

Swift

Document Closed AIMS Election 

Official's 

The guide does not indicate that the 

"Setup Ballot Bar Codes" needs to be 

v.2: 2.8

This documentation shall provide all 

4.3.2009 MDN - Updated Section 

5.2 and Section 12.3.3 with 

5/4/2009: Accept SJ: 

Updated 
VSTL Certification # pending

Page 177 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

Defect Guide, Rev 

12, 3/21/08

modified for ballot style 840 and 

multiples of 840.

For ballot style 840 (and multiples of 

840) the code channel needs to 

manually be corrected by clicking off 

positions 40, 41, 42, 43 and 55 and 

clicking on positions 46, 47, 48 and 54. 

The only position that is not changed is 

position 49. This is the corrected Code 

Channel: 11, 21, 46, 47, 48, 49, 54,

which indicate Sequence 840.

information necessary for system 

use by all personnel who support 

pre-election and election 

preparation, polling place activities 

and central counting activities, as 

applicable, with regard to all system 

functions and operations identified in 

Section 2.3 above.

instructions for manually editing 

the ballot code channel for ES&S 

ballot style 840 and multiples of 

840.

4.23.2009 MDN - Delivered 4/23. 

See response above for a 

description of revisions.

documentation with 

manually editing the 

ballot code channel 

for ES&S ballot style 

840 and multiples of 

840 in "AutoMARK 

Information 

Management System 

Election Officials 

Guide Rev14"

Reject 4/15/09 - No 

AIMS documents 

were delivered

90 3/20/09 J 

Garc

ia

Document 

Defect

Closed System 

Limitations 

Unity v.3.2.0.0 

Feb. 23, 2009

The table in Section 1 Unity System 

Limits does not correctly reflect all 

applicable application limits. 

The table does not reflect the following 

application limits that limit the entire 

system.:

-ERM Database Create allows for 1168 

Contests and 1600 Precincts per Ballot 

Style.

v.1: 7.7.a

Vendors are required to produce 

documentation ...

a. Be sufficient to serve the needs of 

the VSTL, voters, election officials, 

and maintenance technicians;

4.3.2009 MDN - The limiting 

application for both of these issues 

is now listed as HPM. Maximum 

contests per ballot style limitation 

has been updated to 1100. 

Maximum precincts allowed per 

ballot style has been updated to 

1700 (ERM was upgraded in 

version 7.2.0.0 to handle up to 

1800 precincts per ballot style, 

making the HPM limit the 

determining factor for the system). 

The system table has been 

updated and the referenced 

limitation statement has been 

removed from the ERM section as 

ERM limitations for these items are 

never approached because of the 

upstream limits imposed by HPM.

Accept C. Coggins 

4/15/09 Verified in 

System Limitations 

rev 5.0 dated 4/6/09 

that Section 1 and 

3.5 (ERM) have been 

updated with the 

newly defined limits. 

91 3/20/09 Sjaki

leti

Document 

Defect

Closed AutoMARK 

Information 

Management 

System 

Election 

Official‘s 

Guide v12.0

AIMS PC requires to install SanDisk 

drivers, this is missing from the guide. 

v.1: 8.5.a :The vendor shall establish 

such procedures and related 

conventions, providing a complete 

description of those procedures 

used to:

a. Develop and maintain internally 

developed items;

4.3.2009 MDN - Added SanDisk 

Drivers as a software requirement 

to Section 3.2.1. 

4.23.2009 MDN - Delivered 4/23.

See response above for a 

description of revisions.

4/15/09 - No AIMS 

documents were 

delivered

5/4/2009: Accept SJ: 

Updated 

documentation with 

additional drivers and 

software needed for 

AIMS PC in 

"AutoMARK 

Information 

Management System 

Election Officials 

Guide Rev14"

92 3/23/09 K 

Swift

Functional 

Defect

Closed HPM v. 

5.7.0.0

Create Final 

Database

An error was encountered in HPM 

which terminated the application, 

without providing direction for 

resolution. 

v.1: 2.2.5.2.2.b

All voting systems shall meet the 

following requirements for error 

messages:

20090408 GLW -- The HPM and 

ERM versions in Unity 4.0 (now 

being used in Unity 3.2) were 

enhanced and compiled to take 

5/14/09 KS -

Accepted

Verified HPM SOP, 

dated 5/1/109 has 
VSTL Certification # pending

Page 178 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

At the point where the user was 

creating the Final Election Database 

(Create Tabulator Data>Create Final 

Database>OK ) Vol 7 Scenario 1 

(IBVOL07P, an Open Primary), the 

following error was encountered: 

"Program "him" terminated with return 

code 253. COBOL error code:109." 

The application was terminated without 

providing instructions. The user looked 

at the HPM SOP Chapter 36 Error 

Messages. Neither code 253 or 

COBOL error code 109 were listed, 

however, there was no loss of data.

b. All error messages requiring 

intervention by an operator or 

precinct official shall be displayed or 

printed unambiguously in easily 

understood language text, or by 

means of other suitable visual 

indicators; 

v.2: 2.8.5.c

The vendor shall provide 

documentation of system operating 

procedures that meets the following 

requirements:

c. Provides procedures that clearly 

enable the operator to intervene the 

system operations to recover from 

an abnormal system state

advantage of a new Cobol 

Runtime feature that became 

available in version 11 of the 

Cobol development and Runtime 

environment. This feature, set on 

during the compilation of the 

source code, instructs the Cobol 

Runtime to validate, at the time of 

program execution, that the 

composite subscript for a data 

reference does not exceed the 

maximum values possible for the 

specific data item referenced. This 

is commonly referred to in the 

VVSG as bounds checking. 

Should the Cobol Runtime 

determine that the current value of 

a data reference item subscript 

exceeds the stated data boundary, 

the Runtime immediately 

terminates the executing program 

and reports Program Exit Code 

253 and error 109 in the Return 

Code Message Box. The program 

termination occurs before our 

application has the ability to report 

this error in the HPM and/or ERM 

System Log. 

ES&S will add all Program Exit 

Codes (generated by the Cobol 

Runtime) and associated Data 

Reference Error numbers to the 

HPM and ERM SOP‘s along with 

the steps the user should take 

should one of these errors be 

reported by the Cobol Runtime. In 

most cases, the suggested action 

will be to call ES&S Technical 

Support as the occurrence of 

Runtime errors is generally 

indicative of either system 

integration issues and election 

data that far exceeds the 

documented System Limitations. 

DJZ 5-11-09 HPM SOP- Added 

COBOL error tables to the 

Appendix. 

been updated to 

include Cobol error 

tables in Ch. 36: 

Error Messages.

93 3/23/09 K Func- Closed HPM v. The HPM audit log did not report an v.1: 2.2.5.2.1.a: All voting systems 20090408 GLW -- The HPM and /18/09 C Coggins 
VSTL Certification # pending

Page 179 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

Swift tional 

Defect

5.7.0.0

Audit Log

error that resulted in the termination of 

HPM. 

Error Message: "Program "him" 

terminated with return code 253. 

COBOL error code:109." was not 

captured in the HPM audit log, nor any 

other information related to this error, 

however, there was no loss of data.

shall meet the following 

requirements ...of audit records: 

Except where noted, systems shall 

provide the capability to create and 

maintain a real-time audit record. 

This capability records and provides 

the operator of precinct official with 

continuous updates on machine 

status. This information allows 

effective operator identification of an 

error condition requiring intervention 

...

ERM versions in Unity 4.0 (now 

being used in Unity 3.2) were 

enhanced and compiled to take 

advantage of a new Cobol 

Runtime feature that became 

available in version 11 of the 

Cobol development and Runtime 

environment. This feature, set on 

during the compilation of the 

source code, instructs the Cobol 

Runtime to validate, at the time of 

program execution, that the 

composite subscript for a data 

reference does not exceed the 

maximum values possible for the 

specific data item referenced. This 

is commonly referred to in the 

VVSG as bounds checking. 

Should the Cobol Runtime 

determine that the current value of 

a data reference item subscript 

exceeds the stated data boundary, 

the Runtime immediately 

terminates the executing program 

and reports Program Exit Code 

253 and error 109 in the Return 

Code Message Box. The program 

termination occurs before our 

application has the ability to report 

this error in the HPM and/or ERM 

System Log. 

ES&S will add all Program Exit 

Codes (generated by the Cobol 

Runtime) and associated Data 

Reference Error numbers to the 

HPM and ERM SOP‘s along with 

the steps the user should take 

should one of these errors be 

reported by the Cobol Runtime. In 

most cases, the suggested action 

will be to call ES&S Technical 

Support as the occurrence of 

Runtime errors is generally 

indicative of either system 

integration issues and election 

data that far exceeds the 

documented System Limitations. 

DJZ 5-11-09 HPM SOP- Added 

Accept 

Explanation is 

accepted as error 

handling for COBOL 

is consistent with v.1: 

4.2.3 
VSTL Certification # pending

Page 180 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

COBOL error tables to the 

Appendix. 

94 3/24/09 C 

Cog

gins

InformationalClosed ESS System 

Limitations 

v.3.0

(version 

control)

ES&S is submitting 2 separate 

document files for a document title and 

version "System Limitations v.3.0". 

These documents are inconsistent. 

The files named 

U3200_OVR02_SystemLimitations_20

090223 and U3200_LMT_20090211 

have the same document title with the 

Table of Changes reflecting the same 

version and date. 

U3200_OVR02_SystemLimitations_20

090223 does not appear to be 

following the configuration 

management policy as the date on the 

title page is not reflected in the Table of 

Changes. This document contains 

additional system limits in Section 1 

and a new section, Section 2.3 

addressing the VAT. Due to these 

inconsistencies and the differences in 

file names it is unclear if 

U3200_OVR02_SystemLimitations_20

090223 is a replacement for 

U3200_LMT_20090211 . If so, the 

configuration policy does not provide 

guidance on how ES&S staff should 

initiate changes to a baseline name. 

4.3.2009 MDN - Configuration 

Items for this file were not properly 

updated for submission. See the 

Limitations document included with 

the current submission. 

ES&S altered document file 

naming conventions and CM policy 

to leverage Microsoft SharePoint 

as a document configuration 

management tool. The ES&S 

Configuration Management Plan 

has been updated to describe 

revised revision procedures and 

file naming requirements. The 

date and revision number included 

with the Table of Changes traces 

changes to document contents. 

The cover date is an automated 

field that reflects the document 

print date/VSTL submission date. 

See the revised CM Plan for 

additional details.

5.26.2009 MDN - Updated the 

ES&S Configuration Management 

Plan to reference a new file titled 

"U3200_PRE04_TDPOrgandAbstr

acts." This newly added file 

includes a description of TDP 

organization and file naming 

conventions as well as an abstract 

that includes a description of 

included files for each TDP 

section. This file is included under 

the 00_PREFACE section.

6/1/09 CEC Accept 

U3200_PRE04_TDP

OrgandAbstracts 

clarifies the changes 

to the document 

naming

95 4/1/09 K. 

Wils

on

Document 

Defect

Closed Hardening 

Procedures 

for

the Election 

Management

System PC 

3/27/2009

An unavailable tool is identified in the 

procedures.

The tool referred to in Ch 5 step 2: 

ng_scoring_tool-gui-1.0-win32.exe. is 

no longer available: see 

http://members.cisecurity.org/kb/article.

php?id=029

v.2: 2.6.4 The vendor shall provide a 

detailed description of the system 

capabilities and mandatory 

procedures for purchasing 

jurisdictions to ensure secure 

software (including firmware) 

installation to meet the specific 

requirements of Volume I, Section 

6.4 of the Standards. This 

information shall address software 

installation for all system 

components.

4.7.2009 MDN --The procedure for 

hardening validation and reference 

to the cited tool have been 

removed.

Accepted 4/16/2009 

SJ: Updated 

documentation by 

removing the 

unavailable tool in 

Hardening 

Procedures for the 

Election 

Management System 

PC v3.2.0.0 3/27/09

96 4/1/09 K. 

Wils

Document Closed Hardening 

Procedures 

The following items referenced in the 

document need to be delivered for 

v.2: 6.6. b The test agency shall 

examine the vendor's source code 

4.7.2009 MDN -- Added a step to 

Chapter 4 (current page 15) 

Accept 4/16/2009 K. 

Wilson following the 
VSTL Certification # pending

Page 181 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

on Defect for

the Election 

Management

System PC 

dated 

3/27/2009

review (as installation code). Either 

SecuritySetupSelfExtractor.exe or the 

individual files that it may contain such 

as 1) 080407a.inf, 2) secure.reg, 3) 

Registry.pol, 4) secure.cmd, & 5) 

local.cmd.

Following the implementation of the 

procedure in Chapter 4, if any 

executable or scripts resulting from 

non-COTS source code is left on the 

computer, the source code needs to 

meet EAC code review standards.

against the submitted 

documentation during the Physical 

Configuration Audit to verify that the 

software conforms to the vendor's 

specifications

requiring users to delete the 

contents of the c:\temp folder; 

removing all system hardening 

tools after use.

Delivery of the tools to the VSTL is 

forthcoming.

procedure to harden 

the E018 laptop, the 

following files appear 

in the c:\temp folder: 

080407a.inf, 

Registry.pol, 

secsetup.sdb, 

secure.cmd, and 

selfextractorReadMe.

txt (the deletion of 

the temp folder in Ch. 

4 was done before 

the folder was used 

in Ch 5)

97 4/3/09 K. 

Swift

Document 

Defect

Closed HPM SOP, 

Ver. Rel. 

5.7.0.0

1/30/09

The HPM SOP provides conflicting 

instructions for the M650. 

Ch. 2 M650, pg. 22, Step 2; instructs 

user to 'Create Tabulators 

Parameters", and select '650' in the 

Tabulator box. There is no 650 option. 

The SOP references Chapter 23 which 

advises the user that the parameters 

are automatically created when the 

final database is created. 

v.2: 2.8 This documentation shall 

provide all information necessary for 

system use by all personnel who 

support pre-election and election 

preparation, polling place activities 

and central counting activities, as 

applicable, with regard to all system 

functions and operations identified in 

Section 2.3...

4-7-09 DJZ - Took out incorrect 

text in Chapter 2, referring to using 

the Create Tabulators for the 650. 

Put in NOTE : stating (If you are 

using Model 650, the parameter 

file was created when you created 

the final database, so you do not 

need to create tabulator 

parameters. 

Accept C Coggins 

4/15/09

Verified HPM SOP, 

Ver. Rel. 5.7.0.0 

4/3/09 instruction to 

"Create Tabulators 

Parameters" has 

been deleted. 

98 4/6/09 K. 

Swift

Functional

Defect

Closed ERM Printing 

the Precinct 

Summary 

Report 

& ERM SOP, 

Ver. Rel. 

7.5.2.0 

1/30/09

When attempting to print the Precinct 

summary report, an error was 

generated which did not provide 

direction for correction. 

A summary report exceeding 5000 

pages was requested. The following 

error was generated. "Pgm Aero 

terminated with return code 253; Cobol 

error code 109". The application 

closed without identifying the action to 

be performed. The user looked at the 

ERM SOP Error Messages. Neither 

code 253 or COBOL error code 109 

were listed, however, there was no 

loss of data.

v.1: 2.2.5.2.2.e The voting system 

shall meet the following 

requirements for error messages: 

The message cue for all systems 

shall clearly state the action to be 

performed in the event that voter or 

operator response is required. 

20090408 GLW -- The HPM and 

ERM versions in Unity 4.0 (now 

being used in Unity 3.2) were 

enhanced and compiled to take 

advantage of a new Cobol 

Runtime feature that became 

available in version 11 of the 

Cobol development and Runtime 

environment. This feature, set on 

during the compilation of the 

source code, instructs the Cobol 

Runtime to validate, at the time of 

program execution, that the 

composite subscript for a data 

reference does not exceed the 

maximum values possible for the 

specific data item referenced. This 

is commonly referred to in the 

VVSG as bounds checking. 

Should the Cobol Runtime 

determine that the current value of 

a data reference item subscript 

exceeds the stated data boundary, 

the Runtime immediately 

terminates the executing program 

5/14/09 KS -

Accepted

Verified ERM SOP, 

dated 5/1/109, has 

been updated to 

include Cobol error 

tables in Ch. 60: 

Error Messages.
VSTL Certification # pending

Page 182 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

and reports Program Exit Code 

253 and error 109 in the Return 

Code Message Box. The program 

termination occurs before our 

application has the ability to report 

this error in the HPM and/or ERM 

System Log. 

ES&S will add all Program Exit 

Codes (generated by the Cobol 

Runtime) and associated Data 

Reference Error numbers to the 

HPM and ERM SOP‘s along with 

the steps the user should take 

should one of these errors be 

reported by the Cobol Runtime. In 

most cases, the suggested action 

will be to call ES&S Technical 

Support as the occurrence of 

Runtime errors is generally 

indicative of either system 

integration issues and election 

data that far exceeds the 

documented System Limitations. 

DJZ 5-11-09 ERM SOP- Added 

COBOL error tables to the 

Appendix. 

99 4/6/09 K. 

Swift

Functional 

Defect

Closed ERM 7.5.2.0 

Audit Logging 

of Errors 

(V10 S1)

The ERM audit log did not report an 

error that resulted in the termination of 

ERM. 

Error Message: "Pgm Aero terminated 

with return code 253; Cobol error code 

109" was not captured in the ERM 

audit log, nor any other information 

related to this error, however, there 

was no loss of data in ERM.

v.1: 2.2.5.2.1.a: All voting systems 

shall meet the following 

requirements ...of audit records: 

Except where noted, systems shall 

provide the capability to create and 

maintain a real-time audit record. 

This capability records and provides 

the operator of precinct official with 

continuous updates on machine 

status. This information allows 

effective operator identification of an 

error condition requiring intervention 

...

20090408 GLW -- The HPM and 

ERM versions in Unity 4.0 (now 

being used in Unity 3.2) were 

enhanced and compiled to take 

advantage of a new Cobol 

Runtime feature that became 

available in version 11 of the 

Cobol development and Runtime 

environment. This feature, set on 

during the compilation of the 

source code, instructs the Cobol 

Runtime to validate, at the time of 

program execution, that the 

composite subscript for a data 

reference does not exceed the 

maximum values possible for the 

specific data item referenced. This 

is commonly referred to in the 

VVSG as bounds checking. 

Should the Cobol Runtime 

determine that the current value of 

a data reference item subscript 

5/18/09 C Coggins 

Accept

Explanation is 

accepted as error 

handling for COBOL 

is consistent with v.1: 

4.2.3 
VSTL Certification # pending

Page 183 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

exceeds the stated data boundary, 

the Runtime immediately 

terminates the executing program 

and reports Program Exit Code 

253 and error 109 in the Return 

Code Message Box. The program 

termination occurs before our 

application has the ability to report 

this error in the HPM and/or ERM 

System Log. 

ES&S will add all Program Exit 

Codes (generated by the Cobol 

Runtime) and associated Data 

Reference Error numbers to the 

HPM and ERM SOP‘s along with 

the steps the user should take 

should one of these errors be 

reported by the Cobol Runtime. In 

most cases, the suggested action 

will be to call ES&S Technical 

Support as the occurrence of 

Runtime errors is generally 

indicative of either system 

integration issues and election 

data that far exceeds the 

documented System Limitations. 

20090511 DJZ -- ERM SOP 

Added COBO Runtime errors to 

the Appendix.

100 4/6/09 J. 

Garc

ia

Document 

Defect

Closed EDM SOP,

Ver. Rel. 

7.8.0.0 

1/30/09

Election Description maximum field 

length is missing from the 

documentation.

There is a maximum limit of 41 

characters in the Election Description 

field. The Documentation details the 

allowed number of characters for 

Election Name and Election ID 

however, it does not list the allowed 

number of characters for Election 

Description.

v.2: 2.8 This documentation shall 

provide all information necessary for 

system use by all personnel who 

support pre-election and election 

preparation, polling place activities 

and central counting activities, as 

applicable, with regard to all system 

functions and operations identified in 

Section 2.3...

4-7-09 TMT - Updated the 

document to specify a 41 

character limit for the Election 

Description field.

Accept C. Coggins 

4/15/09 Verified on 

pg 63 #3 of EDM 

SOP, Ver. Rel. 

7.8.1.0 4/3/09 the 

field limit of 41 

characters is 

identified

101 4/6/09 J. 

Garc

ia

Functional 

Defect

Closed DS200 FW 

version 

1.3.7.0 

ES010737000

2

&

ES017370025

The Printer boards repeatedly burned 

out on the DS200 scanners with the 

metal ballot boxes.

This was repeated a total of five times 

on two DS200 scanners. The printer 

boards were replaced and firmware 

was loaded. In some instances the 

v.1: 2.5.3.1.b All systems shall 

provide capability to: Produce a 

printed report ...

4.3.2009 MDN - The DS200 meets 

all requirements for withstanding 

ESD and has passed required 

tests. ES&S believes that the 

printer board failures can be traced 

to high static conditions within the 

lag that exceed the system's 

tested limits. The likelihood of 

Accept C Coggins 

4/15/09 Verified the 

"DS200 System 

Maintenance 

Manual" identifies the 

pad and straps. 

These items were not 

observed in use by 
VSTL Certification # pending

Page 184 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

printers would print. At some point the 

report would start to exhibit abnormal 

printing such h as streaks or erratic 

characters. Then a print alarm would 

sound (beeping) and it would not print. 

A printer error instructed the user to 

abort printing. Examination by ES&S 

confirmed that the boards were burned 

out. 

damaging the system during use 

or repair is remote when 

reasonable precautions are 

implemented.

Reasonable precautions include; 

raising the humidity level in the 

lab, using anti-static sprays in the 

lab, wearing an anti-static wrist 

strap when handling system 

components, properly handling 

replacement parts, etc. ES&S 

recommendations for using antistatic pads and wrist straps when 

executing maintenance tasks 

appear in the DS200 System 

Maintenance Manual ―Chapter 1, 

Maintenance Materials,‖ and 

―Chapter 2: Mechanical Checks."

ESS staff when 

performing 

maintenance, 

modem removal or 

repairs on 2/27/09, 

3/17/09 or 3/30/09. 

Repairs performed 

on 4/9/09 by ESS 

staff included the pad 

& strap. 

Raising humidity 

levels and anti-static 

sprays are not 

identified in the TDP 

as required for 

standard 

maintenance or 

operations. 

102 4/6/09 J. 

Garc

ia

InformationalClosed EDM and 

ESSIM field 

limits (Full 

Path to Ballot 

Definition)

When functionally testing the 

documentation changes in discrepancy 

#30 EDM permitted the user to enter a 

Ballot Set name that exceeds the 

ESSIM System limit of 52 characters 

for the file path. This caused an error 

to be generated when opening the 

election in ESSIM.

The tester entered 61 characters into 

the EDM Full Path to Ballot Definition 

field, opened ESSIM, and attempted to 

open the election. An error displayed 

only 52 characters out of the 61 

entered, this provided the information 

necessary to modify the path to ESSIM 

limit. It should be noted that the EDM 

system defaults to a file path with a 

limited number of characters that is 

below the ESSIM 52 character limits. It 

is unlikely that a user would encounter 

this error, but even if they do, the user 

is able to resolve the differences in the 

application field limits. iBeta does not 

consider this inconsistent with the 

requirements of the VSS.

Reject:

4/15/09 C Coggins - While an 

informational issue does not require a 

response, the response provided is 

rejected because the references 

DJZ 4-8-09 - ERM ESSIM 

Chapter 7: New - Added Note to 

state: The limit for characters for 

the Full Path to Ballot Definition 

File created in EDM, is 52 total 

characters in path. More then 52 

characters in the path may cause 

an error. Also in Chapter 46: 

System Messages - Added 

System message for error 

received if more then 52 

characters are used in the Full 

Path to Ballot Definition File 

created in EDM when creating a 

new file. 

TMT - 5.11.09 - ESSIM- The 

comment above should map to 

"ESSIM" rather than "ERM". This 

should cover the ESSIM portion. 

Also EDM Chapter 13: Ballot Sets, 

under Create a Ballot Set, step #7, 

added the sentence that a 

maximum of 52 characters can be 

added to this File. 

5/14/09, KS - Accept

Verified ESSIM SOP, 

dated 4/3/09 contains 

the note/error 

message referenced 

in vendor comments. 

Verified EDM SOP, 

dated 5/11/09, the 

sentence in Ch. 13, 

Step 7 suggesting 

the file name not 

exceed 52 

characters; and the 

removal of the 

statement regarding 

the Ballot Set Name 

limit.

Reject 4/15/09 C 

Coggins
VSTL Certification # pending

Page 185 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

contained in the response were not 

found. Additionally a reference to a 

field limit of 52 characters was found

for the Ballot Set name field on page 

75 #4 of the EDM SOP v.7.8.1.0 

4/3/09. As noted in the ESS response 

the 52 character limit applies to the Full 

Path to Ballot Definition File and not 

the Ballot Set Name field. 

103 4/9/09 K. 

Swift

Document 

Defect

Closed HPM SOP, 

Ver. Rel. 

5.7.0.0

1/30/09

The HPM SOP contains erroneous 

instructions in the Election 

Specification section for the Types 

Description/Include in Total Voting " 

field

Ch. 14, Step 19, pg. 65-66; For the 

"Types Description/Include in Total 

Voting?"; the document states when 

election is defined in the Jurisdiction 

Master System Type, as Central 

Count; the Types Description... field 

will not be active. In testing we 

discovered that the field is active and if 

this field is left blank, then a message 

is displayed during Create Final 

Database that states "Election requires 

at least one included Type Description. 

..."

v.2: 2.8 This documentation shall 

provide all information necessary for 

system use by all personnel who 

support pre-election and election 

preparation, polling place activities 

and central counting activities, as 

applicable, with regard to all system 

functions and operations identified in 

Section 2.3...

MDN 2009.04.16- SOP00_HPM 

Chapter 14 - Addressed with TDP 

revision 3 submitted 04.09.2009 

based on internal review. Step 19 

now includes the verbiage "At 

least one Type description must be 

defined and activated on this 

screen."

5/14/09 KS, Accept

Verified HPM SOP, 

dated 5/11/09; has 

been updated to 

include verbiage as 

stated in vendor 

response.

104 4/9/09 K. 

Swift

Functional 

Defect

Closed ERM v. 

7.5.2.0 

Creating 

results 

database 

v8s1

In HPM when the System Type is set 

to the "Central Count" option in an 

election that is only using an M650 

Central Scanner, an error is 

generated in ERM when creating 

results database.

If the System Type is set to "Central 

Count" in HPM on an election with 

only an M650 (using the M650 limit of 

3750 counters) , opening the election 

in ERM will generate the error "Pgm 

Aero terminated with return code 253; 

Cobol error code 109" and shutdown 

ERM. On attempting to relaunch ERM 

the following messages display:

"Convert Precinct Results File: The 

precincts results file is from older 

software and is being converted." and 

"Error: File: Vol8S1.CTR, Error: #35 -

File does not exist." The election 

cannot proceed. If the System Type is 

v.1: 2.2.6.d An EMS shall generate 

and maintain a database...that 

enables election officials ...to 

perform the following functions: 

generate ballots and electionspecific programs for vote recording 

and vote counting equipment; and

h. generate the post voting reports 

required by section 2.5

MDN 2009.06.16- SOP00_HPM 

Chapter 13 - Updated with TDP 

revision 3 submitted 04.09.2009 

based on internal review. Step 6 

includes the instruction, "From the 

System Type list, select Both. 

This is the only equipment type 

being supported by ES&S. NOTE: 

The Central Count, Precinct Count 

and Mixed system types are not 

supported."

4/23/09 extract from email: The 

109 error during the ERM 

Database create process is the 

result of the implementation of the 

new bounds checker first 

implemented in Unity 4.0 / 3.2. 

When election type is set to 

 ̳Central‘ instructions intended for 

legacy tabulation equipment not 

being certified with this release are 

executed. When the election type 

5/14/09 C Coggins 

Accept that this 

issue is out of scope 

of Unity 3.2.0.0. 

Issue 104 will be 

transferred to Unity 

4.0 as discrepancy 

#1. 
VSTL Certification # pending

Page 186 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

reset to "Both" (i.e. a precinct and a 

central scanner) the election processes 

correctly. However the user is then 

required to complete the definition of a 

non-existence precinct scanner. 

4/20/09 C Coggins iBeta requested 

further clarification of the not supported 

statement. Where is it supported? Is 

there a plan to fix the issue?

is set to  ̳Both‘ in HPM, the ERM 

Database success-fully executes 

intended instructions and stores 

the certification message lines 

entered in HPM into the ERM 

Database. Using election type 

 ̳Both‘ in HPM vs.  ̳Central‘ doesn't 

affect processing in HPM for 

tabulation equipment certified in 

this release. The election type 

setting is only used to disable 

certain menu and screen options 

for precinct based equipment and 

does not affect any tabulator 

parameter files created in HPM or 

data used by ERM to create the 

database. This issue will be fixed 

in Unity 4.0.

105 4/10/09 K. 

Swift

Functional 

Defect

Closed ERM 7.5.2.0 

Audit Logging 

of Errors (V8 

S1)

The ERM audit log did not report errors 

that resulted in the termination of ERM, 

and the startup of ERM. 

Error Message upon termination: "Pgm 

Aero terminated with return code 253; 

Cobol error code 109" 

Error Messages upon launch of ERM: 

"Convert Precinct Results File: The 

precincts results file is from older 

software and is being converted." and 

"Error: File: Vol8S1.CTR, Error: #35 -

File does not exist." 

These messages were not captured in 

the ERM audit log, however, there was 

no loss of data in ERM, as the testers 

did not reach the point of reading 

results into ERM.

v.1: 2.2.5.2.1.a: All voting systems 

shall meet the following 

requirements ...of audit records: 

Except where noted, systems shall 

provide the capability to create and 

maintain a real-time audit record. 

This capability records and provides 

the operator of precinct official with 

continuous updates on machine 

status. This information allows 

effective operator identification of an 

error condition requiring intervention 

...

20090408 GLW -- The HPM and 

ERM versions in Unity 4.0 (now 

being used in Unity 3.2) were 

enhanced and compiled to take 

advantage of a new Cobol 

Runtime feature that became 

available in version 11 of the 

Cobol development and Runtime 

environment. This feature, set on

during the compilation of the 

source code, instructs the Cobol 

Runtime to validate, at the time of 

program execution, that the 

composite subscript for a data 

reference does not exceed the 

maximum values possible for the 

specific data item referenced. This 

is commonly referred to in the 

VVSG as bounds checking. 

Should the Cobol Runtime 

determine that the current value of 

a data reference item subscript 

exceeds the stated data boundary, 

the Runtime immediately 

terminates the executing program 

and reports Program Exit Code 

253 and error 109 in the Return 

Code Message Box. The program 

termination occurs before our 

application has the ability to report 

this error in the HPM and/or ERM 

System Log.

ES&S will add all Program Exit 

5/18/09 C Coggins 

Accept

Explanation is 

accepted as error 

handling for COBOL 

is consistent with v.1: 

4.2.3 
VSTL Certification # pending

Page 187 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

Codes (generated by the Cobol 

Runtime) and associated Data 

Reference Error numbers to the 

HPM and ERM SOP‘s along with 

the steps the user should take 

should one of these errors be 

reported by the Cobol Runtime. In 

most cases, the suggested action 

will be to call ES&S Technical 

Support as the occurrence of 

Runtime errors is generally 

indicative of either system 

integration issues and election 

data that far exceeds the 

documented System Limitations. 

DJZ 5-11-09 ERM SOP- Added 

COBOL error tables to the 

Appendix. 

106 4/10/09 Seat

on

Docum

ent 

Defect

Closed ES&S EDM 

SOP Ver. Rel. 

7.8.0.0 

1/30/09

The EDM Operations Procedures 

documented an incorrect limitation for 

ballot styles within a Absentee precinct.

In Chapter 14 of the documentation it 

states that the EDM Absentee Precinct 

limit for ballot styles per precinct is 40. 

However, the M650 limit is 100 ballot 

styles per absentee precinct. The 

M650 limited of 100 is entered in the 

EDM ballot style per precinct field 

without error.

v.2: 2.8 This documentation shall 

provide all information necessary for 

system use by all personnel who 

support pre-election and election 

preparation, polling place activities 

and central counting activities, as 

applicable, with regard to all system 

functions and operations identified in 

Section 2.3...

TMT - 5.11.09 - EDM - Chapter 14: 

Merge Preferences, changed the 

Absentee Preferences table 

reference to the Maximum Ballot 

Styles Per Precinct column to 100 

characters from 40.

5/14/09 KS - Accept

Verified EDM SOP, 

dated 5/11/09; 

absentee precinct 

limit states 100 in Ch. 

14.

107 4/10/09 Seat

on

Docum

ent 

Defect

Closed ES&S EDM 

SOP Ver. Rel. 

7.8.0.0 

1/30/09

The EDM documentation does not 

state the use of the Assign Sequence 

option.

In Chapter 14 the documentation does 

not mention Assign Sequence and 

there is a screen shot of the Election 

Preferences tab and it does not display 

an option "Assign Sequence" that is in 

the application. the Assign Sequence

is located under the Start Numbering 

heading, on the Election Preferences 

tab in the Merge Preferences option.

v.2: 2.8 This documentation shall 

provide all information necessary for 

system use by all personnel who 

support pre-election and election 

preparation, polling place activities 

and central counting activities, as 

applicable, with regard to all system 

functions and operations identified in 

Section 2.3...

TMT - 5.11.09 - EDM - Chapter 

14: Merge Preferences, see 

Election Preferences for ES&S 

Image Manager screen. The 

screen shot is titled "Election 

Preferences for ES&S Image 

Manager" and has been changed 

to now include the Assign 

Sequence checkbox. This option 

is documented in the table called 

"Election Preferences" in the 

Sequence field in paragraph 2 of 

the Field Definition.

5/14/09 KS - Accept

Verified EDM SOP, 

dated 5/11/09; the 

addition of the screen 

shot containing the 

'Assign Sequence' 

checkbox, and the 

accompanying 

description of use.

108 4/10/09 Seat

on

Docum

ent 

Defect

Closed ES&S HPM 

SOP Ver. Rel. 

5.7.0.0 

1/30/09

The HPM documentation does not 

state the use of the option None in the 

Absentee Type.

In Chapter 16 the documentation 

states, "From the Absentee list, choose 

the applicable absentee type if this 

v.2: 2.8 This documentation shall 

provide all information necessary for 

system use by all personnel who 

support pre-election and election 

preparation, polling place activities 

and central counting activities, as 

applicable, with regard to all system 

DJZ 5-11-09 - SOP00_HPM -

Chapter 16 - Updated page 85, 

and explained None for Precincts. 

None - Will not be flagged as an 

absentee precinct. 

5/14/09 KS - Accept

Verified HPM SOP, 

dated 5/11/09; 

contains a 

description for 

"None" under 

Absentee type.
VSTL Certification # pending

Page 188 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

precinct is

to be designated as an absentee 

precinct." One of the options is "None" 

however it is unclear what none 

means.

functions and operations identified in 

Section 2.3...

109 4/14/09 K. 

Swift

Docum

ent 

Defect

Closed DS200 SOP,

HW v. 1.2.1

FW v. 1.3.7.0

10/17/08

The DS200 documentation does not

list the following error message.

Ch. 11, System Messages does not 

contain the error; "Ballot has been 

counted but not saved to bin. See Poll 

Worker." The problem ended up being 

a ballot was caught up in wires in the 

ballot box. The audit log recorded the 

error as "Ballot jam : ballot counted but 

not saved to bin."

v.2: 2.8.4.d -The vendor shall 

provide documentation of system 

operating features that meets the 

following requirements:

d. Illustrate and describe all status 

indicators and information 

messages.

BB5-11-09 - SOP00_DS200 -

Chapter 11 - Modified the existing 

message to reflect the updated 

verbiage.

5/19/09 Ccoggins 

Accept

Verified message is 

contained in DS200 

May 5, 2009. 

110 4/14/09 S. 

Sivix

ay

Docum

ent 

Defect

Closed ERM SOP, 

Ver. Rel. 

7.5.2.0

1/30/09

The ERM SOP document contains 

obsolete steps. 

Ch. 19 - Close out Precincts with no 

ballots cast, page 110; The window in 

step 1 does not appear when validating 

in ERM.

v.2: 2.8 This documentation shall 

provide all information necessary for 

system use by all personnel who 

support pre-election and election 

preparation, polling place activities 

and central counting activities, as 

applicable, with regard to all system 

functions and operations identified in 

Section 2.3...

DJZ 5-11-09 -Removed screen 

shots the did not refer to the text 

in Chapter 19, page 110. Also 

replaced old screen shots with 

newer ones for the Close out 

Precinct section

5/14/09 KS - Accept

Verified ERM SOP, 

dated 5/5/09; Ch. 19 

has been updated 

with newer screen 

shot and obsolete 

step/screen shot has 

been removed.

111 4/16/09 J. 

Garc

ia

Functional 

Defect

Closed VAT 

A100-00 

WinCE 

5.0.1400

PEB: 1.65, 

SBC: 1.0, FW: 

1.3.2905

The VAT displays an incorrect error 

message when entering an incorrect 

date format.

Testers entered an incorrect date 

format (110309) and a correct date 

format (07:00 AM) however, the error 

that displayed was "Error! Format 

HH:MM AM". 

5/14/09 KS - Reject 

The vendor response does not address 

the discrepancy. The VAT is displaying 

an incorrect error message. When the 

date is entered incorrectly and time is 

entered correctly; the VAT is displaying 

an error against the time. When the 

date is entered correctly and the time is 

entered incorrectly; the VAT is 

displaying an error against the date.

V 1: 2.2.5.2.2.e

All voting systems shall meet the 

following requirements for error 

messages: 

e. The message cue for all systems 

shall clearly state the action to be 

performed in the event that voter or 

operator response is required;

MDN 2009.05.07- AMVAT Poll 

Worker and Jurisdiction GuidesClarified the procedure for 

updating the system time and date 

included under section 6.5.4 of 

AutoMARK 3010 Jurisdiction 

Guide AQS-13-5061-003-R.pdf 

and section 2.1 of AutoMARK 

3010 Poll Workers Guide AQS-13-

5061-002-R.pdf. Instructions now 

explicitly require users to select 

CLEAR prior to entering a new 

time or date setting and select 

APPLY after changing each 

setting. Also added descriptions 

for the correct time and date 

format. 

5.26.09 slm - ES&S strongly 

believes that the System 

Operating Procedures and the 

Readiness Checklist provides the 

end user enough information to 

perform this task correctly. 

Another note is that the system 

does not let you continue or save 

6/1/09 CEC Accept in 

Unity 3.2.0.0 as the 

incorrect date cannot 

be saved; transferred 

to Unity 4.0.0.0 to 

verify correction. 

5/14/09 KS - Reject
VSTL Certification # pending

Page 189 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

an incorrect date. ES&S will 

correct this item in the upcoming 

Unity 4.0 release.

112 4/16/09 J. 

Garc

ia

Docum

ent 

Defect

Closed ES&S 

AutoMARK 

Pollworker's 

Guide 

AQS-13-

5061-002-

R.doc

Rev 8.0 

3/20/08

The documentation does not describe 

the +/- functionality in the printing the 

operation log section.

When attempting to print the audit log 

on the VAT the user selects the text 

box under the Go To Page and a key 

pad is displayed. On the key pad a 

functional option +/- is displayed 

however, there is no information on this 

feature in the documentation. 

v.2: 2.8 This documentation shall 

provide all information necessary for 

system use by all personnel who 

support pre-election and election 

preparation, polling place activities 

and central counting activities, as 

applicable, with regard to all system 

functions and operations identified in 

Section 2.3...

MDN 2009.05.07- AMVAT 

Jurisdiction Guide- The 

AutoMARK uses a canned keypad 

format that includes an 

unsupported +/- button. This 

button cycles between positive (no 

symbol visible) and negative 

(minus sign visible)numeric entry. 

The AutoMARK does not support 

negative entry. Attempting to 

navigate to a page number 

formatted as a negative value 

spawns an error message 

informing the user that the entered 

page number is invalid. Updated 

section 6.5.1 of AutoMARK 3010 

Jurisdiction Guide AQS-13-5061-

003-R.pdf to detail this condition. 

5/14/09 KS - Accept

Verified AM 

Jurisdiction Guide, 

dated 5/7/09, Rev 8; 

has been updated 

with the verbiage 

described in vendor 

response and that 

this is unsupported 

functionality.

113 Closed No issue; the #113 was skipped in 

error

114 4/16/09 K. 

Swift

Functional 

Defect

Closed EDM v. 

7.8.0.0, 

Entering new 

id & password 

(Regress TC)

No password error message was 

generated in Audit Manager when 

creating or in EDM when logging in 

with a password less than 6 or greater 

than 8 characters (see EDM SOP, 

Ch. 55: System Messages)

Passwords for EDM & ESSIM are 

created in Audit Manager. In order to 

validate an error would be generated if 

user IDs were created with passwords 

that did not meet the required 

configuration, passwords less than 6 

(4 or 5 ) and greater than 8 (9) 

characters were entered. No errors 

were generated in either Audit 

Manager or EDM when an ID was 

created or used a non-compliant 

passwords. The only error occurred 

when attempting to create an ID was 

without a password. 

v.2: 6.4.1.a Specific activities to be 

conducted by the ITA shall include: 

A review of the vendor‘s access 

control policies, procedures and 

system capabilities to confirm that all 

requirements of V.1 Sect. 6.2 have 

been addressed completely; 

v.1: 6.2.2 a. Use of data and user 

authorization; & h. Controlled 

access security.

TMT - 5.11.09 - EDM - Chapter 

55: System Messages. Deleted 

these 2 messages from the Error 

Messages table: "Minimum 

Password Length is 6 characters" 

and "Maximum Password Length 

is 8 characters".

5/14/09 KS - Accept

Verified EDM SOP, 

dated 5/11/09; that 

these messages now 

state that they are 

applicable to the 

"iVotronic only". 

From this statement, 

the VSTL assumes 

that 1) there are no 

required password 

configuration 

limitations for ids to 

log onto EDM or 

ESSIM; and 2) the 

6/8 character 

configuration applies 

to setting function 

screen passwords 

(ex: service menu, 

etc.) for the iVotronic.

115 4/16/09 Step

h & 

Kelly

Docum

ent 

Defect

Closed EDM SOP, 

Ver. Rel. 

7.8.1.0, April 

3, 2009 

Ch. 39: 

Text/Referend

um 

EDM application Referendum 

/Question fields do not match the EDM 

SOP

The following appears in EDM, but not 

in the SOP: 

- Text/Referendum menu > 

v.2: 2.8.4.a The vendor shall provide 

documentation of system operating 

features that meets the following 

requirements:

a. Provides a detailed description of 

all input, output, control, and display 

features accessible to the operator 

TMT - 5.11.09 - EDM - Chapter 

39: Text/Referendum. In the 

Create Ballot Text section, edited 

the Question/Text screen to add 

the check boxes and non-grayed 

out options. 

6/12/09 CEC Accept

Verified step #6 

addressed 

Explanation File 

Name field activation 

and #8 addressed 

Rotate Ref/ Question
VSTL Certification # pending

Page 190 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

Referendum/Question section,

- Referendum/Question (checkbox)

- Recall/Retain (grayed out)

- Rotate Ref/Question (grayed out)

- Explanation (grayed out)

The Query Overvote option's in the 

SOP but not in EDM.

5/14/09 KS - Reject - EDM SOP dated, 

5/11/09; Ch. 39 does not explain 

why/how to use the checkboxes: 

Rotate Referendum/Question and 

Explanation. Query Overvote is listed 

in the document, but does not display 

in the application.

6/1/09 Reject CEC -The EDM SOP 

May 26, 2009 does not reflect the step 

2 or step 8 changes identified in the 

5/26/09 response. 

or voter TMT - 5.26.09 - EDM - Chapter 39: 

Text/Referendum. Added note on 

step #2 of Create Ballot Text 

stating that the Explanation check 

box must be checked to use the 

functionality for the Explanation 

File Name field. Also added the 

Rotate Ref/Questions checkbox 

information to step #8. Also 

removed the Query Overvote 

option which is not compatible with 

this version of EDM.

TMT - 6.5.09 - EM SOP - Chapter 

39: Text/Referendum. Added note 

on step #2 of Create Ballot Text 

stating that the Explanation check 

box must be checked to use the 

functionality for the Explanation 

File Name field. Also added the 

Rotate Ref/Questions information 

as the first bullet in step #8. The 

2nd bullet in step 8 refers to the 

Query Undervote; the Query 

Overvote functionality was 

removed. 

6/1/09 CEC Reject

5/14/09 KS - Partial 

Accept & Reject

Verified EDM SOP, 

dated 5/11/09; the 

following checkboxes 

and their use is 

documented in Ch. 

39: Referendum/ 

Question and Recall/ 

Retain.

116 4/20/09 J. 

Garc

ia

Functional 

Defect

Closed Hardware 

Programming 

Manager, 

5.6.2.0

Audit Log

(vol2S4)

An error message that was displayed 

when processing more than the 

allowed number of ballot styles was 

not logged into the audit log.

We attempting to process 101 ballot 

styles in an Absentee Precinct an error 

occurred "Precinct 0004 has more than 

100 ballot styles, only 100 used!". 

However, the error was not logged in 

the system log.

5/14/09 KS - Partial Accept 

Verified EDM SOP, dated 5/11/09; the 

following checkboxes and their use is 

documented in Ch. 39: 

Referendum/Question and 

Recall/Retain.

5/14/09 KS - Reject

Vendor response does not address 

that the error message was not written 

to the system log. Note, HPM 

application did not terminate after 

receiving the message and clicking 

OK. 

v.1: 2.2.5.2.2a All voting systems 

shall meet the following 

requirements for error messages: a. 

The system shall generate, store ... 

all error messages as they occur.

GW 2009.05.07 - In this particular 

case, even though HPM 

determines that precinct 0004 

exceeds the maximum of 100 

ballot styles in a single precinct, 

the program execution continues 

after the user clicks the  ̳OK‘. The 

intent here is to continue on with 

the program execution to see if 

any additional system maximums 

are exceeded in any other 

precincts so that all precincts 

exceeding the maximum can be 

identified and then corrected at 

one time by the HPM user versus 

just telling them about the first 

maximum then terminating the 

program. 

slm 05.26.09 - The message that 

was received during the Vol2, S4 

testing is covered under V1: 

2.2.5.2.3 - Status Message. This 

appears as an informational 

message to the user that the 

6/6/09 Accept JG & 

CEC

As the users has a 

choice to continue or 

not, we agree that 

classification as a 

warning is 

appropriate. 

Warnings need not 

be written to the audit 

log. 

5/14/09 KS - Reject
VSTL Certification # pending

Page 191 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

system was only able to use 100 

ballot styles.

117 4/20/09 Step

h & 

Kelly

Docum

ent 

Defect

Closed EDM SOP, 

Ver. Rel. 

7.8.1.0, 

4/3/09 

Ch. 50: 

Generate 

Ballot Styles

A field listed in the documentation does 

not display in the application.

In the Merge Ballot Sets option, the 

field "Create Ballot Style Information" 

checkbox is not displayed, however it 

is detailed in the document.

5/14/09 KS - Reject

ESSIM was the selected equipment 

type and the field did not display. The 

Note added in Step 3 of Ch. 50 is 

inconsistent with this behavior.

v.2: 2.8.4.a The vendor shall provide 

documentation of system operating 

features that meets the following 

requirements:

a. Provides a detailed description of 

all input, output, control, and display 

features accessible to the operator 

or voter

TMT - 5.11.09 - EDM - Chapter 

50: Generate Ballot Styles.

Added a note in Step 3 stating: 

The Create Ballot Style 

Information check box will appear 

only if you are using the following 

equipment types: DSIM, ESSIM, 

iVotronic bitmap, iVotronic LS, 

Votronic Text.

TMT - 5.26.09 - EDM - Chapter 

50: Generate Ballot Styles. In 

Step #3, changed the note to 

exclude ESSIM as an equipment 

type.

Accept 6/1/09 CEC 

verified step 3 

information about of 

checkbox excludes 

ESSIM

5/14/09 KS - Reject

118 4/20/09 K. 

Wils

on

Docum

ent 

Defect

Closed Hardening 

Procedures 

for the EMS 

PC 4/20/09 

Hardening procedures do not specify 

the folder(s) to lock down on the AIMS 

PC. 

Page 4 identifies that all PC's in the 

EMS environment are hardened 

according to the procedures. 

Jurisdictions may choose to run one or 

all Unity EMS applications on a single 

or multiple stand alone PCs. AIMS 

must run on a separate PC, The 

definition of the folders to lock down 

the AIMS PC are not defined. Access 

control to the AIMS computer is not 

addressed on page 6 item c.

v.1:2.2.1.f If access to a system 

function is to be restricted or 

controlled, the system shall 

incorporate a means of 

implementing this capability.

MM 2009.05.6- Added section 

detailing the requested information 

to the document titled "Hardening 

and Verification of the AutoMARK 

Information Management (AIMS) 

PC."

5/15/09 KS - Accept

Verified Hardening 

Procedures for the 

Election 

Management

System PC, dated 

5/5/09; Pg 9 has 

been updated 

referring the user to 

Ch.2 to harden the 

AIMS PC, explaining 

that only the 

ElectAdmin & 

ElectDefine roles 

should be activated, 

and because the two 

share the same 

privileges, directory 

lock down is not 

necessary.

119 4/20/09 K. 

Wils

on

Docum

ent 

Defect

Closed Hardening 

Procedures 

for the EMS 

PC 4/20/09 

The Role to execute HPM functionality 

is ambiguous

Document states on pg 6 item c that 

HPM is accessed through the 

ElectDefine group, but on page 7, item 

6.c the HPM folder is assigned access 

rights to groups ElectAdmin and 

ElectResults. Based on these 

statements it is unclear what actual 

role is used to execute the HPM 

functionality.

v.1:2.2.1.f If access to a system 

function is to be restricted or 

controlled, the system shall 

incorporate a means of 

implementing this capability.

MM 2009.05.6- Updated the 

relevant step on Page 6 of the 

document to read: 

Folder C:\apps\ess\HPM—assign 

all rights, except ―full‖ for 

ElectAdmin and ElectDefine.

5/15/09 KS - Accept

Verified Hardening 

Procedures for the 

Election 

Management

System PC, dated 

5/5/09; Step 6c on 

pg. 7 has been 

updated per vendor 

response.

120 4/20/09 K. 

Swift

Inform

ational

Closed EDM SOP, 

Ver Rel 

The document is missing relevant 

information in the Text/Referendum 

MM 2009.05.6- The EDM SOP is 

correct. These language tabs are 

5/14/09 KS - Accept

Verified in EDM 
VSTL Certification # pending

Page 192 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

7.8.1.0 4/3/09 section.

Ch. 39 Text/Referendum, does not 

indicate that the language tabs on the 

Question Text window are only used if 

you are using iVotronic equipment.

used for both the iVotronic and Op 

Scan systems.

TMT - 5.11.09 - EDM - Chapter 39: 

Text/Referendum. In the Format 

Text in Additional Ballot 

Languages section, added the 

following sentence right below the 

heading: "Language tabs are 

supported by the following 

equipment types: ESSIM, Optech, 

iVotronic bitmap, LS iVotronic, 

DSIM."

SOP, dated 5/11/09, 

Ch. 39, pg. 301; has 

been updated as 

stated in vendor 

response.

121 4/21/09 sjakil

eti

Functional 

Defect

Close DS200 Audit 

Logging

The DS200 audit log does not reflect 

administrator access. 

On DS200 Machine ID: 

0000es010737002 4/21/2009: The 

administrative mode is password 

protected. After the Election Definition 

was loaded the tester attempted to 

access the administration mode with 

an invalid, blank and a valid 

password. Neither the failed or 

successful attempts were recorded in 

the audit log. 

5/28/09 Reject SJakileti - As v1:.2.5.2 

is applicable to COTS it is an incorrect 

reference to apply to the DS200. As 

the recording of the access 

attempts(successful and unsuccessful) 

are normal or abnormal events, V1: 

2.2.4.1g is a more appropriate 

requirement. 

6/12/09 - CEC The VSTL interprets 

successful and unsuccessful login 

attempts as security events. We 

understand ES&S' position that this is 

a "non-event" and recommend 

referral to the EAC for determination of 

whether logging in as an administrator 

on the DS200 is or is not an "event".

6/19/09 CEC -Reject - EAC Response: 

ES&S will make a change in the code 

to ensure that the log-in and log-out 

events of the administrator are 

captured. iBeta will review the code 

change and regression test the change 

v.1: 2.2.1.a Provide security access 

controls that limit or detect access to 

critical system components to guard 

against loss of system integrity, 

availability, confidentiality, and 

accountability.

v.1: 2.1.5.2 ...operating system audit 

shall be enabled for all session 

openings and closings, for all 

connection openings and closings, 

for all process executions and 

terminations and for the alteration or 

deletion of any memory or file 

object. 

v.1: 2.2.4.1.g To ensure system 

integrity, all systems shall: record 

and report the date and time of 

normal and abnormal events.

SM 2009.05.7 - The DS200 does 

limit or detect access by providing 

messages to the user and 

prohibiting the user to continue on 

in the process. The standard 

quoted does require the event to 

be logged.

ES&S questions if standard v.1: 

2.1.5.2 applies to this instance.

slm 05.26.09 - V1: 2.2.5.3 is a 

COTS requirement. The DS200 

precinct tabulator is not COTS and 

therefore, this requirement does 

not apply.

SMP 2009.6.9 - The VSS uses the 

word 'event' in a broad and nondefinitive manner throughout. 

Because of the standard's 

vagueness in defining what 

constitutes a 'normal' and an 

'abnormal' event, it is left up to the 

interpretation of the standard by 

the manufacturers, the reviewing 

labs, and the governing entity, 

formerly NASED and now the 

EAC, to determine its meaning. 

ES&S makes every attempt to 

record all normal and abnormal 

events in an effort to be 100% 

compliant with the applicable 

standard. In fact, Unity 3.2 alone 

logs over 2300 system event 

messages in its current 

configuration.

6/29/09 Accept CEC 

Per 6/28/09 EAC 

response accepting 

v.1.3.10.0: 

"the EAC has 

decided that iBeta 

can close the 

discrepancy based 

on the EAC's reading 

of the standard. The 

EAC believes that 

the notification of an 

incorrect password in 

this case is a user 

advisory and not an 

error message and 

therefore 

v.1:2.2.5.2.2.a has 

not been violated in 

this case. The EAC 

appreciates iBeta 

bringing this 

discrepancy to their 

attention and ES&S 

fixing the previous 

problem with 

discrepancy #121."

6/26/09 Reject CEC

6/24/09 CEC 

Pending functional 

testing

6/19/09 Reject CEC 

per EAC Response

6/12/09 
VSTL Certification # pending

Page 193 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

to the extent necessary to ensure the 

system functions properly.

6/26/09 CEC Reject v.1.3.10.0

The "Incorrect Password Entered" 

error is not generating the entry in the 

audit log. It is being generated by 

selection of the "Cancel" button.

Scenario 1: User enters an incorrect 

password 

• Error: "Incorrect Password Entered" is 

reported to the user

•User selects the "Try Again" button

• Audit log entry: None

• User can continue to repeat attempts 

without any error being logged.

Scenario 2: User enters an incorrect 

password 

•Error: "Incorrect Password Entered" is 

reported to the user

•Audit log entry: Date and time of the 

"Cancel" button selection: "User Event: 

Admin Password Entry Failure"

VSS: v.1: 2.2.5.2.2.a All voting 

systems shall meet the following 

requirements for error messages: The 

system shall generate, store, and 

report to the user all error messages as 

they occur.

Neither scenario 1 nor 2 is storing the 

error because the audit entry is being 

generated at the time the user selects 

the "Cancel" and not at the time the 

error is generated.

For the DS200, only an authorized 

pollworker has access to the 

Admin function by unlocking the 

power/open poll access door 

followed by entry of the HPM 

generated election specific admin 

access code. Because the 

recording or non-recording of 

access attempts to the DS200 

admin menu is not considered to 

be essential to ensure the physical 

stability and function of the vote 

recording and counting process 

(the fundamental intent of 

V1:2.2.4) ES&S considered this to 

be non-critical and therefore 

classified as a non-event in its 

design. We therefore disagree 

with iBeta's interpretation of the 

constitution of what is a 'normal' 

event.

SMP 2009.6.16 - Additional 

supporting information - The 

DS200 operates in three modes; 

Administrative Mode for preelection day warehouse 

preparation, election day Polls 

Open mode, and post election 

Polls Closed mode. The 

Administrative functions are all 

performed prior to election day for 

system setup and diagnostic 

evaluations including setting the 

system date and time, calibrating 

the scanner, auto-reading ballots, 

and processing test decks. On 

election day, once the polls are 

opened on the DS200, the 

Administration Menu is not 

accessible. 

MDN 2009.06.24 - Updated 

product versioning within the Unity 

3.2.0.0 system overview 

(U3200_OVR00) and System 

Operations Procedure 

(U3200_SOP00_DS200).

SLM 2009.06.24.09 - This is 

addressed in the new trusted build 

Undetermined CEC 

5/28/2009 Rejected 

SJ
VSTL Certification # pending

Page 194 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

of 1.3.10.0.

122 4/21/09 K. 

Wils

on

Functional 

Defect

Closed EDM - (EMS 

Windows 

Con-figuration 

Test step 

E018)

A non-administrator user (ElectResult) 

was able to access and install an 

executable on the EMS PC

Using the CD and the USB an 

executable file was loaded on the EMS 

PC. The executable did not 

automatically run when inserted in the 

applicable drive. However the nonadministrative user was able to double 

click and run the executable. This 

installed on the System 32, System 

Root and C:Windows drives. The 

executable contained a worm that was 

then launched. (Note: As the ESS

procedures recommend but do not 

require the installation of anti-virus 

software the test was run without this 

protection.)

v.1: 2.2.5.3 Third, the system shall 

be configured to execute only 

intended and necessary processes 

during the execution of election

software.

MM 2009.05.7 - This discrepancy 

is addressed in 

U3200_SSS08_System 

Hardening, Chapter 8, "Antivirus/Malware Protection." Section 

reads, "The election PC 

configuration must include 

commercially available and 

standard virus detection software. 

Virus detection software must be 

installed and enabled

on all election system PCs at all 

times. Removable media must 

always be scanned by the PC to 

prevent virus entry and 

propagation."

Strengthened the language from 

"should include commercially 

available," to, "must include.." in 

the current revision.

Accept 6/17/09 J/g

The test was to 

verify that the 

ResultUser could not 

load any application 

or viruses. The 

executable did run 

however the program 

did not get loaded 

onto the PC. 

123 4/21/09 K. 

Wils

on

Functional 

Defect

Closed EDM - (EMS 

Windows 

Con-figuration 

Test step 

E018)

The successful launch of EDM is not 

recorded in the audit log. 

When a non-administrative user 

ResultsUser (ElectResult group) 

launches EDM it is not recorded in 

either the Audit Manager or windows 

security event log. 

v.1: 2.2.5.3 Second, operating 

system audit shall be enabled for all 

session openings and closings, for 

all connection openings and 

closings, for all process executions 

and terminations, and for the 

alteration or deletion of any memory 

or file object. 

v.1: 2.1.5.2 ...operating system audit 

shall be enabled for all session 

openings and closings, for all 

connection openings and closings, 

for all process executions and 

terminations and for the alteration or 

deletion of any memory or file 

object. 

MM 2009.05.7 -- Launch of EDM 

by an authorized user should be 

recorded in the Window Event 

security log if, security logging is 

turned on for that directory as 

outlined in the Windows Hardening 

procedure. Note as per hardening 

instructions the ElectResult group 

should have access to the EDM 

executable denied. (this is covered 

in Chapter 2)

MM 2009.05.20 -- Launch of EDM 

by an authorized user is recorded 

in the Window Event security log 

and Audit Manager, if security 

logging is turned on for that 

directory as outlined in the 

Windows Hardening procedure. 

Note as per hardening instructions 

the ElectResult group should not 

have access to the EDM 

executable denied. (this is covered 

in Chapter 2)

6/5/2009 JG Accept

ResultsUser is 

locked down and can 

not launch the EDM 

application. 

124 4/21/09 K. 

Wils

on

Functional 

Defect

Closed AM & 

Windows 

Logging

(EMS 

Windows 

Copying the election data file on and 

off the EMS PC is not logged in Audit 

Manager or windows audit log.

ResultUser (ElectResult group), a nonv.1: 2.2.5.3 Second, operating 

system audit shall be enabled for all 

session openings and closings, for 

all connection openings and 

closings, for all process executions 

MM 2009.05.8 -- Revised verbiage 

under the Chapter 10 of the 

System Hardening Procedures 

document from the previous 

statement "Follow these steps to 

6/5/2009 JG Accept

ElectData and Unity 

folders are set to 

event viewer. 

ResultUser action 
VSTL Certification # pending

Page 195 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

Con-figuration 

Test step 

E018)

administrator role, copied the 

elecdata\M650.pr file to a USB, 

modified the file name on an external 

PC, and replaced the file. Neither 

the copy or upload was recorded in 

either audit log. 

and terminations, and for the 

alteration or deletion of any memory 

or file object. 

v.1: 2.1.5.2 ...operating system audit 

shall be enabled for all session 

openings and closings, for all 

connection openings and closings, 

for all process executions and 

terminations and for the alteration or 

deletion of any memory or file 

object. 

lock down your directories" to 

instead read, "Follow these steps 

to lock down both the Unity and 

Elecdata directories."

The Audit Manager audit log will 

not record this type of event. 

However, the Unity 3.2. systems 

utilizes the Window Event Viewer 

to record access to system 

directories and election files by a 

system defined user. The 

Hardening Procedures for the 

Election Management System PC 

(Chapter 10, steps 1 to 7) specifies 

the process for which folders and 

events should be selected for 

Window Event Log to record the 

events as intended. 

logged path.

125 4/21/09 K. 

Wils

on

Functional 

Defect

Closed EDM - (EMS 

Windows 

Con-figuration 

Test step 

E018)

Processes that have not been 

identified as necessary can be run 

during execution of the election 

software. 

The tester logged into EDM as a nonadministrator was able to perform tasks 

using Calculator, Notepad and the 

browser. None of these applications 

are not identified as necessary tools for 

operation of EDM. 

v.1:2.2.5.3 Third, the system shall 

be configured to execute only 

intended and necessary processes 

during the execution of election 

software.

20090505 - MDN -- Non-essential 

programs should be removed 

during system hardening. See 

U3200_SSS08_Hardening 

Procedures Chapter 2: Hardening 

and Verification, "Hardening the 

Windows XP Operating System." 

Documentation directs the user to 

"Remove, or disable, all nonelection applications from the 

device

(WordPad, Notepad, games, 

etc.)." Currently listed under step 

5.a.

5/15/09 KS - Accept

Verified Hardening 

Procedures for the 

Election 

Management

System PC, dated 

5/5/09; contains 

ES&S 

recommendation in 

Ch. 2, Step 5a to 

remove all nonessential 

applications.

126 4/23/09 K. 

Wils

on

Functional 

Defect

Closed HPM Modified the HPM log file.. 

logged on as DefineUser. KW.ELG file 

(the audit log for the KW election) 

accessed and records removed. Log 

file was somewhat difficult to find but 

once found easy to modify. When the 

election log was displayed in HPM the 

modification was not detected. There 

was no Windows event log record of 

access nor other audit of access 

found.

6/6/2009 JG Reject The Hardening 

Procedures for the Election 

Management System PC (Chapter 2, 

steps6) states to "assign all access 

rights, except full for ElectAdmin, 

v.1: 2.2.1.d Provide safeguards to 

protect against tampering during 

system repair, or interventions in 

system operations, in response to 

system failure.

v.1:2.2.1.a Provide security access 

controls that limit or detect access to 

critical system components to guard 

against loss of system integrity, 

availability, confidentiality, and 

accountability.

6/19/09 - v.1: 2.2.5.2 & 2.2.5.2.1.e 

are the appropriate requirements. 

v.1: 2.2.5.2 Audit records shall be 

prepared for all phases of election 

operation performed using devices 

controlled by jurisdiction...The 

MM 2009.05.7- HPM will not log 

this event, Windows event viewer 

will record this event, again the 

Windows event viewer will only log 

access to the folder and read/write 

actions. It will not specifically 

record file names, sizes, etc.

The HPM audit log does not record 

this type of event. The Unity 3.2. 

systems utilizes the Window Event 

Viewer to record access to system 

directories and election files by a 

system defined user. The 

Hardening Procedures for the 

Election Management System PC 

(Chapter 10, steps 1 to 7) specifies 

the process for which folders and 

6/22/09 CEC Accept

Accepted based 

upon the EAC 

interpretation

"#126 - The EAC 

reviewed the 

discrepancy decision 

made by iBeta as 

well as ES&S's 

explanation 

regarding the 

discrepancy. The 

EAC applauds the 

VSTL for its diligence 

in citing this 

discrepancy and 

encourages 
VSTL Certification # pending

Page 196 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

ElectDefine, and ElectResult". When 

assigning ElectDefine and ElectResult 

to all access except full the users have 

permission to modify, read & execute, 

and a list folder contents. This allows 

the modification of the audit files. (v.1: 

2.2.4.1 f & h)

6/18/09 JG Reject. At issue, is the 

ability to modify or destroy an audit log. 

While detection is acceptable to other 

areas of security. The standard 

explicitly prohibits altering or destroying 

audit records. The tester was able to 

copy the ELG file; delete part of the 

file; save the file back into the Elecdata 

folder; and reopened the audit log. 

Prior to modification the HPM log had 2 

pages and after the modification the 

HPM log only had 1 page. 

Modification of the log can be done on 

the same PC that Unity resides on 

using Note Pad and Word Pad.

6/19/09 - CEC Response to EAC 

Query: iBeta agrees that v.1: 2.2.4 

limits these Integrity requirements to 

vote recording and counting processes. 

v.1: 2.2.5.2 & 2.2.5.2.1.e are the 

appropriate requirements. 

software shall activate the logging 

and reporting of audit data as 

described in the following sections. 

...v.1: 2.2.5.2.1.e The generation of 

audit record entries shall not be 

terminated or altered by program 

control or intervention of any 

person...

events should be selected for 

Window Event Log to record the 

events as intended.

GLW 2009.06.10

As stated in Chapter 2 of the 

Hardening Procedures, all PC‘s 

loaded with ES&S application 

software and used for the conduct 

of elections should be  ̳secured in 

an appropriate manner to assure 

controlled access and maintain 

chain of custody‘. A properly 

Windows credentialed user, as 

defined in the Hardening 

Procedures, accessed the 

election_name.ELG file from 

outside of ES&S application 

software. Uncontrolled physical 

access (see ES&S System 

Security Specification) to this PC 

had to be obtained and knowledge 

of a Windows Administrator 

defined User ID and Password 

with Administrator assigned 

access rights to the \elecdata 

folder had to be known and used 

to obtain access to this file. If the 

ES&S Hardening Procedures have 

been properly and completely 

performed, all accesses to the 

\elecdata folder and to the 

election_name.ELG file are 

recorded in the PC‘s Windows 

Event Log and may be viewed 

using the Windows Event Viewer. 

Access to the \elecdata folder by 

the three ES&S recommended 

Windows User ID‘s as specified in 

the ES&S Hardening Procedures 

is required because both the 

ElectDefine and ElectResult User 

must be able to create and modify 

files in the \elecdata. ElectAdmin 

is granted access to all folders. 

The \elecdata folder (and its subfolders) is the principle folder path 

used by both Hardware 

Programming Manager (HPM) and 

Election Reporting Manager 

continued rigor in the 

testing of the system. 

However, the 

requirements cited 

are not sufficiently 

clear to sustain the 

discrepancy. The 

EAC found little merit 

in much of ES&S's 

argument regarding 

procedural 

safeguards as a 

means to meet the 

standard cited. 

However, after 

discussion with its 

technical reviewers 

the EAC believes 

that the system as 

currently designed 

meets the standard 

as it is currently 

written. Therefore, 

this discrepancy can 

be resolved. " 

6/18/2009 JG Reject

6/6/2009 JG Reject
VSTL Certification # pending

Page 197 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

(ERM). The election_name.EFC 

output file created from the initial 

election definition and ballot layout 

done in EDM and ESSIM is written 

to the \elecdata folder and is then 

read by HPM to build its initial 

election_name.DAT file. This 

requires ElectDefine to have 

access to the \elecdata folder. 

Additionally, during the processing 

steps performed in HPM, 

additional files are created and 

updated in the \elecdata folder and 

its sub-folders. This too requires 

access to these folders by 

ElectDefine. The ElectResult User 

must have access to the \elecdata 

folder and it‘s sub-folders for 

access to the final 

election_name.DAT file from which 

the ERM results database is 

created (multiple output files) in 

the \elecdata folder. Further 

processing in ERM to accumulate 

and report election results creates 

many additional files in \elecdata 

and its sub-folders. 

127 4/28/09 sjakil

eti

Docum

ent 

Defect

Closed ES&S 

Software 

Design 

Specifications 

Model 650 

v1.0

An import file is not identified in the SW 

design spec. 

Sec 7.1.2. identifies the M650 

imported files are .OFC,.RPT, and 

.PRE. During security testing we are 

seeing 4 files on the zip disk 

.OFC,.RPT,.PRE and .PRF

v.1: 2.2.2.1d include control logic 

and data processing methods 

incorporating parity and check-sums 

(or equivalent error detection and 

correction methods) to demonstrate 

that the system has been designed 

for accuracy

20090505 - MDN -- Updated 

section 7.1.2 of the Model 650 

SDS to include the .PRF file, which 

is a "Precinct Finder File." 

Confirmed that this file is also 

properly documented in Section 

10.5, "System Processes."

5/15/09 KS - Accept

Verified M650 SDS, 

dated 5/12/09; 

Sections 7.1.2 & 10.5 

has been updated to 

include .PRF file

128 4/29/09 J. 

Garc

ia

Docum

ent 

Defect

Closed ES&S ERM

System Ops

Procedures

Ver. Rel. 

7.5.3.0

4/2/09

The documentation has an incorrect 

path for the target field within the 

Security Procedures.

In Part 2 Section 7 number 5 of the 

documentation states "In the Target 

field change erm.cob to uermmngr.cob. 

The entire Target is C:\Program 

Files\rmcobol\runcobol.exe 

uermmngr.cob" However the path 

displayed when executing the security 

procedures is "C:\Program 

Files\Liant\RMCOBOLv11\runcobol.ex

e" Aero.cob.

v.2: 2.8 This documentation shall 

provide all information necessary for 

system use by all personnel who 

support pre-election and election 

preparation, polling place activities 

and central counting activities, as 

applicable, with regard to all system 

functions and operations identified in 

Section 2.3...

DJZ 5-11-09 - Changed the text in 

Part 2, Section 7, number 5. In the 

Target field the path change 

Aero.cob to uermmngr.cob. The 

entire path should read to 

"C\Program 

Files\LIANT\RMCOBOLv11\runcob

ol.exe" uermmngr.cob.

5/14/09 KS - Accept

Verified in EDM 

SOP, dated 5/11/09, 

Ch. 7, Step 5 states 

the correct path as 

stated in vendor 

response.

129 4/29/09 J. 

Garc

Functional Closed M650 

v.2.2.1.0 

The M650 displays an incorrect error 

message when a non precinct level 

V1: 2.2.5.2.2.b

All voting systems shall meet the 

SLM 2009.05.12 - ES&S is waiting 

to respond until the regression test 

Accept 05/21/09 JG

Corrected party 
VSTL Certification # pending

Page 198 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

ia Defect contest is linked to a district type that 

is specific to precinct level races.

When scanning a ballot incorrectly 

linked to a district type that is specific 

to precinct level races the M650 

displays an error "ERROR: Low 

Contrast, Channel D!". 

following requirements for error 

messages: b. All error messages 

requiring intervention by an operator 

or precinct official shall be displayed 

or printed unambiguously in easily 

understood language text, or by 

means of other suitable visual 

indicators; 

is recoding using the correct steps. preference and 

adjusted ballot 

layout to reflect 

correction,. 

Following these 

modifications the 

errors could not be 

recreated.

130 5/1/09 K. 

Wils

on

Functional 

Defect

Closed EDM Security 

(election 

folder)

External copy and replace access to 

critical election folders is not being 

controlled or detected. 

Accessing folder Unity\EDM\VOL8S2 

as a non-admin ( DefineUser) the 

tester copied the entire folder to a USB 

for external analysis and modification. 

Files in the folder have extensions dbf, 

mdx. Using a demo version of 

DBFManager the Candidate.dbf file 

was opened and the names of 

CANDIDATE 1 and CANDIDATE 2 

were swapped. Verified that neither the 

Windows or Audit Manager logged 

activity to copy these files to a USB or 

restore modified files . (Note: The 

purpose of the test was to identify if the 

file manipulation is controlled or 

detected at the point of manipulation 

and by whom. This type of change 

would ultimately be expected to be 

detected in pre-election logic and 

accuracy testing.)

v1:2.2.1.a To ensure security , all 

systems shall: provide security 

access controls that limit or detect 

access to critical system 

components to guard against loss of 

system integrity, availability, 

confidentiality, and accountability

MM 2009.05.12 - Windows Event 

Viewer will log individual system 

user access and specified user 

events to the C:Unity directory and 

sub directories. To configure 

Windows Event Viewer refer to: 

Hardening Procedures for the 

Election Management System PC 

(Chapter 10, steps 1 to 7) for steps 

to enable intended logging.

6/6/2009 JG Accept

Verified the logging 

of activities in the 

Windows event log. 

See Discrepancies 

138 - 140 for greater 

system hardening 

depth

131 5/1/09 sjakil

eti

Securit

y

testing

Closed M650 -

Security 

(installing an 

election)

A modified election can be installed on 

the M650 without detection. 

Before loading the election definition 

on the M650 the tester modified the 

election definition by:

- replacing 0007 to 0999 in 

.PRF(Precinct finder file) file; and 

- replacing 0010-->9999(ballot 

counter -Totals) in RPT file. 

The modified election definition was 

accepted by the M650 . It was 

confirmed by source code review that 

there is no CRC validation for the 

election definition (.RPT,.PRE,.OFC)

v1: 2.2.1a To ensure system 

security, all systems shall provide 

security access controls that limit or 

detect access to critical system 

components to guard against loss of 

system integrity, availability, 

confidentiality, and accountability.

V1: 2.2.2.1.d To ensure vote 

accuracy, all systems shall included 

control logic and data processing 

methods incorporating parity and 

check-sums (or equivalent error 

detection and correction methods) to 

demonstrate that the system has 

been designed for accuracy. 

SLM 2009.05.12 - The PRF file IS 

NOT used by the M650. It is only 

put on the zip disk to have it 

available when results are 

uploaded into ERM. ERM is the 

only application that uses it. 

Therefore, any edits would be 

totally ignored. In the System 

Security Specification, Chapter 5 

referencing Optical Scan Security, 

it covers in great detail the tasks 

and procedures that a local 

jurisdiction must go through to 

secure their central count location. 

In conjunction with this, the user 

will also conduct the Final 

Readiness Checks of the Model 

650 before Election Day. These 

steps are covered in the Model 

Accept 5/26/2009 SJ 

: Verified when 

powered on the 

M650 readiness 

report included check 

election 

options(election 

definition files are 

included with time 

stamp). With this it is 

possible to detect the 

modification
VSTL Certification # pending

Page 199 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

650 Systems Operations 

Procedure, Chapter 5. 

In the Machine Readiness Report 

generated after power up on the 

Model 650, it logs under the check 

election, options the files and the 

date and times they are created or 

modified. It is part of the steps 

listed in the SOP for the 

jurisdiction to verify that these 

dates and times match when the 

election definition was created. 

132 5/1/09 Sjaki

leti

Functional 

Defect

Closed M650 Security 

v. 2.2.1.0 

(reporting 

election 

results)

Modified M650 election results can be 

loaded into ERM without detection. 

After scanning 5 ballots saved results 

to the zip disk. Made some 

modification on some of the files using 

HEX editor.

1. In the .EC file replaced PRECINCT 

by pppppppp, this modified results 

loaded to ERM. No CRC or checksum 

validation errors occurred.

2. In the .log file replaced some bytes 

with zeros ,this modified results loaded 

to ERM. No validation errors occurred.

3. In the .pr file replaced 0001.. by 

999999,this modified results loaded to 

ERM. No validation error occurred.

ERM loads the modified results.

1. 06042009 SJ Accepted

2.06042009 SJ Accepted

3. Rejected 06042009 SJ: Tabulated 

totals are presented in the .pr file. By 

using Hex editor modified totals in one 

precinct by replacing 01 with 02 

,uploaded modified totals into ERM. 

There is no error detection. 

Procedures to secure the central count 

location do not ensure the ultimate 

accuracy of the vote totals. 

6/12/09 CEC Rejected

The system does not incorporate the 

error detection functionality required in 

the VSS. 

6/19/09 EAC Response: The 

discrepancy needs to be resolved. In 

v1: 2.2.1a To ensure system 

security, all systems shall provide 

security access controls that limit or 

detect access to critical system 

components to guard against loss of 

system integrity, availability, 

confidentiality, and accountability.

V1: 2.2.2.1.d To ensure vote 

accuracy, all systems shall included 

control logic and data processing 

methods incorporating parity and 

check-sums (or equivalent error 

detection and correction methods) to 

demonstrate that the system has 

been designed for accuracy. 

SLM 2009.05.12 - The .ec file 

does not contain any results. It is 

an initialization file created by the 

M650 from the EL80 (election 

definition) files that is used by the 

application for tabulation and 

reporting. The edits which were 

made have no affect on results 

processing. If they did, it would 

result in an ERM error.

Although the log file must be 

present, ERM does not use the log 

file entries as part of results 

processing. Edits to it do not 

affect uploading of results. 

If the edit was made to the PR file 

in the field that ERM uses, it 

would have triggered a data 

validation error since a data 

consistency issue would have 

been detected.

In the System Security 

Specification, Chapter 5 

referencing Optical Scan Security, 

it covers in great detail the tasks 

and procedures that a local 

jurisdiction must go through to 

secure their central count location. 

SMP 2009.6.9 - ES&S takes 

security very seriously and our 

systems are designed to operate 

in a secure and reliable manner. 

ERM has numerous measures 

built in to the application to verify 

counts and detect when data may 

have been altered. Some 

examples are: 

6/22/09 CEC Accept

Accepted based 

upon the EAC's 

response

"#132 - The EAC 

reviewed the 

discrepancy decision 

made by iBeta as 

well as ES&S's 

explanation 

regarding the 

discrepancy. The 

EAC applauds the 

VSTL for its diligence 

in citing this 

discrepancy and 

encourages 

continued rigor in the 

testing of the system. 

However, the 

requirements cited 

are not sufficiently 

clear to sustain the 

discrepancy. The 

EAC again found 

little merit in many of 

ES&S's arguments 

regarding procedural 

safeguards as a 

means to meet the 

standard cited. 

However, after 

discussion with its 

technical reviewers 

the EAC believes 

that the system as 

currently designed 
VSTL Certification # pending

Page 200 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

reviewing this discrepancy it was noted 

that this discrepancy also appears to 

be non-compliant with VSS 

V1:2.2.2.1e.

• ERM and M650 counter 

mismatch

• Contest votes exceed ballots 

cast

• Maximum value of the counters 

allowed in ERM have been 

exceeded

• Precinct not on UERM file –

aborting

• Too many local candidates –

aborting 

• .PR file not as expected –

aborting (precinct being updated 

not expected current precinct)

• .PR file has not  ̳OFC‘ – aborting

• Open / Read error messages

The M650 is a central count 

scanner designed to be located in 

a physically-controlled secure area 

accessible only by authorized 

personnel. ES&S recommends all 

ZIP disks also be locked in a 

secure and separate area and 

monitored for check-in and checkout. In addition, the entire area 

should be monitored to ensure it is 

not compromised. When used in 

conjunction with the procedures 

outlined in the ES&S System 

Security Specification, the risk 

level for an unauthorized person 

with unauthorized access to 

modify precise portions of the .PR 

data on a ZIP disk, that would go 

unnoticed by an election 

administrator and undetected by 

ERM is in the lowest category 

when determining total exposure. 

Most critical data is stored in 

binary formats making it extremely 

difficult for an unauthorized user to 

alter the file in such a way that 

would affect the vote totals. There 

is also significant portions of the 

.PR file that is not read by ERM 

and would have no affect if it were 

to be altered.

meets the standard 

as it is currently 

written. Therefore, 

this discrepancy can 

be resolved. "

Rejected 6/12/09 

CEC

Rejected 06042009 

SJ

133 5/5/09 K. 

Swift

Docum

ent 

Defect

Closed EDM SOP, 

Ver. Rel. 

7.8.1.0

The document does not clearly identify 

steps to set up a party selection 

contest for Pick A Party open primary.

v.2: 2.8.5.a The vendor shall provide 

documentation of system operating 

procedures that meets the following 

MDN 2009.5.12 - Updated EDM 

SOP Chapter 50, page 339, "Pick 

a Party" step 1 to specify that a 

6/1/09 CEC Accept 

Verified EDM SOP 

dated 5/26/09 Ch 35 
VSTL Certification # pending

Page 201 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

4/3/09

Ch. 35: 

Straight Party 

Ch. 25, 30, 32 

(Master Office 

file)

The document does not explain that a 

Party Preference contest needs to be 

set up as a straight party contest in a 

Pick a Party open primary. Nor does it 

explain the difference in function. The 

chapters regarding the Master Office 

File, are unclear as to how or where to 

define the Party Preference race.

5/15/09 KS - Reject

Ch. 35 has not been updated to clarify 

that a selection in the Party Preference 

contest does not always mean that 

linked contests will automatically get a 

vote for candidates from the selected 

party in all of the following contests on 

the ballot. 

requirements: a. Provides a detailed 

description of procedures required to 

initiate, control, and verify proper 

system operation;

straight party contest must be 

included in a pick-a-party primary 

election.

TMT 5.26.09 - Chapter 14, Offices 

Preference Tab, under the last 

bullet regarding straight party, 

AND Chapter 35 in the introduction 

added this note: "On an open party 

ballot, the Party Preference race is 

formatted as a Straight Party 

contest. Selections in this contest 

only declare your party; they do 

not automatically select the 

corresponding candidates. You will 

still need to select ballot options on 

your ballot."

address formatting of 

Party Preference as 

a Straight Party 

Contest. 

5/15/09 KS Accept 

Verified EDM SOP, 

dated 5/11/09 

contains the added 

sentence in Ch. 50, 

pg. 339 that explains 

to use the Straight 

Party option to set 

the Party Preference 

contest,

KS, 5/13/09 Reject

134 5/3/09 K. 

Swift

Docum

ent 

Defect

Closed EDM SOP, 

Ver. Rel. 

7.8.1.0

4/3/09

Ch. 39 

Text/Referend

um

Ch. 39 states that 2 text files can be 

created (one for English & one for 

Spanish), however, ESSIM does not 

accept 2 text files.

ESSIM only accepts a single file 

containing English and Spanish text. 

v.2: 2.8.5.a The vendor shall provide 

documentation of system operating 

procedures that meets the following 

requirements:

a. Provides a detailed description of 

procedures required to initiate, 

control, and verify proper system 

operation;

MM 2009.5.12 - This is incorrect 

statement, ESSIM only uses the 

test file that is created or imported 

on the default question/text tab. If 

you are coding a multi lingual 

election you would create a txt file 

with both an English and Spanish 

translation, so both languages 

appear on the ballot. An English 

only text files should be created 

from the English tab, and a 

Spanish only translation created 

from the Spanish tab. ESSIM 

doesn‘t need the English only or 

Spanish only .txt file, but the VAT 

(&iVo)will import them to display 

the voter selected language. 

Accept 5/22/09 J. 

Garcia The issue 

was incorrectly 

reported due to 

misinterpretation of 

the application and 

documentation by the 

test staff.

135 5/7/09 C 

Cog

gins

Functional 

Defect

Closed DS200 -

Modem 

Results 

functions

Simply removing the modem from the 

DS200 does not provide complete 

access control to prevent modem 

usage in Unity 3.2.0.0

ES&S excluded modeming of results 

from Unity 3.2.0.0 supported 

functionality by removing the modem. 

The active modem results button on 

the DS200 permits a user to access 

the code to modem results functions. 

It is understood that if a system does 

not have a modem the function will 

error and not be completed. However 

there is nothing within the Unity 3.2.0.0 

system that would prevent a 

v.1: 6.5.1 Voting system that use 

telecommunications to communicate 

between system components and 

locations are subject to the same 

security requirements governing 

access to any other system 

hardware, software and data 

function. 

V.1: 2.2.1.b Te ensure security, all 

systems shall: provide system 

functions that are executable only in 

the intended manner and order , and 

only under the intended conditions. 

SMP 2009.5.12 - ES&S believes 

we are in full compliance with V.1: 

6.5.1 and V.1: 2.2.1.b. ES&S 

challenges iBeta's opinion that the 

absence of an internal modem 

does not provide adequate control 

to prevent the usage of a modem. 

The absence of the modem 

entirely prevents any possibility of 

external telecommunications from 

the DS200 precinct scanner and is 

the ultimate depth of defense 

against misuse of the system. The 

intended manner in which Unity 

3.2.0.0 and its associated 

components is to be certified and 

Accept 6/5/09 CEC 

EAC Response: 1. 

The EAC will note 

the specifics of this 

issue in our Scope of 

Certification 

Document issued 

and posted on our 

web site once 3.2 

has been certified. 

The notation will 

make it clear that 

ANY use of a modem 

with the Unity 3.2 

configuration as 

tested and certified 
VSTL Certification # pending

Page 202 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

jurisdiction from installing a modem 

and modifying the settings of Unity to 

activate network connections for 

functionality that was purposely 

excluded from certification. While it is 

understood that such an action 

invalidates the EAC certification 

permitting access to the functionality to 

remain active provides insufficient 

depth of defense. 

5/15/09 - In view of the fact that it is the 

EAC's certification that is being 

defended, iBeta is not the appropriate 

judge of the depth to which access to 

this functionality need be protected. 

We therefore recommend that under 

the circumstances outlined in ES&S' 

response this issue be submitted to the 

EAC for "guidance on how to properly 

evaluate conformance" to v.1: 6.5.1 

and 2.2.1.b (EAC Voting System 

Testing and Certification Program 

Manual v.1 section 9.1) . 

used once deployed is to NOT 

support telecommunications in any 

manner. These are the intended 

conditions in which the system is 

to be used and the absence of the 

necessary components 

guarantees the system cannot be 

used in a manner other than the 

certified configuration. 

Telecommunications is clearly 

identified as a non-supported 

function of this certification as is 

evidence by the absence of the 

most essential components to 

support telecommunication from a 

polling site location, the ES&S 

Data Acquisition Manager ("DAM") 

and internal modems from all 

DS200 precinct scanners. Without 

these components, we ensure the 

system is only used in its intended 

manner and under these intended 

conditions.

The active Modem Results button 

displayed during poll closing is 

accompanied with layered edit 

protection informing the user that 

external modem communications 

is not available. 

Telecommunications from the 

polling site will be a supported 

function in the immediate 

subsequent EAC certification of 

Unity 4.0.0.0.

by the EAC will 

immediately 

invalidate the 

certification. 

2. ES&S will issue 

this same notification 

in a special memo to 

be distributed to 

every jurisdiction 

retrofitted, or sold as 

new, the Unity 3.2 

system. 

3. ES&S will provide 

the EAC, on a 

continuing basis, an 

updated list of all 

jurisdictions using the 

EAC certified Unity 

3.2 voting system. 

4. The EAC will 

perform random spot 

checks of 

jurisdictions using 

this system to verify 

that these 

requirements are 

being followed.

136 5/7/09 Step

h &

Kelly

Functional 

Defect

Closed EDM v. 

7.8.0.0, 

Election/Statis

tical Counters

(Regress TC)

An error was displayed when 

attempting to deselect Party by District 

Total statistical counters.

Selected counters in the Party by 

District Totals tabs then Generated the 

Counters. Attempted to remove the 

selected Party by District totals 

counters and a Microsoft error 

appeared, "EDM has encountered a 

problem and needs to close. We are 

sorry for the inconvenience", and 

application closed. The application 

will not permit the user to edit this 

selection.

v.1: 2.3.1.2 b

Ballot formatting is the process by 

which election officials or their 

designees use election 

databases.... All systems shall 

provide a capability for: rapid and 

error-free definition of elections and 

their associated ballot layouts.

MAM 2009.5.26 EDM encountered 

an error and closed because the 

parties by district totals counters 

were not set up correctly (missing 

.DST file). Instruction for setting up 

a .DST file have been added to the 

ES&S Election Data Manager SOP 

Manual, refer to Part 10: Appendix, 

Chapter 56 Setting up a .DST file.

SLM 2009.06.15 - This is not 

supported in Unity 3.2.0.0. This 

will be tested during Unity 4.0.0.0.

Accept 06/19/2009 

JG

Out of scope of Unity 

3.2.0.0 Issue is 

being moved to Unity 

4.0.0.0 discrepancy 

#5.
VSTL Certification # pending

Page 203 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

137 5/7/09 J. 

Garc

ia

Functional 

Defect

Closed M650 

2.2.1.0 

The M650 displays an incorrect total 

number of ballots saved prior to 

selecting the Save button.

After scanning a batch of 42 ballots the 

M650 displayed 33 ballots saved. 

Once the Save button was selected the 

number changed to 34. The test was 

zeroed out and the same batch was 

scanned again. The display showed 

21 ballots saved until the Save button 

was selected. Once the Save button 

saw selected the number changed to 

34. The number of ballots expected to 

be counted was 34 as all blank ballots 

were to be detected and pulled out. 

V1: 2.2.9b 

For all voting systems, each device 

that tabulates ballots shall provide a 

counter that: records the number of 

ballots cast during a particular test 

cycle or election.

SLM 2009.05.12 - ES&S is waiting 

to respond until the regression test 

is recoding using the correct steps. 

Accept 05/21/09 JG

Corrected party 

preference and 

adjusted ballot 

layout to reflect 

correction,. 

Following these 

modifications the 

errors could not be 

recreated.

138 5/8/09 K. 

Wils

on

Securit

y 

testing

Closed ERM - 2.2.1.a

RSN 

database file

Modification of the election results file 

was not logged.

Using test database VOL10S1. 

Logged in as ResultUser (member of 

ElectResult), results were imported 

from DS200 with ballots from precinct 

203 candidates 21-24 were voted 8, 7, 

10 & 5 respectively . The results RSN 

database file was copied. The file was 

moved to another PC, modified 

swapping the votes for candidates 21 & 

22 and 23 & 24. (Only the 4 fields had 

to be modified). The changed file was 

loaded into ERM. As sysadmin, the 

Security Windows events results were 

reviewed. ResultUser was not 

prohibited from accessing the file and 

there was no log record of access to 

the file by ResultUser . (File: 

20090508-erm-files.rtf and associated 

files)

6/6/2009 JG Reject The Hardening 

Procedures for the Election 

Management System PC (Chapter 2,

steps6) states to "assign all access 

rights, except full for ElectAdmin, 

ElectDefine, and ElectResult. When 

assigning ElectDefine and ElectResult 

to all access except full the users have 

permission to modify, Read & Execute, 

and a list folder contents. This allows 

the modification of the files.

v.1: 2.2.1.a Provide security access 

controls that limit or detect access to 

critical system components to guard 

against loss of system integrity, 

availability, confidentiality, and 

accountability.

v.1:2.2.5.3 ... operating system audit 

shall be enabled for all session 

openings and closings, for all 

connection openings and closings, 

for all process executions and 

terminations, and for the alteration 

or deletion of any memory or file 

object.

MM 2009.05.26 - All users system 

users (ElectAdmin, ElectDefine, & 

ElectResults) are required to have 

access to C:electdata folder and 

all files in the root of c:elecdata. 

Windows Event Viewer will log 

individual system user access and 

specified user events (object 

access) to the C:elecdata directory 

and sub directories. To configure 

Windows Event Viewer refer to: 

Hardening Procedures for the 

Election Management System PC 

(Chapter 10, steps 1 to 7) for steps 

to enable intended logging

GLW 2009.06.09

As stated in Chapter 2 of the

Hardening Procedures, all PC‘s 

loaded with ES&S application 

software and used for the conduct 

of elections should be  ̳secured in 

an appropriate manner to assure 

controlled access and maintain 

chain of custody‘. The actions 

taken by the user in this 

discrepancy required the use of a 

 ̳rogue‘ PC with the ES&S ERM 

software application installed and 

access to an exact and complete 

copy of the ERM results database 

for this specific election. A 

properly Windows credentialed 

user, as defined in the Hardening 

Procedures, copied the 

6/18/2009 JG Accept

Note: Testing utilized 

a binary text editor 

and not a secondary 

version of ERM. 

Although the specific 

modification of the 

file is not prevented 

or recorded the 

Window Event Log 

does detect access 

to the Electdata 

folder. Based upon 

the system 

architecture this is 

the extent to which 

detection is available. 

6/6/2009 JG Reject
VSTL Certification # pending

Page 204 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu sLocation Description Requirement ESS Response Validation

election_name.RSN file from the 

primary ERM PC to the  ̳rogue‘ PC 

using native Windows functions. 

Uncontrolled physical access (see 

ES&S System Security 

Specification) to this ERM PC had 

to be obtained and knowledge of a 

Windows Administrator defined 

User ID and Password with 

Administrator assigned access 

rights to the \elecdata folder had to 

be known. Using the rogue 

version of ERM on the  ̳rogue‘ PC, 

candidate totals were adjusted 

using the Manual Entry Update 

procedure. The 

election_name.RSN file was then 

copied from the  ̳rogue‘ PC to the 

primary ERM PC. Since the 

changes made to 

election_name.RSN file were 

performed on a  ̳rogue‘ PC with an 

installed rogue version of ERM, 

the audit log records documenting 

these changes were created in the 

ERM System Log on the  ̳rogue‘ 

PC. However, all accesses to the \elecdata folder and to the 

election_name.RSN file are 

recorded on the primary ERM PC‘s 

Windows Event Viewer if the 

ES&S Hardening Procedures have 

been properly completed. A 

review of the ERM System Log on 

the primary ERM PC would not 

reflect any activity for the 

corresponding timeframe as the 

activity logged in the Windows 

Event Log thus substantiating that 

the accesses to the \elecdata 

folder and to the 

election_name.RSN file were 

performed outside of the system 

and would identify the Windows 

User ID. 

Access to the \elecdata folder by 

the three ES&S recommended 

Windows User ID‘s is required 

because both the ElectDefine and 

ElectResult User must be able to 
VSTL Certification # pending

Page 205 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

create and modify files in this 

folder. ElectAdmin is granted 

access to all folders. The 

\elecdata folder (and it‘s subfolders) is the principle folder path 

used by both Hardware 

Programming Manager (HPM) and 

Election Reporting Manager 

(ERM). The election_name.EFC 

output file created from the initial 

election definition and ballot layout 

done in EDM and ESSIM is written 

to the \elecdata folder and is then 

read by HPM to build it‘s initial 

election_name.DAT file. This 

requires ElectDefine to have 

access to the \elecdata folder. 

Additionally, during the processing 

steps performed in HPM, 

additional files are created and 

updated in the \elecdata folder and 

its sub-folders. This too requires 

access to these folders by 

ElectDefine. The ElectResult User 

must have access to the \elecdata 

folder and it‘s sub-folders for 

access to the final 

election_name.DAT file from which 

the ERM results database is 

created (multiple output files) in 

the \elecdata folder. Further 

processing in ERM to accumulate 

and report election results creates 

many additional files in \elecdata 

and its sub-folders. 

139 5/8/09 K. 

Wils

on

Securit

y 

testing

Closed HPM-2.2.1.a 

CNTLFILE 

user 

identifiers

Access to the CNTLFILE is not logged. 

The CNTLFILE contains the 

obfuscated userids of persons allowed 

to access HPM. Any HPM user can 

access the file and change the user 3 

letter names. None of the tested users 

(DefineUser, ResultUser, AdminUser 

or Administrator) were logged in any 

ESS program or in the Windows Event 

log. The test was to confirm if the file 

could be changed and if so, are 

changes were logged. 

6/6/2009 JG Reject The Hardening 

Procedures for the Election 

v.1: 2.2.1.a Provide security access 

controls that limit or detect access to 

critical system components to guard 

against loss of system integrity, 

availability, confidentiality, and 

accountability.

v.1:2.2.5.3 ... operating system audit 

shall be enabled for all session 

openings and closings, for all 

connection openings and closings, 

for all process executions and 

terminations, and for the alteration 

or deletion of any memory or file 

object.

MM 2009.05.26 - Windows Event 

Viewer will log individual system 

user access and specified user 

events (object access) to the 

C:elecdata directory and sub 

directories. To configure Windows 

Event Viewer refer to: Hardening 

Procedures for the Election 

Management System PC 

(Chapter 10, steps 1 to 7) for steps 

to enable intended logging.

GLW 2009.06.09

As defined and documented in the 

ES&S Hardening Procedures, user 

6/18/2009 JG Accept

Although the specific 

modification of the 

file is not prevented 

or recorded the 

Window Event Log 

does detect access 

to the Electdata 

folder. Based upon 

the system 

architecture this is 

the extent to which 

detection is available. 

6/6/2009 JG Reject
VSTL Certification # pending

Page 206 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

Management System PC (Chapter 2, 

steps6) states to "assign all access 

rights, except full for ElectAdmin, 

ElectDefine, and ElectResult. When 

assigning ElectDefine and ElectResult 

to all access except full the users have 

permission to modify, Read & Execute, 

and a list folder contents. This allows 

the modification of the files.

authentication and access control 

to all ES&S application software is 

done using the native Windows 

Administrative functions. A 

properly configured and hardened 

PC, as defined in the ES&S 

Hardening Procedures, provides 

user authentication and limits 

access for each define user to only 

the required and specified folders. 

A second level of identification (not 

authentication) optionally available 

in HPM allows the HPM System 

Administrator to invoke the 

requirement for each HPM user to 

enter a 3 character identifier each 

time HPM is executed. The 

purpose of this identifier is to 

record on the HPM System Log 

the changes made by each 

defined HPM user, not Windows 

User. The HPM System Log is 

only different in this case by the 

inclusion of the 3 character 

identifier tagged onto each log 

entry. The use of this additional 

identification of HPM users is not 

mandatory and does not change in 

any way the functions performed in 

HPM, other than the inclusion of 

each 3 character identification on 

the HPM System Log.

The actions taken that yielded this 

discrepancy included Windows 

level access and manual 

manipulation to a Control File used 

by HPM to store the HPM System 

Administrator created 3 character 

HPM identifiers. Access to this 

specific file and to the \elecdata 

folder would have been recorded 

in the Windows Event Log and 

indicate the Windows User ID and 

respective timestamp. As 

explained in detail in the response 

to Discrepancy #138, all three 

ES&S recommended Windows 

User roles must have access to 

the \elecdata folder where this 

HPM Control File resides. 
VSTL Certification # pending

Page 207 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

140 5/8/09 K. 

Wils

on

Securit

y 

testing

Closed HPM -

EFC import

An EFC file modified outside of ESSIM 

was successfully imported into HPM. 

The EFC file (the encrypted version of 

the IFC file) was randomly modified at 

location 00E3 replacing 0x89 with 

0x81. The modified file was 

successfully imported into HPM.

6/6/2009 JG Reject The Hardening 

Procedures for the Election 

Management System PC (Chapter 2, 

steps6) states to "assign all access 

rights, except full for ElectAdmin, 

ElectDefine, and ElectResult. When 

assigning ElectDefine and ElectResult 

to all access except full the users have 

permission to modify, Read & Execute, 

and a list folder contents. This allows 

the modification of the files.

6/18/2009 JG Reject The test is to 

validate that the system can identify a 

change has occurred via a check-sum 

or some equivalent. In this case the 

system does not detect a change to the 

EFC (encrypted) file. Although it is 

understood access is required to write 

to the files and that the event log has 

knowledge that something has 

occurred in the folder where the EFC 

file lives, the system is not providing a 

check or equivalent error detection. 

6/22/09 CEC & JG

v.1: 2.2.2.1.d is only applicable to 

voting recording or counting 

equipment. iBeta went back to the 

original test to confirm if it was a valid 

test. The original test had the.efc file 

being randomly modified in order to 

ensure error recovery from an external 

condition that was not electrical or 

mechanical. The purpose of the .efc 

file is to decrypt the .ifc file. It was 

unclear if the random change was 

relevant to the operation of the file. 

Instead of a random modification the 

test was re-executed by first importing 

the file with no changes. The import 

was successful. We then changed the 

v.1:2.2.3.c Recovery from any other 

external condition that causes 

equipment to become inoperable, 

provided that catastrophic electrical 

or mechanical damage due to 

external phenomena has not 

occurred.

6/18/09 The incorrect requirement 

was listed in this discrepancy . The 

correct requirement is v.1: 2.2.2.1.d 

To ensure vote accuracy, all 

systems shall included control logic 

and data processing methods 

incorporating parity and check-sums 

(or equivalent error detection and 

correction methods) to demonstrate 

that the system has been designed 

for accuracy. 

MM 2009.05.26- Windows Event 

Viewer will log individual system 

user access and specified user 

events (obect access) to the 

C:elecdata directory and sub 

directories. To configure Windows 

Event Viewer refer to: Hardening 

Procedures for the Election 

Management System PC 

(Chapter 10, steps 1 to 7) for steps 

to enable intended logging.

GLW 2009.06.09

V1:2.2.3.c appears to deal with 

devices, equipment and data 

processing components that may 

experience non-catastrophic 

failure. It is presumed that 

manually manipulating the 

election_name.EFC file would 

simulate an equipment or device 

non-catastrophic failure and then 

attempting to import that file into 

HPM would result in some sort of 

data integrity issue. 

Given that the data stored in the 

election_name.EFC is mostly 

textual data, it is not unreasonable 

that the manipulation of one byte 

of data would go undetected 

during the import. The purpose of 

the election_name.EFC file is to 

provide to HPM all of the election 

definition data collected and 

entered in EDM and ESSIM. This 

data is used to build the HPM 

election_name.DAT file which is 

the main database of election 

definition in HPM. HPM provides a 

variety of election definition and 

ballot style reports and displays 

that allow the HPM user to validate 

and verify the results of the HPM 

import process. This verification 

step is required to ensure integrity 

of the election definition. 

Additionally, pre-election L&A 

testing of all end-to-end 

processes, from election definition 

to tabulation to results publication 

6/22/09 CEC & JG 

Accept

Reexamination of the 

test identified a flaw 

in the generation of 

corrupted test data. 

The test was rerun 

with appropriately 

corrupted test data 

and the expected 

error and recovery 

was observed. (

6/18/2009 JG Reject

6/6/2009 JG Reject
VSTL Certification # pending

Page 208 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

file name leaving the extension .efc 

unchanged and importing the .ifc into 

HPM. The import was successful. We 

then restored the original .efc file name 

and deleted a couple of lines internal to 

the file. An error was generated and 

the .ifc import was unsuccessful. 

is performed to ensure the 

accuracy of the election. 

As explained in detail in the 

response to Discrepancy #138, all 

three ES&S recommended 

Windows User roles must have 

access to the \elecdata folder 

where the election_name.EFC file 

resides. If the ES&S application 

PC has been properly and 

completely hardened as 

documented in the ES&S 

Hardening Procedures, all 

accesses to this folder and files 

within this folder are tracked by the 

Windows Event Log, whether done 

within or outside of our 

applications. 

Uncontrolled physical access (see 

ES&S System Security 

Specification) to this PC had to be 

obtained and knowledge of a 

Windows Administrator defined 

User ID and Password with 

Administrator assigned access 

rights to the \elecdata folder had to 

be known.

141 5/21/09 J. 

Garc

ia

Docum

ent 

Defect

Closed EDM SOP, 

ver. rel. 

7.8.1.0 

5/11/09

Part 5 Ch. 37

The documentation does not state that 

District Total statistical counters are not 

supported by the DS200.

The documentation does not state that 

Assign District Totals statistical 

counters are only supported for the 

M650 and not supported for the 

DS200.

v.2: 2.8.5.a The vendor shall provide 

documentation of system operating 

procedures that meets the following 

requirements: a. Provides a detailed 

description of procedures required to 

initiate, control, and verify proper 

system operation;

TMT - 5.26.09 - EDM SOP 

Chapter 37: Statistical counters, 

header Assign District totals.

Added the following Note: "Assign 

District Totals statistical counters 

are only supported for the M650 

and not supported for the DS200."

6/8/09 Accept CEC

5/26/09 EDM SOP 

identifies non-support 

of Assign District 

Totals on the DS200

142 5/21/09 S. 

Eato

n & 

K. 

Swift

Inform

ational

Closed M650 SOP 

Procedures 

Firm-ware 

v.2.2.1.0

Hardware 

v.1.1 & 1.2

1/17/08

Message expected did not display 

when ballot was read into M650 upside 

down.

SOP indicates the M650 message 

"Ballot BACKWARDS or UPSIDEDOWN!" will appear if the ballot is 

incorrectly inserted. The message 

received was "Check Timing Track 

Sensor"

20090602 BMJ --

U3200_SOP00_M650 Updated 

"Check Timing Track Sensor" error 

message in Chapter 8, 

Troubleshooting (current page 60) 

to indicate that this error message 

may also indicate an improperly 

inserted ballot.

6/8/09 Accept CEC

Verified rev history 

5/29/09 M650 SOP 

also identifies 

incorrect insertion 

(Note: The data on 

the title page 

indicates 1/30/09, 

does not match the 

revision history date)

143 5/22/09 K. 

Swift 

& J. 

Inform

ational

Closed AIMS - VAT 

preview 

version 

ES&S Configuration Management: 

Preview mode is displaying the 

incorrect version of AIMS.

slm 06.01.09 - The Preview Mode 

is a stand alone program outside 

of AIMS. The Previewer will 

6/12/09 Accept J 

Garcia

Verified the correct 
VSTL Certification # pending

Page 209 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

Garc

ia

number in 

v.1.3.57 When entering Preview Mode, the GUI 

application is displaying a hardcoded 

AIMS version 1.3.2824. 

(Automark32.exe)

display the same version number 

as the VAT firmware. iBeta can 

review this with the Trusted Build 

version 1.3.2906.

version number is 

displayed in the 

preview mode. 

144 5/27/09 J 

Garc

ia

Functional 

Defect

Closed DS200 Ballot 

Scanning

A vote was not correctly recorded.

On Precinct 1580 contest 19 BB19 

candidates 5525 was hand marked 

using the ES&S specified ballot pen. 

The candidate was mark approximately 

50 % of the oval target (squiggled with 

50% ink and 50% white space). The 

mark read as an undervote and not a 

vote for candidate 5525. All other 

votes in the test case were correctly 

recorded. The DS200 report and 

report in ERM reflected the undervote. 

A second scan of the ballots was 

performed with ES&S observers 

present. A vote was read for candidate 

5525. 

6/12/09 CEC 

v.2:4.7.1.1 is an instruction to the labs 

for a test methodology that addresses 

v.1:3.2.1 . The VSS is unclear on what 

action is required when a system 

encounters a misread after reaching 

1,577, 644 ballot positions. The 

standard does permit one misread in 

3,126,404 ballot positions. However it 

is unclear if a system that has 

exceeded 1,577,644 ballot positions 

that does not have more than one 

misread must reach 3,126,404 ballot 

positions. With the EAC authorization 

for reuse we agree that inclusion of 

any testing by SysTest and iBeta is 

appropriate. Use of any other entities 

test results and in a form other than 

ballot positions is not addressed in the 

standard. We recommend referral to 

the EAC. 

v.1: 2.2.2.1c

To insure vote accuracy, all system 

shells: record each vote precisely as 

indicated by the voter and be able to 

produce an accurate report of all 

votes cast. 

v.1: 3.2.1 ... the system shall 

achieve a target error rate of no 

more than one in 10, 000,000 ballot 

positions and a maximum 

acceptable error rate in the test 

process of one in 500,000 ballot 

positions. 

SMP 2009.6.10 - ES&S has been 

unable to determine the cause for 

this vote not being detected in its 

original scan on the DS200. The 

DS200 was able to detect and 

count the vote in all subsequent 

scans in all ballot orientations. We 

were unable to obtain two key 

pieces of information that would 

have lead to an absolute 

conclusion: 1) a Digital Readings 

Report from the original scan, and 

2) the image of the ballot scanned 

that contained the undetected 

mark. The DS200 firmware looks 

at an area slightly larger than the 

oval size for a pixel count to 

determine if a mark is present or 

not. It identifies the oval, then 

ignores the pixels that makes up 

the actual oval to determine if it 

"recognizes" a mark. The DS200 

requires a minimum of 30 pixels 

(out of ~1200 max) in the voter 

response area to trigger the 

Intelligent Mark Recognition 

("IMR") routine that positively 

determines whether a mark is valid 

or marginal. In all subsequent 

scans of the ballot, the DS200 

consistently had readings in the 

400-500 pixel count range 

triggering IMR and resulting in the 

vote being detected and counted. 

As a result of these subsequent 

tests and the absence of the 

Digital Readings Report and 

image, it is our conclusion the only 

possible cause for the IMR to not 

be called is that the pixel count 

was less than the minimum for that 

scan, which is 30. When the pixel 

count is less than 30, the DS200 

sees it as an undervote or no vote, 

which was the result of the original 

test. When IMR is invoked, the 

6/19/09 CEC Accept 

EAC Response: The 

EAC believes that 

with the submission 

of field data from the 

systems use in 

Florida the 

discrepancy is 

resolved as 

supported by the 

total number of 

ballots run during the 

data accuracy testing 

and actual field 

experience.

6/12/09 CEC 

Undetermined
VSTL Certification # pending

Page 210 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu sLocation Description Requirement ESS Response Validation

only results are either a vote 

counted or a marginal mark that is 

returned to the voter with 

notification. In this case, the 

system was set to reject ballots 

containing marginal marks. The 

oval was not fully and properly 

filled in. It was poorly marked, 

squiggled with approximately 50% 

white space space and 50% ink in 

most generally the upper half of 

the oval. This marking, 

undetected on this single pass of 

the ballot, potentially may have 

been undetected due to presence 

of foreign material (e.g., paper 

dust, paper fuzz, other) atop the 

mark during the scan. Such a 

foreign material may have 

obscured all or a sufficient portion 

of the mark. In such a rare case, 

the scanner could have concluded 

the oval as not sufficiently filled in 

to register a vote. The fact that it 

was an incomplete mark, 

squiggles in only the upper portion 

of the oval, IMR not being invoked, 

and all subsequent scans 

consistently reading in the 400-500 

pixel count range well above the 

minimum pixel count necessary to 

invoke IMR, leads to this one 

conclusion.

VSS V2: 4.7.1.1 Data Accuracy, 

states "If the system reads at least 

1,549,703 consecutive ballot 

positions correctly, it will be 

accepted." The DS200 system 

exceeded this requirement in this 

certification by successfully 

recognizing 1,577,644 ballot 

positions with 100% accuracy, fully 

satisfying the requirements for 

Accuracy as defined by the 

standard. 100% vote detection 

accuracy levels were met during 

the formal accuracy tests, and 

through all other functional, 

volume, and stress testing. The 

above described issue occurred 
VSTL Certification # pending

Page 211 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu sLocation Description Requirement ESS Response Validation

only in a single specific ballot for a 

single specific test conducted for 

volume testing designed to 

investigate the scanner's response 

to overload (not an defined 

accuracy test per se). In addition, 

the DS200 recorded 100% 

accuracy all votes cast in 

approximately 35 functional test 

elections performed by SysTest 

Labs and subsequent tests 

performed by iBeta. Since late 

2007 and as recent as April 2009, the DS200 system has 

successfully passed four (4) Mass 

Ballot Tests performed by the 

Florida Division of Elections with 

100% accuracy. Those tests 

consisted of accurate scanning of 

39,778 ballots and 627,131 votes 

without a single accuracy error. 

Those test results are available on 

the Florida Division of Elections 

website at http://election.dos.state

.fl.us/voting-systems/certifiedvoting-system.shtml

It is ES&S' belief there is sufficient 

evidence regarding the accuracy 

of the DS200 as evidenced by the 

successful completion of the 

Accuracy Testing, the accuracy of 

the Functional tests performed by 

both SysTest and iBeta, as well as 

the test results from four State of 

Florida certifications to constitute 

certification approval. ES&S' 

diagnostic research into this 

anomaly concludes there is 

sufficient evidence to determine 

the scanner's ability to detect a 

mark in this case was obscured by 

the presence of a foreign material 

coupled with an incomplete mark 

not in accordance with instructions 

that resulted in the scanner's pixel 

count being below the minimum 

necessary for it to qualify as either 

a valid mark or a marginal mark. 

We believe this lone incident is an example of one of the key issues 
VSTL Certification # pending

Page 212 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

not addressed in the VSS and 

described in the Overview section 

entitled Issues Not Addressed by 

the Revised Standards, Human 

Error Rate vs. System Error Rate, 

stating "the term "error rate" 

applies to errors introduced by the 

system and not by a voter's 

actions, such as a failure to mark a 

ballot according to instructions" 

and "further research on human 

interface and usability issues is 

needed to enable the development 

of Standards for error rates that 

account for human error".

145 5/28/09 C 

Cog

gins

Functional 

Defect

Close DS200 Ballot 

Images

DS200 ballot image files were 

corrupted

While ballot images on a scanner are 

not required by the VSS, the ES&S 

TDP indicates that this is supported 

functionality. During execution of the 

Volume 9 Test Case the ballot image 

files were incomplete or corrupted for 

some of the polling places. A total of 

10 ballots were run in each polling 

place. Incomplete files only included 

the ballot image of the first ballot 

scanned. Corrupted files included the 

1st one or two ballot images and a 

corrupted file. Ten ballot image files 

were correctly generated for the 

majority of polling places. 

v.1: 9.4.1.4 The FCA is an 

exhaustive verification of every 

system function and combination of 

functions cited in the vendor's 

documentation. Through use, the 

FCA verifies the accuracy and 

completeness of the system's TDP.

SMP 2009.6.4 - ES&S has 

determined a faulty internal repartition and reformat procedure 

performed only on SanDisk brand 

2GB USB "thumb" drives to be the 

cause for the inconsistent storage 

of ballot image records and 

occasional file corruption. 

SanDisk USB devices may arrive 

from the factory with SanDisk's U3 

(smart drive) software. In our 

effort to provide our customers 

with consistently partitioned and 

formatted drives, recently ES&S 

took steps to remove the U3 

software when present, re-partition 

and reformat all drives in a 

consistent (but faulty) manner. 

The consequence of this faulty 

procedure has resulted in 

inconsistent and unreliable 

behavior of the 2GB drives when 

storing large amounts of data such 

as the ballot images. iBeta and 

ES&S have confirmed that no vote 

totals were impacted in testing 

performed and was isolated to the 

storing of the ballot images at poll 

close.

ES&S has revised our procedure 

for preparing SanDisk drives and 

updated the appropriate HPM and 

DS200Systems Operations 

Procedures documentation. 

6/29/09 Accept CEC

Verified using out-ofthe-box 2GB USB's 

provided by ESS the 

sample of 10 polling 

places with 100 

precincts stored the 

generated ballot 

images

6/24/09 CEC 

Pending completion 

of review and 

functional testing

6/16/09 - CEC iBeta 

cannot review this 

response as we are 

awaiting delivery of 

supporting 

documentation of the 

changes to the 

manufacturing 

procedures and the 

testing identified. 

Following the 

document review of

the procedures and 

ESS testing , iBeta 

will execute a test 

with a sampling of 

the 2GB drives which 

follow the new 

manufacturing 

procedures. 
VSTL Certification # pending

Page 213 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

In addition, ES&S requests iBeta 

to consider the results of ES&S' 

internal quality assurance tests 

prior to determining the final status 

of this discrepancy. ES&S has 

provided the Test Case and results 

to validate our conclusion and 

remedy for resolving the issue. 

20090617 MDN -- Updated the 

DS200 SOP -- Ch3 (current page 

16) and Ch5 (current page 54) and 

the HPM SOP -- Ch2 (pg20), Ch26 

(pg226) with the following 

verbiage:Warning: Ensure that 

DS200 USB memory devices are 

correctly partitioned and properly 

formatted to the FAT32 file system 

prior to loading devices with 

election configuration files

146 5/29/09 K. 

Wils

on

Inform

ational

Open ESSIM ESSIM continues to warn that audit is 

disabled but does not shut down. 

When the Audit Manager database is 

write locked (to simulate a failure of a 

critical process) ESSIM reports the 

error message "Unable to insert into 

database". ESSIM does not shut down 

but the user gets errors as they attempt 

different actions. 

v.1:2.2.5.3 ...The system shall also 

be configured to halt election 

software processes upon the 

termination of any critical system 

process *such as system audit) 

during the execution of election

software. 

20090608 MDN -- ES&S 

recognizes this informational 

discrepancy and has flagged this 

condition for further examination 

and development in a future voting 

system.

06/24/09 CEC 

Informational issue 

remains open 

because ES&S' 

response does not 

provide a version 

release commitment. 

If a commitment was 

provided the issue 

would be closed and 

transferred to the 

new test effort.

147 5/29/09 K. 

Wils

on

Docum

ent 

Defect

Closed ES&S System 

Functionality 

Description 

ERM Unity v. 

3.2.0.0 

11/20/08 v.1

Functionality dealing with the specific 

requirements of RFI 2008-07 are not 

completely documented

Verified RFI 2008-07 is met but the 

documentation does not reflect how the 

system is handling the RFI 

requirements dealing with:

a) memory is zeroed out prior to 

election counting

b) if memory is not zeroed out then 

election official is warned

c) production of a zero memory report 

to the audit log is associated with the 

above memory zeroization

2008-07 EAC Decision on Request 

for Interpretation (zero report): 

The certification testing of the 

components and system readiness 

by the VSTL shall include attempts 

to initiate an election with non-zero 

totals on counters or residual ballots, 

validating that the  ̳zero‘ report 

procedure will correctly identify and 

warn the election officials of the 

presence of any previously stored 

results which are in a form that may 

be deliberately or accidentally 

processed.

20090605 BBD --

U3200_SFD00_ERM: Updated 

Section 1.2.6 (Verification at the 

Central Location) to contain 

information relevant to EAC VSS 

RFI 2008-07

6/8/09 Accept CEC

Verified ERM SFD 

v.4 addresses 

functionality outlined 

in RFI 2008-07 a 

through c. 

148 5/29/09 K. 

Wils

on

Functional 

Defect

Closed ERM -

Password 

encryption 

algorithm

The key generation is not using 

industry best-practice random number 

generation. 

v.1:6.2.2.e Vendors shall provide a 

detailed description of all system 

access control measures designed 

to permit authorized access to the 

20090608 PMZ -- The ES&S EMS 

system relies on the Windows user 

system to provide user access 

authentication/control. All EMS 

6/19/2009 Accept JG 

(Peer KW)

Verified Windows 

authorized users 
VSTL Certification # pending

Page 214 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

In order to assess the computer 

generated password keys iBeta 

reviewed the source code to verify that 

the key was generated using industry 

best-practice random number 

generation and that the key strength is 

equivalent to or greater than the 

strength of any encryption algorithm 

that utilizes it. Key generation did not 

reflect industry best practice (NIST). 

(Detail of the method is withheld for 

security purposes.)

6/18/09 CEC Reject The response 

does not clearly address how this is 

applicable to the computer generated 

password. 

system and prevent unauthorized 

access, such as ... Computergenerated password keys

v.2:6.4 The ITA shall design and 

perform test procedures that test the 

security capabilities of the voting 

system against the requirements 

defined in v. 1 sect.6. These 

procedures shall focus on the ability 

of the system to detect, prevent, log 

and recover from a broad range of 

security risks as identified in Section 

6 and system capabilities and 

safeguards claimed by the vendor in 

its TDP that go beyond the risks and 

threats identified in section 6. 

PCs must be configured as 

detailed in the system hardening 

procedures manual with access 

restricted both physically and 

through the Windows user controls 

so that only authorized users are 

defined to the system as valid 

users and so that each valid user 

has a unique user identifier and so 

that each has only the access 

required for his/her role. The 

ES&S EMS has additional user 

functional IDs that are used to 

provide a supplemental value to 

the log records; these IDs are 

never used for authentication.

SLM 2009.06.18 - User 

Passwords are managed through 

Windows, which does meet 

industry standards. While the 

ES&S EMS has additional user 

functional IDs, there is no 

requirement for them to be 

encrypted as they are not used for 

authentication. 

access the system.

6/18/09 CEC Reject

149 5/29/09 K. 

Wils

on

Functional 

Defect

Closed ERM -

Blowfish 

algorithm

Algorithm code, which is not being 

called in the application, is present 

without self-test verification. 

In order to assess the access control 

for message encryption iBeta reviewed 

the source code for use of the blowfish 

algorithm to confirm it is self-tested 

and affirmed for usage at least once 

per program instantiation utilizing the 

test vectors in 

http://www.schneier.com/code/vectors.t

xt and that it prints the results of this 

self affirmation test to the audit log. 

iBeta found that although algorithm is 

never called from ERM the code is 

present and no self-test verification is 

performed prior to its usage if it is 

called from another module.

v.1:6.2.2.g Vendors shall provide a 

detailed description of all system 

access control measures designed 

to permit authorized access to the 

system and prevent unauthorized 

access, such as ... Message 

Encryption.

v.2:6.4 The ITA shall design and 

perform test procedures that test the 

security capabilities of the voting 

system against the requirements 

defined in v. 1 sect.6. These 

procedures shall focus on the ability 

of the system to detect, prevent, log 

and recover from a broad range of 

security risks as identified in Section 

6 and system capabilities and 

safeguards claimed by the vendor in 

its TDP that go beyond the risks and 

threats identified in section 6. 

GLW 2009.06.09:

The ES&S Hardening Procedures 

contains the recommended steps 

to harden and protect the PC‘s 

used to install and conduct 

elections using the ES&S 

application software within Unity 

3.2. Proper execution and 

completion of these steps, in 

addition to physical security 

measures as specified in the 

ES&S System Security 

Specification, provide a secure 

computing environment, including 

system access control measures 

and authentication of all users as 

defined by the Windows System 

Administrator. No message 

encryption is used to enhance 

system access controls or prevent 

unauthorized access. No 

transmission of election results is 

included as part of this certification 

event and therefore no encryption 

6/18/09 CEC Accept

Further examination 

confirms that the 

code is applicable to 

the iVotronic. The 

iVotronic is out of 

scope of Unity 

3.2.0.0. This 

discrepancy is 

moved to the Unity 

4.0.0.0 test effort. 

(#4)
VSTL Certification # pending

Page 215 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

is required. 

The ERM modules that use 

Blowfish encryption are invoked 

only if the iVotronic tabulator is 

being supported in the defined 

election. This Unity 3.2 EAC 

certification does not include

qualification of the iVotronic voting 

system. The modules that contain 

Blowfish logic are: ESSCRPT1, 

ESSPEB.DLL, EVENTS.EXE, 

IMAGES.EXE, UNDRVOTE.EXE 

and VioDialog.EXE. These 

modules are included in this 

version of ERM to facilitate the 

addition of the iVotronic voting 

system at a later date. This 

encryption is used to encrypt all 

data written to the iVotronic 

tabulator media: PEB‘s & CF cards

150 5/29/09 K. 

Wils

on

Functional 

Defect

Closed ERM - Audit 

log 

Administrator modifications to the user 

ID and access privileges are not 

recorded in the audit log. 

User identification in ERM appears in 

the first column of the log file when it is 

enabled as per Ch 7 of "ERM System 

Operations Procedures Ver. Rel. 

7.5.3.0 4/3/09" However there is no 

indication in the log that an 

administrator modified (enabled or 

disabled) the user ID system or 

modified the access privileges of the 

specific user ID. By enabling the user 

ID system a jurisdiction may limit user 

privileges within ERM and windows 

login provides the ability to detect the 

authenticated user ID accessing ERM. 

However there is no detection that user 

privileges were changed or a 

corresponding time in the ERM log file 

to trace the Windows authenticated 

user that made modifications to the 

ERM User ID system.

v.1:2.2.1.a Provide security access 

controls that limit or detect access to 

critical system components to guard 

against loss of system integrity, 

availability, confidentiality, and 

accountability.

v.1:2.2.4.1.g Record and report the 

date and time of normal and 

abnormal events;

20090608 PMZ -- The ES&S EMS 

system relies on the Windows user 

system to provide user access 

authentication/control. All EMS 

PCs must be configured as 

detailed in the system hardening 

procedures manual with access 

restricted both physically and 

through the Windows user controls 

so that only authorized users are 

defined to the system as valid 

users and so that each valid user 

has a unique user identifier and so 

that each has only the access 

required for his/her role. The 

ES&S EMS has additional user 

functional IDs that are used to 

provide a supplemental value to 

the log records; these IDs are 

never used for authentication.

Accept 6/17/09 JG

Logged in as an 

administrator and 

added a new USER

ID . The event log 

displayed "Special 

privileges assigned 

to new logon", 

Modified the user 

privileges and the 

event log indicates 

an event had occur.

151 5/29/09 K. 

Wils

on

Functional 

Defect

Closed ERM Audit 

Log

The ERM audit log does not identify an 

import as failed.

The tester replaced the DS200 import 

file with random data and attempted to 

import it into ERM. The file was 

refused. The entry in the audit log 

v.1:2.2.4.1.g Record and report the 

date and time of normal and 

abnormal events;

v.1: 2.2.5.2 The software shall 

activate the logging and reporting of 

audit data as described in the 

GLW 2009.06.09

The steps taken to yield this 

discrepancy include the intentional 

introduction of a corrupt file into 

the ERM vote accumulation 

process used for the DS200. In 

this case, the results file that is 

6/26/09 Accept CEC 

verified in ERM v. 

7.5.4.0 correct 

reporting to audit log.

6/24/09 CEC pending 

functional testing
VSTL Certification # pending

Page 216 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

indicated the start and stop of import 

with no precincts but did not identify 

that an error had occurred. 

6/16/09 CEC Reject

We agree that the system will not 

permit a corrupt file to be imported; the 

error is reported to the user; and that 

the audit log indicates file importation 

started and stopped without importing 

any precincts. However the error is 

not stored, nor is it logged and 

reported in the audit log as indicated in 

the requirement. 

6/19/09 EAC Response: The 

discrepancy needs to be resolved and 

the error message needs to be written 

into the audit log.

following sections. ...v.1: 

2.2.5.2.2.aThey shall generate, store 

and report to the user all error 

messages as they occur

updated on the USB memory 

device resident in the DS200 as 

each ballot is processed on the 

DS200 was manually replaced, 

outside of the ES&S application 

software, with a file containing 

random data before it was 

attempted to be read and updated 

into ERM. The corrupt file was 

immediately identified, before any 

updates were attempted, by the 

low level DLL used by ERM to 

access and retrieve the contents 

from the DS200 USB memory 

device. In this case, the 

ESSM100.DLL performed its 

standard CRC check on the results 

data portion of the DS200 results 

file resident on the USB memory 

device and determined that the 

CRC value stored in the results 

block did not match the computed 

CRC from the respective data 

storage area. When this occurs, a 

 ̳Pack error: CRC error‘ is 

immediately displayed back to the 

ERM user screen with only an  ̳OK‘ 

button for response. When the 

ERM user selects the  ̳OK‘ button, 

processing on the currently 

inserted USB memory device 

immediately terminates and must 

be removed and replaced with the 

next successive DS200 USB 

memory device to be processed. 

This event is tracked by ERM 

using the Precincts Counted / Not 

Counted reports that are available 

within ERM. All precincts defined 

in the polling place for the DS200 

USB memory device would reflect 

a  ̳not counted‘ status. 

Examination of the ERM System 

Log would also show the absence 

of any updates from this USB 

memory device as does the ERM 

user screen displayed during the 

DS200 results update process. It 

is also standard practice to 

manually audit ballots cast by 

6/16/09 CEC Reject
VSTL Certification # pending

Page 217 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

precinct, as determined from the 

election day poll books, back to 

Precinct Reports printed from ERM 

after the conclusion of election 

night processing to validate that all 

precincts report the correct total 

number of ballots cast. 

MDN 2009.06.24 -

U3200_SFD00_ERM -- Added 

new system messages to Section 

1.1.5.1 (current page 13). 

U3200_SDS00_ERM -- Added 

new system messages to Section 

10.7, Appendix G, "ERM LOG File 

.ALG Layout" (beginning on 

current page 557). 

U3200_SOP00_ERM -- Added 

new system messages to Chapter 

60, "Error Messages" (beginning 

on current page 578)

SLM 2009.06.24 - This issue will 

be addressed in the trusted build 

of ERM 7.5.4.0.

152 5/29/09 K. 

Swift

Docum

ent 

Defect

Closed M650 SOP, v. 

2.2.1.0, 

10/17/08 

Ch. 10 Install 

Procedures

M650 installation procedures are 

incomplete.

The installation procedure does not 

address: 

1) Install Model 650 Firmware with 

Green Light Sensors, pg. 109; between 

steps 6 & 7, removal of the zip disk. 

2) Install Model 650 Firmware with Red 

Light Sensors, pg. 109; how to verify 

the installation.

v.2: 2.6.4

The vendor shall provide a detailed 

description of the system capabilities 

and mandatory procedures for 

purchasing jurisdictions to ensure 

secure software (including firmware) 

installation to meet the specific 

requirements of Volume I, Section 

6.4 of the Standards. This 

information shall address software 

installation for all system 

components.

20090602 BMJ -- Pg. 109: 1) 

Install Model 650 Firmware with 

Green Light Sensors, pg. 109; 

between steps 6 & 7, removal of 

the zip disk. BMJ - Added step to 

process where the user is 

prompted to remove the disk. 

2) Install Model 650 Firmware with 

Red Light Sensors, pg. 109; how 

to verify the installation. BMJ -

Added step to process prompting 

the user to power down and restart 

the unit. After restart, they are then 

to check the log printer to verify 

that the firmware version has been 

updated.

6/8/09 Accept CEC

Verified M650 SOP 

5/29/09 addresses 

removal of the zip 

disk and installation 

validation 

153 5/29/09 K. 

Swift

Docum

ent 

Defect

Closed AutoMARK 

VAT Firmware 

and Hardware 

Installation 

Instructions, 

v.1.3.157, 

Rev. History 

v. 2, 05/28/09

AutoMARK VAT installation procedures 

are incomplete.

The installation procedure does not 

address: 

1) Section 5.4, step 3, turning the key 

to the 'Test' position. 

2) Section 5.4 Step 14 referencing 

Appendix B, regardless of errors, as 

the base printer calibrations need to be 

v.2: 2.6.4

The vendor shall provide a detailed 

description of the system capabilities 

and mandatory procedures for 

purchasing jurisdictions to ensure 

secure software (including firmware) 

installation to meet the specific 

requirements of Volume I, Section 

6.4 of the Standards. This 

information shall address software 

20090602 MDN -- Updated the 

following:

1) Section 5.4, step 3-Updated to 

instruct the user to turn the key to 

the TEST position.

2) 5.4, Step 14 -- added verbiage 

to instruct the user to calibrate the 

printer per VSTL item 2

3) Section 7.3, Added images for 

the A200 version of the ES&S 

6/8/09 Accept CEC

Verified v.VAT FW & 

HW Installation 

Instructions v.3 

6/1/09 addresses 

items: 1, 2, 3, 4, 5, 6 
VSTL Certification # pending

Page 218 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

set and the 'Test Ballot' performed to 

fine tune the printer calibration settings. 

3) Section 7.3 Programming the PEB 

steps and photos for the A100.; only 

the A200 is addressed.

4) Section 9.3 Programming Ultrasonic 

Detector Board (USD), pg. 36; Steps 8 

& 22 are unnecessary as the board is 

not connected to the VAT

5) A typo in Section 9.3 Step 9 it does 

not read "...USB MON08 MULTILINK".

6) Section 9.3 Step 12, the user 

checking of "Ignore..." checkbox and 

not the 'User' checkbox. In this case 

Step 13 is unnecessary

installation for all system 

components.

AutoMARK

4) Section 9.3 Removed steps 8 

and 22

5) Corrected identified typo in 

section 9.3. (now step 8)

6) Section 9.3 (now step 11) 

changed the instruction to direct 

the user to check the IGNORE 

box. Removed step 13.

154 6/10/09 J 

Garc

ia

Docum

ent 

Defect

Closed Hardening 

Procedures 

for

the EMS PC

May 5, 2009

The lock-down directory procedures 

are missing user rights.

The ElectDefine user is not to have 

ERM application rights however in step 

6 it does not state to deny ElectDefine 

ERM rights. 

The ElectResult user is not to have 

HPM application rights however in step 

6 it does not state to deny ElectResult 

HPM rights.

V2: 2.6.4

The vendor shall provide a detailed 

description of the system capabilities 

and mandatory procedures for 

purchasing jurisdictions to ensure 

secure software (including firmware) 

installation to meet the specific 

requirements of Volume I, Section 

6.4 of the Standards. This 

information shall address software 

installation for all system 

components.

MM 2009.06.11

Added the following bullets to 

Chapter 2, step 6 (current page 7): 

h. HPM executable- Deny 

execution rights to ElectResults

i. ERM executable- Deny 

execution rights to ElectDefine. 

Accept 6/16/09 JG

Chapter 2, page 7 

step 6 indicates 

further lock down 

procedures of ERM 

and HPM.

155 6/10/09 sjakil

eti

Docum

ent 

Defect

Closed Unity_BECI_3

.2.0.0_2009.0

5.20

The Unity build procedures list 

incorrect COTS download information.

The build procedures incorrect list the 

download locations. Corrected 

locations were provided by the ES&S 

development staff: 

1) HPM COTS installs lists the 

location for msvcrt.dll as Microsoft 

Visual Studio 6.0 Service Pack 3 and 

not Microsoft Visual Studio 6.0 Service 

Pack 5 

2.) HPM COTS installs lists the 

download location for Quartz.dll as 

Windows XP Service Pack1 and not 

Windows XP Security Patch

3) ERM install COTS lists the 

downloaded location for unzip.exe as 

http://sourceforge.net/project/showfiles.

php?group_id=118012 

&package_id=128992&release_id=311

6209 and not ftp://tug.ctan. org/texarchive/tools/zip/info-zip/WIN32/ )

v.1: 8.5.a :The vendor shall establish 

such procedures and related 

conventions, providing a complete 

description of those procedures 

used to:

a. Develop and maintain internally 

developed items;

EAC Voting System Testing and 

Certification Program Manual, v.1.0 

Sec 5.5.1. Demonstrate that the 

software was built as described in 

the Technical Data Package.

20090617 ETS -- Corrected the 

following entries in the table "HPM 

- Install"

1. For msvcrt.dll, corrected the 

entry under "URL for Download or 

source" from Microsoft Visual 

Studio 6.0 Service Pack 3 to 

Microsoft Visual Studio 6.0 Service 

Pack 5.

2. For Quartz.dll, corrected the 

entry under "URL for Download or 

source" from Microsoft Windows 

XP Service Pack 1 to Microsoft 

Windows XP Security Patch.

Corrected the following entries in 

the table "ERM - Install"

1. For unzip.exe, corrected link 

under "File Location (Source)" 

from 

Accept 6/19/2009 JG

Verified the following:

1-Microsoft Visual 

Studio 6.0 Service 

Pack 5

(dated: 3/7/2000 -

same size and 

version, not

able to find one 

dated 3/23/1999) 

2-Microsoft Windows 

XP Security Patch

3-

ftp://tug.ctan.org/texarchive/tools/zip/infoz

ip/

WIN32/

4-ERM 

MSVRTD.DLL 

software 

v.5.0.0.7022.
VSTL Certification # pending

Page 219 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

ERM installs lists the version of 

MSVCRTD.DLL as 6.0.8447.0 and not 

5.0.0.7.022 

http://sourceforge.net/project/show

files.php?group_id=118012&packa

ge_id=128992&release_id=31162

09 to ftp://tug.ctan.org/texarchive/tools/zip/info-zip/WIN32/.

2. For MSVCRTD.DLL, corrected 

the entry under "Software Version" 

from 6.0.8447.0 to 5.0.0.7022

156 6/10/09 K. 

Swift

Docum

ent 

Defect

Closed DS200 

Operating 

System 

Installing/Repl

acing 

Compact 

Flash 

Procedures, 

V. 1.0

The DS200 Installation procedures are 

inaccurate or incomplete.

1) Section 2.2, steps 5 states to 

remove rear cover however the cover 

need not be removed to access the OS 

with Firmware CF card. 

2) Steps 8, & 26 states to remove the 

switch door however the door is not 

removable.

3) Section 2.3, step 3 does not detail 

the procedures to verify proper 

operation of the CF card. 

4) There is no reference to a document 

or instructions to calibrate the DS200 

touch screen. It is not clear if a USB 

must be inserted for the calibration. 

5) There is no reference to a document 

or instructions to load the new 

firmware.

V2: 2.6.4

The vendor shall provide a detailed 

description of the system capabilities 

and mandatory procedures for 

purchasing jurisdictions to ensure 

secure software (including firmware) 

installation to meet the specific 

requirements of Volume I, Section 

6.4 of the Standards. This 

information shall address software 

installation for all system 

components.

MDN 2009.06.11

Ported instructions for firmware 

and OS upgrades to the DS200 

System Maintenance Manual 

(U3200_SMM00_DS200) 

Chapters 7-9. Removed DS200 

07 OS Instl-Repl CF PROC and 

DS200 08 Touch Screen CAL from 

the TDP submission. Procedures 

are duplicated in the System 

Maintenance Manual.

(1) and (2) Section 2.2, now 

included as Chapter 7 of 

U3200_SMM00_DS200, updated 

to reflect current procedures for 

installing the system OS. Opening 

the case is not required. (3) 

Added a warning box instructing 

the user to execute an election test 

(instructions detailed in Chapter 5 

of U3200_SOP00_DS200) to 

verify the system OS and firmware 

are functioning properly. (4) 

Instructions for calibrating the 

touch screen included under 

U3200_SMM00_DS200 Chapter 8. 

Misleading power button image 

that included an inserted USB stick 

within this procedure has been 

replaced with an image that does 

not include the inserted USB 

device. (5) Instructions for 

updating system firmware included 

as Chapter 9 of 

U3200_SMM00_DS200. 

Accept 6/17/09 CEC

Verified DS200 

System Maintenance 

Manual 6/12/09 

chapters 7, 8, & 9 

contain identified 

content. 

157 6/10/09 J 

Garc

ia

Docum

ent 

Defect

Closed AIMS 3010 

Election 

Officials 

Guide AQS13-5001-208-

R

The installation procedures identify an 

inaccurate step. 

The document states to locate the 

AIMS folder in C:/Program Files/AIMS 

and delete the folder. The folder is 

V2: 2.6.4

The vendor shall provide a detailed 

description of the system capabilities 

and mandatory procedures for 

purchasing jurisdictions to ensure 

secure software (including firmware) 

MDN 2009.06.11

AIMS Election Officials Guide --

Updated Section 3.3, beginning on 

current page 32. Removed 

erroneous instruction for deleting 

AIMS files and folders from the 

Accept 6/16/09 CEC

Rev 16 page 32 

identifies the AIMS 

folder is removed by 

the uninstall.
VSTL Certification # pending

Page 220 of 247 (V)2009-30Jun-001(A)

# Date Tes

ter

Type Statu

s

Location Description Requirement ESS Response Validation

Rev 14

4/06/09

already removed when the executing 

the Add or Remove Programs step 

(application "AIMS for ES&S 1.3" )

installation to meet the specific 

requirements of Volume I, Section 

6.4 of the Standards. This 

information shall address software 

installation for all system 

components.

uninstall procedure.

158 6/10/09 J 

Garc

ia

Docum

ent 

Defect

Closed The Hardening document does not 

address restoring to a clean 

environment. 

Removing AIMS ESS DLL is required 

and in order to remove this item the 

user needs a MS windows installer 

clean up tool. Neither the AIMS ESS 

DLL nor not tool is documented.

V2: 2.6.4

The vendor shall provide a detailed 

description of the system capabilities 

and mandatory procedures for 

purchasing jurisdictions to ensure 

secure software (including firmware) 

installation to meet the specific 

requirements of Volume I, Section 

6.4 of the Standards. This 

information shall address software 

installation for all system 

components.

MDN 2009.06.12

Updated the AIMS Election 

Officials Guide Section 3.3, 

beginning on current page 32, with 

instructions for procuring and 

installing the Windows Installer 

Cleanup Utility and procedures for 

using this utility to uninstall AIMS 

ESS.DLL. Added a note to 

Section 3.2 of the ES&S System 

Hardening Procedures (current 

page 9) referencing this procedure 

as a required step when upgrading 

to a new version of AIMS.

Accept 6/16/09 JG

Rev 16 Page 32 

identifies the 

Windows Installer 

Clean up tool and 

how to use the tool.

159 6/12/09 J 

Garc

ia

Functional 

Defect

Closed AIMS 1.3.157

Ballot Viewer/ 

Preview 

Registry message displayed when 

attempting to Preview ballots.

When attempting to preview ballots a 

message displays "No AutoMARK 

Application Path Key found in registry. 

Action cancelled." The ballots could not 

be previewed.

V1: 2.3.4.1.d

All systems shall provide the 

capabilities to:

d. Verify that all ... software function 

correctly.

MDN 2009.06.12

Installing VAT Preview remedies 

the described condition. Updated 

the AIMS Election Officials Guide 

Section 3.2.1, beginning on current 

page 19 with instructions for 

installing VAT Preview.

Accept 6/16/09 JG

Section 3.2.1 of the 

AIMS Election 

Officials Guide states 

to install the VAT 

Preview prior to the 

AIMS install and how 

to install VAT 

Preview.

160 6/12/09 J 

Garc

ia

Inform

ational

Open AIMS build 

date displays 

incorrectly

Configuration Management: The build 

date is incorrectly displayed in AIMS 

and on the VAT ,

AIMS v.1.3.157 was built on 5/26/09. 

The build date in the AIMS Help Menu 

and the VAT "Display Software 

Version" appears as 3/30/09 instead of 

5/26/09. The version number is 

correctly displayed. 

SMP 2009.6.15 - The "build date" 

listed in the AIMS Help>About 

listing is a carry over from the 

previous certification days as a 

method of offering additional 

information for confirming the 

version of software/firmware 

running on they system. The 

"build date" was populated on the 

day the witness build of the 

release was performed. This field 

is no longer relevant under the 

EAC Program and not used in 

either the Trusted Build process 

nor the System Identification 

procedures. ES&S will agree to 

remove this reference in a future 

release of AIMS.

6/24/09 CEC 

Configuration 

management issues 

are identified as 

Informational 

because the VSS 

only requires 

identification of 

inconsistencies.
VSTL Certification # pending

Page 221 of 247 (V)2009-30Jun-001(A)

7.6 Appendix F: Warrant of Accepting Change Control

Responsibility

A copy of the ES& S Unity 3.2.0.0 warrant accepting change control responsibility is attached as a separate

document.
VSTL Certification # pending

Page 222 of 247 (V)2009-30Jun-001(A)

7.7 Appendix G: Trusted Build & Validation Tools Unity 3.2.0.0 

voting system

The ES&S Unity 3.2.0.0 voting system is composed of the hardware, software, and documents identified in section 

3 Voting System Identification. 

IBeta uses a COTS hash program (Maresware) to obtain File Size, MD5 and SHA1 hashes during all witnessed and 

trusted builds. Both algorithms have been validated using the test data from the NIST NSRL website 

(http://www.nsrl.nist.gov/testdata/). This program is widely used in forensic analysis of systems and also used by 

some states to verify their voting software. The MD5 and SHA1 hashes are taken to be consistent with the currently 

distributed NSRL data files which contain the hash resulting from each of those two algorithms.

Listed below are the Source Code Applications reviewed by iBeta for the Final Trusted Builds and Witness of the 

ES&S Unity 3.2.0.0 voting system firmware and software. (NIST Handbook 150-22 4.2.3, 4.13.2, 4.13.4, 5.10.4 

VSS vol. 1: 9.6.2.4)

7.7.1 Witness of the Trusted Build of AIMSCRYPT v. 1.0.0.2

Application/ Component Version Language File Signature

AIMSCRYPT 1.0.0.2a C/C++ AIMSCRYPTInputTB_05262009.hash.txt

Document Prior to the Build Witness:

Vendor Name ES&S

Vendor Consultant(s) (5.6) Rhoda Fox

Witness Name (5.6) Sridevi Jakileti

Witness Title Trusted Builder

Vendor Build Document(s) used and version(s) 1. WinXPwithSP3-DellOptiplexGX520_INST_2009.03.31.pdf

2. AIMSCRYPT Build Environment Compile-Install Guide 

version 1.0

3. Visual vtudio_6.0_EntEdwithSP5_INST_2009.04.30.pdf 

Equipment Used DellOptipllexGX520

iBeta COTS used to clean the build environment disk 

(name and version) (5.6.1.1)

Active KillDisk for DOS V:4.1 Build 2380

iBeta COTS used to generate HASH file signatures 

(name and version)

Mares Hash Ver. 07.08.10.07.12

Construct the build environment (5.6.1.2)

Verify (by signature) that the build environment is 

isolated and controlled by iBeta

Sridevi Jakileti

Witness attests to verifying that the source code being 

built is the source code provided by iBeta

Rhoda Fox

Vendor CM Tool and version Listed in TDP document: Unity 3.2.0.0 Source Code 

Repository & Version

Build Environment Operating System Windows XP Professional v.2002 Service Pack 3 

Build tool(s) and version(s) Microsoft Visual C++ 6.0 

3

rd Party Libraries and Version None

3

rd Party Source Code (COTS) and Version None

3

rd Party DLLs, Drivers, etc. and Version(s) None

Additional file(s) loaded and version(s) None

Record the disk image software version being used Norton GHOST V:11.0

Record the filename of the build environment file 

signature (5.6.1.3) – 

ESS_AIMSCRYPT_PostCots_05262009.hash.txt

Record the filename of the build environment disk 

image –

ESS_AIMSCRYPT_PostCots_05262009.GHO

Verify (by signature) the build environment file signature 

(5.6.1.3)

Sridevi Jakileti

Loading Source Code (5.6.2)

Record the file signature of the source code (5.6.2.1) See above source code table

Verify (by signature) that each file signature of the 

source code loaded matches as documented above 

(5.6.2.1)

Sridevi Jakileti
VSTL Certification # pending

Page 223 of 247 (V)2009-30Jun-001(A)

Method of Build Witness Trusted Build

Record the combined source code and pre-build 

environment file signature (5.6.2.2)

ESS_AIMSCRYPT_PreBuild_05262009.hash.txt

Record the combined source code and pre-build 

environment disk image (5.6.2.3)

ESS_AIMSCRYPT_PreBuild_05262009.GHO

Record the Final Build Version – Unique Identifier AIMSCrypt1.0.0.2

Certification Application Number (if applicable) ESS0701

Document during the Build Witness:

Date / Time Build Initiated 5/26/2009 10.00am

Compiler and Version See Build tools and versions

Application Name AIMSCRYPT

Application Version Order AIMSCrypt1.0.0.2

Obtain Names and Signatures of all persons present 

during build (record below)

Sridevi Jakileti

Rhoda Fox

Issue(s) and Resolution(s) No Issues

Document at Completion of the Build Witness:

Record the disk image of the final build (5.7.3) ESS_AIMSCRYPT_PostCots_05262009.hash.txt

Record file signature of the final build (5.6.3.1) ESS_AIMSCRYPT_PostCots_05262009.GHO

Record the type of unalterable storage media being 

used for installation disk(s) (i.e., CD) – (5.6.3.2)

CD

Record each piece of media that is part of the 

installation disk (each must have a unique identifier) 

(5.6.3.2, 5.7.5)

CD: AutomarkESSCrypt TB05/26(ESSCrypt.dll is input to 

Automark Build)

Record the file signature of the installation disk(s). 

(5.6.3.3, 5.7.5) (include in below archive)

ESS_AIMSCrypt_TBInstall05262009.hash.txt

Record the type of unalterable storage media being 

used for pre-build and post-build archive disk (i.e., CD) 

–

T9 hard drive & backup on T8 final burn to unalterable media 

is pending the EAC Initial Decision

Record each piece of media that is part of the pre-build 

archive disk (each must have a unique identifier) 

(5.6.2.4, 5.7.2, 5.7.3)

T9 hard drive & backup on T8 final burn to unalterable media 

is pending the EAC Initial Decision

Explanation of any significant differences observed No differences

Build Witnesses): Hard copies of the witness signatures are on file at iBeta:

iBeta: Sridevi Jakileti 5/27/2009

ES&S Rhoda Fox 5/27/2009

7.7.2 Witness of the Trusted Build AutoMARK PrinterEngineBoard1.7 (PEB)

Application/ Component Version Language File Signature

PrinterEngineBoard 1.70.0.1 C ESS_AutoMarkPEB_Source_06092009.hash.txt

Document Prior to the Build Witness:

Vendor Name ES&S

Vendor Consultant(s) (5.6) Rhoda Fox

Witness Name (5.6) Sridevi Jakileti

Witness Title Trusted Builder

Vendor Build Document(s) used and 

version(s)

AutoMARK PEB Firmware Compilation Instructions(rev 2 6/9/2009)

Equipment Used DellOptipllexGX520

iBeta COTS used to clean the build 

environment disk (name and version) 

(5.6.1.1)

This build was started by restoring from the AutoMark_05222009 PostCots 

Image ESS_AutoMark_PostCots_05222009.GHO

iBeta COTS used to generate HASH file 

signatures (name and version)

Mares Hash Ver. 07.08.10.07.12

Construct the build environment 

(5.6.1.2)

Verify (by signature) that the build 

environment is isolated and controlled by 

iBeta

Sridevi Jakilet

Witness attests to verifying that the source Rhonda Fox
VSTL Certification # pending

Page 224 of 247 (V)2009-30Jun-001(A)

code being built is the source code 

provided by iBeta

Vendor CM Tool and version Listed in TDP document: Unity 3.2.0.0 Source Code Repository & Version

Build Environment Operating System Windows XP Professional Version 2002 Service Pack 2 

Build tool(s) and version(s) VAIO System Recovery DVD PCG-K23/PCG-K25/PCG-K27 Series

Microsoft Embedded Visual C++ 4.0 Service Pack 4.

Microsoft Visual Studio .NET 2003 Service Pack 2

Keil Software μVision2, C compiler Version 

2.40

Texas Instruments Code Composer Studio. Version 2.0

Cosmic Compiler V 4.1H

Borland C 4.02

Prog08sz Programmer for v 2.05

Atmel Flip v2.4.6

Atmel MCU ISP Software V1.0

Microsoft Access XP/2002

InstallShield 10.5

Microsoft Windows CE With Platform Builder Version 5.0

AutoIT 3.3.0.0

3

rd Party Libraries and Version ACCESSRT.MSI v2002,MSOHELP.exe 10.0.2609.0,Office1.cab v2002,

OSP.MSI v2002,OSP1.cab v2002,FILES\SYSTEM\EXTRACT.exe v2002,

FILES\SYSTEM\MLANG.DAT v2002,FILES\SYSTEM\MLANG.DLL 

5.0.2919.6304,FILES\SYSTEM\MSXML.DLL 5.0.2919.6303,

FILES\SYSTEM\T2EMBED.DLL,0.2.0.69,FILES\WINDOWS\HELP\OSP.HLP 

v2002,IE5\EN\ACTSETUP.cab v2002,IE5\EN\ADVAUTH.cab v2002,

IE5\EN\AOLSUPP.cab v2002,IE5\EN\AXA.cab v2002,IE5\EN\AXA2.cab 

v2002,IE5\EN\AXA3.cab v2002,IE5\EN\BRANDING.cab v2002,

IE5\EN\DCOM95.exe 4.71.1015.0,IE5\EN\DXDDEX.cab v2002, 

IE5\EN\DXMINI.cab v2002,IE5\EN\FONTCORE.cab v2002,

IE5\EN\FONTSUP.cab v2002,IE5\EN\FPESETUP.cab v2002,

IE5\EN\GSETUP95.cab v2002,IE5\EN\GSETUPNT.cab v2002,

IE5\EN\HELPCONT.cab v2002,IE5\EN\HHUPD.cab v2002,

IE5\EN\ICW.cab v2002,IE5\EN\ICWCON.cab v2002,

IE5\EN\IE4MFC40.cab v2002,IE5\EN\IE4SHL95.cab v2002,

IE5\EN\IE4SHLNT.cab v2002,IE5\EN\IE5COMP.exe 5.0.2919.6307

IE5\EN\IE5SETUP.exe 5.0.2919.6307,IE5\EN\IECIF.cab v2002,

IE5\EN\IEDATA.cab v2002,IE5\EN\IEDATAJA.cab v2002,

IE5\EN\IELPKAD.cab v2002,IE5\EN\IELPKAR.cab v2002,

IE5\EN\IELPKIW.cab v2002,IE5\EN\IELPKJA.cab v2002,

IE5\EN\IELPKKO.cab. v2002,IE5\EN\IELPKPE.cab v2002,

IE5\EN\IELPKTH.cab v2002,IE5\EN\IELPKVI.cab v2002,

IE5\EN\IELPKZHC.cab v2002,IE5\EN\IELPKZHT.cab v2002,

IE5\EN\IENT_S1.cab v2002,IE5\EN\IENT_S2.cab v2002,

IE5\EN\IENT_S3.cab v2002,IE5\EN\IENT_S4.cab v2002,

IE5\EN\IENT_S5.cab v2002,IE5\EN\IESETUP.INI v2002,

IE5\EN\IE_EXTRA.cab v2002,IE5\EN\IE_S1.cab v2002,

IE5\EN\IE_S2.cab v2002,IE5\EN\IE_S3.cab v2002,

IE5\EN\IE_S4.cab v2002,IE5\EN\JAAIME.cab v2002,

IE5\EN\KOAIME.cab v2002,IE5\EN\MAILNEWS.cab v2002,

IE5\EN\MDAC_IE5.cab v2002,IE5\EN\MOBILE95.cab v2002, 

IE5\EN\MOBILENT.cab v2002,IE5\EN\MPCDCS.cab v2002,

IE5\EN\MPLAYER2.cab v2002,IE5\EN\MSN_AUTH.cab v2002,

IE5\EN\NM30.cab v2002,IE5\EN\OAINST.exe 4.71.1015.0

IE5\EN\SCAIME.cab v2002,IE5\EN\SETUPNT.cab v2002,

IE5\EN\SETUPW95.cab v2002,IE5\EN\SWDIR.cab v2002,

IE5\EN\SWFLASH.cab v2002,IE5\EN\TCAIME.cab v2002,

IE5\EN\TS95.cab v2002,IE5\EN\TSNT.cab v2002,

IE5\EN\USP10.cab v2002,IE5\EN\VBSCRIPT.cab v2002,

IE5\EN\VGX.cab v2002,IE5\EN\VMX86_01.cab v2002,

IE5\EN\VMX86_02.cab v2002,IE5\EN\VRML2C.exe 4.71.1015.0

IE5\EN\WAB.cab v2002,IE5\EN\WEBFLDRS.cab v2002,

IE5\EN\WPIE5X86.cab v2002,chs.syn 6.1.0.0

chsrom.DLL 6.1.0.0,eci.DLL 6.1.0.0,enu.syn 6.1.0.0

esm.syn 6.1.0.0,jpn.syn 6.1.0.0,jpnrom.DLL 6.1.0.0
VSTL Certification # pending

Page 225 of 247 (V)2009-30Jun-001(A)

kor.syn 6.1.0.0,korrom.DLL 6.1.0.0,ARIALUNI.TTF,BATANG.TTF

MSMINCHO.TTF,PMINGLIU.TTF,sqlxml.MSI c3.0

xblkld3.DLL 3.30.3457.0,Helper.exe,SqlRun.cab,,SqlRun01.MSI

1033dotnetfx.exe 1.1.4322.573

1033,dotnetfxSp1.exe 1.0.0.0,dotnetfxSp1.exe 1.0.871.2738

chs.syn v6.1.0.0,chsrom.DLL v6.1.0.0,eci.DLL v6.1.0.0,

enu.syn v6.1.0.0,esm.syn v6.1.0.0,jpn.syn v6.1.0.0,jpnrom.DLL v6.1.0.0,

kor.syn v6.1.0.0,korrom.DLL v6.1.0.0,chs.syn v6.1.0.0,chsrom.DLL v6.1.0.0,

eci.DLL v6.1.0.0,enu.syn v6.1.0.0,esm.syn v6.1.0.0,FTD2XX.DLL v6.1.0.0,

ftd2xx.inf v6.1.0.0,ftdi_d2xx.DLL v6.1.0.0,jpn.syn v6.1.0.0,jpnrom.DLL 

v6.1.0.0,kor.syn v6.1.0.0,korrom.DLL 

v6.1.0.0,Microsoft.WindowsCE.Forms.DLL v1.0.2268.0,

MSCORLIB.DLL v1.0.2268.0,regflush.exe 

System.DATa.Common.DLL 1.0.2268.0,ICSharpCode.SharpZipLib.DLL 

0.85.1.271,New Text Document.txt 0.85.1.271,SharpZipLib_0855_Bin[1].zip 

0.85.1.271,cabwiz.ddf,Cabwiz.exe 3.1.0.9386,CFResGen.exe 1.0.4128.0,

Makecab.exe

3

rd Party Source Code (COTS) and 

Version

None

3

rd Party DLLs, Drivers, etc. and Version(s) None

Additional file(s) loaded and version(s) None

Record the disk image software version 

being used

Norton GHOST V:11.0

Record the filename of the build 

environment file signature (5.6.1.3) – 

Used the AutoMark PostCots image as Build environment

ESS_AutoMark_PostCots_05222009.GHO

Record the filename of the build 

environment disk image –

Used the AutoMark PostCots image as Build environment

ESS_AutoMark_PostCots_05222009.hash.txt

Verify (by signature) the build environment 

file signature (5.6.1.3)

OK

Loading Source Code (5.6.2)

Record the file signature of the source 

code (5.6.2.1)

see table of source code, above

Verify (by signature) that each file 

signature of the source code loaded 

matches as documented above (5.6.2.1)

OK

Method of Build Witness Trusted Build

Record the combined source code and 

pre-build environment file signature 

(5.6.2.2)

ESS_AutoMarkPEB_PreBuild_06092009.hash.txt

Record the combined source code and 

pre-build environment disk image (5.6.2.3)

ESS_AutoMarkPEB_PreBuild_06092009.GHO

Record the Final Build Version – Unique 

Identifier

PrinterEngineBoard1.7(PEB)

Certification Application Number (if 

applicable)

ESS0701

Document during the Build Witness:

Date / Time Build Initiated 05/21/2009 1.35pm

Compiler and Version See build tools and versions

Application Name AutoMark PrinterEngineBoard

Application Version Order PrinterEngineBoard1.7(PEB)

Obtain Names and Signatures of all 

persons present during build (record 

below)

Sridevi Jakileti

Rhonda Fox

Issue(s) and Resolution(s) No issues

Document at Completion of the Build 

Witness:

Record the disk image of the final build 

(5.7.3)

ES&S AutoMarkPEB TB 06092009 Hardware CD

Record file signature of the final build 

(5.6.3.1)

ESS_AutoMarkPEB_TBInstall06092009.hash.txt

Record the type of unalterable storage 

media being used for installation disk(s) 

(i.e., CD) – (5.6.3.2)

CD
VSTL Certification # pending

Page 226 of 247 (V)2009-30Jun-001(A)

Record each piece of media that is part of 

the installation disk (each must have a 

unique identifier) (5.6.3.2, 5.7.5)

DS200Ancillary install files are (pmb.hex, fw.iic) input to DS200 firmware 

build

Record the file signature of the installation 

disk(s). (5.6.3.3, 5.7.5) (include in below 

archive)

ESS_DS200Ancillary_TBInstall05282009.hash.txt

Record the type of unalterable storage 

media being used for pre-build and postbuild archive disk (i.e., CD) –

T9 hard Drive backed up on T8 (Pending Burning DVD or CD) final burn to 

unalterable media is pending the EAC Initial Decision

Record each piece of media that is part of 

the pre-build archive disk (each must have 

a unique identifier) (5.6.2.4, 5.7.2, 5.7.3)

T9 hard Drive backed up on T8 (Pending Burning DVD or CD) final burn to 

unalterable media is pending the EAC Initial Decision

Explanation of any significant differences 

observed

No differences

Build Witnesses): Hard copies of the witness signatures are on file at iBeta:

iBeta: Sridevi Jakileti 6/9/09

ES&S: Rhonda Fox 6/9/09

7.7.3 Witness of the Trusted Build AutoMARK VAT1.3.2906 &AIMS1.3.157

Application/ Component Version Language File Signature

AM Code 1.0.0.1 VB.net AutoMarkInputTB_05192009.hash.txt

Automark 1.3.2906 VB.net AutoMarkInputTB_05192009.hash.txt

Automark32 1.3.2906 File AutoMarkInputTB_05192009.hash.txt

AutomarkData 1.3.2906 C# AutoMarkInputTB_05192009.hash.txt

AutomarkDataHelperLibrary 1.0.121 C++ AutoMarkInputTB_05192009.hash.txt

AutomarkService 1.0.4 C++ AutoMarkInputTB_05192009.hash.txt

AutomarkStartup 1.0.7 C++ AutoMarkInputTB_05192009.hash.txt

DiagnosticLogger 1.0.105 C++ AutoMarkInputTB_05192009.hash.txt

GetMarks 1.4.9 C++ AutoMarkInputTB_05192009.hash.txt

Makebin 1.1.0.0 C AutoMarkInputTB_05192009.hash.txt

NonVolatileLibrary 1.0.116 C/C++ AutoMarkInputTB_05192009.hash.txt

OperationLogger 1.0.135 C++ AutoMarkInputTB_05192009.hash.txt

PrinterEngineBoard 65.1.3 C AutoMarkInputTB_05192009.hash.txt

ScanDriver 1.542 C++ AutoMarkInputTB_05192009.hash.txt

Scanner 1.48 C, Assembly AutoMarkInputTB_05192009.hash.txt

ScannerPrinterLibrary 1.8.37 C++ AutoMarkInputTB_05192009.hash.txt

SecurityLibrary 1.2.4 C/C++ AutoMarkInputTB_05192009.hash.txt

SwitchInterfaceBoard 1.8.37 C AutoMarkInputTB_05192009.hash.txt

UltrasonicSheetDetector 8.0.1 C, Assembly AutoMarkInputTB_05192009.hash.txt

AIMS

AIMSESS DLL 1.0.1.0 C# AutoMarkInputTB_05192009.hash.txt

AutomarkEncoder 1.0.105 C/C++ AutoMarkInputTB_05192009.hash.txt

MDB 1.3.157 VB AutoMarkInputTB_05192009.hash.txt

SQL Server 1.3.054 SQL AutoMarkInputTB_05192009.hash.txt

Document Prior to the Build Witness:

Vendor Name ES&S

Vendor Consultant(s) (5.6) Rhoda Fox

Witness Name (5.6) Sridevi Jakileti

Witness Title Trusted Builder

Vendor Build Document(s) used and version(s) AutoMARK VAT Software and Firmware Compilation 

Instructions.pdf Rev 16

Equipment Used DellOptipllexGX520

iBeta COTS used to clean the build environment disk 

(name and version) (5.6.1.1)

Active KillDisk for DOS V:4.1 Build 2380

iBeta COTS used to generate HASH file signatures 

(name and version)

Mares Hash Ver. 07.08.10.07.12

Construct the build environment (5.6.1.2)

Verify (by signature) that the build environment is Sridevi Jakileti
VSTL Certification # pending

Page 227 of 247 (V)2009-30Jun-001(A)

isolated and controlled by iBeta

Witness attests to verifying that the source code being 

built is the source code provided by iBeta

Rhoda Fox

Build Environment Operating System Windows XP Professional Version 2002 Service Pack 2 

Vendor CM Tool and version Listed in TDP document: Unity 3.2.0.0 Source Code Repository 

& Version

Build tool(s) and version(s) VAIO System Recovery DVD PCG-K23/PCG-K25/PCG-K27 

Series

Microsoft Embedded Visual C++ 4.0 Service Pack 4.

Microsoft Visual Studio .NET 2003 Service Pack 2

Keil Software μVision2, C 

compiler Version 2.40

Texas Instruments Code Composer Studio. Version 2.0

Cosmic Compiler V 4.1H

Borland C 4.02

Prog08sz Programmer for v 2.05

Atmel Flip v2.4.6

Atmel MCU ISP Software V1.0

Microsoft Access XP/2002

InstallShield 10.5

Microsoft Windows CE With Platform Builder Version 5.0

3

rd Party Libraries and Version ACCESSRT.MSI v2002,MSOHELP.exe 10.0.2609.0,Office1.cab 

v2002,

OSP.MSI v2002,OSP1.cab

v2002,FILES\SYSTEM\EXTRACT.exe v2002,

FILES\SYSTEM\MLANG.DAT 

v2002,FILES\SYSTEM\MLANG.DLL 

5.0.2919.6304,FILES\SYSTEM\MSXML.DLL 5.0.2919.6303,

FILES\SYSTEM\T2EMBED.DLL,0.2.0.69,FILES\WINDOWS\HEL

P\OSP.HLP v2002,IE5\EN\ACTSETUP.cab 

v2002,IE5\EN\ADVAUTH.cab v2002,

IE5\EN\AOLSUPP.cab v2002,IE5\EN\AXA.cab 

v2002,IE5\EN\AXA2.cab v2002,IE5\EN\AXA3.cab 

v2002,IE5\EN\BRANDING.cab v2002,

IE5\EN\DCOM95.exe 4.71.1015.0,IE5\EN\DXDDEX.cab v2002, 

IE5\EN\DXMINI.cab v2002,IE5\EN\FONTCORE.cab v2002,

IE5\EN\FONTSUP.cab v2002,IE5\EN\FPESETUP.cab v2002,

IE5\EN\GSETUP95.cab v2002,IE5\EN\GSETUPNT.cab v2002,

IE5\EN\HELPCONT.cab v2002,IE5\EN\HHUPD.cab v2002,

IE5\EN\ICW.cab v2002,IE5\EN\ICWCON.cab v2002,

IE5\EN\IE4MFC40.cab v2002,IE5\EN\IE4SHL95.cab v2002,

IE5\EN\IE4SHLNT.cab v2002,IE5\EN\IE5COMP.exe 

5.0.2919.6307

IE5\EN\IE5SETUP.exe 5.0.2919.6307,IE5\EN\IECIF.cab v2002,

IE5\EN\IEDATA.cab v2002,IE5\EN\IEDATAJA.cab v2002,

IE5\EN\IELPKAD.cab v2002,IE5\EN\IELPKAR.cab v2002,

IE5\EN\IELPKIW.cab v2002,IE5\EN\IELPKJA.cab v2002,

IE5\EN\IELPKKO.cab. v2002,IE5\EN\IELPKPE.cab v2002,

IE5\EN\IELPKTH.cab v2002,IE5\EN\IELPKVI.cab v2002,

IE5\EN\IELPKZHC.cab v2002,IE5\EN\IELPKZHT.cab v2002,

IE5\EN\IENT_S1.cab v2002,IE5\EN\IENT_S2.cab v2002,

IE5\EN\IENT_S3.cab v2002,IE5\EN\IENT_S4.cab v2002,

IE5\EN\IENT_S5.cab v2002,IE5\EN\IESETUP.INI v2002,

IE5\EN\IE_EXTRA.cab v2002,IE5\EN\IE_S1.cab v2002,

IE5\EN\IE_S2.cab v2002,IE5\EN\IE_S3.cab v2002,

IE5\EN\IE_S4.cab v2002,IE5\EN\JAAIME.cab v2002,

IE5\EN\KOAIME.cab v2002,IE5\EN\MAILNEWS.cab v2002,

IE5\EN\MDAC_IE5.cab v2002,IE5\EN\MOBILE95.cab v2002, 

IE5\EN\MOBILENT.cab v2002,IE5\EN\MPCDCS.cab v2002,

IE5\EN\MPLAYER2.cab v2002,IE5\EN\MSN_AUTH.cab v2002,

IE5\EN\NM30.cab v2002,IE5\EN\OAINST.exe 4.71.1015.0

IE5\EN\SCAIME.cab v2002,IE5\EN\SETUPNT.cab v2002,

IE5\EN\SETUPW95.cab v2002,IE5\EN\SWDIR.cab v2002,

IE5\EN\SWFLASH.cab v2002,IE5\EN\TCAIME.cab v2002,
VSTL Certification # pending

Page 228 of 247 (V)2009-30Jun-001(A)

IE5\EN\TS95.cab v2002,IE5\EN\TSNT.cab v2002,

IE5\EN\USP10.cab v2002,IE5\EN\VBSCRIPT.cab v2002,

IE5\EN\VGX.cab v2002,IE5\EN\VMX86_01.cab v2002,

IE5\EN\VMX86_02.cab v2002,IE5\EN\VRML2C.exe 4.71.1015.0

IE5\EN\WAB.cab v2002,IE5\EN\WEBFLDRS.cab v2002,

IE5\EN\WPIE5X86.cab v2002,chs.syn 6.1.0.0

chsrom.DLL 6.1.0.0,eci.DLL 6.1.0.0,enu.syn 6.1.0.0

esm.syn 6.1.0.0,jpn.syn 6.1.0.0,jpnrom.DLL 6.1.0.0

kor.syn 6.1.0.0,korrom.DLL 

6.1.0.0,ARIALUNI.TTF,BATANG.TTF

MSMINCHO.TTF,PMINGLIU.TTF,sqlxml.MSI c3.0

xblkld3.DLL 3.30.3457.0,Helper.exe,SqlRun.cab,,SqlRun01.MSI

1033dotnetfx.exe 1.1.4322.573

1033,dotnetfxSp1.exe 1.0.0.0,dotnetfxSp1.exe 1.0.871.2738

chs.syn v6.1.0.0,chsrom.DLL v6.1.0.0,eci.DLL v6.1.0.0,

enu.syn v6.1.0.0,esm.syn v6.1.0.0,jpn.syn v6.1.0.0,jpnrom.DLL 

v6.1.0.0,

kor.syn v6.1.0.0,korrom.DLL v6.1.0.0,chs.syn 

v6.1.0.0,chsrom.DLL v6.1.0.0,

eci.DLL v6.1.0.0,enu.syn v6.1.0.0,esm.syn 

v6.1.0.0,FTD2XX.DLL v6.1.0.0,

ftd2xx.inf v6.1.0.0,ftdi_d2xx.DLL v6.1.0.0,jpn.syn 

v6.1.0.0,jpnrom.DLL v6.1.0.0,kor.syn v6.1.0.0,korrom.DLL 

v6.1.0.0,Microsoft.WindowsCE.Forms.DLL v1.0.2268.0,

MSCORLIB.DLL v1.0.2268.0,regflush.exe 

System.DATa.Common.DLL 

1.0.2268.0,ICSharpCode.SharpZipLib.DLL 0.85.1.271,New Text 

Document.txt 0.85.1.271,SharpZipLib_0855_Bin[1].zip 

0.85.1.271,cabwiz.ddf,Cabwiz.exe 3.1.0.9386,CFResGen.exe 

1.0.4128.0,

Makecab.exe, unzip.exe 5.52,

WinCEPB50-060430-2006M04-Armv4I.msi V5.0,WinCEPB50-

060831-2006M08-Armv4I.msi V5.0,WinCEPB50-060228-

2006M02-Armv4I.msi V5.0,WinCEPB50-060131-2006M01-

Armv4I.msi V5.0,WinCEPB50-060731-2006M07-Armv4I.msi 

V5.0,WinCEPB50-060630-2006M06-Armv4I.msi 

V5.0,WinCEPB50-060331-2006M03-Armv4I.msi V5.0,

WinCEPB50-060531-2006M05-Armv4I.msi V5.0,WinCEPB50-

041231-Product-Update-Rollup-Armv4I.msi V5.0,WinCEPB50-

051231-Product-Update-Rollup-Armv4I.msi V5.0,autoit-v3-

setup.exe 3.3.0.0,

ADS_XSCALE_4_2_SDK.msi,

DATA.TAG v2.4.6,data1.cab v2.4.6,

lang.dat v2.4.6,layout.bin v2.4.6,

os.dat v2.4.6,setup.bmp v2.4.6,

SETUP.EXE v2.4.6,SETUP.INI v2.4.6,

setup.ins v2.4.6,setup.lid v2.4.6,

_INST32I.EX_ v2.4.6,_ISDEL.EXE v2.4.6,

_setup.dll v2.4.6,_sys1.cab v2.4.6,

_user1.cab v2.4.6,evc4sp4.exe v4

PL2303.CAT,SER2PL.INF,SER2PL.SYS

DISK.INI v7.09,c51util.dll v7.09,TX51TNY.LIB 

v7.09,CONF_TNY.A51 v7.09,DBG_TINY.DSW v7.09,

GENRTX.BAT v7.09,READ.ME2 v7.09,

RIGHT.A51 v7.09,RTX51TNY.A51 v7.09,

VERS.A51 v7.09

3

rd Party Source Code (COTS) and Version None

3

rd Party DLLs, Drivers, etc. and Version(s) None

Additional file(s) loaded and version(s) None

Record the disk image software version being used Norton GHOST V:11.0

Record the filename of the build environment file 

signature (5.6.1.3) – 

ESS_AutoMark_PostCots_05222009.hash.txt

Record the filename of the build environment disk 

image –

ESS_AutoMark_PostCots_05222009.GHO
VSTL Certification # pending

Page 229 of 247 (V)2009-30Jun-001(A)

Verify (by signature) the build environment file 

signature (5.6.1.3)

Sridevi Jakileti

Loading Source Code (5.6.2)

Record the file signature of the source code (5.6.2.1) see table of source code, above

Verify (by signature) that each file signature of the 

source code loaded matches as documented above 

(5.6.2.1)

Sridevi Jakileti

Method of Build Witness Trusted Build

Record the combined source code and pre-build 

environment file signature (5.6.2.2)

ESS_AutoMark_PreBuild_05272009.hash.txt

Record the combined source code and pre-build 

environment disk image (5.6.2.3)

ESS_AutoMark_PreBuild_05272009.hash.txt

Record the Final Build Version – Unique Identifier AIMS1.3.157

VAT1.3.2906

Certification Application Number (if applicable) ESS0701

Document during the Build Witness:

Date / Time Build Initiated 05/21/2009 1.35pm

Compiler and Version See build tools and versions

Application Name AIMS 

VAT

Application Version Order AIMS1.3.157

VAT1.3.2906

Obtain Names and Signatures of all persons present 

during build (record below)

OK

Issue(s) and Resolution(s) No Issues

Document at Completion of the Build Witness:

Record the disk image of the final build (5.7.3) ESS_AutoMark_PostBuild_05272009.GHO

Record file signature of the final build (5.6.3.1) ESS_AutoMark_PostBuild_05272009.hash.txt

Record the type of unalterable storage media being 

used for installation disk(s) (i.e., CD) – (5.6.3.2)

CD and CF cards on VAT

Record each piece of media that is part of the 

installation disk (each must have a unique identifier) 

(5.6.3.2, 5.7.5)

VAT full 1.3.2906(CF card for VAT machine)05/27/2009

VAT upgrade 1.3.2906(CF card for VAT machine) 05/27/2009

CD:ESS VAT Hardware 1.3.2906 TB05/27/2009

CD:ES&S AIMS & AIMS preview1.3.157 and 1.3.2906 TB 

05/27/2009

AIMS_TBINSTALLS05212009.hash.txt

Record the file signature of the installation disk(s). 

(5.6.3.3, 5.7.5) (include in below archive)

ESS_AutoMark_PostBuild_05272009.GHO

Record the type of unalterable storage media being 

used for pre-build and post-build archive disk (i.e., CD) 

–

T9 hard Drive backed up on T8 (Pending Burning DVD or CD)

final burn to unalterable media is pending the EAC Initial 

Decision

Record each piece of media that is part of the pre-build 

and post-build archive disks (each must have a unique 

identifier) (5.6.2.4, 5.7.2, 5.7.3)

T9 hard Drive backed up on T8 (Pending Burning DVD or CD)

final burn to unalterable media is pending the EAC Initial 

Decision

Explanation of any significant differences observed According to the documentation stating CD labeled with ADA 

Applications and Files(Need to change the documentation), but 

we used in the Build was Trusted COTS 1 CD

From Trusted COTS1 CD copied the unzip.exe to C:\. This is not 

documented in a right place.

Build Witnesses): Hard copies of the witness signatures are on file at iBeta:

iBeta: Sridevi Jakileti 5/27/2009

ES&S: Rhoda Fox 5/27/2009

7.7.4 Witness of the Trusted Build DS200 Ancillary Devices

(PowermanagementMsp430 1.2.0.1 &Scanner C8051 2.11.0.1)

Application/ Component Version Language File Signature

Power Management_MSP430 1.2.0.1a C DS200AncillaryInputTB_05282009.hash.txt

Scanner_C8051 2.11.0.1a C DS200AncillaryInputTB_05282009.hash.txt
VSTL Certification # pending

Page 230 of 247 (V)2009-30Jun-001(A)

Document Prior to the Build Witness:

Vendor Name ES&S

Vendor Consultant(s) (5.6) Dave Herrera

Witness Name (5.6) Sridevi Jakileti

Witness Title Trusted Builder

Vendor Build Document(s) used and version(s) WinXPwithSP3-DellOptiplexGX520_INST_2009.03.31.pdf 

IAREmbeddedWorkbench3.40_INST_2009.04.20.pdf 

KeiluVision3DevelopmentTools4.2007_INST_2009.04.20.pdf 

CypressEZ

USBReferenceDesignKit2.31_INST_2009.04.20.pdf 

DS200AncillaryDevices_BECI_3.2.0.0_2009.05.05.pdf

Equipment Used DellOptipllexGX520

iBeta COTS used to clean the build environment disk 

(name and version) (5.6.1.1)

Active KillDisk for DOS V:4.1 Build 2380

iBeta COTS used to generate HASH file signatures 

(name and version)

Mares Hash Ver. 07.08.10.07.12

Construct the build environment (5.6.1.2)

Verify (by signature) that the build environment is 

isolated and controlled by iBeta

Sridevi Jakilet

Witness attests to verifying that the source code being 

built is the source code provided by iBeta

Dave Herrera

Vendor CM Tool and version Listed in TDP document: Unity 3.2.0.0 Source Code Repository 

& Version

Build Environment Operating System Windows XP Professional Version 2002 Service Pack 3 

Build tool(s) and version(s) Keil μVision3 Development Tools

Cypress CY4611 EZ-USB FX2 Reference Design Kit 

IAR Embedded Workbench EW430

3

rd Party Libraries and Version None

3

rd Party Source Code (COTS) and Version None

3

rd Party DLLs, Drivers, etc. and Version(s) None

Additional file(s) loaded and version(s) Build scripts(unzip.exe, TB-3_CreateExecutables.bat, TB2_LoadSourceCode.bat, TB-0_CheckInputMedium.bat, 

ESSSourceFileList.txt, ESSScriptsFileList.txt, BuildScripts.ini, 

TB-2_LSC-2_ScannerBoard.bat, TB-2_LSC1_PowerManagementBoard.bat, TB-2_LSC0.2_MakeDirectories.bat, TB-2_LSC0.1_SetEnvironmentVariables.bat, TB-3_CE0.01_SetEnvironmentVariables.bat, TB-3_CE1.01_PowerManagementMsp430.bat, TB-3_CE1.02_ScannerC8051.bat, TB-0_CIM0_SetEnvironmentVariables.bat)

Record the disk image software version being used Notron GHOST V:11.0

Record the filename of the build environment file 

signature (5.6.1.3) – 

DS200Ancillary_PostCots_05282009.hash.txt

Record the filename of the build environment disk 

image –

DS200Ancillary_PostCots_05282009.GHO

Verify (by signature) the build environment file 

signature (5.6.1.3)

Ok

Loading Source Code (5.6.2)

Record the file signature of the source code (5.6.2.1) see table of source code, above

Verify (by signature) that each file signature of the 

source code loaded matches as documented above 

(5.6.2.1)

Ok

Method of Build Witness Trusted Build

Record the combined source code and pre-build 

environment file signature (5.6.2.2)

DS200Ancillary_PreBuild_05282009.hash.txt

Record the combined source code and pre-build 

environment disk image (5.6.2.3)

DS200Ancillary_PreBuild_05282009.GHO

Record the Final Build Version – Unique Identifier PowermanagementMsp430 1.2.0.1

Scanner C8051 2.11.0.1

Certification Application Number (if applicable) ESS0701

Document during the Build Witness:

Date / Time Build Initiated 5/28/2009 10.00am
VSTL Certification # pending

Page 231 of 247 (V)2009-30Jun-001(A)

Compiler and Version See Build tools and versions

Application Name DS200ancillary Devices (Power management and Scanner)

Application Version Order PowermanagementMsp430 1.2.0.1

Scanner C8051 2.11.0.1

Obtain Names and Signatures of all persons present 

during build (record below)

Sridevi Jakileti

Dave Herrera

Issue(s) and Resolution(s) No Issues

Document at Completion of the Build Witness:

Record the disk image of the final build (5.7.3) DS200Ancillary_PostBuild_05282009.hash.txt

Record file signature of the final build (5.6.3.1) DS200Ancillary_PostBuild_05282009.GHO

Record the type of unalterable storage media being 

used for installation disk(s) (i.e., CD) – (5.6.3.2)

CD

Record each piece of media that is part of the 

installation disk (each must have a unique identifier) 

(5.6.3.2, 5.7.5)

DS200Ancillary install files are (pmb.hex, fw.iic) input to DS200 

firmware build

Record the file signature of the installation disk(s). 

(5.6.3.3, 5.7.5) (include in below archive)

ESS_DS200Ancillary_TBInstall05282009.hash.txt

Record the type of unalterable storage media being 

used for pre-build and post-build archive disk (i.e., CD) 

–

T9 hard Drive backed up on T8 (Pending Burning DVD or CD)

final burn to unalterable media is pending the EAC Initial 

Decision

Record each piece of media that is part of the pre-build 

archive disk (each must have a unique identifier) 

(5.6.2.4, 5.7.2, 5.7.3)

T9 hard Drive backed up on T8 (Pending Burning DVD or CD)

final burn to unalterable media is pending the EAC Initial 

Decision

Explanation of any significant differences observed No differences

Build Witnesses): Hard copies of the witness signatures are on file at iBeta:

Sridevi Jakileti 5/29/2009

ES&S Dave Herrera 5/29/2009

7.7.5 Witness of the Trusted Build DS200 v.1.3.10.0

Application/ Component Version Language File Signature

DS200 1.3.10.0a C/C++ DS200FirmwareInputTB_05302009.hash.txt

Document Prior to the Build Witness:

Vendor Name ES&S

Vendor Consultant(s) (5.6) Dave Herrera

Witness Name (5.6) Sridevi Jakileti

Witness Title Trusted Builder

Vendor Build Document(s) used and version(s) DS200Firmware_BECI_v1.3.10.0_2009.06.25.pdf

DS200Firmware_PreBuildTaskList_3.2.0.0_2009.06.25.pdf

DS200Firmware_BECI_3.2.0.0_2009.06.25.pdf

Equipment Used Dell Precision 670

iBeta COTS used to clean the build environment disk 

(name and version) (5.6.1.1)

Restored from the DS200 TOS 

PostBuild(DS200TOS_PostBuild_05302009.GHO) as a Build 

environment for DS200 Firmware

iBeta COTS used to generate HASH file signatures 

(name and version)

Mares Hash Ver. 07.08.10.07.12

Construct the build environment (5.6.1.2)

Verify (by signature) that the build environment is 

isolated and controlled by iBeta

Sridevi Jakileti

Witness attests to verifying that the source code being 

built is the source code provided by iBeta

Dave Herrera

Vendor CM Tool and version Listed in TDP document: Unity 3.2.0.0 Source Code Repository 

& Version

Build tool(s) and version(s) Linux From Scratch 6.25

Build Environment Operating System Linux operating system 6.25

3

rd Party Libraries and Version See DS200TOS Trusted Build Doc 3rd Party Libraries and 

versions

3

rd Party Source Code (COTS) and Version None
VSTL Certification # pending

Page 232 of 247 (V)2009-30Jun-001(A)

3

rd Party DLLs, Drivers, etc. and Version(s) See in DS200TOS 3rd party Drivers, etc. and versions

Additional file(s) loaded and version(s) BuildScripts(BuildFirmware1.sh, BuildFirmware2.sh, 

BuildFirmware3.sh, VersionNumbers.txt), PMB.hex and fw.iic 

these are coming from DS200AncillaryDevices trusted Build

Record the disk image software version being used Notron GHOST V:11.0

Record the filename of the build environment file 

signature (5.6.1.3) – 

DS200TOS_PostBuild_05302009.hashl

Record the filename of the build environment disk 

image –

Restored from the DS200 TOS 

PostBuild(DS200TOS_PostBuild_05302009.GHO) as a Build 

environment for DS200 Firmware

Verify (by signature) the build environment file 

signature (5.6.1.3)

Sridevi Jakileti

Loading Source Code (5.6.2)

Record the file signature of the source code (5.6.2.1) see table of source code, above

Verify (by signature) that each file signature of the 

source code loaded matches as documented above 

(5.6.2.1)

Sridevi Jakileti

Method of Build Witness TrustedBuild

Record the combined source code and pre-build 

environment file signature (5.6.2.2)

DS200Firmware_PreBuild_06252009.hashl

Record the combined source code and pre-build 

environment disk image (5.6.2.3)

DS200Firmware_PreBuild_06252009.GHO

Record the Final Build Version – Unique Identifier Ds200_1.3.10.0

Certification Application Number (if applicable) Ds200_1.3.10.0

Document during the Build Witness:

Date / Time Build Initiated 6/25/2009 7.40am

Compiler and Version GCC-4.0.3 (GNU Compiler Collection). This compiler is part of 

the LFS (Linux From Scratch) 6.2-5 Live CD

Application Name DS200

Application Version Order Ds2001.3.9.0

Obtain Names and Signatures of all persons present 

during build (record below)

Sridevi Jakileti

Dave Herrera

Issue(s) and Resolution(s) No Issues

Document at Completion of the Build Witness

Record the disk image of the final build (5.7.3) DS200Firmware_PostBuild_06252009.hashl

Record file signature of the final build (5.6.3.1) DS200Firmware_PostBuild_06252009.GHO

Record the type of unalterable storage media being 

used for installation disk(s) (i.e., CD) – (5.6.3.2)

CF card on DS200 machine

Record each piece of media that is part of the 

installation disk (each must have a unique identifier) 

(5.6.3.2, 5.7.5)

CF card on DS200 for full install(DS200 1.3.10.0 full)

Record the file signature of the installation disk(s). 

(5.6.3.3, 5.7.5) (include in below archive)

USB on DS200 for upgrade(DS200 1.3.10.0 upgrade)

Record the type of unalterable storage media being 

used for pre-build and post-build archive disk (i.e., CD) 

–

T8 hard Drive (Pending Burning DVD or CD) final burn to 

unalterable media is pending the EAC Initial Decision

Record each piece of media that is part of the pre-build 

archive disk (each must have a unique identifier) 

(5.6.2.4, 5.7.2, 5.7.3)

T8 hard Drive (Pending Burning DVD or CD) final burn to 

unalterable media is pending the EAC Initial Decision

Explanation of any significant differences observed No differences

Build Witnesses): Hard copies of the witness signatures are on file at iBeta:

Sridevi Jakileti 6/25/2009

ES&S Dave Herrera 6/25/2009

7.7.6 Witness of the Trusted Build DS200 TOS v.1.0.1.0

Application/ Component Version Language File Signature

BLFS_1.0.1.0_BuildPkg.iso 1.0.1.0 NA- not DS200TOSInputTB_05292009.hash.txt
VSTL Certification # pending

Page 233 of 247 (V)2009-30Jun-001(A)

Application/ Component Version Language File Signature

source code

Document Prior to the Build Witness:

Vendor Name ES&S

Vendor Consultant(s) (5.6) Dave Herrera

Witness Name (5.6) Sridevi Jakileti

Witness Title Trusted Builder

Vendor Build Document(s) used and version(s) LFS_BECI_x86-6.2-5 v2.0

BLFS_BECI_v1.0.0.0 v2.0

Equipment Used Dell Precision 670 

iBeta COTS used to clean the build environment disk 

(name and version) (5.6.1.1)

Active KillDisk for DOS V:4.1 Build 2380

iBeta COTS used to generate HASH file signatures 

(name and version)

Mares Hash Ver. 07.08.10.07.12

Construct the build environment (5.6.1.2)

Verify (by signature) that the build environment is 

isolated and controlled by iBeta

Sridevi Jakileti

Witness attests to verifying that the source code being 

built is the source code provided by iBeta

Dave Herrera

Vendor CM Tool and version Listed in TDP document: Unity 3.2.0.0 Source Code Repository 

& Version

Build Environment Operating System Linux operating system 6.25

Build tool(s) and version(s) Linux From Scratch 6.25

3

rd Party Libraries and Version None

3

rd Party Source Code (COTS) and Version None

3

rd Party DLLs, Drivers, etc. and Version(s) cryptocme-2.0-rhel30.tar.gz 2.0-rhel30, X11R6.9.0-src7.tar.gz 

R6-6.9.0 ,

X11R6.9.0-src6.tar.gz R6-6.9.0 ,gtkmm-2.8.12.tar.bz2 2.8.12 ,

libsigc2.0.18.tar.bz2 2.0.18 ,pango-1.12.3.tar.bz2 1.12.3,

jpegsrc.v6b.tar.gz v6b ,libpng-1.2.12.tar.bz2 1.2.12 ,

giflib-4.1.4.tar.bz2 4.1.4 ,jre-6u3-linux-i586.bin 6u3 ,

gtk+2.8.20.tar.bz2 2.8.20, tiff-3.8.2.tar.gz 3.8.2, 

lcms-1.14.tar.gz 1.14 ,X11R6.9.0-src4.tar.gz R6-6.9.0, 

atk-1.11.4.tar.bz2 1.11.4, X11R6.9.0-src3.tar.gz R6-6.9.0, 

glibmm-2.12.10.tar.bz2 2.12.10,libmng-1.0.9.tar.gz 1.0.9 ,

bootsplash-3.2.tar.bz2 3.2,X11R6.9.0-src5.tar.gz R6-6.9.0 ,

freetype-2.1.10.tar.bz2 2.1.10,glib-2.10.3.tar.bz2 2.10.3,

X11R6.9.0-src1.tar.gz R6-6.9.0 ,fontconfig-2.3.2.tar.gz 2.3.2,

lcms-1.14-gcc343-1.patch 1.14,bootsplash-3.2_makefile.patch 

3.2,

X11R6.9.0-src2.tar.gz R6-6.9.0 ,apache-ant-1.7.0-bin.zip 1.7.0,

jdk-6u3-linux-i586.bin 6u3 ,cairo-1.2.4.tar.gz 1.2.4 ,

xorg-6.9.0-luit_race-1.patch R6-6.9.0 ,libusb-0.1.12.tar.gz 0.1.12, 

gpm-1.20.1-segfault-1.patch 1.20.1 ,openssl-0.9.8dfix_manpages-1.patch 0.9.8d,

cvs-1.11.22.tar.bz2 1.11.22 ,boost_1_34_1.tar.bz2 1.34.1 ,

e2fsprogs-1.38.tar.bz2 1.38 ,gpm-1.20.1-silent-1.patch 1.20.1 ,

unzip552.tar.gz 552 ,busybox-1.2.1.tar.bz2 1.2.1 ,

zip232.tar.gz 232 ,beecrypt-4.1.2.tar.gz 4.1.2,

aumix-2.8.tar.bz2 2.8 ,ctags-5.6.tar.gz 5.6 ,

expat-2.0.0.tar.gz 2.0.0 ,pkg-config-0.20.tar.gz 0.2 ,

blfs-bootscripts-20060910.tar.bz2 20060910,openssh4.5p1.tar.gz 4.5p1,

cvs-1.11.22-zlib-1.patch 1.11.22,usbutils-0.72.tar.gz 0.72 ,

dosfstools-2.11.src.tar.gz 2.11,linux-libc-headers-2.6.12.0.tar.bz2 

2.6.12.0,

gpm-1.20.1.tar.bz2 1.20.1,openssl-0.9.8d.tar.gz 0.9.8d ,

linux-2.6.16.27-utf8_input-1.patch 2.6.16.27 ,udev-096.tar.bz2 

096, 

bootsplash-3.1.6-2.6.15.diff 3.1.6-2.6.15,linux2.6.16.27.tar.bz22.6.16.27, lfslivecd-x86-6.2-5.iso 6.25,

Additional file(s) loaded and version(s) LFS1.sh,BLFS1.sh,BLFS2.sh,BLFS3.sh,
VSTL Certification # pending

Page 234 of 247 (V)2009-30Jun-001(A)

TOS1.sh,VersionNumbers.txt, BLFS_1.0.1.0_BuildPkg.iso

Record the disk image software version being used Notron GHOST V:11.0

Record the filename of the build environment file 

signature (5.6.1.3) – 

DS200TOS_PostCots_05292009.hashl

Record the filename of the build environment disk 

image –

DS200TOS_PostCots_05292009.GHO

Verify (by signature) the build environment file 

signature (5.6.1.3)

Sridevi Jakileti

Loading Source Code (5.6.2)

Record the file signature of the source code (5.6.2.1) see table of source code, above

Verify (by signature) that each file signature of the 

source code loaded matches as documented above 

(5.6.2.1)

Sridevi Jakileti

Method of Build Witness Trusted Build

Record the combined source code and pre-build 

environment file signature (5.6.2.2)

DS200TOS_PreBuild_05292009.hashl

Record the combined source code and pre-build 

environment disk image (5.6.2.3)

DS200TOS_PreBuild_05292009.GHO

Record the Final Build Version – Unique Identifier CfCard_ds200_n1.0.1.0.img

Certification Application Number (if applicable) ESS0701

Document during the Build Witness:

Date / Time Build Initiated 5/29/2009 8.20am

Compiler and Version Listed in TDP document: Unity 3.2.0.0 Source Code Repository 

& Version

Application Name CfCard_ds200_n1.0.1.0 (DS200 Target operating system)

Application Version Order CfCard_ds200_n1.0.1.0.img

Obtain Names and Signatures of all persons present 

during build (record below)

Sridevi Jakileti

Dave Herrera

Issue(s) and Resolution(s) None

Document at Completion of the Build Witness:

Record the disk image of the final build (5.7.3) DS200TOS_PostBuild_05292009.hashl

Record file signature of the final build (5.6.3.1) DS200TOS_PostBuild_05292009.GHO

Record the type of unalterable storage media being 

used for installation disk(s) (i.e., CD) – (5.6.3.2)

CF card on DS200 machine

Record each piece of media that is part of the 

installation disk (each must have a unique identifier) 

(5.6.3.2, 5.7.5)

See DS200

Record the file signature of the installation disk(s). 

(5.6.3.3, 5.7.5) (include in below archive)

ESS_DS200TOS_TBInstall05302009.hash.txt

Record the type of unalterable storage media being 

used for pre-build and post-build archive disk (i.e., CD) 

–

T9 hard Drive backed up on T8 (Pending Burning DVD or CD)

final burn to unalterable media is pending the EAC Initial 

Decision

Record each piece of media that is part of the pre-build 

archive disk (each must have a unique identifier) 

(5.6.2.4, 5.7.2, 5.7.3)

T9 hard Drive backed up on T8 (Pending Burning DVD or CD)

final burn to unalterable media is pending the EAC Initial 

Decision

Explanation of any significant differences observed No differences

Build Witnesses): Hard copies of the witness signatures are on file at iBeta:

Sridevi Jakileti 5/30/2009

ES&S Dave Herrera 5/30/2009

7.7.7 Witness of the Trusted Build Model 650 v.2.2.2.0

Application/ Component Version Language File Signature

M650 2.2.2.0.1 C M650InputTB_05192009.hash.txt

Document Prior to the Build Witness:

Vendor Name ES&S

Vendor Consultant(s) (5.6) Dave Herrera

Witness Name (5.6) Sridevi Jakileti

Witness Title Trusted Build

Vendor Build Document(s) used and version(s) Model650_BECI_3.2.0.0_2009.05.12.pdf

QNX4.25_INST_2009.05.12.doc 
VSTL Certification # pending

Page 235 of 247 (V)2009-30Jun-001(A)

Equipment Used Optiplex G110

iBeta COTS used to clean the build environment disk 

(name and version) (5.6.1.1)

Active KillDisk for DOS V:4.1 Build 2380

iBeta COTS used to generate HASH file signatures 

(name and version)

Mares Hash Ver. 07.08.10.07.12

Construct the build environment (5.6.1.2)

Verify (by signature) that the build environment is 

isolated and controlled by iBeta

Sridevi Jakileti

Witness attests to verifying that the source code being 

built is the source code provided by iBeta

Dave Herrera

Vendor CM Tool and version Listed in TDP document: Unity 3.2.0.0 Source Code Repository 

& Version

Build Environment Operating System QNX operating system4.25

Build tool(s) and version(s) QNX Operating System Runtime 4.25

Photon Runtime 1.14

TCP/IP runtime 4.25

Voyager RunTime 2.00

WATCOM C Compiler

WATCOM C++ Compiler

TCP/IP SDK

3

rd Party Libraries and Version None

3

rd Party Source Code (COTS) and Version None

3

rd Party DLLs, Drivers, etc. and Version(s) None

Additional file(s) loaded and version(s) QNX 4.25 Product Suite (May 2001)

Patch G for QNX 4.25(qnx-4.25-01G.tarx)

Patch D for the TCP/IP Runtime(tcprt-4.25-01D.tarx)

Patch D for the TCP/IP Runtime Documentation(tcprt-4.25-

02D.tarx)

Patch C for the TCP/IP Development Kit(tcptk-4.25-01C.tarx)

Patch D for the TCP/IP Development Kit Documentation(tcptk4.25-02C.tarx)

Record the disk image software version being used Norton GHOST V:11.0

Record the filename of the build environment file 

signature (5.6.1.3) – 

M650-PostCots_05192009.hashl

Record the filename of the build environment disk 

image –

M650_PostCOTS_05192009.GHO

Verify (by signature) the build environment file 

signature (5.6.1.3)

Sridevi Jakileti

Loading Source Code (5.6.2)

Record the file signature of the source code (5.6.2.1) see table of source code, above

Verify (by signature) that each file signature of the 

source code loaded matches as documented above 

(5.6.2.1)

Sridevi Jakileti

Method of Build Witness Trusted Build

Record the combined source code and pre-build

environment file signature (5.6.2.2)

M650_PreBuild_05202009.hashl

Record the combined source code and pre-build 

environment disk image (5.6.2.3)

M650_PreBuild_05202009.GHO

Record the Final Build Version – Unique Identifier M650 2.2.2.0

Certification Application Number (if applicable) ESS0701

Document during the Build Witness:

Date / Time Build Initiated 05/19/2009 8.20am

Compiler and Version See Build tools and versions

Application Name M650

Application Version Order M650 2.2.2.0

Obtain Names and Signatures of all persons present 

during build (record below)

Sridevi Jakileti

Dave Herrera

Issue(s) and Resolution(s) M650 system not able to recognize the ZIP disk(patches). 

Formatted new zip disk, copied the patches to it, it worked fine 

Document at Completion of the Build Witness:

Record the disk image of the final build (5.7.3) M650_PostBuild_05222009.GHO

Record file signature of the final build (5.6.3.1) M650_PostBuild_05222009.hahsl
VSTL Certification # pending

Page 236 of 247 (V)2009-30Jun-001(A)

Record the type of unalterable storage media being 

used for installation disk(s) (i.e., CD) – (5.6.3.2)

Zip disk on M650 machine

Record each piece of media that is part of the 

installation disk (each must have a unique identifier) 

(5.6.3.2, 5.7.5)

M650 2.2.2.0 Install TB5/22/09-->Zip disk

Record the file signature of the installation disk(s). 

(5.6.3.3, 5.7.5) (include in below archive)

ESS_M650_TBInstall05222009.hash.txt

Record the type of unalterable storage media being 

used for pre-build and post-build archive disk (i.e., CD) 

–

T9 hard drive & backup on T8 final burn to unalterable media is 

pending the EAC Initial Decision

Record each piece of media that is part of the pre-build 

archive disk (each must have a unique identifier) 

(5.6.2.4, 5.7.2, 5.7.3)

T9 hard drive & backup on T8 final burn to unalterable media is 

pending the EAC Initial Decision

Explanation of any significant differences observed Section 3.3 in QNX4.25_INST.pdf , 4.5-->modify the b drive 

setting

Fatfsys b=/dev/fd1t11 a=/dev/fd0 & (in the build instead of fdt11 

used only fd1) and not inserted these lines section4.6.(This 

change was made to resolve the troubleshooting with zip disk)

Build Witnesses): Hard copies of the witness signatures are on file at iBeta:

Sridevi Jakileti 5/29/2009

ES&S Dave Herrera 5/29/2009

7.7.8 Witness of the Trusted Build Unity 3.2.0.0

(Audit Manager 7.5.2.0, EDM 7.8.1.0, ERM7.5.4.0, ESSIM7.7.1.0, & HPM 5.7.1.0)

Application/ Component Version Language File Signature

AuditManager 7.5.2.0a VB UnityInputTB_05192009.hash.txt

CB_650 1.2.1.0a C UnityInputTB_05192009.hash.txt

CB_Eagl 1.3.2.0a C++ UnityInputTB_05192009.hash.txt

CB_M100 1.4.1.0a C++ UnityInputTB_05192009.hash.txt

Cb_PEB 1.0.2.0a C++ UnityInputTB_05192009.hash.txt

CB_Rand 1.1.1.0a C++ UnityInputTB_05192009.hash.txt

CF_Utility 9.2.1.0a VB UnityInputTB_05192009.hash.txt

CrcDLL 1.4.2.0a C++ UnityInputTB_05192009.hash.txt

EDM 7.8.1.0a C++ UnityInputTB_05192009.hash.txt

ElectionPackager 1.1.0.0a C++ UnityInputTB_05192009.hash.txt

ERM 7.5.4.0b Cobol UnityInputTB_05192009.hash.txt

ERMDLL 1.1.0.0a C++ UnityInputTB_05192009.hash.txt

ESSCRYPT1 1.1.1.0a C++ UnityInputTB_05192009.hash.txt

ESSCrypt 1.9.3.0a C++ UnityInputTB_05192009.hash.txt

ESSDecpt 1.9.3.0b C++ UnityInputTB_05192009.hash.txt

ESSEagl 1.3.2.0a C++ UnityInputTB_05192009.hash.txt

ESSIM 7.7.1.0a C++ UnityInputTB_05192009.hash.txt

ESSM100 1.7.2.0b C/C++ UnityInputTB_05192009.hash.txt

EssPCMIO 1.1.1.0a C++ UnityInputTB_05192009.hash.txt

ESSPEB 1.0.2.0a C++ UnityInputTB_05192009.hash.txt

ESSXML 2.1.0.0a C++ UnityInputTB_05192009.hash.txt

ESSZIP 2.1.0.0a C++ UnityInputTB_05192009.hash.txt

Events 9.2.1.0a C/C++ UnityInputTB_05192009.hash.txt

ExitWin 1.1.1.0a VB UnityInputTB_05192009.hash.txt

GetAuditData 9.2.1.0a VB UnityInputTB_05192009.hash.txt

HPM 5.7.1.0b Cobol UnityInputTB_05192009.hash.txt

HPMDLL 1.1.0.0a C++ UnityInputTB_05192009.hash.txt

Images 9.2.1.0a C/C++ UnityInputTB_05192009.hash.txt

MakeIBin 9.2.1.0a C++ UnityInputTB_05192009.hash.txt

MPRBoot 2.6.2.0a Assembler UnityInputTB_05192009.hash.txt

MYDLL 1.1.0.1a C UnityInputTB_05192009.hash.txt

PBMtoBMP 1.1.0.1a C++ UnityInputTB_05192009.hash.txt

PCCARD30 3.5.1.0a C++ UnityInputTB_05192009.hash.txt
VSTL Certification # pending

Page 237 of 247 (V)2009-30Jun-001(A)

Application/ Component Version Language File Signature

RegUtil 1.1.0.1a C++ UnityInputTB_05192009.hash.txt

Shell 1.1.1.0a C++ UnityInputTB_05192009.hash.txt

ShellSetup 1.1.1.0a C++ UnityInputTB_05192009.hash.txt

Undrvote 9.2.2.0a C++ UnityInputTB_05192009.hash.txt

VIODialog 9.2.2.0a C/C++ UnityInputTB_05192009.hash.txt

VioWin 9.2.1.0a C++ UnityInputTB_05192009.hash.txt

Win650 2.2.2.0.1 C UnityInputTB_05192009.hash.txt

Document Prior to the Build Witness:

Vendor Name ES&S

Vendor Consultant(s) (5.6) Dave Herrera

Witness Name (5.6) Sridevi jakileti

Witness Title Trusted Builder

Vendor Build Document(s) used and version(s) Unity_BECI_3.2.0.0_2009.05.20

WinXP-CorsairwithVGA_INST_2009.04.22.pdf 

Visual Studio_6.0_EntEdwithSP5_INST_2009.04.30.pdf 

vbAdvance3.1_INST_2009.04.30.pdf 

VisualStudio2005ProEd_INST_2009.04.21.pdf 

RMCOBOLDeveloper11.01_INST_2009.04.29.pdf 

AvocetADXZ180_INST_2009.04.30.pdf 

Codebase6.5Release3_INST_2009.04.21.pdf 

CrystalReports9_INST_2009.04.29.pdf 

PCCardSDKv2.20_INST_2009.04.30.pdf 

Xerces2.7.0_INST_2009.04.25.pdf 

InstallShieldPro7.01_INST_2009.03.23.doc 

InstallShieldExpress2.1_INST_2008.11.30.pdf 

Equipment Used Corsair Orbit PC

iBeta COTS used to clean the build environment disk 

(name and version) (5.6.1.1)

Active KillDisk for DOS V:4.1 Build 2380

iBeta COTS used to clean the build environment disk 

(name and version)

Mares Hash Ver. 07.08.10.07.12

Construct the build environment (5.6.1.2)

Verify (by signature) that the build environment is 

isolated and controlled by iBeta

Sridevi Jakileti

Witness attests to verifying that the source code being 

built is the source code provided by iBeta

Dave Herrera

Vendor CM Tool and version Listed in TDP document: Unity 3.2.0.0 Source Code Repository 

& Version

Build Environment Operating System Windows XP with Service Pack 3

Build tool(s) and version(s) AuditManager: MicroVisual Studio 2005

ALL other unity software: Visual Studio 6.0 Enterprise with 

Service Pack 5

ERM and HPM: RM/COBOL v11.01 

3

rd Party Libraries and Version ESSZIP: zip32.lib; unzip32.lib, 

AuditManager: asycfilt.dll2.40.4275.1, 

Comcat.dll,comct332.ocx6.7.0.8988, 

comdlg32.ocx6.0.84.18,msado25.tlb2.5 2.60.6526.0, 

msadodc.ocx6.0.88.4,

msbind.dll6.0.88.62,mscdrun.dll6.0.88.4

mscomct2.ocx 6.0.88.4 ,mscomctl.ocx 6.0.88.62 

,msderun.dll6.0.88.4

msflxgrd.ocx6.0.84.18,msvbvm60.dll6.0.89.64

oleaut32.dll2.40.4275.1,olepro32.dll5.0.4275.1

stdole2.tlb2.40.4275.1,sysinfo.ocx6.0.81.69

vb6stkit.dll 6.0.84.50, comct332.ocx 

6.7.0.8988,comdlg32.ocx6.0.84.18,

msadodc.ocx6.0.88.4,mscomct2.ocx6.0.88.4,

mscomctl.ocx6.0.88.62,msflxgrd.ocx6.0.84.18,

sysinfo.ocx6.0.81.69,msbind.dll6.0.88.62,

mscdrun.dll6.0.88.4,msderun.dll6.0.88.4,
VSTL Certification # pending

Page 238 of 247 (V)2009-30Jun-001(A)

msrdo20.dll6.0.88.62,msstdfmt.dll6.0.88.4,

rdocurs.dll6.0.88.4,

ESSIM: mfc80.dll8.0.50727.42,

mfc80u.dll8.0.50727.42,mfcm80.dll8.0.50727.42,

mfcm80u.dll8.0.50727.42,msvcm80.dll8.0.50727.42,

msvcp80.dll8.0.50727.42,msvcr80.dll8.0.50727.42,

msvcr71.dll8.0.50727.42,msvcrt.dll6.0.8797.0,

roboex32.dll9.20.534.0,unzip32.dll5.5.2.0,

zip32.dll 2.3.1.0,microsoft.vc80.crt.manifest,

microsoft.vc80.mfc.manifest,

EDM :c4dll.dll1.0.0.1,

crdb_p2bxbse.dll9.2.1.106,crpe32.dll9.2.3.745,

crqe.dll9.2.1.605,crtslv.dll9.2.0.528,

crxf_pdf.dll9.2.1.567,crxf_rtf.dll9.2.0.566,

crxf_wordw.dll9.2.0.566,crxf_xls.dll9.2.1.662,

exportmodeller.dll9.2.1.559,inetwh32.dll7.0.133.0,

msvcr71.dll7.10.3052.4,roboex32.dll9.0.79.0,

u2ddisk.dll9.2.0.541,u2ftext.dll9.2.1.555,

ufmanager.dll9.0.0.1,xerces-c_2_7.dll2.7.0.0, 

ERM:ROBOEX32.DLL

5.0.0.7.022,wh2robo.dll13.10.606.6,asycfilt.dll2.40.4275.1,comc

at.dll4.71.1460.1, comct332.ocx6.7.0.8988,

comdlg32.ocx6.0.84.18,mscomct2.ocx6.0.88.4,mscomctl.ocx6.0

.88.62,MSVBVM50.DLL5.1.43.19,

msvbvm60.dll6.0.89.64,msvcp60.dll6.0.8168.0,

MSVCRTD.DLL6.0.8447.0,oleaut32.dll2.40.4275.1,olepro32.dll5

.0.4275.1,OpenSaveFile.ocx1.0.0.0,

RICHTX32.OCX6.0.88.4,stdole2.tlb2.40.4275.1,

sysinfo.ocx6.0.81.69,tabctl32.ocx6.0.90.43,WSC32.DLL0.0.0.0 

Linker Rev. 3, WSC32.lib, 

HPM: 

HtmlWH.dll7.0.131.0,ROBOEX32.DLL8.0.131.0,asycfilt.dll2.40.

4275.1,

comcat.dll4.71.1460.1,comct332.ocx6.7.0.8988,comdlg32.ocx6.

0.84.18,

MFC42D.DLL6.0.8447.0,mscomct2.ocx6.0.88.4,mscomctl.ocx6.

0.88.62,

msdxm.ocx6.4.9.1128,msvbvm60.dll6.0.89.64,msvcp60.dll6.0.8

168.0,

MSVCP60D.DLL6.0.8168.0,msvcrt.dll6.0.8797.0,MSVCRTD.DL

L6.0.8447.0,

oleaut32.dll2.40.4275.1,olepro32.dll5.0.4275.1,OpenSaveFile.o

cx1.0.0.0,

Quartz.dll6.4.2600.1221,RICHTX32.OCX6.0.88.4,stdole2.tlb2.4

0.4275.1,

3

rd Party Source Code (COTS) and Version Codebase 6.5, Release 3

Modified COTS

Codebase file - d4all.h

3

rd Party DLLs, Drivers, etc. and Version(s) Windows XP with Service Pack 3,MicroVisual Studio 2005

Visual Studio 6.0 Enterprise with Service Pack 5

Yong Dynamic Software vbAdvance 3.1

RM/COBOL v11.01 Compiler

RM/COBOL v11.01 WOW Extensions

64180 macro assembler version 4.01a by 2500 A.D. Software

Crystal Reports 9 Full Developer

Crystal Reports 9 Main Program Files Patch

(05/14/2003)

Crystal Reports 9 Database and Export Patch

(05/14/2003)

OmniDrive SDK v1.01

Sourceforge Info-Zip Zip version 2.32

Sourceforge Info-Zip Unzip version 5.52

Xerces-C 2.7.0

InstallShield Professional version 7.01

InstallShield Express version 2.1
VSTL Certification # pending

Page 239 of 247 (V)2009-30Jun-001(A)

Additional file(s) loaded and version(s) Codebase file - d4all.h(Modified COTS)

Record the disk image software version being used Norton GHOST V:11.0

Record the filename of the build environment file 

signature (5.6.1.3) – 

Unity3.2.0.0_PostCOTS_05212009.hash.txt

Record the filename of the build environment disk image 

–

Unity3.2.0.0_PostCOTS_05212009.GHO

Verify (by signature) the build environment file signature 

(5.6.1.3)

Sridevi Jakileti

Loading Source Code (5.6.2)

Record the file signature of the source code (5.6.2.1) see table of source code, above

Verify (by signature) that each file signature of the 

source code loaded matches as documented above 

(5.6.2.1)

Sridevi Jakileti

Method of Build Witness Trusted Build

Record the combined source code and pre-build 

environment file signature (5.6.2.2)

Unity3.2.0.0_PreBuild_05212009.hash.txt

Record the combined source code and pre-build 

environment disk image (5.6.2.3)

Unity3.2.0.0_PreBuild_05212009.GHO

Record the Final Build Version – Unique Identifier AuditManager 7.5.2.0

EDM 7.8.1.0

ERM 7.5.3.0 (superseded in Supplemental A)

ESSIM 7.7.1.0

HPM 5.7.1.0

Certification Application Number (if applicable) ESS0701

Document during the Build Witness:

Date / Time Build Initiated 5/19/2009 12.45PM

Compiler and Version ERM: WOW Extensions Designer version 11.01, RM/Cobol for 

Windows version 11.01, RM/Cobol Codebridge for Windows, 

Microsoft Visual C++ 6.0, Borland C/C++ 5.01, Microsoft Visual 

Basic 5.0, InstallShield Express 2.12

HPM: Cobol-WOW version 3.12, RM/Cobol for Windows version 

7.50.01, RM/Cobol Codebridge for Windows, Microsoft Visual 

C++ 6.0, Borland C/C++ 5.01, Microsoft Visual Basic 5.0, 

InstallShield Express 2.12

EDM: Microsoft Visual Studio 6.0 SP 5, InstallShield 

Professional 7.01

AuditManger and ESSIM: Microsoft Visual Basic 6.0 SP6, 

InstallShield Professional 7.01

Application Name Unity3.2.0.0

Application Version Order AuditManager 7.5.2.0

EDM 7.8.1.0

ERM 7.5.3.0 (superseded in Supplemental A)

ESSIM 7.7.1.0

HPM 5.7.1.0

Obtain Names and Signatures of all persons present 

during build (record below)

Sridevi Jakileti

Dave Herrera

Issue(s) and Resolution(s) No Issues

Document at Completion of the Build Witness:

Record the disk image of the final build (5.7.3) Unity3.2.0.0_PostCots_05212009.GHO

Record file signature of the final build (5.6.3.1) Unity3.2.0.0_PostCots_05212009.hash.txt

Record the type of unalterable storage media being used 

for installation disk(s) (i.e., CD) – (5.6.3.2)

CD

Record each piece of media that is part of the installation 

disk (each must have a unique identifier) (5.6.3.2, 5.7.5)

Unity3.2.0.0_TBINSTALLS05192009.hash.txt

Record the file signature of the installation disk(s). 

(5.6.3.3, 5.7.5) (include in below archive)

ESS_Unity3.2.0.0_TBInstall05212009.hash.txt

Record the type of unalterable storage media being used 

for pre-build and post-build archive disk (i.e., CD) –

T9 hard Drive backed up on T8 (Pending Burning DVD or CD) 

final burn to unalterable media is pending the EAC Initial 

Decision

Record each piece of media that is part of the pre-build 

archive disk (each must have a unique identifier) 

(5.6.2.4, 5.7.2, 5.7.3)

T9 hard Drive backed up on T8 (Pending Burning DVD or CD) 

final burn to unalterable media is pending the EAC Initial 

Decision

Explanation of any significant differences observed No differences
VSTL Certification # pending

Page 240 of 247 (V)2009-30Jun-001(A)

Build Witnesses): Hard copies of the witness signatures are on file at iBeta:

Sridevi Jakileti 5/27/2009

ES&S Dave Herrera 5/27/2009

7.7.9 Witness of the Trusted Build Unity 3.2.0.0 (Supplemental Build A) - ERM 

v.7.5.4.0

Application/ Component Version Language File Signature

ERM 7.5.4.0b Cobol Unity_ERM_Source_TB06252009.hash.txt

Document Prior to the Build Witness:

Vendor Name ES&S

Vendor Consultant(s) (5.6) Dave Herrera

Witness Name (5.6) Sridevi jakileti

Witness Title Trusted Builder

Vendor Build Document(s) used and version(s) Unity_BECI_3.2.0.0_2009.06.25.pdf

Unity_PreBuildTaskList_3.2.0.0_2009.06.25.pdf

WinXP-CorsairwithVGA_INST_2009.04.22.pdf 

Visual Studio_6.0_EntEdwithSP5_INST_2009.04.30.pdf 

vbAdvance3.1_INST_2009.04.30.pdf 

VisualStudio2005ProEd_INST_2009.04.21.pdf 

RMCOBOLDeveloper11.01_INST_2009.04.29.pdf 

AvocetADXZ180_INST_2009.04.30.pdf 

Codebase6.5Release3_INST_2009.04.21.pdf 

CrystalReports9_INST_2009.04.29.pdf 

PCCardSDKv2.20_INST_2009.04.30.pdf 

Xerces2.7.0_INST_2009.04.25.pdf 

InstallShieldPro7.01_INST_2009.03.23.doc 

InstallShieldExpress2.1_INST_2008.11.30.pdf

Equipment Used Corsair Orbit PC

iBeta COTS used to clean the build environment disk 

(name and version) (5.6.1.1)

Restored from post cots image from Unity3.2.0.0 TB05212009 

(Unity3.2.0.0_PostCOTS_05212009.GHO)

iBeta COTS used to clean the build environment disk 

(name and version)

Mares Hash Ver. 07.08.10.07.12

Construct the build environment (5.6.1.2)

Verify (by signature) that the build environment is 

isolated and controlled by iBeta

Sridevi Jakileti

Witness attests to verifying that the source code being 

built is the source code provided by iBeta

Dave Herrera

Vendor CM Tool and version Listed in TDP document: Unity 3.2.0.0 Source Code Repository 

& Version

Build Environment Operating System Windows XP with Service Pack 3

Build tool(s) and version(s) AuditManager: MicroVisual Studio 2005

ALL other unity software: Visual Studio 6.0 Enterprise with 

Service Pack 5

ERM and HPM: RM/COBOL v11.01 

3

rd Party Libraries and Version ESSZIP: zip32.lib; unzip32.lib, 

AuditManager: asycfilt.dll2.40.4275.1, 

Comcat.dll,comct332.ocx6.7.0.8988, 

comdlg32.ocx6.0.84.18,msado25.tlb2.5 2.60.6526.0, 

msadodc.ocx6.0.88.4,

msbind.dll6.0.88.62,mscdrun.dll6.0.88.4

mscomct2.ocx 6.0.88.4 ,mscomctl.ocx 6.0.88.62 

,msderun.dll6.0.88.4

msflxgrd.ocx6.0.84.18,msvbvm60.dll6.0.89.64

oleaut32.dll2.40.4275.1,olepro32.dll5.0.4275.1

stdole2.tlb2.40.4275.1,sysinfo.ocx6.0.81.69

vb6stkit.dll 6.0.84.50, comct332.ocx 

6.7.0.8988,comdlg32.ocx6.0.84.18,

msadodc.ocx6.0.88.4,mscomct2.ocx6.0.88.4,
VSTL Certification # pending

Page 241 of 247 (V)2009-30Jun-001(A)

mscomctl.ocx6.0.88.62,msflxgrd.ocx6.0.84.18,

sysinfo.ocx6.0.81.69,msbind.dll6.0.88.62,

mscdrun.dll6.0.88.4,msderun.dll6.0.88.4,

msrdo20.dll6.0.88.62,msstdfmt.dll6.0.88.4,

rdocurs.dll6.0.88.4,

ESSIM: mfc80.dll8.0.50727.42,

mfc80u.dll8.0.50727.42,mfcm80.dll8.0.50727.42,

mfcm80u.dll8.0.50727.42,msvcm80.dll8.0.50727.42,

msvcp80.dll8.0.50727.42,msvcr80.dll8.0.50727.42,

msvcr71.dll8.0.50727.42,msvcrt.dll6.0.8797.0,

roboex32.dll9.20.534.0,unzip32.dll5.5.2.0,

zip32.dll 2.3.1.0,microsoft.vc80.crt.manifest,

microsoft.vc80.mfc.manifest,

EDM :c4dll.dll1.0.0.1,

crdb_p2bxbse.dll9.2.1.106,crpe32.dll9.2.3.745,

crqe.dll9.2.1.605,crtslv.dll9.2.0.528,

crxf_pdf.dll9.2.1.567,crxf_rtf.dll9.2.0.566,

crxf_wordw.dll9.2.0.566,crxf_xls.dll9.2.1.662,

exportmodeller.dll9.2.1.559,inetwh32.dll7.0.133.0,

msvcr71.dll7.10.3052.4,roboex32.dll9.0.79.0,

u2ddisk.dll9.2.0.541,u2ftext.dll9.2.1.555,

ufmanager.dll9.0.0.1,xerces-c_2_7.dll2.7.0.0, 

ERM:ROBOEX32.DLL 

5.0.0.7.022,wh2robo.dll13.10.606.6,asycfilt.dll2.40.4275.1,comc

at.dll4.71.1460.1, comct332.ocx6.7.0.8988,

comdlg32.ocx6.0.84.18,mscomct2.ocx6.0.88.4,mscomctl.ocx6.0

.88.62,MSVBVM50.DLL5.1.43.19,

msvbvm60.dll6.0.89.64,msvcp60.dll6.0.8168.0,

MSVCRTD.DLL6.0.8447.0,oleaut32.dll2.40.4275.1,olepro32.dll5

.0.4275.1,OpenSaveFile.ocx1.0.0.0,

RICHTX32.OCX6.0.88.4,stdole2.tlb2.40.4275.1,

sysinfo.ocx6.0.81.69,tabctl32.ocx6.0.90.43,WSC32.DLL0.0.0.0 

Linker Rev. 3, WSC32.lib, 

HPM: 

HtmlWH.dll7.0.131.0,ROBOEX32.DLL8.0.131.0,asycfilt.dll2.40.

4275.1,

comcat.dll4.71.1460.1,comct332.ocx6.7.0.8988,comdlg32.ocx6.

0.84.18,

MFC42D.DLL6.0.8447.0,mscomct2.ocx6.0.88.4,mscomctl.ocx6.

0.88.62,

msdxm.ocx6.4.9.1128,msvbvm60.dll6.0.89.64,msvcp60.dll6.0.8

168.0,

MSVCP60D.DLL6.0.8168.0,msvcrt.dll6.0.8797.0,MSVCRTD.DL

L6.0.8447.0,

oleaut32.dll2.40.4275.1,olepro32.dll5.0.4275.1,OpenSaveFile.o

cx1.0.0.0,

Quartz.dll6.4.2600.1221,RICHTX32.OCX6.0.88.4,stdole2.tlb2.4

0.4275.1,

3

rd Party Source Code (COTS) and Version Codebase 6.5, Release 3

Modified COTS

Codebase file - d4all.h

3

rd Party DLLs, Drivers, etc. and Version(s) Windows XP with Service Pack 3,MicroVisual Studio 2005

Visual Studio 6.0 Enterprise with Service Pack 5

Yong Dynamic Software vbAdvance 3.1

RM/COBOL v11.01 Compiler

RM/COBOL v11.01 WOW Extensions

64180 macro assembler version 4.01a by 2500 A.D. Software

Crystal Reports 9 Full Developer

Crystal Reports 9 Main Program Files Patch

(05/14/2003)

Crystal Reports 9 Database and Export Patch

(05/14/2003)

OmniDrive SDK v1.01

Sourceforge Info-Zip Zip version 2.32

Sourceforge Info-Zip Unzip version 5.52
VSTL Certification # pending

Page 242 of 247 (V)2009-30Jun-001(A)

Xerces-C 2.7.0

InstallShield Professional version 7.01

InstallShield Express version 2.1

Additional file(s) loaded and version(s) Codebase file - d4all.h(Modified COTS)

Build Scripts

Record the disk image software version being used Notron GHOST V:11.0

Record the filename of the build environment file 

signature (5.6.1.3) – 

Unity3.2.0.0_PostCOTS_05212009.hash.txt

Record the filename of the build environment disk image 

–

Restored from post cots image from Unity3.2.0.0 TB05212009 

(Unity3.2.0.0_PostCOTS_05212009.GHO)

Verify (by signature) the build environment file signature 

(5.6.1.3)

Sridevi Jakileti

Loading Source Code (5.6.2)

Record the file signature of the source code (5.6.2.1) see table of source code, above

Verify (by signature) that each file signature of the 

source code loaded matches as documented above 

(5.6.2.1)

Sridevi Jakileti

Method of Build Witness Trusted Build

Record the combined source code and pre-build 

environment file signature (5.6.2.2)

Unity3.2.0.0_PreBuild_06252009.hash.txt

Record the combined source code and pre-build 

environment disk image (5.6.2.3)

Unity3.2.0.0_PreBuild_06252009.GHO

Record the Final Build Version – Unique Identifier ERM7.5.4.0

Certification Application Number (if applicable) ERM7.5.4.0

Document during the Build Witness:

Date / Time Build Initiated 06/25/2009 7.15am

Compiler and Version ERM: WOW Extensions Designer version 11.01, RM/Cobol for 

Windows version 11.01, RM/Cobol Codebridge for Windows, 

Microsoft Visual C++ 6.0, Borland C/C++ 5.01, Microsoft Visual 

Basic 5.0, InstallShield Express 2.12

HPM: Cobol-WOW version 3.12, RM/Cobol for Windows version 

7.50.01, RM/Cobol Codebridge for Windows, Microsoft Visual 

C++ 6.0, Borland C/C++ 5.01, Microsoft Visual Basic 5.0, 

InstallShield Express 2.12

EDM: Microsoft Visual Studio 6.0 SP 5, InstallShield 

Professional 7.01

AuditManger and ESSIM: Microsoft Visual Basic 6.0 SP6, 

InstallShield Professional 7.01

 

Application Name Unity3.2.0.0

Application Version Order ERM7.5.4.0

Obtain Names and Signatures of all persons present 

during build (record below)

Sridevi Jakileti

Dave Herrera

Issue(s) and Resolution(s) No Issues

Document at Completion of the Build Witness:

Record the disk image of the final build (5.7.3) Unity3.2.0.0_PostBuild_06252009.GHO

Record file signature of the final build (5.6.3.1) Unity3.2.0.0_PostBuild_06252009.hash.txt

Record the type of unalterable storage media being used 

for installation disk(s) (i.e., CD) – (5.6.3.2)

CD

Record each piece of media that is part of the installation 

disk (each must have a unique identifier) (5.6.3.2, 5.7.5)

Unity3.2.0.0(ERM) TB062509 Installs

Record the file signature of the installation disk(s). 

(5.6.3.3, 5.7.5) (include in below archive)

Unity3.2.0.0_TBINSTALLS06252009.hash.txt

Record the type of unalterable storage media being used 

for pre-build and post-build archive disk (i.e., CD) –

T8 hard Drive (Pending Burning DVD or CD) final burn to 

unalterable media is pending the EAC Initial Decision

Record each piece of media that is part of the pre-build 

archive disk (each must have a unique identifier) 

(5.6.2.4, 5.7.2, 5.7.3)

T8 hard Drive (Pending Burning DVD or CD) final burn to 

unalterable media is pending the EAC Initial Decision

Explanation of any significant differences observed Restored the PostCots Image from TB05192009

Build Witnesses): Hard copies of the witness signatures are on file at iBeta:
VSTL Certification # pending

Page 243 of 247 (V)2009-30Jun-001(A)

Sridevi Jakileti 6/25/2009

ES&S Dave Herrera 6/25/2009

7.7.10 ES&S Unity 3.2.0.0 System Identification Tools

As identified in Section 5.8 and 5.9 of the US Election Assistance Commission Test and Certification Program Manual

delivery of the System Identification Tools to the EAC is the responsibility of ES&S following the EAC Initial Decision. 

Review of the System Identification Tools is the responsibility of the EAC.
VSTL Certification # pending

Page 244 of 247 (V)2009-30Jun-001(A)

7.8 Appendix H: Amended Test Plan

The Election Systems & Software Unity 3.2.0.0 Voting System VSTL Certification Test Plan and the EAC Approval 

letter of ES&S Unity 3.2 Test Plan version 2.0 are found on the EAC website. 

This test plan was amended during test execution. This amended test plan is attached, ES&S Unity 4.0 VSTL 

Certification Test Plan v. 3.0. Changes are provided in red text for easy identification.
VSTL Certification # pending

Page 245 of 247 (V)2009-30Jun-001(A)

7.9 Appendix I: State Test Reports

No state testing was performed by iBeta concurrent to certification testing.

No state testing was identified by ES&S, the EAC or SysTest Labs. 
VSTL Certification # pending

Page 246 of 247 (V)2009-30Jun-001(A)

7.10 Appendix J ES&S Unity 3.2.0.0 Implementation Statement

A copy of the ES& S Unity 3.2.0.0 implementation statement is attached as a separate document.
VSTL Certification # pending

Page 247 of 247 (V)2009-30Jun-001(A)

7.11Appendix K – EAC Certification Number & Voting System 

Configuration

This report was submitted to the Election Assistance Commission on June 30, 2009. It is pending their acceptance.

No certification number has been issued. When iBeta receives notification that the report is accepted, a revised version of 

the report will be issued. The Certification number will appear here and on the Title page. Any other revisions will be noted 

in the version history

This Certification is for the Voting System Hardware and Software configuration(s) listed in section 3.1.