2675 South Abilene Street, #300, Aurora, Colorado, 80014

Form-E: VSTL Test Report template

ES&S Unity 3.2.0.0 Revision 1 Voting System 

Certification Test Report for DS200 Modifications 

to the EAC Certified ESSUNITY3200

Prepared for 

Election System and Software

11208 John Galt Blvd.

Omaha, NE 68137

EAC Application ESS1002

Version 2.0

iBeta Report Number: (V)2010-30Jun-001(B)

This report has been submitted to the EAC for review and is pending their acceptance. No certification 

number has been issued. When iBeta receives notification that the report is accepted, a revised version of 

the report will be issued. The Certification number will appear here and in the page headers. Any other 

revisions will be noted in the version history.

Trace to Standards

NIST Handbook 150-22

Section 5.5, 5.10.1 through 5.10.3, 5.10.5, 5.10.6

VVSG

Vol. # Section(s) #

1 1.4.1

1 2, 3, 4, 5, 6, & 7

2 1.8.3

2 2, 3, 4, 5, & 6

2 7.4 & 7.5.

2 Appendix B

Test Results in this report apply to the voting system configuration tested. Testing of voting systems that have been 

modified may or may not produce the same test results. This report shall not be reproduced, except in full.

iBeta Quality Assurance is accredited for Voting System Testing:

EAC Lab Code: 0702 - Effective thru 7/16/11
EAC Certification # pending

iBeta Report Number: (V)2010-30Jun-001(B)

ii

Version History

Ver # Description of Change Author Approved by Date

v.1.0 Initial release Carolyn Coggins

Jenn Garcia

Steve Brown

C Coggins

Steve Pearson 

and John Lento 

6/30/10 

v.2.0 Changes initiated by EAC Review comments: 

Significant changes to the report are indentified in 

green. 

Table 1- SOW 11 date 

Table 2 & Section 5.4.1 identify EAC letter

Section 2, Table 11 & Appendix D: Clarify scanner 

limit adjustment in Field Issue 1 and document 

review

Section 2.3.1 add: identify the VSTL test scope

Table 7 - identified HW revision & functionally tested 

ECOs 

Table 9 – identified Field Issue 1 documents

Table 11- clarify withdrawn ENH

Table 12- identify ECOs in functional tests

Section 2.3.3, 4.2. & 5.4 identify steel ballot box and 

clarified de minimus definition

Section 5.1.1 with number of lines of code

Section 5.3 Fix broken header

Appendix D - Added ENH16120 references and 

Spanish translation text

Minor changes (format, typos and spelling corrections) 

remain in black 

Small caps format of all instances of ESSUNITY3200

Carolyn Coggins

Jenn Garcia

Steve Brown

Carolyn Coggins 7/21/10
VSTL Certification # pending

- iii - iBeta Report Number: (V)2010-30Jun-001(B)

TABLE OF CONTENTS

1 INTRODUCTION ............................................................................................................................................ 5

1.1 UNITY 3.2.0.0 REVISION 1 EXCLUSIONS .................................................................................................... 5

1.2 INTERNAL DOCUMENTATION...................................................................................................................... 5

Table 1 Internal Documents........................................................................................................................................ 5

1.3 EXTERNAL DOCUMENTATION..................................................................................................................... 6

Table 2 External Documents ...................................................................................................................................... 6

1.4 TECHNICAL DATA PACKAGE DOCUMENTS .................................................................................................. 9

1.5 TEST REPORT CONTENTS......................................................................................................................... 9

2 CERTIFICATION TEST BACKGROUND .................................................................................................... 10

2.1 TERMS AND DEFINITIONS ........................................................................................................................ 10

Table 3 Terms and Definitions.................................................................................................................................. 10

2.2 PHYSICAL CONFIGURATION AUDIT........................................................................................................... 11

2.2.1 PCA TDP Source Code Review ...................................................................................................... 11

2.2.2 PCA TDP Document Review........................................................................................................... 11

2.2.3 PCA System Configuration Review................................................................................................. 11

2.2.4 Witness Build and Installation.......................................................................................................... 11

2.3 FUNCTIONAL CONFIGURATION AUDIT....................................................................................................... 11

2.3.1 FCA Test Documentation Review ................................................................................................... 11

2.3.2 FCA Functional and System Level Tests ........................................................................................ 11

2.3.3 FCA Hardware Environmental Tests ............................................................................................... 11

3 VOTING SYSTEM IDENTIFICATION .......................................................................................................... 12

3.1 SUBMITTED VOTING SYSTEM IDENTIFICATION .......................................................................................... 12

Table 4 Voting System Name and Version............................................................................................................... 12

Table 5 Voting System Polling Place and Central Count Hardware ......................................................................... 12

Table 6 Voting System EMS Software...................................................................................................................... 12

3.2 VOTING SYSTEM TEST ENVIRONMENT ..................................................................................................... 12

Table 7 Voting System Hardware............................................................................................................................. 12

Table 8 Voting System Software .............................................................................................................................. 13

Table 9 Voting System Technical Data Package Documents................................................................................... 14

Table 10 Other Software, Hardware and Materials................................................................................................... 14

4 VOTING SYSTEM OVERVIEW (CHANGES TO ESSUNITY3200)............................................................. 16

4.1 DS200 FIRMWARE CHANGES ................................................................................................................. 16

Table 11 –DS200 Functional Changes to ESSUNITY3200........................................................................................ 16

4.2 DS200 ENGINEERING CHANGES............................................................................................................. 17

Table 12 Submitted DS200 Engineering Change Orders ......................................................................................... 18

5 CERTIFICATION REVIEW AND TEST RESULTS...................................................................................... 19

5.1 PCA SOURCE CODE REVIEW.................................................................................................................. 19

5.1.1 C and C++ DS200 with Scanner_C85051 Source Code Review Results....................................... 19

5.2 PCA TDP DOCUMENT REVIEW............................................................................................................... 19

5.3 FCA DS200 FUNCTIONAL AND SYSTEM LEVEL TESTING .......................................................................... 19

5.3.1 DS200 Functional and System Level Test Results ......................................................................... 20

5.4 FCA HARDWARE ENVIRONMENTAL TESTING............................................................................................ 20

Table 13 DS200 ECO Hardware Assessments and Required Testing. .................................................................... 20

5.4.1 FCA Hardware Environmental Tests ............................................................................................... 20

6 OPINIONS & RECOMMENDATIONS.......................................................................................................... 21

7 APPENDICES: TEST OPERATION, FINDINGS & DATA ANALYSIS....................................................... 22

7.1 APPENDIX A: CERTIFICATION TEST REQUIREMENTS................................................................................. 22

7.2 APPENDIX B: PCA SOURCE CODE REVIEW ............................................................................................. 22

7.3 APPENDIX C: PCA TDP DOCUMENTATION REVIEW.................................................................................. 22

7.4 APPENDIX D: FCA TESTING.................................................................................................................... 22

7.4.1 FCA Functional and System Level Testing ..................................................................................... 22
EAC Certification # pending

iBeta Report Number: (V)2010-30Jun-001(B)

iv

7.4.2 FCA Hardware Environmental Testing............................................................................................ 28

7.5 APPENDIX E- DISCREPANCY REPORT...................................................................................................... 33

7.6 APPENDIX F: WARRANT OF ACCEPTING CHANGE CONTROL RESPONSIBILITY............................................. 34

7.7 APPENDIX G: TRUSTED BUILD & VALIDATION TOOLS UNITY 3.2.0.0 REVISION 1........................................ 34

7.8 APPENDIX H: AMENDED TEST PLAN ........................................................................................................ 34

7.9 APPENDIX I: STATE TEST REPORTS......................................................................................................... 34

7.10 APPENDIX J: ES&S UNITY 3.2.0.0 IMPLEMENTATION STATEMENT............................................................. 34
EAC Certification # pending

Page 5 of 34 iBeta Report Number: (V)2010-30Jun-001(B)

1 Introduction

This report is submitted to the Election Assistance Commission (EAC) by iBeta Quality Assurance summarizing the 

federal VSTL Certification Testing of the v.1.4.3.0 firmware and hardware engineering changes to the IntElect 

DS200 Precinct Count Scanner (DS200) in the certified ESSUNITY3200 voting system submitted in EAC application 

ESS 1002. The changes submitted in Unity 3.2.0.0 Revision 1 are tested to the EAC Voluntary Voting System 

Guidelines (VVSG 2005). The purpose of this document is to provide an overview of the certification testing and 

findings. The EAC Certificate of Conformance ES&S Unity 3.2.0.0, found on the EAC website, provides the official 

description of the ESSUNITY3200 baseline. It includes the complete list of the systems names, major subsystems, 

and interfacing devices. The Unity 3.2.0.0 Revision 1 test environment is found in Section 3 Voting System 

Identification. Details of the changes are identified in the Section 4 Voting System Overview (Changes to 

ESSUNITY3200)

The DS200 v.1.4.3.0 changes submitted in Unity 3.2.0.0 Revision 1 have also been submitted to the Unity 3.2.1.0 

certification test effort. In the instances where tests or reviews for these certification efforts were identical iBeta 

used the common results.

This certification test effort included a Physical Configuration Audit (PCA) of the changes to the DS200:

Technical Data Package (TDP) documents and the DS200 source code;

Preparation of a Trusted Build from the reviewed source code; and 

Assessment of the engineer changes orders submitted for the DS200 hardware.

Additionally a Functional Configuration Audit (FCA) included:

Review of the test documentation submitted by ES&S of the DS200 changes;

Development of a test plan and test cases for Functional and Hardware Electrical Testing of the DS200 

changes and test results analysis; and

Managing the changed DS200 and the unchanged ESSUNITY3200 baseline test configuration.

Certification testing was performed in compliance with the requirements of VVSG 2005, Volume II National 

Certification Testing Guidelines. The test record included all test executions and reviews. These contained the 

record of requirements completed, deficiencies noted, reports to ES&S, software and manufacturing resolutions, 

validations and documentation of resolutions. iBeta provided ES&S with daily status reports during the test effort.

iBeta Quality Assurance, a limited liability company, is located in Aurora, Colorado. The company is a full service 

software testing laboratory providing Quality Assurance and Software Testing for the business and interactive 

entertainment communities. iBeta Quality Assurance accreditations for the testing of voting systems to the federal 

standards include 

National Voluntary Lab Accreditation Program (NVLAP) Voting System Test Lab (VSTL)

Election Assistance Commission Voting Systems Test Lab (VSTL)

Testing of the DS200 firmware changes was conducted by iBeta in Aurora, Colorado. Non-core hardware 

environmental testing is outside iBeta's accreditation scope as a VSTL. Electrical testing of the engineering 

changes to the DS200 hardware was performed at Criterion Technology, Rollinsville, CO under iBeta's supervision. 

iBeta confirmed sub-contractor Criterion accreditation by the NVLAP for Electromagnetic Compatibility & 

Telecommunications (valid through March 31, 2011) of the VVSG 2005 required test methods. (A list is found at 

the NVLAP linked page.) 

1.1 Unity 3.2.0.0 Revision 1 Exclusions

The Unity 3.2.0.0 Revision 1 does not support elections in Illinois and Hawaii. All other exclusions from the certified 

ESSUNITY3200 voting system remain unchanged.

1.2 Internal Documentation

The documents identified below are iBeta internal documents used in certification testing 

Table 1 Internal Documents

Version # Title Abbreviation Date Author (Org.)

v.07 Voting Certification Master Services 

Agreement Prepared for Election 

MSA contract 11/15/08 iBeta Quality Assurance
EAC Certification # pending

Page 6 of 34 iBeta Report Number: (V)2010-30Jun-001(B)

Version # Title Abbreviation Date Author (Org.)

Software & Services (ES&S)

Statement of Work 11 SOW 11 6/11/10 iBeta Quality Assurance

iBeta VSTL Procedures

v.3.0 Voting Deliverable Receipt Procedure 2/9/10 iBeta Quality Assurance

v.4.0 PCA Document Review Procedure 5/6/10 iBeta Quality Assurance

v.5.0 PCA Source Code Review Procedure 4/30/10 iBeta Quality Assurance

v.5.0 C and C++ Review Criteria 3/2/09 iBeta Quality Assurance

v.0.2 COBOL Review Criteria 3/3/09 iBeta Quality Assurance

v.1.0 Witness Build Procedure 4/07/08 iBeta Quality Assurance

v.3.0 Trusted Build Procedure 4/6/10 iBeta Quality Assurance

v.5.0 Test Case Preparation & Execution 

Procedure

2/9/10 iBeta Quality Assurance

v.6.0 Project Management Voting Procedure 4/12/10 iBeta Quality Assurance

v.5.0 VSTL Test Planning Procedure 2/9/10 iBeta Quality Assurance

v.5.0 VSTL Certification Report Procedure 4/6/10 iBeta Quality Assurance

iBeta Project Documents

4.0 Election Systems & Software Unity 

3.2.0.0 Voting System VSTL Certification 

Test Report (V)2009-30Jun-001(D) *

7/22/09 iBeta Quality Assurance

ES&S Unity 3.2.0.0 Revision 1 PCA and 

FCA Discrepancy Report

6/29/10 iBeta Quality Assurance

v.3.0 ES&S Unity 3.2.0.0 Revision. 1 Voting 

System Certification Test Plan for DS200 

Modifications to the EAC Certified 

ESSUNITY3200

6/30/10 iBeta Quality Assurance

DS200 Functional TC Unity 3.2.0.0 

Revision 1

6/29/10 iBeta Quality Assurance

FCA Environmental Test Case Unity 

3.2.1.0

3/25/10 iBeta Quality Assurance

Unity 3.2.0.0 Revision 1 DS200 Trusted 

Build Read Me

6/14/10 iBeta Quality Assurance

1.0 Trusted Build of the DS200 FW 1.4.3.0 2/12/10 iBeta Quality Assurance

* Public document found on the EAC website

1.3 External Documentation

The documents identified below are external resources used to in certification testing.

Table 2 External Documents

Version # Title Abbreviation Date Author (Org.)

Help America Vote Act* HAVA Oct. 29, 2002 107th Congress

NIST Hdbk 

150 2006

NVLAP Voting System Testing NIST 150 Feb. 2006 NVLAP

NIST Hdbk 

150-22

NVLAP Voting System Testing NIST 150-22 Dec.2005 NVLAP

Voluntary Voting System Guidelines* VVSG Dec. 2005 EAC

EAC Certification of Conformance ES&S 

Unity 3.2.0.0 Election Systems and Software*

ESSUNITY3200

voting system

July 29, 2009 EAC

v.1.0 Testing and Certification Program Manual* Jan. 1, 2007 EAC

v.1.0 Voting System Test Laboratory Program 

Manual*

July 2008 EAC

Unity 3.2 Rev 1 Testing (Field Issue 2 EAC 

email)

6/21/10 EAC
EAC Certification # pending

Page 7 of 34 iBeta Report Number: (V)2010-30Jun-001(B)

Version # Title Abbreviation Date Author (Org.)

DS200 (Field Issue 2 ES&S email) 6/21/10 ES&S

EMC Qualification Test Report Election 

Systems and Software InlElect Precinct Ballot 

Counter DS200 HW Rev. 1.2.1 Report 

Number: 090924-1464

12/3/09 Criterion 

Technology

EMC Qualification Test Report Election 

Systems and Software IntElect Precinct Ballot 

Counter DS200 HW Rev. 1.2.1 Report 

Number: 091130-1503R

3/31/2010 Criterion 

Technology

Voting System Technical Advisory Intermittent 

Freeze/Shutdowns with EAC Certified ES&S 

Unity 3.2.0.0 System

6/25/2010 EAC

EAC Email Response to Unity 3.2.0.0 Rev 1 

Test Plan Status 

6/30/2010 EAC

EAC Letter on De Minimus Changes to Unity 

3.2.0.0

5/25/2010 EAC

RFI & NOC

EAC Decision on Request for Interpretation 2007-

01, Rev. 2 2005 VVSG Vol. 1 Section 3.2.2.1 (e) * 

(AutoMARK Keypad)

Interpretation 

2007-01

5/23/07 EAC

EAC Decision on Request for Interpretation 2007-

02, 2002 Voting Systems Standards, Vol. 1, Section 

4.2.5* (Single character names)

Interpretation 

2007-02

5/14/07 EAC

EAC Decision on Request for Interpretation 2007-

03, 2005 VVSG Vol. 1 Section 3.1.1*(usability test 

review)

Interpretation 

2007-03

9/5/07 EAC

EAC Decision on Request for Interpretation 2007-

04, 2005 VVSG Vol. 1 Section 3.1.3*(Alternate 

language)

Interpretation 

2007-04

10/29/07 EAC

EAC Decision on Request for Interpretation 2007-

05, 2005 VVSG Vol. 1 Section 4.2.1 (Testing Focus 

& Applicability)*

Interpretation 

2007-05

11/6/07 EAC

EAC Decision on Request for Interpretation 2007-

06, 2005 VVSG Vol. 1 Section 4.1.1, 2.1.2c &f, 

2.3.3.3o & 2.4.3c&d. (Recording and reporting 

undervotes) *

Interpretation 

2007-06

11/7/07 EAC

EAC Decision on Request for Interpretation 2008-

01, 2002 VSS Vol. II, 2005 VVSG Vol. II, Section 

4.7.1 & Appendix C* (Temp & power variation 

testing) 

Interpretation 

2008-01

2/6/08 EAC

EAC Decision on Request for Interpretation 2008-

02, Battery Backup for Optical Scan Voting 

machines*

Interpretation 

2008-02

2/19/08 EAC

EAC Decision on Request for Interpretation 2008-

03 (Operating System Configuration) 2002 VSS 

Vol. 1: 2.2.5.3, 4.1.1, 6.2.1.1, Vol. 2:3.5; 2005 

VVSG Vol. 1:2.1.5.2, 5.1.1, 7.2.1, Vol. 2:3.5*

Interpretation 

2008-03

10/3/08 EAC

EAC Decision on Request for Interpretation 2008-

04, 2002 VSS Vol. I, Section 2.3.1.3.1a 2005 

VVSG Vol. II, Section 2.2.1.3a Ballot Production*

Interpretation 

2008-04

5/19/08 EAC

EAC Decision on Request for Interpretation 2008-

05 2002 VSS Vol. I, Section 3.4.2 2005 VVSG Vol. 

I, Section 4.3.2, Durability*

Interpretation 

2008-05

5/19/08 EAC

EAC Decision on Request for Interpretation 2008-

06, 2002 VSS Vol. I, Sections 3.2.2.4c, 3.2.2.5 

2005 VVSG Vol. I, V. 1.0, Sections 4.1.2.4c 

(Electrical Supply), 4.1.2.5 (Electrical Power 

Disturbance) *

Interpretation 

2008-06

8/29/08 EAC

EAC Decision on Request for Interpretation 2008-

07; 2002 VSS Vol. I, Sections, 2.3.4, 2.3.5, 2.3.6, 

2.4.1, 4.4.3, 9.4; 2002 VSS Vol. II, Sections, 3.3.1, 

3.3.2; 2005 VVSG Vol. I, Sections, 2.2.4, 2.2.5, 

2.2.6, 2.3.1, 5.4.3; 2005 VVSG Vol. II, Sections, 

Interpretation 

2008-07

8/27/08 EAC
EAC Certification # pending

Page 8 of 34 iBeta Report Number: (V)2010-30Jun-001(B)

Version # Title Abbreviation Date Author (Org.)

1.3, 3.3.1, 3.3.2*

EAC Decision on Request for Interpretation 2008-

08 2002 VSS Vol. I, 2005 VVSG Vol. I, Glossary * 

(Bar code readers)

Interpretation 

2008-08

8/1/2008 EAC

EAC Decision on Request for Interpretation 2008-

09 (Safety Testing) 2002 VSS Vol. I, Section, 3.4.8 

2005 VVSG Vol. I, Section 4.3.8*

Interpretation 

2008-09

8/25/2008 EAC

EAC Decision on Request for Interpretation 2008-

10 (EFT) 2005 VVSG Vol. I, Sect. 4.1.2.6 2005 

VVSG Vol. II, Sect 4.8*

Interpretation 

2008-10

8/28/2008 EAC

EAC Decision on Request for Interpretation 2008-

12 (Ballot marking Device/ Scope of Testing) 2005 

VVSG Vol. 1: 2.1.5. System Audit 2005 VVSG Vol. 

1: 2.1.5.2 Shared Computing Platform*

Interpretation 

2008-12

12/19/08 EAC

EAC Decision on Request for Interpretation 2009-

001 (VVPAT Accessibility) 2005 VVSG Volume1: 

7.8.2, 7.9.7*

Interpretation 

2009-01

6/25/09 EAC

EAC Decision on Request for Interpretation 2009-

02 (Alternate Languages) 2002 VSS Vol. I: 2.2.1.3a 

Ballot Production; 2005 VVSG Vol. I: 3.1.3 

Alternate Languages*

Interpretation 

2009-02

8/5/09 EAC

EAC Decision on Request for Interpretation 2009-

03 (Battery Back Up for Central Count): RFI 2008-

06 (Battery Back Up for Central Count); 2002 VVSS 

Volume I, Sections 3.2.2.4c, 3.2.2.5; 2005 VVSG 

Volume I, Version 1.0, Sections 4.1.2.4c (Electrical 

Supply), 4.1.2.5 (Electrical Power Disturbance) *

Interpretation 

2009-03

9/28/09 EAC

EAC Decision on Request for Interpretation 2009-

04 (Audit Log Events); 2002 VSS VoI: 2.2.4.1, 

Common Standards, 2.2.5.1 System Audit; 2005 

VVSG VoI: 2.1.4 Integrity, 2.1.5 System Audit, 

2.1.5.1 Operational Requirements, 5.4.3 In-Process 

Audit Records*

Interpretation 

2009-04

9/29/09 EAC

EAC Decision on Request for Interpretation 2009-

05; 2002 VSS Vol. I, Sections, 2.2.7.2 c & d; 2005 

VVSG Vol. I, Sections, 3.2.2.2 c ii & iii*

Interpretation 

2009-05

10/5/09 EAC

EAC Decision on Request for Interpretation 2009-

06 (Temp & Power Variation Tests); 2002 VSS Vol. 

I, Sect. 3.4.3 2002 VSS Vol. II, Sect. 4.7.1, 4.7.2, 

Appx Sec. C.4 2005 VVSG Vol. I, Sect. 4.3.3 2005 

VVSG Vol. II, Sect. 4.7.1, 4.7.3, Appx Sec. C.4 

EAC Decision on RFI 2008-1*

Interpretation 

2009-06

4/6/10 EAC

EAC Decision on Request for Interpretation 2010-

01; 2002 VSS Vol. I, Sections, 3.2.2.8; 2005 VVSG 

Vol. I, Sections, 4.1.2.8* (ESD Voltage Levels)

Interpretation 

2010-01

3/16/10 EAC

EAC Decision on Request for Interpretation 2010-

02 2005 VVSG Vol. I, Section, 5.2.3 d, 5.2.5, 5.2.6, 

5.2.7 b, c, d & e 2005 VVSG Vol. II, Section, 5.4.2* 

(Coding conventions)

Interpretation 

2010-02

5/25/10 EAC

EAC Decision on Request for Interpretation 2010-

03; 2005 VVSG, Volume II, Section 5.4, Section 

5.4.2.a - 5.4.2.v * (Database Coding Conventions)

Interpretation 

2010-03

06/14/2010 EAC

EAC Decision on Request for Interpretation 2010-

04; 2002 VSS Volume I: 2.2.1 Security , 2005 

VVSG Volume I: 2.1.1 Security * (Functional 

Requirements with Respect to Security)

Interpretation 

2010-04

06/24/2010 EAC

Notice of Clarification NOC 07-001: Timely 

Submission of Certification Application*

NOC 07-01 7/17/07 EAC

Notice of Clarification NOC 07-002: VSTL Work 

with Manufacturers Outside of Voting System 

Certification Engagements*

NOC 07-02 7/24/07 EAC

Notice of Clarification: NOC 07-003: State Testing 

Done in Conjunction with Federal Testing within the 

EAC Program*

NOC 07-03 8/6/08 EAC

Notice of Clarification: NOC 07-004: Voting System NOC 07-04 9/5/07 EAC
EAC Certification # pending

Page 9 of 34 iBeta Report Number: (V)2010-30Jun-001(B)

Version # Title Abbreviation Date Author (Org.)

Manufacturing Facilities*

Notice of Clarification 07-05: Voting System Test 

Laboratory (VSTL) responsibilities in the 

management and oversight of third party testing*

NOC 07-05 9/7/07 EAC

Notice of Clarification NOC 08-001: Validity of Prior 

Non-Core Hardware Environmental and EMC 

Testing*

NOC 08-001 3/26/08 EAC

Notice of Clarification: NOC 08-002: Clarification of 

EAC Mark of Certification Requirement*

NOC 08-002 8/30/08 EAC

Notice of Clarification NOC 08-003: Clarification of 

EAC Conformance Testing Requirements for

VSTLs*

NOC 08-003 7/30/08 EAC

Notice of Clarification: NOC 09-001 Clarification of 

the Requirements for Voting System Test 

Laboratories (VSTLs) Development and 

Submission of Test Plans*

NOC 09-001 5/1/09 EAC

Notice of Clarification: NOC 09-002: Clarification of 

EAC Laboratory Independence Requirement*

NOC 09-002 5/4/09 EAC

Notice of Clarification NOC 09-003: Clarification of 

De Minimis Change Determination Requirements*

NOC 09-003 9/19/09 EAC

Notice of Clarification NOC 09-004: Development & 

Submission of Test Reports*

NOC 09-004 11/9/09 EAC

Notice of Clarification NOC 09-005: Development 

and Submission of Test Plans for Modifications to 

EAC Certified Systems*

NOC 09-005 12/2/09 EAC

* Public document found on the EAC website

1.4 Technical Data Package Documents

The Technical Data Package Documents submitted for this certification test effort are listed in Section 3 System 

Identification.

1.5 Test Report Contents

The contents of this Test Report include: 

Section 1: The Introduction- identifies the scope of certification testing.

Section 2 The Certification Test Background identifies the process for the Physical and Functional 

Configuration Audits.

Section 3 The Voting System Identification identifies the system configuration including hardware, software 

and the Technical Data Package documentation.

Section 4 The Voting System Overview identifies the overall design and functionality of voting system.

Section 5 The Certification Review and Test Results are the methods and results of the testing effort.

Section 6 The Opinions & Recommendations of the acceptability of the voting system.

Test Operations, Findings and Data Analysis are in the appendices.

Appendix A: Certification Test Requirement 

Appendix B: Source Code Reviews 

Appendix C: TDP Document Reviews

Appendix D: Test Results

Appendix E: Discrepancy Report

Appendix F: Warrant of Accepting Change Control Responsibility 

Appendix G: Trusted Build and Validation Tools

Appendix H: Amended Test Plan

Appendix I: State Test Reports

Appendix J: ES&S Unity 3.2.0.0 Implementation Statement 
EAC Certification # pending

Page 10 of 34 iBeta Report Number: (V)2010-30Jun-001(B)

2 Certification Test Background

The ESSUNITY3200 baseline was certified by the EAC in July of 2009. ES&S submitted DS200 v.1.4.3.0 firmware 

enhancements and bug fixes and hardware engineering changes. All other applications and hardware of the 

ESSUNITY3200 baseline are unchanged.

Additionally, two reports of field issues were identified to the EAC for the DS200 certified in ESSUNITY3200.

Field Issue 1 is included in Unity 3.2.0.0 Revision 1.

1. The DS200 was sporadically reporting a mark present in row 44 and row 45 of column D on the back of 

the ballot when no actual mark was present. It was found that the specific ballot had been printed with a 

slight skew. Additionally the condition could only be reproduced when the ballot was inserted with a1/8 to 

1/16 inch skew that was slight enough to not generate a rejection of the ballot. The abnormal skew 

revealed that the scanner was reading text slightly outside the channel at this point. Subsequent 

examination by ES&S of this issue determined that in the EACUNITY3200 certified DS200 v.1.13.10.0 

firmware that a constant that limited the scan erroneously allowed the image scan to continue into the 

image of the next column, ES&S Technical Bulleted PRBDS20000006 identifies the issue for jurisdictions 

using DS200 v.1.13.8.0 firmware or the EACUNITY3200 certified v.1.13.10.0. DS200 v.1.4.3.0 was 

submitted in Unity 3.2.1.0 to limit the scan to the column being processed (ES&S Bug16775 and 

Bug16778).

Field Issue 2 is currently under examination by ES&S. It will be addressed in a subsequent certification. The EAC 

has issued a Voting System Technical Advisory on ESSUNITY3200 regarding a "power down" and "freeze". As the 

error messages following the restoration of function did not provide a clear report of the status of the equipment and 

ballots the EAC advisory provides handling instruction. For more information, see the Voting System Technical 

Advisory Intermittent Freeze/Shutdowns with EAC Certified ES&S Unity 3.2.0.0 System posted on the EAC website.

2. DS200 "power down" and "freeze" report was reviewed by iBeta. VSS 2002 error message requirements 

vol.1: 2.2.5.2.2.a and b are not being met. (VSS requirements are listed in Appendix A.) The presence of 

this issue shall be noted in the test report recommendation consistent with the requirements of VVSG 

Vol.2 section B.5. iBeta obtained email confirmation from the EAC and ES&S that there was no 

documented report of any loss or corruption of voting data. 

As part of the EAC Certification application ES&S submitted an implementation statement for Unity 3.2.0.0. Unity 

3.2.0.0 Revision 1 is a revision to the certified ESSUNITY3200 voting system. A copy of this statement is contained 

in the ESSUNITY3200 test report in Appendix J. Daily status reports were sent to ES&S Unity 3.2.0.0 Revision 1 

certification management staff and iBeta project test staff. These reports included project activity status, issues, 

and other relevant information. Status calls were held with the EAC, EAC Reviewers and ES&S.

2.1 Terms and Definitions

The Terms and Definitions identified below are used in this test report. Other applicable Terms and Definitions are 

found in the certified ESSUNITY3200 Test Report.

Table 3 Terms and Definitions

Term Abbreviation Definition

Help America Vote Act HAVA Legislation enacted in 2002 which includes creation 

of the EAC, federal voting standards and 

accreditation of test labs

National Standard Reference 

Library

NSRL Part of NIST that provides software escrow.

National Voluntary Laboratory 

Accreditation Program

NVLAP Part of NIST that provides third-party accreditation 

to testing and calibration laboratories.

Technical Data Package TDP The documentation and code relating to the voting 

system, submitted by the vendor for review.

U.S. Election Assistance 

Commission

EAC U.S. agency established by the Help America Vote 

Act of 2002 to administer Federal elections.

Voluntary Voting System 

Guidelines

VVSG Federal voting system test standards created by 

the EAC. Eventually these will replace the VSS.

Voting System Standards VSS Federal voting system test standards, predecessor 

of the VVSG.

Voting System Test Lab VSTL Lab accredited by the EAC to perform certification 

testing of voting systems.

Voting Variations Significant variations among state election laws 

incorporating permissible ballot content, voting 

options and associated ballot counting logic 
EAC Certification # pending

Page 11 of 34 iBeta Report Number: (V)2010-30Jun-001(B)

2.2 Physical Configuration Audit

The Physical Configuration Audit (PCA) deals with the physical elements of the voting system, including the source 

code, documentation and system configuration reviews, in addition to the witness of the build and installation of the 

reviewed source code. The PCA was only conducted on the changes to the certified ESSUNITY3200 voting system.

2.2.1 PCA TDP Source Code Review

The PCA TDP Source Code Review of the changes to the DS200 in the ESSUNITY3200 voting system was 

performed to verify conformance to VVSG Vol. 1 Sect 5.2 and Vol. 2 Sect. 5. These same changes were submitted 

to the Unity 3.2.1.0 certification test effort. These common results were used in both projects. All results were 

stored in Unity 3.2.1.0. Reviewed results were recorded on Source Code Review sheets (Excel spreadsheets). 

Issues were identified in the review and logged on a Discrepancy Report, after completion of peer review. The 

Discrepancy Report was forwarded to ES&S for correction. At the end of the certification test effort a copy of all 

applicable reviews will be archived in the Unity 3.2.0.0 Revision 1 project.

2.2.2 PCA TDP Document Review

An examination of the Unity 3.2.0.0 Revision 1 TDP was performed to assess any impacts to the ESSUNITY3200

baseline PCA Document Review. This examination consisted of comparison and analysis of the new documents to 

the documents in the certified ESSUNITY3200 voting system. It was determined that none of the changes impacted 

the PCA document review.

2.2.3 PCA System Configuration Review

The PCA System Configuration Review of ES&S Unity 3.2.0.0 Revision 1 was performed to verify conformance to 

VVSG Vol. 1 Sect 9.7.1. Reviewed results are recorded on PCA System Configuration Review sheets (Excel 

spreadsheets). No issues were identified in the review.

2.2.4 Witness Build and Installation

The Witness Build and Installation of the executable code (“trusted build”) for the ES&S Unity 3.2.0.0 Revision 1 

voting system was performed using the reviewed source code per VVSG Vol. 2 Sect 1.8.2.4. Observation of the 

build was documented in the Witness of the Final Build and Code Comparison and contained in Appendix G.

2.3 Functional Configuration Audit

The Functional Configuration Audit was an examination of the hardware and software DS200 changes to the 

ESSUNITY3200 certified voting system. This included review of the ES&S Unity 3.2.0.0 Revision 1 submitted test 

documentation and execution of all required tests.

2.3.1 FCA Test Documentation Review

The FCA Test Documentation Review of the DS200 changes to the ESSUNITY3200 certified voting system 

assessed the level of vendor testing of the voting system to the VVSG Vol. 1 Sect. 2, 3, 4, 5, 6, 7 and 9 

requirements. This assessment was used to define the extent of ES&S' functional testing of the changes and 

identify the VSTL test scope.

2.3.2 FCA Functional and System Level Tests

The DS200 changes submitted in Unity 3.2.0.0 Revision 1 have also been submitted to the Unity 3.2.1.0 

certification test effort. The DS200 firmware updates address internal and field cosmetic and functional 

enhancements and issues. The changes were tested in a single functional test case with seven test scenarios. 

Appendix D details specific information on the Functional Testing. Issues encountered during testing were 

identified in the test record and logged on a Discrepancy Report, after completion of peer review. ES&S resolved 

all discrepancies which did not meet the requirements of the VVSG.

2.3.3 FCA Hardware Environmental Tests

ES&S submitted engineering changes (ECOs) for the ESSUNITY3200 DS200 precinct scanner, the plastic ballot 

box/case and steel ballot box for certification in Unity 3.2.1.0 and Unity 3.2.0.0 Revision 1. All ECOs were reviewed 

and assessed. Those determined to be administrative or having no impact to electrical operation, storage and 

transportation were identified to the EAC as de minimus changes, if they required no other functional testing. 

Changes identified as impacting electrical operation, storage and transportation were retested. There were no 

changes submitted for the ESSUNITY3200 baseline AutoMARK VAT or M650. 
EAC Certification # pending

Page 12 of 34 iBeta Report Number: (V)2010-30Jun-001(B)

3 Voting System Identification

The identification of the ES&S Unity 3.2.0.0 Revision 1 voting system submitted for certification is ultimately documented by 

the EAC. Per their instructions the system identification is found in the EAC Scope of Certification. The hardware, software 

and the Technical Data Package documentation used in the certification test environment are indentified in section 3.2.

3.1 Submitted Voting System Identification

Table 4 Voting System Name and Version

Voting System Name Version

Identified in the EAC Scope of Certification

Table 5 Voting System Polling Place and Central Count Hardware

Hardware OS or Firmware & Version Description 

Identified in the EAC Scope of Certification

Table 6 Voting System EMS Software

Software Applications Version EMS Function Description 

Identified in the EAC Scope of Certification

3.2 Voting System Test Environment

The Voting System Test Environment identifies the specific hardware and software that was used in the test environment. The 

Test Methods in Appendix D identify the specific ES&S 3.2.0.0 Rev 1 voting system software and firmware build installed for 

each test iteration.

Table 7 Voting System Hardware

Hardware OS or Version Manufacturer Description (include functional 

purpose and condition of the 

equipment)

Election Management Hardware

(1) Dell GX260 computer desktop 

with monitor, keyboard & mouse

SN: Tower: 7D0WL21

Windows XP 

Professional 

Version 2002 

SP3

Dell Pre-Vote & Post-Vote: COTS Unity 

PC for the Unity election 

management system

Condition: Good

Precinct Scanner

ES&S intElect DS200

HW 1.2.1

(S/N: 2093900001)

FW 1.4.3.0 ES&S Vote: A Unity Voting System 

precinct count optical scanner paper 

ballot tabulator including a 12-inch 

touch screen display providing clear 

voter feedback and poll worker 

messaging, Condition: Good

ECOs 000628, 000674- Power cord 

shield

Steel Ballot Box

(S/N: C4243)

P/N: 76246 ES&S Vote: Precinct steel ballot box, with 

diverter to segregate ballots into 

multiple chambers as programmed 

in the EMS

Condition: Good

(3) SanDisk 2.0 GB USBs SanDisk COTS: Media for installing election 

definition, recording and reporting 

votes and audit logs

ECO 837 Thumb drive cover

ECO 838 Thumb drive controller

Thermal paper rolls NCR COTS: DS200 reports

Paper Ballots Paper Ballots -

14"

Supplied by ES&S: Ballots for 

DS200 with preprinted election 

content

VL Ballot Marker Pens Marking Device BIC COTS: Ballot Pen to mark paper 

ballots

Other Test Hardware Used to test ECOs in Unity 3.2.1.0

ES&S intElect DS200

HW 1.2.1

SN: ES0107370025

FW 1.3.11.0 ES&S Precinct Count Digital Scanner 

(Modem removed in Unity 3.2.0.0)

Plastic Ballot Boxes 

(HW Rev.1.3)

ES&S Precinct Plastic Ballot Boxes for 

DS200, 
EAC Certification # pending

Page 13 of 34 iBeta Report Number: (V)2010-30Jun-001(B)

Hardware OS or Version Manufacturer Description (include functional 

purpose and condition of the 

equipment)

(1) Bin P/N 94050 

(1) Carrying Case P/N 94051

Emergency Ballot Bin P/N 94325 

(P/N is not marked on the 

Emergency bin)

No Diverter

HW 1.3 

Delkin Thumb Drives: 4GB & 8GB Delkin Delkin Thumb Drives: 4GB & 8GB

COTS: Media for installing election 

definition, recording and reporting 

votes and audit logs

Table 8 Voting System Software

Software Version Manufacturer Identify Hardware

Election Management Software

Election Data Manager (EDM) 7.8.1.0 ES&S EMS software for election definition 

and ballot preparation for M650, 

DS200, and M100

ES&S Ballot Image Manager 

(ESSIM) 

7.7.1.0 ES&S Unity election management system 

desktop publishing tool to layout and 

format paper ballots

Audit Manager (AM) 7.5.2.0 ES&S A Unity election management 

system audit logging software 

application including security and 

user tracking for the Election Data 

Manager and Ballot Image Manager

Hardware Programming Manager 

(HPM)

5.7.1.0 ES&S A Unity election management 

system software application to 

import, format, and convert an 

election file and create election 

definitions for ballot scanning 

equipment

Election Reporting Manager 

(ERM)

7.5.4.0 ES&S A Unity central count software 

application to compile and report 

election results.

Log Monitor 1.0.0.0 ES&S A software application that checks 

the status of the Windows Event Log 

feature and closes all ES&S 

applications if the Event Log feature 

is disabled or not configured 

properly.

Microsoft Windows XP SP3 SP3 Microsoft COTS EMS Operating System

SQLXML 3.0 service pack 

3

Microsoft COTS: XML support for Unity

Windows Internet Explorer 7 Microsoft COTS: AIMS & Unity PC 

Internet Explorer must be resident on 

the PC to contain the latest security 

updates.

Adobe Acrobat v9 9.0 Adobe COTS - Used with ESSIM

Adobe Type Manager 4.1 4.1 Adobe COTS - Used with ESSIM

RM/COBOL v11.01 Runtime 

System

11.01 Liant COTS interpreter software used in 

HPM & ERM

Norton AntiVirus 2005 Symantec Corporation COTS Anti Virus protection for PCs 

and Servers

Optical Scanner

DS200 Firmware

DS200 Power Management 

Firmware

DS200 Scanner Firmware

1.4.3.0

1.2.0.1

2.20.0.0

ES&S Precinct count digital scanner paper 

ballot tabulator including a 12-inch 

touch screen display providing voter 

feedback and poll worker 

messaging. DS200 scanner reads 

marks on both one- and two-sided 

ballots.

Administrators can request custom 

ballot acceptance criteria, which 

ES&S programs onto the scanner’s 

election definition.
EAC Certification # pending

Page 14 of 34 iBeta Report Number: (V)2010-30Jun-001(B)

Table 9 Voting System Technical Data Package Documents

Version # Title Abbreviation Date Author 

(Org.)

5.0 Election Systems & Software System Overview Unity v. 

3.2.0.0 Revision 1

OVR 6/29/10 ES&S

2.0 Election Systems & Software System Limitations Unity v. 

3.2.0.0 Revision 1

6/25/10 ES&S

3.0 Unity 3.2.0.0 Revision 1 System Change Notes None ES&S

None Requirements of the 2005 VVSG Trace to Vendor 

Testing and Technical Data Package

TDP Trace 6/25/10 ES&S

3.0 Election Systems & Software ES&S Software Design 

Specifications DS200 Unity v. 3.2.0.0 Revision 1

DS200 SDS 6/25/10 ES&S

1.0 Election Systems & Software ES&S Software Design 

Specifications ERM Unity v. 3.2.0.0 Revision 1

ERM SDS 6/24/10 ES&S

1.0 Election Systems & Software ES&S System Functionality 

Description DS200 Unity v. 3.2.0.0 Revision 1

DS200 SFD 6/7/10 ES&S

1.0 Election Systems & Software ES&S System Functionality 

Description ERM Unity v. 3.2.0.0 Revision 1

ERM SFD 6/24/10 ES&S

1.0 Election Systems & Software ES&S System Hardware 

Specification DS200 Unity v.3.2.0.0 Revision 1

DS200 SHS 6/7/10 ES&S

None DS 200 Part list 5/12/08 ES&S

None ES&S DS200 System Maintenance Manual Hardware 

Version 1.2.1 Firmware Version 1.4.3.0

6/24/10 ES&S

None ES&S DS200 System Operations Procedures Hardware 

Version 1.2.1 Firmware Version 1.4.3.0

6/24/10 ES&S

None ES&S Election Reporting Manager System Operations 

Procedures Version Release 7.5.3.0

6/28/10 ES&S

None DS200 Precinct Ballot Scanner Election Day Training 

Manual Version Number 1.4.x

6/7/10 ES&S

None DS200 Precinct Ballot Scanner Pre-Election Day Training 

Manual Version Number 1.4.x

6/7/10 ES&S

None ES&S DS200 Ballot Skew Handling 7/15/10 ES&S

None DS200 Ballot Skew Technical Bulletin 10/20/09 ES&S

Table 10 Other Software, Hardware and Materials

Material Material Description Use in the Voting System

Other 

Multiple desktop and laptop PCs A variety of PCs running Microsoft 

operating systems

Supplied by iBeta: Preparation, 

management and recording of test plans, 

test cases, reviews and results

Repository servers Separate servers for storage of test 

documents and source code, 

running industry standards 

operating systems, security and 

back up utilities

Supplied by iBeta: Documents are 

maintained on a secure network server. 

Source code is maintained on a separate 

data disk on a restricted server 

Microsoft Office 2003 & 2007 Excel and Word software and 

document templates

Supplied by iBeta: The software used to 

create and record test plans, test cases, 

reviews and results

SharePoint 2003 TDP and test documentation 

repository

Supplied by iBeta: TDP and test 

documentation repository and 

configuration management tool 

Other standard business application 

software

Internet browsers, PDF viewers 

email

Supplied by iBeta: Industry standard tools 

to support testing, business and project 

implementation

Visual Studio 2003 v.7.1.3808 

(Microsoft)

Build and source code review 

Integrated Development 

Environment

Supplied by iBeta: View source code 

review 

RSM v.6.92 

(M Squared Technologies)

C, C++, Java & C# static analysis 

tool

Supplied by iBeta: identify line counts and 

cyclomatic complexity

Beyond Compare 2 v.2.4.3 (Scooter 

Software)

Comparison utility Supplied by iBeta: used to compare 

file/folder differences

WinDiff 5.1 (Microsoft) Comparison utility Supplied by iBeta: used to compare 

file/folder differences

Knoppix 5.1.0 Hash creation utility Supplied by iBeta: used to generate hash 

signatures on DS200

Symantec Ghost v.11 Image capture tool Supplied by iBeta: used to capture and 
EAC Certification # pending

Page 15 of 34 iBeta Report Number: (V)2010-30Jun-001(B)

Material Material Description Use in the Voting System

test environments

Bart PE 3.1.10a Image capture utility Supplied by iBeta: used with Ghost 

process

Visual Studio 2008 v. 9.0.21022.8 

(Microsoft)

Build and source code review 

Integrated Development 

Environment

Supplied by iBeta: View source code 

review 
EAC Certification # pending

Page 16 of 34 iBeta Report Number: (V)2010-30Jun-001(B)

4 Voting System Overview (Changes to ESSUNITY3200)

The changes to the voting system in Unity 3.2.0.0 Revision 1 identified a change to the DS200 firmware version, non-support 

of Illinois and Hawaii, and engineering change orders to the DS200 hardware and ballot box. There is no other change to the 

overview identified in the ESSUNITY3200 Test Report.

4.1 DS200 Firmware Changes

ES&S submitted a list of firmware enhancements and bug fixes for Unity 3.2.0.0 Revision 1.

Table 11 –DS200 Functional Changes to ESSUNITY3200

ES&S 

Change ID

ES& S 

Category

DS200 

Menu or 

Function 

Change Description Reason for 

Change

Documentation 

Changed

General

BUG15827 Overvote 

handling

System Resolved an issue where the 

scanner failed to divert overvoted 

write-in ballots when the Divert 

Write-ins option was selected

Fixed Issue N/A – Bug Fix

BUG17375 Code Source 

Code

Fix source code discrepancies per 

VSTL review.

Fixed Issue N/A – Bug Fix

BUG17664 Code Source 

Code

Fix source code discrepancies per 

VSTL review.

Fixed Issue N/A – Bug Fix

BUG17666 Reports Status 

Report

Added the protected count to the 

status report that prints 

automatically when a DS200 is reopened for voting.

Fixed Issue N/A – Bug Fix

ENH14725 Interface Display Remove Image Drive icon from 

DS200 if images are not being 

saved

System 

Enhancement

N/A

ENH14726 Interface Display Extend the time that "Thank you for 

voting" displays

System 

Enhancement

N/A

ENH14728 Interface Display Provide clear indication that modem 

transfer was successful

System 

Enhancement

SOP – DS200

ENH14729 Poll Opening Reports Allow multiple zeros tapes to be 

printed before first ballot cast

System 

Enhancement

SOP – DS200 

ENH14730 Interface Audio Change alert beeping to just two 

beeps

System 

Enhancement

N/A

ENH14731 Interface Audio Issue audible alarm when ballot is 

accepted

System 

Enhancement

N/A

ENH14732 Reports Results 

Reports

Repeat machine ID and poll number 

at end of results tape

System 

Enhancement

SOP – DS200

ENH14745 Exception 

Handling

Accept/Reje

ct Ballots

Provide override for Overvote or 

blank ballot rejection

System 

Enhancement

SOP – DS200

ENH15287 Reports Precinct 

Report

Add Early Voting Ballot Styles per 

Precinct Report

System 

Enhancement

SOP – DS200

System Functionality 

Description – DS200

ENH15288 Interface Display Increase Font Size of Thank you for 

Voting message

System 

Enhancement

SOP – DS200

ENH15418 Tabulation Scan 

Accuracy`

Small white dots "hickeys" causing 

read problems

System 

Enhancement

N/A

ENH15890 System Firmware Implement new scanner board 

firmware

System 

Enhancement

System Overview –

Unity 3.2.0.0 Rev1 

ENH15891 Security Counterfeit 

Detection

Implement new administration 

function to calibrate counterfeit 

sensor

System 

Enhancement

SOP – DS200

ENH15892 System Firmware Update scanner client to work with System N/A
EAC Certification # pending

Page 17 of 34 iBeta Report Number: (V)2010-30Jun-001(B)

ES&S 

Change ID

ES& S 

Category

DS200 

Menu or 

Function 

Change Description Reason for 

Change

Documentation 

Changed

new scanner board firmware Enhancement

ENH16085 Interface Display Install New Icons on Welcome 

Screen

System 

Enhancement

SOP – DS200

ENH16120 Interface Display Updated the Over Vote warning 

screen text

System 

Enhancement

SOP – DS200

ENH16211 Reports Reports Print Machine ID & Poll Number in 

Audit Log and after report 

cancellations

System 

Enhancement

SOP – DS200

ENH16231 Audit Admin 

Menu

Enhanced audit logging to log all 

user actions in the Administration 

menu and attempts to access the 

Administration menu.

System 

Enhancement

SOP – DS200

ENH16291 Interface Display DS200: Additional language 

translations for the Over Vote screen

System 

Enhancement

SOP – DS200

ENH16336 Interface Display DS200: Update Spanish language 

translations for the Over Vote screen

System 

Enhancement

SOP – DS200

ENH16382 Tabulation System Expanded support from a maximum 

of 10 Election Day precincts to 18 

precincts.

System 

Enhancement

SOP – DS200

Unity 3.2.0.0 rev1 

System Limitations

ENH17266 Versioning System Update scanner board version System

Enhancement

System Overview – Unity 

3.2.0.0 Revision 1 

(U3200R1_OVR00)

ENH17268 Versioning System Promoted DS200 version 

implemented in Florida for use in 

Unity 3200r1. 

System 

Enhancement

N/A - Versioning

ENH17538 Protected 

Count

System Added a protected count to the 

DS200 firmware. The protected 

count resides on the compact flash 

card in the ES&S firmware partition.

It will increment with every sheet 

accepted and dropped into the ballot 

box. The counter must appear in 

printed reports

System 

Enhancement

SOP – DS200

Field 1

BUG16775 Tabulation System Resolved an issue that caused L&A 

test decks to yield incorrect vote 

totalsFixed Issue N/A – Bug Fix

BUG16782 Scanning System Limit scan to columns being 

processed.

Fixed Issue N/A – Bug Fix

Withdrawn

ENH15009 Security Counterfeit 

Detection

ENH15009 Counterfeit ballot 

detection functionality has been 

withdrawn. The Counterfeit Ballot 

Sensor enhancements ENH15890 to 

15892 remain because the settings 

for this sensor are required for 

correct operation of the DS200.

System 

Enhancement

SOP – DS200

4.2 DS200 Engineering Changes

ES&S submitted engineering changes (ECOs) for the ESSUNITY3200 DS200 Precinct Count Scanner the plastic ballot 

box/case and steel ballot box for certification in Unity 3.2.1.0 and Unity 3.2.0.0 Revision 1. These ECOs fell into two types. 

1. The administrative or hardware changes that don't impact electrical, transportation or storage: Administrative ECOs

covered documentation relevant to the manufacturing administrative process, label changes, production status 

changes, out of scope equipment, documentation updates and drawings. Along with hardware changes that don't 
EAC Certification # pending

Page 18 of 34 iBeta Report Number: (V)2010-30Jun-001(B)

impact electrical, transportation or storage, these ECO's were assessed and determined to require no hardware 

environmental testing.

2. The hardware changes that may impact electrical, transportation or storage: These ECOs submitted addressed endof-life parts and second source suppliers that could impact electrical VVSG test requirements. They required a more 

in-depth assessment by a qualified electrical engineer. 

Table 12 Submitted DS200 Engineering Change Orders

ECO Unity 3.2.1.0 Description ECO Type

Type 1

000315 DS200 Ballot Box Carrying Case -Glue for foam Type 1

000337 DS200 Ballot Bin status change Rev 1.3 Type 1

000342 DS200 Ballot Box BOM status change Type 1

000366 Ballot Box - Retrofit Stock Type 1

000375 Ballot Box carrying case drawings & status change Type 1

000423 Ballot Box -Shipping Type 1

000466 DS200 Ballot Box -Caster Bolt Type 1

000523 Double-coated Tape Type 1

000545 DS200 Image Scanner Cable labels Type 1

000554 Mylar tab Type 1

000562 DS200 Mounting knurling motor process Type 1

000566 DS200 Labels, screw head & clamp size Type 1

000570 DS200 Wire color and screw head changes Type 1

000582 Printer door corner shape and drawings Type 1

000618 Part Number Labels Type 1

*000628 & 000674 Plastic shield for power cord storage Type 1

000665 DS200 Ballot Box Diverter Extender- field retro fit Type 1

000669 DS200 Internal Ballot tub Type 1

836 Steel Ballot Box -Security Pin Type 1

*837 COTS Thumb drive cover Type 1

*838 COTS Thumb drive controller Type 1

839 DS200 -label for compact flash Type 1

845 Steel Ballot Box -Caster inner bearing Type 1

846 DS200 document part number Type 1

851 USB Part Number Type 1

Type 2

000332 DS200 Ballot Box new lock Type 2

000339 DS200 Ballot Box carry case- washer & rivet to hold foam Type 2

000359 DS200 Adding metal bottom edge (BOM & engineering status change) Type 2

841 DS200 EOL Sensor, Power Switch & Capacitor Type 2

843 Steel ballot box -Diverter cable Type 2

844 DS200 EOL parts - diode Type 2

847 DS200 Alternate LCD Backlight Inverter Type 2

000529 DS200 cable, switch, bracket Type 2

000534 DS200 Clamp to chassis Type 2

000535 DS200 Clamps Chassis Tape & holes Type 2

000576 DS200 EOL SMT Power Inductors Type 2

* These ECOs were incorporated into the hardware tested functionally, as identified in Table 7. 
EAC Certification # pending

Page 19 of 34 iBeta Report Number: (V)2010-30Jun-001(B)

5 Certification Review and Test Results

The results and evaluations of the PCA and FCA reviews and tests are identified below. Detailed data regarding the 

Acceptance/Rejection criteria, reviews and tests are found in the appendices.

Appendix A identifies all certification test requirements traced to specific Test Cases

Appendix B identifies the PCA Source Code Review Acceptance/Rejection criteria

Appendix C identifies the PCA TDP Document Review Acceptance/Rejection criteria

Appendix D identifies all FCA Testing Acceptance/Rejection criteria

Appendix E identifies the PCA and FCA Discrepancies reported during review and testing

5.1 PCA Source Code Review

iBeta Quality Assurance reviewed the changes to the DS200 to the VVSG Vol. 1 Sect 5.2 and Vol. 2 Sect. 5, interpreted for 

C/C++ conventions and standards. Source code review issues were identified in a discrepancy report delivered to ES&S. 

Issues identified included missing header and variable declaration comments and undefined constants. These issues were 

reported to ES&S on source code review discrepancy reports. All source code resolution resubmissions from ES&S (BUG 

17375 and 17664 submitted to both Unity 3.2.0.0 Revision 1 and Unity 3.2.1.0) were reviewed and validated. iBeta confirms 

issues were addressed prior to closing the discrepancy.

5.1.1 C and C++ DS200 with Scanner_C85051 Source Code Review Results

Changes to the DS200 were limited to the C/C++ components. The ESSUNITY3200 baseline was modified during the 3.2.1.0 

test effort. The identical results of that review were used in Unity 3.2.0.0 Revision 1. There were a total of 279 changed 

files/functions. Each of the files/functions was 100% reviewed for conformance to the VVSG 2005. Eleven modules with 

instances of non-conformance were reported to ES&S. All discrepancies were comment related. None of the discrepancies 

were against any of the software related VVSG 2005 requirements. ES&S submitted revised code and iBeta validated that all 

discrepancies were resolved.

The file function line count results identified 32374 lines of code:

no files or functions exceeded 240 eLOCs

3.34% were between 60 and 120 lines

.23% were between 120 and 240 lines

the remaining 96.43% were less than 60 lines. 

The source code was found to meet the requirements of the VVSG 2005. The data supporting this review are found in 

Appendix B.

5.2 PCA TDP Document Review

ES&S submitted documentation changes. iBeta Quality Assurance Identified the changes within the TDP to determine if there 

was any impact on the PCA Document Review performed in the ESSUNITY3200 baseline or if the changes in the document 

require verification of a process or functional description of the voting system. It was found that the changes had no impact on 

the ESSUNITY3200 baseline PCA Document Review results found in the certification test report. No additional review was 

required.

5.3 FCA DS200 Functional and System Level Testing

Software changes to the DS200 Firmware were submitted to both Unity 3.2.1.0 and 3.2.0.0 Rev 1. The Unity 3.2.0.0 Revision 

1 Election Management Software (EMS) is unchanged from the certified ESSUNITY3200.

ES&S submitted DS200 Firmware v.1.4.3.0 containing enhancements and a newer version of HPM and ERM in the Unity 

3.2.1.0 test effort. These changes were tested in Unity 3.2.1.0. In that test effort a single test case was created with seven 

scenarios. The election database was created in 3.2.1.0 using the unchanged ESSUNITY3200 baseline versions of AM, EDM 

and ESSIM and then tested with the updates to HPM and ERM.

In the Unity 3.2.0.0 Rev. 1 test effort iBeta restored a PC with the unchanged ESSUNITY3200 baseline EMS software. As the 

identical changes were submitted to both test efforts, the Unity 3.2.1.0 DS200 Functional Test Case and the election database 

were reused. The first three scenarios were repeated using the ESSUNITY3200 baseline HPM and ERM with the DS200 

v.1.4.3.0. Discrepancies 1 & 2 were encountered because the change in ERM submitted in Unity 3.2.1.0 addressed an Illinois 

Overvote reporting issue. The test case was designed to test the Illinois Overvote reporting issue within the ERM in Unity 

3.2.1.0 by using an Illinois test election. ES&S submitted a revision to the System Overview which identified that Unity 3.2.0.0 

Revision 1 voting system does not support elections in Illinois or Hawaii. As a result the election database was modified to 

Idaho. Scenarios 1, 2 and 3 with the modified election location passed and were accepted.

Scenarios 4, 5 and 7 were restricted to DS200 functionality and did not require an end-to-end test for verification. Those 

scenarios were reviewed against the Unity 3.2.1.0 test results and the passed results were accepted. Scenario 6 dealt with 

detection of counterfeit ballots. This optional function was withdrawn in both test efforts due to a failure to detect a counterfeit 

ballot in testing in Unity 3.2.1.0.

A Document Review was conducted of ES&S testing of Field Issue 1 to confirm sufficiency and acceptance of ES&S testing.
EAC Certification # pending

Page 20 of 34 iBeta Report Number: (V)2010-30Jun-001(B)

The EAC has issued a Voting System Technical Advisory regarding a "power down" and "freeze" (Field Issue 2) which 

identifies handling recommendations. For more information see the Voting System Technical Advisory on ES&S Unity 3.2.0.0

posted on the EAC website

5.3.1 DS200 Functional and System Level Test Results

The test and review results of the DS200 changes were accepted. The testing was conducted on the system configuration 

identified in Section 3. Appendix D details specific information on the Functional Testing. Failures, errors, nonconformities 

and anomalies observed in testing are summarized in Appendix E. Documentation of corrections and verification of corrections 

is contained in each summary.

5.4 FCA Hardware Environmental Testing

The ES&S submitted engineering changes (ECOs) for the ESSUNITY3200 DS200 precinct scanner, the plastic ballot box/case

and steel ballot box for certification in Unity 3.2.1.0 and Unity 3.2.0.0 Revision 1. These changes fell into two groups: 

1. The administrative or hardware changes that don't impact electrical, transportation or storage: Administrative ECOs

covered documentation relevant to the manufacturing administrative process, label changes, production status 

changes, out of scope equipment, documentation updates and drawings. Along with hardware changes that don't 

impact electrical, transportation or storage, these ECO's were assessed and determined to require no hardware 

environmental testing. This included the ECOs 000315, 000337, 000342, 000366, 000375, 000423, 000466, 000523, 

000545, 000554, 000562, 000566, 000570, 000582, 000618, 000628, 000665, 000669, 000674, 836, 837, 838, 839, 

845, 846, and 851.

2. The hardware changes that may impact electrical, transportation or storage: Generally addressed end-of-life parts 

and second source suppliers that could impact electrical, transportation or storage VVSG requirements. These 

changes required additional assessment by a qualified electrical engineer.

The results of the assessments and the required testing are identified in the table below.

Table 13 DS200 ECO Hardware Assessments and Required Testing.

ECO # Description of DS200 ECOs Operating- EMC Non-operating 

Transportation & Storage

000332 DS200 Plastic Ballot Box new lock. 4.8.3 Electrostatic Disruption 

EN-61000-4-2

No testing required

000339 DS200 Ballot Box carry case add washer & 

rivet to hold foam during manufacturing 

No testing required No testing required

000359 DS200 Plastic Ballot Box adding metal 

bottom edge

4.8.3 Electrostatic Disruption 

EN-61000-4-2

No testing required

000529 DS200 Carry Case remove micro switch 

bracket and switch cable 

No testing required No testing required

000534 DS200 add clamps to chassis No testing required No testing required

000535 DS200 Tape and holes for attaching clamps, 

no change to wire routing

No testing required No testing required

000576 DS200 End of life SMT Power Inductor No testing required No testing required

841 DS200 add Rod Lens Array, capacitor, and 

protected power switch

All EMC Tests No testing required

843 Steel Ballot Box added a new diverter cable All except 4.8.8 Magnetic Fields 

Immunity EN-61000-4-8

No testing required

844 DS200 end-of-life capacitors, resistors and 

diode 

4.8.3 VSS Electro-static 

Disruption EN-61000-4-2

No testing required

847 DS200 alternate LCD Backlight Inverter All except 4.8.8 Magnetic Fields 

Immunity EN-61000-4-8

No testing required

5.4.1 FCA Hardware Environmental Tests

The DS200 electrical retesting was successfully completed. All tests were passed. Appendix D details specific information on 

the Hardware Environmental Testing. No failures or anomalies were observed.

Engineering changes that did not require environmental retest or any other functional testing were approved as de minimus by 

the EAC. See the ES&S Correspondence on the EAC website EAC Letter on De Minimus Changes to Unity 3.2.0.0.
EAC Certification # pending

Page 21 of 34 iBeta Report Number: (V)2010-30Jun-001(B)

6 Opinions & Recommendations

iBeta Quality Assurance has completed the testing of ES&S Unity 3.2.0.0 Revision 1 voting system.

Testing prescribed by the iBeta test plan or amended test plan was performed as identified. Documentation of any 

divergence from the test plan was included in the amended as run test plan (see Appendix H: Amended Test Plan).

All identified anomalies or failures were reported and resolved (see Appendix E - Discrepancy Report).

Questions with regard to iBeta's interpretation of the test scope were referred to the EAC. The EAC responses were 

incorporated into iBeta's verification of resolution.

The information provided in this report is an accurate representation of the ES&S Unity 3.2.0.0 Revision 1 voting 

system Certification test effort.

Based upon the findings identified in Section 5 Certification Review and Test Results, in our opinion the acceptance 

requirements of Election Assistance Commission Voluntary Voting System Guidelines December 2005 for the changes to the 

DS200 Precinct Count Scanner have been met. In stating this opinion, iBeta notes that the VVSG 2005 Appendix B.5 

identifies that "any uncorrected deficiency that does not involved the loss or corruption of voting data shall not necessarily be 

cause for rejection. Deficiencies of this type may include failure to fully achieve the levels of performances specified in Volume 

1". Section 2 Certification Test Background identifies Field Report 2. The issue identified has been documented by the EAC 

in the Voting System Technical Advisory Intermittent Freeze/Shutdowns with EAC Certified ES&S Unity 3.2.0.0 System. iBeta 

determined that the uncorrected operational deficiency (see Appendix A - VVSG requirement 2.1.5.1.b.i and ii) was permitted 

for recommendation after emails from ES&S and the EAC confirmed that the documented field reports did not identify loss or 

corruption of vote data.

Based upon the test findings, it is our opinion that the acceptance requirements of the VVSG 2005 have been met for the 

DS200 changes submitted for certification testing.

iBeta Quality Assurance recommends that the Election Assistance Commission certifies the DS200 changes submitted in the 

ES&S Unity 3.2.0.0 Revision 1 voting system.

QA Director Voting

June 30, 2010

Note: The system configuration of the certified ESSUNITY3200 explicitly excludes connection to any network (public or private). 

Modification of the hardware or operating system configuration to permit connection to a network invalidates the 

recommendations of this test report.

Unity 3.2.0.0 Revision 1 recommendation excludes Illinois and Hawaii because state specific overvote counting definitions 

prevented the upload of some precincts into this version of ERM during testing. There was no issue with the DS200 identifying 

overvotes.
EAC Certification # pending

Page 22 of 34 iBeta Report Number: (V)2010-30Jun-001(B)

7 APPENDICES: TEST OPERATION, FINDINGS & DATA ANALYSIS

7.1 Appendix A: Certification Test Requirements

Appendix A identifies the test results to the Certification Test Requirement in the original certified ESSUNITY3200 to the VSS 

2002 of and the VVSG 2005 for the changes to the DS200 submitted in the Unity 3.2.0.0 Revision 1 voting system. It is 

contained in the document Appendix A, B, and G of the ES&S Unity 3.2.0.0 Revision 1 Voting System Certification Test Report 

for DS200 Modifications to the EAC Certified ESSUNITY3200.

7.2 Appendix B: PCA Source Code Review

Appendix B identifies the findings of the source code review of the changes to the certified ESSUNITY3200 baseline. It is 

contained in the document Appendix A, B, and G of the ES&S Unity 3.2.0.0 Revision 1 Voting System Certification Test Report 

for DS200 Modifications to the EAC Certified ESSUNITY3200.

7.3 Appendix C: PCA TDP Documentation Review

A comparison of the TDP changes submitted in Unity 3.2.0.0 Revision 1 to the certified ESSUNITY3200 VOTING system was 

performed. The comparison confirmed that the changes had no impact on the ESSUNITY3200 baseline PCA Document 

Review results. No additional PCA TDP Documentation Review was needed for Unity 3.2.0.0 Revision 1 test effort. A 

functional review of the documentation changes was conducted during the DS200 Functional Testing.

7.4 Appendix D: FCA Testing

7.4.1 FCA Functional and System Level Testing

Testing was conducted on the system configuration referenced in the test method. The software and hardware configuration 

is identified in the individual test case document.

Date Test Result Issues Opened Issues Closed Notes

6/21/2010 Rejected Disc # 1, 2 & 3 Scenario 1 Rev 0 - State code IL requires 

ERM update not submitted to Unity 3.2.0.0 

Revision 1

6/22/2010 Rejected Disc # 1, 2 & 3 Scenario 2 Rev 0 - State code IL requires 

ERM update not submitted to Unity 3.2.0.0 

Revision 1

6/24/2010 Accepted Scenario 1 Rev 1 - Retested with state code 

ID.

6/25/2010 Accepted Scenario 2 Rev 1 - Retested with state code 

ID.

6/25/2010 Accepted Scenario 4 - 7 Reuse testing in Unity 3.2.1.0

6/25/2010 Accepted Scenario 3

6/25/2010 Accepted Document Review Field Issue 1

6/29/2010 Accepted Disc # 3 Document Review of ERM SOP identifying 

Error message "Import process error"

6/29/2010 Accepted Disc # 1 & 2 EAC confirmed Unity 3.2.0.0 Revision 1 

restriction of scope excluding IL and HI

Full description of the issues encountered in testing is found in Appendix E- Discrepancy Report.

iBeta Definition DS200 Functional Test Method

Test Case Name DS200 Functional TC

Scope - identifies the 

type of test

The changes to the DS200 submitted in Unity 3.2.0.0 Revision 1 have also been submitted in the 

Unity 3.2.1.0 certification test effort. Seven test scenarios are listed within this test method. 

Scenarios 1, 2 & 3 require execution in Unity 3.2.0.0 Revision 1 to confirm that the changes to the 

DS200 integrate with the ESSUNITY3200 Election Management System. Scenarios 4 through 7 

require no additional testing because they are isolated to the DS200. The ES&S Unity 3.2.1.0 test 

results will be used. In Unity 3.2.1.0 the election creation was performed on a Peer-to-Peer 

configuration (multiple PCs). In Scenarios 1, 2 & 3 election data will be copied to the 

ESSUNITY3200 Election Management System Stand Alone configuration. New media will be 

burned, ballots will be scanned on the DS200 and election results will be read into ERM. Testing 

shall confirm the functional changes in the DS200 firmware v. 1.4.3.0: BUG15827, ENH14725, 

ENH14726, ENH14729, ENH14730, ENH14731, ENH14732, ENH14745, ENH15287, ENH15288, 

ENH15418, ENH15890, ENH15891, ENH15892, ENH16085, ENH16211, ENH16231, ENH16382, 

ENH15418, ENH17538, ENH17666, ENH16120, ENH16291, ENH16336, A Document Review was 
EAC Certification # pending

Page 23 of 34 iBeta Report Number: (V)2010-30Jun-001(B)

iBeta Definition DS200 Functional Test Method

conducted of ES&S testing of BUG16775 & BUG16782 (Field Issue 1) to confirm sufficiency.

(Note: Support of counterfeit ballot detection ENH15009 is withdrawn. Modem functionality is not 

supported in this certification effort (ENH14728); and Unity 3.2.0.0 Revision 1 does not support 

elections in Illinois and Hawaii.

Test Objective The objective is to validate the ES&S functional changes to the DS200 from the ESSUNITY3200

certified baseline. These changes were submitted in Unity 3.2.1.0. Scenario 1 through 3 shall 

confirm that the same results from Unity 3.2.1.0 are observed. Unity 3.2.1.0 results from document 

reviews and testing conducted in Scenarios 4 through 7 shall be used.

Test Variables: 

Volume

Stress

Performance

Error Recovery

Scenario 1:

Reuse the Illinois Open Primary without Party Preference election defined in the Unity 3.2.1.0 test 

effort with the following settings: (This was modified to Idaho as a result of discrepancies 1 and 2)

ENH16382•18 Precincts election day and 1 Polling Place

9 District Types (this does not including Countywide)

18 District Name (two contests to ac district name)

18 ballot style (each precinct has its own ballot style)

14" 36 "standard 14" ballot

2 Partisan contests per style

4 candidates per contest (2 candidates per party for the mayor contest, 4 candidates for the 

Senator contest (total of 216 candidates) 

Write-ins (1 for the Mayor and 2 for the Senator race)

DEM and REP Parties

ENH14745 & BUG15827•The Scanner Options

•Diverter is set for "Overvoted Write-ins & Blank Ballots".

•Ballot Control is set to Query for "Overvoted ballots, Cross voted ballots, Unreadable marks, 

Undervoted ballots and blank ballots"

ENH14725•The DS200 scanner options are set to "Do not save any ballot images" 

Run Election set up reports in EDM

Scenario 2: Same as Scenario 1 except:

ENH15287 •Changes "election day" to "early voting" in HPM by selecting the "Include all Precincts" 

option. The early voting option allows the Poll worker to view the Ballot Style Report.

ENH14725• The DS200 scanner options are set to "save all ballot images"

Scenario 3: Use Scenario 1 test results:

ENH16382•DS200 18 precincts Election Day can exceed the declared System Limit of 10 

precincts. 

Scenario 4: Reuse Unity 3.2.1.0 test results

ENG15418 •ES&S will provide ballots with speckling. Use the DS200 Functional S1 election. Each 

ballot will contain stray marks in the time track (left hand side of the ballot). Marks will emulate 

various levels of white speckling. Ballot will be read on DS200 with v. 1.3.10.0 and re-read on 

version 1.4.3.0

ENH14726 & ENH15288 •After scanning a valid ballot, time how long the "Thank you for voting" 

screen displays. Note the time. Measure the font (text) on the "Thank you for voting" screen 

display. Note the size. Change firmware version and repeat test. Compare times and sizes.

Scenario 5: Reuse Unity 3.2.1.0 test results

ENH17538 & 17666: •Only testing the Protective Counter (for Maryland). Do not need the ballot 

box. ES&S will provide an "updated" test FW version that with a different version number. To test, 

begin scanning ballots. Upgrade the firmware with the test FW provided by ES&S. Continue with 

voting. When the voting is complete, Verify the counter did not reset or lose count and the counter 

appears in the printed reports. Restore the DS200 to the original firmware version. Verify the 

counter did not lose count. 

•Note the counter number from the DS200 and the Initial State report

• Load the election and scan a few (2 to 5) ballots, close the polls and note the counter in the 

results report and audit log. 

•Upgrade the firmware; note the counter on the Initial State report. 

•Re-burn the media for the S1 election, load the election, and scan more ballots. 

•Close the polls. Examine the protected counter on the DS200, Audit log and the Results report. 

•Restore to the original firmware version. • Verify the protected counter did not lose count.

Scenario 6: ENH15009 (ES&S has withdrawn counterfeit ballot detection functionality)

Scenario 7: Use the Unity 3.2.1.0 test results

In Regression REV01 TC Scenario 1 the Overvote translations match ENH16291/16336/16120. 

A description of the 

voting system type & 

operational environment

The Unity 3.2.0.0 Revision 1 EMS is a single PC

• DS200 Precinct Count scanner 

• Steel ballot box with diverter

VVSG 2005 vol. 1 2.1.5.1.b I thru vii, c, 2.1.6, 2.1.7.2, 2.2.1, 2.2.1.3, 2.3.1 thru 2.3.1.2, 2.1.4.j, 2.2.4, 2.3.3.1.b,c,d, 
EAC Certification # pending

Page 24 of 34 iBeta Report Number: (V)2010-30Jun-001(B)

iBeta Definition DS200 Functional Test Method

2.3.3.2, 2.4.1.b, 2.4.3, 2.4.2 thru 2.4.3, 4.1.5.1.d, 4.1.6.1, 4.1.8 thru 4.1.8

VVSG 2005 vol. 2 6.2, 6.2.1, 6.2.2, 6.3, 6.6, 6.7

A4.3.5 Volume (maximum and exceeding more than the maximum number of precincts in a Polling 

Place)

A4.3.5 Volume/Stress (Processing, storing and reporting data when overloading the number of 

precincts in a Polling Place)

A4.3.5 Performance/Recovery (Ballot format handling capability-graceful shut down and recovery 

without loss of data)

A4.3.5 Performance/Recovery (Processing rates-graceful shut down and recovery without loss of 

data)

Hardware, Software 

voting system 

configuration and test 

location

Test Location: iBeta, Aurora, CO

The Unity 3.2.0.0 Revision 1 Voting System consists of the following: 

Audit Manager (AM), Election Data Manager (EDM), ES&S Ballot Image Manager (ESSIM), 

Hardware Program Manager (HPM), Election Reporting Manager (ERM), LogMonitor Service.

1 @ DS200 1-DS200 starting with version 1.3.10.0 and upgrade to version 1.4.3.0 (upgrade 

required for Scenario 5) 

1 Steel ballot box with diverter

Pre-requisites and 

preparation for 

execution of the test 

case. 

Complete the prerequisites: 

• Retrieve the election data databases archived from the DS200 Functional Test Case in Unity 

3.2.1.0, Scenario 2 Rev 02

• Copy the Unity contents from the archive for scenario 1 into the Standalone PC. 

• Copy the Unity and Elecdata contents from the archive for scenario 2 & 3 into the Standalone 

PC.

Getting Started Checks Getting Started: Complete the prerequisites;

Check the voting system to: 

• Verify the test environment and system configuration is documented in the PCA Configuration 

and matches the system used in the 48 hr. temp & power variation test and vendor described 

configuration.

Validate installation of the witnessed build. 

• Testers understand that no change shall occur to the test environment without documentation in 

the test record and the authorization of the project manager

• During testing an operational readiness test was performed. 

• The environment is set up as a standalone configuration. 

Documentation of Test 

Data & Test Results

Test Data: 

- Record all programmed & observed election, ballot & vote data fields and field contents on the 

corresponding tabs to provide a method to repeat the test. 

- Preserve all tabs for each instance the test is run.

Test Results: 

- Enter Accept/Reject on the Test Steps. 

- In Comments enter any deviations, discrepancies, or notable observations. 

- Log discrepancies on the Discrepancy Report and insert the discrepancy number in the 

Comments field of Test Step.

Volume: Paper-based 

voting systems

Processing

Ballot Prep:

Scenario 1) Create DS200 media with the previously defined Test Variables for Scenario 1

Scenario 2) Same as scenario 1 except early voting and ballot images are saved

Scenario 3) Use scenario 1 test results to confirm exceeding the declared system limit of 10 

precincts do not cause system errors or if there are any system errors that cause the EMS ballot 

preparation applications to crash then verify the applications recover without any loss of data. If no 

error is given prior to leaving HPM continue the test through ERM; verify there are no system 

errors.

Scenario 4, 5 & 7) Reuse of Unity 3.2.1.0 testing results

Scenario 6) ENH15009 (ES&S has withdrawn counterfeit ballot detection functionality).

Volume: Scenario 3: ENH16382•Use scenario 1 test results to confirm when exceeding the declared 

system limit of 10 precincts Overloading systems capacity to process, store, and report data. ): 

• Overloading the HPM with more than the declared number of precincts in a single polling place.

Stress Scenario 3: ENH16382•Use scenario 1 test results to confirm when exceeding the declared 

system limit of 10 precincts the system responses to overloading conditions

Performance Scenario 3: ENH16382•Use scenario 1 test results to confirm when exceeding the declared 

system limit of 10 precincts there is no system degradation (Ballot format handling capability and 

Processing rates): 

• The system does not slow down throughout the testing
EAC Certification # pending

Page 25 of 34 iBeta Report Number: (V)2010-30Jun-001(B)

iBeta Definition DS200 Functional Test Method

Error Recovery Voting system gracefully shuts down (no crash) and recovers from errors caused by overloading 

the number of precincts and ballots styles. 

• Processing capabilities-graceful shut down and recover without loss of data

• Critical Status Messages

Readiness Testing and 

Poll Verification

Scenario 1: Voting system is ready for the election: 

Verify the voting system is ready for the election: 

- The election is correctly installed (Election ID, polling place name, precincts) . 

- Test data (run a precinct to validate the system is ready) is segregated from voting data, with no 

residual effect. 

Test confirmation that there are: 

- No hardware/software failures. 

- The device is ready to be activated to accept votes (No Identification of any failures & corrective 

action)•Run 1 precinct to validate the system is ready; confirm the test data is segregated from 

voting data, with no residual effect. Verify totals and audit logs. 

ENH14729•"Polls Opened Menu": Print the zero reports (first of 3 zero reports prior to opening of 

polls) 

ENH16231•a "Failure" error and a "Successful" message are displayed on the DS200; the audit log 

displays the Failure and Successful PW login attempts.

ENH14730•2 beeps and a "successful" message displays once the scanner has been calibrated 

(Calibrate scanner option).

ENH14730 (15891, 15890, 15892)•Calibrate the counterfeit sensor; 2 beeps and a "successful" 

message displays (testing that only the counterfeit sensor can be set up as required, a message is 

displayed as well as 2 beeps can be heard once the counterfeit sensor has been successfully 

calibrated).

ENH14729•The zero report can be printed. (2nd of 3 zero reports prior to opening the polls) The 

report continually displays zeros for the contest, candidates, precincts (all 18), under/overvotes and 

Write-Ins.

ENH15287•The "Polls Opened Menu" doesn't display a "Ballot Style Report" button. (this displays 

only for Early Voting)

• Scan pre-election test ballots

ENH14745 • Select the options to override Overvoted ballots and Blank ballots only, do not select 

any other options. Overvoted and Blank ballots will not prompt an "Accept" or "Reject" message to 

the voter. The ballots will automatically be accepted due to the override. The Cross voted, 

Undervoted and unreadable ballots will provide a prompt to "Accept" or "Reject" and the voter will 

be required to make a selection 

•Tally pre-election test ballots; ballots match the predicted results outlined in the test case

ENH14732 & ENH16211 • run the results tape; at the end of the Results tape and audit log they 

display the "Machine ID" and "Poll Number"

Scenario 2:

Reuse the same election as Scenario 1 except with the changes noted in the Scenario 2 Voting 

Variations

ENH15287• Verify the Ballot Styles Report button on the "Open Polls Menu" screen displays in the 

upper right hand of the icon bar and the report is accurate.

ENH14745 Reset "Override" options: Selection the options to override (undervotes and crossover 

ballots) Verify ballots will automatically be accepted due to the override and "Overvoted W/I 

ballots", "Unreadable marks", and "Blank Ballots" will provide a prompt to "Accept" or "Reject" and 

the voter will be required to make a selection 

ENH14730 • Unplug the DS200 from the ballot box verify 2 beeps are heard. Print a zero report 

and open printer door while printing is occurring. Verify printing will stop and DS200 will beep two 

times

Scenario 3:

•Verify scenario 1 is correctly handled by the system with 18 precincts (exceeding the declared 

system limit of 10 precincts).

Scenario4, 5 & 7 Reuse Unity 3.2.1.0 test results

Scenario 6: ENH15009 ES&S has withdrawn the detection of counterfeit ballots.

Pre- vote:

Opening the Polls 

Verification

Verify the polling place voting system for scenario 1, 2 & 3:

- Zero count report has no results. All test results have been zeroed out during readiness testing.

- Election identification including, Election Name/ID, Precinct ID/Name, Firmware Version

Voting: 

Ballot Activation and 

Casting Verifications

Scenario 1: Using the predetermined vote pattern, reuse previously marked ballots from Unity 

3.2.1.0 and scan ballots.

ENH16231 • attempt to reopen Polls using an incorrect and a correct password. Verify message 

appears on DS200 and in audit log.

ENH14729 •Clear pre-election readiness test totals and run another zero report

ENH14745 •Reset the "Override" options. Selecting the options to override Overvoted ballots and 

Blank ballots only, do not select any other options. Overvoted and Blank ballots will not prompt an 
EAC Certification # pending

Page 26 of 34 iBeta Report Number: (V)2010-30Jun-001(B)

iBeta Definition DS200 Functional Test Method

"Accept" or "Reject" message to the voter. The ballots will automatically be accepted due to the 

override. The Cross voted, Undervoted and unreadable ballots will provide a prompt to "Accept" 

or "Reject" and the voter will be required to make a selection 

ENH14725 & ENH16085 •View the icon bar for each of the icons on the "Welcome screen". Verify 

an X appears on over the disk icon to indicate no images are being saved. Verify ballot status, 

power status, image status, election definition and open polls icons are displayed.

ENH16382 •vote ballots for all 18 precincts and verify the ballots for all 18 precincts are accepted 

without a precinct error

ENH14731 •Verify that there is a beep as each ballot is accepted

ENH15288 • Measure and verify the "Thank you for voting" text is larger. (Compare using old 

v.1.3.10.0 and new v.1.4.3.0 DS200 firmware)

ENH14726 •Time and verify the "Thank you for voting" screen displays 3 to 5 seconds longer. 

(Compare using old v.1.3.10.0 and new v.1.4.3.0 DS200 firmware)

ENH14725 •The Welcome screen displays the Disk icon (on the icon bar) has a small red X (not 

saving ballot images)

ENH16085 • Verify the Welcome screen displays the following on the icon bar, "Battery Status 

Indicator Icon, AC Power Status Indicator Icon, Image Saving Status Indicator Icon, Election 

Definition Status Icon"

Scenario 2: 

ENH14725 •View the icon bar for each of the icons on the "Welcome screen". Verify no X appears 

on over the disk icon to indicate images are being saved. 

All early voting ballots can be scanned without error into the correct ballot bin

Scenario 3: 

ENH16382•Use scenario 1 test results to confirm exceeding the declared system limit of 10 

precincts

Scenario 4: Reuse Unity 3.2.1.0 test results

ENG15418 •ES&S will provide ballots with speckling. Each ballot will contain stray marks in the 

time track (left hand side of the ballot). Marks will emulate various levels of white speckling. Ballot 

will be read on DS200 with v. 1.3.10.0 and re-read on version 1.4.3.0. 

•Scan ballots

ENH14726: After scanning a valid ballot, time how long the "Thank you for voting" screen displays. 

Note the time.

ENH15288: After scanning a valid ballot, measure the font (text) on the "Thank you for voting" 

screen displays. Note the size.

Change the firmware version on the DS200 and scan ballots

ENH14726: After scanning a ballot, time how long the "Thank you for voting" screen displays. Note 

the time.

ENH15288: After scanning a ballot, measure the font (text) on the "Thank you for voting" screen 

displays. Note the size.

Voting: 

Voting System Integrity, 

System Audit, Errors & 

Status Indicators

The system audit provides a time stamped, always available, report of normal/abnormal events 

found within the percentage of sampled test 

Error messages are:

- Generated, stored and reported as they occur. 

- Errors requiring intervention by the voter or poll worker are clearly display issues and action 

instructions in easily understood non-technical text language or with indicators. 

- The text for any numeric codes is contained in the error or affixed to the inside of the voting 

system. 

- Incorrect responses will not lead to irreversible errors. 

- Nested conditions are corrected in the sequence to restore the system to the state before the 

error occurred

Status Messages: 

- Display and report critical status messages are using unambiguous indicators or English 

language text. 

- Non-critical status messages are displayed but do not have to be at the time of occurrence and 

may be numerical codes for subsequent interpretation and reported in unambiguous text.

- Status messages are part of the real-time audit record. 

Scenario 4: Reuse Unity 3.2.1.0 test results

ENH15418 •Using a ballot with an estimated 10% of the timing track scratched out. Scan each 

ballot of the older DS200 version (v. 1.3.10.0) and on the new version. The old version will display 

an error the new version will allow the scanning of the ballot. 

ENH15418 •Using a ballot with an estimated 50% of the timing track scratched out. Scan each 

ballot of the older DS200 version (v. 1.3.10.0) and on the new version. The old version will display 

an error the new version will allow the scanning of the ballot. 
EAC Certification # pending

Page 27 of 34 iBeta Report Number: (V)2010-30Jun-001(B)

iBeta Definition DS200 Functional Test Method

ENH15418 •The white marks (speckles) in the ballot time track will display error "BALLOT 

DRAGGED/Turn Ballot Over and Try Again" on version 1.3.10.0. (Cause: The ballot did not enter 

the feed mechanism smoothly, which caused misalignment during scanning.) . 

End of Scenario 4 testing

Scenario 5: Reuse Unity 3.2.1.0 test results. 

ENH17538 & 17666:

• Scan a few (2 to 5) ballot, close the polls and note the counter in the results report and audit log. 

•Upgrade the firmware and note the counter on the Initial State report. 

•Re-burn the media for the S1 election, load the election, and scan more ballots. •Close the polls. 

•Examine the protected counter on the DS200, Audit log and the Results report.

•Restore to the original firmware version and verify the protected counter did not lose count. 

End of Scenario 5 testing

Scenario 6: ENH15009 ES&S has withdrawn the detection of counterfeit ballots.

Scenario 7 - Reuse results from Unity 3.2.1.0 to confirm ENH16291/16336/16120 Overvote 

translations have been updated.

•You have made too many selections in 1 contest! / ¡Ha realizado demasiados selecciones en 1 

competencia!

•You have made too many selections in N contests! / ¡Ha realizado demasiados selecciones en N 

competencias!

•The contest with too many votes is - / La contienda que tiene demasiados votos es -

•See a poll worker for help. / Vea a un trabajador de votación para obtener ayuda

•Press the button below to return your ballot. / Presione el botón de abajo para que se le regrese 

su boleta.

•See a poll worker for a replacement ballot. / Pida a un trabajador electoral una boleta de 

reposición. 

•Warning! If you press the button below your ballot will be cast and the contest(s) with too many 

votes will not be counted! /¡Advertencia! ¡Si presiona el botón de abajo su boleta será emitida y 

las contiendas que tienen demasiados votos no se contarán. 

•One of the contests with too many votes is: / Una de las competencias con demasiados votos es:

•Your ballot will be returned to you. / Se le devolverá su boleta.

•Correct your ballot/Corrija Su Boleta or "Cast your ballot with errors/Emitir su boleta con errores" 

End of Scenario 7 testing.

Document Review of ES&S Testing. 

BUG16775 & BUG16782 (Field Issue 1)

Review the ES&S assessment, resolution and testing regarding the DS200's sporadic reporting of 

a mark in row 44 and row 45 of column D (back of the ballot) when no actual mark was present. 

Review the assessment of the issue on the DS200 FW (v.1.3.10.0) to confirm that it identified: 

• A very narrow set of specific variables required to generate the error. 

• ES&S demonstrated they were able to consistently and reliably repeat the error; and. 

• ES&S' resolution was consistent with the VVSG. 

Review the resolution testing to confirm that the testing included all conditions and was sufficient to 

accept without additional testing by iBeta.

Variables Identified: Location of the contest from the vertical timing tracks, proximity to the top or 

bottom of the ballot, left side of the column oval placement, extension of the text next to the oval 

the edge of the allowable print area specified in the print layout manuals, and insertion in a skewed 

fashion of ballots with no selection on the last contest in column D.

Resolution: Scanner channel limit adjustments such that ballots with these unique variables 

inserted in a skewed manner will be rejected and require reinsertion.

Post-vote:

Closing the Polls

Scenario 1: Once the polls are closed the voting system

• Printed reports of ballots counted by tabulator

• Reported votes match predicted votes from tabulator with votes and undervotes.

• In the polling place print the summary report with all of the 18 precincts in a single polling place. 

ENH16211 •Cancel printing of audit log only and view the log displays for the "Machine ID" and 

"Poll Number". 

Print audit log

• totals match the predicated reports (using the vote tab)

ENH16211 •Audit Log stops printing after cancellation and displays the "Machine ID" and "Poll 

Number" at the end of the Audit Log

•the Audit Log can be re-printed. The audit log will display the history of this election. The pretest 

and the election day audit log matches the pre-election activities outlined above.

ENH15827 •Overvoted Write-in ballots and blank ballots were separated from the other ballots
EAC Certification # pending

Page 28 of 34 iBeta Report Number: (V)2010-30Jun-001(B)

iBeta Definition DS200 Functional Test Method

Scenario 2 

• Same as Scenario 1 excluding the listed enhancements 

Scenario 3: 

ENH16382•Use scenario 1 test results to confirm exceeding the declared system limit of 10 

precincts

Scenario 6: ENH15009 (withdrawn counterfeit ballot detection), 

Post-vote:

Central Count

Scenario 1: ERM Zero report is printed and no totals display on the report prior to reading in the 

results.

ERM: Vote Consolidation:

ENH14725 • attempt to upload DS200 ballot images and a message displays stating no images 

saved. Ballot images from the DS200 CANNOT be extracted/ viewed. The image was not saved in 

HPM. (Ballot images not saved was set in Test Variables)

•Votes match predicted votes (compare to vote tab. Vote tab is what was used to create paper 

ballots) 

• reports will display election identification

• EL30A - Precinct Report–Group Detail, individual precincts & contest results. 

• Precinct Report contains votes, undervotes & overvotes

• EL45- Election Summary, total number of votes for each candidate/question 

• Verify DS200 SN is displayed in ERM, once the USB flash drive is read into ERM

• Ensure audit logs are accurate & complete. 

Scenario 2: Same as Scenario 1 except ENH14725 • Upload DS200 ballot images. Ballot images 

from the DS200 can be extracted. The image was saved in HPM. (Ballot images saved was set in 

Test Variables) 

Scenario 3:

ENH16382 • Use scenario 1 test results to confirm exceeding the declared system limit of 10 

precincts

Expected Results are 

observed

Review the test result against the expected result:

• Accept: the expected result is observed

• Reject: the expected result of the test case is not observed

• Not Testable (NT): rejection of a previous test step prevents execution of this step, or tested in 

another TC. 

• Not Applicable (NA): not applicable to test scope

Record observations 

and all input/outputs for 

each election;

All inputs, outputs, observations, deviations and any other information impacting the integrity of the 

test results will be recorded in the test case. 

• Any failure against the requirements of the EAC guidelines will mean the failure of the system. 

And shall be reported as such

• Failures will be reported to the vendor as Defect Issues in the Discrepancy Report. 

• The vendor shall have the opportunity to cure all discrepancies prior to issuance of the 

Certification Report. 

• If cures are submitted the applicable test will be rerun. Complete information about the rerun test 

will be preserved in the test case. The cure and results of the retest will be noted in the -

Discrepancy Report and submitted as an appendix of the Certification Report. 

• Operations which do not fail the requirements but could be deemed defects or inconsistent with 

standard software practices or election practices will be logged as Informational Issues on the 

Discrepancy Report. It is the vendor's option to address these issues. Open items will be 

identified in the report.

7.4.2 FCA Hardware Environmental Testing

The DS200 v.1.4.3.0 engineering changes submitted in Unity 3.2.0.0 Revision 1 have also been submitted to the Unity 3.2.1.0 

certification test effort. As the changes are identical iBeta has used the common results. in both test efforts.

Date Test Result Issues Opened Issues Closed Notes - DS200 & Ballot Boxes

11/19/2009 Accept Assessment of ECO 000332:

• Non-operating Tests: Change has no impact on 

transportation or storage. Reuse Unity 3.2.0.0.

• Operating Tests: Change warrants repeating 4.8.3 

VSS Electrostatic Disruption EN-61000-4-2

11/19/2009 Accept Assessment of ECO 000339:

• Non-operating Tests: Change has no impact on 

transportation or storage. Reuse Unity 3.2.0.0

• Operating Tests: Change has no impact on OperatingEMC. Reuse Unity 3.2.0.0

11/19/2009 Accept Assessment of ECO 000359:
EAC Certification # pending

Page 29 of 34 iBeta Report Number: (V)2010-30Jun-001(B)

Date Test Result Issues Opened Issues Closed Notes - DS200 & Ballot Boxes

• Non-operating Tests: Change has no impact on 

transportation or storage. Reuse Unity 3.2.0.0

• Operating Tests: : Change warrants repeating 4.8.3 

VSS Electrostatic Disruption EN-61000-4-2

11/20/2009 Accept Assessment of ECO 000529:

• Non-operating Tests: Change has no impact on 

transportation or storage. Reuse Unity 3.2.0.0

• Operating Tests: Change has no impact on OperatingEMC. Reuse Unity 3.2.0.0

11/11/2009 Accept Assessment of ECO 000843:

• Non-operating Tests: Change has no impact on 

transportation or storage. Reuse Unity 3.2.0.0

• Operating Tests: Change warrants repeating all EMC 

tests & 4.1.2.6 VVSG Electrical Fast Trans EN-61000-4-4.

11/20/2009 Accept Assessment of ECO 000841:

• Non-operating Tests: Change has no impact on 

transportation or storage. Reuse Unity 3.2.0.0

• Operating Tests: : Change warrants repeating all EMC 

test except 4.8.8 VSS Magnetic Fields Immunity EN61000-4-8 & 4.1.2.6 VVSG Electrical Fast Trans EN61000-4-4.

11/20/2009 Accept Assessment of ECO 000844:

• Non-operating Tests: Change has no impact on 

transportation or storage. Reuse Unity 3.2.0.0

• Operation Tests: : Change warrants repeating 4.8.3 

VSS Electrostatic Disruption EN-61000-4-2

11/20/2009 Accept Assessment of ECO 000847:

• Non-operating Tests: Change has no impact on 

transportation or storage. Reuse Unity 3.2.0.0.

• Operating Tests: Change warrants repeating all EMC 

tests & 4.1.2.6 VVSG Electrical Fast Trans EN-61000-4-4.

1/26/2010 Accept Assessment of ECO 000534:

• Non-operating Tests: Change has no impact on 

transportation or storage. Reuse Unity 3.2.0.0.

Non-operating Tests: Change has no impact on 

Operating- EMC. Reuse Unity 3.2.0.0.

1/25/2010 Accept Assessment of ECO 000535:

• Non-operating Tests: Change has no impact on 

transportation or storage. Reuse Unity 3.2.0.0.

Non-operating Tests: Change has no impact on 

Operating- EMC. Reuse Unity 3.2.0.0.

1/15/2010 Accept Assessment of ECO 000576:

• Non-operating Tests: Change has no impact on 

transportation or storage. Reuse Unity 3.2.0.0.

Non-operating Tests: Change has no impact on 

Operating- EMC. Reuse Unity 3.2.0.0.

8/11/2009 -

5/11/2010

Accept Assessment of ECO 000315, 000337, 000342, 000366, 

000375, 000423, 000466, 000523, 000545, 000554, 

000562, 000566, 000570, 000582, 000618, 000628, 

000665, 000669 000674, 836, 837, 838, 839, 845, 846, 

and 851

• Non-operating Tests: Change has no impact on 

transportation or storage. Reuse Unity 3.2.0.0.

Non-operating Tests: Change has no impact on 

Operating- EMC. Reuse Unity 3.2.0.0.

12/15/2009 Accept 4.8.1 VSS Power Disturbance EN-61000-4-11-

successfully executed for ECO#:841, 843, 847

12/11/2009 Accept 4.8.2 VSS Electromagnet Radiated & Conducted 
EAC Certification # pending

Page 30 of 34 iBeta Report Number: (V)2010-30Jun-001(B)

Date Test Result Issues Opened Issues Closed Notes - DS200 & Ballot Boxes

Emissions FCC Part15B successfully executed for 

ECO#:841, 843, 847

12/11/2009 Accept 4.8.3 VSS Electrostatic Disruption EN-61000-4-2 

successfully executed for ECO#:332, 359, 841, 843, 844, 

847

12/12/2009 Accept 4.8.4 VSS Electromagnetic Susceptibility EN-61000-4-3 

successfully executed for ECO#:841, 843, 847

12/16/2009 Accept 4.8.6 VSS Lightening Surge EN-61000-4-5 successfully 

executed for ECO#:843, 841, 847

12/15/2009 Accept 4.8.7 VSS Conducted RF Immunity EN-61000-4-6 

successfully executed for ECO#:843, 841, 847

12/16/2009 Accept 4.8.8 VSS Magnetic Fields Immunity EN-61000-4-8 

successfully executed for ECO#:843, 847

03/25/2010 Accept 4.1.2.6 VVSG Electrical Fast Trans EN-61000-4-4

successfully executed for ECO#:841, 843, 847

The test method description incorporates the assessment, test conditions and details as it was set up in the Unity 3.2.1.0 

Environmental Hardware Test Case and the engineering changes are listed in Section 4.2 DS200 Engineering Changes

Method Detail Environmental Test Method

Test Case Name Environmental Test Case 

Scope - identifies the 

type of test

Assessment and testing of the hardware of the DS200 from ESSUNITY3200:

Identify and assess hardware changes from the certified baseline and engineering change orders to 

determine the extent of testing required, including execution and the provision of test results as 

required.

This set of hardware environmental test cases is outside the scope of iBeta's VSTL accreditation. 

Electrical testing was performed by Criterion Laboratories NVLAP #100396-0 (Electrical) with 

supervision of testing by iBeta. iBeta reviews and documents test records, results and reports to 

confirm testing was performed under an appropriate mode as a voting system and to determine 

acceptance or rejection of some or all testing.

Test Objective Validation of the Unity 3.2.0.0 Revision 1 (identical to Unity 3.2.1.0) DS200 hardware to meet the 

Non-Operating/Operating Environmental test standards of the EAC VSS 2002/VVSG 2005, 

including: 

DS200 Assessment of the ECOs from the ESSUNITY3200 baseline to verify reuse of the NonOperating Transportation and Storage test results from ESSUNITY3200 and test execution of the 

Operating Electrical tests.

Test Variables: Use the test results from testing in Unity 3.2.1.0: 

DS200 - Electrical ECOs from the ESSUNITY3200 baseline (identical to Unity 3.2.1.0) impact on 

operation

Power disturbance disruption - IEC 61000-4-11 (1994-06). 

Electromagnetic radiation- FCC Part 15 Class B requirements - ANSI C63.4. 

Electrostatic disruption - IEC 61000-4-2 (1995-01). 

Electromagnetic susceptibility - IEC 61000-4-3 (1996). 

Electrical fast transient protection - IEC 61000-4-4 (1995-01).(2004-02)

Lightning surge protection - IEC 61000-4-5 (1995-02). 

RF immunity - IEC 61000-4-6 (1996-04). 

AC magnetic fields RF immunity - IEC 61000-4-8 (1993-06).

DS200, reuse the certified baseline in the Unity 3.2.0.0 Test Report- (ECOs have no impact)

MIL-STD810-D:

High temperature method 501.2 Procedures I-Storage maximum 140 F degrees 

Low temperature method 502.2, Procedure I-Storage minimum -4 F degrees 

Temperature & power variations method 501.2 & 502.2 

Humidity method 507.2 

Vibration method 514.3-1 Category 1 Basic Transportation Common Carrier 

Bench handling method 516.3 procedure VI 

Safety OSHA CFR Title 29, part 1910
EAC Certification # pending

Page 31 of 34 iBeta Report Number: (V)2010-30Jun-001(B)

Method Detail Environmental Test Method

A description of the 

voting system type and 

the operational 

environment

Precinct Count scanner/tabulator: Model 200 (DS200)

VSS 2002 vol. 1 3.2.2 thru 3.2.2.14, 3.4.8

VSS 2002 vol. 2 4.6.1.5 thru 4.7.1 & 4.8 RFI 2008-01, 2008-05, 2008-06, 2008-09, 2008-10

Hardware, Software 

voting system 

configuration and test 

location

DS200 Electrical Testing Test Location: Criterion Labs, Rollinsville CO

iBeta provided Criterion with the environmental hardware test case outlining methods for 

preparation of their test plan; iBeta documented the configuration, test environment, lab 

accreditations, tester qualifications, and operational status check performance

iBeta personnel execute the operational status checks and operate the equipment as a voting 

system during the EMI/EMC test execution. 

Pre-requisites and 

preparation for execution 

of the test case. 

DS200 Electrical Testing: Complete the prerequisites;

Validation and documentation of the subcontractor test labs' NVLAP accreditation in the 

specific test method identified in the Test Variables

Record the testers & date

System has been set up as identified in the user manual

Gather any necessary materials or manuals. 

Ensure customization of the test case template is complete

The iBeta approved Operational Status Check script is provided that includes:

o Checking the operation of all buttons, switches and lights

o Opening the polls & running a zero totals report

o Checking appropriate error conditions for correct prompts or responses. (Error 

conditions will depend upon the type of equipment being tested.)

o Accessibility features are operational.

o Power off and on with no loss of function.

Close the polls and print all reports. (Totals & Audit Logs)

Getting Started Checks DS200 Electrical Testing: Check the voting system to: 

Verify the test environment and system configuration is documented in the PCA 

Configuration and matches the vendor described configuration. 

Validate installation of the Trusted Build

Testers understand that no change shall occur to the test environment without 

documentation in the test record and the authorization of the project manager

Confirm the tester understands the recording requirements of the iBeta test case

Operational status check procedures are available and successfully run

An automated script to loop system operation for use during the EMC operational tests exercises all 

necessary functionality.

Documentation of Test 

Data & Test Results

DS200 Electrical Testing: Test Results: 

• Enter Accept/Reject on the Test Steps

• In Comments enter any deviations, discrepancies, or notable observations

Log discrepancies on the Discrepancy Report and insert the number in the Comments

Standard Environmental 

Tests

DS200 Electrical Testing: Follow the test methods in all of the international electrical standards 

listed above to executed the EMC tests 

Expected Results are 

observed

DS200 Electrical Testing: Review the test result against the expected result: 

• Accept: the expected result is observed

• Reject: the expected result of the test case is not observed

• Not Testable (NT): not testable; provide a reason in the comments

• Not Applicable (NA): not applicable to test scope

Record observations and 

all input/outputs for each 

election;

DS200 Electrical Testing: All test results will be recorded in the test case

Any failure against the requirements will mean the failure of the system and shall be reported as 

such. Failures will be reported to the vendor as Defect Issues in the Discrepancy Repot. The 

vendor shall have the opportunity to cure all discrepancies prior to issuance of the Certification 

Report. If cures are submitted the applicable test will be rerun. Complete information about the 

rerun test will be preserved in the test case. The cure and results of the retest will be noted in the 

Discrepancy Report and submitted as an appendix of the Certification Report

Operations which do not fail the requirements but could be deemed defects or inconsistent with 

standard software practices or election practices will be logged as Informational Issues on the 

Discrepancy Report. It is the vendor's option to address these issues. Open items will be 
EAC Certification # pending

Page 32 of 34 iBeta Report Number: (V)2010-30Jun-001(B)

Method Detail Environmental Test Method

identified in the report
EAC Certification # pending

Page 33 of 34 iBeta Report Number: (V)2010-30Jun-001(B)

7.5 Appendix E- Discrepancy Report

# Date Tester Type Status Location Issue Description VVSG Requirement Vendor Response Validation of 

Resolution

1 6/21/10 K. 

Austin/ 

S. 

Brown

Functional 

Defect

Closed ERM 

v.7.5.4.0 -

Uploading 

DS200 

(v.1.4.3.0) 

election 

results

An error occurs in ERM when 

attempting to upload Election results 

from 2 of 18 precincts. 

DS200 election results from 2 of 18 

precincts failed to upload. When 

uploading precincts 2 and 5 ERM

displayed: "Import process error. 

0002-spp record contest totals are 

not correct. The results will not be 

used. Click OK to acknowledge". The 

other16 precincts imported 

successfully. 

v.1: 5.4.4.a Voting systems 

shall meet these reporting 

requirements by providing 

software capable of 

obtaining data concerning 

various aspects of vote 

counting and producing 

printed reports. At a 

minimum, vote tally data 

shall include: Number of 

ballots cast, using each 

ballot configuration, by 

tabulator, by precinct, and 

by political subdivision

SLM 06.29.10 - See Revised System 

Overview • Updated the Notice of 

Unsupported Functionality appearing 

after the document copyright page 

and also the footer to indicate that 

the Unity 3200r1 voting system does 

not support elections for the states of 

Hawaii or Illinois.SLM 06.25.10 -

ES&S does not support the State of 

IL in Unity 3200 Revision 1. This 

error occurred when the election was 

set with the state code of IL. This 

issue has been resolved in Unity 

3.2.1.0.

Accepted 06/29/10 

KA - Verified in 

ES&S Sys 

Overview ver. 5.0 

dated 06/29/10, in 

the Notice of 

Uncertified 

Functionality and 

the footer, that HI 

and IL are listed as 

unsupported 

states.

2 6/23/10 K. 

Austin/ 

S. 

Brown

Functional 

Defect

Closed ERM 

7.5.4.0 -

importing 

DS200 

v.1.4.3.0 

election 

results

ERM failed upload error message 

does not provide clear direction to 

the user.

Error message "Import process error. 

0002-spp record contest totals are 

not correct. The results will not be 

used. Click OK to acknowledge" 

received when attempting to upload 

Election results into ERM does not 

clearly state the action to be 

performed to recover and correct the 

problem. 

V:1. 2.1.5.1.b.v The 

message cue for all voting 

systems shall clearly state 

the action to be performed 

in the event that voter or 

operator response is 

required. 

SLM 06.29.10 - See Revised System 

Overview • Updated the Notice of 

Unsupported Functionality appearing 

after the document copyright page 

and also the footer to indicate that 

the Unity 3200r1 voting system does 

not support elections for the states of 

Hawaii or Illinois.SLM 06.25.10 -

ES&S does not support the State of 

IL in Unity 3200 Revision 1. This 

error occurred when the election was 

set with the state code of IL. This 

issue has been resolved in Unity 

3.2.1.0

Accepted 06/29/10 

KA - Verified in 

ES&S Sys 

Overview ver. 5.0 

dated 06/29/10, in 

the Notice of 

Uncertified 

Functionality and 

the footer, that HI 

and IL are listed as 

unsupported 

states.

3 6/23/20 K. 

Austin/ 

S. 

Brown

Document 

Defect

Closed ERM SOP 

7.5.4.0, 

06/25/200

9

ERM error message is not identified 

in the SOP.

The error message "Import process 

error. 0002-spp record contest totals 

are not correct. The results will not 

be used. Click OK to acknowledge" is

not identified in the ERM SOP 

(06/25/2009, v.7.5.4.0) 

documentation for a user to research.

V:2 2.8.5.c Provides 

procedures that clearly 

enable the operator to 

intervene in system 

operations to

recover from an abnormal 

system state

20100625 MDN - Updated the error 

and audit log listings in the ERM 

SOP, SDS and SFD with the 

specified message. The SOP 

includes the recommended user 

action.

Accepted 06/29/10 

KA - Verified in 

ERM SOP 7.5.4.0 

dated 06/28/10 that 

the error message 

is identified for user 

to research.

4 6/24/10 K. 

Austin/ 

S. 

Brown

InformationalOpen DS200 

v.1.4.3.0

"Thank you for voting" display is 

inconsistent. 

When scanning in ballots with 

Overvotes that include write-ins 

(contest XYZ, vote for 1, candidate 

AA and write-in BB are selected; 

query response to accept the ballot) 

the thank you for voting message is 

not displayed. The ballot is accepted 

and the screen displays for the next 

voter. This is inconsistent with the 

thank you display in other instances 

(accepted ballot with undervotes, no 

errors, overvotes with no write-ins, 

etc.) 

SLM 06.29.10 - ES&S acknowledges 

this finding and will enter it into our 

tracking system to address it in a 

future release of the DS200.
EAC Certification # pending

Page 34 of 34 iBeta Report Number: (V)2010-30Jun-001(B)

7.6 Appendix F: Warrant of Accepting Change Control Responsibility

ES&S' Unity 3.2.0.0 Revision 1 Warrant of Accepting Change Control Responsibility is submitted in a separate 

document. 

7.7 Appendix G: Trusted Build & Validation Tools Unity 3.2.0.0 Revision 1

Appendix G provides documents the Trusted Build and Validation Tools for Unity 3.2.0.0 Revision 1. It is contained 

in the document Appendix A, B, and G of the ES&S Unity 3.2.0.0 Revision 1 Voting System Certification Test 

Report for DS200 Modifications to the EAC Certified ESSUNITY3200.

7.8 Appendix H: Amended Test Plan

The EAC emailed notice to iBeta that the ES&S Unity 3.2.0.0 Rev. 1 Voting System Certification Test Plan for 

DS200 Modifications to the EAC Certified ESSUNITY3200 v.2.0 are found on the EAC website. 

This test plan was amended during test execution. Version 3.0 of the test plan is submitted in a separate document 

with this test report. Changes are provided in red text for easy identification.

7.9 Appendix I: State Test Reports

There were no state test efforts.

7.10 Appendix J: ES&S Unity 3.2.0.0 Implementation Statement

Unity 3.2.0.0 Revision 1 is a revision to the certified ESSUNITY3200 voting system. A copy of the ES&S Unity 

3.2.0.0 implementation statement is attached as a separate document to the ESSUNITY3200 test report.