Election Assistance Commission 

Voting System Test Summary Report

Summary of test Report for testing through 10/22/08 

for Election Systems & Software (ES&S), 

Unity 4.0 Voting System 

Report Number 07-V-ESS-035-CTP-01, Rev 0.3 

July 15, 2009 

Prepared for: 

Vendor Name Election Systems & Software (ES&S) 

Vendor System Election Systems and Software (ES&S) 

Unity 4.0 Voting System

EAC Application No. ESS0702 

Vendor Address 11208 John Galt Boulevard 

Omaha, NE 68137-2364 
 

Statement of use:

This document is a summary report for the testing that SysTest Labs had completed as of 10/22/08. 

The document was compiled and edited by the EAC with the information provided by SysTest Labs. 

Due to the suspension of SysTest’s EAC VSTL accreditation, the ES&S voting system was 

transferred to iBeta for continued testing and certification. iBeta was allowed to re-use portions of 

the testing completed by SysTest. Please refer to the letters of correspondences between the EAC 

and iBeta, which can be found on the EAC’s website at www.eac.gov. This document is to be used 

as a reference for the testing completed by SysTest and used by iBeta. It is NOT to be used in place 

of the approved certification test report from iBeta. 
 

Disclaimer

This document contains confidential information and trade secrets of ES&S. No part of this confidential information and these trade 

secrets should be (a) reproduced; (b) published in any form by any means, electronic or mechanical, including photocopy or 

information storage or retrieval system; or (c) disclosed to any third party, without the express prior authorization of ES&S. 

WARNING! COPY RIGHTED MATERIAL. DO NOT COPY WITHOUT PERMISSION. 

All products and company names are used for identification purposes only and may be trademarks of their respective 

owners. 

Trademarks 

 Intel and Pentium are registered trademarks of Intel Corporation. 

 Microsoft, MS are registered trademarks and Internet Explorer, Windows, MS Windows, Windows NT, 

Windows 2000, Windows 95, Windows 98, Visual C++, Visual Basic, VBX, ODBC, and MFC are trademarks 

of Microsoft Corporation. 

 ES&S’s Unity 4.0 is a trademark of ES&S 

 Audit Manager (AM) is a trademark of Election Systems & Software. 

 Election Data Manager (EDM) is a trademark of Election Systems & Software. 

 Ballot Image Manager (ESSIM) is a trademark of Election Systems & Software. 

 Ballot on Demand (BOD) is a trademark of Election Systems & Software. 

 iVotronic Image Manager (iVIM) is a trademark of Election Systems & Software. 

 Hardware Programming Manager (HPM) is a trademark of Election Systems & Software. 

 Data Acquisition Manager (DAM) is a trademark of Election Systems & Software. 

 Election Reporting Manager (ERM) is a trademark of Election Systems & Software. 

 AutoMARK Information Management System (AIMS) is a trademark of Election Systems & Software.

 AutoMARK Voter Assist Terminal (VAT) A100 & A200 are trademarks of Election Systems & Software. 

 iVotronic is a trademark of Election Systems & Software. 

 Model 100 Precinct Scanner (M100) is a trademark of Election Systems & Software. 

 intElect DS200 Precinct/Central Count Scanner (DS200) is a trademark of Election Systems & Software. 

 Model 650 Central Count Scanner (M650) is a trademark of Election Systems & Software. 

Other Trademarks

 All other products and company names are used for identification purposes only and may be trademarks of their 

respective owners. 

Opinions and Interpretations

EAC interpretations were applied to this VSTL test effort. Interpretations are listed in the relevant section.

Other Labs Performing Hardware Testing

SysTest Labs is responsible for all core voting system tests as identified in NIST NVLAP Handbook 150-22 (2005). The labs listed 

below performed non-core hardware testing for this certification test campaign. 

Laboratory Address Test(s) Date(s) 

SysTest Labs 216 16th St., Suite 700 

Denver, CO, 80202 
 

TABLE OF CONTENTS 

INTRODUCTION..................................................................................................................................... 5

1.1 References ..............................................................................................................................................................5

1.2 Document Overview ..............................................................................................................................................6

CERTIFICATION TEST BACKGROUND .................................................................................................. 7

1.3 PCA - Document and Source Code Reviews .........................................................................................................7

1.4 FCA - Functional & System Testing and Sampling ...............................................................................................7

1.5 Terms, Abbreviations, and Definitions ..................................................................................................................8

SYSTEM IDENTIFICATION................................................................................................................... 10

1.6 Software and Firmware ........................................................................................................................................11

1.7 Equipment (Hardware).........................................................................................................................................16

1.8 Test Materials.......................................................................................................................................................22

1.9 TDP Documents Used to Support Testing ...........................................................................................................22

SYSTEM OVERVIEW............................................................................................................................ 23

CERTIFICATION TEST RESULTS SUMMARY....................................................................................... 30

1.10 Code Review Summary........................................................................................................................................30

1.11 Technical Data Package Review Summary..........................................................................................................33

1.12 Application Functional Test Summary.................................................................................................................34

1.13 System Level (Integration) Test Description........................................................................................................39

1.14 Environmental Hardware Test Summary .............................................................................................................47

1.15 Security/Telecommunications Test Summary......................................................................................................49

1.16 Sampling of Vendor Testing Results....................................................................................................................49

1.17 Maintainability Test Results.................................................................................................................................51

1.18 Accuracy Test Results..........................................................................................................................................51

APPENDIX A –TRUSTED BUILD .......................................................................................................... 59

APPENDIX B – INDEX OF ATTACHMENTS........................................................................................... 61

List of Tables 

Table 1 – Summary of Unity 4.0 Voting System Components.............................................................................................7

Table 2 – Matrix of Terms and Abbreviations provided by the vendor................................................................................8

Table 3 – Matrix of Required Software and Firmware.......................................................................................................11

Table 4 – Matrix of Required COTS Software/Firmware ..................................................................................................13

Table 5 – Matrix of Required Hardware.............................................................................................................................16

Table 6 – Matrix of Test Materials.....................................................................................................................................22

Table 7 – Matrix of System Functional Testing .................................................................................................................26

Table 8 - Matrix of System Level and Other Functional Testing .......................................................................................28

Table 9 – Matrix of Additional Testing..............................................................................................................................29

Table 10 – Source Code Review Assessment..................................................................... Error! Bookmark not defined.
 

Introduction 

SysTest Labs Incorporated is submitted this report as a summary of the certification testing efforts 

for Election Systems & Software (ES&S) Unity 4.0, for testing completed through 10/22/08. The 

purpose of this document is to provide an overview of the certification testing effort and the findings 

of the testing effort up to that date.

This effort included documentation review of the Technical Data Package, source code review, and 

testing of Unity 4.0. Testing consisted of the development of a test plan, managing system 

configurations, executing functional test cases based on the project test requirements, system level 

tests prepared by SysTest Labs and analysis of results. The review and testing was performed at 

SysTest Labs’ Denver, Colorado facility and at ES&S’s facility in Omaha, Nebraska. 

SysTest Labs is a full service third party testing facility, founded in May 1996, from a software testconsulting firm. The specific system testing services offered include: 

 Test Planning and Test Management 

 eBusiness, Client-Server and Stand-alone Application Functional, Compatibility and 

Regression Testing 

 eBusiness and Client-Server Load and Performance Testing 

 Automated Regression Test Development, Consulting, Scripting and Execution 

 Complex, Integrated Test Solutions And Automated Test Harnesses 

 Independent Verification and Validation 

 EAC approved Voting System Test Laboratory 

SysTest Labs maintains partnerships with software test tool vendor companies Segue Software, 

Mercury Interactive, Rational Software, Borland, and Compuware. 

1.1 References 

1. Federal Election Commission Voting System Standards (FEC VSS), April 2002. Volume I 

Performance Standards: Section 1 Introduction 1.6.1; Section 2 Functional Capabilities; Section 

3 Hardware Standards; Section 7 Quality Assurance 7.1, 7.4, 7.6, and 7.7; Section 8 

Configuration Management 8.1; Section 9 Overview of Qualification Tests 9.2, 9.4.1, 9.4.1.1, 

9.4.2, 9.5, 9.5.2, 9.5.2.1, 9.5.2.2, 9.6, and 9.6.1.2. Volume II Test Standards: Section 1 

Introduction 1.3 and 1.4; Section 2 Technical Data Package; Section 3 Functionality Testing 

3.2.1, 3.2.2, 3.2.3, and 3.2.4; Section 6 System Level Integration Testing 6.6 and 6.7; Section 7 

Examination of Vendor Practices for Configuration Management and Quality Assurance 7.5 and 

7.5.1; Appendix A Qualification Test Plan A.2 and A.3. 

2. NIST NVLAP Handbook 150: 2006. 

3. NIST NVLAP Handbook 150-22: 2005. 

4. EAC Testing and Certification Program Manual, United States Election Assistance Commission, 

2006 

5. IEEE Standard for Software Quality Assurance Plans IEEE Std 730-1998, October 20th, 1998. 

6. IEEE Standard for Software Configuration Management Plans IEEE Std 828-1998, June 25th, 

1998. 

7. IEEE Standard for Software Test Documentation IEEE Std 829-1998, December 16th, 1998. 
 

8. IEEE Recommended Practice for Software Requirements Specifications IEEE Std 830-1998, 

October 20th, 1998. 

9. IEEE Standard for Software Unit Testing IEEE Std 1008-1987, December 29th, 1986. 

10. IEEE Standard for Software Verification and Validation IEEE Std 1012-1998, July 20th, 1998. 

11. SysTest Labs Quality System Manual, Revision Rev. 1.5, prepared by SysTest Labs, dated 

December, 05, 2008. 

1.2 Document Overview 

This document contains: 

 The Introduction discusses the application tested/reviewed. 

 The Certification Test Background discusses the testing process. 

 The System Identification identifies hardware and software for the Unity 4.0. 

 The System Overview discusses the functionality of Unity 4.0 software and firmware. 

 The Certification Test Results Summary contains a summary of the testing effort. 

 The Recommendations section contains the analysis of the testing effort conducted so far. 

 The Appendices contain Test Operations, Findings and Data Analysis. They also contain the 

EAC Certification information after the system has been qualified. 

o Appendix A: Trusted Build 

o Appendix B: Index of Attachments 
 

Certification Test Background 

1.3 PCA - Document and Source Code Reviews 

The Physical Configuration Audit (PCA) review of ES&S Unity 4.0 voting system documentation 

submitted in the Technical Data Package (TDP) was executed in order to verify conformance with 

Federal Election Commission Voting System Standards (FEC VSS) April 2002. Source Code was 

reviewed for Unity 4.0. 

All reviews were conduced in accordance with Volume 2, Sections 2.2 through 2.13 and Volume 2, 

Section 6.6 of the FEC VSS April 2002, to demonstrate that the system meets or exceeds the 

requirements of the FEC VSS. Results of the source code review were recorded on Excel 

spreadsheets for each submitted application. Results of the PCA documentation review can be 

found in Attachment B of this Certification Report. Results of the source code review can be found 

in Attachment C. 

Inconsistency or errors in documentation were identified to ES&S in a Discrepancy Report for 

resolution or comment. This Discrepancy Report can be found in Attachment D. 

1.4 FCA - Functional & System Testing and Sampling 

The Functional Configuration Audit (FCA) review of the test documentation submitted by ES&S in 

the TDP was executed in order to verify testing of the voting system requirements defined in Volume 

1 Sections 2, 3, 4, 5, 6, and Volume 2, Section 6.7 of the FEC VSS April 2002. 

SysTest Labs’ standard System Level Test Cases were customized for ES&S and conducted in 

accordance with Volume 2 Section 6, in conjunction with the functional testing. Simulations of 

elections were conducted to demonstrate a beginning-to-end business use case process for Unity 4.0. 

SysTest Labs selected a subset of the Unity 4.0 voting system functionality for functional test 

execution based on high-risk areas, such as Security, Audit log, Tabulating, Transmitting, and 

Accuracy. Any test issues found during the project were identified to ES&S in a Discrepancy Report 

for resolution or comment. ES&S resolved all discrepancies that did not comply with the April 2002 

FEC VSS requirements, which were verified as acceptable after regression testing by SysTest Labs. 

Status Reports were emailed to ES&S. These reports reflected the daily and/or weekly test activities 

including project task status, issues found, and schedule/project management information. All results 

of our FCA testing can be found in Attachment A of this Certification Report. 

Table 1 – Summary of Unity 4.0 Voting System Components 

Software/Firmware Hardware

Election Management System (EMS) 

 Audit Manager 

 Election Data Manager 

 AutoMARK Information 

Management System (AIMS) 

 ES&S Ballot Image Manager 

 Ballot on Demand 

 Compact Flash Multi-Card 

Reader/Writer 

 Automatic Bar Code Reader 

 Hand held bar code scanner 

 iVotronic DRE with a 4-inch RealTime Audit Log printer 

 iVotronic DRE with a 9.5-inch 

Real-Time Audit Log printer 
 

Software/Firmware Hardware

 iVotronic Image Manager 

 Hardware Programming Manager 

 Data Acquisition Manager 

 Election Reporting Manager 

 iVotronic DRE with stand-alone 

printer 

 iVotronic DRE with the 

communication pack 

 AutoMARK Voter Assist Terminal 

 Model 100 precinct scanner with 

steel ballot box 

 intElect DS200 precinct scanner 

with steel & plastic ballot box 

 Model 650 central count scanner 

1.5 Terms, Abbreviations, and Definitions 

The following terms and abbreviations will be used throughout this document: 

Table 2 – Matrix of Terms and Abbreviations provided by the vendor. 

Term Abbreviation Definition 

Audit Manager AM Audit Manager is ES&S’ tracking program for the Unity software 

suite. AM tracks user activity in AM, EDM and ESSIM. 

Automatic Bar Code Reader ABCR The ABCR is a device that audits and recounts the printout 

generated by the iVotronic RTAL printer. The ABCR device 

interfaces with ABCR software installed on a PC to generate reports 

based upon the scanned barcodes from the RTAL printout. 

AutoMARK Information 

Management System AIMS 

Software that facilitates creation of the election database, or 

conversion of a 3rd party election database, for installation on the 

VAT. 

AutoMARK Voter Assist 

Terminal VAT Optical paper ballot marking device for disabled voters and alternative languages 

Ballot On Demand BOD 

Election officials use Ballot on Demand to print test ballots, early 

voting ballots and ballots for polling places that run short of ballots 

on Election Day. 

Binary Logic Input Device 

Alternative accessible appliance that is connected to the AutoMARK 

Voter Assist Terminal through a stereo jack, enabling the voter to 

issue either a yes or no command. These devices may include foot 

pedals and Sip/Puff tubes. 
 

Term Abbreviation Definition 

Data Acquisition Manager DAM 

The ES&S Data Acquisition Manager software is used to transmit 

election results over a network connection from ES&S ballot 

counting equipment to a central count location. 

Delkin USB A USB flash drive to store the scanner’s election definition, audit log 

and other election-specific information. 

Election Data Manager EDM 

Election Data Manager is a database system that stores all of a 

jurisdiction’s precinct, office, and candidate information. It is used 

in conjunction with other Unity software to format and print ballots, 

program ballot scanning equipment, and produce Election Day 

reports. 

Election Reporting Manager ERM 

The Election Reporting Manager is an election results reporting 

program, used to generate paper and electronic reports for poll 

workers, candidates, and the media. ERM can display updated 

election totals on a monitor as ballot data is tabulated and can send 

result reports directly to media outlets over the Internet. ERM is 

designed to support a wide range of ES&S ballot scanning 

equipment and can produce reports for both central count systems 

and precinct count systems. 

ES&S Ballot Image Manager ESSIM ES&S Ballot Image Manager is a publishing tool used to design and 

print ballots with the election information stored in EDM. 

Flash Memory Card FMC The FMC supplies ballot content information to the VAT. 

iVotronic 

The iVotronic is a DRE (direct recording electronic) touch screen 

that displays ballots and records votes. The iVotronic addresses 

accessibility requirements through the use of voice files, font type 

and size, and color combinations. There are two sizes of iVotronics. 

One is the 12 inch with 3 key ADA buttons the other is a 15 inch that 

have a 3, 4 or 6 key ADA buttons. There are also non-ADA 

iVotronics. The iVotronic 6 key allows the use of the sip and puff. 

iVotronic Image Manager iVIM The iVotronic Image Manager enables the user to create and format 

graphic ballot screens for the iVotronic voting device. 

Hardware Programming 

Manager HPM 

Hardware Programming Manager enables the user to import, format, 

and convert the election definition files for ballot scanning 

equipment and DREs. 

intElect DS200 DS200 

The intElect DS200 precinct ballot scanner is part of a jurisdictionwide election tabulating system. Voters make selections and then 

insert their ballots directly into the Model 100 at the polling place. 

The scanner tabulates votes and sorts a ballot as soon as a voter 

inserts it and then feeds the ballot into the attached ballot storage bin 

accepting ballots inserted in any direction and reads both sides of the 

ballot simultaneously. 

Model 100 M100 

The Model 100 precinct ballot scanner is part of a jurisdiction-wide 

election tabulating system. Voters make selections and then insert 

their ballots directly into the Model 100 at the polling place. The 

scanner tabulates votes and sorts a ballot as soon as a voter inserts it 

and then feeds the ballot into the attached ballot storage bin 

accepting ballots inserted in any direction and reads both sides of the 

ballot simultaneously. 

Model 650 M650 

The Model 650 is an optical scan central count counter that is used 

to scan ballots at a central count location. The M650 scan up to 350 

ballots per minute, counts different sizes (11, 14, 17, 19) of ballots 

and can read voting marks on the right or left of the ballot column. 

The M650 prints results reports and saves results to a zip disk. 
 

Term Abbreviation Definition 

PCMCIA PCMCIA card contains the M100 election definition that exactly 

mirrors the ballot contents and issues as defined by election officials. 

Personalized Electronic 

Ballots PEB 

An electronic ballot that a jurisdiction defines for use with the 

iVotronic to open polls, load ballots and collect votes from each 

terminal at the end of an election day. 

Real-Time Audit Log Printer RTAL 

The Real-Time Audit Log Printer records each voter’s actions on a 

paper audit log in real time, including all selections and de-selections 

for the iVotronic. The paper audit log can be viewed but not touched 

by the voter as the paper is behind a clear plastic cover. Prior to 

casting a vote. Under-voted contests and a two-dimension bar code 

of the votes are appended to the audit entries and the paper advances 

out of the view window in either a 9-inch or 4.5-inch window. 

Unity Release N/A The system configuration(s) of ES&S hardware and software voting 

system(s). 

System Identification 

Unity 4.0 was submitted for certification testing with the hardware and software listed below. No 

other ES&S product was included in this test effort. 

The TDP User/Owner manuals that would be part of the certified system delivered to a purchaser of 

the system are as follows: 

System Maintenance Procedures: 

 ES&S DS200 System Maintenance Manual, Hardware Version 1.2.0.0, Firmware Version 

1.2.0.0, November 2, 2007 

 ES&S iVotronic System Maintenance Manual, Hardware Revision iV1.26.15asp, Firmware 

Version 9.2.0.0, August 3, 2007 

 ES&S M100 System maintenance Manual, Version Release 5.4.0.0, Hardware Version, 1.3, 

November, 16, 2007 

 ES&S Model 650 System Maintenance Manual, Firmware Version 2.2.1.0, Hardware 

Version 1.1, November 16, 2007 

System Operation Procedures: 

 ES&S Model 650 System Operations Procedures, Firmware Version 2.2.1.0, Hardware 

Version 1.1, November 16, 2007 

 ES&S Model 100 System Operations Procedures, Firmware Version 5.4.0.0, Hardware 

Version 1.3, November 16, 2007 

 ES&S iVotronic System Operations Procedures, Firmware Version 9.2.0.0, Hardware 

Version 1V1.26.15asp, November 7, 2007 

 ES&S iVotronic Image Manager System Operations Procedures, Version Release 3.1.0.0, 

October 19, 2007 

 ES&S Hardware Programming Manager System Operations Procedures, Version Release 

5.6.0.0, August 17, 2007 
 

 ES&S Image Manager System Operations Procedures, Version Release 7.7.0.0, October 15, 

2007 

 ES&S Election Reporting Manager System Operations Procedures, Version Release 7.4.0.0, 

November 16, 2007 

 ES&S Election Data Manager System Operations Procedures, Version Release 7.8.0.0, 

November 16, 2007 

 ES&S DS200 System Operations Procedures, Firmware Version 1.2.0.0, Hardware Version 

1.2.0.0, November 8, 2007 

 ES&S Data Acquisition Manager System Operations Procedures, Version Release 6.1.1.0, 

November 16, 2007 

 ES&S Audit Manager System Operations Procedures, Version Release 7.5.0.0, November 

12, 2007 

 ES&S Automated Barcode Reader System Operations Procedures, Firmware Version 29, 

Hardware Version Rev B, Software Version 1.3.0.0, HIDCom Version 1.0.0.0, November 

16, 2007 

System Overview: 

 System Overview, Election Systems and Software, Version Number 4.0.0.0, November 15, 

2007 

1.6 Software and Firmware 

Items identified in the table reflect all software and firmware used to perform hardware, software, 

telecommunications, security and integrated system tests. Not all items listed below were required 

to run the Unity 4.0 voting system. However, all items listed were part of the certification test 

effort. 

Table 3 – Matrix of Required Software and Firmware 

Application(s) Manufacturer Version Description 

Audit Manager ES&S 7.5.0.0 Audit Manager provides security and user tracking for 

itself, Election Data Manager and Ballot Image 

Manager. Audit Manager runs in the background of the 

other Unity programs and provides password security 

and a real-time audit log of all user inputs and system 

outputs. Jurisdiction Officials use Audit Manager to set 

Unity system passwords and track user activity. 

Election Data 

Manager 

ES&S 7.8.0.0 Election Data Manager is a single-entry database that 

stores all of a jurisdiction’s precinct, office, and 

candidate information. Election Data Manager is used in 

conjunction with other Unity software to format and print 

ballots, program ballot scanning equipment, and 

produce Election Day reports. 

ES&S Ballot 

Image Manager 

ES&S 7.7.0.0 ESSIM is a desktop publishing tool that is used to 

design and print ES&S paper ballots. ESSIM uses ballot 
 

Application(s) Manufacturer Version Description 

(with Ballot on 

Demand) 

style information created by Unity Election Data 

Manager to display the WYSIWYG ballots. 

Ballot On Demand (BOD) is an accessory program that 

you can use to print individual, Election Day ballots 

directly from ESSIM. 

iVotronic Image 

Manager 

ES&S 3.2.0.0 iVotronic Image Manager (iVIM) is a desktop publishing 

tool that is used to design and generate graphic ballots 

for the iVotronic precinct voting system. iVIM uses ballot 

style information created by Unity Election Data 

Manager to display the WYSIWYG ballots. iVotronic 

Image Manager also allows the user to view the ballot in 

different languages, and create multiple displays for the 

same ballot. Ballots generated by iVotronic Image 

Manager comply with ADA (Americans with Disabilities 

Act) requirements using voice files, specific font type 

and size, and color combinations. 

Hardware 

Programming 

Manager 

ES&S 5.7.0.0 Hardware Programming Manager (HPM) is a complete 

election package that enables the user to import, format, 

and convert the election file; define districts; specify 

election contests and candidates; create election 

definitions for ballot scanning equipment; burn M100 

PCMCIA Cards, DS200 USB memory sticks, M650 zip 

disks, or PEBs; and create the Data Acquisition 

Manager Precinct List. The Hardware Programming 

Manager is primarily used for converting the election 

IFC file for use with the Election Reporting Manager and 

for creating and loading election parameters; however, it 

may also be used for coding the election. The Unity 

Hardware Programming Manager seamlessly programs 

the ES&S election tabulation hardware with electionspecific information retrieved from the Unity Election 

Data Manager (EDM). 

NOTE: Creating an election definition from scratch 

in HPM is not supported in the Unity 4.0 

certification. 

Data Acquisition 

Manager 

ES&S 6.1.3.0 The Unity Data Acquisition Manager (DAM) is a clientserver application that collects election data from ES&S 

voting systems and transmits the data directly from the 

polls or regional sites via modem transmission to the 

host election server for the purpose of results 

accumulation, reporting, and display. 

The Data Acquisition Manager allows users to transfer 

election results from remote polling sites to a 

jurisdiction’s election headquarters. Data Acquisition 

Manager has two software configurations: Data 

Acquisition Manager Remote and Acquisition Manager 

Host. Poll workers use the remote configuration to 

transfer election results to the central collection location. 

Officials at the central site use the host configuration to 

receive election data from polling places. Workers at the 

central location load collected results into Election 

Reporting ManagerTM to format, print, and display final 
 

Application(s) Manufacturer Version Description 

election reports. 

Election 

Reporting 

Manager 

ES&S 7.5.2.0 Election Reporting Manager (ERM) is ES&S’ election 

results reporting program. ERM generates paper and 

electronic reports for election workers, candidates, and 

the media. ERM can also display updated election totals 

on a monitor as ballot data is tabulated, and it can send 

results reports directly to media outlets. Election 

Reporting Manager is designed to support a wide range 

of ES&S ballot scanning equipment and can produce 

reports for both central-count systems and precinctcount systems. 

AIMS ES&S 1.3.57 The AutoMARK Management Information System 

(AIMS) is software that manages all of the information 

required by the AutoMARK Voter Assist Terminal (VAT) 

for an election. The AIMS process starts with a printed 

optical scan ballot. In addition to the printed ballot, files 

produced by ES&S Unity Systems may be imported into 

AIMS, for ease in loading data into the AutoMARK AIMS 

election database. In lieu of the import procedure, 

election specific data may be manually entered into 

AIMS. AIMS writes the election database to a compact 

flash memory card (FMC). This FMC supplies ballot 

content information to the VAT. 

Table 4 – Matrix of Required COTS Software/Firmware 

COTS Application(s) Manufacturer Version Description 

Required COTS software for the Unity 4.0 voting system 

Windows XP Professional Microsoft Corporation 2002 Service Pack 2 COTS software for all 

Applications listed above. 

RM COBOL RUNTIME 

System 

RM/COBOL 11.01 COTS software for the ERM, 

HPM 

Adobe Type Manager 

(includes Adobe Type 

Basics and Adobe Type 

Manager Light) 

Adobe 4.1 COTS software for ESSIM, 

BOD 

OmniDrive USB 

Professional 

Omni No version COTS software for the HPM, 

ERM 

PEB Reader Pivot/ES&S 1.1.0.0 COTS software for HPM, 

ERM 

Non-required COTS software for the Unity 4.0 voting system 

Broadcom Gigabit 

Integrated Controller Broadcom 

9.02.06 COTS software Voyager 

Hand scanner, and Desktop 

PCs. 
 

COTS Application(s) Manufacturer Version Description 

C-Major Audio SigmaTel 

42.xx COTS software Voyager 

Hand scanner, and Desktop 

PCs. 

Conexant D110 MDC Unknown 92 Modem COTS software Voyager 

Hand scanner 

Graphics Media 

Accelerator Driver for 

Mobile 

Intel No version COTS software Voyager 

Hand scanner 

MS Office Professional 

Edition 2003 

(MS Word and Excel 

installed in the setup) 

Microsoft Corporation 11.0.7969.0 COTS software Voyager 

Hand scanner 

O2Micro Smartcard Driver 

O2Micro 2.26.0000 COTS software Voyager 

Hand scanner, and Desktop 

PCs. 

ATI Display Driver ATI No version COTS software for the Server

Dell OpenManage Array 

Manager 

Dell No version COTS software for the Server

DirectX Hotfix – KB839643 Microsoft Corporation No version COTS software for the Server

HP Laser Jet 2300 

Uninstaller 

HP No version COTS software for the Server

Intel® PRO Intelligent 

Installer 

Intel® PRO Network 

Adapters and Drivers 

Intel 2.01.1000 COTS software for the Server

Internet Explorer Q867801 Microsoft Corporation No version COTS software for the Server

LiveUpdate Symantec 

Corporation 

1.7 COTS software for the Server

Symantec AntiVirus Client Symantec 

Corporation 

8.0.0.374 COTS software for the Server

Outlook Express Q823353 Microsoft Corporation No version COTS software for the Server

Windows 2000 Microsoft Corporation Service Pack 4 COTS software for the Server

Windows 2000 

Administration Tools Microsoft Corporation 5.0.0.0000 COTS software for the Server

Microsoft Health Monitor 

2.1 

Microsoft Corporation 2.10.1850.0000 COTS software for the Server

Microsoft Internet Security 

and Acceleration Server 

Microsoft Corporation 3.0.1200 COTS software for the Server

Microsoft Shared Fax Microsoft Corporation 1.0000 COTS software for the Server
 

COTS Application(s) Manufacturer Version Description 

Microsoft Small Business Microsoft Corporation Server 2000 COTS software for the Server

Microsoft Data Access 

Components KB870669 

Microsoft Corporation No version COTS software for the Server

Microsoft.NET Framework Microsoft Corporation 1.1.4322 COTS software for the Server

Windows 2000 Hotfix: 

- KB819696, 

- KB820888, 

- KB822831, 

- KB823182, 

- KB823559, 

- KB82410, 

- KB824141, 

- KB824146, 

- KB825119, 

- KB826232, 

- KB828028, 

- KB828035, 

- KB828741, 

- KB828749, 

- KB835732, 

- KB837001 

- KB839643, 

- KB839645, 

- KB840315, 

- KB841872, 

- KB841873, 

- KB842526, 

Microsoft Corporation

- 20030703.183130 

- 20030604.152521 

- 20030611.114034 

- 20030618.121409 

- 20030627.135515 

- 20030716.151320 

- 20030805.151423 

- 20030823.144456 

- 20030827.151123 

- 20031007.160553 

- 20040122.114409 

- 20031023.142138 

- 20040311.130332 

- 20031023.124056 

- 20040323.171849 

- 

- 20040506.120130 

- 0040519.160457 

- 20040622.153749 

- 20040520.90850 

- 20040610.95344 

- 20040521.202909 

COTS software for the Server

Intel ProEthernet Adapter 

and Software 

Intel No version COTS Software on the 

Desktop PCs 

SeaCOM Unknown No version COTS Software on the 

Desktop PCs 

SoundMAX Unknown No version COTS Software on the 

Desktop PCs 

ATI Software Uninstall 

Utility 

ATI 6.14.10.10.14 COTS Software on the 

Desktop PCs 
 

COTS Application(s) Manufacturer Version Description 

ATI Control Panel ATI 6.14.10.5173 COTS Software on the 

Desktop PCs 

ATI Display Driver ATI 8.20-051110A1-

028793C-Dell 

COTS Software on the 

Desktop PCs 

Conexant D480mdc Unknown 92 modem COTS Software on the 

Desktop PCs 

1.7 Equipment (Hardware) 

Equipment identified in the table reflects all hardware used to perform hardware, software, security 

and integrated system tests. Not all items listed below were required to run the Unity 4.0 voting 

system. However, all items listed were part of this certification test effort. All equipment was 

provided by ES&S. SysTest Labs staff uploaded all executables and installs on the equipment, and 

the equipment and Trusted Build executable and installs were under the control of SysTest Labs. 

Table 5 – Matrix of Required Hardware 

Item Manufacturer Model # Version/Rev Description 

intElect DS200 

(Scanner) 

ES&S DS200 Hardware v. 

1.2.0 & 1.2.1 

DS200 

Firmware 

1.3.7.0 

Power 

Management 

Firmware 

1.2.0.0 

Scanner 

Firmware 

2.11.0.0 

A precinct/central count ballot 

scanner. The scanner accepts 

ballots, tabulates votes, and sorts 

the ballots (if attached to a ballot 

box containing a diverter). 

Steel ballot box 

without diverter 

ES&S N/A N/A A storage receptacle to store 

scanned ballots. (Used with M100 

and DS200). 

Model 100 

(Scanner) – 

ES&S M100 Hardware v. 

1.3.0 

Firmware v. 

5.4.0.0 

A precinct ballot scanner. The 

scanner accepts ballots, tabulates 

votes, and sorts the ballots (if 

attached to a ballot box 

containing a diverter). 

Model 100 ES&S M100 Hardware v. A precinct ballot scanner. The 

scanner accepts ballots, tabulates 
 

Item Manufacturer Model # Version/Rev Description 

(Scanner) – 1.3.0 

Firmware v. 

5.4.0.0 

votes, and sorts the ballots (if 

attached to a ballot box 

containing a diverter). 

Steel ballot box 

w/ diverter) 

ES&S N/A N/A A storage receptacle to sort and 

store scanned ballots. (Used with 

M100 and DS200). 

Plastic ballot box ES&S N/A N/A A storage receptacle to sort and 

store scanned ballots. (Used with 

M100 and DS200). 

Model 650 – Red 

– Left (Scanner) 

ES&S M650 Hardware v. 

1.2 

Firmware v. 

2.2.1.0 

An optical scan central counter 

that is used to scan ballots at a 

central count location. The M650 

prints results reports and saves 

results to a zip disk. 

Model 650 – 

Green – Right 

(Scanner) 

ES&S M650 Hardware v. 

1.1 

Firmware v. 

2.2.1.0 

An optical scan counter that is 

used to scan ballots at a central 

count location. The M650 prints 

results reports and saves results 

to a zip disk. 

Model 650 – 

Green – Left 

(Scanner) 

ES&S M650 Hardware v. 

1.2 

Firmware v. 

2.2.1.0 

An optical scan central counter 

that is used to scan ballots at a 

central count location. The M650 

prints results reports and saves 

results to a zip disk. 

12inch, 3 key 

iVotronic (DRE) –

ES&S 0105-

096-

90659 

Hardware v. 

1.1 

Firmware v. 

9.2.3.0 

A DRE (direct recording 

electronic) touch screen that 

displays ballots and records 

votes. This is 12 inches with 3 

keys ADA buttons. 

12inch, Non-ADA 

iVotronic (DRE) –

ES&S 0105-

096-

90659 

Hardware v. 

1.1 

Firmware v. 

9.2.3.0 

A DRE (direct recording 

electronic) touch screen that 

displays ballots and records 

votes. This is 12 inches with no 

ADA buttons. 

15inch, 3 key 

iVotronic (DRE) 

ES&S 9VDC 

2770mA 

Hardware v. 

1.1 

Firmware v. 

9.2.3.0 

A DRE (direct recording 

electronic) touch screen that 

displays ballots and records 

votes. This is 15 inches with 3 

keys ADA buttons. 
 

Item Manufacturer Model # Version/Rev Description 

15inch, 4 Key 

iVotronic (DRE) – 

ES&S 9VDC 

2770mA 

Hardware v. 

1.1 

Firmware v. 

9.2.3.0 

A DRE (direct recording 

electronic) touch screen that 

displays ballots and records 

votes. This is 15 inches with 4 

keys ADA buttons. 

15inch, 6 key 

iVotronic (DRE) –

ES&S 15” 

9VDC 

2770mA 

Hardware v. 

1.1 

Firmware v. 

9.2.3.0 

A DRE (direct recording 

electronic) touch screen that 

displays ballots and records 

votes. This is a 15 inches with 6 

key ADA buttons. The iVotronic 6 

key allows the use of the sip and 

puff. 

15 inch, NonADA iVotronic 

(DRE) – 

ES&S 0105-

096-

90659 

Hardware v. 

1.1 

Firmware v. 

9.2.3.0 

A DRE (direct recording 

electronic) touch screen that 

displays ballots and records 

votes. This is 15 inches with no 

ADA buttons. 

15 inch 

Supervisor 

iVotronic (RED) –

ES&S 9VDC 

2770mA 

0150-

096-

90659 

Hardware v. 

1.1 

Firmware v. 

9.2.3.0 

Poll workers use supervisor 

equipment to open polls, load 

ballots onto voter PEBs or voting 

terminals, close the polls, and 

print results for the polling place. 

iVotronic RTAL 

Booth 4.5 inch 

window 

Booth: Pivot, 

Printer: Xten 

N/A Hardware v. 

N/A 

Firmware v. 

V012 

The Real-Time Audit Log Printer 

records each voter’s actions on a 

paper audit log in real time on a 

4.5-inch window. This printer is 

attached to a private voting booth. 

iVotronic RTAL 

Booth 9 inch 

window 

Booth: Pivot, 

Printer: Xten 

N/A Hardware v. 

N/A 

Firmware v. 

V012 

The Real-Time Audit Log Printer 

records each voter’s actions on a 

paper audit log in real time on a 

9-inch window. This printer is 

attached to a private voting booth. 

ABCR (Automatic 

Bar Code 

Reader) –

JADAK N/A Hardware v. B 

Firmware v. 29 

The ABCR is a device that audits 

and recounts the barcode printout 

generated by the iVotronic RTAL 

printer. 

Supervisor PEB – Pivot N/A Hardware v. 

N/A 

Firmware v. 

1.7.1.0 

A portable cartridge fitted with an 

infrared communications window 

and a flash memory chip. 

Supervisor PEBs contain specific 

ballot data for each election. They 

open the polls, load the ballot 

onto a voter terminal and enable 
 

Item Manufacturer Model # Version/Rev Description 

the service mode for 

administrative functions. 

Election 

SecurityKey PEB

ES&S N/A Hardware v. 

N/A 

Firmware v. 

1.7.1.0 

The iVotronic utilizes a “Key” PEB 

which requires that a key be 

passed to each iVotronic during 

set up in order to validate that the 

EQC (election qualification code) 

is correct for the election being 

conducted. This “Key” also 

requires that the correct election 

key be resident on each terminal 

before the election data is 

allowed to be unencrypted. 

Voter Activated 

PEB – 3 

Pivot N/A Hardware v. 

N/A 

Firmware v. 

1.7.1.0 

The Voter Activated PEB allows 

the voter to activate a ballot on 

the terminal in complete privacy. 

Communication 

Pack 

with Seiko printer 

Pivot 

Seiko 

N/A 

DPU 

3445 

Hardware v. 

1.1 

A case that contains special 

communications hardware, a 

serial thermal printer, and an 

optional modem for the iVotronic. 

The printer generates paper 

results, and the modem is used to 

transfer results to a central count 

location. 

Printer 

(standalone for 

iVotronic) 

Seiko DPU3445 N/A Standalone printer for the 

iVotronic. 

BOD Printer OkiData 9600 Printer used to print ballots. 

Printer (M650 

Red Left Printer) 

(2)

520 OkiData GE5258

A 

N/A Printer for audit logs and reports 

for the M650. 

Printer (M650 

Green Right 

Printer) – 2

520 OkiData GE5258

A 

N/A Printer for audit logs and reports 

for the M650. 

Printer (M650 

Green Left 

Printer) – 2

Epson Model # 

LQ-590 

P363A N/A Printer for audit logs and reports 

for the M650. 

LaserJet Printer HP 2300N N/A Printer for reports created within 

Unity. 

Router Dlink 1 @ 

DSH-16, 

1 with no 

identificat

1 @ V. B2, 1 

with no 

identification 

Directs and controls the flow of 

data. 
 

Item Manufacturer Model # Version/Rev Description 

ion 

Modem US Robotics 56K 

Sportster 

N/A A device that allows computer 

information to be sent over a 

telephone line. 

Multi-Modem 

Adapters 

(Used in DAM 

PC)

(1 each)

Equinox N/A N/A 4 and 8 port 

Multi-Modem 

Adapters 

(Used in DAM 

PC) 

Digi N/A N/A 4 and 8 Port 

Multi-Modem 

Adapters 

(Used in DAM 

PC) 

Perle N/A N/A 4 and 8 Port 

Multi-Port 

Adapter 

(Used in DAM 

PC) 

SeaLevel N/A N/A 7801 & 7803 – 8 Port 

7406 – 4 Port 

USB PEB 

Reader/Writer 

Pivot M1706 Hardware v. 

1.1 

A device with a USB connection 

used to upload election results 

from a PEB to a PC. 

Hand Bar Code 

Reader 

Voyager MS9544 N/A A device that reads the barcode 

printout generated by the 

iVotronic RTAL printer. 

Omni Drive Omni D707-94 Rev. C1 

USB 1.1 

A device used to read/write data 

to the PCMCIA card. 

Omni Drive 

Professional 

USB2 

Omni D707-94 Rev. A 

USB 2.0 

A device used to read/write data 

to the PCMCIA card. 

SanDisk Reader SanDisk SDDR-91 N/A Used to read data off of a 

SanDisk. 

SanDisk 

ImageMate CF 

Reader 

SanDisk SDDR-92 N/A Used to read data off of a 

SanDisk. 

Zip Disk 

1 Received

iOmega Z250US

BPCMBP

N/A Used to store data. 

Headphones ADID -(ESS) 

N/A -(ES&S 

N/A N/A (ESS) A pair of listening devices joined 

by a band across the top of the 
 

Item Manufacturer Model # Version/Rev Description 

VAT) AKG-K-44 

(ES&S VAT) 

head and worn in or over the 

ears. 

External Volume 

Control Button 

ES&S N/A Hardware v. 

N/A 

Used for controlling the volume 

on the 12 inch 3-Key and 15 inch 

3-Key iVotronics. 

Serial PEB 

Reader 

Pivot N/A Hardware Rev. 

1.1 

Software: N/A 

A device with a serial connection 

used to upload election results 

from a PEB to a PC. The reader 

can also connect to a M100 to 

combine results at the polling 

place. 

UPS Belkin N/A N/A Backup uninterrupted power 

source for the M650 

Sip n Puff Pivot N/A Device used on the iVotronic 6-

key by physically disabled voters 

iVotronic booth Pivot N/A N/A A booth that holds an iVotronic 

terminal and optionally an RTAL 

printer, to ensure voter privacy. 

Dell Laptop D600 

Latitude 

Dell N/A Windows XP 

Professional, 

SP2 

Rev A00 

Intel® Pentium® M processor 

1.60GHz 1.60 GHz, 1.00 GB of 

RAM (Laptop for Remote 

modeming only) 

Post Voting 

(DAM Client Regional Site remote 

only) 

Dell PC 

Pentium® 

Dell N/A Windows XP 

Professional, 

SP2 

4 CPU 2.00GHz, 512MB of RAM 

 (PC System 1) 

(Pre and Post Voting) 

Dell PC 

Pentium® 

Dell N/A Windows XP, 

SP2 

4 CPU 2.80GHz, 2.79 GHz, 1.00 

GB of RAM (PC System 2) 

Pre and Post Voting 

Dell PC 

Pentium® 

Dell N/A Windows XP 

Professional, 

SP2 

4 CPU 2.80GHz, 2.79 GHz, 5.12 

MB of RAM (PC System 3) 

(Post Voting DAM Host only) 

Server (PC) 

PE600SC 

Dell N/A Intel Pentium 4 CPU 1.80 GHz 

AT/AT compatible 

523,763 KB RAM 

Dell Laptop D610 

Latitude 

Dell N/A Windows XP 

Professional, 

SP2 

Intel® Pentium® M processor 

1.73GHz 795MHz, 0.99GB of 

RAM, (Physical Address 

Extension - laptop) 

(Hand Bar Code Reader and 
 

Item Manufacturer Model # Version/Rev Description 

Rev A06 ABCR) 

Multi Compact 

Flash 

Reader/Writer 

(Gang 

Programmer PC) 

Dell N/A Windows XP 

Professional, 

SP2 

Rev A00 

Pentium 4 CPU 

2.80GHz, 2.79 GHz, 512 MB of 

RAM 

(Pre & Post voting) 

VAT ES&S Model # 

A100 

Hardware v 1.0 

Firmware v. 

1.3.2904 

AutoMARK Voter Assist Terminal 

(VAT) is an electronic ballot 

marking device that allows voters 

to electronically mark a ballot, by 

using the touch screen Braille 

keypad or an AT (Assistive 

Technology (Sip and Puff) device. 

VAT ES&S Model # 

A200 

Hardware v 

1.0, and 1.1 

Firmware v. 

1.3.2904 

AutoMARK Voter Assist Terminal 

(VAT) is an electronic ballot 

marking device that allows voters 

to electronically mark a ballot, by 

using the touch screen Braille 

keypad or an AT (Assistive 

Technology (Sip and Puff) device. 

1.8 Test Materials 

Items identified in the table reflect test materials required to perform hardware, software, 

telecommunications, security, accuracy and integrated system tests. 

Table 6 – Matrix of Test Materials 

Item Provided by Details 

Printer paper rolls ES&S RTAL, Communication Pack, M100, DS200 and Seiko Printer 

Zip disks ES&S M650 program media 

USB SanDisk (CF) ES&S Compact Flash card 128, 256 & 512MB 

Blank paper ballot stock ES&S Inches/ballot positions: 11x36, 14x36, 14x48, 17x45, 17x60, 19x51, 19x68 

PCMCIA ES&S PC Cards M100 program media 

USB Memory Stick ES&S DS200 

Head sets ES&S For the VAT and iVotronic 

1.9 TDP Documents Used to Support Testing 

The vendor documents used to support Certification Testing are listed in Attachment A. 
 

System Overview 

The Unity 4.0 Voting System is a configuration consisting of an election management system 

(EMS), three ES&S optical mark scan hardware platforms (M100 precinct scanner, DS200 precinct 

scanner, and M650 central scanner), the iVotronic DRE hardware platform, and the AutoMark Voter 

Assist Terminal (VAT). The election management software includes the following applications: 

Audit Manager, Election Data Manager, AutoMARK Information Management System (AIMS), 

ES&S Ballot Image Manager, Ballot on Demand, iVotronic Image Manager, Hardware 

Programming Manager, Data Acquisition Manager, Automatic Bar Code Reader, and Election 

Reporting Manager. The EMS creates multiple ballot styles, generates election definition reports, 

and consolidates and distributes election results. The system is designed around the pre-election, 

election and post-election activities. 

The ES&S Model 100 (M100) is a precinct-based, voter-activated paper ballot counter and vote 

tabulator. The M100 uses advanced Intelligent Mark Recognition (IMR) visible light scanning 

technology. The M100 has the ability to alert voters to overvoted races, undervoted races, or blank 

ballots. When so alerted, the ballot is returned to the voter providing the opportunity for private 

revisions and the opportunity to recast the ballot. The M100 accepts ballots inserted in any 

orientation – top first, face up; bottom first, face down; etc. Optical sensors simultaneously read both 

sides of the ballot, and accurately record voter selections, as the Counter passes the ballot to the 

integrated ballot box. The M100 also has a feature that provides the user the ability to print a 

combined zero report and a combined results report at a polling place using an iVotronic PEB from 

the same election. A Personalized Electronic Ballot (PEB) is used to activate the iVotronic, open 

polls, load ballots onto the iVotronic and collect all votes from each iVotronic terminal at the end of 

an election day. A PEB reader can be connected to the M100 and combine the iVotronic results with 

those of the M100. All write-in names voted on the iVotronic DRE will display on the combined 

report, as well as the combined totals. The report will display the iVotronic totals separate from the 

M100 totals. If combining the iVotronic results with the M100, the user can transmit results via a 

landline modem. However, transmitting will only send the M100 results; therefore, the PEB must be 

taken to central count for tallying. 

The intElect DS200 is an optical scan paper ballot tabulator designed to be used at the polling place 

level, but can be used in multiple environments, such as a central count device in small jurisdictions 

only. After the voter makes their selections on their paper ballot with an indelible marker, or a 

device that was designed to assist with the marking process, their ballot is inserted into the unit for 

immediate tabulation. Both sides of the ballot are scanned at the same time using a high-resolution 

image-scanning device that produces ballot images that are decoded by a proprietary recognition 

engine. The system includes a large touch screen display to provide clear feedback to the voter on 

the disposition of their ballot. Once the ballot is tabulated and the system updates the vote counters, 

the ballot is dropped into a secure ballot box. The DS200 includes an internal thermal printer for the 

printing of zero reports, log reports, and polling place totals upon the official closing of the polls. 

Like the M100, the DS200 has the ability to alert voters to overvoted races, undervoted races, or 

blank ballots. When so alerted, the ballot is returned to the voter providing the opportunity for 

private revisions and the opportunity to recast the ballot. The DS200 also accepts ballots inserted in 

any orientation – top first, face up; bottom first, face down; etc. Optical sensors simultaneously read 

both sides of the ballot, and accurately record voter selections, as the Counter passes the ballot to the 

integrated ballot box. 
 

The Model 650 is also an optical scan central ballot counter. Jurisdiction workers program the 

scanner for a specific election with an election definition from a Zip disk. After the polls close, poll 

workers transport ballots to a central count location where election officials scan the ballots. The 

Model 650 prints a continuous audit log to a dedicated audit log printer and can print results reports 

directly from the scanner to a second connected printer. The scanner saves results to a Zip disk that 

officials can use to format and print results from a PC running Election Reporting Manager. The 

Model 650 uses an OKI compatible dot matrix printer with a standard parallel input to print reports. 

In addition to the report printer, the M650 supports an additional audit printer. The scanner stops if 

either printer fails. If one printer fails, the audit log automatically switches to the working printer. 

The iVotronic DRE is a touch screen terminal that will display different ballot styles and record 

votes. The iVotronic can display many different ballot styles and has an optional 4-position volume 

control on the main panel and audio button functionality. The iVotronic is available with a 12-inch, 

3-key terminal or a 15-inch 3-key, 4-key, and 6-key terminal. The 3-key terminal contains the 

Select and Scroll buttons; the 4-key terminal contains the Select, Scroll, and Volume Control 

buttons; and the 6-key terminal contains the Scroll buttons, Select button, Volume Control Button, 

Repeat Button, and Info Button; however, in the Unity. 4.0 release, the Repeat and Info buttons are 

not supported. The 15-inch iVotronic supports nine languages and the 12-inch iVotronic terminal 

supports eight languages; however, only Spanish was supported in the Unity 4.0 certification. The 

iVotronic is capable of presenting the ballot to voters in alternative languages—in the case of Unity 

4.0, Spanish. A Personalized Electronic Ballot (PEB) is used to activate the iVotronic, open polls, 

load ballots onto the iVotronic and collect all votes from each terminal at the end of an election day. 

The iVotronic is also capable of presenting a combination audio/visual ballot to the voter. This 

presentation displays the ballot on the iVotronic electronic display and when screen objects are 

selected (race/proposition titles, candidate name etc.), that selected screen object is read to the voter, 

in an audio format, through the use of headphones. Selections are then made through the use of the 

select button on the iVotronic tactile button array. This capability allows use of the terminal for 

voters who may be illiterate or whose native language has no written form. The Real-Time Audit 

Log (RTAL) Printer may be attached to an ES&S Unity 4.0 Voting System iVotronic configuration. 

It records each voter’s actions on a paper audit log in real time. This includes all selections and deselections. The paper audit log is viewed, but not touched by the voter. After a review of the paper 

audit entries and review screen on the iVotronic, a voter casts a vote. Undervoted contests and a 

two-dimension bar code of the votes are appended to the audit entries and the paper advances out of 

the view window. 

Audit Manager (AM) is ES&S’ tracking program for the Unity software suite. AM tracks user 

activity in AM, EDM and ESSIM. 

Election Data Manager (EDM) is a database system that stores all of a jurisdiction’s precinct, office, 

and candidate information. It is used in conjunction with other Unity software to format and print 

ballots, program ballot scanning equipment, and produce Election Day reports. 

ES&S Ballot Image Manager (ESSIM) is a publishing tool used to design and print ballots with the 

election information stored in EDM. 

Election officials use Ballot on Demand (BOD) to print test ballots, early voting ballots and ballots 

for polling places that run short of ballot stock on Election Day. 
 

The iVotronic Image Manager enables the user to create and format graphic ballot screens for the 

iVotronic voting device. 

Hardware Programming Manager enables the user to import, format, and convert the election 

definition files for ballot scanning equipment and DREs. 

The ES&S Data Acquisition Manager software is used to transmit election results over a network 

connection from ES&S ballot counting equipment to a central count location. 

The ABCR is a device that audits and recounts the printout generated by the iVotronic RTAL 

printer. The ABCR device interfaces with ABCR software installed on a PC to generate reports 

based upon the scanned barcodes from the RTAL printout. 

The Election Reporting Manager is an election results reporting program, used to generate paper and 

electronic reports for poll workers, candidates, and the media. ERM can display updated election 

totals on a monitor as ballot data is tabulated and can send result reports directly to media outlets 

over the Internet. ERM is designed to support a wide range of ES&S ballot scanning equipment and 

can produce reports for both central count systems and precinct count systems. 

The Compact Flash Multi-Card Reader/Writer (Gangburner) can be used to copy election data to 

multiple Compact Flash disks and also to read audit data files from multiple compact flash cards to a 

central directory location. 

The ES&S Unity 4.0 Voting System software is designed and developed to: 

 Define election databases for paper or electronic ballots; 

 Install election specific data on appropriate election hardware media (optical scanner or 

DRE); 

 Cast electronic or paper ballots; 

 Provide a voter verifiable paper audit tape for electronic ballots; 

 View and/or print audit reports; 

 Report election totals in the precinct; 

 Transmit results from precinct tabulators and remote regional sites; 

 Tabulate ballots at a central counting site; 

 Send central count vote data via a network (LAN for M650) and landline modem for the 

M100, DS200, and iVotronic) to the Election Reporting Manager (ERM) 

 Report and consolidate votes at a central location. 

Testing was done in conjunction with the ES&S AutoMARK Voting System. Following is a 

description of the AutoMARK systems: 

AIMS is designed and developed to: 
 

 Import election definition files from ES&S Unity 4.0; 

 Manually enter election definition information; 

 Maintain election specific information including precincts, splits, races and candidates; 

 Preview and verify all election data; 

 Write the election database to a secure compact flash memory card (FMC); 

 Record, review, and print real time audit logs. 

The FMC is used to supply ballot content information to the VAT. 

The VAT is an ADA compliant paper ballot-marking component of the ES&S AutoMARK Voting 

System. It enables a user to: 

 Install an election database that corresponds to the printed paper ballot via an FMC; 

 Independently insert a blank paper ballot for either visual or audio display; 

 Independently select candidate and contest options from a visual or audio ballot via a touch 

screen, tactile buttons or an alternative entry binary logic stereo jack input device; 

 Cast a ballot, resulting in the marking and ejection of the paper ballot; 

 Mark a paper ballot such that a Unity 4.0 tabulator programmed with the corresponding 

election database, can read and tabulate it. 

Functionality was tested, as identified in the Test Methodology table below. The following 

functional areas exist for ES&S and Unity 4.0. 

 Table 7 – Matrix of System Functional Testing 

Function Test Methodology 

Ballot Preparation Functions 

a. Ballot preparation subsystem Verify the election is defined for election day, and one 

more precinct/polling place can be defined. 

Before, During & After Processing of Ballots 

b.1. Logic Test – Interpretation of Ballot Styles & 

recognition of precincts 

Verify in Functional Tests: 

Verify voting variation functionality identified by 

ES&S for the ES&S Unity 4.0 voting system (Vol. 1. 

Section 2.2.8.2). 

b.2. Accuracy Tests- Ballot recording/reading 

accuracy 

Verify with the processing of 1,549,703 consecutive 

ballot positions with no errors, or 3,126,404 with one 

error (Vol. 2 Section 4.7.1.1). 

b.3. Status Tests- Equipment statement &memory 

contents 

Verify in Functional Tests: 

Equipment statement & memory contents at the 

corresponding intervals outlined in user documentation 

for the functions a. b.4, c 1-7 and d. 1-8 

b.4. Report Generation – Produce test output data Verify in Functional Tests: 

Clearing Election Totals 

Manual data entry 

Generating a Zero Report 

Testing an Election 

Creating Test Reports 

Clearing Totals for Election Day 
 

Function Test Methodology 

Selecting Reporting Groups 

Loading Scanner Totals 

Producing Election Reports 

Displaying Election Information 

ERM Election Results 

b 5. Report Generation- Produce audit data Verify in Functional Tests: 

System audit reports voting 

Polling Place Functions 

c.1. Opening the polls, accepting & counting ballots Verify in Functional Tests: 

Zero Reports 

Scan paper ballots 

Alerts for over votes and under votes 

c.2. Monitoring equipment status Verify in Functional Tests: 

Equipment status as identified in user documentation 

c.3. Equipment response to commands Verify in Functional Tests: 

Equipment response to all voter and poll worker 

commands as identified in user documentation 

c.4. Generating real-time audit messages Verify Verified in Functional Tests: 

Print audit log 

Each audit message contains a timestamp. 

Election name, software, and firmware are listed at the 

beginning of each audit log. 

Count of ballots processed is included in log of 

uploaded results. 

Error messages. 

Precinct ID is identified for all results pertaining to 

insertions, additions, and deletions. 

c.5: Closing polls and disabling ballot acceptance Verify in Functional Tests: 

Inability to cast additional ballots 

Close of polls 

Inability to scan additional ballots 

c.6. Generating election data reports. Verify in Functional Tests: 

Generation of precinct reports 

c.7. Transfer ballot count to central counting location Verify in Functional Tests: 

Reading the USB from the ERM 

Telecommunication 

c.8. Electronic transmission of election data to 

central count locations

Verify in Functional Tests: 

Confirming transmission, receipt, and validity of data 

interactively and with reports 

Central Count Functions 

d.1.Process ballot deck for > 2 precincts with 3 split 

precincts per precinct for a total of 6 ballot styles 

Verify in Functional Tests: 

Process of ballot decks on the 

d.2. Monitoring equipment status Verify in Functional Tests: 

Equipment status as identified in user documentation 

d.3. Equipment response to commands Verify in Functional Tests: 

Equipment responds to all voter and poll worker 

commands as identified in user documentation 

(Messages generated by the equipment that require an 

action by the voter or poll worker before operation 

continues--as in blank ballots, overvotes, undervotes as 

defined in election setup) 

.4. Integration with peripherals equipment or other See b.3 
 

Function Test Methodology 

data processing systems

d.5. Generating real-time audit messages. See b.4 

d.6. Generating precinct-level election data reports See b.3 

d.7. Generating summary election data reports See b.3 

d.8. Transfer of detachable memory module to the 

processing equipment

See b.3 

d.9. Electronic transmission of data to other 

processing equipment

Verify in Functional Tests: 

Confirming transmission, receipt, and validity of data 

interactively and with reports 

d.10. Producing output data for interrogation by 

external display devices 

Verify in Functional Tests: 

Confirming transmission, receipt, and validity of data 

interactively and with reports where possible 

Table 8 - Matrix of System Level and Other Functional Testing 

Other Functional Testing Test Methodology 

Volume Test 

System’s response to processing more than the expected 

number of ballots/voters per precinct, to processing 

more than the expected number of precincts, or to any 

other similar conditions that tend to overload the 

system’s capacity to process, store, and report data. 

Accuracy Test Case (described previously in this section) 

Stress Tests 

System’s responses to transient overload conditions. 

Subject polling place devices to ballot processing at the 

high volume rates, evaluate software response to 

hardware-generated interrupts and wait states. 

Hardware is tested to limits outside the range of ‘normal’ 

but within specifications for the units. 

Usability Tests 

Responses to input, text syntax, error message content, 

and audit message input 

All System-Level Test Cases 

Accessibility Test 

Exercises system capabilities of voters with disability 

features 

System-Level Test Case GEN 03 

Security Test 

Exercises systems security provisions, unauthorized 

access, deletion or modification of data, audit trail data, 

and modification or elimination of security mechanisms. 

Security Test case for each component (described 

previously in this section) 

Telecommunications Test 
 

Other Functional Testing Test Methodology 

Exercises telecommunications, maintaining data 

integrity, protection against external threats, monitoring 

and responding to external threats, shared operating 

environment, incomplete election returns, and use of 

public communications networks. 

Telecommunications Test case for each component 

Performance Tests 

Tests accuracy, processing rate, ballot format, handling 

capability and other performance attributes claimed by 

Premier 

All System Test Cases 

Recovery Tests 

Exercise system’s ability to recover from hardware and 

data errors. 

Security Test Case 

 Table 9 – Matrix of Additional Testing 

Test Case Execution 

Hi Capacity Ballot Test, 

11X36 ballot 

Using an all fill ballot definition (all left and right ballot 

positions utilized) vote the first and last ballot position in all 

contests on the 11X36 ballot (6 contest w/ 35 candidates). Scan 

the ballot on the scanners (M100, DS200, M650). Utilizing 

random orientations to test all orientations (M100 & DS200). 

Hi Capacity Ballot Test, 

14X36 ballot 

Using an all fill ballot definition (all left and right ballot 

positions utilized) vote the first and last ballot position in all 

contests on the 14X36 ballot (6 contest w/ 35 candidates). Scan 

the ballot on the scanners (M100, DS200, M650). Utilizing 

random orientations to test all orientations (M100 & DS200). 

Hi Capacity Ballot Test, 

14X48 ballot 

Using an all fill ballot definition (all left and right ballot 

positions utilized) vote the first and last ballot position in all 

contests on the 14X48 ballot (6 contest w/ 47 candidates). Scan 

the ballot on the scanners (M100, DS200, M650). Utilizing 

random orientations to test all orientations (M100 & DS200). 

Hi Capacity Ballot Test, 

17X45 ballot 

Using an all fill ballot definition (all left and right ballot 

positions utilized) vote the first and last ballot position in all 

contests on the 17X45 ballot (6 contest w/ 44 candidates). Scan 

the ballot on the scanners (M100, DS200, M650). Utilizing 

random orientations to test all orientations (M100 & DS200). 

Hi Capacity Ballot Test, 

17X60 ballot 

Using an all fill ballot definition (all left and right ballot 

positions utilized) vote the first and last ballot position in all 

contests on the 17X60 ballot (6 contest w/ 59 candidates). Scan 

the ballot on the scanners (M100, DS200, M650). Utilizing 

random orientations to test all orientations (M100 & DS200). 

Hi Capacity Ballot Test, 

19X51 ballot 

Using an all fill ballot definition (all left and right ballot 

positions utilized) vote the first and last ballot position in all 

contests on the 19X51 ballot (6 contest w/ 50 candidates). Scan 

the ballot on the scanners (M100, DS200, M650). Utilizing 

random orientations to test all orientations (M100 & DS200). 

Hi Capacity Ballot Test, 

19X68 ballot 

Using an all fill ballot definition (all left and right ballot 

positions utilized) vote the first and last ballot position in all 

contests on the 19X68 ballot (6 contest w/ 67 candidates). Scan 

the ballot on the scanners (M100, DS200, M650). Utilizing 

random orientations to test all orientations (M100 & DS200). 
 

Test Case Execution 

Expanded Precincts (M100) Using M100 firmware, create PCMCIA card for early voting 

containing 494 precincts on one card. Verify the card can be 

created and read into ERM. 

Expanded Precincts (DS200) Using DS200 firmware, create DS200 USB drive for early 

voting containing 494 precincts on one USB drive. Verify the 

card can be created and read into ERM. 

Auto Recovery Using the iVotronic Auto Recovery procedure v 9.1.0.0, vote an 

election and recover the results from the U2-D chip. (U2-D chip 

is a SanDisk). Manual provided and steps were completed, as 

only a trained ES&S technician completes this procedure. 

Maximum Candidates In ERM load election database "02PNELAN" with more than 

1000 candidates in a precinct. ERM limits 1000 counters in a 

single precinct. Verify that an attempt to load over 1000 

counters gives an error messages stating "Aborted-over 1000 

candidate in precinct: 211 ERM create results database failed. 

Connect election definition HPM and then retry." 

L&A Vote Selected Ballot 

Test 

Using ES&S test case "L&A Vote Selected Ballot Test" to 

verify the logic and accuracy vote selected ballot test. The voter 

selects a particular ballot to vote and that vote logic is applied to 

a select number of ballots designated for the voter to cast. 

L&A Multi-Vote Test Using ES&S test case "L&A Vote Multi-Vote Test" to verify 

the logic and accuracy of the multi vote test. Votes for each 

candidate will increase from one to the next, as in 1, 2, 3, 4, 5, 

etc. 

L&A Vote for One Test Using ES&S test case "L&A Vote For One Test" to verify the 

logic and accuracy of the vote for one test. Each candidate 

within a contest will receive one vote. There will be an 

additional undervote assigned in each contest. 

View Log Using an existing election (GEN01), select a user and verify 

Audit Manager has captured all activity(ies) carried out in all 

applicable applications (EDM, AM, and ESSIM). 

Opening the Polls Functions Use this test case to determine if the DS200 can open polls with 

an election definition that has more than one precinct. The 

HPM Report Level option must be ‘Precinct’. 

M100 with Older Model of 

Plastic Ballot Box 

Using ES&S test case "Model 100 with Plastic Ballot Box" to 

verify that the M100 can process ballots accurately when seated 

in the plastic ballot box. Note: Testing was completed with the 

Plastic Ballot Box; however, it cannot be certified as a part of 

the Unity 4.0 Suite, as it has not gone through Hardware 

Testing. 

Certification Test Results Summary 

1.10 Code Review Summary 

SysTest Labs has reviewed the ES&S software source code for Unity 4.0 to determine the source 

code’s compliance with the FEC VSS for source code according to the FEC VSS April 2002, 

Volume 1 Section 4, 8 and Volume 2 Section 2.2.5.2 and for compliance with ES&S’s internally 

developed coding standards as referenced in Section 1.9 of this document. All results of the source 

code review were documented in Excel spreadsheets. The source code that was delivered to SysTest 

Labs was in the form of: 
 

 Assembly 

 C/C++ 

 C# 

 VB 

 Cobol 

 Java 

The review was conducted for: 

 Software Integrity: The module contains no self-modifying code. Software remains 

unchanged and retains its integrity. The module has defined array dimensions, which are 

positive constant integers (Pointer variables, dynamic memory allocation and management 

are not applicable to Visual Basic.) 

 Modularity: The module has a specific testable function; performs a single function; is 

uniquely named; follows a standard format, has a single entry point; has a single exit point 

(or deviates in an acceptable manner); has error handling; and acceptable module size 

 Control Constructs: Logic flow utilizes standard constructs of the development language 

used; constructs are used consistently throughout the code; logic structure is not overly 

complex, and acceptable use of error handlers. 

 Naming Conventions: Variable and Function names that clearly define the purpose of the 

variable or function. Use of standard notation for variables by type. Use of names that are 

unique for both global and local variables. Use of names that are unique for functions 

(except where it deviates in an accepted manner). 

 Coding Conventions: Use of a standard methodology for the construction of a code 

module. This includes uniform calling sequences, parameter validation, a single executable 

statement per line, and status or error messages. 

 Comments Conventions: Comment Header blocks for the module / function follows a 

standard format in its layout and content. In code comments are clearly delineated and 

readable. 

Attachment C details specific information on the source code review. This information was 

submitted to ES&S for their review and resolution, if necessary, during the course of the 

review. 

Evaluation of Source Code 

The source code was reviewed for compliance per the guidelines defined in FEC VSS 2002 Volume 

2, Section 5.4. Overall, the source code written in [languages] was written adequately in terms of 

the FEC VSS April 2002. The code is generally modular and there is sufficient error handling. 

Readability is good and supports maintainability. 

The reviewer’s assessment is based on the following observations: 

 Software Integrity 

o There were no unbounded arrays. This follows the April 2002 FEC VSS 

requirements for software integrity. 
 

o No instances of self-modifying or dynamically loaded code were observed. 

 Modularity 

o The code is very modular and self-contained. 

o Modules perform only the specified functionality. 

o The requirement of single entry and exit points has been complied with. 

o Modules are small enough to facilitate ease of reading and understanding. 

 Control Constructs 

o Control Constructs used are in accordance with those allowed by the VSS. 

o Most loop control constructs have been appropriately chosen for the logical tasks to 

be accomplished. There are, however, instances of loop constructs written to include 

early termination logic other than by the normal loop exit condition specification. 

The interpretation of the currently written VSS requirement is that this early loop 

termination logic is not disallowed by the VSS but it is a variation of the construct 

other than that described by the standard and was accepted. As the code is currently 

written there will be no problems caused by those loop controls however, future 

changes to the code should be performed with some caution to ensure that the system 

state is stable. 

o Modules have fewer than 6 levels of indented scope. 

o Array boundaries are checked. 

 Naming Conventions 

o Function and variable names are in accordance with the requirements of the VSS. 

o Names differed by more than a single character and have been chosen as to enhance 

the readability of the code. 

o There were no instances of language keywords being used as a name for procedures 

or variables. 

 Coding Conventions 

o Coding conventions employed are in compliance with the requirements of the VSS. 

o Code was well structured and it was clear that it was written with the standards in 

mind. 

 Comments 

o Module headers are in compliance with the requirements of the VSS. 

o In-line comments are sufficient in number and placement to facilitate a good 

understanding of the code. 

o Variables have comments at the point of declaration. 

 On the Application level, no more than 50% of the modules can exceed 60 lines, no more 

than 5% can exceed 120 lines, and none can exceed 240 lines without justification. 

o Most of the functions/modules were within the FEC VSS tolerances; 
 

o 93.4% of the modules had less than 60 lines of code; 

o 5.4% had between 61 and 120 lines of code; 

o 1.2% had between 121 and 240 lines of code; and 

o 0.0% had more than 240 lines of code. 

The source code was reviewed for compliance per the guidelines defined in FEC VSS 2002, 

Appendix E, sections E.2.1 through E.2.6. 

The following line counts for each of the reviewed products were assessed: 

Table 10 – Source Code Review Assessment 

Language Less than 

60 

Between 

60 & 120 

Between 

120& 240 > 240 Total 

Assembly 99.4% 0.6% 0.0% 0.0% 100.0% 

C/C++ 91.2% 7.3% 1.5% 0.0% 100.0% 

C# 100.0% 0.0% 0.0% 0.0% 100.0% 

VB 93.1% 5.8% 1.1% 0.0% 100.0% 

Cobol 96.2% 3.0% 0.8% 0.0% 100.0% 

Java 95.8% 3.5% 0.7% 0.0% 100.0% 

Total % 93.4% 5.4% 1.2% 0.0% 100.0% 

1.11 Technical Data Package Review Summary 

SysTest Labs has reviewed the vendor’s TDP for Unity 4.0 for compliance with the FEC VSS April 

2002 according to Volume 2 Section 2. The specific documents are listed in Section 1.9 of this 

document. 

The review was conducted for the required content and format of the: 

 System Overview: System description and performance

 System Functionality Description: System functional processing capabilities, 

encompassing capabilities required by the Standards and any additional capabilities provided 

by the system, including a simple description of each capability. 

 System Hardware Specification: System Hardware Characteristics, Design and 

Construction 
 

 Software Design and Specification: Purpose and scope, applicable documents, software 

overview, software standards and conventions, software operating environment, software 

functional specification, programming specifications, system database, interfaces and 

appendices. 

 System Security Specification: Access control policy and measures, equipment and data 

security, software installation, telecommunications and data transmission security, elements 

of an effective security program. 

 System Test and Verification Specifications: Development and certification test 

specifications 

 System Operations Procedures: Operation environment, system installation and test 

specifications, operational features, operating procedures, operations support. 

 System maintenance Procedures: Preventative and corrective maintenance procedures, 

maintenance equipment, facilities and support. 

 Personnel Deployment and Training Requirements

 Configuration Management: Configuration management policy, configuration 

identification, procedures for baseline, promotion, demotion and configuration control, 

release process, configuration audits and management resources, 

 Quality Assurance Program: Quality assurance policy, parts and materials special testing 

and examination, quality conformance inspections

 System Change Notes: Changes to the system if it was previously qualified

Evaluation of TDP

The Technical Data Package for Unity 4.0 was found to comply with the April 2002 FEC VSS. 

Attachment B details specific information on the TDP review. This information was submitted to 

ES&S for their review during the course of the project. 

1.12 Application Functional Test Summary 

SysTest Labs executed validation of the ES&S Unity 4.0 Voting System using the standard system 

level test cases where possible. Functional Test scripts for ES&S Unity 4.0 Voting System were 

also executed and testing was conducted on the equipment provided by ES&S, listed in Table 1 – 

Summary of Unity 4.0 Voting System Components. The ES&S Unity 4.0 Voting System was tested 

for Security, Audit Log capabilities, Tabulating, Transmitting, and Accuracy. A listing of the 

Functional Tests completed by SysTest Labs is included Table 9 – Matrix of Additional Testing. 

Evaluation of Functional Testing

The ES&S Unity 4.0 Voting System Functional Test scripts were found to comply with the April 

2002 FEC VSS. Any critical issues found were reported, resolved and re-tested. Table 9 – Matrix 

of additional Testing details specific information on the functional tests. 

SysTest Labs executed functional tests as listed below. Issues identified during testing were reported 

to ES&S in the Unity 4.0 Discrepancy Report. Resolutions were submitted to SysTest Labs, and 

retested to validate acceptable resolution. 
 

Summary of Functional Tests: 

Test Case Name: Readiness Test 

Passed/Failed: Passed 

Summary: Readiness is a functional test case used to validate the entire voting system is ready for 

testing. Readiness testing is used to verify the voting system functions properly, to confirm that the 

equipment has been properly integrated, to obtain equipment status reports, to make sure all voting 

system equipment is properly prepared for an election, and to collect data that verifies equipment 

readiness. The Readiness test case is also used to verify that status and data reports from each set of 

equipment can be obtained, ensure correct installation/interface of all system equipment, verify that 

hardware and software function correctly, and check that consolidated data reports at the polling 

place and higher jurisdictional levels can be generated. 

The Readiness test is executed on initial testing and each time the system is powered down and 

restarted. 

The pass/fail criteria for this test case reflected the system meeting the expected results from the test 

case, and validated the VSS requirements identified in the test case and the Test Plan. 

Test Case Name: Functional Operation Status Test 

Passed/Failed: Passed 

Summary: The Operational test is a comprehensive end-to-end test case(s) verifying the machines’ 

functions are behaving correctly, and operating in the correct modes. SysTest Labs performs the 

operational status check (provided by the vendor) once upon acceptance of the equipment and once 

after all other testing. During this process, all equipment should operate in environmental conditions 

that simulate election use to verify the functional status of the system. Prior to the conduct of each 

of the environmental hardware tests, a supplemental test determines that the operational state of the 

equipment is within acceptable performance limits. 

Summary of instructions for this test case: Turn on power, and allow the system to reach 

recommended operating temperature. Operate the equipment in all modes; demonstrate all functions 

and features that would be used during election operations. Verify all system functions have been 

correctly executed. 

The following vendor provided documentation was utilized: DS200 Precinct Scanner, Pre-Election 

Day Checklist, v 1.x, Release date: 04/2007, Revision date: 04/2007; iVotronicTM Voting System 

Pre-Election Day Operations Checklist, Poll Worker Activated, v 9.2.x, Revision date: 06/2007; 

Model 100 Precinct Scanner, Pre-Election Day Checklist, v 5.x (includes v 5.4), Revision date: 

06/2007; Model 650 Central Scanner, Pre-Election Day Checklist, v 2.x, Revision date: 01/2007; 

Model 650 Pre-Election Day Setup-Handout A, Firmware Version 2.2.1.0, Hardware Version 1.1, 

June 2007. 

All information used in processing the test case was captured. This includes inputs, outputs, 

deviations and any other item that may impact the validation of the test case. Any failure of the test 

against the EAC guidelines is reported and implies failure of the system. Failures are reported as 

defect issues in the discrepancy report and are provided to the manufacturer. Before the final 

Certification Test Report is issued, manufacturers are given the opportunity to correct all 

discrepancies. If the manufacturer submits corrections, retests are performed. 

The first iteration, Rev01, was executed and passed. 
 

Test Case Name: 40HTEST1 

Passed/Failed: Passed 

Summary: 40HTEST1 is a system level test that uses the 2002 Voting System Standards (VSS) 

guidelines to validate required functionality and performance. Testing includes accuracy, ballot 

format handling capability, reporting, and usability of the hardware, software, and procedures in the 

Unity 4.0 voting system, which includes the following: Election Data Manager (EDM); iVotronic 

Image Manager (iVim); Hardware Programming Manager (HPM); Audit Manager (AM); Election 

Reporting Manager (ERM); ES&S Ballot Image Manager (ESSIM); Model 100 Ballot Scanner 

(M100); intElect DS200 Ballot Scanner (DS200); iVotronic DRE; and the Model 650 Optical Scan 

central Count Counter (M650). 

40HTEST1 contained 20 contests and a wide variety of races and candidate positions including: US 

Senator, 2 candidates; Secretary of State, 3 candidates; Auditor of State, 2 candidates; Treasurer of 

State, 2 candidates; US Representative in Congress, 2 candidates; State Representative, 1 candidate; 

Judge of Circuit Court, 1 candidate; Prosecuting Attorney, 2 candidates; County Auditor, 1 

candidate; County Treasurer, 2 candidates; County Sheriff, 1 candidate; County Assessor 1 

candidate; County Commissioner, 1 candidate; County Council Member, 5 candidates; Township 

Trustee, 22 candidates; Township Board Member, 59 candidates; Justice of the Indian Supreme 

Court (Recall/Retain) Y/N; Judge Court of Appeals, 2nd District (Recall/Retain) Y/N; Judge Court of 

Appeals, 4th District (Recall/Retain) Y/N. 

40HTEST1 ballots were voted and report tapes tallied with 100% accuracy; no functional 

discrepancies occurred. The test consisted of Rev01 and passed on the first iteration. 

Test Case Name: 40HTEST3 

Passed/Failed: Passed 

Summary: 40HTEST3 is a functional test case that uses the 2002 Voting System Standards (VSS) 

guidelines to validate required functionality and performance. The object of this test case is to 

verify the on-screen message changes (overvote message is suppressed in the state of Florida) when 

the two different election definitions are utilized containing two different state codes. Testing 

includes accuracy, ballot format handling capability, and usability of the hardware, software, and 

procedures in the following areas of the voting system: Election Data Manager (EDM) v. 7.8.0.0; 

Hardware Programming Manager (HPM) 5.7.0.0; Audit Manager (AM) 7.5.0.0; ES&S Ballot Image 

Manager (ESSIM) 7.7.0.0; Model 100 Ballot Scanner (M100) 5.4.0.0; and intElect DS200 Ballot 

Scanner (DS200) v. 1.3.7.0. 

40HTEST3 consisted of Rev01 and passed on the first iteration. 

Test Case Name: 40HTEST4 

Passed/Failed: Passed 

Summary: 40HTEST4 is a functional test case that uses the 2002 Voting System Standards (VSS) 

guidelines to validate required functionality and performance. The object of this test case is to 

create a subset election in HPM (from an existing election), ensure media can be burned, an election 

loaded on an M650, and a ballot can be accepted. Testing includes accuracy, ballot format handling 

capability, and usability of the hardware, software, and procedures in the following areas of the 

voting system: Election Data Manager (EDM) v. 7.8.0.0; Hardware Programming Manager (HPM) 
 

5.7.0.0; Audit Manager (AM) 7.5.0.0; and the Model 650 Optical Scan central Count Counter 

(M650) v. 2.2.1.0. 

40HTEST4 consisted of Rev01 and passed on the first iteration. 

Test Case Name: 40HTEST5 

Passed/Failed: Passed 

Summary: 40HTEST5 is a functional test case that uses the 2002 Voting System Standards (VSS) 

guidelines to validate required functionality and performance. The object of this test case is to 

verify HPM can accept an Open Primary election with greater than nine Party Preference contests 

and 100 precincts. Testing includes accuracy, ballot format handling capability, and usability of the 

hardware, software, and procedures in the following areas of the voting system: Election Data 

Manager (EDM) v. 7.8.0.0; Hardware Programming Manager (HPM) v. 5.7.0.0; and Audit Manager 

(AM) 7.5.0.0 

40HTEST5 consisted of Rev01 and passed on the first iteration. 

Test Case Name: 3000 PCTS 

Passed/Failed: Passed 

Summary: 3000 PCTS is a functional test case that uses the 2002 Voting System Standards (VSS) 

guidelines to validate required functionality and performance. The object of this test case is to 

verify media can be burned for the iVotronic, a ballot can be loaded, voted, cancelled, and the polls 

can be closed. The election was a General with Straight Party containing more than 150 contests 

and greater than 1300 precincts. 

Testing included ballot format handling capability, usability of the hardware, software, and 

procedures in the following areas of the voting system: Election Data Manager (EDM) v. 7.8.0.0; 

Hardware Programming Manager (HPM) v. 5.7.0.0; Audit Manager (AM) v. 7.5.0.0; ES&S Ballot 

Image Manager (ESSIM) v 7.7.0.0; iVotronic Image Manager (iVim) v. 3.2.0.0; and the iVotronic 

DRE v. 9.2.3.0. 

3000 PCTS consisted of Rev01 and passed on the first iteration. 

Test Case Name: Hi Capacity Ballot Test, 11X36 ballot 

Passed/Fail: Passed 

Summary: SysTest Labs conducted functional system testing on the Unity 4.0 software. The system 

tests were conducted to verify that all the components operate as designed, that the system operates 

as expected in a user environment and other programs associated with Unity 4.0 work correctly. 

Using an all fill ballot definition (all left and right ballot positions utilized) we voted the first and 

last ballot position in all contests on the 11X36 ballot (6 contests w/ 35 candidates). We scanned the 

ballot on the scanners (M100, DS200, M650). Two orientations were run for the M100 and DS200. 

No issues were found in running the 11X36 Hi Capacity Ballot Test. 

Test Case Name: Hi Capacity Ballot Test, 14X36 ballot 

Passed/Fail: Passed 

Summary: SysTest Labs conducted functional system testing on the Unity 4.0 software. The system 

tests were conducted to verify that all the components operate as designed, that the system operates 
 

as expected in a user environment and other programs associated with Unity 4.0 work correctly. 

Using an all fill ballot definition (all left and right ballot positions utilized) we voted the first and 

last ballot position in all contests on the 14X36 ballot (6 contests w/ 35 candidates). We scanned the 

ballot on the scanners (M100, DS200, M650). Two orientations were run for the M100 and DS200. 

No issues were found in running the 14X36 Hi Capacity Ballot Test. 

Test Case Name: Hi Capacity Ballot Test, 14X48 ballot 

Passed/Fail: Passed 

Summary: SysTest Labs conducted functional system testing on the Unity 4.0 software. The system 

tests were conducted to verify that all the components operate as designed, that the system operates 

as expected in a user environment and other programs associated with Unity 4.0 work correctly. 

Using an all fill ballot definition (all left and right ballot positions utilized) we voted the first and 

last ballot position in all contests on the 14X48 ballot (6 contests w/ 47 candidates). We scanned the 

ballot on the scanners (M100, DS200, M650). Four orientations were run for the M100 and DS200. 

No issues were found in running the 14X48 Hi Capacity Ballot Test. 

Test Case Name: Hi Capacity Ballot Test, 17X41 ballot 

Passed/Fail: Passed 

Summary: SysTest Labs conducted functional system testing on the Unity 4.0 software. The system 

tests were conducted to verify that all the components operate as designed, that the system operates 

as expected in a user environment and other programs associated with Unity 4.0 work correctly. 

Using an all fill ballot definition (all left and right ballot positions utilized) we voted the first and 

last ballot position in all contests on the 17X41 ballot (6 contests w/ 44 candidates). We scanned the 

ballot on the scanners (M100, DS200, M650). Four orientations were run for the M100 and DS200. 

No issues were found in running the 17X41 Hi Capacity Ballot Test. 

Test Case Name: Hi Capacity Ballot Test, 17X45 ballot 

Passed/Fail: Passed 

Summary: SysTest Labs conducted functional system testing on the Unity 4.0 software. The system 

tests were conducted to verify that all the components operate as designed, that the system operates 

as expected in a user environment and other programs associated with Unity 4.0 work correctly. 

Using an all fill ballot definition (all left and right ballot positions utilized) we voted the first and 

last ballot position in all contests on the 17X45 ballot (6 contests w/ 44 candidates). We scanned the 

ballot on the scanners (M100, DS200, M650). Two orientations were run for the M100 and DS200. 

No issues were found in running the 17X45 Hi Capacity Ballot Test. 

Test Case Name: Hi Capacity Ballot Test, 17X60 ballot 

Passed/Fail: Passed 

Summary: SysTest Labs conducted functional system testing on the Unity 4.0 software. The system 

tests were conducted to verify that all the components operate as designed, that the system operates 

as expected in a user environment and other programs associated with Unity 4.0 work correctly. 

Using an all fill ballot definition (all left and right ballot positions utilized) we voted the first and 

last ballot position in all contests on the 17X60 ballot (6 contests w/ 59 candidates). We scanned the 
 

ballot on the scanners (M100, DS200, M650). Four orientations were run for the M100 and DS200. 

No issues were found in running the 17X60 Hi Capacity Ballot Test. 

Test Case Name: Hi Capacity Ballot Test, 19X51 ballot 

Passed/Fail: Passed 

Summary: SysTest Labs conducted functional system testing on the Unity 4.0 software. The system 

tests were conducted to verify that all the components operate as designed, that the system operates 

as expected in a user environment and other programs associated with Unity 4.0 work correctly. 

Using an all fill ballot definition (all left and right ballot positions utilized) we voted the first and 

last ballot position in all contests on the 19X51 ballot (6 contests w/ 50 candidates). We scanned the 

ballot on the scanners (M100, DS200, M650). Four orientations were run for the M100 and DS200. 

No issues were found in running the 19X51 Hi Capacity Ballot Test. 

Test Case Name: Hi Capacity Ballot Test, 19X68 ballot 

Passed/Fail: Passed 

Summary: SysTest Labs conducted functional system testing on the Unity 4.0 software. The system 

tests were conducted to verify that all the components operate as designed, that the system operates 

as expected in a user environment and other programs associated with Unity 4.0 work correctly. 

Using an all fill ballot definition (all left and right ballot positions utilized) we voted the first and 

last ballot position in all contests on the 19X68 ballot (6 contests w/ 67 candidates). We scanned the 

ballot on the scanners (M100, DS200, M650). Four orientations were run for the M100 and DS200. 

No issues were found in running the 19X68 Hi Capacity Ballot Test. 

1.13 System Level (Integration) Test Description 

System level testing consisted of a set of standard system level regression tests customized for each 

system configuration. This incorporated end-to-end election scenarios testing the functionality 

supported, as identified in the System Overview. 

SysTest Labs executed System Level Integration Tests for the multiple system configurations of 

ES&S Unity 4.0. The tests validated all supported voting variation functionality and languages in 

mock elections. Testing included validation of the voter facing messages for both audio and visual 

format for all supported languages. Issues identified during testing were reported to ES&S in the 

Discrepancy Report. Resolutions were submitted to SysTest Labs, and retested to validate 

acceptable resolutions. 

Summary of System-Level Test Cases: 

Test Case Name: GEN01 

Passed/Fail: Passed 

Summary: GEN01 is a system-level general election test case that includes core and optional 

functionality and features (as defined in the Test Plan). Accuracy in defining an election through 

tallying results at central count, ballot format handling capability, reporting, and usability of the 

hardware, software and procedures in the entire voting system are verified in this test case. 
 

The election consisted of two precincts with three splits per precinct, 11 contests containing nonpartisan, partisan, recall types A and B, and a proposition/question. A “standard” rotation was 

applied to one of the contests. 

The AM application, which runs in the background and keeps an audit log of all actions performed 

in EDM and ESSIM, was started prior to any test step execution. The election was coded in EDM, 

ballot layout was designed in ESSIM (for paper ballots) and iVIM (for the DRE ballot), and the 

media was created in HPM. Electronic ballots were voted on the iVotronic DRE. Paper ballots 

were scanned on the DS200, M100, and M650 scanners. The results from the DS200, M100, and 

iVotronic were modemed utilizing DAM (Precinct) modem to DAM Host into ERM. The M650 

results were networked to ERM on a LAN via TCP/IP. 

Testing consisted of verifying versions of Software and Firmware, executing the test steps that were 

defined using the System Operations Manuals for all applications and devices, validating various 

reports available from the applications and devices, voting the election using 11 voters for the M100, 

DS200, M650, and 14 voters for the iVotronic, in a predefined and repeatable pattern to simulate 

various voters’ choices, verifying the totals of all reports generated from the individual scanner tapes 

and ERM to a predicted result in the test case, and verification of the VSS requirements as defined in 

the Test Plan. 

Special features that were tested in this election for the M650 include 1) Volatile Flush Header, 

which is used to zero the totals in the volatile memory. Users would utilize this option on the M650 

if they inadvertently counted absentee ballots in a duplicate set of precincts to “flush” or clear both 

the Election Day and absentee ballot counts in that precinct. 2) Creation of 10 node folders, which 

are traditionally used in a network M650 environment. Each M650 is assigned a scanner ID. When 

the M650 operator saves the results, the results are automatically transferred to the election server 

and stored in a node folder with the same number as the scanner ID. 3) M650 early voting group. 

This group is created in the Add/Change Groups of the ERM. Early voting ballots are processed on 

the M650 and then read into the ERM group called Early Voting. For the iVotronic, we defined an 

on screen vote button, which places a vote button display on the iVotronic touch screen. Coded 

ballot functionality was also verified on the iVotronic by ensuring that the ‘Provisional’ prompt 

appeared and separate totals were generated. A coded ballot is given to a voter whose eligibility to 

vote is in question. The iVotronic stores ballot codes in the terminal’s audit data. Election Officials 

can review the coded ballot for eligibility after the polls close. We also verified the “Ballot Control” 

option in HPM for the M100 and DS200. For this election, we set this option to “Reject” the voter 

for any overvoted, cross-voted, blank, or unreadable marks on the ballot(s). 

The pass/fail criteria for this test case were dependent on they system meeting the expected results 

from the test case, and validated the VSS requirements identified in the test case and the Test Plan. 

There were three iterations of the test case, due to discrepancies encountered. Rev 01 failed during 

creation of the media in HPM (see discrepancy # 177). Rev 02 failed utilizing DAM to modem 

iVotronic results into ERM (see discrepancy 240), and while creating reports in ERM (see 

discrepancies 278 and 279). Rev 03 was executed and passed after all of the above discrepancies 

had been resolved, and a new trusted build of DAM was installed and testing successfully executed. 

Test Case Name: GEN02 – Straight Party 

Passed/Fail: Failed – Please refer to iBeta’s Test Report 

Summary: GEN02 is a system level, straight party test case that includes core and optional 

functionality and features (as defined in the Test Plan). The test case is designed to test accuracy in 
 

defining an election, creating election ballots, voting, and tallying results, for a General Election 

with straight party. Ballot format handling capability, reporting, and usability of the hardware, 

software and procedures in the entire voting system are also verified in GEN02. 

The election consisted of a two-page ballot, 7 precincts, and crossover voting, Recalls Type A, B, 

and C, a proposition, questions, and the precinct rotation type. 

The AM application, which runs in the background and keeps an audit log of all actions performed 

in EDM and ESSIM, was started prior to any test step execution. The election was coded in EDM, 

ballot layout was designed in ESSIM (for paper ballots) and iVIM (for the DRE ballots), and the 

media was created in HPM. The electronic ballots were voted on the iVotronic DRE using Seiko 

printer. Paper ballots were scanned on the DS200, M100 and M650 scanners. The results from the 

M100 were read directly into ERM using the PCMCIA card. The results from the DS200 were read 

directly into ERM using the USB flash drive and iVotronic results were read directly into ERM 

using the PEB. The M650 results were networked into ERM using a LAN via TCP/IP (Central). 

Testing consisted of verifying versions of Software and Firmware, executing the test steps that were 

defined using the System Operations Manuals for all applications and devices, validating various 

reports available from the applications and devices, voting the election using 28 voters for the M100, 

DS200, M650, and 24 voters for the iVotronic, in a predefined, repeatable pattern to simulate 

various voters’ choices, and verification of the VSS requirements as defined in the Test Plan. 

Special features that were tested in this election include preventing or disabling the resetting of the 

counter by any person other than authorized persons at authorized points, and verifying the ballot 

count is only visible to designated election officials. 

Coded ballot functionality was also verified on the iVotronic by ensuring that the ‘Provisional’ 

prompt appeared and separate totals were generated. A coded ballot is given to a voter whose 

eligibility to vote is in question. The iVotronic stores ballot codes in the terminal’s audit data. 

Election Officials can review the coded ballot for eligibility after the polls close. 

The pass/fail criteria for this test case were dependent on the system meeting the expected results 

from the test case, and validated the VSS requirements identified in the test case and the Test Plan. 

There have been three iterations of this test case and this test case has not yet passed. 

The first two iterations failed due to discrepancies #177 (any referendum must be set up in the same 

column on the ballot in HPM, or it will not tabulate correctly on the M650) and #239 (Statistical 

Counters did not function correctly on the M650 Grand Totals and Precinct by Precinct reports). 

These discrepancies have since been closed. 

The third iteration was due to discrepancy #297 (the District Canvass report totals were not 

matching our vote data tab). The vendor has responded that documentation needs to be updated. 

Once this documentation can be reviewed and verified, the discrepancy can be closed and the test 

case is expected to pass. 

Test Case Name: GEN02- PA Straight Party 

Passed/Fail: Passed 
 

Summary: GEN02PA is a system-level general election test case with a Pennsylvania straight party 

option. This test case is designed to test accuracy in defining an election, creating election ballots, 

voting, and tallying results for a General Election with Pennsylvania straight party functionality. 

Ballot format handling capability, reporting, and usability of the hardware, software and procedures 

in the entire voting system are also verified in GEN02PA. 

The election consisted of seven precincts with no split precincts; 12 contests containing nonpartisan, partisan, recall types A, B, and C, and a proposition/question. A “Votronic Auto” rotation 

was applied to certain contests. 

The AM application, which runs in the background and keeps an audit log of all actions performed 

in EDM and ESSIM, was started prior to any test step execution. The election was coded in EDM, 

ballot layout was designed in ESSIM (for paper ballots) and iVIM (for the DRE ballots), and the 

media was created in HPM. The electronic ballots were voted on the iVotronic DRE. Paper ballots 

were scanned on the M100 and M650 scanners. The results from the M100 and iVotronic were 

modemed utilizing DAM (Precinct) modem to DAM Host into ERM. The M650 results were read 

directly into ERM using the zip disk. 

Testing consisted of verifying versions of Software and Firmware, executing the test steps defined 

using the System Operations Manuals for all applications and devices, validating various reports 

available from the applications and devices, voting the election using 24 voters for the M100, M650, 

and 23 voters for the iVotronic, in a predefined, random, repeatable pattern to simulate various 

voters’ choices, verifying the totals of all reports generated from the individual scanner tapes and 

ERM to a predicted result in the test case, and verification of the VSS requirements as defined in the 

Test Plan. 

Special features tested in this election include 1) Two-page ballot election per voter and 2) Straight 

party (multi-member board).

The pass/fail criteria for this test case were dependent on the system meeting the expected results 

from the test case, and validated the VSS requirements identified in the test case and the Test Plan. 

GEN02PA consisted of two iterations and three discrepancies were discovered during testing. Upon 

further research, it was discovered that two of the three discrepancies were due to tester error and the 

discrepancies were closed. The third discrepancy was #177 (any referendum must be set up in the 

same column, on the ballot in HPM, or it will not tabulate correctly on the M650). This discrepancy 

consisted of inaccurate instructions of the HPM System Operations document. The vendor corrected 

this document and the added information into the ESSIM Systems Operations document, which 

were reviewed and verified. Upon verification, the discrepancy was closed and the test case passed. 

Test Case Name: GEN03 – Add Languages 

Passed/Failed: Failed– Please refer to iBeta’s Test Report 

Summary: GEN03 is a system-level general election test case that includes core and optional 

functionality and features (as defined in the Test Plan) and Accessibility. Accuracy in defining an 

election through tallying results at central count, ballot format handling capability, reporting, error 

messages, languages (Spanish), usability, and accessibility of the hardware, software and procedures 

in the entire voting system are verified in this test cast. 

The election consisted of 1 precinct with provisional/challenged ballots; 10 contests containing NonPartisan contests (Vote for 1 of M-Sheriff; Proposition/Question-Proposition X), Partisan contests 
 

(Multi-member board, “Vote for 3 of M” race with declared candidates with a voting position 

defined for write-in: City Council), Type D, and Type C. 

Accessibility was tested using the visual, audio, and a sip & puff terminal. The testers simulated the 

disability to test these functions. Audio, Combo (Audio & Visual), and Sip & Puff were 

meticulously tested for correct functionality, usability, and accuracy and all applicable VSS 

requirements. Measurements of the terminal were taken to validate the requirements for 

Accessibility listed in VSS Volume 1 section 2.2.7. 

The secondary language tested during testing was Spanish. It was verified that the electronic display 

did reflect Spanish. Also, it was validated that the appropriate WAV files were played. Accessibility 

was tested in Spanish as well. 

The AM application, which runs in the background and keeps an audit log of all actions performed 

in EDM and ESSIM, was started prior to any test step execution in any of the other applications 

comprising the Unity 4.0 suite. The election was coded in EDM, ballot layout was designed in iVIM 

for the DRE ballot, and the media was created in HPM. The election was voted on the iVotronic 

DRE. Printed reports were collected from the iVotronic terminals and ERM. 

Testing consisted of verifying versions of Software and Firmware, executing the test steps that were 

defined using the System Operations Manuals for all applications and devices, validating various 

reports available from the applications and devices, voting the election using 63 voters (4.0 Testing), 

77 voters (4.0 Heavy Rev. 1), 62 voters (4.0 Heavy Rev. 2) and 29 voters (4.0 Heavy Rev. 3) for the 

iVotronic, in a predefined, random, but repeatable pattern to simulate various voters’ choices, 

verifying the totals of all reports generated from the individual scanner tapes and ERM to a 

predicted result in the test case, and verification of the VSS requirements as defined in the Test Plan. 

Poll worker and Voter facing system messages were also verified against documentation and audit 

logs. 

Special features that were tested in this election include 1) Multi-language ballots (English and 

Spanish), 2) Audio/Visual/Combo ballots, 3) 15” iVotronic with 3-key, 4-Key, 6-Key (6-Key 

supports sip and puff), 4) 12” iVotronic 3-Key, 5) VVPAT printer (Real Time Audit Log- RTAL), 

6) Error Messages and Recovery. The VVPAT printer was tested to verify accuracy of ballots cast 

and verify cancellation of ballots due to any disturbance of the printer. User error messages were 

validated by attempting to cast undervotes, overvotes, blank ballots, type C ballots where the 

question is not answered, attempting to cast ballot prior to reviewing summary, inactivity of 

terminal, and disconnecting the printer cable. 

The pass/fail criteria for this test case were dependent on the system meeting the expected results 

from the test case, and validated the VSS requirements identified in the test case and the Test Plan. 

There were four iterations of the test case executed, due to discrepancies encountered. 

4.0 (Firmware 9.2.0.0) failed during Election Day voting on the DRE. Discrepancies discovered in 

4.0 testing were: 

 #190 Automatically accepting a write-in after a maximum number of characters entered 

 #191 Audio volume level does not stay at the set volume during a combo ballot 

 #193 Inability to return to the ballot after the vote button was pushed on an incomplete ballot 

 #194 to #197 Incorrect audio instructions 
 

 #198 Volume button restarts the instruction and increases volume during summary mode 

 #224 Inability to cast a blank ballot 

 #226 Lack of instruction for the “purple button” 

 #227 Audio stopped playing 

 #235 Instructions Combo screen is not fully displayed on Spanish Ballots 

 #236 Incorrect message when the RTAL printer was unplugged 

 #237 Not all characters displayed on the contest screen in zoom mode 

 #238 The write-in display does not reflect text size selected 

 #245 Incorrect message when the RTAL printer was unplugged 

 #246 Duplicate of #237 

 #247 Incorrect audio message 

 #248 Arrow keys do not work in a combo ballot 

4.0 Heavy Rev 01 (Firmware 9.2.1.0) failed during Election Day voting on the DRE and Printing of 

Reports. During this revision the following discrepancies were corrected and closed: 193, 194, 197, 

224, 226, 227, 245, 246, 247, and 248. The major issue found during 4.0 Heavy Rev. 01 was the 

discovery that the voter could continue to cast a ballot even when the RTAL printer was 

disconnected during the process. Discrepancies discovered during 4.0 Heavy Rev 01 were: 

 #434 Error message was received when inserting a new compact flash and PEB 

 #441 Further instructions for the voter needed for Combo (Audio/Visual) ballot 

 #442 Audio issue 

 #443 Warning message received during the opening polls process 

 #445 Option on “Configure Terminal” menu was not available 

 #448 On a Spanish audio ballot, incorrect instructions were given 

 #449 Votes casted after the RTAL Printer malfunction 

 #450 Incorrect messages received when RTAL printer paper was low 

 #451 Terminal did not behave has the instructions provided. 

 #458 Spanish instructions are not provided while the voter selects ballot type. 

 #464 On a multiple contest, the summary did not display the multiple candidates selected 

 #465 Incorrect audio instructions 

 #466 Incorrect audio instructions 

 #467 Inconsistent instruction during initial navigation instruction 

 #468 Incorrect ballot totals 

 #469 Incorrect time/date provided by ERM compared to the audit log provided by the 

iVotronic. 

ERM Functional discrepancies found:

 #472 Issue with audit data total 

 #473 Incorrect vote data totals 

4.0 Heavy Rev. 02 (Firmware 9.2.2.0) failed during Election Day voting on the DRE and Printing of 

Reports (No new discrepancies were found or closed). The 12” iVotronic terminal could not be 

tested as the vendor was researching discrepancies. As such, discrepancies could not be closed as 
 

they needed validation on the 12” iVotronic terminal. It was discovered at this iteration that #472 & 

#473 were documentation issues. 

4.0 Heavy Rev 03 (Firmware 9.2.3.0) failed during Election Day voting on the DRE and Printing of 

Reports (No new discrepancies were found). The following discrepancies were corrected and 

closed: 443, 448, 449, and 465. At the end of this test cycle, a total of 16 functional discrepancies 

remain open. 

Discrepancies #194, #196, #227, #245, #247 and #248 found in the 4.0 testing, are applicable to the 

accessibility test done concurrently with the GEN03 testing. During GEN03 Heavy Rev 01, the 

following discrepancies were closed for accessibility: #194, #227, #247, and #248. Discrepancies 

#441, #466, and #467 were discovered during these iterations and were applicable to the 

accessibility test. The accessibility test failed with 6 outstanding discrepancies. 

Test Case Name: PRI01 – Open Primary 

Passed/Failed: Passed 

Summary: PRI01 is a system-level primary election test case that includes core and optional 

functionality and features (as defined in the Test Plan) for an Open Primary. The test case is 

designed to test accuracy in defining an election, creating election ballots, voting, and tallying 

results for an Open Primary. Ballot format handling capability, reporting, and usability of the 

hardware, software and procedures in the entire voting system are also verified in PRI01. 

The election consisted of five precincts, 6 contests containing non-partisan, partisan, and a primary 

presidential nomination of candidates. A “Districts by Registered Voters Non-Partisan” rotation was 

applied to one of the contests. 

The AM application, which runs in the background and keeps an audit log of all actions performed 

in EDM and ESSIM, was started prior to any test step execution in any of the other applications 

comprising the Unity 4.0 suite. The election was coded in EDM, ballot layout was designed in 

ESSIM (for paper ballots) and iVIM (for the DRE ballot), and the media was created in HPM. The 

election was voted on paper and the iVotronic DRE. Paper ballots were scanned on the M100 and 

M650 scanners. The results from the M100, and iVotronic were modemed utilizing DAM (Precinct) 

modem to DAM Host into ERM. Results from the M650 were imported from a Zip Disk to ERM. 

Testing consisted of verifying versions of Software and Firmware, executing the test steps that were 

defined using the System Operations Manuals for all applications and devices, validating various 

reports available from the applications and devices, voting the election using 15 voters for the M650, 

16 voters for the M100 (due to ballot being scanned twice), and 15 voters for the iVotronic, in a 

predefined and repeatable pattern to simulate various voters’ choices, verifying the totals of all 

reports generated from the individual scanner tapes and ERM to a predicted result in the test case, 

and verification of the VSS requirements as defined in the Test Plan. 

The pass/fail criteria for this test case were dependent on the system meeting the expected results 

from the test case, and validated the VSS requirements identified in the test case and the Test Plan. 

Only one discrepancy, #177 (any referendum must be set up in the same column on the ballot in 

HPM, or it will not tabulate correctly on the M650), existed in this test case. This discrepancy 

consisted of inaccurate instructions of the HPM System Operations document. The vendor corrected 

this document and added information to the ESSIM Systems Operations document, which was 

reviewed and verified. Upon verification, the discrepancy was closed and the test case passed. 
 

Test Case Name: PRI01 – Pick-a-Party 

Passed/Failed: Failed – Please see iBeta’s Test Report 

Summary: PRI01PP is a system-level primary election test case that includes core and optional 

functionality and features (as defined in the Test Plan) for an Open Primary with Pick-a-Party (aka 

Party Preference). The test case is designed to test accuracy in defining an election, creating 

election ballots, voting, and tallying results for an Open Primary with Pick-a-Party. Ballot format 

handling capability, reporting, and usability of the hardware, software and procedures in the entire 

voting system are also verified in PRI01PP. 

The election consisted of five precincts, 6 contests containing non-partisan, partisan, and a primary 

presidential nomination of candidates. A “Standard (Candidate > Vote for)” rotation was applied to 

one of the contests. 

Testing consisted of verifying versions of Software and Firmware, executing the test steps that were 

defined using the System Operations Manuals for all applications and devices, validating various 

reports available from the applications and devices, voting the election using 55 voters for the M650, 

55 voters for the M100, and 50 voters for the iVotronic, in a predefined, random, but repeatable 

pattern to simulate various voters’ choices, verifying the totals of all reports generated from the 

individual scanner tapes and ERM to a predicted result in the test case, and verification of the VSS 

requirements as defined in the Test Plan for the first iteration of testing. 

The AM application, which runs in the background and keeps an audit log of all actions performed 

in EDM and ESSIM, was started prior to any test step execution in any of the other applications 

comprising the Unity 4.0 suite. The election was coded in EDM, ballot layout was designed in 

ESSIM (for paper ballots) and iVIM (for the DRE ballot), and the media was created in HPM. The 

election was voted on paper and the iVotronic DRE. Paper ballots were scanned on the M100 and 

M650 scanners. The results from the M100, and iVotronic were modemed utilizing DAM (Precinct) 

modem to DAM Host into ERM. Results from the M650 were imported from a Zip Disk into ERM. 

The pass/fail criteria for this test case were dependent on the system meeting the expected results 

from the test case, and validated the VSS requirements identified in the test case and the Test Plan. 

Rev01 failed, as the ERM report totals were not matching our vote data totals (Discrepancies #293-

295). During the second iteration discrepancies #293 & #294 were closed after the iVotronic was 

re-voted and totals were read into ERM. Further validation was needed for discrepancy #295. 

Rev03 passed after receiving clarification, from the vendor, on how to validate the report referenced 

in discrepancy #295 and a new discrepancy (#492) stating there is little to no information provided 

in the documentation to assist in validating the reports. 

Test Case Name: PRI02 – Closed Primary 

Passed/Failed: Passed 

Summary: PRI02 is a system-level test case that includes core and optional functionality and 

features (as defined in the Test Plan) for a Closed Primary Election. The test case is designed to test 

accuracy in defining an election, creating election ballots, voting, and tallying results for a Closed 

Primary. Ballot format handling capability, reporting, and usability of the hardware, software and 

procedures in the entire voting system are also verified in PRI02. 

The election consisted of seven precincts, eight contests containing non-partisan, partisan, and recall 

type D. A “District by Registered Voters” rotation was applied to one of the contests. 
 

The AM application, which runs in the background and keeps an audit log of all actions performed 

in EDM and ESSIM, was started prior to any test step execution in any of the other applications 

comprising the Unity 4.0 suite. The election was coded in EDM, ballot layout was designed in 

ESSIM (for paper ballots) and iVIM (for the DRE ballot), and the media was created in HPM. The 

election was voted on paper and the iVotronic DRE. Paper ballots were scanned on the DS200, and 

M100, and M650 scanners. The results from the DS200, M100, and iVotronic were modemed 

utilizing DAM (Precinct) modem to DAM Host into ERM. The M650 results were transferred to the 

ERM via zip disk. 

Testing consisted of verifying versions of Software and Firmware, executing the test steps that were 

defined using the System Operations Manuals for all applications and devices, validating various 

reports available from the applications and devices, voting the election using 28 voters for the M100, 

DS200, M650, and 28 voters for the iVotronic, in a predefined and repeatable pattern to simulate 

various voters’ choices, verifying the totals of all reports generated from the individual scanner tapes 

and ERM to a predicted result in the test case, and verification of the VSS requirements as defined in 

the Test Plan. 

The pass/fail criteria for this test case were dependent on the system meeting the expected results 

from the test case, and validated the VSS requirements identified in the test case and the Test Plan. 

There were four iterations of the test case executed, due to discrepancies encountered. Rev01 failed 

due to discrepancy #177, which was corrected through documentation. Rev02 failed due to 

discrepancy #176 (crossover voting not accepted), which was rectified by re-coding the election and 

omitting the type of crossover voting we were attempting. In Rev03 we encountered an error 

message that was received as a result of the iVIM application, version 3.1.0.0, being left open for the 

weekend (discrepancy # 249) and due to incorrect vote total on a District Canvassing Report 

(discrepancy #297). In Rev04 we tested a new version of iVIM (3.2.0.0) and were able to correctly 

validate all the totals; therefore, all discrepancies were closed and test case passed. 

1.14 Environmental Hardware Test Summary 

Based upon an examination of the ES&S equipment listed in the vendor’s hardware submitted for 

testing listed in Table 4 – Matrix of Required COTS Software/Firmware and vendor’s 

Hardware Specification, SysTest Labs concluded that the hardware, listed and marked as 

(COTS) in table 4, was COTS (Commercial Off the Shelf). As such it is not subject to 

Environmental Hardware Testing. 

SysTest Labs executed Environmental Hardware testing on the non-COTS hardware listed in Table 

5 – Matrix of Required Hardware. The testing methods and results are located in Attachment H 

Evaluation of Environmental Hardware Testing

Any critical issues found were reported, resolved and re-tested. Attachment H contains the 

hardware environmental reports from SysTest Labs’ EAC approved Hardware Environmental Test 

Subcontractor, Percept Technology Labs. These reports detail specific information on the 

environmental hardware testing.

Test Case Name: Environmental Hardware Test

Passed/Failed: Passed 

Summary: Based upon an examination of the ES&S equipment listed in the vendor’s hardware 

submitted for testing listed in Table 4 – Matrix of Required COTS Software/Firmware and vendor’s 
 

Hardware Specification, SysTest Labs concluded that the hardware, listed and marked as (COTS) in 

table 4, was COTS (Commercial Off the Shelf). As such it is not subject to Environmental 

Hardware Testing. 

Test reports from previous hardware testing performed by accredited NVLAP or A2LA laboratories 

were analyzed to determine if the results would be accepted for certification. The EAC provided 

additional input on required tests in their Notice of Clarification (NOC) 08-001 as follows: 

 For hardware not tested by an accredited laboratory after January 1, 2005, all testing is 

required in this certification effort. 

 All hardware was required to undergo Electrostatic Disruption (ESD) testing. 

Please see the Hardware Test Matrix, ESS retest matrix v1.16.pdf, in attachment H to ascertain 

which tests were accepted from previous hardware test efforts and which tests were performed by 

SysTest Labs in this certification effort. 

Maintainability testing and Accessibility and Human Engineering Evaluation testing was performed 

at SysTest Labs in Denver. 

All tested equipment successfully passed each of the environmental tests to which the equipment 

was subjected. Please see the pertinent test reports as outlined in attachment H. 

Other Labs Performing Hardware Testing 

SysTest Labs is responsible for all core voting system tests as identified in NIST NVLAP Handbook 

150-22 (2005). The labs listed below performed non-core hardware testing for this certification test 

campaign. 

 Environmental Testing was done at SUN APT Laboratories in Longmont, Colorado and 

Percept Technology Laboratories in Boulder, Colorado. 

 EMC Testing was performed at Criterion Laboratories in Rollinsville, Colorado, EMC 

Integrity Laboratories in Longmont Colorado, and NCEE Laboratories in Lincoln, 

Nebraska. 

 Safety Testing was performed at Compliance Integrity Services (CIS) Laboratories in 

Longmont, Colorado and Components Reliability & Safety, Inc. (CRS) Laboratories in 

Broomfield, Colorado. 

Hardware Environmental Testing Assessment 

The acceptance and use of previous hardware environmental testing and certification performed by 

accredited NVLAP or A2LA facilities is based on the following criteria: 

 The configuration of the equipment being presented for testing is substantially identical to 

the equipment that was previously tested and certified and that all changes made to the 

hardware configuration of the equipment being presented for testing, from the hardware that 

was previously tested and certified were confirmed to be de minimis changes. 

 The standards and associated requirements under which the previous testing and certification 

was performed are equal to or more demanding than the current requirements. 

 There have been no significant changes to the test methods. 
 

The labs that completed the hardware environmental testing and certification meet the EAC’s 

requirements for accreditation as defined in NIST Handbook 150-22: 2005 and NIST Handbook 

150-22: 2007. 

Opinions and Interpretations 

EAC Notice of Clarification 08-001 was applied to this VSTL test effort, with the result that all 

hardware equipment was subjected to Electrostatic Disruption testing. 

1.15 Security/Telecommunications Test Summary 

Not applicable for this test summary document. 

1.16 Sampling of Vendor Testing Results 

SysTest Labs executed 11 sample test cases provided by ES&S in the TDP. The tests, test dates, and 

SysTest Labs’ test results are listed below. 

Test Case Name: Expanded Precincts (M100) 

Passed/Failed: Passed 

Summary The Process 100 Cards/200 Memory Sticks is a functional test case that includes core 

features (as defined in the Test Plan). The purpose of this test is to process more than the expected 

number of ballots/voters per precincts, more that the expected number of precincts, and any other 

similar conditions that tend to overload the system’s capacity to process, store and report data. 

As this election was a sample test case, provided by the vendor, SysTest labs was given the media 

and blank ballots. We then hand marked two sets of 195 ballots. One set was scanned on the M100 

and the other set was scanned on DS200. Once voted, all totals were pulled into ERM. 

Testing consisted of verifying versions of Software and Firmware, executing the test steps defined 

using the System Operations Manuals, voting the election using 494 precincts, and validating 

various reports available from the applications and devices. 

The pass/fail criteria for this test case were dependent on the system meeting the expected results 

from the test case, and validated the VSS requirements identified in the test case and the Test Plan. 

Test Case Name: Auto Recovery (iVotronic) 

Passed/Failed: Passed 

Summary: Using the iVotronic Auto Recovery procedure v 9.1.0.0, we voted an election and 

recovered the results from the U2-D chip. (U2-D chip is a SanDisk). The manual was provided and 

steps were completed; however, only a trained ES&S technician should complete this procedure. 

No issues were found in running the Auto Recovery. 

Test Case Name: Maximum Candidates 

Passed/Failed: Passed 

Summary: Load ERM election database "02PNELAN" with more than 1000 candidates in a 

precinct. ERM limits 1000 counters in a single precinct. Verify that an attempt to load over 1000 

counters gives an error messages stating "Aborted-over 1000 candidate in precinct: 211 ERM create 

results database failed. Connect election definition HPM and then retry." The initial attempt to load 
 

the election did not produce the error. In researching the election definition, we found we were 

provided the incorrect election. Once given the correct election definition, the error message was 

received as indicated above. 

Test Case Name: L&A Vote Selected Ballot Test 

Passed/Failed: Passed 

Summary: Using ES&S test case "L&A Vote Selected Ballot Test" to verify the logic and accuracy 

vote selected ballot test. The voter selects a particular ballot to vote and that vote logic is applied to 

a select number of ballots (11,803) designated for the voter to cast. No issues were found in running 

the L&A Vote Selected Ballot Test. 

Test Case Name: L&A Multi-Vote Test 

Passed/Failed: Passed 

Summary: Using ES&S test case "L&A Vote Multi-Vote Test" to verify the logic and accuracy of 

the multi vote test. Votes for each candidate will increase from one to the next, as in 1, 2, 3, 4, 5, 

etc. No issues were found in running the L&A Multi-Vote Test. 

Test Case Name: L&A Vote for One Test 

Passed/Failed: Passed 

Summary: Using ES&S test case "L&A Vote For One Test" to verify the logic and accuracy of the 

vote for one test. Each candidate within a contest will receive one vote. There will be an additional 

undervote assigned in each contest. No issues were found in running the L&A Vote for One Test. 

Test Case Name: View Log

Passed/Failed: Passed 

Summary: Using an existing election (GEN01), select a user and verify Audit Manager has 

captured all activity(ies) carried out in all applicable applications (EDM, AM, and ESSIM). No 

issues were found in running the View Log test case. 

Test Case Name: Opening the Polls Functions

Passed/Failed: Passed 

Summary: Use this test case to determine if the DS200 can open polls with an election definition 

that has more than one precinct. The HPM Report Level option must be ‘Precinct’. No issues were 

found in running the Opening the Polls Functions test case. 

Test Case Name: M100 with Plastic Ballot Box (Note: Testing was completed with the Plastic 

Ballot Box; however, it cannot be certified as a part of the Unity 4.0 Suite, as it has not gone through 

Hardware Testing.)

Passed/Failed: Passed 

Summary: Using ES&S test case "Model 100 with Plastic Ballot Box" to verify that the M100 can 

process ballots accurately when seated in the plastic ballot box. No issues were found in running the 

M100 with Plastic Ballot Box test case. 
 

No critical issues were found in running any sample test cases and all sample test cases passed. 

1.17 Maintainability Test Results 

Test Case Name: Maintainability Test

Passed/Failed: Passed 

Summary: Maintainability is a test case designed to verify the performance of maintenance 

activities, as identified in the System Maintenance Procedures, can be performed by the jurisdiction 

without significant impediment or difficulties. 

System Maintenance Procedures were received for the following hardware: DS200, M650, M100, 

iVotronic DRE, and ABCR (Automatic Bar Code Reader). 

Testing consisted of verifying procedures to include determination of operation status, service of the 

components, repair or replacement of parts, restoration to operation status, confirmation of presence 

of labels and the identification of test points, provision of built-in test, diagnostic circuitry or 

physical indicators of conditions, presence of labels, alarms related to failure, and presence of 

features that allow non-technicians to perform routine maintenance tasks. Also tested was the 

assessment for ease of detecting equipment failure by non-technician, ease of diagnosing problems 

by a technician, rate of false alarms, ease of access to component for replacement, ease of 

performance of adjustment or alignment, ease with which databases updated can be performed by a 

non-technicians and adjustments, alignment, and service of components. 

The testing procedures consisted of using processes described in the Maintenance Manuals to review 

the maintenance procedure(s) for the corresponding product, an examination of the physical 

attributes for reliability and any significant impediment for the performance of the maintenance 

activities to be performed by the jurisdiction, an examination of the additional attributes to assess 

system maintainability, and performance of the activities designated as maintenance activities to the 

corresponding product. 

The pass/fail criteria for this test case were dependent on the system meeting the expected results 

from the test case, and validated the VSS requirements identified in the test case and the Test Plan. 

During testing one issue was encountered. The DS200 System Maintenance Manual (SMM) did not 

advise mechanical checks are only supposed to be completed by a trained technician (discrepancy 

#5). Since then, the vendor has included this issue to the DS200 SMM and the discrepancy was 

closed. The maintainability test passed with no unresolved discrepancies. 

1.18 Accuracy Test Results 

Summary: The Accuracy Test Case is a system level general election with a straight party option. 

This test case verified the iVotronic can accurately and reliably record and store into memory ballots 

incorporating a minimum of consecutive ballot positions and can be counted without error. This test 

was performed in two phases. 

Testing consisted of verifying versions of Software and Firmware, executing the test steps that were 

defined using the System Operations Manuals for all applications and devices, and validating 

various reports available from the applications and devices. 
 

The AM application, which runs in the background and keeps an audit log of all actions performed 

in EDM was started prior to any test step execution in any of the other applications comprising the 

Unity 4.0 suite. The election was coded in EDM, ballot layout was designed in iVIM (for the DRE 

ballot), and the media was created in HPM. The election was voted on the iVotronic (DRE) and the 

results were verified by modeming data from two of the machines using the PEB and ERM. Two of 

the machines utilized the DAM Remote Pack Reader to transmit election results, and the last 

machine used the communication pack to transmit results into ERM. The reports were printed and 

verified. 

The election consisted of 8 contests with 14 parties and 98 candidates, 7 in each party. The object of 

Phase 1 was to verify the iVotronic can accurately and reliably print ballots incorporating a 

minimum of 26,997 ballot positions (including voted and non-voted positions) and that these ballots 

can be mechanically/electronically tabulated without error. The object of Phase 2 was to verify 

iVotronic can accurately and reliably print ballots incorporating a minimum of 1,549,703 ballot 

positions (including voted and non-voted positions) and that these ballots can be 

mechanically/electronically tabulated without error. The ballot positions completed in the Phase 1 

test carried over to and become part of this phase 2 test. The pass/fail criteria for this test case were 

dependent on the system meeting the expected results from the test case, and validated the VSS 

requirements identified in the test case and the Test Plan 

Each component successfully completed component level accuracy testing. Additionally, the ES&S 

ABCR and Voyager optical scanners successfully scanned all tapes and reports were validated. One 

documentation discrepancy was written in response to this Test Case due to a printer busy message 

received while printing out the final results; the tester tried to verify documentation and couldn’t 

locate the error message in the documentation. There was no impact on the system and discrepancy 

number 478 was written. 

Ballot Information 

 Contest Contestants Ballot Positions 

Straight Party 1 14 14

Partisan 8 98 784

Non-Partisan 1 2 2

Total 10 114 800

Totals 

Phase 2 

Phase 1 

Batch 1 Batch 2 

Batch 

3 

Batch 

4 Batch 5 Batch 6 Batch 7 Total 

TOTALS 

Ballots Planned 48 350 350 350 350 350 140 1938

Ballot Position 

Planned 38400 280000 280000 280000 280000 280000 112000 1550400

Ballots Actual 48 350 350 350 350 350 140 1938

Ballot Position 

Actual 38400 280000 280000 280000 280000 280000 112000 1550400
 

Phase 2 

Phase 1 

Batch 1 Batch 2 

Batch 

3 

Batch 

4 Batch 5 Batch 6 Batch 7 Total 

Machine 1 - 

Precinct 1

iVotronic SN 

V5105369-C 

Ballots Planned 48 70 70 70 70 70 31 429

Ballot Position 

Planned 38400 56000 56000 56000 56000 56000 24800 343200

Ballots Actual 48 70 70 70 70 70 31 429

Ballot Position 

Actual 38400 56000 56000 56000 56000 56000 24800 343200

Phase 2 

Phase 1 

Batch 1 Batch 2 

Batch 

3 

Batch 

4 Batch 5 Batch 6 Batch 7 Total 

Machine 2 - 

Precinct 2

iVotronic SN 

V5149331 

Ballots Planned 70 70 70 70 70 31 381

Ballot Position 0 56000 56000 56000 56000 56000 24800 304800

Ballots Actual 70 70 70 70 70 31 381

Ballot Position 

Actual 0 56000 56000 56000 56000 56000 24800 304800

Phase 2 

Phase 1 

Batch 1 Batch 2 

Batch 

3 

Batch 

4 Batch 5 Batch 6 Batch 7 Total 

Machine 3 - 

Precinct 3

iVotronic SN 

V5187814 

Ballots Planned 70 70 70 70 70 31 381

Ballot Position 0 56000 56000 56000 56000 56000 24800 304800

Ballots Actual 70 70 70 70 70 31 381

Ballot Position 

Actual 0 56000 56000 56000 56000 56000 24800 304800

Phase 2 

Phase 1 

Batch 1 Batch 2 Batch Batch Batch 5 Batch 6 Batch 7 Total 
 

3 4 

Machine 4 - 

Precinct 4

iVotronic SN 

V5118248 

Ballots Planned 70 70 70 70 70 31 381

Ballot Position 0 56000 56000 56000 56000 56000 24800 304800

Ballots Actual 70 70 70 70 70 31 381

Ballot Position 

Actual 0 56000 56000 56000 56000 56000 24800 304800

Phase 2 

Phase 1 

Batch 1 Batch 2 

Batch 

3 

Batch 

4 Batch 5 Batch 6 Batch 7 Total 

Machine 5 - 

Precinct 5

iVotronic SN 

V5178132 

Ballots Planned 70 70 70 70 70 16 366

Ballot Position 0 56000 56000 56000 56000 56000 12800 292800

Ballots Actual 70 70 70 70 70 16 366

Ballot Position 

Actual 0 56000 56000 56000 56000 56000 12800 292800

The DS200 Accuracy Test Case is a system level general election. This test case verified the DS200 

can accurately and reliably scan and store into memory a minimum of consecutive ballot positions 

and can be counted without error. This test was performed in two phases. 

Testing consisted of verifying versions of Software and Firmware, executing the test steps that were 

defined using the System Operations Manuals for all applications and devices, and validating 

various reports available from the applications and devices. 

The AM application, which runs in the background and keeps an audit log of all actions performed 

in EDM was started prior to any test step execution in any of the other applications comprising the 

Unity 4.0 suite. The election was coded in EDM, ballot layout was designed in ESSIM for the 

DS200 and the media was created in HPM. This test was created to run in two phases. 

Greater than 100 ballots were scanned when an error occurred on machine 2. Per vendor 

documentation the error “#124 "Ballot Dragged/Turn Ballot Over and Try Again"” gave instructions 

to contact an ES&S technician/representative. After the service technician began troubleshooting it 

was determined the cause of the error was due to a miniscule mark in the timing mark area on the 

ballot on orientations 2 and 4 that caused the ballot to drag. Testing was halted. (See Disc #465). 

These ballots with this mark on them were replaced and a second iteration of the test took place 

(Rev 02). 

During Phase 1 of the second iteration the DS200 accurately and reliably scanned ballots 

incorporating a minimum of 26,997 ballot positions (including voted and non-voted positions) and it 

was confirmed that these ballots can be mechanically/electronically tabulated without error. 
 

During Phase 2, the DS200 accurately and reliably scanned ballots incorporating a minimum of 

1,549,703 ballot positions (including voted and non-voted positions) and it was confirmed that these 

ballots can be mechanically/electronically tabulated without error. The ballot positions completed in 

the Phase 1 test carried over to and became part of this phase 2 test. The pass/fail criteria for this 

test case were dependent on the system meeting the expected results from the test case, and validated 

the VSS requirements identified in the test case and the Test Plan. 

The totals from Machine 1 were modemed directly from the machine into DAM Host and the totals 

from Machine 2 were read into ERM via media. The totals from Machine 1 were printed and 

validated. The reports from ERM totals were printed and validated accurately. 

Actual (reflects using complete batches of 320 ballots each. Batch 1 split between Phase 1 and Phase 2) 

 Requirement 

Target (>= 

Requirement) 

Target 

Machine 

1 

Target 

Machine 

2 

Target 

Machine 3 

(N.A.) 

Target Machine 

4 (N.A.) 

Phase 1 

Ballots N.A. 104 52 52 0 0

ballot 

positions 26,997 27,456 13,728 13,728 0 0

Phase 2 - 19 

batches 

10 

batches 9 batches 

ballots from 

phase 1 N.A. 104 N.A. N.A. N.A. N.A.

ballot 

positions 

from phase 1 N.A. 27,456 N.A. N.A. N.A. N.A.

Ballots N.A. 6,080 3,148 2,828 0 0

ballot 

positions 1,549,703 1,605,120 831,072 746,592 0 0

The M100 Accuracy Test Case is a system level general election. This test case verified the M100 

accurately and reliably recorded and stored into memory ballots incorporating a minimum of 

consecutive ballot positions and counted the ballots without error. This test case was conducted in 

two phases. 

Testing consisted of verifying versions of Software and Firmware, executing the test steps that were 

defined using the System Operations Manuals for all applications and devices, and validating 

various reports available from the applications and devices. 

The AM application, which runs in the background and keeps an audit log of all actions performed 

in EDM was started prior to any test step execution in any of the other applications comprising the 

Unity 4.0 suite. The election was coded in EDM, ballot layout was designed in ESSIM for the 

M100 and the media was created in HPM. This test was created to run in two phases. 

During phase 1 the M100 accurately and reliably scanned ballots incorporating a minimum of 

26,997 ballot positions (including voted and non-voted positions) which allowed the continuation 

onto phase 2. 
 

Phase 2 testing verified the M100 accurately and reliably scanned ballots incorporating a minimum 

of 1,549,703 ballot positions (including voted and non-voted positions). The ballot positions 

completed in phase 1 test carried over and became part of phase 2 testing. The pass/fail criteria for 

this test case were dependent on the system meeting the expected results from the test case, and 

validated the VSS requirements identified in the test case and the Test Plan. 

The totals from Machine 1 were transmitted directly from the unit to the DAM Host and the totals 

were printed and validated. The totals from Machine 2 were read into DAM Remote Pack Reader 

and transmitted, and the totals for Machine 3 were read directly into ERM. These ballots were 

mechanically/electronically tabulated without error and the reports from ERM totals were printed 

and validated accurately. 

There were no problems found during the testing of the M100 and Functional Discrepancy #259 has 

been closed. 

 Requirement 

Target (>= 

Requirement) 

Target 

Machine 1 

Target 

Machine 2 

Target 

Machine 3 

Phase 1 

ballots N.A. 103 35 34 34

ballot positions 26,997 27,192 9,240 8,976 8,976

Phase 2 

ballots from phase 1 N.A. 103 N.A. N.A. N.A.

ballot positions from 

phase 1 N.A. 27,192 N.A. N.A. N.A.

ballots N.A. 5,871 1,923 1,923 1,922

ballot positions 1,549,703 1,549,944 507,672 507,672 507,408

Phase 3 (optional) 

ballots N.A. 5,972 1,991 1,991 1,990

excess ballot positions 

from phase 2 N.A. 241 N.A. N.A. N.A.

ballot positions 1,576,701 1,576,849 525,624 525,624 525,360

The M650 Accuracy Test Case is a system level general election. This test case verified the M650 

accurately and reliably scanned and stored into memory ballots incorporating a minimum of 

consecutive ballot positions and the ballots could be counted without error. This test case was 

conducted in two phases. 

Testing consisted of verifying versions of Software and Firmware, executing the test steps that were 

defined using the System Operations Manuals for all applications and devices, and validating 

various reports available from the applications and devices. 

The AM application, which runs in the background and keeps an audit log of all actions performed 

in EDM was started prior to any test step execution in any of the other applications comprising the 

Unity 4.0 suite. The election was coded in EDM, ballot layout was designed in ESSIM for the 

M650 and the media was created in HPM. This test was created to run in two phases. 
 

This test was performed in two phases. The object of Phase 1 was to verify that the M650 could 

accurately and reliably scan ballots incorporating a minimum of 26,997 ballot positions (including 

voted and non-voted positions) and that these ballots could be mechanically/electronically tabulated 

without error. 

The object of Phase 2 was to verify that the M650 could accurately and reliably scan ballots 

incorporating a minimum of 1,549,703 ballot positions (including voted and non-voted positions) 

and that these ballots could be mechanically/electronically tabulated without error. The ballot 

positions completed in the Phase 1 test carried over to and became part of this phase 2 test. The 

pass/fail criteria for this test case were dependent on the system meeting the expected results from 

the test case, and validated the VSS requirements identified in the test case and the Test Plan. 

The totals from Machine 1 were then read into ERM via media and the totals from Machine 2 were 

saved over a LAN directly to a shared network folder for subsequent reading by ERM. The totals 

from Machine 2 were printed and validated. The reports from ERM totals were printed and 

validated accurately. 

One torn ballot produced an error. SysTest replaced the ballot from a set of extra ballots and testing 

proceeded. The ERM Networked laptop went into sleep mode and the connection was lost. The 

laptop was configured to refrain from sleep mode and testing proceeded. No discrepancy was 

written. Discrepancy #262 was closed. 

Actual (reflects using complete batches of 320 ballots each. Batch 1 split between Phase 1 and Phase 2) 

 Requirement 

Target (>= 

Requirement) 

Target 

Machine 1 

Target 

Machine 

2 

Target 

Machine 

3 (N.A.) 

Target 

Machine 4 

(N.A.) 

Phase 1 

ballots N.A. 104 52 52 0 0

ballot positions 26,997 27,456 13,728 13,728 0 0

Phase 2 - 19 

batches 10 batches 9 batches 

ballots from 

phase 1 N.A. 104 N.A. N.A. N.A. N.A.

ballot positions 

from phase 1 N.A. 27,456 N.A. N.A. N.A. N.A.

ballots N.A. 6,080 3,148 2,828 0 0

Ballot positions 1,549,703 1,605,120 831,072 746,592 0 0

SysTest Labs executed telecommunications tests on all software and hardware components of the 

Unity 4.0 voting system that incorporate either modem or IP connections. Transmission testing 

includes intentional interruption of the communications link to verify recoverability, error reporting, 

and user notification (per the vendor documentation). After the interruption, the data was 

retransmitted successfully and verified. The tests were designed to verify Unity 4.0 meets all 

applicable EAC VSS 2002 telecommunications requirements. 
 

Election Systems and Software’s Data Acquisition Manager (DAM) allows users to transfer election 

results from remote polling sites to a jurisdiction’s election headquarters. Data Acquisition Manager 

has two software configurations: DAM Remote and DAM Host. 

Officials at the central site use the DAM Host configuration to receive election data from polling 

places. The DAM Host requires the setting of several configuration options before a voting device 

or Remote DAM can transmit election results to Election Central. The primary configuration 

elements are: election name, tabulator or DRE device types, shared network folder name for the 

election results, data encryption option, and setting the Precinct Status file. After configuring and 

starting the DAM Host module, the Precinct tabulators or DREs, or DAM Remotes, can then 

transmit election results to Election Central. 

Poll workers use the DAM Remote configuration to transfer election results via Pack Reader and 

Pack Sender to the central collection DAM Host location. The Precincts and polling places have 

three options for getting their collective election results to Election Central. The options are: hand 

carrying of the voting device media for reading at Election Central, reading the tabulator and DRE 

election results media on a Remote DAM for transmission to Election Central, or transmitting the 

election results directly from a tabulator or DRE to the DAM Host at Election Central. All three 

results delivery options were tested during the Accuracy testing. The DAM Remote has several 

configuration parameters including election name, tabulator or DRE type, and the DAM Host 

password. 

All components of the ES&S Unity 4.0 voting system passed relevant telecommunications testing. 

The telecommunications testing includes assessment of the hardware and software performance, the 

design and maintenance characteristics, and the simplicity and flexibility of the components. Any 

critical issues found were reported, resolved and re-tested. 
 

Appendix A –Trusted Build 

Appendix A is not a requirement to this document.

 

Appendix B – Index of Attachments 

Attachment A - FCA - Functional & System Testing and Sampling 

Attachment B - PCA Document Review 

Attachment C – Source Code Review 

Attachment H - Environmental Hardware Test Summary