U. S. ELECTION ASSISTANCE COMMISSION

VOTING SYSTEM TESTING AND CERTIFICATION PROGRAM

1201 New York Avenue, NW, Suite 300

Washington, DC.  20005

July 7, 2010

In response to the ES&S DS200 System Lockup Analysis for Unity v. 3.2.0.0, the EAC finds

that the report provides an insufficient explanation and analysis of the root cause of the

anomaly. The EAC requests clarification of the following questions prior to EAC

acceptance of this report.

Overall issues:

 The title of the document references 3.2.0.0 REV 1. Currently this fix is not part

of that tested configuration, test campaign and report.

 Insufficient details and descriptions of the investigation and test parameters,

setups, raw data, and results to enable the EAC to assess the thoroughness and

completeness of the research.

By Section:

 1.2 – It is not clear what analysis was performed on the returned equipment or if

validation tools were used to check the validly of the installed software.

 1.3 – What hardware problems were examined?

 1.4 – How many machines were tested? How was it determined a battery change

was necessary? If a battery is bad would it cause THE symptom to be exhibited?

How many units had the batteries replaced relative to the units that failed?

 1.5 – To what extent does the core dump alter the certified configuration and

will this feature remain on? If it remains on then how was the software tested?

 1.6 – It is unclear exactly what was done each day in terms of testing. Were tests

repeated multiple times each day on the same unit or only performed once per

day on a unit?

 1.8 – What are the statistics of the “perceptible difference”?

 1.10 – What was done for the “constant operation” and was anyone interacting

with the screen in the chamber? What was the unit chartered to do, and what

were the criteria for no demonstrated symptoms?
 1.16 – We would like a copy of the report “USB traces analysis and

interpretationV1‐1.doc”  

 1.18 – In the second bullet, what parameter caused the x‐call failure?

 1.21 – Was the problem still present even after fixing the memory management

problem and a failing x‐windows call?  

 1.22 – It is stated that the touch test is “most prone” to cause the shutdown

issue. Are there other causes for the shutdown? In the last paragraph, how long

was the “substantial testing” and what were the before and after statistics?

 1.23 – The “Summary” is not a clear and cogent summation of the problem and

the determination of root cause. In addition, the summary also brings new issues

to light. For example, the statement about pressing the “hotspot” on wake up

was never discussed as a possible cause of the shutdown.  

In summary the EAC requests a concise description of the root cause and an

engineering/technical description of how the changes proposed solve the symptom. In

addition the EAC needs to see a mapping of how the various discoveries relate to the

final solution, i.e. memory management, double free and x windows along with a set of

recommended test conditions to replicate the anomaly so that the test case can be

validated to detect if the modification has remedied the problem.

Please let me know if you have any questions regarding this letter or the requested

information.  EAC looks forward to receiving this information so it can work with the

VSTL to adequately test the purposed fix to this issue.   

Brian J. Hancock

Director, Testing and Certification

cc:  iBeta Quality Assurance