EAC Decision on Request for Interpretation 2008-01, 

2002 VSS Vol. II, Section 4.7.1 & Appendix C 

2005 VVSG Vol. II, Section 4.7.1 & Appendix C 

Date: 

February 6, 2008 

Question(s): 

1. How many devices can be utilized to meet these requirements of this test? 

2. How many ballots are required to be cast per hour? 

3. Should the audio interface be tested also during this timeframe if the system is 

capable? 

Section of Standards or Guidelines: 

2002 VVSS Volume II, Section 4.7.1, Appendix C, Section C.4, and 2005 VVSG 

Volume II, Version 1.0, Section 4.7.1, Appendix C, Section C.4 

Background: 

These sections of the Standards and Guidelines deal generally with temperature and 

power variation tests. 

 

The test in Section 4.7.1 is similar to the low temperature and high temperature tests of 

MIL-STD-810D, Method 502.2 and Method 501.2, with test conditions that correspond 

to the requirements of the performance standards. This procedure tests system operation 

under various environmental conditions for at least 163 hours. During 48 hours of this 

operating time, the device shall be in a test chamber. For the remaining hours, the 

equipment shall be operated at room temperature. The system shall be powered for the 

entire period of this test; the power may be disconnected only if necessary for removal 

of the system from the test chamber. 

Operation shall consist of ballot-counting cycles, which vary with system type. An 

output report need not be generated after each counting cycle; the interval between 

reports, however, should be no more than 4 hours to keep to a practical minimum the 
time between the occurrence of a failure or data error and its detection. 

Test Ballots per Counting Cycle: 

Precinct count systems 100 ballots/hour 

Central count systems 300 ballots/hour 

The recommended pattern of votes is one chosen to facilitate visual recognition of the 

reported totals; this pattern shall exercise all possible voting locations. System features 

such as data quality tests, error logging, and audit reports shall be enabled during the test. 

(Specific language contained in the Standard.)

(Appendix C, Section C.4) 

System acceptance or rejection can be determined by observing the number of 

relevant failures that occur during equipment operation. The probability ratio for this 

test is derived from the Exponential probability distribution. This distribution implies 

a constant hazard rate. Therefore, two or more systems may be tested simultaneously 

to accumulate the required number of test hours, and the validity of the data is not 

affected by the number of operating hours on a particular unit of equipment. However, 

for environmental operating hardware tests, no unit shall be subjected to less than two 

complete 24 hour test cycles in a test chamber as required by Volume II, Subsection 

4.7.2. of the Standards

 

Conclusion: 

Question 1. “How many devices can be utilized to meet these requirements of this test?” 

Discussion and Interpretation 

Volume II Appendix C is a general discussion of the application of statistical principles to 

testing under the VSS and VVSG. The proper application of statistical methods must be 

solidly defensible and based upon the science governing each test. 

In the specific paragraph cited it is very helpful to compare the VVS 2002 and the VVSG 

2005 for a critical sentence: 

VSS 2002 Volume II Appendix C.4 

This distribution implies a constant hazard rate. 

VVSG 2005 Volume II Appendix C.4 

This distribution implies a constant hazard rate for equipment failure that is not dependent 

on the time of testing or the previous failures.

In this case the expanded wording in the VVSG 2005 is intended to give additional insight 

into what is meant by a ‘constant hazard rate.’ 

2
The question to be answered is whether the test in 2002 VSS Volume II, Section 4.7.1 is 

looking to reveal constant hazard failures or alternately failures that are time dependent? Mil 

Std.810D Method 501.2 Section I states: 

I-1 PURPOSE. High-temperature chamber tests are performed to determine if material can 

be stored and operated under hot climatic conditions without experiencing physical damage 

or deterioration in performance. 

I-2 ENVIRONMENTAL EFFECTS. High temperatures may temporarily or permanently 

impair the performance of the test item by changing the physical properties or dimensions of 

the material(s) composing it. Examples of some other problems that can occur as the result of 

high-temperature exposure are: 

a. Parts binding from differential expansion of dissimilar materials. 

b. Lubricants becoming less viscous; joints losing lubrication by outward flow of 

lubricants. 

c. Materials changing in dimension, either totally or selectively. 

d. Packing, gaskets, seals, bearings and shafts becoming distorted, binding, and failing 

causing mechanical or integrity failures. 

e. Gaskets displaying permanent set. 

f. Closure and sealing strips deteriorating. 

g. Fixed-resistance resistors changing values. 

h. Electronic circuit stability varying with differences in temperature gradients and 

differential expansion of dissimilar materials. 

i. Transformers and eletromechanical components overheating. 

j. Altering of operating/release margins of relays and magnetic or thermally activated 

devices. 

k. Shortened operating lifetime. 

l. Solid pellets or gains separating. 

m. High internal pressures created within sealed cases of projectiles, bombs, etc. 

n. Burning of explosives or propellants accelerated. 

o. Cast explosives expanded within their cases. 

p. Explosives melting and exuding. 

q. Organic materials tending to discolor, crack, or craze. 

3
Clearly the temperature and power variation test of 4.7.1 is intended to reveal multiple types 

of failures, some of which are time dependent, such as aging effects. Time dependent failure 

mechanisms such as aging effects may not be revealed by multiple units being tested for a 

shorter duration. The extended duration going beyond 48 hours, indicated by the provision 

that the units under test are to remain powered up except for the option of removing from the 

test chamber, provides greater confidence that such effects may be detected. This is 

especially true where there may be a delay or aging from the conditions causing a change 

leading to a failure and the actual failure. 

The Reliability test (Section 4.7.2) and the Accuracy (Section 4.7.1.1) criteria are constant 

hazard or time independent test criteria (The Accuracy criteria for failures are event based but 

the same logic applies). These are tests where the test criteria are valid for any number of 

devices provided the test represents a full operational cycle of all operating modes over the 

specified total hours. The simultaneous test design supports the concept, however, that the 

longer durations can also provide a benefit in detecting time dependent failures as well as the 

constant hazard failures where prior functional, operational, and modular software tests are 

too short in duration to sufficiently detect such issues. 

Therefore, the following criteria are to be used: 

For N test units: 

 N = 1, duration = 163 hours (minimum total hours for the statistical model) 

 N = 2, duration = 85 hours for each unit under test 

 N = 3, duration = 64 hours (48 hrs + 16 hrs) for each unit under test 

 N over 3, duration = 64 hours (48 hrs + 16 hrs) for each unit under test 

 where the 48 hours applies to both non-COTS systems requiring the operational 

environmental test and COTS systems excused from the environmental testing. This 

minimum is subject to change pending further engineering reviews of these tests and actual 

test experience. 

Question 2. “How many ballots are required to be cast per hour?” 

Discussion and Interpretation 

When the voting unit under test is capable of meeting the minimum counting rate the 

minimum ballot counting rate shall be: 

Test Ballots per Counting Cycle 

Precinct count systems 100 ballots/hour 

Central count systems 300 ballots/hour 

4
For system that are not capable of meeting these rates the system shall be tested casting 

ballots on a continuous basis at a rate determined by the ability of the system being 

tested. The rate to be used shall be determined and included in the test plan for the 

system. 

Question 3. “Should the audio interface be tested also during this timeframe if the system 

is capable?” 

Discussion and Interpretation 

An engineering analysis of potential failure modes is necessary to determine how an 

audio interface should be tested. If the possibility of a specific failure mechanism 

affecting audio interface is not identified then it is sufficient to test the audio interface 

periodically during the test. The audio interface should be checked at each extreme of a 

stress being applied in addition to period checks during the course of the test. Results of 

the engineering analysis as well as the test results shall be included in the final voting 

system test report. 

If the engineering analysis identifies a specific failure mechanism that could affect the 

audio interface then checks shall be performed in a way that has the highest probability of 

identifying the failure, should it exist. 

5