U. S. ELECTION ASSISTANCE COMMISSION 

VOTING SYSTEM TESTING AND CERTIFICATION PROGRAM 

1201 New York Avenue, NW, Suite 300 

Washington, DC. 20005 

To: Registered Manufacturers and Voting System Test Laboratories

From: Brian Hancock, Director

Date: March 21, 2012

Re: Automated Source Code Analysis Tools

Over the past year, the EAC received a number of questions regarding the use of automated tools 

to satisfy the source code review requirements of the 2005 VVSG. As you know, the current 

manual source code review methods are expensive and consume an estimated 30-40% of a 

testing campaign’s total time. Manual review may lead to inconsistent application of the source 

code review requirements of the 2005 VVSG. 

The source code requirements of the 2005 VVSG can be divided into the following categories:

• Software integrity

• Software modularity and programming

• Control constructs

• Naming conventions

• Commenting conventions

Three categories of source code related 2005 VVSG requirements can be fully or partially

automated: software integrity, software modularity and programming, and control constructs. 

Determining conformance to the naming convention and commenting conventions requirements 

cannot generically be performed by automated tools, as the 2005 VVSG does not specify a 

particular naming or commenting convention for manufacturers to follow. Human analysis or 

tool customization is required to verify conformance. 

RFI 2010-02 clarifies the binary option of choosing either the coding convention specified by the 

2005 VVSG or an industry-accepted coding standard for a specific language and voting system 

component. Used in the correct context, automated tools can assist in determining conformance 

with the 2005 VVSG’s coding convention or an industry-accepted coding standard. Tools for 

assessing conformance to an industry-accepted coding standard already exist, are widely used 

outside the voting system industry, and are ready to be leveraged. Conversely, tools for assessing 

conformance to the 2005 VVSG’s coding convention will likely need to be developed.
In an effort to test this concept, last year the EAC suggested Wyle Laboratories pilot the use of 

automated tools before endorsing their use within the Program (letter attached). Wyle piloted this 

concept and produced a Test Report for the EAC (Test Report attached). Wyle reported positive 

results in assessing adherence to the 2005 VVSG using automated tools and reported drastic cost 

and time savings. It is important to remember that purely automated testing of code is not 

advisable nor is it the goal of this effort. Both manual and automated review should be used in 

the correct context.

Effective immediately, the EAC supports the use of automated tools whenever possible to assess 

conformance to the coding convention specified with the 2005 VVSG and other industryaccepted coding standards. Additionally, the EAC encourages manufacturers to use these tools 

before entering the EAC’s Certification Program to lower the time and expense associated with 

voting system testing and certification. A specific tool or methodology will not be dictated, nor 

will the EAC validate the correct operation of a tool. VSTLs must determine if a tool functions 

as intended. To assist in determining conformance to the naming conventions and commenting 

conventions categories of requirements using manual source code review methods, the EAC will 

allow VSTLs to sample the submitted source code. The results of the sample review and the 

VSTLs engineering analysis will be provided to the EAC for review, before the EAC makes a 

determination on acceptance. 

Please let me know if you have any additional questions.

Sincerely,

Brian J. Hancock, Director 

Voting System Testing and Certification

Attached: 

• Use of Automated Source Code Review Tools Report (Pilot)

• The use of automated source code review tools
U. S. ELECTION ASSISTANCE COMMISSION

VOTING SYSTEM TESTING AND CERTIFICATION PROGRAM

1201 New York Avenue, NW, Suite 300

Washington, DC. 20005

August 26, 2011

Frank Padilla

Wyle Laboratories

7800 Highway 20 West

Huntsville, Al 35806

Subject: The use of automated source code review tools

Dear Mr. Padilla,

Recently Wyle Laboratories approached the Election Assistance Commission Testing and 

Certification Division about the use automated source code review tools. We understand that Wyle 

would like to begin using automated source code review tools as an alternative to full manual line by 

line examination of source code. The EAC is committed to lowering the cost and reducing the time 

needed to complete certification testing. The use of automated tools may help to achieve both of those 

goals; however, the EAC would like to proceed cautiously. The EAC suggests that Wyle initiate a 

pilot run of these tools before instituting automated source code review on a wider basis. The 

conditions of the pilot are:

• The pilot can be performed on source code currently in our program, entering our program, or 

outside of our program.

• The pilot must focus languages with a well established or frequently used set of coding 

standards, (e.g., Java, C++).

• Wyle must submit a report on the results of the pilot. The report must include:

o Accuracy of the tool;

o Comparison of resources used with automated vs. manual review, (i.e. time and 

materials);

o Ability to test conformity to the 2005 VVSG and other conventions;

o Sampling methods used and percentage of code sampled for comment quality.

If you have additional questions, please do not hesitate to contact me.

Sincerely,

Director, Testing and Certification