Voting System Testing and Certification Program 

U. S. Election Assistance Commission 

1201 New York Avenue, NW, Suite 300 

Washington, DC. 20005 

www.eac.gov 

Certification Program Requirements Matrix (V.5.2) 

The requirements matrix was developed to be a tool to assist in the review of test plans, test 

cases, and test reports for the EAC certification program. This memo explains the structure of 

the matrix and gives guidance on its use. 

I. Structure of the Matrix 

The matrix is constructed using an Excel workbook. The first page, “Document Information,” 

gives the revision level of the matrix itself, the project, and VSTL the matrix is addressing, and 

tracks comments and revisions as the matrix follows the development of the project. The VSTL 

should inaugurate a matrix for each project, identifying itself, the project and the original date 

and primary contact for the first version of the matrix for the project identified. As the matrix is 

revised the primary contract from the organization revising the matrix should be identified on 

this page. 

The second page, “Review Summary,” is designed to track progress as test plans, test cases, or 

test reports are reviewed. Ideally, before approval, all items will be identified as either “Not 

Applicable to this System” or “Test Plan Acceptable.” This page is linked to the other sheets and 

automatically updates as items are marked. No entries should be made on this sheet. 

The third page, “Focus Items,” is for the EAC to complete. The purpose of this page is to 

provide a narrative description of the items of greatest importance in the matrix. The “Focus 

Items” page may link comments from various requirements found elsewhere in the matrix to 

make clear a connection that otherwise may not be apparent. It can also be used to convey a 

unifying theme that may be missed in just reading individual comments. 

The other sheets list the testable requirements in both the 2002 VSS and 2005 VVSG by subject. 

The individual worksheets have 4 major sections. The first set of columns gives the citation and 

information about the requirement. The second group of columns is for use when reviewing a 

test plan. The third group of columns is for reviewing the test cases to be used with the test plan. 

The forth set of columns is for reviewing the test report. 

The following discussion will use the column letters for the worksheets. 

Columns A-F give the testable requirement. 

Last updated: 8/25/2008 1 
Column Purpose 

A-F Requirement 

A Requirement citation in the 2002 VSS 

B Requirement citation in the 2005 VVSG 

C Quote from the standard. 

D Associated test procedure citation in the 2002 VSS 

E Associated test procedure citation in the 2005 VVSG 

F Additional information relevant to the requirement. The purpose of this column is for 

the EAC to identify any notices, interpretations, field experience information or other 

information that the VSTL should be aware of when planning an evaluation. 

G-J Test Plan Review 

G Test Plan Review. This column is for the EAC Technical Reviewer to give the result 

of their review. This column drives the “Review Summary” page Test Plan Review 

column. 

H Test Plan Citation. The VSTL should list the cross reference to the location in the test 

plan that deals with the requirement. 

I EAC Test Plan Review / Notes / Comments / Further Comments / Response. This 

column can be used by the EAC Technical Reviewer to communicate findings, 

questions and/or information about the Test Plan. 

Columns I & J are intended to be used recursively until an item is satisfactorily 

resolved. The EAC Technical Review generally will provide guidance or ask 

questions, particularly on items that are not satisfactorily addressed. The VSTL then 

provides information responding to the review comments. The VSTL may initiate the 

dialogue by providing information it believes will be of help to a reviewer. 

J VSTL / Response / Notes / Comments. This column can be used by the VSTL to 

respond to the EAC comments or give additional information. 

Last updated: 8/25/2008 2 
K-N Test Case/Procedure Review 

K Test Case Review. This column is for the EAC Technical Reviewer to give the result 

of their review. This column drives the “Review Summary” page Test Case Review 

column. 

L Test Case Citation. The VSTL should cite the test procedure or test case that deals 

with the requirement. 

M EAC Test Case Review / Notes / Comments / Further Comments / Response. This 

column can be used by the EAC Technical Reviewer to communicate findings, 

questions and/or information about the Test Cases. 

Columns M & N are intended to be used recursively until an item is satisfactorily 

resolved. The EAC Technical Review generally will provide guidance or ask 

questions, particularly on items that are not satisfactorily addressed. The VSTL then 

provides information responding to the review comments. The VSTL may initiate the 

dialogue by providing information it believes will be of help to a reviewer. 

N VSTL / Response / Notes / Comments. This column can be used by the VSTL to 

respond to the EAC comments or give additional information. 

O-T Test Report Review 

O Changed from Test Plan? (Y/N). This column is for the VSTL to alert the reviewer 

that a change was made from the test plan and that the test report follows a different 

procedure from that which was reviewed. 

P Pass/Fail. This column is for the VSTL to report the result of the evaluation. 

Q Test Report Review. This column is for the EAC Technical Reviewer to give the 

result of their review. This column drives the “Review Summary” page Test Report 

Review column. 

R Test Report Citation. This column is for the VSTL to cross reference from the 

specification to the test report section dealing with the specification. 

Last updated: 8/25/2008 3 
S EAC Test Report Review / Notes / Comments / Further Comments / Response. This 

column can be used by the EAC Technical Reviewer to communicate findings, 

questions and/or information about the Test Cases. 

Columns S & T are intended to be used recursively until an item is satisfactorily 

resolved. The EAC Technical Review generally will provide guidance or ask 

questions, particularly on items that are not satisfactorily addressed. The VSTL then 

provides information responding to the review comments. The VSTL may initiate the 

dialogue by providing information it believes will be of help to a reviewer. 

T VSTL / Response / Notes / Comments. This column can be used by the VSTL to 

respond to the EAC comments or give additional information. 

II. Use of the Matrix 

The matrix should be completed and returned with the test plan, test cases, and test report as a 

tool to assist in the review and resolution of review comments. In addition, the matrix should be 

cited in the test plan and included by reference. This citation is the VSTL’s commitment that the 

information in the matrix is an extension of the test plan. The test procedures and test cases cited 

are thereby incorporated into the test plan by there reference in the matrix. If testing is done 

using different procedures or test cases those changes should be identified in the matrix when the 

test report is submitted. 

The VSTL should provide citation relevant to each requirement for the test plan (H), test 

procedure/cases (L), and, when the test report is submitted, the information required in Columns 

(O), (P), and (R). If additional communication is desired it can be placed in columns (J), (N) or 

(T), as appropriate. 

The EAC Technical Reviewer will use the Test Plan Review (G), Test Case Review (K) and Test 

Report Review (Q) to capture their review of the item and columns (I), (M) & (S) to list 

comments, questions or other communications. 

EAC Technical Reviewers will record the disposition of each item in columns (G), (K) and (Q) 

using the following codes: 

Y - Initial indicator used in the template. Indicates that the same requirement exists 

for the 2002 VSS and the 2005 VVSG. For a specific project it means that no 

initial determination has yet been recorded for this item. 

Y - 02 - Requirement in 2002 VSS but not in 2005 VVSG. 

Y - 05 - Requirement in 2005 VVSG but not in 2002 VSS. 

Last updated: 8/25/2008 4 
Last updated: 8/25/2008 5 

N/A - Requirement does not apply to the system under consideration. For a test case it 

may mean that the test plan adequately addresses the item and a test case is not 

needed. 

NF - Treatment of the item cannot be located in the documentation provided. The 

VSTL should identify where the requirement is treated or provide additional 

documentation on how the requirement will be addressed. 

TA - The Technical Review Team is reviewing the item and has not made an initial 

determination. The Technical Review team lead shall enter a date when the 

review will be complete on the item. 

F - The requirement is not sufficiently addressed. The VSTL should address the 

deficiency and submit the resolution for review. 

EA - The EAC has an action item on this item. An action item could be a need to 

provide further guidance or some other information necessary before a final 

determination can be made on the requirement. 

LA - The VSTL has an action item. Typically this will be used when a requirement 

that has been discussed and a resolution identified. The item will be marked LA 

until the VSTL completes the required action and provides documentation of it to 

the EAC. 

A - Requirement is found acceptable and approved. 

At the end of a successful review all items should be listed as being addresses in a satisfactory 

manner, either “NA” or “A” on the summary sheet.