Elections: Status and Use of Federal Voting Equipment Standards
Published: 2001-10-15T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Events surrounding the last presidential election raised concerns about the people, processes, and technology used to administer elections. GAO has already reported on the scope of congressional authority in election administration and voting assistance to military and overseas citizens. This report focuses on the status and use of federal voting equipment standards, which define minimum functional and performance requirements for voting equipment.
section_title: What GAO Found
paragraphs: The standards define minimum life-cycle management processes for voting equipment developers to follow, such as quality assurance. No federal agency has been assigned explicit statutory responsibility for developing voting equipment standards; however, the Federal Election Commission (FEC) developed voluntary standards for computer-based systems in 1990, and Congress has provided funding for this effort. No federal agency is responsible for testing voting equipment against the federal standards. Instead, the National Association of State Election Directors accredits independent test authorities who test voting equipment against the standards.

section_title: Letter
paragraphs: Events surrounding the last presidential election have brought to light a number of issues about the conduct of elections that extend to the people, processes, and technology used to administer these elections. We were asked by several congressional committees and members to review aspects of elections throughout the United States. In response, we are issuing a series of reports. To date, we have issued reports on the scope of congressional authority in election administration and voting assistance to military and overseas citizens. Other forthcoming reports will examine voting accessibility for people with disabilities; the factors that affected the uncounted votes in the November 2000 presidential election; and the people, processes, and technology challenges affecting the conduct and administration of elections. We are also issuing a capping report that draws upon our extensive body of work to identify the main issues and challenges confronting our nation’s election system and to delineate an analytical framework that Congress could use as it weighs the merits of various reform proposals.
This report focuses on the status and use of federal voting equipment standards. These standards define minimum functional and performance requirements for voting equipment, such as accurately recording votes cast. The standards also define minimum life-cycle management processes for voting equipment developers to follow, such as quality assurance. As agreed with your offices, our objectives were to (1) identify the Federal Election Commission’s (FEC) role regarding voting equipment and assess how well FEC is fulfilling its role and (2) identify the National Association of State Election Directors’ (NASED) process for testing and qualifying voting equipment against FEC’s standards. Details of our objectives, scope, and methodology are contained in appendix I.
subsections: section_title: Background
paragraphs: In 1975 Congress created the FEC to administer and enforce the Federal Election Campaign Act. To carry out this role, FEC discloses campaign finance information, enforces provisions of the law such as limits and prohibitions on contributions, and oversees the public funding of presidential elections.
Within FEC, the Office of Election Administration (OEA) serves as a national clearinghouse for information regarding the administration of federal elections. As such, OEA assists state and local election officials by developing voluntary voting equipment standards, responding to inquiries, publishing research on election issues, and conducting workshops on all matters related to election administration. In addition, it answers questions from the public and briefs foreign delegations on the U.S. election process, including voter registration and voting statistics.
FEC consists of six voting members, appointed by the President and confirmed by the Senate. To encourage nonpartisan decisions, no more than three commissioners can be members of the same political party, and at least four votes are required for most official Commission actions.
FEC’s budget for fiscal year 2001 is $40.4 million, and of that amount, $804,000 is allocated to support OEA functions. FEC has 357 full-time staff, of which 5 are allocated to OEA functions.
subsections: section_title: Voting Equipment Used in the United States
paragraphs: The voting methods used in the United States can be placed into five categories: paper ballots, mechanical lever machines, punch cards, optical scan, and direct recording electronic. The last three methods use computer-based equipment. Three of the five—paper ballots, punch cards, and optical scan—use some kind of paper ballot to record voters’ choices.
Paper Ballot. Voters use a paper ballot listing the names of the candidates and issues and record their choice by placing a mark in a box next to the candidate’s name or issue. After making their choices, the ballots are dropped into a sealed ballot box to be manually tabulated.
Mechanical Lever. Voters pull a lever next to the candidate’s name or issue and the machine records and tallies the votes using a counting mechanism. Write-in votes must be recorded on a separate document. Election officials tally the votes by reading the counting mechanism totals on each lever voting machine.
Punch Card. Voters can use one of two basic types of punch cards— Votomatic or Datavote. In both instances, voters use a computer-readable card to cast their vote. The Votomatic uses a computer-readable card with numbered boxes that correspond to a particular ballot choice. The choices corresponding to those numbered boxes are indicated to the voter in a booklet attached to a vote recording machine, with the appropriate places to punch indicated for each candidate and ballot choice. The voter uses a simple stylus to punch out the box corresponding to each candidate and ballot choice. In the Datavote, the names of the candidates and issues are printed on the card itself—there is no ballot booklet. The voter uses a stapler-like punching device to punch out the box corresponding to each candidate and ballot choice. To tally the votes in both instances, the ballots are fed into a computerized tabulation machine that records the vote by reading the holes in the ballots.
Optical Scan. Voters use a computer-readable paper ballot listing the names of the candidates and issues. The voters record their choices by using an appropriate writing instrument to fill in a box or oval, or complete an arrow next to the candidate’s name or issue. The ballot is then fed into a computerized tabulation machine, which senses or reads the marks on the ballot, and records the vote.
Direct Recording Electronic. Voters use a ballot that is printed and posted on the voting machine or displayed on a computer screen listing the names of the candidates and issues. Voters record their choices by pushing a button or touching the screen next to the candidate’s name or issue. When a voter is finished, the vote is submitted by pressing a vote button, which stores the vote in a computer memory chip. Election officials tally the votes by reading the votes totaled on each machine’s computer chip.
section_title: FEC Has Developed Voting Equipment Standards but Has Not Maintained Them
paragraphs: While neither FEC nor any other federal agency has explicit statutory responsibility to develop voting equipment standards, the Congress has appropriated funds for FEC to develop and update the standards. FEC first issued voting equipment standards in 1990. These standards identify minimum functional and performance requirements for punch card, optical scan, and direct recording electronic voting equipment, and specify test procedures to ensure that the equipment meet these requirements.The functional and performance requirements address what voting equipment should do and delineate minimum performance thresholds, documentation provisions, and security and quality assurance requirements. The test procedures describe three stages of testing: qualification, certification, and acceptance. According to FEC’s standards document:   Qualification testing is the process by which a voting equipment is shown to comply with the requirements of its own design specification and with the requirements of FEC standards.
Certification testing, generally conducted by individual states, determines how well voting equipment conform to individual state laws and requirements.
Acceptance testing is generally performed by the local jurisdictions procuring voting equipment and demonstrates that the equipment, as delivered and installed, satisfies all the jurisdiction’s functional and performance requirements.
The standards are voluntary; states are free to adopt them in whole, in part, or reject them entirely. To date, 38 states require that voting equipment used in the state meet FEC standards either in total or in part.Figure 1 shows these states.
In September 1997, FEC initiated efforts to evaluate its voting equipment standards and identify areas to be updated, and in July 1999, FEC initiated efforts to revise the standards. As part of this revision, FEC has been working closely with state and local election officials and vendors to incorporate industry comments on the draft standards. FEC plans to issue the revised standards in multiple volumes: volume I is to include the functional and performance requirements for voting equipment; volume II is to provide the detailed test procedures, including information to be submitted by the vendor, tests to be conducted to ensure compliance with the standards, and the criteria to be applied to pass the individual tests. Figure 2 depicts FEC’s time frames for revising the standards.
subsections: section_title: FEC’s Voting Equipment Standards Address Most Key Types of Systems Requirements
paragraphs: Organizations such as the Department of Defense and the Institute of Electrical and Electronics Engineers have developed guidelines for various types of systems requirements and for the processes that are important to managing the development of any system throughout its life cycle. These types of systems requirements and processes include, for example:   Security and Privacy Protection. Requirements defining the security/privacy environment, types of security needed (e.g., data confidentiality and fraud prevention), risks the system must withstand, safeguards required to withstand those risks, security/privacy policies that must be met, accountability (i.e., audit trails), and criteria for security certification.
Human Factors. Requirements defining the usability of the system, including considerations for human capabilities and limitations, and the use and accessibility of the system by persons with disabilities.
Documentation. Processes for recording information produced during the system development life cycle, which includes identifying documents to be produced; identifying the format, content, and presentation items for each document; and developing a process for reviewing and approving each document.
Configuration Management. Processes to establish and maintain the integrity of work products through the system development life cycle, including developing a configuration management plan, identifying work products to be maintained and controlled, establishing a repository to maintain and control them, evaluating and approving changes to the work products, accounting for changes to the products, and managing the release and delivery of products.
Quality Assurance. Processes to provide independent verification of the requirements and processes used to develop and produce the system, which include developing a quality assurance plan, determining what system development product and process standards are supposed to be followed, and conducting reviews to ensure that the product and process standards are followed.
While FEC’s 1990 standards satisfy most of these areas, they do not satisfy all. For example, in the area of security, the standards do not address the security/privacy environment in which the voting equipment must operate, the types of security to be provided, the risks the equipment must withstand, the security/privacy policies that must be met, or the criteria for security certification. Further, the standards do not specify requirements for voting equipment usability, taking into account human capabilities and limitations, or the use and accessibility of the voting equipment by persons with disabilities. Table 1 summarizes the types of requirements and processes satisfied in FEC’s 1990 voting equipment standards.
As part of FEC’s current effort to revise the 1990 standards, it has made improvements in all five of the areas in which we identified missing types of requirements and processes. For example, in the area of human factors, the draft standards now include requirements for the use and accessibility of voting equipment by persons with disabilities. Further, for documentation, the draft standards include requirements for identifying documents to be produced; defining the format, content, and presentation items for each document; and developing a process for reviewing and approving each document.  In addition, in the area of security, the standards now address security types, risks, safeguards, policies, accountability, and certification.
While FEC has made improvements, the draft standards do not satisfy two areas—human factors and quality assurance. For example, in the area of human factors, the draft standards do not address requirements for equipment usability, including considerations for human capabilities and limitations. Finally, the draft standards do not yet specify the development of a quality assurance plan or the performance of quality assurance reviews to ensure that the equipment development process requirements are being met. Table 2 summarizes the types of requirements and processes not satisfied in FEC’s 1990 voting equipment standards but satisfied in the draft standards. Appendix III provides a detailed description of the requirements and process types and our complete analysis of FEC’s 1990 voting standards and draft standards.
In the area of quality assurance, FEC stated in its written comments on a draft of this report that its decision to not include quality assurance process reviews in the revised standards was the result of deliberative and collaborative interaction among NASED’s Voting System Committee and FEC staff. In addition, FEC did not include equipment usability because it was determined not to be an area of immediate concern by the election community during FEC’s evaluation of the standards to identify areas to be updated. FEC agrees that equipment usability should be addressed in the standards and has stated that it will fully do so once resources are available. Beyond this stated commitment, FEC has not established any specific plans or allocated specific resources for doing so. Until FEC addresses these missing requirements, the voting equipment standards’ value and utility will be diminished.
section_title: FEC Has Not Proactively Maintained the Voting Equipment Standards
paragraphs: Given the pace of today’s technological advances, standards must be proactively maintained to ensure that they remain current, relevant, and complete. Standards-setting bodies, such as the American National Standards Institute and the National Institute of Standards and Technology, require that standards be revised or reaffirmed at least once every 5 years. This is particularly important with voting equipment standards, which must respond to technological developments if they are to be current, complete, and relevant, and are to be useful to state and local election officials in assuring the public that their voting equipment are reliable.
FEC has not proactively maintained its voting equipment standards. As previously stated, FEC is only now updating the 1990 standards. Because FEC has not proactively maintained the standards, they have become out of date. Vendors are using new technology and expanding voting equipment functions that are not sufficiently covered by the 1990 standards. For example, the 1990 standards do not address election management systems, which are used to prepare ballots and programs for use in casting and tallying votes, and to consolidate, report, and display election results. According to a NASED committee representative and the Independent Test Authority (ITA) responsible for testing election management systems, the lack of adequate standards to address election management software has forced them to interpret the current voting equipment standards to accommodate the development of this new software. Further, according to these representatives, these interpretations have not been documented and formally shared with FEC. As mentioned earlier, FEC is updating its standards, and the draft standards now address election management systems.
FEC officials acknowledge the need to actively maintain the standards, but state that they have not done so because they have not been assigned explicit responsibility. By not ensuring that voting equipment standards are current, complete, and relevant, states may choose not to follow them, resulting in states adopting disparate standards. In turn, this could drive up the cost of voting equipment being designed to multiple standards and produce unevenness among states in the capabilities of voting equipment.
section_title: NASED’s Process for Testing and Qualifying Voting Equipment
paragraphs: No federal agency, including FEC, has been assigned explicit responsibility for testing voting equipment against FEC standards, and no federal agency has assumed this role. Rather, NASED has assumed responsibility for implementing the standards. To do so, NASED established a voting systems committee, which comprises selected state and local election officials and technical advisers. This committee accredits ITAs to test and qualify voting equipment against FEC standards. Figure 3 illustrates the voting equipment standards program, from the development of voting equipment standards through the testing and qualification of voting equipment.
To accredit the ITAs, the NASED committee has developed requirements and procedures, which include provisions for NASED to periodically reaccredit the ITAs and conduct on-site inspection visits, both of which are important to ensuring that the accredited laboratories continue to comply with all requirements. To date, the committee has not reaccredited or inspected ITAs because, according to NASED committee representatives, they rely on the committee’s technical advisers’ ongoing conversations with ITA officials and the officials’ participation in committee meetings to ensure that the ITAs are fulfilling their responsibilities effectively.
Currently, three ITAs are approved to test voting equipment against the FEC standards. In 1994, the NASED committee accredited Wyle Labs to test the hardware and machine-resident software components of proprietary vote cast and tally equipment. In February 2001, Metamor (previously PSINet) applied for accreditation to conduct qualification testing of vote tabulation and election management software.  Also in 2001, SysTest applied for accreditation to conduct qualification testing of voting tabulation and election management software.  While both Metamor and SysTest have been granted an interim approval to test voting equipment, NASED has not yet accredited either.
To test voting equipment, voting equipment vendors submit requests for testing to the ITAs, who then prepare a test procedure. The test procedure details the software and hardware testing requirements that the voting equipment will be tested against and is based on both the FEC voting equipment standards and the vendors’ design specifications.
According to ITA officials responsible for testing voting equipment, the testing process is generally an iterative one. Vendors are provided an opportunity to correct deficiencies identified during testing and resubmit the modified voting equipment for retesting. At the end of testing, the ITA completes a test report and notifies the Election Center that the voting equipment has successfully satisfied testing requirements. The Election Center then assigns a NASED number to the specific equipment model and firmware release that was tested and maintains the list of qualified voting equipment. Each time a vendor issues a new model or software release, the vendor is to submit a request for testing to the ITAs in order to qualify the new model or release. As of July 3, 2001, NASED had qualified 21 models of voting equipment and 7 election management systems, representing 10 vendors. See table 3 for a breakout of the types of equipment qualified.
The ITAs stated that the testing process generally takes about 2 to 3 months. This is contingent, however, upon the vendors having the proper documentation in order. If documentation is missing or incomplete, the process may take longer. According to the ITAs, the cost of qualification testing ranges from $40,000 for vote cast and tally equipment to $75,000 for vote tabulation and election management software.
section_title: Conclusions
paragraphs: While not explicitly provided for in legislation, FEC and NASED have assumed and are performing important roles by developing voting equipment standards and testing and qualifying equipment against these standards, respectively. Given the current pace of technological change for voting equipment, the degree to which these standards are actively maintained and the extent to which they are appropriately applied, can have a direct bearing on the capabilities of voting equipment. This, in turn, can affect the successful conduct of national, state, and local elections. Therefore, it is important that responsibility for these roles be clearly assigned. By doing so, the appropriate federal role in these important areas can be deliberated, decided, and explicitly defined, thereby avoiding another situation where the standards are allowed to become out of date. It is also important that these roles be executed effectively. In the case of FEC’s ongoing update of the standards, this means that requirements for equipment usability, and quality assurance should be developed.
section_title: Matters for Congressional Consideration
paragraphs: As part of the ongoing debate and deliberation over election reform in general, and the federal role in voting equipment standards in particular, the Congress may wish to consider assigning explicit federal authority, responsibility, and accountability for voting equipment standards, including proactive and continuous update and maintenance of the standards. Given that no federal or state entity has been assigned explicit authority or responsibility for testing voting equipment against the FEC standards, the Congress may wish to consider what, if any, federal role is appropriate, regarding implementation of the standards, including the accreditation of ITAs and the qualification of voting equipment.
section_title: Recommendations for Executive Action
paragraphs: To improve the quality of FEC’s voting equipment standards, we recommend that the FEC Commissioners direct the OEA Director to accelerate the development of requirements for equipment usability, including considerations for human capabilities and limitations. To improve the quality of FEC’s current efforts to update the voting equipment standards, we also recommend that the FEC Commissioners direct the OEA Director to develop requirements for quality assurance, including developing a quality assurance plan and conducting quality assurance process reviews.
section_title: Agency Comments and Our Evaluation
paragraphs: In its written comments on a draft of this report (reprinted in appendix II), the FEC Chairman and Vice Chairman stated that FEC generally agrees with most of our observations and recommendations, including that human factors are not being addressed in the revised voting equipment standards and that FEC needs to accelerate their development in future iterations of the standards. Additionally, FEC agreed with our matter for congressional consideration.
Nevertheless, FEC commented that it was concerned with the report’s portrayal of the Commission as being insufficiently proactive in revising voting equipment standards, stating that its efforts have been as timely as possible given certain practical constraints, which it described in a chronology of events and circumstances. FEC also commented that it disagrees with the draft report’s characterization of the Commission’s ongoing efforts to update security and quality assurance standards as incomplete, describing how both areas are being addressed. Subsequent to providing us with its written comments on a draft of this report, FEC also provided us with additional draft standards that address security requirements. Accordingly, we have modified this report, including our recommendations, to reflect this new information.
We do not agree with either of FEC’s other two points of concern. Regarding FEC’s concern with the report’s portrayal of the Commission as being insufficiently proactive in revising voting equipment standards, FEC states in its comments that 7 years elapsed from the time that the standards were first issued in 1990 to the time that FEC first began evaluating them to identify areas that needed to be updated. Further, it states that another 2 years elapsed between the time FEC began evaluating the standards and the time it began updating them. We recognize that FEC is performing, through its own initiative, an important role in developing and updating the standards, and deserves credit for doing so. However, in our view, allowing 9 years to pass before beginning to update the standards, regardless of the practical circumstances that FEC cites, is too long and does not constitute a proactive maintenance process and is the primary reason that the current standards are out of date. Regarding FEC’s disagreement with the report’s characterization of the Commission’s ongoing efforts to update quality assurance standards as incomplete, we do not question, and in fact state in this report, that the draft standards address requirements for quality assurance. However, our main concern is that important and relevant aspects of quality assurance requirements, such as quality assurance plans and process reviews, respectively, are not addressed.
Concerning FEC’s decision to omit quality assurance standards areas from the revised draft standards, we modified this report to reflect FEC’s position that its decision resulted from deliberative and collaborative interaction among NASED and FEC staff and was not, as we were told during the course of our review by the OEA Director, areas that were overlooked.
We are sending copies of this report to the Chairmen and Ranking Minority Members of the Senate Appropriations Subcommittee on Treasury and General Government and the House Appropriations Subcommittee on Treasury, Postal Service, and General Government; the Director of the Office of Management and Budget; and the Chairman and Vice Chairman of FEC. Copies will also be available at our Web site at www.gao.gov. If you have any questions, please contact me at (202) 512- 6240 or by email at hiter@gao.gov. Key contributors to this assignment were Deborah A. Davis, Richard Hung, and Eric Winter.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: The objectives of our review were to (1) identify Federal Election Commission’s (FEC) role regarding voting equipment and assess how well FEC is fulfilling its role and (2) identify the National Association of State Election Director’s (NASED) process for testing and qualifying voting equipment against FEC’s voluntary voting equipment standards.
To identify FEC’s role regarding voting equipment, we researched FEC’s statutory and legislative role in developing and maintaining voting equipment standards. To further identify FEC’s role, we reviewed relevant documents, including the Plan to Update the Voting Systems Standards,the standards update project contract, project work plans, and legislative proposals, and interviewed key FEC officials, including the Director, OEA.
To assess FEC’s voting equipment standards, we examined relevant guidelines and procedures for the development of system requirements. Specifically, we examined the Department of Defense’s Data Item Description for System/Subsystem Specifications; the Institute of Electrical and Electronics Engineers’ Standard 12207 on Software Life Cycle Processes, and the Software Engineering Institute’s Software Development Capability Maturity Model™  and identified 13 types of systems requirements and 3 supporting life-cycle processes that are important in the development of any system. We then compared these types of requirements and processes against FEC’s 1990 voting equipment standards to determine if all key elements were addressed. In those areas where variances were noted, we compared the types of requirements and processes against relevant sections of volumes I and II of the draft standards to determine whether FEC had addressed any of these missing requirements. We only reviewed those portions of the draft standards for which we identified missing types of requirements and processes in the 1990 standards. In addition, our review of the standards did not include validating that the requirements are correct and complete beyond determining whether the standards addressed all of the requirements and process key elements.
To identify NASED’s process for testing and qualifying voting equipment against FEC’s voting equipment standards, we interviewed officials from NASED, the Election Center, and the two independent test authorities (ITA). We also reviewed documentation describing NASED’s process, NASED’s Accreditation of Independent Testing Authorities For Voting System Qualification Testing Handbook, ITAs’ generic test plans, and NASED’s policies, procedures, and by-laws. We also provided a copy of relevant parts of this report to the Chairman of the NASED Voting System Committee for comment. The Chairman stated that the report accurately reflected the NASED process.
We also contacted officials in the State Election Director's offices in each of the 50 states and the District of Columbia to determine which states required that their voting equipment be in compliance with FEC's standards.  We did not verify the officials' responses.
We performed our work at FEC headquarters in Washington, D.C., NASED, the Election Center, and the independent test authorities from March 2001 through September 2001, in accordance with generally accepted government auditing standards.
section_title: Appendix II: Comments From the Federal Election Commission
paragraphs: The following are GAO’s comments on the Federal Election Commission letter dated July 18, 2001.
subsections: section_title: GAO Comments
paragraphs: 1.  See comments 2, 5, and 6. 2.  We do not dispute either the chronology of events provided in FEC’s comments or its statement that it does not have explicit statutory authority to develop and revise the standards. We provide the relevant elements of this chronology in this report. Additionally, we state in this report that FEC has assumed and is performing an important role by developing and revising the standards, despite its lack of explicit statutory responsibility.
We do not agree with FEC’s comment that it has been proactive in updating the voting equipment standards. As FEC acknowledges in its comments, 7 years elapsed from the time the standards were first issued in 1990 to the time that FEC initiated efforts to assess the standards to identify areas that needed to be updated. During that time, considerable experience with the standards was accumulating, as vendors were developing voting equipment to meet the FEC standards and ITAs were testing against them. Since then, additional experience has been gained with the standards as vendors have continued to develop voting equipment to meet the standards, and ITAs have continued to test vendors’ equipment against the standards. For example, we state in this report that ITAs have had to interpret the 1990 standards in the testing process to accommodate vendors’ use of new technologies and expanded equipment functions that are not addressed in the 1990 standards. However, FEC does not formally receive these interpretations, any one of which could be the basis for prompting an update to the standards. In our view, waiting 9 years to begin updating the standards is too long, does not constitute proactive maintenance, and is the primary reason that the current standards are out of date. 3.  FEC is correct in stating that we did not assess all of the revised draft standards areas. However, we disagree that this assessment approach ignores the collaborative and dynamic process of NASED’s Voting Systems Committee and FEC’s staff in overseeing the development of the standards for two reasons. First, this report recognizes that FEC worked closely with state and local election officials in revising the standards. Second, this joint FEC and NASED process has no relevance to our findings that certain standards areas do not address the full range of items associated with well-defined system requirements in these areas. As we state in the objectives, scope, and methodology section of this report, our approach was to assess all of the 1990 standards because they are the standards against which voting equipment are currently being developed and independently tested. In assessing drafts of the updated standards, we assumed that those areas in the 1990 standards that we found to be satisfactory would continue to be satisfactory in the updated standards. As long as our findings are limited to the standards area that we assessed, the issue of whether we assessed all or some of the draft standards is not relevant. 4.  We acknowledge that FEC’s position, as stated in its comments, concerning standards areas omitted from the revised draft standards is that these were based on decisions resulting from deliberative and collaborative interaction among NASED and FEC staff, and were not, as we were told during the course of our review by the OEA Director, areas that were overlooked. Accordingly, we have modified our report to reflect this position. 5.  Subsequent to providing us written comments on a draft of this report, FEC provided us with a copy of volume II of the standards, which includes the tests to be conducted to ensure compliance with the voting equipment standards.  Based on our review of the relevant security sections, the standards satisfy the requirement for security certification.  We have modified our report, including the recommendations, to reflect this new information. 6.  We do not disagree that the draft standards discuss quality assurance and have been strengthened from the 1990 standards. We acknowledge these improvements to the standards in our report. However, as we state, quality assurance includes a number of activities. While FEC’s draft standards include some of these elements, they do not include all of them. Specifically, the draft standards do not address requirements for developing a quality assurance plan and conducting process reviews to ensure that the product and process standards are followed.
section_title: Appendix III: System Requirements and Supporting Life-Cycle Processes and Our Analysis
paragraphs: We identified 13 types of system requirements and 3 supporting life-cycle processes that are often associated with complete system requirements. FEC’s 1990 voting equipment standards satisfied 11 of the 13 system requirements areas and none of the life-cycle processes.  We reviewed FEC’s draft standards for those areas for which we identified variances in the 1990 standards and found that the draft standards had made improvements in all five areas.  However, the draft standards still do not satisfy human factors and quality assurance.  A detailed description of the system requirements areas and our complete analysis follow.
Definition/analysis Required system capabilities based on the purpose of the system; also includes parameters for response times, accuracy, capacities, unexpected/unallowed conditions, error-handling, and continuity of operations. 1990 analysis: Identified areas include ballot definition, candidate/measure selection, vote casting, ballot interpretation, voting reports, accuracy and integrity, processing speed, response times, and error and status messages. Quantitative measures of quality including reliability (perform correctly and consistently), maintainability (easily serviced/repaired/corrected), and availability (accessibility to be operated when needed). 1990 analysis: All identified areas included. Requirements for maintaining a secure system and protecting data privacy, including (1) security/privacy environment in which system must operate; (2) types of security to be provided (e.g., data confidentiality and fraud prevention); (3) risks the system must withstand; (4) safeguards required; (5) security/privacy policies that must be met; (6) accountability the system must provide (i.e., audit trails); and (7) criteria for security certification. 1990 analysis: Access control identified as a security safeguard, and requirements defined for audit records produced by the system to provide accountability. The other areas, however, are not addressed. Draft analysis: In addition to access controls and audit records, the security/privacy environment, the types of security to be provided, the risks the system must withstand, safeguards necessary, security policies, and criteria for security certification are identified. Requirements defining system usability of the system that take into account human capabilities and limitations, along with use and accessibility by persons with disabilities. 1990 analysis: System usability and accessibility by persons with disabilities are not identified. Draft analysis: Requirements for the use and accessibility by persons with disabilities are identified. System usability requirements are not. Characteristics of the interface between the voting system and other systems, including data types, data formats, and timing. 1990 analysis: Removable storage media, communications devices, and printers identified as external interfaces.
Draft Standards satisfied?
Yes Draft Standards satisfied?
Definition/analysis Requirements for system configuration to meet local operational requirements. 1990 analysis: Requirements defined for voting systems programming in accordance with ballot requirements of the election and the jurisdiction in which the equipment will be used. The natural environment that the system must withstand during transportation, storage, and operation, including (1) temperature, (2) humidity, (3) rain, and (4) motion/shock. 1990 analysis: Requirements identified for temperature, humidity, rain, transit drop, and vibration. Any commercial standards that must be used in the system’s development. 1990 analysis: Vendors are instructed to design equipment in accordance with best commercial and industrial practice; software is to be designed in a modular fashion, preferably using a high-level programming language. The system’s physical characteristics, including size, weight, color, nameplates, markings of parts and serial/lot numbers, transportability, and parts interchangeability. 1990 analysis: All requirements identified. Requirements for preventing or minimizing unintended hazards to personnel, property, and the physical environment. 1990 analysis: All systems shall be designed to meet the requirements of the Occupational Safety and Health Administration Requirements for who will use or support the system, such as number of workstations and built-in help/training features. 1990 analysis: Vendors instructed to include information on number of personnel and skill level required to maintain the voting system. Requirements for training devices and materials to be included with the system. 1990 analysis: Vendors instructed to document information required for system use and operator training, and orientation and training of poll workers, user maintenance technicians, and vendor personnel. Requirements for system maintenance, software support, and system transportation. 1990 analysis: Vendors instructed to document information required in these three areas.
Yes Draft Standards satisfied?
Definition/analysis The process of recording information produced during the life-cycle process. Describes and records information about a product, the processes used to develop the product, and provides a history of what happened during the development and maintenance of the product. Includes (1) identification of documents to be produced and delivered to customer or tester, (2) identification of format, content, and presentation items for each document, and (3) review and approval process for each document. 1990 analysis: Requirements identify products to be produced, including the content and format of the documents. Review and approval process not specified. Draft analysis: Products to be produced, including the content and format of the documents, as well as the review and approval process is identified. The process to establish and maintain the integrity of work products throughout the life-cycle process; it involves establishing product baselines and systematically controlling changes to them. The process should include (1) developing a configuration management plan, (2) identifying work products to be maintained and controlled, (3) establishing a repository to maintain and control the work products, (4) evaluating and approving changes to the products, (5) accounting for changes to the work products, and (6) managing the release and delivery of them. 1990 analysis: Includes requirements for (1) identifying work products to be maintained and controlled, (2) evaluating and approving changes to the products, and (3) managing the release and delivery of work products. The standards do not include requirements for developing a configuration management plan, establishing a repository to maintain and control the work products, and accounting for changes to the work products. Draft analysis: All areas identified. The process that provides adequate assurance of the system development process. It typically involves independent review of work products and activities to ensure compliance with applicable development standards and procedures. The process should include (1) developing a quality assurance plan, (2) determining system development product and process standards to be followed, and (3) conducting reviews to ensure that the product and process standards are followed. 1990 analysis: None of these areas specified. Draft analysis: The need to document the hardware and software development process is specified, but a quality assurance plan and quality assurance reviews are not.