Increasing computer interconnectivity—most notably growth in the use of the Internet—has revolutionized the way that our government, our nation, and much of the world communicate and conduct business. While this interconnectivity offers us huge benefits, without proper safeguards, it also poses significant risks to the government’s computer systems and, more importantly, to the critical operations and infrastructures they support. We reported in 2005 that while federal agencies showed improvement in addressing information security, they have also continued to have significant control weaknesses in federal computer systems, which puts federal assets at risk of inadvertent or deliberate misuse, financial information at risk of unauthorized modification or destruction, sensitive information at risk of inappropriate disclosure, and critical operations at risk of disruption. The Federal Information Security Management Act of 2002 requires each agency to develop, document, and implement an agencywide information security program. This program should provide security for the information and information systems that support the operations and assets of the agency, including those provided or managed by another agency, contractor, or other source. Among other things, the program is to include periodic testing and evaluation of the effectiveness of information security policies, procedures, and practices, to be performed with a frequency depending on risk, but no less than annually. The testing is to include management, operational, and technical controls for every system identified in the agency’s required inventory of major information systems. The act also assigns specific responsibilities to OMB and NIST. OMB’s responsibilities include the following: Overseeing agency information security policies and practices, including developing and overseeing the implementation of policies, principles, standards, and guidelines on information security. Reviewing agency information security programs, at least annually. Reporting to Congress annually on agency compliance with FISMA requirements. As part of the reporting process, OMB provides instructions to agencies and their Inspectors General on the annual FISMA reporting requirements. These instructions include performance measures for such things as the number of systems for which security controls have been tested and evaluated in the past year. OMB also uses performance measures to assist in its oversight responsibilities and to annually report to Congress on agencies’ compliance with the requirements of the act. FISMA also directs NIST to develop standards and guidelines for systems other than national security systems. These standards and guidelines instruct agencies on providing an acceptable level of information security for all agency operations and assets and contribute to the testing and evaluation of information security controls within an agencywide information security program. Recognizing the importance of documenting standards and guidelines as part of an agencywide information security program, NIST emphasizes that agencies must develop and promulgate formal, documented policies and procedures in order to ensure the effective implementation of security requirements. NIST standards and guidelines that contain elements applicable to periodic testing and evaluation include the following: Special Publication 800-26, Security Self-Assessment Guide for Information Technology Systems, November 2001. This publication is a self-assessment guide for agencies to use in determining the current status of their information security program. The guide includes a standardized form for reporting the results of system-level assessments and a method for evaluating the effectiveness of the agency’s information security program. The guide also emphasizes the importance of establishing levels of implementation, referred to as the IT security assessment framework. NIST Special Publication 800-26 is effective through the 2006 FISMA reporting period and will be rescinded when Special Publications 800-53A and 800-100 are finalized. Special Publication 800-37, Guide for the Security Certification and Accreditation of Federal Information Systems, May 2004. This guide is to be used for certifying and accrediting nonnational security systems. Developed as part of NIST’s project to promote the development of standards and guidelines to support FISMA, this guide specifies the need for ongoing activities to continuously monitor the effectiveness of security controls. Special Publication 800-53, Recommended Security Controls for Federal Information Systems, February 2005. This publication provides instructions on selecting and specifying security controls for information systems. It also provides the set of security controls that satisfy the depth and breadth of security requirements levied on information systems and provides the fundamental concepts associated with security controls selection and specification, including the identification and use of common security controls. In conducting security assessments, NIST states that assessment results can be used and shared to enhance the efficiency of evaluations and reduce security program costs. Special Publication 800-53A, Guide for Assessing the Security Controls in Federal Information Systems, April 2006. The publication is a second public draft to be used by agencies to assess the effectiveness of security controls employed in federal information systems. NIST establishes methods and procedures to assess the security controls in federal information systems, specifically those controls listed in NIST Special Publication 800-53, Recommended Security Controls for Federal Information Systems. These methods and procedures are designed for agencies to use in determining if the controls are implemented correctly, operating as intended, and producing the desired outcome with respect to meeting the security requirements of the agency. NIST closed acceptance of public comments on this draft on July 31, 2006, and plans to issue a final publication in December 2006. Having well-designed policies is critical for performing effective testing and evaluation of security controls. To assist agencies, OMB and NIST developed instructions, standards, and guidelines for testing and evaluating the controls over information systems. We used the following six elements to evaluate agencies’ policies for periodically testing security controls: 1. Identifying the frequency of periodic testing. 2. Defining roles and responsibilities of personnel performing the testing. 3. Selecting a minimum set of security controls evaluated during periodic tests. 4. Identifying and testing common security controls. 5. Determining the depth and breadth of periodic testing. 6. Including assessment results in remediation plans. The related federal and NIST references are shown in table 1. Agencies’ policies for periodically testing and evaluating security controls have not been adequately designed and effectively implemented. Specifically, none of the federal agencies’ policies fully addressed six important elements included in OMB and NIST guidelines and standards for performing effective security testing and evaluations. In addition, there were weaknesses in the security control assessments for the 30 systems reviewed at the six case study agencies. As a result, agencies have limited assurance that controls are implemented correctly, operating as intended, and producing the desired outcome. In addition, agencies may not be fully aware of security control weaknesses in their systems, thereby leaving the agencies’ operations and systems at risk. Agencies did not fully address six elements important for testing and evaluating security controls in their policies. Specifically, the (1) frequency of periodic testing was not always identified, (2) roles and responsibilities of personnel performing tests often were not clearly defined, (3) selection of a minimum set of security controls evaluated during periodic tests was not always fully addressed, (4) instructions on identification and testing of common security controls were not addressed, (5) instructions on determining the depth and breadth of testing were not included, and (6) descriptions of a process for documenting remedial actions to address deficiencies were not always addressed. Table 2 indicates weaknesses in developing and promulgating formal, documented policies to address the security elements needed for effective testing. FISMA requires agencies to perform—for all major information systems in their inventory—periodic testing and evaluation of the effectiveness of information security policies, procedures, and practices, to be performed with a frequency depending on risk, but no less than annually. Of the 23 agencies’ policies we reviewed, 7 agencies did not require that their security controls (management, operational, and technical) be tested and evaluated at least annually. For example, policies for 3 of the 7 agencies did not specify the frequency of periodic testing. The other 4 agencies identified the frequency of some testing activities—reviewing the overall security program annually, testing standard user account procedures annually, and certifying and accrediting systems at least every 3 years—but did not specify the frequency of periodic testing for other management, operational, and technical security controls. Unless agencies specify the frequency for conducting periodic testing and evaluations at least annually per FISMA, they may not have assurance that controls are being sufficiently evaluated and producing the desired outcome with respect to meeting the security requirements of the agency. NIST 800-37 identifies the roles and associated responsibilities with regard to testing and evaluating information security controls. These roles include the chief information officer, authorizing official, senior agency information security officer, information system owner, and information system security officer. In addition, NIST Special Publication 800-26 specifies that agencies should have procedures in place that identify who is conducting the security testing. Roles and responsibilities of personnel performing testing were not clearly defined in policies for 15 of the 23 agencies. Ten of the 15 agencies did not define roles and responsibilities for personnel performing tests in their policies and the other 5 agencies defined them only partially. For example, one agency defined roles and responsibilities for the system owner but not for other key security personnel such as the chief information security officer and information system security officer. As a result, agency officials may not clearly understand their expected responsibilities and consequently, may not be able to carry out their duties correctly and effectively. Baseline controls are the minimum security controls recommended for an information system based on the system’s security categorization. NIST Special Publication 800-53 provides guidance to agencies for selecting these security controls, which serve as a starting point in determining and designing methods for testing the security controls. NIST specifies that agency security personnel must develop, document, and implement policies for consistent identification, testing, and evaluation of baseline controls. Policies for selecting the minimum security controls evaluated during periodic tests for 11 of the 23 agencies were not always adequate. To illustrate, 7 of the 11 agencies reported having no specific policies or procedures for selecting the minimum baseline security controls, and the other 4 agencies’ policies partially addressed the selection of these controls. For example, one agency’s policy referenced NIST guidance for identifying controls, but it did not first specify the use of the NIST standard when determining the system’s impact level.