Critical Infrastructure Protection: Measures Needed to Assess Agencies' Promotion of the Cybersecurity Framework
Released: 2015-12-17T00:00:00

Letter

Background

U.S. critical infrastructure is made of systems and assets, whether physical or virtual, so vital to the United States that the incapacity or destruction of such systems and assets would have a debilitating impact on national security, economic security, national public health or safety, or any combination of these matters. Critical infrastructure includes, among other things, banking and financing institutions, telecommunications networks, and energy production and transmission facilities, most of which are owned and operated by the private sector. Sector-specific agencies (SSA) are federal departments or agencies with responsibility for providing institutional knowledge and specialized expertise as well as leading, facilitating, or supporting the security and resilience programs and associated activities of their designated critical infrastructure sector in the all-hazards environment.

The Nation Faces an Evolving Array of Cyber- Based Threats

Threats to systems supporting critical infrastructure and federal information systems are evolving and growing. Risks to cyber-based assets can originate from unintentional and intentional threats. Unintentional or non-adversarial threat sources include failures in equipment, software coding errors, or resource depletion, such as accidental actions of employees. They also include natural disasters and failures of critical infrastructure on which the organization depends but are outside of its control. Intentional threats include both targeted and untargeted attacks from a variety of sources, including criminal groups, hackers, disgruntled employees, foreign nations engaged in espionage and information warfare, and terrorists. These threat adversaries vary in terms of the capabilities of the actors, their willingness to act, and their motives, which can include seeking monetary gain or pursuing an economic, political, or military advantage. Table 1 describes the sources of cyber-based threats in more detail.

Cyber threat adversaries make use of various techniques, tactics, and practices—or exploits—to adversely affect an organization’s computers, software, or networks, or to intercept or steal valuable or sensitive information. These exploits are carried out through various conduits, including websites, e-mail, wireless and cellular communications, Internet protocols, portable media, and social media. Further, adversaries can leverage common computer software programs, such as Adobe Acrobat and Microsoft Office, to deliver a threat by embedding exploits within software files that can be activated when a user opens a file within its corresponding program. Table 2 provides descriptions of common exploits or techniques, tactics, and practices used by cyber adversaries.

Federal Policy and Law Address the Protection of Cyber Critical Infrastructure

Because the private sector owns the majority of the nation’s critical infrastructure—such as banking and financial institutions, commercial facilities, and energy production and transmission facilities—it is vital that the public and private sectors work together to protect these assets and systems. Toward this end, federal law and policy assign roles and responsibilities for agencies to assist the private sector in protecting critical infrastructure, including enhancing cybersecurity.

Presidential Policy Directive 21 (PPD-21) assigns roles and responsibilities for the critical infrastructure sectors to the SSAs. The directive identified 16 critical infrastructure sectors and designated associated federal SSAs. Table 3 shows the 16 critical infrastructure sectors and the SSA for each sector.

PPD-21 identified SSA roles and responsibilities to include collaborating with critical infrastructure owners and operators; independent regulatory agencies; and state, local, tribal, and territorial entities as appropriate, as well as providing, supporting, or facilitating technical assistance and consultations for their respective sectors to identify vulnerabilities and help mitigate incidents, as appropriate.

Federal law and policy have also established roles and responsibilities for federal agencies to work with industry to enhance the cybersecurity of the nation’s critical infrastructures. These include Executive Order 13636, the Cybersecurity Enhancement Act of 2014, and the National Infrastructure Protection Plan (NIPP).

Executive Order 13636, Improving Critical Infrastructure Cybersecurity, issued in February 2013, outlines an action plan for improving security for critical cyber infrastructure. This includes, among other things, requirements for NIST to develop a voluntary critical infrastructure cybersecurity framework and performance measures. In developing the cybersecurity framework, NIST is to engage in an open public review and comment process. The order also directs SSAs, in consultation with DHS and other interested agencies, to review the cybersecurity framework and if necessary, develop implementation guidance or supplemental materials to address sector-specific risks and operating environments.

The Cybersecurity Enhancement Act of 2014, enacted in December 2014, established requirements that are consistent with the order regarding NIST’s development of a cybersecurity framework. NIST’s responsibilities in developing the cybersecurity framework under this law include, among other things, identifying an approach that is performance-based, and cost-effective.

In response to Executive Order 13636, NIST issued the Framework for Critical Infrastructure Cybersecurity in February 2014, which is intended to help organizations apply the principles and best practices of risk management to improving the security and resilience of critical infrastructure. The framework proposes a risk-based approach to managing cybersecurity risk and is composed of three parts: the framework core, the framework profile, and the framework implementation tiers.

The framework core is a set of cybersecurity activities, outcomes, and informative references that are common across critical infrastructure sectors, which is to provide guidance for developing individual organization profiles. Through the use of profiles, the framework is intended to help organizations align their cybersecurity activities with business requirements, risk tolerances, and resources. The tiers provide a mechanism for organizations to view and understand the characteristics of their approach to managing cybersecurity risk. (Further information on the framework core is provided in app. II.)

The NIPP defines the overarching approach for integrating the nation’s critical infrastructure protection and resilience activities into a single national effort. DHS developed the NIPP in collaboration with public- and private-sector owners and operators and federal and nonfederal government representatives, including SSAs, from the critical infrastructure community. It details DHS’s roles and responsibilities in protecting the nation’s critical infrastructures and how sector stakeholders should use risk management principles to prioritize protection activities within and across sectors. It emphasizes the importance of collaboration, partnering, and voluntary information sharing among DHS and industry owners and operators, and state, local, and tribal governments.

According to the NIPP, SSAs are to work with their private-sector counterparts to understand cyber risk and develop sector-specific plans that address the security of the sector’s cyber and other assets and functions. The SSAs and their private-sector partners are to update their sector-specific plans based on DHS sector-specific plan guidance issued in 2014.

NIST Developed the Cybersecurity Framework in a Facilitated Manner to Fulfill Responsibilities

NIST used several methods to obtain and incorporate stakeholder views when developing its cybersecurity framework. Respondents to our survey were generally satisfied with NIST’s efforts and methods to develop the framework. Also, NIST met the requirements under Executive Order 13636 and the Cybersecurity Enhancement Act to develop an approach that can help critical infrastructure organizations manage cyber risk.

NIST Obtained and Incorporated Stakeholder Views in Developing the Framework

Executive Order 13636, released in February 2013 required NIST to create a flexible performance-based cybersecurity framework that includes a set of standards, procedures, and processes that align policy, business, and technological approaches to address cyber risks. In addition, under the Cybersecurity Enhancement Act, enacted into law in December 2014, NIST is required to facilitate and support the development of a voluntary set of standards, guidelines, methodologies, and procedures to cost-effectively reduce cyber risks to critical infrastructure.

In February 2014, NIST issued the Framework for Critical Infrastructure Cybersecurity. In developing the framework, NIST used several methods to obtain and incorporate the views of experts from government, industry, and academia. Specifically, it solicited public comments through formal “requests for information,” held workshops with stakeholders to identify and develop elements of the framework, and published a draft version of the framework for further review and comment. Figure 1 summarizes the development of the framework.

NIST began the collaboration process for developing the framework on February 26, 2013, when it issued a formal request for information in the Federal Register to seek comments regarding risk management practices, frameworks, standards, guidelines, and best practices. NIST received 246 unique comments in response to the request from organizations and individuals representing, among others, large companies, associations, sector coordinating councils, federal agencies, universities, and international companies. NIST analyzed the comments received to, among other things, identify common cybersecurity practices and methods to facilitate discussions on the development of the framework.

After the initial request for information, NIST hosted six workshops at various locations across the country to drive the development of the framework. Workshop participants and attendees included critical infrastructure owners and operators, industry associations, individual companies, and government agencies. Table 4 summarizes the NIST workshops.

At each workshop, NIST facilitated discussions with attendees and accepted industry input to collaboratively identify and develop standards and guidelines for the framework. Based on the input from the first four workshops, NIST prepared and published a preliminary draft of the framework for public review and comment following its fourth workshop.

NIST analyzed responses to its request for information, conclusions from workshops, and stakeholder analysis to select components for the framework, such as identifying security common practices, principles, and approaches that supported the objectives of Executive Order 13636. To identify the framework components that reflected the comments received, NIST used input from volunteer industry stakeholders. These stakeholders helped to, among other things, evaluate common themes identified by NIST as well as cybersecurity areas that needed additional exploration to encourage industry engagement in the framework development process. After the initial draft was issued for comment, NIST held the fifth workshop to obtain further stakeholder input. Subsequently, NIST published its final draft of the framework in February 2014.

Following issuance of the framework, on August 26, 2014, NIST issued a second request for information to seek comments on users’ experience with the framework, as part of its efforts to promote use of the framework, and held an additional workshop to obtain information on how organizations learned about and used the framework. NIST received 57 unique comments in response to the second request for information from organizations and individuals that represented, among others, large companies, associations, and sector coordinating councils.

In addition to the framework, NIST developed a roadmap to discuss future plans for the framework, which included identifying areas of improvement of the preliminary framework. To promote the use and adoption of the framework, NIST officials stated they plan to update the framework based on industry feedback and develop guidance on how organizations can use the framework to reduce cybersecurity risks.

Participants Who Responded to Our Survey Were Generally Satisfied with NIST’s Development Efforts

One of NIST’s responsibilities under the Cybersecurity Enhancement Act was to incorporate industry input in the development of the standards and methodologies to manage cybersecurity risks for critical infrastructure. A majority of the 252 respondents to our survey indicated satisfaction with the mechanisms employed by NIST to develop the framework. For example, 186 of 251 respondents indicated that they were “very satisfied” or “satisfied” that NIST provided opportunities for them to provide feedback on the framework during its development process. Similarly, a majority (170 of 187) indicated the workshops hosted by NIST were “very” or “somewhat effective” in engaging industry involvement in development of the framework. Appendix III provides additional details on survey respondents’ evaluations of NIST’s collaborative approach to developing the framework.

NIST’s Cybersecurity Framework Meets Requirements

NIST implemented the requirements for development of a cybersecurity approach as required by Executive Order 13636 and the Cybersecurity Enhancement Act of 2014.

Executive Order 13636 required NIST to develop among other things a flexible, repeatable, performance-based, and cost-effective approach to help owners and operators of critical infrastructure identify, assess, and manage cyber risk. In addition, NIST was to develop a set of standards that align policy, business, and technological approaches to address cyber risks.

Similar to Executive Order 13636, the Cybersecurity Enhancement Act of 2014 authorizes NIST to, among other things, facilitate and support the development of a voluntary set of standards, guidelines, methodologies, and procedures to cost-effectively reduce cyber risks to critical infrastructure. In carrying out these activities, NIST is required to identify, among other things, a flexible, repeatable, performance-based, and cost- effective approach that can be voluntarily adopted to help identify, assess, and manage cyber risks.

To ensure the framework could assist owners and operators with their cyber risk as called for by the executive order and the act, NIST created implementation tiers to allow organizations to determine their cybersecurity risks and identify processes that align to their business approaches to manage those risks. According to the framework, implementation tiers describe how cybersecurity risk is managed by an organization and the degree to which its risk management practices exhibit key characteristics defined in the framework. There are four tiers discussed in the framework, with each one building upon the previous tier: Partial (Tier 1), Risk Informed (Tier 2), Repeatable (Tier 3), and Adaptive (Tier 4).

Further, based on our analysis, the framework supported and developed by NIST is intended to be:

Flexible: The NIST framework can be modified to an organization’s cybersecurity condition and needs. Its processes also provide for a flexible and risk-based implementation of the framework and can be used with a broad array of cybersecurity risk management processes.

Repeatable: Under the framework, organizations are able to create a framework profile to compare their current cybersecurity state to their targeted state. This allows for identification and prioritization of improvement opportunities within the context of a continuous and repeatable process that can be assessed as the organization moves toward its targeted state.

Performance-based: Organizations can use the framework implementation tiers to measure progress for managing cybersecurity risk. Specifically, the implementation tiers allow organizations to track their performance in managing cyber risks from an informal, reactive response to an approach that is agile and risk-informed. For example, in order to track performance, organizations are encouraged to identify their desired tier according to organizational goals and to progress to higher tiers to continue reducing cybersecurity risks.

Cost-effective: The framework’s implementation tiers can be used to allow organizations to evaluate their current cyber activities to determine if adoption of cybersecurity risk management practices was sufficient given their mission and regulatory requirements. Additionally, these tiers allow organizations to determine activities that are most important to their critical services, allowing them to prioritize expenditures to maximize the impact of their investment in cybersecurity.

Further, in accordance with the executive order and the act, the framework is voluntary. The framework emphasizes that the information it contains is guidance for individual organizations to use to improve their cybersecurity posture. According to NIST, the implementation tier approach allows for organizations to determine activities that are most important to critical service delivery as well as prioritize expenditures to maximize the impact of their cybersecurity investment.

Federal Entities Are Promoting the Cybersecurity Framework, but DHS Is Not Measuring the Effectiveness of Its Efforts

DHS, SSAs, and NIST have promoted adoption and use of the cybersecurity framework by critical infrastructure owners and operators through a variety of methods. Specifically, DHS established a program, in accordance with Executive Order 13636, to encourage adoption of the framework and has taken a number of actions, including disseminating guidance and working with stakeholders to promote it. However, DHS is not measuring the effectiveness of these actions in order to evaluate the results of its activities and programs. For their part, SSAs for most sectors are developing tailored guidance for implementing the framework in their sectors, and NIST has promoted the framework through public events and its website.

DHS’s Voluntary Program Promotes the Framework but Is Not Measuring the Effectiveness of Efforts

DHS, as required by Executive Order 13636, established the Critical Infrastructure Cyber Community (C) Voluntary Program in February 2014 to support the voluntary adoption of the framework. The program is intended to enhance critical infrastructure cybersecurity and encourage the adoption of the NIST framework. According to DHS C Voluntary Program has framework promotion actions broken out into three phases. Phase 1 is focused on outreach and building awareness of the framework. Phase 2, which was initiated in 2015, involves entity capability building, where DHS promotes the framework to specific types of entities, such as academia, business, and state governments, and highlighting resources to assist in implementing the framework from DHS, other federal agencies, and the private sector. Phase 3 is to facilitate the creation of communities of interest around critical infrastructure cybersecurity.

To promote awareness of the framework during phase 1, DHS launched a website with guidance and links to resources to assist critical infrastructure organizations interested in implementing the framework. DHS also developed a webinar series in January 2015 to provide organizations additional resources for addressing and improving cybersecurity risk management practices. Several past webinars are available on demand on the website. DHS Cstated that they specifically targeted promotional efforts to entities identified as having nationally critical assets. According to officials, as of October 1, 2015, the C Voluntary Program had held 256 briefings since November 2013.

For phase 2, which began in 2015, DHS C these included resources for academia; business (large, midsize, and small); federal government; and state, local, tribal and territorial governments. For example, to assist in identifying cyber resilience and practices, DHS points to its cyber resiliency review. According to officials, as of October 1, 2015, the resiliency review had been downloaded from the site over 6,500 times. Further, to assist with cybersecurity incident detection, DHS identified the Cyber Information Sharing and Collaboration Program, which is intended to share incident information. The C Voluntary Program also developed a Small and Midsize Business Toolkit, which contains a number of resources to help entities recognize and address cybersecurity risks, including information for startups and a guide to free tools intended to provide cybersecurity assistance. The toolkit, which was posted on the program’s website in May 2015, had received over 2,000 downloads as of October 1, 2015, according to officials.

Among the respondents to our survey who indicated that the Cresponses out of 112 responding to the question indicated that the C Voluntary Program promotional activities were “very” or “somewhat” effective in encouraging the use of the framework.

DHS C Voluntary Program was assisting SSAs and 10 sectors in developing framework implementation guidance. According to DHS, sector-specific implementation guidance will provide each sector with a tailored approach on how to best use the framework within their industry and make it easier for organizations to (1) use the framework to secure themselves against cyber risk, (2) learn about available tools and resources and approaches that can support cybersecurity risk management and the framework, and (3) gain an understanding of how different sectors and industries are approaching cybersecurity risk management broadly.

DHS C According to DHS officials, Phase 3, which is to facilitate the creation of communities of interest around DHS cybersecurity initiatives, began in 2015 and will continue into 2016. DHS officials stated that the intended outcome of this phase is to facilitate the development of self-sustaining communities by continuing to provide forums for information sharing among critical infrastructure owners and operators across the nation. DHS officials stated that phase 3 will be accomplished through a series of regional events, webinars, and development of new resources that promote information sharing and community building.

DHS Is Not Measuring the Effectiveness of Its Framework Promotion Efforts

Performance measurement involves identifying performance goals and measures, establishing performance baselines by tracking performance over time, identifying targets for improving performance, and measuring progress against those targets. As we have previously reported, according to leading practices in the federal government and in industry, organizations should measure performance in order to evaluate the success or failure of their activities and programs. In addition, the Standards for Internal Control in the Federal Government, known as the “Green Book,” sets internal control standards for federal entities. Those standards state that internal control monitoring should occur and that the quality of performance over time should be assessed.

DHS C Voluntary Program documentation identifies three metrics that align with program goals: (1) making resources accessible, (2) increasing participation by entities with the CHowever, none of these metrics indicate the effectiveness of the program’s efforts to promote adoption of the framework, and program officials are not otherwise measuring or tracking how effective those efforts or materials are in encouraging individuals and organizations to voluntarily adopt the framework. For example, they are not tracking what percentage of an individual sector they have promoted to or how effective their efforts and guidance are at encouraging the use of the framework by entities within a critical infrastructure sector.

According to DHS C Voluntary Program officials, they have not established these metrics or monitoring mechanisms because DHS has focused on getting as much information and resources out as possible. However, without understanding whether its promotional efforts are effective, the C Voluntary Program and NIST personnel or resources, speaking at industry conferences, and providing information during sector coordinating council meetings. For example, within the water sector, the Environmental Protection Agency (EPA) and the sector coordinating council are working to understand how entities within the sector are managing risk, including using the framework. Specifically, EPA is supporting the council by providing contractor and subject matter expert support as the council develops metrics and administers a survey to the sector through 2016. EPA officials stated that once the sector coordinating council receives and analyzes the information received from the survey, it will provide the information to EPA. Another example is the transportation systems sector, where DHS, as the co-SSA, held several promotional activities for Cybersecurity Month in October 2015.

Respondents to our survey who indicated that their SSA promoted use of the framework stated that they were usually encouraged to use the framework as a result. Specifically, 60 of the 82 respondents who responded to that question indicated that SSA promotional activities were “very” or “somewhat” effective in encouraging the use of the framework.

In addition, most SSAs have determined whether to develop framework implementation guidance to address sector-specific risks and operating environments for their sector. Executive Order 13636 requires SSAs to determine whether or not it is necessary for their sector to develop sector- specific framework implementation guidance, and SSAs for 15 of the 16 sectors had made this determination as of October 2015. Of the 15 sectors, 5 have completed and released framework implementation guidance. These framework implementation guides were created by SSAs, sector stakeholders, or a combination of both. For example, the energy sector was the first sector to produce implementation guidance, and the SSA worked with sector stakeholders to produce it. The health care and public health sector had a sector stakeholder provide a crosswalk between their risk framework and the NIST cybersecurity framework. HHS officials representing the health care and public health SSA stated that they are beginning an effort to create further implementation guidance for the sector’s seven subsectors.

Seven other sectors have begun drafting implementation guidance. According to officials, these sectors were finalizing their guidance and had involved their sector stakeholders to review it.

For the food and agriculture sector, Department of Agriculture and Food and Drug Administration officials representing the SSAs for the sector stated that they had yet to begin drafting framework implementation guidance and were exploring the creation of sector guidance with their stakeholders.

The Departments of Defense (DOD) and the Treasury, as SSAs for the defense industrial base and financial services sectors, respectively, decided not to create sector-specific framework implementation guidance. According to DOD officials representing the defense industrial base SSA, the department determined, after discussions with the defense industrial base sector coordinating council, that implementation guidance is not needed. The DOD officials stated that there was a consensus within the sector coordinating council that sector companies would voluntarily adopt the framework as they determined to be necessary. Treasury officials representing the SSA for the financial services sector stated that they determined, with input from the sector, that implementation guidance should not be created by Treasury as the financial services SSA. According to the Treasury officials this decision was made in cooperation with the financial services sector coordinating council, and they concluded that implementation guidance was not necessary due to the regulatory structure of the sector.

DHS and the General Services Administration (GSA), which are the co- SSAs for the government facilities sector, have yet to determine if sector implementation guidance should be developed. A GSA official representing the SSA for the government facilities sector stated that there are metrics in the sector-specific plan that will allow GSA to gather information from the sector on its promotional and implementation needs and use that information to best meet the needs of the sector. According to DHS and GSA officials they are waiting until this information is gathered and assessed before discussing whether sector-specific implementation guidance will be needed. DHS and GSA officials do not have a specific time frame for when the information will be gathered or for when they will make a decision. Without a decision by DHS and GSA under Executive Order 13636 whether guidance is needed to address sector-specific risks and operating environments, implementation of the framework in the government facilities sector may be hindered.

NIST Continues to Promote the Framework

Although not specifically required to by Executive Order 13636 or the Cybersecurity Enhancement Act of 2014, NIST has continued its efforts to promote the framework. NIST encouraged the use of the framework at workshops and public events it hosted and updated its website to provide information on upcoming events and resources related to the framework. Specifically, NIST’s website for the framework (www.nist.gov/cyberframework) provides a list of publically available resources to assist entities interested in using the framework. The website includes guidance and tools created by federal and private sector entities for implementing the framework. NIST encourages entities to submit publically available framework implementation guidance that they create so that NIST can provide information about it with the other guidance and tools already highlighted. The website also lists upcoming events where NIST officials will provide framework information and perspectives. Past speaking events from across the country are also listed with links to the event webpage and in some cases the NIST presentation slides.

Respondents to our survey who indicated they had been promoted to by NIST noted that they were encouraged to use the framework as a result. Specifically, 102 responses out of 132 indicated that NIST promotional activities were “very” or “somewhat” effective in encouraging the use of the framework.

Additionally, NIST officials stated they are following implementation of the framework, and a section on the NIST website is dedicated to accounts from entities of how they have implemented the framework. As of October 2015, the section listed an Intel use case for framework implementation and a link to a podcast on how the University of Pittsburgh is using the framework.

NIST officials added that they are seeking feedback on the framework. Specifically, officials stated that they are researching how organizations are applying the framework methodologies, what value they are obtaining from the framework, and what challenges they are facing in implementing the framework. NIST officials stated they are currently focused on the overall effectiveness of the framework for the benefit of those using it to improve future versions. They further stated that they continue to ask stakeholders when an appropriate time to update the framework would be. To solicit this feedback, officials stated that they are asking individuals attending NIST presentations and may issue a request for information through the Federal Register regarding whether or not the critical infrastructure community believes that it is necessary to begin updating the framework.

Conclusions

NIST has generally fulfilled its requirements, established in Executive Order 13636 and the Cybersecurity Enhancement Act, to develop a cybersecurity framework for adoption by critical infrastructure sectors. By using a collaborative process for developing the framework, NIST has helped ensure that the resulting guidance, standards, and methodologies, if effectively implemented, can help cost-effectively reduce cyber risks to critical infrastructure.

To facilitate the voluntary adoption of the framework by critical infrastructure owners and operators, DHS, sector-specific agencies, and NIST are taking a variety of actions. However, while DHS has established a program dedicated to encouraging the framework’s adoption, without establishing metrics to assess the effectiveness of these efforts, it has less assurance that it is meeting its objectives. In addition, while most SSAs have determined the need for sector-specific guidance to implement the framework, DHS and GSA have yet to meet this requirement for the government facilities sector, which may hinder adoption of the framework in this sector.

Recommendations for Executive Action

To better facilitate adoption of the NIST Framework for Improving Critical Infrastructure Cybersecurity, we are making the following recommendations: We recommend that the Secretary of Homeland Security direct officials responsible for the Critical Infrastructure Cyber Community Voluntary Program to develop metrics for measuring the effectiveness of efforts to promote and support the framework.

We also recommend that the Secretary of Homeland Security and the Administrator of GSA set a time frame for determining the need for sector-specific guidance to implement the framework in the government facilities sector.

Agency Comments and Our Evaluation

We provided a draft of this report to the Departments of Agriculture, Commerce, Defense, Energy, Health and Human Services, Homeland Security, Transportation, and the Treasury and to GSA and EPA.

In written comments signed by the Director, Departmental GAO-OIG Liaison Office (reprinted in app. IV), DHS concurred with our two recommendations. DHS agreed with the need to further refine and mature metrics for measuring the effectiveness of efforts to promote and support the framework. DHS also provided details about efforts to track output metrics addressing the CIn addition, officials from the Departments of Commerce, Health and Human Services, and Homeland Security provided technical comments via e-mail that have been addressed in this report as appropriate. The Departments of Agriculture, Defense, Energy, Transportation, and the Treasury and EPA responded via e-email that they had no comment on the report.

We are sending copies of this report to the appropriate congressional committees; the Secretaries of Agriculture, Commerce, Defense, Energy, Health and Human Services, Homeland Security, Transportation, and the Treasury; the Administrators of the Environmental Protection Agency and General Services Administration; and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-6244 or wilshuseng@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix VI.

Appendix I: Objectives, Scope, and Methodology

The objectives of our review were to determine the extent to which (1) the National Institute of Standards and Technology (NIST) facilitated the development of voluntary standards and procedures to reduce cyber risks to critical infrastructure, and (2) federal agencies promote the standards and procedures to reduce cyber risks to critical infrastructure.

To determine how NIST facilitated the development of voluntary standards and procedures for critical infrastructure, we reviewed and analyzed the actions taken by NIST to develop its Framework for Improving Critical Infrastructure Cybersecurity. In addition, we analyzed Executive Order 13636, issued in February 2013, and the Cybersecurity Enhancement Act of 2014, enacted in December 2014, to identify key NIST responsibilities for developing a cybersecurity framework. We analyzed documents and performed interviews with NIST officials to assess its collaborative efforts with industry stakeholders in soliciting input in the development of the framework, including workshops it hosted and the website it set up to disseminate updates on the framework. Specifically, we reviewed documentation and videos of the six workshops hosted by NIST intended to obtain industry, academia, and government representative feedback in the development of the framework, in addition to NIST’s two requests for information to the public for input on cybersecurity standards and methodologies. We also analyzed the resulting framework to assess whether NIST had fulfilled its responsibilities under law.

Additionally, to address this objective, we conducted a web-based survey of individuals who (1) provided written comments with contact information in response to a NIST request for information notice or (2) registered for at least one of the workshops hosted by NIST to develop the framework. There were 2,082 individuals in the population that we targeted, and to make the survey as inclusive as possible we sent the survey request to all of them. The questionnaire included questions about the effectiveness of NIST’s collaborative efforts in fulfilling requirements to develop the framework using an open and public comment process. To minimize errors arising from differences in how questions might be interpreted and to reduce variability in responses that should be qualitatively the same, we conducted pretests with critical infrastructure representatives over the telephone. Based on feedback from these pretests, we revised the questionnaire to improve the clarity of the questions. An independent survey specialist within GAO also reviewed a draft of the questionnaire prior to its administration.

After completing the pretests, we administered the survey to the NIST workshop attendees and request for information respondents on August 10, 2015, notifying them that our online questionnaire would be activated within a couple of days. On August 18, 2015, we sent a second e-mail message to these individuals, informing them that the questionnaire was available online and providing them with unique passwords and usernames. We collected responses through August 24, 2015. We were able to obtain 252 completed questionnaires, a 12 percent response rate, in the time available for survey fieldwork. Because we do not know if the answers that nonrespondents would have given would materially differ from those that did respond, our results can only represent the views of those who did respond. Their views are not generalizable to the registrant and respondent population as a whole.

To address our second objective, we reviewed and analyzed actions and documentation related to promoting the framework by the nine sector- specific agencies (SSAs) responsible for the 16 critical infrastructure sectors established in Presidential Policy Directive-21, including the Department of Homeland Security (DHS), and NIST. For DHS, we analyzed agency documentation and the website of its Critical Infrastructure Cyber Community (C) Voluntary Program to identify the framework promotional guidance and tools provided to the critical infrastructure sectors. Also, we analyzed the metrics and information being used by the DHS CTo analyze NIST’s promotional efforts, we analyzed documentation and interviewed relevant NIST officials. We reviewed the NIST framework website to understand how NIST was informing the public about its public events to promote the framework and presenting entities with guidance to implement the framework for other agencies, sectors, and third parties.

In addition, we surveyed individuals who responded to NISTs requests for information and registered for one of the workshops hosted by NIST for the development of the framework to identify and evaluate the effectiveness of DHS, SSA, and NIST efforts to promote the framework. We also interviewed federal officials from DHS’s C Voluntary Program, the nine SSAs representing the 16 critical infrastructure sectors, and NIST regarding their efforts to promote the framework and create sector- specific framework implementation guidance.

We conducted this performance audit from February 2015 to December 2015 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: The Core of NIST’s Cybersecurity Framework

The National Institute of Standards and Technology (NIST) Framework for Improving Critical Infrastructure Cybersecurity is intended to help organizations apply the principles and best practices of risk management to improving the security and resilience of critical infrastructure. The framework proposes a risk-based approach to managing cybersecurity risk and is composed of three parts: the framework core, the framework profile, and the framework implementation tiers.

The framework core includes a listing of functions, categories, subcategories, and informative references that describe specific cybersecurity activities the framework identified as common across all critical infrastructure sectors. According to NIST, the framework core represents a common set of activities for managing cybersecurity risk. The framework also states that while it is not exhaustive, it is extensible, allowing organizations, sectors, and other entities to use subcategories and informative references that are cost-effective and efficient and that enable them to manage their cybersecurity risk.

Table 6 includes the five functions and 22 categories of the framework core, and Table 7 includes information for one of the categories, asset management, as described in the NIST framework and appendix A of the framework. The information presented here represents how each function has categories, subcategories, and informative references. For more information on the framework, framework core, and categories, see the NIST framework website at www.nist.gov/cyberframework.

Table 7 provides an example of the subcategories and related informative references for a single category of the identify function: asset management.

Appendix III: Survey Questions and Responses on Development of the Cybersecurity Framework

The following table presents selected questions and responses from our survey of registrants at the National Institute of Standards and Technology’s (NIST) workshops for developing the NIST Framework for Improving Critical Infrastructure Cybersecurity and commenters who responded to NIST’s request for information. There were 2,082 individuals in the population that we targeted, and to make the survey as inclusive as possible we sent the survey request to all of them. We obtained 252 completed questionnaires, a 12 percent response rate, in the time available for survey fieldwork. Because we do not know if the answers that nonrespondents would have given would materially differ from those that did respond, our results represent the views of those who did respond. They are not generalizable to the registrant and respondent population as a whole. Not all questions were applicable to or otherwise answered by the respondents to our survey. Our results represent those answering each specific question.

Appendix IV: Comments from the Department of Homeland Security

Appendix V: Comments from the General Services Administration

Appendix VI: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Michael W. Gilmore, Assistant Director; Sher’rie Bacon; Wilfred Holloway; Lee McCracken; David Plocher; Carl Ramirez; and Jeffrey Woodward made key contributions to this report.