Critical Infrastructure Protection: Measures Needed to Assess Agencies' Promotion of the Cybersecurity Framework
Published: 2015-12-17T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: U.S. critical infrastructures, such as financial institutions and communications networks, are systems and assets vital to national security, economic stability, and public health and safety. Systems supporting critical infrastructures face an evolving array of cyber-based threats. To better address cyber-related risks to critical infrastructure, federal law and policy called for NIST to develop a set of voluntary cybersecurity standards and procedures that can be adopted by industry to better protect critical cyber infrastructure.
The Cybersecurity Enhancement Act of 2014 included provisions for GAO to review aspects of the cybersecurity standards and procedures developed by NIST. This report determines the extent to which (1) NIST facilitated the development of voluntary cybersecurity standards and procedures and (2) federal agencies promoted these standards and procedures. GAO examined NIST's efforts to develop standards, surveyed a non-generalizable sample of critical infrastructure stakeholders, reviewed agency documentation, and interviewed relevant officials.
section_title: What GAO Found
paragraphs: In accordance with requirements in a 2013 executive order which were enacted into law in 2014, the National Institute of Standards and Technology (NIST) facilitated the development of a set of voluntary standards and procedures for enhancing cybersecurity of critical infrastructure. This process, which involved stakeholders from the public and private sectors, resulted in NIST's Framework for Improving Critical Infrastructure Cybersecurity . The framework is to provide a flexible and risk-based approach for entities within the nation's 16 critical infrastructure sectors to protect their vital assets from cyber-based threats. To develop the framework in a collaborative manner, NIST solicited input from sector stakeholders through a formal request for information and conducted multiple workshops with critical infrastructure owners and operators, industry associations, government agencies, and other stakeholders. Participants GAO surveyed were generally satisfied with the approach NIST took to develop the framework. Further, the framework meets the requirements established in federal law that it be flexible, repeatable, performance-based, and cost-effective. For example, the framework contains multiple implementation “tiers,” which allows it to be adapted to an organization's specific conditions and needs.
Agencies with responsibilities for supporting protection efforts in critical infrastructure sectors (known as sector-specific agencies), and NIST have promoted and supported adoption of the cybersecurity framework in the critical infrastructure sectors. For example, the Department of Homeland Security (DHS) established the Critical Infrastructure Cyber Community Voluntary Program to encourage adoption of the framework and has undertaken multiple efforts as part of this program. These include developing guidance and tools that are intended to help sector entities use the framework. However, DHS has not developed metrics to measure the success of its activities and programs. Accordingly, DHS does not know if its efforts are effectively encouraging adoption of the framework.
Sector-specific agencies have also promoted the framework in their sectors by, for example, presenting to meetings of sector stakeholders and holding other promotional events. In addition, all of the sector-specific agencies except for DHS and the General Services Administration (GSA), as co-SSAs for the government facilities sector, had decided whether or not to develop tailored framework implementation guidance for their sectors, as required by Executive Order 13636. Specifically, DHS and GSA had not yet set a time frame to determine whether sector-specific implementation guidance is needed for the government facilities sector. By not doing so, DHS and GSA may be hindering the adoption of the cybersecurity framework in this sector.
section_title: What GAO Recommends
paragraphs: GAO recommends that DHS develop metrics to assess the effectiveness of its framework promotion efforts. In addition, DHS and GSA should set a time frame to determine whether implementation guidance is needed for the government facilities sector. DHS and GSA concurred with the recommendations.

section_title: Letter
paragraphs: The nation’s critical infrastructure provides the essential services–such as  banking, water, and electricity–that underpin American society, and it  relies extensively on computerized systems and electronic data to carry  out its missions. The cyber threat to critical infrastructure continues to  grow and represents a serious national security challenge. Foreign  malicious actors have directly attacked and extracted highly sensitive  materials from the networks of government agencies and major critical  infrastructure companies.
Due to the cyber-based threats to federal systems and critical  infrastructure, the persistent nature of information security vulnerabilities,  and the associated risks, we continued to designate federal information  security as a government-wide high-risk area in our most recent biennial  report to Congress, a designation we have made in each report since  1997. In 2003, we expanded this high-risk area to include the protection  of critical cyber infrastructure and we continued to do so in the most  recent update to our high-risk list.
To better address these cyber-related risks, the President issued  Executive Order 13636, Improving Critical Infrastructure Cybersecurity,  on February 12, 2013. This order aimed to enhance the security and  resilience of the nation’s critical infrastructure and to maintain a cyber  environment that encourages efficiency, innovation, and economic  prosperity while promoting safety, security, business confidentiality,  privacy, and civil liberties. It called for, among other things, the Director of  the National Institute of Standards and Technology (NIST) to lead the  development of a voluntary risk-based cybersecurity framework that  would comprise a set of industry standards and best practices to help  organizations manage cybersecurity risks. In addition, the Cybersecurity  Enhancement Act of 2014 was enacted in December 2014 to authorize,  among other things, NIST to facilitate and support the development of a  voluntary set of standards to reduce cyber risks to critical infrastructure.
The Cybersecurity Enhancement Act of 2014 also included a provision for  us to review, in a series of reports, various aspects of the cybersecurity  standards and procedures developed by NIST. Our objectives for this  review were to determine the extent to which (1) NIST facilitated  development of voluntary standards and procedures to reduce cyber risks  to critical infrastructure, and (2) federal agencies promoted the standards  and procedures to reduce cyber risks to critical infrastructure.
To determine the extent to which NIST facilitated development of  voluntary standards and procedures to reduce cyber risks to critical  infrastructure, we reviewed the methodology and process used by NIST  to create the Framework for Improving Critical Infrastructure  Cybersecurity. We examined how NIST developed the framework,  including how industry comments were integrated into the document. In  addition, we surveyed participants in the framework’s development. Our  population for this survey included individuals who (1) provided written  comments with contact information in response to a NIST request for  information notice or (2) registered for at least one of the workshops  hosted by NIST to develop the framework. There were 2,082 individuals  in the population that we targeted, and we sent the survey request to all  of them and received 252 completed surveys. We also interviewed  relevant NIST officials involved in the framework’s development and  federal officials from agencies with lead roles in critical infrastructure  protection efforts, referred to as sector-specific agencies (SSA),  representing all 16 critical infrastructure sectors established in federal  policy. Also, we evaluated the framework against the requirements from  Executive Order 13636 and the Cybersecurity Enhancement Act of 2014  to determine whether these criteria were implemented in the framework.
To determine the extent to which federal agencies, including the SSAs  and NIST, promoted the standards and procedures to reduce cyber risks,  we analyzed documents and the website of the DHS Critical Infrastructure  Cyber Community (C) Voluntary Program to assess the framework  promotional guidance and tools provided to the critical infrastructure  sectors. We also collected and analyzed relevant documentation from the  SSAs about efforts to promote the framework. In addition, we analyzed  the metrics and information being used by the DHS CWe conducted this performance audit from February 2015 to December  2015 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives. Appendix I discusses our  objectives, scope, and methodology in greater detail.
subsections: section_title: Background
paragraphs: U.S. critical infrastructure is made of systems and assets, whether  physical or virtual, so vital to the United States that the incapacity or  destruction of such systems and assets would have a debilitating impact  on national security, economic security, national public health or safety, or  any combination of these matters. Critical infrastructure includes, among  other things, banking and financing institutions, telecommunications  networks, and energy production and transmission facilities, most of  which are owned and operated by the private sector. Sector-specific  agencies (SSA) are federal departments or agencies with responsibility  for providing institutional knowledge and specialized expertise as well as  leading, facilitating, or supporting the security and resilience programs  and associated activities of their designated critical infrastructure sector in  the all-hazards environment.
subsections: section_title: The Nation Faces an Evolving Array of Cyber- Based Threats
paragraphs: Threats to systems supporting critical infrastructure and federal  information systems are evolving and growing. Risks to cyber-based  assets can originate from unintentional and intentional threats.  Unintentional or non-adversarial threat sources include failures in  equipment, software coding errors, or resource depletion, such as  accidental actions of employees. They also include natural disasters and  failures of critical infrastructure on which the organization depends but are  outside of its control. Intentional threats include both targeted and  untargeted attacks from a variety of sources, including criminal groups,  hackers, disgruntled employees, foreign nations engaged in espionage  and information warfare, and terrorists. These threat adversaries vary in  terms of the capabilities of the actors, their willingness to act, and their  motives, which can include seeking monetary gain or pursuing an  economic, political, or military advantage. Table 1 describes the sources  of cyber-based threats in more detail.
Cyber threat adversaries make use of various techniques, tactics, and  practices—or exploits—to adversely affect an organization’s computers,  software, or networks, or to intercept or steal valuable or sensitive  information. These exploits are carried out through various conduits,  including websites, e-mail, wireless and cellular communications, Internet  protocols, portable media, and social media. Further, adversaries can  leverage common computer software programs, such as Adobe Acrobat  and Microsoft Office, to deliver a threat by embedding exploits within  software files that can be activated when a user opens a file within its  corresponding program. Table 2 provides descriptions of common  exploits or techniques, tactics, and practices used by cyber adversaries.
section_title: Federal Policy and Law Address the Protection of Cyber Critical Infrastructure
paragraphs: Because the private sector owns the majority of the nation’s critical  infrastructure—such as banking and financial institutions, commercial  facilities, and energy production and transmission facilities—it is vital that  the public and private sectors work together to protect these assets and  systems. Toward this end, federal law and policy assign roles and  responsibilities for agencies to assist the private sector in protecting  critical infrastructure, including enhancing cybersecurity.
Presidential Policy Directive 21 (PPD-21) assigns roles and  responsibilities for the critical infrastructure sectors to the SSAs. The  directive identified 16 critical infrastructure sectors and designated  associated federal SSAs. Table 3 shows the 16 critical infrastructure  sectors and the SSA for each sector.
PPD-21 identified SSA roles and responsibilities to include collaborating  with critical infrastructure owners and operators; independent regulatory  agencies; and state, local, tribal, and territorial entities as appropriate, as  well as providing, supporting, or facilitating technical assistance and  consultations for their respective sectors to identify vulnerabilities and  help mitigate incidents, as appropriate.
Federal law and policy have also established roles and responsibilities for  federal agencies to work with industry to enhance the cybersecurity of the  nation’s critical infrastructures. These include Executive Order 13636, the  Cybersecurity Enhancement Act of 2014, and the National Infrastructure  Protection Plan (NIPP).
Executive Order 13636, Improving Critical Infrastructure Cybersecurity,  issued in February 2013, outlines an action plan for improving security for  critical cyber infrastructure. This includes, among other things,  requirements for NIST to develop a voluntary critical infrastructure  cybersecurity framework and performance measures. In developing the  cybersecurity framework, NIST is to engage in an open public review and  comment process. The order also directs SSAs, in consultation with DHS  and other interested agencies, to review the cybersecurity framework and  if necessary, develop implementation guidance or supplemental materials  to address sector-specific risks and operating environments.
The Cybersecurity Enhancement Act of 2014, enacted in December  2014, established requirements that are consistent with the order  regarding NIST’s development of a cybersecurity framework. NIST’s  responsibilities in developing the cybersecurity framework under this law  include, among other things, identifying an approach that is performance-based, and cost-effective.
In response to Executive Order 13636, NIST issued the Framework for  Critical Infrastructure Cybersecurity in February 2014, which is intended  to help organizations apply the principles and best practices of risk  management to improving the security and resilience of critical  infrastructure. The framework proposes a risk-based approach to  managing cybersecurity risk and is composed of three parts: the  framework core, the framework profile, and the framework implementation  tiers.
The framework core is a set of cybersecurity activities, outcomes, and  informative references that are common across critical infrastructure  sectors, which is to provide guidance for developing individual  organization profiles. Through the use of profiles, the framework is  intended to help organizations align their cybersecurity activities with  business requirements, risk tolerances, and resources. The tiers provide  a mechanism for organizations to view and understand the characteristics  of their approach to managing cybersecurity risk. (Further information  on the framework core is provided in app. II.)
The NIPP defines the overarching approach for integrating the nation’s  critical infrastructure protection and resilience activities into a single  national effort. DHS developed the NIPP in collaboration with public- and  private-sector owners and operators and federal and nonfederal  government representatives, including SSAs, from the critical  infrastructure community. It details DHS’s roles and responsibilities in  protecting the nation’s critical infrastructures and how sector stakeholders  should use risk management principles to prioritize protection activities  within and across sectors. It emphasizes the importance of collaboration,  partnering, and voluntary information sharing among DHS and industry  owners and operators, and state, local, and tribal governments.
According to the NIPP, SSAs are to work with their private-sector  counterparts to understand cyber risk and develop sector-specific plans  that address the security of the sector’s cyber and other assets and  functions. The SSAs and their private-sector partners are to update their  sector-specific plans based on DHS sector-specific plan guidance issued  in 2014.
section_title: NIST Developed the Cybersecurity Framework in a Facilitated Manner to Fulfill Responsibilities
paragraphs: NIST used several methods to obtain and incorporate stakeholder views  when developing its cybersecurity framework. Respondents to our survey  were generally satisfied with NIST’s efforts and methods to develop the  framework. Also, NIST met the requirements under Executive Order  13636 and the Cybersecurity Enhancement Act to develop an approach  that can help critical infrastructure organizations manage cyber risk.
subsections: section_title: NIST Obtained and Incorporated Stakeholder Views in Developing the Framework
paragraphs: Executive Order 13636, released in February 2013 required NIST to  create a flexible performance-based cybersecurity framework that  includes a set of standards, procedures, and processes that align policy,  business, and technological approaches to address cyber risks. In  addition, under the Cybersecurity Enhancement Act, enacted into law in  December 2014, NIST is required to facilitate and support the  development of a voluntary set of standards, guidelines, methodologies,  and procedures to cost-effectively reduce cyber risks to critical  infrastructure.
In February 2014, NIST issued the Framework for Critical Infrastructure  Cybersecurity. In developing the framework, NIST used several methods  to obtain and incorporate the views of experts from government, industry,  and academia. Specifically, it solicited public comments through formal  “requests for information,” held workshops with stakeholders to identify  and develop elements of the framework, and published a draft version of  the framework for further review and comment. Figure 1 summarizes the  development of the framework.
NIST began the collaboration process for developing the framework on  February 26, 2013, when it issued a formal request for information in the  Federal Register to seek comments regarding risk management  practices, frameworks, standards, guidelines, and best practices. NIST  received 246 unique comments in response to the request from  organizations and individuals representing, among others, large  companies, associations, sector coordinating councils, federal agencies,  universities, and international companies. NIST analyzed the comments  received to, among other things, identify common cybersecurity practices  and methods to facilitate discussions on the development of the  framework.
After the initial request for information, NIST hosted six workshops at  various locations across the country to drive the development of the  framework. Workshop participants and attendees included critical  infrastructure owners and operators, industry associations, individual  companies, and government agencies. Table 4 summarizes the NIST  workshops.
At each workshop, NIST facilitated discussions with attendees and  accepted industry input to collaboratively identify and develop standards  and guidelines for the framework. Based on the input from the first four  workshops, NIST prepared and published a preliminary draft of the  framework for public review and comment following its fourth workshop.
NIST analyzed responses to its request for information, conclusions from  workshops, and stakeholder analysis to select components for the  framework, such as identifying security common practices, principles, and  approaches that supported the objectives of Executive Order 13636. To  identify the framework components that reflected the comments received,  NIST used input from volunteer industry stakeholders. These  stakeholders helped to, among other things, evaluate common themes  identified by NIST as well as cybersecurity areas that needed additional  exploration to encourage industry engagement in the framework  development process. After the initial draft was issued for comment, NIST  held the fifth workshop to obtain further stakeholder input. Subsequently,  NIST published its final draft of the framework in February 2014.
Following issuance of the framework, on August 26, 2014, NIST issued a  second request for information to seek comments on users’ experience  with the framework, as part of its efforts to promote use of the framework,  and held an additional workshop to obtain information on how  organizations learned about and used the framework. NIST received 57  unique comments in response to the second request for information from  organizations and individuals that represented, among others, large  companies, associations, and sector coordinating councils.
In addition to the framework, NIST developed a roadmap to discuss future  plans for the framework, which included identifying areas of improvement  of the preliminary framework. To promote the use and adoption of the  framework, NIST officials stated they plan to update the framework based  on industry feedback and develop guidance on how organizations can  use the framework to reduce cybersecurity risks.
section_title: Participants Who Responded to Our Survey Were Generally Satisfied with NIST’s Development Efforts
paragraphs: One of NIST’s responsibilities under the Cybersecurity Enhancement Act  was to incorporate industry input in the development of the standards and  methodologies to manage cybersecurity risks for critical infrastructure. A  majority of the 252 respondents to our survey indicated satisfaction with  the mechanisms employed by NIST to develop the framework. For  example, 186 of 251 respondents indicated that they were “very  satisfied” or “satisfied” that NIST provided opportunities for them to  provide feedback on the framework during its development process.  Similarly, a majority (170 of 187) indicated the workshops hosted by NIST  were “very” or “somewhat effective” in engaging industry involvement in  development of the framework. Appendix III provides additional details  on survey respondents’ evaluations of NIST’s collaborative approach to  developing the framework.
section_title: NIST’s Cybersecurity Framework Meets Requirements
paragraphs: NIST implemented the requirements for development of a cybersecurity  approach as required by Executive Order 13636 and the Cybersecurity  Enhancement Act of 2014.
Executive Order 13636 required NIST to develop among other things a  flexible, repeatable, performance-based, and cost-effective approach to  help owners and operators of critical infrastructure identify, assess, and  manage cyber risk. In addition, NIST was to develop a set of standards  that align policy, business, and technological approaches to address  cyber risks.
Similar to Executive Order 13636, the Cybersecurity Enhancement Act of  2014 authorizes NIST to, among other things, facilitate and support the  development of a voluntary set of standards, guidelines, methodologies,  and procedures to cost-effectively reduce cyber risks to critical  infrastructure. In carrying out these activities, NIST is required to identify,  among other things, a flexible, repeatable, performance-based, and cost- effective approach that can be voluntarily adopted to help identify,  assess, and manage cyber risks.
To ensure the framework could assist owners and operators with their  cyber risk as called for by the executive order and the act, NIST created  implementation tiers to allow organizations to determine their  cybersecurity risks and identify processes that align to their business  approaches to manage those risks. According to the framework,  implementation tiers describe how cybersecurity risk is managed by an  organization and the degree to which its risk management practices  exhibit key characteristics defined in the framework. There are four tiers  discussed in the framework, with each one building upon the previous  tier: Partial (Tier 1), Risk Informed (Tier 2), Repeatable (Tier 3), and  Adaptive (Tier 4).
Further, based on our analysis, the framework supported and developed  by NIST is intended to be:
Flexible: The NIST framework can be modified to an organization’s  cybersecurity condition and needs. Its processes also provide for a  flexible and risk-based implementation of the framework and can be  used with a broad array of cybersecurity risk management processes.
Repeatable: Under the framework, organizations are able to create a  framework profile to compare their current cybersecurity state to their  targeted state. This allows for identification and prioritization of  improvement opportunities within the context of a continuous and  repeatable process that can be assessed as the organization moves  toward its targeted state.
Performance-based: Organizations can use the framework  implementation tiers to measure progress for managing cybersecurity  risk. Specifically, the implementation tiers allow organizations to track  their performance in managing cyber risks from an informal, reactive  response to an approach that is agile and risk-informed. For example,  in order to track performance, organizations are encouraged to  identify their desired tier according to organizational goals and to  progress to higher tiers to continue reducing cybersecurity risks.
Cost-effective: The framework’s implementation tiers can be used to  allow organizations to evaluate their current cyber activities to  determine if adoption of cybersecurity risk management practices was  sufficient given their mission and regulatory requirements.  Additionally, these tiers allow organizations to determine activities that  are most important to their critical services, allowing them to prioritize  expenditures to maximize the impact of their investment in  cybersecurity.
Further, in accordance with the executive order and the act, the  framework is voluntary. The framework emphasizes that the information it  contains is guidance for individual organizations to use to improve their  cybersecurity posture. According to NIST, the implementation tier  approach allows for organizations to determine activities that are most  important to critical service delivery as well as prioritize expenditures to  maximize the impact of their cybersecurity investment.
section_title: Federal Entities Are Promoting the Cybersecurity Framework, but DHS Is Not Measuring the Effectiveness of Its Efforts
paragraphs: DHS, SSAs, and NIST have promoted adoption and use of the  cybersecurity framework by critical infrastructure owners and operators  through a variety of methods. Specifically, DHS established a program, in  accordance with Executive Order 13636, to encourage adoption of the  framework and has taken a number of actions, including disseminating  guidance and working with stakeholders to promote it. However, DHS is  not measuring the effectiveness of these actions in order to evaluate the  results of its activities and programs. For their part, SSAs for most sectors  are developing tailored guidance for implementing the framework in their  sectors, and NIST has promoted the framework through public events  and its website.
subsections: section_title: DHS’s Voluntary Program Promotes the Framework but Is Not Measuring the Effectiveness of Efforts
paragraphs: DHS, as required by Executive Order 13636, established the Critical  Infrastructure Cyber Community (C) Voluntary Program in February 2014  to support the voluntary adoption of the framework. The program is  intended to enhance critical infrastructure cybersecurity and encourage  the adoption of the NIST framework. According to DHS C Voluntary Program has  framework promotion actions broken out into three phases. Phase 1 is  focused on outreach and building awareness of the framework. Phase 2,  which was initiated in 2015, involves entity capability building, where DHS  promotes the framework to specific types of entities, such as academia,  business, and state governments, and highlighting resources to assist in  implementing the framework from DHS, other federal agencies, and the  private sector. Phase 3 is to facilitate the creation of communities of  interest around critical infrastructure cybersecurity.
To promote awareness of the framework during phase 1, DHS launched a  website with guidance and links to resources to assist critical  infrastructure organizations interested in implementing the framework.  DHS also developed a webinar series in January 2015 to provide  organizations additional resources for addressing and improving  cybersecurity risk management practices. Several past webinars are  available on demand on the website. DHS Cstated that they specifically targeted promotional efforts to entities  identified as having nationally critical assets. According to officials, as of  October 1, 2015, the C Voluntary Program had held 256 briefings since  November 2013.
For phase 2, which began in 2015, DHS C these included resources for  academia; business (large, midsize, and small); federal government; and  state, local, tribal and territorial governments. For example, to assist in  identifying cyber resilience and practices, DHS points to its cyber  resiliency review. According to officials, as of October 1, 2015, the  resiliency review had been downloaded from the site over 6,500 times.  Further, to assist with cybersecurity incident detection, DHS identified the  Cyber Information Sharing and Collaboration Program, which is intended  to share incident information. The C Voluntary Program also developed  a Small and Midsize Business Toolkit, which contains a number of  resources to help entities recognize and address cybersecurity risks,  including information for startups and a guide to free tools intended to  provide cybersecurity assistance. The toolkit, which was posted on the  program’s website in May 2015, had received over 2,000 downloads as of  October 1, 2015, according to officials.
Among the respondents to our survey who indicated that the Cresponses out of 112 responding to the question indicated that the C  Voluntary Program promotional activities were “very” or “somewhat”  effective in encouraging the use of the framework.
DHS C Voluntary Program was assisting  SSAs and 10 sectors in developing framework implementation guidance.  According to DHS, sector-specific implementation guidance will provide  each sector with a tailored approach on how to best use the framework  within their industry and make it easier for organizations to (1) use the  framework to secure themselves against cyber risk, (2) learn about  available tools and resources and approaches that can support  cybersecurity risk management and the framework, and (3) gain an  understanding of how different sectors and industries are approaching  cybersecurity risk management broadly.
DHS C  According to DHS officials, Phase 3, which is to facilitate the creation of  communities of interest around DHS cybersecurity initiatives, began in  2015 and will continue into 2016. DHS officials stated that the intended  outcome of this phase is to facilitate the development of self-sustaining  communities by continuing to provide forums for information sharing  among critical infrastructure owners and operators across the nation.  DHS officials stated that phase 3 will be accomplished through a series of  regional events, webinars, and development of new resources that  promote information sharing and community building.
subsections: section_title: DHS Is Not Measuring the Effectiveness of Its Framework Promotion Efforts
paragraphs: Performance measurement involves identifying performance goals and  measures, establishing performance baselines by tracking performance  over time, identifying targets for improving performance, and measuring  progress against those targets. As we have previously reported,  according to leading practices in the federal government and in industry,  organizations should measure performance in order to evaluate the  success or failure of their activities and programs. In addition, the  Standards for Internal Control in the Federal Government, known as the  “Green Book,” sets internal control standards for federal entities. Those  standards state that internal control monitoring should occur and that the  quality of performance over time should be assessed.
DHS C Voluntary Program documentation identifies three metrics that  align with program goals: (1) making resources accessible, (2) increasing  participation by entities with the CHowever, none of these metrics indicate the effectiveness of the  program’s efforts to promote adoption of the framework, and program  officials are not otherwise measuring or tracking how effective those  efforts or materials are in encouraging individuals and organizations to  voluntarily adopt the framework. For example, they are not tracking what  percentage of an individual sector they have promoted to or how effective  their efforts and guidance are at encouraging the use of the framework by  entities within a critical infrastructure sector.
According to DHS C Voluntary Program officials, they have not  established these metrics or monitoring mechanisms because DHS has  focused on getting as much information and resources out as possible.  However, without understanding whether its promotional efforts are  effective, the C Voluntary Program and NIST personnel or resources,  speaking at industry conferences, and providing information during sector  coordinating council meetings. For example, within the water sector, the  Environmental Protection Agency (EPA) and the sector coordinating  council are working to understand how entities within the sector are  managing risk, including using the framework. Specifically, EPA is  supporting the council by providing contractor and subject matter expert  support as the council develops metrics and administers a survey to the  sector through 2016. EPA officials stated that once the sector  coordinating council receives and analyzes the information received from  the survey, it will provide the information to EPA. Another example is the  transportation systems sector, where DHS, as the co-SSA, held several  promotional activities for Cybersecurity Month in October 2015.
Respondents to our survey who indicated that their SSA promoted use of  the framework stated that they were usually encouraged to use the  framework as a result. Specifically, 60 of the 82 respondents who  responded to that question indicated that SSA promotional activities were  “very” or “somewhat” effective in encouraging the use of the framework.
In addition, most SSAs have determined whether to develop framework  implementation guidance to address sector-specific risks and operating  environments for their sector. Executive Order 13636 requires SSAs to  determine whether or not it is necessary for their sector to develop sector- specific framework implementation guidance, and SSAs for 15 of the 16  sectors had made this determination as of October 2015. Of the 15  sectors, 5 have completed and released framework implementation  guidance. These framework implementation guides were created by  SSAs, sector stakeholders, or a combination of both. For example, the  energy sector was the first sector to produce implementation guidance,  and the SSA worked with sector stakeholders to produce it. The health  care and public health sector had a sector stakeholder provide a  crosswalk between their risk framework and the NIST cybersecurity  framework. HHS officials representing the health care and public health  SSA stated that they are beginning an effort to create further  implementation guidance for the sector’s seven subsectors.
Seven other sectors have begun drafting implementation guidance.  According to officials, these sectors were finalizing their guidance and  had involved their sector stakeholders to review it.
For the food and agriculture sector, Department of Agriculture and Food  and Drug Administration officials representing the SSAs for the sector  stated that they had yet to begin drafting framework implementation  guidance and were exploring the creation of sector guidance with their  stakeholders.
The Departments of Defense (DOD) and the Treasury, as SSAs for the  defense industrial base and financial services sectors, respectively,  decided not to create sector-specific framework implementation guidance.  According to DOD officials representing the defense industrial base SSA,  the department determined, after discussions with the defense industrial  base sector coordinating council, that implementation guidance is not  needed. The DOD officials stated that there was a consensus within the  sector coordinating council that sector companies would voluntarily adopt  the framework as they determined to be necessary. Treasury officials  representing the SSA for the financial services sector stated that they  determined, with input from the sector, that implementation guidance  should not be created by Treasury as the financial services SSA.  According to the Treasury officials this decision was made in cooperation  with the financial services sector coordinating council, and they concluded  that implementation guidance was not necessary due to the regulatory  structure of the sector.
DHS and the General Services Administration (GSA), which are the co- SSAs for the government facilities sector, have yet to determine if sector  implementation guidance should be developed. A GSA official  representing the SSA for the government facilities sector stated that there  are metrics in the sector-specific plan that will allow GSA to gather  information from the sector on its promotional and implementation needs  and use that information to best meet the needs of the sector. According  to DHS and GSA officials they are waiting until this information is  gathered and assessed before discussing whether sector-specific  implementation guidance will be needed. DHS and GSA officials do not  have a specific time frame for when the information will be gathered or for  when they will make a decision. Without a decision by DHS and GSA  under Executive Order 13636 whether guidance is needed to address  sector-specific risks and operating environments, implementation of the  framework in the government facilities sector may be hindered.
section_title: NIST Continues to Promote the Framework
paragraphs: Although not specifically required to by Executive Order 13636 or the  Cybersecurity Enhancement Act of 2014, NIST has continued its efforts to  promote the framework. NIST encouraged the use of the framework at  workshops and public events it hosted and updated its website to provide  information on upcoming events and resources related to the framework.  Specifically, NIST’s website for the framework  (www.nist.gov/cyberframework) provides a list of publically available  resources to assist entities interested in using the framework. The  website includes guidance and tools created by federal and private sector  entities for implementing the framework. NIST encourages entities to  submit publically available framework implementation guidance that they  create so that NIST can provide information about it with the other  guidance and tools already highlighted. The website also lists upcoming  events where NIST officials will provide framework information and  perspectives. Past speaking events from across the country are also  listed with links to the event webpage and in some cases the NIST  presentation slides.
Respondents to our survey who indicated they had been promoted to by  NIST noted that they were encouraged to use the framework as a result.  Specifically, 102 responses out of 132 indicated that NIST promotional  activities were “very” or “somewhat” effective in encouraging the use of  the framework.
Additionally, NIST officials stated they are following implementation of the  framework, and a section on the NIST website is dedicated to accounts  from entities of how they have implemented the framework. As of October  2015, the section listed an Intel use case for framework implementation  and a link to a podcast on how the University of Pittsburgh is using the  framework.
NIST officials added that they are seeking feedback on the framework.  Specifically, officials stated that they are researching how organizations  are applying the framework methodologies, what value they are obtaining  from the framework, and what challenges they are facing in implementing  the framework. NIST officials stated they are currently focused on the  overall effectiveness of the framework for the benefit of those using it to  improve future versions. They further stated that they continue to ask  stakeholders when an appropriate time to update the framework would  be. To solicit this feedback, officials stated that they are asking individuals  attending NIST presentations and may issue a request for information  through the Federal Register regarding whether or not the critical  infrastructure community believes that it is necessary to begin updating  the framework.
section_title: Conclusions
paragraphs: NIST has generally fulfilled its requirements, established in Executive  Order 13636 and the Cybersecurity Enhancement Act, to develop a  cybersecurity framework for adoption by critical infrastructure sectors. By  using a collaborative process for developing the framework, NIST has  helped ensure that the resulting guidance, standards, and methodologies,  if effectively implemented, can help cost-effectively reduce cyber risks to  critical infrastructure.
To facilitate the voluntary adoption of the framework by critical  infrastructure owners and operators, DHS, sector-specific agencies, and  NIST are taking a variety of actions. However, while DHS has established  a program dedicated to encouraging the framework’s adoption, without  establishing metrics to assess the effectiveness of these efforts, it has  less assurance that it is meeting its objectives. In addition, while most  SSAs have determined the need for sector-specific guidance to  implement the framework, DHS and GSA have yet to meet this  requirement for the government facilities sector, which may hinder  adoption of the framework in this sector.
section_title: Recommendations for Executive Action
paragraphs: To better facilitate adoption of the NIST Framework for Improving Critical  Infrastructure Cybersecurity, we are making the following  recommendations:  We recommend that the Secretary of Homeland Security direct officials  responsible for the Critical Infrastructure Cyber Community Voluntary  Program to develop metrics for measuring the effectiveness of efforts to  promote and support the framework.
We also recommend that the Secretary of Homeland Security and the  Administrator of GSA set a time frame for determining the need for  sector-specific guidance to implement the framework in the government  facilities sector.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to the Departments of Agriculture,  Commerce, Defense, Energy, Health and Human Services, Homeland  Security, Transportation, and the Treasury and to GSA and EPA.
In written comments signed by the Director, Departmental GAO-OIG  Liaison Office (reprinted in app. IV), DHS concurred with our two  recommendations. DHS agreed with the need to further refine and mature  metrics for measuring the effectiveness of efforts to promote and support  the framework. DHS also provided details about efforts to track output  metrics addressing the CIn addition, officials from the Departments of Commerce, Health and  Human Services, and Homeland Security provided technical comments  via e-mail that have been addressed in this report as appropriate. The  Departments of Agriculture, Defense, Energy, Transportation, and the  Treasury and EPA responded via e-email that they had no comment on  the report.
We are sending copies of this report to the appropriate congressional  committees; the Secretaries of Agriculture, Commerce, Defense, Energy,  Health and Human Services, Homeland Security, Transportation, and the  Treasury; the Administrators of the Environmental Protection Agency and  General Services Administration; and other interested parties. In addition,  the report is available at no charge on the GAO website at  http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-6244 or wilshuseng@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in appendix VI.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: The objectives of our review were to determine the extent to which (1) the  National Institute of Standards and Technology (NIST) facilitated the  development of voluntary standards and procedures to reduce cyber risks  to critical infrastructure, and (2) federal agencies promote the standards  and procedures to reduce cyber risks to critical infrastructure.
To determine how NIST facilitated the development of voluntary  standards and procedures for critical infrastructure, we reviewed and  analyzed the actions taken by NIST to develop its Framework for  Improving Critical Infrastructure Cybersecurity. In addition, we analyzed  Executive Order 13636, issued in February 2013, and the Cybersecurity  Enhancement Act of 2014, enacted in December 2014, to identify key  NIST responsibilities for developing a cybersecurity framework. We  analyzed documents and performed interviews with NIST officials to  assess its collaborative efforts with industry stakeholders in soliciting  input in the development of the framework, including workshops it hosted  and the website it set up to disseminate updates on the framework.  Specifically, we reviewed documentation and videos of the six workshops  hosted by NIST intended to obtain industry, academia, and government  representative feedback in the development of the framework, in addition  to NIST’s two requests for information to the public for input on  cybersecurity standards and methodologies. We also analyzed the  resulting framework to assess whether NIST had fulfilled its  responsibilities under law.
Additionally, to address this objective, we conducted a web-based survey  of individuals who (1) provided written comments with contact information  in response to a NIST request for information notice or (2) registered for  at least one of the workshops hosted by NIST to develop the framework.  There were 2,082 individuals in the population that we targeted, and to  make the survey as inclusive as possible we sent the survey request to all  of them. The questionnaire included questions about the effectiveness of  NIST’s collaborative efforts in fulfilling requirements to develop the  framework using an open and public comment process. To minimize  errors arising from differences in how questions might be interpreted and  to reduce variability in responses that should be qualitatively the same,  we conducted pretests with critical infrastructure representatives over the  telephone. Based on feedback from these pretests, we revised the  questionnaire to improve the clarity of the questions. An independent  survey specialist within GAO also reviewed a draft of the questionnaire  prior to its administration.
After completing the pretests, we administered the survey to the NIST  workshop attendees and request for information respondents on August  10, 2015, notifying them that our online questionnaire would be activated  within a couple of days. On August 18, 2015, we sent a second e-mail  message to these individuals, informing them that the questionnaire was  available online and providing them with unique passwords and  usernames. We collected responses through August 24, 2015. We were  able to obtain 252 completed questionnaires, a 12 percent response rate,  in the time available for survey fieldwork. Because we do not know if the  answers that nonrespondents would have given would materially differ  from those that did respond, our results can only represent the views of  those who did respond. Their views are not generalizable to the registrant  and respondent population as a whole.
To address our second objective, we reviewed and analyzed actions and  documentation related to promoting the framework by the nine sector- specific agencies (SSAs) responsible for the 16 critical infrastructure  sectors established in Presidential Policy Directive-21, including the  Department of Homeland Security (DHS), and NIST. For DHS, we  analyzed agency documentation and the website of its Critical  Infrastructure Cyber Community (C) Voluntary Program to identify the  framework promotional guidance and tools provided to the critical  infrastructure sectors. Also, we analyzed the metrics and information  being used by the DHS CTo analyze NIST’s promotional efforts, we analyzed documentation and  interviewed relevant NIST officials. We reviewed the NIST framework  website to understand how NIST was informing the public about its public  events to promote the framework and presenting entities with guidance to  implement the framework for other agencies, sectors, and third parties.
In addition, we surveyed individuals who responded to NISTs requests for  information and registered for one of the workshops hosted by NIST for  the development of the framework to identify and evaluate the  effectiveness of DHS, SSA, and NIST efforts to promote the framework.  We also interviewed federal officials from DHS’s  C Voluntary Program,  the nine SSAs representing the 16 critical infrastructure sectors, and  NIST regarding their efforts to promote the framework and create sector- specific framework implementation guidance.
We conducted this performance audit from February 2015 to December  2015 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
section_title: Appendix II: The Core of NIST’s Cybersecurity Framework
paragraphs: The National Institute of Standards and Technology (NIST) Framework  for Improving Critical Infrastructure Cybersecurity is intended to help  organizations apply the principles and best practices of risk management  to improving the security and resilience of critical infrastructure. The  framework proposes a risk-based approach to managing cybersecurity  risk and is composed of three parts: the framework core, the framework  profile, and the framework implementation tiers.
The framework core includes a listing of functions, categories,  subcategories, and informative references that describe specific  cybersecurity activities the framework identified as common across all  critical infrastructure sectors. According to NIST, the framework core  represents a common set of activities for managing cybersecurity risk.  The framework also states that while it is not exhaustive, it is extensible,  allowing organizations, sectors, and other entities to use subcategories  and informative references that are cost-effective and efficient and that  enable them to manage their cybersecurity risk.
Table 6 includes the five functions and 22 categories of the framework  core, and Table 7 includes information for one of the categories, asset  management, as described in the NIST framework and appendix A of the  framework. The information presented here represents how each function  has categories, subcategories, and informative references. For more  information on the framework, framework core, and categories, see the  NIST framework website at www.nist.gov/cyberframework.
Table 7 provides an example of the subcategories and related informative  references for a single category of the identify function: asset  management.
section_title: Appendix III: Survey Questions and Responses on Development of the Cybersecurity Framework
paragraphs: The following table presents selected questions and responses from our  survey of registrants at the National Institute of Standards and  Technology’s (NIST) workshops for developing the NIST Framework for  Improving Critical Infrastructure Cybersecurity and commenters who  responded to NIST’s request for information. There were 2,082  individuals in the population that we targeted, and to make the survey as  inclusive as possible we sent the survey request to all of them. We  obtained 252 completed questionnaires, a 12 percent response rate, in  the time available for survey fieldwork. Because we do not know if the  answers that nonrespondents would have given would materially differ  from those that did respond, our results represent the views of those who  did respond. They are not generalizable to the registrant and respondent  population as a whole. Not all questions were applicable to or otherwise  answered by the respondents to our survey. Our results represent those  answering each specific question.
section_title: Appendix IV: Comments from the Department of Homeland Security
section_title: Appendix V: Comments from the General Services Administration
section_title: Appendix VI: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Michael W. Gilmore, Assistant  Director; Sher’rie Bacon; Wilfred Holloway; Lee McCracken; David  Plocher; Carl Ramirez; and Jeffrey Woodward made key contributions to  this report.