Critical Infrastructure Protection: DHS List of Priority Assets Needs to Be Validated and Reported to Congress
Released: 2013-03-25T00:00:00

Letter

Background

Pursuant to the Homeland Security Act of 2002, as amended, DHS has responsibility for the protection of the nation’s critical infrastructure. Within DHS, the Office of Infrastructure Protection is responsible for critical infrastructure protection and resilience and leads the coordinated national effort to mitigate risk to the nation’s critical infrastructure, which includes working with public and private sector infrastructure partners.The Office of Infrastructure Protection also has the overall responsibility for coordinating implementation of the NIPP across the 18 critical infrastructure sectors; overseeing the development of Sector-Specific Plans; providing training and planning guidance to SSAs and asset owners and operators on protective measures to assist in enhancing the security of infrastructure within their control; and helping state, local, tribal, territorial, and private sector partners develop the capabilities to mitigate vulnerabilities and identifiable risks to their assets.

Within the Office of Infrastructure Protection, IASD manages the NCIPP. According to DHS, the main goals of the NCIPP are to (1) identify the infrastructure that if disrupted or destroyed could significantly affect the nation’s public health and safety, economic, or national security; (2) increase the accuracy of infrastructure prioritization efforts used to inform DHS resource allocation decisions; and (3) focus planning, foster coordination, and support preparedness efforts for incident management, response, and restoration activities among federal, state, and private sector partners. Critical infrastructure identified through the program includes several thousand level 1 or level 2 assets and systems. The levels are used to enhance decision making related to infrastructure protection and can include a range of businesses or assets in a local geographic area, such as refineries, water treatment plants, or commercial facilities, as well as the information and data systems that ensure their continued operation.

Consistent with the generally voluntary critical infrastructure protection approach identified in the NIPP, according to DHS, the success of the NCIPP relies upon the voluntary contributions and cooperation of public and private sector partners from the infrastructure protection community. To compile the NCIPP list, consistent with statutory requirements, IASD conducts a voluntary annual data call to solicit nominations to the list from state homeland security and federal partners. To submit nominations, partners are to develop realistic scenarios for infrastructure that meet specific criteria developed by IASD. Consistent with the consequence categories identified in the NIPP risk management framework, NCIPP nominations are to meet minimum specified consequence thresholds outlined in the annual data call for at least two of the following four categories: fatalities, economic loss, mass evacuation length, and degradation of national security. After nominations are submitted, according to DHS guidance, IASD conducts a multiphase adjudication process intended to give state and federal partners the opportunity to review IASD’s preliminary decisions and submit additional information to support nominations that were not initially accepted, before IASD finalizes the NCIPP list.

Uses of the NCIPP List

The NCIPP list is used to establish risk management priorities. According to the NIPP, prioritizing risk management efforts provides the basis for understanding potential risk mitigation benefits that are used to inform The NCIPP list, which identifies planning and resource decisions.nationally significant critical infrastructure based on consequences, informs the NIPP risk management prioritization process. The NIPP risk management prioritization process involves analyzing risk assessment results to determine which critical infrastructure faces the highest risk so that management priorities can be established. The NCIPP list is also used to, among other things:

Allocate Homeland Security Grants. Within DHS, FEMA uses the number of assets included on the NCIPP list, among other data, in its risk formula for allocating State Homeland Security Program (SHSP) and UASI grant funds. The SHSP and UASI provide funding to states and cities, respectively, to support a range of preparedness activities to prevent, protect against, respond to, and recover from acts of terrorism and other catastrophic events. While the number of critical infrastructure a state or city has on the NCIPP list is used to determine the allocation of SHSP and UASI grant funds, there is no requirement that states or cities use these grant funds to enhance protection of these assets. For fiscal year 2012, FEMA allocated $294 million in SHSP funding to all 50 states, the District of Columbia, Puerto Rico, American Samoa, Guam, the Northern Mariana Islands, and the U.S. Virgin Islands. Additionally, in fiscal year 2012, FEMA allocated approximately $490 million in UASI funding to the nation’s 31 highest-risk cities.

Prioritize Voluntary Critical Infrastructure Protection Programs. The Office of Infrastructure Protection’s Protective Security Coordination Division (PSCD) uses the NCIPP list and other inputs to prioritize its efforts to work with critical infrastructure owners and operators and state and local responders to (1) assess vulnerabilities, interdependencies, capabilities, and incident consequences, and (2) develop, implement, and provide national coordination for protective programs. Related to these efforts, PSCD has deployed the aforementioned PSAs in 50 states and Puerto Rico to locations based on population density and major concentrations of critical infrastructure. PSAs use the NCIPP list to prioritize outreach to level 1 and level 2 assets in their area of jurisdiction for participation in DHS’s voluntary security survey and vulnerability assessment programs, such as the Enhanced Critical Infrastructure Protection and Site PSAs are also often called upon by state Assistance Visit programs.homeland security advisers to assist them in nominating assets to the NCIPP list.

Inform Incident Management Planning and Response Efforts. DHS uses information collected during the NCIPP process and the NCIPP list to inform and prioritize incident management planning and response efforts. When an incident occurs, DHS officials pull information from a variety of sources, including the database of assets nominated to and accepted on the NCIPP list, to identify critical infrastructure in the affected area. IASD then prioritizes this information in an infrastructure of concern list to guide incident response efforts. The infrastructure of concern list includes any critical infrastructure affected by the event, which may include level 1 or level 2 assets.DHS components, including FEMA and PSAs, who use it on the ground to guide local incident response efforts.

DHS Has Made Changes to NCIPP List Criteria, but Has Not Identified the Impact of These Changes or Validated Its Approach

DHS has made several changes to its criteria for including assets on the NCIPP list. These changes initially focused on introducing criteria to make the lists entirely consequence based, with subsequent changes intended to introduce specialized criteria for some sectors and assets. DHS’s changes to the NCIPP criteria have changed the composition of the NCIPP list, which has had an impact on users of the list. However, DHS does not have a process to identify the impact of these changes on users nor has it validated its approach for developing the list.

DHS Made Several Changes to the List Criteria and Format

DHS’s initial approach for developing the NCIPP list differed by asset level. According to the Homeland Security Act, as amended, DHS is required to establish and maintain a prioritized list of systems and assets that the Secretary determines would, if destroyed or disrupted, cause national or regional catastrophic effects. The criteria for level 1 assets focused on consequences—the effects of an adverse event. The criteria for level 2 assets focused generally on capacity—the number of people that use an asset or output generated by an asset, such as the number of people that occupy a commercial office building, the daily ridership of a mass transit system, or the number of people served by a water utility. DHS officials told us that the level 1 consequence-based criteria and thresholds were initially established at the beginning of the program at the discretion of the Assistant Secretary for Infrastructure Protection, who sought to identify infrastructure that the destruction of which could be expected to cause impacts similar to those caused by the attacks of In contrast, the initial level 2 September 11 and Hurricane Katrina.criteria were generally capacity based in order to identify the most critical assets within each of the 18 sectors. However, the capacity-based criteria often differed by sector, making it difficult to compare criticality across sectors and therefore identify the highest-priority critical infrastructure on a national level.

In 2009, DHS changed the level 2 criteria to make the NCIPP list entirely consequence based, a change that brought its approach more into line with statutory requirements and, consistent with the NIPP risk management framework, allowed for comparison across sectors. The new level 2 criteria match the level 1 consequence-based criteria— fatalities, economic loss, mass evacuation length, or national security impacts—but with lower threshold levels than those used to identify level To be included on the NCIPP list, an asset must meet at least 1 assets.two of the four consequence thresholds, and is included on the list as either level 1 or level 2 depending on which consequence thresholds it meets. As figure 1 shows, the level 1 thresholds are higher than level 2 thresholds and therefore represent the most nationally critical assets.

According to officials and agency documents, DHS changed the level 2 criteria to be consequence based for several reasons. First, NCIPP program officials stated that they changed the criteria to align the list with statutory requirements. Specifically, DHS interpreted the statute’s requirement that it identify assets that “would, if destroyed or disrupted, cause national or regional catastrophic effects,” as a call for consequence-based criteria.of assets prioritized using capacity-based criteria demonstrated that the initial level 2 criteria were not sufficient to fully identify assets capable of causing catastrophic events. Second, program officials stated that they changed the criteria to allow for comparisons across sectors, which is consistent with the NIPP. The NIPP states that using a common approach with consistent assumptions and metrics increases the ability to make comparisons across sectors, different geographic regions, or different types of events. Third, DHS also changed the criteria to improve the utility of the list. According to the NCIPP guidance, prior to 2009, assets designated as level 2 on the list experienced instability—assets being added and removed from year to year—which frustrated efforts to use the list for risk management planning and engagement, while assets designated as level 1 on the list—which had always been consequence based—remained relatively stable year to year.

Foot-and-mouth disease (FMD) is a highly contagious viral disease of cloven-hoofed animals such as cattle, swine, and sheep. Infected animals develop a fever and blisters on their tongue and lips, and between their hooves. Many animals recover from a FMD infection, but the disease leaves them debilitated and causes losses in meat and milk production. FMD does not have human health implications. According to the U.S. Department of Agriculture, a 2001 outbreak of FMD in the United Kingdom resulted in the slaughter of millions of animals and economic losses conservatively estimated at $14.7 billion. See GAO, Homeland Security: Actions Needed to Improve Response to Potential Terrorist Attacks and Natural Disasters Affecting Food and Agriculture, GAO-11-652 (Washington D.C.: Aug. 19, 2011).

DHS is currently reevaluating the agriculture and food sector-specific criteria because, according to officials, the specialized criteria created a great deal of inconsistency in the agriculture and food assets and systems included on the NCIPP list year to year.

In 2010, DHS also made adjustments to the NCIPP criteria to account for high-risk assets that may not always meet the consequence criteria by introducing the Catastrophic Economic Impacts Project and the Threats to Infrastructure Initiative. Under the Catastrophic Economic Impacts Project, infrastructure that meets only the level 1 consequence threshold for economic impact, but no other criteria, is added to the list as a level 2 asset. DHS officials explained that the project was added to account for instances when economic impact may be the primary impact. For example, the officials noted that a collapse of the U.S. financial system would likely not cause a large number of prompt fatalities or evacuations, but would cause catastrophic national impacts nonetheless. Meanwhile, the Threats to Infrastructure Initiative allows infrastructure that has received a specific, credible threat from a malicious actor, but otherwise would not meet NCIPP list criteria, to be added to the list as a level 2 asset. Unlike the other NCIPP criteria, the Threats to Infrastructure Initiative focuses on the threat to infrastructure rather than the consequences that may result from a specific event, which could complicate comparisons across assets and sectors. DHS officials told us that infrastructure with specific and credible threats were always included on the NCIPP list, but were historically added based on information from the intelligence community. The addition of the initiative allowed states to nominate critical infrastructure under the same scenario based on state and local intelligence information, such as that collected by fusion centers. According to DHS officials, they adjudicate Threats to Infrastructure Initiative nominations by determining whether the threat to an asset is specific and credible. As of fiscal year 2012, approximately 60 assets and systems have been added to the NCIPP list as a result of these new criteria.

In 2009, DHS also changed the format of the NCIPP list by expanding the type of infrastructure that could be nominated to the list to include clusters and systems of critical infrastructure in an effort to characterize the relationship among some infrastructure, such as dependencies and interdependencies, which was consistent with the statute and NIPP. According to the NCIPP guidance, clusters or systems of critical infrastructure are made up of two or more associated or interconnected assets or nodes that can be disrupted through a single event, resulting in regional or national consequences that meet the NCIPP criteria thresholds. An asset is a single facility with a fixed location that functions as a single entity (although it can contain multiple buildings or structures) and meets the NCIPP criteria by itself. A node is a single facility, similar to an asset, that does not meet the NCIPP criteria individually but does meet the criteria when grouped with other nodes or assets in a cluster or system. Figure 2 provides an illustration of an asset, a node, a cluster, and a system.

Because nodes do not meet the NCIPP criteria on their own, they are not included on the NCIPP list, but are identified on a separate list that is associated with the NCIPP list. For example, a group of nodes or assets making up a cluster would be listed on the NCIPP list under the name of the cluster, such as the ABC Cluster, but one would have to consult the associated list of nodes to identify the specific facilities that make up the listed cluster. The concept of clusters and systems is consistent with the statute and NIPP risk management framework. The law states that the prioritized list of critical infrastructure shall contain both systems and assets included in the national asset database, and the NIPP states that to the extent possible, risk assessments should assess the dependencies and interdependencies associated with each identified asset, system, or network. According to DHS, they recognized a need to identify clusters of critical infrastructure in 2008 after Hurricanes Gustav and Ike damaged a group of refineries that resulted in a nationally significant supply disruption of certain petrochemicals used across a wide range of industries.

DHS Has Not Identified the Impact of Changes in Criteria on List Users or Validated Its Approach for Developing the List

The changes DHS made to the NCIPP criteria in 2009 and 2010 changed the number of assets on and the composition of the NCIPP list. The total number of assets, clusters, and systems on the NCIPP list decreased from more than 3,000 in fiscal year 2009 to fewer than 2,000 in fiscal year 2011. The introduction of clusters and systems resulted in a separate list of thousands of nodes associated with the NCIPP list. Specifically, more than 2,500 additional facilities were included on the first nodes list in fiscal year 2011, and almost 4,000 facilities were included on the nodes list for fiscal year 2012. Figure 3 shows the relative changes in the number of assets, clusters, and systems on the NCIPP list and associated nodes list for fiscal years 2007 through 2012.

Additionally, the criteria changes also resulted in a change in the distribution of assets, clusters, or systems included on the NCIPP list by sector. Figure 4 shows that, among other sectors, the distribution of assets in the agriculture and food and defense industrial base sectors experienced large increases as a percentage distribution of the list from fiscal years 2009 to 2011, while for the same period, the energy and transportation sectors experienced large decreases. It also shows that the distribution of assets in the agriculture and food sector continued to increase as a percentage distribution of the list from fiscal years 2011 to 2012, while for the same period, the chemical sector experienced a large decrease.

Our analysis shows that changes to the NCIPP list can have an impact on users of the list, specifically, FEMA’s allocation of UASI grant funds and PSAs’ ability to prioritize outreach and conduct site visits for its protection programs. Our analysis of the FEMA risk formula shows that a change in the number of NCIPP-listed assets located in a city has an impact on a city’s relative risk score. Our analysis also shows that current UASI grant allocations are strongly associated with a city’s current relative risk Therefore, a change in the number of NCIPP-listed assets score.located in a city can have an impact on the level of grant funding it receives.approximately $490 million in UASI grant funds to the 31 cities with the highest relative risk scores out of 102 eligible cities nationwide. Our analysis of FEMA’s risk formula showed that, at the minimum, if the number of level 2 assets is increased or decreased by as few as two for each city, it would change the relative risk score for 5 of the 31 cities that received fiscal year 2012 UASI grant funding. Such a change could result in increased or decreased grant funding allocations for the affected cities. The changes in the relative risk scores tend to affect cities in the middle to the bottom of the top 31 list because there is generally a larger gap between the relative risk scores of those cities at the top of the list than those in the middle to bottom of the list. However, even a small change in grant funding could have an impact on a city, especially if that city does not traditionally receive other federal assistance as compared with cities with higher risk scores.

For example, in fiscal year 2012, FEMA allocated We previously reported that changes to the NCIPP list have presented challenges to managing DHS programs, particularly the voluntary security survey and assessment programs managed by PSCD. In May 2012, we reported that PSCD was unable to track the extent to which it conducted security surveys and vulnerability assessments on NCIPP level 1 and level 2 assets because of (1) inconsistencies between the databases used to identify the high-priority assets and to identify surveys and assessments completed, and (2) the change in the format and organization of the NCIPP list that converted some assets previously listed as level 1 or level 2 into a cluster or system.fiscal year 2012 NCIPP list, DHS has begun to assign unique numerical identifiers to each NCIPP asset, cluster, and system, which officials told us has helped DHS track how many security surveys and vulnerability assessments it conducts on high-priority assets. The officials also told us that they anticipate fewer challenges associated with the list since the Beginning with the number of assets, clusters, and systems on the NCIPP list has remained relatively stable from fiscal years 2011 to 2012. However, as discussed earlier, the number of nodes associated with the NCIPP list has increased substantially, growing from more than 2,500 in fiscal year 2011 to almost 4,000 in fiscal year 2012, which could further challenge PSA’s ability to conduct outreach and prioritize site visits to critical infrastructure for its protection programs.

PSCD officials in Washington, D.C. further told us that they do not have criteria establishing how PSAs should assess an NCIPP cluster or system that may contain many different nodes. The number of nodes in an NCIPP cluster or system can vary from two to several dozen and may be geographically dispersed. For example, one PSA told us that nodes in the same cluster may not have the same owner and could be part of a multistate system. Another PSA said that because several nodes in a system may not be the same (i.e., different types of facilities, different facility owners, or located in different areas), he generally conducts an assessment of each node in order to consider an assessment of a system complete. He explained that the facilities would have to be identical in order to conduct a single assessment for separate nodes, which he noted is rarely the case. Because it is difficult to prioritize which nodes within clusters or systems may be the most important for conducting assessments, the increase in the number of nodes associated with the NCIPP list could have the effect of complicating PSA efforts to conduct outreach to and assessments on the nation’s highest-priority infrastructure. PSCD officials told us they view this as a challenge, but they do not characterize it as a significant challenge. Further, they stated that while the treatment of nodes within NCIPP clusters or systems has not been specifically addressed in current program policies or guidance, they do not believe that this challenge has affected their ability to effectively prioritize facilities to receive security surveys and assessments. In January 2013, a PSCD official told us that PSCD is considering new guidance that would clarify how PSAs should approach nodes when conducting outreach or prioritizing visits for voluntary protection programs.

DHS does not have a process for identifying the impact of changes to the list on its users and has not reviewed the impact of these changes on users. However, program officials told us that they work closely with the primary users of the list to understand how the data are used. According to officials, they recognize that changes to the NCIPP list may have an impact on users of the list, but they consider these impacts to be minor. For example, one program official told us that the changes in the number of level 1 and level 2 assets rarely have a significant effect on the amount of grant funding allocated to states or cities, because of the additional inputs considered in the FEMA risk formula that determine the grant allocations. However, as previously demonstrated through our analysis, even small changes to the NCIPP list counts can have an impact on UASI grant allocations when accounting for all of the additional inputs considered in FEMA’s risk formula. The officials also recognized that changes from the fiscal year 2009 to fiscal year 2011 NCIPP lists, which significantly reduced the number of assets on the list, required PSCD to reset its performance metrics for conducting its voluntary security survey and assessment programs. However, officials told us that the assets on the NCIPP list have remained relatively stable since fiscal year 2011; therefore, the officials believe that changes to the list would have a minor impact on PSAs’ outreach activities. While our analysis shows that the number of assets on the NCIPP list remained fairly constant from fiscal year 2011 to 2012, it also shows that the number of nodes on the associated nodes list continued to grow and almost doubled during this time. As discussed, the increase in nodes may complicate PSA efforts to conduct outreach to and assessments on the nation’s highest-priority infrastructure. Additionally, the officials told us that, internally, changes to the NCIPP list do not have an impact on DHS’s ability to identify and prioritize critical infrastructure during an incident because the list is just one of many information sources they consult when developing an event- specific infrastructure of concern list to guide incident response efforts.

While the change to an entirely consequence-based list created a common approach to identify infrastructure and align the program with the statute and NIPP, recent and planned criteria changes to accommodate certain sectors and assets represent a departure from this common approach, which could hinder DHS’s ability to compare infrastructure across sectors. For example, the agriculture and food sector has criteria that are different from those of all other sectors. Furthermore, DHS has not validated its approach to developing the list to ensure that it accurately reflects the nation’s highest-priority critical infrastructure.

The NIPP calls for risk assessments—such as NCIPP efforts—to be complete, reproducible, documented, and defensible to produce results that can contribute to cross-sector risk comparisons for supporting investment, planning, and resource prioritization decisions. Table 1 provides a description of these core criteria for risk assessments.

DHS could not provide documentation explaining how the threshold levels were established, such as the methodology for developing the NCIPP criteria or the analysis used to support the criteria, because, according to agency officials, the agency undertook an information technology change in the spring of 2012 that resulted in the loss of agency e-mails and program documentation. Nevertheless, as previously noted, officials told us the criteria and thresholds were established at the discretion of the Assistant Secretary for Infrastructure Protection. Program officials noted that they review the list on an annual basis but that the list has not been independently verified and validated by an external peer review. These officials believe a peer review would enable DHS to determine whether its efforts to develop the NCIPP list are based on analytically sound methodology and whether it has appropriate procedures in place to ensure that the NCIPP list is defensible and reproducible.

We have previously reported that peer reviews are a best practice in risk management and that independent expert review panels can provide objective reviews of complex issues. An independent peer review to validate the NCIPP criteria and list development process would better position DHS to reasonably assure that, consistent with the NIPP risk management framework, federal and state partners that use the NCIPP list have sound information when making risk management and resource allocation decisions. According to the NIPP, having sound information for making those decisions is critical for focusing attention on those protection and resiliency activities that bring the greatest return on investment.

In August 2012, NCIPP program officials told us they would like to establish a peer review to validate the program because officials believe the list has stabilized and now consider the program to be in a “maintenance phase.” In December 2012, the program director told us that IASD drafted and submitted a proposal to the Assistant Secretary for Infrastructure Protection in November 2012 that proposed different approaches for reviewing the NCIPP, including a peer review of the criteria used to decide which assets and systems should be placed on the list and the process for doing so. At that time, DHS officials said that they could not provide a copy of the draft proposal because it had not been approved by management. As of January 2013, IASD told us that the proposal had not been submitted to the Assistant Secretary for Infrastructure Protection as originally discussed, that it was unclear when the proposal would be submitted, and that it remained uncertain whether a peer review would be approved.

The National Research Council of the National Academies has also recommended that DHS improve its risk analyses for infrastructure protection by validating the models and submitting them to external peer review. According to the council, periodic reviews and evaluations of risk model outputs are important for transparency with respect to decision makers. These reviews should involve specialists in modeling and in the problems that are being addressed and should address the structure of the model, the types and certainty of the data, and how the model is intended to be used. Peer reviews can also identify areas for improvement. As we have previously reported, independent peer reviews cannot ensure the success of a model, but they can increase the probability of success by improving the technical quality of projects and the credibility of the decision-making process. Thus, an independent peer review would better position DHS to provide reasonable assurance that the NCIPP criteria and list development process is reproducible and defensible given the recent and planned changes, and that critical infrastructure protection efforts are being prioritized on the nation’s highest-priority infrastructure as intended by the NIPP risk management framework.

DHS Has Taken Actions to Improve Its Consultation with States and SSAs to Address Challenges Developing the NCIPP List

DHS has taken various actions to work with states and SSAs, consistent with statutory requirements and the NIPP, to identify and prioritize critical infrastructure. However, officials representing selected states and SSAs have mixed views about their experiences adjusting to DHS’s changes to the NCIPP. DHS recognizes that states, in particular, face challenges— such as resource and budgetary constraints—associated with nominating assets to the NCIPP list, and has taken actions to address these challenges and reduce the burden on states.

DHS Has Taken Various Actions to Work with States and SSAs on Developing the List

In recent years, DHS has taken actions to improve its outreach to states and SSAs to obtain their input on changes to the NCIPP. In 2009, DHS’s outreach to states and SSAs consisted of issuing a memorandum to obtain input on the proposed change to consequence-based criteria. Since 2009, DHS has taken various actions to address state nomination challenges and to reduce the burden on states. For example, in 2009, DHS revised its list development process to be more transparent and provided states with additional resources and tools for developing their NCIPP nominations. Specifically, once states submit their NCIPP nominations, DHS is to make preliminary adjudication determinations based upon the NCIPP criteria, then provide its preliminary adjudication results (whether a nomination was accepted or not) and why the decision was made. Next, DHS is to allow states an opportunity to request reconsideration of the nomination for which they could provide additional documentation clarifying the eligibility of the infrastructure. Figure 5 shows the revised NCIPP list development process, including the nomination, adjudication, and reconsideration phases.

In addition to revising the adjudication process, DHS took several actions intended to improve the nomination process in recent years. First, according to DHS’s 2011 data call guidance, DHS provided on-site assistance from subject matter experts to assist states with identifying infrastructure and disseminated a lessons learned document providing examples of successful nominations to help states improve justifications for nominations. Second, DHS has taken action to be more proactive in engaging states and SSAs in ongoing dialogue on proposed criteria changes and improving the NCIPP process and resulting list. For example, in 2010, DHS hosted the Food and Agriculture Criticality Working Group established through the Food and Agriculture Sector Government Coordinating Council—consisting of over 100 participants (including DHS, states, and SSAs)—to discuss the aforementioned modification of the criteria to make it more applicable to the agriculture and food sector. As discussed earlier, DHS and its state and SSA partners are currently reevaluating the agriculture and food sector-specific criteria, and the SSAs held a meeting in December 2012 to discuss updating and adding additional criteria. In addition, in July 2011, DHS established a working group composed of state and SSA officials to solicit feedback on the nomination process and recommend actions to improve the quality of the NCIPP list in preparation for the 2013 data call. DHS officials told us that much of the feedback received from states and SSAs centered on DHS improving communication and guidance throughout the data call—for example, updating the guidance with additional information on criteria. DHS also planned long-term studies, such as requesting input from partners on modifying criteria thresholds. DHS officials told us that they conducted extensive outreach to states and SSAs to encourage participation in the NCIPP working group including extending the submission deadlines multiple times, funding an on-site meeting with the partners, and hosting webinars and conference calls. However, according to DHS officials, DHS has since disbanded the working group because of lack of state and SSA participation and DHS budget constraints.

Officials Representing Selected SSAs and States Have Mixed Views on the NCIPP Nomination Process

Despite DHS’s outreach efforts, homeland security officials representing selected states and SSAs have mixed views on the NCIPP nomination process because of program changes, such as the aforementioned change to consequence-based criteria. Overall, the SSA officials we interviewed had more positive views of the NCIPP nomination process than the state officials we interviewed.

SSA officials representing five of the eight sectors we interviewed told us that they believe it is very easy or moderately easy to nominate assets to the NCIPP list. However, officials representing three sectors said that they believe it is moderately difficult or very difficult to nominate assets to the list because of various factors. For example, one SSA official told us that the diversity and complexity of the sector’s assets makes it difficult to determine which assets meet the NCIPP criteria. Also, one SSA official stated that the online tool that the SSA uses to nominate assets to the NCIPP list requires detailed information, such as latitude and longitude coordinates, that may not be available for assets with unique characteristics.

By contrast, most state officials we contacted reported that it is difficult to nominate assets to the NCIPP list using the consequence-based criteria, and two officials said that they are considering whether to continue to participate in the NCIPP process. Homeland security officials representing 13 of the 15 states told us that they believe that the nomination process is moderately difficult or very difficult, while officials representing 2 states told us that they believe the nomination process is neither easy nor difficult. For the 13 states where officials told us that they believe the nomination process is moderately difficult or very difficult, officials representing 5 states told us that not having the capability and resources to develop scenarios to support consequence-based criteria (such as conducting economic analysis) are the major factors contributing to the time-consuming and difficult process of submitting nominations when the criteria changed. Officials from 2 states told us that their states no longer plan to nominate infrastructure to the NCIPP list because of the time and effort required to make nominations.

DHS Is Working to Address Challenges Facing States

DHS officials told us that they recognize that some states are facing challenges participating in the NCIPP program (as we previously identified in our discussions with state officials) and, according to officials, they are working to help them address some of these challenges. For example, DHS officials said that they recognized that the change to consequence-based criteria was difficult because it required states to invest considerable resources to make nominations. However, they also believe that other factors may influence states’ willingness to participate, such as (1) some state officials may believe that all critical infrastructure has been captured for the state and sector, (2) some state officials may believe that the benefits of participating—such as access to grant funding—have diminished and there is no longer an incentive to participate, and (3) the NCIPP data call process is voluntary and state partners do not have to participate if they do not wish.

DHS has taken several steps to minimize the burden on state partners. First, DHS is conducting a more limited annual data call wherein all assets identified on the previous list are generally carried forward onto the subsequent list and states are asked to provide nominations of (1) critical infrastructure not accepted during the previous data call or (2) critical infrastructure not previously nominated but that partners believe merits consideration. In fiscal year 2013, 13 state or territorial partners participated in the data call. DHS officials question whether, given current budget constraints facing state and federal partners, there is a need to conduct an annual data call. In our past work, we have reported that, with our nation facing serious, long-term fiscal challenges, a reevaluation of federal agencies’ operations has never been more important than it is Consistent with our past work, DHS officials told us that they today.considered whether the costs of conducting an annual data call outweigh the benefits, since only minor updates are being made to the NCIPP list. In addition, one state official observed that, in a resource-constrained environment, states can no longer afford to conduct the NCIPP data call because it diverts resources from critical infrastructure protection partnership and coordination activities that could increase state and regional resilience, such as states maintaining their own list of high- priority critical infrastructure. In response, according to DHS officials, DHS is working to minimize major changes to the consequence-based NCIPP criteria, and thus, does not anticipate making any major changes to the NCIPP criteria that would cause a burden on state resources. Finally, DHS officials also told us that they have begun to take additional actions to enhance state participation, including developing and organizing a webinar with PSAs and state officials as they execute the data call. DHS is also working collaboratively with the State, Local, Tribal and Territorial Government Coordinating Council to develop a guide to assist states with their efforts to identify and prioritize their critical infrastructure.

DHS Could Not Verify That It Met Requirements to Report to Congress on the NCIPP

DHS has prepared documents describing the national asset database and the prioritized critical infrastructure list; however, DHS could not verify that it has delivered these documents for purposes of meeting its statutory requirement to report this information to the congressional committees specified in the law. Pursuant to the 9/11 Commission Act, which amended title II of the Homeland Security Act, DHS is required to report annually to the Committee on Homeland Security and Governmental Affairs of the Senate and the Committee on Homeland Security of the House of Representatives on, among other things, any significant challenges in compiling the database or list and, if appropriate, the extent to which the database or list has been used to allocate federal funds to prevent, reduce, mitigate, or respond to acts of terrorism.Although DHS was able to compile documents on the database and list for fiscal years 2008 through 2011 that generally contain the information on which DHS is to report, officials from DHS and the Office of Infrastructure Protection told us they were uncertain whether the documents were delivered to the requisite congressional committees because they do not have records to indicate that the documents were delivered. According to a DHS official, the DHS document tracking system includes notes on the intended delivery of the fiscal year 2008 and 2010 documents and a note regarding delivery of the fiscal year 2009 document, but the system does not contain a record to verify that the documents were delivered, i.e., that the transactions actually occurred. Staff from both committees could not find evidence of the documents. One staff member also conducted a search of congressional archives for the 109th, 110th, and 111th Congresses and found no records of receiving the statutorily required reports from DHS.

We reviewed the DHS documents intended to fulfill the statutory reporting requirements for fiscal years 2008 through 2011 and found that they generally contain information consistent with the statutory requirements. For example, the documents generally included an overview of the NCIPP list development process and changes, if any, from the previous year; challenges compiling the list; and how the list is used. Table 2 shows key elements of each document and how they match up with the statutory requirements.

Nevertheless, absent an approach to verify the delivery of the statutorily required reports on the database and list to the requisite committees of Congress, DHS cannot ensure that it has provided the committees with necessary information in a timely manner. The Standards for Internal Control in the Federal Government calls for compliance with applicable laws and regulations and for the accurate, timely, and appropriate documentation of the transactions. An approach to verify the timely delivery of required reports to the requisite committees of Congress, such as documenting or recording the transactions, would better position DHS to ensure that it is in compliance with its statutory reporting requirements, thereby providing the committees information needed to perform oversight.

Conclusions

DHS efforts to identify and prioritize infrastructure continue to evolve, and the department has taken important actions to focus its prioritization approach on consequences, consistent with statutory requirements and the NIPP risk management framework. However, in recent years, DHS introduced new criteria for select sectors and non-consequence-based criteria to account for some assets, which could hinder DHS’s ability to compare assets across sectors in order to identify the nation’s highest- priority critical infrastructure. Given the magnitude of the changes DHS has made to the criteria for including infrastructure on the list, validation of the NCIPP list development approach could provide DHS managers and infrastructure protection partners more reasonable assurance that the list captures the highest-priority infrastructure that, if destroyed or disrupted, could cause national or regional catastrophic effects. NCIPP program officials told us they would like to have the NCIPP reviewed to validate the criteria used to decide which assets and systems should be placed on the list, but they have not yet submitted a proposal for this review to the Assistant Secretary for Infrastructure Protection. An independent, external peer review would better position DHS to provide reasonable assurance that its approach is reproducible and defensible, and that infrastructure protection efforts are being prioritized on the nation’s highest-priority critical infrastructure as intended by the NIPP risk management framework. Finally, it is unclear if DHS has met statutory annual reporting requirements regarding the NCIPP lists because DHS is unable to verify the delivery of these required reports. As a result, DHS cannot ensure that it is fulfilling its statutory reporting obligations and may not be providing the requisite congressional committees with the information needed to effectively oversee the program, particularly with regard to the allocation of scarce federal resources.

Recommendations for Executive Action

To better ensure that DHS’s approach to identify and prioritize critical infrastructure is consistent with the NIPP risk management framework and that DHS is positioned to provide reasonable assurance that protection and resiliency efforts and investments are focused on the nation’s highest-priority critical infrastructure, we recommend that the Assistant Secretary for Infrastructure Protection, Department of Homeland Security, take the following action: commission an independent, external peer review of the program with clear project objectives for completing this effort.

To ensure that DHS is in compliance with its statutory reporting requirements and provides decision makers with the information necessary to perform program oversight, we recommend that the Secretary of Homeland Security, take the following action: develop an approach, such as documenting or recording the transaction, to verify the delivery of the statutorily required annual reports on the database and list to the requisite congressional committees.

Agency Comments and Our Evaluation

We provided a draft of this report to the Secretary of Homeland Security for review and comment. In its written comments reproduced in Appendix III, DHS agreed with both of our recommendations.

With regard to our first recommendation that DHS commission an independent, external peer review of the program with clear project objectives for completing this effort, DHS stated that a peer review would enable DHS to determine whether the NCIPP list is based on analytically sound methodology and whether appropriate procedures are in place to ensure that the list is defensible and reproducible. Specifically, DHS stated that it plans to commission and complete an independent peer review of the NCIPP process by the end of the fourth quarter of fiscal year 2014. If fully implemented, to include a review by independent experts to validate the criteria and process DHS uses to decide which assets and systems should be placed on the NCIPP list as we described in this report, DHS’s planned efforts will address the intent of this recommendation.

With regard to our second recommendation that DHS develop an approach, such as documenting or recording the transaction, to verify the delivery of the statutorily required annual reports on the database and list to the requisite congressional committees, DHS stated that it has a system in place to track the development and approval of congressional reports, but DHS confirmed that it does not currently have a standard procedure for verifying that the congressional reports are delivered. DHS stated that its Office of Legislative Affairs will develop and implement a standard operating procedure for tracking the delivery of annual reports on the database and the list. DHS did not provide an estimated completion date for this effort. If fully implemented, DHS’s planned efforts will address the intent of this recommendation.

DHS also provided technical comments which we incorporated, as appropriate.

We are sending copies of this report to the Secretary of Homeland Security, the Under Secretary of the National Programs Protection Directorate, selected congressional committees, and other interested parties. In addition, the report is available at no charge on GAO’s website at http://www.gao.gov. If you or your staff have any questions about this report, please contact Stephen L. Caldwell at (202) 512-8777 or caldwells@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix IV.

Appendix I: Critical Infrastructure Sectors

This appendix provides information on the 18 critical infrastructure sectors and the federal agencies responsible for sector security. The National Infrastructure Protection Plan (NIPP) outlines the roles and responsibilities of the Department of Homeland Security (DHS) and its partners—including other federal agencies. Within the NIPP framework, DHS is responsible for leading and coordinating the overall national effort to enhance protection via 18 critical infrastructure sectors. Homeland Security Presidential Directive/HSPD-7 and the NIPP assign responsibility for critical infrastructure sectors to sector-specific agencies (SSA). On February 12, 2013, the President issued Presidential Policy Directive/PPD-21 that, among other things, reduced the number of critical infrastructure sectors from 18 to 16. As an SSA, DHS has direct responsibility for leading, integrating, and coordinating efforts of sector partners to protect 11 of the 18 critical infrastructure sectors. The remaining sectors are coordinated by eight other federal agencies. Table 3 lists the SSAs and their sectors as they existed before any reorganization of the critical infrastructure sectors affected by the issuance of PPD-21.

Appendix II: Objectives, Scope, and Methodology

To address our first objective—determine the extent to which DHS changed its criteria for developing the National Critical Infrastructure Prioritization Program (NCIPP) list, identified the impact, if any, of these changes, and validated its approach—we reviewed the Implementing Recommendations of the 9/11 Commission Act of 2007 (9/11 Commission Act), which, by amending title II of the Homeland Security Act of 2002, required the Secretary of DHS to establish and maintain a national database of systems and assets determined to be vital and the loss, interruption, incapacity, or destruction of which would have a negative or debilitating effect on the economic security, public health, or safety of the United States, any state, or any local government, or as otherwise determined appropriate for inclusion by the Secretary. In addition, the 9/11 Commission Act required the Secretary of DHS to establish and maintain a single prioritized list of systems and assets included in the national database that the Secretary determines would, if destroyed or disrupted, cause national or regional catastrophic effects.

We also reviewed DHS guidelines issued to states and SSAs from 2007 through 2012 that included details on the NCIPP list development process, to determine how DHS’s criteria and process for developing the list changed year to year. We then obtained and analyzed the NCIPP lists finalized for fiscal years 2007 through 2012 to determine the total number of high-priority assets by state and the change in distribution of high- priority assets by sector year to year. We used our analysis to select 8 of the 18 sectors—the banking and finance, defense industrial base, chemical, energy, transportation systems, agriculture and food, government facilities, and dams sectors. We chose these sectors to obtain a mix of sectors that (1) experienced the largest and smallest percentage change in the distribution of assets on the NCIPP list between fiscal years 2009 and 2011 because of program changes DHS made during this period, and (2) have an SSA located within or outside DHS. The information from our analysis of these sectors is not generalizable to the universe of all sectors. However, it provides valuable insights into yearly changes in the distribution of assets on the NCIPP list among a diverse group of sectors. On February 12, 2013, the President issued Presidential Policy Directive/PPD-21 that, among other things, reduced the number of critical infrastructure sectors from 18 to 16. To assess the reliability of the data, we reviewed existing documentation about DHS’s data system, which houses the data application used to create the NCIPP list, and spoke with knowledgeable agency officials responsible for maintaining the system and data application. While we determined that the data were sufficiently reliable to provide a general overview of the program, we included data limitations from our previous work in this report, where appropriate. We also interviewed officials in the Infrastructure Analysis and Strategy Division (IASD), which is part of the Office of Infrastructure Protection in DHS’s National Protection and Program Directorate, who are responsible for managing the NCIPP to identify DHS’s rationale for changing the criteria.

In addition, to address the first objective, we reviewed our prior reports as well as DHS Inspector General reports on protection and resiliency prioritization efforts and spoke with program officials who use the list from DHS’s Protective Security Coordination Division (PSCD), the Federal Emergency Management Agency (FEMA), and the Federal Bureau of Investigation to determine how they use the NCIPP list and the impact changes to the NCIPP list have had, if any, on their ability to use the list during fiscal years 2007 through 2012. In addition to interviewing program officials from PSCD headquarters, we also conducted interviews with nine of DHS’s protective security advisors (PSA)—one from each of the nine PSA regions—to discuss their contributions to the NCIPP list, how they use the list to prioritize their activities, and actions NCIPP management has taken to solicit their feedback regarding the program.The results from our interviews are not generalizable to the universe of PSAs but provide specific examples of how PSAs use the list and insights on the effect changes have had on their activities.

Although the FEMA UASI grant formula is the same as the FEMA State Homeland Security Program (SHSP), we focused our sensitivity analysis on the UASI grant because this grant is allocated to only a subset of the nation’s 100 most populous urban areas— referred to as metropolitan statistical areas (MSA)—each year, whereas by law, each state and territory are required to receive a minimum allocation of the SHSP funds each year. For ease of reporting, we will refer to UASI grant recipients as cities rather than MSAs. for these 31 cities. We then re-ran the risk formula using these revised NCIPP level 2 infrastructure counts, while holding all other data inputs constant, which resulted in a change to the relative risk score rankings for 5 of the top 31 cities. We also performed additional statistical analysis of the FEMA risk formula and data that showed UASI grant allocations are strongly associated with a city’s current risk score, even when accounting for the influence of the previous year’s grant allocations. Based on our prior work with the FEMA UASI grant risk formula and interviews with FEMA officials about its data sources and quality assurance procedures, we determined that the data were sufficiently reliable for the purposes of this report. Last, we met with IASD officials to discuss actions they have taken to identify the impact of changes, if any, on users of the list, and compared these actions with applicable criteria in the NIPP and Standards for Internal Control in the Federal Government to determine if they were consistent.

Regarding our second objective—to determine the extent to which DHS worked with states and SSAs to develop the NCIPP list—we reviewed relevant provisions of the 9/11 Commission Act and the guidelines DHS issued to state homeland security advisers and SSAs to solicit nominations of high-priority infrastructure for inclusion on the NCIPP list.

We also conducted interviews with officials from 10 SSAs and 15 state homeland security offices to obtain federal and state perspectives on DHS’s change to consequence-based criteria and coordination of the NCIPP program, as well as their views on nominating to and using the list. The SSA officials we interviewed represented the 8 sectors selected during our analysis for the first objective. Specifically, DHS was the SSA for 4 of the sectors—the chemical, dams, government facilities, and transportation systems sectors. The Departments of Energy, Defense, and the Treasury were the SSAs for 3 sectors—the energy, defense industrial base, and banking and finance sectors, respectively. Two SSAs, the Department of Agriculture and the Food and Drug Administration, share responsibility for the agriculture and food sector. The state homeland security officials we interviewed represented 15 states—California, Georgia, Illinois, Hawaii, Oklahoma, Maine, Mississippi, Nevada, New Jersey, New York, Texas, Virginia, Washington, West Virginia, and Wisconsin. We selected these states because they contained a range in the number of assets on the NCIPP The list and represented at least 1 state from each of 9 PSA regions.sector and state interviews are not generalizable to the universe of infrastructure sectors and states contributing to the NCIPP list. However, our selection combined with DHS policy guidance, further informed us about DHS efforts to manage the NCIPP program across a spectrum of states and partners nationwide. Finally, we interviewed IASD officials to discuss actions DHS had taken to consult with state and federal partners (as identified in program guidelines and based on our interviews with states and SSAs), and compared their responses with applicable criteria in the NIPP, Standards for Internal Control in the Federal Government, and relevant statutory provisions.

With regard to our third objective—determine the extent to which DHS reported to the requisite committees of Congress on the NCIPP—we reviewed the statutory requirement that DHS report annually to the Senate Committee on Homeland Security and Governmental Affairs and the House Committee on Homeland Security on the national asset database and prioritized critical infrastructure list. We also spoke to staff members representing both committees to determine if the committees received the statutorily required reports. Last, we interviewed DHS officials to discuss efforts to provide these reports to the committees and obtained and reviewed documents on the national asset database and prioritized critical infrastructure list that were intended to meet statutory reporting requirements to determine if these efforts were consistent with relevant statutory provisions and Standards for Internal Control in the Federal Government.

We conducted this performance audit from May 2012 to March 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix III: Comments from the Department of Homeland Security

Appendix IV: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, John F. Mortin, Assistant Director, and Andrew M. Curry, Analyst-in-Charge, managed this assignment. Chuck Bausell, Mona Nichols-Blake, Aryn Ehlow, Katherine M. Davis, Michele C. Fejfar, Eric D. Hauswirth, Mitchell B. Karpman, Thomas F. Lombardi, and Janay Sam made significant contributions to the work.

Related GAO Products

Critical Infrastructure Protection: Preliminary Observations on DHS Efforts to Assess Chemical Security Risk and Gather Feedback on Facility Outreach. GAO-13-421T. Washington, D.C.: March 14, 2013.

Critical Infrastructure Protection: An Implementation Strategy Could Advance DHS’s Coordination of Resilience Efforts across Ports and Other Infrastructure. GAO-13-11. Washington, D.C.: October 25, 2012.

Critical Infrastructure Protection: Summary of DHS Actions to Better Manage Its Chemical Security Program. GAO-12-1044T. Washington, D.C.: September 20, 2012.

Critical Infrastructure Protection: DHS Is Taking Action to Better Manage Its Chemical Security Program, but It Is Too Early to Assess Results. GAO-12-567T. Washington, D.C.: September 11, 2012.

Critical Infrastructure: DHS Needs to Refocus Its Efforts to Lead the Government Facilities Sector. GAO-12-852. Washington, D.C.: August 13, 2012.

Critical Infrastructure Protection: DHS Is Taking Action to Better Manage Its Chemical Security Program, but It Is Too Early to Assess Results. GAO-12-515T. Washington, D.C.: July 26, 2012.

Critical Infrastructure Protection: DHS Could Better Manage Security Surveys and Vulnerability Assessments. GAO-12-378. Washington, D.C.: May 31, 2012.

Critical Infrastructure Protection: DHS Has Taken Action Designed to Identify and Address Overlaps and Gaps in Critical Infrastructure Security Activities. GAO-11-537R. Washington, D.C.: May 19, 2011.

Critical Infrastructure Protection: DHS Efforts to Assess and Promote Resiliency Are Evolving but Program Management Could Be Strengthened. GAO-10-772. Washington, D.C.: September 23, 2010.

Critical Infrastructure Protection: Update to National Infrastructure Protection Plan Includes Increased Emphasis on Risk Management and Resilience. GAO-10-296. Washington, D.C.: March 5, 2010.

The Department of Homeland Security’s (DHS) Critical Infrastructure Protection Cost-Benefit Report. GAO-09-654R. Washington, D.C.: June 26, 2009.

Information Technology: Federal Laws, Regulations, and Mandatory Standards to Securing Private Sector Information Technology Systems and Data in Critical Infrastructure Sectors. GAO-08-1075R. Washington, D.C.: September 16, 2008.

Risk Management: Strengthening the Use of Risk Management Principles in Homeland Security. GAO-08-904T. Washington, D.C.: June 25, 2008.

Critical Infrastructure: Sector Plans Complete and Sector Councils Evolving. GAO-07-1075T. Washington, D.C.: July 12, 2007.

Critical Infrastructure Protection: Sector Plans and Sector Councils Continue to Evolve. GAO-07-706R. Washington, D.C.: July 10, 2007.

Critical Infrastructure: Challenges Remain in Protecting Key Sectors. GAO-07-626T. Washington, D.C.: March 20, 2007.

Homeland Security: Progress Has Been Made to Address the Vulnerabilities Exposed by 9/11, but Continued Federal Action Is Needed to Further Mitigate Security Risks. GAO-07-375. Washington, D.C.: January 24, 2007.

Critical Infrastructure Protection: Progress Coordinating Government and Private Sector Efforts Varies by Sectors’ Characteristics. GAO-07-39. Washington, D.C.: October 16, 2006.

Information Sharing: DHS Should Take Steps to Encourage More Widespread Use of Its Program to Protect and Share Critical Infrastructure Information. GAO-06-383. Washington, D.C.: April 17, 2006.

Risk Management: Further Refinements Needed to Assess Risks and Prioritize Protective Measures at Ports and Other Critical Infrastructure. GAO-06-91. Washington, D.C.: December 15, 2005.