Critical Infrastructure Protection: DHS Efforts to Assess Chemical Security Risk and Gather Feedback on Facility Outreach Can Be Strengthened
Released: 2013-04-05T00:00:00

Letter

Background

The CFATS program is intended to secure the nation’s chemical infrastructure by identifying and protecting high-risk chemical facilities. Section 550 of the DHS appropriations act for fiscal year 2007 requires DHS to issue regulations establishing risk-based performance standards for the security of facilities that the Secretary determines to present high levels of security risk. The CFATS rule was published in April 2007 and Appendix A to the rule, published in November 2007, listed 322 chemicals of interest and the screening threshold quantities amount for each.

ISCD has direct responsibility for implementing DHS’s CFATS rule, including assessing risks and identifying high-risk chemical facilities, promoting effective security planning, and ensuring that high-risk facilities meet the applicable risk-based performance standards through site security plans approved by DHS. ISCD is managed by a Director and operates five branches that are, among other things, responsible for (1) information technology operations; (2) policy and planning; (3) providing compliance and technical support; (4) inspecting facilities and enforcing CFATS regulatory standards; and (5) managing logistics, administration, and chemical security training. From fiscal years 2007 through 2012, DHS dedicated about $442 million to the CFATS program. In fiscal year 2012, DHS was authorized 242 full-time equivalent positions.

The CFATS Rule and Process

DHS’s CFATS rule outlines a specific process for administering the program. Any chemical facility that possesses any of the 322 chemicals in the quantities that meet or exceed the threshold quantity outlined in Appendix A of the rule is required to use DHS’s Chemical Security Assessment Tool (CSAT)—a web-based application through which owners and operators of chemical facilities provide information about the facility. Once a facility is registered in CSAT, owners and operators are to complete the CSAT Top Screen—which is the initial screening tool or document whereby the facility is to provide DHS various data, including the name and location of the facility and the chemicals and their quantities at the site. DHS is to analyze this information using its risk assessment approach, which is discussed in more detail below, to initially determine whether the facility is high risk. If so, DHS is to notify the facility of its preliminary placement in one of four risk-based tiers—tier 1, 2, 3, or 4. Facilities preliminarily placed in any one of these tiers are considered to be high risk, with tier 1 facilities considered to be the highest risk. Facilities that DHS initially determines to be high risk are required to then complete the CSAT security vulnerability assessment, which includes the identification of potential critical assets at the facility and a related vulnerability analysis. DHS is to review the security vulnerability assessment and notify the facility of DHS’s final determination as to whether or not the facility is considered high risk, and if the facility is determined to be a high-risk facility, about its final placement in one of the four tiers.

Once assigned a final tier, the facility is required to use CSAT to submit a site security plan or participate in an alternative security program in lieu of a site security plan. The security plan is to describe the security measures to be taken to address the vulnerabilities identified in the vulnerability assessment, and identify and describe how security measures selected by the facility are to address the applicable risk-based performance standards. DHS then is to conduct a preliminary review of the security plan to determine whether it meets the regulatory requirements. If these requirements appear to be satisfied, DHS is to issue a letter of authorization for the facility’s plan. DHS then is to conduct an authorization inspection of the facility and subsequently determine whether to approve the security plan. If DHS determines that the plan does not satisfy CFATS requirements, DHS then notifies the facility of any deficiencies and the facility must submit a revised plan correcting them. If the facility fails to correct the deficiencies, DHS may disapprove the plan. Following approval, DHS may conduct further inspections to determine if the facility is in compliance with its approved security plan.Figure 1 illustrates the CFATS regulatory process.

ISCD uses a risk assessment approach during the early stages of the regulatory process to develop risk scores to assign chemical facilities to a final tier. According to an ISCD document that describes how ISCD develops its CFATS risk score, the risk score is intended to be derived from estimates of consequence (the adverse effects of a successful attack), threat (the likelihood of an attack), and vulnerability (the likelihood of a successful attack, given an attempt). The ISCD risk assessment approach is composed of three models, each based on a particular security issue: (1) release, (2) theft or diversion, and (3) sabotage, depending on the type of risk associated with the 322 chemicals of interest listed in Appendix A of the CFATS rule. For release, the model assumes that a terrorist will release the chemical of interest at the facility and then estimates the risk to the surrounding population. For theft or diversion, the model assumes that a terrorist will steal or have the chemical of interest diverted to him or herself and then estimates the risk of a terrorist attack using the chemical of interest in a way that causes the most harm at an unspecified off-site location. For sabotage, the model assumes that a terrorist will remove the chemical of interest from the facility and mix it with water, creating a toxic release at an unspecified off- site location, and then estimates the risk to a medium-sized U.S. city. Once ISCD estimates a risk score based on these models, it assigns the facility to a final tier.

ISCD Has Assigned Thousands of Facilities to Tiers, but ISCD’s Approach to Risk Assessment Does Not Reflect All Risk Elements

Since 2007, ISCD has assigned about 3,500 high-risk chemical facilities to final tiers and has taken action to identify and address problems with its risk-tiering approach. However, ISCD’s risk-tiering approach does not reflect all elements of risk. Specifically, ISCD is to assess risk using estimates of the consequences, threat, and vulnerability associated with a terrorist attack, but ISCD does not consider key elements of risk, such as economic consequences or facility vulnerability consistent with the NIPP and the CFATS rule. ISCD recognizes that its tiering approach is not complete and continues to mature and has begun to take actions to assess its approach, including commissioning an expert panel.

ISCD Has Tiered Thousands of High-Risk Chemical Facilities and Resolved Some Problems Using Its Risk Assessment Approach to Assign Tiers

In July 2007, ISCD began reviewing information submitted by the owners and operators of approximately 40,000 facilities. By January 2013, ISCD had designated about 4,400 of the 40,000 facilities as high risk and thereby covered by the CFATS rule. ISCD had assigned about 3,500 of those facilities to a final tier, of which about 90 percent were tiered because of the risk of theft or diversion. The remaining 10 percent were tiered because of the risk of release or the risk of sabotage. In addition, about 900 of the 4,400 facilities had been assigned to preliminary tiers and were to be assigned a final tier once ISCD processed data from the facility using ISCD’s risk assessment approach. ISCD officials noted that the number of tiered facilities and their individual tiers is likely to be fluid over time as changes in chemical holdings, production, processes, storage methods, or use occur. Table 1 shows the number and percentage of facilities assigned a final tier as of January 2013.

Over the last 2 years, ISCD has identified problems with the way the release chemicals model assigns chemical facilities to tiers and has taken or begun to take action to address those problems. In February 2011, ISCD managers were notified by contracting officials responsible for running the model that some chemical facilities had been placed in an incorrect final tier because this model included incorrect data about the release of high-risk chemicals of interest. In June 2011, ISCD officials adjusted the model, lowering the tier for about 250 facilities, about 100 of which were subsequently removed from the CFATS program. In September 2012, ISCD officials stated that they were confident that the adjustment helped make this model more accurate.

However, in October 2012, ISCD officials stated that they had discovered another anomaly that they were working to correct. Specifically, ISCD officials said that they had uncovered a defect that led the model to exclude population density calculations for about 150 facilities in states or U.S. territories outside the continental United States, including Alaska, Hawaii, Puerto Rico, and Guam. In December 2012, ISCD officials said that they had made adjustments to the model to resolve this issue. They added that they expected that once data from the approximately 150 facilities were assessed, no more than 11 of the approximately 150 facilities would be affected by a change to their tier. ISCD officials said that as of February 2013, upon further examination, they expect that about 2 facilities will be affected. However, those two facilities were already tiered for other chemicals covered by CFATS, and ISCD officials did not expect those facilities’ respective tiers to change.

ISCD’s Risk Assessment Approach Does Not Consider All Elements of Risk

ISCD has tiered thousands of facilities using its current risk assessment approach, but ISCD’s risk assessment approach is not mature because it does not consider key elements of risk from the NIPP and the CFATS rule. According to the NIPP, which, among other things, establishes the framework for managing risk among the nation’s critical infrastructure, risk is a function of three components—consequence, threat, and vulnerability—and a risk assessment approach must assess each component for every defined risk scenario. Furthermore, the CFATS rule calls for ISCD to review consequence, threat, and vulnerability information in determining a facility’s final tier. However, ISCD’s risk assessment approach does not fully consider all of the core criteria or components of a risk assessment, as specified by the NIPP, nor does it comport with parts of the CFATS rule.

ISCD’s risk assessment approach does not currently conform to the NIPP and is not consistent with the CFATS rule because it does not yet fully consider consequence criteria when assessing risk associated with a terrorist attack. The NIPP states that at a minimum, consequences should focus on the two most fundamental components—human consequences and the most relevant direct economic consequences. Like the NIPP, the CFATS rule states that chemical facilities covered by the rule are those that present a high risk of significant adverse consequences for human life or health, or critical economic assets, among other things, if subjected to terrorist attack, compromise, infiltration, or exploitation.

Our review of ISCD’s risk assessment approach and discussions with ISCD officials showed that the approach is currently limited to focusing on one component of consequences—human casualties associated with a terrorist attack involving a chemical of interest—and does not consider consequences associated with economic criticality. ISCD officials told us that, at the inception of the CFATS program, they did not have the capability to collect or process all of the economic data needed to calculate the associated risks and they were not positioned to gather all of the data needed. They said that they collect basic economic data as part of the initial screening process in the CSAT; however, they would need to modify the current tool to collect more sufficient data. This contrasts with other DHS components, like the U.S. Coast Guard and the Transportation Security Administration, which have gathered and assessed economic data as part of some critical infrastructure risk assessment efforts.

ISCD officials stated that they have begun to have discussions with other DHS components, like the U.S Coast Guard, about their approach to risk assessment. They also said that they recognize that the economic consequences part of their risk-tiering approach will require additional work before it is ready to be introduced. They noted that the preamble to the November 2007 CFATS rule stated that they would defer incorporating economic criticality until a later date. In September 2012, ISCD officials told us that they had engaged Sandia National Laboratories to examine how ISCD could gather needed information and determine the risk associated with economic impact, but this effort is in the initial stages, with an expected completion date of June 2014. ISCD officials added they are uncertain about how Sandia National Laboratories’ efforts will affect their risk assessment approach.

ISCD’s risk assessment approach is also not consistent with the NIPP because it does not consider threat for the majority of regulated facilities. According to the NIPP, risk assessments should estimate threat as the likelihood that the adversary would attempt a given attack method against the target. Like the NIPP, the CFATS rule requires that, as part of site vulnerability assessment process, facilities conduct a threat assessment, which is to include a description of the internal, external, and internally- assisted threats facing the facility and that ISCD review the site vulnerability assessment as part of the final determination of a facility’s tier. showed that (1) ISCD is inconsistent in how it assesses threat using the different models because while it considers threat for the 10 percent of facilities tiered because of the risk of release or sabotage, it does not consider threat for the approximately 90 percent of facilities that are tiered because of the risk of theft or diversion; and (2) ISCD does not use current threat data for the 10 percent of facilities tiered because of the risk of release or sabotage.

6 C.F.R. §§ 27.215, .220.

ISCD did not have documentation to show why threat had not been factored into the formula for approximately 90 percent of facilities tiered because of the risk of theft or diversion. However, they pointed out that the cost of adding a threat analysis for these facilities might outweigh the benefits of doing so because it may not provide the increased specificity and level of details to justify the cost. Officials further explained that the model assumes that a terrorist would remove the chemical of interest and use it offsite and ISCD cannot predict where a chemical of interest would be used as a result of theft or diversion. Nonetheless, it is inconsistent for ISCD to not consider threat for the theft or diversion risk model, given that the assumptions about an attack are similar to those considered under the sabotage model—that is, both models assume that a terrorist would use a chemical of interest at an offsite, undisclosed location. This extra level of specificity would be useful for ISCD’s overall risk assessment efforts given that about 90 percent of facilities are regulated because of the theft or diversion security issue. ISCD officials said that given the complexity of assessing threat for theft or diversion, they are considering reexamining their approach.

Regarding the other 10 percent—facilities tiered because of the risk of release or sabotage—ISCD documents showed that both models consider threat data based primarily on the location of the facility. Nonetheless, ISCD could use more current data to estimate threat among these facilities. Our review showed that ISCD is using 5-year-old threat data based on metropolitan statistical areas (MSA) to estimate threat for those facilities even though these data are updated annually by DHS for purposes of the Urban Areas Security Initiative program. ISCD officials said that they were unaware that threat data they were using were out of date and said they would explore the feasibility of using updated threat scores. Current threat data would provide a more complete and accurate threat profile for release or sabotage and might aid in ISCD’s overall risk assessment efforts.

ISCD’s risk assessment approach is also not consistent with the NIPP because it does not consider vulnerability when developing risk scores.

According to the NIPP, risk assessments should identify vulnerabilities, describe all protective measures, and estimate the likelihood of an adversary’s success for each attack scenario. Similar to the NIPP, the CFATS rule calls for ISCD to review facilities security vulnerability assessments as part of its risk-based tiering process. This assessment is to include the identification of potential security vulnerabilities and the identification of existing countermeasures and their level of effectiveness in both reducing identified vulnerabilities and meeting the aforementioned risk-based performances standards.

Our review of the risk assessment approach and discussions with ISCD officials showed that the security vulnerability assessment—the primary CSAT application ISCD uses to assess risk—contains numerous questions aimed at assessing vulnerability and security measures in place. These include questions about the accessibility of the facility to an attacker, the capability of the security force to respond to an attack, and security controls related to potential cyber attacks. However, although facilities are required to respond to these questions, ISCD officials told us that they have opted not to use the data facilities provide because it is “self-reported” data—data that are not validated by ISCD—and ISCD officials have observed that facility owners and operators tend to either overstate or understate some of the vulnerability information provided; thus making it not useful for tiering purposes. ISCD officials agreed that the risk assessment approach does not assess differences in vulnerability from facility to facility and location to location because it does not use any vulnerability data. Thus, ISCD’s risk assessment approach treats every facility as equally vulnerable to a terrorist attack regardless of location and on-site security.

ISCD officials told us that they consider facility vulnerability, but primarily at the latter stages of the CFATS regulatory process particularly with regard to the development and approval of the facility site security plan and the inspection process. With regard to site security plans, ISCD officials stated that even though facility data are not currently used to tier facilities based on their response in the security vulnerability assessment, they view the responses as valuable because they prompt facilities’ thinking about vulnerability before they prepare their site security plan or alternative security program. Regarding inspections, ISCD officials stated that they believe that once security plans are authorized and approved, the inspection process could enable ISCD to assess facilities’ vulnerabilities and gauge their progress mitigating those vulnerabilities. Because ISCD has completed a limited number of authorization inspections (56 as of December 2012), it is too early to tell how they plan to use this self-reported vulnerability information. However, ISCD officials indicated that it might be used to help make decisions about the use of inspection resources, especially since they do not anticipate retiering facilities based on their efforts to mitigate risk.

ISCD Has Begun to Take Actions to Examine How its Approach Could be Enhanced and Could Take Additional Steps to Help Ensure That it is Complete and Validated

ISCD has begun to take some actions to examine how its risk assessment approach can be enhanced. For example, in addition to engaging Sandia National Laboratories to develop the framework for assessing economic consequences discussed earlier, ISCD has commissioned a panel of subject matter experts to examine the strengths and weaknesses, if any, of its current risk assessment approach. ISCD officials stated that the panel’s work is intended to focus on whether ISCD is heading in the right direction and they view it as a preliminary assessment. According to ISCD’s task execution plan, the objectives of this assessment are to (1) convene a panel of subject matter experts involved in chemical safety and security, (2) hold one or more working group meetings focused on assessing and providing feedback on the current models and (3) provide a report on the strengths, weaknesses, and issues on the current models. The plan calls for the panel to provide actionable recommendations on potential improvements to the CFATS models, but the panel is not to develop alternative CFATS models nor formally validate or verify the current CFATS risk assessment approach— steps that would analyze the structure of the models and determine whether they calculate values correctly. ISCD officials stated that they believe that the review process would include some steps to assess whether the models are methodologically sound and reliable. In February 2013, after the panel was convened, ISCD officials also stated that they provided information to the panel about various issues that they might want to consider, among them (1) how to address vulnerability in the models given ISCD concerns about data quality and (2) what the appropriate variables to use, if any, are for threats associated with theft or diversion, as discussed earlier.

ISCD is moving in the right direction by commissioning the panel to identify the strengths and weaknesses of its risk assessment approach and the results of the panel’s work could help ISCD identify issues for further review and recommendations for improvement. The results of the panel’s efforts represent one piece of information ISCD will have to consider, moving forward, to ensure that the risk assessment approach is complete within the context of the NIPP risk management framework and the CFATS rule. For instance, in addition to any recommendations coming out of the panel’s work, the development of a mature risk assessment approach would require that ISCD consider and act upon the results of Sandia National Laboratories work on economic consequences. Likewise, ISCD would need to consider the issues we identified, such as not using up-to-date threat data, or how vulnerability could be used in the final tiering process.

ISCD will need to develop an overall plan designed to incorporate the results of these various efforts to revise and enhance its risk assessment approach to fully address each of the components of risk— consequences, threat, and vulnerability—to better align them with the NIPP and the CFATS rule. A plan, complete with milestones and time frames, is consistent with standard practices for project management, which state that managing a project involves, among other things, developing a timeline with milestone dates to identify points throughout the project to reassess efforts under way to determine whether project changes are necessary. ISCD would then be better situated to provide a more complete picture of its approach for developing and completing its review of steps needed to address each component of ISCD’s risk assessment approach and actions needed to make it fully conform to the NIPP and the CFATS rule. It also would provide ISCD managers and other decision makers with insights into (1) ISCD’s overall progress and (2) a basis for determining what, if any, additional actions need to be taken.

In addition, given the significant consequences of a terrorist attack on a chemical facility, after ISCD completes these actions, commissioning an independent peer review to assess its revised risk assessment approach, including a complete verification and validation of the models would help ensure that the revised model is sound and facilities are appropriately tiered. In our past work, we reported that peer reviews are a best practice in risk management and that independent expert review panels can provide objective reviews of complex issues. We reported that peer reviews should, among other things, address the structure of the model, the types and certainty of the data, and how the model is intended to be used. Furthermore, the National Research Council of the National Academies has recommended that DHS improve its risk analyses for infrastructure protection by validating the models and submitting them to external peer review. According to the National Research Council of the National Academies, peer reviews should include validation and verification to ensure that the structure of the models is both accurate and reliable.

As we have previously reported, independent peer reviews cannot ensure the success of a risk assessment approach, but they can increase the probability of success by improving the technical quality of projects and the credibility of the decision-making process. Thus, a peer review that is commissioned after ISCD revises its approach and incorporates all of the elements of risk would enable peer reviewers to consider a more complete risk assessment approach and provide the opportunity to fully verify and validate it. After ISCD has developed a more mature risk assessment approach, a subsequent peer review would provide better assurance that ISCD can appropriately identify and tier chemical facilities, better inform CFATS planning and resource decisions; and provide the greatest return on investment consistent with the NIPP.

ISCD Revised Its Security Plan Review Process, but Plan Approvals Could Take Years

ISCD has revised its site security plan review process to address concerns expressed by ISCD managers that the original process was overly complicated and included bottlenecks that slowed the review time. ISCD officials said that they believe the current security plan review process, implemented in July 2012, is an improvement over the prior versions. However, they did not collect or track data on the prior review processes, so the improvement between the previous review processes and the current process cannot be measured. Going forward, ISCD has recently implemented a plan to measure various aspects of the process, but it will take time before ISCD can establish baseline measures. Nonetheless, given the rate at which ISCD intends to review and approve security plans, we estimate that it could take about 7 to 9 years to complete reviews of plans for approximately 3,120 facilities that, as of January 2013, had been assigned a final tier but had not yet had their security plans reviewed and approved.

ISCD Revised Its Security Plan Review Process because of ISCD Managers’ Concerns

ISCD has made various revisions to its security plan review process. Under the CFATS rule, once a facility is assigned a final tier, it is to submit a site security plan to describe security measures to be taken and how they will address applicable risk-based performance standards. The November 2011 internal memorandum that discussed various challenges facing the program noted that ISCD had not approved any security plans and stated that the process was overly complicated, did not leverage ISCD’s resources, and created bottlenecks. In addition, the memorandum stated that revising the process was a top program priority because the initial security plan reviews were conducted in a manner inconsistent with the spirit and intent of the CFATS authorizing legislation—that is, plan reviewers used the risk-based standards as prescriptive criteria rather than as standards for developing an overall facility security strategy. According to ISCD, the initial reviews were conducted using the 18 risk-based standards as prescriptive criteria because ISCD had not developed guidance for reviewers of facility plans to use when considering the merits of those plans. ISCD officials told us that they had been working on a solution prior to the internal memorandum being finalized in November 2011. They also pointed out that the action plan that was intended to address the challenges outlined in the memorandum, developed in early 2012, included an action item devoted to improving the security plan review process.

ISCD has implemented two revisions to the security plan review process since October 2011. According the ISCD officials, the first revision was called the interim review process, which was intended to be a “holistic” review whereby individual reviewers were to consider how layers of security measures met the intent of each of the 18 standards. This was a departure from the original review process which generally used the performance standards as specific criteria. Under the interim process, ISCD assigned portions of each facility’s plan to security specialists (e.g., cyber, chemical, and physical, among others) who reviewed plans in a sequential, linear fashion. Using this approach, plans were reviewed by different specialists at different times culminating in a quality review. ISCD officials told us that the interim process was unsustainable, labor- intensive, and time-consuming, particularly when individual reviewers were looking at pieces of thousands of plans that funneled to one quality reviewer.

In July 2012, ISCD stopped using the interim process and began using the revised review process. The current process entails using contractors, teams of ISCD employees (physical, cyber, chemical, and policy specialists), and ISCD field office inspectors who are to review plans simultaneously using the holistic approach developed earlier. Figure 2 shows the revised security plan review process as of July 2012.

ISCD officials said that they believe the revised security plan review process is a “quantum leap” forward, but did not capture data that would enable them to measure how, if at all, the new process is more efficient (i.e., less time-consuming) than the former processes. ISCD officials explained that one of the more time-saving beneficial aspects of the new process involves field inspectors interacting with the facilities when the review of the security plan results in an unfavorable outcome. Now, when ISCD identifies a security plan that contains deficiencies, such as missing or unclear information about a security measure, the plan is to be immediately returned to the facility and ISCD is to schedule a compliance assistance visit whereby field inspectors work with the facility to resolve any issues identified. According to ISCD officials, this approach contrasts with the past practices whereby ISCD would continue to review the entire plan even when problems were identified early and not return the plan to the facility until the review was complete, resulting in longer reviews.

Officials also noted that by using the revised process, ISCD has realized the value of (1) moving from a single person reviewing every plan sequentially to a team approach, and (2) understanding that security plans do not have to be perfect in order to issue authorization letters and conduct authorization inspections. Regarding the latter, ISCD officials noted that ISCD has begun issuing authorization letters with conditions to inform facilities that their plans provide sufficient information to schedule an inspection. For example, one authorization letter noted that ISCD had not yet determined whether or not the plan satisfied the cyber security risk-based performance standard, and stated that additional information would be gathered during the authorization inspection.

Also, when the revised process was implemented in July 2012, all authorization letters include a condition noting that ISCD has not fully approved the personnel surety risk-based performance standard of plans because ISCD has not yet determined what the facilities are to do to meet all aspects of personnel surety. ISCD believes issuing authorization letters with conditions, rather than waiting until all conditions are met, enables inspectors to visit facilities sooner so that ISCD can approve plans more quickly.

Moving forward, ISCD intends to measure the time it takes to complete parts of the new process and has recently implemented a plan to measure various aspects of the process. Specifically, ISCD’s Annual Operating Plan, published in December 2012, lists 63 performance measures designed to look at various aspects of the site security plan review process—from the point the plans are received by ISCD to the point where plans are reviewed and approved. For example, ISCD plans to collect data on (1) the percentage of facilities with authorization inspections completed within 90 days of security plan authorization for tier 1 and 2 facilities and within 120 days of security plan authorization for tier 3 and 4 facilities, and (2) the number of high-risk facilities in total and by tier that have approved security plans. Collecting data to measure performance about various aspects of the security plan review process is a step in the right direction, but it may take time before the process has matured to the point where ISCD is able to establish baselines and assess its progress. As of February 2013, ISCD is beginning to gather data at points in the process to establish baselines and measure performance and has a goal of reviewing some of the measures and data associated with them monthly, quarterly, or annually, depending on the measure.

Security Plan Reviews Could Take Years to Complete, but ISCD is Examining How it Can Accelerate the Review Process

ISCD actions to revise its security plan reviews may result in improvements over the prior processes, but it could take years to review plans for thousands of facilities that have been assigned a final tier—a factor which ISCD hopes to address by examining how it can accelerate the review process. According to ISCD officials, between July 2012 and December 2012, ISCD had approved 18 security plans, with conditions such as the aforementioned personnel surety qualification. ISCD officials told us that, moving forward, they anticipate that the revised security plan review process could enable ISCD to approve security plans at a rate of about 30 to 40 a month. Furthermore, ISCD officials noted that the approval rate could reach 50 plans a month in the third quarter of fiscal year 2013 as the review process becomes more efficient. However, ISCD estimates that under a best case scenario the revised review process could take about 6 months to approve a plan—assuming the plan would not have to be sent back to the facility for revision. ISCD estimates further show that under a worst case scenario the revised process could take as long as 20 months to approve a plan—assuming the plan would have to be sent back to the facility for revisions. Regardless, ISCD officials told us that they would likely be able to increase production because staff are overcoming the learning curve associated with the revised process.

Using ISCD’s estimated approval rate of 30 to 40 plans a month, we calculated that it could take anywhere from 7 to 9 years to complete reviews and approvals for the approximately 3,120 plans submitted by facilities that have been final-tiered that ISCD has not yet begun to review. Figure 3 shows the estimated number of years it could take to approve all of the security plans for the approximately 3,120 facilities that, as of January 2013, had been final-tiered assuming an approval rate of 30 to 40 plans a month.

It is important to note that our 7 to 9 year estimate does not include other activities central to the CFATS mission, either related to or aside from the security plan review process. Specifically, our estimate does not include time required to: review security plans for about 900 facilities that have yet to be assigned a final tier; and review approved security plans to resolve issues relating to personnel surety, which cannot be fully accomplished until after ISCD decides how to conduct the terrorist ties portion of personnel surety.

Finally, our estimate does not include developing and implementing the compliance inspection process, which is intended to ensure that facilities that are covered by the CFATS rule are compliant with the rule, within the context of the 18 performance standards. ISCD officials estimate that the first compliance inspections would commence in September 2013, which means that the CFATS regulatory regime would likely be fully implemented for currently tiered facilities (to include compliance inspections) in 8 to 10 years. According to ISCD officials, they are actively exploring ways to expedite the speed with which the backlog of security plans will be cleared such as potentially leveraging alternative security programs, re-prioritizing resources, and streamlining the inspection and review requirements. ISCD officials added that they plan to complete authorizations inspections and approve security plans for tier 1 facilities by the first quarter of fiscal year 2014 and for tier 2 facilities by the third quarter of fiscal year 2014.

ISCD Has Increased its Efforts to Communicate and Work with Facilities and May Have an Opportunity to Systematically Gather Feedback on Its Outreach Efforts

ISCD’s efforts to communicate and work with owners and operators to help them enhance security at their facilities have increased since the CFATS program’s inception in 2007, particularly in recent years. The various trade associations representing facility owners and operators who responded to our query on ISCD’s outreach had mixed views about the effectiveness of ISCD’s efforts to communicate with them over various aspects of the program. Most of the trade associations that responded stated that ISCD seeks and receives informal feedback on its communication efforts, but ISCD stated that it has not developed a systematic approach to solicit feedback to assess the effectiveness of its outreach activities. ISCD is currently developing a strategic communication plan which may create an opportunity for ISCD to explore how it can obtain systematic feedback on its outreach.

ISCD’s External Communication Efforts with Facilities have Increased Since 2007

Since 2007, ISCD has taken various actions to communicate with facility owners and operators and various stakeholders—including officials representing state and local governments, private industry, and trade associations—to increase awareness about CFATS and these efforts have increased as the program has matured. From fiscal years 2007 through 2009, most of ISCD’s communication efforts entailed outreach with owners and operators and stakeholders through presentations to familiarize them with CFATS; field visits with federal, state, and local government and private industry officials; and compliance assistance visits at facilities that are intended to assist them with compliance or technical issues. By 2010 and in subsequent years, ISCD revised its outreach efforts to focus on authorization inspections during which inspectors visited facilities to verify that the information in their security plans was accurate and complete and other outreach activities including stakeholder outreach. Table 2 shows the number of outreach activities performed by ISCD from fiscal year 2007 through the first quarter of fiscal year 2013.

According to ISCD officials, the increase in outreach was intended to facilitate the development of site security plans and occurred for various reasons. First, according to ISCD officials, during the early years of the program, regulated facilities did not require as much assistance because they were generally engaged in the development of their Top Screens and security vulnerability assessments. Second, officials said that, as ISCD matured, its ability to track outreach activities improved when ISCD’s tracking system was automated in June 2010—during the early years of the program, outreach reporting was manual. Third, ISCD officials stated that ISCD staffing increases made it possible to perform more outreach to regulated stakeholders. They said that, prior to fiscal year 2009, field staff consisted of about 30 staff detailed from DHS’s Federal Protective Service and in subsequent years ISCD increased the size of its field staff to more than 100.

Selected Trade Associations Had Mixed Views about ISCD Efforts to Communicate with Owners and Operators

Our analysis of industry trade associations’ responses to questions we sent them about the program showed mixed views about ISCD’s efforts to communicate with owners and operators through ISCD outreach efforts. Whereas 3 of the 11 trade associations that responded to our questions indicated that ISCD’s outreach program was effective in general, 3 reported that the effectiveness of ISCD’s outreach was mixed, 4 reported that ISCD’s outreach was not effective, and 1 respondent reported that he did not know. Our analysis also showed that trade associations that responded, in general, viewed specific types of ISCD outreach to be either effective or of mixed effectiveness (fig. 4 shows our analysis of trade association responses to questions about specific types of ISCD’s outreach activities).

We also analyzed trade association responses with regard to the usefulness of ISCD outreach activity in terms of their members’ understanding of performance standards, tiering approach, and data collection requirements, as shown in figure 5.

Our analysis of trade association responses to our questions showed that opinions were mixed regarding the usefulness of outreach activities related to helping regulated facilities understand the Risk-Based Performance Standards. Specifically, trade association responses showed that 5 of the 11 associations that responded indicated that outreach was useful or had mixed usefulness and 4 associations reported that it was not useful. For example, among those trade associations that indicated that outreach on the performance standards was useful, one reported that outreach activities have helped members better understand the standards and another reported that performance standards guidance and early presentations were also helpful. Conversely, among those that indicated they did not believe outreach on the Risk-Based Performance Standards was useful, one cited a lack of ISCD training on the standards and another reported that ISCD needed to provide more assistance to explain what would or would not be compliant using the standards.

Nearly all (9 of 11) trade association respondents indicated that ISCD’s outreach to members was not useful in helping them understand the tiering approach used to determine the risk levels of regulated facilities, which they viewed as not being transparent. respondents reported that the lack of transparency can hinder facility owners and operators’ ability to properly make risk reduction decisions; and 6 respondents reported that there are instances where tiering results contradict the facility’s perception of what is “high risk” or conflict with results at similar facilities with similar chemical holdings and population densities in surrounding communities. One member company of 1 trade association reported that industry has repeatedly asked for information on the tiering process without success. ISCD officials told us that they do not currently provide regulated facilities with details about the CFATS risk- tiering approach but noted that one of the tasks of the aforementioned expert panel review is to make recommendations regarding what additional tiering-related information should be provided to facilities. DHS has designated some parts of the risk models as Chemical-terrorism Vulnerability Information—which warrants special treatment for handling and sharing, including assessing whether or not there is a need to know—and other parts of the models as Secret. ISCD officials also stated that they have not received a formal request by facilities asking ISCD to recheck assigned tiering levels or to re-evaluate the output of the risk models because facilities believe the methodology used is faulty.

One respondent said that he was not well positioned to comment on the usefulness of ISCD’s tiering-related outreach activities because of a lack of information from owners and operators the association represented. Another respondent indicated that that the association was not aware that ISCD had conducted outreach in this area.

According to these officials, the primary request from facilities is to know how their facilities were tiered, and industry requests for information mainly come from facilities that either believe they should not be tiered or tiered facilities that are questioning why a neighboring facility has not been tiered.

Our query of selected trade associations also showed that they generally had mixed views about ISCD’s outreach on data collection requirements. Specifically, 4 of the 11 association officials that provided responses to our query reported that ISCD’s outreach efforts had either been useful or had been of mixed usefulness in enabling them to better understand and comply with the ISCD’s data collection requirements; 3 reported that outreach on data collection requirements had not been useful; and 4 did not answer the question. Among the 4 respondents that found outreach on data collection requirements to be useful or of mixed usefulness, 1 noted that ISCD had done more outreach specifically on elements of the site security plan, which has broadened stakeholders understanding of the type of information DHS is looking for and options they may not have previously considered. Of the 3 that reported ISCD’s outreach on data collection requirements had not been useful, 1 trade association respondent reported that, in 2009, industry representatives had suggested that ISCD produce a document providing tips and suggestions for completing site security plans and reported that it took 3 years for ISCD to produce such a document; similarly, 2 member companies of another trade association reported that ISCD’s outreach had not been useful in increasing understanding of data collection requirements, and 1 cited the experience of an individual that seemed confused on much of the information being requested.

Trade Associations We Contacted Expressed Concern about the Burden of Responding to Data Collection Requirements which ISCD Has Plans to Address

Most of the trade associations that responded to our questions also expressed concern about CSAT data collection requirements specifically with regard to various applications such as the Top Screen, vulnerability assessment, and site security plan. Specifically, our analysis of responses by the 11 trade associations indicated that nearly all (9 of 11 respondents) believed that the CSAT data collection effort was burdensome for regulated owners and operators; the other 2 trade associations provided mixed responses. Nine industry trade associations reported that the CSAT data submission requirements take significant resources (such as time and personnel) to prepare, and 10 questioned the value of the tools in reducing risk or increasing security.association official reported that completing the Top Screen, vulnerability assessment, and security plan data collection effort required over 200 person hours. Another association official reported that the vulnerability assessment data collection requirements were very burdensome and noted that one member’s security plan covered 1,400 questions and was nearly 300 pages long.

ISCD officials acknowledged that CSAT can be burdensome and they intend to introduce improvements to CSAT to assist facilities in developing and submitting their Top Screens, security vulnerability assessments, and site security plans. In ISCD’s December 2012 Annual Operating Plan, one action item calls for the revision of the CSAT based on engagement with industry in order to create a more efficient and effective tool. ISCD officials stated that they estimate that revisions to CSAT will be in place sometime in 2014. According to these officials, improvements being considered for CSAT applications include a reduction in the number of text-based responses and narrative information required in the vulnerability assessment and the site security plan; the inclusion of more drop-down menu options, which is also expected to improve data analysis; and a reduction of the number of repetitive questions, for example, in the site security plan. ISCD officials noted that CSAT can be burdensome when facilities have to reenter data from one document to the next and stated that they are looking to revise CSAT so that information already entered on one document automatically populates data fields covering the same question in the next. ISCD officials also told us that one of the actions ISCD plans to take is to hold three meetings, or roundtable discussions, with industry officials at various locations beginning at the end of February 2013. They said that these discussions are intended to obtain input from industry officials on how CSAT can be improved. ISCD officials said that in making revisions to the CSAT they will consider eliminating unnecessary data requirements, but stated that they may decide to continue to request the data—even if they are not used for risk tiering—because doing so may be helpful to facilities as they prepare their security plans.

ISCD Does Not Seek Systematic Feedback on the Effectiveness of Its Outreach Efforts

ISCD seeks informal feedback on its outreach efforts but does not systematically solicit feedback to assess the effectiveness of outreach activities, and it does not have a mechanism to measure the effectiveness of ISCD’s outreach activities. Trade association officials reported that in general ISCD seeks informal feedback on its outreach efforts and that members provide feedback to ISCD. Association officials further reported that ISCD has encouraged association members to contact local ISCD inspectors and has hosted roundtable discussions and meetings where members of the regulated community provide feedback, suggest improvements, or make proposals regarding aspects of the CFATS program such as site security plans, alternative security programs, gasoline storage site risks, and the personnel surety program. Furthermore, according to ISCD officials, while feedback is solicited from the regulated community generally on an informal basis, inspectors and other staff involved in ISCD’s outreach activities are not required to solicit feedback during meetings, presentations and assistance visits, and inspectors are also not required to follow-up with the facilities after assistance visits to obtain their views on the effectiveness of the outreach.

ISCD, as part of its annual operating plan, has a priority for fiscal year 2013 to develop a strategic communications plan intended to address both internal and external communication needs including industry outreach. One goal in the plan is to maintain robust communication and outreach. In addition, the annual operating plan contains 27 monitoring and performance measures that address outreach program activities, but only one of these calls for ISCD to solicit feedback to assess and measure the effectiveness of the program. The NIPP states that when the government is provided with an understanding of private sector information needs, it can adjust its information collection, analysis, synthesis, and dissemination activities accordingly. We have previously reported on the benefits of soliciting systematic feedback. Specifically our prior work on customer service efforts in the government, systematic feedback from regulated facility owners and operators to among other things, determine the kind and quality of services they want and also determine their level of satisfaction with existing services including outreach may benefit to organizations like ISCD that service the public.

As ISCD develops its strategic communication plan, options for gathering feedback on outreach possibly could include using formal surveys reviewed and approved by the Office of Management and Budget (OMB) soliciting feedback from regulated under the Paperwork Reduction Act; owners and operators as a part of after-action reviews conducted at assistance visits, meeting and presentations; working with trade associations or other representatives of the regulated community to design and conduct member surveys; or working with members of various critical infrastructure sectors, such as the chemical or energy sectors, to develop and conduct surveys of sector owners and operators. Given the mixed perspectives of the trade associations we queried, systematic feedback about outreach activities might better position ISCD to identify problems and target changes to its outreach efforts, if necessary, or improve CFATS program outcomes in general. Doing so would also be consistent with Standards for Internal Control in the Federal Government, which call for top level reviews of actual performance and the establishment and review of performance measures and indicators.

Conclusions

ISCD has taken action to assign facilities to risk based tiers, revise its process to review site security plans, and work with facilities to improve security. However, three factors could affect program operations as ISCD moves forward:

The first factor is a risk assessment approach that is not yet complete because it does not consider all of the elements of risk called for by the NIPP and the CFATS rule. ISCD has begun to take some actions to develop a more robust risk assessment approach, but ISCD would be better positioned to assess risk if it developed an overall plan, with milestones and time frames, incorporating the results of the various efforts to more fully address each of the three components of risk— consequence, threat, and vulnerability—and take actions to enhance the current risk assessment approach. ISCD has commissioned an expert panel to identify the strengths and weaknesses of the current risk assessment approach and the results of the panel’s work could help ISCD identify issues for further review and recommendations for improvements. This effort and the results from it represent one component of the various efforts ISCD will have to consider moving forward to ensure that the risk assessment approach is complete per the NIPP and the CFATS rule. After ISCD has developed and completed its efforts to enhance its risk assessment approach by using the results of the current expert panel’s efforts as well as incorporating the issues we identified along with the Sandia National Laboratories’ work on economic consequences, an independent peer review would provide better assurance that ISCD can appropriately identify and tier chemical facilities, better inform CFATS planning and resource decisions; and provide the greatest return on investment consistent with the NIPP and CFATS rule.

The second factor is ISCD’s ability to measure its progress reviewing site security plans under its revised review process. ISCD has developed measures to assess its progress reviewing site security plans and has recently implemented a plan to measure various aspects of the process. ISCD’s efforts appear to be a step in the right direction, but, it will take time for ISCD to collect enough data to develop baselines and begin measuring its performance. While it could take years before ISCD can review and approve the site security plans currently in its queue, ISCD intends to explore ways that it can accelerate the process. Thus, we are not making recommendations at this time.

The third factor is exploring opportunities to establish a mechanism to systematically gather feedback to measure the effectiveness of ISCD outreach efforts with facility owners and operators. This includes ISCD efforts to communicate with owners and operators on various aspects of the program, such as the development of site security plans, and work with them to better understand how and whether data collection requirements are burdensome and can be reduced. Doing so would be consistent with (1) the NIPP which states that once the government understands private sector information needs, it can adjust its information collection, analysis, synthesis, and dissemination activities accordingly, and (2) our past work on the benefits of soliciting feedback to determine customer level of satisfaction with existing services, including outreach. In addition, by obtaining systematic feedback on its outreach efforts, ISCD might be better positioned to (1) identify problems and target program changes, if necessary, and generally improve ISCD efforts to communicate and work with facility owners and operators, and (2) measure its performance consistent with our Standards for Internal Control in the Federal Government.

Recommendations for Executive Action

To better assess risk associated with facilities that use, process, or store chemicals of interest consistent with the NIPP and the CFATS rule, we recommend the Secretary of Homeland Security direct the Under Secretary for NPPD, the Assistant Secretary for IP, and Director of ISCD to take the following two actions: develop a plan, with timeframes and milestones, that incorporates the results of the various efforts to fully address each of the components of risk and take associated actions where appropriate to enhance ISCD’s risk assessment approach consistent with the NIPP and the CFATS rule, and conduct an independent peer review, after ISCD completes enhancements to its risk assessment approach, that fully validates and verifies ISCD’s risk assessment approach consistent with the recommendations of the National Research Council of the National Academies.

To enhance ISCD efforts to communicate and work with facilities, we recommend that the Secretary of Homeland Security direct the Under Secretary for NPPD, the Assistant Secretary for IP, and the Director of ISCD to explore opportunities and take action to systematically solicit and document feedback on facility outreach consistent with ISCD efforts to develop a strategic communication plan.

Agency Comments and Our Evaluation

We provided a draft of this report to the Secretary of Homeland Security for review and comment, which are reprinted in appendix II. In its written comments, DHS agreed with our recommendations and stated that it has efforts underway that will address them. Regarding our first recommendation that ISCD develop a plan, with timeframes and milestones, to fully address each of the components of risk and take associated actions where appropriate to enhance ISCD’s risk assessment approach consistent with the NIPP and the CFATS rule, DHS plans to document all processes and procedures related to the risk assessment approach and conduct an internal DHS review of the complete risk assessment process, among others things, to ensure that all elements of risk are included in the risk assessment approach. Regarding our second recommendation that ISCD conduct an independent peer review, after it completes enhancements to its risk assessment approach, that fully validates and verifies ISCD’s risk assessment approach consistent with the recommendations of the National Research Council of the National Academies, DHS agreed that a peer review that includes validation and verification steps would be a worthwhile endeavor, once any changes that result from the aforementioned review of the risk assessment approach are implemented. Regarding our third recommendation that ISCD explore opportunities and take action to systematically solicit and document feedback on facility outreach consistent with ISCD efforts to develop a strategic communication plan, DHS plans to explore such opportunities to make CFATS-related outreach efforts more effective for all stakeholders. DHS also provided technical comments, which we incorporated as appropriate.

We are sending copies of this report to interested congressional committees and to the Secretary of the Department of Homeland Security. In addition, the report will be available at no charge on GAO’s website at http://www.gao.gov. If you or your staff members have any questions about this report, please contact me at (202) 512-9610 or caldwells@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

This report is a follow-on engagement of work we completed in July 2012 about the actions the Department of Homeland Security’s (DHS) Infrastructure Security Compliance Division (ISCD) took related to an internal memorandum that cited an array of challenges—including human capital and administrative issues—that hindered the implementation of the Chemical Facility Anti-Terrorism Standards (CFATS) program. This report discusses the extent to which DHS has assigned chemical facilities to risk-based tiers and assessed its approach for doing so, revised the process to review security plans, and communicated and worked with facilities to help improve security.

To determine the extent to which DHS has assigned chemical facilities to risk-based tiers and assessed its approach for doing so, we reviewed ISCD applications and documents including web-based Chemical Security Assessment Tools (CSAT) applications—such as the Top- Screen and security vulnerability assessment—used to collect security information from facilities, the ISCD risk assessment approach used to determine a facility’s risk tier, policies and procedures on tiering, as well as a sample copy of a facility’s Top Screen and security vulnerability assessment. In addition, we outlined the risk tiering models associated with different security issues—release, theft or diversion, and sabotage. We also reviewed DHS memoranda detailing the challenges with the release security issue model and ISCD’s work to improve it. We compared our review to various criteria including the risk framework outlined in the CFATS statute and rule, the National Infrastructure Protection Plan (NIPP), and risk modeling best practices as outlined by the National Research Council of the National Academy of Sciences to determine if ISCD’s risk assessment approach comports with these criteria and if not, where gaps or deficiencies exist. We also obtained data from ISCD’s CSAT system that showed how many of the roughly 3,500 facilities, which ISCD has determined to be regulated by CFATS, were placed in each of the 4 risk-based tiers and what their related security issue(s) was. Because data in the system change regularly, these data represent a snapshot of finally tiered facilities as of January 24, 2013. To assess the reliability of the data we obtained from CSAT, we reviewed system documentation, compared similar datasets for consistency, and interviewed knowledgeable ISCD officials about system controls and determined that the CSAT data were sufficiently reliable for the purposes of this report. We also reviewed documents related to ISCD’s ongoing review of the risk assessment approach including the statement of objectives, task execution plan, and terms of reference and compared these documents with the criteria for peer review as laid out by the National Research Council of the National Academy of Sciences as well as past GAO work on peer review.understanding of the risk assessment approach, we interviewed DHS and contracting officials knowledgeable about the methodology behind the models and their strengths and weaknesses. We also interviewed DHS and contracting officials responsible for the review about the review’s scope and progress, and to corroborate and confirm our understanding of the review.

To corroborate and confirm our To determine the extent to which DHS has revised its process to review security plans, we reviewed and analyzed documents, where available, including the CFATS statute and rule, the November 2011 internal memorandum, DHS’s Risk-Based Performance Standards Guidance, ISCD security plan review policies and procedures, security plan review memoranda, and the instructions used by facilities to prepare and submit security plans. As a part of our review of security plan documents, we identified progress made and challenges encountered by ISCD as the security plan review process evolved. To confirm our understanding of the security plan review process and how it has evolved, we gathered and analyzed statistics pertinent to the process to determine how many site security plans had been reviewed, authorized, and approved as a percent of the total number of security plans submitted. We then used the number of plans approved, along with ISCD officials’ estimates of how many plans they intend to approve per month beginning in calendar year 2013 to estimate how long it might take ISCD to review and approve site security plans for the approximately 3,500 currently regulated facilities. We also interviewed ISCD officials about the security plan review process to corroborate the information obtained from ISCD documents and the results of our analyses.

To determine the extent to which DHS has communicated and worked with facilities to help improve security, we reviewed documents (e.g., ISCD’s standard operating procedures for conducting compliance assistance visits, inspections, and help desk support). We also reviewed ISCD data on outreach activities—such as the type and number of field visits to facilities and presentations to industry—for fiscal year 2007 through November 30, 2012 that we obtained from CHEMS. As noted above, we determined that data from the CHEMS system were sufficiently reliable for the purposes of this report. In addition, we contacted officials representing 15 trade associations whose members are facilities regulated by CFATS to obtain their perspectives on DHS’s efforts to communicate and work with facility owners and operators to help improve security. We selected these 15 trade associations because they are listed in an annex to the NIPP as those with which DHS works on a regular basis on chemical security matters. According to this annex, working with these industry associations is a more manageable number of contact points through which DHS can coordinate activities with a large number of the asset owners and operators in the chemical sector. According to the NIPP, a Sector Coordinating Council is the principal entity under which owners and operators of critical infrastructure can coordinate with the government on a wide range of protection activities and issues. The Chemical Sector Coordinating Council represents a significant majority of the owners and operators in the chemical sector. We sent a list of open- ended questions to representatives at each of the 15 trade associations and received responses from 11 trade association representatives. We analyzed, categorized, and summarized these responses by using a systematic content analysis of the open-ended responses to determine the trade association views. As a part of our analysis, two analysts reviewed the responses, developed categories to be used for the content analysis, and worked together to categorize each open-ended answer. A third analyst reviewed this categorization and verified that the answers were placed in the appropriate categories. Any disagreements regarding the categorization of the answers were subsequently reconciled. The information we obtained from the 11 trade associations that responded is not generalizable to the universe of chemical facilities covered by CFATS; however, it does provide insights into DHS efforts to perform outreach and seek feedback on the implementation of the CFATS rule. We compared the results of our analysis of the responses received from the trade associations and other audit work related to ISCD’s outreach efforts to various criteria, including CFATS statute and rule, the NIPP, past GAO work on industry outreach, and internal control standards to determine if DHS’s outreach efforts comport with these criteria, and if not, determined where gaps exist. We also interviewed knowledgeable ISCD officials regarding their outreach efforts.

We identified three limitations that should be considered when using our results. First, as noted in our previous work, documentary evidence about the development of the CFATS program is, for the most part, not available. Program officials did not maintain records of key decisions made in the early years of the program and the basis for these decisions. This applies particularly to the risk tiering methodology and the security plan review process. Furthermore, ISCD officials told us many of the individuals involved in these decisions are no longer at ISCD. During discussions, the current management team qualified that much of what they told us about these decisions is their best guess of what happened and why. This limits our ability to fully assess the risk model and compare the original site security plan review process with the process currently in place. Second, with regard to our work with facility owners and operators, we limited our review to industry associations primarily due to time constraints and the large number of owners and operators in the chemical sector. As a result, we could not generalize our findings to the universe of CFATS-regulated facility owners and operators. We mitigated this limitation by contacting 15 industry associations that represented a wide range of CFATS regulated chemical facilities. Third, given that ISCD had only recently begun to approve security plans, our scope did not include the facility compliance inspection process (which is based on the results of the approved security plans).

We conducted this performance audit from October 2012 through April 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Department of Homeland Security

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, John F. Mortin, Assistant Director, and Ellen Wolfe, Analyst-in-Charge; Chuck Bausell; Jose Cardenas; Michele Fejfar; Jeff Jensen; Tracey King; Marvin McGill; made significant contributions to the work.

Related GAO Products

Critical Infrastructure Protection: An Implementation Strategy Could Advance DHS’s Coordination of Resilience Efforts across Ports and Other Infrastructure. GAO-13-11. Washington, D.C.: October 25, 2012.

Critical Infrastructure Protection: Summary of DHS Actions to Better Manage Its Chemical Security Program. GAO-12-1044T. Washington, D.C.: September 20, 2012.

Critical Infrastructure Protection: DHS Is Taking Action to Better Manage Its Chemical Security Program, but It Is Too Early to Assess Results. GAO-12-567T. Washington, D.C.: September 11, 2012.

Critical Infrastructure: DHS Needs to Refocus Its Efforts to Lead the Government Facilities Sector. GAO-12-852. Washington, D.C.: August 13, 2012.

Critical Infrastructure Protection: DHS Is Taking Action to Better Manage Its Chemical Security Program, but It Is Too Early to Assess Results. GAO-12-515T. Washington, D.C.: July 26, 2012.

Critical Infrastructure Protection: DHS Could Better Manage Security Surveys and Vulnerability Assessments. GAO-12-378. Washington, D.C.: May 31, 2012.

Critical Infrastructure Protection: DHS Has Taken Action Designed to Identify and Address Overlaps and Gaps in Critical Infrastructure Security Activities. GAO-11-537R. Washington, D.C.: May 19, 2011.

Critical Infrastructure Protection: DHS Efforts to Assess and Promote Resiliency Are Evolving but Program Management Could Be Strengthened. GAO-10-772. Washington, D.C.: September 23, 2010.

Critical Infrastructure Protection: Update to National Infrastructure Protection Plan Includes Increased Emphasis on Risk Management and Resilience. GAO-10-296. Washington, D.C.: March 5, 2010.

The Department of Homeland Security’s (DHS) Critical Infrastructure Protection Cost-Benefit Report. GAO-09-654R. Washington, D.C.: June 26, 2009.

Information Technology: Federal Laws, Regulations, and Mandatory Standards to Securing Private Sector Information Technology Systems and Data in Critical Infrastructure Sectors. GAO-08-1075R. Washington, D.C.: September 16, 2008.

Risk Management: Strengthening the Use of Risk Management Principles in Homeland Security. GAO-08-904T. Washington, D.C.: June 25, 2008.

Critical Infrastructure: Sector Plans Complete and Sector Councils Evolving. GAO-07-1075T. Washington, D.C.: July 12, 2007.

Critical Infrastructure Protection: Sector Plans and Sector Councils Continue to Evolve. GAO-07-706R. Washington, D.C.: July 10, 2007.

Critical Infrastructure: Challenges Remain in Protecting Key Sectors. GAO-07-626T. Washington, D.C.: March 20, 2007.

Homeland Security: Progress Has Been Made to Address the Vulnerabilities Exposed by 9/11, but Continued Federal Action Is Needed to Further Mitigate Security Risks. GAO-07-375. Washington, D.C.: January 24, 2007.

Critical Infrastructure Protection: Progress Coordinating Government and Private Sector Efforts Varies by Sectors’ Characteristics. GAO-07-39. Washington, D.C.: October 16, 2006.

Information Sharing: DHS Should Take Steps to Encourage More Widespread Use of Its Program to Protect and Share Critical Infrastructure Information. GAO-06-383. Washington, D.C.: April 17, 2006.

Risk Management: Further Refinements Needed to Assess Risks and Prioritize Protective Measures at Ports and Other Critical Infrastructure. GAO-06-91. Washington, D.C.: December 15, 2005.