The Department of Homeland Security’s Risk Assessment Methodology: Evolution, Issues, and Options for Congress
Released: 2007-02-02T00:00:00

Summary:
As early as his Senate confirmation hearing, Department of Homeland Security (DHS) Secretary Michael Chertoff advocated a risk-based approach to homeland security. Secretary Chertoff has stated "DHS must base its work on priorities driven by risk" and, increasingly, risk assessment and subsequent risk mitigation have influenced all of the department's efforts intended to enhance our nation's ability to prevent, respond to, and recover from future terrorist attacks and natural disasters. While the practice of risk analysis may be advanced in the insurance and financial industries, it is relatively less developed in the homeland security field. Although there are numerous reasons that account for this dynamic, two primary reasons include (1) the dynamic nature of terrorism and ability of terrorists to adapt to successful countermeasures, and (2) the lack of a rich historical database of terrorist attacks, which necessitates a reliance on intelligence and terrorist experts for probabilistic assessments of types of terrorist attacks against critical assets and/or regions. This report begins with an overview of the evolution of risk assessment methodologies from the Department of Justice in FY2002 to DHS in FY2007, and then discusses the discipline of risk management and risk assessment as applied to Homeland Security Grant Program (HSGP).
Terrorism risk analysis and assessment do not exist in a vacuum. Risk is analyzed and assessed as a means to mitigate or "buy down" risk over time by developing certain capabilities across the country. At DHS, the State Homeland Security Grant Program is the primary tool the agency has to influence the behavior of State and local partners to take actions that reduce what both parties agree are the risks of a terrorist attack and to respond effectively to such an attack, or other catastrophe. Regardless of the complexity of the risk assessment methodology, due to the inherent uncertainties associated with assessing risk in a dynamic counterterrorism context, some level of flexibility in managing risk may be necessary. Empirical data on historical terrorist attacks in the United States may, therefore, continue to play an important role in resource allocation to reduce risk.
This report presents several risk assessment and related grant program options for congressional consideration: (1) maintain the status quo in the inextricably linked areas of risk assessment and grant allocation, (2) draft a national impact assessment to understand return on investment of the approximately $12 billion of HSGP spent by FY2008, (3) enhance the transparency of the risk allocation methodology to state and local governments, and (4) develop a comprehensive and long-term strategy for managing, assessing and mitigating risk. To achieve these goals, the department could opt to consider procedural or organizational changes. Possible approaches are discussed in the report's final section. This report may be updated.

subsections: section_title: Introduction
paragraphs: As early as his Senate confirmation hearing, Homeland Security Secretary Michael Chertoff advocated a risk-based approach to homeland security. Under Secretary Chertoff's direction, the use of risk assessment has become pervasive throughout DHS. Increasingly, risk assessment and subsequent risk mitigation efforts influence many aspects of the department's work intended to enhance our nation's ability to prevent, respond to, and recover from future terrorist attacks and natural disasters. Indeed, Secretary Chertoff has stated "DHS must base its work on priorities driven by risk."
The purpose of this report is to analyze how DHS assesses risk. In the absence of sound risk assessment methods, the prioritization of homeland security activities at the federal, state, and local level is problematic. If DHS is to "prevent terrorist attacks within the United States," one of its primary statutory missions, it needs to assess risk in an accurate manner. However, risk assessment does not occur in a vacuum; the end goal is to reduce and mitigate risk. All of DHS's employees work to reduce risk, respond to a terrorist attack or natural disaster should one occur, and/or protect the country by preventing dangerous materials or individuals from crossing U.S. borders. The primary tool DHS has to "buy down" or minimize risk and to influence the behavior of State and local public safety and law enforcement officials who collectively represent substantial "force multipliers" is the Homeland Security Grant Program. Others have written extensively about DHS grant programs and the allocation of such programs across the country.
The purpose of this report is not to re-construct grant program research, but to examine the concept of DHS risk assessment itself and how the evolution of risk assessment flows through the DHS grant programs. The report has three sections (1) the evolution of risk assessment from the Department of Justice in FY2002 to DHS in FY2007, (2) fundamental questions about risk analysis as applied to homeland security, and (3) possible options for Congress. It will examine strategic questions about risk, and how risk is defined and distinguishable from other terms, such as vulnerability. Finally, the report will discuss a possible range of approaches for Congress with respect to DHS risk assessment practices, DHS's organization to assess risk, and the implementation of risk mitigation efforts using the DHS grant tool.
Given the focus of this report, an analysis of the DHS's risk assessment methodology through the lens of the homeland security grant process, some background information on the grant process is necessary. As previously stated, the risk assessment process cannot be examined in isolation. Rather, the context of the homeland security grant program is discussed to illuminate the homeland security risk assessment methodology and its implementation throughout various homeland security initiatives. This report may be updated.
section_title: Background
paragraphs: In FY2004, the allocation of homeland security grant monies inspired debate in states across the country. One often-reported anecdote noted that Wyoming's FY2004 State Homeland Security Grant Program (SHSG) award was $14,360,000, while New York and California received $78,827,000 and $133,964,000, respectively. On its face, it seemed intuitive that New York and California would receive more money than Wyoming. But when examined in light of 2004 census bureau estimates, it appears that Wyoming received approximately $28.34 in SHSG funding per capita while New York and California received $4.10 and $3.73 per capita, respectively. The rationale behind the disbursement of funds seemed counterintuitive to many, especially given the recent attacks and continued plots against locations in New York and California, to include the 1993 World Trade Center Bombing, the 1994 Blind Sheikh plot, the Millennium plot against Los Angeles International Airport (LAX), and the September 11 th attacks, amongst others. Numerous interested stakeholders at all levels of government sought to learn more about the homeland security grant allocation methodology and process.
 Figure 1 provides a time line to track major milestone events in the evolution of risk assessment in a homeland security context.
Risk experts appear to agree that all communities have some level of risk from terrorism. Yet, homeland security officials acknowledge that it is impossible to protect every target and harden every community to the extent that they become impervious to future attacks. It seems clear that it is necessary, from a national perspective, to identify the areas and entities across the country most at risk and to work to reduce that risk. What is less clear is the best way to evaluate relative homeland security risk, and establish an acceptable level of risk while attempting to close the most dramatic gaps between risk and capabilities. What follows is a chronological overview of the DHS risk assessment methodology examined through the prism of the Homeland Security Grant Program.
subsections: section_title: Evolution of the DHS Risk Assessment Methodology
paragraphs: The federal government's approach to distributing funds to State/local governments to enhance the latter's ability to prepare for and respond to terrorist acts has evolved in the last six years. It is important to understand the genesis of this grant program and the reactions to each stage of its development in order to better comprehend the current methodology. The evolution of the grant program and the risk methodologies it employs has occurred against the backdrop of the transformation of the nation's understanding of 'homeland security' itself. Borne out of the September 11 attacks, the term 'homeland security' and the department designed to enhance it, were initially solely terrorism-focused. With time, and other catastrophic incidents, the focus of the department expanded to include a range of potentially destabilizing, non-terrorism threats, such as natural disasters. This evolution in mission has significant ramifications for the calculation of the threat aspect of the risk formulas utilized to allocate some of the homeland security grant funds, as will become evident in the following section's overview of grant allocation and related risk methodologies.
Over the years, there have been numerous criticisms from various groups over how risk is assessed and, as a result, DHS grants are allocated. Following the FY2004 homeland security grant allocation process, the 9/11 Commission (hereafter Commission) weighed in on the funding controversy when it issued the following recommendation in its final report, published in late July 2004:
Homeland security assistance should be based strictly on an assessment of risks and vulnerabilities. Now, in 2004, Washington, D.C., and New York City are certainly at the top of any such list. We understand the contention that every state and city needs to have some minimum infrastructure for emergency response. But federal homeland security assistance should not remain a program for general revenue sharing. It should supplement state and local resources based on the risks or vulnerabilities that merit additional support. Congress should not use this money as a pork barrel.
The Commission report asks a second question: "Can useful criteria to measure risk and vulnerability be developed that assess all the many variables?" The Commission lists a variety of factors that should be considered in the assessment of "threats and vulnerabilities" to include "population, population density, vulnerability, and the presence of critical infrastructure within each state." The Commission suggests that the federal government should then require each State that receives such funds to provide an "analysis based on the same criteria to justify the distribution of funds [with]in that State." The Commission understood that the "benchmarks" chosen to evaluate a site's threat and vulnerability "...will be imperfect and subjective; [but] they will continually evolve."
Given the criticisms associated with the DHS risk assessment methods, a look at congressional interest in risk assessment as it relates to the homeland security grant programs may be instructive.
section_title: Risk Assessment-Related Legislative Activity13
paragraphs: Concurrent with the Commission's critique and internal efforts within DHS to move to a more risk-based approach, Members of Congress put forth a series of bills and amendments to the Homeland Security Act that sought to reform the criteria for distributing homeland security grant funds. Each effort sought to remedy the perceived issues associated with the homeland security risk assessment process. Some suggested the creation of new oversight or coordination bodies. Most importantly, for purposes of this report, each bill and amendment proposed changes to reduce guaranteed allotments and enhance the percentage of funding allocated based on risk. To varying detail, each legislative initiative suggested definitions or approaches to evaluate risk with regards to homeland security. The Appendix provides additional information on the legislative initiatives referenced in this section.
Understanding the criticisms of the DHS risk assessment process and the proposed congressional remedies, an analysis of how the various stages of risk assessment have evolved over time may be useful.
section_title: Risk Assessment: Stages of Development
paragraphs: There have been at least three stages in the evolution of risk assessment methodology as it pertains to homeland security. These stages and unique developments within each era are summarized below.
subsections: section_title: Stage I: R=P
paragraphs: This period covers from FY2001, when the Department of Justice (DOJ) had primary responsibility for assessing risk, to FY2002-FY2003, when this responsibility was transferred to DHS. This first stage of risk assessment could be characterized as early stage developmental. During this period, risk was generally assessed and measured according to population numbers. In short, risk (R) was equated to population (P).
section_title: Stage II: R=T+CI+PD
paragraphs: This period covers from FY2004 to FY2005. During this period, the importance of critical infrastructure, population density and a number of other variables was included in the assessment of risk. However, the formula for risk remained additive and "risk-like," as probabilities were not an essential element of the risk assessment process. Risk was assessed as the sum of threat (T), critical infrastructure (CI), and population density (PD).
section_title: Stage III: R=T*V*C=T*(V&C)
paragraphs: This period covers from FY2006 to today, a time when probability of particular events was systematically introduced into the formula. As will be discussed more in-depth below, FY2006 also marked another important departure from the previous risk assessment methodology: For the first time, when calculating risk, DHS chose to examine both risk to assets and geographic areas. With the swearing in of Michael Chertoff as Secretary of the DHS in February 2005, the department underwent both organizational and strategy-related changes. Concurrent with DHS's reorganization, Secretary Chertoff announced that a new risk-based methodology would dictate department al activities and all future federal funds would be distributed accordingly. This new approach to allocating the remaining funds required an assessment of risk using a formula that considers the threat to a target/area, multiplied by vulnerability (V) of the target/area, multiplied by consequence (C) of an attack on that target/area. As a result, the risk assessment formula became R=T*V*C. Variables were no longer additive, but were multiplied, implying weighting of variables and some assessment of the likelihood that certain events would occur.
section_title: The Current Process
subsections: section_title: FY2007
paragraphs: The FY2007 Homeland Security Grant Guidance describes the DHS approach to risk assessment as:
Risk will be evaluated at the Federal level using a risk analysis model developed by DHS in conjunction with other Federal entities. Risk is defined as the product of three principal variables:
•     Threat (T) —the likelihood of an attack occurring
•     Vulnerability and Consequence (V&C) —the relative exposure and expected impact of an attack
Although DHS continues to discuss its risk methodology in terms of the R=T*V*C formula, it appears as if the department is treating vulnerability (V) and consequence (C) as an amalgamated, single variable as depicted in Figure 2 . As mentioned above, due to difficulties associated with differentiating vulnerability values across areas and states, according to DHS it has, in effect, assigned a value of one to vulnerability. As a result, while three variables may formally remain in the formula, in effect only two exist for FY2007. In addition, significant changes to the underlying elements of each variable were made for the FY2007 process.
While the FY2007 HSGP Guidance does not provide additional detail as to the specifics of the risk methodology, a separate document, the FY2007 DHS Grant Programs Overview , accompanying the Guidance sent to state homeland security leaders does provide greater transparency into how risk is assessed. In the FY2007 DHS Grant Programs Overview , the weighting of each variable is provided and includes a description of the underlying data-sets supporting the calculation for each variable. As demonstrated in Figure 2 the vulnerability and consequence variables of the risk methodology now include the sub-elements of population index (comprising 40% of the risk methodology), a national infrastructure index (15%), an economic index (20%), and a nation security index (5%).
In FY2007, DHS's manner for determining threat (20% of the risk methodology) underwent a significant change in how intelligence and investigative information was analyzed. DHS's Office of Intelligence and Analysis for the first time undertook an historical analysis of threats to the representative UASI cities that spanned from the attacks of September 11, 2001, to the release of the FY2007 Homeland Security Grant Guidance. Prior to FY2007, in supporting the Homeland Security Grant effort, DHS evaluated threats to cities for the preceding year only and did not consider historical threat trends. For FY2007, DHS also initiated an effort whereby the cities deemed most at risk were placed in four tiers based on assessed level of threat.
It should be noted that DHS's efforts to evaluate and analyze threats only consider federal government intelligence and investigative information. To date, State and local intelligence and investigative information are not systematically considered in DHS's assessment of threats to a given locality. It could be argued that the establishment of the State and local fusion centers may assist in ensuring relevant State and local threat information is considered in future federal government risk analysis efforts.
section_title: The Current State
paragraphs: The evolution of the DHS risk assessment process and formula, as summarized in Table 1 , continues to spark additional questions and some concerns in the following areas: the transparency of the risk assessment process; the implications of an evolving risk formula; minimum grant allotments; and the responsibility for buying down risk.
subsections: section_title: Transparency
paragraphs: The additional information provided by the department in FY2007 should allow applicants of DHS grant funds to have a better understanding of the types of information contained in the underlying data-sets and how each is assessed and weighted during the risk assessment process. While this transparency in the methodology may satisfy some grant process critics, others remain concerned with the formula's effectiveness in meeting the needs of those most at risk.
section_title: Risk Formula Evolution
paragraphs: With the adoption of R=T*V*C, many see FY2006 as the first significant change to DHS's risk assessment methodology. Some observers could express concern that continued changes to the methodology will not allow the United States to establish a baseline of risks to the nation, thus jeopardizing any attempts to spot current trends or forecast future security concerns. Others might view the changes to the methodology as steps toward improving the risk assessment process and suggest that as DHS's understanding of risk evolves and its access to data increases, the associated methodology will stabilize and provide a sound foundation from which to make analytic and grant decisions.
section_title: Guaranteed Minimums
paragraphs: Some homeland security observers suggest that future congressional or executive branch changes to DHS's risk-based formula should include the elimination of the disbursement of guaranteed funding minimums to all states and municipalities. Noting that HSGP grants "enhance States, territories, and Urban Areas ability to prevent, protect against, respond to, and recover from terrorist attacks, major disasters, and other emergencies," others may argue that the continuation of a minimal level of funds to all states might be beneficial in shoring up vulnerabilities or buying equipment that can equally respond to man-made and natural threats to a jurisdiction, citizens, property, or government functions. Other commentators may maintain that disbursing a significant portion of the funds without regard to the risk level of a given locality will continue to impair the prevention, preparedness, and response capabilities of those cities deemed highest at risk.
section_title: Responsibility for Reducing Risk and Federal Grant Levels
paragraphs: Since its inception, DHS's risk-based formula for distributing funds to state and local communities has been a source of frustration for members of the federal, state, and local governments and those who assess post-9/11 counterterrorism program implementation efforts. Some homeland security observers suggest that it is unrealistic to expect grant levels to continue to increase as U.S. budget concerns weigh on future appropriations. Others might note that as at-risk jurisdictions continue to shore up previously known vulnerabilities they will require less federal funding due to a lowering of their risk profile.
 Table 1 below provides a cursory overview of the evolution of the DHS grant and risk assessment formula from FY2001 through FY2007.
To inform the ongoing congressional debate on risk assessment as it flows through the DHS grant program, the following section provides an assessment of some of the policy questions associated with risk assessment in a homeland security context.
section_title: Risk Assessment26 and Resource Allocation—Macro Questions
paragraphs: The overview of the evolution of the risk assessment methodology as it pertains to homeland security grant allocations highlights a number of questions at the macro level, such as how to best measure risk, that might be considered before contemplating questions of allocation. As mentioned above, the Commission recommended that the allocation of grants should be based on risk and (emphasis added) vulnerability. Vulnerability is but one of three elements of risk, as defined by DHS. In recent Senate testimony, former Congressman and Vice Chair of the 9/11 Commission Lee Hamilton suggested that experience serve as a guide for risk assessment and resource allocation. Mr. Hamilton noted three elements worthy of consideration when allocating homeland security spending: (1) historical and empirical data on what has been attacked not only within the United States but overseas—Washington, DC; New York City, New York; Madrid, Spain; and London, England—all large cities; (2) Al Qaeda statements—according to Mr. Hamilton—"...So far as we know they (Al Qaeda) continue to target symbols of American power"; and (3) the best available intelligence. While this approach is reasonable and based on sound logic, some might argue that broader questions and a more anticipatory approach may need to be considered to arrive at some credible and predictive value for future risk.
According to a recent RAND study, the following three questions might be addressed by policymakers before resource allocation decisions are made:
Should resources be allocated based on risk, risk reduction, or some other basis? How can terrorism risk be estimated? What are the tolerable levels of risk?
Another fundamental question in this area is "what is the risk to" and "from what sources does the risk originate?" Is the risk to people, infrastructure, the economy, or all of the above? Is the source of risk acts of terrorism, or the broader "all-hazards" approach, where the interests lie in responding to "incidents of national significance," as defined in the National Response Plan ? Does DHS, as suggested by the department 's Inspector General, "need to continue refining its risk-based approach to award first responder grants that ensure the areas and assets that represent the greatest vulnerability to the public are as secure as possible?"
With respect to the threat element of the risk equation, to what extent is unique data collected by state and local officials being incorporated into the threat element of risk? State and local law enforcement and public safety personnel provide substantial amounts of data to DHS and other federal entities with the understanding that the information will be used, in part, to assess threat. Yet, according to a DHS official, the methodology for incorporating that data are under-developed and, as a result, the data is not currently incorporated into threat assessment at the federal level in any systematic and meaningful manner. It may be possible that the emergence and proliferation of state, local, and regional intelligence fusion centers could become a funnel through which such state and local data could be aggregated and provided to the federal government in a manner that would allow it to contribute to threat assessment, an element of the risk equation that is weighted at 20% in FY2007.
The answers to these questions can have a fundamental impact on how grants are allocated. While the risk management process may be similar whether the source of risk is a hurricane or a terrorist attack, arguably, the inputs provided into the risk assessment model will be far different. DHS guidance shows that both the UASI and LETPP Programs are largely designed to provide state and local governments with funds to prepare and protect against as well as respond to and recover from acts of terrorism. While this purpose also exists in the SHSP, it has the additional purpose of supporting the implementation of the National Preparedness Goal. The other two grants currently under the Homeland Security Grant Program umbrella, the Metropolitan Medical Response System (MMRS) and the Citizen Corps Program (CCP), are almost completely focused on preparedness for post-event response. Consistent with a need to ensure all phases of a catastrophe are considered and program objectives are clearly defined, the DHS Inspector General found that "the department must incorporate sound risk management principles and methodologies to successfully prepare for, respond to, recover from, and mitigate acts of terrorism and natural disasters." In short, while DHS's risk assessment methodology is largely geared toward countering terrorism, the results of the assessment, along with other factors, such as effectiveness, are used for purposes which go beyond terrorism.
Once the fundamental questions of "risk to" and "risk from what" are answered, it is necessary to form a methodology to measure relative risk and to draft and implement a strategy to reduce it. To this end and from an economic efficiency perspective, it could be argued that the optimal manner in which homeland security grants might be allocated would be according to a comparative analysis of how historical homeland security grants have actually reduced risk. From September 11, 2001, through FY2008, approximately $12 billion will have been provided to state and local governments by DHS to prepare for and respond to terrorist attacks and other disasters. A central question that may be asked is what has been the rate of return, as defined by identifiable and empirical risk reductions, on this $12 billion investment? It does not appear, however, that there is an established methodology to engage in such analyses, nor are the data sets necessary for such analyses well-developed. According to one DHS official, while the department is planning to assess the impact of DHS grants on risk reduction in FY2008, it has been somewhat constrained by resources and the absence of a methodology to conduct such an assessment. As a result, some might argue, the next best method to allocating resources is to assess, measure and rank relative risk. This is, in essence, the approach currently being used by DHS.
section_title: Risk Management and Assessment: Complex Activities
paragraphs: The concept of risk—how to define, assess, and manage it—is relatively complex. According to DHS, "...Risk is classically represented as the product of a probability of a particular outcome and the results of that outcome." As mentioned above, it was not until FY2006 that DHS moved from a risk-like or additive approach to assessing risk to one that is guided by more classically defined or probabilistic methods of assessing risk. As will be expanded upon below, DHS differentiates between two different, but complementary types of risk: asset-based risk and geographic-based risk. Because DHS is assessing risk as a means to allocate resources to buy down risk, it is imperative, according to DHS, that its risk calculations be relative. That is, "...Because of the dynamic nature of temporal valuations in the many elements that figure into risk, an absolute value, even if it could be calculated, would be meaningful for a very limited time." Moreover, DHS differentiates between risk analysis and risk management. According to the Society for Risk Analysis:
Risk analysis is broadly defined to include risk assessment, risk characterization, risk communication, risk management, and policy relating to risk, in the context of risks of concern to individuals, to public and private sector organizations, and to society at a local, regional, national, or global level. Risk analysis seeks to inform, not to dictate, the complex and difficult choices among possible measures to mitigate risks.
Risk management is a continual process or cycle in which risks are identified, measured and evaluated; countermeasures are then designed, implemented and monitored to see how they perform, with a continual feedback loop for decision-maker input to improve countermeasures and consider tradeoffs between risk acceptance and avoidance. Risk assessment pertains to the quantification or measurement of identified risk and probabilistic assessment that certain risks will manifest themselves. According the Government Accountability Office (GAO), risk assessment is:
the process of qualitatively or quantitatively determining the probability of an adverse event and the severity of its impact on an asset. It is a function of threat, vulnerability and consequence. A risk assessment may include scenarios in which two or more risks interact to create greater or lesser impact. A risk assessment provides the basis for the rank ordering of risks and for establishing priorities for countermeasures.
The practice of risk management is well-developed within the insurance, engineering, finance, and political risk industries. It is clear, however, that risk management remains relatively immature in its application to the homeland security field. Some might that argue the implementation of risk assessment and management in the homeland security and counterterrorism fields may be more complex than in its industrial application where the primary objective is to protect against financial loss. Financial loss is, of course, one element of assessing and mitigating risk in a homeland security context, but of equal if not more importance are threats to human health and strategic national missions, among other factors. According to DHS, the following issues must be taken into consideration in the assessment of risk in the homeland security context:
Historical Data . In the insurance or financial sectors, the assessment of risk benefits from a rich and voluminous set of data which can be mined for patterns of historical behavior. While there are various governmental and non-governmental databases on terrorism, these data sources are relatively less robust. As a result, the subjective judgment of intelligence and terrorism experts become relatively more important in the projection of likely threat scenarios directed against categories of assets and/or geographic areas. Risk " Inheritance . " Because grant candidates include states and Urban Areas, and individual assets exist in both spaces, risk can be "inherited" from one candidate to another. For example, the risk score for a port will be "inherited" by the city and state in which that port is located. As such, according to DHS, it utilizes various mathematical techniques, including weighting, normalization and order of computation to control for such unique factors.
There are numerous other complicating factors associated with assessing risk in the homeland security context. Notwithstanding a common framework for assessing risk at an aggregate level, one of the central problems is that risks need to be defined at a micro level—for example, the risk to a certain asset or geographic area, given terrorist capabilities and intentions—to be very useful. At least with respect to assessing risk from terrorism, the nature of the risk is dynamic, as terrorists continually monitor successful countermeasures and adapt their targets, tactics, and modes of operation to surmount the countermeasures. Moreover, as alluded to above, a related problem is the absence of a definitive answer to the question of "how much risk is acceptable?" Secretary Chertoff has been open and frank in discussing the department 's risk-based approach—that is—the fact that the country has to accept some level of risk, as it is not feasible to protect against every real or perceived risk. Yet that level of acceptable risk, the threshold over which federal resources will be dedicated to managing risk, is not yet defined. In short, the successful risk reduction measures of today, may not necessarily be as successful in the near-to-medium term.
As will be outlined below, even if there is agreement on the central elements of risk, these elements are not necessarily independent of one another, thus requiring a sophisticated understanding of how each of the elements or variables of risk are interdependent. Stochastic and sensitivity tests for each variable and regression between variables may be of some utility in understanding how strong the relationships between variables are. For FY2007, as mentioned above, DHS has in effect assigned a value of one to vulnerability. While understandable at some level, this essentially eviscerates any interplay between vulnerability and consequence by having the effect of weighting the consequences of such an attack at 80%. It could be argued that if the vulnerability of a particular asset is exceedingly low, regardless of how grave the consequence, the risk to the asset may also be very low and, therefore, allocating relatively scarce homeland security resources to such an asset may be inefficient and ineffective.
Insofar as measuring risk is concerned, it could be argued that it is essential to identify the primary drivers of risk and collect the most appropriate data to quantify those risks. Collecting and measuring data that is readily available, but not central to risk yields quantifiable risk scores, yet some could argue that the results would be indefensible and relatively meaningless. If data which drive risk are not currently being collected, perhaps in the short-term such data deficiencies might be clearly recognized and controlled for in calculating risk. Reducing a variable value to zero or one based on the difficulty of collecting appropriate data to measure that variable should only be used as a temporary, stop-gap technique, as invariably such practices result in inaccurate risk assessments. Moreover, the level of confidence decision-makers have in data collected to assess risk is important. Resource allocation could be based, for example, solely on population figures, a statistic for which high confidence data exists. However, detriments of such a system are that population, in and of itself , may not necessarily be a terrorist target. If the population is not particularly dense, or exists in an area of marginal national economic impact or exists in an area where there are few critical national infrastructure assets, the population may not necessarily be a target for various terrorist groups.
There may be a balance to be struck between a risk assessment methodology that is too simple and one that is too complex. The question is what is the appropriate balance, and how a consistent methodology can be applied to a dynamic set of terrorist threats. Some homeland security observers might argue that the development of a long-term risk assessment strategy implemented by a strong DHS or broader government risk assessment analytic center that has as its sole mission the study of risk—its inputs and assessment of risk reduction results—may prove highly useful to help achieve this balance. Regardless of the complexity of the risk assessment methodology, due to the inherent uncertainties associated with assessing risk in a dynamic counterterrorism context, some level of flexibility in managing risk may be necessary. Empirical data based on historical terrorist attacks in the United States may, therefore, continue to play an important role in resource allocation designed to buy down risk. Some might argue that such an approach constitutes a "rearview mirror" or reactive perspective. Others might argue that unless and until reliable intelligence can demonstrate otherwise, historical attack patterns, informed by the best available current and strategic intelligence, will remain an essential risk assessment indicator.
section_title: Risk Assessment: Some Critical Drivers and Perspectives
paragraphs: Numerous factors drive risk and are essential to understanding risk assessment and management. This section will provide some basic definitions common to the risk assessment and management field. Although they are often used interchangeably, the terms associated with risk assessment have unique, though related, meanings. The most recent data available is included in the FY 2007  Homeland Security Grant Program — Grant Guidelines and Application Kit , in which it is stated that:
Risk will be evaluated at the Federal level using a risk analysis model developed by DHS in conjunction with other Federal entities. Risk is defined as the product of three principal variables: (1) Threat, or the likelihood of a type of attack occurring, (2) Vulnerability, or the relative exposure of an attack and (3) Consequence, or expected impact of an attack.
As alluded to above, DHS also differentiates between the following two types of risk, the attributes of which are depicted in Figure 3 and Figure 4 .
Asset-Based Risk "...employs strategic threat estimates from the Intelligence Community of an adversary's intent and capability to attack different types (e.g. chemical plants, stadiums, commercial airports) using different methods of attack. The vulnerability of each asset type to each attack method is analyzed to yield the form of attack most likely to be successful." Geographically-Based Risk "..considers general characteristics associated with a geographic area independent of the assets that exist within that area. This type of risk evaluates reported threats (credible and less credible), law enforcement activity (FBI and ICE terrorism case data and suspicious incidents).... Vulnerability factors considered include international border, number of international visitors and port channel length. The consequences of an attack on that area are then estimated to include human health...economic...strategic mission... and psychological impacts."
In short, as alluded to above, the DHS formula for assessing risk, whether it is asset-based or geographic-based, is: Risk = Threat*Vulnerability*Consequence , otherwise expressed as,
This formula is central not only to the allocation of DHS grant programs, but to all of the department's activities, as Secretary Chertoff has stated. It is important to define these variables for a number of reasons:
Without a common understanding of the lexicon, it is difficult to assess risk at the strategic and tactical levels. In order to gather the appropriate data which serves as an input to the risk assessment process, state and local agencies must understand how DHS is defining the elements of risk. In the absence of an understanding of each of these elements individually, it becomes increasingly difficult to comprehend how they are inter-related and inter-dependent.
There are numerous DHS elements, including the U.S. Coast Guard, Office for Domestic Preparedness and the Homeland Infrastructure Threat and Risk Analysis Center (HITRAC), among others, that are responsible for risk analysis and management. An example of how one component defines and practices risk assessment is instructive. HITRAC is the entity within DHS that is tasked with combining intelligence threat data as assessed and accessed by the DHS Office of Intelligence and Analysis with infrastructure vulnerabilities. According to a HITRAC Representative:
While inherently the most subjective component of the risk equation, threat of enemy attack is derived from study of enemy intent and capability. Intent of this adversary is assessed after study of all available information about they want to accomplish by attacking the United States. We work with our partners in the intelligence community to understand as much as we are able about the terrorists' goals, plans, and desires. We match what we know about the intentions of the adversary with information we have about what the enemy is capable of accomplishing. For this part of the equation we rely both on what we see the enemy discussing, recruiting and training for as well as lessons learned from overseas attacks....
This is just one example of how a single entity within DHS is approaching risk and specifically defining its components. It should be noted that although DHS headquarters has adopted a particular risk methodology, it is unclear how pervasive that approach has become outside headquarters, specifically within agencies brought under the department in 2004.
Given the evolution of DHS's risk assessment methodology and associated complexities translating risk assessments into well-targeted allocations of HSGP funds to buy down risk, there are a number of possible approaches for Congress to consider in this area.
section_title: Possible Approaches for Congress
paragraphs: More than natural disasters, assessing risk emanating from manmade actions is an extremely difficult task. Methodological tension is created when attempting to apply a quantitative formula to human-driven activities that require subjective assessments of enemy capabilities and intentions. Were a truly effective risk assessment tool to be created to help decision-makers manage risk, it would have to recognize that "management of risk is not elimination of risk." Whether focused on an "all-hazards" or counterterrorism approach, tools that attempt to quantify risk will always be inexact. However, sound data, a well thought-out formula, and consistent application of the methodology are important when attempting to measure terrorism risk to the U.S. and systematically buy down the risk to a particular location or asset. Such clarity and consistency are particularly important as the funds granted based on the DHS's risk methodology are the primary tools the federal government has to influence the behavior of state and local partners who will be the first on the scene of a terrorist attack and will be responsible for returning the community to pre-incident conditions. Congress has a number of apparent options concerning DHS's risk methodology efforts, including the following:
subsections: section_title: Maintaining Status Quo
paragraphs: Congress may wish to maintain the current policy and practices associated with DHS's existing risk assessment methodologies, and their affect on HSGP allocations. Some might argue that in the absence of measures to assess historical effectiveness of DHS grant programs, changing formulas and methodologies may be premature. Others might argue that with more than $12 billion worth of investment aimed at risk reduction and preparedness, state and local governments should have achieved a level of preparedness and capability that can allow room for negotiation on financial burden-sharing with the federal government for those programs deemed worthy of future support.
section_title: National Impact Assessment
paragraphs: By FY2008, more than $12 billion will have been provided to states, localities, and regions to buy down risk and enhance preparedness and capabilities to prevent a terrorist attack or to respond to such an attack or natural disaster should one occur. While audits have been conducted to determine how allocated funds have been spent, a national assessment of how much risk has been reduced as a result of such expenditures has not been undertaken. How much has risk been bought down? What investments have yielded the highest rate of return? What is the risk profile of each grant recipient moving forward? How are their existing capabilities measured against extant risk? What capabilities gaps exist, and how can resources best be targeted to address those gaps? There are at least two possible precursors to the drafting of such an impact assessment: (1) a defensible methodology that can (a) reasonably define and measure risk, (b) provide a means for measuring how developing capabilities are reducing that risk, and (c) illustrate how to identify specific capability gaps which might serve as an input for future allocation of homeland security grants; and (2) articulation of this methodology, including the data necessary to conduct such an assessment, to grant recipients. With the results of such an assessment, federal, state, local, and regional authorities might arguably be in a better position to understand the most effective and efficient way to target relatively scarce homeland security resources.
section_title: Further Enhance Transparency
paragraphs: While safeguarding the intelligence, law enforcement, and other sensitive information weighted and analyzed through DHS's risk methodology, disclosure of the mathematical equation used to determine threat, vulnerability, and consequence may allow all applicants and stakeholders to understand and have a basis to confirm or challenge the results prior to funds being allocated. It could be argued that providing this level of detail regarding the methodology and underlying equation may allow those who would seek to attack U.S. facilities to reverse-engineer the formula, thus increasing the probability of a successful terrorist attack. Others might maintain that allowing the risk formula equations to be revealed would encourage state and municipalities to manipulate the data provided to DHS, thus increasing their chances of receiving additional funding without a sufficient risk-based justification. Homeland security observers could counter these arguments by suggesting that though there may be the potential for those wishing to take advantage of the transparency of the system, the positives include possible increases in information sharing between DHS and state and local governments due to an understanding of how data is used and as such result in increased confidence in the other entity. Some could argue more transparency would allow DHS to more confidently allocate resources, as enhanced transparency may reduce the surprised outcries that seem to arise with each cycle's award announcements. This argument is based on the assumption that grant applicants that are comfortable with the risk assessment process and familiar with the data streams used to calculate risk. As a result, the applicants may be less likely to be surprised by their jurisdiction's ranking and awards.
section_title: Development of a Risk Strategy Both Within DHS and Throughout All Government Agencies
paragraphs: Since the establishment of DHS in March of 2003, the department's risk formula has evolved. Though it could be argued that these changes are indicative of a maturing organization and process, it is possible that the lack of a coherent, long-term, overarching risk strategy, which forms the foundation of departmental activities, could have negative repercussions for buying down risk. Without a clearly articulated risk methodology based on fundamentals intrinsic to risk, yet adaptive to changing threats, a baseline understanding of the nation's risk profile may never be achieved and the department's risk assessment process could potentially be vulnerable to budget fluctuations and political influence. This is especially important given the apparent division of risk assessment responsibilities throughout various offices and directorates within the department.
Arguments can be made that such an overarching risk philosophy needs to be adopted throughout the federal government. In a December 2005 report on homeland security risk management, GAO concluded that
for the results of a risk management system to be meaningful and useful, all related agencies should be using similar methods. If agencies' methods are not compatible, then comparisons between agencies become difficult and sector or national risk assessments becomes less reliable. In our earlier work, we concluded that a structured, systematic approach to risk management offers the best assurance that activities designed to protect the homeland and combat the effects of terrorism will produce the most effective and efficient results.
A cohesive risk strategy and agreement on core terms amongst disparate agencies is desirable because many aspects of the risk assessment process are dependent on functions performed by agencies outside the department. However, the necessity of common definitions and standards goes beyond the federal government. As states and localities continue to provide information to be included in the risk assessment process, to include, information on critical infrastructure sites within their respective jurisdictions and, eventually, investigative information, the rationale for attempting to develop national-wide risk assessment strategy at all levels of government becomes stronger.
section_title: Appointment of a DHS Risk Assessment Manager (RAM)
paragraphs: With regards to DHS's risk assessment efforts, the department might potentially create a Risk Assessment Manager (RAM) position charged with better integrating the various elements working on aspects of the risk assessment process. In addition, the RAM might be assigned the responsibility of creating and subsequently implementing a department-wide strategic risk strategy. Such a position could be in an advisory capacity to the Secretary or entail operational and oversight functions of a permanent DHS Risk Assessment Center (see below). The creation of a RAM within DHS responsible for coordinating all inter-departmental risk methodology activities would provide Congress, other federal government agencies, and state and local homeland security leaders with a single person accountable for explaining the complexities of future risk management strategy efforts and a specific office to receive suggestions regarding improving current processes. The RAM could also serve in a liaison capacity to ensure external agencies are familiar with DHS's approach to risk and facilitate agreement on key terms and processes amongst other agencies.
section_title: Creation of a DHS Risk Advisory Board (RAB)
paragraphs: As previously stated, Secretary Chertoff has made it clear that risk assessment underlies all elements of the department's operations. Risk management and assessment are disciplines which are relatively well-developed across the private sector. Moreover, within the U.S. government, there are numerous experts on risk assessment. To ensure that the Secretary is getting the best possible advice as to how DHS should continue to refine its risk management activities, a formal board of senior-level risk management professionals might be established to advise the Secretary. While not having program management responsibilities, the Risk Advisory Board (RAB) might advise the Secretary on the best risk management practices across industry and government. It could also lead the DHS effort, with substantial input from a potential Risk Assessment Center (see below), to draft a long-term risk management strategy.
section_title: Creation of a Permanent Risk Assessment Center (RAC)
paragraphs: While the proposed RAB would operate on a strategic level, it could be beneficial for DHS to examine its current efforts to apply risk strategy to its various programs and initiatives. Risk is central to DHS's operations. DHS may not necessarily have the appropriate resources dedicated full-time to (1) pro-actively assess the dynamic drivers of risk, (2) lead the collection of the right types of data to assess risk, and (3) develop a methodology to analyze how effectively past homeland security grant investments have "bought down" risk. These tasks are relatively complex and, it could be argued, require the formation of a group of professional methodologists whose sole function is risk assessment. While elements of this capability may exist now within the Preparedness Directorate, no single group has this sole responsibility. For example, HITRAC is charged largely with mapping vulnerabilities to threats, which is an essential function unto itself.
There are several potential benefits offered by a risk center: First, a permanent center would likely help DHS to think strategically about the current risk assessment process. Second, continued attention to this issue and sufficient time to address it would probably allow DHS to create more effective assessment tools and use those multiple tools in tandem to analyze various risk areas. Third, the risk center would potentially allow DHS to draw on the existing expertise and resources of all the offices and divisions within DHS, as well as external entities, such as other Intelligence Community agencies.
section_title: Implement 9/11 Commission Recommendation
paragraphs: As mentioned above, the 9/11 Commission recommended that "...homeland security assistance should be based strictly on an assessment of risks and vulnerabilities...But federal homeland security assistance should not remain a program for general revenue sharing. It should supplement state and local resources based on the risks or vulnerabilities that merit additional support." Some homeland security observers could interpret this literally to mean that after six years and $12 billion of homeland security investments, most states should be at some minimal level of security and capability. Therefore, some might argue, the time may be appropriate to revisit the USA PATRIOT Act formula which results in 40% of the total DHS grant funding being allocated based on formula which is not based primarily on risk. Others might argue that until a methodology is developed to ascertain how prior years' grant allocations have decreased each state's risk levels, it may be premature to alter the formula. Questions that might be addressed when considering this options include
What duration did Congress originally intend when it created the DOJ and now DHS homeland security grants? What measures are in place to ensure that state and local governments are spending resources in a manner that is consistent with congressional intent? To what extent, if at all, has congressional oversight yielded any indications that state and local governments have come to view homeland security grants as entitlements? Has DHS or Congress entered into discussions with state and local governments about sustainable burden-sharing arrangements with respect to state and local programs assessed as being worthy of continued financial support?
section_title: Treat Terrorism Prevention Grants Uniquely
paragraphs: Secretary Chertoff recently stated that one of the unique areas in which the DHS can add value is in the area of prevention. He stated, "...obviously, when it comes to terrorism, our best solution is a solution that prevents a terrorist act before it actually comes about. And a critical element in that is our early warning system, which is intelligence...." Notwithstanding this statement, a review of aggregate budget data for homeland security expenditures suggests that less than 1% of what the U.S. government spends on homeland security is dedicated to intelligence and warning, an essential element in the prevention component of homeland security. While there are many similarities in the response capability, whether the response be to a successful terrorist attack or natural disaster, terrorist acts can be prevented, natural disasters cannot. Information related to meteorology is different from intelligence related to national security. The threat element of the risk reduction formula is what differentiates terrorism from all other hazards. As mentioned throughout this report, terrorist threats are dynamic and evolve over time; some might argue the risk assessment methodology and attendant grant allocation process should be as agile as the adversary against which its resources are directed. DHS currently has an Intelligence Enterprise Strategic Plan , and the FY2007 grant application kit provides guidance for state, local and regional intelligence fusion centers. Yet, the linkages between these two documents and the grant process, some would argue, is tenuous. One of the Homeland Security Presidential Directive (HSPD)-8 derived "Universal Tasks" is prevention. Drawing upon the National Strategy for Combating Terrorism, Congress may ultimately consider recommending that DHS provide a specific and articulable strategy and approach to terrorism prevention, which would include a focus on how the grant allocation process is tailored to fully leverage intelligence across levels of government to prevent terrorist acts.
subsections: section_title: Appendix. Legislative Activity on DHS Risk Formula for Grants