Transit Security Grant Program: DHS Allocates Grants Based on Risk, but Its Risk Methodology, Management Controls, and Grant Oversight Can Be Strengthened
Published: 2009-06-08T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: From fiscal years 2006 through 2008, the Department of Homeland Security (DHS) has allocated about $755 million dollars to transit agencies through its Transit Security Grant Program (TSGP) to protect transit systems and the public from terrorist attacks. GAO was asked to evaluate the extent to which (1) TSGP funds are allocated and awarded based on risk; (2) DHS has allocated, awarded, and distributed TSGP grants in accordance with statutory deadlines and leading practices for collaborating agencies; and (3) DHS has evaluated the effectiveness of the TSGP and its investments. To address these objectives, GAO reviewed the TSGP risk model, fund allocation methodology and program documents, such as TSGP guidance, and interviewed DHS and transit officials, among other steps.
section_title: What GAO Found
paragraphs: DHS has used a risk analysis model to allocate TSGP funding and award grants to higher-risk transit agencies, although transit agency officials have expressed concerns about changes that have occurred since the TSGP's inception, such as revised priorities. The TSGP risk model includes all three elements of risk--threat, vulnerability, and consequence--but can be strengthened by measuring variations in vulnerability. DHS has held vulnerability constant, which limits the model's overall ability to assess risk and more precisely allocate funds. Although the Transportation Security Administration (TSA) allocated about 90 percent of funding to the highest-risk agencies, lower-risk agency awards were based on other factors in addition to risk. In addition, TSA has revised the TSGP's approach, methodology and funding priorities each year since 2006. These changes have raised predictability and flexibility concerns among transit agencies because they make engaging in long-term planning difficult. DHS met the statutory timeline requirements for allocating and awarding grants, but the two agencies that manage the TSGP--TSA and Federal Emergency Management Agency (FEMA)--lack defined roles and responsibilities, and only 3 percent of the funds awarded for fiscal years 2006 through 2008 have been spent as of February 2009. There is no documentation articulating the roles and responsibilities of the agencies, and grant information has not been passed between the two agencies which affected TSA's ability to share grant status information with transit agencies. DHS met statutory deadlines for releasing grant guidance and acting upon applications, but management and resource issues have resulted in delays in approving projects and making funds available, including (1) lengthy project negotiations between transit agencies and TSA; (2) a backlog of required environmental reviews; and (3) a reported lack of personnel to conduct required reviews. As a result, according to FEMA records, as of February 2009, transit agencies have spent about $21 million of the $755 million that has been awarded for fiscal years 2006 through 2008. This spending rate is, in part, caused by agencies receiving authorization to spend grant dollars late in the grant period. Despite concerns over delays, FEMA has not communicated time frames for providing funding. In April 2004, GAO reported that timely grant awards are imperative to provide intended benefits. DHS has reported taking some actions to address delays, including shortening project approval times and hiring staff, but the effectiveness of these efforts is unknown. Although FEMA has taken initial efforts to develop measures to assess the effectiveness of its grant programs, TSA and FEMA lack a plan and related milestones for developing measures specifically for the TSGP, and thus DHS does not have the capability to measure the effectiveness of the program or its investments. Without such a plan, it will be difficult for TSA and FEMA to provide reasonable assurance that measures are being developed to assess the effectiveness of the program as intended. While FEMA is responsible for the financial controls and audits of the TSGP, it does not have a mechanism to systematically collect data and track grant projects throughout the grant process. As a result, FEMA cannot assess whether awards are timely or funds are being used effectively to reduce risk and increase transit system security

section_title: Letter
paragraphs: American transit passengers, who take approximately 34 million trips each  weekday, rely on mass transit and passenger rail systems to provide  efficient, reliable, and safe transportation. However, terrorist attacks on  mass transit systems around the world—such as the 2005 attack on  London’s underground rail and bus systems, which resulted in 52 fatalities  and over 700 injuries—highlight the vulnerability of mass transit systems  and the need for increased focus on securing these systems. In an effort to  strengthen the security of the nation’s mass transit and passenger rail  systems against risks associated with potential terrorist attacks, the  Department of Homeland Security (DHS) has awarded grant funding to the  nation’s transit agencies that DHS has deemed to be of highest risk.  From  2003 to 2008, DHS provided over $1 billion in federal grant funding to U.S.  mass transit and passenger rail agencies for a variety of security activities,  including developing security plans, purchasing or upgrading security  equipment, and providing security training to transit employees. In fiscal  year 2009, DHS plans to award an additional $373 million to mass transit  and passenger rail agencies through the Transit Security Grant Program  (TSGP) to protect the traveling public from acts of terrorism, major  disasters, and other emergencies. Furthermore, the American Recovery  and Reinvestment Act of 2009 appropriated an additional $150 million to  DHS for the TSGP.
Since fiscal year 2005, the federal government has provided funding for  mass transit and passenger rail security through the TSGP—one of six  grant programs that constitute DHS’s transportation security grant  portfolio. The TSGP provides funds to owners and operators of mass  transit and passenger rail systems (which include intracity bus, commuter  bus, and all forms of passenger rail, including Amtrak) to protect critical  surface transportation infrastructure. While the TSGP provides funding  for mass transit, passenger rail (Amtrak) and other systems, this report  focuses exclusively on funds provided for mass transit because they  represent most of the funding.
You requested that we evaluate the TSGP, including the risk analysis  model and risk-based allocation methodology used to guide grant awards,  as well as the management of the program. Specifically, this report  addresses the extent to which  1.  TSGP funds are allocated and awarded based on risk, and grant  requirements have changed since 2006;  2.  DHS has allocated, awarded, and distributed TSGP grants in  accordance with statutory deadlines and leading practices for  collaborating agencies; and  3.  DHS has evaluated the effectiveness of the TSGP as well as  investments made using funds awarded through the TSGP.
To assess the extent to which TSGP funds were allocated and awarded  based on risk and grant requirements have changed since 2006, we  analyzed DHS documents, including those related to the TSGP risk  analysis model for fiscal years 2007 through 2008, TSGP guidance, and  TSGP priorities, and attended TSGP presentations held by the  Transportation Security Administration (TSA) and Federal Emergency  Management Agency (FEMA). To provide a basis for examining DHS’s  efforts to carry out risk management principles, we compared DHS’s risk- based methodology—which includes its TSGP risk analysis model—to  both the National Infrastructure Protection Plan (NIPP) and the risk  management framework that we developed based on best practices and  other criteria. We also reviewed the steps that TSA and FEMA took to  ensure the reliability of the risk model by interviewing officials  responsible for managing the model as well as reviewing DHS’s  documentation on the model. We determined that the model’s inputs and  results were sufficiently accurate for our purposes. To assess the extent to  which grant requirements have changed since 2006, we interviewed TSA  and FEMA officials about the TSGP grant determination process used in  fiscal years 2006, 2007, and 2008 and about the changes made to the  process for fiscal year 2009. In 2006, TSA began managing the policy  aspect of the TSGP, such as establishing grant priorities. For this reason,  we focused our review of the TSGP from this point through the beginning  of the fiscal year 2009 grant process.
To review whether DHS has administered the TSGP in accordance with  statutory deadlines and leading practices for collaborating agencies, and  to determine the status of grant expenditures, we reviewed TSGP  guidance, applicable laws, and grant project data from TSA for fiscal years  2006 through 2008. We also obtained grant approval data from TSA and  financial data from FEMA that allowed us to examine whether statutory  deadlines in the DHS appropriations acts were being met, as well as the  time that elapsed between award notification and grant disbursement. To  verify the reliability of the project approval dates and funding amounts  provided by TSA, we compared them to the total funding provided to each  region and found that these amounts matched. For the purposes of our  report, we concluded that TSA data on project approval dates and funding  amounts were sufficiently reliable. We also analyzed any policies and  procedures in place for managing the program with criteria on leading  practices for collaborating agencies and our Standards for Internal  Control in the Federal Government. Additionally, we conducted site  visits at, or held teleconferences with, a total of 30 mass transit operators  in the United States that represent 75 percent of the nation’s total mass  transit and passenger rail ridership to solicit their perspectives on the  management of the grant process. During these site visits, we interviewed  transit agency grant management personnel as well as state administrative  agency personnel from nine states who were responsible for administering  the TSGP grants at the state level. We used specific criteria to select  these mass transit and passenger rail agencies, including eligibility for  grant funding, high levels of ridership, and a diversity of risk levels as  determined by TSA. While the information we obtained from these 30  transit agencies cannot be generalized to all transit agencies, it enhanced  our understanding of the types of projects initiated using TGSP funds and  the period of time that elapsed between the grant award and the receipt of  such funds by transit agencies.
To determine the extent to which DHS has evaluated the effectiveness of  the TSGP as well as investments made using funds awarded through the  TSGP, we analyzed DHS’s strategic plan for transportation security and  available performance data and measures related to the grant program. To  determine what performance measurement data DHS had collected that  TSA or FEMA could use to understand the progress of the TSGP, we  interviewed grant officials from TSA’s Transportation Sector Network  Management Office and FEMA’s Grants Program Directorate. To verify the  reliability of the FEMA data on the amount of TSGP funding distributed  and held, we compared the data to state administrative agency and transit  agency records. Any discrepancies in the data were resolved through  discussions with FEMA officials. For the purposes of our report, we  concluded that FEMA data on distribution and held funding amounts were  sufficiently reliable. We also compared TSA’s and FEMA’s efforts to  evaluate their programs with guidance on performance measurement  contained in previous GAO reports.
We conducted this performance audit from September 2007 to June 2009  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
subsections: section_title: Overview and Vulnerabilities of U.S. Mass Transit Systems
paragraphs: Mass transit includes four main components––heavy rail, commuter rail,  light rail, and bus. Heavy rail systems—subway systems like New York  City’s transit system and Washington, D.C.’s metro—typically operate on  fixed rail lines within a metropolitan area and have the capacity for a  heavy volume of traffic. Commuter rail systems typically operate on  railroad tracks and provide regional service (e.g., between a central city  and adjacent suburbs). Light rail systems are typically characterized by  lightweight passenger rail cars that operate on track that is not separated  from vehicular traffic for much of the way. Large bus transit service is  characterized by vehicles powered by diesel, gasoline, battery, or  alternative fuel engines contained within the vehicle. According to the  American Public Transportation Association (APTA), 10.7 billion trips  were taken on mass transportation in 2008 —the highest number of trips  taken on U.S. mass transportation in 52 years. According to TSA, transit  officials, and transit experts, certain characteristics of mass transit  systems, such as multiple access points and limited barriers to access,  make them inherently vulnerable to terrorist attack and therefore difficult  to secure. High ridership, expensive infrastructure, economic importance,  and location in large metropolitan areas or tourist destinations also make  them attractive targets for terrorists because of the potential for mass  casualties and economic damage.
subsections: section_title: Grant Funding for Mass Transit Security
paragraphs: Because of the expense of operating and securing a transit system, the  costs are often shared among several entities. According to the  Department of Transportation’s (DOT) Federal Transit Administration  (FTA), almost all U.S. mass transit systems receive funds from public and  private sector sources to maintain a public service that is provided and  managed locally, since revenues from customer fares, on average, account  for 40 percent of system operating costs. For example, FTA provides  financial assistance to public transportation through its Large Urban Cities  Grant Program which provides funding to urban areas through a formula- based allocation. Owners and operators of public transit systems are also  responsible for ensuring the security of their systems. According to a 2004  APTA survey, transit agencies had more than $6 billion in transit security  investment needs.
To help defray the costs of securing U.S. transit systems, DHS has  provided transit security grant funding to transit agencies since 2003.  Beginning in fiscal year 2005, the DHS appropriations acts have provided  annual appropriations for mass transit security, including the TSGP, which  focused specifically on mass transit security. Table 1 outlines the TSGP  allocations for fiscal year 2006 through fiscal year 2009.
Both DHS appropriations acts and the Implementing Recommendations of  the 9/11 Commission Act (9/11 Commission Act) outline requirements for  security funding for mass transit and provide timelines for the issuance of  grant program guidance and decisions. In addition to appropriating  funding to the TSGP, DHS appropriations acts have provided deadlines for  the issuance of grant guidance, the application period, and when DHS  must act on applications. The 9/11 Commission Act required the Secretary  of Homeland Security to establish a program for making grants to eligible  public transportation agencies for security improvements, and DHS  fulfilled this requirement through the TSGP. Although the TSGP  considered risk prior to the passage of the 9/11 Commission Act, the act  created additional requirements for the TSGP, including that recipients of  public transportation funds be selected based on risk and that projects  address items identified in security assessments or plans. It also outlined  permissible use of funds and placed a limitation on the percentage of  funds used for operational costs.
Responsibility for administering mass transit security funding has changed  numerous times within DHS since 2003. DHS’s Office of Domestic  Preparedness administered the Urban Area Security Initiative (UASI) grant  program from fiscal year 2003 to fiscal year 2005. During fiscal year 2006,  the administration of the TSGP was transferred to TSA and the Office of  Grants and Training within DHS’s Preparedness Directorate. TSA became  the lead federal agency for determining the security priorities eligible for  funding and developing the criteria for evaluating applications, while  DHS’s Office of Grants and Training became responsible for grant  management. The Post-Katrina Emergency Management Reform Act of  2007 transferred most offices within the Preparedness Directorate into  FEMA; however, policy responsibilities, such as setting grant priorities  and funding decisions, remained with TSA. As a result, during fiscal year  2007, the Office of Grants and Training was transferred to FEMA. In fiscal  year 2008, FEMA’s Grant Programs Directorate became responsible for  administering TSGP grants.
section_title: Risk Management Practices Associated with the TSGP
paragraphs: Risk management has been endorsed by Congress, the President, the  Secretary of Homeland Security, GAO, and others as a way to direct finite  resources to areas that are most at risk of terrorist attack. Risk  management is a continuous process that includes the assessment of  threats, vulnerabilities, and consequences to determine what actions  should be taken to reduce or eliminate one or more of these elements of  risk. DHS released the NIPP, which created, in accordance with Homeland  Security Presidential Directive 7 (HSPD-7), a risk-based framework. The  NIPP, issued in 2006 and updated in 2009, sets forth guidance for agencies  with critical infrastructure protection responsibilities, such as TSA, for the  prioritization of protection initiatives and investments across sectors to  ensure that government and private sector resources are applied where  they offer the most benefit for mitigating risk.
TSA created six transit security fundamentals that it states are the  foundations for a successful security program, and the agency uses these  fundamentals to prioritize TSGP projects. For the fiscal year 2008 and  2009 grant cycles, TSA established a systematic process to rank these  priorities in awarding grant funds. To do this, TSA established project  effectiveness groupings—groups of project types that TSA ranked in order  of priority based on their ability to reduce risk—into which transit agency  projects were placed. For example, in fiscal year 2008, there were four  possible groupings (project types) for agency projects. The highest priority  for that year focused on projects aimed at developing security plans and  providing employee security training. See appendix II for the project  effectiveness groupings for fiscal year 2008.
DHS uses a risk model to help determine the transit agencies eligible for  TSGP funds. Both TSA and FEMA share responsibility for the TSGP risk  model, with TSA providing most of the data inputs to the model that is  managed by FEMA. The TSGP’s risk methodology is similar to the  methodology used to determine eligibility for other DHS state and local  grant programs. For example, the methodology for determining basic  eligibility for the TSGP is derived from the UASI grant program—both  models identify and use the same urban areas and both the UASI and the  TSGP risk models calculate risk scores for each urban area. See appendix  III for additional details on the TSGP model.
section_title: The Process for Awarding TSGP Funds
paragraphs: There are three stages of the TSGP grant cycle; allocation, award, and  distribution, as discussed in figure 1. TSGP grant guidance is created  annually by TSA and FEMA and provides an overview of the TSGP, the  application materials needed to apply for funding under the program, and  DHS management requirements.
Tier II gencie compete for  nd. Agencie’ project re reviewed  y NRP nd Exective Commitee  nd rnked based on rind  other fctor.
TSA nd T I gennd. DHS nno llon, bu  not flied.
When trit gencielfill ll grnt reqirement nd  FEMA complete review, fnd  re released. their project when the rd   nnonced.
Tier I gencieve 90 d to detil  how they will implement the project.
For fiscal year 2007, DHS had 75 days to release the grant guidance once the appropriations act had  passed, and for fiscal year 2008, DHS had 30 days to release the grant guidance once the  appropriations act had passed.
Beginning in fiscal year 2009, Tier I transit agencies submit specific project information for award  decisions prior to DHS award announcements.
section_title: Allocation
paragraphs: Using the TSGP risk analysis model, DHS develops risk scores, which are  used to identify the highest-risk regions and the transit agencies within  those regions that are eligible for funding. These regions are then placed  into one of two tiers based on their risk scores to determine initial funding  allocations; however, these allocations may change when DHS begins  reviewing projects.
Tier I: DHS determines the regions at the highest risk of a terrorist  attack and selects transit agencies within those regions eligible to  receive Tier I funding. Each Tier I region is given a target allocation  based on its share of risk (as determined by the model). Each region,  through discussions among transit agencies and TSA officials in the  regional transit security working groups (RTSWG), decides which  projects to fund on a collaborative basis. Each Tier I region has a  RTSWG that includes eligible transit agencies, law enforcement  agencies, and Amtrak (if stations exist in the region).
Tier II: Lower-risk regions and certain transit agencies in those regions  make up the Tier II group. The Tier II allocation is a set amount of  funding allocated for all Tier II regions combined. Transit agencies in  this tier apply for funding on a competitive basis—whereby their  projects are evaluated against all other Tier II agency projects  proposals, instead of funding decisions being determined  collaboratively, as with the Tier I RTSWGs.
After DHS announces target allocation amounts through the release of  the grant guidance, Tier I and Tier II transit agencies have 45 days from  the release of the guidance to apply for funding.
section_title: Award
paragraphs: During the award process, DHS evaluates transit agencies’ projects and  determines which projects to fund, although the evaluation process for  Tier I and Tier II agencies is different. Once the application period closes  for Tier I and II, the DHS appropriations act states that DHS has 60 days to  act upon the application, which DHS has defined as the length of time  taken to review the applications, make the award decisions, and announce  final allocations. In fiscal year 2008, as shown in figure 2, DHS created a  three-part scoring methodology for evaluating projects that included an  oject quality  agency’s risk score, its project effectiveness grouping, and a project quality  score, that included a regional collaboration factor.    score, that included a regional collaboration factor.
In fiscal year 2009, DHS used a similar scoring methodology, although this  methodology was applied differently to Tier I and Tier II agencies, as  described below.
DHS has encouraged all TSGP applications to have a regional coordination component  that demonstrates an investment strategy based on a regional security strategy. Many Tier I  and Tier II regions have more than one transit agency operating, so coordination of federal  TSGP investments is encouraged and is reflected in the regional collaboration component  of the overall project score.
Tier I: In fiscal year 2006, each region’s eligible transit agencies competed  for the target allocation. Under this process, transit agencies’ applications  were reviewed by an NRP consisting of subject matter experts from DHS,  TSA, and FTA. An executive committee, consisting of senior officials from  TSA, reviewed the NRP recommendations, the Secretary of Homeland  Security made the final selections for funding, and awards were  announced.
In fiscal year 2007, DHS introduced a new process to award grants to Tier I  agencies that involved direct negotiations between TSA and each RTSWG  to identify the grant-funded projects that TSA would approve. Under this  approach—known as the cooperative agreement process—award  announcements were made, and transit agencies had 90 days to submit to  DHS their investment justifications which provided additional details on  how the transit agency would implement the awarded security projects.  TSA collaborated with the transit agencies to finalize the investment  justifications.  Once these steps were completed, TSA officially approved  the projects.
In fiscal year 2008, TSA applied a more systematic approach for  determining project funding. For Tier I regions, project scores were  determined by weighing various factors, including the project  effectiveness grouping, the transit agencies’ risk scores, and a regional  collaboration factor. According to TSA officials, project quality was not as  important a factor for Tier I agencies because TSA participated in the  project development process through the RTSWG, which they believed  helped ensure quality.  DHS made award announcements to Tier I agencies  based on the project concepts discussed in the RTSWG; however, as in the  fiscal year 2007 process, final project approval was not completed at that  time.  DHS officials plan to use the same procedures in fiscal year 2009,  except that they expect investment justifications to be completed prior to  the award announcements and final project approvals are to be completed  at the time of award.
Tier II: DHS has used a competitive approach for awarding funds to Tier  II agencies. From fiscal year 2006 to fiscal year 2008, projects submitted  were reviewed and ranked by an NRP. After reviewing the transit agency  risk scores and submitted projects, the panel developed a recommended  slate of projects, including proposed funding amounts. An executive  committee consisting of senior officials from TSA, DHS, and FTA then  reviewed the recommendations as well as the risk scores of the transit  agencies. The Secretary of Homeland Security made the final selections  for funding, and then funding was announced. The evaluation criteria used  by the panel have evolved from fiscal years 2006 through 2008, as shown in  appendix V. Unlike Tier I agencies, Tier II agencies received their final  project funding amount for each fiscal year at the time of award  announcement. DHS plans to continue with this approach for Tier II  agencies for fiscal year 2009.
section_title: Distribution
paragraphs: TSGP funds cannot be disbursed to transit agencies until FEMA ensures  the agency’s compliance with federal grant management requirements,  such as the National Environmental Policy Act. Since fiscal year 2008, TSA  has approved transit agency projects (for both Tier I and II projects) and  then forwarded them to FEMA’s Grant Programs Directorate (GPD) for  review. GPD is responsible for ensuring that all grant projects adhere to  federal grant requirements, including all environmental and historical  preservation (EHP) requirements. FEMA’s Office of Environmental and  Historical Preservation (OEHP) assists with the EHP reviews. GPD  reviews projects identified as having limited EHP impacts, while OEHP  reviews projects needing a more extensive environmental and historical  review. Until FEMA is satisfied that all requirements have been met, no  grant funding can be released to transit agencies to begin projects.  However, once funds are awarded, transit agencies must complete the  grant project within the designated performance period for the grant year.  The TSGP’s performance periods have ranged from 24 to 36 months  depending on the grant year and project type to be completed. FEMA has  discretion to extend the performance period, if necessary.
For Tier I regions with multiple states, one state administrative agency is  designated for the entire region. The DHS Appropriations Act for fiscal  year 2009 required funding to be provided directly to the transit agencies,  removing the state administrative agency from the grant process. As a  result, going forward transit agencies will be responsible for all state  administrative agency duties, including submitting grant applications.
section_title: DHS Uses Elements of Risk to Allocate and Award Funds to Transit Agencies, but the Risk Model Can Be Strengthened and Transit Stakeholders Expressed Concerns about Funding Flexibility
paragraphs: DHS has established an approach for allocating and awarding TSGP funds  using a risk model that incorporates the elements of risk and is intended to  allocate funding to the highest-risk regions and transit agencies; however,  the model could be strengthened to measure variations in vulnerability  across regions. Furthermore, TSA revised its process and focus for the  TSGP on numerous occasions since 2006, but transit stakeholders  expressed concern about these revisions and their impact on funding  flexibility.
subsections: section_title: DHS’s TSGP Risk Model Incorporates Elements of Risk, but Could Be Strengthened to Measure Variations in Vulnerability
paragraphs: DHS uses a model to assess the risk to each transit agency region that  includes the three elements of risk––threat, vulnerability, and  consequence; however, the model does not measure variations in  vulnerability, which limits the model’s overall ability to assess risk. As we  reported in June 2008, measuring vulnerability is considered a generally  accepted practice in assessing terrorism risk. However, DHS did not  specifically measure vulnerability for each region and the associated  transit agencies in the model. DHS reported that it did not measure  region and transit agency vulnerability because it lacked data on the  differences in vulnerability among transit agencies. Therefore, DHS  decided to hold this variable constant in the risk formula. However,  holding vulnerability constant may be problematic because, for example, a  region may be highly vulnerable to one mode of attack but have a low level  of vulnerability to another depending on a variety of factors, such as  countermeasures already in place.
TSA officials acknowledged the need to incorporate vulnerability into the  risk model as a method for refining the results, but cautioned that  measuring variations in vulnerability would require time and resources. As  a result, officials reported that they were considering using transit agency  vulnerability assessment results as a source of vulnerability information.  To do this, FEMA officials acknowledged that they must be able to  consistently compare assessments across agencies and regions, which  may prove difficult given the variations in scope and methodology of these  assessments. A FEMA official stated that the risk model is designed to  incorporate other data, including vulnerability information, when it  becomes available. A TSA official noted that TSA is considering looking  into past vulnerability assessments and its Baseline Assessment for  Security Enhancements (BASE) reviews for vulnerability information that  might be used in the model. TSA officials also remarked that they  consider ridership to be the major known vulnerability factor. A TSA  official remarked that ridership represents the number of people exposed  by an attack, which is a proxy for the openness of the system, station, or  both. However, the risk model also uses ridership to measure  consequence, so its link to vulnerability does not add additional  information about how risk may vary across regions. Without accounting  for variations in vulnerability, the effectiveness of the risk analysis model  may be limited in that it may not fully consider important differences in  regions and transit systems that could affect their vulnerability to attack  and the risk scores may not be as precise. A more precise risk analysis  could affect the allocations of funds to Tier I or Tier II regions because  allocation is determined in part by the risk share.
section_title: The TSGP Risk Model Allocates Funding between Tier I and II and among Tier I Regions Based on Risk
paragraphs: Using its TSGP risk model, DHS placed transit agencies into one of two  tiers based on the risk of a terrorist attack occurring within a region, and  then allocated funding to those tiers based on risk. In the fiscal year 2007  model, Tier I represented approximately 80 percent of the total risk of all  regions assessed by the model, and Tier II represented the other 20  percent. In the fiscal year 2008 model, Tier I represented approximately  93 percent of the total risk to all regions assessed by the model, and Tier II  represented the other 7 percent.  Our analysis of the risk model and the  funding allocated through the TSGP for fiscal years 2007 and 2008 showed  that almost 90 percent of grant funds were allocated to the highest-risk  transit agencies—that is, those agencies in Tier I. Furthermore, during  fiscal years 2007 and 2008, the funding allocated to Tier I regions was  based on a region’s risk share, which was determined by its share of the  total risk for all Tier I regions in the model.  Our analysis of the three grant  cycles between fiscal year 2007 and 2008 showed that almost 90 percent of  grant funds were allocated to the highest risk transit agencies—that is,  those agencies in Tier I. Tier II received approximately 10 percent of the  grant funds.
After DHS allocated funds to Tier I regions, transit agencies worked within  their respective RTSWGs in negotiating with TSA to identify which  projects would be funded with the target allocation—known as the  cooperative agreement process.  TSA officials believe that the cooperative  agreement process ensured project quality because under this approach  TSA was able to work closely with transit agencies to develop security  projects. Additionally, in an effort to ensure that grant money is spent on  worthwhile projects, the grant guidance permits TSA to transfer funding  among regions if fewer quality applications are submitted from one region  and higher-priority security projects exist elsewhere. As a result, during  fiscal year 2008, TSA transferred funds between Tier I regions and from  Tier II to Tier I regions.  Although TSA worked with each Tier I region  during the fiscal year 2008 grant cycle, TSA officials reported that some  regions did not submit enough projects that exceeded the minimum  project score required to receive funding. As a result, one Tier I region saw  a reduction in its target allocation. According to TSA officials, these  reductions occurred because they did not want to fund poor quality  projects just because funds were available in a particular region. As a  result, in fiscal year 2008, Tier I gained an additional $13.7 million from  Tier II, and $7.5 million from the Freight Rail Security Grant Program, for a  total of $21.2 million. Five of the eight Tier I regions received awards  above their target allocations such as the New York City region, which  received $21 million more than its target allocation. The San Francisco  Bay Area, which was the only Tier I region to see a reduction, received  $2.8 million less than its target allocation.
section_title: Although Allocation to Tiers Was Risk Based, Funding of Tier II Transit Agency Awards Was Tied to DHS’s Assessments of Project Quality
paragraphs: Although DHS allocated funding to tiers based on risk, the specific Tier II  transit agency awards were not closely linked to risk. Unlike its  cooperative agreement process used to award funds for Tier I agencies,  DHS uses a competitive awards process for Tier II agencies and does not  negotiate the approval of security projects with the Tier II agencies as it  does with the Tier I agencies. Before fiscal year 2008, the executive  committee considered agency risk after the NRP had scored the agency  projects based on their investment justifications; however, the risk score  was not part of a standard methodology or formula for determining  funding. This process changed in fiscal year 2008 when TSA began using  Tier II agency risk scores as one part of its three-part scoring methodology  to determine project competitiveness.  Because applicants compete for  Tier II funds on a project-by-project basis, Tier II grant awards were not  solely based on transit agency risk. Rather, other factors also determined  grant funding. Specifically, our review of the NRP scores showed that  project quality was a major factor in determining if an agency received  grant funding. For example, a lower-risk agency with a high-quality  project was more likely to receive funding than a higher risk agency with a  low-quality project, based on the NRP’s assessment.
TSA reported that Tier II agencies submitted projects with proposed  investments totaling $37 million during fiscal year 2008, although DHS  initially awarded $16.9 million of the total $36 million allocated to Tier II  agencies. TSA officials reported that this occurred because many projects  were ineligible because of such things as insufficient information, lack of  live monitoring for closed-circuit television projects, or a focus on law  enforcement instead of security. Because there were not enough high- quality projects submitted to fulfill the $36 million allocation for Tier II,  according to TSA, the Secretary of Homeland Security made the decision  to recompete—that is, allow agencies to resubmit projects for funding— for an additional $6 million. To accomplish this, TSA provided written  feedback to Tier II agencies that received partial funding or no funding  from the initial fiscal year 2008 grant cycle and invited them to reapply for  the $6 million. The initial and recompeted TSGP funding for fiscal year  2008 resulted in DHS awarding about $23 million to all Tier II agencies.  DHS officials stated that the decision to recompete $6 million ensured that  the fiscal year 2008 funding for Tier II agencies was equal to the amount of  funding Tier II agencies received in fiscal year 2007.  TSA officials stated  that all eligible projects recommended by the NRP were funded with the  initial $16.9 million. However, TSA officials commented that during the  recompete, there were more eligible requests than funding available  because of their efforts to provide feedback on unsuccessful applications.  Transit agencies submitted $9.1 million worth of eligible projects for the $6  million in funds, thus projects were funded based on total project scores  until the funds were exhausted. TSA officials noted that several initially  deficient applications were modified based on feedback, resubmitted, and  then approved.
section_title: TSA Has Revised Its Grant Project Focus and Scoring Methodology since 2006, Raising Transit Stakeholder Concerns about Flexibility
subsections: section_title: Transit Agency Stakeholders Expressed Concerns about Shifts in Grant Project Focus since 2006
paragraphs: The types of projects eligible for funding and the specific projects TSA has  focused on have changed each grant year since 2006—making long-term  planning difficult, according to officials we interviewed from 8 of 30  transit agencies and numerous stakeholders at TSGP after-action  conferences held in September and October 2008. These changes, such as  the projects that would receive priority for funding, concerned transit  agencies because they meant that the agencies had to change their  proposals in some cases. For example, results from 28 TSA BASE reviews  completed from December 2006 through January 2007 indicated that  security training was an area needing improvement at many transit  agencies and was a critical vulnerability that needed to be addressed  immediately. As a result, after DHS released the fiscal year 2007 grant  guidance in January 2007, TSA officials notified all transit stakeholders in  February 2007 that the top funding priority for fiscal year 2007 would be  changed to training for key frontline employees. TSA informed the transit  agencies that this training would be given elevated priority when the  investment justifications were evaluated for funding merit, and projects  that included training would be funded ahead of other projects. While this  change may have been necessary to adjust to a changing security  environment, the change resulted in transit agencies having less than 2  weeks to decide whether they wanted to change their grant applications  and refocus them on this priority area. See appendix VI for a listing of  grant priorities for fiscal years 2006 through 2009.
section_title: Grant Project Scoring Methodology Has Changed, and Stakeholders Have Expressed Concerns Regarding Flexibility
paragraphs: Another change in the grant program that transit stakeholders expressed  concern about occurred in the fiscal year 2008 grant cycle when DHS  changed its methodological approach for evaluating applications.  Before  fiscal year 2008, the NRP evaluated Tier II grant projects for project  quality––including how those projects addressed the grant priorities. In  contrast, Tier I grant projects were determined by negotiations between  TSA and the RTSWG. However, in 2008 DHS introduced a new scoring  methodology for Tiers I and II, which was explicitly outlined in DHS’s  grant guidance that year. According to TSA officials, the change in scoring  methodology was based on stakeholder feedback that DHS be more  transparent and clear about funding priorities and exactly how projects  would be prioritized and ranked.
However, 28 of 40 transit stakeholders we interviewed (30 transit agencies  and 10 state administrative agencies) and numerous stakeholders at TSGP  after action conferences held in September and October 2008 noted that  the TSGP provides limited flexibility to pursue projects that have been  identified as transit agency security needs. Officials from one state  administrative agency said that prioritizing security projects puts forward- thinking agencies at a disadvantage because if they have already  completed projects that address TSA’s highest funding priorities, then  obtaining funding for alternative projects is difficult. Transit officials from  one agency said the grant priorities provide incentives for agencies to  potentially buy things they do not want or need, and that these  technologies will eventually just sit on the shelf. TSA officials stated that  the TSGP is a limited fund that must be allocated to best maximize the use  of scarce resources based on risk. TSA officials also reported that they  receive requests in excess of available funding, and therefore cannot fund  all eligible requests, necessitating a prioritization and ranking schema and  clear guidance on allowable project types.
Officials from five large Tier I transit agencies that have chemical  biological detection systems, or would like to install such systems,  expressed concerns that they could no longer receive funding to install  these detection systems. The TSGP listed chemical and biological  detection as an allowable expense for the grant program from fiscal years  2005 through 2007; however, TSA did not fund chemical and biological  projects during fiscal years 2007 and 2008 and listed them as an  unallowable expense for the first time in the fiscal year 2008 guidance.  TSA made this determination because its threat reports and security  assessments determined that improvised explosive devices (IED) and  improvised incendiary devices (IID) are the most common means of  attacking mass transit, and the training of frontline employees needed to  be addressed immediately. However, in fiscal year 2009 chemical and  biological detection systems became eligible, and TSA officials stated that  they may fund chemical and biological detection systems for fiscal year  2009 because some agencies have demonstrated that they can use this  technology effectively and restoring this eligibility may allow agencies to  enhance their response and recovery capabilities.
Similar concerns over flexibility were outlined in recommendations from  the Mass Transit Security Sector Coordinating Council to the Government  Coordinating Council led by TSA in December 2007. The transit industry  members of the council were concerned about the imbalance among the  priorities listed in the fiscal year 2007 grant guidance and noted that transit  agencies are in the best position to determine the balance of funding  between capital and operating initiatives. They specifically noted that  more predictability and flexibility in implementing priorities cited in the  grant guidance is needed to allow agencies to engage in long-term planning  of security initiatives, allowing agencies to more easily fund projects on a  multiyear basis. According to TSA officials, the collaborative efforts  between TSA and eligible transit agencies in the Tier I regions, combined  with the project effectiveness groupings that cite eligible security  enhancement measures in a prioritized listing, are intended to enhance  predictability and flexibility. In an effort to improve the TSGP, TSA and  FEMA held a conference in September 2008 to obtain feedback from  transit agencies and state administrative agency officials on the fiscal year  2008 grant cycle. At that conference, transit agency stakeholders  continued to express concerns about the need for greater flexibility and  that funding decisions should be informed by the regional strategies that  they have put into place.
For fiscal year 2009, DHS has reported changing the scoring methodology  to address transit agency concerns over limited flexibility. Specifically,  DHS added a grouping for other mitigation activities that allows some of  the project types that were previously excluded. Furthermore, DHS has  not explicitly excluded any type of project and has enabled transit  agencies to explain to DHS the priority groupings into which they believe  their project should be placed. The decision about project placement,  however, continues to lie with DHS and projects that fall outside of the  established project effectiveness groupings are given the lowest-priority  score. While this change could alleviate transit agencies’ concerns about  limited flexibility, it is too soon to determine whether it will address  agency concerns and allow them to secure funding for their highest  security needs.
section_title: TSA and FEMA Lack Documented Roles and Responsibilities for Administering the TSGP, and Although Statutory Timeline Requirements Were Met, Little Money Has Been Expended
paragraphs: DHS has met the statutory timeline requirements in allocating and  awarding grants. However the two agencies that manage the TSGP—TSA  and FEMA—lack defined roles and responsibilities, and the approval of  grant projects and completion of administrative requirements for grants  awarded in fiscal years 2006 through 2008 took many months.  Additionally, delays also occurred after projects were passed to FEMA for  administrative and environmental reviews because of backlogs and  reported resource constraints. TSA and FEMA have attempted to address  these delays by approving projects earlier in the grant process, issuing  guidance, and adding resources. Because of these delays, project funds  were often not available to transit agencies for months, and in some cases  years, after being awarded, and as a result, only 3 percent of grant money  has been spent as of February 2009.
subsections: section_title: TSA and FEMA Have Not Defined Roles and Responsibilities for Managing the TSGP
paragraphs: While TSA and FEMA share responsibility for managing the TSGP, the two  agencies have not defined and documented their roles and responsibilities  in a memorandum of understanding (MOU), or through similar means.  TSA’s responsibilities fall primarily in the award process and include,  among other things, identifying grant priorities, while FEMA’s  responsibilities include administering the grant management process to  ensure compliance with applicable laws, rules, and regulations. The roles  and responsibilities of the two agencies related to the award and  postaward processes are in the grant guidance. For example, the guidance  states that FEMA has the lead for designing and operating the  administrative mechanisms needed to manage the grant program.  However, there is no documentation articulating the working arrangement  between the two agencies.  For example, it is not part of FEMA’s  procedures to notify TSA when funding is released to the state  administrative agencies and transit agencies, despite TSA officials  reporting several requests for access to this information. As a result, TSA  officials reported that because they do not have this information, it is  difficult for them to respond when transit agencies contact them with  questions about their grants.
As we reported in October 2005, many agencies face a range of barriers  when they attempt to work collaboratively. To enhance and maintain  effective collaboration, we reported that agencies engage in practices such  as establishing joint strategic plans to achieve common outcomes as well  as instituting compatible policies, procedures, and other means to operate  across agency boundaries. Additionally, agencies can strengthen their  commitment to work collaboratively by articulating their agreements in  documents, such as MOUs, interagency guidance, or interagency planning  documents. Standards for Internal Control in the Federal Government  also requires agencies to delegate authority and responsibility throughout  their organizations. Articulating roles and responsibilities for managing  the TSGP could strengthen TSA and FEMA’s ability to ensure that  activities, processes, and resources are aligned to achieve a common  outcome and ensure smooth coordination during the grant process. TSA  officials stated that a formal MOU and guidance documents between TSA  and FEMA would be beneficial, while FEMA officials stated that they  believed the two agencies are working together effectively.
section_title: DHS Met Statutory Requirements to Release Grant Guidance and Act on Grant Applications, Although Some Projects Were Approved Months after the Award
paragraphs: DHS met the requirements of the TSGP to release grant guidance and act  on grant applications as defined by DHS; however, additional agency  actions are to be completed before specific transit agency projects and  funding levels are approved and transit agencies can begin projects. Since  fiscal year 2007, DHS appropriations acts have established timelines for  DHS to release the TSGP guidance and act upon transit agency  applications. For fiscal years 2007 through 2009, DHS met the  requirements to release the grant guidance within 75 days for fiscal year  2007 and 30 days for fiscal years 2008 and 2009. The appropriations acts  also set timelines for DHS to act upon the grant applications within 60  days, but until 2009, this did not include approving projects. DHS policy  defined the requirement to act upon grant applications as reviewing the  applications, making the award decisions, and announcing final  allocations. DHS met the requirements to act upon the grant applications  within 60 days, as defined by DHS, for fiscal years 2007 and 2008. While  there are specific statutory deadlines for releasing grant guidance and  acting on grant applications, there are no statutory deadlines once the  projects are approved and are passed to FEMA for review and funding  release to transit agencies.
However, even though allocation amounts were announced by DHS within  the statutory time frames during fiscal years 2006 through 2008, none of  the Tier I regions had their projects approved by TSA at the time of award  because TSA procedures allowed for approval after the award. For  example, during fiscal year 2007, TSA did not begin approving Tier I  projects until more than 5 months after the award date. One Tier I region  did not receive project approval for its fiscal year 2007 grant projects until  November 2008, or 15 months after the award date. As such, although DHS  met the statutory deadlines for acting upon grant applications within the  time frames established in legislation, project approval was not yet  completed. In contrast, all Tier II agencies involved in the competitive  process, which evaluates all projects at once, had all of their projects  approved by TSA when the awards were announced.
Delays in approving grant projects after awards were announced have  been attributed to TSA and the transit agencies involved in the cooperative  agreement process taking months to agree upon projects.  According to  TSGP grant guidance, the cooperative agreement process is valuable  because it provides greater flexibility and allows TSA to work directly with  transit agencies to quickly adapt to changes as situations arise during the  grant cycle. However, this cooperative process has also resulted in  significant time passing between the award and final project approval  dates. According to TSA data, during the fiscal year 2006 grant cycle, the  average project took 9.7 months to receive approval. During the initial  grant cycle in fiscal year 2007, the average project approval took 7.1  months. During the supplemental grant cycle in fiscal year 2007, the  average project approval took 5.5 months. Furthermore, at the time of  our review, there was still one Tier I region whose project from a previous  grant cycle had not yet been approved. Specifically, as of January 2009, a  Chicago region project totaling $2.9 million had not been approved from  the fiscal year 2006 grant cycle even though the fiscal year 2006  performance period ended in March 2009. In contrast, Tier II agencies  involved in the competitive process have their projects approved at the  time of the award and thus do not experience these delays.
TSA officials stated that some of the delays were caused by a provision in  the DHS appropriations act for fiscal year 2009, which provided that the  program could not include a cost share requirement for grants made  available for fiscal years 2008 and 2009. According to TSA officials, the  removal of this cost share requirement caused a disruption because some  transit agencies had to modify their projects, their budgets, or both, which  resulted in final project approval and disbursement delays. One state  administrative agency official in a Tier I region said that delays in funding  approval make program performance period extensions a necessity.  Further, the official stated that because some projects are complex and  involve multiple partners, delays can have a ripple effect and slow project  completion. In addition, as grant program periods are extended, it is  possible for multiple grant years to occur simultaneously, making them a  greater challenge to manage effectively. A TSA official reported that as of  late March 2009, all Tier I projects for fiscal year 2008 were approved.
A TSA official said that TSA has made progress in managing project  approval time frames by changing some of its procedures for fiscal year  2009, but also noted that some of the delays in previous years could be  attributed to transit agency procedures as well. For example, a TSA  official noted that some transit agencies are required to have projects  approved by their boards of directors or state legislatures—efforts which  contributed to the length of time between award and project approval. For  example, one state administrative agency official said that transit agencies  cannot begin projects until state legislatures approve the projects. The  official noted that this process can take time, especially if the legislature is  not in session. According to TSA, during fiscal year 2009 funds are to be  awarded directly to individual transit agencies; therefore, when DHS  announces the awards, each transit agency’s funding amount must be  finalized at that time. On April 8, 2009, in conjunction with the award  announcement, DHS issued final allocation amounts for transit agencies  for fiscal year 2009. As a result of this administrative change, TSA officials  noted that they expected the project approval letters to be sent to FEMA  soon after the award announcement.
section_title: Postaward Delays Can Also Be Attributed to FEMA’s Backlog of Environmental Reviews and Reported Resource Constraints
paragraphs: Once TSA approves projects and award amounts are finalized, FEMA  takes responsibility for ensuring compliance with federal requirements;  however, backlogs in FEMA’s review processes have resulted in delays in  distributing project funding. One requirement that has caused delays  involves ensuring compliance with the National Environmental Policy Act,  which requires the consideration of the environmental impacts of  proposed actions as well as reasonable alternatives to those actions.  FEMA’s GPD works in conjunction with FEMA’s OEHP to complete the  Environmental and Historical Preservation (EHP) reviews of each project.  GPD reviews projects that have no, or limited, EHP impacts, and OEHP  reviews those projects needing a more comprehensive environmental and  historical preservation review. Before April 2007, DHS’s Office of Grants  and Training and TSA shared responsibility for managing the TSGP.  According to OEHP officials currently managing the EHP review process,  when FEMA assumed responsibility for administering the TSGP in April  2007, they discovered that the EHP requirements had not been fully  integrated into the TSGP and that there was a lack of institutional  knowledge among DHS’s staff about how to manage the EHP process and  TSGP requirements. This lack of experience, in combination with the  lengthy process of collecting the necessary EHP information from grant  applicants, led to a backlog of EHP reviews from fiscal years 2005 through  2007.
According to FEMA officials, there is a need for additional personnel to  address the EHP backlog and other anticipated workload issues.  According to GPD officials, the backlog created by pending EHP reviews  led to a sizable workload for GPD’s limited staff. In addition, GPD officials  estimated that when transit agencies begin applying directly to FEMA for  TSGP grants in fiscal year 2009, instead of going through their state  administrative agencies, this approach will generate a fivefold increase in  TSGP applications as individual transit agencies apply rather than state  administrative agencies. In February 2009, GPD officials reported that  several efforts are under way to manage their workload. For example,  GPD expects to hire six more program analysts—in addition to the two  already in place––to manage the expected workload increase. FEMA  officials also reported in February 2009 that they expected to have these  new staff hired and in place by March or April 2009. Additionally, GPD  reported that it augmented its staff with contractor support in December  2007, to reduce the time for EHP reviews and expedite the release of  funds. In March 2009, GPD officials said that they planned to expand the  contract within 2 months to include another person for EHP support. They  also reported that they are in the process of conducting a workforce study,  to commence in late spring 2009, to determine staffing needs for the  additional workload, and expect to have this study completed by the end  of fiscal year 2009.
section_title: EHP Backlog Remains Despite FEMA’s Efforts to Address It
paragraphs: In a separate effort to address the backlog of EHP reviews, in 2007 OEHP  developed new guidance for conducting environmental reviews. The new  guidance is aimed at addressing the backlog and heavy workload brought  about by the integration of GPD grants into FEMA by focusing GPD and  OEHP staff resources on project reviews with the greatest potential for  environmental impact. FEMA officials reported that the backlog prior to  the release of the guidance resulted in projects taking several weeks to  several months for EHP approval, depending on the complexity and level  of review. Officials also reported that the internal processing time has  improved by 50 percent since the EHP guidance was released, and the  guidance has also helped to identify the need for EHP training for external  and internal stakeholders. See appendix VII for FEMA’s EHP review  project types. Additionally, DHS revised its grant guidance for fiscal year  2009 to clarify to grant recipients the EHP information that they should  submit so that FEMA can begin reviewing their projects. The intention of  this revision was to reduce the amount of time between collecting the  information and beginning the EHP review process.
Despite these efforts, there remains a backlog of grant projects awaiting  review. According to FEMA officials, as of March 2009, 72 projects were  still in review, accounting for $88 million. Twenty-four were projects from  fiscal year 2006, and 48 projects from fiscal year 2007. FEMA officials  further noted that a large number of these projects were in EHP review. As  of March 2009, FEMA’s EHP regulations were disaster focused, and have  not been revised since 1996—before DHS existed. FEMA officials reported  in March 2009 that the agency would revise its environmental regulations  to be more inclusive of all types of projects, including non disaster  homeland security grants, that FEMA funds. However, FEMA did not have  a timeline for when the new regulations would be published. Best  practices for project management call for milestone dates, among other  factors, in carrying out a project successfully. Establishing milestones  could help FEMA ensure that revisions to its environmental regulations  are conducted as management intended.
section_title: A Small Amount of Grant Funds Has Been Spent Because of the Length of Time to Make Funds Available
paragraphs: From fiscal years 2006 through 2008, DHS awarded about $755 million in  transit security grants; however, as of February 2009, only about $21  million, or 3 percent, of this total had been expended by transit agencies  largely because of TSA’s lengthy cooperative agreement process, the EHP  backlog, and delays in receiving disbursement approval from FEMA. As  of February 2009, for fiscal years 2006 through 2008, approximately $334  million dollars has been distributed to transit agencies and approximately  $421 million is still being held pending review (with the majority of the  held funds from fiscal year 2008). As might be expected, more recent fiscal  years showed higher unexpended balances. However, low grant  expenditures by transit agencies was commonly reported across all TSGP  grant years, as shown in figure 3, and are related to many transit agencies  receiving authorization to spend their grant dollars near the end of each 2  to 3 year grant performance period. FEMA officials reported that transit  agencies may choose to draw down their award at any time during the  performance period.
Our analysis of TSA project approval and FEMA grant adjustment notices  (GAN) from fiscal year 2006 or release of funds memos for fiscal year 2007  showed that it could take up to 20 months for transit agencies to receive  approval to begin projects, which accounted for a significant portion of  the grant performance period. FEMA used GANs and release of funds  memos to notify the state administrative agency and the transit agency  that they may begin a project.  In fiscal year 2006, state administrative  agencies may have received more than one GAN for each project. The first  GANs were to notify the state administrative agencies to “obligate and  expend” the funds, which meant that they could begin the projects.  However, this did not mean that they could draw down any funding. Only  upon receipt of the “obligate, expend, and draw down” GAN could the  funds be withdrawn. This two-part GAN process created some confusion  among transit stakeholders and, in fiscal year 2007, FEMA clarified the  GAN process. In fiscal year 2008, FEMA changed this procedure again to  include the use of a single release of funds memo, which allowed transit  agencies to draw down funds. See figure 4 for the average amount of time  it took for transit agencies to receive approval from TSA and FEMA to  om TSA and FEMA to  begin projects after the grant award date.  begin projects after the grant award date.
In addition to the delay between announcing awards and obtaining final  project approvals, 25 of 40 transit stakeholders we interviewed, including  state administrative agency officials, also reported time delays in receiving  their grant monies. Furthermore, numerous transit stakeholders attending  the TSGP after-action conferences raised concerns about the time it took  to receive awarded funds after projects were approved, and stated that  they believed the process was broken. They also reported that they  believed the performance period needed to start when the GANs were  received, not when awards were announced. For example, during fiscal  year 2006, one Tier I transit agency was awarded $4 million for a new  integrated security response center, but the agency did not receive  approval to begin the project until June 11, 2008. As a result, unless this  transit agency receives an extension, it will have less than 10 months to  complete the project to stay within the original 30-month performance  period. A transit agency official told us that the agency requested an  extension until June 30, 2010, to complete this project, and was awaiting  FEMA’s response. In addition, in December 2008, a state administrative  agency official for one state sent a request to FEMA for a 2-year extension  to the performance period for the entire state’s fiscal year 2006 TSGP grant  because the “delays from the federal level have left many of these projects  without a chance of success during the performance period.”
Transit stakeholders also said that concerns about funding delays have  hampered their ability to effectively plan for and manage projects. For  example, one transit agency official said that because of delays in  receiving grant funding the agency is constantly seeking extensions, which  are often not approved for longer than 3 months. In addition, another  transit agency official stated that state procurement processes can take  additional time to complete, which can also reduce the amount of  available time to complete the project within the performance period.   FEMA officials noted that 2006 was an unusual year for the grant program  because the multiple GANs they issued to state administrative agencies  resulted in confusion among transit agencies about when projects could  begin or when they could start spending money. As a result of the delays  encountered in the fiscal year 2006 grant process, FEMA officials stated in  March 2009 that they were notifying transit agencies of one-year  extensions for all fiscal year 2006 grants that were set to end on March 31,  2009. Despite the concerns over funding delays, FEMA has not established  or communicated time frames for providing grant funding to transit  agencies once projects have been approved by TSA. In April 2004, we  reported that timely awarding of grant funds is imperative to provide the  intended benefit of the grant program. Additionally, the purpose of the  TSGP is to provide funding to owners and operators of transit systems to  protect critical surface transportation infrastructure and the traveling  public. Ensuring the timely distribution of grant funds is essential for  ensuring that transit system owners and operators receive necessary funds  early enough in the performance period to complete their security  projects.
section_title: Additional Steps Needed to Develop Performance Measures to Assess TSGP Grant Project Effectiveness and to Fulfill Administrative Responsibilities
subsections: section_title: TSGP Lacks a Plan and Related Milestones for Developing Performance Measures
paragraphs: While the purpose of the TSGP is to provide funds to protect critical  surface transportation infrastructure and the traveling public, the program  lacks a plan and related milestones for developing measures to track  progress toward achieving program goals. While FEMA reported that it  was beginning to develop measures to better manage its portfolio of  grants, TSA and FEMA have not collaborated to produce performance  measures for assessing the effectiveness of TSGP-funded projects, such as  how funding is used to help protect critical transportation infrastructure  and the traveling public from possible acts of terrorism. Further, FEMA  does not yet have performance measures in place for its administrative  duties, such as measuring the time taken to complete reviews of financial  and administrative requirements. As we reported in October 2005, to  enhance and maintain effective collaboration, agencies should engage in  practices to achieve common outcomes and establish compatible policies,  procedures, and other means to operate across agency boundaries.  Additionally, according to best practices for project management, the  development of a project management plan—which defines how the  project is executed, monitored and controlled, and closed—is a key  element of project management. Best practices for project management  also call for milestone dates, among other factors, in carrying out a project  successfully.
FEMA officials reported in October 2008 that while they were in the  process of establishing baselines and targets for measures identified  through the Program Assessment Rating Tool (PART) requirement,  additional work was needed to develop meaningful measures. FEMA  officials stated that performance measures for the TSGP are likely to focus  on the increased security capabilities of the transit agencies, such as the  number of canine teams a transit agency deploys. In addition, FEMA has  also been developing a cost-to-capability assessment that officials report  will allow them to analyze grant program accomplishments from fiscal  years 2003 through 2007. Still in its early stages, the cost-to-capability  assessment focuses on efforts to measure a jurisdiction’s capability to  prevent and respond to various types of disasters compared to a target  level of capability. Although TSA has lead responsibility for surface  transportation security, a TSA grant program official stated that TSA does  not have any role in FEMA’s cost-to-capability assessment and only  learned about it in late 2008. This official also reported that the assessment  raised some concerns as it might not be tailored appropriately to each  transportation mode.
TSA officials reported that they are considering using the BASE review  and TSA inspectors to develop and monitor performance measures for the  TSGP; however, TSA officials reported not taking any action to develop  performance measures because of resource constraints for managing the  program. As we have reported, federal programs contributing to the same  or similar results should collaborate to ensure that goals are consistent  and, as appropriate, program efforts are mutually reinforcing. Until TSA  and FEMA collaborate to develop a plan with related milestones, it will be  difficult for the agencies to provide reasonable assurance that measures  are being developed to ensure that the program is achieving its stated  purpose of protecting critical surface transportation infrastructure.
section_title: FEMA Lacks Mechanisms to Systematically Collect Data and Track Grant Activities for Administrative Purposes
paragraphs: FEMA is responsible for conducting both a budget review and  programmatic review of grant projects including reviews of EHP  requirements.  However, despite this role, FEMA does not have a  mechanism for systematically collecting data on the status of individual  grant projects throughout this review process, including tracking the  status of the reviews it conducts and the release of funds to transit  agencies. Although FEMA has systems to track financial information  related to all of its grant programs, these systems do not allow FEMA to  track the status of grant reviews, such as EHP reviews. As a result, GPD  staff reported that they created a spreadsheet to track this information,  including identifying when TSGP funds were released once requirements  were met. Under this tracking process, each program analyst was  responsible for maintaining accurate records in the spreadsheet. However,  TSA did not have access to it and, until February 2009, the information  was not monitored for accuracy. Further, we found inconsistencies  between FEMA’s spreadsheets and data collected through FEMA’s  financial systems, including the amount of funding being held pending  EHP and other reviews. For example, we found that the total amount of  funds on hold in the GPD internal spreadsheets was not equal to the hold  amounts in FEMA’s financial systems. GPD officials told us that FEMA’s  financial systems were the official record for the awards.
A FEMA official reported that there are multiple information systems  involved in managing the TSGP. FEMA is in the process of implementing a  new consolidated grants management system—expected to be operational  for the TSGP by October 2009. According to FEMA, the new system is to  include functions that support the application process and is expected to  be fully operational throughout DHS in 2011. Although the system will not  initially support the tracking of grant disbursements, FEMA officials  reported that their intention is to have the system support these functions  in future releases. However, FEMA officials did not have a specific date for  when these capabilities would be available. In addition, as of March 2009,  there was no mechanism for TSA to gain access to grant review or  financial information, even though TSA officials reported requesting  information regarding when funds were released to transit agencies so that  they could track this information. Standards for Internal Control in the  Federal Government requires agencies to ensure that pertinent  information is recorded and communicated to management and others  within the entity in a form and within a time frame that enables them to  carry out their internal control and other responsibilities. Moreover,  systematically collecting data on the status of grant projects throughout  the grant process could strengthen FEMA and TSA’s ability to effectively  manage the program.
Similarly, the GPD is responsible for the financial controls and audits of  the TSGP to ensure that funds are appropriately disbursed and used in  accordance with grant requirements. However, the agency does not have a  plan for targeting its monitoring activities related to the use of grant funds  once projects have been implemented. GPD officials said that their office  conducts on-site visits to transit agencies to collect information on the use  of grant funds, but because of a lack of staff resources, their efforts have  mostly been limited to the largest Tier I transit agencies that either have  not spent their grant funding or were not able to complete projects within  the designated grant period. GPD officials said that they also conduct  document reviews, including reviewing quarterly financial reports,  progress reports, and special conditions to release funds. Although they  reported having limited resources, GPD officials said that they were able  to conduct approximately 24 site visits during fiscal years 2006 through  2008, attend numerous RTSWG meetings, and interact with transit  agencies at conferences as part of their efforts to monitor the awards.  GPD officials also reported creating a monitoring tool for the fiscal year  2007 grant cycle to be used during on-site visits, and officials stated that  the agency plans to modify the tool each grant year based on the specific  grant requirements for that year. GPD officials also reported that the tool  has been used by GPD program analysts during their site visits.
While GPD’s monitoring tool will likely strengthen the agency’s ability to  monitor grant activities, GPD lacks a plan to delineate how and when this  monitoring will take place. GPD officials acknowledged that a robust  monitoring plan needs to be implemented with processes in place to  ensure that the agency visits each transit agency at least once a year.  According to grants management best practices, monitoring grantee  performance helps ensure that grant goals are reached, and it is important  that agencies identify, prioritize and manage potential at-risk recipients.  For example, one federal agency with grant-making responsibilities has  created monitoring plans that include criteria to perform risk assessments,  which consider factors affecting a grantees ability to effectively manage  grants. This information could be used to prioritize monitoring activities  given GPD’s limited personnel. In addition, in September 2006 we reported  on the value of feedback provided through performance monitoring plans  and tools such as site visits. Moreover, TSA officials stated that their  agency currently has no role in the oversight of grant expenditures, but  believed that the use of its inspectors to provide grant oversight would be  a key component of the overall approach to mass transit security. TSA’s  surface transportation security inspectors, who are located throughout the  United States, interact with transit agencies for other purposes on a  regular basis, and could be used for on-site monitoring. In October 2005,  we reported that leveraging resources is vital to achieving effective  collaboration. A monitoring plan would provide GPD with a road map for  how it will carry out its monitoring activities to help ensure that it is  effectively using its limited resources. In addition, by working  collaboratively with TSA and its surface inspectors, who have security  expertise, GPD could leverage existing resources to ensure that transit  agencies are complying with security specifications set out in TSGP grant  guidance and the agencies’ own investment justifications.
section_title: Conclusions
paragraphs: As terrorist attacks on transit systems overseas have made clear, even  with a variety of security precautions in place, mass transit systems that  move high volumes of passengers daily remain vulnerable to attack. Risk  management has been endorsed by the federal government as a way to  direct finite resources to those areas at greatest risk of a terrorist attack.  While DHS uses a risk-based process to allocate funding for the TSGP,  without considering possible variations in vulnerability in the risk model,  the risk scores developed through the model are not as precise as they  could be, which could affect the allocation of funds to Tier I and Tier II  agencies. In addition, articulating roles and responsibilities for managing  the TSGP could strengthen TSA and FEMA’s ability to ensure that  activities, processes, and resources are aligned to achieve a common  outcome and ensure smooth coordination during the grant process.  Further, TSA’s delays in approving projects and FEMA’s backlog of project  reviews are contributing to delays, which negatively affect the ability of  transit agencies to complete their projects within grant performance  periods. However, TSA has made changes to the project approval process  for fiscal year 2009, which resulted in all projects being approved at the  same time as the grant award announcement. FEMA has also reported  plans to modify its approach for managing the administrative requirements  of the TSGP, including revising its environmental regulations to be more  inclusive of all the types of projects, including nondisaster homeland  security grants. While FEMA has not reported a time frame for completing  this process, establishing milestones to complete this modification could  help FEMA ensure that revisions to its environmental regulations are  conducted in a timely manner.
We have also previously reported on the importance of performance  monitoring in grant programs. Monitoring the implementation of TSGP  grant projects is vital to ensure that transit agencies are complying with  security specifications set out in the TSGP guidance and in the agencies’  own investment justifications. A monitoring plan that details how and  when monitoring will take place could improve GPD’s ability to plan for  this important oversight function and help it ensure that it is effectively  using its limited resources. A monitoring plan, which includes a method  for leveraging TSA resources, would also put GPD in a better position to  monitor grant implementation by working collaboratively with TSA to  leverage the security expertise of TSA’s surface transportation security  inspectors which will help FEMA address its resource limitations related  to monitoring. In addition, while FEMA’s consolidated grants management  system should allow FEMA to better manage data collection, the system  being developed is not expected to allow FEMA to collect data on the  status of grant activities throughout the grant process or to provide TSA  with access to this information, both of which are vital to ensuring  effective program management. Moreover, until the system is established  and able to track TSGP grants to allow for effective oversight and  management of TSGP funds, FEMA could benefit from establishing an  interim process that tracks the necessary information and share this  information with TSA, its TSGP partner. Finally, performance measures  are fundamental to the successful management of federal programs. As we  have reported, federal programs contributing to the same or similar results  should collaborate to ensure that goals are consistent and, as appropriate,  program efforts are mutually reinforcing. Until TSA and FEMA collaborate  to develop a plan with related milestones for jointly measuring the  effectiveness of TSGP, it will be difficult for the agencies to provide  reasonable assurance that measures are being developed to ensure that  the program is achieving its stated purpose of protecting critical surface  transportation infrastructure and that accountability and effective  stewardship of public resources exist. Finally, the absence of information  on the expected time frames for making funds available to transit agencies  once projects are approved can hinder transit agency efforts to design and  implement projects within the designated performance periods of the  grant.
section_title: Recommendations for Executive Action
paragraphs: We are making seven recommendations to help strengthen the  implementation and oversight of the TSGP.
To strengthen DHS’s methodology for determining risk, we are  recommending that the Secretary of Homeland Security develop a cost- effective method for incorporating vulnerability information into future  iterations of the TSGP risk model.
To strengthen the administration, oversight, and internal controls of the  TSGP, we are recommending that the Secretary of Homeland Security  direct TSA and FEMA to take the following four actions:    Define TSA’s and FEMA’s respective roles and responsibilities for  managing the TSGP in an MOU or similar document.
Develop a cost-effective plan for monitoring the use of grant funds  once projects have been implemented, including a strategy for  leveraging resources that could allow TSA surface transportation  security inspectors to assist in monitoring the grant projects to ensure  that the projects meet the security requirements set out in TSGP  guidance.
Develop an interim solution to systematically collect data and track  grant activities until FEMA’s grants management system can perform  these functions, and ensure that both agencies have access to these  data.
Collaborate to develop a plan and milestones for measuring the  effectiveness of the TSGP and its administration.
In addition, we recommend that the Secretary of Homeland Security direct  FEMA to take the following actions:    Establish a time frame for revising environmental regulations to be  more inclusive of nondisaster homeland security grant programs.
Establish and communicate time frames for making funds available to  transit agencies once FEMA receives project approvals from TSA.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to DHS and DOT for review and  comment. DOT did not provide comments. DHS provided written  comments on May 15, 2009, which are reprinted in appendix VIII. In  commenting on the report, DHS reported that it concurred with all seven  recommendations and discussed actions it has taken or planned to take to  implement them.
With regard to our first recommendation that DHS strengthen its  methodology for determining risk by developing a cost-effective method  for incorporating vulnerability information into future iterations of the  TSGP risk model, DHS concurred with the recommendation and said that  it would make appropriate adjustments in the fiscal year 2010 grant cycle.
DHS concurred with our second recommendation that TSA and FEMA’s  respective roles for the administration and oversight of the TSGP be  defined and documented in an MOU or similar document. DHS reported  that TSA and FEMA will work collaboratively to develop the MOU before  the fiscal year 2010 grant cycle and share it with external stakeholders to  ensure that the responsibilities and relationships between TSA and FEMA  are clear.
DHS also concurred with our third recommendation that it develop a cost- effective plan for monitoring the use of grant funds and leverage TSA  surface transportation security inspectors to assist in monitoring these  projects. Specifically, DHS reported that FEMA would work toward  developing a cost-effective monitoring plan to include the use of surface  transportation security inspectors in such instances when their transit  security expertise would be appropriate for monitoring grant program  functions. Because FEMA would be utilizing TSA personnel with  numerous other responsibilities to help with this monitoring, it is  especially important that the two agencies work together to coordinate  this effort and conduct the monitoring as efficiently as possible. For  example, TSA’s surface transportation security inspectors currently  monitor transit agencies through the BASE reviews and could monitor  grant implementation concurrently with those reviews.
DHS stated that it concurred with our fourth recommendation that TSA  and FEMA develop an interim solution to systematically collect data and  track grant activities until FEMA’s grants management system can perform  these functions, and ensure that both agencies have access to these data.   DHS also stated that TSA and FEMA will identify appropriate channels for  data collection and tracking as well as information sharing so that both  agencies have access to all appropriate information to ensure accurate and  consistent record keeping. In addition, DHS reported that it has taken  action to modify FEMA tracking logs and project spreadsheets to collect  additional information to track projects to improve its collection and  tracking of grant information. However, given that FEMA does not know  when the grants management system will be able to systematically collect  data and track grant activities, it is critical that FEMA develop and  implement this interim solution to collect and track key grant information  as quickly and accurately as possible.
With regard to our fifth recommendation that TSA and FEMA collaborate  to develop a plan and milestones for measuring the effectiveness of the  TSGP and its administration, DHS stated that it concurred with the  recommendation. DHS reported that a collaborative written plan with  established goals and milestones will be designed and implemented as part  of the MOU or other formal agreement between TSA and FEMA.
DHS concurred with our sixth recommendation that FEMA establish a  time frame for revising environmental regulations that consider  nondisaster homeland security grant programs. However, in its comments  on this recommendation, DHS stated that FEMA’s environmental  regulations apply to nondisaster grants. We did not intend to suggest that  the regulations did not currently apply to nondisaster grants.  Rather, we  are recommending that FEMA establish a time frame for completing its  plans to revise regulations that are currently focused on emergency  management program issues to be more inclusive of the types of issues  associated with nondisaster grant programs. In response to this comment,  and to clarify our point, we revised the recommendation to reflect that the  environmental regulations apply to all FEMA grant programs, but that  FEMA should establish time frames for revising the regulations to be more  inclusive of nondisaster grants. DHS also stated that FEMA is currently  working with DHS to update these procedures and is targeting completion  of this effort for the end of calendar year 2009. However, DHS noted that  these efforts could be extended if delays occur because of additional time  needed to complete procedural changes.
With regard to our seventh recommendation that FEMA establish and  communicate time frames for making funds available to transit agencies  once FEMA receives project approvals from TSA, DHS concurred.  Specifically, DHS also reported that FEMA will make every reasonable  effort to establish and communicate time frames for releasing funds to  TSGP grantees once FEMA receives approval of grant projects from TSA.   However, DHS noted that the release of funds often depends on the  responsiveness of grantees in submitting required documents and thus  FEMA would work proactively to obtain required information. DHS also  reported that FEMA would release grant funds within 3 to 5 days, if all  required EHP and budget information is received from grantees, and  appropriate clearances are provided by OEHP and the FEMA financial  analyst. However, our recommendation also intended that FEMA establish  timeframes for when its internal reviews would be completed once it  receives all of the required documents to facilitate a timely distribution of  TSGP awards. FEMA’s OEHP already has time frames for completing its  EHP review process and a related performance metric to assess its  effectiveness in meeting these time frames. Establishing such time frames  for its other internal reviews and communicating those to transit agencies  could help improve transit agency efforts to implement projects within the  designated performance periods of the grant.
As agreed with your office, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies to the Secretary of Homeland  Security, the Secretary of Transportation, the Director of the Office of  Management and Budget, and interested congressional committees.  The  report also is available at no charge on the GAO Web site at  http://www.gao.gov.
If you or your staff have any questions concerning this report, please  contact me at (202) 512-8777 or lords@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on the  last page of this report. Key contributors to this report are listed in  appendix VIII.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: The objectives of this report were to determine the extent to which (1)  Transit Security Grant Program (TSGP) funds are allocated and awarded  based on risk, and grant requirements have changed since 2006; (2) the  Department of Homeland Security (DHS) has allocated, awarded, and  distributed TSGP grants in accordance with statutory deadlines and  leading practices for collaborating agencies; and  (3) DHS has evaluated  the effectiveness of the TSGP as well as investments made using funds  awarded through the TSGP.
To determine the extent to which TSGP funds are allocated and awarded  based on risk, we analyzed guidance documents outlining best practices  for effectively implementing a risk management framework, including the  DHS National Infrastructure Protection Plan (NIPP), the Transportation  Security Sector Specific Plan (TS-SSP), and GAO’s risk management  framework. We obtained the Transportation Security Administration  (TSA) and the Federal Emergency Management Agency’s (FEMA) risk  analysis model for the TSGP for fiscal years 2007 and 2008. We analyzed  the model for these fiscal years to determine the process by which DHS  used the model to estimate risk—by incorporating threat, vulnerability,  and consequence information—as well as how the model was used to  divide regions into Tier I (higher risk) and Tier II (lower risk), and make  allocations to tiers and regions and the extent to which these allocations  were tied to the region’s or transit agency’s share of risk. We also  interviewed officials from TSA and FEMA as well as FEMA’s contractor,  Digital Sandbox, to understand what information was included in the  model and how the model was managed between the two agencies. We did  not evaluate the quality of the information or data included in the model,  but instead evaluated the model for how it incorporated the required  elements of risk. We determined the reliability of the model by discussing  methods of entering and maintaining data with agency officials. On the  basis of these discussions, and our review of the processes used to collect  the data, we determined that the data were sufficiently reliable for the  purposes of this report.
To determine the extent to which grant requirements have changed since  fiscal year 2006, we analyzed TSA’s grant guidance and grant priorities for  fiscal years 2006 through 2009, and attended TSA and FEMA presentations  to transit agencies prior to the release of the grant guidance as well as  after-action conferences for the fiscal year 2008 grant cycle.  Additionally,  we interviewed TSA and FEMA officials about the TSGP grant  determination process used in fiscal years 2006, 2007, and 2008––including  TSA’s scoring methodology for Tier I and II and the national review panel  criteria used for Tier II––and about the changes made to the process for  fiscal year 2009. We also interviewed 30 mass transit and passenger rail  operators that have applied for, received grant funding, or both to gain  their perspectives on how the grant requirements have changed since  fiscal year 2006 and the impact that these changes have had on the grant  process. The agencies we interviewed represent 75 percent of the nation’s  total mass transit and passenger rail ridership based on information we  obtained from the Federal Transit Administration’s National Transit  Database and the American Public Transportation Association. We  selected this nonprobability sample of transit agencies based on                (1) varying levels of ridership, (2) eligibility to receive TSGP grants,          (3) varying levels of risk (Tier I versus Tier II), (4) expert  recommendation, and (5) geographic dispersion. Because we selected a  non-probability sample of mass transit and passenger rail agencies, the  information obtained from these site visits cannot be generalized to all  transit agencies nationwide. Table 2 lists the mass transit and passenger  rail agencies we included in our interviews.
During site visits to mass transit and passenger rail agencies, we  interviewed grant managers and transit agency security officials  responsible for developing TSGP grant applications. Further, we  interviewed state administration agency officials directly involved in the  TSGP to determine how the administration of the program worked  between the state administration agencies and TSA and FEMA. We  discussed the TSGP, either in person or by teleconference, with the SAA’s  in Washington, D.C., and the following states: Washington, Illinois,  Minnesota, California, Texas, Georgia, Florida, Massachusetts, and New  York.
To determine the extent to which DHS has allocated, awarded, and  distributed TSGP grants in accordance with statutory deadlines and  leading practices for collaborating agencies, we reviewed a variety of  applicable laws, guidelines, and best practices. To determine DHS’s  compliance with statutory deadlines, we analyzed TSGP requirements in  the DHS appropriations acts for fiscal years 2007, 2008, and 2009 against  DHS’s TSGP actions to release grant guidance and act upon grant  applications. Additionally, we interviewed officials from FEMA’s Grants  Preparedness Directorate (GPD) and the Office of Environmental and  Historical Preservation (OEHP) to determine what actions were being  taken to meet the requirements of the National Environmental Policy Act.  To determine DHS’s compliance with federal guidance, we compared  FEMA controls for the TSGP, including how grant monies are monitored  through FEMA’s financial systems and spreadsheets, with criteria in  Standards for Internal Controls in the Federal Government. To  determine the extent to which DHS followed leading practices for  collaborating agencies, we compared plans and procedures in place  between TSA and FEMA to manage the program with criteria in our  October 2005 report.
To determine the status of grant funding since 2006, we reviewed the  length of time between grant allocation and grant distribution. This  required reviews of extensive grant documentation, including reviewing  original grant award dates for fiscal years 2006 through 2008, analyzing  grant project approval dates from TSA, reviewing grant adjustment notice  (GAN) and release of funds memos from FEMA, as well as grant  distribution and drawdown information from FEMA’s financial system and  internal spreadsheets. We compared this information against the records  of three state administrative agencies for states with large Tier I transit  agencies to determine the accuracy of the dates and financial information  we gathered and returned to FEMA for explanations when we found  discrepancies. We also reviewed grant guidance and grant requirements to  determine the performance period during which agencies had to spend  grant funding. Additionally, we interviewed TSA grant management  officials and FEMA GPD and OEHP officials to gain additional information  on how the grant process works at each stage––allocation, award, and  distribution. Our analysis also included interviews with officials from the  transit agencies listed in table 2 to gain additional information on how  grants are allocated and awarded as well as the length of time involved to  complete the grant process.
To determine the extent to which DHS has evaluated the effectiveness of  the TSGP as well as investments made using funds awarded through the  TSGP, we reviewed the following documents for guidance on performance  measures for infrastructure protection grant programs as well as for any  measures related to the TSGP: the National Preparedness Guidelines, the  NIPP, the TS-SSP–mass transit modal annex, and the TSGP grant  guidance. Additionally, we reviewed the guidance on leading practices for  collaborating agencies as well as best practices for project management.  To determine whether TSA or FEMA had implemented any measures for  the administration of the TSGP, we interviewed TSA grant management  officials as well as officials in FEMA’s GPD and OEHP. Finally, to identify  the extent to which TSA and FEMA are measuring TSGP investments, we  reviewed the Office of Management and Budget’s Program Assessment  Rating Tool, which identified baselines and targets for measures for the  infrastructure protection grants.
We conducted this performance audit from September 2007 to June 2009  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Fiscal Year 2008 Project Effectiveness Groupings
subsections: section_title: Training, operational deterrence, drills, public awareness activities
paragraphs: DHS-approved behavior recognition detection courses  Immediate actions for security threats/incidents  Employee security threat assessments (e.g., background  checks)
section_title: Appendix III: TSGP Risk Analysis Model
paragraphs: The TSGP risk model accounts for risk to both intracity rail (subway and  commuter rail) and bus systems. The rail and bus scores are combined to  determine the total transit risk for the region. Within each mode, the threat  index accounts for 20 percent of the total risk score while the vulnerability  and consequence indexes account for 80 percent. DHS’s measurement of  vulnerability and consequence is mainly a function of the consequences of  a successful terrorist attack, represented by a population index, the total  number of trips made on a system in a given year, and a national  infrastructure index, which focuses on critical assets that if attacked  would cause severe losses of life because of their particular vulnerabilities  and damage mechanisms. Figure 5 shows the TSGP risk model.
section_title: Appendix IV: TSGP Tier I and II Regions
paragraphs: Table 3 shows the Tier I regions for 2009, and table 4 shows the Tier II  regions for 2009.
section_title: Appendix V: Tier II National Review Panel Criteria, 2006 through 2008
subsections: section_title: 1. Rail and bus specific project types and
paragraphs: Bus project types included inventory control improvements, increased perimeter security, training  and awareness, emergency response and preparedness, and implementation of technology-driven  surveillance. Rail project type included use of passive measures, development and enhancement of  improvised explosive devices, mitigation capabilities, and mitigation of high conseuence risks.
section_title: Appendix VI: TSGP Priorities 2006 through 2009
subsections: section_title: Protection of underwater and other deep-bore tunnels and associated track mileage from attacks
paragraphs: Same as Rail.
Same as Rail.
Same as Rail.
Projects include interoperable communications, evacuation plans, and antiterrorism security  enhancement measures for low-density stations.
section_title: Appendix VII: FEMA’s Environmental and Historic Preservation Review Project Types
subsections: section_title: Purchases such as vehicles, patrol boats, ID cards, handheld or portable equipment, and navigation or communication equipment for vehicles, boats, or other mobile units
section_title: Appendix IX: GAO Contact and Staff Acknowledgments
subsections: section_title: Acknowledgments
paragraphs: In addition to the contact named above, Dawn Hoff, Assistant Director;  Daniel Klabunde, Analyst-in-Charge; and Martene Bryan, Senior Analyst,  managed this assignment. Jason Berman, Charlotte Gamble, and Su Jin  Yon made significant contributions to the work. Chuck Bausell and  William Chatlos assisted with design, methodology, and data analysis.  Linda Miller and Lara Kaskie provided assistance in report preparation,  and Tracey King provided legal support.