Surface Transportation: Competitive Grant Programs Could Benefit from Increased Performance Focus and Better Documentation of Key Decisions
Published: 2011-03-30T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: In February 2009, the American Recovery and Reinvestment Act (Recovery Act) appropriated $1.5 billion for discretionary grants for capital investments in surface transportation projects of national and regional significance, including highways, transit, rail, ports, and others. The act required the Department of Transportation (DOT) to develop criteria to award these grants--known as the Transportation Investment Generating Economic Recovery (TIGER) grants--and to meet several statutory requirements. GAO was asked to review (1) the criteria and process used to evaluate applications and award grants, (2) the outcome of the process, and (3) the extent to which DOT communicated information to applicants and the public. GAO reviewed documentation of the award process and selection documentation and interviewed key DOT officials.
section_title: What GAO Found
paragraphs: DOT developed criteria to evaluate TIGER applications, such as improving the state of repair of critical infrastructure, reducing fatalities and injuries, and increasing economic competitiveness by improving the efficient movement of workers or goods. GAO has called for a more performance-oriented approach to funding surface transportation and has recommended that a merit-based competitive approach--like TIGER--be used to direct a portion of federal funds to transportation projects of national and regional significance. This is a departure from the formula-based approach regularly used for surface transportation in which funds are largely returned to their state of origin and states have considerable flexibility in selecting projects for these funds--an approach that can potentially result in projects of national or regional significance that cross state lines and involve more than one transportation mode not competing well at the state level for these funds. DOT provides over $40 billion annually in formula funds to states and urbanized areas for highway and transit projects; by contrast, TIGER provided $1.5 billion on a one-time basis. However, TIGER was part of the Recovery Act, which was intended to provide economic stimulus across the nation, and the act required TIGER to balance using a competitive approach with achieving an equitable geographic distribution of funds. DOT has proposed a discretionary grant program like TIGER in its fiscal year 2012 budget, which means that DOT and Congress have the opportunity to consider how to balance the goals of merit-based selection of projects with geographic distribution of funds. Of the 51 applications that received awards, 26 were from the highly recommended applications advanced by the Evaluation Teams and the other 25, which received one-third of TIGER funds, were from the recommended applications advanced by the Control and Calibration Team. While DOT thoroughly documented the Evaluation Teams' assessments and the Review Team's memo described the strengths of projects recommended for award, it did not document the Review Team's final decisions and its rationale for selecting recommended projects for half the awards over highly recommended ones. Internal documentation of the Review Team's deliberations was limited to draft minutes from the team's initial assessments of projects. These draft minutes, which were not complete and never finalized or approved, reflect questions Review Team members raised about the strengths and weaknesses of various applications. For example, the Review Team questioned the extent to which financial commitments of project partners had been secured, whether projects were "ready-to-go," or whether a project's economic benefits were overstated. However, these questions did not necessarily reflect the reason a project was ultimately recommended or rejected for award. In addition, DOT officials told us that some highly recommended projects were not selected to achieve an equitable geographic distribution of award funds. In particular, DOT officials stated that some highly recommended projects from the Central and Western regions were rejected to prevent these regions from being overrepresented and that they advanced recommended projects from the South because projects initially selected for award underrepresented this region. DOT's TIGER program externally communicated outcome information similar to other Recovery Act competitive grant programs GAO examined, including the Review Team's memo to the Secretary and the amount of funding awarded. As with most other programs, DOT did not publish the reasons for the Review Team's decisions or why some applications were selected while others were rejected. GAO found no requirements for federal programs to externally communicate the reasons for their selection decisions and federal agencies rarely publicly disclose the reasons for their selection decisions.

section_title: Letter
paragraphs: In February 2009, facing what was generally reported to be the most  serious economic crisis since the Great Depression, Congress enacted the  American Recovery and Reinvestment Act of 2009 (Recovery Act) to,  among other things, preserve and create jobs, promote economic recovery  across the nation, and invest in transportation and other infrastructure to  provide long-term economic benefits. The Recovery Act appropriated $48  billion for transportation investments, including $1.5 billion for  discretionary grants available to state and local governments, including  organizations such as transit agencies, port authorities, and metropolitan  planning organizations, among others, and to be administered by the  Department of Transportation (DOT) for capital investments in surface  transportation, including highway, transit, rail, port, and other projects.  The Transportation Investment Generating Economic Recovery (TIGER)  discretionary grants were designed to fund merit-based projects expected  to have a significant impact on the nation, a metropolitan area, or a region.  In making awards, the legislation directed DOT to address several  statutory requirements. In December 2009, Congress appropriated $600  million to DOT for a “TIGER II” discretionary grant program similar to the  original TIGER program’s structure and objectives. DOT announced the  projects selected for TIGER grants in February 2010 and projects selected  for TIGER II grants in October 2010.
You expressed interest in DOT’s evaluation and selection process for the  TIGER grants and the extent to which information about that process was  made public. In response, we examined (1) the criteria and process  established by DOT to evaluate TIGER applications and award grants; (2)  the outcomes of DOT’s process to evaluate TIGER applications and award  grants; and (3) the extent to which DOT communicated the TIGER criteria,  evaluation process, and outcomes to applicants and the public. This report  also addresses, in appendix I, lessons learned from TIGER that were  applied in TIGER II.
To carry out this work, we reviewed the act and its requirements for  TIGER grant awards and the department’s criteria for selecting awardees  as published in the Federal Register and other TIGER guidance materials,  such as the DOT Evaluation Score Sheets that defined what characteristics  would meet each criterion. We reviewed documentation such as the  TIGER grant selection process summary and DOT’s March 2009 Financial  Assistance Guidance Manual, which provides guidance on discretionary  grant processes. We analyzed (1) available data from the applications such  as the amount of the request, transportation mode, and region; (2) the  initial ratings from a competitive review focused on project merits; (3)  which applications were advanced for further review and which were not;  and (4) recommendations of senior staff that described the strengths and  weaknesses of projects recommended for award. However, because DOT  did not document its reasons for key decisions, our review was limited to  draft minutes that were not finalized or approved and that summarized  initial assessments of advanced projects. To gain insight into award  decisions, we asked DOT officials to reconstruct and discuss why projects  were recommended or rejected for award, and, while this offers some  insight, such information has significant limitations. Specifically, it is  testimonial in nature and reflects officials’ recollections several months  after TIGER grants were announced. It may therefore contain a greater  level of uncertainty and error than documentation created while decisions  were made. We assessed DOT’s level of public communication regarding  the criteria and evaluation process for TIGER grants by comparing it to  Office of Management and Budget guidance on publishing selection  criteria. We also compared DOT’s communication of TIGER grant awards  to other Recovery Act discretionary grant programs. Finally, we reviewed  TIGER II guidance and training resources to identify changes between  TIGER and TIGER II. We also interviewed 8 of the 51 TIGER awardees,  selecting them based on funding level, region, transportation mode, and  jurisdiction size. We conducted this performance audit from June 2010  through March 2011 in accordance with generally accepted government  auditing standards. Those standards require that we plan and perform the  audit to obtain sufficient, appropriate evidence to provide a reasonable  basis for our findings and conclusions based on our audit objectives. We  believe that the evidence obtained provides a reasonable basis for our  findings and conclusions based on our audit objectives. For more  information on our scope and methodology, see appendix II.
subsections: section_title: Background
paragraphs: The Recovery Act provided the Secretary of Transportation $1.5 billion for  the purpose of awarding discretionary grants on a competitive basis.  Eligible projects included capital investments in roads, highways, bridges,  or transit; passenger and freight rail; and port infrastructure, as well as  bicycle and pedestrian-related improvements. The TIGER program’s  purpose was to fund merit-based projects that would provide a significant  impact on the nation, a metropolitan area, or a region and required DOT to  develop criteria to evaluate the merits of and select grants that would meet  the goals of the program. The Recovery Act also directed the Secretary to  meet several statutory requirements in making the final award selections  including:  1.  ensuring an equitable geographic distribution of funds;  2.  achieving an appropriate balance in addressing the needs of urban and  3.  giving priority to projects for which federal funding would be required  to complete an overall financing package that includes nonfederal  sources; and  4.  giving priority to projects that are expected to be completed within 3  years of enactment of the act and obligating all TIGER funds by  September 30, 2011.
The Recovery Act also allowed DOT to provide up to $200 million to  support projects eligible for federal credit assistance. The legislation  required that DOT make individual awards of no less than $20 million and  no more than $300 million. However, the legislation gave the department  discretion to waive the minimum grant size for the purpose of funding  significant projects in smaller cities, regions, or states, and DOT opted to  do so in certain cases.
Traditionally, federal surface transportation funding has been primarily  delivered through formula grant programs—about $40 billion annually— based on distributions prescribed by federal statute. In addition, to  address concerns about the “equity” of how federal aid is distributed  among states, Congress has included legislative provisions for geographic  distribution in every surface transportation reauthorization act since 1982.  For highway programs, DOT’s distribution formulas include provisions to  ensure that states receive a guaranteed portion—92 percent since fiscal  year 2008—of the estimated share of taxes highway users in each state  contributed for a subset of highway programs. The grant funds are then  administered by the state or passed through an intermediary or  subrecipient, such as a local government.
Compared with formula programs, discretionary grant programs are rarely  used to distribute surface transportation funding. In a discretionary grant  program, agencies rely on a competitive process in which Congress gives  award discretion to federal agencies to review applications in light of  legislative and regulatory requirements as well as published selection  criteria established for a program. The review process gives agencies the  discretion to determine which applications best address the program  requirements and are, therefore, most worthy of funding. The TIGER  program was a new discretionary grant program for DOT, wherein DOT  had to establish criteria and an evaluation process that could be used to  assess applications from several different transportation modes. By  including the requirement that awards achieve an equitable geographic  distribution, among others, DOT had to design a process that addressed  both competitive selection criteria and statutory requirements.
The Recovery Act set short time frames for DOT to implement the TIGER  program so that these funds—like other programs in the Recovery Act— would produce a stimulative effect on the economy. Specifically, the act  required DOT to announce all projects selected to be funded by TIGER  grants by February 17, 2010, 1 year after enactment and to obligate all  TIGER funds by September 30, 2011. DOT published its Notice of Funding  Availability on June 17, 2009; applications were due by September 15,  2009, and awards announced by the statutory deadline (see fig. 1 for a  timeline of TIGER activities).
The Consolidated Appropriations Act for fiscal year 2010 appropriated  $600 million to DOT for National Infrastructure Investment grants in  support of a TIGER II discretionary grant program. Similar to the TIGER  program’s structure and objectives, these grants were to be awarded on a  competitive basis for projects that are expected to have a significant  impact on the nation, a metropolitan area, or a region. TIGER II had many  of the same statutory requirements that TIGER had, as well as some  additional ones, including a requirement that DOT ensure selection of a  variety of transportation modes.
section_title: DOT Developed Comprehensive Merit- Based Selection Criteria and a Competitive Selection Process to Evaluate TIGER Applications and Meet Statutory Requirements
subsections: section_title: DOT Established Criteria for Awarding Competitive Grants That Support Merit- Based Investments in Transportation Infrastructure
paragraphs: To meet the Recovery Act requirement that DOT develop criteria to  evaluate the merits of TIGER program applications and select grants, DOT  published criteria in a May 2009 interim Notice of Funding Availability  after the passage of the Recovery Act, and in June 2009, when DOT  published its final Notice of Funding Availability, we evaluated the criteria.  We concluded that DOT had followed key federal guidance and standards  for developing selection criteria. For example, DOT’s criteria clearly  indicated that projects should produce long-term benefits such as  improving the state of repair of existing transportation infrastructure,  reducing fatalities and injuries through safety investments, and increasing  economic competitiveness by improving the efficient movement of  workers or goods. In addition, a benefit-cost analysis was generally  required to determine if a project’s expected benefits outweighed its  costs.
Developing rigorous criteria for discretionary grants is important because  criteria focus the competitive selection process and helps agencies, like  DOT, address national and regional priorities and achieve the highest  possible return on federal investments. As we have reported, many federal  surface transportation programs do not effectively address key challenges  because federal goals and roles are unclear, programs lack links to  performance, and some programs do not use the best tools and  approaches to ensure effective investment decisions. For these and other  reasons, surface transportation funding remains on GAO’s high-risk list.  Our previous work has called for a more performance-oriented approach  to funding surface transportation, and in particular policies that ensure  that goals are well-defined and focused on the federal interest and that  recipients of federal funds are accountable for results. Specifically, we  have recommended that a criteria-based selection approach—like that  developed in TIGER—be used to direct a portion of federal funds in  programs designed to select transportation projects with national and  regional significance.
Such an approach—one rarely used to fund surface transportation— represents a significant departure from the formula-based approach  regularly used to fund the nation’s surface transportation program.  Formula programs distribute over $40 billion annually to states and  urbanized areas for highway and transit projects (compared with the $1.5  billion one-time appropriation provided for TIGER). In fiscal year 2009,  this included almost $36 billion for highway infrastructure projects  through the Federal Highway Administration (FHWA) and approximately  $10 billion in transit grants to urbanized areas and states through the  Federal Transit Administration. The Federal-Aid Highway Program in  particular poses considerable challenges to introducing a merit-based  performance orientation for selection of projects of national or regional  significance. This is because this program distributes funding through a  complicated process in which the underlying data and factors are  ultimately not meaningful because they are overridden by other provisions  designed to yield a largely predetermined outcome—that of returning  revenues to their state of origin. Moreover, once the funds are  apportioned, states have considerable flexibility to reallocate them among  highway and transit programs. As we reported in June 2010, this flexibility,  coupled with a rate-of-return orientation, essentially means that the  Federal-Aid Highway Program functions, to some extent, as a cash  transfer, general purpose grant program. This formula-based approach  can potentially result in meritorious projects of national or regional  significance—in particular those involving multiple modes of  transportation or those that cross state boundaries—not competing well at  the state level for available funds.
TIGER selection criteria reflected federal interest in specific goals, such as  improving the state of repair of transportation infrastructure. Specifically,  DOT developed and applied two primary criteria—(1) long-term outcomes  and (2) job creation and economic stimulus—and two secondary criteria— innovation and partnerships. DOT further defined its primary and  secondary criteria with the concepts described in table 1 to help TIGER  reviewers determine how well a proposed project aligned with each  criterion. DOT described these criteria in its final Notice of Funding  Availability, noting that primary criteria were weighted more heavily than  secondary criteria, while the concepts defining each selection criterion  were weighted equally.
The criteria were designed to help DOT reviewers and applicants  determine which projects were closely aligned with the goals of the TIGER  program. Some criteria assessed the direct effects—such as reductions in  travel time and the number of fatalities and injuries—that are common  metrics used in evaluating the performance of transportation projects. For  example, to assess whether a project achieved a “state-of-good repair”  within the long-term outcomes criterion, evaluators had to determine if a  project was relevant to regional, state, or local efforts to maintain  transportation facilities; if failing to rehabilitate the condition of  infrastructure would threaten future economic growth and stability; and if  a sustainable source of revenue was available for the long-term operation  and maintenance of the infrastructure, among other issues. The Beartooth  Highway Reconstruction Project in Park County, Wyoming, a project  awarded $6 million, proposed to improve the state of repair of a 7-mile  segment of a highway, including replacing a critical bridge in deficient  condition, connecting Yellowstone National Park with the Shoshone  National Forest by completing its reconstruction. FHWA deemed this  segment of highway inadequate and substandard in 1994.
Other criteria were intended to help DOT assess the potential for a project  to produce indirect effects such as improved quality of life, coordinated  economic development, and better land use. One factor also within the  long-term outcomes criterion—fostering livable communities or  “livability”—represented a new focus for DOT projects. DOT defined  livability as:    enhancing mobility through the creation of more convenient  transportation options for travelers,  increasing modal connectivity between various transit and other  transportation options,  improving accessibility to transportation for economically disadvantaged  populations, and    coordinating transportation and land-use planning decisions.
For example, the Saint Paul, Minnesota, Union Depot Multi-Modal Transit  and Transportation Hub, a project awarded $35 million, was given funding  to renovate the city’s historic Union Depot to colocate Amtrak, intercity  bus carriers, local buses, light rail services, taxis, and bicycle  accommodations, as well as offer new space for commercial development.  The award announcement indicated that colocating these transportation  services would increase connectivity between transportation modes and  create commercial space that would promote economic growth and  redevelopment in the downtown area. For more discussion of how DOT  defined its criteria, see appendix III, which shows the score sheet  Evaluation Teams used to assess applications.
To meet the Recovery Act’s direction that the Secretary meet several  statutory requirements in making the final award selections, DOT  published and made potential applicants aware of these requirements in  its Notice of Funding Availability. In addition, DOT developed internal  guidance to clarify these requirements in the award process. For example,  according to officials, DOT defined achieving an equitable geographic  distribution of funds by establishing four regions based on a methodology  that accounted for population sizes, geographic proximities, and the  existing distribution of federal surface transportation formula funds. In  addition, DOT sought to ensure within regions that awards were not  clustered in one or two states, but were reasonably well distributed within  a region. To give priority to projects for which federal funding would be  required to complete an overall financing package, DOT gave priority to  projects that included significant state, local, or private co-investment,  required projects to demonstrate “independent utility,” meaning that  projects created a complete and operable segment that would produce  significant transportation benefits upon completion, according to officials.  In some cases, this meant DOT funded a project in its entirety, while in  others it funded a segment of a larger application as long as that segment  resulted in complete and operable infrastructure.
section_title: DOT Established a Competitive Selection Process for Evaluating Applications That Included Merit-Based Assessments and Addressed Statutory Requirements
paragraphs: DOT’s process for competitively selecting applications involved several  teams of reviewers—Evaluation Teams assessed and rated applications  and a senior-level Review Team made final award recommendations. In  addition, other teams evaluated the consistency of the ratings and  assessed the accuracy of applicants’ economic analyses and project  readiness. The TIGER selection process is described in figure 2.
DOT used 10 Evaluation Teams of five reviewers each—primarily career  employees with technical knowledge—who represented the different DOT  operating administrations, including the Federal Highway Administration,  Federal Railroad Administration, Federal Transit Administration, the  Maritime Administration, and the Office of the Secretary of Transportation  (OST). This team design meant that applications were reviewed by an  intermodal team that included members with subject matter expertise  from several different transportation modes. Although applications were  assigned randomly, DOT did ensure that at least one team member had  expertise in the mode presented in the application. The teams assessed  over 1,450 applications that requested almost $60 billion, and each team  evaluated approximately 150 applications.
Evaluation Team members were directed to select projects that they  judged had the greatest potential to meet the primary and secondary  criteria. Individual team members provided a rating of “highly  recommended,” “recommended,” “not recommended,” or “negative” for  each of the elements defining the primary and secondary criteria—for  instance, state of good repair, livability, and others—and an overall score  based on these criteria. Individuals also drafted short narratives  supporting their assessment. Table 2 presents the definition of each  adjectival rating  Once the team members completed their individual evaluations, the team  met as a whole to come to consensus on an overall team rating for each  application and a narrative describing their assessment of each project.  The Evaluation Teams prioritized applications receiving an overall team  rating of highly recommended and advanced these projects to the Review  Team for further evaluation.
In determining the overall project rating, DOT’s guidance encouraged  Evaluation Teams to identify and advance for further review projects that  best met the merit-based criteria. These applications were to be ranked  “highly recommended” and were to be subject to additional review by  additional teams on a wide range of factors—a time-consuming process  that needed to be reserved for a smaller group of applications. DOT’s  guidance to individual Evaluation Team members indicated they should in  general give an overall rating of highly recommended to projects that  receive a highly recommended in multiple selection criteria and that a  negative score on any of the selection criteria reduced the likelihood that  the project would receive a highly recommended overall rating.  Furthermore, DOT’s guidance stated that Evaluation Teams generally  should advance projects that received an overall highly recommended  score from four to five of the individual team members. Those receiving  three highly recommended overall scores were to be advanced only on a  case-by-case basis in consultation with other teams involved in the review  process. Projects receiving one to two highly recommended overall scores  generally were not to be advanced. Finally, DOT’s guidance noted that  Evaluation Teams should not advance any project unable to demonstrate a  likelihood of significant long-term benefits in the long-term outcome  criterion.
As the Evaluation Teams’ primary responsibility was to conduct a merit- based technical review of applications based on the criteria DOT  developed, according to DOT officials, they were not responsible for  addressing other factors in the TIGER review:    The Evaluation Teams were directed to consider information presented in  the applications—including project benefits and costs and the project’s  completion of National Environmental Policy Act requirements—but not  confirm its accuracy. Evaluation Teams were told that separate Economic  Analysis and Environmental Teams would determine the accuracy of the  benefits and costs and would validate projects’ environmental readiness.
The Evaluation Teams were not responsible for ensuring that applications  selected would meet the Recovery Act’s statutory requirements, including  achieving an equitable geographic distribution of funds and balancing the  needs of urban and rural communities. The teams did contribute to  prioritizing projects expected to be completed within the 3-year time  frame as part of project readiness, but they did not have to ensure projects  met this requirement. Finally, with regard to prioritizing applications in  which TIGER funding would complete a funding package, while the  Evaluation Teams could make recommendations on funding levels and  whether segments of a project (rather than the entire project) should be  funded, determining what level of funding to present to the Secretary of  Transportation as part of an award fell primarily to the senior-level Review  Team.
subsections: section_title: Control and Calibration Team
paragraphs: A Control and Calibration Team—led by a Deputy Assistant Secretary for  Policy with two staff members from OST’s Office of Policy—also reviewed  and advanced applications, and it did so both during the Evaluation  Teams’ assessments as well as later in the process when the Review Team  identified projects for award. According to DOT officials, the Control and  Calibration Team advanced applications primarily in two ways: It used a  statistical analysis to assess the ratings across the 10 Evaluation Teams  and ensure that projects of similar types and quality were advanced  consistently to the Review Team. This analysis was also intended to make  certain that there were no significant disparities in ratings among the  different transportation modes—an issue that, while not a requirement in  TIGER, officials believed was worth monitoring given TIGER’s unique  approach.
The Control and Calibration Team also advanced projects at the request of  the Review Team. In several cases, the Review Team asked to assess  projects of similar types in an effort to ensure that the most meritorious  projects of this type were selected for award. For instance, the Review  Team requested an analysis of the effect on port projects of the expansion  of the Panama Canal as well as a side-by-side comparison of all streetcar  applications and projects on Indian Reservations and federal lands. The  Review Team also asked the Control and Calibration Team to identify  additional projects to help them meet statutory requirements such as  geographic distribution and providing some funding in the form of credit  assistance. In response, the Control and Calibration Team, in consultation  with the Evaluation Team leads, identified additional projects beyond  those initially advanced by the Evaluation Teams for the Review Team to  consider, which resulted in additional projects being advanced that  received an overall ranking from the Evaluation Teams of recommended  rather than highly recommended.
section_title: Economic Analysis and Environmental Analysis Teams
paragraphs: DOT required applicants to include a description of the status of  environmental approvals as well as information on the project’s benefits  and costs. Applications advanced to the Review Team were reviewed by  an Environmental Analysis Team that assessed each advanced project’s  ability to substantially meet federal environmental readiness requirements.  In addition, an Economic Analysis Team composed of nine DOT  economists—including the Chief Economist and economists from relevant  operating administrations—assessed the economic analysis from each  advanced application to determine whether the analysis was “useful” or  “not useful” in its presentation of information and variables considered  and whether the total benefits of a proposed project were reasonably  likely to outweigh its costs. DOT required applicants to provide different  types of information of benefits and costs depending on the amount the  application requested. Specifically, projects requesting more than $100  million were required to calculate the net benefits of a project, indicate the  value assigned to qualitative benefits, and describe the methodology used  to arrive at this calculation. DOT directed applicants requesting more than  $20 million and less than $100 million to provide estimates of expected  benefits in the five long-term outcomes. Applicants requesting less than  $20 million did not have to submit a benefit-cost analysis. The Economic  Analysis and Environmental Analysis Teams presented their findings to  the Review Team, which considered this information along with other  factors in its assessment of applications.
section_title: Review Team
paragraphs: The Review Team consisted of 12 senior DOT staff, including the Deputy  Secretary, Under Secretary, three Assistant Secretaries, the Chief of Staff,  the General Counsel, and Administrators from the cognizant operating  administrations—the Federal Highway Administration, Federal Railroad  Administration, Federal Transit Administration, Maritime Administration,  and the Research and Innovative Technology Administration. This team  assessed all applications advanced by the Evaluation Teams and Control  and Calibration Team. The Review Team was responsible for addressing  four broad areas:    First, it was responsible for ensuring that the award recommendations  made to the Secretary, taken as a whole, met all statutory requirements,  including ensuring an equitable geographic distribution of funds, balancing  the needs of urban and rural communities, prioritizing projects for which  federal funding would complete an overall funding package that included  nonfederal sources, and prioritizing projects that could be completed  within 3 years of the act’s enactment.
Second, it assessed the merits of advanced projects by considering the  TIGER criteria applied by the Evaluation Teams and whether project  benefits outweighed costs. It accomplished this by receiving technical  presentations from the Evaluation Team leaders and the Economic  Analysis Team.
Third, it had to ensure that potential awardees were in fact eligible, ready- to-go, and that information in the application—such as expected  benefits—was accurate. To accomplish this, the Review Team requested  more information on some advanced projects. For example, in some cases,  validating environmental readiness required a follow-up conversation with  applicants to obtain clarification about the documentation submitted or  assurances provided in their applications.
Fourth, it recommended to the Secretary of Transportation which projects  to fund and whether an application should receive partial or full funding.
The Review Team’s initial assessments were conducted during a series of  meetings that occurred over about 2 months. In each meeting, the Review  Team evaluated about 6 to 12 projects, discussed project strengths and  weaknesses, identified areas for clarification or follow-up, and ranked  each project in a tier based on the likelihood that the team would fund the  project. At the conclusion of its assessment, the Review Team developed a  memo with a final list of projects that it recommended for award. This  memo included a description of each project’s strengths, benefits, and  how the project aligned with TIGER criteria. This memo was sent to the  Secretary of Transportation who approved all the recommended projects  and announced the TIGER recipients and award amounts in February  2010.
section_title: DOT Applied Competitive Selection Criteria and Addressed Statutory Requirements; However, It Did Not Document Key Decisions
subsections: section_title: The Review Team Assessed 166 Advanced Projects and Selected 51 Projects for Award
paragraphs: Of the 1,457 applications submitted, 8 percent or 115 applications received  an overall team rating of highly recommended and were advanced by the  Evaluation Team for further review. These 115 applications made requests  for funding that totaled about $7.7 billion—about five times the $1.5 billion  available for award. About 33 percent of the 1,457 applications received an  overall team rating of recommended. The remaining 59 percent received a  not recommended or negative rating or were excluded from evaluation for  reasons such as eligibility, readiness, or other factors that made the  application not acceptable to receive funding. Table 3 shows how all  applications were rated by the Evaluation Teams.
In addition to projects advanced by the Evaluation Team, the Control and  Calibration Team advanced 50 recommended projects and 1 not  recommended project to the Review Team, for a total of 166 advanced  projects. As noted, the Review Team ultimately selected for approval by  the Secretary of Transportation 51 of the 166 advanced projects for award.  The Secretary approved awards for each of these 51 applications.
TIGER awards were distributed across the country with 41 states and the  District of Columbia receiving awards and with roughly equal funding  levels (from 20 to 27 percent of funding) going to the four geographic  regions DOT established, as shown in figure 3. Awardees represented a  balance between the needs of urban and rural communities, as both  groups received awards in about the same proportion as applications  submitted and advanced. However, rural projects tended to receive  smaller awards and received 11 percent of the total funds. The average  TIGER award was just under $30 million. The largest award was $105  million for a large freight rail project in two states that would improve  intermodal domestic rail service. The smallest award was $3.15 million for  a project to improve a road and waterfront bike path in Vermont. Figure  3 shows the distribution of awards by region, jurisdiction size,  transportation mode, and funding level.
While there was no requirement to distribute awards across different  modes of transportation, TIGER funding supported highway, transit, rail,  port, and other projects—including bridge replacements, streetcar lines,  and bicycle-pedestrian networks (see table 4). Transit projects received  the most funding with 12 projects receiving $469 million. Although TIGER  grants were not awarded to intercity passenger rail projects, freight rail  projects received about 25 percent of the total award funds. Although  these projects received less funding overall than transit, funding levels  tended to be higher per project with 5 freight projects receiving a total of  $354 million.
In addition, TIGER-funded projects were eligible for federal credit  assistance through the Transportation Infrastructure Finance and  Innovation Act (TIFIA), which provides direct loans, loan guarantees, and  standby lines of credit to finance surface transportation projects of  national and regional significance. TIGER funds used for TIFIA grants  allow DOT to make a smaller financial commitment to support much  larger projects—specifically, DOT estimates that each dollar of federal  funds can provide up to $10 in TIFIA credit assistance. DOT offered TIFIA  awards to five applicants in the amount of either $10 million or $20 million  each. One applicant had applied for and received a TIFIA award, and four  applicants, while applying for a regular grant, were offered an opportunity  to use the grant funds as a TIFIA award, at their discretion. Three of these  four applicants opted to take the award as a grant and one took a TIFIA  award.
DOT officials said that, as required by the Recovery Act and part of DOT’s  partnership criterion, applicants who had secured funding commitments  from third parties such as state and local government and private industry  fared better in the TIGER process. Specifically, DOT noted in the Review  Team’s memo to the Secretary that 11 of the 51 awardees had arranged  funding partnerships for their projects and were seeking TIGER funding to  complete a funding package. While all TIGER awards were directed to  projects that applicants indicated could be completed as a result of the  award, these 11 awardees received 40 percent of the awarded funds. See  appendix IV for additional information on the awardees and selection  process and appendix V for information on the status of obligations and  outlays for TIGER grants.
section_title: Half the Awardees Were Chosen from Applications Advanced by the Evaluation Teams While the Other Half Were Advanced by the Control and Calibration Team
paragraphs: The Review Team selected 26 awardees from the pool of 115 highly  recommended applications advanced by the Evaluation Teams’  competitive review process. These applicants received about $950 million  of the funds. The other 25 applications selected for award, which received  about $549 million of TIGER funds, were from the pool of applications  advanced by the Control and Calibration Team. Applications advanced by  the Control and Calibration Team included 50 that received an overall  rating of recommended from the Evaluation Teams and one that received  a not recommended rating: Cincinnati’s streetcar project. DOT officials  said this project was advanced to the Review Team because it provided  additional context for the Review Team’s analysis of streetcar projects,  and it was not awarded a TIGER grant. Figure 4 shows the results of the  TIGER selection process, including applications advanced by the  Evaluation Teams and the Control and Calibration Team.
The recommended projects advanced by the Control and Calibration Team  tended to fare less well in the technical review process. As mentioned  earlier, each project received an overall rating from individual Evaluation  Team members as well as a consensus overall rating from the team as a  whole. The recommended projects advanced by the Control and  Calibration Team not only received lower overall consensus ratings from  the Evaluation Teams (recommended versus highly recommended), they  also received fewer overall highly recommended ratings from individual  Evaluation Team members. For example, as shown in table 5, the 51  projects advanced by the Control and Calibration team received a highly  recommended overall rating from individual team members less than one- fourth of the time. By comparison, projects advanced by the Evaluation  Teams received highly recommended overall ratings from individual team  members about two-thirds of the time.
Because the Review Team was responsible for considering a wider range  of factors than the Evaluation Teams, it is not unreasonable to expect that  the Review Team’s deliberations could produce a different result.  However, while DOT thoroughly documented the Evaluation Teams’  assessments and the reasons for its decisions and the Review Team’s  memo to the Secretary described the strengths and benefits of projects  recommended for award, DOT did not document the Review Team’s  reasons for its decisions, including the reasons for selecting recommended  projects over highly recommended ones. Most significantly, DOT did not  document Review Team meetings in which final decisions to recommend  or reject a project for award were made. Documentation of the Review  Team’s deliberations was limited to draft minutes from the team’s initial  assessments of advanced projects—a process that occurred in meetings  held over a period of about 2 months. The minutes were never finalized or  approved and were provided to us in draft form.
We analyzed these draft minutes and found that they reflected questions  Review Team members raised about the strengths and weaknesses of  various applications—questions consistent with TIGER criteria and  requirements. For instance, the Review Team raised questions about the  following projects, none of which received an award:    The extent to which financial commitments of project partners had been  secured. For instance, the Coos Bay Rail Line Rehabilitation Project in  Oregon proposed rehabilitating track so that a shortline railroad could  serve regional industrial operations, distribution facilities, and marine  terminals around the Coos Bay harbor and other locations in southwest  Oregon. The project would also have reconnected these facilities to the  national rail system. The Review Team raised questions about the project’s  financial commitments—specifically, whether there would be any  significant cost-sharing by the state or other commitments to the project.
Whether projects were sufficiently ready-to-go. For example, the North  Corridor Commuter Rail Project in North Carolina proposed to upgrade 25  miles of rail to permit faster passenger operations as well as construct new  passenger and maintenance facilities and acquire additional equipment.  The Review Team was concerned that this project would require an  environmental impact statement to satisfy the environmental readiness  requirements, which could substantially delay the project’s initiation.
Whether a project’s economic benefits were overstated. The West  Shoreway Project in Cleveland, Ohio, proposed to reconstruct 2.5 miles of  limited access highway along Cleveland’s lakefront into a boulevard with  six intersections, providing improved waterfront access. However, the  Economic Analysis Team characterized the analysis provided by the  applicant as “not useful” and therefore insufficient to demonstrate that the  project’s benefits exceeded its cost. In response, the Review Team asked  whether more data could be found on the potential benefits and economic  merits of the project. According to DOT officials, producing a useful  benefit-cost analysis was a challenge for many TIGER applicants. (See  appendix I for a discussion of steps DOT took to improve applicant  benefit-cost analyses in TIGER II.)
Our review of these narratives suggests that, on the whole, the team asked  reasonable questions and raised some valid concerns about many of the  projects they did not recommend for award. However, DOT officials told  us that these questions did not necessarily reflect the reason a project was  ultimately recommended or rejected for award. Furthermore, some of the  draft minutes simply noted that the Review Team did not see what made a  certain project more compelling than similar projects—a comment that  yields limited insight into why certain projects were selected and others  rejected.
To gain insight into the Review Team’s final decisions, we asked DOT  officials with the Control and Calibration Team overseeing the TIGER  process to reconstruct and discuss the reasons why projects were  recommended or rejected for award by the Review Team. While these  discussions offered some insight, such information has significant  limitations. Specifically, it is testimonial in nature and reflects officials’  recollections from several months after the completion of the TIGER grant  awards. It may therefore contain a greater level of uncertainty and error  than documentation created while decisions were being made.
According to these DOT officials, one important factor affecting award  decisions was the need to achieve an equitable geographic distribution of  funds (which, as noted earlier, DOT defined as distributing funding in  roughly equal amounts across four regions and without concentrating  projects in any one state within a region). DOT met this requirement by  rejecting some highly recommended projects to limit the number of  awards to regions that would have been overrepresented and by making  awards to recommended projects in regions that would have been  underrepresented. Specifically, officials stated that that 15 highly  recommended projects in the West and the Central regions were rejected  to limit the awards to these regions. In addition, although the Evaluation  Teams advanced 23 highly recommended projects from the South, the  Review Team recommended only 2 of these projects for award. DOT  officials indicated that these projects were rejected for a wide range of  reasons such as limited financial partnerships or the availability of other  funding sources such as tolls or user fees to support the project, among  others. As a result, the South was underrepresented for awards. To  address this, officials told us they advanced 14 additional Southern region  applications that had received lower overall ratings of recommended from  the Evaluation Teams, and 6 of these 14 recommended projects received  an award.
While the DOT’s draft minutes and our discussions with OST officials  provide some insight into the deliberations of the Review Team, because  there was no internal documentation from the Review Team meetings in  which final decisions to recommend or reject projects for award were  made, DOT cannot definitively demonstrate the basis for its award  selections, particularly the reasons why recommended projects were  selected for half the awards over highly recommended ones. For example,  without documentation DOT cannot demonstrate why statutory  requirements such as geographic distribution and other priorities such as  projects being ready-to-go and documenting their benefit costs analysis  could not have been achieved by selecting $1.5 billion of applications from  among the $7.7 billion in highly recommended applications advanced by  the Evaluation Teams. As our previous work has noted, documentation of  agency activities is a key part of accountability for decisions.  Furthermore, DOT’s Office of Inspector General (OIG) has published  several documents in which it raised questions about DOT’s discretionary  grant selections, noting that projects were not always selected based on  the relative priority assigned in a technical review. According to the OIG,  when decisions to fund projects deemed to be of lower-priority in a  technical review over higher-priority projects, a more thorough review and  analysis of project alternatives and documentation of the rationale used to  support decisions are necessary. DOT’s March 2009 Financial Assistance  Guidance Manual also stresses the importance of documenting such  decisions. This manual provides a standardized set of procedures for DOT  in processing and awarding grants, and it states that decisions not to fund  projects with the highest priority from a technical review shall be  documented. The absence of an insightful internal record of award  decisions and the reasons why final selections differ from the priorities  recommended from the technical review can give rise to challenges to the  integrity of the decisions made. DOT’s lack of documentation of key  decisions—particularly those in which it selected recommended projects  for award over those receiving a highly recommended rating in the  technical evaluation—makes it vulnerable to criticism that projects were  selected for reasons other than merit.
section_title: DOT Communicated the Criteria and Selection Process to Applicants and the Public but Provided More Limited Information on Award Decisions
paragraphs: DOT externally communicated information on the TIGER evaluation  criteria and selection process, and some of the applicants we interviewed,  each of which received awards, said they understood the criteria and  found DOT’s guidance helpful. According to grant policies and guidance  from the Office of Management and Budget, funding announcements that  clearly state selection criteria promote competition and fairness in the  selection of grantees. DOT’s Notice of Funding Availability for TIGER  included information on all of the statutory requirements and competitive  criteria that DOT used to evaluate the applications, as well as the relative  weights of the competitive criteria, and some of the applicants we  interviewed specifically said they understood the criteria and found the  information clear. For example, a Burlington Vermont Waterfront  Transportation Improvements North project official said that they were  able to find information on all of the criteria and application process  through publicly available sources, including the DOT Web site that posted  several “Questions and Answers” regarding TIGER. Several applicants we  interviewed said that DOT officials responded to questions in writing  online for the benefit of all applicants.
DOT made less information publicly available on the outcome of its  selection process—for instance, it did not publish the reasons for the  Review Team’s decisions or why some applications were selected while  others were rejected. However, in our review, we did not find any  requirements or guidance instructing federal programs to publicly disclose  the reasons for their selection decisions. Congress and the President have  emphasized the need for accountability, efficiency, and transparency and  have made these a central principle of the Recovery Act, but the act did  not define the attributes of transparency or how deep into the deliberative  process an agency’s actions should be transparent.
To assess the extent to which DOT publicly communicated outcome  information, we compared the information TIGER externally  communicated to the information communicated by 22 other similar  Recovery Act competitive grant programs (for a list of these programs, see  app. II). Only one of the programs communicated more outcome  information on technical scores and comments. Although it was not  required in the Recovery Act to do so, the Department of Education’s Race  to the Top grant fund published all of its ratings and decisions regarding  its applicants on its Web site, including the application, the score sheet  summarizing how the application was rated, narratives on the application  that describe the ratings, the application’s progression through the  selection process, and whether each applicant received an award.
In addition to these Recovery Act grant programs, we also compared the  TIGER program to the Federal Transit Administration’s discretionary New  Starts program—the federal government’s primary program for supporting  capital investments in rail and bus rapid transit systems. Like Race to the  Top, DOT’s New Starts program also published all scores. However, the  evaluations in these two programs were not structured identically to  TIGER. Specifically, New Starts did not have a second round of review and  used the ratings from its evaluative process as the basis for recommending  awards to Congress. Race to the Top used a different approach from  TIGER to gain further insight into applicants being considered for award— namely, it invited small groups of applicants to give oral presentations to a  panel before awards were finalized.
DOT officials told us they took actions to provide feedback to applicants  but have not yet developed a strategy for disclosing additional information  to the public. Specifically, officials noted that they provided one-on-one  discussions between DOT staff and applicants that requested feedback on  their TIGER applications. However, DOT officials also told us they  recognize that they will need to make the process more transparent. For  instance, officials told us they are exploring plans to increase the  program’s level of communication with the public for TIGER II and future  discretionary grant programs, although they have not yet decided what  additional information they would make available. Officials said they are  considering providing a summary abstract for each TIGER II application  that describes the project and its strengths and also indicates the rationale  for why it was selected or not selected. Officials said this approach could  increase transparency, show accountability for DOT’s decisions, and offer  an opportunity to improve applications in subsequent discretionary  programs. However, DOT officials also expressed concern that public  disclosure of considerations or opinions—favorable or unfavorable— taken into account by individuals or groups during the application review  and selection processes could hamper deliberation in future discretionary  grant selection processes.
Although DOT is not required to make this kind of complete and detailed  information public, in not doing so, it may be missing an opportunity to  better meet Congress’ and applicants’ needs. TIGER is a unique program  that distributes surface transportation funds based on merit and  performance across many modes of transportation on a competitive  basis—a new approach for DOT. TIGER also made federal investments in  projects like ports and freight rail infrastructure that rarely compete for  federal transportation funds. Congress expressed an interest in continuing  this new approach when it enacted TIGER II, and DOT has proposed a  new $2 billion discretionary grant program in its fiscal year 2012 budget  modeled after TIGER. Were DOT to make additional information on its  selection decisions publicly available, Congress would have more  information to help them better understand the basis on which the funding  is being distributed, and thus would have additional information about the  merits of this new approach and more confidence in the outcome. In  addition, the demand for TIGER funds was substantial, with over 1,450  applications received. Were DOT to make additional information on its  selection decisions publicly available, potential applicants would have  better information on how to develop and submit well-developed projects  that address significant regional and national transportation challenges.
section_title: Conclusions
paragraphs: The TIGER program represented an important step toward investing in  projects of regional and national significance on a merit-based,  competitive basis. Allocating federal funding for surface transportation  based on performance in general, and directing some portion of federal  funds on a competitive basis to projects of national or regional  significance in particular, is a direction we have recommended to more  effectively address the nation’s surface transportation challenges.  TIGER—and the TIGER II program that followed—was a novel approach  to funding surface transportation in that it distributed funds across many  modes of transportation and allowed projects like ports and freight  railroads that rarely compete for existing federal transportation funds to  participate. While Congress, when it enacted TIGER II, and the  Administration have expressed an interest in this new approach, the role  of discretionary grants in the funding the nation’s overall surface  transportation program is evolving. Formula funding is—and will likely  continue to be—the primary mechanism for distributing federal funds for  surface transportation. Congress has struck a careful balance in formula  programs to achieve equity among the states in how surface transportation  funds—in particular, highway funds—are distributed and to allow states to  select projects that reflect state and local priorities. There is a natural  tension between providing funding based on merit and performance and  providing funds on a formula basis to achieve equity among the states.  Consequently, meritorious projects of national or regional significance, in  particular those involving multiple modes of transportation or those that  cross geographic boundaries, may not compete well at the state level for  formula funds. Given that the Recovery Act was intended to create and  preserve jobs and promote economic recovery nationwide, Congress  believed it important that TIGER grant funding be geographically  dispersed. In the future, however, surface transportation competitive grant  programs provide Congress the opportunity to consider the appropriate  balance between funding projects based on merit and performance and  providing funds to achieve equity among the states.
TIGER was a new program for DOT, and the Recovery Act set short time  frames for establishing and administering the program. DOT met these  deadlines and developed a sound set of criteria to evaluate the merits of  applications and select grants that would meet the goals of the program.
Furthermore, it maintained good documentation of the criteria-based  evaluation conducted by its Evaluation Teams in the technical review and  effectively communicated information about its criteria to applicants—an  important step in promoting competition and fairness. By thoroughly  documenting how its technical teams considered and applied the criteria,  clearly communicating selection criteria to applicants, and publicly  disclosing some information on the attributes of the projects that were  selected, DOT took important steps to build the framework for future  competitive programs and its institutional capacity to administer them.  This foundation is important if there are going to be future rounds of  TIGER or similarly structured programs. Congress needs to have the best  information on how well the TIGER program has worked, and DOT needs  to gain the confidence of Congress and the public so that it can fairly and  expertly administer a multi-modal, multi-billion dollar discretionary  program.
The absence of documentation—in particular, the lack of documentation  regarding decisions to select recommended projects for half the awards  over highly recommended ones—can give rise to challenges to the  integrity of the decisions DOT made and subject it to criticism that  projects were selected for reasons other than merit. Documenting key  decisions, as good internal control practices and DOT’s guidance already  require, could provide a roadmap for administering future competitive  grant programs and help build confidence in DOT’s institutional ability to  administer this type of program. Furthermore, while federal agencies  rarely publicly disclose the reasons for their selection decisions in a  competitive review process, the uniqueness of the TIGER approach and  DOT’s limited experience with it suggests that publicly disclosing  additional information about selection decisions would give Congress a  better basis to assess the merits of this new approach and the information  it needs to judge whether and how to continue with it. Given DOT’s  concerns about the potential effect on internal deliberations, the decision  of what information to disclose publicly is best made by DOT in  consultation with the Congress, balancing DOT’s concerns with the  Congress’ need for information.
section_title: Matter for Congressional Consideration
paragraphs: If Congress enacts competitive discretionary grant programs such as the  TIGER program in the future, it may wish to consider balancing the goals  of funding projects through merit-based selection with achieving an  equitable geographic distribution of funds by establishing thresholds and  other mechanisms to limit, as appropriate, the influence of geographic  considerations.
section_title: Recommendations for Executive Action
paragraphs: To ensure that future rounds of the TIGER program or other similarly  structured competitive grant programs are accountable to Congress,  transparent to the public, and provide meaningful feedback to applicants,  we recommend that the Secretary of Transportation (1) document key  decisions for all major steps in the review of applications, particularly the  reasons for acceptance or rejection of applications and decisions in which  lower-rated applications are selected for award over higher-rated  applications, and (2) in consultation with the Congress, develop and  implement a strategy to disclose information regarding award decisions.
section_title: Agency Comments
paragraphs: GAO provided DOT with a draft of this report for its review and comment.  DOT officials provided technical comments via e-mail which we  incorporated as appropriate. DOT officials stated the department would  consider our recommendations.
As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies of this report to  congressional subcommittees with responsibilities for surface  transportation issues and the Secretary of Transportation. In addition, this  report will be available at no charge on GAO’s Web site at  http://www.gao.gov  If you or your staff have any questions about this report, please contact me  at (202) 512-2834 or herrp@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. GAO staff who made significant contributions to this report  are listed in appendix VI.
section_title: Appendix I: Lessons Learned from TIGER and Applied in TIGER II
paragraphs: The Department of Transportation (DOT) identified several challenges  during the Transportation Investment Generating Economic Recovery  (TIGER) evaluation process that it sought to address as “lessons learned”  in the TIGER II process. In TIGER, many projects were given a full review  multiple times before DOT determined their eligibility and readiness. The  Evaluation Teams tried to identify projects that were not eligible or were  not ready-to-go with respect to environmental approvals or securing other  financial commitments. However, some of these applications still  advanced to the Review Team and were thus reviewed again by the  Economic Analysis Team or the Control and Calibration Team before their  final status was determined. The solution in TIGER II was to implement a  pre-application process that required applicants to provide documentation  that would determine if each project or planning activity was (1) eligible,  (2) ready-to-go, and (3) had secured the necessary nonfederal funding  match claimed in the application. As stated in the TIGER and TIGER II  Notice of Funding Availability (NOFA), DOT officials required  documentation such as project schedules, environmental and legislative  approvals, state and local planning, and technical and financial feasibilities  for review. DOT officials said this pre-application process provided more  assurance that the final applications being reviewed were eligible and  ready-to-go, which improved efficiency and allowed teams to focus on the  merits of qualified applications.
A second challenge resulted from the fact that applications were randomly  assigned to the 10 Evaluation Teams, which also caused DOT to have to  review them a number of times. As a result of the random assignment,  each Evaluation Team reviewed projects from a mix of transportation  modes, which prevented a side-by-side comparison of the merits and  benefits for similar types of projects. Although the Evaluation Teams  assessed all applications and advanced only highly recommended projects  for further review, the Review Team wanted to review certain projects that  had significant similarities to ensure that the most meritorious projects  had in fact been advanced. For example, the Control and Calibration Team  advanced five recommended streetcar applications and several  recommended port projects that cited benefits from the expansion of the  Panama Canal. The Review Team reevaluated them all to determine which  projects were the most meritorious. The solution in TIGER II was to assign  similar types of projects to Evaluation Teams so that the initial review  assessed the merits of applications from the same types of projects side- by-side at the beginning of the process. Projects advanced for further  review were evaluated all together without respect for mode. DOT officials  told us this change allowed them to substantially improve the efficiency of  the review process and make awards expeditiously in TIGER II.
A third challenge in TIGER was for DOT to meet the statutory requirement  to prioritize awards to projects for which federal funding would complete  a funding package—a challenge, in part, because of the high demand for  TIGER funds and the statutory requirement that funds be equitably  distributed across the nation. In TIGER, DOT interpreted this requirement  to mean that it would give priority to projects that included substantial co- investment, and that it would fund discrete project segments from within  larger applications as long as these segments demonstrated “independent  utility,” which DOT defined as a segment of a larger application that  provided significant transportation benefits and created an operable  project when completed. However, although the TIGER NOFA did state it  would consider one or more components of a large project that met  selection criteria, it did not explicitly state that DOT would consider  funding project segments. Further, DOT officials said that the extent to  which applications provided information on how projects could be  segmented varied. As a result, DOT had to contact some applicants the  Review Team was considering for award to discuss whether projects  could be segmented to achieve operable and complete projects and at  what funding level. In TIGER II, DOT provided explicit guidance in the  NOFA defining independent utility and the potential for large applications  to receive partial funding. Further, TIGER II guidance requires the  applicant to identify and clearly describe the benefits of each discrete  project segment and how this segment aligns with selection criteria.
According to DOT officials, a final challenge in TIGER was that many  applications evaluated by the Economic Analysis Team were deemed to  not have useful analyses of expected project benefits and costs. Further,  DOT economists stated that many applicants also substituted economic  impact analyses, which typically focus on local and regional benefits  rather than national benefits. Because of this, the Economic Analysis  Team individually assessed the economic analyses from the 166 advanced  applications to determine the actual benefits and costs to present accurate  information to the Review Team, which they stated was a time-consuming  process. DOT officials thought the limited usefulness of applicants’  economic analyses was largely a consequence of applicants lacking  familiarity with how to properly conduct such analyses. For TIGER II,  DOT provided specific benefit-cost guidance that roughly accounted for  about one-third of the information in the TIGER II NOFA. DOT also  provided question and answer webinars on how to conduct a benefit-cost  analysis. Lastly, DOT provided training seminars for applicants explaining  the differences between benefit-cost and other analyses, the standards for  conducting proper benefit-cost analyses, and the characteristics of a useful  benefit-cost analysis. DOT also indicated in these sessions and in its  guidance that not including useful benefit-cost analyses might be the basis  for denying an award. As a result, DOT officials hoped to improve the  quality of the benefit-cost analyses applicants provided and that it would  be able to conduct an efficient assessment of them.
section_title: Appendix II: Objectives, Scope, and Methodology
paragraphs: To determine what criteria and processes were used in evaluating  applications and making award selections, we reviewed goals and  objectives for the TIGER program provided in the law, the Federal  Register, and DOT documents. We also reviewed guidance and  documentation of training provided to individuals serving as reviewers in  the grant evaluation process. Further, we reviewed prior GAO work  evaluating DOT’s TIGER grant criteria and guidance developed by DOT  and the Office of Management and Budget on leading practices for  managing discretionary grant programs.
To describe the outcomes of DOT’s evaluation process, we requested  documentation of the Evaluation and Review Teams’ assessments of  applications. Due to the level of documentation DOT maintained, our  ability to analyze this information differed substantially between the two  teams.
Evaluation Teams: DOT maintained thorough documentation of the  Evaluation Teams’ reviews, including individual team member and team  ratings, as well as associated narratives describing the strengths and  weaknesses of each application. We assessed the reliability of these data  by interviewing the leaders of each of the 10 Evaluation Teams, as well as  DOT officials who helped develop the process for Evaluation Teams to  record their assessments. Ratings of each application by each individual  team member were recorded in spreadsheets along with narratives  describing the individual team member’s assessment. Then, when the  entire Evaluation Team met to discuss scores and develop team ratings,  these scores and accompanying narratives were also recorded in a  spreadsheet. We reviewed these data for missing information, errors, and  other indicators of reliability. We determined that the data were  sufficiently reliable for the purposes of this report. We used these data to  summarize information on the initial pool of applicants, those advanced to  the Review Team, and those selected for an award. Specifically, we  categorized projects by award amount, region, transportation mode,  jurisdictional size, and other measures to describe patterns in the  characteristics of the projects and patterns of awards.
Review Team: DOT provided draft minutes that were never finalized or  approved from 15 of the 16 Review Team meetings. In each of these  meetings, the Review Team evaluated about 6 to 12 projects and made an  initial assessment and as needed identified issues in need of follow-up,  such as confirming financial commitments. The minutes did not include  what we understood was the final meeting to rate the remaining  applications (about 15 projects did not appear in the draft minutes). The  minutes also did not include information about the final award decisions.
Because DOT did not document the rationale for selecting recommended  projects for award over highly recommended ones, to gain insight into  award decisions, we asked DOT officials to reconstruct and discuss why  projects were recommended or rejected for award. This approach  provided some insight, but such information also has significant  limitations. Specifically, it is testimonial in nature and reflects officials’  recollections from several months after the completion of the TIGER grant  awards. It may therefore contain a greater level of uncertainty and error  than documentation created while decisions were being made.
We also examined the extent to which DOT’s competitive selection  process helped DOT to achieve statutory requirements for TIGER as  defined in the American Recovery and Reinvestment Act of 2009  (Recovery Act). We accomplished this by analyzing data on geographic  location, jurisdiction size, and other factors among applications submitted,  advanced, and awarded. We summarized information on the extent to  which DOT officials documented their decisions to advance applications  for the next round of review, and to select applications for an award.
To determine the extent to which DOT communicated the criteria,  process, and outcomes to applicants and the public, we compared  application review documents, including technical evaluation guidance  and information used to make awards decisions, with information  communicated to potential applicants through the Federal Register. In  addition, we interviewed DOT program officials, including technical  evaluation panelists, as well as examined outreach presentations and  documentation to understand the level of communication between  potential applicants and DOT officials regarding the TIGER process and  outcomes. We also interviewed a judgmental selection of TIGER awardees  to get their perspective on DOT’s level of communication. We spoke to  eight awardees because they could discuss the entire process and  outcomes. We judgmentally selected awardees based their funding level,  region, transportation mode, and jurisdiction size. Because this was not a  random or representative sample, the views of these applicants cannot be  generalized. We also reviewed other Recovery Act discretionary grant  program Web sites—22 in total—as well as DOT’s New Starts  discretionary grant program to determine the types of information these  programs make publicly available, and compared this level of  communication to the TIGER grant program. Table 6 lists these programs.
To determine what challenges DOT faced and the steps DOT took to  address them in TIGER II, we interviewed relevant DOT officials involved  in the TIGER evaluation process, including all 10 of the Evaluation Team  leaders and 2 members of the Review Team. We also reviewed DOT  webinars, reviewed guidance DOT issued in the form of “Question and  Answer” documents on the TIGER Web site, and reviewed supplemental  guidance. We also made comparisons between the TIGER and TIGER II  guidance documents to understand what had been changed or clarified  from the first round of review to the second.
We conducted this performance audit from June 2010 through March 2011  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix III: Standard Evaluation Sheet Used by Evaluation Teams to Evaluate Applications against TIGER Criteria
section_title: Appendix IV: Applications Received and Projects Selected for Funding
subsections: section_title: Composition of Initial Applicant Pool
paragraphs: DOT’s TIGER program received 1,457 applications requesting almost $60  billion. The applications were distributed among the regions with the  South submitting both the highest number of applications and the highest  percentage of the total funding request. See table 7 for a summary of the  regional distribution of the applications.
Highway project applications by far represented the largest number of  applicants by project type comprising over half the number of applicants  and amount requested. Transit projects accounted for the second largest  percentage with about 15 percent of applications and almost one-fifth of  the amount requested. See table 8 for a summary of the distribution of  project types among the applications submitted.
Considering the size of jurisdiction, urban areas made up the majority of  applications and funding requests with rural areas representing slightly  over one-quarter of applications and slightly less than one-quarter of  funds.
section_title: Composition of Applications Advanced for Further Review
paragraphs: From this initial applicant pool, DOT officials from both the Evaluation  Teams and the Control and Calibration Team advanced 166 applications  requesting approximately $11 billion in TIGER funds for further review. Of  these, DOT advanced just over one-fifth of applications from the  Northeast, Central, and South regions. The West region, however, had  more applications advanced than the other regions. The Northeast,  Central, and West regions requested around the same proportion of funds  requested by forwarded applications, but the South requested substantially  more, around one-third of the total. The requested amounts for advanced  projects were roughly in line with the funding requests for all applicants  within each region. As discussed in the report, decisions on which  applications to advance were a combination of the applications’ ratings  against DOT’s selection criteria and other factors such as geographic  distribution requirements. See table 9 for a summary of the regional  distribution of the advanced applications.
Approximately 40 percent of the advanced applications were for highway  projects. Transit projects were about one-quarter of the total advanced  applications and requested funds. Rail, port, and other projects each had  about one-tenth of applications forwarded. These applicants also  requested less funding overall. See table 10 for a summary of the  distribution of project types among the advanced applications.
Both urban and rural applications were advanced in roughly the same  proportion in number and funding level requested as they had applications  submitted.
section_title: Composition of Selected Projects
paragraphs: DOT officials selected 51 projects to receive about $1.5 billion in TIGER  funds. Of the many characteristics of the TIGER awardees, there was one  discernable trend with respect to which projects were selected to receive  the most funds. DOT officials said that awardees with significant leveraged  funds from third parties fared well in the process. For example, while DOT  noted leveraging or partnerships for 11 of the 51 awardees in its decision  rationale document, these 11 awardees received 40 percent of the  distributed funds. DOT officials said that this trend occurred because they  selected applicants that could leverage funding from third parties,  allowing DOT to complete funding packages for more applicants. In  conjunction with the tendency to select leveraged projects, DOT also  awarded partial funding to all but six applicants. The average amount of  funds made available was about half of what had been requested with  some applicants receiving only 3 percent of what they had requested.  However, DOT officials said that all awards went to applicants that could  complete a project with independent utility.
TIGER awards ranged from 20 percent to 27 percent of funds awarded  over the four geographic regions. The four regions received roughly the  same percentage of TIGER funds with the Northeast and the West  receiving 27 percent each and the South receiving 20 percent. Of note,  while the South requested 39 percent of funds in the initial application  pool and 33 percent of funds for advanced applications, it ultimately  received about one-fifth of the funding. This was due in part to questions  about the economic benefits of projects and additional funds of some  highly recommended Southern applications, as well as lower evaluative  ratings for Southern projects overall. See table 11 for a summary of the  regional distribution of the awardees.
Funding awards among selected project types ranged from 7 percent to 31  percent. For example, transit projects received about 31 percent of the  total funds, whereas port projects received 7 percent of the total. See table  12 for a summary of the distribution of project types among the TIGER  awardees.
There was no discernable trend for urban or rural projects in terms of the  number of projects selected for funding, as both locality groups received  funding amounts in about the same proportions as applications submitted  and advanced. However, rural projects tended to be smaller and received  only 11 percent of the TIGER funds.
section_title: Appendix V: Status of Implementation of TIGER Grants
paragraphs: As of February 2011, DOT had executed all but two of the 59 grant  agreements and obligated $1,468 million in TIGER funds. In addition, DOT  has outlayed $39 million of the obligated funds. DOT delegated  responsibility for completing grant agreements, obligating funds, and  overseeing the projects to the operating administrations based on the  project’s transportation mode and the applicant. The Federal Railroad  Administration (FRA) and Maritime Administration have executed all of  their grant agreements and obligated all of their funds. The two  outstanding grant agreements are both Transportation Infrastructure  Finance and Innovation Act (TIFIA) projects, which required additional  time to complete. For example, one project in Colorado opted to use its  $10 million grant as TIFIA assistance, and doing so required additional  traffic and revenue studies before the agreement could be finalized.  Recovery.gov tracks the amount of funds obligated by state, but not the  grant agreements.
DOT divided oversight responsibilities among the operating  administrations, depending upon the transportation mode of the project in  each grant agreement as well as prior institutional relationships between  the applicant and DOT. Specifically, DOT considered the capacity of the  operating administrations and whether there was an existing oversight  relationship between a grantee and a federal agency. For example, some  rail projects were delegated to the Federal Highway Administration  (FHWA) because FHWA has existing relationships with the relevant states  and freight railroads, and FRA has limited capacity for overseeing grants,  especially with the large volume of High Speed and Intercity Passenger  Rail grants FRA is overseeing. Also, projects such as improving grade  crossings can be overseen by FHWA because FHWA has a history of  oversight with these types of projects as they affect both rail and  highways.
According to officials, DOT is interested in performance measurement and  is taking steps to begin building knowledge on this subject across the  department. Our prior work has shown that measuring performance  allows organizations to track the progress they are making toward their  goals and gives managers crucial information on which to base their  organizational and management decisions. TIGER presents an  opportunity to focus on performance and results in transportation  projects, and DOT officials incorporated performance measurements into  grant agreements. Officials believe that every TIGER project can  incorporate and collect performance measures, but regions and states  have varying capabilities. For example, localities with existing data  collection programs and resources may be able to collect and manage  performance information; however, other localities may find such a task  more challenging, according to DOT officials. In these cases, DOT has  asked these awardees to collect information that would serve as a  precursor to performance measurements, which should help to build  capacity and experience among these localities. DOT is currently  developing and experimenting with the best methods for measuring  objectives and collecting data, and it is working collaboratively with  applicants to weigh different options for performance measurements.
section_title: Appendix VI: GAO Contact and Staff Acknowledgments
subsections: section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, GAO staff who made major  contributions to this report are Steve Cohen (Assistant Director), Joah  Iannotta (analyst-in-charge), Carl Barden, Aisha Cabrer, David Hooper,  Sarah Jones, SaraAnn Moessbauer, Amy Rosewarne, and Max Sawicky.