Public Transit: Length of Development Process, Cost Estimates, and Ridership Forecasts for Capital-Investment Grant Projects
Published: 2014-05-30T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: FTA provides funds to transit project sponsors to build new or extensions to existing fixed-guideway transit systems through the Capital Investment Grant program. This program funds New, Small, and Very Small Starts projects—funds that are based partly on the project's total estimated cost. For example, for New Starts, project capital costs exceed $250 million or the program contribution exceeds $75 million; for Small Starts, capital costs are less than $250 million and the program contribution is less than $75 million. The pre-construction development process for these projects includes various steps between the time when a project sponsor identifies the project to be funded and the formal award of FTA construction funds. During this process, the scope, capital cost, and ridership estimates can change.
The Moving Ahead for Progress in the 21st Century Act mandated that GAO biennially review these types of projects. This report describes (1) the length of the development process across these projects and the factors affecting the length, (2) capital cost-estimate changes throughout this process, and the factors contributing to the changes, and (3) how project sponsors forecast ridership, including support that FTA provides. GAO analyzed pertinent laws, regulations, agency guidance, and FTA data for the 32 New, Small, and Very Small Starts projects initiated and funded from 2005 to 2013, prior to recent changes in program processes. GAO interviewed FTA staff and project sponsors. DOT reviewed a draft of this report and provided technical comments, which were incorporated as appropriate.
section_title: What GAO Found
paragraphs: For the 32 New Starts, Small Starts, and Very Small Starts projects funded from 2005 to 2013 that GAO reviewed, the length of the development process varied substantially, from as little as 2 to as long as 14 years, based on GAO's analysis of data from the Federal Transit Administration (FTA) and project-sponsors. GAO found that the development process took 3 to 14 years to complete for New Starts projects, 3 to 12 years for Small Starts projects, and 2 to 11 years for Very Small Starts projects. The length of the process is generally driven by factors that are often unique to each project, including (1) the extent of local-planning activities prior to formal approval for funding, (2) the extent and availability of local and financial support, and (3) the extent of FTA oversight activities. For example, sponsors of 17 of the 32 projects GAO reviewed stated that activities to secure local funding contributed to the length of the development process. FTA has taken some steps to streamline this process. For example, in January 2012, FTA eliminated the requirement for the development of a hypothetical alternative that served as a basis of comparison to evaluate a proposed project.
GAO found that capital cost estimates for New Starts, Small Starts, and Very Small Starts projects during the development process generally did not change substantially prior to the award of federal funding. For 23 of the 32 projects GAO reviewed, the final cost estimated prior to receiving federal funding was within 10 percent of the original cost estimates. The remaining 9 projects varied by as much as 41 percent lower and 55 percent higher than the estimates used at the end of the development process. Several project sponsors told us that, when changes did occur, it was a result of changing market conditions and FTA's recommending that sponsors increase project costs to cover unforeseen events, among other factors. For example, officials at the Valley Transportation Authority, located in Santa Clara, California, stated that FTA recommended that it increase the project's cost by $100 million to cover unforeseen events.
New and Small Starts project sponsors whom GAO interviewed generally forecast ridership using regional travel models prepared by metropolitan-planning organizations (MPO). Specifically, 8 out of the 9 New Starts project sponsors and 3 out of 4 Small Starts project sponsors GAO spoke with use these travel models. For example, for a Portland, Oregon, streetcar project, the project sponsor used travel forecasts prepared by the Portland MPO. The other New Starts and Small Starts project sponsors use actual transit-ridership data from surveys of regional transit riders; and a statewide travel model, respectively. On the other hand, FTA procedures permit sponsors of Very Small Starts projects to essentially demonstrate, through a detailed counting of riders of existing public transportation in the project's corridor, that the proposed project will service at least 3,000 transit riders on an average weekday. FTA has taken a number of actions to support the development of ridership forecasts. These include, among other actions, providing funding to state agencies and MPOs to help them collect travel data and develop forecasting procedures and providing technical support, such as reviews of final forecasts. GAO interviewed 13 New Starts and Small Starts project sponsors and most said that FTA's technical assistance, which includes reviewing the ridership forecasts, was generally helpful.

section_title: Letter
paragraphs: The Federal Transit Administration’s (FTA) Capital Investment Grant  program, which includes the New Starts and Small Starts programs,  provided, as we found in a 2012 report, about $15.1 billion in federal  capital funds over the past 8 fiscal years to help many states, cities, and  localities plan and build new “fixed-guideway” systems or extensions to  existing fixed-guideway systems. Since the early 1970s, the federal  government has provided a large share of the nation’s capital investment  through the New Starts program. The Safe, Accountable, Flexible,  Efficient Transportation Equity Act—A Legacy for Users (SAFETEA-LU)  created the Small Starts program for funding lower-cost fixed-guideway- system projects in 2005. FTA created the Very Small Starts project  category for funding very low cost projects at the same time. While MAP- 21 made certain revisions to the program, because this report covers  projects under SAFTEA-LU prior to the implementation of MAP-21  changes, we focus on SAFETEA-LU related requirements. Although FTA  has provided funding for New, Small, and Very Small Starts projects,  these projects are designed and implemented by project sponsors,  typically local transit agencies. The project sponsors often coordinate with  local metropolitan-planning organizations (MPOs) in designing and  implementing these projects, and FTA awards funding to project sponsors  upon completion of the pre-construction development process, which  includes a range of local policy development and decision-making  activities, where the specific transit corridor and project are identified, and  then refined as design progresses. FTA evaluates and rates the projects  selected, and, ultimately, awards funding. The scope and estimates of  capital costs and ridership can change as the local project sponsor  refines the project before FTA awards funding.
The Moving Ahead for Progress in the 21st Century Act (MAP-21)  mandated GAO to biennially review FTA’s processes and procedures for  evaluating, rating, and recommending new fixed-guideway capital  projects and core-capacity improvement projects and the Department of  Transportation’s (DOT) implementation of such processes and  procedures. In this report, we focus on the 2005 through 2013 time  period to identify (1) the extent to which the length of the development  process varies across New Starts, Small Starts, and Very Small Starts  projects and what factors affect the length of this process; (2) the extent  to which capital cost estimates for New Starts, Small Starts, and Very  Small Starts projects change throughout the development process, and  what factors contribute to the changes; and (3) how project sponsors  forecast ridership, including any support that FTA provides in helping  them develop these forecasts.
To address all of these objectives, we reviewed and summarized relevant  laws, such as SAFETEA-LU, FTA circulars, and policy guidance, as well  as our body of work on the Capital Investment Grant Program. To  determine the extent to which the length of the development process  varied across New Starts, Small Starts, and Very Small Starts projects  and the extent to which capital cost estimates for these projects changed  throughout the development process, we collected and analyzed  schedule milestone data and capital-cost estimate data from FTA on all  32 of the New Starts, Small Starts and Very Small Starts projects that had  been: (1) approved to enter the preliminary engineering phasedevelopment process following the implementation of SAFETEA-LU in  October 2005 and (2) awarded a grant agreement prior to the March 2013  implementation of the current surface transportation legislation, MAP-21.  To verify and assess the reliability of the data compiled by FTA, we  compared it to project data we received from project sponsors we  interviewed. We resolved any discrepancies with the data with FTA  headquarters staff, and we determined that the data were sufficiently  reliable for the purposes of this report. To provide insight on the factors  contributing to the project’s timeline trends and challenges and the  project’s cost-estimate changes and to obtain information on how   of the  ridership forecasts are developed, we interviewed 23 project sponsors  representing 30 of the 32 projects. The information obtained from these  interviews is not generalizable to all New Starts, Small Starts and Very  Small Starts projects. We also interviewed FTA officials to determine the  support that FTA provides to help project sponsors develop ridership  forecasts. Further details on our scope and methodology can be found in  appendix I.
We conducted this performance audit from August 2013 to May 2014 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: Under SAFETEA-LU, FTA’s primary source of funding for new fixed- guideway capital projects or extensions to existing fixed-guideway-transit  systems was the Capital Investment Grant program. Within the Capital  Investment Grant program, project sponsors typically applied for funding  as either a New Starts or Small Starts project. FTA’s New Starts projects  under SAFETEA-LU were defined as new fixed guideway or extensions to  existing fixed guideway capital projects with a total capital cost of $250  million or more or a Capital Investment Grant program contribution of $75  million or more. The Small Starts program was created by SAFETEA-LU  in 2005 to provide a more streamlined evaluation and rating process for  lower-cost and less complex projects, defined as new fixed-guideway  projects, extensions to fixed guideways or corridor-based bus projects  whose estimated capital cost was under $250 million and whose Capital  Investment Grant program contribution was under $75 million. Within the  Small Starts program, as defined in SAFETEA-LU, FTA created a  category for very low cost Small Starts projects, known as Very Small  Starts. These projects must contain the same elements as Small Starts  projects and also contain the following three features: (1) location in  corridors with more than 3,000 existing riders per average weekday who  will benefit from the proposed project, (2) have a total capital cost of less  than $50 million for all project elements, and (3) have a per-mile cost of  less than $3 million, excluding rolling stock (such as train cars, buses,  etc). As part of the application process, sponsors of New Starts, Small  Starts, and Very Small Starts projects are expected to identify local  sources of funding to contribute to the project along with federal funding  provided through both the Capital Investment Grant program and  potentially other sources of federal funding.
The steps in the development process depend on whether a project is a  New Starts project or a Small or Very Small Starts project (see fig.1).
New Starts. Under SAFETEA-LU, sponsors of New Starts projects  were required by statute to go through a planning and development  process. In the alternatives analysis phase, project sponsors identified  the transportation needs in a specific corridor and evaluated a range  of alternatives to address the locally identified problems in that  corridor. Project sponsors completed the alternatives analysis phase  by selecting a locally preferred alternative (LPA). Subsequently,  during the preliminary-engineering phase, project sponsors refined the  design of the locally preferred alternative and its estimated costs,  benefits, and impacts. Further, under the National Environmental  Policy Act of 1969 (NEPA), as amended, and implementing   regulations, New Starts project sponsors were required to complete  the NEPA environmental review process to receive Major Capital  Investment program funding. When the preliminary -engineering  phase was completed and federal environmental requirements are  satisfied, FTA may approve the project’s advancement into final  design, if the project obtained an acceptable rating under the statutory  evaluation criteria and met other readiness requirements. For a  project to receive funding, FTA needed to recommend it for a full  funding grant agreement (FFGA) in the President’s budget.
Small Starts. Under SAFETEA-LU, the development process for Small  Starts was condensed by combining the preliminary-engineering and  final-design phases into one “project development” phase. When  projects applied to enter project development, FTA evaluated and  rated them according to the statutory criteria. Under SAFETEA-LU,  there were fewer statutory criteria specified for Small Starts projects  compared to New Starts projects. Either using annual appropriations  or existing FTA appropriations that remain available, FTA provided  funding for Small Starts projects in one of two ways: through project- construction grant agreements (PCGA) or single-year construction  grants when the Small Starts funding request was less than $25  million. For a project to receive funding, FTA needed to recommend  it in the President’s budget.
Very Small Starts. Very Small Starts projects also progressed through  a single-project development phase and were evaluated and rated on  the same project criteria as Small Starts projects. However, they  qualified for automatic medium or better ratings, which required  submittal of less data to FTA, because they had sufficient existing  transit ridership in the corridor and met low cost parameters to  “warrant” them for satisfactory ratings. FTA provided funding for Very  Small Starts projects through PCGAs or single-year construction  grants. For a project to receive funding, FTA needed to recommend it  in the President’s budget.
Under SAFETEA-LU, any transit project that fit the definition of a new  fixed-guideway capital project or extension to an existing fixed-guideway  project was eligible to compete for funding under the Capital Investment  Grant program that provides funding for New Starts, Small Starts, and  Very Small Starts projects. Such projects included:
Commuter rail—systems that operate along electric or diesel- propelled railways and provide train service for local, short distance  trips between a central city and adjacent suburbs.
Heavy rail—systems that operate on electric railways with high- volume traffic capacity and are characterized by separated rights-of- way, sophisticated signaling, high platform loading, and high-speed,  rapid-acceleration rail cars operating singly or in multi-car trains on  fixed rails.
Light rail—systems that operate on electric railways with light-volume  traffic capacity and are characterized by shared or exclusive rights-of- way, or low or high-platform-loading, single or double-car trains, and  overhead electric lines that power rail vehicles.
Streetcars—systems that are similar to light rail, but distinguishable  because they are usually smaller and designed for shorter routes,  more frequent stops, and lower travel speeds.
Bus rapid transit (BRT)—bus systems that vary in design, but  generally included service enhancements to attract riders and provide  similar transit-related benefits as rail transit, characterized by  improvements such as dedicated bus lanes, improved stations,  improved vehicles, off-vehicle fare collection, special branding of the  service, and frequent service, among other things.
Pub. L. No. 112-141, § 20008, 126 Stat. 405 (2012). As noted previously, this report  describes the project development process in effect from October 2005 through March  2013, prior to the implementation of changes from MAP-21.
New Starts project. FTA has not yet implemented these changes fully,  but has issued some guidance on how these changes will affect the  program.in the future, though FTA officials told us that there is no firm date on  when the various policy changes will take effect.
FTA plans to conduct additional rule-making on MAP-21 topics  FTA and its project-management oversight contractors are to provide  oversight during the development process. FTA maintains its  headquarters in Washington, D.C., with 10 regional offices throughout the  continental United States, and employs about 500 employees to oversee  and provide funding for a variety of transit programs and initiatives,  including for New Starts, Small Starts, and Very Small Starts projects.  FTA and its contractors are to conduct oversight reviews throughout the  project’s planning and design as well as before FTA recommends a  project for funding; these reviews are to include an evaluation of the  project’s risk, scope, cost, schedule, and project management plan, as  well as the project sponsor’s technical capacity and capability. Project  sponsors also submit periodic updates to FTA on different aspects of  major projects, such as the cost, expected schedule, and projected  ridership of the project. In addition, proposed projects are required to  complete the NEPA environmental review processes in order to receive  federal funding. Specifically, NEPA and implementing regulations require, among other things, an environmental review document with  information on alternative courses of action and a consideration of social,  economic, and environmental impacts of the proposed transportation  improvement.
section_title: The Length of the Development Process for Projects Varies Substantially Depending upon Project-Specific and Local Factors
subsections: section_title: Length of the Development Process Varies Substantially across Projects
paragraphs: Among the 32 transit projects we reviewed, we found significant variation  in the length of time sponsors of New Starts, Small Starts, and Very Small  Starts projects needed to complete the development process (see fig. 2).  Specifically, for the approved projects we examined, the development  process ranged from 2 years for a 6.8 mile Bus Rapid Transit project in  Monterey, California, to over 14 years for a 30 mile commuter rail project  in Denver, Colorado. The wide range of time needed to plan, design, and  secure funding for these transit projects prior to construction is similar to  the range of time that is generally considered necessary to plan and  design other types of capital projects. For example, studies have  suggested that for major highway projects the activities leading up to the  construction of a highway may vary from 1 year for a minor project to 7 to  14 years for a major project.
The variation across projects is attributable, in part, to conditions and  factors specific to each project.
For some projects, the development process was lengthy. For  example, for the Eagle Commuter Rail Line project, the development  process, beginning with the selection of the locally preferred  alternative in 1997 until the project was awarded an FFGA, lasted  over 14 years. Project sponsors stated that they did not pursue entry  into preliminary engineering until after completing further investment  studies that expanded the scope of the project in the early 2000s and  securing funding through a local referendum in 2004. In addition, prior  to entering the Capital Investment Grant pipeline, officials worked to  finalize the technology for the project and secure approval from the  project sponsor’s board of directors. Officials stated that once the  project was approved into the pipeline in 2009, the project progressed  quickly through the remainder of the process, and the project sponsor  secured an FFGA approximately two and a half years after  acceptance into preliminary engineering.
For sponsors of the Mason Corridor project, which successfully  completed a Small Starts bus rapid-transit line from Aspen to  Glenwood Springs, Colorado, the development process extended  over 11 years as a result of challenges related to, among other things,  securing funding for the project and obtaining agreement for the  project among local stakeholders.
However, for other projects, the length of the development process was  comparatively shorter. For example, the development process for the 7.3- mile, 10 station, Portland-Milwaukie light-rail New Starts project lasted  about 4 years. In this case, project sponsors stated that they encountered  no major obstacles during this time, though they noted that the process  was extended by 6 months while the project sponsor identified additional  local funds and reduced to project’s scope in response to lower than  anticipated federal funding for the project. We will discuss in more detail  the general types of factors that affected the length of the development  process later in the report.
In general, larger projects, such as those that applied for funding as New  Starts projects, required more time to progress through the development  process than smaller projects, such as those that apply for funding as  Small and Very Small Starts projects. On average, the development  process was 17 months longer in duration for New Starts projects than for  Small Starts projects and 12 months longer than for Very Small Starts  projects. Specifically, according to our analysis of FTA and project  sponsor data, we found that New Starts projects took about 3 to 14 years  to complete the development process, Small Starts projects took about 3  to 12 years, while Very Small Starts projects took about 2 to 11 years.  According to FTA officials, the length of the development process is  unique for each project, and generally depended upon the project’s  specific characteristics, such as scope, corridor location, and availability  of local funding, among other factors.
Some of the variability across the New Starts, Small Starts, and Very  Small Starts projects resulted from activities that took place later in the  process, after the locally preferred alternative was selected and before  the project was formally accepted into FTA’s pipeline. The “pipeline” is a  sub-component of the overall development process and is defined as the  period of time between when a project is accepted into the preliminary- engineering (New Starts) or project-development (Small and Very Small  Starts) phase and the final award of construction funding by FTA.  Depending on the project, the time between the selection of the locally  preferred alternative and entry into the pipeline took as little as a few  months to over a decade. According to project sponsors, activities during  this period included revising the project scope, securing local funding, and  preparing to enter into the project pipeline, among other things. Once a  project was been accepted into the pipeline, we found that the length of  the process was similar across all three project categories, and generally  lasts from 2 to 5 years, and averages about 3 years (see fig. 3). However,  within each of the three types of projects, the length of time in the pipeline  for an individual project varied widely depending on the project’s specific  characteristics. Furthermore, as previously discussed, FTA officials stated  it is difficult to characterize an “average” project, as each proposed transit  project has its own unique project characteristics, physical-operating  environment, and challenges.
While there was substantial variation in the length of the overall  development process within and across transportation modes, the  variation in the Capital Investment Grant pipeline duration was similar  across the modes of the projects we reviewed. (See table 1.) For  example, light rail projects required 3 to 10 years to complete the  development process, while commuter rail projects required 5 to 14 years  to complete this process. As previously noted, a portion of the variability  in length of the development process was due to activities that occur after  a locally preferred alternative is selected but before FTA accepted the  project into the pipeline. However, projects generally required 2 to 5 years  to progress through the pipeline, regardless of the mode proposed by a  project sponsors.
section_title: Local Factors Substantially Affect the Length of the Development Process
paragraphs: Our review found that local factors, specific to each project, were  generally the primary elements that determined the development  process’s length. Furthermore, our prior work has found that some of  these of factors—particularly obtaining project funding and community  support—also commonly affect the length of time to complete other types  of capital projects, including highway projects.
Local Financing: Project sponsors noted that securing local funding,  such as through local sales taxes and referendums, can be  challenging. We previously found that local funding remains a  substantial component of the overall funding for New Starts, Small  Starts, and Very Small Starts projects. Sponsors of 17 of 32 of the  projects we reviewed stated that activities to secure local funding  contributed to the length of the development process. For example,  the project sponsor of the Mason Corridor BRT stated that securing  local commitment was particularly challenging and extended the  development process by about 7 years. The project sponsor selected  a bus rapid transit as a locally preferred alternative in 2000, but was  unable to secure local funding until 2007. The time needed to identify  and secure local funding was a significant factor in extending the  development process over 11 years, as federal funding for the project  was contingent upon the project’s sponsor securing a local funding  source. Similarly, the project sponsor of the Mid-Jordan light rail  project in Salt Lake City, Utah, stated that securing local funding for  the project delayed its development by about a year. According to the  project’s sponsor, while it selected the locally preferred alternative in  2005, the project sponsor did not enter into preliminary engineering  until 2007, when the project had secured funding through a local  referendum in 2006 that increased the local sales tax.
Local Community Support: The development process can also be  extended as a result of efforts project sponsors undertake to secure  local community approval for a project. Sponsors of 12 of the 32 of the  projects we reviewed stated that community support for their project  affected the length of the development process. For example, project  sponsor officials who oversaw the development of a BRT project in  Northern California stated that the major hurdle in the development of  the project was overcoming some community opposition to the  planned route that arose in 2008 after the selection of the locally  preferred alternative. Specifically, the alignment of the project was  scheduled to go through a residential area where residents had  historically opposed the location of a bus route. According to the  project sponsor, the change provoked some community opposition to  the project, and as a result, the alignment of the project was modified.  Officials estimated that the development process was extended by  about 3 to 6 months. However, community support can take a  significant amount of time to develop and sustain. For example,  officials from the project sponsor overseeing the construction of the  High Capacity Transit Corridor heavy rail project in Honolulu worked  for 10 years—starting before the locally preferred alternative was  selected—to develop support for the project.
Stakeholder Coordination: Coordinating with other local government  agencies as well as other transportation providers can also be  challenging and may affect the length of the development process.  Specifically, for 8 of the 32 projects we reviewed, sponsors stated that  the process of coordinating with other stakeholders extended the  development process. For example, project sponsors overseeing the  development of a light rail project in Charlotte, North Carolina, had to  coordinate with local freight-rail operators and Amtrak to relocate  service to minimize disruption to Amtrak’s existing service. The  project’s design was modified as a result of these negotiations, which  extended the project’s development nearly 7 months, according to the  project’s sponsor. Project sponsor officials stated that they did not  anticipate the complexity of the negotiations with the railroad  operators and, accordingly, noted that earlier coordination with these  operators might have accelerated the project timeline modestly.
Environmental review. The effect of a proposed project on the local  environment, as well as steps required by law to mitigate  environmental impacts from the proposed project, may also affect the  duration of the development process. Specifically, under the NEPA  environmental review process, project sponsors may measure the  impact of different alternatives by the extent to which the alternative  meets the project’s purpose and need, and is consistent with the  goals and objectives of any local urban planning. The NEPA  environmental review process also requires federal agencies to  evaluate and in some cases prepare detailed statements assessing  the environmental impact of and alternatives to major federal actions  significantly affecting the environment. However, according to FTA  officials and project sponsors, this process can be time consuming.  Specifically, both FTA officials and project sponsors for 8 of the 32  projects we reviewed noted that the required NEPA environmental  review process, may add time to the development process. For  example, officials from one project sponsor stated that it took nearly  two and a half years to complete the NEPA process.
Scope and configuration changes. The factors described above may  also result in revisions to the project’s scope and configuration, which  may in turn extend the development process. Revisions to a project’s  design ranged from minor alterations to pedestrian access to a project  to changes to a project’s proposed alignment or service route. Project  sponsors for 7 of the 32 projects we reviewed identified changes in  scope as a factor affecting the development process, which  sometimes resulted from one of the factors described above. For  example, for the Charlotte light rail project described above, in  addition to the design changes arising from coordination with local  stakeholders, recession-related reductions in the sales-tax revenue  funding the project forced the project sponsors to further revise the  project, thus extending the overall development process.
section_title: FTA Has Taken Steps and Identified Future Opportunities to Reduce the Duration of the Development Process
paragraphs: According to some project sponsors we interviewed, FTA assistance is  generally helpful in completing the development process, though they  noted that the duration of some oversight reviews can be lengthy. We  have previously found FTA and its oversight approach have improved  sponsor’s management of their projects. However, for 12 of the 32  projects we reviewed, project sponsors stated that some types of  oversight reviews can be time-intensive and extend the development  process, sometimes by weeks or months. For example, project sponsors  for 2 of the 32 projects we reviewed cited FTA’s risk assessment as a  requirement that affected the length of the development process.addition, sponsors of 4 of the 16 Very Small Starts projects we reviewed  speculated that some of the longer review times for smaller projects may  have been a result of FTA’s initial uncertainty in how it would implement  the simplified review process for Very Small Starts projects. Conversely,  one project sponsor noted that because much of the development  process is driven by local factors, there was not much FTA could have  done to accelerate the process.
Finding the right balance between protecting federal investments through  project management oversight and advancing projects through the  development process is challenging. We have previously found that a  balance exists between expediting project development and maintaining  the rigor and accountability over the development of New Starts, Small   Furthermore, we have previously  Starts, and Very Small Starts projects. found that FTA’s oversight efforts help the agency ensure that a federally  funded transit project’s scope, schedule, and cost are well developed and  that the project’s design and construction conform to applicable statutes,  regulations, and guidance.take longer than expected, because project sponsors sometimes provide  information that is incomplete or inaccurate, resulting in additional review  time and delays. While FTA has acknowledged that the process can be  lengthy and frustrating, FTA has taken some steps over the last several  years to further streamline the development process.
We also previously found that reviews may  In its January 2013 final rule implementing some MAP-21 changes,  FTA eliminated the requirement for the development of a baseline  alternative, removing the requirement to compare a proposed project  to a hypothetical alternative.reviewed stated that development of a baseline alternative was a  time- and resource-consuming part of the development process.
Project sponsors for 3 of 32 projects we
The January 2013 final rule also allows proposed projects to  automatically receive a satisfactory rating on a certain rating and  some evaluation criteria based on the project’s characteristics or the  characteristics of the project corridor. For example, for Small Starts  projects, if the operating and maintenance cost of the proposed  project is less than 5 percent of current system-wide operating and  maintenance cost, the project qualifies for automatic medium or better  rating on its local financial commitment evaluation. FTA officials told  us that they plan to explore expanding the types of projects that may  prequalify for automatic ratings.
In September 2013, FTA introduced a new tool to assist project  sponsors in estimating ridership on their projects. According to FTA  officials, the tool, known as the Simplified Trips-on-Project Software  (STOPS), may help to significantly shorten the time projects’ sponsors  need to develop ridership estimates. We will discuss ridership  estimation for projects later in this report.
FTA estimates these changes could reduce the development process  time for projects by six months or more. as more MAP-21 requirements are formally implemented through the  rule-making process, it may identify additional efficiencies in the  development process.
section_title: The Majority of Capital Cost Estimates Did Not Change Significantly during the Development Process, and Changes That Did Occur Were due to Various Factors
paragraphs: We found that capital cost estimates for New Starts, Small Starts, and  Very Small Starts projects during the development process generally did  not change substantially prior to the award of federal funding. Project  sponsors told us that cost estimate changes occurred as a result of  changing market conditions, FTA’s application of additional project  contingencies and scope modifications, among others. However, most  estimates did not change much from the initial capital cost estimated  upon entry into the development process.
subsections: section_title: While Cost Estimates Changed Significantly for Several Projects, the Majority of Capital Cost Estimates Did Not
paragraphs: The majority of cost estimates of the projects we reviewed did not change  significantly. For 23 of the 32 projects we reviewed, the original cost  estimated upon entry into the Capital Investment Grant pipeline was  within 10 percent of the final cost estimated prior to receiving federal  funding. The original capital cost estimates for the remaining 9 projects  varied by as much as 41 percent lower and 55 percent higher from the  estimates used at the end of the development process. Of those projects,  4 were New Starts, 3 were Small Starts, and 2 were Very Small Starts  projects. Figure 4 shows the range of cost changes for these projects.
While the majority of the capital cost estimates did not change  significantly during the development process, some estimates did change.   However, we did not assess project sponsors’ cost-estimating  procedures, or related FTA policies, and how that might have contributed  to the cost estimates that did change. But, as noted in our previous  reports, federal agencies have experienced challenges in conducting cost  estimating—some of the agencies’ programs cost more than expected  and deliver results that do not satisfy all requirements.may experience some of those same challenges. Reliable capital-cost   Project sponsors  estimates are necessary for the New Starts program for a number of  reasons:  to support decisions about funding one capital improvement project  over another, to develop annual funding requests to Congress, to evaluate resource requirements at key project-development  decision points, and  to develop performance measurement baselines.
We plan to examine FTA’s and project sponsors’ implementation of best  practices for developing and managing capital program costs in future  work on the Capital Investment Grant program.
section_title: Several Factors Have Contributed to Cost Estimate Changes
paragraphs: Our review identified a number of factors that led to cost estimate  changes during the development process, as described below. In some  cases, a combination of factors contributed to cost estimate changes.
Economic and Market Conditions. Nine project sponsors stated that  economic conditions, such as the recession from 2007 to 2009, likely  increased competition for some of its contracts and created a bidding  environment favorable to the agency for reducing costs. For example,  Livermore Amador Valley Transit Authority (located in Livermore,  California) officials stated that because of the recession, companies  submitted lower bids than initially anticipated on each of the four  major construction contracts associated with the project. According to  the officials, the project finished about $4.5 million under its approved  budget, due in large part to the recession. The Utah Transportation  Authority (UTA) also stated that the recession affected the cost  estimate over the development process. The officials said that the  recession created competition that helped reduce the construction  costs associated with these projects due to a reduced demand for  construction and contracting services. However, these types of  projects are also sensitive to changes in material prices. For example,  right before FTA awarded the grant for the Mid-Jordan project, the  cost of steel increased substantially, adding $1.5 million to the cost of  the overall project.
Contingency levels. According to project sponsors, capital cost  estimates for 6 projects increased as a result of FTA’s risk and  contingency reviews. For example, officials at Valley Transportation  Authority (VTA) (located in Santa Clara, California) stated that, as part  of FTA’s risk assessment review, the project-management oversight  contractor recommended an increase in the contingency amount for  the project by $100 million. VTA officials further stated that  contingency amounts fluctuated throughout the development process  as the design of the project was further refined. We have previously  found that FTA’s risk reviews have helped to improve project  sponsors’ controls over project costs and provided FTA with a better  understanding of the issues surrounding projects, such as the  potential problems that could lead to cost increases.reviews to analyze whether the project sponsor has included a  sufficient level of contingency within their cost estimate.
Scope and Configuration Changes. Project sponsors stated that the  scope of 12 projects was reduced or increased significantly during the  development process, changes that led to capital cost-estimate  changes. For example, the Minneapolis Metropolitan Council stated  that its project had a $24 million increase from preliminary engineering  to FFGA, and $15.6 million of the increase in capital cost that was  attributable to inclusion of three at-grade infill stations.
Refined Cost Estimate as Project Progressed: Because the majority of  project estimates were developed in the planning stage, they will  continue to change as part of the development process. For example,  the Denver Regional Transportation District stated that the capital cost  estimate for their project decreased because as the project advanced  through the project development process, the cost estimators had a  better idea of the project’s scope and design, which led to more  accurate cost estimating. Generally, the more information that is  known about a project, the more accurate and less variable the  estimate is expected to be. We have previously found that cost  estimates are based on many assumptions and are expected to  change as project requirements are clarified.
section_title: Project Sponsors Use Regional Travel Models to Forecast Ridership, and FTA Provides Funding and Technical support
subsections: section_title: Project Sponsors Generally Forecast Ridership Using Regional Travel Models
paragraphs: Project sponsors rely on support from MPOs to develop their ridership  forecasts. According to FTA officials, most travel-forecasting procedures  are maintained by MPOs. The MPOs produce travel forecasts as they  prepare transportation plans for metropolitan areas and assess the plans’  conformity with federal air-quality requirements. Based on a  Transportation Research Board (TRB) study on metropolitan travel  forecasting, MPOs estimate future travel demand and analyze the  impacts of alternative transportation investment situations using  computerized travel-demand-forecasting models. According to this study,  forecasts derived from travel models enable policy makers to make  informed decisions on investments and policies relating to the  transportation system. In a 2009 report, we found that these MPO travel  models are complex and require inputs of extensive current information  on roadway and transit system characteristics and operations, as well as  current and forecasted demographic information. Creating and  operating the models requires a high degree of technical training and  expertise. However, we also found in 2009 that some MPOs face  challenges in travel demand forecasting, including a lack of technical  capacity and data necessary to conduct the complex transportation  modeling required to meet planning needs. The TRB also noted that  MPOs face a much broader and more complex set of requirements and  needs in their travel modeling.
By and large, New and Small Starts project sponsors whom we  interviewed generally use the regional travel models of the project  sponsor’s local MPO to forecast ridership.
Eight out of the nine New Starts project sponsors reported using MPO  travel models. For example, officials from the Regional Transportation  District (Denver, CO) said that the local MPO’s (Denver Regional  Council of Government) approved regional travel-demand model is  used to develop the Regional Transportation District’s ridership  forecasts. Officials from the Utah Transit Authority (Salt Lake City,  Utah) also used a regional travel model maintained by the Wasatch  Front Regional Council—the MPO for the Salt Lake City area. The  model incorporates information from highway usage, rail, and other  mass transit ridership, as well as transit rider surveys. However, one  project sponsor, Sound Transit (Seattle, WA) used the incremental  method to forecast its ridership. This method essentially uses actual  transit ridership data, which includes, among other data, observed  origins and destinations of transit users and surveys of region-wide  transit riders.
Three out of four Small Starts project sponsors use travel models  developed by the local MPO. For example, for the Portland, Oregon,  Streetcar Loop project, the Tri-County Metropolitan Transportation  District of Oregon (TRIMET) used travel forecasts prepared by the  Portland Metropolitan Planning Organization. According to TRIMET  officials, the model includes and is continually updated with  employment and population data, as well as, data on roadway and  transit routes. According to these officials, the MPO travel model is  one of the more sophisticated ridership models for an urban area.  One project sponsor used its statewide travel model to forecast  ridership, instead of a local MPO travel model. According to the  Montachusett Regional Transit Authority (Fitchburg, MA), it used a  local travel model which was a component of the overall  Massachusetts state travel model to forecast ridership.
Project sponsors that use regional travel models to forecast transit  ridership for New Starts and Small Starts projects are required to test the  forecasts for accuracy against current data describing actual transit  ridership, per FTA requirements. To implement this test, the travel models  are used to prepare a forecast of current ridership using current  population, employment, highway conditions, and transit service levels.   According to FTA, comparisons of these current-year forecasts against  current-year data demonstrate the extent to which the models grasp  actual ridership patterns and support improvements to the models when  errors are evident. When the models are able to pass the tests, they are  then ready to make forecasts for the proposed project.
FTA procedures permit Very Small Starts project sponsors to document  current transit volumes in the project corridor and thereby avoid the need  to prepare ridership projections for the project. As previously mentioned,  according to FTA, one of the key requirements for a Very Small Starts  project is that at least 3,000 existing transit riders will use the proposed  project on an average weekday. Through this requirement, FTA can  ensure that the proposed project will have sufficient ridership and produce  enough travel benefits to be considered cost-effective without having to  do detailed travel forecasts or other complicated analysis to prove the  project is justified. To adequately document the required number of  existing transit riders, the sponsoring agency must conduct a detailed  counting of riders of existing public transportation in the project corridor,  and estimate the number of existing riders that will use the Very Small  Starts project. FTA guidance requires that the counts be conducted on  existing routes serving the project corridor that either: operate on the street segments where the Very Small Start will on streets parallel or nearby that will be rerouted to operate on the  Very Small Starts street segments after the project is completed.
For example, the Los Angeles County Metropolitan Transportation  Authority (Metro) developed its ridership projections for its two Very  Small Starts bus rapid transit projects we reviewed, based on actual  experience with another bus rapid-transit service. According to Metro  officials, in order to validate ridership projections, Metro used data  collected from its Automatic Passenger Counter system on the  existing bus rapid-transit service. Metro officials told us that automatic  passenger counters are installed on every bus in its fleet to provide  accurate passenger ridership data.
FTA has endorsed two alternative approaches for developing ridership  forecasts that rely less on travel models and more on current data on  actual travel patterns.  1.  Incremental Methods rely on rider survey data to describe current  transit ridership patterns. This method essentially focuses on changes  in transit ridership caused by proposed projects and by growth in  population and employment. According to FTA officials, in corridors  where transit is well established, these incremental methods offer a  quick, and possibly more reliable, ridership-forecasting approach.  2.  Simplified Trips-on-Project Software package (STOPS) which FTA  released in September 2013, is an approach that local agencies can  use instead of, or in conjunction with, metro-area models. STOPS  uses data from the Census Transportation Planning Package  (currently from the 2000 decennial census) to replace some  component models and provides already calibrated models of transit- versus-auto choice. For local agencies whose travel models are not  ready to provide reliable forecasts for transit projects, STOPS offers  an alternative that can avoid the need for project sponsors to perform  data collection and model updates, processes that can sometimes  take as long as 2 years to complete. Instead, using STOPS,  developing ridership forecasts can take as little as 2 weeks.
We did not assess the adequacy of any of these travel models. However,  the TRB study noted that there is no single approach to travel forecasting  or set of procedures that is correct for all applications or all MPOs.  Additionally, the study stated that FTA is to be commended for taking  steps to ensure quality in the travel forecasting methods used for major  project planning. In particular, the study noted that FTA’s initiatives to  ensure the quality of New Starts ridership forecasting have been useful in  uncovering weaknesses and that FTA has taken a strong role in  improving modeling practice.
section_title: Uncertainties Create Challenges for Developing Ridership Forecasts
paragraphs: According to FTA officials, regardless of the approach project sponsors  use to forecast ridership, all ridership forecasts have uncertainties. FTA  officials identified at least two areas of uncertainty:
Data inputs that are forecasts. Travel models require information on  population, employment, household incomes, transit service levels,  transit fares, highway capacity, and other influences on travel  patterns. Consequently, ridership forecasts for future years are  grounded in predicted future conditions rather than data on actual  conditions. For distant years and in rapidly growing metro areas,  uncertainties in these predictions can be large.
Optimism. Sponsors and planners of new transit projects anticipate  good outcomes. As a result, optimistic assumptions are common on  such things as operating speeds, accessibility to stations, and the  amount of new development within a given area. Travel models tend  to compound this across-the-board optimism in many ways leading to  forecasts that may be much more optimistic than any one of the  inputs, and this optimism may lead sponsors to reject less-than- hoped-for ridership projections and search for ways to increase the  projections.
Some project sponsors we interviewed also identified the following  challenges affecting ridership estimates:
The difficulty in developing accurate population and employment  growth estimates.
The unpredictability of gas price levels on ridership. For instance,  higher prices will encourage higher ridership and a large decline in  prices will discourage ridership. One project sponsor told us that the  economy has a significant effect on ridership, and more specifically,  the economy affects the price of gas and cost of parking, which in turn  affects ridership.
section_title: FTA Supports the Development of Ridership Forecasts through Funding and Technical Support
paragraphs: FTA has taken a number of actions to support the development of  ridership forecasts. These include the following:  Funding. According to FTA officials, the agency contributes funding to  state agencies and MPOs to support, among many other activities, the  collection of travel data and the development of travel-forecasting  procedures. MPOs receive annual funding from both the Federal Highway  Administration and FTA, in addition to state matching funds. Nationally,  FTA’s share of this funding is about $129 million for fiscal year 2014.
Technical support. FTA told us that, since the inception of the Capital  Investment Grant program, it has filled at least one staff position with a  nationally recognized expert in travel forecasting who is responsible for  assisting project sponsors in the development of travel forecasts and for  oversight of Capital Investment Grant project ridership forecasts. FTA has  also allocated approximately two full-time staff to oversight activities.  These activities include the following:
Technical assistance in travel forecasting methods development.
According to FTA, at the invitation of local agencies, FTA staff  provides comments, participates in peer-review panels, and engages  in ongoing discussions with local project sponsors and their  contractors during the development of new travel forecasting  procedures for metropolitan areas.
Early reviews of methods and assumptions. FTA officials also stated  that the agency encourages project sponsors and their contractors to  meet with FTA staff early in the preparation of forecasts in support of  proposed projects. These officials said that this early engagement  identifies potential problems with forecasting methods and planning  assumptions at a point in time when these issues can be dealt with  efficiently—essentially avoiding late surprises when project sponsors  have finished their forecasts.
Reviews of final travel forecasts. Before a proposed project is  approved for entry into preliminary engineering (New Starts) or project  development (Small Starts), FTA staff review the travel forecasts  submitted by project sponsors in support of these projects. Staff  document any significant uncertainties found in the forecasts and  make recommendations to FTA’s Office of Planning and Environment  regarding acceptance of the forecasts as sufficiently reliable for the  agency’s use in project evaluation and rating.
We interviewed 13 New Starts and Small Starts project sponsors, and a  majority (7) said that FTA’s technical assistance, which includes  reviewing ridership forecasts, was generally helpful. For example, an  official from the Metropolitan Council (Minneapolis and St. Paul, MN) told  us that he has found it useful that FTA reviews its ridership forecasts for  different projects. In particular, it can be very insightful to have FTA ask  probing questions regarding forecasts, a process that gives project  sponsors a quality check on the veracity of their ridership-forecast  numbers. In another example, officials from Valley Metro (Phoenix, AZ)  told us that FTA provided them assistance for 9 months as the ridership  forecasting was being developed, assistance that helped them deliver a  credible document for evaluation and rating. Furthermore, another project  sponsor said that the FTA team that reviewed its ridership projections  was both thorough and timely with its reviews.
Requirement for testing of travel models. In 2007, FTA required that local  travel models used to forecast transit ridership for New Starts and Small  Starts project be tested for accuracy against current data describing  actual transit ridership. According to FTA, the requirement ensures that  local methods used to prepare ridership forecasts submitted to FTA have  been demonstrated to have a basic grasp of current local transit ridership.  FTA officials said that the 2013 policy guidance on the Capital Investment  Grant program continues this requirement.
section_title: Agency Comments
paragraphs: We provided DOT with a draft of this report for review and comment. DOT  provided technical comments, which we incorporated as appropriate.
We are sending copies of this report to interested congressional  committees and the Secretary of the Department of Transportation. In  addition, this report will be available at no charge on GAO’s web site at  http://www.gao.gov. If you or your staff have any questions or would like  to discuss this work, please contact me at (202) 512-2834 or  wised@gao.gov. Contact points for our Offices of Congressional  Relations and Public Affairs may be found on the last page of this report.  Individuals making key contributions to this report are listed in appendix  IV.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: The Moving Ahead for Progress in the 21st Century Act (MAP-21)  mandated GAO to biennially review FTA’s processes and procedures for  evaluating, rating, and recommending new fixed-guideway capital  projects and core capacity improvement projects and the Department of  Transportation’s (DOT) implementation of such processes and  procedures. In this report, we identify (1) the extent to which the length of  the development process varies across New Starts, Small Starts and  Very Small Starts projects and what factors affect the length of this  process, (2) the extent to which capital cost estimates for New Starts,  Small Starts, and Very Small Starts projects change throughout the  development process, and what factors contribute to the changes, and (3)  how project sponsors forecast ridership, including any support that FTA  provides in helping them develop these forecasts.
To address all of these objectives, we reviewed and summarized relevant  laws, such as The Safe, Accountable, Flexible, Efficient Transportation  Equity Act: A Legacy for Users (SAFETEA-LU), FTA circulars and policy  guidance, as well as our body of work on the Capital Investment Grant  Program. To determine the extent to which the length of the development  process varies across New Starts, Small Starts, and Very Small Starts  projects and the extent to which capital cost estimates for these projects  change throughout this process, we collected and analyzed project  milestone data and cost estimate data (see apps. II and III) from FTA staff  and analysis of FTA Annual Reports on Funding Recommendations for  fiscal years 2008 through 2014. We included all 32 New Starts, Small  Starts, and Very Small Starts projects that had been: 1) approved to enter  preliminary engineering following SAFETEA-LU (October 2005) and (2)  awarded a grant agreement prior to the implementation of MAP-21  (March 2013). To verify and assess the reliability of the data compiled by  FTA, we compared it to project data we received from project sponsors  we interviewed. We resolved any discrepancies with the data with FTA  headquarters staff, and we determined that the data were sufficiently  reliable for the purposes of this report.
To provide insight on the factors contributing to project’s timeline trends  and challenges and project cost-estimate changes and to obtain  information on how ridership forecasts are developed, we interviewed 23  project sponsors representing 30 of the 32 projects. Table 2 lists the  New Starts, Small Starts, and Very Small Starts project sponsors we  interviewed for our review. The information obtained from these  interviews is not generalizable to all New Starts, Small Starts, and Very  Small Starts projects. We also interviewed FTA officials to determine the  support that FTA provides to help project sponsors develop ridership  forecasts.
We conducted this performance audit from August 2013 to May 2014 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Key Milestone Dates and Cost Estimates for Selected New Starts Projects Reviewed by GAO (Dollars in Millions)
paragraphs: Appendix II: Key Milestone Dates and Cost  Estimates for Selected New Starts Projects  Reviewed by GAO (Dollars in Millions)
section_title: Appendix III: Key Milestone Dates and Cost Estimates for Selected Small and Very Small Starts Projects Reviewed by GAO (Dollars in Millions)
paragraphs: Appendix III: Key Milestone Dates and Cost  Estimates for Selected Small and Very Small  Starts Projects Reviewed by GAO (Dollars in  Millions)
subsections: section_title: Project
section_title: Appendix IV: GAO Contact and Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Brandon Haller (Assistant  Director), Antoine Clark, Catherine Colwell, Dwayne Curry, Geoffrey  Hamilton, Terence Lam, Jason Lee, Karen Richey, Amy Rosewarne,  Kelly Rubin, and Matt Voit made key contributions to this report.