Public Transportation: Opportunities Exist to Improve the Communication and Transparency of Changes Made to the New Starts Program
Published: 2005-06-28T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Transportation Equity Act for the 21st Century (TEA-21) and subsequent legislation authorized about $13.5 billion in guaranteed funding for the Federal Transit Administration's (FTA) New Starts program, which is used to select fixed guideway transit projects, such as rail and trolley projects, and to award full funding grant agreements (FFGA). GAO assessed the New Starts process for the fiscal year 2006 cycle. GAO identified (1) the number of projects that were evaluated, rated, and proposed for new FFGAs and the proposed funding commitments in the administration's budget request; (2) changes FTA has made to the New Starts application, evaluation, rating, and oversight processes since the fiscal year 2001 evaluation cycle and how these changes have been communicated to project sponsors; and (3) how FTA developed the measures used to evaluate and rate projects from the criteria outlined in TEA-21 and how those measures are used in the rating process.
section_title: What GAO Found
paragraphs: For the fiscal year 2006 evaluation cycle, FTA evaluated and rated 27 projects and identified 4 projects that were expected to be ready for new FFGAs before the end of fiscal year 2006 and an additional 6 projects that may be eligible for other funding outside of FFGAs. The administration's fiscal year 2006 budget proposal requests $1.5 billion for the New Starts program, a request similar to that of the past 2 years. FTA has made 16 changes to the New Starts application, evaluation, rating, and oversight processes since the fiscal year 2001 cycle that were primarily intended to make the process more rigorous and systematic. Seven of the 16 changes underwent rulemaking, including providing formal notice to the transit industry and soliciting comment, while 9 changes did not. FTA officials said that these nine changes are consistent with the existing regulations governing the New Starts process or relate to the project development oversight process rather than the evaluation and rating process and, therefore, in their opinion, do not need to undergo formal rulemaking. By not consistently soliciting public opinion, however, FTA is missing an opportunity to obtain stakeholder buy-in, increase the transparency of the New Starts process, and lessen potential difficulties project sponsors face in implementing the changes. Many of the measures FTA uses to evaluate and rate New Starts projects have evolved over time, with industry input, through formal rulemaking and informal efforts, such as workshops and reports. Although both TEA-21 and FTA's New Starts program regulations emphasize the importance of using a multiple-measure approach for evaluating projects, FTA assigns weight to all three financial criteria but only two of the five project justification criteria in developing a project's rating. FTA officials said that they do not use the other three project justification criteria--which are specified in TEA-21--because the measures fail to distinguish among projects. Project sponsors we interviewed offered suggestions for improving all of the project justification measures, and FTA has efforts underway to improve some of the measures.

section_title: Letter
paragraphs: Much of the federal government’s share of new capital investment in mass  transportation since the early 1970s has come through the Federal Transit  Administration’s (FTA) New Starts program, which awards full funding  grant agreements (FFGA) for fixed guideway rail, bus rapid transit, and  ferry projects across the country. An FFGA establishes the terms and  conditions for federal participation in a project, including the maximum  amount of federal funds available for the project, which by statute cannot  exceed 80 percent of its net cost. Since fiscal year 1998, the New Starts  program has provided state and local agencies with more than $8.8 billion  to help design and construct transit projects throughout the country.
Under the Transportation Equity Act for the 21st Century (TEA-21) and  subsequent amendments and other legislation, Congress authorized  approximately $13.5 billion in New Starts commitment authority through  2005. Even though the level of program funding is higher than it has ever  been, demand for these funds has also been extremely high. For that  reason, TEA-21 directed FTA to prioritize projects for funding by  evaluating, rating, and recommending potential projects on the basis of  specific financial and project justification criteria—including mobility  improvements, cost-effectiveness, environmental benefits, and operating  efficiencies. In applying these criteria, TEA-21 directed FTA to consider a  number of additional factors, including land use and congestion relief.  Furthermore, TEA-21 required FTA to issue regulations for the evaluation  and rating process, which FTA did in December 2000.
TEA-21 also requires us to report each year on FTA’s processes and  procedures for evaluating, rating, and recommending New Starts projects  for federal funding and on the implementation of these processes and  procedures. This report discusses (1) the number of projects that were  evaluated, rated, and proposed for new FFGAs for the fiscal year 2006  evaluation cycle and the proposed funding commitments for New Starts in  the administration’s fiscal year 2006 budget request; (2) changes that FTA  has made to the New Starts application, evaluation, rating, and oversight  processes since TEA-21 and how those changes have been communicated  to project sponsors; and (3) how FTA developed the measures it uses to  evaluate and rate projects from the criteria identified in TEA-21 and how  the agency uses these measures in the evaluation process. To address these  objectives, we reviewed the administration’s fiscal year 2006 budget  request; the legislation and regulations governing the New Starts process;  and FTA’s annual New Starts reports, reporting instructions, and other  program guidance and documentation. We also interviewed FTA officials  and representatives from the American Public Transportation Association,  the New Starts Working Group, and the projects that were rated in the  fiscal year 2006 evaluation cycle. In addition, we attended FTA’s meetings  with project sponsors—the New Starts Roundtables—in April and May  2005. We conducted our work from November 2004 through May 2005 in  accordance with generally accepted government auditing standards. (See  app. I for more information about our scope and methodology.)
subsections: section_title: Background
paragraphs: TEA-21 authorized a total of $36 billion in “guaranteed” funding for a  variety of transit programs, including financial assistance to states and  localities to develop, operate, and maintain transit systems. Under one of  these programs, New Starts, FTA identifies and selects fixed-guideway  transit projects for funding—including heavy, light, and commuter rail;  ferry; and certain bus projects (such as bus rapid transit). FTA generally  funds New Starts projects through FFGAs, which establish the terms and  conditions for federal participation in a New Starts project and also define  a project’s scope, including the length of the system and the number of  stations; its schedule, including the date when the system is expected to  open for service; and its cost.
To obtain an FFGA, a project must progress through a local or regional  review of alternatives and meet a number of federal requirements,  including providing information for the New Starts evaluation and rating  process (see fig. 1). As required by TEA-21, New Starts projects must  emerge from a regional, multimodal transportation planning process. The  first two phases of the New Starts process—systems planning and  alternatives analysis—address this requirement. The systems planning  phase identifies the transportation needs of a region, while the alternatives  analysis phase provides information on the benefits, costs, and impacts of  different corridor-level options, such as rail lines or bus routes. The  alternatives analysis phase results in the selection of a locally preferred  alternative—which is intended to be the New Starts project that FTA  evaluates for funding, as required by statute. After a locally preferred  alternative is selected, project sponsors submit a request to FTA for entry  into the preliminary engineering phase. Following completion of  preliminary engineering and federal environmental requirements, the  project may be approved by FTA to advance into final design, after which  the project may be approved by FTA for an FFGA and proceed to  construction, as provided for in statute. FTA oversees grantee management  of projects from the preliminary engineering phase through construction  and evaluates the projects for advancement into each phase of the process,  as well as annually for the New Starts report to Congress. We have  recognized the New Starts program as a good model that the federal  government could use for approving other transportation projects.
To help inform administration and congressional decisions about which  projects should receive federal funds, FTA assigns ratings based on a  variety of financial and project justification criteria, as defined by its  program regulations, and then assigns an overall rating. These criteria are  identified in TEA-21 and reflect a broad range of benefits and effects of the  proposed project, such as cost-effectiveness, as well as the ability of the  project sponsor to fund the project and finance the continued operation of  its transit system (see fig. 2). Projects are rated at several points during the  New Starts process—as part of the evaluation for entry into preliminary  engineering and final design, and yearly for inclusion in the New Starts  annual report.
FTA assigns the proposed project a rating of high, medium-high, medium,  low-medium, or low for each criterion, then assigns a summary rating for  local financial commitment and project justification. Finally, FTA develops  an overall project rating of “highly recommended,” “recommended,” “not  recommended,” or “not rated.”  (See table 1 for the criteria FTA uses to  evaluate projects.) The exceptions to this process are statutorily “exempt”  projects, which are those that request less than $25 million in New Starts  funding. These projects are not required to submit project justification  information—although FTA encourages them to do so—and do not receive  ratings from FTA; thus, the number of projects in preliminary engineering  or final design may be greater than the number of projects evaluated and  rated by FTA.
As required by statute, the administration uses the FTA evaluation and  rating process, along with the stage of development of New Starts projects,  to decide which projects to recommend to Congress for funding.  Although many projects receive an overall rating of “recommended” or  “highly recommended,” only a few are proposed for FFGAs in a given fiscal  year. FTA proposes “recommended” or “highly recommended” projects for  FFGAs when it believes that the projects will be able to meet certain  conditions during the fiscal year that the proposals are made. These  conditions include the following:  All non-New Starts funding must be committed and available for the  project.
The project must be in the final design phase and have progressed to the  point where uncertainties about costs, benefits, and impacts (e.g.,  environmental or financial) are minimized.
The project must meet FTA’s tests for readiness and technical capacity,  which confirm there are no cost, project scope, or local financial  commitment issues remaining.
section_title: FTA Identified Four New Projects for FFGAs and Requested $1.5 Billion for Fiscal Year 2006
paragraphs: Of the 34 projects in preliminary engineering or final design for the fiscal  year 2006 cycle, 27 were evaluated and rated and 7 were statutorily exempt  from the rating process. Four projects were recommended for funding with  the expectation that they would be ready for new FFGAs before the end of  fiscal year 2006, and an additional 6 projects were identified as potentially  receiving a recommendation for New Starts funding outside of FFGAs. The  administration’s fiscal year 2006 budget proposal requests a total of $1.5  billion for the New Starts program, an amount similar to requests for the  past 2 fiscal years. (See app. II for the administration’s 2006 budget  proposal and FTA’s project ratings.)
subsections: section_title: FTA Evaluated and Rated 27 Projects and Proposed 4 for FFGAs in Fiscal Year 2006
paragraphs: FTA’s Annual Report on New Starts:  Proposed Allocations of Funds for  Fiscal Year 2006 (“annual report”) listed a total of 34 projects in  preliminary engineering and final design, and FTA evaluated and rated 27 of  them. Seven were statutorily exempt from being rated because they  requested less than $25 million in New Starts funding. Of the 27 projects  that were rated, 2 were highly recommended, 12 were recommended, 8  were not recommended, and 5 were designated “not rated.” In its annual  report, FTA said that projects were designated as “not rated” because they  did not submit the required information or because of FTA’s continuing  concerns about the reliability of the transportation benefits forecast for  these projects. According to FTA, a principal source of these concerns was  inconsistent assumptions used in defining the baseline alternative and the  proposed New Starts project, making it difficult to isolate the impacts of  the proposed project. In some cases, the local travel demand models were  inconsistent with FTA guidance and good planning practice. FTA is  currently working with the sponsors of these projects to improve the  forecasts.
As shown in figure 3, the combined number of recommended and highly  recommended projects declined sharply from 27 in the fiscal year 2003  evaluation cycle to 14 in the fiscal year 2004 evaluation cycle, while the  combined number of not recommended and not rated projects rose from 6  to 18. As we previously reported, this was primarily due to difficulties  project sponsors encountered when implementing the new cost- effectiveness measure—the Transportation System User Benefits (TSUB)  measure—and adjusting to FTA’s preference policy that favors projects  that seek a federal New Starts share of no more than 60 percent of the total  project costs. According to FTA, the information that was provided by  project sponsors in their reports on TSUB highlighted previously unknown  problems with many local models that forecast travel demand. FTA first  incorporated both of these changes beginning with the fiscal year 2004  evaluation cycle. Project ratings generally improved (i.e., there were more  projects with recommended ratings and fewer with not recommended  ratings) in the fiscal year 2005 evaluation cycle as project sponsors  improved their financial plans, grew more comfortable with the new cost- effectiveness measure, and made corrections and improvements to their  models that forecast travel demand, according to FTA. While the  combined number of not rated and not recommended projects was  approximately the same in the fiscal year 2005 and 2006 evaluation cycles,  the number of projects receiving ratings of at least recommended has  decreased slightly again. Unlike in previous years, however, there was no  obviously identifiable reason for this change, except for the fact that some  projects moved out of the ratings pool and into construction, according to  FTA.
The fiscal year 2006 rating cycle saw the smallest total number of projects  in preliminary engineering and final design during the TEA-21 period. The  number of projects evaluated and rated has decreased slightly every year,  from 42 projects in the fiscal year 2000 rating cycle to 27 projects in the  most recent cycle. During this same time frame, the number of exempt  projects grew steadily through the fiscal year 2004 cycle but has been  declining ever since. FTA officials suggested that these trends could be the  result of a combination of factors. First, many of the projects that TEA-21  authorized have worked their way through the New Starts process and  obtained FFGAs, and thus are not subject to the annual evaluation and  rating process. At the same time, new projects entered preliminary  engineering each year and were rated for the first time. FTA officials  speculated that additional projects would be included in reauthorization  legislation and would enter the New Starts pipeline over the course of the  reauthorization period. Second, FTA has increased the level of scrutiny it  applies to projects attempting to advance from alternatives analysis to  preliminary engineering to help ensure that only the strongest projects  enter the New Starts pipeline. Third, FTA has worked with project sponsors  to reduce the number of inactive projects in the New Starts pipeline.  Fourth, since the time TEA-21 was enacted, some state and local  transportation agencies have been feeling the impact of local budget  constraints, making it more difficult to secure local funding for proposed  New Starts projects. This could be exacerbated by FTA’s policy of favoring  projects that seek a federal New Starts share of no more than 60 percent of  the total project costs. As we reported in 2004, several project sponsors  told us that FTA’s push for a lower federal New Starts share would likely  affect their decision to advance future transit projects. Therefore, we  recommended that FTA examine the impact of the preference policy on  projects in the evaluation process. FTA is considering how to conduct such  an examination.
FTA’s evaluation process informed the administration’s recommendation to  fund four projects that are expected to be ready for new FFGAs before the  end of fiscal year 2006, including Charlotte, South Corridor Light Rail  Transit; New York, Long Island Rail Road East Side Access; Phoenix,  Central Phoenix/East Valley Light Rail Transit Corridor; and Pittsburgh,  North Shore Light Rail Transit Connector. The total capital cost of these  four projects is estimated to be $9.9 billion, of which the total federal New  Starts share is expected to be $3.6 billion. FTA executed an FFGA for the  Phoenix project in January 2005 and for the Charlotte project in May 2005.  The other two projects are expected to be ready for FFGAs before the end  of fiscal year 2006. According to FTA, the remaining projects that received  overall ratings of recommended or highly recommended do not yet pass  FTA’s readiness tests for FFGAs.
The administration also proposed reserving $158.6 million in New Starts  funding for final design and early construction activities for as many as six  “other projects,” including San Diego, Mid-Coast Light Rail Transit  Extension; Denver, West Corridor Light Rail Transit; New York, Second  Avenue Subway; Washington County (Oregon), Wilsonville to Beaverton  Commuter Rail; Dallas, Northwest-Southeast Light Rail; and Salt Lake City,  Weber County to Salt Lake City Commuter Rail. These six projects were in  or nearing final design, received overall highly recommended or  recommended ratings, and had cost-effectiveness ratings above “low.”  According to FTA officials, no other projects met these criteria. The annual  report did not specify amounts for particular projects to ensure that the  project is moving forward as anticipated prior to making specific funding  recommendations to Congress, according to FTA officials, because projects  may encounter unexpected financial or other obstacles that slow their  progress. For example, FTA told us that the sponsor for one of the projects  is considering a significant expansion of the project scope, which would  put it back in preliminary engineering and render it ineligible for the funds  FTA proposed reserving for the “other projects” for fiscal year 2006.  Reserving funds for these projects without specifying a particular amount  for any given project will allow the administration to make “real time”  funding recommendations when Congress is making appropriations  decisions. FTA does not anticipate that all of the six projects will be  recommended for funding in fiscal year 2006.
section_title: Administration’s Proposed Fiscal Year 2006 Budget Requests Similar Amount of Funding to Previous Years
paragraphs: The administration’s fiscal year 2006 budget proposal requests that $1.5  billion be made available for the New Starts program, an amount similar to  that requested in the last two fiscal years. Figure 4 illustrates the specific  budget allocations the administration has proposed for fiscal year 2006,  including the following:  $634.6 million would be allocated among the 16 projects with existing  grant agreements,  $590 million would be allocated to four projects expected to be  proposed for new FFGAs,  $158.6 million would be allocated to as many as six “other” projects to  continue project development, and   $122.5 million is reserved to be allocated among projects in preliminary  engineering, at the discretion of Congress and as provided in law.
Anticipated FFGAs ($590 million)
Existing FFGAs ($635 million) extensions to existing systems that are ferry boats or ferry terminal facilities or that are approaches to  ferry terminal facilities.
FTA had approximately $2.2 billion in commitment authority—that is, the  amount of funding authorized for New Starts projects—remaining as of  May 2005. TEA-21 and subsequent amendments and legislation provided  FTA the authority to make about $13.5 billion in funding commitments for  New Starts projects. Surface transportation programs, including the New  Starts program, were scheduled to expire in September 2003, but have  subsequently been extended through June 2005. According to FTA  officials, the commitment authority for fiscal year 2006 and beyond will be  addressed in the next surface transportation authorization legislation.  However, FTA officials told us that there will not be sufficient commitment  authority remaining to execute all four proposed FFGAs until additional  commitment authority is provided through congressional authorization.
section_title: FTA Has Implemented a Number of Changes to the New Starts Program, Some Without Project Sponsors’ Input
paragraphs: FTA has made 16 changes to the New Starts application, evaluation, rating,  and project development oversight processes since the fiscal year 2001  evaluation cycle—the first full evaluation and rating cycle after the  enactment of TEA-21. FTA has used a variety of written and electronic  methods to communicate information about these changes, although it  primarily relies on reporting instructions, roundtables, and workshops. For  nine of these changes, FTA did not publish information about the change in  the Federal Register or institute a rulemaking process and for six of these  nine changes did not provide any avenues for public review and comment  prior to implementing the changes. FTA has said that all of the changes it  has made are consistent with the evaluation framework outlined in the  existing regulations governing the New Starts process. Some project  sponsors we interviewed thought certain changes helped to improve the  process; however, a considerable number of project sponsors expressed  concern that they did not have an opportunity to comment on many of the  changes before they were implemented and have experienced considerable  challenges while attempting to incorporate some of the changes. For  example, some changes have required project sponsors to devote  additional agency resources, as well as time.
subsections: section_title: FTA Has Made 16 Changes Intended to Improve the Application, Evaluation, Rating, and Project Development Oversight Processes since the Fiscal Year 2001 Evaluation Cycle
paragraphs: We identified 16 changes that FTA made to the New Starts application,  evaluation, rating, and project development oversight processes since the  fiscal year 2001 evaluation cycle. These changes range from requiring that  projects undergo a risk assessment to instituting new practices for funding  recommendations. (See table 2 for complete list of changes.) For example,  FTA made two significant changes to the evaluation and rating process for  the fiscal year 2004 evaluation cycle. First, FTA implemented the TSUB  measure as a variable in the calculation of cost-effectiveness and mobility  improvements. The new measure is intended to calculate the change in  the amount of travel time and costs that people incur for taking a trip. This  is a more comprehensive measure than the old “cost per new rider”  measure because it takes into account a broader set of benefits to transit  riders, including new and existing transit riders. Second, in response to  language contained in appropriations committee reports, FTA instituted a  preference policy favoring projects that seek a federal New Starts share of  no more than 60 percent of the total project cost. As shown in table 2, FTA  implemented changes for a variety of reasons, including simplifying the  New Starts process and focusing more on results and performance.  Although the impetus for each change varied, FTA officials stated that, in  general, all the changes were intended to make the process more rigorous,  systematic, and transparent. For example, the requirement for project  sponsors to develop a plan for evaluating the impacts of the project and the  accuracy of travel forecasts—that is, a Before and After study—will  identify lessons learned and hold transit agencies accountable for results.  In our previous work we have commented on a number of these changes  and made some recommendations for improving the New Starts process.  (See app. III for a list of recommendations from previous reports.)
Some project sponsors we interviewed thought that certain changes helped  to improve the process. For instance, seven project sponsors stated that  the “make the case” document helped them focus on the key benefits of the  projects and three said it helped them “sell” the project to local  decisionmakers or the public. In addition, many project sponsors  acknowledged that the TSUB measure is more comprehensive than the old  “cost per new rider” measure. However, project sponsors expressed a  variety of concerns about the effects the changes had on the steps required  to complete the application process and about FTA’s implementation of the  changes. For instance, for the “make the case” document, 15 project  sponsors stated that FTA did not create clear expectations, 10 noted that  FTA prepared no written guidance, and 5 indicated that FTA did not provide  specific examples or templates. A few of these project sponsors also  stated that they did not feel like they understood what FTA wanted for the  “make the case” document, and eight said that they had to produce multiple  iterations of the document. Twelve project sponsors also mentioned they  were not clear about how FTA was using the document. In addition, when  asked what effect the implementation of the TSUB measure had on their  project, 13 of 26 project sponsors said it made the application process more  expensive, and 20 of 26 said that this measure required them to spend  significantly more time to complete the application. Similar comments  were made by project sponsors about other changes, including the risk  assessment and Before and After study requirements. For example, when  asked what effect the risk assessment requirement has had on their project,  4 of the 26 project sponsors said it resulted in a more rigorous or  systematic evaluation and rating process, 7 said it made the process more  expensive, and 5 said it delayed their project. Three project sponsors noted  that each of the individual requirements add to the overall workload and  eventually result in the application process becoming a full-time project.  However, some project sponsors observed that as they become more  familiar with each change and as FTA issues more guidance, such as for  using FTA’s software to calculate the TSUB value, the change has become  less burdensome.
subsections: section_title: In Spring 2005 FTA Introduced New Cost-Effectiveness Practice and Technical Changes to the Cost-Effectiveness Calculation
paragraphs: In March 2005, FTA announced a new practice for the New Starts program  whereby the administration will generally target funding recommendations  to projects able to achieve at least a medium or higher cost-effectiveness  rating. FTA announced this change through a “Dear Colleague” letter sent  to FTA grantees, including current New Starts project sponsors. The letter  was also posted on the home page of FTA’s Web site. The administration’s  previous policy had been to recommend projects for funding that received  at least a medium-low cost-effectiveness rating, provided that they met all  other criteria and project readiness requirements. In the letter to project  sponsors, FTA explained that the impetus for change was the concerns that  GAO, the Office of Management and Budget, the Department of  Transportation’s Inspector General, and Congress raised about  recommending projects for funding that had medium-low cost- effectiveness ratings. In the letter, and in subsequent communication to  project sponsors, FTA also stated that the practice applies to a  recommendation for funding and not to the way the project’s overall rating  is determined. For example, a project can still receive an overall  recommended project rating with a medium-low cost-effectiveness rating  and can advance from preliminary engineering to final design with this  rating; however, as a general rule, the project would not be recommended  for funding by the administration. FTA has said that this new practice will  help it to further prioritize and distinguish among projects for federal  funding, which is important given current fiscal challenges and the  resulting need to maximize the benefit of every federal dollar invested in  transportation.
According to FTA officials, the four New Starts projects with anticipated  FFGAs for the fiscal year 2006 cycle (see app. II) will not be affected by the  new practice. However, FTA has said that the six New Starts projects that  are categorized as “other projects” in the fiscal year 2006 annual report— and therefore eligible for a portion of the $158.6 million in New Starts funds  FTA reserved—will be subject to the new cost-effectiveness funding  recommendation practice and will continue to be subject to this practice  when they apply for an FFGA. According to FTA, two of the six “other  projects” currently do not meet this standard, and FTA is working with  them to improve their cost-effectiveness rating. Six project sponsors we  interviewed expressed concern as to whether they would be able to make  the necessary modifications to their projects in order to earn a higher cost- effectiveness rating by the next evaluation cycle—which begins in August  2005. Also, seven project sponsors commented that FTA puts too much  emphasis on the cost-effectiveness measure, and four indicated that the  new cost-effectiveness practice increases this emphasis. FTA officials  stated that FTA and the administration consider more than just cost- effectiveness in making funding recommendations, noting that every  project that received a not recommended rating in the last 2 years did so  because of its poor financial summary rating.
In addition to the new cost-effectiveness practice in the March “Dear  Colleague” letter, FTA also proposed five technical changes to the way cost- effectiveness is calculated and asked for industry comment on these  proposed changes by April 1, 2005. All comments were posted on the  Department of Transportation’s online docket and were available for  public review. The proposed changes included (1) adjusting the cost- effectiveness rating breakpoints (i.e., low, low-medium, medium, medium- high, high) for inflation, and possibly applying a regional index in an effort  to address cost differences across the country; (2) permitting project  sponsors to utilize either a 2025 or 2030 planning horizon to be consistent  with metropolitan planning organizations’ regional planning processes; (3)  permitting standardized costs and the proposed adjustments to useful life  estimates, which clarify and lengthen these estimates for a number of  assets; (4) permitting modal constants for new guideway modes as a means  of enabling travel models to estimate the effect of improvements to transit  service quality beyond the time and cost measures already accounted for in  the travel models (such as comfort and reliability); and (5) excluding some  soft costs (e.g., administrative expenses) from the calculation of  annualized capital costs for the purpose of calculating cost-effectiveness.  FTA officials told us that they wanted to obtain industry comment on the  proposed changes and to use this feedback to help decide which proposed  changes to adopt. Officials from FTA also said the deadline for industry  comments was driven by the reporting deadlines for the New Starts annual  evaluations and the need to promptly release reporting instructions for the  News Starts program. On the basis of FTA’s review of the proposed changes  and project sponsor responses, FTA announced in an April 29th “Dear  Colleague” letter its decision to incorporate three of the five proposed  changes for the fiscal year 2007 cycle—adjusting rating breakpoints for  inflation, permitting the 2030 design year forecast, and permitting the use  of the standardized cost categories’ useful life assumptions to calculate  annualized capital costs for the purpose of calculating cost-effectiveness.  FTA stated that further research is needed on the other two proposed  changes. FTA incorporated the three technical changes into its fiscal year  2007 reporting instructions, which were released to project sponsors on  May 3, 2005.
section_title: FTA Uses Multiple Methods to Communicate Information about Changes
paragraphs: FTA communicates information about changes to project sponsors through  a variety of different methods, including annual reporting instructions,  “Dear Colleague” letters to project sponsors, conversations with project  sponsors, roundtables and workshops with FTA officials and project  sponsors, and the FTA Web site. FTA officials indicated that they most  heavily rely on regulations, reporting instructions, and roundtables and  workshops to communicate information about changes to the New Starts  application, evaluation, rating, and project development oversight  processes. In particular, FTA officials told us that they use the workshops  as a two-way communication vehicle, during which they can explain the  New Starts application and evaluation process to project sponsors. The  workshops are usually held two to four times a year in conjunction with a  transit industry conference. For example, during the American Public  Transportation Association (APTA) legislative conference in early March  2005, FTA officials held a workshop to discuss the New Starts process as  well as the administration’s new cost-effectiveness funding  recommendation practice. The agency also holds two New Starts  roundtables each year to explain the application and evaluation process  and allow project sponsors the opportunity to have an open discussion  with FTA officials as well as share information on best practices. FTA used  this year’s New Starts roundtables in New York and San Francisco to,  among other things, respond to questions about the recently introduced  changes. FTA’s annual reporting instructions also describe changes to the  application and evaluation process, in addition to providing guidance to  project sponsors on preparing their New Starts submittal. For instance, in  the reporting instructions for the New Starts report for the fiscal year 2005  evaluation cycle, FTA introduced the change to the weighting system used  to calculate the project justification rating.
Project sponsors told us that they generally learn about changes through  one or more of FTA’s communication methods, although they have varying  views on the effectiveness of the different methods. Most project sponsors  we interviewed said that they typically view the roundtables and  workshops and conversations with FTA officials as generally or very  effective methods for learning about changes. However, some concerns  were also raised about these communication methods. First, three project  sponsors stated that they feel compelled to attend all of FTA’s workshops  and roundtables in order to keep up with the changes to the New Starts  process because these meetings are the primary methods FTA uses to  introduce changes. Others noted that they have received inconsistent  information about changes depending on the source. As a result, 18 of 26  project sponsors stated they prefer to obtain information about the  changes directly from FTA officials—and 11 of these project sponsors  stated that they prefer to receive the information in writing for  documentation purposes. Eleven of the 26 project sponsors we interviewed  said that FTA’s Web site was generally or very ineffective as a method for  communicating information about changes. Project sponsors cited a  variety of challenges in using the Web site as a source of information about  changes to the New Starts program. For example, some project sponsors  stated the Web site was difficult to navigate and had a poor search engine,  while others stated they had difficulty finding information about the New  Starts program and that in some cases the information they retrieved from  the Web site was not up to date. Several project sponsors indicated that  maintaining a central repository for all information related to the New  Starts program on FTA’s Web site would be helpful.
section_title: FTA Has Not Consistently Used the Public Notice or Rulemaking Process to Introduce Changes or Solicit Industry Comment on Proposed Changes
paragraphs: Of the 16 changes that FTA has made to the New Starts application,  evaluation, rating, and project development oversight processes since the  fiscal year 2001 evaluation cycle, seven underwent rulemaking, including  providing formal notice to the transit industry and soliciting public  comment, while nine changes did not (see table 2). The Freedom of  Information Act requires federal agencies to publish in the Federal Register  notice of changes to programs and the Administrative Procedure Act sets  out the rulemaking process—which would include notifying the public and  soliciting and considering comments, among other things—required to  make changes to agencies’ rules and procedures. FTA last undertook  rulemaking for the New Starts program in 1999 at the direction of TEA-21 and issued regulations in December 2000. Officials from FTA told us that  they have not amended their regulations to incorporate the nine changes  made to the New Starts process since that time because, in their opinion,  the changes are within the framework of the current regulations. FTA  officials also said that some of the changes—such as the risk assessment  requirement—involve project development oversight and do not affect the  evaluation and rating process; therefore, in their view, the changes do not  need to be included in program regulations. However, by making changes  outside of the regulatory process, FTA has missed an opportunity to obtain  formal public input on the proposed changes, which would increase the  transparency of the agency’s decision-making process and ensure that the  views of project sponsors and other interested parties are considered.  There are a range of options available to FTA for obtaining industry input,  from the more formal rulemaking process to less formal ways of soliciting  comment. In those instances where FTA determines that a formal  rulemaking process is unnecessary, it could provide project sponsors an  informal opportunity to review and comment on any substantive changes  proposed for the New Starts program, as FTA recently did when it solicited  public comment on proposed technical changes to the rating process.
Of the nine changes that were not published in the Federal Register and did  not undergo rulemaking, six were made without providing other avenues  for public review and comment prior to their implementation. For example,  FTA did not seek industry input before implementing the risk assessment  requirement, project justification criteria weights, and the “make the case”  changes to the document. The limited opportunities to review and  comment on proposed changes have resulted in implementation problems,  according to some project sponsors. In addition, FTA did not provide  project sponsors the opportunity to comment on the administration’s  recently announced cost-effectiveness funding recommendation practice.  Many project sponsors expressed concern about the way FTA has  developed and implemented changes to the New Starts process. For  example, 14 of the 26 project sponsors with whom we spoke said that they  have generally not been given an opportunity to offer input into prospective  changes before they are implemented. Many project sponsors noted that  the March “Dear Colleague” letter was a step in the right direction in that it  was one of the first times since the last rulemaking that FTA had sought  their input on proposed changes. However, five project sponsors  commented that FTA did not give them sufficient time to review and  comment on the proposed changes. In addition, FTA did not publish these  changes in the Federal Register and provide at least 30 days’ notice, the  minimum time typically required when changes are subject to the  Administrative Procedure Act. Congressional committees have also  expressed concerns about the transparency by which FTA makes changes  to the evaluation and rating process, as shown by the bills to reauthorize  surface transportation programs—introduced in early 2005—that offer a  more formalized approach for providing notice to and soliciting comment  from project sponsors. In particular, the House reauthorization bill would  require that FTA provide notice and an opportunity for comment at least 60  days before issuing any “nonregulatory substantive changes.” The Senate  proposal states that FTA should issue periodic descriptions of the rating  criteria and allow for public comment. (See app. IV for information about  other changes proposed by the House and Senate.)
Project sponsors offered a number of suggestions for improving FTA’s  communication about changes to the New Starts process. For example,  nine project sponsors told us that FTA should provide them with an  opportunity to offer input into a change before it is made. Thirteen project  sponsors mentioned that FTA should provide them with more lead time  when the agency requires them to implement a change. For example, a  number of project sponsors expressed concern that the recent technical  changes to the cost-effectiveness calculation announced in April 2005 were  made too close to when New Starts information is due—June for travel  forecasts and “make the case” documents and August for the remaining  application materials. In addition, eight project sponsors expressed that  FTA should develop a standard schedule for when it announces and  implements changes to the New Starts process. For example, one project  sponsor suggested that FTA announce all changes at its New Starts  roundtable a year before the changes are to be implemented.
Our previous body of work on organizational transformation indicates  that communication with stakeholders should be a top priority for any  agency, is most effective when done early and often in any process, and is  central to forming the partnerships that are needed to develop and  implement an organization’s strategies. An effective communication  strategy should facilitate an honest two-way exchange with, and allow for  feedback from, stakeholders. This communication is central to forming the  effective internal and external partnerships that are vital to a program’s  success. For the past 2 years, FTA officials have announced at the  roundtables that they are planning to develop and implement a strategy for  improving communication among FTA offices and between FTA and  project sponsors. FTA officials have told us that the strategy could include  providing a single FTA point of contact for project sponsors and improving  the distribution of policy, guidance, and procedures, perhaps using tools  such as a listserve or webinar, as it did when it sought comments from  project sponsors on the recent cost-effectiveness changes. FTA has  implemented new communications tools, such as the webinar, but has not  developed a comprehensive communications strategy.
section_title: New Starts Measures Have Evolved Over Time, but Not All Measures Count Toward a Project’s Rating
paragraphs: Many of the measures FTA uses to evaluate and rate New Starts projects  predate TEA-21 and have evolved over time. In developing these measures,  FTA has historically sought industry input by way of formal rulemaking as  well as outreach sessions, workshops, and reports. Although both TEA-21  and FTA’s current New Starts program regulations emphasize the  importance of using a multiple-measure approach for evaluating projects,  FTA assigns a 50 percent weight to both the cost-effectiveness and the land  use criteria when developing the project justification summary rating. The  other three project justification criteria are not weighted, although the  mobility improvements criterion is used as a “tie-breaker” when the  average of the cost-effectiveness and land use ratings falls equally between two ratings (e.g., between “medium” and “medium-high”). According to  FTA officials, FTA does not use these criteria because the underlying  measures have weaknesses that diminish their use in distinguishing among  projects. Project sponsors we interviewed offered suggestions for  improving all of the project justification measures.
subsections: section_title: Congress Directed FTA to Use Multiple Criteria in Evaluating and Rating Projects
paragraphs: Through the Intermodal Surface Transportation Efficiency Act of 1991  (ISTEA) and TEA-21, Congress has directed FTA to use multiple criteria in  evaluating and rating New Starts projects. In particular, TEA-21 identifies a  series of financial and project justification criteria that reflect a broad  range of benefits and effects of the proposed projects. The financial criteria  include the share of non-New Starts funding and the capital and operating  finance plans. The project justification criteria identified by TEA-21 include  mobility improvements, cost-effectiveness, operating efficiencies, and  environmental benefits. TEA-21 also identifies several additional statutory  “considerations” to the evaluation process, including land use issues. TEA- 21 directs FTA to “evaluate and rate the project as ‘highly recommended,’  ‘recommended,’ or ‘not recommended,’ based on…the project justification  criteria,” among other things. FTA has recognized and acknowledged the  congressional intent for New Starts projects to be evaluated using multiple  measures. For instance, in a January 2005 report to Congress, FTA states  that it is “clear that Congress intends FTA to evaluate projects based on  more than cost-effectiveness criteria” and that the “statutory framework is  consistent with the concept that a wide range of benefits should be  considered in evaluating projects.” “It is important to note that the measure for cost-effectiveness is not intended to be a single,  stand-alone indicator of the merits of a proposed new starts project. It is but one part of the  multiple measure method that FTA uses to evaluate project justification under the statutory  criteria. While cost-effectiveness is an important consideration, so are mobility  improvements, environmental benefits, and the other factors described both in TEA-21 and  elsewhere in this rule.”
FTA has repeated the importance of using a multiple measure approach in  many reports and other documents since then, including its annual reports  to Congress. The New Starts regulations identify the measures FTA will use  to evaluate projects from the project justification criteria outlined in TEA- 21. Figure 5 shows the project justification criteria identified by TEA-21  and the associated measures identified by FTA in the New Starts  regulations. (See app. V for more detail on these measures.)
section_title: FTA Used Industry Input to Develop Measures for All Criteria Identified in TEA-21
paragraphs: The measures FTA uses to evaluate and rate New Starts projects have  evolved over time, beginning years before TEA-21, through an iterative  process involving Congress and the transit industry. FTA (then known as  UMTA) introduced the first system for rating New Starts projects in 1984.  At that time, projects were rated on cost-effectiveness (cost per new rider)  and local financial commitment. Through ISTEA in 1991, Congress added  mobility improvements, environmental benefits, and operating efficiencies  to the list of criteria FTA should use to evaluate projects and also added  land use policies as an additional factor for consideration.
FTA circulated a policy paper in 1994, asking for public comment on its  proposed measures and procedures for assessing projects to address the  requirements laid out in ISTEA. The agency received 31 responses from  transit operators, metropolitan planning organizations, state departments  of transportation, and other interested parties. FTA used these responses in  finalizing its criteria and measures in a notice published in 1996 in the  Federal Register. In the 1994 policy paper, FTA also solicited comments on  the appropriateness of using a multiple-measure method for evaluating  projects. Respondents generally agreed that this was appropriate, although  they were split on how (or whether) the various criteria should be  weighted. FTA formally adopted the multiple-measure approach in 1996.
TEA-21 required FTA to issue regulations for the evaluation and rating  process, and FTA used that rulemaking opportunity to revise several of the  project justification measures. Before and during the rulemaking process,  FTA conducted outreach sessions around the country, soliciting comments  on its processes and procedures for managing the New Starts program. FTA  issued a Notice of Proposed Rulemaking in 1999 and received comments  from 41 individuals and organizations. In response to the comments, FTA  made several changes to the project justification measures. For example,  many of these commenters objected to the “cost per new rider” measure  for cost-effectiveness, saying that the focus on new riders ignores benefits  provided to other riders, which may bias the measure against cities with  “mature” transit systems. In response, FTA replaced it with the incremental  cost per hour of TSUB measure (to capture benefits to both new and  existing riders). The agency also added a mobility improvements measure  for employment near stations to complement the existing low-income  households near stations measure in response to industry comments that a  system that is located near low-income households is of little use to  residents unless it can also provide access to employment and other  activity centers.
Since issuing the regulations, FTA has continued efforts to fine-tune and  improve some of the project justification measures. For example, FTA  convened a panel of experts from the Urban Land Institute to discuss the  land use measures and is seeking industry input on possible changes. FTA’s  Strategic Business Plan for Fiscal Year 2005 includes deliverables to (1)  develop a land use measure that is more quantifiable, offers a better basis  for distinguishing among projects, and provides grantees information with  which to improve their projects and (2) develop a methodology for  measuring the congestion relief benefits of New Start projects. Both are  scheduled to be completed by summer 2005. FTA officials told us that the  fiscal year 2005 program plan included research funds for these efforts. In  addition, as discussed previously, FTA recently instituted technical  modifications to the cost-effectiveness measure.
section_title: FTA Does Not Use All Project Justification Criteria in Determining Ratings Because of Perceived Weaknesses in the Measures
paragraphs: Despite the requirement to use multiple measures, FTA does not use three  of the five project justification criteria in calculating a project’s rating.  Specifically, FTA currently assigns a weight of 50 percent each to the cost- effectiveness and land use criteria; the other three project justification  criteria are not assigned weights (see fig. 6). In its annual report for fiscal  year 2006, FTA stated that it assigns individual ratings for the other three  project justification criteria and reports them in the annual report, but  these ratings “are not considered in the determination of an overall project  justification rating.” FTA does, however, use all three financial  commitment criteria in developing a project’s rating.
FTA officials told us that they do not use the mobility improvements,  environmental benefits, and operating efficiencies criteria in determining  the project justification summary rating because the measures do not, as  currently structured, provide meaningful distinctions among competing  New Starts projects. Many project sponsors we interviewed had similar  views, noting that individual projects are too small to have much impact on  the whole region or the whole transit system. For example, one of the  environmental benefits measures requires project sponsors to measure the  project’s impact on the annual number of tons of emissions forecast for the  region for various pollutants. FTA officials also said that they do not assign  weight to the mobility improvements measures in determining the project  justification rating because they believe that the employment and low- income household measures do not meaningfully distinguish among  projects, and the user benefits are already captured in the cost- effectiveness measure. Therefore, to count the user benefits as part of the  mobility improvements would result in a double-counting of the benefits.  In contrast, FTA officials told us that the cost-effectiveness and land use  measures help to make meaningful distinctions among projects. For  example, according to FTA, existing transit supportive plans and policies  demonstrate an area’s commitment to transit projects and are a strong  indicator of a project’s future success.
Because FTA officials believe that these project justification measures do  not provide meaningful distinctions among projects and are not factored  into a project’s rating, FTA does not stringently evaluate the projects on the  associated criteria. For instance, according to FTA’s New Starts reporting  instructions, every project must submit data on operating efficiencies and  automatically receives a score of medium on that criterion. Similarly, a  project that is in a nonattainment area for a pollutant and that  demonstrates a projected decrease in that pollutant gets a high  environmental benefits rating, while a project in an attainment area that  demonstrates a decrease gets a medium environmental benefits rating.
The New Starts regulations do not specify the weights that should be  assigned to each project justification criterion. FTA sought industry input  on whether there should be a weighting system in the last rulemaking  process; however, according to FTA officials, the agency did not receive  input that was useful to inform its policy. Consequently, the regulations are  silent on the weights that should be assigned to each criterion and do not  prohibit FTA from treating various criteria as more important than other  criteria. FTA instituted its current weighting system with the fiscal year  2004 evaluation cycle after determining that the operating efficiencies,  environmental benefits, and mobility improvements measures do not  effectively distinguish among projects and that there is overlap among the  mobility and cost-effectiveness measures. Because the regulations do not  set forth the weights that should be assigned to each criterion, FTA did not  amend the regulations when it instituted the current weighting system.
Although FTA does not use all of the project justification criteria identified  in TEA-21 to calculate project ratings, project sponsors must submit  information for all five project justification criteria and FTA publishes this  information for each project in the New Starts annual report. FTA officials  offered several reasons for requiring project sponsors to continue to  provide this information. First, TEA-21 requires FTA to consider these  criteria when evaluating projects. Even though not all of the project  justification criteria count toward the rating, FTA officials told us that they  review and consider all five criteria and, therefore, are operating within the  evaluation framework established in TEA-21 and the New Starts  regulations. Second, FTA officials said that they believe that mobility  improvements, environmental benefits, and operating efficiencies are  important and should be a part of the evaluation process, even though FTA  does not yet have measures for these criteria that help make distinctions  among projects for the purpose of rating and funding decisions. No  measures are specified in TEA-21, which only describes the criteria FTA  should use to evaluate and rate projects, so FTA has the flexibility to revise  the measures as needed. FTA officials stated that they continue to examine  and pursue options to improve the measures and that FTA has committed  approximately $500,000 of its fiscal year 2005 research funds to continue its  work to improve the New Starts measures. The officials also said that they  would wait to change these measures until legislation governing the New  Starts program is reauthorized. They said that they will have to institute a  formal rulemaking process at that time and would use that opportunity to  solicit public comment on the New Starts evaluation and rating process.  Third, FTA officials told us that information on these three criteria is  presented in the annual report and may be useful to Congress and local  decisionmakers. A number of project sponsors we spoke to expressed  frustration that they must prepare and submit information for all the  measures for the five project justification criteria even though some of this  information does not contribute to the projects’ ratings.
section_title: Project Sponsors and Others Identified Strengths and Weaknesses of the Measures and Suggested Improvements
paragraphs: Project sponsors we interviewed, as well as FTA, the Department of  Transportation’s Inspector General, and other industry experts have  identified various strengths and weaknesses of the project justification  measures used to evaluate the New Starts projects. For example, all of  these sources acknowledge that FTA’s measure of cost-effectiveness does  not capture benefits that accrue to highway users as more people switch to  the improved transit system and highway congestion decreases. According  to the department’s Inspector General, the omission of highway travel time  savings means that the benefits from proposed projects that convey  significant travel time savings for motorists are not recognized in the  selection process. FTA noted that current local models used to estimate  future travel demand for New Starts are incapable of estimating reliable  highway travel time savings as a result of the New Starts project. According  to the department’s Inspector General, this limitation is due to unreliable  local data on highway speeds. FTA is working with the Federal Highway  Administration to study ways to remedy this problem. Table 3 shows the  strengths, weaknesses, and other concerns most commonly mentioned by  the New Starts project sponsors we interviewed.
In addition, in a recent report on the benefits and costs of transportation  improvements, we identified challenges in measuring the benefits and  costs of transit investments—some of which are relevant to the measures  used by FTA to evaluate New Starts projects. For example, desirable  changes in land use are indirect benefits of a transportation investment,  which are difficult to forecast and quantify. We also reported that social  benefits such as reductions in environmental costs—including reduced  emissions—were difficult to quantify and value. Additionally, we reported  that there is great variation in the models local transportation planning  agencies use to develop travel forecasts (which underlie many of the New  Starts measures), producing significant variation in forecast quality and  limiting the ability to assess quality against the general state of practice.  Some experts have also found that travel demand models tend to predict  unreasonably bad conditions in the absence of a proposed highway or  transit investment. In particular, travel forecasting does not contend well  with land-use changes or effects on nearby roads or other transportation  alternatives that result from transportation improvements or growing  congestion. Before conditions get as bad as they are forecasted, people  make other changes, such as residence or employment changes, to avoid  the excessive travel costs.
Our previous work has stated that agencies successful in measuring  performance had performance measures that, among other things,  demonstrate results and cover multiple priorities. Specifically, successful  measures (1) are aligned with agencywide goals and mission and are  clearly communicated throughout the organization; (2) are clearly stated,  with a name and definition that are consistent with the methodology used  to calculate the measures; (3) have a measurable target; (4) are objective;  (5) are reliable; (6) cover core program activities; (7) have limited overlap  with other measures; (8) provide balance in ensuring that various priorities  are covered; and (9) address governmentwide priorities, such as quality  and cost of service. For example, measures that are not objective may  result in performance assessments that are systematically over- or  understated, and a lack of balance could create skewed incentives when  measures overemphasize some goals. While these successful attributes  were developed specifically for performance measures, they also could be  useful in determining how to improve other types of measures, such as  those FTA uses to evaluate and rate New Starts projects.
The sponsors of the 26 projects we interviewed had many suggestions for  improving the project justification measures. These suggestions ranged  from adding measures to the mobility improvements, environmental  benefits, and land use criteria to changing the way in which operating  efficiencies are calculated. The most commonly cited suggestions are listed  in table 4, but we did not determine whether the suggestions were  appropriate or feasible. FTA officials told us that they received limited  suggestions for specific measures or methodologies for mobility  improvements, environmental benefits, and operating efficiencies during  the rulemaking process.
section_title: Conclusions
paragraphs: TEA-21 and FTA’s New Starts regulations clearly state that New Starts  projects should be evaluated and rated using multiple criteria. Further, the  statute and regulations identify the project justification criteria that should  be used in the evaluation process, including mobility improvements,  environmental benefits, operating efficiencies, cost-effectiveness, and land  use, as well as the financial commitment criteria that should be used in the  evaluation process. The regulations also identify the measures that will be  used to operationalize the criteria. Although FTA uses all three financial  criteria, in practice, FTA uses only two of the project justification criteria— cost-effectiveness and land use—to calculate a project’s overall rating.  FTA’s reliance on two of the five project justification criteria, coupled with  the recent cost-effectiveness practice for funding—which further  emphasizes one criterion—is drifting away from the multiple-measure  evaluation and rating process outlined in TEA-21 and the current New  Starts regulations. According to FTA officials, FTA does not assign weights  to environmental benefits, operating efficiencies, and mobility  improvements because of weaknesses in the measures and the overlap of  some measures that could result in double-counting of benefits. FTA has  made notable progress in improving the measures for cost-effectiveness  and land use, including seeking advice from experts and the transit  industry and conducting pilot tests since the enactment of TEA-21.
However, given that FTA has been statutorily directed to also use  environmental benefits, operating efficiencies, and mobility improvements  in evaluating and rating projects, it is imperative that FTA either pay  additional attention to improving these three criteria so that they can be  more explicitly used in the evaluation process or explicitly demonstrate the  linkages between these criteria and the measures used.
FTA has made a number of changes intended to enhance the rigor of the  program over the past 6 years, and some of these changes, such as the  Before and After study, risk assessment, and cost-effectiveness practice  could help FTA hold project sponsors accountable for results and maximize  the benefits of each dollar invested. However, FTA could improve the  transparency of changes made to the New Starts program by giving project  sponsors an opportunity to review and comment on any substantive  changes before they are implemented. The Freedom of Information and  Administrative Procedure Acts establish formal processes for notifying the  public and making changes to federal programs, including soliciting  comments about proposed changes. If FTA officials determine that a formal  rulemaking process is unnecessary, FTA could still provide project  sponsors an opportunity to review and comment on any substantive  changes proposed for the New Starts program, potentially avoiding some of  the implementation problems that have occurred in the past. In addition,  the review and comment period could help FTA improve the proposed  changes as well as gain industry buy-in and support for changes—elements  that are critical for success.
FTA could also strengthen its communication efforts by improving its Web  site so that project sponsors view it as a viable source for obtaining  information about changes to the New Starts program. The Web site could  provide a central forum for comprehensive, up-to-date information on the  New Starts program and could also be useful for publicizing FTA’s activities  to improve the application, evaluation, rating, and oversight processes.  Much of this information is already available on FTA’s Web site. However,  the information that remains is scattered in different locations and many  project sponsors told us that it was difficult to locate needed information.  Making the information easier to find could help reduce confusion about  the New Starts process.
section_title: Recommendations for Executive Action
paragraphs: To ensure that the New Starts regulations reflect FTA’s current evaluation  and rating process, and to ensure that FTA’s New Starts evaluation process  and policies are objective, transparent, and follow the intent of federal  statute, we recommend that the Secretary of Transportation direct the  Administrator, FTA, to take the following four actions:  ensure that the agency’s regulations governing the New Starts  evaluation and rating process reflect FTA’s current weighting practices  for the criteria when the regulations are revised;  improve the measures for evaluating New Starts projects so that all five  project justification criteria can be used in determining a project’s  overall rating, or provide a crosswalk in the agency’s New Starts  regulations showing clear linkages between the criteria outlined in  statute and the criteria and measures used in the rating process;  publish future changes to the New Starts program in the Federal  Register and subject future changes to the New Starts program to the  rulemaking process or, at a minimum, a 30-day informal review and  comment period, as appropriate. As part of this process, the agency  should develop criteria for determining which changes should be  subject to the rulemaking process, as outlined in federal statute, or to an  informal review and comment period. At a minimum, these criteria  could include changes that impose new reporting requirements or new  analysis on project sponsors, changes to the principles used to  recommend projects for funding, and changes to the weights assigned to  the criteria used to evaluate and rate projects. The criteria should be  communicated to Congress and to project sponsors and others in the  transit community; and  consolidate information and guidance related to the New Starts program  in one location on the agency’s Web site and regularly review this  information to ensure it is up to date and easy to access.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to the Department of Transportation for  review and comment. Officials from the Department and FTA indicated that  they generally agreed with the report’s findings, conclusions, and  recommendations. FTA officials also provided technical clarifications,  which we incorporated as appropriate.
We are sending copies of this report to congressional committees with  responsibilities for transit issues; the Secretary of Transportation; the  Administrator, Federal Transit Administration; and the Director, Office of  Management and Budget. We also will make copies available to others  upon request. In addition, this report will be available at no charge on  GAO’s Web site at http://www.gao.gov.
If you or your staff have any questions on matters discussed in this report,  please contact me at siggerudk@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. GAO contacts and key contributors to this report are listed in  appendix VI.
section_title: Scope and Methodology
paragraphs: To address our objectives, we reviewed the administration’s fiscal year  2006 budget request, the Federal Transit Administration’s (FTA) annual  New Starts report, FTA’s New Starts regulations, FTA’s 2004 and 2005  reporting instructions for the New Starts program, federal statutes  pertaining to the new Starts program, and previous GAO reports. We also  interviewed FTA officials and representatives from the American Public  Transportation Association, New Starts Working Group, and the  Association of Metropolitan Planning Organizations. In addition, we  attended FTA’s New Starts roundtables with project sponsors in New York  and San Francisco in April and May 2005, respectively.
We also conducted semistructured interviews with project sponsors from  the 26 projects that were evaluated and rated in the fiscal year 2006  evaluation cycle (see table 5). These interviews were designed to gain  project sponsors’ perspectives on a number of topics, including the manner  in which FTA communicates changes to the New Starts application,  evaluation, rating, and project development oversight processes; the  impact of the changes that FTA has made to the application, evaluation,  rating, and project development oversight processes since the fiscal year  2001 evaluation cycle—the first full evaluation and rating cycle after the  enactment of the Transportation Equity Act for the 21st Century (TEA-21);  and the measures FTA uses to evaluate projects. To verify the clarity, length  of time of administration, and understandability of the interview questions,  as well as to determine whether respondents had sufficient knowledge and  information to answer the questions, we pretested the questions with three  project sponsors. We made changes to the content and format of the final  set of interview questions as a result of these pretests. After conducting the  interviews with sponsors from all 26 projects, we used a content analysis to  systematically determine project sponsors’ views on key interview  questions and identify common themes in project sponsors’ responses. Two  analysts reached consensus on the coding of the responses, and a third  reviewer was consulted in case of disagreements, to ensure that the codes  were reliable.
To ensure the reliability of information presented in this report, we  interviewed FTA officials about FTA’s policies and procedures for  compiling the New Starts annual reports, including FTA’s data collection  and verification practices for New Starts information. Specifically, we  asked them whether their policies and procedures had changed  significantly since we reviewed them for our 2004 report on New Starts.   FTA officials told us that there were no significant changes in their data  collection and verification policies and procedures for New Starts  information. Therefore, we concluded that the FTA information presented  is sufficiently reliable for the purposes of this report.
We conducted our work from November 2004 through May 2005 in  accordance with generally accepted government auditing standards,  including standards for data reliability.
section_title: Projects with Existing FFGAs and Projects in Preliminary Engineering and Final Design in the Fiscal Year 2006 Cycle
paragraphs: Projects with existing full funding grant agreements (FFGAs)
Kansas City—Southtown Bus Rapid Transit  (BRT)
section_title: Status of Previous GAO Recommendations for Improving the New Starts Evaluation Process
paragraphs: The Federal Transit Administration (FTA) should  prioritize the projects it rates as highly  recommended or recommended and ready to  receive New Starts funds.
FTA officials told us that their recently implemented  cost-effectiveness practice was partly in response to  this recommendation. FTA will generally not  recommend funding for a project that does not  achieve at least a “medium” cost-effectiveness rating.
FTA should make commitment authority allocated to  projects for which the federal funding commitments  have been withdrawn available for all projects  competing for New Starts funding (i.e., “release” the  funding).
In fiscal year 2003, FTA released $157 million in  commitment authority reserved for a Los Angeles  Mid-City subway project that had been suspended for  more than 3 years.
FTA should amend its regulations governing the  level of federal funding share for projects to reflect  its policy of favoring projects that request less than  60 percent New Starts funding.
FTA disagreed with this recommendation, arguing  that its preference policy is not a legally binding  requirement and therefore should not be reflected in  the New Starts regulations. However, the agency did  change the way it characterized the policy in its fiscal  year 2006 New Starts annual report. This document  states that FTA “generally” will not recommend  funding for projects that request more than a 60  percent New Starts share of funding.
FTA should issue additional guidance to transit  agencies describing FTA’s expectations regarding  the local travel forecasting models and the specific  type of data FTA requires to calculate the  Transportation System User Benefits measure.
FTA completed several actions and has other ongoing  efforts to address the recommendation. (1) FTA  provided additional guidance in its reporting  instructions, clarifying the practices that must be  followed in preparing the Transportation System User  Benefits measure. (2) FTA provided additional  guidance in July 2003, on how it would evaluate the  measures in the fiscal year 2005 rating process. (3)  FTA held several special workshops for transit  agencies on Transportation System User Benefits,  travel forecasting, and development of transit  alternatives. (4) FTA initiated a proactive outreach to  project sponsors to help them prepare and evaluate  the measure, identifying weaknesses in their  proposed measure and suggesting improvements. (5)  FTA has a number of ongoing planned improvements  to its guidance, including developing a users guide to  the Summit software, developing case studies and  exemplary practices, and updating its guidance on  travel forecasting for New Starts projects.
FTA should clearly explain the basis on which it  decides which projects will be recommended for  funding outside of full funding grant agreements,  and what projects must do to qualify for such a  recommendation. These explanations should be  included in FTA’s annual New Starts report and  other published New Starts guidance.
In its fiscal year 2006 New Starts report, FTA  described its criteria for recommending projects for  funding outside of full funding grant agreements.  These criteria include projects that were in or nearing  final design, received overall highly recommended or  recommended ratings, and had cost-effectiveness  ratings above a “low.”
FTA should examine the impact of its preference  policy on projects currently in the evaluation  process, as well as projects in the early planning  stages, and examine whether its policy results in  maximizing New Starts funds and local  participation.
FTA has not implemented this recommendation.
section_title: Key New Starts Provisions Contained in House and Senate Reauthorization Bills
paragraphs: Establishes a “Small Starts” program for projects  develop a streamlined evaluation process for projects  requesting less than $75 million in New Starts funds.
Eliminates the “exempt” classification for projects  requesting between $25 million and $75 million in New  Starts funding, with a total project cost of less than  $200 million. These projects would be evaluated  through a streamlined rating process. requesting less than $25 million in New Starts funding  and allows FTA to analyze and rate all projects through  a streamlined process.
Maintains the “exempt” classification for projects  requesting less than $25 million in New Starts funding.
Expands New Starts funding eligibility to include  rapid transit operating in nonexclusive lanes)  requesting less than $75 million to be eligible for New  Starts funding. nonfixed-guideway projects with a majority of fixed- guideway components seeking between $25 million  and $75 million as part of its “Small Starts” initiative.
Maintains the current rating system. recommended,” “recommended,” and “not  recommended” ratings) to five levels of ratings:  “high,”  “medium-high,” “medium,” “medium-low,” and “low.”
Maintains the maximum New Starts share at 80  percent for individual projects (in contrast to the  administration’s reauthorization proposal, which would  lower the maximum New Starts share to 50 percent).
Maintains the maximum New Starts share at 80  percent for individual projects.
Specifically prohibits FTA from requiring a nonfederal  share that is more than 20 percent of the project’s cost.
Requires FTA to provide notice and an opportunity for  review and evaluation process and criteria and allow  for public comment. comment at least 60 days before issuing any  nonregulatory substantive changes.
section_title: FTA’s Project Justification Measures for Evaluating and Rating New Starts Projects
paragraphs: Transportation System User Benefits per  project passenger mile (normalized travel time  savings)
section_title: GAO Contacts and Staff Acknowledgments
subsections: section_title: GAO Contacts
section_title: Staff Acknowledgments
paragraphs: In addition to the individuals named above, other key contributors to this  report were Bert Japikse, Jessica Lucas-Judy, Sara Ann Moessbauer, and  Gary Stofko.
section_title: Related GAO Products
paragraphs: Highway and Transit Investments:  Options for Improving Information  on Projects’ Benefits and Costs and Increasing Accountability for  Results. GAO-05-172. Washington, D.C.:  January 24, 2005.
Mass Transit:  FTA Needs to Better Define and Assess Impact of Certain  Policies on New Starts Program. GAO-04-748. Washington, D.C.:  June 25,  2004.
Mass Transit:  FTA Needs to Provide Clear Information and Additional  Guidance on the New Starts Ratings Process. GAO-03-701. Washington,  D.C.:  June 23, 2003.
Mass Transit:  Status of New Starts Program and Potential for Bus Rapid  Transit Projects. GAO-02-840T. Washington, D.C.:  June 20, 2002.
Mass Transit:  FTA’s New Starts Commitments for Fiscal Year 2003. GAO- 02-603. Washington, D.C.: April 30, 2002.
Mass Transit:  FTA Could Relieve New Starts Program Funding  Constraints. GAO-01-987. Washington, D.C.: August 15, 2001.
Mass Transit:  Implementation of FTA’s New Starts Evaluation Process  and FY 2001 Funding Proposals. GAO/RCED-00-149. Washington, D.C.:   April 28, 2000.
Mass Transit:  Challenges in Evaluating, Overseeing, and Funding Major  Transit Projects. GAO/T-RCED-00-104. Washington, D.C.: March 8, 2000.
Mass Transit:  “Mobility Improvements” Is One of Many Factors Used to  Evaluate Mass Transit Projects. GAO/RCED-00-6R. Washington, D.C.:   October 15, 1999.
Mass Transit:  Status of New Starts Transit Projects With Full Funding  Grant Agreements. GAO/RCED-99-240. Washington, D.C.:  August 19, 1999.
Mass Transit:  FTA’s Progress in Developing and Implementing a New  Starts Evaluation Process. GAO/RCED-99-113. Washington, D.C.:  April 26,  1999.