Public Transportation: Preliminary Information on FTA's Implementation of SAFETEA-LU Changes
Released: 2006-06-27T00:00:00

Letter

Background

SAFETEA-LU authorized over $52 billion for federal transit programs, including the New Starts and JARC programs, from fiscal year 2005 through fiscal year 2009. SAFETEA-LU authorized $7.9 billion for the New Starts program and $727 million for the JARC program. Both of these programs are managed by FTA.

The New Starts program is a discretionary grant program for investments in new fixed-guideway projects. Under the statutorily-defined evaluation process for the New Starts program, FTA identifies and selects fixed- guideway transit projects—including heavy, light, and commuter rail; ferry; and busway projects—for funding. FTA generally funds New Starts projects through full funding grant agreements (FFGA), which establish the terms and conditions for federal participation in a New Starts project and also define a project’s scope, including the length of the system and the number of stations; the project’s schedule, including the date when the system is expected to open for service; and the project’s cost. To obtain an FFGA, a project must progress through a local or regional review of alternatives and meet a number of federal requirements, including providing information for the New Starts evaluation and rating process. As required by SAFETEA-LU, New Starts projects must emerge from a regional, multimodal transportation planning process. The first two phases of the New Starts process—systems planning and alternatives analysis— address this requirement. The systems planning phase identifies the transportation needs of a region, while the alternatives analysis phase provides information on the benefits, costs, and impacts of different corridor-level options, such as rail lines or bus routes. The alternatives analysis phase results in the selection of a locally preferred alternative— which is intended to be the New Starts project that FTA evaluates, as required by statute. After a locally preferred alternative is selected, project sponsors seek FTA’s approval for entry into the preliminary engineering phase. Following completion of preliminary engineering and federal environmental requirements—and assuming New Starts requirements continue to be met—FTA may approve the project’s advancement into final design, after which FTA may approve the project for an FFGA and proceed to construction, as provided for in statute. FTA oversees grantees’ management of projects from the preliminary engineering phase through construction and evaluates the projects for advancement into each phase of the process, as well as annually for the New Starts report to Congress.

To help inform administration and congressional decisions about which projects should receive federal funds, FTA assigns ratings based on a variety of financial and project justification criteria, and then assigns an overall rating. For the fiscal year 2007 evaluation cycle, FTA used the financial and project justification criteria identified in TEA-21. These criteria reflect a broad range of benefits and effects of the proposed project, such as cost-effectiveness, as well as the ability of the project sponsor to fund the project and finance the continued operation of its transit system (see fig. 1). FTA assigns the proposed project a rating for each criterion, then assigns a summary rating for local financial commitment and project justification. Finally, FTA develops an overall project rating. Projects are rated at several points during the New Starts process—as part of the evaluation for entry into preliminary engineering and final design, and yearly for inclusion in the New Starts annual report that is submitted to Congress.

More recent than New Starts, the JARC program was created in 1998 in order to support the nation’s welfare reform goals. Without adequate transportation, welfare recipients face significant barriers in moving from welfare to work. In 1998, we reported that three-fourths of welfare recipients live in central cities or rural areas, while two-thirds of new entry-level jobs are located in suburbs. Public transportation facilities often offer limited or no access to many of these jobs. JARC, which is administered by FTA, was designed to fill these gaps in transportation services for low-income individuals.

JARC is intended to increase collaboration among transit agencies, local human service agencies, nonprofit organizations, and others and to improve the mobility of low-income individuals seeking work. Programs selected to receive grants—including the expansion of public transportation routes, ridesharing activities, and promotion of transit voucher programs—are designed to assist low-income individuals in accessing employment opportunities and related services, such as child care and training.

SAFETEA-LU’s Changes to the New Starts Program Range from Identifying New Evaluation Criteria to Establishing the Small Starts Program

SAFETEA-LU made changes to the New Starts program that range from identifying new evaluation criteria to establishing the Small Starts program. FTA has taken some initial steps in implementing these changes, including issuing an ANPRM for the Small Starts program and guidance for the New Starts program, both in January 2006. The Small Starts program is a new component of the New Starts program and is intended to expedite and streamline the application and review process for small projects. The transit community, however, questioned whether the program, as outlined in the ANPRM, would streamline the process. In its January 2006 guidance, FTA also identified and sought public input on possible changes to the New Starts program that would affect traditional New Starts projects, or large starts, such as revising the evaluation process to incorporate the new evaluation criteria identified by SAFETEA-LU. FTA also identified possible implementation challenges, including how to distinguish between land use and economic development criteria in the evaluation framework.

FTA Has Started to Implement SAFETEA-LU Changes and Will Continue to Do So through the Rulemaking Process

SAFETEA-LU introduced eight changes to the New Starts program, codified an existing practice, and clarified federal funding requirements. These changes range from the creation of the Small Starts program to introducing new evaluation criteria. For example, SAFETEA- LU added economic development to the list of criteria that FTA must use in the New Starts evaluation process. In addition, SAFETEA-LU codified FTA’s requirement that project sponsors conduct before and after studies for all completed projects. SAFETEA-LU also clarified the federal share requirements for New Starts projects. In particular, SAFETEA-LU states that the federal share for a New Starts project may be up to 80 percent of the project’s net capital project cost, unless the project sponsor requests a lower amount. SAFETEA-LU also prohibits the Secretary of Transportation from requiring a nonfederal share of more than 20 percent of the project’s total net capital cost. This language addresses FTA’s policy of favoring projects that seek a federal New Starts share of no more than 60 percent of the total cost. FTA instituted this policy beginning with the fiscal year 2004 evaluation cycle in response to language contained in appropriation committee reports. Table 1 describes SAFETEA-LU provisions for the New Starts program and compares them to TEA-21’s requirements.

FTA has taken some initial steps in implementing SAFETEA-LU changes. For example, in January 2006, FTA published the New Starts policy guidance and, as will be discussed later, the ANPRM for the Small Starts program. FTA will continue to implement the changes outlined in SAFETEA-LU through the rulemaking process over the next year and a half. Specifically, in response to SAFETEA-LU changes, FTA is developing a Notice of Proposed Rulemaking (NPRM) for the New Starts and Small Starts programs. FTA plans to issue the NPRM in January 2007, with the goal of implementing the final rule in January 2008. Figure 2 shows a time line of FTA’s actual and planned implementation of SAFETEA-LU changes.

Small Starts Program Is Intended to Offer a Streamlined Process, but Transit Community Members Question Whether It Will Do So

A significant SAFETEA-LU change was the creation of the Small Starts program. The Small Starts program is a discretionary grant program for public transportation capital projects that (1) are corridor-based, (2) have a total cost of less than $250 million, and (3) are seeking less than $75 million in federal Small Starts program funding. The Small Starts program is a component of the existing New Starts program, but, according to the conference reports accompanying SAFETEA-LU, is intended to provide project sponsors with an expedited and streamlined evaluation and rating process. Table 2 compares New Starts and Small Starts program requirements.

In January 2006, FTA published an ANPRM to give interested parties an opportunity to comment on the characteristics of and requirements for the Small Starts program. In its ANPRM, FTA suggests that the planning and project development process for proposed Small Starts projects could be simplified by allowing analyses of fewer alternatives for small projects, allowing the development of evaluation measures for mobility and cost- effectiveness without the use of complicated travel demand modeling procedures in some cases, and possibly defining some classes of pre- approved low-cost improvements as effective and cost-effective in certain contexts. FTA also sought the transit community’s input on three key issues in its ANPRM, including eligibility, the rating and evaluation process, and the project development process. For each of these issues, FTA outlined different options for how to proceed, and then posed a series of questions for public comment, including the following questions on the rating and evaluation process: How should the evaluation framework for New Starts be changed or adapted for Small Starts projects?

How might FTA evaluate economic development and land use as distinct and separate measures?

How might FTA incorporate risk and uncertainty into project evaluations for Small Starts?

What weights should FTA apply to each measure?

FTA’s ANPRM for Small Starts generated a significant volume of public comment. While members of the transit community were supportive of some proposals for the Small Starts program, they also had a number of concerns. In particular, the transit community questioned whether FTA’s proposals would, as intended, provide smaller projects with a more streamlined evaluation and rating process. As a result, some commenters recommended that FTA simplify some of its original proposals in the final NPRM to reflect the smaller scope of these projects. For example, several project sponsors and industry representatives thought that FTA should redefine the baseline alternative as the “no-build” option and make the before and after study optional for Small Starts projects to limit the time and cost of project development. In addition, others were concerned that FTA’s proposals minimized the importance of the new land use and economic development evaluation criteria introduced by SAFETEA-LU, and they recommended that the measures for land use and economic development be revised.

Since FTA does not plan to issue its final rule for the New Starts and Small Starts programs until early 2008, FTA issued proposed interim guidance for the Small Starts program in June 2006 to ensure that project sponsors would have an opportunity to apply for Small Starts funding and be evaluated in the upcoming cycle (i.e., the fiscal year 2008 evaluation cycle, which begins in August 2006). The proposed interim guidance describes the process that FTA will use to evaluate proposed Small Starts projects to support the decision to approve or disapprove their advancement to project development and the decision to recommend projects for funding, including whether proposed projects are part of a broader strategy to reduce congestion in particular regions. In addition, although not required by SAFETEA-LU, FTA introduced a separate eligibility category within the Small Starts program for “Very Small Starts” projects in the proposed interim guidance. Small Starts projects that qualify as Very Small Starts are projects that do not include the construction of a new fixed guideway, are in corridors with existing riders who will benefit from the proposed project and number more than 3,000 on an average weekday, including at least 1,000 riders who board at the terminal stations, and have a total capital cost of less than $50 million and less than $3 million per mile (excluding rolling stock).

According to the proposed interim guidance on the Small Starts program, FTA intends to scale the planning and project development process to the size and complexity of the proposed projects. Therefore, Very Small Starts projects will undergo a very simple and streamlined evaluation and rating process. Small Starts projects that do not meet all three criteria for Very Small Starts projects will be evaluated and rated using a framework similar to that used for traditional, or large starts, New Starts projects. However, FTA officials have indicated that this evaluation and rating framework would be modified, for example, to include only those criteria listed in the statute. FTA is seeking public input on the Small Starts proposals contained in the proposed interim guidance through July 9, 2006. FTA plans to review the comments received and issue its final interim guidance for the Small Starts program by August 2006. This guidance will govern the program until the final rule is issued.

FTA Also Identified Possible Changes to the New Starts Program in Response to SAFETEA-LU as well as Implementation Challenges

In response to SAFETEA-LU, FTA identified possible changes to the New Starts program that would affect traditional New Starts projects, or large starts, in its January 2006 guidance. According to FTA, some of SAFETEA- LU provisions could lead to changes in the definition of eligibility, the evaluation and rating process, and the project development process. In the guidance, FTA outlines changes it is considering and solicits public input, through a series of questions, on the potential changes. For example, FTA identified two options for revising the evaluation and rating process to reflect SAFETEA-LU’s changes to the evaluation criteria. The first option would extend the current process to include economic development impacts and the reliability of cost and ridership forecasts. Specifically, FTA suggested that economic development impacts and the reliability of forecasts simply be added to the list of criteria considered in developing the project justification rating. The second option would be to develop a broader process to include the evaluation criteria identified by SAFETEA- LU and to organize the measures to support a more analytical discussion of the project and its merits. According to FTA, the second option would broaden the evaluation process beyond a computation of overall ratings based on individual evaluation measures and develop better insights into the merit of a project than are possible from using the quantified evaluation measures alone. (See app. I for a description of the different changes FTA is considering.)

FTA also identified potential challenges in implementing some of SAFETEA-LU changes in its guidance. In particular, FTA described the challenges of incorporating and distinguishing between two measures of indirect benefits in the New Starts evaluation process—land use and economic development impacts. For example, FTA noted that its current land use measures (e.g., land use plans and policies) indicate the transit- friendliness of a project corridor both now and in the future, but they do not measure the benefits generated by the proposed project. Rather, they describe the degree to which the project corridor provides an environment in which the proposed project can succeed. According to FTA’s guidance, FTA’s evaluation of land use does not include economic development benefits because FTA has not been able to find reliable methods of predicting these benefits. FTA further stated that because SAFETEA-LU introduces a separate economic development criterion, the potential role for land use as a measure of development benefits becomes even less clear given its potential overlap with the economic development criterion. In addition, FTA noted that many economic development benefits result from direct benefits (e.g., travel time savings), and therefore, including them in the evaluation could lead to double counting the benefits FTA already measures and uses to evaluate projects. Furthermore, FTA noted that some economic development impacts may represent transfers between regions rather than a net benefit for the nation, raising questions as to whether these impacts are useful for a national comparison of projects. To address some of the challenges, FTA suggested that an appropriate strategy might be to combine land use and economic development into a single measure.

We have also reported on many of the same challenges of measuring and forecasting indirect benefits, such as economic development and land use impacts. For example, we noted that it is challenging to predict changes in land use because current transportation demand models are unable to predict the effect of a transportation investment on land-use patterns and development, since these models use land-use-forecasts as inputs into the model. In addition, we noted that certain benefits are often double counted when evaluating transportation projects. In particular, indirect benefits, such as economic development, may be more correctly considered transfers of direct user benefits or economic activity from one area to another. Therefore, estimating and adding such benefits to direct benefits could constitute double counting and lead to overestimating a project’s benefits. Despite these challenges, experts told us that evaluating land use and economic development impacts is important, since they often drive local transportation investment choices.

FTA received a large number of written comments on its online docket in response to its proposed changes. (See app. I for common comments submitted for each proposed change.) While members of the transit community were supportive of some proposals, they expressed concerns about a number of FTA’s proposed changes. For example, a number of commenters expressed concerns about FTA’s options for revising the evaluation process, noting that both proposals deemphasized the importance of economic development and land use. Some commenters also noted that land use and economic development should not be combined into a single measure and that they should receive the same weight as cost-effectiveness in the evaluation and rating process.

SAFETEA-LU Transformed the JARC Program from a Discretionary to a Formula-based Program

SAFETEA-LU made a number of changes to the JARC program, the most notable of which was the creation of a formula to distribute JARC funds. Whereas funds for JARC projects were congressionally designated in recent years, SAFETEA-LU’s formula distributes funds to states and large urbanized areas. This is a significant change because some states and urbanized areas will receive substantially more funds than under the discretionary program, while others will receive substantially less. In addition, the formula program will result in some areas receiving JARC funds that had not received them in the past. Other JARC changes resulting from SAFETEA-LU include the ability to use a portion of JARC funds for planning activities and the removal of a restriction on the JARC funding available for reverse commute projects, which are designed to help individuals in urban areas access suburban employment opportunities. FTA has worked to develop guidance to help JARC recipients implement these changes by soliciting comments and input through program notices and listening sessions beginning in November 2005. FTA issued interim JARC guidance in March 2006 and is currently working to develop draft final guidance for the program. Final guidance for JARC is expected later this year. Two potential challenges for FTA as it moves forward will be to issue final JARC guidance in a timely manner and to determine its plan for oversight of the JARC program.

SAFETEA-LU Created a Formula for Distributing JARC Funds and Amended Other Key Aspects of the Program

A key SAFETEA-LU change to the JARC program was the creation of a formula to distribute JARC funds. Under TEA-21, JARC was a discretionary grant program for which FTA competitively selected JARC projects and, more recently, awarded funds for congressionally designated projects. Under SAFETEA-LU, states and large urbanized areas have been apportioned funding for JARC projects through a formula based on the number of low-income individuals and welfare recipients in each area. This is a significant change because some states and urbanized areas will receive substantially more funds than under the discretionary program, while others will receive substantially less. In addition, the formula program will result in some areas receiving JARC funds that had not received them in the past. Forty percent of JARC funds each year are required to be apportioned among states for projects in small urbanized and other than urbanized areas, and the remaining 60 percent are required to be apportioned among urbanized areas with a population of 200,000 or more. The governor of each state must designate a recipient for JARC funds at the state level to competitively select and award funds for projects in small urbanized and other than urbanized areas within the state. In large urbanized areas, the recipient must be designated by the governor, local officials, and publicly owned operators of public transportation.

In addition to creating a formula for distributing JARC funds, SAFETEA- LU also extended a JARC requirement related to coordinated planning to additional FTA programs and made a number of other changes to key aspects of the JARC program. In the past, JARC projects were required to be part of a coordinated public transit-human services transportation plan; a similar requirement is included in SAFETEA-LU. However, this requirement will apply in fiscal year 2007 to two other FTA programs that provide funding for transportation-disadvantaged populations. In addition, recipients in states and urbanized areas that select JARC projects must now certify that their selections were based on this plan. Another change resulting from SAFETEA-LU is the ability of a recipient to use up to 10 percent of its JARC allocation for administration, planning, and technical assistance, and the expansion of the definition of eligible activities to include planning as well as capital and operating activities. SAFETEA-LU also removed a restriction on the amount of funding available for reverse commute projects to help individuals in urban areas gain access to suburban employment opportunities. Table 3 compares key JARC provisions under SAFETEA-LU and TEA-21.

Some of these changes address issues that we have raised in past reports on JARC and the coordination of transportation services for transportation-disadvantaged populations. For example, in 2004 we reported that a majority of the JARC grantees we spoke with supported a proposal to use grant funds for administrative, planning, and technical assistance activities, because these activities could increase coordination with potential partners. In 2003, we also reported that some federal and state officials believed that providing financial incentives or mandates for coordination was one way to improve the coordination of transportation services among federal programs. In addition, officials of one metropolitan planning organization that we spoke to about changes to the JARC program also note that the change to a formula program may better facilitate cooperation between organizations. They explained that the required coordinated plans for JARC projects became irrelevant in the past when JARC funds were congressionally designated.

FTA Has Developed Interim JARC Guidance and Plans to Issue Final Program Guidance Later This Year

FTA has been working to develop guidance to help JARC recipients implement changes to the program. In November 2005, FTA published a notice of changes for FTA programs, including JARC. This notice provided information on the JARC program and solicited public comment on aspects of the program such as technical assistance needs and the coordinated planning process. FTA also held five public listening sessions across the country in December 2005 on a number of programs, including JARC, to obtain comments and input on the questions and issues that should be included in future guidance. In March 2006, drawing on the information FTA received through comments and the listening sessions, it released interim JARC guidance for fiscal year 2006 and proposed strategies for fiscal year 2007, and sought comments to assist in the development of program guidance. FTA received more than 200 comments on this notice, and the comments addressed a variety of issues, including the coordinated planning requirement for JARC and other programs and the selection of designated recipients. For example, several private operators of transportation services have requested that FTA include language that private transportation operators be involved in the coordinated planning process. A number of comments have also addressed whether there would be a potential conflict of interest in having a provider of transportation services also serve as the designated recipient that will select JARC projects for funding. FTA officials have indicated that they plan to address many of the issues raised in the comments in draft final guidance for JARC that they plan to release later this summer. FTA plans to solicit comments on the draft final guidance and issue final guidance for JARC later this year. Figure 3 presents a time line for FTA’s implementation of changes to the JARC program.

Through our preliminary work, we have identified two challenges that FTA may encounter as it moves forward in its implementation of changes to JARC. One potential challenge for FTA will be to ensure that it develops JARC guidance in a timely manner so that JARC recipients can implement the program. Officials from one metropolitan planning organization we spoke with about JARC changes noted that the guidance will be important because it will address questions that JARC recipients have raised about the program’s implementation and to which they have received conflicting answers from FTA headquarters and regional staff. A publicly available schedule of FTA deliverables related to SAFETEA-LU’s implementation stated that draft final guidance for JARC was anticipated between May and July 2006. However, FTA officials told us that they now expect to issue the draft final guidance in late July or early August. This change reflects FTA’s extension of the comment period for the March 2006 notice by 1 month to receive additional comments, and the submission of more than 100 comments on or after the last day of the comment period. The additional comments raised a number of issues for FTA to consider, according to FTA officials. While FTA has stated that criteria in the final guidance will not apply retroactively to issued grants so that areas can proceed with JARC projects, FTA officials as well as officials from an association that represents metropolitan planning organizations have told us that some recipients of JARC funds will likely wait for final program guidance before proceeding. In addition, few states and urbanized areas have taken formal steps to apply for fiscal year 2006 funds. As of late May, 5 states had notified FTA of their designated recipient for JARC funding, and 1 of the 152 urbanized areas that receive a JARC apportionment had obligated fiscal year 2006 JARC funds, according to FTA officials.

Another potential challenge for FTA in moving forward will be to determine its plan for overseeing the JARC program. FTA officials have told us that they are still developing this plan, and that at a minimum they expect to use routine grant management tools—such as progress reports and site visits—to oversee JARC recipients. In its interim guidance, FTA also indicates that it intends to use existing oversight mechanisms from the federal urbanized area and nonurbanized area formula programs, such as triennial reviews and state management reviews. However, FTA officials acknowledge they need to determine how to incorporate JARC grant recipients into these oversight processes.

Transparency, Communication, and Accountability Issues Will Continue to Be Important

Our past work suggests that transparency, communication, and accountability issues will be important as FTA moves forward in implementing SAFETEA-LU changes to the New Starts and JARC programs. Like SAFETEA-LU, TEA-21 required GAO to regularly review the New Starts and JARC programs. Since 1998, we have issued numerous reports on these programs, and many of the reports contained recommendations to FTA on ways to improve the implementation of these programs. SAFETEA-LU addressed some of these issues, and FTA has also taken steps to resolve some of them. Nevertheless, given the number of changes that are being made to both programs, continued focus on improving transparency, communication, and accountability will be important.

In our recent reports on the New Starts program, we noted several cases in which FTA could have improved the program’s transparency. Typically, these cases dealt with FTA’s decisions not to seek public input on proposed policy changes before they were implemented. In our 2005 report, we found that FTA had made 16 changes to the New Starts process since fiscal year 2001, but had not published information about the changes in the Federal Register or instituted a rulemaking process for 9 of the changes; moreover, for 6 of the 9 changes, FTA did not provide any avenues for public review and comment. For example, during the fiscal year 2004 cycle, FTA instituted a preference policy in its ratings process favoring current and future projects that do not request more than a 60 percent federal funding share. However, FTA did not amend its regulations to reflect this change in policy or its existing procedures, and the public did not have an opportunity to comment on the impact of the change prior to its adoption.

SAFETEA-LU addressed our past concerns about the transparency of the New Starts program by requiring FTA to publish for notice and comment any proposals that make significant changes to the New Starts program. FTA has already implemented this requirement. For instance, earlier this year, FTA gave the transit community an opportunity to review and comment on proposed procedural changes (i.e., nonregulatory changes) to the New Starts process as well as possible changes FTA was considering for the New Starts program in the future. Although members of the transit community expressed concerns about some of FTA’s proposed changes in their comments, project sponsors and industry representatives repeatedly told us that they appreciated the opportunity to review and comment on the proposals. FTA officials have also stated that they have been pleased with the review and comment process, noting that it helps to ensure that FTA’s guidance is more complete, more responsive to stakeholders’ needs, and more likely to take into account on-the-ground realities.

We have also previously reported shortfalls in FTA’s communication of New Starts program changes to project sponsors that in several cases, have resulted in implementation problems. For example, in our 2003 report, we noted that a number of project sponsors were unable to calculate a valid Transportation System User Benefits (TSUB) value, and as a result, their projects received a “not rated” rating for the cost- effectiveness criterion. Project sponsors commented that they would have benefited from additional guidance and technical support on how to generate the required data for the TSUB measure. Similarly, during the fiscal year 2005 evaluation cycle, FTA introduced a requirement for project sponsors to submit a “make the case” document to articulate the benefits of a proposed New Starts project. FTA officials intended to use the document to help interpret data produced by the local travel forecasting models, but FTA did not prepare any written guidance on what information to include or provide report templates. Without such information, project sponsors stated that they did not understand what should be included in the document or how it would be used, and FTA officials later acknowledged that many of the submissions did not meet their expectations.

SAFETEA-LU addressed these communication problems by requiring that FTA routinely publish policy guidance. Specifically, SAFETEA-LU requires that FTA publish policy guidance for comment and response no later than 120 days after the enactment of SAFETEA-LU, each time significant changes are made, and at least every 2 years. FTA responded to this requirement by publishing policy guidance for the New Starts program in January 2006 and soliciting public comments on the proposed changes outlined in the guidance. Furthermore, in its January guidance, FTA included possible long-term changes to the large starts component of the New Starts program that FTA is considering. FTA stated that it hoped to use the policy guidance as a forum for discussing possible changes with the transit community so that FTA could take the community’s comments into account when developing the NPRM for the New Starts program. In addition, FTA held multiple listening sessions across the country, during which officials told project sponsors about proposed changes to the New Starts program and their rationale for implementing these changes. Most of the project sponsors and industry representatives we interviewed told us that they appreciated FTA’s efforts to solicit their feedback and to encourage an open discussion about the proposed changes.

Finally, we have identified steps for increasing the accountability of the New Starts and JARC programs. For example, we previously reported that outcome evaluations of completed transit and highway projects were not usually conducted to determine whether proposed outcomes were achieved. We noted that because outcome evaluations are not usually completed, agencies miss an opportunity to learn from the successes and shortcomings of past projects to better inform future planning and decision making and increase accountability for results. FTA also identified such evaluations as an opportunity to hold agencies accountable for results and identify lessons learned, and therefore, starting in fiscal year 2003, FTA required project sponsors to complete before and after studies for completed New Starts projects. SAFETEA-LU codified the requirement for before and after studies, and required that these studies (1) describe and analyze the impacts of the new fixed guideway capital project on transit services and transit ridership, (2) evaluate the consistency of predicted and actual project characteristics and performance, and (3) identify sources of differences between predicted and actual outcomes. In addition, SAFETEA-LU included several provisions, including the following, that emphasize the accuracy and consistency of project cost and ridership estimates in the New Starts process: SAFETEA-LU requires the Secretary of Transportation to consider the reliability of the forecasting methods used by New Starts project sponsors and their contractors to estimate costs and ridership as part of the New Starts evaluation process.

SAFETEA-LU allows the Secretary of Transportation to provide a higher grant percentage than requested by the project sponsor if the net cost of the project is not more than 10 percent higher than the net cost estimated at the time the project was approved for advancement into preliminary engineering and the ridership estimated for the project is not less than 90 percent of the ridership estimated for the project at the time the project was approved for advancement into preliminary engineering.

SAFETEA-LU requires the Secretary of Transportation to submit an annual report to congressional committees analyzing the consistency and accuracy of cost and ridership estimates made by contractors to public transportation agencies developing new projects.

Likewise, we have raised issues associated with FTA’s measurement of the JARC program’s results and made recommendations for improvement. In April 2002, we testified that FTA had not yet completed its evaluation of the JARC program or reported to Congress, as TEA-21 required. We also expressed concerns about FTA’s plan to evaluate the program using one performance measure—the number of accessible employment sites— because it would not allow FTA to fully address key aspects of the program or criteria for selecting grantees. We reiterated these concerns in our December 2002 report and recommended that FTA report to Congress on the results of its evaluation of JARC, as required by law, and consider as part of its evaluation of the effectiveness of the JARC program in meeting both of its goals. Our most recent review of the JARC program concluded that the data used in FTA’s 2003 evaluation of the JARC program lacked the consistent, generalizable, and complete information needed to draw any definitive conclusions about the program as a whole. According to FTA, it has faced obstacles in evaluating the JARC program primarily because grantees have had difficulty collecting and reporting information on their programs. SAFETEA-LU requires the Secretary of Transportation to evaluate the JARC program and submit a report describing the results of this study to Congress by August 2008. Specifically, the Secretary must conduct a study to evaluate the effectiveness of the grant program and the effectiveness of recipients making grants to subrecipients. FTA has already begun to take some steps to meet its evaluation requirements, even prior to issuing its final program guidance. These steps may also address some of the concerns we previously raised about FTA’s evaluation of the JARC program. For example, FTA has identified new performance measures and goals, developed a preliminary performance evaluation framework to guide its data collection efforts, and is currently in the process of researching options for simplifying its data collection system and reducing the reporting requirements for grantees.

Mr. Chairman, this concludes my statement. I would be pleased to answer any questions that you or other Members of the Subcommittee may have at this time.

Contact Information

For further information on this testimony, please contact Katherine Siggerud at (202) 512-2834 or siggerudk@gao.gov. Individuals making key contributions to this testimony include Vidhya Ananthakrishnan, Nikki Clowers, John Finedore, Lauren Heft, Daniel Hoy, Jessica Lucas-Judy, Nancy Lueke, and Kimanh Nguyen.

Appendix I: FTA’s Proposed Changes to the New Starts Program

In its January 2006 guidance, the Federal Transit Administration (FTA) identified possible changes to the New Starts program in response to the Safe, Accountable, Flexible, Efficient Transportation Equity Act: A Legacy for Users (SAFETEA-LU). According to FTA, some of SAFETEA-LU’s provisions may lead to changes in the program’s definition of eligibility, evaluation and rating process, and project development process. The following table summarizes the changes FTA has proposed in these three areas, FTA’s rationale for the proposed changes, and the transit community’s response to the proposed changes.

Definition of a fixed guideway: FTA asks whether a Bus Rapid Transit project is a “fixed guideway” project and whether it should fund high-occupancy vehicle (HOV) projects to the degree that they provide benefits to public transit riders.

A fixed guideway has not been specifically defined in the statute.

The current definition of fixed guideway works well; thus, FTA should make no changes.

A minimum percentage of the guideway (e.g., 30-75 percent) should be dedicated in order for a project to get funding.

HOV projects should be funded by the Federal Highway Administration.

Project evaluation and ratings process Evaluation framework: FTA proposes two options for revising the evaluation framework. Option 1 would extend the current framework to include economic development impacts and the reliability of forecasting methods for costs and ridership. Option 2 would be a broader framework that incorporates the new evaluation factors specified by SAFETEA-LU and, according to FTA, organizes the measures to support a more informative, analytical discussion of the project and its merits for New Starts funding.

The current evaluation framework might be improved upon. concerns because they continue to define cost-effectiveness only in terms of mobility. enough weight to land use and economic development.

Nature of the problem or opportunity evaluation measure: FTA asks whether measures that represent the nature of the problem or the opportunity the proposed projects are designed to address should be included in the evaluation framework, and how FTA should evaluate or rate projects that address significant transportation problems compared with projects that take advantage of opportunities to improve service.

New Starts projects are intended to solve specific transportation problems, take advantage of opportunities to improve transportation services, or support economic development.

Funding should be available for projects seeking to shape economic development or to provide a solution to mobility problems.

Economic development impacts measure: FTA identifies two options for characterizing economic development benefits: (1) regional economic benefits and (2) station area development impacts. FTA sought comment on whether there was preference for either option, as well as on how to evaluate economic development and land use as distinct and separate measures.

SAFETEA-LU identified economic impacts as a new evaluation criterion. better isolate the effect of the transit project. There are too many other variables associated with regional economic benefits.

FTA should use both regional and station area economic benefits.

Land use and economic development should be separate measures and carry as much weight as cost- effectiveness.

Differentiating between land use and economic development is difficult.

Mobility benefits measure: FTA proposes to measure mobility by using a combination of user benefits per passenger mile and project ridership. FTA also asked whether other measures of mobility benefits could be used.

The measure of mobility benefits ought to capture as many benefits as possible.

FTA should continue to work toward capturing transportation benefits to highway users in a project corridor.

FTA should analyze the impact of non-home-based trips, trips generated by special events, and automobile trips not taken because of enhanced pedestrian activity established in a project corridor.

Mobility for transit dependents measure: FTA proposes to measure mobility for transit dependents by the share of user benefits accruing to the passenger in the lowest income stratum compared with the regional share of the lowest income stratum. FTA asked whether this proposed measure would cause any implementation difficulties, and whether there were other measures FTA should consider.

Since low-income populations and households without access to automobiles depend critically on the public transportation system to provide basic mobility, access to jobs, health care and other critical services, projects that improve transit services for these populations have special merit.

An implementation difficulty would be the inconsistencies in regional travel demand models—that is, some models are based on income, others on automobile ownership, and some on both.

FTA’s previous measure—percentage of low income households in the project corridor—is somewhat imprecise.

Environmental benefits measure: FTA proposes to continue using the same environmental benefits measure, which uses the projected change in regional vehicle miles traveled to estimate the change in various harmful types of vehicle emissions and energy consumption.

SAFETEA-LU maintained environmental benefits as an evaluation criterion.

FTA should retain its current measure of environmental benefits.

Operating efficiency measure: FTA proposes removing this measure as a separate evaluation criterion, relying instead on an evaluation of cost-effectiveness to address the statutory criterion. According to FTA, the impact of the project on operating and maintenance costs is captured in the calculation of cost-effectiveness.

The current measure—projected systemwide change in operating cost per passenger mile—does not distinguish among proposed projects.

FTA should use the cost- effectiveness evaluation measure to address the operating efficiency criterion.

Cost-effectiveness measure: FTA proposes to broaden the current cost-effectiveness measure to include nontransportation benefits, such as economic development benefits, land use impacts, and mobility benefits to transit dependents. FTA also suggests using two cost- effectiveness measures—one for the forecast year as is done today and the second calculated for the year the project opens.

The current measure of cost-effectiveness does not capture non-transportation benefits. measure would increase the time and cost of project development.

FTA should use the consumer price index, not the gross domestic product index, to adjust the dollar value of the cost-effectiveness threshold.

Financial capabilities measure: FTA proposes changing the way the financial rating factors related to uncertainty are incorporated into the evaluation process. Specifically, FTA suggests using the project sponsor’s ability to absorb funding shortfalls and cost overruns as an explicit measure of financial risk.

SAFETEA-LU identifies the following factors that FTA must use in evaluating financial capability: (1) the reliability of forecasting methods for costs and ridership, (2) existing grant commitments, (3) the degree to which funding sources are dedicated, (4) debt obligations of the project sponsor, and (5) the non-New Starts funding share.

It is unclear from the guidance who is responsible for assessing the reliability of financial forecasts.

The emphasis placed on the reliability of the financial forecast should correlate to the stage of project development.

Reliability of forecasts measures: FTA proposes to assess the risk and uncertainty inherent in project evaluation. Specifically, FTA plans to evaluate the uncertainty associated with the nature and severity of the problem, as well as individual measures of project merit and cost-effectiveness measures.

SAFETEA-LU requires that the reliability of the forecasting methods used to estimate costs be considered in the evaluation of New Starts projects.

Proposal is confusing. assessments suggests that the proposal would require substantial effort with little reduction in uncertainty.

FTA should place significant weight on the project sponsor’s ability to enhance the reliability of forecasts through the proven quality control methods.

Development of project ratings: Currently, FTA develops separate ratings for project justification and local financial commitment, and then derives an overall project rating from these component ratings using decision rules. FTA proposes to use a similar process for rating projects. However, FTA states that the reliability of forecasts needs to be incorporated into the ratings process, and suggests different options for accomplishing this, such as using probability weightings or using uncertainty indicators to decide the outcome for ratings at the margins. FTA also seeks input about the weights that should be assigned to each measure.

SAFETEA-LU requires that the reliability of the forecasting methods used to estimate costs be considered in the evaluation of New Starts projects.

Economic development and land use should receive the same weight as cost-effectiveness.

Local endorsement of the financial plan: FTA proposes to require that project sponsors specify all proposed sources of funding in the financial plan, and that the sponsoring agency provide a letter endorsing the proposed financial strategies and amounts of planned funding by those agencies identified as funding sources.

SAFETEA-LU requires that FTA ensure that proposed New Starts projects are supported by an acceptable degree of local financial commitment and resources.

Securing an endorsement will be overly burdensome and delay project development.

FTA has experienced situations in which a project’s financial plans state that local agencies will provide funding, but in reality those local agencies do not support the project plan. sponsors receive financial commitments.

Hard to fully secure funding commitments in preliminary engineering and final design.

Approval of the baseline alternative: FTA proposes to maintain the current approval process and definition of the baseline alternative. However, FTA asks whether the baseline can be more clearly defined and whether there is a way to report on the benefits of the project including the benefits attributable to the difference between the no-build and the baseline alternatives.

There has been significant confusion over the definition of the baseline alternative.

More clarity needed on how FTA defines baseline alternative. should not be driven by FTA.

On-board transit survey: FTA is considering requiring that a recent survey of transit riders be used to inform the technical work completed during alternatives analysis. FTA suggests that “recent” could be defined as within the 5 years preceding a request to enter preliminary engineering.

Data on current ridership patterns are essential to the development of reliable forecasts.

Surveys are expensive and may be unnecessary in some areas.

FTA should consider other means of collecting data on ridership, such as electronic fare collection data and small sample surveys.

Preliminary engineering purpose and exit criteria: FTA is considering defining the preliminary engineering phase as the process of finalizing the project’s scope, cost, and financial plan such that (1) all environmental impacts are identified and adequate provisions are made for their mitigation in accordance with National Environmental Policy Act (NEPA), (2) all major or critical project elements are designed to the level that no significant unknown impacts relative to their costs will result, and (3) all cost estimating is complete to a level of confidence necessary for the sponsor to implement the financing strategy.

Since the completion of preliminary engineering for proposed projects represents the completion of nearly all the steps needed to make a final decision on the actual implementation of the proposed project, the information for making that final decision must be reliable.

Need a clearer definition of preliminary engineering phase to help project sponsors target resources.

Design costs will be frontloaded, thereby increasing the costs of preliminary engineering.

Project reaffirmation by the metropolitan planning organization (MPO): FTA is considering requiring that the sponsoring agencies reaffirm their adoption of the project in its final configuration and costs into the MPO’s long range transportation plan as part of the application to advance the project to final design.

Before a project is approved for advancement into preliminary engineering, the project must be adopted by the MPO into its long-range transportation plan. However, a project’s scope and costs may change during the preliminary engineering phase. Thus, this requirement would ensure that a revised project still conforms to the MPO’s transportation plans and financial investment strategies.

Creates another step that will increase time and cost of project development.

Duplicates sponsors’ ongoing work with the MPO and provides no added certainty.

Will likely have limited impact on local financial endorsement.

Inconsistent with Federal Highway Administration regulations.

New Starts funding share incentives: FTA asks how it should implement the provision in SAFETEA-LU that would give FTA discretion to provide a higher percentage of New Starts funding than that requested by the project sponsor as an incentive to produce reliable ridership and cost estimates.

SAFETEA-LU allows the Secretary to provide a higher grant percentage than requested by the project sponsor if (1) the net cost of the project is not more than 10 percent higher than the net cost estimated at the time the project was approved for advancement into preliminary engineering, and (2) the ridership estimated for the project is not less than 90 percent of the ridership estimated for the project at the time the project was approved for advancement into preliminary engineering.

Incentive money should be invested back into the New Starts program.

Incentive should focus on the project’s outcomes like project impacts.

Related GAO Products

New Starts reports and testimonies

Opportunities Exist to Improve the Communication and Transparency of Changes Made to the New Starts Program. GAO-05-674. Washington, D.C.: June 28, 2005.

Mass Transit: FTA Needs to Better Define and Assess Impact of Certain Policies on New Starts Program. GAO-04-748. Washington, D.C.: June 25, 2004.

Mass Transit: FTA Needs to Provide Clear Information and Additional Guidance on the New Starts Ratings Process. GAO-03-701. Washington, D.C.: June 23, 2003.

Mass Transit: Status of New Starts Program and Potential for Bus Rapid Transit Projects. GAO-02-840T. Washington, D.C.: June 20, 2002.

Mass Transit: FTA’s New Starts Commitments for Fiscal Year 2003. GAO-02-603. Washington, D.C.: April 30, 2002.

Mass Transit: FTA Could Relieve New Starts Program Funding Constraints. GAO-01-987. Washington, D.C.: August 15, 2001.

Mass Transit: Implementation of FTA’s New Starts Evaluation Process and FY 2001 Funding Proposals. GAO/RCED-00-149. Washington, D.C.: April 28, 2000.

Mass Transit: Status of New Starts Transit Projects With Full Funding Grant Agreements. GAO/RCED-99-240. Washington, D.C.: August 19, 1999.

Mass Transit: FTA’s Progress in Developing and Implementing a New Starts Evaluation Process. GAO/RCED-99-113. Washington, D.C.: April 26, 1999.

Job Access and Reverse Commute: Program Status and Potential Effects of Proposed Legislative Changes. GAO-04-934R. Washington, D.C.: August 20, 2004.

Welfare Reform: Job Access Program Improves Local Service Coordination, but Evaluation Should Be Completed. GAO-03-204. Washington, D.C.: December 6, 2002.

Welfare Reform: DOT Has Made Progress in Implementing the Job Access Program but Has Not Evaluated Impact. GAO-02-640T. Washington, D.C.: April 17, 2002.

Welfare Reform: Competitive Grant Selection Requirement for DOT’s Job Access Program Was Not Followed. GAO-02-213. Washington, D.C.: December 7, 2001.

Welfare Reform: GAO’s Recent and Ongoing Work on DOT’s Access to Jobs Program. GAO-01-996R. Washington, D.C.: August 17, 2001.

Welfare Reform: DOT Is Making Progress in Implementing the Job Access Program. GAO-01-133. Washington, D.C.: December 4, 2000.

Welfare Reform: Implementing DOT’s Access to Jobs Program in Its First Year. GAO/RCED-00-14. Washington, D.C.: November 26, 1999.

Welfare Reform: Implementing DOT’s Access to Jobs Program. GAO/RCED-99-36. Washington, D.C.: December 8, 1998.

Welfare Reform: Transportation’s Role in Moving from Welfare to Work. GAO/RCED-98-161. Washington, D.C.: May 29, 1998.

Other transit-related reports

Highway and Transit Investments: Options for Improving Information on Projects’ Benefits and Costs and Increasing Accountability for Results. GAO-05-172. Washington, D.C.: January 24, 2005.

Transportation Disadvantaged Populations: Some Coordination Efforts Among Programs Providing Transportation Services, but Obstacles Persist. GAO-03-697. Washington, D.C.: June 30, 2003.

Transit Labor Arrangements: Most Transit Agencies Report Impacts Are Minimal. GAO-02-78. Washington, D.C.: November 19, 2001.

Mass Transit: Many Management Successes at WMATA, but Capital Planning Could Be Enhanced. GAO-01-744. Washington, D.C.: July 3, 2001.

Transit Grants: Need for Improved Predictability, Data, and Monitoring in Application Processing. GAO/RCED-00-260. Washington, D.C.: August 30, 2000.

This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.