Federal Transit Programs: Federal Transit Administration Has Opportunities to Improve Performance Accountability
Released: 2010-11-17T00:00:00

Letter

Background

Federal Transit Programs

Federal transit funds are distributed to recipients by formula, through specific designations within the legislation, or on a discretionary basis through individual grant programs. FTA administers its programs according to the programmatic framework for the nation’s transportation infrastructure specified in SAFETEA-LU. FTA works in partnership with the states and other grant recipients to administer federal transit programs, providing financial assistance, policy direction, technical expertise, and some oversight. State and local governments are ultimately responsible for executing most federal transit programs by matching and distributing federal funds and by planning, selecting, and supervising infrastructure projects and safety programs—all in accordance with federal requirements. The eight federal transit programs we reviewed, which represent 97 percent of FTA’s total fiscal year 2010 grant funds (excluding funds provided under the American Recovery and Reinvestment Act) and their respective goals are as follows: The Urbanized Area Formula Grant Program makes funding available to urbanized areas and to states for public transportation capital projects and operating assistance for equipment and facilities in urbanized areas and for transportation-related planning. Funds are allocated based on a multitiered formula, which separates urban areas with populations under 200,000 from those with populations of 200,000 or more. Funds are first apportioned based on a formula provided in law to designated recipients (typically metropolitan planning organizations or a state or regional authority that is responsible for capital projects and for financing and directly providing public transportation). Designated recipients allocate the apportionment among eligible transit service providers in the urbanized area, and then transit providers may submit grant applications to FTA.

The Nonurbanized Area Formula Grant Program provides funding to states and Indian tribes for the purpose of supporting public transportation capital projects and operating costs for equipment and facilities in rural areas with populations of less than 50,000. Funds are first apportioned to states and Indian tribes based on a formula provided in law, then states and Indian tribes may submit applications to FTA. Grant recipients may allocate funds to state or local governmental authorities, or an operator of public transportation or intercity bus service.

The Fixed-Guideway Modernization Program makes federal resources available for the purposes of modernizing or improving existing fixed- guideway systems. Funds are first apportioned to urbanized areas according to a seven-tier formula. Public transit entities (transit authorities and other state and local governmental authorities) in those urbanized areas to which the funds are apportioned may submit applications to FTA.

The Bus and Bus Related Equipment and Facilities Program provides capital assistance for new and replacement buses, related equipment, and facilities for expansion and maintenance purposes. The projects funded by this program are generally determined by Congress. Funds may be provided only to state and local governmental authorities. The Bus Testing Facility Program—which has a single facility for testing new bus models for maintainability, reliability, safety, performance, and other factors—is also administered under this program.

The New Starts Program provides funds for construction of new fixed- guideway systems or extensions to existing fixed-guideway systems. These grants are awarded for specific projects by congressional directive based on FTA recommendations. Funds may be provided only to state and local governmental authorities.

The New Freedom Program provides funding for capital and operating expenses for new public transportation services and alternatives beyond those required by the Americans with Disabilities Act of 1990 that are designed to assist individuals with disabilities. Funds are first apportioned to designated recipients and states based on a formula provided in law. Designated recipients and states to which funds have been apportioned may submit grant applications to FTA. The recipients of these grants, in cooperation with the MPO, are responsible for conducting an areawide competitive solicitation for grant applications from subrecipients (i.e.: state and local governmental authorities, private nonprofit organizations, or transit service operators).

The Job Access and Reverse Commute (JARC) Program provides funding for capital, planning, and operating expenses for projects that transport low-income individuals to and from jobs and activities related to employment and for reverse commute projects. Funds are first apportioned to designated recipients and states based on a formula provided in law. Designated recipients and states to which the funds have been apportioned may submit grant applications to FTA. The recipients of these grants, in cooperation with the MPO, are responsible for conducting an areawide competitive solicitation for grant applications from subrecipients (i.e.: state and local governmental authorities, private nonprofit organizations, or transit service operators).

The Special Needs of Elderly Individuals and Individuals with Disabilities Program provides funding to states for the purpose of assisting private nonprofit groups in meeting the transportation needs of elderly individuals and individuals with disabilities when current transportation services are unavailable, insufficient, or inappropriate for meeting these needs. Funds are first apportioned to states based on a formula administered by the Secretary of Transportation that considers the number of elderly individuals and individuals with disabilities in each state. States forward an annual Program of Projects (POP) and grant applications to FTA. States may allocate apportioned funds to a private nonprofit organization or a governmental authority.

While most people in the United States travel by motor vehicle, demand for public transit continues to grow. According to the U.S. Census Bureau, the number of people who commute to work using public transit grew by about 1 million from 2005 to 2008 to reach approximately 7.21 million persons. Annual federal, state, and local investment in transit grew nearly 77 percent between 1997 and 2006 to total almost $31 billion. During the recent economic recession, however, investment in transit did not keep pace with demand, according to the American Public Transportation Association (APTA), a leading industry organization. Moreover, according to a March 2010 survey conducted by APTA, because of the economic downturn, more than half of the country’s largest transit agencies have recently cut service and/or raised fares and many are also facing budget shortfalls in 2011.

Transit organizations vary widely in size from large systems with multiple modes of transportation, such as the New York City Metropolitan Transit Authority, to smaller organizations, including single-vehicle paratransit services. The governance structure for these organizations varies and includes, among others, entities that are units or departments of (1) city, county, or local governments; (2) state governments; (3) private for-profit corporations; and (4) university or college systems.

Performance Accountability

Accountability for performance varies from grant program to grant program. For the purposes of this report, performance accountability is defined as the mechanisms by which individuals or organizations are held accountable for meeting specified performance-related expectations, such as goals. Performance measurement is the ongoing monitoring and reporting of program accomplishments, particularly progress toward established goals. In this report, we have focused on specific mechanisms that are meant to encourage performance—such as rewards given or penalties imposed. GPRA requires agencies governmentwide to establish agency and program goals and performance measures, and to report on their results in achieving those goals for all their program activities, including grant programs. Performance measures may address the type or level of program activities conducted (process-oriented measures), the direct products and services delivered by a program (output-oriented measures), and/or the results of those products and services (outcome-oriented measures). Performance information, for the purposes of this report, refers to data on performance resulting from performance measurement. To improve both grant performance and grantee performance accountability, Congress, granting agencies, and grantees can use performance accountability mechanisms, which are provisions or specific actions that can be taken at different points in the grant life cycle.

Some Federal Transit Programs Distribute Funds Based Partly on Performance, but Opportunities to Improve Grant Recipients’ Performance Accountability Remain

Of the eight SAFETEA-LU transit programs we reviewed, two are generally funded by congressional direction, while the remaining six are funded through legislatively defined grant formulas.

Funding for Nonformula Programs under SAFETEA-LU, Including New Starts, Is Based Partly on Performance

Funding for the New Starts Program and the Bus and Bus Related Equipment and Facilities Program—the two non-formula-based SAFETEA- LU programs we reviewed—is based partly on performance.

New Starts is often cited as a model for other federal transportation programs because its grant evaluation process is used to distinguish among proposed investments, though GAO and others have also identified challenges related to the complexity and cost of this process. New Starts funding is allocated by Congress to specific projects according to FTA’s recommendations to Congress. FTA’s recommendations are based on statutorily identified criteria such as projected performance on land use, cost effectiveness, and other criteria. By necessity, the New Starts proposed projects are evaluated based on projected performance on the selected criteria, rather than actual performance, because the projects have not yet started.

For the Bus and Bus Related Equipment and Facilities Program, funding is generally directed by Congress, except most recently in fiscal year 2007, when, according to agency officials, FTA allocated funding in part based on transit agency performance on ridership and bus load (i.e., actual ridership as compared with total capacity).

Only a Small Percentage of Federal Transit Funding for Formula Programs under SAFETEA-LU Is Awarded Based on Performance Considerations

According to our analysis, the SAFETEA-LU formulas require that about 5 percent, on average, of fiscal year 2010 funding be awarded based on transit agency performance for the six formula programs we reviewed (see table 1). The remaining grant funding distribution is based on infrastructure-related factors, such as total route miles and revenue vehicle miles, or demographic factors, as shown in table 1. Performance measures include measures of costs per passenger mile or vehicle revenue miles per capita because they measure agencies’ activities against capacity, demand, or costs. To illustrate this, using the example of costs per passenger mile, an agency with low costs that is also providing a low level of service (i.e., a low number of passenger miles) is not performing as well as an agency with comparable population and population density that is able to provide a high number of passenger miles while keeping costs per passenger mile low.

One of the six formula grant programs we reviewed—the Urbanized Area Formula Grant—bases funding, in part, on performance, per statutory requirements, while the other five formula programs we reviewed are funded entirely based on demographic and infrastructure-related factors as shown in table 1. Under the Urbanized Area Formula Grant Program, approximately 8 percent of the funding, on average, is apportioned based on transit agency performance, although the actual percentage of an individual transit agency’s funding based on performance varies. This program apportions funds to two groups of agencies, as follows: For transit agencies in urbanized areas with populations under 200,000, (small cities), funding is apportioned based mostly on demographic data. However, urbanized areas may be eligible for additional incentive funding, which is apportioned in the following way: For each of six performance measures for which the urbanized area performs above the average performance of larger urbanized areas, small city urbanized areas may receive additional funding in the apportionment. In the end, only 1 percent of total funding for this grant program is apportioned in this way.

For urbanized areas with populations of 200,000 or greater, funding is apportioned in part based on infrastructure-related measures and in part through an incentive tier measuring passenger miles per dollar of operating costs. The remaining 7 percent of performance-based funding for this program is apportioned in this way. However, the incentive for an agency to change its behavior to increase its grant apportionment is small compared with its operating costs. GAO’s simulations of the incentive tier apportionment show that a 10 percent increase in an urbanized area’s passenger miles (while holding the area’s operating costs and data for all other urbanized areas constant) typically results in additional funding of less than 0.5 percent of operating costs for the area. Several aspects of the design of the transit formula incentive tier contribute to the relatively small size of the incentive available to a transit agency. First, the overall level of funding apportioned under the incentive tier is relatively small, both as a fraction of total federal aid for the Urbanized Area Formula Grant Program and also as a fraction of current operating expenses. Second, because all the transit agencies within an urbanized area are included in the passenger miles and operating costs of the urbanized area, the incentive apportionment to a single transit agency may be affected by changes in the behavior of other agencies within the urbanized area. Additionally, even if a transit agency’s increased performance results in an increased apportionment amount for the urbanized area, there is no guarantee that that transit agency will benefit from the increased apportionment, as the designated recipient is responsible for allocating the apportioned funds among area agencies. As we discuss later in this report, when an incentive is not of sufficient value, the expected return does not outweigh the expected risk or costs, and recipients are not motivated to pursue the performance improvement.

Once Grants Are Awarded, FTA Collects Performance Information on the Programs as Required, but Generally Does Not Use Performance Data to Evaluate the Effectiveness of FTA’s Grant Programs

FTA has developed some performance measures and targets related to DOT’s strategic goals. Some of these strategic goals and performance measures are related to internal DOT and FTA performance (e.g., organizational excellence); others relate to how well DOT and FTA programs are performing. FTA tracks performance against a set target. For example, in DOT’s Performance and Accountability Report and FTA’s budget documents, FTA provides information on the following goals and measures: Related to DOT’s safety goal, FTA measures transit fatalities and injuries per 100 million passenger-miles traveled.

Related to DOT’s reduced congestion goal, FTA measures the average percentage change in ridership for the 150 largest transit agencies, as well as the percentage of bus fleets and rail stations compliant with the Americans with Disability Act of 1990.

Also related to DOT’s reduced congestion goal, FTA measures the number of jobs made accessible by the JARC program.

In addition to measuring and reporting performance in these areas, FTA has implemented several initiatives aimed at improving grantee performance. For instance, FTA’s Transit State of Good Repair program, which focuses on ensuring that local transit systems are maintained in a state of good repair, initiated a national discussion about the condition of the nation’s infrastructure as an issue posing pressing challenges for most of the nation’s transit systems. The goals of this initiative are to raise awareness of state of good repair issues throughout the industry, bring stakeholders together to assess and measure the scope of maintenance problems, and explore creative approaches to financing necessary repairs and upgrades for aging transportation assets. Additionally, FTA recognizes high-performing transit agencies with nonfinancial awards through various programs. For example, through 2008, FTA recognized transit agencies that experienced significant increases in public transportation ridership. The program was suspended in 2009 in response to the overall trend of decreasing ridership.

FTA also collects performance data from about 2,000 transit agencies through the National Transit Database (NTD) and other reporting requirements. The reporting requirements vary somewhat by program, as follows: Recipients of Urbanized Area Formula Grants and Nonurbanized Area Formula Grants are required to submit information on performance and other information to the National Transit Database. This includes information on ridership, vehicle revenue miles, vehicle information (such as the age and type of the vehicle), and financial information, among other categories.

Recipients of several other transit grants are required to report performance information specific to the goals of the program to help FTA fulfill its obligations under GPRA. For example, recipients of the Special Needs of Elderly Individuals and Individuals with Disabilities grants are required to report data on gaps in service filled and rides provided for elderly individuals or individuals with disabilities as a result of the grant. JARC recipients are required to report information on the number of jobs made accessible and the number of rides provided as a result of JARC funding. Recipients of the New Freedom grant program are required to report on services and rides provided to individuals with disabilities and changes made to infrastructure, technology, or vehicles that affect the availability of transportation services as a result of the New Freedom projects implemented in the current reporting year.

FTA has set targets for some broad goals and has some initiatives aimed at increasing grantee performance, but does not, in general, use the performance data it collects to evaluate the effectiveness of its grant programs. FTA officials stated that FTA’s strategic goals and objectives are generally determined based on their relationship to DOT’s strategic goals as well as the priorities of senior FTA officials. We have previously reported that agencies can use performance information to, among other things, identify problems or weaknesses in programs, to try to identify factors causing the problems, and to modify a service or process to try to address problems. FTA officials use collected performance data to track overall program trends and performance on a national level, but not to determine the link between specific FTA program activities and changes in performance. For example, FTA reports performance statistics to Congress through its biennial report, Status of the Nation’s Highways, Bridges, and Transit: Conditions and Performance. This report includes the change over time for statistics such as average operating speed, vehicle use statistics, frequency and reliability of services, condition of transit assets and infrastructure, and safety, among others. However, FTA does not explore the potential reasons for a change in performance over time. Similarly, FTA officials stated that FTA does not conduct analysis of changes in transit agency performance or set targets for the measures that are incentivized in grant formulas to determine what effect incentives have on transit agency performance (i.e., the extent to which the incentive funding is having the desired outcome). As a result, FTA is missing a valuable opportunity to evaluate the end results of its program activities and SAFETEA-LU funding formulas. Information on the effectiveness of formula incentive programs, in particular, is necessary for determining how and when to make changes to funding mechanisms in the future.

Performance Accountability Mechanisms Can Improve Performance of Transit Agencies, and although They Potentially Present Disadvantages, Most of These Can Be Mitigated by Following Key Strategies

Potential Advantages of Using Performance Accountability Mechanisms Are Encouraging Good Performance, Deterring Poor Performance, and Helping Agencies Allocate Limited Funds

Through our literature review, we identified major financial and nonfinancial mechanisms for making federal transit programs more performance based. Tables 2 and 3 describe the three types of mechanisms we found and their potential advantages and provide examples of how various federal and state agencies or programs use them.

In short, these mechanisms create a link between performance and benefits by either rewarding good performance or penalizing poor performance. However, the extent to which a reward or penalty motivates performance depends on its importance to the agency—if a financial reward or penalty is too small, for example, it may not affect behavior. At the same time, a reward that is too small to influence a large agency may be significant to a smaller agency.

Potential Disadvantages to Using Performance Mechanisms Are That They Can Burden Transit Agencies, Produce Inequitable Results, and Raise Other Concerns

Our literature review found not only advantages associated with performance accountability mechanisms but some potential downsides and concerns as well.

Most notably, to demonstrate that an incentive has been achieved or that a penalty is not warranted, agencies have to gather, maintain, and analyze data, and these tasks require resources. Therefore, to the extent possible, the measure on which an incentive or penalty is awarded or assessed should be based on readily available information, or information that could be readily obtained. If not, it could place a burden on the agency’s resources.

Additionally, when penalizing organizations for not meeting requirements, there are concerns that (1) an organization may be penalized before it has a chance to address its performance problems, and (2) a punitive approach to performance accountability may be taken rather than a constructive approach. For example, even if penalties are employed to promote performance accountability, there should be a constructive, collaborative approach to performance improvement that both precedes and follows the penalty.

Our literature review also identified other disadvantages that relate specifically to implementing financial performance accountability mechanisms. These include the following: The zero-sum nature of performance-based allocations, particularly under constrained resources, can cause inequity, as programs can improve and still receive nothing. For example, all recipients might improve their performance, so that those improving the least might not be rewarded. Also, under resource constraints, performance-based allocations are not likely to provide a meaningful incentive to improve performance or may not be possible. For instance, a basic reason behind tying the allocation of funding to performance is that this will provide an incentive for funding recipients to perform better (to either generate additional funds or avoid financial penalties). However, according to a 2004 study on the use of performance standards and measures for public transportation systems, there is not much evidence that the various formulas used to fund transit agencies actually produce this result. One reason is that even if a single transit system were to make significant performance improvements, the current apportionment formulas do not result in much of a funding change from year to year (in part because of the fact that the change in one system is but a small fraction of the statewide or nationwide numbers that feed into the formulas).

Well-funded grant recipients, with the staff and resources to implement more sweeping changes, may be rewarded at the expense of poorly funded ones that may have more pressing operational concerns. Moreover, for poorly funded recipients, the amount of improvements required may not be realistically attainable through changes within the recipients’ control. For example, according to the previously mentioned study done in 2004, some transit agencies would like to see an increase in the funding provided from local, as opposed to state and federal, sources. However, a funding stream that includes a requirement for contribution of local dollars could be particularly difficult to accomplish for rural systems with limited revenue or for systems where local leaders do not support a contribution of local dollars. This could result in a transit system being penalized financially for a decision made outside its control, with an ensuing decline in service quality. To be fair, factors well beyond the control of the transit agency are often drivers of performance (e.g., local economic problems leading to a drop in employment that in turn leads to a drop in ridership).

Grant recipients may be charged with meeting conflicting goals. For example, if an important local goal for a transit agency is to increase geographic service coverage, the achievement of this goal might have negative impacts on performance as commonly defined—efficiency and/or effectiveness. For a transit system that wants to develop new evening or weekend service, generally such service will be less productive than weekday service. Performance-based funding is likely to provide a disincentive for providing such service.

If not designed correctly, performance mechanisms can introduce perverse incentives. For example, regarding a federal employment program that has used incentive awards—the Workforce Investment Act’s Title I-B Program for Dislocated Workers—we reported in 2009 that local agency officials receiving grants under the program may be reluctant to provide services to job seekers that are less likely to find and maintain a job, because the incentive award is tied to achieving performance levels in placement and retention. This is contrary to the program’s objective, which is to assist all job seekers in obtaining employment.

Finally, when considering the use of performance accountability mechanisms for transit agencies, DOT officials noted that transit grant recipients operate within the context of their local, state, and regional government structures and funding sources. Therefore, some changes may need to be enacted by other organizations to improve transit service, and it may be appropriate to provide incentives to these other government structures as well.

Key Strategies Can Help Overcome Some Disadvantages of Performance Accountability Mechanisms and Help Ensure Successful Implementation of Those Mechanisms

Our earlier work discusses how five key strategies collectively facilitate the effective selection, design, and implementation of performance accountability mechanisms. They are as follows: 1. Ensure mechanisms are of sufficient value. The value of the rewards and penalties—whether financial or nonfinancial—and the cost of improved performance are adequate to motivate desired behaviors and provide a meaningful return to both the grantor and the grantee. 2. Periodically renegotiate and revise mechanisms and measures. Provide for and use the flexibility to reevaluate performance accountability mechanisms and associated performance measures at regular, scheduled intervals and allow time to learn from each cycle to improve performance. 3. Ensure appropriate measures are selected. Measures should represent performance that is within the grantee’s sphere of influence, and can reasonably be achieved and evaluated within the specified time frame, and should be tested over time to minimize the potential for unintended consequences and perverse incentives. Additionally, performance data should be tested and validated to make sure they are credible, reliable, and valid. Absent these attributes, organizations lack the basis for sound decisions about rewards and penalties. 4. Ensure grantor and grantee technical capacity. Grantors and grantees should have the necessary knowledge about performance accountability mechanisms and the ability to effectively implement them. 5. Ensure phased implementation. Allow time to design, test, and revise measurement systems before linking them to accountability mechanisms.

These five key strategies can help mitigate the disadvantages we have previously discussed related to using performance accountability mechanisms, as shown in table 4.

Some of these strategies address multiple disadvantages. For example, table 4 shows that ensuring appropriate measures are selected could mitigate four of the six disadvantages. In addition, we found in our literature review that to ensure financial stability, many sources advocated for providing transit agencies with a baseline funding amount that would enable grantees to know the minimum (or a baseline amount) that they would receive each year, and then providing additional funds that would be based on performance. Although not a key strategy, this more specific option would help address the following three disadvantages: the zero-sum nature of performance-based allocations, well-funded grant recipients being rewarded at the expense of poorly funded ones, and grant recipients being charged with meeting conflicting goals.

Appropriateness of and Success in Using Performance Accountability Mechanisms Depend on Program Goals, but Little Is Known about Using These Mechanisms in Transit Programs

Although performance mechanisms can provide advantages, and the disadvantages can be mitigated, they are still not suitable for all programs. As previously discussed, the eight federal transit programs that we reviewed are designed to accomplish different goals, and the use of performance accountability mechanisms is better suited to some than to others. For example, discretionary grant programs such as New Starts, which are designed to award funds to the best potential performers, may be better suited to the use of performance accountability mechanisms, while formula programs, such as the Urbanized Area Formula Grant Program and Fixed-Guideway Modernization Program, which are intended to support the financial stability of transit agencies or distributional equity, may be less well suited. We have found that performance accountability mechanisms need to be tailored to specific situations, since not all mechanisms are appropriate to all situations, and there is no “one-size-fits- all” solution to performance accountability. Furthermore, according to FTA, transit systems vary in size, resources, and community needs, making it less productive to judge all systems on a common set of performance measures or performance accountability mechanisms.

Even though mechanisms are not all appropriate for all situations, performance accountability mechanisms still hold promise. As we have described in the previous section of this report, DOT is requiring some transportation grants to be based on performance, with good results in some departments. For example, NHTSA used performance information on alcohol-related injuries and fatalities to target grant funding and specific program strategies to states with the highest impaired driver rates. However, DOT has only required transit grants to be minimally based on performance. Expanding the use of performance accountability mechanisms in the area of transit could help make transit grants more performance based. The performance accountability mechanisms described in this report suggest opportunities for FTA to study their impact and potentially use them to make transit grants more performance based—i.e., to identify grantee performance problems, look for solutions, and make other important management decisions, per leading practices.

Congress also has a role to play in increasing the use of performance mechanisms in transit, as part of its process of setting national priorities and allocating the resources to achieve them. Congress has a number of opportunities to initiate the use of performance mechanisms, such as when it establishes or reauthorizes a program, during the annual appropriations process, and during hearings focused on program and agency operations. Moreover, our 2006 report on performance accountability mechanisms recommended that the federal Office of Management and Budget encourage and assist federal agencies in working with Congress to expand the effective use of performance accountability mechanisms when federal grant programs are being created or reauthorized. As noted earlier in this report, Congress and the administration are currently debating reauthorization of the entire surface transportation program, including transit programs.

However, without FTA analysis of the appropriateness, feasibility, and potential impact of using various transit performance mechanisms, Congress may lack the information needed to identify and implement the most effective mechanisms and better help transit agencies maximize their potential. Furthermore, because each mechanism may not be suited to every program, such analysis may require careful study to ensure the best link between mechanisms and transit programs.

Selected Transit Agencies Use Performance Measurement to Varying Degrees and See a Role for the Federal Government in Transit but Cited Challenges in Linking Performance with Planning and Decision Making

All Transit Agencies We Interviewed Measure Performance in Some Categories, but the Extent to Which They Measure It in Others Varies Widely

While officials from all 12 of the transit agencies we interviewed told us they measure performance in certain categories, the extent to which they measure other categories varied widely (see table 5).

Officials at all 12 transit agencies told us that they measure performance in the categories of customer satisfaction, ridership, on-time performance, and safety.

Officials from 7 of the transit agencies told us that they measure performance in the category of environmental impact and energy use. Officials from 1 of the agencies told us that they do this by tracking greenhouse gas emissions from the agency’s fuel consumption based on methodology published in an FTA report. Officials at 4 other agencies said that they had studied or planned to study this category, but they do not regularly measure performance in this category.

Officials from 4 of the transit agencies told us they measure performance in the category of economic development. According to one of these officials, to do so the agency tracks the value of property developments around transit stations. Officials from 4 other agencies told us that while their agencies do not regularly measure economic development performance, they had either periodically studied the agency’s economic impact or were planning to do so. For example, officials from 1 of these agencies said that they had studied, among other things, the direct and indirect impact of their agency’s current and planned operations on the agency’s local economy by tracking factors such as the agency’s capital investments and procurement of goods and services. However, they do not regularly measure performance in this category.

The transit agencies we interviewed determined performance measures based on a variety of factors, including the following: Officials from all of the U.S. transit agencies said that federal requirements provide the basis for at least some of their performance measures. These officials added that their agencies are required to report performance data to FTA’s NTD and, hence, determine certain measures of performance based on NTD requirements, as discussed earlier.

Officials from 10 of the transit agencies said that local priorities were used to determine at least some of their performance measures. For example, officials from 1 agency said that the local metropolitan planning organization required that the agency report performance on farebox recovery, which measures the agency’s recovery of its costs from fares.

Officials from 8 of the transit agencies added that they determined some of the performance measures based on industry best practices or their agency’s internal needs, such as measures used to track progress toward an agency’s strategic goals.

Officials from 4 of the U.S. transit agencies said that state reporting requirements, such as outlined in state legislation on their agency, also formed the basis for determining some of their performance measures.

Officials from 2 of the transit agencies noted that public concerns about various issues led to the development of some performance measures. Specifically, officials from 1 agency told us that passenger concerns about their security while using the transit system led the agency to allocate more funding for its transit police force and to develop and monitor the agency’s performance on a “customer security index.” Officials from another said that concern about the status of a new transit development led the agency to regularly measure and publicize performance information on the development’s status.

Challenges to Measuring Performance

Transit agency officials we interviewed cited a variety of challenges associated with measuring performance. Officials from 8 of the agencies stated that measuring performance on outcomes was particularly challenging. For example, officials from 5 of these agencies told us that it is particularly challenging to determine a method for measuring economic impact or to obtain relevant data. In particular, officials from 1 agency added that measuring the agency’s impact on employment generation is challenging because it is difficult to isolate the agency’s effect on employment. Similarly, officials from 5 of the transit agencies said it was challenging to measure environmental impact generally because it was difficult to determine a method for measuring it or to obtain relevant data. For example, an official from 1 agency said that it is difficult to measure how much pollution a bus generates compared with the pollution from passenger vehicles that the bus is displacing, in part because emissions from these passenger vehicles can vary widely and relevant data are difficult to obtain. Moreover, we were told by officials from 7 of the transit agencies that it is challenging to measure performance because performance is affected by numerous factors that are out of a transit agency’s control, like economic conditions or weather. For example, an official from 1 agency noted that ridership is affected by economic conditions, such as employment. Officials from another agency also said that ridership data can be influenced by the elimination of funding for a route because, according to the officials, the elimination of routes makes it difficult to measure annual changes in overall ridership. Additionally, officials from 1 agency told us that some transit systems operate in environments that an agency has less control over, particularly bus systems, which can make measuring performance on these systems more difficult than for dedicated systems like subways. For example, the officials said that buses, unlike subways, can travel different distances on any given day, making it difficult to measure performance on a month-to- month basis.

Performance Measurement Affects Agencies’ Planning and Decision Making, but Other Factors Also Play a Role

Officials from all 12 of the transit agencies we interviewed told us that they use performance information for agency planning and decision- making purposes. Officials at 9 of the agencies said that performance information is used in decisions or planning related to funding allocations. Officials from 1 agency explained that performance-based funding allocations were critical because they help to inform the agency’s annual budget process and address problems, such as deterioration of capital assets, before they can have a negative impact on the agency’s performance. Officials from 2 transit agencies said that information on customer satisfaction helped them understand customer perceptions about their system, which informed decisions and planning regarding issues like funding allocations and setting of strategic goals. For example, officials from 1 of these agencies said customer perceptions about the transit system’s security led the agency to make decisions and plans that included allocating more funding toward transit police, developing a customer security performance measure, and setting a strategic goal of increasing customer security. Similarly, officials from the other transit agency said that measures of customer satisfaction regarding system cleanliness led their agency to allocate funding for cleaning the system and to set a strategic goal associated with system cleanliness.

While performance affects some transit agency decisions and planning, other factors are also considered. For example: Funding availability: Officials from half of the transit agencies said that funding shortfalls made it difficult to link performance measurement to decisions and planning. For example, officials from 2 of these agencies said that the recent economic recession had reduced sales tax revenue and consequently agency funding, resulting in reduced service or plans for expansion.

Political priorities: Officials from half of the transit agencies added that local political priorities also challenge the agencies’ ability to link performance to decisions and planning. These officials said that among other reasons, elected officials can affect decisions and planning for routes or funding allocation based on political priorities rather than performance information.

Transit Agencies and Experts See a Role for the Federal Government in Transit, but See Challenges to Imposing National Performance Measures

Officials from the 12 transit agencies and experts from the five organizations we interviewed suggested a variety of changes to the federal role in transit: Investment in existing transit infrastructure: Officials from 3 of the transit agencies and experts from two of the organizations suggested that the federal government address the declining condition of transit agency infrastructure by focusing on state of good repair needs. Officials from 2 of these transit agencies and one of the organizations believed that federal grants for transit too often focused on new capital investments rather than updating and maintaining existing infrastructure.

Short duration of federal grant funding: Officials from 3 of the transit agencies stated that the duration of federal transit programs’ grant funding made it difficult for them to conduct long-term strategic planning. According to officials from 2 of these agencies, this long-term planning would benefit if grants were provided for 3- to 5-year periods.

Flexibility in implementing federal transit grants: Some of the transit agency officials and experts we spoke with stated that the federal government should provide greater flexibility to transit agencies or local transportation planning officials to decide how federal grants should be used. Specifically, officials from 5 of the transit agencies and experts from three of the organizations told us that federal grants for transit systems should be less specific to particular modes, such as bus or rail. Officials from 2 of the transit agencies and experts from three of the organizations also told us that transit agencies or local transportation planning officials should have greater autonomy to determine which transportation solution would be most efficient for their circumstances.

Current SAFETEA-LU programs provide funding for accomplishing goals such as modernizing or improving existing transit equipment and facilities, providing capital assistance for new transit systems or expansion to existing systems, expanding access to elderly persons and persons with disabilities, and transporting low-income individuals to and from jobs. During interviews, experts from four of the organizations and officials from 7 of the transit agencies told us that the federal government should set goals that reflect broader issues, such as promoting economic development or improving air quality, in part to address the differences between transit agencies’ circumstances and priorities. However, officials from 3 of the agencies and experts from three of the organizations told us that the goals should address transit-specific issues, such as state of good repair needs. As discussed earlier, federal transit programs focus on these issues to some extent. The transit agency officials and transit experts we interviewed offered a variety of suggestions for consideration in developing federal transit program goals, including the following: encourage economic growth and access to jobs; increase the safety of transit systems; provide increased mobility for populations, particularly in regions with encourage modal shift from automobiles to public transit; and promote livable communities around transit that reduces dependence on foreign oil, encourage economic growth, and addresses environmental challenges.

All of the transit agencies and expert organizations we interviewed foresaw challenges to imposing national performance accountability mechanisms on transit agencies, some of which we identified earlier in our review. Officials from 10 of the agencies and experts from three of the organizations told us that it would be challenging to account for the varying priorities of transit agencies, such as differences in the size or the density of the population they serve, or the ways in which they measure performance. For example, officials from 4 of the agencies noted a lack of consistency in how performance is measured, which makes it difficult to compare agencies’ performance. The agency officials and transit experts cited a variety of other challenges that they said should be considered when implementing a more performance-based federal transit system, including the following: Some transit agencies, particularly in small cities or rural areas, lack personnel or technical resources to measure performance.

Some federal support should be based on need rather than performance because of factors that could make it difficult for some agencies to meet performance goals or maintain performance standards. For example, officials from 1 of the transit agencies said that if the federal support was based on performance alone, then areas with smaller populations would not be positioned as well as agencies in larger, more populated areas to perform on measures like passengers per mile or recovery of costs from fares. Experts from one organization added that some funding for transit agencies should be allocated if there is a risk of decreased performance in the future in areas such as safety.

For some categories of performance, like economic development, data are more difficult to obtain or performance is more difficult to measure.

Goals attached to a federal performance mechanism could outweigh the transit needs or priorities based on local circumstances.

Some of the transit agencies and experts we interviewed provided suggestions for implementing federal performance accountability mechanisms that would mitigate some of the challenges they discussed. For example, Officials from 1 transit agency suggested that a voluntary pilot program be implemented to test a rating system that would measure participating agencies’ median performance over several years on a series of performance indicators. They said such a system would allow the federal government to assess how each agency is performing over time, rather than comparing agencies with differing circumstances or evaluating agencies based on a single performance measure.

An official from 1 transit agency suggested that the federal government support the development of a national transit benchmarking system in which transit agencies could learn best practices from other agencies to improve their performance, rather than applying national performance targets that would have to account for the many differences among transit agencies. The official cited an established benchmarking system in which transit agencies can confidentially share their performance information and compare it against that of other agencies in similar groups, including large subway systems, medium subway systems, and bus systems.

Experts from two of the organizations also told us that the federal government should offer capacity development opportunities such as technical assistance to address the lack of resources or potential burden that some transit agencies could experience with additional federal requirements for performance measurement.

Conclusions

In our 2008 report on surface transportation, we recommended that Congress consider instituting processes to make grantees more accountable by establishing more performance-based links between funding and program outcomes. Implementing links between transit funding and performance through the use of financial performance accountability mechanisms and other tools and approaches, as outlined in this report, could help create incentives to local transit agencies to improve their performance. Although we have identified challenges to making the federal transit program more performance based, we have also identified ways to mitigate those challenges—through the use of strategies for implementing performance accountability mechanisms. As we have illustrated through examples and through the concerns and suggestions of local transit agency and expert officials, performance accountability mechanisms are not one-size-fits-all. Nevertheless, with careful consideration, the mechanisms and strategies highlighted in this report can be combined to achieve a more accountable federal transit program and thus lead to improved outcomes in transit and achievement of federal goals.

One FTA program, New Starts, is frequently cited as a model for other federal transportation programs, because of its use of a rigorous and systematic evaluation process to distinguish among proposed investments based on projected performance. Other FTA programs have afforded limited performance accountability, and thus little incentive to ensure federal transit grants are being used effectively and efficiently. Because of the lack of incentives of sufficient value to grantees in formula programs, the federal transit grant system has limited ability to influence the activities and priorities of local grantees and to ensure that the goals and priorities of the federal transit program are achieved. Further decreasing the federal government’s ability to positively affect the performance of local transit agencies is FTA’s limited analysis of the extent to which transit grants, current formula incentives, and FTA program activities have had an effect on transit agency performance. The Government Performance and Results Act of 1993, as amended, emphasized managing results and identifying opportunities to improve performance and increase accountability. By not fully measuring and analyzing the effects of transit grant programs and FTA program activities on local transit agency performance, FTA is missing a valuable opportunity to assess both its own activities and the transit grant programs for needed changes.

As Congress and DOT prepare for the upcoming surface transportation reauthorization, which includes the reauthorization of all federal transit programs, the department and FTA have an opportunity to enhance the performance of transit grant programs by incorporating performance accountability mechanisms into those programs’ legislative requirements. Doing so could increase the federal government’s ability to ensure that federal transit grants are used efficiently and effectively and achieve the goals established by Congress, keeping in mind the challenges and strategies for mitigating those challenges that we have identified. Since little is currently known about applying these mechanisms to the transit sector, FTA could help fill that knowledge gap by analyzing the feasibility and potential impacts of options for using such mechanisms in federal transit programs and report that information to Congress to help inform the congressional deliberations on the ongoing surface transportation reauthorization.

Recommendations for Executive Action

To enhance the performance of federal transit programs, we recommend that the Secretary of Transportation direct the FTA Administrator to study and report to Congress on options for adding performance accountability mechanisms to transit grant programs. FTA should strive to provide such information in time to be considered during the reauthorization of the federal transit programs.

To improve FTA’s ability to determine the extent to which transit grants, current formula incentives, and FTA program activities have had a positive effect on local transit agency performance—and to the extent that FTA finds current data sources to be appropriate and reliable for such purposes—we recommend that the Secretary of Transportation direct the FTA Administrator to further analyze and use data submitted to FTA by local transit agencies. As part of this analysis, FTA may want to identify and evaluate, when applicable, the extent to which transit grant programs are accomplishing their established goals, the areas of performance in which FTA should concentrate its program activities to increase the performance of local transit agencies and the federal transit program in general, and the extent to which formula incentives and other performance accountability mechanisms have influenced the activities and performance of local transit agencies.

Agency Comments and Our Evaluation

We provided a draft of this report to DOT for review and comment. DOT officials provided technical clarifications, which we incorporated into the report as appropriate, and DOT said it would consider our recommendations.

We are sending copies of this report to the Secretary of Transportation, the Administrator of the Federal Transit Administration, and appropriate congressional committees. This report is also available at no charge on the GAO Web site at http://www.gao.gov.

If you have any questions about this report, please contact me at 202-512- 2834 or wised@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Major contributors to this report are listed in appendix II.

Appendix I: Scope and Methodology

To address the first objective, on the extent to which federal transit programs incorporate performance in funding distribution and evaluating transit program effectiveness, we selected and reviewed 8 of 18 federal transit programs that account for nearly 97 percent of all Federal Transit Administration (FTA) grant funds, excluding funds provided under the American Recovery and Reinvestment Act. With respect to the distribution of transit funds in these programs, we analyzed legislation and FTA documents and interviewed FTA officials to determine the current factors used to establish the level of grant funding under each of the transit programs and the extent, if any, to which the funding is linked to performance. In assessing the extent to which programs are funded based on performance, we compared the factors used to allocate funding to the programs against a definition of performance measures, which requires the measurement of activities against capacity, demand, or costs. To simulate incentive tier apportionments for the Urbanized Area Formula Grant Program’s bus tier, we analyzed FTA apportionment documents and National Transit Database (NTD) data. Concerning the extent to which performance information is incorporated into efforts to evaluate the effectiveness of transit programs, we also analyzed FTA documents and interviewed FTA officials. We relied on criteria found in the Government Performance and Results Act, Transportation Research Board reports, and other sources, which indicate, among other things, that performance- based systems should have defined goals and objectives, link performance measures and program activities, have steps to monitor and report on performance, integrate performance results into decision making, and review and update performance systems periodically. We did not assess NTD or other FTA data to determine its reliability or appropriateness for use in assessing the performance of the Safe, Accountable, Flexible, Efficient Transportation Equity Act: A Legacy for Users (SAFETEA-LU) transit programs.

To address the second objective, on mechanisms that exist for making federal transit programs more performance based and strategies for supporting their successful implementation, we conducted a literature review to identify pertinent studies and reports and interviewed transit agency officials and industry experts. We selected studies and reports that focused on performance accountability mechanisms and general practices that facilitated the effective use of accountability mechanisms or provided examples of using performance accountability mechanisms. Our literature search covered studies published from 2004 onward and was largely drawn from major electronic databases in transportation, economics, and other fields (e.g., Transportation Research Information Services, EconLit, ProQuest, WorldCat, and other databases) and from our past work in surface transportation and performance measures and management. We also interviewed transit agency officials (see list of agencies in table 6) and a variety of industry experts to obtain their views on performance accountability mechanisms, their advantages and disadvantages, and strategies for mitigating any disadvantages and for helping support their successful implementation. We identified industry experts through a literature review and interviews with officials from FTA. Collectively, these experts were knowledgeable about the transit issues that we were addressing. In addition to the literature review, the examples of agencies or programs where the mechanisms have been proposed for or actually used were identified from other sources, including our previous work and interviews with transit agency officials and experts.

To address the third objective, describing how selected U.S. and foreign transit agencies incorporate performance measurement into their planning and decision making and their views on the federal role in transit, we conducted semistructured interviews with 12 transit agencies—10 from the United States and 2 from foreign countries (see table 6). We identified transit agencies for our analysis through a literature review and interviews with officials from FTA and industry organizations. We further narrowed the list by reviewing the transit agencies’ Web sites for information about their use of performance measurement to ensure the transit agency would be able to discuss performance measurement. The 10 U.S. agencies were selected to ensure geographic distribution and variation in the size of the population served. To select the 2 foreign transit agencies for our review, we narrowed the list to include agencies that offer multiple modes of transit and that were available to speak with us in English. Because the agencies were selected as a nonprobability sample, the results cannot be generalized to all transit agencies. However, the information from these agencies is illustrative of the ways in which transit agencies can incorporate performance measurement into their planning and decision- making processes. We also conducted interviews with industry experts, as previously discussed, to obtain their views on the role of the federal government in transit We conducted this performance audit from November 2009 through November 2010 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: GAO Contact and Staff Acknowledgments

Acknowledgments

In addition to the contact named above, Rita A. Grieco, Assistant Director; Elizabeth H. Curda; Dwayne Curry; Robert J. Dinkelmeyer; Katherine E. Hamer; James Leonard; Amy Rosewarne; Sara Ann W. Moessbauer; and Mindi G. Weisenbloom made key contributions to this report.