Surface Transportation: DOT Is Progressing toward a Performance-Based Approach, but States and Grantees Report Potential Implementation Challenges
Released: 2015-01-16T00:00:00

Letter

Background

Most of the federal funding for highway and transit infrastructure and safety improvements is provided by FHWA, FTA, and NHTSA.

FHWA provides about $38 billion each year, or nearly 93 percent of its overall budget, to states to design, construct and maintain the nation’s roadway and bridge infrastructure through the federal-aid highway program. These funds are distributed through annual apportionments established by statutory formulas. FHWA oversees the federal-aid highway program, primarily through its 52 division offices located in each state, the District of Columbia, and Puerto Rico. FHWA division offices have 10 to 61 staff each, depending on the size of the state’s highway program and other factors. As of October 2014, FHWA had approximately 2,900 staff—about two-thirds in the field and the remaining third at FHWA headquarters in Washington, D.C.

FTA provides approximately $10.5 billion each year for transit project grants to states, transit agencies, and MPOs. The grant funding provided by FTA represents 98 percent of its overall $10.7 billion budget; the remaining portion of the budget funds the administration and oversight of its various grant programs by about 500 federal employees in its Washington, D.C., headquarters and 10 regional offices.

NHTSA provides about $500 million per year, or about 60 percent of its annual budget of approximately $800 million, to state highway safety offices through grants. NHTSA awards these grants to state highway safety offices to support projects that focus on mitigating behavioral factors associated with crashes and improving traffic safety information systems, among other things. The remaining third of NHTSA’s budget supports various highway and vehicle safety research and enforcement programs. NHTSA employs about 600 employees in its Washington, D.C. headquarters; its 10 regional offices; and its Vehicle Research and Test Center.

States develop a variety of plans as part of their overall transportation programs. Each state, as well as the District of Columbia and Puerto Rico, is required by statute to carry out a continuing, cooperative, and comprehensive statewide transportation planning process that requires periodic updates of plans. The statewide transportation planning process addresses both urbanized and nonmetropolitan areas of the state and includes both highway and transit needs. As part of the statewide planning process, states must develop a long-range statewide transportation plan and a state transportation improvement program (STIP). The long-range statewide transportation plan establishes a state’s strategic vision and direction for its transportation investments for at least a 20-year period. The STIP is the state program of transportation projects covering at least a 4-year period that are to be supported with federal surface transportation funds, as well as regionally significant projects requiring an action by FHWA or FTA, whether or not the projects are federally funded. At least every 4 years, state DOTs are required by statute to submit an updated STIP to FHWA and FTA for review and approval, in which the state certifies that the transportation planning process has been carried out in accordance with federal planning requirements. In urbanized areas, MPOs are also required to produce a long-range transportation plan, referred to as a metropolitan transportation plan, and a transportation improvement program (TIP). We have previously found that the cycle and frequency of updates of these plans vary across states and MPOs. In addition, each state is required to develop a strategic highway safety plan to, among other things, provide for performance management and identify strategies that cover all aspects of highway safety, which include (1) infrastructure improvements; (2) behavioral approaches, such as education and enforcement efforts meant to change drivers’ behavior; and (3) emergency medical services. Furthermore, each year each state is required to develop a highway safety plan, describing the state’s highway safety program and the activities they plan to undertake. According to NHTSA, states have voluntarily included measurable goals and objectives in their highway safety plans since 2010, enabling states to track the status of highway safety implementation efforts and monitor progress.

Prior to enactment of MAP-21 in 2012, DOT’s surface transportation funding programs and statewide planning generally were not linked to performance-based outcomes, although some states used performance measures, most notably in the safety area. Historically, most federal surface transportation funds have been distributed through formulas that have only an indirect relationship to infrastructure needs, and many have no relationship to outcomes or grantees’ performance. FTA had incorporated performance-based principles into some of its formula grant programs to allocate funding. We previously found that about 5 percent of fiscal year 2010 funding for six formula grant programs we reviewed, on average, was required to be awarded based on transit agency performance. In a 2010 report, we noted that DOT’s oversight of the statewide planning process focused on state DOTs’ compliance with planning process requirements and did not consider transportation planning outcomes. At the state level, we found that states and MPOs have incorporated some performance measures and targets into their plans, though the use of measures and targets varied depending on the area. For example, we found nearly all state DOTs had safety performance measures and quantifiable targets related to these measures, due, in part, to the federal requirement that state DOTs develop a strategic highway safety plan that establishes statewide goals and objectives to reduce highway fatalities and serious injuries. In other areas, such as measuring truck freight traffic, vehicle congestion, and emissions, we noted states generally did not incorporate performance measures into their plans. However, even in areas such as safety where states had used performance measures, we reported that only a few states made significant attempts to integrate performance measurement into their statewide planning process to inform investment decisions.

Since 2008, we have recommended that Congress consider reexamining and refocusing surface transportation programs to include performance- based principles. In particular, we have noted that tracking specific outcomes that are clearly linked to program goals could provide a strong foundation for holding grant recipients responsible for achieving federal objectives and measuring overall program performance. Furthermore, we have previously found that data-driven decision-making leads to better results, and that if agencies do not use performance measures and performance information to track progress to track progress toward goals, they may be at risk of failing to achieve their goals. In addition, we found through our prior work that performance information can be used across a range of management functions to improve results, from setting program priorities and allocating resources to taking corrective action to solve program problems.

MAP-21 included a number of statutory requirements related to transforming the surface transportation system to a performance-based approach, for the operating administrations, as well as for states and other grantees, in many cases holding grantees accountable for the first time. For instance, MAP-21 directed the Secretary of Transportation to establish performance measures related to highway safety, asset condition, and highway system performance, among other things. In addition, states and other grantees are required to establish targets related to the performance measures established by DOT and to collect data, and track progress against those targets. States and other grantees are also required to report on progress toward achieving their targets. For example, beginning in 2016, states are required to report every 2 years to DOT on the progress they have made in achieving their FHWA targets. Further, in 2017, 5 years after the passage of MAP-21, DOT is required to report to Congress on progress made by states and other grantees in achieving their targets. In addition to establishing a performance-based approach for existing programs under NHTSA, FHWA, and FTA, MAP-21 expanded FTA’s safety oversight role and directed it to establish performance criteria for its new safety responsibilities, establish performance measures for state of good repair, develop a national transit safety plan, and develop requirements for local agency safety plans, among other things.

To implement statutory requirements, such as those contained in MAP- 21, Congress has provided regulatory agencies, such as DOT, with the authority and responsibility for developing and issuing regulations. The basic process by which all federal agencies develop and issue regulations is spelled out in the Administrative Procedures Act (APA). This act establishes procedures and broadly applicable federal requirements for rulemaking, including informal rulemaking, also known as notice and comment rulemaking. (See fig .1) Among other things, the act generally requires agencies to publish a notice of proposed rulemaking in the Federal Register. After giving interested persons an opportunity to comment on the proposed rule by providing “written data, views, or arguments,” the agency may then publish the final rule. In addition to the requirements under the act, proposed rules are often subject to internal and external deliberations and reviews, including official management approval within an agency, and to review and approval by the Office of Management and Budget (OMB).

DOT Has Made Progress in Moving toward a National Performance-Based System, but Multiple Factors Affect Progress

DOT Has Made Extensive Outreach Efforts to Communicate with States and Other Grantees

Since MAP-21 was enacted in July 2012, DOT has conducted an extensive outreach effort to prepare states and grantees for a national performance-based approach to surface transportation. DOT worked closely with national transportation organizations and grantees to share information and collect input about the potential performance-based requirements. DOT has:

Established a performance working group in August 2012, comprised of FHWA, FTA, and other DOT officials as well as representatives from nine national transportation organizations. According to DOT officials, the working group, which meets on a quarterly basis, has served as a forum for DOT to coordinate outreach efforts with national organizations, and also provided national organizations with a better understanding of the potential performance-based requirements and implementation schedule.

Conducted webinars, organized regional workshops, and facilitated peer exchanges and national organization conferences to share information with grantees. For instance, FHWA and FTA sponsored a roundtable on MAP-21 during an Association of Metropolitan Planning Organizations (AMPO) conference and also participated in Transportation Research Board annual meetings.

Begun development of a technical assistance program and training curriculum on transportation performance management to assist grantees with the transition to a performance-based approach. For example, according to FHWA officials, FHWA has begun developing a technical assistance program that will include resources such as the development of guidebooks and evaluation tools, and identification of best practices. Once FHWA finalizes the rulemakings, FHWA will provide on-site assistance to its grantees through the technical assistance program. Additionally, FHWA has begun developing a training curriculum for states and MPOs that will be available once the performance-based approach is formally established, according to FHWA officials. FHWA also conducted a national assessment of state readiness to help inform its technical assistance program and training curriculum.

Established guidance to help communicate how certain MAP-21 performance requirements will be implemented. FHWA and FTA, for instance, jointly issued policy guidance to clarify the requirement that public transportation providers must be represented on the board of MPOs that serve as transportation management areas.

Sought formal public input on potential approaches for implementing the MAP-21 requirements. For example, FTA issued an advance notice of proposed rulemaking (ANPRM), for its rules related to transit safety plans and transit asset management to solicit feedback and obtain input from grantees on the content of potential performance- based approaches to transit.

A majority of grantee officials we spoke to stated that DOT’s outreach efforts have helped them understand the potential requirements and mechanics of implementing a national performance-based system. For example, officials from the New York State Association of Metropolitan Planning Organizations noted that DOT officials have provided helpful and extensive information, including responses to frequently asked questions on the FHWA website to help support the development of the performance-based approach and keep grantees informed. Similarly, officials from Washington state noted that as a result of DOT’s outreach efforts, the rules that have been released for public comment have contained few unanticipated requirements.

Rules to Guide the Performance-Based Approach Are under Development

DOT is formally establishing requirements for states and other grantees to implement the performance-based approach envisioned by MAP-21 through the rulemaking process. (See table 1.) Specifically, DOT plans to establish performance measures, as well as target-setting and reporting requirements for grantees for a range of transportation activities. In some cases, MAP-21 required DOT to use the rulemaking process to implement performance-based requirements and established certain deadlines; however, in other cases, MAP-21 did not establish deadlines for implementing performance-based requirements. For example, while MAP-21 required FTA to issue a rule for target-setting and reporting requirements in transit agency safety plans, it did not establish a deadline for the issuance of a final rule. In addition, DOT officials stated that they may provide additional guidance to states and other grantees outside of the rulemaking process. For example, FTA officials stated that FTA plans to establish a National Transit Safety plan and establish minimum safety performance criteria for transit agencies as required by MAP-21, through a policy document, rather than through the formal rulemaking process.

DOT’s operating administrations have made varying degrees of progress developing their associated performance rules. (See fig. 2.) As of January 5, 2015, five out of nine rules have been available for public comment, and the remaining four rules are still under development.

NHTSA issued an interim final rule for public comment in January 2013 but has not established a timeline for issuing a final rule. According to NHTSA officials, the agency is awaiting enactment of a new surface transportation reauthorization act before deciding on a timeframe since the timing and content of such a statute could require NHTSA to revise its interim rule.

FHWA has released three of its five proposed rules for public comment. FHWA released two proposed rules on safety for public comment in March 2014 and is currently evaluating the comments it received. In addition, on January 5, 2015, FHWA released a notice of proposed rulemaking on pavement and bridge performance measures for public comment. FHWA plans to release its remaining two rules for public comment by April 2015.

FTA’s planned rules related to transit asset management and transit agency safety plans remain under development. FTA has not released either of its rules for public comment, although FTA did issue an ANPRM to solicit public input about the contents of its proposed rules. FTA officials said it plans to release proposed rules for transit asset management for public comment in June 2015, and a proposed rule for transit agency safety plans in August 2015.

FTA and FHWA have jointly released for public comment a notice of proposed rulemaking related to performance-based metropolitan and statewide planning. The public comment period for this proposed rule closed in October 2014, and FHWA and FTA are currently evaluating the comments received on this proposed rule.

In developing these rules, DOT has missed five out of six performance- related rulemaking deadlines established in MAP-21. However, DOT officials declined to provide specific dates for when some of the rules will be finalized. FHWA plans to finalize the two safety rules and the metropolitan and statewide planning rule developed with FTA by September 2015. However, FHWA has not established when its remaining rules would be finalized. FTA officials stated that it has not determined when its two rules would be finalized. FHWA and FTA officials we spoke with stated they could not provide a more specific date when the rulemaking process would be completed for the rules that have not yet been released for comment, noting the challenges of predicting the effect of the public comments submitted in response to its proposed rulemaking. As part of finalizing the rules, DOT plans to establish specific deadlines for grantees’ implementation of the MAP-21 performance target- setting and reporting requirements. NHTSA is awaiting enactment of a new surface transportation reauthorization to see if it includes any additional requirements pertaining to its grants programs before deciding on a timeframe.

Multiple Factors Affect DOT’s Progress in Developing a National Performance-Based Approach

DOT’s progress in developing a national performance-based approach has been affected by the extent to which MAP-21 transformed surface transportation programs, the length and complexity of the rulemaking process, and DOT’s varying levels of experience implementing a performance-based approach.

MAP-21 began the transformation of the nature and structure of surface transportation programs by establishing a comprehensive performance-based approach for NHTSA, FHWA, and FTA, in many cases, holding states and other grantees accountable for performance for the first time. Historically, most federal surface transportation funds have been distributed through formulas that have only an indirect relationship to infrastructure needs, and many have no relationship to outcomes or grantees’ performance. In general, funds have been distributed without regard to performance, and therefore, it has been difficult to determine whether federal spending has improved the performance of the nation’s surface transportation infrastructure. As part of this transformational effort to shift from a process-based approach to a performance-based approach, MAP-21 required DOT to establish national performance measures that will be used by states and other grantees to set targets and report on progress in meeting those targets. However, state and grantee officials told us that in some instances, there is no broad consensus among DOT, states, MPOs, and transit grantees on how to implement a performance-based approach. While some states and grantees have been measuring performance for years, state and grantee officials we spoke with told us that their approaches have been tailored to their state and local interests and priorities, as opposed to national goals.

In addition to moving toward a performance-based approach to surface transportation, MAP-21 created a number of new responsibilities for DOT. For example, every four years, DOT is required to review and certify the process used by each state to develop its asset management plan, which identifies strategies that help the state make progress toward its targets for asset condition, among other things. In addition, FHWA and NHTSA are responsible for evaluating and approving the process by which initial targets will be set by states in their state highway safety plans. FTA has additional responsibilities for overseeing the safety of the nation’s transit system. As part of these responsibilities, FTA plans to develop an oversight approach concurrently with the development of performance measures to evaluate federal transit safety. While FTA does not have a formal role in evaluating targets related to the MAP- 21 performance measures, grantees must report to FTA on their progress meeting targets, and identify the following year’s targets.

The rulemaking process is lengthy and complex. Given the length of time a rule typically requires to be developed under the APA and extent to which MAP-21 transformed surface transportation programs, the rulemaking deadlines established by MAP-21 may have been ambitious. In our prior work, we found that rulemakings can range from 1 to nearly 14 years to complete, depending on a number of factors. More specifically, we found that a rulemaking takes an average of four years to complete, based on a sample of 16 rulemakings from four federal agencies, including DOT. Within this sample, these four federal agencies took an average of about 2 ½ years to develop and issue a proposed rule. In contrast, the timeline for MAP-21 generally gave DOT approximately 1 to 1 ½ years to develop and issue some of the proposed rules.

The number and extent of comments received may affect the duration and complexity of the rulemaking process. Specifically, agencies are required to respond to comments that have been received in response to a notice of proposed rulemaking. We have previously found that when agencies did respond to public comments, they often made changes to the rules. FHWA officials stated that they received a substantial number of comments on their safety performance measures rule, and it will take time to address these comments and revise the rule if necessary. Similarly, FTA officials stated that they received over 2500 pages of comments on FTA’s ANPRM, and are reviewing and using the comments to develop FTA’s transit safety and asset management plan rules. In addition, after public release of four of the five proposed rules published thus far, DOT extended the public comment period, which may delay the remaining milestones in the rulemaking process up through the publication of the final rule.

Furthermore, DOT’s rules are interrelated, so delays in establishing performance measures may cause delays in completing the process for moving to a performance-based approach. Until DOT finalizes the performance measures required by MAP-21, states and grantees will be unable to establish targets and report on progress made in relation to those performance measures. For example, three FHWA rules develop performance measures, which directly affect state target- setting and reporting requirements for grantees. (See fig. 3.) Delays in establishing performance measures may also affect information on grantee progress reported to Congress. For example, in MAP-21, Congress directed states to report biennially to DOT, beginning in October 2016, on the progress made in achieving the targets set under DOT’s performance-based approach. MAP-21 also directed DOT to report to Congress in October 2017 on the progress states and MPOs have made in achieving their targets. However, DOT officials acknowledged that the reports due to Congress in October 2017 may contain limited information depending on the effective dates of the MAP-21 performance requirements.

There are varying levels of experience with performance measurement across DOT’s operating administrations. For example, NHTSA has been collaborating with grantees and the Governors Highway Safety Association (GHSA) to develop performance measures in its highway safety grant programs since 2008. According to NHTSA officials, in 2010, states voluntarily began reporting performance measures, targets, and progress meeting these targets in their highway safety plans. NHTSA uses these measures as a basis for reporting to Congress about highway safety. In contrast, FHWA and FTA have had more limited experience in applying a performance-based approach to its grant programs. For example, federal-aid highway program funding has historically been distributed through a complicated process based on statutory formulas. Prior to MAP-21, FTA had incorporated performance-based principles into some of its formula grant programs to allocate funding. We previously found that about 5 percent of fiscal year 2010 funding for six formula grant programs we reviewed, on average, was awarded based on transit agency performance. We found that, after these grants were awarded, FTA was required to collect performance data, but FTA typically did not use the performance data to evaluate the effectiveness of its formula grant programs.

Notwithstanding the factors affecting DOT’s effort to implement the performance-based approach, most states and grantees we spoke with continue to take steps to implement performance-based principles into their transportation programs, though some of these grantees are awaiting further guidance from DOT. Notably, in 2014, FHWA, through its division offices, made an assessment of states’ readiness to implement MAP-21 requirements and, in a draft summary report, stated that most states are actively involved and taking steps to prepare for MAP-21 implementation. State DOTs have also worked with national transportation organizations to develop criteria and broad guidelines for performance measures and to evaluate how they can be applied by members at the state or local levels. For example, officials from the American Association of State Highway and Transportation Officials (AASHTO) and state DOTs have been active in the development and deployment of transportation performance management guidelines. This resulted in six overarching principles on how national-level performance measures should be developed and implemented, which guided AASHTO recommendations on these topics that were submitted to FHWA in November 2012. However, a few of the grantees we spoke with stated that they are waiting for final rulemakings and rollout of national performance measures prior to implementing a performance-based approach. These officials noted that due to the uncertainty about the content of the transit-related rules, they are prioritizing local requirements over the potential MAP-21 requirements.

States and Other Grantees Reported Potential Data and Cost Challenges to Implementing DOT’s Performance-Based Approach

Officials from states and other grantee agencies and representatives from national organizations we interviewed reported a variety of challenges to implementing a performance-based approach, of which data and cost challenges were most frequently cited. Specifically, officials and representatives indicated data availability, collection, and analysis would be significant challenges for their agencies. Officials and representatives also frequently cited implementation costs, some associated with addressing the data challenges, although they acknowledged that they had not or could not estimate the financial impact because DOT’s rulemakings have not been finalized.

States and Other Grantees Reported Potential Data Availability, Collection and Analysis Challenges

Officials and representatives we interviewed anticipated facing challenges related to the need for data, including their availability, collection and analysis. We have previously found that implementing a performance accountability framework necessitates agencies having to collect and analyze data that are useful and relevant. In describing these challenges, state and other grantee officials we interviewed acknowledged that since DOT’s rulemakings had not been finalized yet, it was unclear the extent to which DOT’s requirements may exceed their current data and performance-based efforts.

Representatives from 5 national organizations and officials from 18 state and other grantee agencies we interviewed noted that data may not be available to measure progress, in part because there is a lack of consensus on what or how to measure certain performance areas. For example, state and grantee officials told us they were concerned that data may not be available to measure congestion, in part, because there is a lack of consensus among states and grantees regarding how to measure it. In the absence of consensus, states and other grantees have used a variety of metrics to measure congestion, which could complicate the development of national congestion measures. Furthermore, while there has been a heightened importance placed on congestion data in rural areas due to the MAP-21 requirements and the increased freight traffic in rural areas resulting from oil and gas development, most congestion data have been historically focused on urban areas. Accordingly, a number of grantees we spoke with told us that available data sets may not reflect congestion in rural areas. In addition, some safety data may not be available. For example, while a national database for transportation fatalities exists, no national data source currently exists for serious injuries. Although NHTSA and GHSA developed and issued guidelines for defining serious injuries in 2012, states currently define and collect crash data differently which makes it hard to aggregate data. For example, Florida DOT officials told us their definition of a serious injury varied from other states, making their data inconsistent with other states’. In its proposed rulemaking on safety performance measures, FHWA acknowledged the need for a consistent definition of serious injuries, and proposed steps—including a system for classifying serious injuries—to address this challenge.

Data collection also poses a challenge for states and grantees. Officials from 19 states and other grantees, and representatives from 6 of the national organizations we interviewed reported the following types of potential challenges associated with the data collection:

States may have to collect additional data. For example, MAP-21 expanded the definition of the National Highway System, adding 50,000 miles including many roads owned and operated by local agencies; officials from some states noted that they have not previously collected data on these roads. In addition, FHWA has proposed that states collect data on the average annual daily traffic on all public roads. According to some state officials we spoke with, however, this would be a substantial undertaking. For example, New York DOT officials told us the state currently has adequate data for only about 37 percent of its public roads. Officials from some state DOTs also stated that they may have to collect additional data on local roads, based on their review of the requirements in FHWA’s proposed rulemaking on the Highway Safety Improvement Program. For example, officials from the Montana, North Dakota, South Dakota, and Wyoming DOTs stated the requirements in FHWA’s proposed rulemakings may require the collection of additional data on low- volume roads, such as all gravel and other unpaved roads in their states.

Collecting and standardizing performance data may require additional collaboration among state and local transportation providers. For example, as part of standardizing the approach to collecting data, grantee officials from Michigan reported that it would be difficult to obtain consistent data from rural bus agencies on the condition of their assets. Similarly, officials from one Florida MPO anticipated challenges in obtaining data from transit providers since the MPO has limited means for ensuring transit agencies collect and report data to them. These officials stated they are working with these transit operators to improve cooperation. Also, in order to standardize their approach to collecting data, grantees stated that they may need to revise their collection protocols. However, officials from some states suggested that this could pose a significant challenge because such protocols necessarily involve new training for agencies, such as police departments, to ensure that data is collected uniformly or could involve establishing or revising some thresholds.

Some performance data may be proprietary and expensive to obtain from private transportation providers. We previously found that state and local officials had limited data to quantify rail-related freight impacts (such as traffic delay times or costs) on local communities. State and grantee officials we interviewed suggested that they may need to obtain data from private sources, which in some cases may prove challenging. For example, according to MPO officials in Florida, freight data necessary to measure congestion (specifically on the volume of rail transport through local areas) are proprietary, and it may be difficult or expensive to access this data set from operators. Similarly, officials from the New York State Association of MPOs also raised challenges about collecting performance data on freight, because of their expense and propriety nature. We have previously found that private railroads are often unwilling to provide detailed information on train counts, lengths, speed, and times to local communities to help them quantify traffic impacts at highway-grade rail crossings.

Moreover, representatives from four national transportation organizations and officials from 10 states and other grantees we interviewed stated that even when data can be collected, current staffing levels, areas of expertise and data systems may not support the analyses that may be required. For example, representatives from the National Association for Regional Councils (NARC) told us that ensuring MPOs have sufficient staff capable of monitoring program performance would be a challenge. We previously found that some MPOs have limited expertise or staff to analyze performance data. Furthermore, during the course of our current review, officials from several MPOs told us that they rely heavily on their state DOTs to analyze data. Additionally, some state officials noted that their data systems may not be set up to report or break out analyses according to DOT’s anticipated requirements. For example, South Carolina officials stated that, to conduct the analysis potentially required by DOT’s rulemaking, they may need to revise their existing legacy database systems or invest in a new database system. FHWA officials have acknowledged that the data analysis capabilities of some states and grantees are limited, noting in FHWA’s assessment of state readiness that 35 states may face data integration challenges and that 28 states may face data processing challenges.

While several state and other grant recipient officials we spoke with reported anticipating few or no data-related challenges, many of these officials raised concerns about how data and performance measurement would be ultimately used by DOT. Some state officials expressed concern that DOT’s approach will direct states to develop targets based on outdated national data, as opposed to more current data available to the states. For example, NHTSA requires that states develop targets for highway safety based on data collected and submitted into FARS. However, the lag between when these data are collected and submitted by states and finalized by NHTSA is approximately 24 months. As a result, officials from several states, as well as AASHTO and GHSA representatives, expressed concern that states may be required to set targets based on outdated FARS data, rather than focus on setting targets based on the more current data collected by states, and that DOT would be using outdated data to determine whether states are making significant progress in meeting their targets. Officials and representatives also questioned if performance measurement should or would be tied to DOT’s funding decisions, which some of them opposed, because they reported that it would be difficult to compare the transportation and transit priorities and operating environments of different states and transit agencies. Officials from several states further underscored to us that performance was a function of available funding and investments, which has not kept pace with the level of deteriorating infrastructure. Moreover, officials from some states and other grantees raised concerns that DOT’s requirements might impede or hinder their approaches that have evolved over time to meet state and local needs. For example, Washington state officials anticipated that many of their existing performance management programs may be broader in scope than MAP-21 performance measures and provide more detailed information necessary to support state investment decision processes. The officials noted that the effort required to comply with the DOT national performance requirements in MAP-21 may affect the state’s ability to support its own state performance measurement and investment decision process. These officials reported being concerned with how DOT will integrate legacy performance programs in states like Washington into the new framework.

DOT officials acknowledged that states and grantees will face some data challenges as part of the transition to the performance-based approach, though DOT officials noted they have taken some steps to anticipate these challenges. For example, DOT has encouraged and worked with states to share data, best practices, and templates for reporting. In addition, DOT officials stated that their overall approach has emphasized the use of existing data sets and plans to further detail its approach to ensuring data availability through its upcoming rulemakings. In addition, FHWA has used information gathered through its assessment of state readiness to help inform its technical assistance program, which DOT anticipates will address some of the above mentioned challenges.

Implementation Costs Could Vary and Be a Significant Challenge for Some States and Grantees

Representatives from 5 national organizations and officials from 22 of the 29 states and other grantees we spoke to reported potential challenges with the costs of implementing a national performance measurement system. According to FHWA officials, the total implementation costs of any one of its proposed rulemakings to states and grantees in any one year will not exceed $100 million for any one rule. For example, in March 2014, FHWA estimated the 10-year costs associated with the eight new requirements proposed in its Safety Performance Measure rulemaking (see table 1) were approximately $67 million. About $26 million of this is estimated to be one-time costs incurred by state DOTs and MPOs to standardize and update data collection and reporting systems, as well as the increase in level of effort from labor to establish and report targets. In calculating this cost, DOT assumed that a portion of MPOs, approximately half, would establish their own targets and that a portion would adopt state DOT targets. FHWA acknowledged that larger MPOs may have more resources available to develop performance targets and that it believed smaller MPOs (those with populations under 200,000) would have no incremental costs as they would adopt state-established targets. DOT’s estimates for other rulemakings reflect the associated costs of activities (e.g., planning and data collection) beyond what is currently performed by states, MPOs, and providers of public transportation.

On a per-state or agency basis, the cost of implementation may vary depending on the characteristics of each grantee, and some states reported that these costs may exceed the average cost estimated by DOT. One MPO official reported that her organization believed DOT’s estimate that the cost of administering oversight of transportation planning would increase no more than 15 percent was low. According to officials from some of the national organizations and officials from about one-third of states and grantees we interviewed, implementation costs to hire additional staff or allow for staff time and travel may be burdensome, particularly for smaller or more rural states and grantees. Officials from the New York State Association of MPOs explained that hiring and training new staff to collect and analyze performance data would require additional funding and that smaller MPOs would be more acutely affected than the larger MPOs, particularly those that do not currently conduct sophisticated analysis with the performance data. One rural transit provider we interviewed also relayed concerns about having to potentially hire an additional staff person to serve as a chief safety officer in order to meet this requirement being considered by FTA. Officials from some planning organizations we interviewed underscored their lack of planning professionals and how the few individuals employed by their agencies were already responsible for fulfilling multiple roles.

Implementation costs may also vary by grantees’ experience in different surface transportation areas. In summer 2014, FHWA queried its division offices about the readiness of states to implement performance-based approaches to transportation projects in their states. According to the division offices, in the areas of congestion, air quality and freight, only 20 to 40 percent of states had adopted performance management practices, and only about 10 to 20 percent of states are very highly prepared to carry out performance-based programs in these areas. For example, in Nevada, FHWA officials reported that state officials may need some additional assistance to integrate data and develop metrics for congestion, freight, and air quality. On the other hand, state DOTs and other grantees may be better positioned to meet DOT’s performance measurement requirements in other areas, such as safety, pavement, and bridge condition. According to FHWA’s assessment, about 40 to 50 percent of the states are ready to carry out a performance-based program in some areas, such as safety, pavement, and bridge condition, where, historically, measures and targets have been used by states to evaluate program performance. According to the division offices, in these areas about 60 to 70 percent of the states integrate performance into their long- range plans, develop quantifiable targets to guide investment decision making, and develop plans that document their strategies to achieve targets.

According to AASHTO, some of its members have concerns that collecting data to support these rules may reduce the portion of their federal highway grants available to fund transportation projects over the long term. While these costs are eligible for federal planning funds, officials from several grantee agencies said they do not expect additional federal funding for implementation. Representatives from several national organizations and officials from some states and other grantees we interviewed emphasized that DOT should, whenever possible, leverage existing resources or data or procure data sources itself. Representatives from the American Public Transportation Association suggested that DOT consider the data that it will be asking transit agencies to collect, pointing to recent FTA-sponsored asset management pilot projects that demonstrated the data collection process can be burdensome and should be based on the types of information that states and local transit agencies currently collect. Similarly, representatives from the National Association of Development Organizations also underscored the cost of acquiring and analyzing data and the need to structure data collection around existing sources. DOT officials agreed that data availability, collection, analysis and associated costs present potential challenges, and stated that in order to address this situation, DOT has emphasized the use of existing data sets and intends to further detail its approach to ensuring data availability through its upcoming rulemakings.

Concluding Observations

DOT’s effort to develop a performance-based approach for its various surface transportation grant recipients represents a significant shift for the Department, transforming the federal involvement in surface transportation to a more outcome-oriented approach, as we have recommended since 2008. In many cases, the shift to a performance- based approach will hold states and other grantees accountable for performance for the first time. Such an approach could provide a strong foundation for holding grant recipients responsible for achieving federal objectives and measuring overall program performance. However, the development and implementation of a performance-based approach is not without challenges. Notably, as NHTSA’s implementation of a performance-based approach has demonstrated, this broader effort of incorporating performance-based principles into the overall surface transportation program may take longer than Congress originally anticipated. In addition, while many grantees have been evaluating the performance of transportation projects for many years, experiences with performance measurement varies and even the most experienced grantees may face challenges aligning their individual approaches with DOT’s national approach. Nevertheless, given the billions of dollars provided annually in federal funding to improve the nation’s highway and transit infrastructure and safety—and competing demands for that funding—the ability to assess the outcomes of federal surface transportation programs and hold grant recipients accountable for results could help all parties more efficiently allocate federal funds to address specific challenges and areas of concern.

Agency Comments

We provided a draft of this report to DOT for review and comment. DOT provided technical comments, which we incorporated as appropriate. We are sending copies of this report to interested congressional committees and the Secretary of Transportation. In addition, this report will be available at no charge on GAO’s website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-2834 or flemings@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

The objectives of our review were to determine: (1) the progress that DOT has made in developing a national performance-based approach to surface transportation and (2) the challenges states and other grantees report facing in implementing this approach.

To address these objectives, we reviewed scholarly/peer-reviewed, government, trade, and non-profit literature from 2011 through 2014, the time period just before and since MAP-21 was enacted. The reviewed literature included those published by the National Cooperative Highway Research Program, which is sponsored by FHWA and American Association of State Highway and Transportation Officials (AASHTO) and presented at the Transportation Research Board annual conference. The literature examined local and national transportation performance based approaches. We also reviewed DOT-conducted performance management case studies and workshop summaries and reports that were publicly available, as well as, our body of work on performance management in the federal government and transportation programs.

To examine the progress that DOT has made in developing a national performance-based approach for surface transportation programs, we reviewed related requirements and timetables in MAP-21 for FHWA, FTA, and NHTSA as well as interim and proposed rulemakings developed by these operating administrations in response to these requirements. We did not evaluate the potential effectiveness of the proposed rules because the content may change before rules are finalized in response to comments submitted by grantees and other parties. We also analyzed DOT documents, information obtained through interviews with DOT officials, guidance issued through the rulemaking process, and information from grantee interviews. We also analyzed information from the Report on DOT Significant Rulemakings, issued monthly, which provides the original, actual, and projected dates for the major milestones in the rulemaking process to understand the progress FHWA, FTA, and NHTSA have made developing the performance-based approach. In the event that actual dates were unavailable, projected dates, obtained directly from the Report on DOT Significant Rulemakings were used to calculate the length of milestones. We interviewed FHWA, FTA, and NHTSA officials to understand the factors affecting DOT’s progress in developing the performance-based approach.

To understand the challenges states and other grantees face in implementing a performance-based approach to surface transportation, we analyzed comments submitted to DOT by and interviewed representatives from seven national transportation organizations (such as AASHTO, Association of Metropolitan Planning Organizations, and American Public Transportation Association), which broadly represent state and local transportation, transit, and planning interests and perspectives, both in urban and rural communities.

We also reviewed the comments submitted to DOT and conducted semi- structured interviews with state and local officials from 29 grantee agencies in 13 states—Florida, Maryland, Michigan, Montana, New York, North Dakota, Oregon, South Carolina, South Dakota, Virginia, Washington, Wyoming, and Utah. We developed criteria—population growth trends, population size, geographic location, transit infrastructure age, and experiences managing and measuring transportation project performance—which we used to select a cross-section of grantee agencies (i.e., state transportation agencies, state highway safety offices, metropolitan and rural planning organizations, and transit agencies). We also relied on suggestions from trade organizations and DOT to identify agencies that would meet our criteria and to identify specific state and local officials. Based on these criteria, in four states, we interviewed officials from the state department of transportation, one or more metropolitan planning organizations, local transit providers, and DOT regional oversight offices; in the remaining nine states, we interviewed officials from at least one state transportation agency, planning organization or transit agency in that state. In addition to discussing anticipated challenges, state and grantee officials also provided information about state and local transportation planning processes, the experience and extent of available performance management data, and DOT’s progress to date with its outreach and rulemaking. The results of interviews with state and local officials from these 29 grantee agencies cannot be generalized to all federal transportation grantees; however, they provided insight into the types of challenges that some grantees anticipate in implementing DOT’s future rulemaking and overall performance based approach, as well as, perspectives on DOT’s outreach and rulemaking progress.

To further understand the challenges that states and other grantees may face, we also reviewed the draft results of a questionnaire DOT administered to its 52 FHWA Division Offices assessing states’ readiness to implement a performance-based approach. This questionnaire was administered from July through August 2014, and its purpose was to assist FHWA in how to best design program support for states in meeting requirements under DOT’s future performance-based approach.

We conducted this performance audit from May 2014 to January 2015 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Department of Transportation Rulemakings for Selected MAP-21 Performance Requirements

Appendix II: Department of Transportation Rulemakings for Selected MAP-21 Performance Requirements The Statewide, Metropolitan, and Non-Metropolitan Planning rulemaking has been developed jointly with the Federal Transit Administration.

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the individual named above, Steve Cohen (Assistant Director), Sara Vermillion (Assistant Director), Timothy Bober, Melissa Bodeau, Elizabeth Curda, Bert Japikse, Terence Lam, Sara Ann Moessbauer, Josh Ormond, Amy Suntoke, Friendly Vang-Johnson, and Matt Voit made key contributions to this report.