Surface Transportation: A Comprehensive Plan Could Facilitate Implementation of a National Performance Management Approach
Published: 2017-07-27T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Since 2008, GAO has highlighted the need to demonstrate the outcomes of the billions of dollars the Department of Transportation (DOT) provides to states and other grantees for surface transportation programs. MAP-21 included provisions for DOT and its grantees to move toward a performance-based approach, transforming federal surface transportation programs by holding states and other grantees accountable for results, in many cases for the first time.
GAO was asked to review DOT's implementation of TPM. This report focuses on FHWA, which administers the largest grant program of the three DOT agencies involved, and (1) examines the progress made in developing rules to establish a national performance-based approach and (2) evaluates how FHWA is guiding the transition to TPM, among other objectives. GAO reviewed proposed and final rules and information on rulemaking activities and interviewed FHWA officials, national transportation organizations, and a non-generalizable sample of 10 state departments of transportation, among others. States were selected based on factors such as geographic distribution, population, and urban and rural characteristics.
section_title: What GAO Found
paragraphs: The Federal Highway Administration (FHWA) recently issued the last of six interrelated rules to implement a new performance-based approach to federal surface transportation grant programs. Three of the six rules establish 17 total performance measures in the areas of safety, pavement and bridge conditions, and performance of the surface transportation system (congestion, freight movement, reliability, and air quality). Rulemaking was the first of three stages to implement the transportation performance management (TPM) approach, which was required by the Moving Ahead for Progress in the 21st Century Act (MAP-21) in 2012 (see fig.). The agency issued the first of the six rules in March 2016. In January 2017, the last two final rules were issued and became effective in May 2017, with part of one rule related to greenhouse gas emissions delayed indefinitely.
FHWA is developing tools and training to help implement TPM but has not yet developed a comprehensive plan to guide implementation. The agency has begun to help states and regional metropolitan planning organizations (MPO) prepare to set targets and begin tracking performance as part of the second implementation stage. However, the agency has not clearly articulated strategic goals for the TPM transition, developed an implementation plan showing how the various efforts under way relate to each other and when they will be completed, or clearly communicated the approach to states and MPOs. For example, the agency has not identified strategic goals describing what each stage of the transformation is intended to accomplish and how the various efforts under way will help accomplish those goals. While FHWA has stated in its rulemaking that it wants to “communicate a national performance story,” it has not identified specific processes or activities that will be essential to achieving these results. Developing, communicating, and reinforcing the goals of a transformation, as well as specific activities and timelines for achieving those goals, are among the key practices that GAO has identified for major organizational transformations. An implementation plan could help the agency better articulate the goals of the transformation, make the best use of the performance information states will begin reporting, and help assess the effects of federal funding to improve investment decision-making.
section_title: What GAO Recommends
paragraphs: FHWA should develop a TPM implementation plan that includes goals and specific actions and timelines, and publicly communicate that plan. DOT concurred with the recommendation. The agency also provided technical comments, which were incorporated as appropriate.

section_title: Letter
paragraphs: The nation’s surface transportation system moves people and freight, is  critical to the economy, and affects the daily lives of most Americans.  However, the system is under growing strain, and the cost to repair and  upgrade the system to meet current and future demands is estimated in  the hundreds of billions of dollars. The U.S. Department of Transportation  (DOT) annually provides billions of dollars to states and other grantees to  improve the nation’s highway and transit infrastructure and safety. DOT  provides federal funding to these grantees, including states, transit  agencies, and regional metropolitan planning organizations (MPO),  primarily through programs administered by three of DOT’s operating  administrations: The Federal Highway Administration (FHWA), which  administers the federal-aid highway program that provides billions to  states for highway and bridge projects; the National Highway Traffic  Safety Administration (NHTSA), which provides grant funding to support  state highway safety programs; and the Federal Transit Administration  (FTA), which administers a variety of grant programs that provide funding  for the development and expansion of bus, rail and other transit projects,  among other things. In fiscal year 2015, DOT provided approximately $49  billion to grantees for surface transportation projects through the  programs administered by FHWA, FTA, and NHTSA, of which FHWA  accounted for approximately $40 billion in formula funding.
We have previously found it difficult to determine whether DOT’s federal  spending improved the performance of the nation’s surface transportation  infrastructure because funding was only indirectly related to infrastructure  needs and unrelated to outcomes or performance. Beginning in 2008, we  recommended that Congress consider a fundamental reexamination of  federal surface transportation programs to clarify federal goals and roles,  establish links to performance and outcomes, and improve investment  decision making. In particular, we noted that tracking specific state  outcomes that are clearly linked to federal program goals could provide a  strong foundation for holding grant recipients responsible for achieving  federal objectives and measuring the overall performance of these  programs. Funding the nation’s surface transportation system has been  on GAO’s High Risk List since 2007.
The Moving Ahead for Progress in the 21st Century Act (MAP-21),  enacted in 2012, adopted a performance-based approach to funding the  federal government’s surface transportation grant programs. This  approach represents a transformation that, once fully implemented, is  expected in many cases to hold grantees—including state departments of  transportation and MPOs—accountable for performance for the first time.  This approach also represents a significant shift for DOT and its operating  administrations, transforming the federal involvement in surface  transportation to a more outcome-oriented approach. For example, states  that receive FHWA funding will be required to measure and track bridge  and pavement conditions, and undertake action should conditions fall  below minimum thresholds. Additionally, MAP-21 required that the  federal-aid highway program focus on several national goals including  goals related to infrastructure condition, transportation safety, and system  reliability. For example, one national goal is to maintain highway  infrastructure assets in a state of good repair.
Shortly after the enactment of MAP-21 and to meet certain requirements  to establish performance measures and standards for transportation  performance management (TPM), DOT’s operating administrations began  developing a number of rules. Among other things, MAP-21 required  target setting by states and established reporting requirements for  grantees. Under the implementing regulations, states and other grantees  must set annual or multi-year targets based on the performance  measures and report annually or biennially on the progress in meeting  these targets. Finally, for highway safety, MAP-21 requires DOT to  evaluate each state’s performance on targets states have set in their  highway safety plans and biennially report such evaluations to Congress.  In 2015, we found that DOT had made progress in moving toward a  national performance-based approach to surface transportation, but that  states and other grantees anticipated potential data and cost challenges  with implementing the new approach.
You asked us to review issues related to DOT’s implementation of the  national approach to TPM. This report focuses on FHWA because it  represents the vast majority of federal surface transportation spending,  and this report:  1.  examines the progress FHWA has made in developing rules to  establish a national performance-based approach to highway  transportation and the nature of those rules,  2.  identifies challenges states anticipate in implementing the  3.  evaluates how FHWA is guiding the transition to national  transportation performance management.
To determine the nature of FHWA’s progress in developing TPM rules  and to evaluate the steps the agency is taking to guide the transition, we  reviewed surface transportation performance management requirements  for DOT and its modal administrations, focusing on FHWA’s role in  implementing MAP-21 requirements. We focused primarily on the rules  related to the condition and performance of the federal highway system,  reviewing the six rules specific to FHWA’s efforts to carry out  performance-based requirements. We also reviewed the 2015 Fixing  America’s Surface Transportation (FAST) Act, which maintained the  performance-based approach. At the time of our review, some of rules  were proposed and in the notice and comment period; others were in the  process of being issued as final rules; and some were final. Given the  ongoing nature of the rulemaking process, we reviewed the most current  version of each rule available at the time of our review. In addition, we  reviewed agency fact sheets, guidance, presentations, and outreach  materials pertaining to the various rules and documents outlining  implementation activities as well as planned efforts for the post- rulemaking stages of TPM implementation. We also met with DOT  officials and FHWA’s TPM office officials to discuss the status of the  agency’s efforts to meet the performance goals and complete the  rulemaking process as well as the plans for the remaining stages of  implementation. In addition, we met with representatives from eight  national transportation industry associations representing states, transit  agencies, MPOs, and other planning entities to discuss their perspectives  on FHWA’s TPM implementation efforts. This included, for example,  representatives from the American Association of State Highway and  Transportation Officials (AASHTO) standing committee on performance  management. We also met with experts from two academic institutions,  selected because they specialize in transportation data analysis, are  familiar with the new performance management requirements, and have  experience working with multiple state DOTs and MPOs on these issues.  We also reviewed our prior work on organizational mergers and  transformations and evaluated FHWA efforts against key practices  identified in this work, focusing on selected key practices that were  directly applicable to the agency’s approach to implementing  transportation performance management.
To identify states’ anticipated implementation challenges, we focused on  three of the six FHWA rules that relate to: pavement and bridge conditions (Pavement and Bridge rule);  the performance of the surface transportation system, including freight  movement and air quality (System Performance rule); and  the transportation-planning process involving states and MPOs  (Planning rule).
We selected FHWA’s Pavement and Bridge and System Performance  rules because they contain new performance measures for the states  related to infrastructure conditions and performance. We selected the  joint FHWA and FTA Planning rule because it outlines the framework  states and MPOs must implement to integrate the performance-based  approach into their transportation planning processes. This framework  directly relates to how states and MPOs will establish specific targets for  each of the required performance measures. We excluded another  performance measure rule on safety—the only other rule among the six  that established performance measures—because states already have  experience tracking and reporting similar safety data. In addition, our past  work found that states generally do not anticipate significant challenges in  this area. For the Pavement and Bridge, Planning, and System  Performance rules, we reviewed the final rules, including DOT’s summary  of significant issues raised and changes made in response to comments.  The Pavement and Bridge and Planning final rules included summaries of  comments submitted by 42 and 36 state DOTs, respectively, as well as  MPOs. For the System Performance rule, which was in the notice of  proposed rulemaking phase during the bulk of our engagement, we also  reviewed and analyzed comments submitted by the 36 state DOTs that  submitted comments. We also met with state DOTs’ officials and FHWA’s  division office officials in 10 selected states to discuss their perspectives  on anticipated implementation challenges associated with TPM in general  and these three rules in particular. The selected states were Alaska,  Arizona, Florida, Illinois, Maine, Mississippi, New York, Rhode Island,  South Dakota, and Texas. We selected these states to include a range in  terms of geographic distribution, population size, urban and rural  characteristics, and experience managing and measuring transportation  performance. In addition, we interviewed the eight national organizations  and academic experts discussed above. The information gathered from  the selected states and industry associations cannot be generalized to all  federal surface transportation grantees; however, they provided insight  into the types of implementation challenges that some grantees  anticipate. We also reviewed recent academic literature from 2013  through 2017—representing the time period since MAP-21 was  enacted—as well as our body of work on performance management in  the federal government and transportation programs.
We conducted this performance audit from May 2016 to July 2017 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: DOT provides billions of dollars to states and other grantees annually to  improve the nation’s highway and transit infrastructure and safety. Most of  this federal funding is provided by FHWA, FTA, and NHTSA. FHWA  provides the vast majority of federal surface transportation funds—about  $40 billion each year to states to design, construct, and maintain the  nation’s roadway and bridge infrastructure through the federal-aid  highway program. This program includes the nation’s National Highway  System, which consists of approximately 220,000 miles of the nearly 1- million miles of roadways eligible for federal aid, including the 47,000-mile  Interstate Highway System. Historically, most surface transportation funds  are distributed through annual apportionments established by statutory  formulas that take into account a number of factors, including the  estimated share of taxes highway users in each state contribute to the  Highway Trust Fund. We have previously reported that these formulas  have only an indirect relationship to infrastructure needs, and many have  no relationship to outcomes or grantees’ performance.
MAP-21 addresses several of our past recommendations—by articulating  national goals and adopting a performance-based approach to funding  surface transportation projects. The act established seven national  surface transportation goals for the federal-aid highway program:
Safety—to achieve a significant reduction in traffic fatalities and  serious injuries on all public roads.
Infrastructure condition—to maintain the highway infrastructure asset  system in a state of good repair.
Congestion reduction—to achieve a significant reduction in  congestion on the National Highway System.
System reliability—to improve the efficiency of the surface  transportation system.
Freight movement and economic vitality—to improve the National  Highway Freight Network, strengthen the ability of rural communities  to access national and international trade markets, and support  regional economic development.
Environmental sustainability—to enhance the performance of the  transportation system while protecting and enhancing the natural  environment.
Reduced project delivery delays—to reduce project costs, promote  jobs and the economy, and expedite the movement of people and  goods by accelerating project completion through eliminating delays in  the project development and delivery process, including reducing  regulatory burdens and improving agencies’ work practices.
The act and its implementing regulations set forth a three-stage process  for the performance-based surface transportation transformation across  multiple modes:  1.  Rulemaking: DOT must establish performance measures and  2.  Set and track performance targets: states, MPOs, and other grantees  must set annual or multi-year targets based on these performance  measures and states report on their progress on meeting such targets  to DOT, and  3.  Evaluate performance: DOT must evaluate whether states have met  or made significant progress toward their targets.
States have one year from the effective date of the performance measure  final rules to establish their own performance targets for those measures.  MPOs then have 180 days after the respective state’s DOT sets its  targets to establish their own targets or agree to support the state’s  target. If states do not make significant progress toward meeting some  targets, they must report action they will undertake to achieve their  targets. Generally, the targets established will cover a 4-year reporting  period and will be reported biennially. MAP-21 also mandated that states  begin reporting their progress toward targets to DOT no later than  October 2016. In addition, the legislation directed DOT to report to Congress in October 2017 on the effectiveness of performance-based  planning as a tool for guiding transportation investments and the  effectiveness of the performance-based planning process of each state.  The performance-based approach remains essentially unchanged in the  FAST Act—the most recent surface transportation reauthorization  measure—enacted in December 2015.
In administering approximately $40 billion a year to grantees for the  federal-aid highway program, FHWA manages a variety of grant  programs that provide funding for the development of infrastructure  projects, and jointly with FTA, administers the implementation of the  statewide and metropolitan planning requirements carried out by states  and MPOs, discussed below. FHWA oversees the federal-aid highway  program primarily through its 52 division offices located in each state, the  District of Columbia, and Puerto Rico. To help lead the agency’s efforts to  implement a performance-based approach, FHWA established the office  of Transportation Performance Management in 2013 to help oversee the  rulemaking process and the agency’s implementation efforts.
Regulatory agencies, such as DOT, have authority and responsibility for  developing and issuing regulations. The basic rulemaking process is  spelled out in the Administrative Procedures Act. Among other things, the  act establishes procedures and broadly applicable federal requirements  for informal rulemaking. In addition, the act generally requires agencies  to publish a notice of proposed rulemaking in the Federal Register. After  giving interested persons an opportunity to comment on the proposed rule  by providing “written data, views, or arguments,” and considering those  comments, the agency may then publish the final rule. In our prior work,  we found that rulemakings can range from 1 to nearly 14 years to  complete, depending on a number of factors. More specifically, we found  that a rulemaking takes an average of 4 years to complete, based on a  sample of 16 major or other significant rulemakings from four federal  agencies, including DOT.
TPM will be carried out as part of the transportation planning process  prescribed by statute for state DOTs and MPOs. This process addresses  both urbanized and nonmetropolitan areas of the state and includes  planning and coordination for both highway and transit needs jointly  administered by FTA and FHWA. Under this process, states engage in  continuing cross-agency coordination and planning that requires periodic  updates of specific plans. For example, states develop a variety of plans  for overall transportation programs, including a long-range statewide  transportation plan and a state transportation improvement program. In  urbanized areas, MPOs are also required to produce a long-range  transportation plan, referred to as a metropolitan transportation plan, and  a transportation improvement program. While MAP–21 left the basic  framework of the transportation-planning process largely untouched, the  act introduced critical changes to the process by requiring grantees such  as states, MPOs, and transit agencies to move toward more  performance-based highway and transit programs, and linking investment  priorities to the achievement of performance targets. Under the new TPM  approach, states and MPOs must coordinate on the target-setting and  transportation-planning process, and states are required to integrate  performance targets into both existing planning processes and plans.  TPM also seeks to ensure that the transportation-planning process  provides a performance-based approach to decision-making in support of  national goals.
section_title: FHWA Recently Completed the Rulemaking Stage, Establishing a National Approach to Transportation Performance Management
paragraphs: FHWA has recently completed the first of the three-stage TPM  implementation process—rulemaking, setting and tracking performance  targets, and evaluating performance—issuing all six interrelated rules that  establish both the performance measures focused on the national  transportation goals and the overarching performance-management  framework (see fig.1). The agency issued the first of the six rules in  March 2016. In January 2017, the last two rules were issued and became  effective in May 2017, with part of one rule delayed indefinitely.  Completing the rulemaking process was a substantial undertaking for  FHWA, as none of the rules had been completed when we last reported  on this topic in 2015. With the completion of the six final rules, the  framework for a performance-based approach to highway transportation  management is now in place.
FHWA is responsible for six rules related to the condition and  performance of the federal highway system, one of which is administered  jointly with FTA (see table 1). These six rules are interrelated because  some establish specific performance measures while others relate to the  transportation-planning processes or specific federal programs related to  the performance measures. For example, three rules established the national performance measures (17 measures in total) to be used by  state DOTs and MPOs, including rules focused on specific infrastructure  condition measures, while other rules outline how the performance-based  approach will be carried out through the transportation-planning process  that occurs between states and MPOs.
The 17 performance measures focus on key indicators of (1) safety, (2)  pavement and bridge condition, and (3) performance of the surface  transportation system (reliability, congestion, freight movement, and air  quality). Generally, these performance measures correspond to national  federal-aid highway program goals and performance areas set forth in  MAP-21. For example, one performance measure—the percentage of  pavements on the Interstate System in good condition—corresponds to  the “pavement condition on the Interstate System” performance area and  to the national goal to “maintain the highway infrastructure asset system  in a state of good repair.” See appendix I for additional information on the  17 performance measures and the corresponding goals and performance  areas.
Many of these measures are fundamentally new to states while a few  other measures are similar to information some states already have  experience analyzing and tracking. For example, in 2015 we found that  only 20 to 40 percent of states had adopted performance management  practices in the areas of congestion, air quality and freight. In contrast,  states have been working to develop safety performance measures in  highway safety grant programs since 2008. The final System  Performance measure rule establishes six measures based on MAP-21  requirements to assess the performance of the Interstate and non- Interstate National Highway System, traffic congestion, freight movement,  and emissions. For example, two of these measures are to carry out the  National Highway Performance Program, including a measure of the  percent of person-miles traveled on reliable Interstate system roadways.  The Pavement and Bridge rule establishes six measures and contains  minimum thresholds for pavement and bridge conditions, and if a state  falls below those thresholds it must spend a portion of its annual federal  funding for improvements.
While DOT began the rulemaking process shortly after MAP-21 was  enacted, it has taken several years for the rules to be developed and  move through the process, resulting in the recent issuance of the final  rules. For example, DOT completed the first two rules in March of 2016 and issued the last two of the six rules in January 2017. The last two rules  took effect on May 20, 2017. As we noted in 2015, MAP-21 established a  deadline for promulgating a rulemaking for five of the rules; however, no  such deadlines existed for one of the rules. At that time, we found that  DOT had missed five performance-related rulemaking deadlines  established in MAP-21; however, given the extent of regulatory changes  required and the length and complexity of the rulemaking process, these  deadlines may have been ambitious. For example, in response to the  proposed System Performance rule, DOT received more than 8,800  public comments, which the agency then had to review and respond to  before finalizing the rule.
section_title: Most Significant Challenges States Anticipate Are Analyzing and Reporting on Complex Data Required by the System Performance Rule
paragraphs: In their comments on the System Performance, Pavement and Bridge,  and Planning rules as well as in interviews we conducted with selected  states, states anticipated that the complexity of the data and the  capabilities needed to use that data to set performance targets and  measure progress under the System Performance rule as the most  pressing challenge in implementing the performance-based approach. In  contrast, states anticipated few challenges implementing the Pavement  and Bridge rule and some challenges related to the Planning rule.  According to selected states and others, anticipated data challenges in  implementing the System Performance rule include the size of the data  set required, the process of combining data sources, and other issues  that may result in states facing significant additional costs and difficulties.  In the final rule, FHWA took steps to respond to these anticipated  challenges, which were raised in public comments, by revising some  performance measures and eliminating others. Nonetheless, some  stakeholders we spoke to believe that FHWA did not fully address some  significant concerns. FHWA officials stated that revisions made in the final  rule along with new data improvements and planned technical assistance  adequately address data concerns expressed by the stakeholders.
subsections: section_title: States Identified Various Challenges in Implementing System Performance Rule
paragraphs: In comments in response to the proposed System Performance rule and  in our interviews with officials in selected states, states reported that the  anticipated costs and difficulties associated with collecting, analyzing, and  reporting data associated with the rule was the major challenge affecting  TPM implementation. The System Performance rule includes six  performance measures related to highway system performance, freight  movement, and congestion. Twenty-nine of the 36 state DOTs that  commented on the proposed rule and 8 of the 10 states we met with cited  the System Performance rule’s reliance on an FHWA-prescribed data set  as the major source of the anticipated challenges. FHWA provides this  data set, the National Performance Management Research Data Set  (NPMRDS), to states and MPOs, which will then use it to calculate and  set targets and measure progress for four of the six performance  measures established in the rule. FHWA selected this data set to  calculate the metrics for travel time and speed-based measures to  promote consistency and coverage at a national level.
Specifically, anticipated challenges related to the dataset included: (1) its  size, (2) the process necessary to combine disparate traffic volume and  speed data, and (3) the quality of the data.
Size of the data set—According to AASHTO representatives and  academic experts from two institutions we met with, the sheer size of  the data set and server space required will affect their ability to  analyze the data and will be a challenge for many states. AASHTO,  for example, said reporting on the measures will require states to  develop computerized applications to analyze millions, in some cases  hundreds of millions of data points, principally in order to identify  areas where traffic is congested. Officials from one state’s DOT said  their staff does not have the training or experience to conduct the data  analysis necessary to report congestion bottlenecks and congestion  with a data set as large as NPMRDS. Five of the 10 state DOTs we  interviewed raised similar concerns about their ability to analyze the  large dataset.
Process of combining data sources—Four of the six measures in the  System Performance rule require estimates of delay or estimates of  the reliability of person miles traveled or truck travel times. Data will  need to be combined from two different data sources before these  performance measures can be calculated, and most state officials and  academic experts we spoke to agree that this initial step will be  challenging. The two data sources include traffic volume data from the  Highway Performance Monitoring System—which states have been  using for many years to monitor highway conditions—and the  NPMRDS data containing highway traffic speeds. For example, one  state said that combining the two sets of data—a process typically  called “conflation”—requires manual and highly technical adjustments  to the underlying data. This process is particularly challenging  because it requires advanced technical expertise. Some state DOTs  also emphasized that they have limited experience using the data,  which will necessitate significant additional work to perform the  required calculations.
Data quality—Some states expressed concerns with the quality and  completeness of the NPMRDS data. For example, Michigan DOT  officials expressed concerns about the amount of missing travel time  data. In the Detroit area in particular, more than 50 percent of the data  are missing for some routes. These officials noted they do not believe  sound investment decisions will be made if they are based on  incomplete data.
Officials we met with in most states, many comments by states on the  proposed rule, and the national MPO association stated that they  anticipate added costs and burdens related to implementing the System  Performance rule, given the technical capabilities and expertise  necessary to mitigate these data issues and use the NPMRDS data set.  In comments on this rule, AASHTO noted that overall, the measures and  calculation methods are considered to be overly complex even by those states that have been viewed as leaders in developing performance  measures and related tools, such as Missouri, Texas, and Washington.  Similarly, a national MPO association told us that the vast majority of  MPOs do not have the capacity to conduct their own analysis of regional  system performance and may need to rely on the state to do the required  analyses. For example, 28 of the 36 state DOTs that commented on the  proposed rule and 7 out of our 10 selected states anticipate technical,  human resource, or funding challenges associated with the data  requirements. Several states specifically noted they would need outside  expertise, such as academic experts or contractors, or additional  resources such as training, hardware, or software for staff to address  these data challenges, which can be costly responses for states facing  limited resources and budget constraints. For example, officials from one  state DOT that is considered to be very experienced with transportation  performance measurement commented that to implement the rule, state  DOTs and MPOs must find additional employees and funding to complete  the extensive workload needed to collaborate with other agencies, as the  rule requires for the transportation planning process. Officials from  another experienced state DOT also commented that the use of  NPMRDS requires data-management and statistical-evaluation skills that  are not commonly found within DOT and MPO staffs. Thus, these officials  noted using these data for analytical and reporting requirements triggers  the need to invest in capabilities outside of their organizations, such as  working with universities to provide the expertise needed to provide  ongoing data analysis and data management of the NPMRDS or  contracting for expensive proprietary systems that could handle the  computational- and analytical-reporting challenges.
section_title: States Anticipate Fewer Challenges with Pavement and Bridge and Planning Rules
paragraphs: In contrast to the System Performance rule, states that commented on  the final rules and the selected states we met with generally did not  anticipate major challenges in implementing the Pavement and Bridge  final rule. According to AASHTO, the impact on states’ implementing the  Pavement and Bridge rule will be less significant and onerous because  most states have been measuring pavement and bridge conditions for  years. FHWA has also previously reported that most states are already  actively involved in preparing to implement legal requirements in these  more mature program areas. According to FHWA, states have historically  expended a large amount of funds and resources collecting and analyzing  pavement and bridge data, and they are more experienced with the  analyses necessary to calculate these measures than they are for the  new System Performance rule.
Because the TPM rules are interrelated, elements of the final Planning  rule also affect implementation of the System Performance rule. Given  this effect, states and MPOs expressed some general concerns in their  comments on the final rules and in interviews with how state and local  planning activities will be carried out. Under the state transportation  planning process, setting and agreeing upon performance targets for the  17 performance measures necessitates coordination between a state and  MPOs in the state. Once states set their performance targets, generally  MPOs will have 180 days to decide whether to set their own targets  based on conditions and priorities in their respective planning areas, or to  agree to the state-level targets. Thirteen of 36 state DOTs and the MPO  national organization expressed concern with the level of coordination  required to set and agree on targets. Three state DOTs commented on  the Planning rule that setting performance targets will be a significant  challenge for interstate MPOs that have membership in multiple states.  This is due to each state’s having different resources and different  transportation policies, goals, and priorities. According to FHWA,  commenters also said that it is not uncommon for states, MPOs, and  operators of public transportation to have different priorities that may  conflict with each other, potentially leading to conflicts when setting  performance targets and attempting to implement them. In response to  these comments, FHWA and FTA emphasize in the final Planning rule the  importance of state DOTs and MPOs engaging in early and ongoing  interagency coordination during their performance-based planning and  programming. Given that states and MPOs are just beginning the target- setting process for the Pavement and Bridge and System Performance  rules, the extent to which engaging in early and ongoing coordination  throughout the planning process will help resolve situations where states  and MPOs may have differing priorities driving their preferred  performance targets is not yet clear.
section_title: FHWA Took Steps to Address Challenges in the Final System Performance Rule, but Concerns Remain
paragraphs: In response to comments received on the proposed System Performance  rule, FHWA stated that it had taken steps to reduce the complexity of the  requirements, including eliminating, revising and simplifying measures,  and reducing the burden of compliance. FHWA eliminated some  measures and merged others in the final rule. For example, four of the six  measures in the final rule require the use of data derived from NPMRDS,  compared to seven in the proposed rule. FHWA also took steps to  simplify the required data processing and calculation of the metrics,  including changes to specific data requirements such as how travel time  intervals are calculated, which agency officials say effectively reduces the  amount of data analysis necessary. The agency also stated its  commitment to working with state DOTs and MPOs to establish a joint  effort to acquire services and tools that will help with activities such as  data processing, analysis, measure calculation and reporting.
According to transportation industry association representatives and  some states we met with, the changes made in the final System  Performance rule do not address most of the expected data challenges  associated with the process of combining data sources, the size of the  data, and states lack of experience with the data. Specifically, AASHTO  representatives told us that while the final rule made some technical  changes to specific data requirements, overall, significant data  challenges—related to both the size of the data set and the process of  combining the speed and volume data sets—remain. AASHTO  representatives also said that the new measures in the final rule are not  responsive at all to their comments and although FHWA made changes to  the rule—such as changing the weighting of the travel time measures  from system miles to person-miles traveled, which is an adjustment to the  measure itself—overall, the final rule is ultimately more complicated. After  the final System Performance rule was issued, officials from two state  DOTs we met with said that the final rule did not substantively change the  concerns identified in the original comments and that many of the  challenges in implementing the rules remain.
In response to these concerns, FHWA officials said that they made  significant alterations in the final rule to the measures in direct response  to the extensive and substantive comments received on the proposed  rule. The officials also stated that revisions made in the final rule along with new NPMRDS improvements, planned joint efforts with the states,  and technical assistance adequately address data concerns expressed by  the stakeholders. For example, FHWA officials stated that the changes  made reduce the size of the data set by approximately two-thirds. In  addition, after issuing the final System Performance rule, FHWA executed  a new contract in April 2017 containing updates to the NPMRDS data that  officials said would also help address anticipated data challenges.  According to agency officials, the new NPMRDS data will substantially  reduce the workload and complexity for state DOTs and MPOs. This new  data set will help simplify the initial step of combining the two data  sources necessary to begin calculating four of the System Performance  measures, reducing the technical burden on the states. FHWA officials  stated that data provided under this new contract will be available for  states’ use in July 2017. Recognizing that some states have limited  experience with the type of data analysis that will be necessary to  calculate the performance measures, officials stated that FHWA is  committed to providing technical assistance to help alleviate burdens  associated with states’ lack of experience with the data.
section_title: DOT Has Taken Steps to Guide the Transition to TPM but Does Not Have a Comprehensive Implementation Plan
paragraphs: To guide the transition to national transportation performance  management, FHWA has engaged in extensive outreach with states and  MPOs throughout the initial, rulemaking stage of TPM implementation  and continues to develop a range of tools, trainings, and workshops to  help prepare states and MPOs for key activities related to the second  stage—setting and tracking performance targets. The agency has also  begun to determine its internal roles and responsibilities for both the  target-setting and evaluation stages of the transition and has taken steps  to meet its congressional reporting requirements in the interim. However,  the agency has: not clearly defined its overarching goals for the transition, not developed an implementation plan showing how the various  efforts underway relate to each other and when they will be  completed, nor  clearly communicated its approach to states and MPOs.
Without these elements, states and MPOs may struggle to fulfill their  performance management responsibilities and FHWA may find it  challenging to effectively use its resources to assist states with  implementation challenges, to evaluate their performance, and ultimately  to use that information, as the final performance measure rules state, to  “communicate a national performance story” and assess the impacts of  federal funding.
subsections: section_title: DOT Continues to Develop Tools and Training for Implementation
paragraphs: As we found in 2015, during the initial rulemaking stage of TPM  implementation, FHWA engaged in extensive communication and  outreach with states and MPOs. FHWA is currently developing tools and  training to advance the second—setting- and tracking-performance  targets—and third—evaluating performance—stages of TPM  implementation. These efforts build upon the agency’s earlier  communication and outreach efforts. While the agency continues to use  the webinars, workshops, and trainings employed during the rulemaking  stage, DOT has changed the substantive focus of these efforts to address  the second implementation stage. For example, the focus has shifted to  helping states and MPOs understand and implement specific  performance measure requirements, data-management approaches, and  target-setting practices. FHWA’s efforts include the following:
Webinars: Beginning in 2012, FHWA used webinars to communicate  information related to the rulemaking process. More recently, the  agency has begun hosting webinars focused on data use and target  setting. For example, FHWA conducted a webinar in July 2016 on the  basics of target setting that defined key terms, explained the five  steps in the target-setting process, and provided a preview of a new  course on effective target setting. In addition, a target-setting webinar  on the final safety rule in September 2016 included an overview of  FHWA’s recommended process for coordination on safety targets.
Workshops: In June 2015, FHWA began facilitating peer-to-peer  workshops, including a 2016 data management session, a topic that  will play a critical role in TPM implementation as states use data to set  appropriate targets and track progress toward them. In addition,  FHWA has conducted workshops on safety target setting and  coordination and has begun to conduct workshops that focus on how  to implement technical aspects of the Pavement and Bridge and  System Performance rules now that they are in effect.
Web-based assessment tools: In December 2016, FHWA released a  new web resource and guidebook that state and local transportation  agencies can use to assess their transportation performance  management approach and identify steps they might take to  implement or improve that approach. The web resource includes a  self-assessment tool states and MPOs can use to determine their  level of maturity in TPM areas such as target setting, performance- based planning, and data usability and analysis. It also provides  additional resources states and MPOs can use to strengthen their  approach where needed in these areas.
Trainings: FHWA and the National Highway Institute are developing 8  training courses covering topics such as the target-setting process for  specific TPM elements, but many of these are new or were delayed  due to the regulatory review. For example, the “Steps to Effective  Target Setting for TPM” and the “Performance-based Programming  and Planning” trainings are newly available and being delivered in  summer 2017. According to FHWA, trainings pertaining to the rules  that took effect in May 2017—”TPM for Pavement” and “TPM for  Bridges”—will be available in the fall of 2017. As of July, 2017, 5 of  the 8 courses were available.
FHWA also has efforts under way to guide its internal TPM agency  activities. These efforts are designed to organize its staff, ensure the  agency is prepared to make its statutorily mandated report to Congress  on the status of TPM efforts by October 2017, and to lay the groundwork  for receiving and displaying information from states and MPOs when the  time comes for them to report their targets and for DOT to evaluate  states’ performance.
Developing internal guidance: FHWA issued internal guidance  describing the roles and levels of responsibility headquarters, division  office, and resource center staff will assume, including assuring data  quality and influencing target setting. This document outlined  preliminary responsibilities in February, 2015 and FHWA officials  acknowledged more work remains to be done to solidify these roles  as TPM evolves.
Preparing to meet mandatory reporting requirements: Despite delays  in the rulemaking process, FHWA has taken steps to assist states in  meeting their statutory reporting requirements. For example, to assist  states in reporting progress toward achieving targets to DOT by  October 1, 2016, as mandated, the agency issued guidance to states  on how to submit preliminary information to meet requirements in the  absence of the finalized Pavement and Bridge and System  Performance rules. According to FHWA officials, they have also  developed two web portals for states to use to report on progress for  the Highway Safety Improvement Program performance requirements  and other measure areas.
Planning to assess state readiness: FHWA also planned and  developed a draft survey that would both provide the agency with  information on the status of TPM practice across the states and  provide specific examples that would enable FHWA to meet its 2017  requirement to report to Congress. Officials said they also planned to  use the survey results to identify further assistance and guidance  needs. The officials also told us that they want to administer a similar  survey in 2 years to assess states’ ability to meet TPM requirements  and identify the major challenges that have arisen. While the survey  was initially planned for the end of 2016, it was delayed during the  review process and remained under review as of June 2017. Agency  officials were unsure when the survey would ultimately be  administered, but said they had identified alternate sources of  information to use to assess states’ progress in order to meet  congressional reporting requirements. For example, FHWA has been  collecting information from its division offices on how states and  MPOs have been working on TPM-related issues.
Planning web-based performance reporting tools: FHWA is also in the  preliminary stages of planning a web portal that states and MPOs can  use to submit their performance reports to the agency and a website  that the agency plans to use to publicly display state and MPO targets  and actual performance data. Both the Pavement and Bridge and  System Performance final rules state that the website will “likely  include infographics, tables, charts, and descriptions of the  performance data that state DOTs report to FHWA.” Agency officials  said the site will show both targets and actual performance numbers  and will eventually be expanded to a DOT-wide site incorporating  performance figures from other modes of transportation. The officials  said that they plan to share prototypes of the web portal and website  for states’ comments and review during fall 2017 and to have design  completed by October 2018.
section_title: DOT Has Not Developed a Comprehensive Plan to Guide Implementation Efforts
paragraphs: The new performance-based approach to transportation is a  transformational shift, impacting billions of dollars in federal transportation  funds, including almost $40 billion in FHWA grant funding to states and  holding states and grantees accountable for results, many for the first  time. In 2003, we identified key practices from major private- and public- sector organizational mergers, acquisitions, and transformations that  federal agencies could implement to successfully manage a  transformation. We found that several key practices can help federal  agencies implement needed transformational changes, such as  transforming to a more results-oriented culture. Some of these key  practices are applicable to FHWA’s approach to implementing TPM. In  particular, establishing a coherent mission and integrated strategic goals  that are clear to both agency employees and grantees from the outset is  critical to guide a successful transformation and involves determining  strategies and resources to effectively accomplish the goals of the  transformation. Developing, communicating and constantly reinforcing the  mission and strategic goals of the transformation gives employees a  sense of what the organization intends to accomplish and helps them  determine what they must do differently to help the organization achieve  success. In 2003, we also identified setting implementation goals and a  timeline as a key practice, given that transformations are lengthy  processes that can take years to complete, and noted the importance of  focusing on a set of principles and priorities at the outset of the  transformation. In addition, we noted that identifying the processes that  will be used to achieve results should focus attention clearly on critical  stages of the transformation and determine essential activities that must  be completed by certain dates. Furthermore, our past work on  organizational transformations found that early, effective, and ongoing  communication is essential to implementing a successful transformation,  as it engages employees and stakeholders and builds an understanding  of the purpose of planned changes. As principles of internal control in the  federal government, management should internally and externally  communicate the necessary quality information to achieve objectives.
While FHWA has developed tools and trainings to advance both the  rulemakings and implementation, the agency has not pulled all of this  together into a comprehensive plan to guide its efforts through the final  two stages of implementation. Specifically, DOT has not:
Articulated a mission and overarching goals to guide the transition:  While national goals for the transportation system were established in  MAP-21, FHWA has not coherently articulated the mission of TPM or  specific strategic goals related to each stage of the TPM transition in a  comprehensive manner. For example, the agency has not identified  strategic implementation goals describing what the transformation is  intended to accomplish and how the various efforts the agency has  underway will help accomplish those goals. Each of the final  performance measure rules states that TPM will provide FHWA the  ability to “better communicate a national performance story” and to  assess the impacts of federal funding. MAP-21 also provided that  performance based planning and programming will improve project  decision-making, thereby ensuring efficient investment of federal  funds. However, FHWA has not identified specific processes or  essential activities that will be used to achieve these results. For  example, the agency has not explained the process it will use to  analyze states’ performance data to assess the impacts of federal  funding on the surface transportation system. Furthermore, FHWA  has not described how the individual tools and trainings it has been  developing contribute to a comprehensive approach to achieving the  national goals and planning outcomes. If FHWA does not provide a  sense of what it intends to accomplish or determine what must be  done differently to help achieve successful TPM implementation,  states may not see how TPM is intended to help them improve their  transportation planning and decision-making processes.
Developed an implementation plan and timelines: FHWA has not  developed and made public a comprehensive plan describing specific  deliverables or actions the agency intends to take to address its  ultimate goals, identified how these planned activities will help move  the agency and its grantees through the two remaining stages of the  transition, or established timelines for when these activities will be  carried out. For example, FHWA did not make clear at the outset of  the transition how it would prepare grantees or focus on a key set of  principles and priorities for implementation, a key practice for  successful transformations. As a result, states have not had a plan  that lays out the various efforts that FHWA is taking and plans to take  to help address anticipated challenges. While FHWA has begun to  develop individual plans to outline implementation efforts for some  specific rules, the agency has not developed an overarching plan that  would describe the range of efforts across the six rules or explicitly  stated what actions it plans to take to facilitate the implementation of  these rules. For example, officials from one state DOT told us that  they were unsure how the transition would work in practical terms  because all the presentations FHWA provided concerning future plans  had been very general. While DOT will begin assessing states’  performance in the third implementation stage, the agency has not  described how it will evaluate performance nationwide. For example,  while FHWA officials told us they are developing a website to display  state targets and performance data, they have not outlined specific  plans for analyzing, displaying, or using the data in a way that will  provide practical insights into the condition of the nation’s surface  transportation system or better communicate a national performance  story. Furthermore, the agency has not developed a timeline  identifying when it will initiate and conclude specific activities, making  it difficult for state and MPOs to know when the tools, training, and  guidance FHWA is developing to help them meet the new  requirements will be available. For example, officials from one state  DOT told us they struggled to begin complying with new requirements  after rules were finalized because they had to wait months for specific  subject-matter guidance to be released.
Communicated its approach: FHWA has not communicated its overall  approach to TPM to states and MPOs—the grantees that are  responsible for carrying out several of the performance management  requirements. While the majority of state DOT officials we met with  were generally appreciative of the communication the agency  provided during the rulemaking stage and found the various webinars  to be valuable in conveying the status of agency efforts in the first  stage of implementation, the agency’s broader plans for full TPM  implementation were unclear. For example, officials we spoke with  from 9 out of the 10 states said they were not aware of FHWA’s plans  for TPM beyond the rulemaking process. As such, states and MPOs  may not have a sense of the purpose behind the significant changes  they will need to make to support this transformation of the surface  transportation system or know how to support the agency’s objectives  of communicating a national performance story and assessing the  impacts of federal funding. For example, officials from one state DOT  said they were not aware of any FHWA documentation that would  constitute a concrete plan for developing a framework that would  provide an in-depth look at the nation’s infrastructure.
FHWA officials stated they have not developed an implementation plan  with goals and timeframes because they have been focused on  completing the rulemakings. Agency officials stated that any efforts to  formalize an implementation plan would need to come after all the rules  took effect—which happened in May 2017—as well as completion of the  previously discussed survey effort. Agency officials also acknowledged  that an implementation plan would be helpful, but challenging to develop.  In the absence of a timely developed implementation plan, DOT may not  be ideally positioned to coordinate its technical assistance activities or  address implementation challenges anticipated by states—such as the  data-related challenges discussed earlier. An implementation plan could  also help the agency make the best use of the performance information  states will begin reporting to improve transportation decision-making,  communicate a national performance story, and assess the impacts of  federal funding. If grantees do not have a clear understanding of the  specific implementation goals and processes that their TPM efforts will  contribute to or how FHWA plans to use their performance information, it  may be difficult for them to support implementation.
section_title: Conclusions
paragraphs: As our past work has demonstrated, it is increasingly important to  improve the effectiveness of surface transportation programs by  establishing links to performance and measuring progress toward clear  national goals in order to maximize the use of available resources. TPM  represents a fundamental transformation in the way the agency will hold  grant recipients responsible for achieving congressionally established  goals and measure overall program performance. While the first stage of  implementation, the rulemaking process, took nearly 5 years to complete,  FHWA is now at a critical junction, preparing for the second and third  implementation stages—target setting and performance evaluation.  States and other grantees anticipate significant challenges associated  with implementation of these new requirements, including specific  challenges related to the complexity of the data and the processes  necessary to calculate and track certain performance measures. It  remains to be seen whether and to what extent revisions made to the final  rule and other actions by FHWA adequately addresses concerns  expressed by the stakeholders.
As the states begin the process necessary to agree upon and set  performance targets for each of the required TPM performance  measures, they could benefit from a plan outlining the agency’s  overarching approach to TPM and how the various actions the agency  has taken during the initial stage of implementation are related to one  another and will help prepare states for implementation. Without a formal  implementation plan guiding and coordinating FHWA’s efforts, FHWA  may struggle to articulate the goals and purpose of the transformation, tell  a national performance story, and identify and address the activities best  suited to help states and MPOs overcome anticipated implementation  challenges or identify future efforts to pursue to help achieve specific  TPM implementation goals. With the issuance of the interrelated final  rules and of states’ beginning the target-setting process, there is an  opportunity for FHWA at this critical junction to articulate an overarching  approach to TPM, including specific steps the agency plans to take to  operationalize the transformation, time frames for key activities and  efforts, and specific actions the agency will take to help states and other  grantees address anticipated challenges.
section_title: Recommendation for Executive Action
paragraphs: To better position DOT and FHWA to effectively guide the transformation  of federal surface transportation programs to a more performance- oriented approach, and to help states and MPOs overcome anticipated  challenges, we recommend that the Secretary of Transportation direct the  Administrator of FHWA to take the following two actions:  1.  Develop a formal TPM implementation plan to include overarching  implementation goals, specific actions FHWA plans to take to help  states and MPOs successfully implement TPM, and corresponding  timelines.  2.  Publicly communicate this plan and approach to build a shared  understanding of the goals and purpose of the transformation with its  grantees.
section_title: Agency Comments
paragraphs: We provided a draft of this report to DOT for review and comment. In its  comments, reproduced in appendix II, the agency concurred with our  recommendation. DOT also provided technical comments, which we  incorporated as appropriate.
We are sending copies of this report to the Secretary of Transportation  and the Administrator of the Federal Highway Administration. In addition,  the report will be available at no charge on the GAO website at  http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-2834 or flemings@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in appendix III.
section_title: Appendix I: Performance Management Rules and Performance Measures
paragraphs: System Performance Rule  National Goal  Congestion Reduction: To achieve a significant  reduction in congestion on the National  Highway System  System Reliability: To improve the efficiency of  the surface transportation system  Freight Movement and Economic Vitality: To  improve the National Highway Freight Network,  strengthen the ability of rural communities to  access national and international trade  markets, and support regional economic  development  Environmental Sustainability: To enhance the  performance of the transportation system while  protecting and enhancing the natural  environment.
Performance of the Interstate  System  Performance of the non- Interstate NHS  Freight movement on the  Interstate System  Traffic congestion  3.  Truck travel time reliability index  4.  Annual hours of peak-hour excessive delay  5.  Percent of non-single occupancy vehicle  6.  Total emission reductions  and Bridge Condition for the National Highway Performance Program (Pavement and Bridge); and  Assessing Performance of the National Highway System, Freight Movement on the Interstate  System, and Congestion Mitigation and Air Quality Improvement (System Performance). These rules  were created in response to the Moving Ahead for Progress in the 21st Century Act (MAP-21), which  required the Secretary of Transportation to establish measures states could use to assess their  performance in the 12 areas listed in this table. These performance measures are intended to create  a framework to support the seven national goals established by MAP-21. Part of the System  Performance final rule pertaining to the greenhouse gas measure was delayed indefinitely. 82 Fed.  Reg. 22879 (May 19, 2017). FHWA plans to publish another notice of proposed rulemaking regarding  the greenhouse gas measure at a later date.
section_title: Appendix II: Comments from the Department of Transportation
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the individual named above, Steve Cohen (Assistant  Director); Maria Wallace (Analyst in Charge); Delwen Jones; Alex  Lawrence; Crystal Wesco; Sarah Wilson; and Elizabeth Wood made key  contributions to this report.