Peer Exchange Workshop on the 

“Perfect World of Measuring 

Congestion” 

 

Workshop Summary Report 

Washington, D.C. 

December 17-18, 2013 

FHWA-HOP-14-009
i

TABLE OF CONTENTS

WORKSHOP OVERVIEW AND OBJECTIVES...............................................................................................1

PERFECT WORLD FOR MEASURING CONGESTION .................................................................................2

IMPLEMENTATION HURDLES AND NEEDS...............................................................................................3

Data Needs..............................................................................................................................................3

External Factors......................................................................................................................................4

Data Analysis Tools ...............................................................................................................................4

STEPPING STONES TO IDEAL STATE ........................................................................................................5

1. Decision-making Context..............................................................................................................5

2. Measure Definition and Calculation ...........................................................................................5

3. Customer Engagement..................................................................................................................6

4. Data Needs......................................................................................................................................6

5. Analysis Tools and Methods ........................................................................................................6

6. Facility Coverage............................................................................................................................7

7. Traveler Choices.............................................................................................................................7

PROJECT SUGGESTIONS .............................................................................................................................7

CONCLUSIONS AND NEXT STEPS..............................................................................................................8

APPENDIX A: WORKSHOP AGENDA .........................................................................................................9

APPENDIX B: WORKSHOP PARTICIPANTS..............................................................................................10

APPENDIX C: WHITE PAPERS...................................................................................................................11
1

WORKSHOP OVERVIEW AND OBJECTIVES

The Federal Highway Administration’s (FHWA’s) Operations Performance Measurement 

Program (http://www.ops.fhwa.dot.gov/perf_measurement) sponsored a Peer Exchange 

Workshop titled “the Perfect World of Measuring Congestion” held in Washington, D.C. on 

December 17-18, 2013. The focus of the workshop was to identify the “perfect world” of 

measuring congestion from an operations perspective, as well as possible research and 

implementation efforts that could be undertaken in the next five years by FHWA to get there.

The workshop agenda and a list of participants are included in Appendix A and B, respectively. 

The results of the workshop are being used to provide input to FHWA’s Operations Performance 

Measures and Management Program five-year Road Map. The Road Map will include research 

and implementation projects that FHWA could fund over the next five years as well as related 

projects that other groups (other FHWA programs, TRB, etc.) might want to undertake. 

In an effort to stimulate discussion at the workshop, four brief white papers were prepared and 

distributed prior to the workshop. These white papers are included in Appendix C.
2

PERFECT WORLD FOR MEASURING CONGESTION

The morning session of December 17th focused on attendee perspectives regarding what the 

perfect world of congestion measurement might look like. The Operations Performance

Management Capability Maturity Model (OPMCMM) (presented in white paper #1 (see 

Appendix C)) was used to facilitate the discussion.

Five prevalent themes emerged that workshop attendees would like to see in a “perfect world” of 

measuring congestion for operations: 

1. Flexible Decision-Making Framework: Public-agency transportation staff are often 

inundated with questions from varied stakeholders who have specific needs or decisions 

to make. The perfect world will have a flexible decision-making framework that is 

suitable for communicating to several audiences – travelers, operators, planners, 

and political/decision-makers. 

2. Automated Monitoring at all Geographic Levels: The perfect world will have 

monitoring everywhere for system needs, but allows for filtering out specific/local 

transportation project/program improvements. This continuous multimodal 

monitoring would provide benefit-cost information for specific projects (rural or urban), 

and allow automated report creation. 

3. Data Consistency: There is a need for consistency in data set formats within and across 

agencies. The perfect world will have consistency in data formats to improve 

understanding of the data. 

4. Measures and Methods Consensus: In some urban areas and states, there is a difference

of opinion on performance measures, thresholds (to define congestion), appropriate 

targets for the measures, calculation procedures and assumptions used. The perfect 

world will have consensus on the measures and calculation methods. 

5. Fusion of Mode-Specific Datasets: Thorough transportation decision-making requires 

mode-specific data. The perfect world will have a fusion and redundancy of modespecific data sets across roadway facilities for decision-making in a multimodal and 

multi-agency environment.
3

IMPLEMENTATION HURDLES AND NEEDS

After discussing the vision for the perfect world of congestion measurement for operations,

workshop attendees identified implementation hurdles and needs in the areas of data, external 

factors and data analysis tools. These needs are listed in the sections below.

DATA NEEDS

Workshop attendees identified the following data needs (“new” or “improvements on existing”):

 Travel characteristics (needed for all modes on roadways)

o Volume data

 Person-volume

 Density information

 Data by lane

o Latent/induced demand

o Real-time speed

o Capacity

 Information on causes of congestion

 Trip Information

o Multimodal information (transit, bicyclist, and pedestrian data) – and how to fuse 

modes together

o Origin-destination data 

 Improved spatial coverage

o Rural areas

o Arterials

 Data governance guidance

o Data business plan, data standardization

 Data use information

o How are users using the data

o What do they look at? How are they changing their decisions? 

 Accessibility/Livability data

 How to fuse crowd-sourced and sensor data
4

EXTERNAL FACTORS

Workshop attendees identified the following external factors that serve as hurdles to performance 

monitoring because they may lead to fluctuations in performance measure values. 

 Weather

 Economic factors

o Employment rates

o Population growth

o Gross state product (or gross metropolitan product)

 Demand (where is it now? – temporally and spatially)

 Accessibility/livability must be incorporated and considered

 Quantitative extent of tourism and visitors (e.g., from Chamber of Commerce)

 Military base volumes/impacts

External factors are further discussed in white paper #4 (see Appendix C).

DATA ANALYSIS TOOLS

Workshop attendees identified the following analysis tools or provided the following 

observations for overcoming analysis barriers for measuring congestion for operations:

 FHWA National Performance Management Research Data Set (NPMRDS) is valuable, but a 

national tool is needed to analyze the dataset

 The University of Maryland has developed many data analysis tools (see 

http://www.cattlab.umd.edu/?portfolio=ritis) for the I-95 Corridor Coalition

 Many DOTs not present at the workshop likely need help in telling their message 

 Tools must model/adjust/forecast and be more nimble
5

STEPPING STONES TO IDEAL STATE

Based on discussions throughout the workshop, seven primary themes emerged as “stepping 

stones” to get to the ideal state of congestion measurement for operations. These seven theme 

areas are: 

1) Decision-making context; 

2) Measure definition and calculation; 

3) Customer engagement; 

4) Data needs; 

5) Analysis tools and methods; 

6) Facility coverage; and 

7) Traveler choices. 

A brief discussion of each of these theme areas is included in the sections below. 

1. DECISION-MAKING CONTEXT

There is a need to better understand the context in which decisions are being made and the 

evolving nature of these decisions. In light of this information, there is a need for transportation 

professionals to understand their role in the decision-making process for varied audiences 

(travelers, operators, planners, policy makers), and to understand that their credibility is at stake 

when providing information to these audiences. 

There is a link between the operations decisions that can be made and the quality of the data 

upon which those decisions are being made. Because of this link, data availability and value is 

important. In some cases, there is a need for a culture change within transportation agencies to 

place a high value on quality data for decision-making. 

2. MEASURE DEFINITION AND CALCULATION

There is a need for consistency in how the industry defines, calculates and applies congestion 

measures for operations. Consistency – or at a minimum clear documentation – of measure 

calculation steps and methods used to identify congestion thresholds is important. Transparency 

in reliability measure calculation and application is even more critical because reliability 

measures are based upon distributions rather than averages. 

Related to measure definition and calculation is the need for consideration of multimodal 

elements in the performance measures and targets. Workshop attendees acknowledge the need 

for guidance on setting targets, while seeking flexibility in target-setting. Consideration of land 

use, accessibility and emissions impacts is important in multimodal measure definition and 

target-setting. 
6

3. CUSTOMER ENGAGEMENT

There is a need to understand the best practices and tools for communicating with customers –

particularly the four audiences previously mentioned (travelers, operators, planners, policy 

makers). Several workshop attendees expressed the importance of communication. There are 

different methods for communication to varied audiences, including panels, focus groups, 

smartphone apps, social media, etc. Likewise, the public audiences have spatially-differing 

community values and needs. It is important to anticipate needs of varied audiences and be 

proactive, while also managing expectations. 

4. DATA NEEDS

There was extensive discussion by workshop attendees about data needs and needs related to 

data integration to improve congestion measurement for operations. A list of specific data needs 

was generated by attendees (see prior section of this documentation). Integration of multiple 

sources of data to meet decision needs is often required. There is a need for reducing the 

“friction” of data exchange through common application programming interfaces (APIs), 

segment definitions, location referencing, etc. A better understanding is needed for best 

practices of data use for reporting external factors (e.g., induced demand, land use change, 

economic activity, employment or changing demographics) to provide context and correlation 

with performance activities.

Within public transportation agencies, incorporating data governance and disseminating data 

policies is important so all users understand the data and use limitations. Workshop attendees 

also discussed the possibility of public-private and public-public partnerships for data acquisition 

when mutual gains are possible. 

5. ANALYSIS TOOLS AND METHODS

There is a need for more guidance on performing before-and-after evaluations of operational 

treatments. More specifically, attendees were interested in better understanding when and where 

evaluations should be performed, the integrity of the underlying data sources, how to define the 

baseline for analysis, how to control for external factors, etc. Attendees discussed the concern 

with consistency in data sources (e.g., probe data) over time and how to handle that in beforeafter evaluations. There is a related need and interest in ultimately having analysis tools and 

methods that are integrated between the project-level, corridor-level and system-level. Attendees 

expressed interest in predictive analysis tools for planning and preliminary engineering purposes 

that incorporate external factors and can assist in target setting.
7

6. FACILITY COVERAGE

There is a need for better data on arterials and in rural areas to improve congestion measurement 

for operations. In the arterial environment, there is a pressing need for both better data, and also 

multimodal analysis methods. In the rural environment, there is a need for better corridor 

coverage where there is a focus on congestion issues. One specific difficulty is that the 

geographic length of traffic message channels (TMCs) used by private-sector speed providers are 

often many miles. Locations of localized rural congestion are therefore “washed-out” over these 

longer reporting segments. Rural coverage is also valuable because many rural corridors are 

critical for goods movement, tourism, and economic development. 

7. TRAVELER CHOICES

There is a need for information about how transportation system users are using and reacting to 

operational treatments such as traffic information and controls. Data are needed for all system 

users – travelers as well as freight shippers. As an example, understanding how travelers behave 

or react to traffic information and controls can help transportation professionals better understand 

what type(s) of traffic information messaging cause a particular type of mode shift. 

PROJECT SUGGESTIONS

The concluding activities on Day 2 of the Peer Exchange Workshop included developing project 

ideas to help get to the “perfect world” of congestion measurement for operations. The 

following project suggestions were identified during the discussion. 

 Compile terminology and develop glossary: Development of a guidebook of common 

terminology for operations performance measurement, including methods for calculating 

measures using standard inputs. 

 Guidance on the target setting process: Development of a guidebook including a synthesis 

and guidance at both the program and project level for target setting. 

 Identifying a process for performance-based decision-making: Development of a process for 

decision-making, which might include a predictive modeling tool, identifying the value of 

data in decision-making, and recognizing tradeoffs with limited resources. 

 Understanding best practices for customer engagement: Perform synthesis of best practices 

for engaging customers (e.g., panels, focus groups, social media, etc.) and include decisionmakers as customers. 

 Investigation and documentation of external factors that can impact operations performance 

measurement: Identify key attributes that should be tracked and reported and provide 

guidance on the process (extension of peer exchange background white paper #4). 

 Synthesis and guidance on non-traditional performance measures: Non-traditional measures 

are ones that capture land use, multimodal aspects, and environmental effects (e.g., 

Greenhouse Gases). 
8

 Technical Support for National Performance Management Research Data Set (NPMRDS): 

Includes user’s forums and data crosswalk and integration issues (particularly with fusing 

NPMRDS to existing data sets). Project could be established as a pooled fund study. 

 Identification of additional benefits and opportunities with private-company data: 

investigation of the potential for private-company data sources beyond just travel time and 

origin-destination data (e.g., individualized data from private providers for traveler choices). 

 Improved arterial data: Guidance is needed to identify available data and calculation 

procedures for performance measures in the arterial environment (particularly the nonfreeway National Highway System). 

 Development of data governance policies for public agencies: A guidance document that 

identifies best practices for managing and sharing data within and among public agencies. 

 Analysis tool for computing required performance statistics: A pooled fund for a “barebones” analysis tool to compute required performance statistics. 

 Guidance on before-after evaluations of operational treatments: Guidebook to perform 

effective before-after studies of operational projects and programs including integrity of data 

sources, defining baseline, controlling for external factors, etc.

 Best practices online library on performance monitoring and management: Development and 

support of a web “presence” where this information could be queried and accessed. 

 Value-added threshold: Better defining those facilities (using an average daily traffic [ADT] 

or ADT per lane threshold on arterials, for example) that are important for regional 

management and traveler information. 

CONCLUSIONS AND NEXT STEPS

The focus of the December 17-18, 2013 Peer Exchange Workshop was to identify the “perfect 

world” of measuring congestion from an operations perspective. The Peer Exchange Workshop 

highlighted several themes and corresponding needs that are critical to meet to advance 

performance monitoring and management for operations toward this “perfect world.” These 

themes and needs have been summarized in this document. The next steps will involve crafting 

these needs identified in the workshop into a Road Map for FHWA’s Operations Performance 

Measures Program. 
9

APPENDIX A: WORKSHOP AGENDA

Peer Exchange Workshop on the “Perfect World of Measuring Congestion” 

Federal Highway Administration, Office of Operations

December 17-18, 2013

USDOT Conference Room #6

1200 New Jersey Ave, SE, Washington, DC

Dec 17th AM

8:30a to 12p Session 1: Setting the Stage and Expectations

 Introductions

 Overview of FHWA project

 Workshop goal: provide input for FHWA 5-year research plan.

 Brief summary of white papers, focusing on capability maturity framework.

Session 2: Visioning/ Ideal State/Best Practices

 Open discussion by all participants.

 If time/resources/data were unconstrained, how would we structure 

performance monitoring and management programs?

 Address data, tools, external variables, etc.

Dec 17th PM

1p to 4:30p Session 3: Barriers, Hurdles, Unresolved Issues

 What are barriers to the “Ideal State” discussed before lunch?

 Don’t limit to big or small barriers, or easily solved barriers, just compile a 

listing of most important.

Session 4: Stepping Stones to the Ideal State

 Provide structure and priority to barrier removal.

 What barriers, when removed or mitigated, could provide the most costeffective advances in operations performance management practices?

Dec 18th AM

8:30a to 

11:30a

Session 5: Crafting a Research Plan

 Summarize yesterday’s discussion of barriers and hurdles, gather 

feedback/confirmation.

 Discuss logical research projects that address priority barriers.

 Identify related efforts and stakeholders.

Session 6: Closure and Next Steps

 Summarize research project concepts, gather feedback/confirmation.

 Distribution of workshop notes to participants.

 Discuss future avenues for involvement and peer exchange.
10

APPENDIX B: WORKSHOP PARTICIPANTS

No. Name Organization

1 Jesse Beurk Delaware Valley Regional Planning Commission

2 Daniela Bremmer Washington State DOT

3 Mara Campbell Missouri Department of Transportation

4 Michael Chamberlain Texas DOT

5 Mark Demidovich Georgia DOT

6 Rick Dowling Kittelson

7 Bill Eisele Texas A&M Transportation Institute

8 Mike Fontaine Virginia DOT/VCTIR

9 Brian Gardner Federal Highway Administration

10 Mark Hallenbeck University of Washington

11 Trish Hendren Washington Metropolitan Area Transit Authority

12 Brian Hoeft Las Vegas Regional Transportation Commission

13 William Hyman Transportation Research Board/SHRP 2 Reliability

14 Jim Kranig Minnesota DOT

15 Doug Laird Federal Highway Administration

16 Jeff Lindley Federal Highway Administration

17 Mena Lockwood Virginia DOT

18 Tim Lomax Texas A&M Transportation Institute

19 Rich Margiotta Cambridge Systematics

20 Doug McLeod Florida DOT

21 Andrew Meese Metro Washington Council of Governments

22 Harlan Miller Federal Highway Administration

23 Keith Nichols Hampton Roads Transp. Planning Organization

24 Michael Pack University of Maryland

25 Karl Petty Iteris

26 Joan Sollenberger Caltrans

27 Pete Stephanos Federal Highway Administration

28 Paul Szatkowski Virginia DOT

29 Rich Taylor Federal Highway Administration

30 Shawn Turner Texas A&M Transportation Institute

31 Stan Young University of Maryland
11

APPENDIX C: WHITE PAPERS

To stimulate discussion at the Workshop, four “white papers” on key performance monitoring 

and management topics were prepared and distributed in advance. These papers were not 

intended as an exhaustive treatment of the topics, but a discussion of important issues, question, 

and practices. The four white papers are as follows:

1. Framework for Gauging State-of-the-Practice in Performance Monitoring and 

Management. This paper presents a framework that can be used to classify best practices 

in performance monitoring. The framework is based on these categories:

a. Performance measures (content and form); 

b. Performance management (agency culture); 

c. Data used;

d. Transportation modes; 

e. Facility and trip coverage; and

f. Traveler preferences and tradeoffs consideration. 

2. Advancing Best Practices in Performance Monitoring: How and Where to Take the 

Next Steps? This paper described ways in which best practices in performance 

monitoring (in terms of data types, coverage, measures, etc.) could be further improved in 

the next five to ten years.

3. Connecting the Dots: How to Better Link Project-Level Performance Monitoring to 

Policy-Level Performance Management? This paper outlined important considerations 

for performance monitoring and management at varying levels of detail to meet different 

decision-making needs.

4. Operations Performance Management: How Should External Events and Trends 

be Considered? This paper identified the key external influences that may lead to 

fluctuations in performance measure values. These external influences include things like 

changes in travel demand, economy, development and land use patterns, housing cost and 

school quality, travel technology (e.g., connected vehicles or telecommuting), etc.
Paper #1: Framework for Gauging State-of-the-Practice Page | 1 

In Performance Monitoring and Management December 10, 2013 

Paper #1: Framework for Gauging State-of-the-Practice in Performance 

Monitoring and Management 

Prepared for: 

Peer Exchange Workshop on the “Perfect World of Measuring Congestion” 

FHWA Office of Operations 

Prepared by: 

Texas A&M Transportation Institute and Battelle 

FINAL 

December 10, 2013 

1. Introduction 

This background paper is paper number one (1) of four (4) developed in preparation for the 

Peer Exchange Workshop to be held on December 17-18, 2013 in Washington, D.C. The 

objective of this paper is to describe a framework to gauge the state-of-the-practice in 

performance monitoring. The framework includes examples, best practices, reporting and 

typical data sources. These examples discuss how current congestion measures are used to 

support transportation investment decisions including operational strategies. 

The paper begins by presenting a summary of the proposed framework developed by the Texas 

A&M Transportation Institute (TTI) for gauging current best practices. The framework was 

informed by observation of general industry practices related to performance monitoring. The 

remainder of the report supports these observations with additional details and examples. 

2. Highlights of Framework for Gauging State-of-the-Practice 

Observation of practices for performance monitoring and management provides evidence of a 

varying degree of the extent that performance data are used in decision-making for 

transportation projects and programs. Figure 1 introduces an Operations Performance 

Management Capability Maturity Model (OPMCMM) to identify characteristics of the 

performance management maturity continuum, which is evident in practice. The OPMCMM 

provides a method for grading agency practices on several characteristics important for 

performance management to be a seamless practice in a transportation organization. The six 

(6) categories are 1) performance measures (content and form), 2) performance management 

(agency culture), 3) data, 4) modes, 5) facility and trip coverage, and 6) traveler preferences 

and tradeoffs. The proposed framework illustrates four (4) “ribbon categories” of bronze, silver, 

gold and platinum to illustrate maturity level for the given characteristics. 

The capability maturity model illustrated in Figure 1 was inspired by the capability maturity 

model applied to systems operations and management described in SHRP 2 Report L06 

(Institutional Architectures to Improve Systems Operations and Management).1

 While the 

SHRP 2 Report L06 used four (4) levels, here “ribbon levels” are proposed for the maturity 

scale. The following are general highlights about the OPMCMM and observations from selected 

practices: 

                                                            

1 http://onlinepubs.trb.org/onlinepubs/shrp2/SHRP2_S2-L06-RR-1.pdf
Paper #1: Framework for Gauging State-of-the-Practice Page | 2 

In Performance Monitoring and Management December 10, 2013 

 In the OPMCMM, performance monitoring and management evolves to 

interactive/automated performance results, becomes data-driven using real-time or near 

real-time information, and has a performance connection between project-level and 

system-wide performance. 

 At the bronze level of the OPMCMM, data sets are “siloed” in agencies and become 

more transparent, accessible and connected with maturity. 

 The platinum level of the OPMCMM is characterized by complete modal information and 

understanding not only how you traveled, but how/where you really wanted to travel. 

 There are a number of external sources and transportation-related contributing factors to 

congestion that are not currently imbedded in typical performance practice (e.g., 

economy, societal factors [development patterns, housing prices], weather, incidents, 

work zones, connected/autonomous vehicles) – these external factors will be described 

in background white paper #4. 

The characteristics and examples suggested in Figure 1 are described further in the remainder 

of this paper. Background paper #2 will further describe how the industry can get to the “perfect 

world” (platinum level) identified in Figure 1.
Page | 3

December 10, 2013

In Performances Monitoring and Paper #1: Framework for Gauging State of the Practice

Management

Figure 1. Illustration of Proposed Operations Performance Management Capability Maturity Model for Gauging Current Practice

Perf. 

Measures

(Content):

Limited, project-specific

“after” study; snapshot of outcome 

measures (averages)

(e.g., GA TRIP evaluation, 

CA Traffic Signal Program)

Only required 

(“siloed”) reporting

Family of output and outcome

measures in some areas of

organization

Perf. 

Mgmt

(Culture):

Minimal perf

mgmt interest; performed 

only as required

Isolated champions of

perf mgmt; nothing coordinated

across any agency “silos”

Evidence of entire portions of agency

implementing and making decisions with 

perf measures; evidence of “datainformed” process to select projects 

(e.g., WA Gray Notebook)

Perf mgmt is ubiquitous in 

agency culture; entire agency speaks 

“perf mgmt language” – it feeds 

mgmt decision-making; “data-driven” decisions 

using real-time information

All “ribbon levels” typically use speed, volume and roadway inventory information at spatial and temporal scales of interest. As the industry matures to 

the “platinum level,” real-time performance management and decision-making are possible. 

All “ribbon levels” typically use travel time-based or delay performance measures, including total delay, delay per mile, travel time index, planning time 

index, etc. Measures are computed at the spatial and temporal scales of interest. 

Degree to which performance data

are used in decision-making

0%

100%

Modes:

Mode areas are 

“siloed” in agency; limited 

communication

Some ability to 

capture/estimate 

mode shifts within 

“silo” of interest

Ability to capture

mode shifts across 

entire agency

Ability to capture mode 

shifts over time and across

the agency, including

intermodal considerations

(freight and person)

Bronze

Silver

Gold

Platinum

Operations Performance Management Capability Maturity Model (OPMCMM)

Engineering judgment/

intuition/anecdotal

Network performance

reporting Data-informed Data-driven

Project-specific

performance

Feedback loop in 

decision-making

Data: “Siloed” datasets; 

no connections

Connected with other agency

datasets (volume, crashes, pavement, etc.); 

graphical user interface (GUI) to visualize/query 

data for decision-making

Dataset connections possible – requires 

programming/processing

Facility and

Trip 

Coverage:

Limited temporal/ 

spatial coverage 

in “silos”

Selected modes/facilities

and temporal coverage

(e.g., TX 100 Roadways, 

IN/MD Mobility Reports)

All modes, all facilities, 

all days,

all times covered

In addition to gold level, ability to 

capture diversion from traveler

info or control strategies

Limited ability to 

capture trip preferences

or revealed behavior

(e.g., new Google Maps®

preferences feature)

Some ability in selected 

agency “silos” to capture some 

trip preferences

Technological methods

identified/used to capture 

trip preferences or revealed behavior 

Agency ability to gauge or capture 

traveler preferences and revealed 

behavior (i.e., how you traveled and

How/where you really wanted to travel)

Traveler 

Preferences/

Tradeoffs:

(e.g., TTI UMR, FHWA’s UCR) (e.g., I-95 Corridor Coalition Future 

Performance Activities)

Full/seamless family of 

output/outcome measures across organization; 

real-time link (and decision-making) between 

project-level and system reporting; 

averages and reliability

Evidence of some coordination of datasets 

across traditional agency “silos”
Paper #1: Framework for Gauging State-of-the-Practice Page | 4 

In Performance Monitoring and Management December 10, 2013 

3. Current Practice in Measuring Congestion 

The ultimate goal of measuring congestion is improved transportation investment decisionmaking. Initially, performance measures provide a baseline of conditions, and, over time, allow 

for trend analyses of what types of transportation investments are working and where. These 

performance insights inform future investment decisions for transportation projects, programs 

and strategies. 

An evaluation of the current practice in measuring congestion provides evidence that there are 

varying degrees to which performance data are used in transportation investment decisionmaking. This varied nature of the profession is captured in Figure 1 (the OPMCMM). Current 

practice can be viewed on a scale of increasing performance management maturity, and from 

left to right in Figure 1, there is increased maturity in performance management. At the left end 

of the scale there is a minimal degree of performance data used in decision making, and this 

improves as the reader tracks their eyes to the right along the graphic at the top of Figure 1. 

For perspective, the graphic identifies four (4) “ribbon categories” (bronze, silver, gold, and 

platinum) from left to right as the degree of performance data are used in decision-making. 

Just below the graphic at the top of Figure 1 are short characterizations to describe the 

OPMCMM continuum. While there are four (4) ribbon categories, there are six (6) general 

short-description observations related to data in decision-making along the maturity continuum. 

At the far left where there is zero performance data used in decision-making, decisions are 

made with engineering judgment, are highly intuitive and/or based on anecdotal evidence. 

Progressing to the right are specific projects, characterized by a “snapshot” of performance for a 

project evaluation that is not really connected to system-wide performance activities. As one 

progresses along the maturity scale, network performance reporting is encountered. Moving 

further along the maturation scale, the next step includes a feedback loop in decision-making 

(i.e., an evaluation process is in place after decisions are made to evaluate the decision and the 

process). The next steps include data-informed decisions – not entirely data-driven, but data 

play a key role – and are then followed by data-driven decisions from performance data (in near 

real-time or real-time). 

This section of the paper describes the following characteristics of the performance 

management maturity scale highlighted in Figure 1: 

 Performance measures (content and form); 

 Performance management (agency culture); 

 Data used; 

 Transportation modes; 

 Facility and trip coverage; and 

 Traveler preferences and tradeoffs consideration. 

Selected examples of different characteristics and ribbon levels are also provided in the 

following section. 

 
Paper #1: Framework for Gauging State-of-the-Practice Page | 5 

In Performance Monitoring and Management December 10, 2013 

Performance Measures (Content and Form) at Each Ribbon Level 

All ribbon levels typically use travel time-based or delay performance measures (e.g., total 

delay, delay per mile, travel time index, planning time index, etc.). Measures are computed at 

the spatial and temporal scales of interest. 

 Bronze Level: Characterized by limited project and program evaluations. Typically these 

project evaluations are for a specific intersection analysis or segment of road where a 

project or program will be implemented (before study) or to evaluate the impacts after 

implementation (after study). The evaluation informs the public agency if the 

project/program was successful and whether improvements are needed. Typically it is 

just a snapshot of average (rather than reliability) outcome performance measures. 

 Silver Level: At this level only mandated or legislated (required) reporting is performed 

by the agency, and these activities are “siloed” into the divisions or groups of the agency 

required to report. 

 Gold Level: At this level measures captured include both public sector efforts (output) 

and performance results from the field (outcome) in selected areas of the organization 

(e.g., operations section of the agency reports on the number of motorist assistance 

patrols on the freeway system in a particular urban area [output measure] as well as 

average travel time information on the urban roadway system [outcome measure]). 

 Platinum Level: Full and seamless family of output and outcome measures used by the 

agency. There is a real-time link between project-level and system reporting that 

facilitates real-time (or near real-time decision-making). With continuous data readily 

available at this level, reliability measures can be easily produced in addition to average 

conditions. 

Performance Management (Agency Culture) at Each Ribbon Level 

Equally important to the measures themselves is the culture of the transportation agency in 

adapting performance management into their decision-making processes; therefore, the 

OPMCMM includes this characteristic. 

 Bronze Level: Minimal performance management interest in the agency. Performance 

measurement only performed when mandated or legislated as required. Performed to 

“check a box.” 

 Silver Level: Isolated champions emerge in the agency with strong interest in 

performance management. Any performance management activities still siloed. 

 Gold Level: Evidence of entire portions of the agency implementing and making 

decisions with performance measures. There is evidence of “data-informed” processes 

to select projects. 

 Platinum Level: Performance management is ubiquitous in agency culture. Entire 

agency speaks “performance management language” fluently, and it feeds management 

decision-making. Decisions are “data-driven” using real-time information. 
Paper #1: Framework for Gauging State-of-the-Practice Page | 6 

In Performance Monitoring and Management December 10, 2013 

Data Used 

All ribbon categories along the maturity scale typically use speed, volume and roadway 

inventory data at spatial and temporal scales of interest. As the industry matures to the 

“platinum level,” real-time performance management and decision-making are possible. 

 Bronze Level: “Siloed” datasets with no connections. Data are sought for the project 

evaluation or analysis at hand and are typically acquired from different groups within the 

agency. 

 Silver Level: Dataset connections are possible, but it requires programming/processing. 

 Gold Level: Similar to the silver level where dataset connections are possible, but it 

requires programming/processing. At the gold level there is evidence of some 

coordination of datasets across agency “silos.” 

 Platinum Level: At the platinum level of the maturity scale, all database types are 

connected (e.g., volume, crashes, pavement quality, etc.) and available through 

automated methods (e.g., relational database, GIS, etc.). 

Transportation Modes 

This characteristic of the OPMCMM relates to an agency’s ability to understand multimodal and 

intermodal trip characteristics. 

 Bronze Level: Mode knowledge and related data and information are “siloed” in the 

agency and there is limited communication across modes in the organization. 

 Silver Level: Some ability to capture or estimate mode shifts within agency “silo” of 

interest. 

 Gold Level: Ability to capture mode shifts across the entire agency. 

 Platinum Level: The ability to capture mode shifts over time and across the agency, 

including intermodal considerations (i.e., freight and person movement). 

Facility and Trip Coverage 

The geographic scope of travel is captured in this characteristic. 

 Bronze Level: Limited temporal and spatial coverage available for performance 

monitoring. 

 Silver Level: Selected modes and/or facilities and related temporal coverage. 

 Gold Level: All modes, all facilities, all days, and all times covered. 

 Platinum Level: In addition to all gold level characteristics, also ability to capture 

diversion from traveler information or control strategies. 
Paper #1: Framework for Gauging State-of-the-Practice Page | 7 

In Performance Monitoring and Management December 10, 2013 

Traveler Preferences and Tradeoffs 

The final characteristic identified in Figure 1 relates to an agency’s ability to capture traveler 

preferences and tradeoffs. 

 Bronze Level: Limited ability to capture trip preferences or revealed behavior. “Revealed 

behavior” relates to methods or technologies to provide insights or understanding about 

how and where particular trips are made from start to finish. 

 Silver Level: Some ability in selected agency “silos” to capture some trip preferences or 

revealed behavior. 

 Gold Level: Technological methods identified/used to capture trip preferences. 

 Platinum Level: An agency has the ability to gauge or capture traveler preferences and 

revealed behavior. This means an agency can identify how you traveled, and has 

information about how you really wanted to travel. This makes the important distinction 

that travelers must use the built system (and modes present), but may really want to 

take other routes/modes if they were available. 

The platinum level as defined here is admittedly far off; however, it is important to keep an eye 

on the fact that as transportation professionals better understand traveler trip preferences, this 

will help them develop the system for all users. 

4. Selected Examples of Different Characteristics and Ribbon Levels 

There are several examples provided in Figure 1 to help the reader identify where some 

selected current practice and activities might fall on the OPMCMM framework. These examples 

are only intended to start a discussion about how the OPMCMM can be used to grade different 

types of agency performance management activities. 

OPMCMM Characteristic: Performance Measures (Content and Form) – Bronze Level 

There are numerous examples of performance activities at the bronze level, characterized by 

project-specific studies. Below are a few transportation operations examples recently 

highlighted in FHWA’s 2012 Urban Congestion Trends Report.2

 These types of evaluations are 

project specific and provide a snapshot of performance. Table 1 includes examples of 

performance measures (content and form) at the bronze level. 

 

                                                            

2 http://ops.fhwa.dot.gov/publications/fhwahop13016/index.htm
Paper #1: Framework for Gauging State-of-the-Practice Page | 8 

In Performance Monitoring and Management December 10, 2013 

Table 1. Examples of Performance Measures (Content and Form) at the Bronze Level

(source: FHWA’s 2012 Urban Congestion Trends Report)

Towing and Recovery Incentive Program (TRIP) (Atlanta, Georgia) 

Description TRIP provides monetary incentives for timely clearance of crashes involving commercial vehicles. A 

recent Georgia Department of Transportation (GDOT) study evaluated TRIP, and found that it allowed 

roadway opening at least 2 hours and 45 minutes faster than in 2007 (before TRIP). Other key 

findings were that TRIP decreased average incident cost by 70 percent, saved over $9 million in 

delay, wasted fuel and emissions from inception to 2009, and results in a benefit of nearly $11 for 

each $1 spent. 

Data Modeling of “pre-TRIP incidents” using clearance times to estimate benefits if TRIP were in place. 

Measures Costs of delay, wasted fuel, emissions, benefit-to-cost ratio. 

Traffic Light Synchronization Program (California) 

Description Caltrans recently funded the Traffic Light Synchronization Program (TLSP) at $250 million from 

successful passage in November 2006 of California’s Proposition 1B (Transportation). Several 

projects have been implemented and have either met or exceeded estimated benefits. Two examples 

of benefits are in San Ramon, CA where ASCTs were installed on Crow Canyon Road and Bollinger 

Canyon Road. Both locations resulted in reductions in travel time, fuel consumption, emissions and 

collisions. 

Data Data sources include travel time from travel time runs, traffic volume measurement, and database of 

crashes. 

Measures Travel time, total delay, stopped time, congested time, fuel consumption, and emissions. Fuel 

consumption and emissions were estimated from SYNCHRO software. 

OPMCMM Characteristic: Performance Management (Agency Culture) – Gold Level 

Table 2 includes an example of performance management (agency culture) at the gold level of 

the Washington State DOT’s Gray Notebook.

Table 2. Examples of Performance Management (Agency Culture) at the Gold Level 

(Source: WSDOT’s Gray Notebook)3

WSDOT’s Gray Notebook 

Description The Washington DOT’s Gray Notebook (GNB) is the basis of WSDOT’s external performance 

reporting. It is recognized nationally for setting a high standard for accessible and accurate updates 

on programs and projects. The report is released quarterly with sections clearly related to the six 

statewide transportation policy goals of safety, preservation, mobility (congestion relief), environment, 

economic vitality and stewardship. 

Data and 

Measures 

The report uses a number of data sources and performance measures across the six policy goal 

areas. WSDOT’s Performance Dashboard (page vii of the GNB Edition 50) highlights key mobility 

measures as annual weekday vehicle-hours of delay, average clearance times for major incidents, 

percentage of ferries departing on time and percentage of Amtrak Cascades trips arriving on time. 

                                                            

3 http://www.wsdot.wa.gov/Accountability/default.htm
Paper #1: Framework for Gauging State-of-the-Practice Page | 9 

In Performance Monitoring and Management December 10, 2013 

OPMCMM Characteristic: Data – Silver Level 

Table 3 includes examples of the “data” characteristic at the silver level of TTI’s Urban Mobility 

Report and FHWA’s Urban Congestion Reports.

Table 3. Examples of Data at the Silver Level (Sources: TTI’s Urban Mobility Report, 

FHWA’s Urban Congestion Reports)4

TTI’s Urban Mobility Report 

Description The Urban Mobility Report reports congestion in all U.S. urban areas. In the 2012 Urban Mobility 

Report, trend data are available from 1982 to 2011. Detailed congestion statistics are provided for 

101 urban areas, and summary statistics are provided for the 498 urban areas throughout the U.S. 

Statistics are presented in tables by urban areas population size. The report is widely quoted on 

congestion and associated costs. 

Data The Urban Mobility Report is powered by huge datasets, including: 1) FHWA Highway Performance 

Monitoring System (HPMS) volume data, 2) INRIX 15-minute annual average speed data (850,000 

miles of road), 3) FHWA Freight Analysis Framework commodity value and tonnage for trucks. 

Measures Delay per auto commuter, total delay, travel time index, excess fuel per auto commuter (gallons), 

congestion cost per auto commuter (wasted time and fuel), truck congestion cost, truck delay, 

planning time index, carbon dioxide production, truck commodity value, commuter stress index, and 

total peak period travel time. 

FHWA’s Urban Congestion Reports

Description The Urban Congestion Report is produced quarterly to characterize emerging traffic congestion and 

reliability trends at the national and city level. The reports currently include 19 urban areas in the U.S. 

Data State DOT archived traffic operations data. Data for those roadways that are instrumented with traffic 

sensors for the purposes for real-time traffic management are included in the dataset. 

Measures Congested hours, travel time index, planning time index. 

 

                                                            

4 http://mobility.tamu.edu/ums, http://www.ops.fhwa.dot.gov/perf_measurement/ucr/index.htm
Paper #1: Framework for Gauging State-of-the-Practice Page | 10 

In Performance Monitoring and Management December 10, 2013 

OPMCMM Characteristic: Data – Gold Level 

Table 4 includes an example of the “data” characteristic at the gold level of a future performance 

activity with the I-95 Corridor Coalition. 

Table 4. Examples of Data at the Gold Level (Sources: I-95 Corridor Coalition Website)5

I-95 Corridor Coalition Future Performance Activities 

Description One example that incorporates some aspects of this level is the planned performance measurement 

activities of the I-95 Corridor Coalition. The Coalition’s website discusses the ongoing development of 

“a corridor-wide, web-based visual analytics monitoring system for identifying major bottlenecks, 

reporting travel time reliability and displaying other congestion measures using private-sector vehicle 

probe data fused with agency incident/event data where available. This system demonstrates how 

states can create a congestion monitoring program using a variety of data sources.” The system will 

allow the users to view both real-time and historical performance at various zoom levels 

Data Real-time INRIX speed data and agency incident/event data (where available) 

Measures Travel time index, travel time reliability, hours of congestion per mile and buffer index 

OPMCMM Characteristic: Facility and Trip Coverage – Silver Level 

Table 5 includes an example of the facility and trip coverage characteristic at the silver level 

including TxDOT’s 100 Most Congested Roadways List.

Table 5. Examples of Facility and Trip Coverage at the Silver Level (Sources: TxDOT’s 

100 Most Congested Roadways List)6

TxDOT’s 100 Most Congested Roadways List 

Description Since 2009, Texas DOT has sponsored TTI to produce a list of the most congested roadway sections 

in the state on the TxDOT website. The two agencies have developed an approach that combines 

annual speed archive data from private companies with basic roadway geometry, and traffic counts 

published in the TxDOT statewide roadway inventory file (RHiNo) to calculate congestion-related 

performance measures. The list is used for Texas DOT to program dollars to address the worst traffic 

locations across the State. 

Data Annual average traffic speeds from private-sector company and basic geometry and traffic volume 

from TxDOT roadway inventory. 

Measures Annual hours of delay per mile, annual hours of truck delay per mile, Texas congestion index (form of 

travel time index), planning time index, commuter stress index, annual congestion cost, and truck 

congestion cost. 

The Indiana Mobility Report7

 and the Maryland State Highway Mobility Report8 also represent 

examples of the facility and trip coverage characteristic at the silver level. Both reports use 

private-sector speed data to estimate mobility performance measures. Both reports include 

evaluations and mobility improvements due to specific projects in the state; therefore identifying 

the mobility benefits of specific transportation investments to further inform future decisionmaking. 

                                                            

5 http://www.i95coalition.org/i95/Default.aspx

6 http://www.txdot.gov/inside-txdot/projects/100-congested-roadways.html

7 http://docs.lib.purdue.edu/imr/4/

8 http://www.roads.maryland.gov/OPPEN/2013_Maryland__Mobility.pdf
Paper #1: Framework for Gauging State-of-the-Practice Page | 11 

In Performance Monitoring and Management December 10, 2013 

The 2013 Maryland State Highway Mobility Report goes a step further by integrating the speed 

data with volume information to compute delay and associated wasted time and fuel costs due 

to congestion (using the same methodology as TTI’s Urban Mobility Report). Maryland’s Report 

also includes the planning time index reliability measure. 

OPMCMM Characteristic: Traveler Preferences and Tradeoffs – Bronze Level 

Google Maps® has a new preferences feature that allows you to make maps of places that 

matter to you, allows you to save places to find them quickly later, and allows you to rate places 

you know to discover new places you might like. This is just an example of the types of Internet 

tools that can provide information about the types of places people want to go and perhaps how 

they choose to get there. Market tools and technologies will continue to evolve that can help 

agencies better understand traveler preferences and where travelers really want to go. 
Paper #2: Advancing Best Practices in Performance Monitoring: Page | 1 

How and Where to Take the Next Steps? December 10, 2013 

Paper #2: Advancing Best Practices in Performance Monitoring: 

How and Where to Take the Next Steps? 

Prepared for: 

Peer Exchange Workshop on the “Perfect World of Measuring Congestion” 

FHWA Office of Operations 

Prepared by: 

Texas A&M Transportation Institute and Battelle 

FINAL 

December 10, 2013 

1. Introduction 

This is one of four papers prepared for the Peer Workshop on Operations Performance 

Measures to be held on December 17-18, 2013 in Washington, D.C. The objective of this paper 

is to describe ways in which best practices in performance monitoring (in terms of data types, 

coverage, measures, etc.) could be further improved in the next 5 to 10 years. This paper is 

intended to stimulate discussion at the December Peer Workshop and is not intended as an 

exhaustive treatment of this topic area. In fact, workshop participants are encouraged to add 

their own ways to advance best practices in performance monitoring and management. 

This paper will use the same six categories as Paper #1, which are: 

1. Performance measures; 

2. Performance management (agency culture); 

3. Data; 

4. Modes; 

5. Trip and facility coverage; and 

6. Traveler preferences and tradeoffs. 

For each of these six categories, several speculative ideas are presented for discussion that 

could advance best practices within that particular category. 

2. Category 1: Performance Measures

This category refers to the types of performance measures, as well as the measures 

themselves. Also included in this section are calculation procedures and input parameters. 

 User-centric (trip-based) view of performance/congestion: Trip-based measures have 

been proposed as a way to better connect with individual experiences. Many traveler 

information interfaces have the ability to build and customize common travel routes. So in 

addition to facility- and system-based measures, would a “personal congestion calculator” 

on public agency performance reporting sites aid in building credibility with the general 

commuting public? With the increasing coverage of network-wide travel times, this could be 

fairly easy to program. But does it help with public comprehension and understanding? 

 Standardized or consensus definition of delay: Clearly defined user delay calculation 

procedures could help to ensure consistency among publicly reported performance 

statistics. For example, imagine the confusion that would result if each state climatology 
Paper #2: Advancing Best Practices in Performance Monitoring: Page | 2 

How and Where to Take the Next Steps? December 10, 2013 

department defined a different temperature scale. Minnesotans might define 10° F as 

freezing cold, whereas Floridians might define freezing cold at 50° F. Should user delay be 

treated as a physical property like temperature, and then targets be used to capture the 

desired goal or user perception? Even in our example, the only way that “freezing cold” is 

comparable between the two states is because of the consensus temperature measurement 

scale of Fahrenheit. 

 Standardizing measure calculation details: As with clearly defining delay, it would be 

helpful to clearly define data processing and calculation details to ensure consistency 

among reported performance measures. For example, it has been shown that time mean 

speed calculations provide different results than space mean speed calculations. It has also 

been shown that “snapshot” travel times provide different results than “simulated traversal” 

travel times. Similarly, the calculation of reliability measures uses percentiles from a travel 

time distribution—but what range of dates and times should be included in the distribution? 

 Market segmentation/differentiation of value of user delay: All transportation system 

user delay does not have the same value. Given that delay has been proposed as one of 

the measures for MAP-21 performance reporting, it would be helpful to have a better, more 

current understanding of how transportation system users value delay. Commuter and 

freight/truck delay costs are the most common differentiation. However, these are very 

broad categories that contain a very wide variety of users with even wider value judgments 

of delay. For example, some motorists view their morning commute as “wake up” time or 

their evening commute as “wind-down” time. With high-speed Internet connectivity, some 

transit riders extend their work hours into their commuting time, or extend their personal time 

(e.g., pleasure reading) into their commuting time. Even within the freight category, it has 

been hypothesized that delay may not have nearly as much value as travel time reliability. 

3. Category 2: Performance Management (Agency Culture)

This category refers to the agency implementation of performance measures, and how deeply 

integrated performance measures are in agency decision-making and other agency processes. 

 Professional capacity for data analytics: Big Data analytics is becoming more common in 

many different areas; transportation is no exception. Recent advances have got us squarely 

into the “big data” era in terms of the amount of data generated. However, some 

transportation departments lag behind in their ability to store, manipulate, process, and 

report on these Big Data sets for performance reporting. General “data analytics” training 

courses are available regardless of specific data application (e.g., through organizations like 

The Data Warehouse Institute). Ideally, however, a few training courses could be developed 

and delivered for transportation performance measure-specific data analytics. 

 Recognition of performance goals and targets: Performance goals and targets are 

understood and embraced by all employees. The performance goals are not something that 

is only published in an annual report or website, but something that permeates the culture 

and day-to-day decisions of the agency. 

 Individual or workgroup incentives for meeting targets: Meeting or exceeding 

performance targets could be incentivized (through bonuses, awards, etc.) at the employee, 

workgroup, or department level. Meeting performance targets could be prioritized more 

highly than adherence to a static, non-creative process. 
Paper #2: Advancing Best Practices in Performance Monitoring: Page | 3 

How and Where to Take the Next Steps? December 10, 2013 

 Culture of evaluation: Agencies should regularly conduct “before-after” evaluations that 

quantify the impacts of capital or operational improvements. These evaluation results should 

be incorporated into a feedback loop that better informs the policy, planning and 

programming process. This is the subject of Paper #3 in this series (titled “Connecting the 

Dots”). 

4. Category 3: Data

This category refers to data that is required for calculating performance measures or putting 

performance measures in the correct context. 

 Better system-wide data on flows and throughput: Based on private sector 

advancements in recent years, system-wide travel times and speeds are now available. 

Similar advancements are needed in flow data (i.e., traffic counts by vehicle class). Many 

public agencies have fixed-point sensors that monitor the most congested freeways, but less 

commonly on rural or free-flowing highways and arterial streets. Additionally, where fixedpoint sensors are installed, inadequate maintenance sometimes limits data availability. It 

may be that flow sensors for performance monitoring may only be needed between major 

interchanges or cross streets, a spacing that is wider than most current operations-based 

installations. 

 Readily available data on management actions: Historic/analytic data should be available 

for all transportation infrastructure management and control strategies. For example, all 

displayed signal phasing, ramp meter sequences, dynamic lane assignments, managed 

lane prices, traveler information, highway condition and road closure information, ozone 

alerts, etc. This “strategy parameters” data could be used to evaluate and calibrate 

management strategies. 

 Readily available data on external variables and influences: There are many influences 

on transportation demand outside of what many agencies currently consider (the subject of 

Paper #4 in this series). Ideally, these external variables and influences are identified and 

somehow incorporated into performance monitoring and target setting processes. 

5. Categories 4 and 5: Mode, Trip, and Facility Coverage

This category refers to the mode, trip, and facility coverage characteristics of performance 

measures and the supporting data. Many of the advancements in this category relate to 

gathering better individualized but anonymous data about person trips. With individualized but 

anonymous data, one can see an individual trip trace or origin-destination, but has no idea who 

made that trip. 

 Trip patterns: Trip-based measures are sometimes seen as more customer-centric than 

facility-based measures. Both are needed, but being able to identify actual trips, trip 

patterns, and trip experience would provide a better understanding of system performance. 

Actual trips may only be known about a select subset of all travelers who choose to opt in. 

 Habitual trip patterns: There is benefit to understanding habitual trip patterns for unique 

but anonymous travelers or goods. For example, for those travelers who choose to opt in, 

demographic and socioeconomic information could be known. For goods movement 

shippers who decide to opt in, basic commodity information could be known. However, “opt 
Paper #2: Advancing Best Practices in Performance Monitoring: Page | 4 

How and Where to Take the Next Steps? December 10, 2013 

in” processes could create sample bias that must be addressed in experimental design or 

data analysis. 

 Deviations from habitual trip patterns: Once habitual trip patterns are known for a 

selected subset of travelers and/or goods, then deviations from habitual patterns can be 

identified, as well as causes for these deviations. For example, did a phone call or text 

message from a family member trigger the deviation (e.g., run errand on way home from 

work)? Did a traveler information alert prompt a detour to a less congested route? Or was a 

managed/toll lane used on this commute because a mother was running late for the big 

soccer game on her calendar? 

 Capturing mode shifts: To accurately capture the effects of various demand management 

strategies, better information is required about mode shifts. For example, did an employerbased demand management strategy actually reduce motor vehicle trips or create a shift to 

more other modes? To take this one step further, one could also measure the trips not taken 

(i.e., telecommuting, Internet shopping, etc.). This capability would require knowing habitual 

trip patterns, as well as planned trip patterns and preferences (i.e., does this individual 

usually prefer to shop brick-and-mortar or online?). 

 Going beyond door-to-door trip measurement: In a perfect world, door-to-door trip 

information could be extended to an actual trip end inside a building, such as when a 

walking trip begins inside a large building or complex. This capability may be more prevalent 

in the future as mapping companies extend their reach (using WiFi signal recognition) inside 

buildings. 

 Inferring trip purpose: Trip purpose is known or can be inferred with high confidence from 

trip ends. Associations or “friendships” between companion travelers can be determined 

from similar trip ends and trip traces, thereby allowing one to determine carpooling or 

ridesharing, as well as companion use of public transit. Capturing long-distance travel is 

another possibility. 

6. Category 6: Traveler Preferences and Tradeoffs

This category refers to information about traveler preferences and tradeoffs that could be used 

to better understand trip-based information and characteristics. 

 More individualized data on travelers and their preferences: Private sector 

advancements in recent years have resulted in greater availability of system-wide travel 

times and speeds. However, some of the “richness” of this data has been lost in its 

aggregation among multiple users. Knowing more detailed yet anonymous traveler 

preferences (and perhaps demographics) could help our understanding of trip making and 

how we measure its performance. Some of this information is more important for 

transportation demand management, but the benefits of more individualized, trip-centric 

data could also flow into performance monitoring. The benefits of this anonymous yet 

individualized information have already been recognized within USDOT’s Data Capture and 

Management program (see Figures 1 and 2). 

It is not immediately clear how this measurement capability could be scaled to a national 

level to represent a significant sample of travelers. Most likely, though, it would require some 

type of incentive for travelers to voluntarily participate and provide their individual but 
Paper #2: Advancing Best Practices in Performance Monitoring: Page | 5 

How and Where to Take the Next Steps? December 10, 2013 

anonymous information and preferences. It could be similar to smart phone apps that are 

currently being used in limited research tests and pilot deployments, whereby financial 

incentives (e.g., coupons, free traveler info or services). Another option could be 

implementation within the mileage-based user fee process, whereby travelers or shippers 

who decide to opt in and provide certain types of individualized information receive a credit 

towards their mileage-based user fee. 

Figure 1. Data Sources Envisioned within US DOT Data Capture and Management 

Program, 

with Individualized Traveler Data Highlighted inside Red Ellipse

(Source: Real-Time Data Capture and Management Program Vision: Objectives, Core Concepts and 

Projected Outcomes, April 2010, 

http://www.its.dot.gov/data_capture/datacapture_management_vision1.htm) 

 
Paper #2: Advancing Best Practices in Performance Monitoring: Page | 6 

How and Where to Take the Next Steps? December 10, 2013 

Figure 2. Evolution of Data Environments in US DOT Data Capture and Management 

Program, 

with Individualized Traveler Data Highlighted inside Red Ellipse 

(Source: Real-Time Data Capture and Management Program Vision: Objectives, Core Concepts and 

Projected Outcomes, April 2010, 

http://www.its.dot.gov/data_capture/datacapture_management_vision1.htm) 

7. Conclusions

This paper identified several different ways in which practices could be further advanced within 

the areas of performance measurement and management. The possibilities offered in this paper 

are meant to stimulate discussion at the December Peer Workshop and are not considered to 

be an exhaustive list. In fact, as workshop participants are skimming this paper prior to the 

December Workshop, it is hoped that they will add any thoughts or ideas to the margins of this 

paper, and bring those ideas and possibilities for discussion at the Workshop. 
Paper #3: Connecting the Dots: How to Better Link Project-Level Page | 1 

Performance Monitoring to Policy-Level Performance Management? December 10, 2013

Paper #3: Connecting the Dots: How to Better Link Project-Level Performance 

Monitoring to Policy-Level Performance Management? 

Prepared for: 

Peer Exchange Workshop on the “Perfect World of Measuring Congestion” 

FHWA Office of Operations 

Prepared by: 

Texas A&M Transportation Institute and Battelle 

FINAL 

December 10, 2013 

1. Introduction 

This is one of four papers prepared for the Peer Workshop on Operations Performance 

Measures to be held on December 17-18, 2013 in Washington, D.C. The objective of this paper 

is to outline the important considerations for performance monitoring and management at 

varying levels of detail to meet different decision-making needs. This paper is intended to 

stimulate discussion at the December Peer Workshop and is not intended as an exhaustive 

treatment of this topic area. 

2. Continuum of Performance Management 

Performance measures can be used in a very wide range of transportation decisions, from 

making real-time traffic signal adjustments at a single intersection, to making multi-billion dollar 

transportation investment decisions over the next 20 years in a state of 38 million people. 

In some cases, performance measures are also used to determine the effectiveness of 

improvements (through before-and-after evaluations). For example, did the incident 

management program improve incident response and clearance times, and further, did it reduce 

congestion and improve reliability? If certain strategies are more effective than others, than 

those strategies are more likely to be deployed in the future. 

In many cases, performance measures are used to provide situational awareness. For example, 

is congestion getting better or worse? Which locations have the most congestion? What are the 

trends over time? 

In other cases, performance measures are used to guide transportation investments. For 

example, where are the most congested or least reliable highways, and therefore the highest 

return on highway investment? Performance measures can also be used for multimodal 

alternatives analysis and tradeoff. For example, what combination of land use policies, 

operations and management strategies, public transit, and highway investments will produce the 

most favorable performance outcome? 

It is clear that performance measures are used by many different audiences for many different 

types of decisions (see Figures 1 and 2). There may or may not be discrete boundaries between 

these different types of decisions; instead, a continuum exists. In some agencies, even the lines 

between “operations” and “planning” become less clear.
Paper #3: Connecting the Dots: How to Better Link Project-Level 

Performance Monitoring to Policy-Level Performance Management? 

Figure 1. Performance Measures Provide Answers to Questions at Several Levels 
Paper #3: Connecting the Dots: How to Better Link Project-Level 

Performance Monitoring to Policy-Level Performance Management? 

Figure 2. General Characterization of Performance Reporting Parameters 
Paper #3: Connecting the Dots: How to Better Link Project-Level Page | 4 

Performance Monitoring to Policy-Level Performance Management? December 10, 2013

Several questions arise when considering the use of performance measures in such a wide 

range of decisions: 

 How varied are the performance measures used for “microscopic” vs. “macroscopic” 

decisions? 

 If different performance measures are used at different decision levels, how can one 

ensure logical consistency between decisions made at “micro” and “macro” levels? In 

other words, do performance measures at the “micro” level “tell a different story” than 

performance measures at the “macro” level? 

 If the same or very similar performance measures are used throughout the different 

levels, can the same or very similar datasets be used for performance-based decisions 

at these different levels? 

 Is it necessary to measure performance at all these different levels? Can we just 

measure everything at the “micro” level? 

We will explore these questions and other issues in more detail at the Peer Workshop in midDecember. Workshop participants are encouraged to share their perspectives and experiences 

with performance-based decisions at their respective agencies. 

3. Illustrative Example 

Specific examples are usually best to help illustrate key concepts. This section includes two 

examples1

 that illustrate different ends of the spectrum in regards to performance measurement: 

1. A project-specific performance evaluation of I-465 in Indianapolis, Indiana. 

2. A statewide ranking of annual performance trends in Indiana. 

The first example illustrates the congestion reduction impacts of a specific transportation 

improvement called Accelerate I-465, a series of geometric design improvements and capacity 

additions along an 11-mile section of I-465. However, the more detailed nature of this 

performance assessment is equally applicable to before-and-after operational improvements. 

Figure 3 shows a color-coded speed diagram that visually indicates the congestion at several 

interchanges for all months in 2011, while Figure 4 shows the same speed diagram for all 

months in 2012. The congestion improvement from 2011 to 2012 is readily apparent, as the 

2012 diagram has fewer yellow blocks (indicating speeds of 45 to 54 mph) and more green 

blocks (indicating speeds of 55 to 64 mph). The congestion improvements are also quantified in 

terms of several quantitative performance measures; however, these “qualitative” illustrations 

(i.e., speed diagrams) are a helpful visual aid that provides time- and location-specific detail. 

                                                            

1 Both examples and all associated graphics are from the 2012 Indiana Mobility Report, available at

http://docs.lib.purdue.edu/imr/.  
Paper #3: Connecting the Dots: How to Better Link Project-Level 

Performance Monitoring to Policy-Level Performance Management? 

Figure 3. 2011 Speed Diagram for I-465 (During Construction) 

Source: 2012 Indiana Mobility Report, http://docs.lib.purdue.edu/imr/. 
Paper #3: Connecting the Dots: How to Better Link Project-Level 

Performance Monitoring to Policy-Level Performance Management? 

Figure 4. 2012 Speed Diagram for I-465 (After Construction) 

Source: 2012 Indiana Mobility Report, http://docs.lib.purdue.edu/imr/. 
Paper #3: Connecting the Dots: How to Better Link Project-Level Page | 7 

Performance Monitoring to Policy-Level Performance Management? December 10, 2013

Figure 5 illustrates the second example, which provides a statewide perspective on the most 

improved Interstate segments based on 2011-2012 changes in the travel time deficit. The 

section of I-465 that showed the significant improvement in Figures 3 and 4 is ranked as #16 in 

the Top 20 Most Improved segments across Indiana’s monitored roadway system. 

Figure 5. Top 20 Most Improved Performance (Based on Change in Travel Time Deficit, 

2011-2012) 

Source: 2012 Indiana Mobility Report, http://docs.lib.purdue.edu/imr/. 

Figure 6 provides a systemwide context for specific improvements along I-465. The leftmost 

chart shows distance-weighted congestion hours, and the I-465 congestion quantities are 

shown as the dark and light purple slivers (a small proportion of the overall congestion). 

Similarly, the rightmost chart shows total travel time deficit, and the I-465 congestion quantities 

are shown in dark and light purple. Figure 6 appears to be an effective way to “connect the dots” 

and make the link between project-specific benefits and system-wide performance. 

Figure 6. Specific Facility Improvements Shown in Statewide Context 

Source: 2012 Indiana Mobility Report, http://docs.lib.purdue.edu/imr/. 

 
Paper #3: Connecting the Dots: How to Better Link Project-Level Page | 8 

Performance Monitoring to Policy-Level Performance Management? December 10, 2013

4. Findings and Conclusions 

The previous section showed two illustrative examples of performance measurement that were 

at different ends of the spectrum in regards to level of detail. Figures 3 and 4 visually illustrated 

time- and location-specific congestion reduction impacts of a specific project on I-465. Figures 5 

and 6 provided a “big picture” view of system (i.e., statewide) performance, and showed the I465 project in this system-wide context. 

Figure 6 is a general characterization of performance reporting parameters. Only three levels 

are shown in this graphic for the sake of clarity. In practice, however, there is a continuum of 

level of detail and information requirements, and these may vary between different agencies 

depending upon its decision-making process. 

There are several other performance reporting efforts in the U.S. in which one could find similar 

examples that span a range of detail, from specific facilities/projects to system-wide. The best 

practices appear to have these characteristics: 

 Project-specific examples that clearly show the benefits of specific transportation 

improvements in easily-understood terms. These examples may be qualitative (e.g., 

visual) and/or quantitative. These project-specific examples are more detailed and are 

likely to help decision-makers relate to real-world examples. However, project-specific 

examples don’t provide the “big picture” in terms of overall system performance. 

 System-wide statistics are necessary to show the “big picture” view for higher-level 

decision-makers. System-wide trends over multiple years are also desirable, even if all 

of the change may not be fully attributable to specific transportation improvements. 

However, system-wide reporting is not ideal for showing specific problem areas or 

specific causes. 

 Showing specific improvements in the context of overall system changes (as 

shown in Figure 6) is important to logically connect specific projects to the overall system 

performance. By providing this context, decision-makers can see what impact specific 

projects have on the overall problem. 

 Using the same or logically similar performance measures at different levels of 

detail helps provide continuity and consistency between specific project impacts and 

overall system performance. For example, Figures 3 and 4 used speeds as a 

performance measure, while Figures 5 and 6 used travel time deficit. In this case, 

speeds and travel times are logically similar and provide continuity between different 

levels of reporting. 

 Ideally, one could use the same data for performance reporting at all levels of 

detail, from project specific to system-wide. Due to current limitations in data, this may 

not always be feasible in current practice. 

 Aggregate, system-wide reporting is more likely to be influenced by external 

variables (the subject of another white paper for this workshop) outside of public agency 

control. Conversely, project-specific evaluations are more likely to control for these 

external variables to isolate the impacts of the investment or strategy. 
Paper #4: Operations Performance Management: Page | 1 

How Should External Events and Trends be Considered? December 10, 2013

Paper #4: Operations Performance Management: How Should External Events 

and Trends be Considered? 

Prepared for: 

Peer Exchange Workshop on the “Perfect World of Measuring Congestion” 

FHWA Office of Operations 

Prepared by: 

Texas A&M Transportation Institute and Battelle 

FINAL 

December 10, 2013 

1. Introduction 

There are many independent variables (or external factors) for which state DOTs, metropolitan 

planning organizations and USDOT have little control over. These variables may affect 

performance measures or trends. Therefore, when using performance measures for reporting 

and/or decision-making, these variables must either be incorporated or their impact recognized 

in the performance of the transportation system and in the results of project evaluations. 

For many of these variables, the relative change may be more important than the actual value. 

For example, in an operations context, it may be more important to the evaluators to know the 

change in employment in a region and how that might have affected operations rather than 

knowing the base year number of workers. Adjustments in calculation procedures or 

communication can be made to incorporate some of these factors, but others can only be 

recognized and their possible effect communicated. 

This paper identifies the key external influences that may lead to fluctuations in performance 

measure values. It also includes proposed adjustments to alleviate these fluctuations. These 

variables will be discussed in more detail at the workshop. When developing the proposed 

adjustments, the following elements were considered: Can we generally accommodate the 

variable? Can we connect it to congestion and reliability measures? Can we use the information 

to explain why the measure is changing? Can we use the measures to make investments, policy 

changes or practice adjustments that will reduce congestion? What needs to happen to achieve 

this level of analysis? 

This paper is designed to help achieve the following two workshop objectives: 

 Design of a “platinum standard” monitoring program; and 

 Development of a practical FHWA research roadmap by the end of the workshop. 

In the text below, a recommendation accompanies each variable to describe how the 

phenomenon or issue should be accommodated. 

 Explanatory information – describes a variable that has an effect on transportation 

system performance, but cannot be included in the analytical construct of the measure. 

 Modifying factors – includes variables that can be accommodated with a change in the 

measure or the calculation procedures 

 
Paper #4: Operations Performance Management: Page | 2 

How Should External Events and Trends be Considered? December 10, 2013

2. The Big Picture Variables 

The independent variables included in this document may have an effect on performance 

measures. Specifically, the variables may impact the changes in performance measurement 

values between reporting periods. While these variables may not be reported in every case, 

they should be in the menu of questions that are asked. There may also be some contexts 

where the variables are more important: regional vs. corridor analyses, monthly vs. annual 

summaries, operating decisions vs. summary reports. 

 Economy –The 2008 recession once again illustrated the important role that nontransportation actions or occurrences play in congestion levels. Local recessions have 

caused drops in the congestion level in the past – see the big California cities in 2000 

with the dot.com bubble burst. Measures such as those below can be included as a 

regular component of operations reports to explain the role of changes in the economy – 

both good and bad: 

o Gross metropolitan product – the local version of gross domestic product 

o Total salary and wages – essentially the job-related portion of GMP 

o Employment - number of jobs 

o Unemployment rate 

o Population 

o Gas and diesel fuel prices 

 Recommendation: Include an economy measure as an overarching system 

explanation unless there is significant sub-regional variation in employment changes 

(e.g., large assembly plant or corporation ceases operations). 

 Societal – Decisions made to improve the quality of life or enhance economic 

opportunity often have an effect on system operations. These may be regional, corridor 

or neighborhood level changes and may be accompanied by changes in operation. As 

operating policies, practices and technologies are deployed, it is also important to 

capture other ‘outside the right-of-way’ events or phenomena that affect operations. This 

need not be a comprehensive investigation of the urban condition, and it may be 

reported only occasionally or displayed as an appendix/additional information element; 

some examples include the following. 

o Development Patterns – Density, magnitude and mix of land uses would be 

typical descriptors. Part of most long range metro plans is to concentrate more 

population and jobs into dense neighborhoods and centers and to move more 

people by transit and carpools. These changes are typically estimated to support 

more economic development and person travel for a given level of congestion. 

Vehicle ownership rates and vehicle use might be very useful to explain what is 

causing changes in several types of performance measures. 

o Housing Cost – Separate from, but related to, development patterns is the issue 

of housing cost. This could be ownership or rental cost – the key element is to 

capture the changes in cost that might tend to shift commuting and other travel 

patterns. 

o School Quality – Changes in school quality may cause changes in residential 

and job patterns that might result in vehicle travel changes. Much of the effect of 

this variable might be illustrated by changes in vehicle-miles of travel (see below) 

but the causation might be important. Suburban real estate agents reportedly (1) 
Paper #4: Operations Performance Management: Page | 3 

How Should External Events and Trends be Considered? December 10, 2013

identify school quality as a cause of home purchases, explaining some of the 

‘suburban sprawl’ that occurs despite long and unreliable commute times. 

o Generational Differences in Travel – Travel data from the first decade of the 

21st Century appear to indicate changes in the travel patterns of the younger end 

of the workforce. While some of this change is related to differences in 

employment numbers and rates, there also appears to be increased use of 

electronic means (such as telework or teleshop) for making trips and greater use 

of shared ride services among younger travelers. Additional descriptive 

information and alterations to trend calculation methods may be required if the 

‘millennial travel shift’ is an enduring part of transportation. 

o Connected Travel – In placing a value on extra travel time due to congestion or 

unreliability, the role of travelers being wirelessly connected should be 

considered. The value adjustment(s) may be addressed in other research, but 

the penalty to a traveler, especially one on public transit or in a shared-ride, may 

not be as significant as in the past. The fact that travelers can work during this 

time may allow them to accept worse travel conditions, with no decrease in 

quality of life or economic condition. 

 Recommendation: Examine the potential role of these variables annually and 

include an explanation of the possible role in changing congestion or reliability levels 

and in altering the trends or values of the key performance measures. 

3. Specific Variables 

These variables might have a direct effect on the travel speed dataset. An operator typically 

has relatively little ability to affect these but performance measures can be significantly affected 

by them. 

 Changes in Travel Demand – More or fewer travelers and/or vehicles can alter the 

context of several measures. In addition to the broad measure of total travel, the 

following sources might also be considered to better understand the effect that changes 

in vehicle-miles or person-miles of travel may have on congestion and reliability: 

o Latent demand is an associated issue; operations treatments can improve travel 

conditions and draw traffic volume from other routes, causing the improvement to 

appear to be less significant than if demand were constant. 

o Effect of TDM programs which may change the vehicle volume, trip departure 

times or travel route; or trips that were not made due to electronic substitution 

o Mode Share – If an agency is trying to get transit-oriented development and 

mode shift, mode share should be measured. 

 Recommendation: Examine the primary route and any routes that might be affected 

– typically parallel routes but if signal timing will be revised, significant crossing 

roadways should also be examined. 

 Person volume – If a particular approach or strategy will cause shifts to buses and 

carpools, person volume should be accommodated in the measure. 

 Recommendation: Include an estimate of person volume in all reports; collect data 

on person-volume to support estimates when usage changes (e.g., high-occupancy 

vehicle lanes). 
Paper #4: Operations Performance Management: Page | 4 

How Should External Events and Trends be Considered? December 10, 2013

 Parallel routes – Major streets or other freeways should be included in the analysis if 

they will be substantially affected by an improvement. If the effect is not substantial they 

should be included in an explanation. 

 Recommendation: Examine congestion and reliability of any routes that might be 

affected by a corridor improvement. 

 Weather, Incidents and Road Work - The location and timing of events that affect 

system performance (including incident clearance data) should be linked with congestion 

data to improve performance management. The base case should include descriptive 

information as explanatory variables.

 Recommendation: Initially, include explanatory variables of these events in any 

report -- this may be as simple as number of rain days or incidents. As the datasets 

for these elements are integrated, there may be an ability to parse the sources of 

congestion using a set of allocation rules. 

 Connected vehicles or autonomous vehicles – At some point these will begin to 

affect operating performance of the system and therefore the measures and the data. 

 Recommendation: Develop a description of the presence of infrastructure and 

vehicle attributes to characterize the reason for changes in operations performance 

measures.

 Modal Accessibility – The ease of use of transit, walking and bike modes is an 

important aspect of describing the role of single occupancy vehicle (SOV) alternatives. 

Ease of transit use, etc. 

 Recommendation: An explanatory measure such as number or percent of urban 

residents with access to a nearby transit stop may be a useful initial measure. Other 

elements such as sidewalks, bike lanes and paths, bikeshare, car sharing should 

also be described if they begin to play significant role in the region or corridor. 

4. Aggregation Level 

The more disaggregate the measures are, the more likely they are to capture effects of 

improvements. But they cannot be too disaggregate or too narrow or the “spillover effects” may 

be missed. For example, freeway improvements could improve freeway speeds and throughput, 

which could draw traffic from a parallel arterial, thereby also improving the arterial operations. 

 Recommendation: There are a number of guidance documents that can aid 

performance measurement professionals in developing a consistent dataset. The 

specific actions depend on the uses for the measures and users should expect that 

some changes will be necessary as measurement, management and investment 

decisions evolve. 

5. Customer Expectations and Target Setting 

Comparisons of urban, suburban and rural congestion are often a part of regional congestion 

discussions. At the statewide level, similar comparisons are made between large and small 

urban area congestion levels. These often are generated by differences in expectations and, to 
Paper #4: Operations Performance Management: Page | 5 

How Should External Events and Trends be Considered? December 10, 2013

date, all the technical ‘solutions’ have been found wanting in the public discussion of how the 

comparisons should be reflected in project or program investments. 

 

 Recommendation: The use of the data and measures will be key in the decision 

about this element. At a broad regional scale, the long range planning process 

should be used to investigate public opinion on expectations; the level of 

‘unacceptable congestion’ can be used to develop performance measure targets. 

For system reporting – such as a freeway operations report, a good first step may be 

choose some target level (for example, the speed at which the maximum volume 

occurs) and explain the use of that target. It appears that a map of acceptable 

congestion levels will be needed to identify where system improvements are needed, 

and where the community has decided to not aggressively attack congestion in lieu 

of other attributes. 

REFERENCES 

(1) Nonlinear Effects of School Quality on House Prices. Abbigail J. Chiodo, Rubén 

Hernández-Murillo, and Michael T. Owyang. Federal Reserve Bank of St. Louis Review, 

May/June 2010, 92(3), pp. 185-204. 
FHWA-HOP-14-009