Practical Guide for Quality 

Management of Pavement 

Condition Data Collection

U.S. Department of Transportation

Federal Highway Administration
Notice

This document is disseminated under the sponsorship of the U.S. Department of Transportation 

in the interest of information exchange. The U.S. Government assumes no liability for the use of 

the information contained in this document. This report does not constitute a standard, 

specification, or regulation.

The U.S. Government does not endorse products or manufacturers. Trademarks or 

manufacturers’ names appear in this report only because they are considered essential to the 

objective of the document.

Quality Assurance Statement

The Federal Highway Administration (FHWA) provides high-quality information to serve 

Government, industry, and the public in a manner that promotes public understanding. Standards 

and policies are used to ensure and maximize the quality, objectivity, utility, and integrity of its 

information. FHWA periodically reviews quality issues and adjusts its programs and processes 

to ensure continuous quality improvement.

ii

Technical Report Documentation Page

1. Report No. 2. Government Accession No. 3. Recipient’s Catalog No.

4. Title and Subtitle

Practical Guide for Quality Management of Pavement Condition Data Collection

5. Report Date

February 2013

6. Performing Organization Code

7. Authors

Linda M. Pierce, Ginger McGovern, and Kathryn A. Zimmerman

8. Performing Organization Report 

No.

9. Performing Organization Name and Address

Applied Pavement Technology, Inc.

115 W. Main Street, Suite 400 

Urbana, IL 61801

10. Work Unit No. (TRAIS)

11. Contract or Grant No.

DTFH61-07-D-00028

12. Sponsoring Agency Name and Address

U.S. Department of Transportation

Federal Highway Administration

1200 New Jersey Avenue, SE 

Washington, DC 20590

13. Type of Report and Period 

Covered

Final Document

October 2011 to February 2013

14. Spons oring Age ncy Code

15. Supplementary Notes

FHWA COTR: Thomas Van 

16. Abstract 

An effective pavement management system depends on reliable, accurate, and complete information. Having quality 

pavement management data is directly linked to the ability of the pavement management system to contribute to the 

development of reasonable and reliable recommendations and decisions regarding an agency’s pavement network. 

Pavement condition data are one of the key components of a pavement management system. Pavement condition data 

are used to model pavement performance, to trigger various actions ranging from maintenance to rehabilitation to 

reconstruction, to evaluate program effectiveness, and to satisfy many other purposes. While there are many different 

methodologies used for assessing pavement condition, ranging from manual surveys to fully automated procedures, the 

need for quality data remains the same.

Agencies take a number of steps to ensure and verify data quality, including calibration of the data collection equipment or 

the inspection teams, incorporating quality control sections that are reinspected to assess repeatability, and verifying 

reasonableness and completeness of the pavement condition survey. The ability to evaluate and determine the quality of 

pavement condition data is essential for establishing the accuracy and reliability of analyses made using pavement condition

The Federal Highway Administration (FHWA) sponsored the development of a Practical Guide on Quality Management 

Procedures for network-level pavement condition data. The Practical Guide provides information related to the 

development and implementation of a QM program, incorporating proven QM practices, and showcasing examples or case 

studies using pavement condition data from a variety of state DOTs.

Key Words

Pavement management, pavement condition assessment, 

quality management, quality control, quality assurance, 

quality acceptance. 

18. Distribution Statement

No restrictions. 

19. Security Classif. (of this report)

Unclassified

20. Security Classif. (of this page)

Unclassified

21.No of Pages

170

22. Price

N/A

Form DOT F 1700.7 ( 8 - 7 2 ) Reproduction of completed page authorized


Practical Guide for Quality Management of Pavement Condition Data Collection

v

TABLE OF CONTENTS

Executive Summary..................................................................................................................... ix

1. Introduction ........................................................................................................................... 1

Purpose................................................................................................................................... 1

Background............................................................................................................................ 1

Scope...................................................................................................................................... 2

Audience ................................................................................................................................ 3

Document Organization......................................................................................................... 3

How to Use This Guide ......................................................................................................... 4

Related Documents................................................................................................................ 5

2. Location Referencing System............................................................................................... 6

Introduction............................................................................................................................ 6

Importance of Location Referencing Systems....................................................................... 6

Characteristics of a Location Referencing System................................................................ 6

Types of Referencing Methods.............................................................................................. 7

3. Network-Level Condition Data Collection........................................................................ 11

Introduction.......................................................................................................................... 11

Data Collection Overview ................................................................................................... 11

Data Uses............................................................................................................................. 12

Survey Types and Technology ............................................................................................ 13

Data Items Collected............................................................................................................ 16

4. Principles of Data Quality Management........................................................................... 28

Introduction.......................................................................................................................... 28

Principles and Terminology................................................................................................. 28

Definitions ........................................................................................................................... 29

The Benefits of a Data Quality Management Program........................................................ 30

Data Quality Management Cycle......................................................................................... 31

Maintaining the Data Quality Management Process ........................................................... 31

Cost-Effectiveness of Data Quality Management Procedures............................................. 32

5. Data Quality Management Plan......................................................................................... 34

Introduction.......................................................................................................................... 34

Importance of a Data Quality Management Plan................................................................. 34

Quality Management Plan for Data Collection.................................................................... 35

6. Data Quality Standards and Acceptance Criteria............................................................ 39

Introduction.......................................................................................................................... 39

Resolution, Accuracy, and Repeatability............................................................................. 40

Reference Values................................................................................................................. 41

Data Variability.................................................................................................................... 43

Statistical Analysis............................................................................................................... 44

Acceptance Criteria.............................................................................................................. 44
Practical Guide for Quality Management of Pavement Condition Data Collection

vi

Corrective Actions............................................................................................................... 45

Other Quality Requirements................................................................................................ 46

7. Quality Control.................................................................................................................... 48

Introduction.......................................................................................................................... 48

Personnel Training............................................................................................................... 48

Equipment Calibration......................................................................................................... 50

Control, Verification, and Blind Site Testing...................................................................... 52

Distress Rating and Video Checks....................................................................................... 59

Data Processing, Handling, and Database Checks .............................................................. 60

Corrective Action................................................................................................................. 63

8. Acceptance ........................................................................................................................... 65

Introduction.......................................................................................................................... 65

Analysis of Control, Verification, and Blind Site Testing................................................... 65

Global Database Checks...................................................................................................... 65

Sampling Checks ................................................................................................................. 66

9. Quality Management Reporting ........................................................................................ 71

Introduction.......................................................................................................................... 71

QC Documentation and Reporting ...................................................................................... 71

Acceptance Documentation and Reporting ......................................................................... 72

10. Additional Quality Management Tools............................................................................. 74

Introduction.......................................................................................................................... 74

Automated Software Data Checks....................................................................................... 74

Geographic Information Systems ........................................................................................ 74

Time-Series Comparisons.................................................................................................... 74

Pilot Data Collection............................................................................................................ 76

Independent Verification ..................................................................................................... 76

11. Conclusions and Recommendations .................................................................................. 77

References.................................................................................................................................... 78

Glossary of Terms....................................................................................................................... 85

Appendix A. Quick Reference to Quality Management Plan................................................ 91

Appendix B. Data Quality Management Plan Template........................................................ 94

Appendix C. Case Study— British Columbia MoTI............................................................ 109

Appendix D. Case Study— Louisiana DOTD ....................................................................... 120

Appendix F. Case Study — Pennsylvania DOT.................................................................... 141

Appendix G. Quality Control and Acceptance Checklist..................................................... 154
Practical Guide for Quality Management of Pavement Condition Data Collection

vii

LIST OF FIGURES

Figure 1. Route-mile (km) point................................................................................................. 8

Figure 2. Route-reference post.................................................................................................... 8

Figure 3. Link-node. ................................................................................................................... 9

Figure 4. Route-street reference.................................................................................................. 9

Figure 5. High precision, low accuracy .................................................................................... 28

Figure 6. High accuracy, low precision .................................................................................... 28

Figure 7. Data QM cycle........................................................................................................... 32

Figure 8. Summary of QM activities. ....................................................................................... 37

Figure 9. Example pavement condition survey scope. ............................................................. 39

Figure 10. Use of quality control and acceptance processes by U.S. and provincial highway 

agencies..................................................................................................................... 49

Figure 11. Schematic of rod and level survey. ........................................................................... 56

Figure 12. Close-up of Dipstick® profiler......................................................................................

Figure 13. Example of Dipstick® profiler................................................................................... 56

Figure 14. SSI CS8800 walking profiler. ................................................................................... 57

Figure 15. SurPro 3500 walking profiler. ................................................................................... 57

Figure 16. Walking Profiler G2 .................................................................................................. 57

Figure 17. Control site data for pre-qualification of service providers. ..................................... 75

LIST OF TABLES

Table 1. Document organization. .............................................................................................. 3

Table 2. Common QM questions............................................................................................... 4

Table 3. Location referencing method key aspects. .................................................................. 7

Table 4. Network- and project-level data collection. .............................................................. 12

Table 5. Condition survey data collection and frequency. ...................................................... 13

Table 6. Summary of agency pavement condition data collection.......................................... 16

Table 7. Network-level surface deterioration by pavement type. ........................................... 17

Table 8. Examples of data resolution requirements for different protocols. ........................... 40

Table 9. Example of agency pavement condition data quality requirements.......................... 41

Table 10. Pennsylvania DOT batch data acceptance criteria. ................................................... 45

Table 11. Example of agency control site requirements. .......................................................... 54

Table 12. Example of agency testing of verification sites......................................................... 54

Table 13. Summary of agency distress rating checks................................................................ 59

Table 14. Summary of agency video checks. ............................................................................ 60

Table 15. Summary of agency data/database checks. ............................................................... 61

Table 16. Agency expected distress values. .............................................................................. 62
Practical Guide for Quality Management of Pavement Condition Data Collection

viii

LIST OF ACRONYMS

AASHO – American Association of State Highway Officials

AASHTO – American Association of State Highway and Transportation Officials

ASTM – American Society for Testing and Materials

DCV – Data collection vehicle

DMI – Distance measuring instrument

DOT – Department of Transportation

FHWA – Federal Highway Administration

FWD – Falling weight deflectometer

GIS – Geographic information system

GPR – Ground penetrating radar

GPS – Global positioning systems

HPMS – Highway Performance Monitoring System

INO – Institut National d’Optique

IRI – International Roughness Index

ISO – International Organization of Standards

ISTEA – Intermodal Surface Transportation Efficiency Act

KML – Keyhole markup language

LADAR – Laser radar

LIDAR – Light detection and ranging

LRM – Location referencing method

LRS – Location referencing system

LTPP – Long-Term Pavement Performance

MEPDG – Mechanistic-Empirical Pavement Design Guide

MLRS – Multi-level referencing system

MoTI – Ministry of Transportation

NCHRP – National Cooperative Highway Research Program

NHS – National Highway System

NIST – National Institute of Standards and Technology

PCI – Pavement Condition Index

PCR – Pavement Condition Rating

PMS – Pavement Management System

PQI – Pavement Quality Index

PSI – Present Serviceability Index

PSR – Present Serviceability Rating

PWL – Percent within limits

QC – Quality control

QM – Quality management

RWD – Rolling wheel deflectometer

TQM – Total quality management
Practical Guide for Quality Management of Pavement Condition Data Collection

ix

EXECUTIVE SUMMARY

Pavement condition data is a critical component of any pavement management system. 

Establishing a quality management (QM) plan for pavement condition data collection will aid in 

achieving reliable, accurate, and complete condition data and will address steps to take when

dealing with data quality issues. Without a documented plan, agencies are less likely to apply 

QM activities consistently from year to year and assess the effectiveness of the techniques used.

This guide outlines a process for systematically implementing QM practices throughout the 

pavement condition data collection effort. It describes the roles and responsibilities for 

successful QM of pavement condition data collection and presents examples of practices 

currently in use by transportation agencies.

Creating and maintaining an effective QM program for pavement condition data collection includes

specifying the data collection rating protocols to be used, establishing quality standards and 

acceptance criteria, identifying responsibilities, performing quality control (QC) activities,

monitoring and testing for acceptance, taking timely and appropriate corrective actions, and 

performing QM reporting. Each of these is discussed briefly below.

Data Collection Rating Protocols

The foundation of a successful QM plan is the definition of methods, standards, and protocols to 

be used in collecting pavement condition data. While agencies typically have well-defined 

procedures, there is variability between which data elements are included and what protocols are 

used. Pavement condition rating protocols/guides should clearly define the distress types, 

severity levels, rating methods (e.g., count, length, or area), reporting interval, and the method to 

be used to compute condition values. Failure to understand and communicate any of these can 

negatively impact the usefulness of the data that the agency receives.

Quality Standards and Acceptance Criteria

The QM plan establishes and documents the data quality requirements for all deliverables. The 

agency must specify realistic and attainable quality standards for each data item collected at the 

network-level. The specific measures that will be used to determine acceptable data quality 

should also be identified.

The data quality requirements in a QM plan typically define the level of resolution, accuracy, and 

repeatability for each data element. Resolution refers to the level of detail—specified in absolute 

terms—such as rut depth measured to the nearest inch (mm) or International Roughness Index (IRI)

measured to the nearest inch/mile (m/km). The resolution should be fine enough to track pavement 

condition deterioration adequately and support agency decisions, but must also reasonably reflect 

technological limitations for network-level data collection. Accuracy refers to the closeness of a 

measurement to an accepted ground truth or reference value. Requirements for accuracy can be 

specified in absolute values, percent, standard deviation, or other statistical measure. Repeatability 

refers to the comparison of repeated measurements of the same section under the same or similar 

conditions. Acceptance criteria define the allowed variability of the data for accuracy and 
Practical Guide for Quality Management of Pavement Condition Data Collection

x

repeatability and may also specify the percentage of data that must comply with the data quality 

standards.

Identification of Responsibilities

The QM plan outlines the various activities and who is responsible for that activity during data 

collection (this applies to data collected by both the agency and/or service provider). For 

example, the entity (agency division or service provider) collecting the data has the tools and 

resources to influence the quality of the data, so the QC activities should be under the purview of 

those collecting the data; alternatively, the agency pavement management division is in the best 

position to assess data acceptability since they are the ultimate owner of the data. The QM plan 

should identify the staffing, roles, and responsibilities for QC and acceptance, including 

reporting, documentation, and tracking/resolution of problems.

Quality Control

QC includes those activities performed to assess and adjust production processes to obtain the 

desired level of quality data (Flintsch and McGhee 2009). Common techniques used for QC

pavement condition data collection include equipment calibration and method acceptance;

personnel training; control and verification site testing; distress rating checks; and data reduction 

and processing checks.

• Equipment Calibration and Method Acceptance – A key feature of the QM plan is the 

requirement for equipment calibration and method acceptance. Testing equipment is

calibrated and testing methods and analysis are accepted prior to data collection and 

checked periodically thereafter to verify that the equipment is functioning according to 

expectations and that the collection and analysis methods are being followed. The 

agency may also have requirements for equipment and rater (i.e., person 

conducting/reviewing the pavement condition survey) certification.

• Personnel Training – Personnel training for pavement condition data collection, rating, 

and data reduction is an important QC element in the QM plan. Field crews must learn 

how to calibrate, operate, and troubleshoot complex equipment; raters must learn the 

proper protocols and pass competency tests; and data reduction personnel must learn how 

to process, compile, properly format, segment, and check the data for errors. Some 

agencies require a formal “certification” of the raters and equipment operators to verify 

that they have the required knowledge and skills.

• Control Site Testing – Control, verification, and/or blind site testing are critical QM 

activities that are performed prior to and periodically throughout data collection. Control 

sites are roadway segments whose pavement condition have typically been measured by 

the agency or a third party personnel for use as a reference value or “ground truth.” Data 

collected during the pavement condition survey are compared against the reference 

values to verify proper collection procedures and continued calibration of the equipment. 

In this way, control sites are used to assess the adequacy of the QC processes.

• Distress Rating Checks – The QC program for pavement condition data collection

typically includes random sample audits, inter-rater reproducibility, and data checks for 

accuracy and repeatability of the results. Random samples of the pavement condition 
Practical Guide for Quality Management of Pavement Condition Data Collection

xi

data may be selected and checked by the lead rater or QC personnel. If the pavement 

condition ratings do not meet quality standards, corrective action is taken, such as retraining of raters or retesting.

• Data Reduction and Processing Checks – Pavement condition data collection crews 

typically perform a series of data checks in the field; once submitted, the pavement 

condition data is reviewed in the office for accuracy and completeness. In the office, the 

pavement condition data and images are processed according to standard procedures and 

analyzed for quality issues.

After data collection and determination of condition ratings are complete, the final 

database is compiled and segmented according to the agency’s location referencing 

system (LRS). At this time, segment lengths are checked against the master routing file 

to look for any missing segments. Final checks of the database typically include 

verifying proper format, checking for missing data, and screening the entire database for 

errors. In addition, some agencies may also include time-series comparisons and 

geographic information system (GIS) checks.

Acceptance

Agencies use a variety of techniques to inspect the pavement condition data and assess its quality 

before acceptance. Control and/or verification site testing is often used before and during the 

pavement condition survey to assess and monitor the adequacy of the QC process. Global 

checks, sampling, and time-series comparisons are typically used to check the quality of the

delivered data. Typical global checks include inspecting for data that are out of expected ranges, 

missing segments or data elements, and statistical analysis to check for data inconsistencies. 

Other acceptance testing might include re-analyzing or resurveying a sample of the sections and 

GIS checks. The QM plan should establish the timeframe or recurring frequency for performing 

data acceptance checks.

Corrective Action

The QM plan should specify the corrective action to be taken if data are found not to meet the 

quality requirements. This may include equipment calibration, additional rater training, and recollection or rerating of pavement sections. It is important that the agency clearly define 

corrective actions prior to collection rather than waiting until a problem is discovered. For 

service provider contracts, the agency and service provider should discuss and agree upon the 

corrective action prior to conducting the pavement condition survey.

For those agencies that utilize service providers for conducting the condition survey, contract 

specification typically require that any discrepancies in data, condition assessment, and image 

quality be jointly investigated by the agency and service provider or addressed by the service 

provider, at no cost, and to the satisfaction of the agency.

Reporting

Documentation of the QC and acceptance procedures should be performed during each phase of 

data collection. Reporting is an important component of the overall QM program as it enables 

problem tracking and continuous improvement of the quality process. QM reporting also enables 
Practical Guide for Quality Management of Pavement Condition Data Collection

xii

the agency to refer back to previous reports, keep track of related problems, and take steps to 

prevent the same issues from recurring.

Summary

The above information is further described in the Guide. The guide also includes illustrations of 

several QM features related to pavement condition data collection through the use of six case 

studies based on implemented agency practices. In addition, the QM procedures of four agencies 

(British Columbia Ministry of Transportation and Infrastructure, Louisiana Department of 

Transportation and Development, and Oklahoma and Pennsylvania Department of 

Transportation) are summarized in the Appendices.

Having pavement condition data that accurately represents the condition of the pavement 

network will improve the agency’s ability to provide reasonable, timely, and reliable 

preservation and rehabilitation recommendations. As noted by the Virginia Department of 

Transportation, implementation of a QM plan can provide (Shekharan et al. 2006):

• Better compliance with external data requirements.

• Better credibility within the organization.

• Better integration with other internal agency data.

• Cost-savings from more appropriate treatment recommendations.

• Improved accuracy and consistency of data.

• Improved decision support for managers.

• Increased accuracy in reporting deficient pavements.

• Increased accuracy in reporting existing condition indices.

• Increased accuracy of budget need determinations.
Practical Guide for Quality Management of Pavement Condition Data Collection

1

1. INTRODUCTION

Purpose

The purpose of this Practical Guide for Quality Management of Pavement Condition Data 

Collection (QM Practical Guide) is to provide transportation agencies with the necessary tools, 

procedures, and practices for developing, using, and/or modifying a QM plan for network-level 

pavement condition data collection. This guide outlines a process for systematically 

implementing QM practices throughout the data collection effort. It describes the roles and 

responsibilities for successful QM of the data and presents the practices currently in use by 

transportation agencies.

Although several states have well-established and documented QM procedures in place, many 

others do not. For agencies with an existing QM plan, this guide will serve as a reference to 

check the completeness of their current plan. For agencies just beginning to develop or adopt 

QM procedures, this guide will help with the implementation of a comprehensive QM plan.

The QM program should be managed as a formal process to ensure the quality of pavement 

condition data. The QM program should be considered a “living” process, with periodic 

assessment and improvement over time. As staffing and equipment changes occur, changes in 

the QM program may be warranted. Effective implementation requires continuous assessment 

and adherence to the QM Plan. 

Background

Pavement behavior and performance is highly variable due to many factors, such as pavement 

structural design, climate, traffic, materials, subgrade, and construction quality. These factors 

contribute to changes in pavement performance that are reflected in the results of a pavement 

condition survey. Minimizing the impact of data variability on pavement condition data helps 

ensure that survey results reflect real changes in pavement performance rather than variations in 

data due to poor data quality.

Pavement condition data quality supports a wide variety of decisions and has direct and indirect 

impacts on agency processes. Some of the major uses of pavement condition data include:

• Characterizing current condition.

• Developing models of predicted pavement deterioration.

• Projecting future conditions.

• Developing treatment recommendations, timing, and cost.

• Preparing and prioritizing annual and multi-year work programs.

• Allocating resources between regions and/or assets.

• Analyzing the impacts of various budget and treatment scenarios.

• Analyzing performance of different pavement designs and/or materials.
Practical Guide for Quality Management of Pavement Condition Data Collection

2

Data variability has a cascading impact on pavement deterioration prediction, treatment timing, 

and resulting budgets. In order to make accurate predictions of pavement deterioration, Larson, 

Sami, and Luhr (2000) expressed a vision for pavement data quality in which “variability for 

each data element must be smaller than the year-to-year change in that element.” In practice, this 

level of accuracy has often proved to be difficult to achieve in network-level data collection. In 

some cases – such as newly-constructed concrete pavements – the data elements may change 

very little from one year to the next. In other cases, the technology and methods of data 

collection simply do not allow for a higher level of accuracy.

Data for pavement condition assessment may also be combined and converted into condition 

indices to describe current condition. The magnitude of the impact of data variability on 

characterization of the current condition depends on many factors; including the distress deduct 

values for index calculation and the manner in which quantities of distresses are determined. As 

an example, just a one percent difference in the area of low-severity fatigue cracking can make a 

12 point difference in the 100-point pavement condition index (PCI) calculation defined in 

ASTM D6433, Standard Practice for Roads and Parking Lots Pavement Condition Index 

Surveys. Such a large difference may result in a completely different treatment recommendation 

and have a significant impact on the associated cost.

Excessive data variability makes the accurate prediction of pavement deterioration difficult. 

Large data variability shows up as “noise” and can cause incorrect assumptions about pavement 

deterioration rates with resulting treatment recommendations that do not match field conditions.

Treatment rules are also sensitive to variability in distress type and severity and different 

treatments may be recommended because of data variability. Variability may result in a segment 

receiving a treatment earlier or later than is optimal, and when the analysis period covers a 10 to 

20 year time frame, the impact of variability becomes magnified. For example, if poor quality 

data predicts a segment to have an 18 year rather than a 25 year life expectancy, that type of 

inaccuracy can result in much higher projected budget needs. Thus, the quality of data collected 

can have a dramatic impact on the planning and programming decisions made by an agency.

It may also be beneficial for an agency to conduct a sensitivity analysis to gain a better 

understanding of the impact of data variability on its decision support system. A network-wide 

variability assessment of distress types and severities is recommended. Such an analysis 

explores and quantifies the impact of variability on indices, prediction models, treatment rules, 

and budgets. If certain distress types are found to be highly variable or the impact of variability 

great, the weight of these may need to be adjusted in index calculations.

Scope

The QM Practical Guide focuses on QM processes―including quality control (QC) and 

acceptance procedures―and the roles and associated responsibilities of both the agency and, 

when applicable, the service provider. It describes in detail the concepts and essential 

procedures of an effective QM plan and how they relate to the final quality of the data.

The information presented here covers a range of data collection survey types and should be 

tailored to support the needs and practices of a particular agency. Real agency examples are 

presented throughout the QM Practical Guide, a quick reference to QM plans is provided in 
Practical Guide for Quality Management of Pavement Condition Data Collection

3

Appendix A, a template for development of a QM plan is provided in Appendix B, and four 

agency QM procedures are summarized in Appendices C through F.

Audience

The QM Practical Guide is intended for highway and local transportation agencies responsible 

for network-level pavement condition data collection. It provides guidance to agencies that do 

not currently have a QM plan and those that can benefit from improvements to their existing QM 

process. The QM Practical Guide is also a handbook for anyone needing to know more about 

QM procedures for pavement condition data collection.

Document Organization

The QM Practical Guide is organized into the sections described in table 1.

Table 1. Document organization.

Chapter Title Description

1 Introduction Gives the purpose, background, scope, audience, and 

organization of the QM Practical Guide.

2 Location Referencing 

Systems Presents methods of geospatially locating the data.

3 Network-Level Data 

Collection Background

Presents an overview of the data collection process, the 

types of surveys conducted, data items collected, and 

rating protocols used.

4 Principles of Data Quality 

Management 

Presents the principles, definitions, and key concepts of 

data QM.

5

Development and 

Implementation of a Data 

Quality Management Plan

Presents an overview of the key steps to develop and 

implement a comprehensive QM plan.

6 Data Quality Standards and 

Acceptance Criteria

Describes the process used to establish data quality 

standards and acceptance criteria.

7 Quality Control Presents the key activities utilized for QC.

8 Acceptance Describes the procedures used for acceptance.

9 Quality Management 

Reporting

Describes the procedure for documenting all phases of 

the QM process.

10 Additional Quality 

Management Tools

Presents additional tools that can aid or automate the 

QM process.

11 Conclusions and 

Recommendations

Presents conclusion about the QM plan development 

process and provides recommendations for successful 

implementation of a QM plan.

Appendix A Quick Reference to QM Plan Provides a quick overview of the major procedures in a 

QM plan and the responsible party for each.

Appendix B Data Quality Management 

Plan Template Provides a template for the development of a QM plan.
Practical Guide for Quality Management of Pavement Condition Data Collection

4

Table 1. Document organization (continued).

Chapter Title Description

Appendix C Case Study―British 

Columbia

Summary of the QM procedures for the British 

Columbia Ministry of Transportation and Infrastructure 

(MoTI).

Appendix D Case Study―Louisiana

Summary of the QM procedures for the Louisiana 

Department of Transportation and Development 

(DOTD).

Appendix E Case Study―Oklahoma Summary of the QM procedures for the Oklahoma 

Department of Transportation (DOT).

Appendix F Case Study―Pennsylvania Summary of the QM procedures for the Pennsylvania 

DOT.

Appendix G Quality Control and 

Acceptance Checklist

Summary of the key features for quality control and 

acceptance of pavement condition data collection.

How to Use This Guide

The QM Practical Guide can be used by highway and local transportation agencies in the 

development and implementation of a data collection QM plan. Answers to the following 

common questions can be found in the chapters of the QM Practical Guide outlined in table 2.

Table 2. Common QM questions.

Question Location

1. What types of data collection surveys and technologies can be 

covered by a QM plan? Chapter 3

2. How is data quality managed? Chapter 4

3. What are the key features to include in a QM plan? Chapter 5

4. How do I define and measure data quality? Chapter 6

5. How accurate does my condition data need to be? Chapter 6

6. What steps can be taken before and during data collection to 

ensure data quality? Chapter 7

7. What steps can be taken after data collection to evaluate the 

quality of the data? Chapter 8

8. What are the basic procedures of a QM plan and who is 

responsible for each? Appendix A

9. What should a data QM plan look like? Appendix B

10. Have any agencies developed good QM plans? Appendices C-F
Practical Guide for Quality Management of Pavement Condition Data Collection

5

Related Documents

The two other major studies related to network-level pavement condition data collection 

performed under the National Cooperative Highway Research Program (NCHRP) include:

• NCHRP Synthesis of Highway Practice 334 – Automated Pavement Distress Collection 

Techniques (McGhee 2004).

• NCHRP Synthesis of Highway Practice 401 – Quality Management of Pavement 

Condition Data Collection (Flintsch and McGhee 2009).

Together with the American Association of State Highway and Transportation Officials

(AASHTO) and the American Society for Testing and Materials (ASTM) data collection 

protocols and standards, these documents provide background and guidance on QM of networklevel pavement data collection.
Practical Guide for Quality Management of Pavement Condition Data Collection

6

2. LOCATION REFERENCING SYSTEM

Introduction

Pavement management systems rely on data from a variety of sources (e.g., roadway inventory, 

traffic, materials, and construction). This data must be available and managed so that it can be 

readily accessed by decisionmakers at all levels (and by all divisions) of the transportation 

agency. The ability to obtain data related to specific roadway segments requires a location 

referencing system (LRS). An LRS is used for locating objects along a roadway and for 

referencing those objects to each other. In general, the LRS includes identification of a known 

point (e.g., mile or kilometer post), direction (e.g., increasing or decreasing), and distance (i.e., 

length and/or offset) (HTC 2002).

Importance of Location Referencing Systems

The LRS allows for the integration and visualization of multiple sources of information and data 

for a specific location and, as such, is an important part of any management system in areas with 

geographical diversity. For management of a pavement network, an LRS provides a means of 

linking specific roadway attributes and conditions to a location and can provide a visual display 

of the information and data for analysis and reporting. Given these critical functions, location 

references must be considered as part of a QM program to ensure that this information is 

properly considered in the analysis.

Characteristics of a Location Referencing System

Ten core functional requirements of LRS were identified from NCHRP Project 20-27(3),

Workshop on Functional Specifications for Multimodal, Multi-dimensional Transportation 

Location Referencing Systems. The following, in general, describes the core requirements of an

LRS (Adams, Koncz, and Vonderohe 2001):

1. Ability to locate, place, and position objects and events in three dimensions and time 

relative to the roadway network.

2. Accommodate a time reference to relate the database to the real world and provide the 

ability to transform the data among different time referencing methods. As a result, a 

known time, most commonly Greenwich Time, is associated with the data.

3. Allow data transformation among linear, nonlinear, and time referencing methods 

without loss of accuracy, precision, and resolution.

4. Support mapping capabilities.

5. Support the display and analysis of objects and events in multiple three-dimensional and 

time resolutions.

6. Support the navigation of objects, in near real-time and contingent on various criteria, 

along the transportation network.

7. Support regeneration of objects and network states over time and maintain the network 

event history.

8. Support association of error measures with space and time data at the object level.

9. Store and express object-level metadata to guide general data use.
Practical Guide for Quality Management of Pavement Condition Data Collection

7

10. Support time relationships among objects and events and support the time delay of events 

(i.e., the difference in time between scheduled events and actual events occurring at a 

particular location).

Types of Referencing Methods

The following sections provide a brief overview of location, spatial, and multi-level referencing 

methods.

Location Referencing Methods

Location referencing methods (LRM) include route-mile (km) point, route-reference post, linknode, and route-street reference, all of which are appropriate for managing data related to linear 

features such as a roadway network. The basic methods and key aspects of LRM used for 

roadway networks are shown in table 3.

Table 3. Location referencing method key aspects (TAC 1997, FHWA 2001).

Location Referencing 

Method Key Aspects

Route-mile (km) point

(see figure 1)

• Each route is assigned a unique name or value (e.g., Main Street, 

State Route 199).

• The beginning of the route is defined.

• Distance is measured from a given or known point to the 

referenced location.

• Route-mile (km) posts are not physically identified in the field.

Route-reference post

(see figure 2)

• Uses signs posted in the field to indicate known locations.

• Benefit over the route-mile (km) post is the elimination of 

problems associated with change in route length (e.g., due to 

realignment).

Link-node

(see figure 3)

• Specific physical features are identified as nodes (e.g., 

intersections, cross streets).

• Each node is assigned a unique identifier or number.

• Links are defined as the length between nodes.

Route-street reference

(see figure 4)

• Local streets are used to identify roadway features.

• Feature is recorded on one street at a specified distance and 

direction from another street.
Practical Guide for Quality Management of Pavement Condition Data Collection

8

Figure 1. Route-mile (km) point (FHWA 2001).

Figure 2. Route-reference post (FHWA 2001).
Practical Guide for Quality Management of Pavement Condition Data Collection

9

Figure 3. Link-node (FHWA 2001).

Figure 4. Route-street reference.

Spatial Referencing Methods

A spatial referencing method locates transportation features (or objects) using global positioning 

systems (GPS) to known locations. Coordinate systems use two or more spatial references (e.g., 
Practical Guide for Quality Management of Pavement Condition Data Collection

10

x, y, and z; latitude, longitude, and elevation; or State plane coordinates and elevation). Spatial 

reference methods are used within a GIS.

Multilevel Referencing Systems

Many agencies are moving to multilevel location referencing systems (MLRS) following the 

business model provided by Adams, Koncz, and Vonderohe (2001). An MLRS provides a base 

network capable of integrating information from multiple disparate LRS, such as county-routelog mile (km), street name-address, and/or intersection-offset systems. The MLRS provides a 

transformation mechanism that allows for a common linear description of a network that can 

relate all of the other supporting systems. This is extremely important given that in many 

agencies, systems have developed over time in different divisions for different purposes, and 

based on different LRS bases. As an example, the planning division may use one LRS for 

description of traffic data collection locations, while accident statistics are maintained on a 

completely different LRS by a different agency division. As agencies seek to view and manage 

assets and information across institutional “stove-pipes,” integration of existing systems into an

MLRS provides a better means of visualizing and managing features and data more efficiently.
Practical Guide for Quality Management of Pavement Condition Data Collection

11

3. NETWORK-LEVEL CONDITION DATA COLLECTION

Introduction

Pavement condition data is a critical component of a pavement management system. It is this 

data, collected consistently and over a period of time, that enables the characterization of current 

network condition, triggering of pavement preservation and rehabilitation treatments and/or 

strategies, and prediction of future conditions. Network condition data―combined with 

inventory, traffic, and cost data―allows a pavement management system to analyze and 

compare pavement sections to find the most cost-effective and beneficial combination of sections 

and treatments.

As the needs and uses of network-level condition data evolve, so has the technology to collect it. 

The following paragraphs discuss the evolution of this effort and the ensuing technology 

development.

Data Collection Overview

Under the 1991 Federal Transportation Authorizing legislation, the Intermodal Surface 

Transportation Efficiency Act (ISTEA) required all Federal-aid roads to be managed by 

pavement management systems. This requirement spurred the adoption of new technologies in 

order to collect the data needed for pavement management system network analyses. Although 

the requirement was later repealed, most State highway agencies continue to collect networklevel condition data and use pavement management system principles to manage their road 

network. In 2012, the Moving Ahead for Progress in the 21st Century Act (MAP-21) was 

enacted into law and provides over $105 billion for fiscal years 2013 and 2014 (FHWA 2012). 

MAP-21 creates a performance-based and multimodal program and establishes new requirements 

for setting performance targets for Interstate pavement (and bridges on the National Highway 

System [NHS]) condition as part of an Asset Management Plan.

Network-level pavement condition data are typically collected in large volumes and often, 

though not always, at highway speeds. The techniques that enable collection over a large 

network in a relatively short period of time use modern (and still evolving) technologies that 

automate much of the data acquisition and processing effort. Such technologies and procedures 

allow agencies to collect and report pavement condition data on a more frequent schedule and 

are typically more cost-effective than manual techniques. Many agencies collect sensor data 

(i.e., roughness, rut depth, and faulting via transverse and longitudinal profile) on an annual or 

bi-annual basis and distress data (i.e., fatigue cracking, longitudinal cracking, and patching) on a 

less frequent basis (McGhee 2004).

The pavement condition data items collected for network-level decisions differ somewhat from 

those used for project-level decisions. For example, International Roughness Index (IRI), rut 

depth, faulting, and surface distress are collected at the network-level by many agencies but 

structural capacity (which is not currently collected at high speeds) is collected primarily at the 

project-level (Flintsch and McGhee 2009). Both types of data support decisionmaking, but 

project-level data is often used to refine the network-level pavement management system
Practical Guide for Quality Management of Pavement Condition Data Collection

12

treatment recommendations. Table 4 further illustrates the details of data collection for projectand network-level roadways.

Table 4. Network- and project-level data collection (Flintsch and McGhee 2009).

Aspect Network-level Project-level

Uses

• Planning

• Programming

• Budgeting

• Pavement management system

treatment triggers, identification of 

candidate projects, life cycle cost 

analysis

• Network-level condition reporting

• Mechanistic-Empirical Pavement 

Design Guide (MEPDG)

calibration

• Project scope

• Refine pavement management 

system treatment recommendations

• MEPDG calibration

Data Items 

Typically 

Collected

• IRI

• Rut depth

• Faulting

• Cracking

• Punchouts

• Patching

• Joint condition

• Raveling

• Bleeding

• Surface texture

• Detailed crack mapping and other 

distresses

• Structural capacity (e.g., falling 

weight deflectometer [FWD])

• Joint load transfer

• Base/soils characterization (e.g., 

ground penetrating radar, cores, 

trenches)

Other Items 

Collected 

Concurrently

• Video

• GPS coordinates

• Geometrics (e.g., curve, grade, 

elevation, cross slope)

• Other assets (e.g., bridges, signals)

• Events (e.g., construction zones, 

railroad crossings)

• Drainage conditions

• Appurtenances (e.g., sign and 

guardrail location and condition)

• Geometrics (e.g., curve, grade, 

elevation, cross slope)

Speed • Typically highway speeds • Walking or slower speeds

Data Uses

Traditionally, network-level pavement condition data has primarily been collected for use in an 

agency’s pavement management decision process. If an agency does not have a formal 

pavement management system, the data is still used to support planning and programming of 

pavement preservation, rehabilitation, and reconstruction activities. Within a pavement 

management system, the data are used to determine current network conditions, predict future 

conditions based on various budget scenarios, and recommend a range of possible treatments for 

each segment of roadway over an analysis period. Because the recommended treatments are 
Practical Guide for Quality Management of Pavement Condition Data Collection

13

based on network-level data―as opposed to a more detailed, project-level investigation―the 

pavement management system recommendations are further refined with project-level data to 

determine the true project scope. This distinction between network- and project-level data is 

important in discussions of pavement condition data quality.

Increasingly, network-level pavement condition data are being used for more than pavement 

management systems analyses or treatment decisions. Other common uses for network-level 

pavement condition data include Highway Performance Monitoring System (HPMS) reporting, 

asset management, and calibration of the MEPDG. It should be noted that each of these uses 

may have differing requirements for data quality.

Survey Types and Technology

While early efforts at data collection typically involved manual surveys, advancements in 

computing technology and data storage have enabled more efficient collection and processing of 

network-level condition data. As a result, methods and frequencies for data collection have 

developed over time to take advantage of these capabilities.

Survey Frequency

Transportation agencies collect network-level data using a variety of methods and monitoring 

frequencies. Table 5 provides examples of the data types collected and frequency of collection 

for various highway agencies.

Table 5. Condition survey data collection and frequency.

Agency Condition Data Collected Frequency

British Columbia MoTI Surface distress, rut depth, and IRI

Primary system every 2 years; 

secondary system every 2 to 4

years; and selected side roads 

every 4 years

Colorado DOT Cracking, rut depth, and IRI Annually

Florida DOT Surface distress, faulting, rut depth, 

and IRI Annually

Idaho DOT Surface distress, rut depth, and IRI Annually

Indiana DOT Surface distress, rut depth, and IRI Annually

Iowa DOT Cracking, rut depth, faulting, Dcracking, joints spalling, and IRI Every 2 years

Kentucky Transportation 

Cabinet

Surface distress, faulting, rut depth, 

and IRI Annually

Louisiana DOTD Cracking, patching, faulting, rut 

depth, and IRI Annually

Long-Term Pavement 

Performance (LTPP)

Surface distress, faulting, rut depth, 

and longitudinal profile Every 2 years
Practical Guide for Quality Management of Pavement Condition Data Collection

14

Table 5. Condition survey data collection and frequency. (continued).

Agency Condition Data Collected Frequency

Maryland State Highway 

Administration (SHA) Cracking, rut depth, and IRI Annually

Nebraska Department of 

Roads (DOR)

Surface distress, faulting, rut depth, 

and IRI Annually

New Mexico DOT Surface distress and faulting Annually

North Carolina DOT Surface distress, faulting, rut depth, 

and IRI

Annually on interstate and 

primary roads

Oklahoma DOT Surface distress, faulting, rut depth, 

and IRI

NHS every year and nonNHS every 2 years

Oregon DOT Surface distress, faulting, rut depth, 

and IRI Annually

Pennsylvania DOT Surface distress, faulting, rut depth, 

and IRI Annually

Virginia DOT Surface distress, rut depth, and IRI Annually

Washington DOT Surface distress, faulting, rut depth, 

and IRI Annually

In addition, as of 2010, the Federal Highway Administration (FHWA) requires that IRI be 

collected annually on roads comprising the NHS, which includes interstates, while the non-NHS 

routes may still be collected on a 2-year cycle (FHWA 2010). It is expected that this change in 

the required reporting cycle has influenced the frequency with which State highway agencies 

collect pavement condition data. In addition, there are new pavement condition data items that 

are required for the HPMS submittal, including rut depth, faulting, and cracking data. For 

additional details on HPMS data collection and reporting, see the HPMS Field Manual (FHWA 

2010).

Manual, Semi-Automated, and Automated Surveys

Data collection technology is one of the most rapidly evolving areas of pavement management. 

The development and application of ultrasonic, infrared, laser sensors, and high-speed computer 

processing have contributed greatly to the ability of transportation agencies to collect large 

volumes of pavement condition data quickly and efficiently. More recently, line and area scan 

digital video cameras have facilitated fully or semi-automated crack detection. The following 

briefly describes the primary methods for collecting pavement condition data.

• Manual surveys – Manual surveys are conducted by walking or traveling at slow speed 

and noting the existing surface distress. Manual surveys may be limited to selected 

segments or span the entire roadway length. Distresses are generally recorded on paper, 

but there is an increasing trend to enter the survey results directly into computers or handheld devices. Rut depth and/or faulting are typically estimated by taking manual spot 

measurements.
Practical Guide for Quality Management of Pavement Condition Data Collection

15

• Automated surveys – Automated surveys typically incorporate the use of vans fitted 

with equipment (e.g., lasers, high-speed cameras, and computers) specifically designed 

for collecting pavement and roadway features. Digital images of the transverse and 

longitudinal profiles of the roadway surface are captured at highway speeds for use in 

assessing pavement condition. Data and images collected through automated surveys 

require processing using either fully or semi-automated methods.

- Semi-automated – For semi-automated processing, the resulting images are 

viewed at workstations by personnel trained to rate visible cracks and other 

distresses. Proprietary software packages are used for displaying the images and 

recording distresses. Sensor data are processed for determining rut depth, IRI, 

and faulting.

- Fully automated – Fully automated processing includes using the collected 

images and pattern recognition technology for automatically (i.e., no user 

interference) detecting distress. A number of service or equipment providers have 

developed or are developing systems that use video and/or laser technology to 

detect and classify pavement cracking in real-time at highway speeds. Other 

systems capture the pavement images first and use automated post-processing to 

detect and classify cracks. As with semi-automated processing, the sensor data is 

used to determine rut depth, IRI, and faulting.

While fully automated and semi-automated technologies have gained wide acceptance in 

pavement condition data collection, manual (including walking and windshield) surveys are still 

used by many highway agencies in the United States and the Canadian provinces as well as local 

agencies. Based on a survey conducted by McGhee (2004) and updated by other sources 

(FHWA 2008; Zimmerman and McKinney 2011; Zimmerman and McKinney 2012), 44 of 65

transportation agencies (50 State highway agencies, the LTPP Program, Eastern Federal Lands, 

Puerto Rico, District of Columbia, and 11 Canadian provinces) collect pavement condition data 

using automated pavement condition data collection vehicles, while 21 agencies conduct a 

windshield-based survey (table 6). In addition, of the 44 agencies that collect automated 

pavement condition data, 14 agencies report that the data is processed using fully automated 

methods, while the remaining 30 agencies conduct semi-automated analysis. It should be noted 

that the majority of agencies collect profile data for determining IRI, rut depth, and faulting 

using automated vehicles either as part of or independent of the distress survey.

Many transportation agencies have been collecting network-level pavement condition data for 20

years or more and collectively have used a variety of technologies. While data quality has 

largely improved in step with technology advances, it has also resulted in data consistency 

issues. As an example, the first automated rut depth measurement systems used three or five 

sensors to measure the distance from a rut bar to the pavement surface while the latest 

technology uses lasers to measure the transverse profile using 1,000 or more points across the 

pavement surface. The resulting calculated rut depths from the lasers are often substantially 

greater than those measured using three or five points. Thus an agency has decisions to make 

regarding the use of the new data in pavement performance curves, treatment triggers, and 

condition reporting. These types of consistency issues are not negligible and must be addressed 

continually as technology evolves.
Practical Guide for Quality Management of Pavement Condition Data Collection

16

Table 6. Summary of agency pavement condition data collection (McGhee 2004; FHWA 

2008; Zimmerman and McKinney 2011; Zimmerman and McKinney 2012).

Method Number of Agencies

Agency Vendor Total

Data 

Collection

Automated 23 21 44

Windshield 19 2 21

Data 

Processing

Fully Automated 7 7 14

Semi-Automated 16 14 30

Reporting Interval

While technology has enabled the collection and processing of data points for calculating rut 

depth, joint faulting and IRI at very close longitudinal spacing (as small as 1 in [25 mm]), the 

interval used to summarize and report the data must be practical and useful to the agency. 

Therefore, condition reporting intervals are typically some fraction of a mile (km) and intervals 

of 0.01 to 1 mi (0.016 to 1.6 km) are common. Another option is to report the data aggregated to 

the pavement management analysis section length (McGhee 2004).

Data Items Collected

Information collected as part of a network-level data collection effort may involve many items, 

but there is a fairly standard set of condition data typically collected, including roughness, 

rutting, faulting, and surface distress (table 7). Other information, such as right-of-way imagery 

may augment this data and provide information related to other assets (e.g., guardrail, signs, and 

structures).

Roughness

Virtually all highway agencies collect network-level roughness data through automated means

(McGhee 2004). While older technology included the Mays meter and other response-type road 

roughness measurement equipment, newer systems use non-contact sensors to collect 

longitudinal profile data at highway speeds. The longitudinal profiles are used to calculate the 

IRI statistic according to designated standards (e.g., AASHTO R 43, Standard Practice for 

Quantifying Roughness of Pavements and ASTM E1926, Standard Practice for Computing 

International Roughness Index of Roads from Longitudinal Profile Measurements).
Practical Guide for Quality Management of Pavement Condition Data Collection

17

Table 7. Network-level surface deterioration by pavement type.

Pavement Type Network-Level Data Items

Asphalt

• Roughness

• Rut depth

• Transverse cracking

• Fatigue (wheelpath or load-related) cracking

• Non-load related (block, edge, or construction joint) cracking

• Shoving or distortion

• Potholes and/or patching 

• Bleeding

• Raveling

• Polishing

Composite

(asphalt over concrete)

• All distresses listed for asphalt pavements

• Reflective cracking

Jointed Concrete

• Roughness

• Faulting

• Slab cracking (transverse and/or longitudinal)

• Scaling

• Polishing

• Map cracking (or alkali-silica reactivity)

• Durability cracking (D-cracking)

• Joint spalling and/or pumping

• Joint seal damage

• Blowups

• Patching

Continuously 

Reinforced Concrete

• Roughness

• Punchouts and/or patching

• Longitudinal cracking

Gravel

• Potholes

• Washboarding

• Loose aggregate or dust

Rut Depth

Rut depth is another surface distress that is measured by most, if not all, highway agencies, often 

concurrently with the longitudinal profile. However, while the measurement of longitudinal 

profile and calculation of IRI have been largely standardized, the methods used to measure rut 

depth still vary greatly between agencies. Many agencies own data collection vehicles outfitted 

with three or more individual ultrasonic or laser sensors mounted across the front or rear bumper 

of the data collection vehicle. Newer collection vehicles project lasers across the roadway and 

collect more than a thousand data points transversely. The AASHTO standard (AASHTO R 48, 

Standard Practice for Determining Rut Depth in Pavements) for measuring and reporting rut 

depths specifies a minimum number of sensors; thus, all systems using at least five sensors will 

be in accordance with AASHTO R 48.
Practical Guide for Quality Management of Pavement Condition Data Collection

18

Faulting

Faulting is most often calculated from the same longitudinal profiles used to calculate IRI. The 

AASHTO protocol for joint faulting measurement (AASHTO R 36, Standard Practice for 

Evaluating Faulting of Concrete Pavements) is recommended for HPMS submittal, but highway 

agencies often rely on the protocols defined by the service provider or equipment manufacturer 

(McGhee 2004).

Cracking and Other Surface Distresses

There is variability among highway and local transportation agencies in the collection of 

pavement surface distress. Some distresses that are prevalent in one area of the country are not 

significant in others due to variations in climate or construction materials and practices (i.e., 

rutting in the southwest and thermal cracking in the northern United States). While the FHWA, 

AASHTO, and ASTM have all issued standards for the terminology, definitions, and data 

collection techniques, there is still variation in the distress types and collection methods used by 

highway and local transportation agencies.

Condition Indices

Raw pavement condition data are typically converted into indices for use in pavement 

management systems. Various distresses and severities are often combined to form an index that 

represents a certain type of distress. For example, a fatigue index for asphalt pavements may 

incorporate various levels of fatigue cracking, wheelpath patching, and potholes. These indices

represent a condition state and can be used to rank pavement sections, trigger treatments, or 

predict future conditions.

While many transportation agencies collect individual pavement distresses at the network level

and then use those to create various individual indices, others collect an overall condition 

indicator, such as present serviceability rating (PSR), present serviceability index (PSI), or 

pavement condition index (PCI) (Ganesan et al. 2006). The PSR, developed in the 1960s at the 

American Association of State Highway Officials (AASHO) Road Test, was long required by the 

FHWA for State highway agencies’ annual HPMS submittals. PSR was a subjective rating of a 

pavement’s ability to serve the traffic as intended and was based largely on ride quality as 

experienced by the rater. Because of its subjectivity, the PSR was difficult to reproduce. Later, 

a more objective measure, the PSI, was developed as a way to calculate overall pavement 

condition based on measurements of roughness, rut depth, and cracking. Many State highway

agencies adopted the use of the PSR or PSI as an overall indicator as they developed their 

pavement management system.

The PCI is a more complex indicator originally developed by the U.S. Army Corps of Engineers

and later standardized in ASTM D5340, Standard Test Method for Airport Pavement Condition 

Index Surveys. The PCI is a numerical value between 0 and 100 that is calculated from a visual 

survey of pavement distress on a sample of the network. Various distress/severity combinations 

result in points deducted from the starting value of 100. Some agencies modified the PCI 

calculations to use only the distresses prevalent on their pavement network.
Practical Guide for Quality Management of Pavement Condition Data Collection

19

In addition to standardized condition indices created for widespread use, a number of State 

highway and local transportation agencies have developed their own unique overall condition 

index, often termed a pavement quality index (PQI) or some other designation. For example:

• Ohio DOT calculates a PQI based on measured pavement roughness and a pavement 

condition rating (PCR).

• Minnesota DOT combines a ride quality index and surface rating to derive a PQI.

• South Carolina DOT uses their pavement distress index and the PSI to calculate a PQI.

• Oklahoma DOT combines individual indices, such as ride, rut depth, and functional and 

structural indicators, to calculate a PQI.

• Nebraska DOR calculates a serviceability index, which is a combination of visual distress 

and rut depth or faulting.

Other Data Items

Other pavement condition data that may be collected at the network-level include friction, 

structural capacity, and macro texture. Network-level friction data are less commonly collected

due to the associated cost of data collection and analysis. Some agencies use skid trailers to 

perform locked-wheel skid testing according to ASTM E274, Standard Test Method for Skid 

Resistance of Paved Surfaces Using a Full-Scale Tire. If friction testing is conducted at the 

network-level, testing is typically conducted on 2- or 3-year intervals. Macro texture is a 

property related to friction and is relatively inexpensive to collect since it can be measured with 

the same sensors used to collect IRI. However, surface friction is determined by both micro 

texture and macro texture, which limits direct use of macro texture-only values.

Although quite desirable from a pavement management perspective, structural capacity testing is 

infrequently performed at the network level due to time and cost. Current test methods most 

often use static FWD equipment to measure pavement deflections under a dropped load. The 

FWD must be stationary in the test lane for a short period of time as the test is performed and

traffic control is needed to protect the equipment and crew. Testing may be performed at predetermined intervals, such as every 0.10 or 0.50 mi (0.16 to 0.80 km) along the route. Ground 

penetrating radar (GPR) and coring may be needed to determine pavement layers and enable 

calculation of pavement structural properties. The coring operation is often performed separately 

and also requires traffic control. A newer technology, called the rolling wheel deflectometer 

(RWD), has been under development for a number of years to collect deflection measurements at 

highway speeds. The Virginia DOT has investigated the use of RWD as a network-level 

screening tool to identify areas needing more detailed testing (Diefenderfer 2010).

A number of other data items are frequently collected at the network level concurrently with 

pavement condition. While not directly related to the pavement condition, many of these are 

needed by State highway agencies to fulfill Federal reporting requirements, and others are 

desirable for planning, programming, or inventory purposes. These include horizontal and 

vertical curves, longitudinal grade, elevation, cross slope, and global positioning system data 

(i.e., latitude and longitude). Some of these data items are collected using the same lasers and 

accelerometers that are used to collect pavement condition data. Others use equipment that can 

be easily installed on the data collection vehicles.
Practical Guide for Quality Management of Pavement Condition Data Collection

20

Roadway Events and Other Assets

Certain roadway events need to be recorded in case an agency wishes to exclude some data from 

consideration in the pavement management system. These may include bridges, railroad 

crossings, construction zones, and lane deviations (i.e., when the data collection vehicle must 

move out of the collection lane for some reason). These events may be recorded by the field 

crew or entered in the database by the rating personnel after collection.

Increasingly, agencies are using the network-level condition data collection process as an 

opportunity to collect inventory or condition information on other roadway assets, such as signs, 

signals, striping, guardrail, and bridge clearances. Many of these are extracted from video 

captured during data collection, but others are collected with additional equipment on the data 

collection vehicle.

Video

Many network-level data collection vehicles use video as part of the distress rating process. 

Downward-facing cameras collect pavement images that are stitched together to form a 

continuous record of the pavement surface. Special lighting is often used to illuminate any 

shadows on the pavement surface. The distresses visible in the pavement images are categorized 

and classified according to agency distress rating protocols. Typically, agencies also collect

images with at least one forward-facing camera and sometimes side and/or rear-facing cameras. 

These cameras are often used to assist in verifying location or to collect other assets.

The technology used to collect pavement and roadway images is continually evolving. The first 

pavement distress images were captured using 35 mm film, and this method is still used for the 

LTPP program. VHS tapes were used subsequently by many agencies and service providers. 

The current capture method most often used is digital imaging stored on tapes or recorded 

directly to computer hard drives. As described by McGhee (2004), digital images “lend 

themselves to automated analysis because of their ability to analyze variations in grayscale as 

those variations relate to pavement features.” Whether the distresses are rated using a fully- or 

semi-automated process, digital images provide a number of advantages in storing and accessing 

images.

The digital image capture technologies currently in use include area scan and line scan imaging. 

Area scan cameras capture an area of pavement, typically 6 to 12 ft (1.8 to 3.6 m) wide and 10 to 

15 ft (3.0 to 4.6 m) in length, meaning two cameras may be needed to cover the entire lane 

width. Line scan cameras capture a single line of pixels across the lane at a time and build the 

pavement image line by line as the vehicle progresses longitudinally along the road. Future 

developments in imaging technology may allow for a 3-dimensional representation of the 

pavement surface through the use of laser radar (i.e., light detection and ranging [LIDAR] and

laser radar [LADAR]).

Distress Rating and Data Collection Protocols

Most transportation agencies have well-defined and documented protocols for evaluating 

pavement condition; however, there is a great deal of variability between agencies in distress 

definitions and post-processing summaries. For example, some agencies collect only the length 
Practical Guide for Quality Management of Pavement Condition Data Collection

21

of each type of cracking, while others collect the type, severity, and extent. Each agency 

typically will have a distress rating manual (often with photos of distresses) and other 

documentation of the collection methods and protocols to be used.

Efforts to standardize data collection have been ongoing since the 1980s (AASHTO 2001). Both 

ASTM and AASHTO have led the development of standards related to pavement management

definitions, distress protocols, and data collection techniques. These standards are not always 

separate and may reference another standard. For example, the AASHTO standard for 

quantifying pavement roughness references an ASTM standard for collecting pavement profile. 

The FHWA specifies the use of AASHTO standards for the collection of IRI, rut depth, faulting, 

and asphalt pavement cracking for the annual HPMS submission by State highway agencies.

Pavement condition data collection and rating protocols should clearly define the distress types, 

severity levels, rating methods (i.e., count, length, or area), and the reporting interval. In 

addition, some protocols specify the method to be used to compute condition values, such as a 

quarter-car simulation to compute IRI or a 5-point stringline method to compute rut depth. 

Failure to understand and communicate any of these requirements can negatively impact the 

consistency and usefulness of the data that the agency receives.

AASHTO Protocols

In the early 2000s, AASHTO proposed and later adopted standards of practice for collecting

network-level IRI, rut depth, cracking in asphalt pavements, and faulting in jointed concrete 

pavements. The purpose of these standards was to help produce consistent results for use in 

network-level pavement management. Each of the standards documents how to perform the 

measurement, the reporting interval, QC procedures, and certification of equipment (if 

applicable). Since that time, additional standards have been developed for other aspects of 

pavement condition data collection. These have been adopted on a temporary basis for up to 8

years, during which time AASHTO may convert them to full standards or opt to discontinue

them. The AASHTO standards (designated as “R”) and provisional standards (designated as 

“PP”) most relevant to network-level data collection include:

• AASHTO M 328, Standard Specification for Inertial Profiler. Defines the required 

attributes of an inertial profiler, including equipment requirements, mounting and 

installation details, and profiler precision and bias statements (in accordance with 

AASHTO R 56, Standard Practice for Certification of Inertial Profiling Systems).

• AASHTO PP 67, Quantifying Cracks in Asphalt Pavement Surfaces from Collected 

Images Utilizing Automated Methods. Describes procedures for quantifying cracking 

distress at the network-level in asphalt pavement surfaces utilizing automated processing 

of images. Any functionally adequate equipment or software that involves minimal 

human intervention can be used to process the images; however, significant human 

review is acceptable. A sampling of images or 100 percent coverage can be used.

• AASHTO PP 68, Collecting Images of Pavement Surfaces for Distress Detection. 

Describes procedures for collecting images of pavement surfaces using automated 

methods to detect distress for both network- and project-level analysis. Any functionally
Practical Guide for Quality Management of Pavement Condition Data Collection

22

adequate equipment can be used to collect the images, but they are to be collected 

utilizing a platform traveling at or near the prevailing highway speed.

• AASHTO PP 69, Determining Pavement Deformation Parameters and Cross Slope from 

Collected Transverse Profile. Describes a method for deriving pavement deformation 

parameters such as rut depth and cross-slope in pavement surfaces using a transverse 

profile. Any equipment or procedure with the acceptable accuracy can be used, and the 

data will typically be processed using a collection of algorithms in a computer.

• AASHTO PP 70, Collecting the Transverse Pavement Profile. Describes a method for 

collecting pavement transverse profile, including its relationship to a level horizontal 

reference, in pavement surfaces using automated measurement devices. The profile can 

subsequently be used to quantify cross-slope, edge drop off, and pavement distresses such 

as rut depth. Any equipment or procedure with the acceptable accuracy can be used; 

however, this standard addresses data collection using a measurement device traveling at 

or near the posted speed limit.

• AASHTO R 36, Evaluating Faulting of Concrete Pavements. Describes a method for 

estimating faulting in the outside wheelpath in jointed concrete pavements using either 

manual or automated measurements. Requires reporting of the maximum fault value, to 

the nearest mm, and total number of transverse joints and transverse cracks with 

measurable faulting over a summary interval of 0.06 m (0.1 km).

• AASHTO R 40, Standard Practice for Measuring Pavement Profile Using a Rod and 

Level. Describes a method for collecting pavement profile using conventional survey 

equipment. Profiles are measured using relative elevation differences.

• AASHTO R 41, Standard Practice for Measuring Pavement Profile Using a Dipstick®. 

Describes a method for collecting pavement prolife using the Face Technologies 

Dipstick. Profiles are measured using relative elevation differences.

• AASHTO R 43, Quantifying Roughness of Pavements. Describes a method for 

estimating roughness from a single longitudinal profile in each wheelpath. IRI is 

calculated from each profile and the average of the two is reported as the roughness for 

the section. This standard references ASTM E950, Standard Test Method for Measuring 

the Longitudinal Profile of Traveled Surfaces with an Accelerometer Established Inertial 

Profiling Reference as the method by which to measure the profile. It requires reporting 

in metric units to the nearest 6.3 in/mi (0.1 m/km) over a summary interval of 0.06 mi 

(0.1 km). Requires agencies to develop a plan that includes, at a minimum, personnel 

qualification and training, equipment accuracy and calibration records, and ongoing QC 

program. Additional, non-mandatory guidelines are given for development of a plan, 

including agency certification of data collection personnel, equipment maintenance and 

testing program, regular testing of verification sections, and time-series comparisons of 

IRI data.

• AASHTO R 48, Determining Rut Depth in Pavements. Describes a method for 

estimating rut depth in pavement surfaces from transverse profile measurements using a 

minimum of five points and the wire method for calculation. Any equipment or 

procedure with the acceptable accuracy can be used. It requires reporting of maximum 

and average rut depth to the nearest mm and rut depth stratification (2, 3, or 4 level) for 

each summary interval of 0.06 mi (0.1 km).
Practical Guide for Quality Management of Pavement Condition Data Collection

23

• AASHTO R 55, Quantifying Cracks in Asphalt Pavement Surface. Describes procedures 

for quantifying cracking in both wheelpath and non-wheelpath areas of asphalt pavement 

surfaces using automated or manual methods. Any equipment or procedure with the 

acceptable accuracy can be used. This standard requires a plan that addresses personnel 

qualification/certification/training, equipment calibration/maintenance/testing, monthly 

testing of validation sections, and time-series comparisons of ratings.

• AASHTO R 57, Standard Practice for Operating Inertial Profilers and Evaluating 

Pavement Profiles. Describes the procedures for operating and verifying calibration of 

an inertial profiling system.

ASTM Standards

The following ASTM standards have been developed to support the collection of pavement 

condition data at the network-level.

• ASTM E950, Standard Test Method for Measuring the Longitudinal Profile of Vehicular 

Traveled Surfaces with an Accelerometer Established Inertial Profiling Reference. 

Establishes methods for evaluating and classifying the accuracy of inertial profilers based 

on the sampling interval, vertical measurement resolution, precision, and bias.

• ASTM E1166, Standard Guide for Network Level Pavement Management. Provides an 

outline of the basic components of a pavement management system, including LRS, data 

collection and database managements, analysis, implementation, operation, and 

maintenance.

• ASTM E1926, Standard Practice for Computing International Roughness Index from 

Longitudinal Profile Measurements. Defines the standard for computing IRI from a 

longitudinal profile based on a quarter-car simulation model. 

• ASTM E1656, Standard Guide for the Classification of Automated Pavement Condition 

Survey Equipment. Outlines a method to classify equipment that operates at traffic 

speeds and collects longitudinal profile, transverse profile, or cracking of the pavement 

surface.

• ASTM E1703, Test Method for Measuring Rut-Depth of Pavement Surfaces Using a 

Straightedge. Describes a method for manually measuring rut depth using a 6 ft to 12 ft 

(1.8 to 3.6 m) straightedge and a gauge graduated to 1 mm or finer. 

• ASTM D6433, Standard Practice for Roads and Parking Lots Pavement Condition Index 

Surveys. Describes a method for the determination of road and parking lot pavement 

condition through visual surveys using the PCI method (developed by the U.S. Army 

Corps of Engineers) of quantifying pavement condition.

Long Term Pavement Performance

The LTPP Distress Identification Manual (Miller and Bellinger 2003) is a rating protocol that 

was developed as a research tool for the Strategic Highway Research Program to enable 

collection of uniform distress data on hundreds of test sections across the country. At the time of 

publication in 1993, the manual was the first of its kind to provide a common language to 

describe a uniform method for measuring pavement distresses. While recognized as a researchlevel tool, the LTPP Distress Identification Manual (Miller and Bellinger 2003) has been used by 
Practical Guide for Quality Management of Pavement Condition Data Collection

24

a number of highway agencies, including the Colorado and Oregon DOTs, as a starting place in 

developing State-specific distress rating manuals.

Highway Performance Monitoring System

As previously described, the HPMS is an annual national performance reporting tool for the 

FHWA. The HPMS Field Manual specifies the data items, including pavement condition, which

must be reported by all State highway agencies. While it does not create any new protocols for 

collecting and reporting pavement condition data, the field manual does specify which protocols 

should be followed for each item. For HPMS submittal, IRI data is to be collected in accordance 

with AASHTO R 43, rut depth in accordance with AASHTO R 48, and faulting data in 

accordance with AASHTO R 36. For cracking length and percent, AASHTO PP 67 and the 

LTPP Distress Identification Manual (Miller and Bellinger 2003) are recommended as guides.

Agency Specific

As previously stated, most highway agencies have developed agency-specific pavement 

condition data collection procedures and distress rating protocols. While some have adopted the 

AASHTO protocols for the collection and reporting of IRI, rut depth, and faulting―as required 

for HPMS submittal―other condition data is largely collected according to individualized 

protocols in each agency. Each agency distress rating manual is unique and may contain 

additional information helpful for data collection. Examples of agency distress rating manuals 

include:

• British Columbia Ministry of Transportation and Infrastructure (MoTI), Pavement 

Surface Condition Rating Manual (BCMoTI 2012).

• Metropolitan Transportation Commission (MTC 2002).

• Minnesota DOT Distress Identification Manual (MNDOT 2003).

• Nebraska DOR Pavement Maintenance Manual (NDOR 2002).

• North Carolina DOT Pavement Condition Survey Manual (NCDOT 2010).

• Oregon DOT Pavement Distress Survey Manual (ODOT 2010).

• Texas DOT Pavement Management Information System Rater’s Manual (TXDOT 2010).

• Utah DOT Pavement Preservation Manual – Part 2, Pavement Condition Data (UDOT 

2009).

An example of an agency rating manual and collection protocol is further discussed in Case 

Study No. 1 (see Appendix C for a more detailed version of the British Columbia MoTI Case 

Study).
Practical Guide for Quality Management of Pavement Condition Data Collection

25

Case Study No. 1. 

Rating Manual and Data Collection Protocol 

British Columbia MoTI 

The British Columbia MoTI Pavement Surface Condition Rating Manual provides data collectors

explicit instruction on how to identify and rate distresses as well as collect roughness and rut depth 

measurements. The manual goes into great detail on the classification and rating of surface 

distresses, with diagrams, photos, and insight into possible causes of the distresses. The manual 

was developed by a committee of rehabilitation and design personnel and later updated with field 

input. The same document specifies the data quality requirements

and QC tests to be performed on high speed network-level

surveys (BCMoTI 2012).
Practical Guide for Quality Management of Pavement Condition Data Collection

26

Case Study No. 1. 

Rating Manual and Data Collection Protocol 

British Columbia MoTI (continued) 
Practical Guide for Quality Management of Pavement Condition Data Collection

27

Case Study No. 1. 

Rating Manual and Data Collection Protocol 

British Columbia MoTI (continued) 
Practical Guide for Quality Management of Pavement Condition Data Collection

28

4. PRINCIPLES OF DATA QUALITY MANAGEMENT

Introduction

Data QM is a growing field of science related to information systems. While QM concepts, such 

as conformance to specifications and meeting customers’ expectations, have long been applied to 

manufacturing, industry, and even highway construction, the application of QM to data quality is

relatively new. Organizations such as the International Organization of Standards (ISO), the 

National Institute of Standards and Technology (NIST), and others have advanced the concept of

applying QM principles to data to the point that data QM is now a recognized academic field of 

study.

Principles and Terminology

The concepts and principles associated with the measurement of data quality have evolved 

significantly in the last few decades. The traditional approach to data quality could be called an 

“error approach,” whereas current methods could better be characterized as an “uncertainty 

approach (ISO 2008).” With the traditional error approach, there was an assumed single “true”

or reference value, and the objective of measurement was to get as close as possible to that true 

value. With the newer uncertainty approach, there is an assigned interval, or range, of 

reasonable values with an acknowledgment of the uncertainty and finite amount of detail that can 

be measured (ISO 2008).

The traditional approach to data quality describes deviations away from a true value that are due 

to random and systematic errors. Random errors result in dispersion, or low precision, around a 

reference value, while systematic errors shift the observed mean of a series of measurements 

away from the actual value, resulting in bias or low accuracy (figures 5 and 6). With the large 

number of measurements taken for network-level pavement condition data collection, random 

errors tend to offset each other and systematic errors become the most important influence on

data quality (Flintsch and McGhee 2009). However, it should be noted that the combined effect 

of random and systematic errors may result in a large total error in any single measurement.

Figure 5. High precision, low

accuracy

Figure 6. High accuracy, low

precision
Practical Guide for Quality Management of Pavement Condition Data Collection

29

The newer data quality concepts put forth by ISO, NIST, and others replace precision, bias, and 

accuracy terminology with trueness and uncertainty. Trueness refers to the closeness of 

agreement between the mean of a large number of measurements and a true value, and

uncertainty characterizes the reasonable dispersion of measured values. Measures of trueness 

and uncertainty are often described by standard deviations, confidence intervals, or more 

complex statistical or mathematical computations than traditional measures of data accuracy.

Whether using traditional or newer data quality concepts and terminology, the objective remains

to characterize the closeness of a measurement to an accepted reference value or interval of 

values. Most of the pavement condition data quality standards currently used by transportation 

agencies incorporate the traditional terminology of precision, accuracy, and repeatability, along 

with other aspects of data quality, including completeness and consistency.

Definitions

Data QM uses many of the same terms as those used in describing QM for industrial production 

and highway construction. Terms such as QC, acceptance and independent assurance (IA) are all 

applicable to data QM. The following definitions, unless otherwise noted, were taken from 

AASHTO R 10 (AASHTO 2011), see also the Glossary of Terms at the end of this document:

• Acceptance – The process whereby all factors used by the agency (i.e., sampling, testing, 

and inspection) are evaluated to determine the degree of compliance with contract 

requirements and to determine the corresponding value for a given product. Also referred 

to as verification when used to validate the collected data.

• Accuracy – The degree to which a measurement, or the mean of a distribution of 

measurements, tends to coincide with the true population mean. When the true 

population mean is not known, as is the case with pavement data collection, the degree of 

agreement between the observed measurements and an accepted reference standard 

(ground truth) is typically used to quantify the accuracy of the measurements (TRB 

2002).

• Bias – An error, constant in direction, that causes a measurement, or the mean of a 

distribution of measurements, to be offset from the true population mean.

• Calibration – A set of operations that establish, under specified conditions, the 

relationship between values of quantities indicated by a measuring instrument or 

measuring system, or between values represented by a material measure or a reference 

material, and the corresponding values realized by standards.

• Independent assurance (IA) – Activities that are an unbiased and independent 

evaluation of all the sampling and testing (or inspection) procedures used in the quality 

assurance program.

• Precision – The degree of agreement among a randomly selected series of measurements; 

or the degree to which tests or measurements on identical samples tend to produce the 

same results.

• Quality – The degree of excellence of a product or service; the degree to which a product 

or service satisfies the needs of a specific customer; or the degree to which a product or 

service conforms with a given requirement.
Practical Guide for Quality Management of Pavement Condition Data Collection

30

• Quality control (QC) – The system used by a contractor to monitor, assess, and adjust 

its production or placement processes to ensure that the final product will meet the 

specified level of quality. Quality control includes sampling, testing, inspection and 

corrective action (where required) to maintain continuous control of a production or 

placement process.

• Repeatability – Degree of variation among the results obtained by the same operator 

repeating a test on the same material. The term repeatability is therefore used to 

designate test precision under a single operator.

• Reproducibility – Degree of variation among the test results obtained by different 

operators performing the same test on the same material.

• Resolution – The smallest change in a quantity being measured that causes a perceptible 

change in the corresponding indication (ICO 2008).

• Validation – The mathematical comparison of two independently obtained sets of data 

(e.g., agency acceptance data vs. contractor data) to determine whether it can be assumed 

they came from the same population.

The Benefits of a Data Quality Management Program

An effective pavement management system depends on reliable, accurate, and complete 

information. Quality pavement condition data is directly linked to the ability of the pavement 

management system to produce reasonable, timely, and reliable recommendations regarding an 

agency’s pavement network. Increasingly, pavement managers realize that money is wasted and 

poor decisions are made when data are substandard. Confidence in data is eroded and people

within the organization will tend to work around poor quality data. The savings from using good 

data comes from more accurate decisions and lower life cycle cost for maintaining the 

pavements. AASHTO recognizes some of the benefits of QM in the pavement condition data 

collection program as increased “credibility, cost-effectiveness, and overall utility of the PMS” 

(AASHTO 2001). However, after a certain point, increasing the reliability of data significantly 

increases the cost of collecting it.

Virginia DOT studied the effects of implementing a comprehensive quality monitoring plan and 

found three primary benefits (Shekharan et al. 2006):

• Increased accuracy in reporting existing condition indices as demonstrated by changes in 

the condition categories by as much as 60 percent.

• Increased accuracy in reporting deficient pavements by as much as 30 percent. 

• Increased accuracy of budget need determinations, illustrated by a cost correction of 

more than $18 million for interstate pavement maintenance recommendations.

Virginia DOT concluded that without a comprehensive QM plan, “agencies may be under or 

overestimating maintenance and rehabilitation needs by 25 percent or more” (Shekharan et al. 

2006). Shekharan et al (2006) also stated that the essential benefits of developing a QM plan for 

pavement condition data include:
Practical Guide for Quality Management of Pavement Condition Data Collection

31

• Improved accuracy and consistency of data.

• Better credibility within the organization.

• Better compliance with external data requirements.

• Better integration with other internal agency data.

• Cost-savings from more appropriate treatment recommendations.

• Improved decision support for managers.

Data Quality Management Cycle

Management of data quality is based on many of the same principles as other QM processes, 

such as Total Quality Management (TQM) and the Deming cycle of “Plan, Do, Check, and Act” 

for quality enhancement. Wang (1998) identified four phases that are essential in the QM cycle 

to ensure high quality data: define, measure, analyze, and improve. Expanding these concepts to

pavement condition data collection includes (also shown in figure 7):

• Define data quality – Identify the acceptable levels of resolution, accuracy, and 

repeatability.

• Plan and implement QC – Develop and implement a set of procedures to produce, 

check, and ensure data of acceptable quality.

• Perform acceptance tests and evaluate results – Perform tests to compare delivered 

data to acceptability metrics. 

• Take corrective action – Take steps to re-collect or reprocess data as needed to achieve 

data acceptance standards.

• Report on data quality – Document the data quality standards, protocols, equipment, 

personnel, collection and processing methods, QC, acceptance tests, and results.

• Improve the process – Use the knowledge and experienced gained to modify processes 

as needed to improve data quality.

It should be noted that the steps in the QM cycle incorporates a feedback process so that the 

collection team evaluates data quality continually throughout the collection and makes any 

needed process modifications as soon as it becomes evident.

Maintaining the Data Quality Management Process

The power of data QM stems from the continued application of the quality cycle each time data 

are collected. Even well-constructed QM programs are only effective when they are well 

maintained. A static QM plan will eventually lose its effectiveness, therefore the plan should 

continue to evolve as opportunities for improvement are identified and implemented.

The QM process should be accessible and understandable to all those involved in the data 

collection effort. The ability to provide employee feedback to the QM plan can motivate staff to 

participate in finding new ways to improve quality.
Practical Guide for Quality Management of Pavement Condition Data Collection

32

Define Data 

Quality

Plan and 

Implement 

Acceptance

Perform 

Acceptance 

Tests & 

Evaluate 

Results

Take 

Corrective 

Action

Report on 

Data 

Quality

Improve 

the Process

Figure 7. Data QM cycle (modified from Wang 1998). 

Cost-Effectiveness of Data Quality Management Procedures

While agencies would like to have the highest possible data quality, there are practical limits to 

the cost-effectiveness of QM procedures and the optimal quality level is not necessarily the 

highest. The net benefit of quality data can be measured using models that link varying levels of 

data quality (and the associated costs) with economic outcomes. The data quality costs include 

all of the costs associated with planning, implementing, and evaluating data quality. The 

benefits of high quality data can be thought of as the avoidance of costs that would be incurred 

with failures of data quality. The costs of poor quality data include re-collecting, reprocessing, 

or re-rating data. For the user of such inaccurate data, costs include those associated with 

incorrect maintenance, rehabilitation, or reconstruction decisions. 

The result of the data QM program can mean a significant reduction in pavement treatment costs 

for an agency. In 2003, agencies that outsourced data collection paid an average of $50 per mile 

($80 per km) for complete sensor and distress data and video collection (McGhee 2004). An 

agency might spend the equivalent of $10,000 to $15,000 annually per lane mile ($16,000 to 

$24,000 per lane-km) for pavement maintenance, rehabilitation, and reconstruction treatments

over the life of a pavement. Even a reduction of one percent in the annual cost of pavement 

treatments would more than offset the entire cost of data collection.

Different data QM activities, tools, and methods may be more or less cost effective. For 

example, prevention of data errors through improved training is often a more cost-effective 

approach than screening for and correcting errors once they have been made. In general, 
Practical Guide for Quality Management of Pavement Condition Data Collection

33

automated and preventive data QM procedures tend to be more cost-effective than laborintensive corrective procedures.
Practical Guide for Quality Management of Pavement Condition Data Collection

34

5. DATA QUALITY MANAGEMENT PLAN

Introduction

A data QM plan is a document that defines the acceptable level of data quality and describes how 

the data collection process will ensure this level of quality in its deliverables and processes. QM

activities ensure that:

• Data will meet agreed-upon standards and requirements.

• Work processes are performed as documented.

• Non-conforming data are identified and appropriate corrective action is taken.

Data QM plans apply to data collection deliverables and work processes. QM plans should 

include QC, and acceptance criteria. QC activities by the data collection team (agency or 

vendor) are necessary to monitor data quality and resolve errors as they arise. The QC activities 

help to monitor and verify that the data collection processes are being followed and are effective

in obtaining quality standards. Finally, acceptance activities assist in verifying that the data 

collection deliverables meet the defined quality standards.

There are many templates available for development of a QM plan and an example – adapted 

from the University of Wisconsin, Department of Information Technology – is provided in 

Appendix B (UW 2012).

Importance of a Data Quality Management Plan

The quality of network-level pavement condition data is vital not only for pavement management 

purposes (i.e., assessing network-level condition, predicting pavement performance, establishing 

condition indices, and so on), but also for an ever expanding list of potential applications, such as 

calibration of the MEPDG, incorporation into HPMS, forensic investigations, and development 

of asset management performance measures. Therefore, the ability to evaluate and determine the 

quality of pavement condition data is essential for establishing the accuracy and reliability of 

analyses made using pavement condition data.

Though it is desirable to collect the highest quality pavement condition data possible, the 

improvements in quality must be balanced with the additional required effort, time, and budget. 

It is also recognized that the quality of needed pavement condition data may be a function of the 

intended use of the data. This can range from research-level analysis to development and 

reporting of key performance measures (e.g., statewide pavement condition and percent of 

smooth roads). Pavement condition data is no longer used just for reporting network-level 

condition, but has expanded into other areas, such as planning and programming preservation 

and rehabilitation activities and capital improvements as well as asset management applications. 

As the use of pavement condition data expands within and to other areas beyond pavement 

management, the ability to quantify pavement condition data quality through QM procedures 

becomes increasingly important.

As outlined by Flintsch and McGhee (2009), consistently achieving a quality product or service 

requires the implementation of a formal approach to organize, manage, and control quality. This 
Practical Guide for Quality Management of Pavement Condition Data Collection

35

approach should include methods, techniques, tools, and model problem solutions. A QM plan 

helps ensure adequate procedures are in place to reduce or eliminate systematic and random 

errors. A QM plan is an efficient framework for dealing with quality issues, and without a 

documented plan, agencies are less likely to consistently apply QM activities from year to year 

or assess the effectiveness of the techniques utilized. A formal plan is critical to continuing the 

QM cycle and for ongoing improvement over time.

Quality Management Plan for Data Collection

Since high quality network-level pavement condition data is vital to a number of pavement 

applications, improving data quality by establishing a QM plan to address QC and acceptance of 

collected pavement condition data is essential. The QM plan establishes data quality criteria, 

acceptable levels of variability, and procedures designed to limit variability as much as possible. 

In addition, a large number of highway agencies use service providers for pavement condition 

data collection; therefore, a QM plan plays a critical role in ensuring that the data is collected

correctly and accurately and that the correct data collection process is repeatable from year to 

year. QM processes should be in place throughout the life of the pavement condition survey—

before, during, and after production—and should include activities undertaken by the agency and 

any pavement condition data collection service provider. The QM activities apply to pavement 

condition data collected using manual, semi-automated, or automated methods.

Responsibility for QM of the pavement condition data collection lies with the agency and the 

service provider (when applicable). The QC and acceptance procedures are used to ensure that 

the collected pavement condition data meet or exceed quality standards. In addition, the 

acceptance criteria should be met prior to the pavement condition data being accepted for use.

The key features of the plan include setting data quality standards, directing activities to achieve 

those quality standards (including monitoring and corrective actions), measuring pavement 

distress, and reporting the results. These key features include:

• Define data collection/rating protocols – A critical component of the QM plan is 

establishing the methods, standards, and protocols to be used in collecting the data. 

Pavement condition rating protocols/guides should clearly define the distress types, 

severity levels, rating methods (e.g., count, length, or area), reporting interval, and the 

method to be used to compute condition values. Failure to understand and communicate 

any of these can negatively impact the usefulness of the condition data.

• Establish data quality standards – An important first step in the QM plan is to establish

and document the data quality requirements for all deliverables. Quality standards should

specify realistically attainable accuracy, completeness, precision, repeatability, 

reproducibility, and resolution criteria for each data item at the network level. The 

specific measures that will be used to determine adequate data (and video) quality should 

be identified.

• Identification of responsibilities – While there is considerable overlap in QM activities 

(such as checking data) during data collection; the QM plan should outline the person(s) 

responsible for each activity. For example, data collection personnel are responsible for 

data QC since they produce the data and have the tools and resources to influence the 
Practical Guide for Quality Management of Pavement Condition Data Collection

36

quality of those data; alternatively, the user (e.g., pavement management group) is in the 

best position to assess data acceptability because this entity is the ultimate owner of the 

data. The QM plan should identify the staffing, roles, and responsibilities for QC and 

acceptance, including problem reporting, documentation, and tracking.

• Personnel training programs – Training for data collection, rating, and reduction 

personnel is an important QC element in the QM plan. Crews must learn how to 

calibrate, operate, and troubleshoot complex equipment; raters must learn the proper 

protocols and pass competency tests; and data reduction personnel must learn how to 

compile the data in the proper format and check for errors. Some agencies require a 

formal “certification” of the pavement distress raters and equipment operators to verify 

that they have the needed knowledge and skills.

• Equipment calibration and method acceptance – A key feature of QM plans is the 

requirement for equipment calibration and method acceptance. Testing equipment is

calibrated and testing methods and analysis are verified prior to data collection and 

periodically thereafter to ensure that the equipment is functioning according to 

expectations and that the collection and analysis methods are being followed. Testing of 

control, blind, or verification sites are used for QC and acceptance before and during 

production. Other validation techniques include oversampling or cross-measurements 

and reanalyzing or resurveying a sample of the sections.

• Data inspection – The QM plan should establish the timeframe or recurring frequency 

for performing specific data checks. In general, data inspection checks are performed 

during production for QC and acceptance when the data are submitted. Typical data 

checks include network-level checks for ratings that are out of expected ranges, checks 

for detecting missing segments or data elements, and statistical analysis to check for data 

inconsistencies.

• Corrective action – The QM plan should specify the corrective action to be taken if data 

are not found to meet quality requirements. This may include re-collecting or rerating 

pavement sections, and it is important that the agency and service provider discuss and 

agree upon the corrective actions upfront rather than waiting until a problem is 

discovered.

• Quality management reporting – To complete the QM cycle, it is important that 

documentation of the QC and acceptance procedures be performed during all phases of 

the data collection survey. Reporting is an important component of the overall QM 

program as it enables tracking of problems and continuous improvement of the quality 

process. QM reporting enables the agency to refer back to previous reports, keep track of 

related problems, and take steps to prevent the same issues from reccurring.

A flowchart of the QM activities is further illustrated in figure 8.
Practical Guide for Quality Management of Pavement Condition Data Collection

37

Figure 8. Summary of QM activities (adapted from Flintsch and McGhee 2009).

Start Here

Highway Agency

 Scope of data collection  Delineation of responsibilities

 Time schedule  Data collection requirements  Data collection manuals and protocols  Quality control requirements

 Data acceptance criteria based on accuracy, completeness, 

precision, repeatability, reproducibility, and resolution  Other specific requirements and review

Contract out data 

collection?

No, In-House

Yes

Service Provider Selection

 Request for proposals (such as scope of services, time frame, 

responsibilities, minimum qualifications, data collection approach, 

QC requirements, acceptance requirements, deliverables, and so 

on)  Proposal selection

 

Equipment/Method 

Validation Process

 Selection of 

control sites

 Test control sites

 Meeting unifying 

criteria

Initial Start-Up Process

 Personal training / 

certification

 Equipment 

calibration / 

certification / 

inspection

Control site 

adequate?

Plan & Set up Prior to Data Collection

No

Yes

Pilot Data 

Collection

 Pilot 

feedback

Production Data Collection

 Real time in-vehicle 

data checks  Daily raw data checks 

for completeness, and 

reasonableness

 Image quality

 File & project tracking 

/ documentation

Data Processing

 Automatic

 Semiautomatic

 Manual

During Data Collection
Practical Guide for Quality Management of Pavement Condition Data Collection

38

Figure 8. Summary of QM activities (continued).

QC Checks

 Periodic data (daily, weekly, 

monthly) checks for

completeness and 

reasonableness

 Rater consistency checks

 Equipment validation testing

Acceptance

 Periodic data checks for 

completeness and 

reasonableness

 Blind (or known) control site 

validation testing to ensure data 

collected meeting required 

acceptance criteria

 Sampling to check whether data 

meeting the tolerance 

requirements

R t

Corrective Actions

 e.g., re-run process & re-survey 

as necessary

QC

Adequate?

No

Yes

During Data Collection (continued)

Data Delivering QC Checks

 Field data review, software 

checks if applicable  Review for missing segments 

(such as GIS based) or data 

elements

 Time series comparisons

 Independent validation: 

sampling and statistical 

i

Corrective Actions

 e.g., re-process or re-collect 

some sections or survey missing 

sections

Deliverables

 Data tables

 Images

 Reports

Databases

 PMS database

 Asset management database

 Video database Go back to data collection process

Acceptance?

No

Yes

After Data Collection

No further changes
Practical Guide for Quality Management of Pavement Condition Data Collection

39

6. DATA QUALITY STANDARDS AND ACCEPTANCE CRITERIA

Introduction

Establishing data quality standards and acceptance criteria is an important early step in any QM 

plan. Therefore, each agency should formalize and document their data quality requirements, 

including the criteria for acceptance of data. This information is generally documented in the 

agency-developed pavement condition survey scope. An example of a pavement condition 

survey scope is shown in figure 9. In the survey scope, the agency compiles a detailed list of 

sections to be tested, often including the beginning and ending descriptions of the segments (e.g., 

highway junctions or cross streets), as well as global positioning system (GPS) latitude and 

longitude coordinates, when available. These detailed descriptions play an important role by 

maximizing the surveyor’s ability to locate the proper pavement segments to be surveyed. 

Pavement Condition Survey Scope

A. Number of miles to be surveyed. 

a. Increasing, decreasing, or both directions for interstate, multi-lane, divided, 

and two lane highways.

b. For automated pavement condition data collection, specify image collection 

(e.g., right of way images on all routes). 

B. Data to be collected, units of measure, and reporting distance (e.g., every 0.10 mi 

[0.16 km]): 

a. GPS coordinates (longitude, latitude, and elevation), when applicable. 

b. International Roughness Index (IRI) (inch per mile or meter per kilometer).

c. Bridges (count). 

d. Concrete pavements:

i. Transverse cracking (linear feet).

ii. Longitudinal cracking (linear feet). 

iii. Joint faulting (inch). 

iv. Patching (square feet).

v. Blowups (square feet).

vi. Punchouts (square feet).

e. Asphalt-surfaced pavements:

i. Alligator cracking (square feet).

ii. Block cracking (linear feet).

iii. Longitudinal cracking (linear feet).

iv. Transverse cracking (linear feet).

v. Rutting (inch). 

vi. Patching (square feet).

vii. Blowups (square feet).

viii. Potholes (count).

Figure 9. Example pavement condition survey scope (adapted from LADOTD 2011).

In addition to providing a list and description of sections to be tested, some agencies also prepare 

a database for use during the condition survey. The database typically contains columns for all 
Practical Guide for Quality Management of Pavement Condition Data Collection

40

of the data elements to be collected and rows for each reporting interval (e.g., every 0.01 mi

[0.02 km] or every 0.25 mi [0.40 km]). The agency populates the database cells related to 

location (e.g., route identifier, direction, and milepost) and other data elements such as number 

of lanes, functional classification, and expected pavement type. The remaining cells are 

populated by the agency staff (for agency-conducted surveys) or by the service provider during 

the condition survey. In this way, a properly formatted database is provided for use, thus helping 

to minimize pavement condition data quality issues.

As a practical matter, the quality standards will largely depend on the agency’s intended use of 

the data to support decisionmaking. Some agencies will use individual distresses (or an index 

based on an individual distress) to trigger treatment options within a pavement management 

system. As an example, rut depth over 0.5 in (12.7 mm) might trigger an asphalt overlay. Other 

agencies may use only an overall index, such as PCI, to report and model pavement condition,

trigger treatment options, and analyze budgetary needs. Knowledge of the eventual use of each 

data item, trigger points, and any grouping of data is important when determining both the 

resolution and quality needed. An agency might perform a sensitivity analysis of data items in 

the pavement management system to determine data quality impacts.

Resolution, Accuracy, and Repeatability

The data quality requirements in a QM plan typically define the level of resolution, accuracy, 

and repeatability for each data element. Resolution refers to the level of detail―specified in 

absolute terms―such as rut depth measured to the nearest mm or IRI measured to the nearest 

inch/mile. The resolution specified should be fine enough to track pavement deterioration 

adequately and support agency decisions but must reasonably reflect technological limitations

for network-level collection. Equipment manufacturers can provide practical guidance on the 

resolution capabilities of equipment. As shown in table 8, different protocols or reporting 

requirements have varying data resolution requirements.

Table 8. Examples of data resolution requirements for different protocols.

Data Item Required Data Resolution

HPMS LTPP AASHTO

IRI 1.0 in/mi (0.016 m/km) 0.6 in/mi (0.01m/km) 6 in/mi (0.1 m/km)

Rut depth 0.10 in (2.54 mm) 0.04 in (1 mm) 0.04 in (1 mm)

Fault height 0.10 in (2.54 mm) 0.04 in (1 mm) 0.04 in (1 mm)

Accuracy (or bias) refers to the closeness of a measurement to an accepted ground truth or 

reference value. Requirements can be specified in absolute values, percent, standard deviation, 

or other statistical measure. For example, table 9 includes the accuracy and precision criteria for 

several highway agencies. In addition, Case Study No. 2 illustrates an example of the data 

quality standards for the Oklahoma DOT. Appendix E contains a summary of the QC and 

acceptance process used by the Oklahoma DOT.
Practical Guide for Quality Management of Pavement Condition Data Collection

41

Table 9. Example of agency pavement condition data quality requirements.

Agency Data Element Accuracy Precision

British Columbia 

MoTI (BCMoTI 2012)

IRI ± 10 percent of Class I 

profile survey

± 6.3 in/mi (0.1 m/km)

standard deviation for 5 

runs

Rut depth ± 0.12 in (3 mm) of 

manual survey

± 0.12 in (3 mm)

standard deviation for 5 

runs

Pavement distress 

index (PDI)

± 1 PDI of manual 

survey (0 to 10 scale)

± 1 standard deviation of 

the PDI for 5 runs

Alabama DOT

(ALDOT 2010)

IRI ± 5 percent of control 

section

± 1 in/mi (0.02 m/km) 

average of 5 passes

Rut depth ±0.1 in (2.5 mm) of 

manual survey

± 0.01 in (0.25 mm) 

average of 5 passes

Faulting ±0.1 in (2.5 mm) of 

manual survey

± 0.01 in (0.25 mm) 

average of 5 passes

Virginia DOT

(Diefenderfer 2010)

IRI ±5 percent of agency 

reference value < 5 percent of 10 runs

Rut Depth ± 0.08 in (2.0 mm) of 

agency reference value < 5 percent of 10 runs

Once resolution, accuracy, and repeatability requirements have been established, they should be 

periodically re-evaluated to ensure that they adequately represent the capabilities of the current 

technology and assure the reliability of the data. As a minimum, the data quality requirements

should be evaluated every 5 years.

Reference Values

An important aspect of data quality standards is the establishment of reference values or “ground 

truth” at control and blind sites. Control site reference values are typically established by the 

agency and evaluated using manual measurement techniques or the “most appropriate” 

technology (note: techniques used for manual measurement of control sites are described further 

in Chapter 7). The control sites are used to carry out a series of measurements under more or 

less repeatable conditions. The reference values are assumed to represent the true pavement 

condition and are used for comparison to the values measured periodically using network-level 

automated or production methods at the same control sites. This is one method of evaluating 

whether quality standards for accuracy are being met.
Practical Guide for Quality Management of Pavement Condition Data Collection

42

In assigning a reference value, it is assumed that no mistakes were made in performing the 

measurement, the equipment and method used have low error, and the resulting value is a 

“correct” reflection of conditions in the field. However, it should be recognized that adherence 

to the best procedure results in a reference value with some degree of uncertainty and that even 

the most accurate ground truth is only temporarily accurate, as conditions change with time. 

In choosing the methods to measure and establish reference values, it is important to understand 

the technology and procedures that will be used to collect each network-level data item. For 

example, rut depth measured with a 5-point rut bar will provide very different values than that 

measured with Institut National d’Optique (INO) lasers. The two methods cover different lane 

widths and use a significantly different number of data points to calculate rut depth. The agency 

Case Study No. 2 

Example of Data Quality Standards 

Oklahoma DOT 

The Oklahoma DOT has contracted for pavement condition data collection since the 1990s. In 2001, 

Oklahoma DOT began implementation of a pavement management system and recognized the 

importance of being able to measure data quality. As part of its new QM procedures, Oklahoma DOT 

worked with the data collection contractor to understand the limits of the available technology and 

then formalized its data quality standards.

During collection, Oklahoma DOT personnel meet the data collection field crew at control or 

verification sites on a weekly basis. Using the quality standards shown below, sensor data (IRI, rut, 

and fault) from the sites are checked for accuracy and repeatability. If more than one vehicle is 

collecting data on the project, the data is also checked for reproducibility. Upon delivery of final data, 

samples of distress ratings are checked for accuracy. The GPS coordinates for the beginning of each 

control section are compared for accuracy with Oklahoma DOT-provided coordinates. All GPS 

coordinates (collected every 0.01 mi) are plotted on a state highway system shape file using a GIS and 

visually inspected for obvious errors or omissions. 

Data Element

Required Minimum 

Accuracy

Required Resolution 

(Measure to the 

Nearest)

Required Minimum 

Repeatability

International 

Roughness Index

± 5 percent compared to 

dipstick or Class I profiler 1 in/mi (0.02 m/km) ± 5 percent run to run for three repeat runs

Rut Depth ± 0.08 in (2.0 mm) compared 

to manual survey 0.01 in (0.25 mm) 

± 0.08 in (2.0 mm) 

run to run for three 

repeat runs

Faulting ± 0.08 in (2.0 mm) compared 

to manual survey 0.01 in (0.25 mm) 

± 0.08 in (2.0 mm)

run to run for three 

repeat runs

Distress Ratings ± 10 percent compared to 

agency ratings N/A N/A

GPS Coordinates 0.00005 degrees compared to 

agency provided coordinates 0.000001 degree N/A
Practical Guide for Quality Management of Pavement Condition Data Collection

43

may choose to try and replicate the production method (including its limitations) or may instead 

try to establish the most accurate reference value possible. It is important that the agency also 

understand the limitations and implications of the selected manual (or other) method of 

measuring the pavement condition. These factors should be explicitly taken into consideration 

when choosing the method of establishing reference values.

It is particularly challenging to establish reference values for surface distress ratings, such as 

cracking. The subjectivity of distress ratings makes data accuracy more difficult to ascertain. 

For these data, the reference values may be a consensus-based ground truth estimate. The 

method used to rate distresses will again have a direct bearing on the resulting reference values. 

For example, the ability to detect fine cracks in asphalt surfaces is a type of discrepancy that 

often occurs between manual and automated measurements. 

To the extent possible, reference values should be established for all data items to be collected. 

For example, agencies could use handheld GPS equipment at a number of sites to use as a check 

against the collected latitude and longitude data. Curve and grade values could be obtained from 

recent construction survey plans and pavement (or surface) type could be visually observed at a 

number of locations. 

Data Variability

Besides the inherent variability in pavement condition data, there are external sources of 

variability that influence data measurement. These include data collection equipment or method, 

rater consistency, inter-rater uniformity, time, and data referencing, processing, or handling 

(Morian, Stoffels, and Frith 2001). 

External sources of variability in sensor data (e.g., IRI, rut depth, faulting, and macro texture) 

often are related to the equipment used to collect the data. The type of sensor used (e.g., laser, 

ultrasonic, and scanning laser) may have a different sensor footprint, lane coverage width, 

number of data points used in the calculations, and can be affected by roadway environmental 

conditions―including weather, contaminants on the pavement, or pavement texture. Wheel path 

wander and/or lateral position of the vehicle are often a source of data variability related to 

operation of the equipment. Referencing errors lead to data variability if the location of the data 

is even slightly off. Raw data must be processed according to the protocols specified by the 

agency and any inconsistency in the method used will result in data variability. These are just a 

few examples of external sources of data variability, and it is important that agencies spend the 

time to learn more about this subject. Without a broad understanding of the sources of data 

variability, the agency may have unreasonable expectations or may improperly assess data 

quality. 

Sources of variability in distress ratings can be related to image quality (e.g., resolution and 

contrast) if distresses are rated from pavement video. In the case of windshield surveys, the 

rater’s ability to see the roadway clearly and the speed of the survey vehicle have an impact on 

data variability. The subjective and complicated nature of the rating process (i.e., the rater must 

correctly identify both the type of distress and the severity) leads to data variability. Studies 

have shown that consistency in determining severity levels is particularly difficult to achieve 

(FHWA 2000; Goodman 2000). 
Practical Guide for Quality Management of Pavement Condition Data Collection

44

Statistical Analysis

Verification of pavement condition data assessment often consists of reanalyzing or resurveying 

a sample of pavement sections. The sample size should consist of a sufficient number of 

pavement sections to ensure that the selected samples represent the population (for each 

pavement family) and that there is a sufficient number of sections to verify the specified 

measurement accuracy (Flintsch and McGhee 2009). For pavement condition data collection, 

the sample size used in the QC process usually ranges from 2 to 10 percent (Flintsch and 

McGhee 2009). Typically, agencies select sample size based on previous experience or 

determine the optimal sample size using statistical techniques based on the desired accuracy and 

the degree of risk (Flintsch and McGhee 2009). Sample size can be determined using the 

following equation: 

n = �

zσ 2⁄ σ

E �

2

(Eq 1) 

where:

n = sample size

zσ/2 = Standard Normal Distribution

 = 1.960 (α = 0.05, or 95 percent confidence interval) 

 = 1.645 (α = 0.10, or 90 percent confidence interval)

σ = population standard deviation

E = tolerable bias

Data checks between QC or acceptance samples and production surveys (for the same roadway 

section) often include evaluation of the differences between the mean condition values (e.g., 

comparison of IRI, individual distresses, and/or composite index). The comparison of means 

typically includes a paired t-test to determine if the production survey is consistently under or 

overestimating pavement condition. Other statistical evaluations have also been used. For 

example, the British Columbia MoTI uses the kappa statistic (see Appendix C), the Alabama 

DOT uses a Pearson’s r correlation (ALDOT 2010), and the Nebraska DOR uses a multivariate 

factor analysis (NDOR 2009).

Acceptance Criteria

After establishing quality standards, the agency must also set the criteria for acceptance of the 

data. The acceptance criteria specify the limits of data variability, most often by a allowing a 

certain percentage or standard deviation above or below the reference value. Acceptance criteria

may also specify the percent of data checked that must be within the acceptable limits. For 

example, some agencies require that 100 percent of the data tested or sampled meet the specified 

tolerances, but others require a certain percent within limits (PWL). An example of a PWL batch 

acceptance criteria used by Pennsylvania DOT on a 2.5 percent sample of data is shown in table 

10. 
Practical Guide for Quality Management of Pavement Condition Data Collection

45

Table 10. Pennsylvania DOT batch data acceptance criteria (PennDOT 2011). 

Reported Value Initial Criteria

Percent 

Within Limits Action if Criteria Not Met

IRI ± 25 percent1 95 percent Reject deliverable

Individual distress 

severity combination ± 30 percent1 90 percent Feedback on potential bias or drift in ratings and retrain on definitions

Total fatigue cracking ± 20 percent1 90 percent Reject deliverable

Total non-fatigue 

cracking ± 20 percent1 90 percent Reject deliverable

Total joint spalling ± 20 percent1 90 percent Reject deliverable 

JCP transverse 

cracking2 ± 20 percent1 90 percent Reject deliverable

Location – segment /

offset

Correct segment 

surveyed 100 percent Return deliverable for correction

Location – section 

begin ± 40 ft (12 m) 95 percent Return deliverable for correction and systems recheck

Panoramic images Legible signs 80 percent Report problem – reject subsequent 

deliverables 1 within agency measured value.

2 JCP – jointed concrete pavement. 

Corrective Actions

The corrective actions to be taken during QC and acceptance should be agreed on and specified 

in the QM plan. The acceptable tolerances for each data item should be interpreted so that when 

variability limits are exceeded, the collection or rating process should stop and the equipment or 

rater should not be used for production until the problem is resolved. Re-calibration of 

equipment or re-training of personnel may be needed to correct the problem. 

In rare cases, variability levels are exceeded but production may continue on a trial basis. This 

decision should only be taken after careful consideration and always with the explicit agreement 

of the agency. It is essential that decisionmakers possess the knowledge and skills necessary to 

assess the factors involved in excessive variability and make a sound judgment based on all the 

available information. This deviation from the QC and acceptance protocol should be 

approached with caution and changes documented. 

During the acceptance phase, the most common corrective actions specified are rejection of the 

deliverable and re-collection, re-rating, or reprocessing of the item. Due to expense, this is the 

least desirable time to discover a problem with data or video quality. For this reason, most QM 

plans strongly emphasize QC and acceptance activities in order to mitigate the risk of producing 

data of unacceptable quality. All corrective actions should be documented in the final QM 

report.
Practical Guide for Quality Management of Pavement Condition Data Collection

46

Other Quality Requirements

Acceptance criteria should also be specified for any non-data deliverables, such as video. Image 

quality is often described in subjective terms. 

Many agencies specify requirements for the clarity and brightness of video images. These 

requirements are more subjective in nature and typical acceptance criteria for panoramic or rightof-way images could include the minimum percent of a control section with clear images (e.g., 

signs readable), no debris on the lens or housing in the image, adequate exposure, and images in 

the correct order and direction. For pavement images, the criteria usually include proper 

exposure, sufficient sharpness to enable distress identification, and correct “stitching” of images. 

The acceptance criteria might also specify the maximum number of allowable consecutive 

images of less than acceptable quality.

An example of video image quality requirements from the Louisiana DOTD is provided in Case 

Study No. 3. The complete Louisiana DOTD QC and acceptance process is provided in 

Appendix D. 
Practical Guide for Quality Management of Pavement Condition Data Collection

47

Case Study No. 3. 

Quality of Video Images 

Louisiana DOTD 

The Louisiana DOTD has been conducting automated pavement condition surveys since 1995. 

Surveys are conducted by pavement condition assessment service providers who are required to 

collect images for quantification of pavement surface distress as well as distance to overhead 

obstructions. Specifically related to video images, the service provider is required to review the 

pavement and right-of-way (ROW) images during data collection and at the end of each day to 

ensure:

Feature Pavement

Image

ROW

Image

Clarity  

Minimal missed/skipped images  

Proper lighting  

Correct stitching  ---

Upon receipt, the video images are checked by the Louisiana DOTD to ensure that the collected 

images have acceptable levels of clarity, brightness/darkness, and completeness. Per LADOTD 

2012c, video image review includes: 

• Image clarity―all images should be clear and highway signs easily read. Most 

highway distresses should be evident in all views. There should be minimal or no debris 

in the cameras’ viewing path.

• Image brightness/darkness―images are not to be collected during hours when it is too 

dark (rule-of-thumb: if street lights or security lights are on, then it is too dark). It has 

been found that during poor lighting conditions, the images become very grainy and 

seem to be out of focus, or it results in a “black out,” which can cause a control section 

to be rejected. In addition, if the data collection occurs just before a rain storm, the dark 

clouds may not allow the proper amount of light to enter the camera, and the subsequent 

image(s) will be of poor quality. 

• Dry pavement―control section should not have any standing water during testing; 

otherwise, the control section will be rejected. As a result, data collection should be 

halted during a rain storm. If rain drops are allowed to accumulate on the protective 

glass, the images will be of poor quality due to the lack of clarity and sharpness.

• Image replay―images should play sequentially and in the correct order. The data 

collection vehicle should give the impression that it is traveling in a forward direction.

• Missing images―there should be minimal or no missing images. Any control section 

that contains substitute images should be rejected. 
Practical Guide for Quality Management of Pavement Condition Data Collection

48

7. QUALITY CONTROL

Introduction

QC includes those activities performed to assess and adjust production processes to obtain the 

desired level of quality of pavement condition data (Flintsch and McGhee 2009). This chapter 

focuses on the tools and techniques used for data collection, whether conducted using in-house 

staff or by service providers, to ensure the production of high-quality data.

The QC plan should detail the procedures to be followed during the pavement condition survey 

and include such items as (see Appendix G for QC and acceptance checklist):

• Training automated distress collection crews and distress raters.

• Equipment setup and calibration.

• Field testing control and verification sites.

• Real-time data checks.

• Internal validity checks.

• Quality checks during data reduction.

• Corrective action.

It is interesting to note that many of these procedures are currently in use by U.S. highway 

agencies routinely performing network-level data collection activities. Figure 10 illustrates the 

percent of highway agencies (U.S. State and Canadian provincial agencies) using the various QC

activities.

Personnel Training

One of the key QC activities is training the personnel that will be collecting the data and/or 

rating the distresses. In some cases, the collection and rating are performed simultaneously―

with one technician driving while another logs distresses using a keyboard. In other cases, the 

crew collects sensor data and pavement video images, which are used later to rate visible 

pavement distresses. Sometimes, visible distresses are rated from the shoulder of the road (a 

low-speed windshield survey) while other condition data (such as IRI and rut depth) are collected 

at highway speeds by a field crew. Thus, the training given the collection crew will vary greatly 

depending on the method of collection and rating. For this discussion, the issue of training the 

data collection field crew and rating personnel will be treated separately.

Distress rating technicians (for either manual or semi-automated surveys) undergo extensive 

training before production rating begins. Typically, agency and service providers conduct 

training for pavement condition distress rating in-house; however, very few have certification 

programs that raters must pass prior to commencing the condition survey. Two exceptions 

include the North Carolina DOT, which provides rater certification training, and the Louisiana 

DOTD, which requires the service provider to certify pavement raters. However training is 

carried out or evaluated, pavement condition raters should be instructed in the agency’s unique 

distress rating protocols. In addition, raters should conduct and be evaluated on verification sites 
Practical Guide for Quality Management of Pavement Condition Data Collection

49

specific to the pavement condition data collection. Distress ratings performed by the survey 

crews should be compared to those of the lead rater for accuracy and consistency. Cross-rater 

checks are also performed to ensure all raters produce similar results. If any raters are found to 

be deficient, additional training may be required.

Figure 10. Use of quality control and acceptance processes by U.S. and provincial 

highway agencies (Flintsch and McGhee 2009)

The training program for automated data collection crews is also extensive. Data collection

technicians must learn how to operate and troubleshoot complex computer hardware/software, 

video, and automotive systems. Their training typically includes calibrating the equipment, 

monitoring the data and video systems in real time, and understanding the multitude of factors 

that can affect data quality during the collection process. They should also be trained in startup 

checks (e.g., laser calibration, accelerometer “bounce test”, distance measuring instrument 

[DMI]) that are performed daily before collection begins to ensure proper functioning of 

equipment and software. They must also learn checks of systems and data that are performed at 

the end of collection every day (e.g., data completeness, image and data quality) and other 

systems checks that are performed on a weekly or periodic basis (e.g., DMI and GPS). Finally, 

field technicians are trained to review data in the field to look for inconsistencies and to check 
Practical Guide for Quality Management of Pavement Condition Data Collection

50

error logs. This allows the crews to spot problems, take corrective actions, and re-collect the 

segments before leaving the area. Field technicians should keep a proper log of all checks to 

submit to the project manager as documentation.

Equipment Calibration

The complexity of current automated technologies used for network-level data collection makes 

it imperative that equipment be properly adjusted and calibrated prior to the start of surveys. 

This equipment is capable of collecting large volumes of data very quickly, but if the equipment 

is out of calibration or improperly configured for the data collection, the data may be useless for 

the intended purpose. Attention must be given to the configuration and calibration of equipment 

and confirmation of proper operation after data collection if there is to be confidence in the 

quality of the data.

Equipment calibration and configuration requirements will likely be specific to the piece of 

equipment in use. As such, specification of requirements may be difficult in a standardized QM 

plan. If data are collected by an agency using a set piece of equipment, the procedures and 

requirements may be more easily specified. However, if collected by a service provider, the 

agency may have to work closely with the provider to establish minimum requirements for 

configuration, calibration, periodic checks, and documentation.

Any pavement condition data collection equipment and its subsystems (e.g., DMI, GPS, or video 

images) should be calibrated and checked prior to initializing the pavement condition survey. 

Control sites with known length and condition values can be used to calibrate the pavement 

condition data collection equipment. Certain subsystems, such as the DMI, can be calibrated in 

the field, if needed, while others (typically laser sensors) will require additional laboratory 

calibration.

Configuration and Calibration

The equipment used for automated network-level data collection consists of complex sensors, 

computers, GPS units, and video systems installed on the data collection vehicle (DCV). The 

components are configured according to agency requirements and calibrated as specified by the 

manufacturer. Some of the DCV components, such as laser sensors, are sealed and cannot be 

calibrated except by the manufacturer and thus should be replaced when they malfunction. For 

the components that can be calibrated, the agency typically performs initial calibration prior to 

mobilization and re-checks the equipment in the field at the agency’s control sites.

Calibration of the laser profiling system should include laser sensor checks and block tests to 

ensure the accuracy of the height sensors, accelerometer calibration, “bounce tests” to verify 

proper functioning of the height sensors and accelerometers, and distance calibration to ensure 

accuracy of the DMI. Calibration of the DMI and some accelerometers occurs during field

testing, and each should be checked and recalibrated on a regular basis. The DMI calibration 

includes testing on a course of known length, and should be of sufficient length to minimize 

calibration errors. Generally, agencies check accelerometer’s daily during startup operations. 

Certain tests, such as the “block test,” which checks the vertical height sensor, can detect

problems; however, the vertical height sensor requires manufacturer calibration. If a sensor is 

replaced, the block test should be run to verify that the replacement sensor is performing 
Practical Guide for Quality Management of Pavement Condition Data Collection

51

accurately. Agencies typically conduct “bounce tests” and other equipment tests on a weekly 

basis.

In addition to calibrating the equipment, agencies must also establish the list of data items that 

will be used in the measurement calculations. Onboard software generally analyzes the IRI, 

faulting, and rut statistics from the raw longitudinal and transverse profiles. Although there are 

many ways to compute IRI, the AASHTO and ASTM standards incorporate calculations to 

simulate the response of a quarter-car traveling at a specified speed and wavelength for filtering

the raw profile. Likewise, onboard systems also analyze faulting and rut depth measurements.

Additional field testing, at startup and during collection, includes setting and checking the 

quality of the video camera. After collection, parameters are set for the automated processing 

and images are sequenced and “stitched” (in the case of pavement video) for proper playback.

Data collection teams use the collection databases and detailed section descriptions to properly 

locate the section “begin” points during data collection. The data collection team uses these files 

to route data collection efficiently to avoid unnecessary delays or downtime for the field crews.

Verification of equipment and method continues throughout the data collection phase by repeat 

testing of the control sites (described in a subsequent section of this chapter). If any problems 

are found during collection, the equipment is recalibrated or replaced and/or the raters are retrained.

Distress rating methods must also be calibrated before production begins. Raters are trained in

the agency’s specific rating protocols and their ratings are compared against each other and 

against those of a lead (or QC) rater. An adequate number of distress rating control sites for each 

pavement type should be used in the pre-collection phase to ensure that the distress rating 

method is in accordance with agency requirements. Further training is performed until results 

are within tolerances.

Real-Time Monitoring, Daily or Periodic Checks

Field crews have checklists of daily checks that must be done at the beginning of each day, 

including checking the computer systems, camera enclosures, tire pressure, and lasers. In 

addition, the field crew should check video monitors periodically throughout the day to watch for 

any changes in video quality. Most DCV onboard systems enable real-time monitoring of IRI, 

rut depth, faulting, DMI, grade, cross slope, and GPS as it is being collected. These systems 

have audible and visual warnings displayed if data values are out of normal range (either high or 

low) or other problems are detected. The crew checks the onboard road sections and reference 

point database during collection to ensure the proper segments are being collected. Shapefiles, 

which are used to store geometric location and associated attribute information, may also be 

loaded into the computer to facilitate identification of collection segments. Shapefiles are 

particularly useful to the data collection crew members who may not be familiar with sample 

section locations.

At the end of each day of data collection, field technicians back up data and review data files to 

look for irregularities. Log files keep track of all error messages and section averages may be 
Practical Guide for Quality Management of Pavement Condition Data Collection

52

calculated. These, as well as samples of video, are reviewed for any issues that need corrective 

action. The field crew fills out a daily log of activities, including QC performed. Reports are 

uploaded and reviewed by QC personnel in the office.

Control, Verification, and Blind Site Testing

Control, verification, and blind site testing are critical QM activities that are performed before 

production and then periodically throughout the production phase of the data collection. Control 

sites are segments of road whose condition data have been measured by the agency (or third 

party) personnel. Manual measurements are not required, but measurement should be performed

using the best available and practical techniques. This data is used as a reference value or 

“ground truth” to compare against data collected during production and verifies proper collection 

procedures and continued calibration of the equipment. The agency may also use the same or 

different control sites to rate visible distresses (such as cracking and raveling) manually to 

compare with the production distress ratings. In this way, control sites are used to assess the 

adequacy of the QC processes being used during data collection.

Pre-production control site testing is performed to verify equipment calibration and method

acceptance before collection begins. Some agencies may require prospective service providers

to pre-qualify by testing local control sites and submitting the data with their proposal. At the 

beginning of data collection, the collection crew will run the control sites an agreed upon number 

of times (typically three or five). The average of the runs is used to compare to the reference 

values to assess whether equipment is properly calibrated and the proper protocols are being 

followed. If control sites are passed, the collection crew will be ready to start production data 

collection. If not, diagnoses are performed until the issues are resolved. During collection, 

control sites and/or verification sites are retested at regular intervals. If more than one DCV is 

used, the control site testing demonstrates reproducibility of the multiple DCVs.

For distress ratings, control sites are used in the pre-production phase to train and calibrate raters 

in the proper application of the rating protocols. The procedures are similar whether ratings are 

to be performed through semi-automated means or by windshield surveys. For fully automated 

distress rating, the control sites are used to calibrate and adjust the computer algorithms that are 

used during data reduction.

Control Site Selection and Setup

Agencies typically establish multiple control sites that are representative of different pavement 

types and distresses found on the network. Control sites should have varying levels of distress,

and it is possible to use the same control sites for many years. This is beneficial to the agency 

both for time savings (i.e., the sites may not need to be manually measured every year) and the 

ability to compare data history at the sites.

Control sites can be used to validate sensor and/or distress data, but often a site that is selected 

for one is not necessarily well-suited for the other. For example, significant cracking in the 

wheel paths can interfere with accurate and repeatable IRI or rut depth measurements. 

Therefore, it may be desirable to limit the amount of cracking on control sites to be used for 

sensor data validation. Control sites should have measurable (i.e., some minimum) amounts of 

distress. This is because a certain range of variability will be allowed in the data quality 
Practical Guide for Quality Management of Pavement Condition Data Collection

53

requirements, and if the amount of distress is low, the accuracy of the measurement may be more 

difficult to verify. For example, if a control site has an average rut depth of 0.50 in (12.7 mm)

and the required accuracy is ±0.10 in (2.5 mm), the acceptable measurement range would be 0.40 

to 0.60 in (10.2 to 15.2 mm). But on a site with average rut depth of 0.20 in (5.0 mm), the 

acceptable range would be 0.10 to 0.30 in (2.5 to 7.5 mm). In percentage terms, measurements 

on the first site have allowable variance of ± 20 percent, while on the second site, that tolerance 

would be ± 50 percent.

Other suggested requirements for control site selection are that they be straight (i.e., tangent 

sections) and flat (i.e., no curves or super elevation), have adequate lead in for acceleration and 

deceleration, and be safe for the collection crew to turn around to make multiple runs. Because 

of these requirements, control sites might not be used to check the accuracy of curve and grade

with automated equipment. Multi-lane divided highways are often selected as control sites to 

minimize traffic disruption.

Set up of control sites involves marking and measuring the control site location and length by the 

agency or third party. Control sites can vary in length but typically range from 0.5 to 1 mi (0.8 

to 1.6 km). If longer sites are used, the reference values are usually established using automated 

measurements. The beginning of a control site is marked (often with paint on the shoulder), a 

specified length is manually measured, and the ending location is marked. Detectable targets can 

also be placed at the beginning and endpoints to automatically trigger data recording. Lane 

markings must be in good condition to ensure the crew can discern the proper segment for 

collection.

As part of the data quality requirements, many agencies specify the evaluation of a control site 

during the pre-production phase. Control sites are used to calibrate data collection equipment 

and validate raters, as well as to establish ground truth of pavement distress (including rut depth, 

roughness, and faulting). Any discrepancies between the ground truth and the condition survey 

can be resolved prior to initiating the production survey. Table 11 provides examples of agencyspecified control site requirements.

Verification and Blind Sites

Verification sites are similar to control sites except they have not been measured manually by the 

agency. For sensor data, a verification site is tested shortly after the collection crew has 

demonstrated calibration at a control site, and this data is then accepted as the reference value for 

future repeat testing of the site. Verification sites are typically run multiple times, and the 

average of the multiple runs is compared to the reference value to determine accuracy and 

repeatability. Verification sites can also be used for windshield distress rating since crews can be 

sent back at intervals during data collection to perform repeat ratings of sites.
Practical Guide for Quality Management of Pavement Condition Data Collection

54

Table 11. Example of agency control site requirements.

Agency Number of 

Sites

Site

Length Other Details

British Columbia MoTI

(BCMoTI 2012) 4 (asphalt) 0.5 mi (0.8 km)

• Selected using prior year’s survey data or 

control sections

Louisiana DOTD

(J. A. Horne, personal 

communication 2012

4 (asphalt

4 (JCP)

4 (CRCP1)

0.5 mi 

(0.8 km)

• Service provider is also required to 

evaluation prior to proceeding to the next 

district

Oklahoma DOT

(ODOT 2010)

2 (asphalt)

2 (JCP)

0.5 mi 

(0.8 km)

• Used as part of the scoring of the service 

provider’s proposal

Pennsylvania DOT

(J. L. Arellano, personal

communication 2012)

4 (asphalt)

2 (JCP)

~0.5 mi 

(0.8 km)

• Service provider must run each testing 

vehicle prior to acceptance for 

production testing

Virginia DOT

(Shekharan et al. 2006)

8 (asphalt)

2 (JCP)

2 (CRCP)

Variable

• Calibrate distress rating process

• Establish precision and bias for 

roughness, rut depth, and distress 1 CRCP – continuously reinforced concrete pavement

Verification sites, like control sites, can be used for subsequent collection cycles if no treatment 

has been performed, so the agency has a history of the reference values at the site. Verification 

sites are typically spread geographically throughout the data collection area while control sites 

are often centrally located. Many agencies require control or verification site testing at regular 

intervals (e.g., weekly) throughout production data collection as specified in the QC plan. This

repeat testing is also used to demonstrate continued calibration of the equipment/method. Table 

12 provides an example of agency practices for verification testing.

Table 12. Example of agency testing of verification sites.

Agency Verification Testing

British Columbia MoTI (BCMoTI 

2012)

• One site every 3 days.

• For contracts longer than 30 days, re-evaluate to verify 

repeatability.

Colorado (CDOT 2012)

• Virtual review of eight asphalt and two concrete segments 

from first 500 mi (800 km), randomly selected.

• Each region reviews an additional three to six sites.

Louisiana (Fillastre 2012) • 5 percent of collected sections.

Maryland (MDSHA 2011)

• IRI and rut depth once a month, no less than 3 times during 

survey.

• Compare cracking index with previous year.

Nebraska (NDOR 2009) • 10 percent of pavement segments spot checked in field.

Oklahoma (ODOT 2010) • Weekly evaluation of either the verification or control site (6

to 10 per survey year).

Pennsylvania (PennDOT 2011) • 125 blind sites.

• Random sample of 2.5 percent of all segments surveyed.
Practical Guide for Quality Management of Pavement Condition Data Collection

55

Although not quite as common, agencies may also use blind sites whose locations are not 

disclosed to the data collection team in advance. As collection is completed in an area 

containing one of these unknown or blind sites, the agency requests the data for that segment of 

the network. The agency will have rated the distresses or manually measured the sensor data 

elements in advance to establish the reference values. The data collection team then submits the 

data, which is checked by an agency lead or QC rater.

Establishing Reference Values

There are many different methods and types of equipment that can be used by an agency or third 

party to measure and establish control site reference values for sensor data. While the manual 

measurement of discrete data points on the control site will likely be more accurate than the 

automated method, there will typically be fewer total data points collected. Consideration should 

be given to the number of data points needed to characterize the data item sufficiently. The 

methods used for manual measurement should result in a reference value that is equal to or better 

in accuracy than that which can be collected using automated means. Some of the common 

methods of manual measurement to establish reference values include:

• IRI – Establishing a reference value for IRI typically involves manually measuring a 

profile in each wheelpath and then calculating IRI for the left and right wheel paths from 

the profiles. A rod and level survey (figure 11) is one method used to manually measure 

the longitudinal profile and while accurate (classified by ASTM as a Class I profile), this 

method is also one of the slowest. A Face Dipstick® (figures 12 and 13) or walking 

profilers (figures 14 through 16), which are other types of Class I profilers, are often used 

to collect the longitudinal profile from which IRI can be calculated. The Dipstick® 

measures changes in elevation between two support legs as the operator pivots and 

“walks” the instrument along a survey line. The walking profiler automates the pivot 

process so that the operator pushes the machine along the roadway at a slow walking 

speed. The rolling model of the Dipstick® consists of three wheels that establish a 

reference line and displacement recorded at 1-in (25 mm) intervals and can operate at 

about 3 mph (5 km/h). After collecting the profile, computer algorithms are used to filter 

out specified wavelengths, and the calculation of IRI is based on the appropriate or 

desired simulation method (i.e., quarter- or half-car simulation).

• Rut depth – One common method for manually measuring rut depth is to use a 

straightedge along with a ruler or other gauge. In a typical procedure, the straightedge is 

centered over one wheelpath, and the vertical distance to the deepest part of the rut is 

measured with the ruler or gauge. The use of a rut wedge with incremented steps enables 

quicker and more repeatable measurements. The straightedge is then moved to the other 

wheelpath and the corresponding rut measured. Agencies have commonly used 4- or 6-ft 

(1.2 to 1.8 m) straightedges to measure rut depth. There are many variations to this 

procedure (such as sliding a straightedge from one edge line of the lane to the other or 

using a longer straightedge) that may result in somewhat different rut depth 

measurements. The wire method for calculating rut depth will also yield a different result 

than the straightedge method. A walking profiler or Dipstick® can also be used to collect 

a transverse profile for calculation of rut depth. If pivoting equipment is used, it should 

be realized that there will be a maximum of 12 data points for the transverse profile 

across the lane that can be used to calculate rut depth. It is strongly recommended that 
Practical Guide for Quality Management of Pavement Condition Data Collection

56

the agency consider the characteristics of the transverse profile (e.g., total width, number 

of data points, and so on) that will be produced by the automated equipment during 

network-level collection when deciding on the manual procedure to measure rut depth 

and establish the reference values.

Figure 11. Schematic of rod and level survey (Sayers and Karamihas 1998).

Figure 12. Close-up of Dipstick® profiler Figure 13. Example of Dipstick® profiler

(photo courtesy of WSDOT). (photo courtesy of WSDOT).
Practical Guide for Quality Management of Pavement Condition Data Collection

57

Figure 14. SSI CS8800 Figure 15. SurPro 3500 Figure 16. Walking Profiler 

(www.smoothroad.com) (www.surpro.com) G2 (www.arrb.com.au) 

walking profiler. walking profiler.

• Faulting – Manual measurements of joint and cracking faulting are often performed 

using a straightedge along with a ruler or gauge or by means of a fault meter. A 

straightedge is placed on the higher slab (approach or leave) at the joint and the vertical 

distance to the lower slab is measured with a ruler or gauge. A fault meter is a semiautomated method of measuring joint faults. The fault meter is placed on the leave side 

of the joint and a probe contacts the slab across the joint to measure the fault height. The 

Georgia fault meter incorporates a digital readout that displays positive or negative faults. 

As with rut depth measurements, the agency should recognize the characteristics and 

methods of the automated technology that will be used for the network-level collection of 

faulting. For example, most systems cannot distinguish between a faulted joint and a 

faulted transverse crack, so it is important to select control sites with no transversely 

cracked slabs. 

Manual ratings of cracking, potholes, raveling, joint deterioration, and other distresses can be 

performed on sites specifically selected for distress ratings. A walking survey of distresses may 

be conducted by agency personnel from the shoulder of the road. If data collection will use 

images to rate distresses, the agency may choose to rate distresses using the same images. 

Distress validation sites should be rotated or replaced on a regular basis to ensure that raters or 

operators are not repeating known values from prior surveys. 

Control Site Testing Procedures

Control site testing is usually performed prior to collection and then at some regular interval 

(e.g., weekly) during the data collection phase. If data collection is contracted, a representative 

of the agency may choose to be present at the control site testing and may even ride along in the 

DCV to observe. Prior to collection, a control site might be run three to five times consecutively

and the “run-to-run” average and other statistics (e.g., standard deviation) calculated and 

compared to both the data quality standards for accuracy and repeatability. If there are any 
Practical Guide for Quality Management of Pavement Condition Data Collection

58

concerns about data quality that arise during the initial control site testing, they should be 

resolved before collection is allowed to begin.

During collection, the field crew will return periodically and run control sites (typically three 

runs at each site) to verify continued calibration of the equipment and methods. If outsourced, 

the agency may wish to be present each time control sites are run or have the results sent to them 

electronically within a specified number of days. It is not uncommon for a DCV to fail the 

criteria for a data item at one or more control sites at various times throughout the collection 

phase. All personnel involved with the data collection process should openly discuss the results 

and work together to identify the possible causes and solutions. There may be equipment or 

software malfunctions, driver or operator error, weather-related problems, or the control site may 

have interfering distresses. It is also possible that the “ground truth” was incorrectly measured or 

an inappropriate technique used. All of these possibilities and others must be considered and a 

decision reached jointly about how to proceed. 

If the data quality analysis consistently shows discrepancy between the reference values and the 

data collectors’ control, verification, or blind site data, further investigation must often be 

performed to discover the cause. Discussions with the data collector about collection methods 

for the specific data elements in question are the best place to start.

If data collection is outsourced, the service provider may be using different data collection 

technology than the agency. Automated collection of sensor data is achieved using technology 

that collects hundreds or thousands of data points which may be more than the agency used 

during development of the reference values. The interpretation of sensor data to characterize 

roughness, rutting, or faulting is also an important consideration. Therefore, it is imperative to 

understand how different data collection techniques may result in substantially different 

measurements. For example, a three-point rut calculation may result in a substantially different 

maximum rut depth than a five-point or thousand-point laser-measured rut depth measurement. 

The data collector may be able to export longitudinal and transverse profiles from its collection 

system. If so, the agency can use those profiles to compare to its own. The agency should then 

use the data collector’s profiles to calculate IRI, rutting, or faulting and compare their results 

with those produced by the data collector’s methods. This is a good way to identify differences 

caused by calculation method and those related to collection technology. 

Once the agency has a full understanding of different measurement technique implications, it can 

then decide how best to characterize “ground truth” and establish reference values for its sites.

Tracking Test Results

After reference values have been established, a spreadsheet can be used for tracking the results of 

repeat testing of control, verification, or blind sites. The reference values are entered and 

tolerances for each data item are calculated and displayed in the spreadsheet. Each time the data 

collection crew runs a control site, the resulting data (summarized at the reporting interval) is 

copied into the spreadsheet and automatically compared to the reference values. The data 

collection crew and project manager then know quickly if the control site rating was passed or 
Practical Guide for Quality Management of Pavement Condition Data Collection

59

failed. Plots of the data are also sometimes helpful in identifying problems or possible changes 

in the control site condition. 

Distress Rating and Video Checks

In addition to QM processes for sensor data, distress rating and video data collection practices 

have additional potential requirements for QC. Areas where QC and acceptance considerations 

may be applied include setup of the processing software, audits of distress rating results and 

methods, and checks on image quality where applicable. Each of these is discussed below. 

Rating Software Setup

One method of controlling quality in distress ratings is to define values for each collected data 

element. Context-based validation rules can be programmed into the rating software so that 

inconsistent or illogical data, based on the event context, cannot be entered. For example, the 

software may prohibit entering asphalt pavement distresses on concrete pavements. Drop-down 

menus can also be used to minimize the key strokes and simultaneously improve data accuracy. 

Distress Rating Audits

The QC program for distress ratings includes random sample audits, inter-rater reproducibility, 

and repeat test checks. For manual and semi-automated methods, as raters complete a batch of 

ratings, random samples of the data are selected and checked by the lead rater or QC personnel. 

If the ratings do not meet quality standards, the entire batch should be re-evaluated. Table 13 

provides a summary of agency distress rating checks (these are in addition to individual distress 

and indices checks). 

Table 13. Summary of agency distress rating checks. 

Agency Distress Rating Checks

Louisiana DOTD 

(see Appendix D) 

• Missing high severity distresses.

• Missing 5 or more low/medium severity distresses. 

• Incorrect distress type or severity. 

• Over-rating (identifying a distress when one is not present).

Oklahoma DOT

(see Appendix E)

• Duplicates and range for IRI, rut depth, faulting, and macro texture.

• Ranges of individual and combinations of distress. 

• Maximum patch length. 

• Non-matching distress types to pavement type.

• Expected number of railroad crossings and bridge segments.

Pennsylvania DOT

(see Appendix F)

• Comparison of historical condition data.

• Comparison of maintenance costs.

Inter-rater reliability (also called cross-rater) means that two or more competent raters using the 

same protocols on the same sample sections should get the same results. Repeat test reliability 

(also called intra-rater) checks are performed by giving a single rater a sample of pavement to rerate later in the data collection phase and comparing the ratings with the previous ones. If 

similar results are not produced on any of these checks, further training and rerating of batches 

should be performed. A logic check is another type of check performed on distress. A logic 
Practical Guide for Quality Management of Pavement Condition Data Collection

60

check looks for consistency between the types of distresses rated and pavement type (e.g., 

asphalt cracking on asphalt surfaced pavements) or for a range of values. 

Video Checks

Video monitors are checked in real-time during collection by the crew to ensure proper image 

clarity, lighting, aim, and focus and to ensure that nothing on the camera enclosures are 

obscuring visibility. Routine checks of sample video are also performed by QC personnel who 

quickly notify field crews of any problems. Table 14 provides a summary of video quality 

checks conducted by the Louisiana DOTD. 

Table 14. Summary of agency video checks. 

Agency Clarity Brightness/

Lighting

Missed

Images

Image

Stitching

Location

Reference

Alabama DOT

(ALDOT 2010)

 

Colorado DOT 

(CDOT 2012) 

 1

Louisiana DOTD 

(see Appendix D)

   

Oklahoma DOT

(see Appendix E)

   2

1 ID flag/counter continuity 2 Segment begin and end points

Data Processing, Handling, and Database Checks

Data collection crews may send (or upload) a shipment of data on a daily or weekly basis to a 

central office. Data submissions should occur on a frequent basis so that if problems exist, they 

are identified and resolved quickly, before too much additional data is collected. As the data 

arrives, it is reviewed for accuracy and completeness, and if problems are found, the crew is 

given quick feedback so that corrective actions can be taken. Data and images (when applicable) 

are processed according to standard procedures and are analyzed for quality issues at that time.

After collection and rating are complete, the final database(s) is compiled. This involves taking 

data that may have been collected continuously, performing segmentation, assigning location 

information, and loading into a database shell or compiling the final database. The data 

collection team typically performs a series of checks on the data during this process. 

Segmentation

Data are typically collected continuously along a route and segmented later according to the 

agency’s LRS. The segmentation process has the potential to introduce sizable errors to a 

segment’s assigned beginning point. This is often due to the fact that segment lengths in the 

agency’s records often do not exactly match the actual lengths in the field. The collected data 

may be made to fit the agency’s official segment lengths by removing some data or stretching the 

data to fit the specified length. At this point, the lengths of segments collected are checked 
Practical Guide for Quality Management of Pavement Condition Data Collection

61

against the master routing file to look for any missing segments. GIS may also be used to plot 

the collected data on the road network shape file to check the accuracy of the segmentation 

process and of the collected latitude and longitude data. 

Database Checks

Before delivery, a series of checks should be conducted on the compiled database. Final checks 

of the database typically include verifying proper format, checking for missing data, and 

screening the entire database for errors. This may involve logic checks and identifying data 

patterns such as a consecutive series of zeroes, null, repeating, or error values, or random values 

that are much larger or smaller than expected (i.e., out of range). Time-series comparisons with 

previous years’ data can also be conducted to identify possible errors in the final database. 

Examples of agency database checks are summarized in table 15. In addition, table 16 provides 

agency examples of typical expected values for individual distress. 

Table 15. Summary of agency data/database checks. 

Agency Data/Database Checks

British Columbia MoTI 

(see Appendix C)

• Data exists for all road segments.

• Traversal definitions for all road segments.

• Data file structure.

• Start and end boundaries for all road segments.

• Lane references and chainages according to the provided data files.

• Null and negative values.

• Minimum and maximum tolerance parameters.

Colorado DOT 

(CDOT 2012) 

• Duplicated records.

• Missing segments.

• Wrong highway limits.

• Missing highways.

• Wrong pavement type. 

• Highway not in network. 

• Wrong raw data value (expected maximum exceeded).

Iowa DOT

(IADOT n.d.) 

• Location description (route, direction, begin/end).

• Missing condition data.

• Number of missing consecutive segments.

Louisiana DOTD 

(see Appendix D) 

• Pavement type and pavement texture from previous year.

• Sudden change in roughness and rut depth. 

• High quantities of distress with low roughness values. 

• High roughness values with low quantities of distress.

• Reasonable maximum extent of distress.

• Segments marked as a construction zone and lane deviation.

• Bridge segments not rated as bridges.

• Sections with longer lengths than specified. 

• Sections with insufficient lead in/lead out pavement length.

• Incomplete data.
Practical Guide for Quality Management of Pavement Condition Data Collection

62

Table 15. Summary of agency data/database checks (continued). 

Agency Data/Database Checks 

Oklahoma DOT

(see Appendix E)

• District, direction, chainage, and event codes.

• Correct data types. 

• Agency-supplied section information. 

• GPS begin point tolerance range and GPS duplicates. 

• Surface collected versus agency-supplied pavement type. 

• Geometric data ranges. 

• Missing video log data.

Pennsylvania DOT

(see Appendix F)

• Duplicate records.

• Verify dates of testing

• Missing location reference data. 

• Non-numeric data in a numeric field. 

• Zero slab/joint counts for concrete pavements.

• Missing segments.

• Construction, bridge, lane deviation, and miscellaneous flags. 

• Matching location data for turn-back and closed-to-traffic roadways.

• Surface type. 

• Condition versus segment length.

Table 16. Agency expected distress values (CDOT 2012; NDOR 2009). 

Distress Colorado

DOT1

Nebraska

DOT2

Oklahoma 

DOT3

IRI 800 in/mi

(12.6 m/km)

+ 190 in/mi

(3 m/km)

20 – 600 in/mi

(0.3 – 9.5 m/km)

Rut depth 1.5 in

(38.1 mm)

+ 0.2 in

(5 mm)

0 – 1.25 in

(0 – 32 mm)

Faulting --- + 0.04 in

(1 mm)

0 – 0.8 in

(0 – 20 mm)

Fatigue cracking 7,000 sq-ft

(650 sq-m) --- ---

Transverse cracking 150 (count) --- ---

Longitudinal cracking 3,000 ft

(914 m) --- ---

Corner breaks 50 (count) --- ---

1 Maximum expected value for a 0.1 mi (0.16 km) control section length. 2 From previous year’s survey. 3 Expected range for a 0.01 mi (0.016 km) pavement segment length. 

Case Study No. 4 provides an example of the quality control process used by the Nebraska DOR 

for semi-automated pavement condition data collection. 
Practical Guide for Quality Management of Pavement Condition Data Collection

63

Case Study No. 4. 

Data Collection Quality Control 

Nebraska Department of Roads 

The Nebraska DOR collects annual pavement condition data using agency staff. Five pavement raters 

(three full-time and two part-time) perform distress ratings from pavement images. Roughness and 

faulting data are collected by agency staff using laser profilers. Rut depth is collected using both a 3-

point laser and an INO laser. If the INO data is considered out of range, the 3-point rut data is used 

(NDOR 2009). 

The QC process includes calibration of the profiler’s laser sensors, accelerometers, and DMI; control 

site testing; real-time system checks; and time-series comparisons.

• Equipment Calibration – Periodic equipment checks and calibration are part of the quality 

management process. When measured from a level reference plane, the data collection 

software automatically calculates a height correction factor. The collection system uses dual 

accelerometers which are also automatically calibrated using the data collection software. The 

DMI is tested periodically over a predetermined distance and correction factors are calculated 

as needed. While in collection mode, an onboard computer performs continuous system 

checks to ensure that all components are operational.

• Control Sites – Control sites (or correlation sections) are used to perform monthly evaluations 

of profilers during data collection. The test sections (typically 1.0 mi [1.6 km] in length) 

represent different surface types and varying levels of roughness and rut depth. Profilers are 

checked individually for repeatability and against each other for reproducibility. A maximum 

variability of ± 5 percent is allowable.

• Global Checks – QC of sensor data also involves checks in the office for duplicates and/or 

gaps in the data and investigation of any rut depth where the difference between left and right 

rut depth is greater than 0.25 in (6 mm). Other global database checks include inspecting for 

out of date data, zeros or blanks, large drops in serviceability index, correct section limits, and 

begin/end points. 

• Sampling – QC of distress ratings involves field checks of about a 10 percent sample of the 

ratings. In addition, all new projects (those constructed in the last year) are checked to ensure 

visual ratings and profile data from the new construction are used.

• Time-Series – A restoration index (RI) is used to describe pavement condition. The RI is 

based on the type, extent, and severity of distresses observed on bituminous pavement 

sections. As part of the QC checks, a time-series comparison is made of the previous year’s 

RI data for each bituminous road section and any change greater than 10 percent is verified. 

Additional time-series checks are made to compare the current and previous year’s IRI, rut 

depth, and faulting data (as applicable). Any changes in IRI greater than 8 in/mi (0.13 m/km), 

changes in faulting greater than 0.06 in (15 mm), or changes in rut depth greater than 1 in (25 

mm) are investigated.
Practical Guide for Quality Management of Pavement Condition Data Collection

64

Corrective Action

The corrective actions taken during QC will vary depending on the type of problem found, the 

suspected or known cause of the issue, and whether the problem is thought to be isolated or 

systemic. Initial calibration and real-time data checks on board the data collection vehicle 

provide immediate feedback on the proper function of equipment. End of day checks also 

uncover problems that can be corrected quickly. Issues related to equipment are typically 

resolved by replacing defective hardware, recalibration, or re-training of personnel in correct 

operation of the equipment. Any data that was collected with faulty or improperly calibrated or 

configured equipment will most often be re-collected but occasionally it is an issue (such as DMI 

calibration) that can be corrected post-collection. 

Data quality issues related to training of distress raters is often discovered during QC 

comparisons (or distress rating audits) of sample distress ratings. The QC plan will often specify 

that the rater be retrained and retested until satisfactory results are achieved. The common 

corrective action taken would be re-rating of batches by a more experienced rater.

Field testing of control and verification sites for QC provides another opportunity to discover 

data quality issues and take immediate corrective actions. Sensor data is compared to both 

reference values and previously collected data to identify any discrepancies in accuracy or 

repeatability. The data collection crew is usually required to cease collection until the cause of 

the problem can be identified. Corrective action often consists of re-collecting any segments 

surveyed since the last successful testing of control or verification sites. If problems with 

distress ratings are found through control, verification, and/or blind site testing, the corrective 

action typically requires retraining of distress raters and re-rating of a batch of data. 

Data quality issues that are discovered during data reduction and final database checks are often 

more difficult to correct. Some issues, such as improper segmentation or incorrect data type, can 

be corrected fairly easily but others, such as missing data, would require the data collection 

vehicle to return to the area for re-collection. The corrective action is usually taken on a case-bycase basis, depending on the nature, severity, extent, and cause of the problem. 
Practical Guide for Quality Management of Pavement Condition Data Collection

65

8. ACCEPTANCE

Introduction

Acceptance activities are performed by the agency to determine if deliverables have met the 

established quality standards. Acceptance testing should be tailored to adequately encompass all 

deliverables and check for format, accuracy, completeness, consistency, and/or other quality 

criteria as appropriate. In the following sections, the most common acceptance checks are 

described in more detail (see Appendix G for QC and acceptance checklist). 

Analysis of Control, Verification, and Blind Site Testing

Periodic testing of control sites is one of the most commonly used QM tools. Control, 

verification, and blind site testing are used for both QC and acceptance. The testing process is 

used during the collection phase to verify continued calibration of equipment and/or methods and

the results of the site testing are used for acceptance by the agency. The site testing data is

compared to the data quality standards for accuracy, repeatability, and/or reproducibility and the 

acceptance criteria applied. 

If the site testing data does not meet criteria, corrective action should be taken. This may include 

equipment recalibration and re-collection of the data since the last successful site testing. In the 

case of distress ratings, it may also include rejection of a batch of data. For this reason, it is 

desirable to conduct frequent site testing so that the quantity of data collected or produced 

between testing is more manageable in case data needs to be re-collected or distresses re-rated. 

Global Database Checks

Upon receiving the final condition database or batches of data/video, the agency typically 

performs a series of checks of the entire database. These checks may be performed as manual 

screening, a set of saved queries executed individually, or an automated or semi-automated series 

of checks for errors. For example, the British Columbia MoTI (2012) conducts a thorough 

manual review of the submitted data to verify completeness, proper file structure, start and end 

points for all segments, and correct lane and length according to inventory. In addition, the data 

is screened for null or negative values and range. Data may be cross-checked against other data 

sources (e.g., inventory), checked for compliance with a required format or for errors related to 

location, completeness, and consistency, or checked for logical ranges for each data item. 

Incorrect data due to poor calibration is more difficult to identify in global database screening 

because the errors are smaller and often form no recognizable pattern.

Data Format

In many cases, data format is the first check performed because the remaining queries may not 

function correctly if the data is not in the proper format. If a database shell was provided, the 

delivered database should still be checked for correct field names and data format (such as a 

number to the correct decimal places, text, and so on). Once correct format has been confirmed

or corrections made, the manual or automated checks or queries of the data can be performed.
Practical Guide for Quality Management of Pavement Condition Data Collection

66

Location Accuracy 

The segments file provided for data collection is used to check location accuracy and 

completeness of the data. A GIS may also be used to plot the data and look for location 

inaccuracies or missing segments (discussed further in Chapter 10). It is desirable to resolve 

location and segmentation discrepancies before proceeding with other data quality checks 

because subsequent checks may use other agency data (such as road inventory or previous years’ 

condition data) that are tied to the LRS. 

Data Completeness

Besides checking for segments that may have been missed during collection, checks are also 

performed to discover any data that was inadvertently not included in the final database. Null or 

default values are sometimes an indication of missing data. Another completeness check that 

may be performed is to compare the total number of events recorded by segment (such as bridge 

counts) to look for missing data that should have been recorded. 

Data Consistency

Data consistency checks are performed to look for data that does not make sense in some way. 

One of the first consistency checks often performed is to compare the surface type recorded 

during data collection to the pavement type. An inconsistency may be due to a treatment 

performed or may indicate a problem with the LRS. A context-based validation test can be used 

to compare distress types recorded versus pavement types (e.g., cracked slabs can only be rated 

on jointed concrete pavements). A type of logic test would be to check for repeat values when 

not normally expected (e.g., exact same IRI value repeated for five consecutive records). 

Another example of a logic test would include checking for the correct type of cross slope (either 

normal or reverse) on left or right horizontal curves. 

Data Range

Global database checks are often performed to screen for out of range data elements. There 

should be a set of default rules established for what values are considered “normal” for each data 

element and what triggers investigation. For example, an IRI less than 30 in/mi (0.5 m/km) or 

greater than 300 in/mi (4.7 m/km) may warrant examination. Data range checks might also 

relate to the rating protocol, for example, the length of raveling cannot be longer than the 

segment length or the area of patching cannot be greater than the segment area. The agency 

should have a defined minimum and maximum expected value for each data item. 

Sampling Checks

Due to the large number of records in most network-level pavement condition databases, 

checking each individual record for quality would not be practical. Sampling of data, 

particularly distress ratings, for QC and/or acceptance testing is a common QM procedure 

adopted by many agencies. Detailed examination of random samples of a portion of the data 

also enables the agency to make an estimation of the likelihood of errors in the whole database.

A sample of distress ratings is taken as representing the quality of the entire batch of data from 

which the sample was taken. Therefore, if problems are found, the entire batch is considered 
Practical Guide for Quality Management of Pavement Condition Data Collection

67

suspect and may be rejected. The agency will often take further samples from the batch to try 

and verify the extent of the problem. 

Sampling Methods

Samples can be random, systematic, stratified, clustered, or some combination of those, all of 

which can be used in QM procedures (Ong, Noureldin, and Sinha 2010). Systematic sampling 

involves selecting a fixed or periodic interval (e.g., the first 500 ft [152 m]). Stratified sampling 

divides the entire database into non-overlapping groups (such as pavement type), and then 

samples (random or non-random) are taken from each group. Cluster sampling is a more 

complex technique where the data are divided into clusters (close to each other), and some data 

are then chosen at random or by some other method. Cluster sampling might be done to save 

travel time if an agency were going to check values in the field. With random sampling, each 

record has an equal chance of being selected, and a random number generator is often used to 

ensure true randomness of the sample. 

Determination of Sample Size

When conducting sampled checks, a key consideration that must be addressed is the size of the 

sample for adequate representation of the population and verification of required measurement 

accuracy. For network-level pavement condition data collection, sample size typically ranges 

from 2 to 10 percent, although this may also be dependent on the size of the network. The 

sample size may be larger for a smaller network but will generally not be less than about 5 

percent. A survey of pavement data collection vendors found that 29 percent of the vendors 

review 2 to 5 percent of the data, another 29 percent of the vendors review 6 to 10 percent, and 

42 percent of the vendors review more than 10 percent of the data as part of their regular QC 

practices (Flintsch and McGhee 2009). Selection of sample size may be based on prior 

experience or using statistical techniques considering the desired accuracy and acceptable degree 

of risk that the data may not be representative of the population. However the sample size is 

selected, minimum sample size requirements should be specified prior to the beginning of data 

collection and in consideration of cost impacts. 

Distress Rating Sampling Checks 

Checks of distress ratings are a manual process in which samples of data are visually inspected 

by the agency (or independent party) for accuracy of the ratings. If production ratings were 

performed by windshield survey, the acceptance checks will usually be done by windshield or 

walking survey. If semi- or fully automated distress reduction methods were used, acceptance 

checks will often be performed by viewing the pavement images. Some software programs 

overlay the distress ratings directly on the pavement image to allow viewing of the image and 

ratings together. The distress ratings in some software packages are color coded for severity 

level and distress type, which facilitates easier review. 

As the distress rating samples are checked, the agency should keep a log of all data checked and 

the results of the inspection. Since distress rating checks are very time-consuming, it is not 

uncommon for the agency to begin sampling checks before data collection/rating is complete. 

Because of this overlap, there is opportunity to re-collect or resurvey any sections with data that 

do not meet quality criteria.
Practical Guide for Quality Management of Pavement Condition Data Collection

68

Other Sampling Checks

Sampling is most commonly used for distress rating checks but can be used to examine any data 

elements. The agency may or may not have actual data or reference materials to compare to the 

sample. For example, a sample of collected horizontal curve radius data may be compared to the 

original construction plans, if available. Samples of GPS coordinates can be checked using 

Google EarthTM mapping service. If reference material is not available, the inspector may view 

video of the road segments and perform logic or consistency checks. 

Video Quality Checks 

Samples of forward-facing and pavement (downward-facing) video are checked for clarity, 

proper exposure, color balance, stitching, and sequencing. An improperly focused or mounted 

camera or debris on the camera housing may cause the clarity of the image to be unacceptable. 

Most video cameras can automatically compensate for changes in levels of light by opening or 

closing the iris, but sometimes quick changes in lighting can cause image problems. Also, if 

video was collected too late in the day or on an overcast day, image quality may be 

compromised. Improper color balance of the images is sometimes found to be a problem. 

Pavement video is often (but not always) illuminated by artificial lighting, such as strobes or 

infrared laser. Improper synchronization of the light source and cameras can cause exposure 

problems. If artificial illumination is not used, the vehicle may cast a shadow on the pavement, 

which impedes viewing of distresses. Many systems use two cameras to collect the left and right 

halves of the lane, and the frames are stitched together to provide the complete lane width. 

Although this process is automated, there may be problems with improperly stitched images. 

Both forward-facing and pavement image cameras must be sequenced and organized properly 

within the video database to play in the correct order. Samples of video batches are usually 

checked with the viewing software to verify correct playback.

Resurveying

Some agencies contract for data collection but also use agency-owned data collection vehicles to 

resurvey samples of the network for data quality evaluation. For instance, the Pennsylvania 

DOT office for Roadway Inventory and Testing Section (RITS) conducts a check of more than 

1,300 mi (2,092 km) (or 5 percent) of the annual survey conducted by the service provider using 

the Pennsylvania DOT-owned automated data collection vehicle (Pennsylvania DOT 2011). 

Pennsylvania DOT collects IRI, rut depth, and distresses on the segments for comparison to the 

service provider’s data. The acceptance practice conducted by the Pennsylvania DOT is further 

described in Case Study No. 5. A complete description of the Pennsylvania DOT QC and 

acceptance process is provided in Appendix F. 
Practical Guide for Quality Management of Pavement Condition Data Collection

69

Case Study No. 5. 

Acceptance Process 

Pennsylvania DOT 

The Pennsylvania DOT contracts for the collection of condition data on approximately 27,000 mi (43,452 

km) of roadway annually (PennDOT 2011). Roughness, rut depth, faulting, GPS, distress ratings, and 

video are all collected concurrently by the service provider. As part of the acceptance process, 

Pennsylvania DOT performs the following evaluations:

• Calibration Sites – Six distress rating calibration sites must be tested by the service provider 

prior to beginning network data collection. To establish ground truth, three Pennsylvania DOT 

raters perform distress ratings and the ratings are averaged. Tolerance for the data is ± 10 percent 

of ground truth and if larger discrepancies are found, Pennsylvania DOT works with the service 

provider to resolve differences in interpretation. 

• Verification Sites – Two roughness and rut depth verification sites are used prior to and during 

data collection. Pennsylvania DOT uses agency profilers to measure roughness and manually 

measures rut depth on the sites. Service provider data is expected to be within ± 10 percent of 

ground truth. To verify repeatability and reproducibility of roughness measurements, each of the 

service provider’s data collection vehicles performs five repeat runs of the roughness sites. After 

the initial verification runs, the roughness sites must be re-tested on a monthly basis during 

collection.

• Blind Sites – In 2008, Pennsylvania DOT began using “blind” sites to assess the service 

provider’s images, roughness data, and distress ratings. Pennsylvania DOT selects segments 

statewide before collection begins and requests the data when the service provider’s weekly 

report indicates a segment has been collected. A total of 100 segments were used as blind sites in 

the 2010 collection. Acceptable variation for accuracy is ±10 percent compared to Pennsylvania 

DOT data. Whenever possible, historical data is also used for comparison. The results of blind 

site testing are analyzed to uncover any bias in the service provider’s distress rating process.

• Sampling – Up to 2.5 percent stratified random sample of the delivered data is selected to 

evaluate quality of distress ratings, roughness, and rut depth data. Pennsylvania DOT uses the 

pavement images to rate distresses and compare to the service provider’s ratings. Individual 

distresses are compared to Pennsylvania DOT ratings and a percent within limits (PWL) criteria 

is applied. A 3-year time-series comparison of distresses and rut depth is performed. In addition, 

5 percent of the annual survey mileage is re-collected by Pennsylvania DOT using in-house 

equipment and staff for comparison to vendor data. 

• Needs Comparison – A maintenance cost analysis is performed on a 5-percent sample to 

compare the needs that result using the service provider’s ratings versus those that result from 

using Pennsylvania DOT ratings. A maintenance allocation program is run using both sets of 

ratings to determine dollar needs for each segment. These are compared to see how closely the 

service provider’s needs match Pennsylvania DOT’s and whether there is a bias toward higher or 

lower severity ratings. Pennsylvania DOT also compares the treatments recommended using the 

two sets of data.

• Global Checks – Batches of data are checked for data format, surface type, segment length, and 

event flags.

Practical Guide for Quality Management of Pavement Condition Data Collection

71

9. QUALITY MANAGEMENT REPORTING

Introduction

QM reporting encompasses documentation of the QC and acceptance procedures performed 

during all phases of data collection. There is an old adage in the quality world: if it wasn’t 

documented, it didn’t happen. Reporting is an important component of the overall QM program 

as it facilitates continuous improvement of the quality process. QM reporting enables the agency 

to refer back to previous reports, keep track of related problems, and take steps to prevent the 

same issues from reoccurring.

The type and amount of reports required will be dependent on the quality processes employed. It 

is recognized that reporting and documentation may be considered tedious, and sometimes of 

little value. However, the importance of proper reporting cannot be overstated. Reporting 

provides a mechanism for continued improvement of the process, through documentation of 

what worked and what did not. Over time, as processes and quality requirements develop and 

mature, the agency will recognize real gains in terms of reduced level of effort and monetary 

savings as a result of documented quality processes and results. For these reasons, an agency 

should give equal consideration to documentation of each step of the process when developing a 

QM process.

Process documentation should be retained in an organized filing system for ready reference in 

the future. This filing system may be paper-based, electronic, or a combination of the two. 

There should be clearly defined roles and responsibilities for the person or persons charged with 

maintaining this file, so that people updating quality reports know where the information resides

and information remains current and complete.

QC Documentation and Reporting

Chapter 7 provided a list of potential considerations for QC. Each of those activities provides an 

opportunity for documentation that the activity occurred, when it occurred, who or what was 

reviewed, the results of the review, follow up actions required, and who completed the review. 

This is minimum critical information in any quality report. Additional information may record 

comments about the process with recommendations for possible changes (e.g., where the defined 

process may not fit the reality of the data collection effort), contributing factors (e.g., inclement 

lighting or weather), or other observations that may be helpful in interpretation of the report.

In addition to the more general items discussed above, the following more specific 

considerations might be included as part of QC reports:

• Equipment and key personnel used during data collection.

• Documentation of initial and continuing calibration/checks/maintenance for field 

equipment, any equipment problems, and corrective actions taken.

• Schedule adherence and the reasons for any changes.

• Documentation of collection procedures and protocols used.
Practical Guide for Quality Management of Pavement Condition Data Collection

72

• Reporting of any variances in standard operating procedures or changes in collection 

methods made in the field.

• Applicable guidance documents.

• Reporting of all control, verification, and blind site testing and results.

• Documentation of all QC activities.

• Analysis of all rater checks and intra- or inter-rater comparisons.

• Log of all quality issues identified through QC activities and corrective actions taken.

• Copies of all correspondences.

In addition to activity-specific reports, consideration should be given to an end-of-data collection

overview report, which summarizes the results of the quality achieved during data collection. 

This report should provide a summary of the tests conducted and the findings as compared to the 

quality metrics established for data collection.

Acceptance Documentation and Reporting

Chapter 8 provided an overview of acceptance processes. Like the QC above, each of the 

activities associated with acceptance is an opportunity for documentation of findings and results. 

The acceptance process typically is the agency’s responsibility, and as such this documentation 

can serve as an audit of the agency’s quality process for conducting the network-level pavement 

condition survey.

The acceptance report may include:

• A description of quality standards and acceptance criteria.

• A description of control, verification, and blind sites and reference values used.

• An analysis of control, verification, and blind site testing results.

• Documentation of all global database checks performed and the results.

• Documentation of all sampling checks and the results.

• Documentation of all other acceptance checks and the results.

• A log of all quality issues identified through acceptance checks and corrective actions

taken.

• Recommendations for improvements.

An example of the LTPP program QM reporting process is provided in Case Study No. 6.
Practical Guide for Quality Management of Pavement Condition Data Collection

73

Case Study No. 6. 

Quality Management Reporting 

FHWA LTPP Program 

The FHWA LTPP program invested significant resources in developing QC systems to address 

the variety of data sources and technologies employed in the program and in the documentation 

of the results of those QC and acceptance systems. With an ultimate goal of providing a 

research-quality data set for ongoing studies, FHWA staff and contractors developed quality 

management plans documenting a variety of quality checks. In general, the major categories of 

QC systems developed include equipment calibration procedures, equipment calibration checks, 

operator training and certification, post data collection reviews, data screening from external 

sources, and formal quality control management procedures. FHWA staff charged regional 

contractors with development of quality control plans that would specify and document efforts 

for quality control. FHWA and the technical support services contractor audited the QC 

completion efforts. FHWA and contractor staff conducted acceptance testing on data before, 

during, and after entry into the database. Each of these processes resulted in forms of 

documentation retained as part of the regional contractor or FHWA files. 

• Equipment calibration – Calibration procedures are defined in protocols developed by 

LTPP, which may reference ASTM or AASHTO specifications. Documentation 

includes daily equipment checks (tires pressures, “bounce tests,” buffer warm-ups, and 

so on), monthly and annual calibrations, problem reports, and daily operation reports.

• Operator training and certification – Documentation of periodic certification and 

recertification for distress surveyors and equipment operators; confirmation of 

procedural reviews semi-annually through regional QC plans; periodic peer review 

audits of equipment, paperwork, and procedures; and operator coordination meetings to 

review updates to protocols, equipment operations, and general subjects related to the 

activity.

• Post data collection reviews – Documentation of logic checks on data returned to the 

regional offices; validation of mathematical summaries on distress surveys; and time 

series data reviews for reasonableness and consistency. 

• Data screening – Review and documentation of data coming in from outside sources, 

including environmental data, traffic data, and inventory data. Much of the 

documentation was provided as part of Data Analysis Operations Feedback Reports, 

which document the nature of the data concern and the process used to address it. 

• Data quality studies – FHWA asked those investigating the data, which could include 

LTPP contractors or external sources, to document data concerns through data analysis 

feedback reports. These reports document the nature of the concern, the process for 

resolution, and the disposition of the issue for the program. 

There is a large body of knowledge about the program available on the FHWA LTPP web-site 

(www.fhwa.dot.gov/pavement/ltpp/index.cfm), with the most relevant document to this 

discussion being the Long-Term Pavement Performance Compliance with Department of 

Transportation Information Dissemination Quality Guidelines (FHWA-HRT-08-065). 
Practical Guide for Quality Management of Pavement Condition Data Collection

74

10. ADDITIONAL QUALITY MANAGEMENT TOOLS

Introduction

The QM tools discussed in this chapter are perhaps less commonly used by agencies and/or 

service providers because of the additional time, cost, or, in some cases, expertise required to 

implement them. However, all have shown value in supplementing the standard QM procedures 

and should be considered for inclusion in a QM plan.

Automated Software Data Checks

Many agencies perform a series of quality checks on the entire database as described in Chapter 

8. This is typically done using a set of queries stored for use with the database. Because of the 

multitude of queries to be run and the large size of most condition databases, a few agencies have 

automated the process either completely or partially. For example, the Colorado DOT (2012) 

uses a computer program to check for duplicate records, missing segments, incorrect pavement 

type, and other errors. The Oklahoma DOT uses a Microsoft Access-based tool that enables the 

user to execute the queries in a logical sequence against smaller subsets (i.e., field districts) of 

the database (Wolters, McGovern, and Hoerner 2006).

Geographic Information Systems

GIS, as used in the context of asset management, are tools designed to integrate data and 

cartography. GIS software provides a platform for examining, visualizing, and managing 

pavement data. The condition survey data elements can be visualized on a map as long as the 

data has been located geographically. For example, GIS can be used to plot the collected data on 

a shape file of the road network to check the accuracy of the segmentation process and the 

collected latitude and longitude data. If a segment has been missed, a faulty beginning point 

assigned, or the data otherwise improperly segmented, it is often readily apparent by visualizing 

the data using the GIS. The ability to examine the data visually is useful in many ways, such as 

comparing data from each side of a divided highway, or comparing radius of curvature with the 

map display of the location.

A newer development in the use of GIS as a QM data tool involves creating keyhole markup 

language (KML) files from the condition and inventory data and importing them for use with a 

browser-based GIS such as Google EarthTM mapping service. The ability to use an Internet 

application to display pavement data onto the road network along with satellite images is proving 

to be very helpful in checking the data.

Time-Series Comparisons

Although not quite as common as other types of data checks, many agencies do perform 

comparisons of data collected in previous years with the current data. Time series comparisons 

enable the analysis of data trends to identify any unexpected changes that may indicate data 

quality issues. For instance, the Nebraska DOR (2009) compares the current and previous year’s 

data and investigates any sections with an increase in faulting or rut depth greater than 0.20 in (5 

mm). New Mexico DOT (2011) examines changes in distress severities and expects to see a 

maximum increase of one level in the highest severity category in 1 year (and no decrease in 

highest severity category). British Columbia MoTI (2012) compares samples of data to previous 
Practical Guide for Quality Management of Pavement Condition Data Collection

75

years’ data for a detailed assessment of changes. The Maryland State Highway Administration 

(MDSHA 2011) investigates any sections with changes in the cracking index greater than 10 

points between years. The Louisiana DOTD (LADOTD 2011) compares current to previous data 

and investigates any areas where pavement conditions (IRI, rut depth, faulting, or distress 

quantities) indicate acute improvement or deterioration. All of these are examples of routine 

time series checks on data that can provide a quick and clear indication of a problem that may 

exist.

Pre-Qualification of Service Providers

In discussions with various highway agencies, there is an increasingly wider requirement for 

service providers to be pre-qualified through control site data collection and submission. Under 

this model, the prospective service providers are required to collect control site data at their own 

expense and within a certain window of time and submit the data in order to be considered for 

the contract. This gives the agency an opportunity to evaluate each service provider’s technical 

capabilities before selection. Agencies may provide a small database shell to facilitate receiving 

the control site data in the proper format. Figure 17 shows an example of rut depth data for one 

0.5-mi (0.80 km) control site from five different prospective service providers. In this manner, a 

visual representation of the prospective service provider results can be compared to the agency 

manual measurements for use in the contract award process.

Figure 17. Control site data for pre-qualification of service providers.

-2.54

0.00

2.54

5.08

7.62

10.16

12.70

15.24

-0.10

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.00 0.10 0.20 0.30 0.40 0.50

Average Rut Depth (in)

Average Rut Depth (mm)

Distance, mi (km)

Provider 1 Provider 2 Provider 3

Provider 4 Provider 5 Agency Manual

(0.16) (0.32) (0.48) (0.64) (0.80)
Practical Guide for Quality Management of Pavement Condition Data Collection

76

Pilot Data Collection

Many agencies begin any new data collection survey with a “pilot” data collection of some small 

percentage of the network but enough of sufficient length to represent the various pavement 

types. The submission of the pilot data enables review of the data format and quality before 

actual production begins. After collection and processing, the data can be reviewed and 

feedback provided to the data collection team on the need to revise data collection or provide

raters with additional training. Once all data has received approval, network-level collection can 

begin.

Independent Verification

Another data verification method that is infrequently used is independent verification. 

Independent verification is an unbiased and independent evaluation of data quality that is 

performed by someone other than the entity that collected or is receiving the data. Independent 

verification is performed in a manner similar to that of agency construction acceptance testing

and its purpose is to verify the effectiveness of the QC process. The independent evaluator 

reviews or resurveys samples of the collected data and compares them to the data quality 

standards.

Independent verification involves a schedule of sampling and inspection by qualified personnel. 

If both data collection and independent verification are performed in-house, the independent 

verification should be done by personnel (and equipment) not involved in the actual data 

collection or distress ratings. The sampling frequency employed for independent verification can 

be a percentage of the total data collected or can be based on the amount of data collected by 

each data collection vehicle or rated by each distress rater. Statistical methods can be used to 

determine the appropriate sample size for independent verification purposes or an agency may 

require that a predetermined sample percentage, such as 5 to 10 percent, be evaluated.

It is essential that the independent verification process detect and report deficiencies in a timely 

manner. When a comparison of production and independent verification data reveals significant 

differences, corrective actions should be implemented.
Practical Guide for Quality Management of Pavement Condition Data Collection

77

11. CONCLUSIONS AND RECOMMENDATIONS

The QM Practical Guide will assist highway or local transportation agencies in developing, 

refining, and/or maintaining a QM process. QM of data collection is a long-term, continuous 

effort that must be updated periodically. A QM plan is the main tool used to guide an agency’s 

systematic application of QM principles. Only if the agency is committed to the process and 

invests the needed resources to ensure the QM plan is implemented and maintained, will the 

effort truly pay off.

The techniques and tools described in the QM Practical Guide represent data collection QM 

practices used by U.S. highway agencies, Canadian provincial agencies, and pavement condition 

data service providers. If an agency does not have a formal QM plan, the QM Practical Guide

will help in developing and implementing such a plan. For agencies that have already adopted a 

QM plan, the Guide’s content can be used to update and refine the current plan.

As one of the first steps, the agency should evaluate their current QC and acceptance procedures. 

Beginning with data quality requirements and continuing through to QM reporting, the agency 

should assess the adequacy and completeness of each phase of its current process. It might help 

to review the quick reference guide provided in Appendix A and look for areas that may be 

missing or improved in the agency. Many examples of agency practices were presented in the 

QM Practical Guide, but if these do not meet an agency’s needs, they should develop their own 

procedures, keeping in mind the resources needed for implementation.

Following the self-assessment, agencies should create and adopt a thorough and formal QM plan 

or refine an existing plan as needed. The QM plan template presented in Appendix B is a 

blueprint for agencies to build upon. Although QM plans are not required, they should be 

developed and used for many reasons. Documentation of the QM requirements, procedures, and 

expectations are a key step in assigning accountability for process implementation and the 

resulting data quality. QM plans help mitigate risk, identify opportunities for innovation and 

improvement, and aid staff in key decisionmaking at all stages of the pavement condition data 

survey. These are just a few of the benefits of maintaining a thorough QM plan.

As an agency begins its formal QM efforts, it should secure buy-in from all staff (agency and 

service provider, when applicable) involved in the pavement condition data survey. Without 

engaging the entire data collection team and developing consensus on the importance of QM, the 

QM effort will be difficult to implement.
Practical Guide for Quality Management of Pavement Condition Data Collection

78

REFERENCES

Adams, T. M., N. A. Koncz, and A. P. Vonderohe. 2001. Guidelines for the Implementation of 

Multimodal Transportation Location Referencing Systems. NCHRP Report 460. Transportation 

Research Board, Washington, DC. Accessed October 18, 2012. 

http://onlinepubs.trb.org/onlinepubs/nchrp/nchrp_rpt_460.pdf.

Alabama Department of Transportation. 2010. Network-Level Pavement Condition Data 

Collection Procedure. ALDOT-414-04. Alabama Department of Transportation, Montgomery, 

AL. Accessed October 18, 2012. 

http://www.dot.state.al.us/mtweb/Testing/testing_manual/doc/pro/ALDOT414.pdf.

American Association of State Highway and Transportation Officials (AASHTO). 2012. 

Standard Practice for Evaluating Faulting of Concrete Pavements. AASHTO R 36-12. 

American Association of State Highway and Transportation Officials, Washington, DC.

American Association of State Highway and Transportation Officials (AASHTO). 2011. 

Standard Practice for Definition of Terms Related to Quality and Statistics as Used in Highway 

Construction. AASHTO R 10-06. American Association of State Highway and Transportation 

Officials, Washington, DC.

American Association of State Highway and Transportation Officials (AASHTO). 2010. 

Standard Specification for Inertial Profiler. AASHTO M 328-10. American Association of 

State Highway and Transportation Officials, Washington, DC.

American Association of State Highway and Transportation Officials (AASHTO). 2010. 

Quantifying Cracks in Asphalt Pavement Surfaces from Collected Images Utilizing Automated 

Methods. AASHTO PP 67-10. American Association of State Highway and Transportation 

Officials, Washington, DC.

American Association of State Highway and Transportation Officials (AASHTO). 2010. 

Collecting Images of Pavement Surfaces for Distress Detection. AASHTO PP 68-10. American 

Association of State Highway and Transportation Officials, Washington, DC.

American Association of State Highway and Transportation Officials (AASHTO). 2010. 

Determining Pavement Deformation Parameters and Cross Slope from Collected Transverse 

Profile. AASHTO PP 69-10. American Association of State Highway and Transportation 

Officials, Washington, DC.

American Association of State Highway and Transportation Officials (AASHTO). 2010. 

Collecting the Transverse Pavement Profile. AASHTO PP 70-10. American Association of 

State Highway and Transportation Officials, Washington, DC.

American Association of State Highway and Transportation Officials (AASHTO). 2010. 

Standard Practice for Measuring Pavement Profile Using a Rod and Level. AASHTO R 40-10. 

American Association of State Highway and Transportation Officials, Washington, DC.

American Association of State Highway and Transportation Officials (AASHTO). 2010. 

Standard Practice for Measuring Pavement Profile Using a Dipstick. AASHTO R 41-05. 

American Association of State Highway and Transportation Officials, Washington, DC.
Practical Guide for Quality Management of Pavement Condition Data Collection

79

American Association of State Highway and Transportation Officials (AASHTO). 2010. 

Standard Practice for Determining Rut Depth in Pavements. AASHTO R 48-10. American 

Association of State Highway and Transportation Officials, Washington, DC.

American Association of State Highway and Transportation Officials (AASHTO). 2010. 

Quantifying Cracks in Asphalt Pavement Surface. AASHTO R 55-10. American Association of 

State Highway and Transportation Officials, Washington, DC.

American Association of State Highway and Transportation Officials (AASHTO). 2010. 

Standard Practice for Certification of Inertial Profiling Systems. AASHTO R 56-10. American 

Association of State Highway and Transportation Officials, Washington, DC.

American Association of State Highway and Transportation Officials (AASHTO). 2010. 

Standard Practice for Operating Inertial Profilers and Evaluating Pavement Profiles. AASHTO 

R 57-10. American Association of State Highway and Transportation Officials, Washington, 

DC.

American Association of State Highway and Transportation Officials (AASHTO). 2007. 

Standard Practice for Quantifying Roughness of Pavements. AASHTO R 43-07. American 

Association of State Highway and Transportation Officials, Washington, DC.

American Association of State Highway and Transportation Officials (AASHTO). 2001a. 

Pavement Management Guide. American Association of State Highway and Transportation 

Officials, Washington, DC.

American Society for Testing and Materials (ASTM). 2008. Standard Practice for Computing 

International Roughness Index of Roads from Longitudinal Profile Measurements. ASTM

E1926. American Society for Testing and Materials, West Conshohocken, PA.

American Society for Testing and Materials (ASTM). 2009. Standard Test Method for 

Measuring the Longitudinal Profile of Vehicular Traveled Surfaces with an Accelerometer 

Established Inertial Profiling Reference. ASTM E950. American Society for Testing and 

Materials, West Conshohocken, PA.

American Society for Testing and Materials (ASTM). 2009. Standard Guide for Network Level 

Pavement Management. ASTM E1166. American Society for Testing and Materials, West 

Conshohocken, PA.

American Society for Testing and Materials (ASTM). 2010. Standard Test Method for 

Measuring Rut Depth of Pavement Surfaces Using a Straightedge. ASTM E1703. American 

Society for Testing and Materials, West Conshohocken, PA.

American Society for Testing and Materials (ASTM). 2011. Standard Test Method for Airport 

Pavement Condition Index Surveys. ASTM D5340. American Society for Testing and 

Materials, West Conshohocken, PA.

American Society for Testing and Materials (ASTM). 2011. Standard Practice for Roads and 

Parking Lots Pavement Condition Index Surveys. ASTM D6433. American Society for Testing 

and Materials, West Conshohocken, PA.
Practical Guide for Quality Management of Pavement Condition Data Collection

80

American Society for Testing and Materials (ASTM). 2011. Standard Test Method for Skid 

Resistance of Paved Surfaces Using a Full-Scale Tire. ASTM E274. American Society for 

Testing and Materials, West Conshohocken, PA.

American Society for Testing and Materials (ASTM). 2011. Standard Guide for Classification 

of Automated Pavement Condition Survey Equipment. ASTM E1656. American Society for 

Testing and Materials, West Conshohocken, PA.

British Columbia Ministry of Transportation and Infrastructure (BCMoTI). 2012. Network 

Level Automated Pavement Surface Condition Surveys. British Columbia Ministry of 

Transportation, Victoria, British Columbia, Canada. Accessed November 19, 2012. 

http://www.th.gov.bc.ca/publications/const_maint/2012_pavement.pdf.

Colorado Department of Transportation (CDOT). 2012. Quality Assurance Protocol for 

Verifying Pavement Management Condition Data. Colorado Department of Transportation, 

Denver, CO. 

Diefenderfer, B. K. 2010. Investigation of the Rolling Wheel Deflectometer as a Network-Level 

Pavement Structural Evaluation Tool. VTRC 10-R5. Virginia Transportation Research Council. 

Charlottesville, VA. Accessed November 19, 2012. 

http://www.virginiadot.org/vtrc/main/online_reports/pdf/10-R5.pdf.

Federal Highway Administration (FHWA). 2000. Variability of Pavement Distress Data from 

Manual Surveys. FHWA-RD-00-160. Federal Highway Administration, McLean, VA. 

Accessed November 19, 2012. http://www.fhwa.dot.gov/pavement/pub_details.cfm?id=9.

Federal Highway Administration (FHWA). 2001. Implementation of GIS-Based Highway 

Safety Analyses: Bridging the Gap. FHWA-RD-01-039. Federal Highway Administration, 

Washington, DC. Accessed October 19, 2012. 

http://www.fhwa.dot.gov/publications/research/safety/1039.pdf.

Federal Highway Administration (FHWA). 2008. Pavement Management Systems Peer 

Exchange Program Report. Federal Highway Administration, Washington, DC. Accessed 

November 19, 2012. https://www.fhwa.dot.gov/asset/pmspeer/pmspeer.pdf.

Federal Highway Administration (FHWA). 2010. Highway Performance Monitoring System 

Field Manual. Federal Highway Administration, Washington, DC. Accessed November 1, 

2012. http://www.fhwa.dot.gov/policyinformation/hpms/fieldmanual/.

Federal Highway Administration (FHWA). 2012. Moving Ahead for Progress in the 21st

Century Act (MAP-21) A Summary of Highway Provisions. Federal Highway Administration, 

Washington, DC. Accessed December 18, 2012. 

http://www.fhwa.dot.gov/map21/docs/map21_summary_hgwy_provisions.pdf.

Fillastre, C. 2012. Pavement Management Data to Support Planning. Presentation at FHWA 

Pavement Management Peer Exchange – Atlanta, GA. Louisiana Department of Transportation, 

Baton Rouge, LA.
Practical Guide for Quality Management of Pavement Condition Data Collection

81

Flintsch, G. and K. K. McGhee. 2009. NCHRP Synthesis 401: Quality Management of 

Pavement Condition Data Collection. Transportation Research Board, Washington, DC. 

Accessed November 19, 2012. http://onlinepubs.trb.org/onlinepubs/nchrp/nchrp_syn_401.pdf.

Ganesan, V. P. K., S. M. Stoffels, J. Arellano, and D. A. Morian. “Use of LTPP Data to Verify 

the Acceptance Limits Developed for PennDOT Pavement Distress Data.” Improving 

Pavements with Long-Term Pavement Performance: Products for Today and Tomorrow. No. 

01036914. Federal Highway Administration, Washington, DC. Accessed December 13, 2012. 

http://www.fhwa.dot.gov/publications/research/infrastructure/pavements/ltpp/06109/.

Goodman, S. 2000. C-LTPP Surface Distress Variability Analysis. Canadian Strategic 

Highway Research Program, Ottawa, Ontario, Canada. Accessed November 19, 2012. 

http://www.cshrp.org/products/surface-distress.pdf.

HTC Infrastructure Management Ltd (HTC). 2002. Location Referencing Systems Guide for 

Road Controlling Authorities. The Association of INGENIUM, Auckland, NZ. Accessed 

October 19, 2012. http://www.lpcb.org/lpcbdownloads/data_collection/2002_rims_location_referencing_guide.pdf.

International Organization for Standardization (ISO). 2000. Quality Management Systems –

Fundamentals and Vocabulary. ISO 9001:2000. International Organization for Standardization, 

Geneva, Switzerland.

International Organization for Standardization (ISO). 2008. International Vocabulary of 

Metrology. International Organization for Standardization, Geneva, Switzerland.

Iowa Department of Transportation (IADOT). n.d. Automated Pavement Distress Data 

Collection Services, Request for Proposal. Iowa Department of Transportation, Ames, IA.

Joint Committee for Guides in Metrology (JCGM). 2008. Evaluation of Measurement Data—

Guide to the Expression of Uncertainty in Measurement. Bureau International des Poids et 

Measures, Sevres, France.

Landers, S., M. Robson, and L. Cowe Falls. 2001. “Development of Quality Assurance and 

Control Procedures for Network Level Contract Pavement Surface Condition Surveys.” 

Proceedings, 5th International Conference on Managing Pavements. University of Washington,

Seattle, WA. Accessed October 18, 2012. 

http://pavementmanagement.org/ICMPfiles/2001069.pdf.

Larson, C. D., N. Sami, and D. Luhr. 2000. “Structured Approach to Managing Quality of 

Pavement Distress Data: Virginia Department of Transportation Experience.” Transportation 

Research Record 1699. Transportation Research Board, Washington DC.

Louisiana Department of Transportation and Development (LADOTD). 2012a. Louisiana 

Cracking and Patching Protocol for Concrete Pavements. Louisiana Department of 

Transportation and Development, Baton Rouge, LA.
Practical Guide for Quality Management of Pavement Condition Data Collection

82

Louisiana Department of Transportation and Development (LADOTD). 2012b. Louisiana 

Cracking and Patching Protocol for Asphalt Surface Pavements. Louisiana Department of 

Transportation and Development, Baton Rouge, LA.

Louisiana Department of Transportation and Development (LADOTD). 2012c. Quality Control 

by Data Collection Vendor and Quality Acceptance by LADOTD. Louisiana Department of 

Transportation and Development, Baton Rouge, LA.

Louisiana Department of Transportation and Development (LADOTD). 2011. Data Collection 

QA-QC and Acceptance. Louisiana Department of Transportation and Development, Baton 

Rouge, LA.

Maryland State Highway Administration (MDSHA). 2011. Pavement Management Data QA 

QC Procedure. Maryland State Highway Administration, Annapolis, MD.

McGhee, K. H. 2004. NCHRP Synthesis 334: Automated Pavement Distress Collection 

Techniques. Transportation Research Board, Washington, DC. Accessed November 19, 2012. 

http://onlinepubs.trb.org/onlinepubs/nchrp/nchrp_syn_334.pdf.

Metropolitan Transportation Commission (MTC). 2002. Pavement Condition Index Distress 

Identification Manual. 3rd Edition. Metropolitan Transportation Commission, Oakland, 

CA. Accessed February 25, 2013. http://www.mtcpms.org/products/index.html.

Metropolitan Transportation Commission (MTC). 1986. Distress Identification Manual for 

Asphalt and Surface Treatment Pavements. 2nd Edition. Metropolitan Transportation 

Commission, Oakland, CA. Accessed November 19, 2012. 

http://www.mtcpms.org/publications/asphalt%20PCI%20book.pdf.

Miller, J. S. and W. Y. Bellinger. 2003. Distress Identification Manual for the Long-Term 

Pavement Performance Program (4th Revised Edition). Federal Highway Administration, 

Washington, DC. Accessed November 19, 2012. 

http://www.tfhrc.gov/pavement/ltpp/reports/03031/03031.pdf.

Minnesota Department of Transportation (MNDOT). 2003. Distress Identification Manual. 

Minnesota Department of Transportation, Maplewood, MN. Accessed November 19, 2012. 

http://www.dot.state.mn.us/materials/manuals/pvmtmgmt/distressmanual.pdf.

Montgomery, D. C., G. C. Runger, and N. F. Hubele. 2010. Engineering Statistics, 5th Edition. 

John Wiley & Sons, Inc., Hoboken, NJ.

Morian, D., S. Stoffels, and D. Frith. 2001. Quality Management of Pavement Performance 

Data. Transportation Research Board, Washington, DC.

Nebraska Department of Roads (NDOR). 2009. Pavement Management Systems. Nebraska 

Department of Roads, Lincoln, NE. accessed November 19, 2012. 

http://www.nebraskatransportation.org/mat-n-tests/pdfs-docs/npms.pdf.
Practical Guide for Quality Management of Pavement Condition Data Collection

83

New Mexico Department of Transportation (NMDOT). 2011. Pavement Distress Data Quality 

Management Plan. New Mexico Department of Transportation, Santa Fe, NM.

North Carolina Department of Transportation (NCDOT). 2010. Pavement Condition Survey 

Manual. North Carolina Department of Transportation, Raleigh, NC. Accessed November 19, 

2012. 

http://www.ncdot.gov/doh/pmu/PavementInfo/pcsman/2010_Pavement_Condition_Survey_Man

ual.pdf.

Oklahoma Department of Transportation (ODOT). 2005. Pavement Management Distress 

Rating Guide. Oklahoma Department of Transportation, Oklahoma City, OK.

Oklahoma Department of Transportation (ODOT). 2009. Evaluation of Pavement Management 

Data Quality. Oklahoma Department of Transportation, Oklahoma City, OK.

Oklahoma Department of Transportation (ODOT). 2010. Request for Proposals. Oklahoma 

Department of Transportation, Oklahoma City, OK.

Ong, G. P., S. Noureldin, and K. C. Sinha. 2010. Automated Pavement Condition Data 

Collection Quality Control, Quality Assurance, and Reliability. Purdue University, West 

Lafayette, IN. Accessed December 2012, 

http://docs.lib.purdue.edu/cgi/viewcontent.cgi?article=2636&context=jtrp.

Oregon Department of Transportation (ODOT). 2010. Pavement Distress Survey Manual. 

Oregon Department of Transportation, Salem, OR. Accessed November 19, 2012. 

http://www.oregon.gov/ODOT/HWY/CONSTRUCTION/docs/pavement/Distress_Survey_Manu

al.pdf?ga=t.

Pennsylvania Department of Transportation (PennDOT). 2010. Automated Pavement Condition 

Survey Field Manual. Publication 336. Pennsylvania Department of Transportation, Harrisburg, 

PA. Accessed November 19, 2012. 

ftp://ftp.dot.state.pa.us/public/PubsForms/Publications/Pub%20336.pdf.

Pennsylvania Department of Transportation (PennDOT). 2011. Systematic Technique to 

Analyze and Manage Pennsylvania Pavements (STAMPP) QA Report. Pennsylvania Department 

of Transportation, Harrisburg, PA.

Sayers, M. W. and S. M. Karamihas. 1998. The Little Book of Profiling: Basic Information 

about Measuring and Interpreting Road Profiles. University of Michigan Transportation 

Research Institute, Ann Arbor, MI. Accessed November 19, 2012. 

http://www.umtri.umich.edu/content/LittleBook98R.pdf.

Shekharan, R., D. Frith, T. Chowdhury, C. Larson, and D. Morian. 2006. The Effects of a 

Comprehensive QA/QC Plan on Pavement Management. Transportation Research Board, 

Washington, DC.
Practical Guide for Quality Management of Pavement Condition Data Collection

84

Texas Department of Transportation (TXDOT). 2010. Pavement Management Information 

System Rater’s Manual. Texas Department of Transportation, Austin, TX. Accessed November 

19, 2012. ftp://ftp.dot.state.tx.us/pub/txdot-info/cst/raters_manual.pdf.

Transportation Association of Canada (TAC). 1997. Pavement Design and Management Guide. 

Transportation Association of Canada, Ottawa, Ontario.

Transportation Research Board (TRB). 2002. Transportation Research Circular E-C037: 

Glossary of Highway Quality Assurance Terms. Transportation Research Board, Washington, 

DC. Accessed November 19, 2012. http://onlinepubs.trb.org/onlinepubs/circulars/ec037.pdf.

University of Wisconsin (UW). Template-Quality Management. Project Management Advisor. 

University of Wisconsin, Department of Information Technology. Accessed August 1, 2012. 

http://www.pma.doit.wisc.edu/templates.html.

Utah Department of Transportation (UDOT). 2009. Pavement Preservation Manual – Part 2, 

Pavement Condition Data. Utah Department of Transportation, Salt Lake City, UT. Accessed 

November 19, 2012. http://www.udot.utah.gov/main/uconowner.gf?n=11034818796417575.

Wang, R. Y. 1998. A Product Perspective on Total Data Quality Management. Vol. 41, No. 2. 

Communications of the Association for Computing Machinery, New York, NY.

Wolters, A. S., G. McGovern, and T. Hoerner. 2006. “Development of a Tool to Assess the 

Quality of Collected Pavement Management Data.” Transportation Research Record No. 1974.

Transportation Research Board, Washington, DC.

Zimmerman, K and B. McKinney. 2012. FHWA Pavement Management Peer Exchange 

Atlanta, Georgia. Unpublished. Federal Highway Administration, Washington, DC.

Zimmerman, K. and B. McKinney. 2011. FHWA Pavement Management Peer Exchange 

Raleigh, North Carolina. Unpublished. Federal Highway Administration, Washington, DC.
Practical Guide for Quality Management of Pavement Condition Data Collection

85

GLOSSARY OF TERMS

AASHO Road Test: Roadway test that was constructed near Ottawa, IL to evaluate pavement 

performance subjected to truck loads with known magnitude and frequency.

AASHTO: American Association of State Highway and Transportation Officials.

Acceptance plan: An agreed-upon method of evaluating the acceptability of the pavement 

condition data (Flintsch and McGhee 2009).

Acceptance testing: The activities required to determine the degree of compliance of the 

pavement data collected with contract requirements (Flintsch and McGhee 2009).

Acceptance: The process whereby all factors used by the agency (i.e., sampling, testing, and 

inspection) are evaluated to determine the degree of compliance with contract requirements and 

to determine the corresponding value for a given product (AASHTO 2011).

Accuracy: The degree to which a measurement, or the mean of a distribution of measurements, 

tends to coincide with the true population mean (AASHTO 2011).

Automated data collection: Process of collecting pavement condition data by the use of 

imaging technologies or other sensor equipment (McGhee 2004).

Automated data processing: The reduction of pavement condition (surface distresses, such as 

cracking and patching, or pavement condition indices, such as IRI) from images or other sensors. 

The process is considered fully automated if the pavement condition (e.g., distress) is identified 

and quantified through techniques that require either no or very minimal human intervention 

(e.g., using digital recognition software capable of recognizing and quantifying cracks on a 

pavement surface) (McGhee 2004).

Bias: An error, constant in direction, that causes a measurement, or the mean of a distribution of 

measurements, to be offset from the true population mean (AASHTO 2011).

Blind site: Control site whose location is unknown to the data collection team.

Calibration – A set of operations that establish, under specified conditions, the relationship 

between values of quantities indicated by a measuring instrument or measuring system, or 

between values represented by a material measure or a reference material, and the corresponding 

values realized by standards (AASHTO 2011).

Certified technician: a technician certified by some agency as proficient in performing certain 

duties (AASHTO 2011).

Cluster sampling: the random selection of clusters (i.e., groups or bundles) from a population. 

For each cluster selected, random sampling of the contained elements may then be performed 

(AASHTO 2011).

Control sites: Control sites are roadway segments whose condition data have typically been 

measured by agency or third party personnel for use as a reference value or “ground truth.” Data 

collected during the production phase is compared against the reference values to verify proper 

collection procedures and continued calibration of the equipment. In this way, control sites are 

used to assess the adequacy of the QC processes.

Corrective action: Improvements/adjustments to an organization's processes taken to eliminate 

causes of non-conformities or other undesirable situations.
Practical Guide for Quality Management of Pavement Condition Data Collection

86

Data processing: Covers all of the activities that are conducted to convert the raw data collected 

in the field surveys to useful information (Flintsch and McGhee 2009).

Distance Measurement Instrument (DMI): A transducer used to determine the longitudinal 

distance that the measurement vehicle has traveled. 

Distress rating: Measurement of the extent and severity of distress (e.g., cracking, patching, 

faulting, and rut depth) present on a roadway surface.

Falling Weight Deflectometer (FWD): An impact load device used to deliver a transient 

impulse load to the pavement surface and measure the resultant pavement response (its 

deflection) by a series of sensors. See ASTM D4694, Standard Test Method for Deflections with 

a Falling-Weight-Type Impulse Load Device.

Federal Highway Administration (FHWA): A division of the U.S. Department of 

Transportation that funds research and oversees the distribution of federal highway funding.

Geographic information system (GIS): System designed to capture, store, manipulate, 

analyze, manage, and present all types of geographical data.

Global Positioning System (GPS): Satellite-based navigation system.

Ground Penetrating Radar (GPR): A testing device that emits short pulses of radio wave 

energy that travel through the pavement structure and creates echoes at boundaries of dissimilar 

materials, such as at an asphalt-base interface. See AASHTO R 37, Standard Practice for 

Application of Ground Penetrating Radar (GPR) to Highways.

Ground truth: See reference value.

Highway Performance Monitoring System (HPMS): A national-level highway information 

system that includes data on the extent, condition, performance, use, and operating 

characteristics of the nation's highways. The HPMS contains administrative and extent of 

system information on all public roads, while information on other characteristics is represented 

in HPMS as a mix of universe and sample data for arterial and collector functional systems. 

Limited information on travel and paved miles is included in summary form for the lowest 

functional systems (FHWA HPMS website –

http://www.fhwa.dot.gov/policyinformation/hpms.cfm).

Independent assurance (IA): Activities that are an unbiased and independent evaluation of all 

the sampling and testing (or inspection) procedures used in the quality assurance program

(AASHTO 2011).

International Roughness Index (IRI): A statistic used to estimate the amount of roughness in a 

measured longitudinal profile. The IRI is computed from a single longitudinal profile using a 

quarter-car simulation. See ASTM E1926, Standard Practice for Computing International 

Roughness Index of Roads from Longitudinal Profile Measurements.

Inter-rater reliability: Competent raters using the same protocols on the same roadway 

sections, get the same results. It is also known as cross-rater reliability.

Location referencing method (LRM): A location reference method consists of a mechanism to 

find and state the address of a point by referencing it to a known point. Its purpose is to

communicate the location of a point through an address (TRB 1974).

Location referencing system (LRS): The total set of procedures for determining and retaining 

a record of specific points along a roadway. The system includes the location referencing 
Practical Guide for Quality Management of Pavement Condition Data Collection

87

method(s), together with the procedures for storing, maintaining, and retrieving location 

information about points and segments on the highways (TRB 1974).

Logic check: Consistency between the rated distress type and the pavement type (i.e., assuring 

that the rated distress matches the pavement type).

Macrotexture: A surface texture defined by wavelengths of 0.02 to 2 in (0.5 to 50 mm) and 

vertical amplitudes between 0.005 and 0.8 in (0.13 to 20 mm). Defined by mixture properties 

(shape, size, and aggregate gradation) of an asphalt paving material and finishing/texture (depth, 

width, spacing, and direction of tining/grooving) of a concrete pavement material.

Manual data collection: Pavement condition data collection through processes where people 

are directly involved in the observation or measurement of pavement properties without the 

benefit of automated equipment (e.g., visual surveys and faultmeters) (McGhee 2004).

Microtexture: A surface texture defined by wavelengths less than 0.02 in (0.5 mm) and vertical 

amplitudes between 0.04 and 20 mils. Defined by the surface properties of the aggregate 

particles.

National Highway System (NHS): Consists of roadways important to the nation's economy, 

defense, and mobility (FHWA NHS website –

http://www.fhwa.dot.gov/planning/national_highway_system/). 

Network-level data: Data supporting pavement management decisions on a roadway network 

or system basis (Flintsch and McGhee 2009).

Pavement Condition Index (PCI): A numerical rating resulting from a pavement condition 

survey that represents the severity of surface distresses. See ASTM D6433, Standard Practice 

for Roads and Parking Lots Pavement Condition Index Surveys.

Pavement condition indicator: A measure of the condition of an existing pavement section at a 

particular point in time. This indicator may be a specific measure of a pavement condition 

characteristic (e.g., smoothness or cracking severity and/or extent) or an index defined for a 

single distress (e.g., cracking), for multiple distresses (e.g., PCI), or for the overall pavement 

condition (Flintsch and McGhee 2009).

Pavement condition: An evaluation of the degree of deterioration and/or quality of service of 

an existing pavement section at a particular point in time, either from an engineering or user 

(driver) perspective. The condition as it is perceived by the user is often referred to as functional 

condition. The estimated ability of the pavement to carry the load is referred to as structural 

condition (Flintsch and McGhee 2009).

Pavement performance: The history of pavement condition indicators over time or with 

increasing axle load applications (TRB 2002).

Percent within limits (PWL): The percentage of the lot falling above the lower specification 

limit (LSL), beneath the upper specification limit (USL), or between the USL and LSL

(AASHTO 2011).

Precision: The degree of agreement among a randomly selected series of measurements; or the 

degree to which tests or measurements on identical samples tend to produce the same results

(AASHTO 2011).

Present Serviceability Index (PSI): An index derived by formula for estimating the 

serviceability rating from measurements of physical features of the pavement.
Practical Guide for Quality Management of Pavement Condition Data Collection

88

Present Serviceability Rating (PSR): A definition of pavement serviceability based on 

individual observations.

Project-level data: Data supporting pavement management decisions on a discrete project or 

roadway segment basis (Flintsch and McGhee 2009).

Quality audits: The process of systematic examination of a quality system carried out by an 

internal or external quality auditor or an audit team. It is a key element in the ISO quality system 

standard to verify that the institution has clearly defined internal quality monitoring procedures 

linked to effective action (Flintsch and McGhee 2009).

Quality control (QC): The system used by a contractor to monitor, assess, and adjust its 

production or placement processes to ensure that the final product will meet the specified level of 

quality. Quality control includes sampling, testing, inspection and corrective action (where 

required) to maintain continuous control of a production or placement process (AASHTO 2011).

Quality control plan: A document that describes the process to be followed for delivering the 

level of pavement condition data quality required. This plan typically includes data quality 

objectives (precision, accuracy, completeness, etc.), organization and responsibility, sampling 

procedures, equipment requirements (calibration, verification, etc.), processing of the QC data, 

statistical analysis to be conducted, reporting, documentation of potential problems, and remedial 

solutions (Flintsch and McGhee 2009).

Quality management plan: A document that specifies the quality management procedures and 

resources that will be used and how the process will be implemented and assessed for 

effectiveness (adapted from ISO 2000).

Quality system: The organizational structure, procedures, processes, and resources needed to 

implement QM to meet the quality objectives (Flintsch and McGhee 2009).

Quality: The degree of excellence of a product or service; the degree to which a product or 

service satisfies the needs of a specific customer; or the degree to which a product or service 

conforms with a given requirement (AASHTO 2011). 

Random sample: sample in which each increment in the lot has an equal probability of being 

chosen (AASHTO 2011).

Reference value: A value that serves as an agreed-upon reference for comparison, and which is 

derived as a theoretical or established value, based on scientific principles, an assigned or 

certified value, based on experimental work of some national or international organization, or a 

consensus or certified value, based on collaborative experimental work under the auspices of a 

scientific or engineering group (AASHTO 2011). Reference value is also known as ground 

truth.

Repeatability: Degree of variation among the results obtained by the same operator repeating a 

test on the same material. The term repeatability is therefore used to designate test precision 

under a single operator (AASHTO 2011).

Reproducibility: Degree of variation among the test results obtained by different operators 

performing the same test on the same material (AASHTO 2011).

Resolution: The smallest change in a quantity being measured that causes a perceptible change 

in the corresponding indication (ICO 2008).
Practical Guide for Quality Management of Pavement Condition Data Collection

89

Semi-automated data collection/processing: Process of collecting pavement condition data 

using imaging technologies or other sensor equipment but involving significant human input 

during the processing and/or recording of the data (Flintsch and McGhee 2009).

Stratified sampling: A sampling procedure whereby samples are randomly obtained from each 

sublot (AASHTO 2011). When subpopulations within an overall population vary, it is 

advantageous to sample each subpopulation (stratum) independently. Stratification is the process

of dividing members of the population into homogeneous subgroups before sampling.

Systematic sampling: Statistical method involving the selection of elements from an ordered 

sampling frame. The most common form of systematic sampling is an equal-probability method. 

In this approach, progression through the list is treated circularly, with a return to the top once 

the end of the list is passed.

Time-history: A set of successive periodic measurements of pavement condition over time on 

the same roadway sections. This time-history can be used to determine pavement performance 

(Flintsch and McGhee 2009).

Validation: The mathematical comparison of two independently obtained sets of data (e.g., 

agency acceptance data vs. contractor data) to determine whether it can be assumed they came 

from the same population (AASHTO 2011).

Practical Guide for Quality Management of Pavement Condition Data Collection

91

 

APPENDIX A. QUICK REFERENCE TO QUALITY

MANAGEMENT PLAN

Practical Guide for Quality Management of Pavement Condition Data Collection

93

Figure A-1. Quick reference to QM plan

Data Quality Definition

• Resolution

• Accuracy

• Repeatability

• Format

• Acceptance Criteria

• Corrective actions

Define Rating & 

Collection Protocols

• Distress rating protocols

• Sensor data protocols

• Other data items or video

Personnel Training

• Field technician training

• Rater training and

calibration

Equipment Setup & 

Calibration

• Protocols

• Calibration

• Verification

Control , Blind, 

Verification Site Setup

• Select and mark sites

• Reference values

Control Site Testing

• Accuracy

• Repeatability

Quality Control 

• Real-time & daily checks 

of data & equipment

• Control, verification, blind

site testing

• Checks for completeness

and reasonableness

• Inter-rater comparisons

Quality Acceptance 

Checks

• Global database checks

• Distress sampling checks

• Other checks – GIS, 

time-series, and so on

Quality Monitoring 

Checks

• Monitor control, blind,

verification site testing

• Check video samples

• Check distress samples

• Check other data 

samples

Quality Acceptance 

Checks

• Global database checks

• Distress sampling checks

• Other checks – GIS,

time-series, and so on

Pre-Production Production Post-Production

Agency/Pavement 

Management

Data Collection Team Independent Party

Quality Acceptance 

Checks

• Global database checks

• Distress sampling checks

• Other checks – GIS, 

time-series, and so on

Routing & Scheduling

Corrective Actions

• Re-survey

• Re-rate

Quality Reporting

Quality Reporting

Corrective Actions Quality Reporting

• Re-survey

• Re-rate
Practical Guide for Quality Management of Pavement Condition Data Collection

94

APPENDIX B. DATA QUALITY MANAGEMENT PLAN TEMPLATE

Adapted from the University of Wisconsin,

Template-Quality Management (UW 2012)
Practical Guide for Quality Management of Pavement Condition Data Collection

95

Network-Level Pavement Condition Data Collection

Quality Management Plan

Agency Name

Prepared By: J.R. Smith – Pavement Management Engineer

Date: July 1, 2012

Version No: 1.0
Practical Guide for Quality Management of Pavement Condition Data Collection

96

Document Change Control

The following is the document control for revisions to this document.

Version 

Number Date of Issue Author(s) Brief Description of Change 

0.9 11/15/2011 J.R. Smith Preliminary draft

1.0 01/27/2012 J.R. Smith Updated for 2012 deliverables

Definitions

The following are definitions of terms, abbreviations, and acronyms used in this document.

Term Definition
Practical Guide for Quality Management of Pavement Condition Data Collection

97

TABLE OF CONTENTS

1. Quality Management Approach.

2. Deliverables, Protocols, and Quality Standards.

3. Quality Control.

4. Acceptance.

5. Quality Team Roles and Responsibilities.

6. Quality Reporting Plan.

7. Acceptance of QM Plan.
Practical Guide for Quality Management of Pavement Condition Data Collection

98

1. QUALITY MANAGEMENT APPROACH

The purpose of managing quality is to validate that the deliverables are completed with an 

acceptable level of quality. Quality management (QM) assures the quality of the data collection

deliverables and describes the processes and procedures to be used for ensuring quality.

The QM plan identifies key activities, processes, and procedures for ensuring quality. Below is a 

brief explanation of each of the sections of the QM plan that follow.

Section 2.

Deliverables, Protocols, 

and Quality Standards

The data collection deliverables subject to quality review, protocols 

used to collect, and quality standards that are the measures used to 

determine a successful outcome for a deliverable. The criteria to 

describe when each deliverable is considered complete and correct 

are defined by the pavement management engineer. Deliverables are 

evaluated against these criteria before they are formally approved.

Section 3.

Quality Control (QC)

The QC activities that monitor, provide feedback, and verify that the 

data collection deliverables meet the defined quality standards.

Section 4.

Acceptance

The acceptance testing that will be used to determine if quality 

criteria are met and corrective actions that will be taken for any 

deliverables not meeting criteria.

Section 5.

Quality Team Roles and 

Responsibilities

The quality-related responsibilities of the data collection team.

Section 6.

Quality Reporting Plan

The documentation of all QM activities―including quality standards, 

QC, acceptance, and corrective actions―and the format of the final 

QM report.

Section 7.

Acceptance of QM Plan Signature page for acceptance of the QM Plan.
Practical Guide for Quality Management of Pavement Condition Data Collection

99

2. DELIVERABLES, PROTOCOLS, AND QUALITY 

STANDARDS

Note: this information is further described in Chapter 3 of the QM Guide.

The key deliverables, protocols used for collection, and associated quality standards are 

described below. Quality standards define, when applicable, the resolution, accuracy, and 

repeatability or other standards that will be used to determine the quality of each deliverable. 

See Section 4 for the Acceptance Testing Plan.

Deliverable Protocols Resolution

Accuracy 

(compared to 

reference value)

Repeatability 

(for three repeat 

runs)

IRI (left, right, and 

average) AASHTO 1 in/mi ± 5 percent ± 5 percent

Rut depth (average and 

maximum) AASHTO 0.01 in ± 0.06 in ± 0.06 in

Faulting (average of 

faults over 0.2 in) AASHTO 0.01 in ± 0.06 in ± 0.06 in

GPS (latitude and 

longitude) N/A 0.00001 degree ± 0.00005

degree

± 0.00005

degree

Cross slope N/A 0.1 percent ± 0.5 percent ± 0.5 percent

Longitudinal grade N/A 0.1 percent ± 0.5 percent ± 0.5 percent

Radius of curvature N/A 1 ft ± 10 percent ± 10 percent

Distress ratings

Agency distress 

rating manual

(2009)

Varies ± 10 percent N/A

Location of segment N/A N/A

All assigned 

segments 

surveyed & 

assigned correct 

location

N/A

Segment begin point N/A 0.01 mi ± 0.05 mi N/A

Panoramic images N/A N/A

Signs legible, 

proper exposure 

and color balance

N/A

Pavement images N/A N/A

1/8 in wide 

cracking visible 

on asphalt and 

concrete 

pavements

N/A
Practical Guide for Quality Management of Pavement Condition Data Collection

100

3. QUALITY CONTROL

Note: this information is further described in Chapter 7 of the QM Guide.

The focus of QC is on data collection deliverables and processes. QC monitors the deliverables 

to verify that they are of acceptable quality and are complete and correct. The following table 

identifies:

• The major deliverables that will be tested for satisfactory quality level.

• The quality expectations for the deliverables.

• The QC activities that will be executed to control and monitor the quality of the 

deliverables.

• How often or when the QC activities will be performed.

Deliverable

Quality 

Expectations QC Activity Frequency/Interval

IRI, rut depth, 

faulting, GPS 

coordinates, cross 

slope, longitudinal 

grade, horizontal and 

vertical curves

95 percent

compliance with 

standards

Initial equipment configuration, 

calibration, verification Pre-collection

Daily equipment checks and

monitor real-time Daily

End of day data review Daily

Control, blind, or verification 

testing Weekly

Inspect uploaded data samples Daily

Inspect processed data Daily

Final data review Prior to delivery

Distress ratings

95 percent

compliance with 

standards

Initial rater training Pre-collection

Control site rating calibration Pre-collection

Intra-rater checks Weekly

Inter-rater checks Weekly

Final data review Prior to delivery
Practical Guide for Quality Management of Pavement Condition Data Collection

101

Deliverable

Quality 

Expectations QC Activity Frequency/Interval

Location of segment

and begin point

100 percent

compliance with 

standards

Mileage review Daily

Comparison with the master route 

file Weekly

GIS comparison Prior to delivery

Final data review Prior to delivery

Panoramic and 

pavement images

98 percent

compliance with 

standards of each 

control section and 

not more than 5 

consecutive 

images failing to 

meet criteria

Startup checks, real-time 

monitoring, and field review Daily

Uploaded samples review Weekly

Final review Prior to delivery
Practical Guide for Quality Management of Pavement Condition Data Collection

102

4. ACCEPTANCE

Note: this information is further described in Chapter 8 of the QM Guide.

The focus of acceptance is to validate that deliverables meet the established quality standards. 

Following is a description of acceptance testing, the frequency to be performed, and corrective 

actions for items that fail to meet criteria.

Deliverable

Acceptance 

(Percent 

Within Limits) Acceptance Testing & Frequency

Action if Criteria 

Not Met

IRI, rut depth, 

faulting, cross 

slope, 

longitudinal 

grade, 

horizontal and 

vertical curves

95 percent

Weekly control, verification, and blind site 

testing. Global database check for range, 

consistency, logic, and completeness and 

inspection of all suspect data. 5 to 10

percent sample inspection upon delivery. 

Use of GIS for further inspection.

Reject deliverable; 

data must be recollected.

Distress ratings 95 percent

Global database check for consistency, 

logic, completeness. 5 to 10 percent

sample inspection upon delivery.

Return deliverable 

for correction.

GPS

coordinates 100 percent

Weekly control, verification, and blind site 

testing. Plot on base map using GIS upon 

delivery.

Return deliverable 

for correction.

Location of 

segment and 

segment begin 

point

100 percent

Plot on base map using GIS. Global 

database check of accuracy and 

completeness.

Return deliverable 

for correction.

Panoramic and 

pavement 

images

98 percent of 

each control 

section and not 

more than 5 

consecutive 

images failing 

to meet criteria

Weekly inspection of control, blind, or 

verification site video. 5 to 10 percent

sample inspection upon delivery.

Reject deliverable; 

images must be recollected.
Practical Guide for Quality Management of Pavement Condition Data Collection

103

5. QUALITY TEAM ROLES & RESPONSIBILITIES

The following identifies the quality-related responsibilities of the data collection team and lists

specific quality responsibilities.

Team Role Assigned Resource Quality Management Responsibilities

Agency Manager J. R. Smith – Pavement 

Mgt. Engineer

• Set quality standards, acceptance criteria, and 

corrective actions. 

• Approve each deliverable per quality 

standards.

• Approve resolution of quality issues.

• Assess effectiveness of QM procedures.

• Recommend improvements to quality 

processes.

Agency Assistant 

Manager

A. T. Bell –

Transportation Specialist

• Communicate weekly with data collection 

manager.

• Submit acceptance exceptions log to data 

collection team.

• Supervise manual measurement of control, 

verification, and blind sites. 

• Establish reference values with data 

collection team.

• Monitor schedule adherence.

• Supervise acceptance checks.

• Monitor resolution of quality exceptions 

reported to data collection team.

• Prepare QM report.

Agency Staff

B. Wilson, S. Davis -

Transportation 

Technicians

• Observe and maintain records of control, 

verification, bind site testing. Analyze and 

document results.

• Perform data and video acceptance checks

and document results.

• Perform GIS checks and document results.

• Maintain acceptance log and submit quality 

exceptions to agency assistant manager. 

Data Collection 

Manager D. L. Jones

• Assure deliverables meet broad set of data 

quality requirements.

• Communicate weekly with agency assistant 

manager.

• Assure quality issue resolution and report 

results to agency assistant manager.
Practical Guide for Quality Management of Pavement Condition Data Collection

104

Team Role Assigned Resource Quality Management Responsibilities

Quality Manager R. M. Williams

• Assure practice of QC measures in QM plan.

• Assure proper protocols used.

• Assure training plan addresses all personnel 

skill levels.

• Assure reviews by Distress Rating Lead, 

Data Reduction Lead, and Video Lead.

• Assure performance of all quality audits and 

reporting of all data quality exceptions using 

QC log.

• Assure correction of all quality issues and 

changes in procedures as needed. 

• Perform and document final deliverables 

quality review.

• Compile documentation of all QC activities.

Equipment Manager J. C. Adams • Assure and document initial equipment 

configuration, calibration, and verification.

Field Crew Lead M. B. Jones

• Perform daily and/or periodic equipment 

start-up checks, tests, inspections, and 

calibrations.

• Perform daily review of data logs and video 

samples.

• Assure real-time monitoring of data and

video quality.

• Assure performance of weekly control, 

verification, or blind site testing.

• Assure documentation of all field QM

activities and reporting of any problems using

QC log.

Distress Rating Lead C. D. McGee

• Perform and document initial rater training 

and assure raters adequately trained in 

protocols.

• Document testing of raters on initial control 

site calibration.

• Perform and document quality audits, 

including intra- and inter-rater checks. 

Report any problems using QC log.

• Perform retraining as needed.
Practical Guide for Quality Management of Pavement Condition Data Collection

105

Team Role Assigned Resource Quality Management Responsibilities

Data Reduction Lead F. V. Ross

• Perform and document checks of total 

mileage, segment lengths, and comparison 

with master route file. 

• Assure and document GIS checks of segment 

location and completeness. 

• Document quality audits of uploaded and 

processed data. Report any problems using 

QC log.
Practical Guide for Quality Management of Pavement Condition Data Collection

106

6. QUALITY REPORTING PLAN

Note: this information is further described in Chapter 9 of the QM Guide.

The data collection manager will monitor quality through QC activities and report data quality

exceptions as part of weekly status reporting, or more frequently if conditions warrant. Quality 

is monitored through acceptance testing, and quality issues are reported to the data collection 

team as soon as issues are discovered.

The QC log is used by the data collection team to itemize, document, and track to closure items 

reported through QC process.

QC Log

ID Number

Review 

Date

Deliverable

Reviewed

Location 

Information Findings Resolution

Resolution 

Date

QC-1

QC-2

QC-3

QC-4

The acceptance log is used by the pavement management engineer or independent assurer to 

itemize, document, and track to closure items reported through the acceptance process.

Acceptance Log

ID Number

Review 

Date

Deliverable

Reviewed

Location 

Information Findings Resolution

Resolution 

Date

Accept-1

Accept-2

Final QM Reporting

Data Collection Team – Upon delivery of the final database and other deliverables, the data 

collection team provides a copy of the QC logs, a summary of scope and schedule (including any 

deviations from the planned schedule), a list of the collection vehicles and personnel used on the 

project, documentation of equipment calibration and maintenance, results of all control, 

verification, and blind site testing, and documentation of other problems encountered (not listed 

on the QC log) and corrective actions taken.

Pavement Management Engineer – Upon acceptance of the final database and all other 

deliverables, the Pavement Management Engineer prepares a draft Quality Management Report 
Practical Guide for Quality Management of Pavement Condition Data Collection

107

and when applicable, provides a copy to the service provider (who reviews and provides 

feedback). This report will include a summary of scope and schedule, description of control, 

verification, blind site testing (including reference values and analysis of results), description of 

all global and sampling tests performed and the results, and recommendations for improvement.
Practical Guide for Quality Management of Pavement Condition Data Collection

108

7. AGENCY & DATA COLLECTOR QM PLAN ACCEPTANCE

Quality Management Plan accepted by the Agency Manager:

__________________________________________ Date: ________________________

Agency Manager Name & Title

Quality Management Plan accepted by the Data Collection Manager:

__________________________________________ Date: ________________________

Data Collection Manager Name & Title
Practical Guide for Quality Management of Pavement Condition Data Collection

109

APPENDIX C. CASE STUDY— BRITISH COLUMBIA MOTI
Practical Guide for Quality Management of Pavement Condition Data Collection

110

Introduction

The British Columbia MoTI has contracted automated network-level pavement surface condition 

surveys since 1993 (Landers, Robson, and Cowe Falls 2001). Automated condition surveys 

include surface distress, rutting, and profile measurements, as well as collecting right-of-way 

(ROW) video images. In cooperation with service providers, the British Columbia MoTI QC

procedures were developed so that potential issues could be addressed in real time, thus 

providing a more rapid response during data collection. In addition, the British Columbia MoTI

QC procedures are intended to provide a realistic evaluation of the service provider’s 

capabilities, while minimizing the burden to the Ministry for implementing and monitoring the 

process (Landers, Robson, and Cowe Falls 2001).

Data Collection

The British Columbia MoTI Pavement Surface Condition Rating Manual (BCMoTI 2012) 

documents the technical specifications that guide the data collection and QC requirements for 

high speed network-level pavement surface condition surveys (BCMoTI 2012). The Manual was 

originally released in 1994 and subsequently updated in 2002 and 2009 to include field 

experience and input from data collection service providers. The fourth release of the Manual in 

2012 updated current practices and the application of the rating system to support Ministry asset 

management needs.

Pavement condition data is collected on approximately 25,000 lane-mi (40,000 lane-km) on its 

main highway network. Condition data is averaged and reported over 164 ft (50 m) intervals. 

Collected condition data are shown in table C-1.

Table C-1. Condition data collected by the British Columbia MoTI (BCMoTI 2012).

Category Distress Types

Cracking

• Longitudinal wheel path cracking

• Longitudinal joint cracking

• Pavement edge cracking

• Transverse cracking

• Meandering longitudinal cracking

• Alligator cracking

Deformation • Rutting (calculated from rut depth profile data)

Defects • Bleeding

• Potholes

Roughness • International Roughness Index (IRI)

Quality Control Requirements

The British Columbia MoTI’s QC program is divided into two phases: (1) initial QC where the 

service providers’ methods and equipment are initially calibrated and (2) production acceptane

where the survey is monitored to ensure continuing compliance using blind verification sites.
Practical Guide for Quality Management of Pavement Condition Data Collection

111

Initial Quality Control

The initial QC tests have two objectives. The first objective is to achieve concordance between 

the service provider and the Ministry distress nomenclature and to develop agreement on the 

severity and extent levels described in the British Columbia MoTI Surface Distress Rating 

System Manual (BCMoTI 2012). The second objective is to test field safety procedures and to 

determine that all service provider instrumentation is operating properly prior to receiving 

authorization to commence data collection.

Control Site Selection

Four control sites are selected for the initial QC evaluation. The control sites are 2,460 ft (750 

m) in length, which includes an 820 ft (250 m) lead-in. The sites exhibit a representative variety 

of distress types, a range of pavement deterioration levels, and surface types with the intent of 

being representative of the actual network-level survey conditions. The prior year’s survey data, 

combined with field reconnaissance, are used to select the sites. The same control sites are used 

for subsequent QC testing if there has been no rehabilitation in the intervening year. If possible, 

control sites are selected in close proximity of each other to facilitate the efficiency of automated 

data collection.

Advance Manual Surveys

Manual surface distress surveys are performed according to the Ministry’s rating manual,

including crack mapping for each 164-ft (50 m) pavement segment interval. Rut depth is 

determined by taking manual transverse profile measurements in each wheel path at 33-ft (10 m) 

intervals using a rut-measurement gauge (figure C-1). A pavement profile survey is conducted 

using a Class I profiler, conforming to ASTM E 950, to establish the true longitudinal profile as 

shown in figure C-2. Multiple passes of the Class I profiler are conducted to ensure consistent 

and reliable results.

On-Site Review

The purpose of the on-site review is to ensure that the service provider is proficient with the 

British Columbia MoTI surface distress rating methodology. This review requires the service 

provider to complete a pavement distress survey at the start-up site and to summarize the 

identified distresses. The service provider survey results are compared to the manual ratings 

collected by the agency to assess the distress rater's findings and keyboarding ability as well as 

data processing algorithms. The service provider and the British Columbia MoTI personnel then 

walk the test site comparing the semi-automated results to the manual ratings. Any discrepancies

are discussed and resolved between the service provider and British Columbia MoTI personnel.

Surface Distress Rating Tests―Windshield Survey

Windshield distress surveys are conducted at all four test sites, and the survey vehicle completes 

a series of five runs over each site. The distress ratings are generated at 164-ft (50 m) intervals 

and compared to the manual values for each run. The distress types evaluated during the on-site 

review are included in the distress severity and rating form, an example of which is shown in 

table C-2.
Practical Guide for Quality Management of Pavement Condition Data Collection

112

Figure C-1. Rut depth survey (BCMoTI 2012).

Figure C-2. Class I roughness survey (BCMoTI 2012).
Practical Guide for Quality Management of Pavement Condition Data Collection

113

Table C-2. Example of distress severity and rating form (BCMoTI 2012).

Chainage

Longitudinal Wheel 

Path Cracking

(LWT)

Longitudinal Joint 

Cracking (LJC)

Pavement Edge 

Cracking (PEC)

Transverse Cracking

(TC)

Meandering 

Longitudinal 

Cracking (MLC)

Alligator Cracking

(AC)

Potholes (POT)

Distortion (DST)

Bleeding (BLD)

S1 D2 S D S D S D S D S D S D S D S D

0 – 164 ft

(0 – 50 m) 1 3 2 3

164 – 328 ft

(50 – 100 m) 1 3 2 3

328 – 492 ft

(100 – 150 m) 1 3 2 3

492 – 656 ft

(150 – 200 m) 2 3

656 – 820 ft

(200 – 250 m) 2 3

820 – 984 ft

(250 – 300 m) 2 3

984 – 1148 ft

(300 – 350 m) 3 5 2 3 2 2

1148 – 1312 ft

(350 – 400 m) 3 5 1 2

1312 – 1476 ft

(400 – 450 m) 3 5 3 3 1 2

1476 – 1640 ft

(450 – 500 m) 2 5 3 3

1 Severity: 1 = low, 2 = moderate, and 3 = high. 2 Density: 1 = few (< 10 percent), 2 = intermediate (10 to 20 percent), 3 = frequent (20 to 50 percent),

4 = extensive (50 to 80 percent), and 5 = throughout (80 to 100 percent).

Three criteria are used to assess the service providers’ surface distress rating ability and include 

(BCMoTI 2012):

• Pavement Distress Index (PDI)―calculated using a modified version of the ASTM 

D6433 PCI value. The PDI value (0 to 10 scale) is used to compare the service-provider

survey results to the manual survey results and assess the repeatability of the serviceprovider’s vehicle for each of the five passes.

• Keystroke totals―compares the distress severity and extent (for each distress type) of

the service provider survey results to the British Columbia MoTI manual surveys. This 

comparison is conducted at a very detailed level and is used as a diagnostic tool to assess 

the rating accuracy and to highlight discrepancies. Each distress severity/extent 

combination (three levels of severity and five levels of extent) is analyzed for:

̶ A particular distress type that is being rated too severely.
Practical Guide for Quality Management of Pavement Condition Data Collection

114

̶ A particular distress that is being rated at a different density level.

̶ A particular distress that is missed altogether.

• Kappa Statistic―used to evaluate the level of agreement between the manual 

benchmark survey and the automated rating for surface distress. The Cohen's weighted 

Kappa Statistic is used to observe the variability between multiple distress datasets (see 

equation 1).

K = 1 − ∑ ∑ wijxij

k

j=1 ki=1

∑ ∑ wijmij

k

j=1 ki=1

(1)

where:

K = Kappa Statistic

k = number of codes

wij = weighted value

xij = observed value

mij = expected value

The Kappa Statistic allows for the introduction of weightings for different distress types, 

severities, and densities depending upon the agency practices. The range of the Kappa 

Statistic is shown in table C-3.

Table C-3. Kappa Statistic range.

Kappa Statistic Strength of Agreement

< 0.00 Disagreement

0.00 Chance agreement

0.01 to 0.20 Slight Agreement

0.21 to 0.40 Fair Agreement

0.41 to 0.60 Moderate Agreement

0.61 to 0.80 Substantial Agreement

0.80 to 0.99 Almost Perfect Agreement

1.00 Perfect Agreement

An example of the Kappa Statistic evaluation is provided in tables C-4 though C-6. 

Table C-4 includes the QC results from the manual survey conducted by the British 

Columbia MoTI, while table C-5 is the automated survey results conducted by the service 

provider. The data is evaluated using the Kappa Statistic and the results of this analysis 

are provided in table C-6. Table C-6 indicates that there is substantial agreement (see 

table C-3 for strength of agreement for each Kappa Statistic range) between the agency 

and service provider ratings for severity and type, density and type, and the overall 

agreement factor.
Practical Guide for Quality Management of Pavement Condition Data Collection

115

Table C-4. QC control site pavement distress analysis – British Columbia MoTI.

Note

1. Sev = Severity: 1 = low, 2 = moderate, and 3 = high.

2. Den = Density: 1 = few (< 10 percent), 2 = intermediate (10 to 20 percent), 3 = frequent (20 to 50 percent),

4 = extensive (50 to 80 percent), and 5 = throughout (80 to 100 percent).

Table C-5. QC control site pavement distress analysis – service provider.

Note

1. Sev = Severity: 1 = low, 2 = moderate, and 3 = high.

2. Den = Density: 1 = few (< 10 percent), 2 = intermediate (10 to 20 percent), 3 = frequent (20 to 50 percent),

4 = extensive (50 to 80 percent), and 5 = throughout (80 to 100 percent).

Table C-6. Kappa Statistic results.

Variables Kappa

Severity and Type 0.746

Density and Type 0.739

Overall Agreement Factor 0.790

The average PDI value, keystroke totals, and Kappa Statistic are evaluated from the distress 

ratings for both the manual and service provider surveys. The PDI is used as the main criteria to 

BCMoTH Manual Survey Data

Start End LWT LJC PEC TC MLC AC POT DST BLD Segment

m m Sev Den Sev Den Sev Den Sev Den Sev Den Sev Den Sev Den Sev Den Sev Den PDI

0 50 3 3 3 2 3 3 3 1 3.0

50 100 3 3 2 4 3 3 3 1 2.8

100 150 3 3 2 1 3 3 2 1 3.7

150 200 3 3 2 3 3 3 2 1 3.3

200 250 3 3 2 3 3 3 2 1 3.3

250 300 3 3 3 3 2 1 3.9

300 350 3 2 2 1 2 3 5.8

350 400 2 1 1 3 2 3 6.3

400 450 2 2 2 1 3 3 5.6

450 500 2 1 3 3 6.2

Average PDI = 4.4

Automated Survey Data

Start End LWT LJC PEC TC MLC AC POT DST BLD Segment

m m Sev Den Sev Den Sev Den Sev Den Sev Den Sev Den Sev Den Sev Den Sev Den PDI

0 50 3 3 3 3 2 1 3.9

50 100 3 3 3 3 3 1 3.7

100 150 3 3 3 3 2 1 3.9

150 200 3 3 3 3 2 1 3.9

200 250 3 3 3 3 2 1 3.9

250 300 3 3 2 3 2 1 4.1

300 350 3 3 3 3 3 3 2 1 3.0

350 400 2 3 3 3 2 1 4.9

400 450 2 2 3 3 5.9

450 500 2 2 3 3 5.9

Average PDI = 4.2
Practical Guide for Quality Management of Pavement Condition Data Collection

116

assess accuracy and repeatability, while the keystroke summaries and the Kappa Statistic are 

used as additional diagnostic tools.

Roughness Tests

The service-provider survey vehicle conducts a total of five runs of each control site. The IRI 

values for each wheel path are determined on 164-ft (50 m) intervals and compared to the 

manual values. The average IRI value is determined for each wheel path over the 1,640-ft (500 

m) test site.

Rut Depth Tests

The rut depth values for each wheel path are generated at 164-ft (50 m) intervals and compared 

to the manual values for each run. The average rut depth value is determined for each wheel 

path for the 1,640-ft (500 m) test site.

Acceptance Criteria for Initial Quality Control Tests

The acceptance criteria for surface distress, roughness, and rut depth measurements are provided 

in table C-7.

Table C-7. Initial QC criteria (BCMoTI 2012).

Category Criteria Acceptance Criteria Value

Surface 

Distress

Measure PDI value (0 to 10 scale)

Calculation 1,640 ft (500 m) average based on 164 ft (50 m) values

Unit Lane

Accuracy ± 1 PDI value of manual survey (0 to 10 scale)

Repeatability ± 1 standard deviation of the PDI values for five runs

Roughness

Measure IRI

Calculation 1,640 ft (500 m) average based on 164 ft (50 m) values

Unit Outside wheel path

Accuracy ± 10 percent of Class I profile survey

Repeatability ± 6.3 in/mi (0.10 mm/m) standard deviation for five runs

Rutting

Measure Rut depth

Calculation 1,640 ft (500 m) average based on 164 ft (50 m) values

Unit Averaged for both wheel paths

Accuracy ± 0.12 in (3 mm) of manual survey

Repeatability ± 0.12 in (3 mm) standard deviation for five runs

If the service provider fails to meet the acceptance criteria, they are required to resolve any 

issues until the acceptance criteria are met and the British Columbia MoTI representative is 

satisfied. Issue resolution may include, but is not limited to (BCMoTI 2012):

• Conduct additional on-site discussions with British Columbia MoTI personnel.

• Repeat the condition survey.
Practical Guide for Quality Management of Pavement Condition Data Collection

117

• Conduct equipment repairs/modifications.

• Retrain and/or replace rating staff.

Production Survey Acceptance

The service provider’s condition rating and equipment are closely monitored during the 

production surveys through the use of blind sites. The blind sites are located on roadway 

sections in each of the British Columbia MoTI Regions. All blind sites are manually surveyed in 

advance by British Columbia MoTI and are of unknown location to the service provider.

Blind Site Locations

The number and location of the blind sites are based on the pavement condition survey length

and the service provider schedule of data collection. Blind sites are located such that the service 

provider is evaluated each day for the first 2 to 3 days of the pavement condition survey, and 

then every 3 days pending satisfactory performance. Similar to the initial QC sites, blind sites 

are 2,460 ft (750 m) in length, including an 820 ft (250 m) lead-in, and exhibit a representative 

sample of distress types and severities encountered during the pavement condition survey.

Advance Manual Surveys

Manual surface distress, roughness, and rut depth measurements are conducted at each blind site 

prior to the production surveys. A single rater is used for all of the blind site evaluations to 

provide a consistent benchmark for the manual surveys.

Monitoring Process

Each day of the production survey, the service provider is required to update the British 

Columbia MoTI on the survey progress. During these updates, the service provider is notified 

whether or not they have passed over a blind site during the previous day’s testing. The service 

provider is required to submit a distress rating report summarizing the pavement distress 

(including IRI and rut depth) over the length of the blind site. Because of possible referencing 

differences, the service provider is required to submit 0.6 mi (1.0 km) of pavement condition 

data within 820 ft (250 m) of either side of the blind site location, as illustrated in figure C-3.

Figure C-3. Example of blind site reporting limits.

The service provider must successfully meet the blind site acceptance criteria prior to receiving 

approval to proceed with the production survey.

Reporting Limits

0.6 mi (1 km)

Outside lane

Inside lane

1,640 ft

(500 m)

Blind Site

820 ft

(250 m)

820 ft

(250 m)
Practical Guide for Quality Management of Pavement Condition Data Collection

118

Retest Initial QC Sites

If the production surveys are more than 1 month in duration or expand over multiple regions, the 

service provider is required to rerun the four initial QC sites each month of the production survey 

and prior to testing a new region. Retesting of the initial QC sites helps to ensure that the service 

provider equipment is operating properly and the pavement raters are correctly applying the 

British Columbia MoTI distress rating system.

Acceptance Criteria for Blind Site QC Tests

The QC acceptance criteria for the surface distress, roughness, and rut depth measurements for 

blind site are shown in table C-8.

Table C-8. Blind site QC criteria (BCMoTI 2012).

Category Criteria Acceptance Criteria Value

Surface 

Distress

Measure PDI value (0 to 10 scale)

Calculation 1,640 ft (500 m) average based on 164 ft (50 m) values

Unit Lane

Accuracy ± 1 PDI value of manual survey

Roughness

Measure IRI

Calculation 1,640 ft (500 m) average based on 164 ft (50 m) values

Unit Outside wheel path

Accuracy ± 10 percent of Class I profile survey

Rutting

Measure Rut depth

Calculation 1,640 ft (500 m) average based on 164 ft (50 m) values

Unit Averaged for both wheel paths

Accuracy ± 0.12 in (3 mm) of the manual survey

If the service provider is unable to meet the blind site QC criteria, the following activities, and 

possibly others, may be required (BCMoTI 2012):

• Conduct additional on-site discussions with British Columbia MoTI.

• Review digital images with British Columbia MoTI.

• Repeat pavement condition surveys.

• Conduct equipment repairs/modifications.

• Retrain and/or replace rating staff.

Acceptance of Submitted Data

Upon receipt of the service provider pavement condition survey results, British Columbia MoTI

conducts a three-step process for accessing the submitted data. The three-step process includes 

manual and system checks of the surface distress, roughness, and rut depth data files.
Practical Guide for Quality Management of Pavement Condition Data Collection

119

1. Manual Review―involves the manual review of the submitted data files, which includes

(BCMoTI 2012):

• Checking that data exists for all road segments.

• Verifying highway traversal definitions for all road segments.

• Ensuring correct data file structure.

• Verifying start and end boundaries for all road segments.

• Checking all lane references and chainages according to the provided data files.

• Screening all data for null and negative values.

• Screening all data according to minimum and maximum tolerance parameters.

Any noted discrepancies are provided to the service provider for correction.

2. Prior Year Comparison―compares the current year survey data to the prior year’s 

survey data. This comparison is conducted to determine if there are any unexpected 

changes in pavement condition from the prior year’s survey.

3. Pavement Management System Data Upload Tests―involves uploading the collected 

data to British Columbia MoTI’s pavement management system. The pavement 

management system includes internal standardized and user-defined verification tests, 

which are run once the data has been uploaded. An error log report is processed; input 

data is corrected, as needed; and the corrected data is reloaded into the pavement 

management system.
Practical Guide for Quality Management of Pavement Condition Data Collection

120

APPENDIX D. CASE STUDY— LOUISIANA DOTD
Practical Guide for Quality Management of Pavement Condition Data Collection

121

Introduction

The Louisiana DOTD has conducted automated pavement condition surveys since 1995 using 

pavement condition assessment service providers. In 2007, distance to overhead obstructions 

(e.g., bridges, and sign structures) and geometric data (i.e., cross slope, shoulder drop-offs, and 

vertical and horizontal curves) were added to the pavement condition survey.

Data Collection

Approximately 20,000 directional mi (32,000 km) of pavement condition data are collected 

biennially during the data collection cycle. Both directions are collected on interstates and multilane divided highways, and one direction is collected for two-lane highways. Pavement distress 

is collected in accordance with Louisiana DOTD cracking and patching protocols (LADOTD 

2012a, LADOTD 2012b). Data collected as part of the network-level pavement condition data is

reported for every 0.100 mi (0.161 km) of the surveyed length and are shown in table D-1.

Table D-1. Louisiana DOTD network-level condition survey 

(LADOTD 2012a; LADOTD 2012b).

General Data Asphalt Pavements Concrete Pavements

• GPS coordinates 

(longitude, latitude 

and elevation)

• Bridges (count)

• Distance to overhead 

obstructions

• Geometric data

• Alligator cracking

• Random (block, 

longitudinal, and 

transverse) cracking

• Rut depth

• Patching

• Blowup

• Potholes

• IRI

• Transverse cracking

• Longitudinal cracking

• Joint faulting

• Concrete patching

• Blowups

• Punchouts

• IRI

Collected data should be reviewed for completeness at the end of each day. Louisiana DOTD

requires that the service provider deliver the following data on a weekly basis:

• ROW images.

• Raw data from the data collection vehicle’s electronic sensors (i.e., rutting, IRI, faulting, 

and GPS data).

• Equipment calibrations test results (e.g., distress manifestation index, rut measurement 

device, and video foot print).

• Electronic sensor verification results.

The service provider is responsible for checking all data/images prior to delivery to Louisiana 

DOTD and should rectify all issues discovered by Louisiana DOTD.
Practical Guide for Quality Management of Pavement Condition Data Collection

122

Louisiana DOTD also provides user access to the Roadware VisiDataTM (see figure D-1) 

information. With Roadware VisiDataTM, the user is able to view full roadway width images and 

distress data (e.g., IRI and crack map).

Figure D-1. Example Roadware VisiDataTM information (Fillastre 2012).

Quality Control Requirements

Louisiana DOTD requires the service provider to submit QC plans that will ensure that data are 

collected within specified precisions, including preliminary activities (i.e., developing the QC 

plan, conducting personnel training/certification, and equipment calibration), control sites, data

checks, and final documentation delivery.

Preliminary Activities

Develop Quality Control Plan

The service provider’s equipment is checked against an agency profiler and a Class I profiling 

instrument (e.g., Dipstick) before beginning testing. During production, the service provider is 

required to use QC sections of known IRI, rutting, and faulting values.
Practical Guide for Quality Management of Pavement Condition Data Collection

123

Personnel Training/Certification

Louisiana DOTD requires inter-rater training to be completed by the service provider and the 

results submitted to Louisiana DOTD for review. This training helps ensure that a rater exhibits 

understanding of protocols by correct distress identification and classification and consistency of 

rating distresses across multiple raters. Key personnel are identified in the data collection

request for proposal (RFP), and the service provider is required to disclose all certifications and 

achievements in their proposal, including education background and achievements of key 

personnel, and current and past clients for references. In addition, data collection vehicle crew

are required to be trained so they have a good understanding of the data collection protocols, 

especially the correct method of data collection on control sections. 

Equipment Calibration

Equipment calibration is conducted before the initiation of the data collection activities and 

periodically thereafter to verify that the equipment is functioning according to expectations and 

that the collection and analysis methods are being followed. The service provider recommends

the subsystems and equipment for data collection and is responsible for proper calibration. All 

equipment is calibrated according to the manufacturer’s recommendations. The DMI is 

calibrated on segments with a known/surveyed length. It is also important that all operating 

procedures pertaining to data collection used by the service provider are documented and 

standardized to ensure consistent results.

Control Sites

Data verification by testing control or verification sites and conducting data checks are used for 

QC before and during production. The service provider collects the required data on each 

control site (minimum of three runs) to prove repeatability of the electronic measurements. 

Electronic data is compared to previous year’s data collection to ensure data consistency and 

validity. It is recommended that the control site be inspected each time the service provider

starts a new district, or each time the data collection vehicle leaves the state. The service 

provider is mandated to re-collect control section data from the previous week’s collection to 

verify that the equipment is in calibration.

During data collection, the service provider ensures that a control section is ready for data 

collection. Louisiana DOTD identifies a list of situations when the data can be collected 

(LADOTD 2012c) that includes the following:

• A control section should not be collected if the data collection vehicle is forced to collect 

the majority of the data while traveling towards the sun.

• A control section should not have excessive water on the roadway.

• A control section should not be collected during inclement weather.

• A control section should only be collected during daytime hours and when there is 

enough daylight to collect it in its entirety.
Practical Guide for Quality Management of Pavement Condition Data Collection

124

Data Checks

To ensure that erroneous data are not being collected, in-vehicle, real-time data checks are 

performed for rutting, IRI, GPS, faulting, and DMI data to ensure that it is within the required

tolerances. In addition, pavement distress data (i.e., images and processed results) are provided 

to the Louisiana DOTD for review and evaluation. The Louisiana DOTD reviews approximately

5 percent of the control section length and segments the samples into 0.10-mi (0.16 km) 

increments. For example, a control section with a 10 mile (16 km) length would result in 5 

samples each 0.1 mi (0.16 km) in length. Sampled section images are checked using the vendorsupplied proprietary software and reviewed for such items as:

• Missing a high severity distress.

• Missing 5 or more low/medium severity distress.

• Incorrect distress type or severity, or over-rating (indicating that a distress is present 

when actually there is no distress).

Figures D-2 and D-3 illustrate a screen capture of the Visidata for a control section sample. The 

upper portion of figure D-2 provides details of the pavement section including location (e.g., 

district, route, beginning and ending mile post), pavement type, average IRI, and average rut 

depth. The lower portion of figure D-2 provides the details of the identified distresses by distress 

type (alligator cracking, longitudinal cracking, and transverse cracking), and severity (low, 

medium, and high). The lower portion of figure D-2 is used to assess whether or not the 

distresses shown in figure D-3 are adequately reported.

Unlike the pavement images, the processed data is not sampled; instead Microsoft Access 

queries are run to check for data inconsistencies. Electronic data checks include:

• Changes in pavement type from the previous year’s survey.

• Changes in pavement texture from the previous year’s survey.

• Sudden changes in roughness (major improvement/deterioration).

• Sudden changes in rut depth (major improvement/deterioration).

• High quantities of distress with low roughness values.

• High roughness values with low quantities of distress.

• Check for reasonableness of the maximum extent of distress. For example, if fatigue 

cracking is present in both wheel paths for a section length of 0.10 mi (0.16 km), and a 

wheel path is considered to be 3 ft (0.9 m) wide, the resulting extent of fatigue cracking 

would be 3,168 sq-ft (295 sq-m). In this example, the service provider data for fatigue 

cracking should not exceed 3,168 sq-ft (295 sq-m).

• Review all segments that are marked as a construction zone.

• Review all segments that are marked as a lane deviation.

• Review segments that are identified as a bridge, but the service provider data does not 

indicate a bridge location.

• Review control sections that are found to have a longer lengths than specified.
Practical Guide for Quality Management of Pavement Condition Data Collection

125

• Review control sections where the service provider did not collect the required 0.10 mile 

(0.16 km) lead in/lead out pavement length.

• Review pavement segments with incomplete data collection.

Figure D-2. Example of Visidata evaluation (LADOTD).
Practical Guide for Quality Management of Pavement Condition Data Collection

126

Figure D-3. Example of Visidata evaluation (LADOTD).

Pavement surface and ROW images are observed during and after a day’s collection to prevent

and minimize re-runs. In addition to the data checks, pavement surface and ROW images are 

checked by the Louisiana DOTD for clarity, for ensuring that there are minimal missed or 

skipped images, for proper lighting, and for the correct stitching of pavement images.

During the Louisiana DOTD review of the service provider submitted data and images, any

problems with the distress identification, the processed data and results, and the image quality

are defined (e.g., the extent of the problem is determined) and documented. The documented 
Practical Guide for Quality Management of Pavement Condition Data Collection

127

findings are summarized in a report and provided to the service provider. As stated in the 

Louisiana DOTD service contract, the service provider is obligated to fix all issues, to the 

satisfaction of the Louisiana DOTD, at no cost to the agency; this includes re-rating and reprocessing the data and, if necessary, recollecting the offending control section.

The data collection vehicle is also checked daily for proper calibration, operation, and 

maintenance. All calibration, operation, and maintenance efforts are performed in accordance 

with the manufacturer recommendations, or as outlined in the standard operating procedures for

the equipment. Calibration, operation, and maintenance effort activities are documented in 

writing and submitted to the Louisiana DOTD as they occur.

Final Deliverables

The final QC deliverables submitted to the Louisiana DOTD encompass the entire data 

collection efforts documenting inter-rater consistency, data collection vehicle equipment checks, 

control/calibration/verification site results, and data collection vehicle calibration documentation. 

Specifically, the items to be submitted by the service provider for QC requirements include 

(LADOTD 2012c):

• All reports (i.e., inter-rater consistency, data collection vehicle equipment checks, 

control/calibration/verification site results, and data collection vehicle calibration 

documentation).

• All correspondences relating to the project.

• Explanations of abnormal calibrations.

• Data collection schedule adherence or changes.

• A listing of the data collection vendor’s key project personnel.

• Project-related issues that were addressed during the data collection.

• Recommendations for improvements.

Acceptance Requirements

Louisiana DOTD performs QC of the data collected by the service providers through 

comprehensive checks for three data components: ROW images, pavement images and rated 

distresses, and database checks (LADOTD 2012c). Detailed descriptions of the QC

requirements are presented in Louisiana DOTD’s QC write-up. The following sections 

summarize the most important requirements.

Right-of-Way Images

Evaluation of the ROW images includes a review of image quality, checks to ensure that the 

correct sections have been collected, verification that all required sections have been sampled, 

and confirmation that all control sections have the correct lead-in and lead-out lengths. Each of 

these is further described in the following sections.
Practical Guide for Quality Management of Pavement Condition Data Collection

128

Image Quality

The following items are checked by the agency to ensure that the collected images have 

acceptable levels of clarity, brightness/darkness, and completeness (LADOTD 2012c):

• Image clarity―all images should be clear and highway signs easily read. Most highway 

distresses should be evident in all views. There should be minimal, or no, debris in the 

cameras’ viewing path.

• Image brightness/darkness―images are not to be collected during hours when it is too 

dark (rule-of-thumb: if street lights or security lighting are lit, then it is too dark). It has 

been found that during poor lighting conditions, the images become very grainy and seem 

to be out of focus, or it results in a “black out,” which can cause a control section to be 

rejected. In addition, if the data collection occurs just before a rain storm, the dark clouds 

may not allow the proper amount of light to enter the camera, and the subsequent 

image(s) will be of poor quality.

• Dry pavement―control section should not have any standing water during testing; 

otherwise, the control section will be rejected. As a result, data collection should be 

halted during a rain storm. If rain drops are allowed to accumulate on the protective 

glass, the images will be of poor quality due to the lack of clarity and sharpness.

• Image replay―images should play sequentially and in the correct order. The data 

collection vehicle should give the impression that it is traveling in a forward direction.

• Missing images―there should be minimal or no missing images. Any control section 

that contains substitute images should be rejected.

Correct Data Collection

The control section manual or approved equivalent, contains beginning and ending descriptions 

of all control section locations, and also indicates the length and route number. The Louisiana 

DOTD has several internal tools available to aid in checking the ROW to determine if the data 

collection vehicle tested the correct control section. These tools include:

• Louisiana DOTD GIS Proposed and Active DOTD Construction Projects web page.

• Louisiana DOTD Project and Highway Information web page.

• Control Section Manual.

• Highway maps.

• Peers.

Specifically, the following items are checked by the agency to ensure correct data collection 

(LADOTD 2012c):

• The beginning and ending of the control section are checked to ensure that the data 

collection vehicle started and ended at the correct location. If the beginning or ending of 

the control section is determined to be incorrect, then the control section should be 

rejected.
Practical Guide for Quality Management of Pavement Condition Data Collection

129

• The images for the first 0.10 mi (0.16 km) should be played and checked, while the 

distress images should be sampled throughout the entire control section.

• The lengths, as determined by the control section manual and the service provider, should 

coincide to be within less than 5 percent difference. If the difference is more than this, 

then the control section should be carefully reviewed to ensure that the data collection 

vehicle tested the correct control section.

• Most control sections should have a 0.10 mi (0.16 km) lead-in and lead-out. Only the 

ROW images are collected for the lead-in and lead-out. This helps determine if the data 

collection vehicle began and ended at the correct beginning and ending locations.

Pavement Images/Rated Distresses

Prior to reviewing video images for rating pavement distress, the Louisiana DOTD recommends 

that raters conduct a thorough review of the pavement distress protocols for asphalt and concrete 

(LADOTD 2012a; LADOTD 2012b). Evaluation of pavement images includes the ability to 

easily view distress type and severity and to establish that all sample sections have been properly 

evaluated with the appropriate distress rating. Each of these factors is further described in the 

following sections, which are extracted from the Louisiana DOTD references.

Image Quality

All images should be clear and the distress type and severity be easily identifiable. The 

camera(s) should be able to quickly adjust to varying lighting conditions. For example, when the 

data collection vehicle is on an asphalt road and has crossed a concrete bridge, the camera(s) 

may “white out” from the higher degree of light reflection. Conversely, when the data collection 

vehicle exits a concrete bridge onto an asphalt pavement, the camera(s) may “black out.”

Pavement images should be synchronized with the ROW images. The images should play in the 

correct order, and, if two or more cameras are used, the images should be “seamless” both 

transversely and longitudinally. The pavement type and texture should correspond to the 

pavement type and texture that is shown in the ROW view.

To make the process as efficient and accurate as possible, the ratings utilize three different colors 

to represent the three different severity levels: green (low severity), yellow (medium severity), 

and red (high severity). As shown in figure D-4, different markings represent different 

distresses: transverse (a single, solid line that travels from the left side to the right side), 

longitudinal (a single, solid line that travels from the bottom to the top), alligator (a single, long 

dashed line that travels from the bottom to the top), block (a single, single dashed line that 

travels from the bottom to the top), and patch (an area drawn marked, using a cross hatch 

pattern).
Practical Guide for Quality Management of Pavement Condition Data Collection

130

Figure D-4. Example of pavement distress rating (Fillastre 2012).

Alligator (low)

Transverse (low)

Alligator (high)

Longitudinal (medium)

Transverse (low)
Practical Guide for Quality Management of Pavement Condition Data Collection

131

Sampling

All sections are sampled to ensure that the rating corresponds to the present distress type and 

severity. Different methods are used to detect whether or not the control section was properly 

rated, including the following (LADOTD 2012c):

• The reviewer can open a grid with the quantified data, which will allow the review of the 

numerical amount of a specific distress.

• The reviewer can write an Structured Query Language (SQL) statement in the “Where” 

field to query the database. The reviewer can then review the filtered segments and 

review the ratings.

• The reviewer can open the inventory view and view all distresses that were rated within a 

specific segment.

• The reviewer can randomly sample the control section, which is the quickest and most 

efficient process for checking the ratings. Using this method, the reviewer can determine 

if the distresses were identified correctly, especially where distress type and severity are 

concerned.

• The control section should be sampled at a rate of approximately 5 percent, as defined in 

table D-2. If any errors are found, then the reviewer should perform a more thorough 

review of the ratings so that most or all of the errors can be found and reported back to 

the service provider.

Table D-2. Control site sampling frequency (LADOTD 2012c).

Control Section Length

Sample Frequency

(tenth mile to be checked)

≤ 1 mi

(1.6 km) 2

> 1 and ≤ 5 mi

(> 1.6 and ≤ 8.0 km) 3

> 5 and ≤ 10 mi

(> 8.0 and ≤ 16.1 km) 5

> 10 and ≤ 15 mi

(> 16.1 and ≤ 24.1 km) 8

> 15 and ≤ 30 mi

(> 24.1 and ≤ 48.3 km) 13

> 30 mi

(48.3 km) 5 percent of control section length

Database Checks

The database checks used by Louisiana DOTD include searches for missing data, out of 

tolerance data, and abrupt pavement condition changes when compared to data collected from 

previous years. Any pavement section discrepancies are further investigated. In addition, GPS 

data is plotted on a GIS map for comparison to existing accepted GPS data. Finally, the database 

is imported into the pavement management software to validate data format and data trends. 

Any issues are documented and reported to the service provider for resolution.
Practical Guide for Quality Management of Pavement Condition Data Collection

132

APPENDIX E. CASE STUDY— OKLAHOMA DOT
Practical Guide for Quality Management of Pavement Condition Data Collection

133

Introduction

Since 2001, the Oklahoma DOT has implemented and refined its pavement management system 

(PMS) to provide the agency with an effective decision support tool for prioritizing pavement 

projects. Oklahoma DOT contracts for annual network-level collection of pavement condition 

data, geometric data, and video. Data are processed through a combination of automated and 

semi-automated methods.

Data Collection

Oklahoma DOT maintains approximately 12,300 centerline mi (19,800 centerline km) of 

highways on the non-toll State highway system. Condition data for interstates and NHS routes 

are collected annually and the remainder of the system is collected every 2 years. Data are 

collected over 100 percent of the length of the state-maintained network and reported at 0.01 mi 

(0.02 km) increments. The pavement condition data collected are shown in table E-1.

Table E-1. Pavement condition data items collected (ODOT 2010).

General Data Asphalt and 

Composite Pavements

Jointed Concrete 

Pavements

Continuously 

Reinforced Concrete 

Pavements (CRCP)

• Surface type

• Pavement 

macrotexture

• Roadway 

geometrics (cross 

slope, radius of 

curvature, and 

longitudinal 

grade)

• GPS coordinates

• Roadway events 

(bridges, railroad 

crossings, 

approach slabs, 

construction, lane 

deviations, and 

detours)

• IRI

• Rut depth

• Transverse 

cracking 

• Fatigue cracking 

• Miscellaneous 

cracking

• Asphalt patching

• Raveling

• IRI

• Fault, average

• Transversely 

cracked slabs

• Longitudinally 

cracked slabs

• Multi-cracked slabs

• Spalled joints 

• D-cracked joints

• Corner breaks

• Asphalt patching

• Concrete patching

• Number of joints

• IRI

• Longitudinal 

cracking 

• Punchouts

• Asphalt patching

• Concrete patching

Data Collection and Distress Rating Protocols

Oklahoma DOT specifies the use of the AASHTO Standards for the collection of IRI, rutting, 

and faulting. However, Oklahoma DOT modifies the Standards by requiring that the results be 

reported in U.S. customary units and at a reporting interval of 0.01 mi (0.02 km) (ODOT 2010).

The 2005 Oklahoma DOT Pavement Management Distress Rating Guide provides thorough 

and clear descriptions of Oklahoma DOT’s definitions of pavement distresses, guidelines for

conducting the condition rating, and recording the distress data. One objective of the Distress 

Rating Guide is to achieve consistent, accurate, and repeatable distress ratings for use in the 
Practical Guide for Quality Management of Pavement Condition Data Collection

134

pavement management system. The Distress Rating Guide gives general instruction to raters 

about which lanes to rate and provides a clear description and visual depiction of each severity 

category for each type of distress.

Data Quality Requirements 

Oklahoma DOT has worked with its service providers to understand the limits of the available 

technology and establish realistic data quality requirements. The agency has established 

requirements for the accuracy, resolution, and repeatability of the collected data. Accuracy 

refers to the deviation of the data collected by the service provider compared to that item collected 

or provided by the agency as ground truth. Table E-2 includes the minimum requirements for data 

quality specified in the latest Oklahoma DOT request for proposals.

Table E-2. Data quality requirements (ODOT 2010).

Data Element Required Minimum 

Accuracy

Required Resolution 

(Measure to the 

Nearest)

Required Minimum 

Repeatability

IRI

± 5 percent compared 

to dipstick or Class I 

profiler

1 in/mi

(0.01 mm/m)

± 5 percent run to run 

for three repeat runs

Rut depth

± 0.08 in (2 mm)

compared to manual 

survey

0.01 in 

(0.25 mm)

± 0.08 in (2 mm) run 

to run for three repeat 

runs

Faulting

± 0.08 in (2 mm)

compared to manual 

survey

0.01 in

(0.25 mm)

± 0.08 in (2 mm) run 

to run for three repeat 

runs

Distress ratings

± 10 percent compared 

to Oklahoma DOT 

ratings

N/A N/A

GPS coordinates

0.00005 degrees 

compared to Oklahoma 

DOT provided 

coordinates

0.000001 degree N/A

Pre-Qualification of Service Providers

Oklahoma DOT requires prospective service providers to participate in a demonstration to 

collect specified condition data items at four Oklahoma DOT control sites (ODOT 2010). The 

four 0.50-mi (0.8 km) long control sites are located on a four-lane divided highway in a county 

in the central part of the state. Two sites are jointed concrete pavement, while the other two 

sites are asphalt pavement. The beginning and ending points of each site are marked on the 

outside shoulder.

Each service provider is required to collect and submit the following (ODOT 2010):

• Video log images—provide the pavement (downward-facing) and two ROW views (one 

forward and slightly right, one forward and slightly left) for the entire length of each control 

site. ROW views should be collected and presented at intervals of 0.005-mi (0.008 km) or 
Practical Guide for Quality Management of Pavement Condition Data Collection

135

200 images per mile (125 images per km) for each view. The pavement view should provide 

continuous 100 percent coverage of the driving lane. The service provider may choose the 

image resolution, but all images should be in jpeg format.

• GPS data—provide latitude and longitude in degrees and decimals of a degree to six decimal 

places for the beginning of each 0.01-mi (0.02 km) interval for the entire length of each 

control site.

• IRI data—provide IRI in U.S. Customary units for the left and right wheel paths and the 

average of both wheel paths at a data summary interval of 0.01 mi (0.02 km). Collect IRI 

according to AASHTO PP 37-04, but use a data summary interval of 0.01 mi (0.02 km) and 

report the results in U.S. Customary units. 

• Rut depth data—for the asphalt pavement control sites, provide left rut depth, right rut

depth, average rut depth, maximum rut depth, and the percent of rut depth measurements that 

are less than 0.5 in (12.7 mm), in U.S. Customary units, for each 0.01-mi (0.02 km) interval.

Rut depth measurements should be taken at a maximum spacing of 10.56 ft (3.2 m)

longitudinally for a minimum of five measurements per wheel path every 0.01 mi (0.02 km). 

Rut depth should be collected in accordance with AASHTO PP 38-00, Standard Practice for 

Determining Maximum Rut Depth in Asphalt Pavements, using a minimum of eleven sensors 

and a data summary interval of 0.01-mi (0.02 km), and report the results in U.S. Customary 

units.

• Faulting data—for the jointed concrete control sites, provide the average fault, maximum 

fault, number of faults, and standard deviation for each 0.01-mi (0.02 km) interval. Collect 

faulting in accordance with AASHTO PP 38-00, using a data summary interval of 0.01 mi 

(0.02 km) and report the results in U.S. Customary units. 

• Geometric data—for each control site, provide longitudinal grade, cross slope, and curve 

radii in U.S. Customary units for each 0.01-mi (0.02 km) interval. 

• Distress data—provide processed pavement distress ratings for the control sites using the 

Oklahoma DOT Distress Rating Guide. Aggregate and report distress data at 0.01-mi (0.02 

km) intervals.

Oklahoma DOT evaluates the video and control site data and uses this information as part of 

the scoring of the service providers’ proposals. Over the years, Oklahoma DOT has found the 

requirement for submission of control site data to be invaluable in evaluating the technical 

capabilities of service providers.

Collection File and Database Shell

Oklahoma DOT furnishes the service provider with a collection file that describes each 

segment (called control sections) to be collected along with a physical description and GPS 

coordinates of the beginning and ending point of each. In addition, the agency furnishes a 

shape file of the network that the service provider can use with GIS software to visualize the 

segments. The service provider uses these files to route the collection efficiently and to 

compare against the collected segments to make sure none were missed. Oklahoma DOT also 

provides a database shell with records for each 0.01 mi (0.02 km) of each control section. The 

shell has the proper format for each variable so the service provider only has to populate the 

shell with the collected data. 
Practical Guide for Quality Management of Pavement Condition Data Collection

136

Quality Control Plan Requirements

Oklahoma DOT requires the service provider to submit a QC plan that covers all data elements and 

includes procedures to detect and correct equipment malfunctions, data processing errors, and errors 

in data accuracy, resolution, and repeatability in a timely fashion (ODOT 2010). The QC plan must 

include a description of when and how the checks will be made, the qualifications of those 

conducting the checks, the percentage of data that will be checked, and how errors will be reported 

and corrected. The plan typically includes QC checks at all stages of the data collection, processing,

reduction, and delivery processes. Some of the QC procedures include control and verification site 

testing, inter-rater consistency testing, and numerous checks of data quality and completeness.

Control and Verification Site Testing

Control and verification site testing is a particularly important component of the QC process. Initial 

reference values for control sites were established by agency manual measurement of rutting and 

faulting. Oklahoma DOT contracted with an independent party for Dipstick profile measurement and 

calculation of IRI. After several years of manual measurement, Oklahoma DOT subsequently began 

utilizing the data provided by multiple service providers during pre-qualification control site testing 

to update the reference values.

Oklahoma DOT requires initial testing of control sites before production begins and weekly testing 

of either control or verification sites thereafter. Oklahoma DOT personnel meet the field crew at the 

site and ride with them in the data collection vehicle to observe testing. Upon completion of the 

testing, the crew loads a spreadsheet with the data and video onto a flash drive for the agency to take 

back to the central office and inspect. The agency analyzes the data, compares it to all previous 

site testing, and notifies the service provider quickly of any issues found. Data collection must 

be stopped until all issues are resolved.

Video samples of the control or verification sites are also evaluated weekly to ensure the quality of 

the collected video. Some of the factors found to affect video quality have included angle of the 

camera in relation to the position of the sun, automatic iris functioning, and vibration-dampening 

capability of the camera mount. Over time, the agency has found that observing control site testing 

has allowed for the identification of some of the factors that can affect the quality of sensor data 

measurements. In particular, rut bar width, wind speed and resultant vehicle wander at the time of 

collection, and extreme cold temperatures have been found to impact data quality (Wolters, 

McGovern, and Hoerner 2006).

Acceptance

Oklahoma DOT performs a wide variety of quality checks and/or analyses of the submitted 

data, both during and after data collection. During collection, the agency accompanies the field 

crew to monitor the control and verification site testing. After collection, Oklahoma DOT 

performs automated checks, sampling of data and video, and GIS checks.

QA Tool

Oklahoma DOT’s checks of the final database initially involved running many individual 

queries to check for out of range, inconsistent, or missing data and performing sampling checks 

of distress ratings. After several years of instituting more and more checks, Oklahoma DOT 

decided a tool to automate the process as much as possible would be beneficial and time-saving 
Practical Guide for Quality Management of Pavement Condition Data Collection

137

(Wolters, McGovern, and Hoerner 2006). As a result, Oklahoma DOT contracted for the 

development of a Visual Basic Application (VBA) tool based on Microsoft Access to automate 

a series of checks on the entire database. The main menu screen for the QA Tool is shown in 

figure E-1.

Figure E-1. QA Tool menu screen.

Checks are run by field district and are arranged in a logical order according to Oklahoma DOT 

workflow. The QA Tool combines the data checks into the following four groups:

• Preliminary checks—checks of allowable values for general information such as field 

district, direction, chainage, event codes, correct data types, agency-supplied section 

information, GPS tolerance range for section beginning points, GPS duplicates, surface 

collected versus agency-supplied pavement type, geometric data ranges, and missing 

video log data.

• Sensor data checks—checks for duplicates and expected range for IRI, rutting, faulting, 

and macro texture data. Expected ranges include:

- IRI: 20 to 600 in/mi (0.3 to 9.5 m/km).
Practical Guide for Quality Management of Pavement Condition Data Collection

138

- Rut depth: 0 to 1.25 in (32 mm).

- Faulting: 0 to 0.8 in (20 mm).

- Macro texture: 0 to 0.10 in (2.7 mm).

• Distress rating checks—checks for expected ranges of individual and combinations of 

distresses. The Oklahoma DOT uses a pavement segment length of 52.8 ft (16.1 m), 

therefore, the expected range in the extent of pavement distress include:

- Number of cracked slabs: 4 (15 ft [4.6 m] joint spacing).

- Alligator cracking: 52.8 ft (16.1 m).

- Patching: 636 ft (194 m), measured over the full lane width and length.

• Special checks—checks for maximum asphalt patch length, non-matching distress types 

(distresses not associated with a surface type), and expected number of railroad crossings 

and bridges per segment.

As the checks are performed, the QA Tool interface displays the records with suspected 

problems and allows the user to make adjustments to the data if desired. If the discrepancy 

appears to be isolated, the Oklahoma DOT will adjust the data in the record. However, in some 

cases, when the problem has been found to be widespread, the Oklahoma DOT has requested 

the service provider to re-rate or correct the suspect data. All problems, as well as the action 

taken to resolve the issue, are documented in the spreadsheet log of the QA Tool and included 

in the final QM Report. In addition, category and summary reports of the results of the QA 

checks can be exported from the QA Tool.

Video Sampling

As portions of the collection are completed, the service provider begins shipping portable 

external hard drives loaded with video images to the Oklahoma DOT. Oklahoma DOT uses the 

service provided video log software to view the images, check segment beginning and ending 

points, and assess overall video quality. One or two routes that traverse the entire state are 

selected and “driven” in their entirety with the video to see if the segments line up correctly and 

are continuous. Any problems found are quickly reported to the service provider so that, if 

needed, sections can be re-collected before the crew finishes the project and leaves the state.

GIS

Oklahoma DOT also uses GIS to check the submitted data. The agency plots the collected data 

points on a base map of the network to check for missing data or inaccurate beginning or 

ending points. This practice has been useful in finding errors introduced in the segmentation 

process.

Quality Management Reporting

Upon completion of all quality checks, Oklahoma DOT compiles an annual Evaluation of 

Pavement Management Data Quality report (ODOT 2009). This report is an important part of 

the overall QM process as it allows the agency to track quality issues from year to year, provide 

feedback to the service provider, and suggest changes to improve data quality.
Practical Guide for Quality Management of Pavement Condition Data Collection

139

The data quality evaluation report contains the following sections:

• Background—summary of the project scope, information about the service provider, 

miles (kilometers) and portions of the network that were collected, and data and other 

deliverables.

• General information—collection schedule, data collection vehicles and technology 

used, and administrative (communication) problems encountered. 

• Evaluation of sensor data quality—description of control and verification sites, 

establishment of reference values, procedure for initial and subsequent control and 

verification testing, and results of testing for each data collection vehicle.

• Evaluation of other sensor data quality—summary of issues identified with the quality 

of location, geometric, and GPS data.

• Evaluation of distress data quality—description of distress rating quality issues 

identified during screening with the QA Tool and confirmed by inspection of samples.

• Evaluation of video quality—description of the views collected and problems identified 

by sampling.

• Conclusions—summary of the problems encountered and corrective actions or other 

resolution.

• Appendices—reference values for control and verification sites, results of each control 

and verification site testing for each data collection vehicle, graphs of IRI from control 

sites testing, and a log of suspected quality issues found with the QA Tool, including the 

results of investigation and action taken.

Oklahoma DOT provides this report to the service provider and discusses the findings and 

possible ways to improve quality. 

Corrective Actions

In practice, many data quality issues can be corrected without re-collection, but the service 

provider is ultimately held responsible for resurveying, in a timely manner, any segments of 

roadway for which the delivered data do not meet the specified quality standards. The agency 

withholds 2.5 percent of the total contract amount pending final acceptance of data quality.

The agency keeps a log of all data issues found through the QA Tool and other acceptance testing 

or checks (e.g., control or verification site testing, GIS). If a problem is identified with the 

sensor data and the vehicle is still collecting data, the service provider is required to stop 

collection. The problem is investigated and the data collected since the last successful control or 

verification site run must often be re-collected.

If a problem is discovered after collection has been completed, the agency discusses with the 

service provider the best way to correct the data. If the problem relates to distress ratings, the 

data is often re-rated. Problems related to segmentation, count of special items (e.g., railroad 

crossings), and geometric data sometimes can be corrected during data reduction by the service 

provider. Other times, the data is not easily corrected and the agency must decide whether to 
Practical Guide for Quality Management of Pavement Condition Data Collection

140

require the service provider to re-mobilize and re-collect the data. Another option is to have 

those segments resurveyed the following year if the same service provider is to be used.
Practical Guide for Quality Management of Pavement Condition Data Collection

141

APPENDIX F. CASE STUDY — PENNSYLVANIA DOT
Practical Guide for Quality Management of Pavement Condition Data Collection

142

Introduction

Prior to 1997, the Pennsylvania DOT assessed pavement condition for the “Systematic 

Technique to Analyze and Manage Pennsylvania’s Pavements” (STAMPP) program using 

manual surveys. Since then, the Pennsylvania DOT has contracted with service providers for 

automated pavement condition assessment. Contract requirements include collection of video 

images (forward-facing and roadway surface), profile testing for determining IRI and rut depth, 

and location of roadway appurtenances (e.g., signs, rumble strip, and barriers).

Pennsylvania DOT reports that the STAMPP program provides (PennDOT 2011):

• Uniform evaluation of statewide pavement condition.

• Regular monitoring of the overall pavement network condition.

• A method of establishing county-level condition rankings.

• Optimization of investments.

• Reporting for the Additional State Funds from the Highway Maintenance Appropriation

program.

• Data to support pavement design, materials, rehabilitation, and maintenance activities.

• Assistance in project selection.

Data Collection

The service provider automated data collection is conducted annually on the interstate and NHS 

routes and on a 2-year cycle for all non-NHS routes, for a total of approximately 27,000 mi

(43,000 km) each year. Pavement condition assessment is conducted in accordance with the 

Automated Pavement Condition Survey Field Manual Procedure (PennDOT 2010). In addition, 

Pennsylvania DOT owns an automated vehicle, which had been used for conducting QC testing, 

but more recently was used to evaluate warranty projects (see figure F-1). Due to personnel 

availability, Pennsylvania DOT is currently using the service provider’s images to conduct QC

testing.

The Pennsylvania DOT also conducts annual high-speed profile testing, using agency-owned 

equipment, on all interstate routes and on all agency-maintained, newly surfaced roadways. 

Testing of newly surfaced pavements assures that the year-end IRI analyses reflect pavement 

profile surface improvements that occurred during the year.

The survey manual also includes guidelines and requirements for the location referencing system 

and survey technique. Video logging is only conducted on asphalt and jointed plain concrete 

pavements. Condition surveys for CRCP, unpaved roads, shoulders, guide rail, and drainage are 

performed manually. The collected and reported data are shown in table F-1.
Practical Guide for Quality Management of Pavement Condition Data Collection

143

Figure F-1. Automated data collection vehicle.

Table F-1. Network-level condition survey data (PennDOT 2010).

General Data Asphalt Pavements Concrete Pavements

(distress type and extent)

• Location (by county,

state route, segment,

offset, latitude, and

longitude, determined by 

differential mode GPS)

• Optional data

- Geometric 

information 

(horizontal and 

vertical curve data, 

grade, traffic 

signals, cross-slope, 

and super elevation)

- Rumble strip 

locations, other 

feature types, and 

locations

• Fatigue cracking

• Transverse cracking

• Miscellaneous cracking

• Edge deterioration

• Bituminous patching

• Raveling/weathering

• Left edge joint

• Rut depth

• IRI

• Joint faulting

• Broken slabs

• Transverse joint spalling

• Transverse cracking

• Longitudinal cracking

• Longitudinal joint 

spalling

• Bituminous patching

• Concrete patching

• Rut depth

• IRI
Practical Guide for Quality Management of Pavement Condition Data Collection

144

Pavement surface profile data is collected using a high-speed profiler in accordance with ASTM 

E950. Longitudinal profile, for IRI determination, is measured at least every 6.0 in (152 mm) for 

both the inside and outside wheel paths (69 in [175 cm] apart). IRI is reported as the average of 

both wheel paths for each 528 ft (161 m) length. For jointed concrete pavements, the IRI data is 

also used for determining the presence of broken slabs. For the broken slab analysis, the profile 

data is evaluated using a 20-ft (6 m) moving window. When the distress data indicates the 

presence of a broken slab, the IRI data is used in the calculation of the broken slab severity 

rating. Surface rutting is collected in both wheel paths independently and reported for each 

wheel path. For automated equipment, the maximum allowable spacing between rut depth 

sampling intervals is 30 ft (9 m). Sampling performed and reported more often is acceptable.

Each sample is assigned to one of the following three severity levels (PennDOT 2010):

• Low average rut depth: ≥ 0.25 in (6 mm) and < 0.5 in (12 mm)

• Medium average rut depth ≥0.5 in (12 mm) and < 1.0 in (25 mm)

• High average rut depth ≥1.0 in (25 mm)

The length of rutting for each severity level is measured in each wheel path, thereby it is possible 

to have a total reported rut depth equal to twice the segment length.

The digital images are made available for viewing on two web-based applications:

• The online VideoLog is an internet (and intranet) application that allows the user to view 

roadway images according to County, State Route, Direction, and Segment, as shown in 

figure F-2.

• VisiDataTM is only available to a limited number of Pennsylvania DOT employees and 

allows for viewing and querying Automatic Road Analyzer (ARAN) processed data 

synchronized with the captured video images, as illustrated in figure F-3.

Quality Control Processes

The Pennsylvania DOT office for Roadway Inventory and Testing Section (RITS) conducts a 

QC check of more than 675 mi (1,086 km) (or 2.5 percent) of the annual miles surveyed by the 

service provider using the downward images from the service provider. The QC process is 

conducted on IRI, rut depth, and all distress types listed in table F-2. The QC process includes 

evaluation of calibration sites, blind verification sites, and 2.5 percent of randomly sampled sites.

Calibration Sites

Evaluation of the calibration sites is conducted by both Pennsylvania DOT and the service 

provider prior to beginning the network-level condition survey. The service provider distress, 

rutting, and IRI results are compared to those determined by Pennsylvania DOT. The service 

provider results, for all distress measurements, must be within 10 percent of the values 

determined by the Pennsylvania DOT. Any discrepancies are flagged for review, verified, and 

resubmitted by the service provider. An example of the total distress comparison is shown in 

table F-3.
Practical Guide for Quality Management of Pavement Condition Data Collection

145

Figure F-2. Example image of VideoLog.

Figure F-3. Example image of Roadware VisiDataTM.
Practical Guide for Quality Management of Pavement Condition Data Collection

146

Table F-2. Example comparison of total distress (PennDOT 2011).

Pavement 

Type Distress

Number of Occurrences Service Provider Rating is

More Severe

(> 10 percent)

Similar

(± 10 percent)

Less Severe

(> 10 percent)

Jointed 

Concrete 

(2 segments)

Broken slab 0 2 0

Longitudinal joint spalling 0 2 0

Transverse cracking 0 2 0

Transverse joint spalling 0 0 2

Left wheel path rutting 0 2 0

Right wheel path rutting 0 2 0

Joint faulting 0 2 0

Bituminous 

(4 segments)

Edge deterioration 0 2 2

Fatigue cracking 0 4 0

Miscellaneous cracking 0 4 0

Transverse cracking 0 4 0

Left edge joint 0 2 2

Left wheel path rutting 0 2 2

Right wheel path rutting 1 0 3

Note: Shaded text indicates discrepancy between service provider and Pennsylvania DOT rating, requiring 

further evaluation.

In this example, the service provider tended to report less transverse joint spalling than 

Pennsylvania DOT but was within the required tolerance (± 10 percent) for all other concrete 

distresses. For bituminous pavements, the service provider tended to report less edge 

deterioration, left edge joint distress, and left wheel path rutting for two of the four calibration 

sites, and tended to under report right wheel path rutting on three calibration sites and over report 

on one calibration site. In this example, the service provider would be required to re-evaluate 

and determine the cause of the discrepancies.

In addition, Pennsylvania DOT also compares the individual distress severity and extent between 

the service provider and Pennsylvania DOT survey results. An example of this comparison is 

shown in table F-3.

The analysis shown in table F-3 indicates that the service provider reported the distress severity 

within the specified limits (± 10 percent) for the majority of the four bituminous calibration sites. 

Discrepancies are noted when 50 percent or more of the distress severity and type exceeds the 10 

percent limits. Therefore, discrepancies would be noted for low-severity fatigue cracking, lowseverity left edge joint deterioration, low-severity left wheel path rutting, and low-severity right 

wheel path rutting. The service provider is required to re-evaluate and address all noted 

discrepancies.
Practical Guide for Quality Management of Pavement Condition Data Collection

147

Table F-3. Example comparison of individual distress severity

and extent (PennDOT 2011).

Distress Type Severity

Number of Occurrences Service Provider Rating is

More Severe

(> 10 percent)

Similar

(± 10 percent)

Less Severe

(> 10 percent)

Edge 

deterioration

Low 1 2 1

Medium 0 4 0

High 0 3 1

Fatigue 

cracking

Low 3 1 0

Medium 0 2 2

High 0 3 1

Miscellaneous 

cracking

Low 1 3 0

Medium 0 4 0

High 0 4 0

Transverse 

cracking

Low 0 4 0

Medium 0 4 0

High 0 4 0

Left edge joint 

deterioration

Low 0 2 2

Medium 0 4 0

High 0 4 0

Left wheel 

path rutting

Low 0 2 2

Medium 0 4 0

High 0 4 0

Right wheel 

path rutting

Low 1 2 1

Medium 0 3 1

High 0 3 1

Note: Shaded text indicates discrepancy between service provider and Pennsylvania DOT 

rating, requiring further evaluation.

Blind Verification Sites

Blind verification site testing and comparisons are conducted during network-level distress data 

collection. The location of the blind verifications sites are disclosed in the bi-weekly summary 

report after the site(s) has been tested by the service provider. As with the calibration sites, the 

service provider survey results are compared to the values reported by the Pennsylvania DOT 

and must be within 10 percent of the Pennsylvania DOT ratings. Any discrepancies are flagged 

for review, verified, and resubmitted by the service provider. Comparison tables similar to those 

shown in tables F-2 and F-3 are prepared for the blind verification sites. An additional benefit of 

evaluating the blind verification sites is the ability to determine potential bias in service provider 

condition assessment. For example, table F-4 summarizes the individual severity level distress 

type comparison for jointed concrete pavements. In general, the service provider distress 

evaluations on the blind verification sites tended to agree with distress severity levels determined 

by Pennsylvania DOT; however, the service provider consistently under-reported low-severity 

transverse joint spalling. Based on further investigation, it was determined that the service 

provider was not consistently following the Pennsylvania DOT definition for joint spalling.
Practical Guide for Quality Management of Pavement Condition Data Collection

148

Table F-4. Pennsylvania DOT example comparison of jointed concrete pavement 

distress―blind verification sites (PennDOT 2011).

Distress Type Severity

Number of Occurrences Service Provider Rating is

More Severe

(> 10 percent)

Similar

(± 10 percent)

Less Severe

(> 10 percent)

Broken slab

Low 0 18 0

Medium 0 18 0

High 0 18 0

Longitudinal 

joint spalling

Low 0 18 0

Medium 0 18 0

High 0 18 0

Transverse 

cracking

Low 2 14 2

Medium 0 15 3

High 1 17 0

Transverse 

joint spalling

Low 0 8 10

Medium 0 17 1

High 0 18 0

Faulted joint Medium 1 17 0

High 0 18 0

Note: Shaded text indicates discrepancy between service provider and Pennsylvania DOT 

rating, requiring further evaluation.

Random Sites

A total of 2.5 percent of random roadway segments are selected by Pennsylvania DOT for the 

evaluation of the service provider’s network-level condition data collection effort. Pennsylvania 

DOT defines the first batch of service provider collected data as random sites, which includes 

sites located on interstate routes I-78, I-81, and I-83 (pavement segments within close proximity 

of Harrisburg, PA). As with the calibration and blind verification sties, the service provider 

survey results on the random sites are compared to the values reported by Pennsylvania DOT. 

The random site evaluation include comparisons of service provider and Pennsylvania DOT 

condition data, service provider historical condition data, and resulting dollars needed for 

maintenance.

Comparison of Condition Data

Upon receipt of a given batch (ranging from 720 to 5,000 mi [1,159 to 8,047 km]) of service 

provider condition results, Pennsylvania DOT conducts an immediate analysis of the received 

data to ensure that it meets the required acceptance criteria. Data are checked to ensure that the 

IRI and other specified distresses, location data, and video images are in conformance with the 

acceptance criteria shown in table F-5.
Practical Guide for Quality Management of Pavement Condition Data Collection

149

Table F-5. Pennsylvania DOT random sample acceptance criteria (PennDOT 2011).

Reported Value Initial

Criteria

Percent

Within

Limits

Action Criteria if Not Met

IRI ± 25 percent1 95 Reject deliverable

Individual distress severity 

combination ± 30 percent1 90 Provide feedback on potential bias or drift in rating and retrain on distress definitions

Total fatigue cracking ± 20 percent1 90 Reject deliverable

Total non-fatigue cracking ± 20 percent1 90 Reject deliverable

Total joint spalling ± 20 percent1 90 Reject deliverable

Jointed concrete pavement 

transverse cracking ± 20 percent1 90 Reject deliverable

Location – segment and 

offset Correct segment 100 Return to service provider for correction

Location – section begin ± 40 ft (12 m) 1 95 Return to service provider for correction 

and system check

Panoramic images Legible signs 80 Report problem and reject subsequent 

deliverable 1 within agency measured value.

Comparison of Service Provider Historical Data

Individual distress data are plotted using current and the previous 2 years of condition data. Any 

discrepancies noted in the current year data are returned to the service provider for review of all 

segments included in the batch. Figure F-4, for example, shows the total percent of jointed 

concrete pavement distresses plotted over a 3-year period. This analysis indicates good 

consistency (i.e., reasonable trends) in the service provider data. However, additional review 

was warranted due to a higher than expected increase in transverse joint spalling.

Figure F-4. Example distress comparison (redrawn from PennDOT 2011).

0

2

4

6

8

2008 2009 2010

Percent of Total

Year

Long. Joint Spall Tran. Crack Tran. Joint Spall

Joint Faulting Broken Slab Long. Crack
Practical Guide for Quality Management of Pavement Condition Data Collection

150

Comparison of Resulting Dollars Needed for Maintenance

This comparison is used to determine which batch(es) of data provided by the service provider 

are more likely to have the greatest difference in maintenance dollar needs compared to the 

values determined by the Pennsylvania DOT. In this analysis, the average of differences in 

maintenance costs are calculated from the Pennsylvania DOT and service provider condition 

data. For example, a negative number indicates that the service provider rated the distress more 

severely, while a positive number indicates that the service provider rated the distress less 

severely than the Pennsylvania DOT. For the example shown in figure F-5, there is relatively 

good consistency between the service provider and Pennsylvania DOT results since the 

difference in calculated maintenance costs are relatively low (within ± $5,000). This example 

also illustrates that the service provider tended to rate the majority of batches less severely than 

Pennsylvania DOT.

A second analysis of the data batches includes an evaluation of the precision of service provider 

data, which is defined as the differences between the service provider-estimated maintenance 

needs and those determined by the Pennsylvania DOT. For this evaluation the difference in 

calculated costs is plotted for each batch of data along with the random and blind site results (see 

figure F-6). From this graph, the Pennsylvania DOT is able to determine which batches of data 

have the highest difference in costs. Although Pennsylvania DOT does not have a defined cost 

difference that would be considered significant, this graph is helpful in identifying which 

pavement segments may require further investigation. For the example shown in figure F-6, the 

difference in cost tended to be greater in the earlier data deliverables (i.e., batches A through D) 

but was reduced in later batches; this in part is due to the feedback provided to the service 

provider during the analysis of the blind sites.

Figure F-5. Example consistency plot (redrawn from PennDOT 2011).

DOT ratings are 

more severe

Service provider 

ratings are more 

severe

-$6,000

-$4,000

-$2,000

$0

$2,000

A B C D E H I Pilot

Sites

Blind

Sites

Difference in calculated cost

Batch ID
Practical Guide for Quality Management of Pavement Condition Data Collection

151

Figure F-6. Example precision plot (redrawn from PennDOT 2011).

In addition, Pennsylvania DOT also generates tables summarizing the difference in total dollar 

needs by pavement type and by individual treatment group (tables F-6 and F-7, respectively) for 

all pavement segments included in the network-level condition assessment.

Table F-6. Example total dollar needs (modified from PennDOT 2011).

Pavement

Type

Service provider > 

PennDOT $

Service provider = 

PennDOT

Service provider < 

PennDOT Total Δ

Needs

($M) # of 

Segments

Δ $ 

Needs

($M)

# of 

Segments

Δ $ 

Needs

($M)

# of 

Segments

Δ $ 

Needs

($M)

Concrete 6 0.08 18 0 59 -1.11 -1.03

Bituminous 274 1.54 417 0 138 -1.55 -0.01

Total 280 1.62 435 0 197 -2.66 -1.04

Note: Positive cost values indicate the service provider rated the pavement condition more severely than 

Pennsylvania DOT.

For both jointed concrete and bituminous pavements, table F-6 indicates that 280 segments (or 

31 percent) result in a higher costs estimate using service provider-based data rather than data 

provided by the Pennsylvania DOT, 435 segments (or 48 percent) result in equivalent costs, and 

197 segments (or 22 percent) result in lower costs. In addition, the service provider-based 

condition data results in a lower jointed concrete pavement total treatment cost of approximately 

$1.03 million, a lower bituminous total treatment cost of approximately $10,000, and a lower 

total pavement treatment cost of approximately $1.04 million as compared to using condition 

data collected by the Pennsylvania DOT.

$0

$10,000

$20,000

$30,000

$40,000

$50,000

A B C D E H I Pilot

Sites

Blind

Sites

Difference in calculated cost

Batch ID
Practical Guide for Quality Management of Pavement Condition Data Collection

152

Table F-7 provides a similar comparison as shown in table F-6, but by pavement type and 

treatment group.

Table F-7. Example comparison of treatment groups (modified from PennDOT 2011).

Treatment

Group

Number of Segments Service 

provider

Difference

($M)

Service 

provider >

PennDOT

Service 

provider = 

PennDOT

Service 

provider < 

PennDOT

Bituminous Pavements

 Reconstruction 0 0 0 0.00

 Major Rehabilitation 0 69 8 -1.08

 Minor Rehabilitation 6 69 0 0.35

 Seal Coat 0 3 5 0.02

 Routine Maintenance 23 646 0 0.74

Concrete Pavements

 Reconstruction 0 2 2 -0.47

 Major Rehabilitation 0 3 3 -0.62

 Preservation 0 7 0 0.01

 Routine Maintenance 1 65 0 0.05

Total 30 864 18 -0.53

Note: Positive cost values indicate where the service provider rated the pavement 

conditions more severely than Pennsylvania DOT.

Data Edits

Upon completion of the QC evaluation and correction of condition data discrepancies by the 

service provider, Pennsylvania DOT conducts a series of data edit checks prior to uploading to 

the Roadway Management System (RMS) database. These checks include the following 

sequential activities:

1. Duplicate records―duplicate records are noted for correction by the service provider 

and removed from the service provider input file for processing of additional data checks.

2. Survey year―survey year, month, and day are checked to verify dates of testing. Any 

data records with noted errors are written to the service provider error file for correction.

3. Invalid key―invalid key errors include: missing county number, invalid county 

number, missing state route and/or missing segment number, non-numeric data in a 

numeric field, invalid survey date, and invalid state route and/or segment number. In 

addition, zero slab/joint counts for concrete pavements.

4. Missing segment― confirms that there are no missing parent segments.

5. Construction, bridge, lane deviation, and miscellaneous flags―identifies if the proper 

coding has been applied to roadway segments that do not require condition assessment. 

However, in some instances, the service provider may rate the roadway, and this data 

check also ensures that a proper length has been assigned.
Practical Guide for Quality Management of Pavement Condition Data Collection

153

6. Administrative data―includes verifying that the state route data matches the service 

provider data for turn-back and closed-to-traffic roadways.

7. Surface type―confirms that the service provider has correctly recorded the roadway 

segment surface type (including bridge decks).

8. Condition versus segment length―for each roadway segment, the extent of pavement 

severities (by distress type) are summed and compared to the segment length. The 

segment length is subtracted from the distress length, and any values greater than 2.7 ft 

(0.8 m) are flagged for correction.
Practical Guide for Quality Management of Pavement Condition Data Collection

154

APPENDIX G. QUALITY CONTROL AND ACCEPTANCE CHECKLIST
Practical Guide for Quality Management of Pavement Condition Data Collection

155

Pre-Production

Data Quality Standards Yes No

Data collection and distress rating protocols clearly defined and documented?  

Data format, resolution, accuracy, and repeatability specified?  

Acceptance criteria and sampling/evaluation plan specified?  

Corrective actions specified?  

Personnel Training

Collectors trained on agency collection protocols?  

Collectors trained in equipment calibration and start up checks?  

Collectors trained in equipment operation, maintenance, and troubleshooting?  

Raters trained in workstation operation (automated)?  

Raters trained in distress rating protocols and certified (when applicable)?  

Raters tested against lead rater, other raters, and re-trained as needed?  

Equipment Configuration and Calibration

Equipment and software configured according to agency collection protocols?  

Checks/calibration of lasers, DMI, GPS, and computer equipment?  

Conduct sensor “block test”?  

Conduct accelerometer “bounce test”?  

Check video equipment for proper clarity and exposure?  

Test Control and Verification Sites

Adequate number and type of sites chosen?  

Reference values appropriately established?  

Equipment calibration checked by control site testing and promptly reported?  

Distress raters checked for proper application of agency distress protocols?  

During Production

Periodic and Real-Time Data Checks

Sensors, DMI, GPS, and other on board systems functioning properly?  

Camera enclosures are properly aligned and functioning?  

Check tire pressures?  

Ensure video/image quality  

Monitor real-time displays for out of range data/malfunctioning equipment?  

Periodic block and bounce tests, and DMI check  

Control, Verification, and Blind Sites

Sites tested periodically for data accuracy, repeatability, and reproducibility?  

Raters verified through periodic re-rating of distress sites?  

Results tracked and compared against previous testing?  
Practical Guide for Quality Management of Pavement Condition Data Collection

156

End of Day/Other Yes No

Back up and review data for missing data and irregularities  

Review images for clarity and lighting  

Complete log of daily activities and document all QC activities  

Check incoming batches of data for accuracy and completeness  

Distress Rating Checks

Periodic re-rating of batches?  

Periodic re-testing against lead rater or QC rater?  

Periodic cross-rater checks?  

Random samples of ratings checked?  

Post-Production

Global Database Checks

Check data format and completeness  

Check beginning and end of segment location accuracy and total lengths  

Check that the collected pavement type matches the expected pavement type  

Check that data is within expected ranges  

Check that distress types match pavement types  

Check for null, repeating, or error values  

Distress Rating Sample Checks

Comparison with previous year’s data (e.g., large changes in distress)  

Verify that the distress type or severity have been correctly identified  

Verify that the maximum allowable extent of distress has not been exceeded  

Other Data Checks

Comparison with previous year’s data (e.g., large changes in distress)  

Check segments collected using GIS  

Verify that the maximum extent of distress has not been exceeded  

Video Checks

Review image clarity, brightness/lighting, color balance  

Check for any missing images  

Ensure that images are stitched together properly  

Images play in correct order  

Quality Management Reporting

Document equipment and personnel used on project  

Record equipment calibration, checks and maintenance  

Record and report control, verification, and blind site testing  

Document rater checks  

Record and report results of distress rating checks  

Record and report results of global database checks  

Record and report results of all quality issues  