Traffic Safety Data: State Data System Quality Varies and Limited Resources and Coordination Can Inhibit Further Progress
Released: 2010-04-15T00:00:00

Letter

Background

To help identify priorities for highway and traffic safety programs, states maintain six core types of traffic safety data systems: vehicle, driver, roadway, crash, citation and adjudication, and injury surveillance (see table 1). Organizations responsible for implementing and maintaining these systems vary among states, but generally include highway safety offices, law enforcement agencies, motor vehicle offices, courts, emergency medical service (EMS) providers, and others.

While state funds are generally the primary source of funding to implement and maintain these systems, states also use federal funds. SAFETEA-LU provides the Section 408 grant program with the most authorized funding exclusively for traffic safety data systems. Administered by NHTSA, this grant program authorized $34.5 million annually from fiscal year 2006 through 2009. For fiscal year 2009, all 50 states and D.C., received funding through the Section 408 grant program, with amounts ranging from $346,262 to $2.3 million. As stated in SAFETEA-LU, goals of this program are to encourage states to adopt and implement effective programs to: improve the timeliness, consistency, completeness, accuracy, accessibility, and integration of traffic safety data; evaluate the effectiveness of efforts to make such improvements; link these state traffic safety data systems with other data systems within the state; and improve the compatibility of the state data system with national and other state data systems to enhance the ability to observe and analyze national trends in crash occurrences, rates, outcomes, and circumstances.

To receive funding through the Section 408 grant program, states must meet certain requirements, including establishing a traffic records coordinating committee (TRCC), demonstrating measurable progress toward meeting goals and objectives identified in a multi-year highway safety data and traffic records systems strategic plan, and certifying that an assessment of the state traffic records system has been performed within the last 5 years (see table 2).

Among these requirements for the Section 408 grant program, a state TRCC serves to guide and make decisions about traffic safety data systems within the state. The Section 408 grant program requires states to include technical experts on the TRCC, including representatives from highway safety, highway infrastructure, law enforcement and adjudication, public health, injury control, motor carrier agencies, and other stakeholders. In addition to a technical-level TRCC, some states have also established an executive-level TRCC, which can include a manager or director—rather than technical—representatives from state organizations.

To determine state eligibility for the Section 408 grant program and progress toward meeting goals and objectives set forth in a strategic plan, NHTSA has developed six performance measures of data system quality: timeliness, consistency, completeness, accuracy, accessibility, and integration (see table 3). While performance measure definition and relative significance may vary for each system within a state depending on the state’s baseline, goals and objectives, NHTSA officials are working to provide examples of these performance measures to make it easier for states to measure progress. NHTSA expects to finalize these improvements in April 2010.

Traffic records assessments are an evaluation of states’ traffic safety data systems, which includes discussions of how systems met NHTSA’s performance measures. A NHTSA technical team or private sector contractors conduct assessments for states using a “peer” review approach. Technical teams recommended by NHTSA conduct most assessments. The teams are generally composed of five assessors that states approve to conduct the assessment. These assessors have demonstrated expertise in major highway safety program areas, such as law enforcement, engineering, driver and vehicle services, injury surveillance systems, and general traffic records development, management, and data use. The peer review team generally takes about 5 days to complete an assessment, including interviews with state officials, preparing the assessment report, and conducting a final briefing with state officials (see fig. 1). Assessors and NHTSA officials described the principal document to guide the traffic records assessment process as the Traffic Records Program Assessment Advisory, which was updated in 2006, and for the purposes of this report will be referred to as the 2006 Advisory. The format of traffic records assessments was updated to reflect changes made to the original advisory. The principle change made to the assessment format is that the sections describing traffic safety data systems are now combined with previously separate sections describing the information quality.

Besides the Section 408 grant program, SAFETEA-LU authorized other NHTSA grant programs, such as the Section 402 State and Community Highway Safety Grants and the Section 406 Safety Belt Performance Grants, which states can use for any traffic safety purpose, including traffic safety data improvement projects. Also, the Federal Highway Administration (FHWA), the Federal Motor Carrier Safety Administration (FMCSA), and other federal agencies—such as the Centers for Disease Control and Prevention (CDC) and the Department of Homeland Security—have provided support to state traffic safety data projects. For example, the Highway Safety Improvement Program has provided funding to help states achieve a significant reduction in traffic fatalities and serious injuries on public roads through the implementation of infrastructure-related highway safety improvements, which can include traffic safety data projects. A new program is FHWA’s Crash Data Improvement Program (CDIP), which is designed to assist states in developing or improving methods of assessing the quality of their crash data. As part of CDIP, a technical team performs an assessment of a state’s crash data system and then produces a report with recommendations on the establishment of performance measures. FHWA officials reported that after the completion of the assessment, states are eligible to receive up to $50,000 in funding from FHWA to implement recommendations of the report. At the time of this report the program was in its beginning stages and three states had participated so far.

State Traffic Data System Quality Varies, but the Full Extent of Data System Quality Is Difficult to Determine

State Traffic Data System Quality Varies by System and Performance Measure

Data system quality also varies by performance measure. For example, across all traffic safety data systems, states met the consistency performance measure 72 percent of the time, but met the data integration measure only 13 percent of the time (see fig. 3). The comparatively high level of consistency in state data systems may result from states using uniform reporting forms, such as uniform crash, citation, and EMS reports that are consistent with nationally accepted and published guidelines and standards. According to state officials, integrating data systems can be difficult due to older and outdated system design and obtaining cooperation from different data managers. Assessors said that integration is difficult to measure and report on. Further, state and other officials described integration as one of the last performance measures that states tend to focus on in creating high-quality traffic safety data systems while timeliness, accuracy, and completeness are addressed first.

In addition to data system quality varying by system type and by performance measure, our analysis revealed differences in the extent to which individual state systems met various performance measures.

Vehicle and Driver Systems: Vehicle and driver systems met at least 60 percent of the performance measures; specifically, 38 vehicle systems and 31 driver systems met four or more of the six performance measures. Vehicle systems performed best in the area of timeliness—completely meeting that performance measure in 45 states—while driver systems met the accessibility performance measure in 35 states. State officials cited multiple reasons why state vehicle and driver systems may be high- performing compared to other data systems, such as (1) these data systems need to be reliable and customer-oriented since the public has contact with the systems through vehicle registrations and driver license applications; and (2) these data systems generate revenue for states through fees and other charges for vehicle and driver licenses. In one state we visited, revenue collected through the Bureau of Motor Vehicles for motor vehicle licenses and fees amounted to over $90 million in 2009.

Despite the general ability of driver and vehicle systems to meet most performance measures, only seven driver and five state vehicle systems met the integration performance measure. State officials said that integrating driver and vehicle systems with other traffic safety data systems is difficult due to the age of some systems. For example, in one state we visited the vehicle database is 30 years old and has no ability to electronically communicate or integrate with other data systems. In addition, 31 state driver systems met the performance measure for completeness of data. Based on assessments we reviewed, one reason why all states did not have complete driver data may be that some states do not collect previous driver histories from other states for non-commercial drivers. In order to meet the performance measure for completeness, driver histories must be included for all licensed drivers in particular adverse actions received by drivers in other states, either while licensed elsewhere or driving in other states. In addition, having complete records for drivers promotes safety for law enforcement officers conducting roadside traffic stops. For example, an officer can determine whether the driver that he or she has pulled over has a warrant out for his or her arrest or a suspended license, and with access to vehicle data, can find out if the driver is in a stolen vehicle. With this information the officer can better prepare for the interaction, whereas the officer may be more at risk without it.

Roadway Systems: Roadway systems met almost half of the performance measures; specifically, they performed best in consistency—38 states met the performance measure—but, less than half of the states met the performance measure of completeness. According to one assessor and a state official, roadway data plays an important role in state planning. This may lead some states to collect such data consistently. However, in several states we visited, state officials only collected and inventoried roadway characteristics for the state maintained roadways, but less for locally maintained and other roadways, which may contribute to roadway data incompleteness. Nationally, locally maintained roads account for about 77 percent of all public roads, while state maintained roads represent about 20 percent of the total road mileage. In Idaho, of the over 47,000 miles of roadway in the state, the Idaho Transportation Department is responsible for collecting and maintaining data on about 5,000 miles of these roads. The remaining approximately 42,000 miles are the responsibilities of local road authorities. As GAO has previously reported, most states have not developed roadway inventory data for locally maintained roads because they do not operate and maintain those roads and are concerned about costs and time frames involved in collecting the data. In addition, state officials reported that they collect the amount of data on locally maintained roads that are required for the national database—the Highway Performance Monitoring System (HPMS)—which consists of all data collected and updated by states on selected highway segments across the United States. Because of this, officials said detailed data are not collected for all roadways. One effect of incomplete roadway data is that location data for some crashes will make the identification of hazardous locations difficult or impossible and can also prevent states from fully identifying and reporting on potential remedies for hazardous locations and estimating the costs of those remedies.

Crash Systems: While state crash data systems met about as many performance measures as not, our analysis showed, and we have previously reported, that state crash data systems varied considerably in the extent to which they met NHTSA’s performance measures. For example, crash data systems in five states met all six performance measures, while systems in six states did not meet any of the performance measures. In addition, systems in 27 states met two or fewer of the six performance measures. Also, according to our analysis, crash systems in 32 states met the consistency performance measure. Several of the states that we visited had uniform crash report forms used by law enforcement to report vehicle crashes, which may contribute to the consistency of crash data. Traffic records assessors, NHTSA officials, and state officials said that states have tended to focus on improving crash data systems, in part, due to crash data having a clearer link to improving public safety than other traffic safety data systems. However, systems in 23 states did not meet the performance measure for crash data accuracy. Manual data entry and a lack of electronic edit checks could lead to less accurate data, which can inhibit meaningful analysis. For example, in one state we visited, law enforcement officers provided incorrect longitude coordinate data using Geographic Information Systems (GIS) equipment. This human error resulted in inaccurate crash location data; in multiple instances, the computer program located crashes in China.

Citation and Adjudication Systems: For citation and adjudication systems, about as many performance measures were met as were not met; specifically, systems in 18 states met four or five performance measures, while systems in 21 states met one or none of the six performance measures. However, 17 percent of the time, the extent to which the performance measure was met was unknown. Citation and adjudication systems performed best in consistency—38 systems met this performance measure. Similar to crash data, the adoption of uniform citation forms may have improved consistency for this system. However, about half of state citation and adjudication systems did not meet accessibility and completeness performance measures, and only one state met the integration performance measure. These performance measures may be difficult for some states to meet due to the high number of jurisdictions that states rely on to report data or because a statewide citation system may not exist. For example, Georgia officials said that the state has nearly 800 different courts—about 400 of which are municipal courts, which handle most traffic violations—each with its own court data system. There is no comprehensive collection of citation data in the state, and the state has a limited ability to require jurisdictions to submit data. Georgia officials said that citation and adjudication data are relatively incomplete because some courts do not report all data. Also, if states do not have an electronic citation system, even police departments with the ability to submit citations electronically must submit their citations on paper. For example, a law enforcement officer from one state we visited said that his department has the capability to electronically submit citations, but must still print out citations to submit them to the state because the state is not able to electronically receive citations.

Injury Surveillance Systems: Less than half of the performance measures were met, but similar to the citation and adjudication systems, the extent to which performance measures were met was unknown 17 percent of the time. Systems in 39 states met 3 to 0 performance measures, while systems in 12 states met four to six. In addition, within injury surveillance systems, 25 states met the performance measure for accuracy and 30 states met the performance measure for consistency. This may be attributed to training provided to those responsible for data entry. For example, one state hospital administration provides training to data entry staff on how to enter cases into the state data system properly. In contrast, 40 state injury surveillance systems did not meet the performance measure for integration. Assessors and one state official reported that the multiple components necessary for a state injury surveillance data system make meeting various performance measures more difficult than for other data systems. According to the 2006 Advisory for traffic records assessments, a complete injury surveillance system typically has five components: pre- hospital (i.e., EMS), trauma, emergency department, hospital in- patient/discharge, and rehabilitation to track injury causes, magnitude, costs, and outcomes. Officials said that maintaining multiple components often requires that several departments contribute data, which can make data management difficult. For example, in Minnesota, the EMS Regulatory Board collects EMS data, the Minnesota Hospital Association collects patient discharge information, and the Minnesota Department of Health maintains the Minnesota Trauma Data Bank, which contains trauma and mortality data, all of which are reported to the Minnesota Department of Health. In addition, systems in several states have only some components of a fully functioning injury surveillance system in place or have system components that are just in the beginning stages of development.

Many Traffic Records Assessments Are Incomplete or Inconsistent; therefore, the Full Extent of Data System Quality Is Difficult to Determine

Although NHTSA’s implementing guidance for the Section 408 program states that a traffic records assessment should be an in-depth, formal review of a state’s highway safety data and traffic records system, our analysis revealed instances where assessments were incomplete or inconsistent. We assigned “unknown” codes where no other categorization was possible due to limited or otherwise absent information in a state traffic records assessment, which includes both incomplete and inconsistent performance measure descriptions. The results of our analysis were that 49 of 51 traffic records assessments had at least 1 area out of 36 (six state traffic data systems multiplied by six performance measures) for which the extent to which a system met a performance measure was unknown. Incomplete or inconsistent information could limit the usefulness of these assessments to state officials and make it difficult to ascertain the full extent of data system quality. NHTSA officials said that they review traffic records assessments for quality and that they have accepted all state assessments as adequate to fulfill the statutory requirement included in NHTSA’s Section 408 grant program implementing guidance. NHTSA officials said that they are currently beginning work with a contractor to study the assessments. While the contract to study the assessments includes a component to examine state traffic records assessments for effectiveness and utility, the main objective is to review state traffic records programs and data systems from states that have had at least two traffic records assessments and identify any improvements or degradations that occurred between the two assessments. In addition to the contract, NHTSA officials reported starting other activities, which will include updating related advisory documents, increasing participation of other DOT administrations, aligning traffic records assessments with other similar NHTSA program assessments, determining the most effective frequency for requiring assessments, incorporating all performance measures identified in advisory documents, and developing a more robust list of assessors for states. As these efforts are in the beginning or planning stages, it is too soon to tell how they will impact the traffic records assessment process.

Our review of traffic records assessments showed that for those traffic records assessments that had any unknown areas, the number of unknown areas ranged from 1 to 18 out of a possible 36, but most assessments had five or fewer unknown areas. Of the 49 assessments we coded with unknown areas, 27 had between 1 and 3 unknown areas and 6 had 10 or more (see fig. 4). Out of the total 1,836 codes that we assigned across all 51 assessments, 226 (about 12 percent) were coded as unknown.

Our coding analysis revealed that the frequency of unknown areas is greater in the updated assessment format compared to the prior assessment format. Of the 51 assessments we reviewed, 11 (22 percent) were in the updated assessment format. Despite the lower number of assessments in the updated format, proportionally, we coded about three times as many areas as unknown in the updated assessment format than the prior format. The updated traffic records assessment format, which is based on the 2006 Advisory, is less tied to NHTSA’s Section 408 grant program implementing guidance than previously. The 2006 Advisory describes what characteristics state traffic safety data systems should have, but unlike NHTSA’s implementing guidance and the prior advisory, in several areas it does not include a discussion of each of the six performance measures as they relate to each of the six data systems. For example, the 2006 Advisory notes that data should be timely and includes an example of a quality control measure for timeliness, but unlike the prior advisory, does not establish a specific time frame by which timeliness can be assessed. The 2006 Advisory also does not expressly discuss the accessibility performance measure for five of the six traffic safety data systems. This means that for five of the six data systems, the 2006 Advisory addresses only four of the six performance measures.

As previously noted, several assessments were incomplete, meaning that there was not enough information provided to determine the extent to which a state had met a performance measure. There was one instance in which an assessment lacked performance measure evaluation information on that state’s entire injury surveillance system since “representatives of the various medical data systems were not present during Traffic Records Assessment. Therefore no information related to timeliness, consistency, completeness, accuracy, accessibility, and integration…could be presented in report.” In other instances, we were unable to make a determination based on information provided in the assessment. For example, in 14 traffic records assessments it was unclear whether citation and adjudication data were timely. In another assessment, the timeliness of injury surveillance data was explained as the timeliness of EMS arrival time as opposed to the timeliness of when the injury data are available for analysis. Incomplete injury surveillance data may lessen a state’s ability to track injury causes, magnitude, costs, and outcomes.

Some incomplete assessments may result from differing views on the value of various performance measures between NHTSA officials and traffic records assessors. According to NHTSA officials, the six performance measures across the six data systems are of equal importance in the context of assessing a state’s qualification for subsequent year Section 408 grant funding. NHTSA officials also said that making progress in one data system or performance measures is not more highly valued than making progress in another. Additionally, NHTSA officials said that part of the value of assessments is that they provide information on all areas of states’ traffic safety data systems. In contrast, some of the assessors we interviewed questioned the value of some or all of NHTSA’s performance measures for the various state traffic safety data systems. For example, assessors said that information on the integration performance measure was not valuable because it is difficult to measure. Others said that injury surveillance data assessors focus on integration more than the other performance measures and that one of the most important findings in the injury surveillance section of a traffic records assessment is how it integrates with other traffic safety data systems. Furthermore, some assessors reported that they do not evaluate certain performance measures if it appears that nothing has changed in a state since the last assessment, and that some performance measures and traffic safety data systems are not as important as others.

As noted previously, the principal document used by assessors as a guide for the traffic records assessment process is the 2006 Advisory. The purpose of this guidance is to provide states with guidance on the necessary contents, capabilities, and quality of data in a traffic records system and to be a description of an ideal system, not to describe what information should be included in a traffic records assessment. Furthermore, as opposed to the previous advisory, the 2006 Advisory explicitly discusses some, but not all of the six performance measures for each traffic records systems. Given that, per Section 408 grant program requirements, assessments are conducted every 5 years, there is merit in having clearer guidance that assessments include all performance measures to update state officials on their traffic safety data systems, even if such an update explains that nothing has changed since the last assessment.

In addition to completeness concerns, some traffic records assessments are inconsistent, meaning that information provided in one part of the assessment describing the extent to which a state met a performance measure was inconsistent with information provided elsewhere in the assessment. For example, one assessment described the performance measure of consistency as “…not to be an issue in that a uniform citation is used and there are a relatively small number of police agencies …that submit traffic citations.” However, later on the same page in the accuracy section it was noted that “The Court indicated that officer reporting is not consistent and more training is needed to assure that charging documents and affidavits of probable cause are completed correctly. Additional training could help to assure uniformity of submissions.” Another assessment explained, “Information provided during the assessment interviews indicated that the data are timely; the latest data available for analysis is 2006.” Upon review, the available data were at least a year old since the traffic records assessment was conducted in 2008. However, NHTSA guidance suggests that all injury data be available in a comparable time frame for the crash data, which is preferably within 90 days of a crash.

Despite these limitations, traffic records assessments remain vital in helping states identify problems, develop plans, and prioritize projects to improve traffic safety data systems (see fig. 5). For example, Minnesota officials used recommendations made in a traffic records assessment, along with the strategic plan, to prioritize traffic records projects. In addition to being useful to states for making traffic records improvements, NHTSA officials emphasize traffic records assessments as valuable for strategic planning purposes. NHTSA officials added that the traffic records assessment process is important because it provides an independent look at the quality of traffic safety data systems, helps determine where priorities should lie, and guides states on targeting limited resources. Assessors and state officials also emphasized the value of traffic records assessments for states. For example, one assessor said that traffic records assessments serve as a tool and guideline for states in how to move forward with traffic safety data systems and to promote a data-driven approach by balancing stakeholder interests with priorities highlighted by data.

In contrast, state officials reported that while information captured in traffic records assessments is useful, the more specific information such as problem identification, definitions of performance measures and data analysis recommendations included in FHWA CDIP assessments has additional benefits. Although CDIP began in 2008 and only three states have currently participated, officials in states where both a traffic records and CDIP assessment were conducted said that the information included in CDIP assessments was more in-depth and specific. CDIP assessments are conducted in a similar manner as traffic records assessments, take roughly the same amount of time to conduct, and cover all six performance measures identified in NHTSA’s implementing guidance, but focus only on a state’s crash data system. CDIP assessments include recommendations and particular steps or methods states can take to potentially improve their crash system.

By contrast, assessors identify problems in traffic records assessments but state officials said that traffic records assessments generally do not provide specific strategies for ways to improve the traffic safety data systems. State officials reported that an assessment with information as specific as that provided in CDIP assessments would be valuable to have for each of their traffic safety data systems. In addition, several state officials said that insufficient time is spent conducting traffic records assessments to produce an in-depth, detailed report. In one state, traffic records assessment officials spent only 10 minutes with the team representing one of the six data systems.

States Have Demonstrated Progress in Some Data Systems, but Face Resource and Coordination Challenges

States Have Demonstrated Some Progress Nationwide

Information collected by NHTSA from the states shows that 49 states and D.C. have demonstrated progress in improving the quality of all six traffic safety data systems. States have demonstrated progress in all six traffic safety data systems, as well as across all six performance measures. It is important to note that reported state progress is not equivalent to achieving a high-quality traffic safety data system; rather, such progress represents steps toward that end goal. Of the possible 36 areas in which to demonstrate progress, by system and by performance measure, states demonstrated progress in 23 areas to NHTSA from fiscal year 2008 through fiscal year 2009 (see table 4). To remain eligible for Section 408 grant funding states must demonstrate measurable progress related to achieving the goals and objectives of a state’s multi-year highway safety data and traffic records strategic plans. NHTSA officials reported that states can fulfill this requirement by demonstrating progress in one performance measure for one data system per year. For example, a state might report progress involving the performance measure of completeness within the roadway data system. States can and have reported more than one area of progress. NHTSA does not require states to report all progress toward improving traffic safety data systems and, as a result, states may be making progress that is not reported. Additionally, NHTSA does not always accept every area of progress that a state reports if the state demonstrates sufficient progress in at least one area; therefore, state progress may be understated. Sometimes NHTSA cannot verify that progress has taken place in all reported areas, due to a lack of evidence or incomplete information. For example, Maine officials reported five areas of progress to NHTSA for fiscal year 2009 and NHTSA officials accepted four of those areas. West Virginia officials reported four areas of progress, one of which NHTSA officials accepted. While NHTSA officials reported that demonstrated progress does not represent all progress that states are making, it serves as a useful approximation for the areas in which states are making progress. Moreover, NHTSA officials said that in regards to qualifying for Section 408 grant funding, the most important development is that states are making some progress in improving traffic safety data systems.

State progress, for the 2 most recent fiscal years, may reflect some trends identified by our analysis of the extent to which state traffic safety data systems met NHTSA performance measures. For example, states demonstrated the least progress in the vehicle and driver data systems (7 of the 164 total areas of progress listed in table 4). This may reflect that vehicle and driver systems already met most performance measures, as shown in our coding analysis. In contrast, states have demonstrated progress for crash data systems more often than other systems. Out of the 164 instances states have demonstrated progress, 89—over half—involved improvements to state crash data systems. This may indicate heightened state efforts to improve crash data systems due to these systems not meeting various performance measures, as shown in our analysis. Furthermore, state and NHTSA officials, as well as assessors, reported that states have focused on improving crash systems.

Progress has resulted from states pursuing small- and large-scale projects to improve traffic safety data systems. For example, some progress has resulted from smaller-scale projects, such as printers for citations or online tutorials. NHTSA officials said that they have encouraged states to use Section 408 grant program funding to support near term, quick projects, recognizing that large-scale projects might require significant, additional time or funds. However, some state officials said that smaller- scale projects are less likely to immediately lead to substantial improvements in the overall quality of state traffic safety data systems.

Support for large projects also depends on state funding in addition to Section 408 grant program funding awarded to a state. For example, Virginia has expended over $900,000 in state and local funding on the Traffic Record Electronic Data System (TREDS) project, which integrates federal, state, and local data; provides law enforcement the ability to collect and submit crash data electronically; reduces manual entry of data; provides enhanced analysis capabilities and increases accessibility for data users; among other things. Thus far, NHTSA has awarded Virginia approximately $2.5 million in Section 408 grant funding.

For the states that we visited, federal assistance has helped states to improve traffic safety data systems. Officials in all eight states that we visited stressed the important role of the Section 408 grant program to improve traffic safety data and have used this and other federal funding to implement projects. Officials reported that while state funding makes up the majority of support for traffic safety data projects, without Section 408 grant program or other federal funding some projects would have happened much more slowly, or not at all. NHTSA officials estimated that for every dollar provided through Section 408 grant funding, states spend an additional $4 for traffic safety data projects. Below are examples of state projects that have used federal funding.

Timeliness–Several states have implemented or are currently working on projects to transition from manual to electronic reporting of data. Electronic reporting reduces reliance on paper processes and can increase the speed of submission and eventual availability of data for analysis. Minnesota has undergone such a transition for crash data. Minnesota officials said that in 2009 over 90 percent of the state’s crash reporting was submitted electronically to its crash database. This includes all crash reports from Minnesota’s State Highway Patrol. Electronic submission has helped the state submit and finalize all data in Minnesota’s crash database within 30 days. This represents an improvement from the 6-week backlog to enter crash data that Minnesota experienced in 2003.

Completeness–To improve the completeness of crash data, officials in three states that we visited reported using diagram software to help law enforcement officers depict crashes. Officers generate these diagrams by entering information electronically at the scene of a crash (see fig. 6). The diagram increases completeness by including visual information like the position of the vehicle(s), location of damage, intersection layout, and other crash features, such as trees and pedestrians. Using crash diagram software, officers can edit information before completing and submitting the diagram as part of the crash report.

Consistency–Some states have improved consistency by adopting uniform reporting forms and increasing compliance with national guidelines. In 2007, Virginia revised its crash data collection form using guidance from NHTSA and guidelines captured in the Model Minimum Uniform Crash Criteria. Virginia officials reported that the form revision increased compliance with the Model Minimum Uniform Crash Criteria from 55 to 80 percent. Also, Georgia’s Emergency Medical Services Information System has used a revised form that includes approximately 300 data elements— as opposed to the previous form, which had 103. This revised form is “gold compliant” with National EMS Information System guidelines. Approximately 30 percent of Georgia’s EMS agencies are still using the previous forms, but state officials expect a continued transition to the new form.

Accuracy–To improve the accuracy of roadway data, including roadway features such as bridge locations, some states have explored projects available through GIS and other technology. Maine’s Department of Transportation has created the Maine Department of Transportation Map Viewer System, which will eventually become available to a variety of state data users. This system integrates existing GIS technologies into a viewer screen where users can view roadway data and update information to increase accuracy. Users of the viewer system can also select and change which data are displayed and view photographs of a particular section of roadway to illustrate local features (see fig. 7).

In another example, one law enforcement jurisdiction that we interviewed installed video data recorders in police vehicles. These devices record scenes to the front or rear of a vehicle. Uses of these recorders include reviewing crash footage to verify information and ensure that crash, driver, and vehicle data are accurate (see fig. 8).

Accessibility–Several state and local jurisdictions we met with, including those in Maine, Minnesota, and Ohio, have completed projects to make traffic safety data more accessible to users. For example, data captured by Ohio’s Location Based Response System (LBRS) is available to data users and other citizens on the Internet.

Ohio officials reported that this has increased accessibility to roadway information, and reduced public requests for roadway data. In addition to state Department of Transportation officials, LBRS users have included County Emergency Management Agencies, utilities, and county engineers. In addition to LBRS, other projects have included jurisdictions incorporating new technologies to make crash, driver, citation, and vehicle data more accessible in law enforcement vehicles. Figure 9 provides examples of other, completed projects that have increased the accessibility of various data systems for law enforcement officials.

Integration–To improve the integration of traffic safety data systems with one another, 19 states participate in the Crash Outcome Data Evaluation System (CODES) effort. Facilitated and supported by NHTSA, CODES seeks to better link and otherwise integrate crash and injury surveillance data. Such integration can result in state officials better understanding the medical consequences of traffic crashes and the types of injuries that certain crashes are likely to produce. Of the states that we visited, Georgia, Maine, Minnesota, Ohio, and Virginia participate in the CODES project. Ohio officials reported that the most extensive linkage between injury surveillance systems in the state has happened through the CODES program, which has established links between EMS, trauma, and crash data. Virginia officials cited CODES in helping to submit and link data to other organizations including the Department of Motor Vehicles.

States We Visited Face Resource and Coordination Challenges, but Some Have Implemented Strategies to Help Address These Challenges

While states have demonstrated progress, a number of overarching challenges exist to improving traffic safety data systems. This is due in part to the complexity and multifaceted nature of trying to establish traffic safety data systems. The Section 408 grant program is designed to improve six, oftentimes completely separate, state traffic safety data systems. We have previously reported that overhauling one outdated data system can be both challenging and expensive, particularly when integrating a new system with existing legacy systems. State officials in all states that we visited also reported that just maintaining one data system requires significant funding, time, or other limited resources. Therefore, trying to make simultaneous improvements to multiple traffic safety data systems can magnify these challenges.

Limited Resources: Officials in all the states that we visited identified limited resources as a significant challenge in state efforts to improve traffic safety data systems. Some of the most frequently cited limitations in funding and human capital resources are discussed below.

Limited funding. According to state officials, making improvements to one data system can cost tens of millions of dollars. Therefore, obtaining funding necessary to make improvements to six state traffic safety data systems is a challenge. As we previously reported, while traffic safety data grants have provided states with funding to improve traffic safety data systems and complete associated projects, the cost of developing and maintaining data systems can exceed Section 408 program grant amounts. While state officials reported that state funding supports most of the cost of traffic safety data projects, NHTSA and officials in five out of eight states we visited indicated that traffic safety data system improvements are not among the highest state priorities due to budgetary constraints or limited interest. The recent economic recession has amplified state funding limitations for data projects.

Moreover, a state’s legislative process may delay funding for traffic safety data projects. Even in instances where funding is available, some traffic safety data improvements require state legislative action or approval to move forward on contracting, design, and implementation processes. Infrequent state legislative sessions can heighten delays in receiving approval to spend awarded federal funding. For example, according to state officials, the legislature in one state we visited meets every other year, which can delay approval of spending of federal grant and other funding on traffic safety data projects and contribute to carry over of funds. In another state, major technology projects must first be approved by the state’s information technology authority. The project planning involved to obtain state approval can make some projects cost prohibitive. For example, the state wanted to update the injury surveillance system 4 years prior, but had to obtain approval first, which resulted in delays in implementation and a doubling of the project’s costs.

Limited human capital resources. States that rely on paper crash and citation forms require manual, time-consuming data entry, which can strain resources and lead to backlogs in data. For example, the Texas Department of Transportation assumed responsibility for the state’s crash data system in 2007 from another state department, and also assumed responsibility of a backlog of some 3 million crash reports over a 5-year period that needed to be entered into the data system (see fig. 10). The accumulated backlog was the result of the state’s use of a manual crash data system designed in the 1970s prior to implementing the state’s electronic crash data system in 2008. According to a Texas Department of Transportation briefing report, the manual process was inefficient, resource intensive, and not conducive to the timely dissemination of data.

In some states, there are only a few staff that manage a state’s traffic safety data programs and grants. This is significant because state officials reported that grant applications are time consuming and difficult to balance with other key job responsibilities. In one instance, a state we visited had to return federal grant funding because it did not have available staff resources to effectively manage the grant and associated project. A regional NHTSA official also reported that the turnover and training of new, state staff can be a challenge, particularly when staff must be trained on the specifics of the Section 408 grant program due to limited institutional knowledge. Furthermore, NHTSA officials reported that regional meetings have helped state officials obtain contacts and share leading practices, but state budget restrictions have curtailed these meetings, removing this training opportunity and resource. NHTSA officials reported that they have recently begun using online webinars as an alternative for national, state, and regional audiences.

Training individuals is an important component in ensuring the collection of high quality traffic safety data, as recommended in several traffic records assessments. However, a number of state officials told us that training on data collection may be limited due to funding and resource constraints, such as staff resources and travel expenses. In several states, officials reported that the local law enforcement officers collecting the data may not fill out a crash report completely or accurately, or submit the form in a timely fashion, which may lead to instances where crash data are inaccurate, incomplete, and untimely.

Officials in several states reported that information technology resources are limited and that state agencies often have to share staff with technical expertise between different data systems and projects. Due to limited internal technical expertise, some states have used contractors, but state officials reported that this can be expensive. Also, some states have a limited list of contractors a state will approve or technologies that the contractor can offer. For example, officials from one state we visited reported that the technologies provided by contractors were not completely compatible with existing local traffic safety data systems, which limited its usefulness. However, we have previously reported that hiring a contractor can help states obtain the technical expertise needed to efficiently integrate data systems.

In light of these challenges, some states have implemented strategies to overcome resource limitations. For example, North Carolina’s Governor’s Highway Safety Program office took a series of targeted, incremental steps to first focus on improving the quality of two traffic safety data system performance measures—specifically timeliness and accuracy—in each system before working on other performance measures, such as integration. State officials emphasized the importance of focusing on the “basics” and working from there, rather than starting with the most complicated improvements. For example, North Carolina initially used Section 408 grant program funding to create a guidebook that provides consolidated information on all six traffic safety data systems and their status. The guidebook enabled state officials to identify the most pressing needs among all six traffic safety data systems and target limited resources. Although the primary function of the guidebook was to increase the accessibility of data system information, it also helped state officials recognize the need to integrate traffic safety data systems to increase data accessibility between data systems. Accordingly, North Carolina has an active project to complete the linkage of crash and injury surveillance data. Although the amount of Section 408 grant program funding is small compared to state funding, North Carolina officials explained that the program is a catalyst for progress by sometimes supporting smaller projects like the guidebook, which then pave the way for larger projects, such as integrating data systems.

According to state officials and one assessor, another strategy that one state has used to overcome limited funding and staff was to contract out the management of its centralized crash data system. For the state, this project was revenue neutral because it does not require additional funds for continued maintenance, as the contracted vendor receives payment by selling crash reports and data extracts to interested parties. The profit gave the vendor an incentive to work diligently with law enforcement agencies to ensure reports are complete, accurate, and submitted in a timely fashion to the central data system. There was also a built in incentive for the law enforcement agency that submits the crash reports as it also receives a reimbursement of 67 percent of the cost of each report sold. As a result of contracting out the crash data system, the state eliminated an annual cost of over $1 million for staffing, consulting, and system maintenance, and no longer requires annual federal funds to help support the system. This funding has since been redirected to hire additional state troopers and add additional staff where needed.

Coordination Issues: Officials in all states we visited identified coordination issues that presented challenges in improving state traffic safety data. Some of the most frequently cited coordination issues are discussed below. “Stove-piped” agencies. Custodians of the different state traffic safety data systems are oftentimes housed in different state offices or agencies. A number of federal officials, state officials, and assessors reported instances of unwillingness to share data between various offices because of the “stove-piped” structure where there is little interaction between traffic safety data stakeholders. Furthermore, we heard from state officials and assessors that there is not always a clear understanding of the relationship among all six traffic safety data systems. Typically most of the data systems are housed within a state’s Department of Transportation or Department of Public Safety, which can compound coordination challenges for data systems housed elsewhere (e.g., injury surveillance data).

Privacy concerns. According to state officials and assessors, federal and state privacy laws can limit accessibility and sharing of certain traffic safety data. A traffic records assessment for one state that we visited reported that restrictions placed on release of crash data in general, and of personal identifiers in the crash data for use by analysts within state government offices, posed major barriers to crash data analysis. This assessment also reported that these restrictions do not affect the state’s ability to generate reports such as annual crash reports or most ad hoc analyses of the state’s crash experience, but does limit the state’s ability to perform more detailed crash problem identification and to support research into the safety implications of specific laws or policies.

Decentralized state governance structures. State governance structures can further complicate coordination efforts. For example, decentralized court systems such as those found in two states we visited make it difficult for the state to collect adjudication data from lower-level courts. Addressing such governance issues can take many years. For example, Minnesota officials said that the state has worked to centralize its court system over a 15-year process. State officials and assessors also reported that there is little incentive for jurisdictions, agencies, and individuals to collect and submit data in a timely fashion. Some states have mandated deadlines for the submission of data, but these deadlines are not always adhered to by all agencies required to report. Although some states have the ability to sanction those jurisdictions that do not submit data, this option is not always used.

Federal and state officials, as well as assessors, told us that executive-level TRCCs, which include key decisionmakers such as agency directors, can help technical-level TRCCs overcome a variety of coordination and resource impasses. The technical-level TRCCs have been one of the successes of the Section 408 grant program and NHTSA officials and officials in nearly all of the states that we visited praised TRCC activities in bringing state stakeholders together, establishing important relationships, and moving traffic safety data systems forward. While technical-level TRCCs may lack the authority to implement certain decisions or traffic safety data projects, according to NHTSA officials, several assessors, and state officials, the authority associated with executive-level TRCCs can help prioritize traffic safety data improvements and coordinate efforts. For example, assessors explained that if a data manager refuses to share data, an executive-level TRCC could compel data sharing. They also said that the involvement and support of executive-level decision makers can raise the profile of traffic safety data projects, which do not always receive much attention, and provide the necessary leadership to complete traffic safety data improvement projects. NHTSA officials also noted that executive-level TRCCs can help states commit resources to traffic safety data projects. For example, officials in one state reported that information technology staff sometimes have not prioritized traffic safety data projects due to limited resources. However, executive-level TRCC representatives in that state have the authority to target and dedicate these sometimes limited information technology resources to traffic safety data projects. Figure 11 depicts some of the advantages of an executive-level TRCC.

Officials from North Carolina, a state we visited with an executive-level TRCC, reported that the state’s executive-level TRCC oversees all highway safety issues and fills the role of “champion” for the state’s initiatives. All of the technical-level TRCC’s activities are reported to the executive-level TRCC. The executive-level TRCC helps the technical-level TRCC prioritize issues, provide assistance on legislative initiatives or interagency projects requiring significant resources. Currently, the state is developing new traffic safety data projects that will require legislation to be passed for funding. A state official said that executive-level TRCC endorsement and support will be necessary to pass the legislation.

While advantageous, currently few states have established executive-level TRCCs. A NHTSA study recommended that states should have both a technical-level and executive-level TRCC to be successful, but as previously noted, only technical-level TRCCs are required for states to be eligible for funding under the Section 408 grant program. Based on estimates from one traffic records assessor, as of November 2009, nine states had an executive-level TRCC. Several traffic records assessments, however, have recommended that states establish executive-level TRCCs to help improve traffic safety data systems.

Rural and Urban Areas May Face Distinct Challenges

Rural and urban areas across the country faced some distinct challenges in improving traffic safety data systems. As previously discussed, some state roadway data systems do not include locally maintained roadway data, which may include rural road data, and therefore do not provide a full picture of a state’s roadway system. As previously reported, many states have not developed roadway inventory data for locally maintained roads because they do not operate and maintain those roads, and are concerned about the possible costs and time frames involved in obtaining these data. As a result, states may have difficulty applying a data-driven, strategic approach to highway safety. In addition, despite the higher proportion of fatalities occurring in rural areas, officials in one state expressed concerns that a proportional amount of state traffic safety funding is not allocated to reflect this higher fatality level. We have also previously reported that limited data on rural roads can hinder state efforts in funding and addressing its top traffic safety priorities. However, some states are working to improve data on non-state owned roadways, including rural roads. For example, Ohio’s LBRS established a partnership between state and local governments and has allowed Ohio’s Department of Transportation to expand roadway data to include more comprehensive roadway information. Figure 12 depicts a map of Ohio’s Clark County, showing the 188 percent increase in the number of located crashes available for analysis through LBRS. This increase largely consisted of crashes occurring on locally maintained roadways.

An additional challenge involves the volume of vehicle crashes affecting when and how much data rural and urban areas can submit. For example, state officials in two states we visited reported that rural areas submit crash data more regularly due to lower volumes. According to one assessor, some large urban law enforcement agencies have refused to report crash data, leading to gaps that limit the state’s ability to make decisions that effectively target resources. In contrast, officials in three states we visited reported that urban areas find it more difficult to submit crash data in a timely manner due to the large volumes of reports filed. Further, some cities have their own discrete crash data systems due to their high crash rates. According to officials in one state we visited, though large cities may have their own crash records systems, the system may not be linked to the state crash data system and contributes to a large number of missing crash reports.

Some rural areas face additional challenges due to limited technology options. The lack of telecommunications services, such as access to the Internet, limits the ability of local jurisdictions to electronically submit data to state data systems, which can reduce the timely submission of data. We have previously reported that the cost of providing telecommunications services is higher in rural areas than in urban areas, in part due to lack of infrastructure. For some rural jurisdictions, even when the technology is available, it may not be cost effective to use due to lower volumes of traffic safety data submitted per year.

Officials from states we visited reported on some strategies being implemented to overcome some of the challenges for rural and urban areas. For example, in one state we visited, the state’s highway safety office provided funding to equip state highway patrol vehicles in rural areas with mobile data terminals. Currently, roughly 70 to 80 percent of state highway patrol vehicles in rural areas have these terminals, which have increased the timeliness of crash reports submitted in the state. In another state, the state legislature created an organization to oversee funding for rural and locally maintained roadways. This organization had the mission of helping local agencies receive funding specifically targeted at locally maintained roadways. Lastly, officials in another state we visited reported an increase in the electronic submission of crash reports when the state required at least a certain percentage of crash reports to be electronically submitted in order to qualify for the Section 402 grant funding. State officials identified certain urban areas that were either underreporting crashes or not electronically submitting crash reports and then worked with these jurisdictions to improve submission rates. Since 2007, one urban area in this state has increased its electronic submission of crash reports from 44 percent to nearly 100 percent by the end of 2009. Overall, 91 percent of all crashes are now being electronically submitted in this state.

Conclusions

Improving state traffic safety data systems is critical to state efforts to use data-driven approaches to improve traffic safety and reduce traffic fatalities and injuries. The Section 408 grant program has helped states to improve the quality of traffic safety data systems across NHTSA’s six performance measures. Despite this progress, however, almost all states have traffic safety systems that do not meet one or more performance measures. The wide range of quality we found in state traffic safety data systems underscores the importance of state traffic records assessments in helping states to plan and prioritize improvements to traffic safety data systems. However, incomplete and inconsistent information in the assessments limits the usefulness of the assessments, which, according to NHTSA’s implementing guidance, should be an “in-depth, formal review of a state’s highway safety data and traffic records system.” Furthermore, assessments in the updated traffic records assessment format currently being used often do not systematically evaluate each of the six performance measures as they relate to each of the six data systems. Improving the completeness and consistency of assessments would help states more accurately identify problems and effectively target limited resources. NHTSA officials recognize the importance of these assessments for states and are taking steps to identify improvements to some aspects of the assessment process. However, NHTSA’s efforts to review the assessment process and the effectiveness and utility of traffic records assessments are in the early stages. Based on NHTSA’s statement of work for the study, the contract includes a component to examine state traffic records assessments for effectiveness and utility and identify any improvements or degradations of traffic safety data quality. However, it is unclear whether this review will evaluate the overall content and quality of the information provided in the assessments to the level of specificity that may be needed.

States face various resource and coordination challenges, which make further progress in improving the quality of traffic safety data systems difficult. State officials we spoke with noted several strategies to address these challenges. One of these strategies—establishing an executive-level TRCC—can potentially address multiple resource and coordination challenges. Specifically, an executive-level TRCC can be a helpful tool for states to prioritize traffic safety data improvements, coordinate efforts, and overcome impasses. Although the Section 408 grant program requires that states have a technical-level TRCC, it does not require states to establish an executive-level TRCC. The establishment of an executive-level TRCC holds promise, but we did not fully assess its value for states as it was beyond the scope of this report.

Recommendations for Executive Action

We recommend that the Secretary of Transportation direct the NHTSA Administrator to take the following two actions: Ensure that traffic records assessments provide an in-depth evaluation that is complete and consistent in addressing all performance measures across all state traffic safety data systems. As part of NHTSA’s ongoing initiatives to improve the traffic records assessment process, specific efforts could include revisiting available assessment guidance, the frequency and manner in which assessments are conducted, and NHTSA’s assessment review process.

Study and communicate to Congress on the value of requiring states to establish an executive-level TRCC in order to qualify for Section 408 grant funding.

Agency Comments

We provided a draft of this report to DOT for review and comment. DOT officials agreed with the findings and recommendations in the report and offered technical corrections that we incorporated, as appropriate. Regarding the recommendation to ensure that traffic records assessments provide an in-depth evaluation that is complete and consistent, the officials noted that NHTSA has begun several initiatives to identify opportunities to improve the assessment process and provide the states with a more effective assessment document.

We are sending copies of this report to the Secretary of Transportation and interested congressional committees. The report is also available at no charge on GAO’s Web site at http://www.gao.gov.

If you or your staff have any questions concerning this report, please contact me on (202) 512-2834 or flemings@gao.gov. Contact points for our offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

In response to your request, this report provides information on the status of the quality of state traffic safety data systems. In particular, we sought to identify (1) the extent to which state traffic safety data systems meet National Highway Transportation Safety Administration’s (NHTSA) performance measures for assessing the quality of data systems, and (2) what progress states have made in improving traffic safety data systems and what challenges remain.

To identify the extent of state traffic safety data systems meeting NHTSA’s performance measures, we analyzed the most recent traffic records assessments for each of the 50 states and the District of Columbia (D.C.). A traffic records assessment is a state document that contains findings and recommendations on the quality of a state’s traffic safety data systems, among other things. Assessments are conducted or updated at least every 5 years as one of the eligibility requirements for Section 408 grant program funding. At least three team members reviewed each assessment and coded the extent to which a state’s six traffic safety data systems met each of NHTSA’s six performance measures—timeliness, consistency, completeness, accuracy, accessibility, and integration. After individual team members independently coded the data quality of assigned state traffic records assessments, the three member sub-group met to discuss the coding categories and reached consensus on the final coding category assignment for each performance measure. Independently coding, initial unanimous agreement was reached 37 percent of the time amongst the three coders before discussions to reach consensus. Across states, initial unanimous agreement was as high as 58 percent for one state, but for two states there was no unanimous agreement for any of the coding categories. Within the performance measures there was also a range of initial unanimous agreement. For vehicle information timeliness, individual coders reached unanimity for 37 states, including D.C. (73 percent). The lowest level of initial unanimity (14 percent, or seven states) occurred within the injury surveillance system’s accuracy performance measure.

Throughout this document we use the term “coding category” to refer to the extent to which a data system meets an individual performance measure. We created broad categories based on information presented in state traffic records assessments; these coding categories are not precise measurements of the extent to which data systems met performance measures. We assigned numbers to correlate with the coding categories defined below: 0 – Did not meet or minimally met performance measure (i.e., negligible, 0 to 5 percent). The state did not meet or minimally met a particular performance measure based on the available evidence. The state clearly did not meet, or the state met the performance measure to a negligible extent. For example, one state’s crash data timeliness was described as, “At present, crash data entry is experiencing a 12-month backlog. This is due to delays at every step in the process from initial crash reporting through final data entry and the multi-step/multi-stop process that is used in handling crash reports. The delays are having an impact on highway safety analysis and decision making in the state.” Since the criteria for crash timeliness is that the information should be available within a time frame to be currently meaningful for effective analysis of the state’s crash experience, preferably within 90 days, this performance measure area was coded as a zero. 1 – Marginally met performance measure (i.e., slightly, to a limited extent, greater than 5 to 50 percent). The state met the performance measure at some level above “minimally,” but not to a significant extent. The state met the performance measure to a slightly, or to a very limited extent. For example, one state’s citation and adjudication data consistency was described as, “Although there is a uniform traffic citation for [the state], not all agencies use it in the same manner. has opted to use it differently than the rest of the state. Since is a state with a court administrator that oversees each court, there seems to be some consistency in the way cases are adjudicated… is recorded at the courts is controlled so that each court records the same information.” Since the criteria for citation consistency is that all jurisdictions should use a uniform traffic citation form, and the information should be uniformly reported throughout all enforcement jurisdictions, this performance measure area was coded as a one. 2 – Generally met performance measure (i.e., significant extent, for the most part, greater than 50 to 95 percent). For the most part, the state met the performance measure, but with some limitations. For example, one state’s vehicle data accuracy was described as, “…in transition. The Department of Motor Vehicles has used Vehicle Identification Number (VIN) Analysis Software to enhance accuracy, but the descriptive information about vehicles was taken from registration and title applications. Beginning in 2006, the Department of Motor Vehicles has been entering the body style and descriptive information from VIN decoding and has been upgrading the descriptions to VIN decoded entry when re-titling vehicles.” The criteria for vehicle system accuracy includes that the state should employ methods for collecting and maintaining vehicle data that produces accurate data and should make use of current technologies designed for these purposes; therefore, this performance measure area was coded as a two. 3 – Completely met performance measure (i.e., fulfills or satisfies the condition, greater than 95 to 100 percent). The state fully met all aspects of the performance measure, and if any limitation was identified it was not material in nature. For example, one state’s roadway data accessibility was described as, “Data are accessed through the Roadway Information Management System and Integrated Transportation Management System. Various reports are produced on a daily basis for use both within the Department of Transportation (DOT) and for use by consultants, businesses and the general public.” Since the criteria for roadway accessibility is that the information should be readily and easily available to the principal users of these databases containing the roadway information for both direct (automated) access and periodic outputs (standard reports) from the files, this performance measure was coded as a three. 9 – Unknown. By “unknown” we mean that no other categorization was possible. This may be due to limited information preventing categorization, or that such information is absent. For example, one state’s roadway data integration was described as, “The integration of road and crash files seems to be adequate for present uses within [the state’s Department of Transportation].” This limited information did not directly address the integration of roadway data. In another example, a state’s injury surveillance data completeness and accuracy was described as, “Data completeness and data accuracy were not able to be evaluated during our assessment.” Due to the absent information, these performance measure areas were coded as a nine.

The extent to which a state has met a performance measure is considered a reflection of data system quality. Throughout this report, in instances where a performance measure was coded as a zero or a one the performance measure is considered not met, whereas, if a two or a three was assigned the performance measure is considered met. After we concluded the coding of the assessments, we conducted a series of statistical analysis. Analysis included answering the following questions: Overall frequency of each coding category (0, 1, 2, 3, 9).

Frequency of each coding category for each of the six data systems (0, 1, 2, 3, 9).

Frequency of each coding category for each of the six performance measures (0, 1, 2, 3, 9).

Frequencies by measure and system (total of 36 sets of frequencies).

Sum “score” for each state (excluding the coding category 9).

Frequency of the coding category 9 in the new assessment format as compared with the old format (which includes a section dedicated to “Information Quality”).

Percent of states with one or more 9s and total percent of the time a 9 was assigned.

The number of times a state scored a 3 or 2 (completely or substantially) in each system. Provided as the number of states with zero 2s or 3s in each system, one 2 or 3 in each system, etc.

The number of times a state scored a 0 or 1 (not met or marginally) in each system. Provided as the number of states with zero 1s or 2s in each system, one 0 or 1 in each system, etc.

The extent to which a state has met a performance measure is a reflection of data system quality. In addition to our analysis of state traffic records assessments, this objective was informed through documentary and testimonial evidence gathered on site visits. We collected and reviewed relevant advisories and guidance related to traffic safety. We also interviewed federal, state, and local officials, data users, and other experts to obtain perspectives on the quality of traffic safety data. However, we did not factor these other information sources into our traffic records assessment coding analysis.

To identify the progress states have made in improving traffic safety data systems and to determine what challenges remain, we reviewed states’ progress in meeting performance measures reported to NHTSA and in state documents, such as State Highway Safety Strategic Plans. We conducted site visits to eight states: Georgia, Idaho, Maine, Minnesota, North Carolina, Ohio, Texas, and Virginia. We selected these states based on a number of factors, including NHTSA recommendations, fatality rates, population, roadway ownership, prevalence of rural roads, and geographic diversity. NHTSA officials also provided input on states that they believed encompassed a wide range of traffic safety data system quality. During our site visits we interviewed state officials to identify progress in improving the quality of traffic safety data and associated systems. To identify state challenges in improving data systems, we conducted a literature review of past GAO work and other relevant studies. We also conducted in-depth interviews with state officials responsible for data systems, and collectors and users of state traffic safety data during our state site visits. Additionally, we spoke with NHTSA, national industry associations representing the different data systems, and experts in the field to inform our analysis of the primary challenges states face, as well as to inform us of state efforts to address these challenges. We compiled all the various interviews and conducted an analysis to identify the most frequently cited challenges.

Appendix II: Additional Data Analysis

The following table represents all values associated with our coding analysis of traffic records assessments for 50 states and D.C. We calculated scores for all 50 states and D.C. by adding the number of points received by a state. The total number possible was calculated by multiplying the number of systems (6) by the number of performance measures (6) by the number of possible points available per measure (3). This resulted in a maximum score of 108 points that states could receive based on the quality of their traffic safety data systems.

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Sara Vermillion (Assistant Director), Matt Cail (Analyst-in-Charge), Emily Eischen, Brandon Haller, Delwen Jones, Catherine Kim, Kirsten Lauber, Hannah Laufe, Josh Ormond, and Crystal Wesco made key contributions to this report.