Traffic Safety Data: State Data System Quality Varies and Limited Resources and Coordination Can Inhibit Further Progress
Published: 2010-04-15T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Traffic crashes kill or injure millions of people each year. High-quality traffic safety data is vital to allocate resources and target programs as the Department of Transportation's (DOT) National Highway Traffic Safety Administration (NHTSA) and states work to improve traffic safety through data-driven approaches. To qualify for federal funding, states must submit plans which include fatality and crash data analyses to identify areas for improvement. This requested report provides information on (1) the extent to which state traffic safety data systems meet NHTSA performance measures for assessing the quality of data systems, and (2) progress states have made in improving traffic safety data systems, and related challenges. To conduct this work, GAO analyzed state traffic records assessments, visited eight states, and interviewed federal officials and other traffic safety experts.
section_title: What GAO Found
paragraphs: GAO's analysis of traffic records assessments--conducted for states by NHTSA technical teams or contractors at least every 5 years--indicates that the quality of state traffic safety data systems varies across the six data systems maintained by states. Assessments include an evaluation of system quality based on six performance measures. Across all states, GAO found that vehicle and driver data systems met performance measures 71 percent and 60 percent of the time, respectively, while roadway, crash, citation and adjudication, and injury surveillance data systems met performance measures less than 50 percent of the time. Also, data system quality varies by performance measure. For example, across all data systems, states met the performance measure for consistency 72 percent of the time, but states met the integration performance measure 13 percent of the time. According to NHTSA, assessments should be in-depth reviews of state traffic safety data systems; however, in some cases, incomplete or inconsistent information limits assessment usefulness. Of the 51 assessments we reviewed, 49 had insufficient information to fully determine the quality of at least one data system. Furthermore, an updated assessment format has resulted in more frequent instances of insufficient information. Despite varying state traffic safety data system performance, data collected by NHTSA show that states are making some progress toward improving system quality. All states GAO visited have implemented projects to improve data systems, such as switching to electronic data reporting and adopting forms consistent with national guidelines. However, states face resource and coordination challenges in improving traffic safety data systems. For example, custodians of data systems are often located in different state agencies, which may make coordination difficult. In addition, rural and urban areas may face different challenges in improving data systems, such as limited technology options for rural areas or timely processing of large volumes of data in urban areas. States GAO visited have used strategies to overcome these challenges, including establishing an executive-level traffic records coordinating committee, in addition to the technical-level committee that states are required to establish to qualify for traffic safety grant funding. An executive-level committee could help states address challenges by targeting limited resources and facilitating data sharing.

section_title: Letter
paragraphs: In 2008, about 37,000 people were killed on public roadways in the United  States and another 2.3 million were injured. While these fatality and injury  statistics are some of the lowest in decades, high-quality traffic safety data  remains vital to the Department of Transportation’s (DOT) National  Highway Traffic Safety Administration (NHTSA) and state efforts to  further improve traffic safety. State officials increasingly use data-driven  approaches to allocate resources and target programs to improve traffic  safety, as well as to avoid incurring financial penalties. For example, in  2007 state departments of transportation were required to submit plans to  qualify for federal funding, which included state fatality and crash data  analyses to identify a state’s highway safety hazards. To support data- driven efforts, the Safe, Accountable, Flexible, Efficient Transportation  Equity Act: A Legacy for Users (SAFETEA-LU) authorized $138 million for  NHTSA’s Section 408 Traffic Safety Information System Improvement  (Section 408) grant program from fiscal years 2006 through 2009. States  can use Section 408 grant funding to improve the quality of six core types  of traffic safety data systems—crash, driver, vehicle, roadway, citation and  adjudication, and injury surveillance. Congress is considering whether and  in what form to reauthorize the Section 408 grant program as part of the  next surface transportation reauthorization act. As requested, this report  provides information on (1) the extent to which state traffic safety data  systems meet NHTSA performance measures for assessing the quality of  data systems, and (2) progress states have made in improving traffic safety  data systems, and related challenges.
To identify the extent to which state traffic safety data systems met  NHTSA performance measures, we analyzed the most recent traffic  records assessments for each of the 50 states and the District of Columbia  (D.C.) and from that information determined and coded the extent to  which a state’s six traffic safety data systems met each of NHTSA’s six  performance measures—timeliness, consistency, completeness, accuracy,  accessibility, and integration. Throughout this document we use the term  “coding category” to refer to the extent to which a data system meets an  individual performance measure and reported these categories as met, did  not meet, or unknown. We created these broad coding categories based on  information presented in state traffic records assessments. These  categories are not precise measurements of the extent to which data  systems met performance measures, but provide a reflection of data  system quality. See appendix I for a full description of our coding category  definitions, data analysis, and methodology.
To identify the progress states have made in improving traffic safety data  systems and to determine what challenges remain, we reviewed states’  reported progress in meeting performance measures required by NHTSA  and in state documents, such as highway safety data and traffic records  strategic plans. We conducted site visits to eight states: Georgia, Idaho,  Maine, Minnesota, North Carolina, Ohio, Texas, and Virginia. We selected  these states based on a number of factors, including NHTSA  recommendations, fatality rates, population, roadway ownership,  prevalence of rural roads, and geographic diversity. We also reviewed  project funding and other information from the eight states that we visited  to provide examples of how states are improving traffic safety data  systems. In addition, we interviewed state officials about their progress in  improving the quality of traffic safety data and associated systems. To  identify state challenges in improving data systems, we conducted in-depth  interviews during our state site visits with officials responsible for data  systems, as well as data collectors and users. We spoke with NHTSA  officials, national industry association representatives, and other experts  in the field to inform our analysis of the challenges states face and  strategies to address them. We compiled all of the interviews and  identified the most frequently cited challenges.
We performed our work from May 2009 to April 2010 in accordance with  generally accepted government auditing standards. Those standards  require that we plan and perform the audit to obtain sufficient and  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
subsections: section_title: Background
paragraphs: To help identify priorities for highway and traffic safety programs, states  maintain six core types of traffic safety data systems: vehicle, driver,  roadway, crash, citation and adjudication, and injury surveillance (see  table 1). Organizations responsible for implementing and maintaining  these systems vary among states, but generally include highway safety  offices, law enforcement agencies, motor vehicle offices, courts,  emergency medical service (EMS) providers, and others.
While state funds are generally the primary source of funding to  implement and maintain these systems, states also use federal funds.  SAFETEA-LU provides the Section 408 grant program with the most  authorized funding exclusively for traffic safety data systems.  Administered by NHTSA, this grant program authorized $34.5 million  annually from fiscal year 2006 through 2009. For fiscal year 2009, all 50  states and D.C., received funding through the Section 408 grant program,  with amounts ranging from $346,262 to $2.3 million. As stated in  SAFETEA-LU, goals of this program are to encourage states to adopt and  implement effective programs to:  improve the timeliness, consistency, completeness, accuracy,  accessibility, and integration of traffic safety data;   evaluate the effectiveness of efforts to make such improvements;  link these state traffic safety data systems with other data systems  within the state; and  improve the compatibility of the state data system with national and  other state data systems to enhance the ability to observe and analyze  national trends in crash occurrences, rates, outcomes, and  circumstances.
To receive funding through the Section 408 grant program, states must  meet certain requirements, including establishing a traffic records  coordinating committee (TRCC), demonstrating measurable progress  toward meeting goals and objectives identified in a multi-year highway  safety data and traffic records systems strategic plan, and certifying that  an assessment of the state traffic records system has been performed  within the last 5 years (see table 2).
Among these requirements for the Section 408 grant program, a state  TRCC serves to guide and make decisions about traffic safety data systems  within the state. The Section 408 grant program requires states to include  technical experts on the TRCC, including representatives from highway  safety, highway infrastructure, law enforcement and adjudication, public  health, injury control, motor carrier agencies, and other stakeholders. In  addition to a technical-level TRCC, some states have also established an  executive-level TRCC, which can include a manager or director—rather  than technical—representatives from state organizations.
To determine state eligibility for the Section 408 grant program and  progress toward meeting goals and objectives set forth in a strategic plan,  NHTSA has developed six performance measures of data system quality:  timeliness, consistency, completeness, accuracy, accessibility, and  integration (see table 3). While performance measure definition and  relative significance may vary for each system within a state depending on  the state’s baseline, goals and objectives, NHTSA officials are working to  provide examples of these performance measures to make it easier for  states to measure progress. NHTSA expects to finalize these  improvements in April 2010.
Traffic records assessments are an evaluation of states’ traffic safety data  systems, which includes discussions of how systems met NHTSA’s  performance measures. A NHTSA technical team or private sector  contractors conduct assessments for states using a “peer” review  approach. Technical teams recommended by NHTSA conduct most  assessments. The teams are generally composed of five assessors that  states approve to conduct the assessment. These assessors have  demonstrated expertise in major highway safety program areas, such as  law enforcement, engineering, driver and vehicle services, injury  surveillance systems, and general traffic records development,  management, and data use. The peer review team generally takes about 5  days to complete an assessment, including interviews with state officials,  preparing the assessment report, and conducting a final briefing with state  officials (see fig. 1). Assessors and NHTSA officials described the principal  document to guide the traffic records assessment process as the Traffic  Records Program Assessment Advisory, which was updated in 2006, and  for the purposes of this report will be referred to as the 2006 Advisory. The  format of traffic records assessments was updated to reflect changes made  to the original advisory. The principle change made to the assessment  format is that the sections describing traffic safety data systems are now  combined with previously separate sections describing the information  quality.
Besides the Section 408 grant program, SAFETEA-LU authorized other  NHTSA grant programs, such as the Section 402 State and Community  Highway Safety Grants and the Section 406 Safety Belt Performance  Grants, which states can use for any traffic safety purpose, including  traffic safety data improvement projects. Also, the Federal Highway  Administration (FHWA), the Federal Motor Carrier Safety Administration  (FMCSA), and other federal agencies—such as the Centers for Disease  Control and Prevention (CDC) and the Department of Homeland  Security—have provided support to state traffic safety data projects. For  example, the Highway Safety Improvement Program has provided funding  to help states achieve a significant reduction in traffic fatalities and  serious injuries on public roads through the implementation of  infrastructure-related highway safety improvements, which can include  traffic safety data projects. A new program is FHWA’s Crash Data  Improvement Program (CDIP), which is designed to assist states in  developing or improving methods of assessing the quality of their crash  data. As part of CDIP, a technical team performs an assessment of a state’s  crash data system and then produces a report with recommendations on  the establishment of performance measures. FHWA officials reported that  after the completion of the assessment, states are eligible to receive up to  $50,000 in funding from FHWA to implement recommendations of the  report. At the time of this report the program was in its beginning stages  and three states had participated so far.
section_title: State Traffic Data System Quality Varies, but the Full Extent of Data System Quality Is Difficult to Determine
subsections: section_title: State Traffic Data System Quality Varies by System and Performance Measure
paragraphs: Data system quality also varies by performance measure. For example,  across all traffic safety data systems, states met the consistency  performance measure 72 percent of the time, but met the data integration  measure only 13 percent of the time (see fig. 3). The comparatively high  level of consistency in state data systems may result from states using  uniform reporting forms, such as uniform crash, citation, and EMS reports  that are consistent with nationally accepted and published guidelines and  standards. According to state officials, integrating data systems can be  difficult due to older and outdated system design and obtaining  cooperation from different data managers. Assessors said that integration  is difficult to measure and report on. Further, state and other officials  described integration as one of the last performance measures that states  tend to focus on in creating high-quality traffic safety data systems while  timeliness, accuracy, and completeness are addressed first.
In addition to data system quality varying by system type and by  performance measure, our analysis revealed differences in the extent to  which individual state systems met various performance measures.
Vehicle and Driver Systems: Vehicle and driver systems met at least 60  percent of the performance measures; specifically, 38 vehicle systems and  31 driver systems met four or more of the six performance measures.  Vehicle systems performed best in the area of timeliness—completely  meeting that performance measure in 45 states—while driver systems met  the accessibility performance measure in 35 states. State officials cited  multiple reasons why state vehicle and driver systems may be high- performing compared to other data systems, such as (1) these data  systems need to be reliable and customer-oriented since the public has  contact with the systems through vehicle registrations and driver license  applications; and (2) these data systems generate revenue for states  through fees and other charges for vehicle and driver licenses. In one state  we visited, revenue collected through the Bureau of Motor Vehicles for  motor vehicle licenses and fees amounted to over $90 million in 2009.
Despite the general ability of driver and vehicle systems to meet most  performance measures, only seven driver and five state vehicle systems  met the integration performance measure. State officials said that  integrating driver and vehicle systems with other traffic safety data  systems is difficult due to the age of some systems. For example, in one  state we visited the vehicle database is 30 years old and has no ability to  electronically communicate or integrate with other data systems. In  addition, 31 state driver systems met the performance measure for  completeness of data. Based on assessments we reviewed, one reason why  all states did not have complete driver data may be that some states do not  collect previous driver histories from other states for non-commercial  drivers. In order to meet the performance measure for completeness,  driver histories must be included for all licensed drivers in particular  adverse actions received by drivers in other states, either while licensed  elsewhere or driving in other states. In addition, having complete records  for drivers promotes safety for law enforcement officers conducting  roadside traffic stops. For example, an officer can determine whether the  driver that he or she has pulled over has a warrant out for his or her arrest  or a suspended license, and with access to vehicle data, can find out if the  driver is in a stolen vehicle. With this information the officer can better  prepare for the interaction, whereas the officer may be more at risk  without it.
Roadway Systems: Roadway systems met almost half of the performance  measures; specifically, they performed best in consistency—38 states met  the performance measure—but, less than half of the states met the  performance measure of completeness. According to one assessor and a  state official, roadway data plays an important role in state planning. This  may lead some states to collect such data consistently. However, in  several states we visited, state officials only collected and inventoried  roadway characteristics for the state maintained roadways, but less for  locally maintained and other roadways, which may contribute to roadway  data incompleteness. Nationally, locally maintained roads account for  about 77 percent of all public roads, while state maintained roads  represent about 20 percent of the total road mileage. In Idaho, of the over  47,000 miles of roadway in the state, the Idaho Transportation Department  is responsible for collecting and maintaining data on about 5,000 miles of  these roads. The remaining approximately 42,000 miles are the  responsibilities of local road authorities. As GAO has previously reported,  most states have not developed roadway inventory data for locally  maintained roads because they do not operate and maintain those roads  and are concerned about costs and time frames involved in collecting the  data. In addition, state officials reported that they collect the amount of  data on locally maintained roads that are required for the national  database—the Highway Performance Monitoring System (HPMS)—which  consists of all data collected and updated by states on selected highway  segments across the United States. Because of this, officials said detailed  data are not collected for all roadways. One effect of incomplete roadway  data is that location data for some crashes will make the identification of  hazardous locations difficult or impossible and can also prevent states  from fully identifying and reporting on potential remedies for hazardous  locations and estimating the costs of those remedies.
Crash Systems: While state crash data systems met about as many  performance measures as not, our analysis showed, and we have  previously reported, that state crash data systems varied considerably in  the extent to which they met NHTSA’s performance measures. For  example, crash data systems in five states met all six performance  measures, while systems in six states did not meet any of the performance  measures. In addition, systems in 27 states met two or fewer of the six  performance measures. Also, according to our analysis, crash systems in  32 states met the consistency performance measure. Several of the states  that we visited had uniform crash report forms used by law enforcement  to report vehicle crashes, which may contribute to the consistency of  crash data. Traffic records assessors, NHTSA officials, and state officials  said that states have tended to focus on improving crash data systems, in  part, due to crash data having a clearer link to improving public safety  than other traffic safety data systems. However, systems in 23 states did  not meet the performance measure for crash data accuracy. Manual data  entry and a lack of electronic edit checks could lead to less accurate  data, which can inhibit meaningful analysis. For example, in one state we  visited, law enforcement officers provided incorrect longitude coordinate  data using Geographic Information Systems (GIS) equipment. This human  error resulted in inaccurate crash location data; in multiple instances, the  computer program located crashes in China.
Citation and Adjudication Systems: For citation and adjudication  systems, about as many performance measures were met as were not met;  specifically, systems in 18 states met four or five performance measures,  while systems in 21 states met one or none of the six performance  measures. However, 17 percent of the time, the extent to which the  performance measure was met was unknown. Citation and adjudication  systems performed best in consistency—38 systems met this performance  measure. Similar to crash data, the adoption of uniform citation forms may  have improved consistency for this system. However, about half of state  citation and adjudication systems did not meet accessibility and  completeness performance measures, and only one state met the  integration performance measure. These performance measures may be  difficult for some states to meet due to the high number of jurisdictions  that states rely on to report data or because a statewide citation system  may not exist. For example, Georgia officials said that the state has nearly  800 different courts—about 400 of which are municipal courts, which  handle most traffic violations—each with its own court data system. There  is no comprehensive collection of citation data in the state, and the state  has a limited ability to require jurisdictions to submit data. Georgia  officials said that citation and adjudication data are relatively incomplete  because some courts do not report all data. Also, if states do not have an  electronic citation system, even police departments with the ability to  submit citations electronically must submit their citations on paper. For  example, a law enforcement officer from one state we visited said that his  department has the capability to electronically submit citations, but must  still print out citations to submit them to the state because the state is not  able to electronically receive citations.
Injury Surveillance Systems: Less than half of the performance measures  were met, but similar to the citation and adjudication systems, the extent  to which performance measures were met was unknown 17 percent of the  time. Systems in 39 states met 3 to 0 performance measures, while systems  in 12 states met four to six. In addition, within injury surveillance systems,  25 states met the performance measure for accuracy and 30 states met the  performance measure for consistency. This may be attributed to training  provided to those responsible for data entry. For example, one state  hospital administration provides training to data entry staff on how to  enter cases into the state data system properly. In contrast, 40 state injury  surveillance systems did not meet the performance measure for  integration. Assessors and one state official reported that the multiple  components necessary for a state injury surveillance data system make  meeting various performance measures more difficult than for other data  systems. According to the 2006 Advisory for traffic records assessments, a  complete injury surveillance system typically has five components: pre- hospital (i.e., EMS), trauma, emergency department, hospital in- patient/discharge, and rehabilitation to track injury causes, magnitude,  costs, and outcomes. Officials said that maintaining multiple components  often requires that several departments contribute data, which can make  data management difficult. For example, in Minnesota, the EMS  Regulatory Board collects EMS data, the Minnesota Hospital Association  collects patient discharge information, and the Minnesota Department of  Health maintains the Minnesota Trauma Data Bank, which contains  trauma and mortality data, all of which are reported to the Minnesota  Department of Health. In addition, systems in several states have only  some components of a fully functioning injury surveillance system in place  or have system components that are just in the beginning stages of  development.
section_title: Many Traffic Records Assessments Are Incomplete or Inconsistent; therefore, the Full Extent of Data System Quality Is Difficult to Determine
paragraphs: Although NHTSA’s implementing guidance for the Section 408 program  states that a traffic records assessment should be an in-depth, formal  review of a state’s highway safety data and traffic records system, our  analysis revealed instances where assessments were incomplete or  inconsistent. We assigned “unknown” codes where no other categorization  was possible due to limited or otherwise absent information in a state  traffic records assessment, which includes both incomplete and  inconsistent performance measure descriptions. The results of our  analysis were that 49 of 51 traffic records assessments had at least 1 area  out of 36 (six state traffic data systems multiplied by six performance  measures) for which the extent to which a system met a performance  measure was unknown. Incomplete or inconsistent information could  limit the usefulness of these assessments to state officials and make it  difficult to ascertain the full extent of data system quality. NHTSA officials  said that they review traffic records assessments for quality and that they  have accepted all state assessments as adequate to fulfill the statutory  requirement included in NHTSA’s Section 408 grant program implementing  guidance. NHTSA officials said that they are currently beginning work  with a contractor to study the assessments. While the contract to study the  assessments includes a component to examine state traffic records  assessments for effectiveness and utility, the main objective is to review  state traffic records programs and data systems from states that have had  at least two traffic records assessments and identify any improvements or  degradations that occurred between the two assessments. In addition to  the contract, NHTSA officials reported starting other activities, which will  include updating related advisory documents, increasing participation of  other DOT administrations, aligning traffic records assessments with other  similar NHTSA program assessments, determining the most effective  frequency for requiring assessments, incorporating all performance  measures identified in advisory documents, and developing a more robust  list of assessors for states. As these efforts are in the beginning or planning  stages, it is too soon to tell how they will impact the traffic records  assessment process.
Our review of traffic records assessments showed that for those traffic  records assessments that had any unknown areas, the number of unknown  areas ranged from 1 to 18 out of a possible 36, but most assessments had  five or fewer unknown areas. Of the 49 assessments we coded with  unknown areas, 27 had between 1 and 3 unknown areas and 6 had 10 or  more (see fig. 4). Out of the total 1,836 codes that we assigned across all  51 assessments, 226 (about 12 percent) were coded as unknown.
Our coding analysis revealed that the frequency of unknown areas is  greater in the updated assessment format compared to the prior  assessment format. Of the 51 assessments we reviewed, 11 (22 percent)  were in the updated assessment format. Despite the lower number of  assessments in the updated format, proportionally, we coded about three  times as many areas as unknown in the updated assessment format than  the prior format. The updated traffic records assessment format, which is  based on the 2006 Advisory, is less tied to NHTSA’s Section 408 grant  program implementing guidance than previously. The 2006 Advisory  describes what characteristics state traffic safety data systems should  have, but unlike NHTSA’s implementing guidance and the prior advisory,  in several areas it does not include a discussion of each of the six  performance measures as they relate to each of the six data systems. For  example, the 2006 Advisory notes that data should be timely and includes  an example of a quality control measure for timeliness, but unlike the prior  advisory, does not establish a specific time frame by which timeliness can  be assessed. The 2006 Advisory also does not expressly discuss the  accessibility performance measure for five of the six traffic safety data  systems. This means that for five of the six data systems, the 2006  Advisory addresses only four of the six performance measures.
As previously noted, several assessments were incomplete, meaning that  there was not enough information provided to determine the extent to  which a state had met a performance measure. There was one instance in  which an assessment lacked performance measure evaluation information  on that state’s entire injury surveillance system since “representatives of  the various medical data systems were not present during  Traffic  Records Assessment. Therefore no information related to timeliness,  consistency, completeness, accuracy, accessibility, and integration…could  be presented in  report.” In other instances, we were unable to make a  determination based on information provided in the assessment. For  example, in 14 traffic records assessments it was unclear whether citation  and adjudication data were timely. In another assessment, the timeliness  of injury surveillance data was explained as the timeliness of EMS arrival  time as opposed to the timeliness of when the injury data are available for  analysis. Incomplete injury surveillance data may lessen a state’s ability to  track injury causes, magnitude, costs, and outcomes.
Some incomplete assessments may result from differing views on the  value of various performance measures between NHTSA officials and  traffic records assessors. According to NHTSA officials, the six  performance measures across the six data systems are of equal  importance in the context of assessing a state’s qualification for  subsequent year Section 408 grant funding. NHTSA officials also said that  making progress in one data system or performance measures is not more  highly valued than making progress in another. Additionally, NHTSA  officials said that part of the value of assessments is that they provide  information on all areas of states’ traffic safety data systems. In contrast,  some of the assessors we interviewed questioned the value of some or all  of NHTSA’s performance measures for the various state traffic safety data  systems. For example, assessors said that information on the integration  performance measure was not valuable because it is difficult to measure.  Others said that injury surveillance data assessors focus on integration  more than the other performance measures and that one of the most  important findings in the injury surveillance section of a traffic records  assessment is how it integrates with other traffic safety data systems.  Furthermore, some assessors reported that they do not evaluate certain  performance measures if it appears that nothing has changed in a state  since the last assessment, and that some performance measures and traffic  safety data systems are not as important as others.
As noted previously, the principal document used by assessors as a guide  for the traffic records assessment process is the 2006 Advisory. The  purpose of this guidance is to provide states with guidance on the  necessary contents, capabilities, and quality of data in a traffic records  system and to be a description of an ideal system, not to describe what  information should be included in a traffic records assessment.  Furthermore, as opposed to the previous advisory, the 2006 Advisory  explicitly discusses some, but not all of the six performance measures for  each traffic records systems. Given that, per Section 408 grant program  requirements, assessments are conducted every 5 years, there is merit in  having clearer guidance that assessments include all performance  measures to update state officials on their traffic safety data systems, even  if such an update explains that nothing has changed since the last  assessment.
In addition to completeness concerns, some traffic records assessments  are inconsistent, meaning that information provided in one part of the  assessment describing the extent to which a state met a performance  measure was inconsistent with information provided elsewhere in the  assessment. For example, one assessment described the performance  measure of consistency as “…not  to be an issue in that a  uniform citation is used and there are a relatively small number of police  agencies …that submit traffic citations.” However, later on the same page  in the accuracy section it was noted that “The Court indicated that officer  reporting is not consistent and more training is needed to assure that  charging documents and affidavits of probable cause are completed  correctly. Additional training could help to assure uniformity of  submissions.” Another assessment explained, “Information provided  during the assessment interviews indicated that the data are timely; the  latest data available for analysis is 2006.” Upon review, the available data  were at least a year old since the traffic records assessment was  conducted in 2008. However, NHTSA guidance suggests that all injury data  be available in a comparable time frame for the crash data, which is  preferably within 90 days of a crash.
Despite these limitations, traffic records assessments remain vital in  helping states identify problems, develop plans, and prioritize projects to  improve traffic safety data systems (see fig. 5). For example, Minnesota  officials used recommendations made in a traffic records assessment,  along with the strategic plan, to prioritize traffic records projects. In  addition to being useful to states for making traffic records improvements,  NHTSA officials emphasize traffic records assessments as valuable for  strategic planning purposes. NHTSA officials added that the traffic records  assessment process is important because it provides an independent look  at the quality of traffic safety data systems, helps determine where  priorities should lie, and guides states on targeting limited resources.  Assessors and state officials also emphasized the value of traffic records  assessments for states. For example, one assessor said that traffic records  assessments serve as a tool and guideline for states in how to move  forward with traffic safety data systems and to promote a data-driven  approach by balancing stakeholder interests with priorities highlighted by  data.
In contrast, state officials reported that while information captured in  traffic records assessments is useful, the more specific information such  as problem identification, definitions of performance measures and data  analysis recommendations included in FHWA CDIP assessments has  additional benefits. Although CDIP began in 2008 and only three states  have currently participated, officials in states where both a traffic records  and CDIP assessment were conducted said that the information included  in CDIP assessments was more in-depth and specific. CDIP assessments  are conducted in a similar manner as traffic records assessments, take  roughly the same amount of time to conduct, and cover all six  performance measures identified in NHTSA’s implementing guidance, but  focus only on a state’s crash data system. CDIP assessments include  recommendations and particular steps or methods states can take to  potentially improve their crash system.
By contrast, assessors identify problems in traffic records assessments but  state officials said that traffic records assessments generally do not  provide specific strategies for ways to improve the traffic safety data  systems. State officials reported that an assessment with information as  specific as that provided in CDIP assessments would be valuable to have  for each of their traffic safety data systems. In addition, several state  officials said that insufficient time is spent conducting traffic records  assessments to produce an in-depth, detailed report. In one state, traffic  records assessment officials spent only 10 minutes with the team  representing one of the six data systems.
section_title: States Have Demonstrated Progress in Some Data Systems, but Face Resource and Coordination Challenges
subsections: section_title: States Have Demonstrated Some Progress Nationwide
paragraphs: Information collected by NHTSA from the states shows that 49 states and  D.C. have demonstrated progress in improving the quality of all six traffic  safety data systems. States have demonstrated progress in all six traffic  safety data systems, as well as across all six performance measures. It is  important to note that reported state progress is not equivalent to  achieving a high-quality traffic safety data system; rather, such progress  represents steps toward that end goal. Of the possible 36 areas in which to  demonstrate progress, by system and by performance measure, states  demonstrated progress in 23 areas to NHTSA from fiscal year 2008 through  fiscal year 2009 (see table 4). To remain eligible for Section 408 grant  funding states must demonstrate measurable progress related to achieving  the goals and objectives of a state’s multi-year highway safety data and  traffic records strategic plans. NHTSA officials reported that states can  fulfill this requirement by demonstrating progress in one performance  measure for one data system per year. For example, a state might report  progress involving the performance measure of completeness within the  roadway data system. States can and have reported more than one area of  progress. NHTSA does not require states to report all progress toward  improving traffic safety data systems and, as a result, states may be  making progress that is not reported. Additionally, NHTSA does not  always accept every area of progress that a state reports if the state  demonstrates sufficient progress in at least one area; therefore, state  progress may be understated. Sometimes NHTSA cannot verify that  progress has taken place in all reported areas, due to a lack of evidence or  incomplete information. For example, Maine officials reported five areas  of progress to NHTSA for fiscal year 2009 and NHTSA officials accepted  four of those areas. West Virginia officials reported four areas of progress,  one of which NHTSA officials accepted. While NHTSA officials reported  that demonstrated progress does not represent all progress that states are  making, it serves as a useful approximation for the areas in which states  are making progress. Moreover, NHTSA officials said that in regards to  qualifying for Section 408 grant funding, the most important development  is that states are making some progress in improving traffic safety data  systems.
State progress, for the 2 most recent fiscal years, may reflect some trends  identified by our analysis of the extent to which state traffic safety data  systems met NHTSA performance measures. For example, states  demonstrated the least progress in the vehicle and driver data systems (7  of the 164 total areas of progress listed in table 4). This may reflect that  vehicle and driver systems already met most performance measures, as  shown in our coding analysis. In contrast, states have demonstrated  progress for crash data systems more often than other systems. Out of the  164 instances states have demonstrated progress, 89—over half—involved  improvements to state crash data systems. This may indicate heightened  state efforts to improve crash data systems due to these systems not  meeting various performance measures, as shown in our analysis.  Furthermore, state and NHTSA officials, as well as assessors, reported  that states have focused on improving crash systems.
Progress has resulted from states pursuing small- and large-scale projects  to improve traffic safety data systems. For example, some progress has  resulted from smaller-scale projects, such as printers for citations or  online tutorials. NHTSA officials said that they have encouraged states to  use Section 408 grant program funding to support near term, quick  projects, recognizing that large-scale projects might require significant,  additional time or funds. However, some state officials said that smaller- scale projects are less likely to immediately lead to substantial  improvements in the overall quality of state traffic safety data systems.
Support for large projects also depends on state funding in addition to  Section 408 grant program funding awarded to a state. For example,  Virginia has expended over $900,000 in state and local funding on the  Traffic Record Electronic Data System (TREDS) project, which integrates  federal, state, and local data; provides law enforcement the ability to  collect and submit crash data electronically; reduces manual entry of data;  provides enhanced analysis capabilities and increases accessibility for  data users; among other things. Thus far, NHTSA has awarded Virginia  approximately $2.5 million in Section 408 grant funding.
For the states that we visited, federal assistance has helped states to  improve traffic safety data systems. Officials in all eight states that we  visited stressed the important role of the Section 408 grant program to  improve traffic safety data and have used this and other federal funding to  implement projects. Officials reported that while state funding makes up  the majority of support for traffic safety data projects, without Section 408  grant program or other federal funding some projects would have  happened much more slowly, or not at all. NHTSA officials estimated that  for every dollar provided through Section 408 grant funding, states spend  an additional $4 for traffic safety data projects. Below are examples of  state projects that have used federal funding.
Timeliness–Several states have implemented or are currently working on  projects to transition from manual to electronic reporting of data.  Electronic reporting reduces reliance on paper processes and can increase  the speed of submission and eventual availability of data for analysis.  Minnesota has undergone such a transition for crash data. Minnesota  officials said that in 2009 over 90 percent of the state’s crash reporting was  submitted electronically to its crash database. This includes all crash  reports from Minnesota’s State Highway Patrol. Electronic submission has  helped the state submit and finalize all data in Minnesota’s crash database  within 30 days. This represents an improvement from the 6-week backlog  to enter crash data that Minnesota experienced in 2003.
Completeness–To improve the completeness of crash data, officials in  three states that we visited reported using diagram software to help law  enforcement officers depict crashes. Officers generate these diagrams by  entering information electronically at the scene of a crash (see fig. 6). The  diagram increases completeness by including visual information like the  position of the vehicle(s), location of damage, intersection layout, and  other crash features, such as trees and pedestrians. Using crash diagram  software, officers can edit information before completing and submitting  the diagram as part of the crash report.
Consistency–Some states have improved consistency by adopting uniform  reporting forms and increasing compliance with national guidelines. In  2007, Virginia revised its crash data collection form using guidance from  NHTSA and guidelines captured in the Model Minimum Uniform Crash  Criteria. Virginia officials reported that the form revision increased  compliance with the Model Minimum Uniform Crash Criteria from 55 to 80  percent. Also, Georgia’s Emergency Medical Services Information System  has used a revised form that includes approximately 300 data elements— as opposed to the previous form, which had 103. This revised form is “gold  compliant” with National EMS Information System guidelines.  Approximately 30 percent of Georgia’s EMS agencies are still using the  previous forms, but state officials expect a continued transition to the new  form.
Accuracy–To improve the accuracy of roadway data, including roadway  features such as bridge locations, some states have explored projects  available through GIS and other technology. Maine’s Department of  Transportation has created the Maine Department of Transportation Map  Viewer System, which will eventually become available to a variety of state  data users. This system integrates existing GIS technologies into a viewer  screen where users can view roadway data and update information to  increase accuracy. Users of the viewer system can also select and change  which data are displayed and view photographs of a particular section of  roadway to illustrate local features (see fig. 7).
In another example, one law enforcement jurisdiction that we interviewed  installed video data recorders in police vehicles. These devices record  scenes to the front or rear of a vehicle. Uses of these recorders include  reviewing crash footage to verify information and ensure that crash,  driver, and vehicle data are accurate (see fig. 8).
Accessibility–Several state and local jurisdictions we met with, including  those in Maine, Minnesota, and Ohio, have completed projects to make  traffic safety data more accessible to users. For example, data captured by  Ohio’s Location Based Response System (LBRS) is available to data users  and other citizens on the Internet.
Ohio officials reported that this has increased accessibility to roadway  information, and reduced public requests for roadway data. In addition to  state Department of Transportation officials, LBRS users have included  County Emergency Management Agencies, utilities, and county engineers.  In addition to LBRS, other projects have included jurisdictions  incorporating new technologies to make crash, driver, citation, and vehicle  data more accessible in law enforcement vehicles. Figure 9 provides  examples of other, completed projects that have increased the  accessibility of various data systems for law enforcement officials.
Integration–To improve the integration of traffic safety data systems with  one another, 19 states participate in the Crash Outcome Data Evaluation  System (CODES) effort. Facilitated and supported by NHTSA, CODES  seeks to better link and otherwise integrate crash and injury surveillance  data. Such integration can result in state officials better understanding  the medical consequences of traffic crashes and the types of injuries that  certain crashes are likely to produce. Of the states that we visited,  Georgia, Maine, Minnesota, Ohio, and Virginia participate in the CODES  project. Ohio officials reported that the most extensive linkage between  injury surveillance systems in the state has happened through the CODES  program, which has established links between EMS, trauma, and crash  data. Virginia officials cited CODES in helping to submit and link data to  other organizations including the Department of Motor Vehicles.
section_title: States We Visited Face Resource and Coordination Challenges, but Some Have Implemented Strategies to Help Address These Challenges
paragraphs: While states have demonstrated progress, a number of overarching  challenges exist to improving traffic safety data systems. This is due in  part to the complexity and multifaceted nature of trying to establish traffic  safety data systems. The Section 408 grant program is designed to improve  six, oftentimes completely separate, state traffic safety data systems. We  have previously reported that overhauling one outdated data system can  be both challenging and expensive, particularly when integrating a new  system with existing legacy systems. State officials in all states that we  visited also reported that just maintaining one data system requires  significant funding, time, or other limited resources. Therefore, trying to  make simultaneous improvements to multiple traffic safety data systems  can magnify these challenges.
Limited Resources: Officials in all the states that we visited identified  limited resources as a significant challenge in state efforts to improve  traffic safety data systems. Some of the most frequently cited limitations in  funding and human capital resources are discussed below.
Limited funding. According to state officials, making improvements to  one data system can cost tens of millions of dollars. Therefore, obtaining  funding necessary to make improvements to six state traffic safety data  systems is a challenge. As we previously reported, while traffic safety data  grants have provided states with funding to improve traffic safety data  systems and complete associated projects, the cost of developing and  maintaining data systems can exceed Section 408 program grant  amounts. While state officials reported that state funding supports most  of the cost of traffic safety data projects, NHTSA and officials in five out of  eight states we visited indicated that traffic safety data system  improvements are not among the highest state priorities due to budgetary  constraints or limited interest. The recent economic recession has  amplified state funding limitations for data projects.
Moreover, a state’s legislative process may delay funding for traffic safety  data projects. Even in instances where funding is available, some traffic  safety data improvements require state legislative action or approval to  move forward on contracting, design, and implementation processes.  Infrequent state legislative sessions can heighten delays in receiving  approval to spend awarded federal funding. For example, according to  state officials, the legislature in one state we visited meets every other  year, which can delay approval of spending of federal grant and other  funding on traffic safety data projects and contribute to carry over of  funds. In another state, major technology projects must first be approved  by the state’s information technology authority. The project planning  involved to obtain state approval can make some projects cost prohibitive.  For example, the state wanted to update the injury surveillance system 4  years prior, but had to obtain approval first, which resulted in delays in  implementation and a doubling of the project’s costs.
Limited human capital resources. States that rely on paper crash and  citation forms require manual, time-consuming data entry, which can  strain resources and lead to backlogs in data. For example, the Texas  Department of Transportation assumed responsibility for the state’s crash  data system in 2007 from another state department, and also assumed  responsibility of a backlog of some 3 million crash reports over a 5-year  period that needed to be entered into the data system (see fig. 10). The  accumulated backlog was the result of the state’s use of a manual crash  data system designed in the 1970s prior to implementing the state’s  electronic crash data system in 2008. According to a Texas Department of  Transportation briefing report, the manual process was inefficient,  resource intensive, and not conducive to the timely dissemination of data.
In some states, there are only a few staff that manage a state’s traffic  safety data programs and grants. This is significant because state officials  reported that grant applications are time consuming and difficult to  balance with other key job responsibilities. In one instance, a state we  visited had to return federal grant funding because it did not have  available staff resources to effectively manage the grant and associated  project. A regional NHTSA official also reported that the turnover and  training of new, state staff can be a challenge, particularly when staff must  be trained on the specifics of the Section 408 grant program due to limited  institutional knowledge. Furthermore, NHTSA officials reported that  regional meetings have helped state officials obtain contacts and share  leading practices, but state budget restrictions have curtailed these  meetings, removing this training opportunity and resource. NHTSA  officials reported that they have recently begun using online webinars as  an alternative for national, state, and regional audiences.
Training individuals is an important component in ensuring the collection  of high quality traffic safety data, as recommended in several traffic  records assessments. However, a number of state officials told us that  training on data collection may be limited due to funding and resource  constraints, such as staff resources and travel expenses. In several states,  officials reported that the local law enforcement officers collecting the  data may not fill out a crash report completely or accurately, or submit the  form in a timely fashion, which may lead to instances where crash data are  inaccurate, incomplete, and untimely.
Officials in several states reported that information technology resources  are limited and that state agencies often have to share staff with technical  expertise between different data systems and projects. Due to limited  internal technical expertise, some states have used contractors, but state  officials reported that this can be expensive. Also, some states have a  limited list of contractors a state will approve or technologies that the  contractor can offer. For example, officials from one state we visited  reported that the technologies provided by contractors were not  completely compatible with existing local traffic safety data systems,  which limited its usefulness. However, we have previously reported that  hiring a contractor can help states obtain the technical expertise needed to  efficiently integrate data systems.
In light of these challenges, some states have implemented strategies to  overcome resource limitations. For example, North Carolina’s Governor’s  Highway Safety Program office took a series of targeted, incremental steps  to first focus on improving the quality of two traffic safety data system  performance measures—specifically timeliness and accuracy—in each  system before working on other performance measures, such as  integration. State officials emphasized the importance of focusing on the  “basics” and working from there, rather than starting with the most  complicated improvements. For example, North Carolina initially used  Section 408 grant program funding to create a guidebook that provides  consolidated information on all six traffic safety data systems and their  status. The guidebook enabled state officials to identify the most pressing  needs among all six traffic safety data systems and target limited  resources. Although the primary function of the guidebook was to increase  the accessibility of data system information, it also helped state officials  recognize the need to integrate traffic safety data systems to increase data  accessibility between data systems. Accordingly, North Carolina has an  active project to complete the linkage of crash and injury surveillance  data. Although the amount of Section 408 grant program funding is small  compared to state funding, North Carolina officials explained that the  program is a catalyst for progress by sometimes supporting smaller  projects like the guidebook, which then pave the way for larger projects,  such as integrating data systems.
According to state officials and one assessor, another strategy that one  state has used to overcome limited funding and staff was to contract out  the management of its centralized crash data system. For the state, this  project was revenue neutral because it does not require additional funds  for continued maintenance, as the contracted vendor receives payment by  selling crash reports and data extracts to interested parties. The profit  gave the vendor an incentive to work diligently with law enforcement  agencies to ensure reports are complete, accurate, and submitted in a  timely fashion to the central data system. There was also a built in  incentive for the law enforcement agency that submits the crash reports as  it also receives a reimbursement of 67 percent of the cost of each report  sold. As a result of contracting out the crash data system, the state  eliminated an annual cost of over $1 million for staffing, consulting, and  system maintenance, and no longer requires annual federal funds to help  support the system. This funding has since been redirected to hire  additional state troopers and add additional staff where needed.
Coordination Issues: Officials in all states we visited identified  coordination issues that presented challenges in improving state traffic  safety data. Some of the most frequently cited coordination issues are  discussed below.  “Stove-piped” agencies. Custodians of the different state traffic safety data  systems are oftentimes housed in different state offices or agencies. A  number of federal officials, state officials, and assessors reported  instances of unwillingness to share data between various offices because  of the “stove-piped” structure where there is little interaction between  traffic safety data stakeholders. Furthermore, we heard from state officials  and assessors that there is not always a clear understanding of the  relationship among all six traffic safety data systems. Typically most of the  data systems are housed within a state’s Department of Transportation or  Department of Public Safety, which can compound coordination  challenges for data systems housed elsewhere (e.g., injury surveillance  data).
Privacy concerns. According to state officials and assessors, federal and  state privacy laws can limit accessibility and sharing of certain traffic  safety data. A traffic records assessment for one state that we visited  reported that restrictions placed on release of crash data in general, and of  personal identifiers in the crash data for use by analysts within state  government offices, posed major barriers to crash data analysis. This  assessment also reported that these restrictions do not affect the state’s  ability to generate reports such as annual crash reports or most ad hoc  analyses of the state’s crash experience, but does limit the state’s ability to  perform more detailed crash problem identification and to support  research into the safety implications of specific laws or policies.
Decentralized state governance structures. State governance structures  can further complicate coordination efforts. For example, decentralized  court systems such as those found in two states we visited make it difficult  for the state to collect adjudication data from lower-level courts.  Addressing such governance issues can take many years. For example,  Minnesota officials said that the state has worked to centralize its court  system over a 15-year process. State officials and assessors also reported  that there is little incentive for jurisdictions, agencies, and individuals to  collect and submit data in a timely fashion. Some states have mandated  deadlines for the submission of data, but these deadlines are not always  adhered to by all agencies required to report. Although some states have  the ability to sanction those jurisdictions that do not submit data, this  option is not always used.
Federal and state officials, as well as assessors, told us that executive-level  TRCCs, which include key decisionmakers such as agency directors, can  help technical-level TRCCs overcome a variety of coordination and  resource impasses. The technical-level TRCCs have been one of the  successes of the Section 408 grant program and NHTSA officials and  officials in nearly all of the states that we visited praised TRCC activities in  bringing state stakeholders together, establishing important relationships,  and moving traffic safety data systems forward. While technical-level  TRCCs may lack the authority to implement certain decisions or traffic  safety data projects, according to NHTSA officials, several assessors, and  state officials, the authority associated with executive-level TRCCs can  help prioritize traffic safety data improvements and coordinate efforts. For  example, assessors explained that if a data manager refuses to share data,  an executive-level TRCC could compel data sharing. They also said that  the involvement and support of executive-level decision makers can raise  the profile of traffic safety data projects, which do not always receive  much attention, and provide the necessary leadership to complete traffic  safety data improvement projects. NHTSA officials also noted that  executive-level TRCCs can help states commit resources to traffic safety  data projects. For example, officials in one state reported that information  technology staff sometimes have not prioritized traffic safety data projects  due to limited resources. However, executive-level TRCC representatives  in that state have the authority to target and dedicate these sometimes  limited information technology resources to traffic safety data projects.  Figure 11 depicts some of the advantages of an executive-level TRCC.
Officials from North Carolina, a state we visited with an executive-level  TRCC, reported that the state’s executive-level TRCC oversees all  highway safety issues and fills the role of “champion” for the state’s  initiatives. All of the technical-level TRCC’s activities are reported to the  executive-level TRCC. The executive-level TRCC helps the technical-level  TRCC prioritize issues, provide assistance on legislative initiatives or  interagency projects requiring significant resources. Currently, the state is  developing new traffic safety data projects that will require legislation to  be passed for funding. A state official said that executive-level TRCC  endorsement and support will be necessary to pass the legislation.
While advantageous, currently few states have established executive-level  TRCCs. A NHTSA study recommended that states should have both a  technical-level and executive-level TRCC to be successful, but as  previously noted, only technical-level TRCCs are required for states to be  eligible for funding under the Section 408 grant program. Based on  estimates from one traffic records assessor, as of November 2009, nine  states had an executive-level TRCC. Several traffic records assessments,  however, have recommended that states establish executive-level TRCCs  to help improve traffic safety data systems.
section_title: Rural and Urban Areas May Face Distinct Challenges
paragraphs: Rural and urban areas across the country faced some distinct challenges in  improving traffic safety data systems. As previously discussed, some state  roadway data systems do not include locally maintained roadway data,  which may include rural road data, and therefore do not provide a full  picture of a state’s roadway system. As previously reported, many states  have not developed roadway inventory data for locally maintained roads  because they do not operate and maintain those roads, and are concerned  about the possible costs and time frames involved in obtaining these data.  As a result, states may have difficulty applying a data-driven, strategic  approach to highway safety. In addition, despite the higher proportion of  fatalities occurring in rural areas, officials in one state expressed concerns  that a proportional amount of state traffic safety funding is not allocated to  reflect this higher fatality level. We have also previously reported that  limited data on rural roads can hinder state efforts in funding and  addressing its top traffic safety priorities. However, some states are  working to improve data on non-state owned roadways, including rural  roads. For example, Ohio’s LBRS established a partnership between state  and local governments and has allowed Ohio’s Department of  Transportation to expand roadway data to include more comprehensive  roadway information. Figure 12 depicts a map of Ohio’s Clark County,  showing the 188 percent increase in the number of located crashes  available for analysis through LBRS. This increase largely consisted of  crashes occurring on locally maintained roadways.
An additional challenge involves the volume of vehicle crashes affecting  when and how much data rural and urban areas can submit. For example,  state officials in two states we visited reported that rural areas submit  crash data more regularly due to lower volumes. According to one  assessor, some large urban law enforcement agencies have refused to  report crash data, leading to gaps that limit the state’s ability to make  decisions that effectively target resources. In contrast, officials in three  states we visited reported that urban areas find it more difficult to submit  crash data in a timely manner due to the large volumes of reports filed.  Further, some cities have their own discrete crash data systems due to  their high crash rates. According to officials in one state we visited, though  large cities may have their own crash records systems, the system may not  be linked to the state crash data system and contributes to a large number  of missing crash reports.
Some rural areas face additional challenges due to limited technology  options. The lack of telecommunications services, such as access to the  Internet, limits the ability of local jurisdictions to electronically submit  data to state data systems, which can reduce the timely submission of  data. We have previously reported that the cost of providing  telecommunications services is higher in rural areas than in urban areas,  in part due to lack of infrastructure. For some rural jurisdictions, even  when the technology is available, it may not be cost effective to use due to  lower volumes of traffic safety data submitted per year.
Officials from states we visited reported on some strategies being  implemented to overcome some of the challenges for rural and urban  areas. For example, in one state we visited, the state’s highway safety  office provided funding to equip state highway patrol vehicles in rural  areas with mobile data terminals. Currently, roughly 70 to 80 percent of  state highway patrol vehicles in rural areas have these terminals, which  have increased the timeliness of crash reports submitted in the state. In  another state, the state legislature created an organization to oversee  funding for rural and locally maintained roadways. This organization had  the mission of helping local agencies receive funding specifically targeted  at locally maintained roadways. Lastly, officials in another state we visited  reported an increase in the electronic submission of crash reports when  the state required at least a certain percentage of crash reports to be  electronically submitted in order to qualify for the Section 402 grant  funding. State officials identified certain urban areas that were either  underreporting crashes or not electronically submitting crash reports and  then worked with these jurisdictions to improve submission rates. Since  2007, one urban area in this state has increased its electronic submission  of crash reports from 44 percent to nearly 100 percent by the end of 2009.  Overall, 91 percent of all crashes are now being electronically submitted in  this state.
section_title: Conclusions
paragraphs: Improving state traffic safety data systems is critical to state efforts to use  data-driven approaches to improve traffic safety and reduce traffic  fatalities and injuries. The Section 408 grant program has helped states to  improve the quality of traffic safety data systems across NHTSA’s six  performance measures. Despite this progress, however, almost all states  have traffic safety systems that do not meet one or more performance  measures. The wide range of quality we found in state traffic safety data  systems underscores the importance of state traffic records assessments  in helping states to plan and prioritize improvements to traffic safety data  systems. However, incomplete and inconsistent information in the  assessments limits the usefulness of the assessments, which, according to  NHTSA’s implementing guidance, should be an “in-depth, formal review of  a state’s highway safety data and traffic records system.” Furthermore,  assessments in the updated traffic records assessment format currently  being used often do not systematically evaluate each of the six  performance measures as they relate to each of the six data systems.  Improving the completeness and consistency of assessments would help  states more accurately identify problems and effectively target limited  resources. NHTSA officials recognize the importance of these assessments  for states and are taking steps to identify improvements to some aspects of  the assessment process. However, NHTSA’s efforts to review the  assessment process and the effectiveness and utility of traffic records  assessments are in the early stages. Based on NHTSA’s statement of work  for the study, the contract includes a component to examine state traffic  records assessments for effectiveness and utility and identify any  improvements or degradations of traffic safety data quality. However, it is  unclear whether this review will evaluate the overall content and quality of  the information provided in the assessments to the level of specificity that  may be needed.
States face various resource and coordination challenges, which make  further progress in improving the quality of traffic safety data systems  difficult. State officials we spoke with noted several strategies to address  these challenges. One of these strategies—establishing an executive-level  TRCC—can potentially address multiple resource and coordination  challenges. Specifically, an executive-level TRCC can be a helpful tool for  states to prioritize traffic safety data improvements, coordinate efforts,  and overcome impasses. Although the Section 408 grant program requires  that states have a technical-level TRCC, it does not require states to  establish an executive-level TRCC. The establishment of an executive-level  TRCC holds promise, but we did not fully assess its value for states as it  was beyond the scope of this report.
section_title: Recommendations for Executive Action
paragraphs: We recommend that the Secretary of Transportation direct the NHTSA  Administrator to take the following two actions:    Ensure that traffic records assessments provide an in-depth evaluation  that is complete and consistent in addressing all performance measures  across all state traffic safety data systems. As part of NHTSA’s ongoing  initiatives to improve the traffic records assessment process, specific  efforts could include revisiting available assessment guidance, the  frequency and manner in which assessments are conducted, and NHTSA’s  assessment review process.
Study and communicate to Congress on the value of requiring states to  establish an executive-level TRCC in order to qualify for Section 408 grant  funding.
section_title: Agency Comments
paragraphs: We provided a draft of this report to DOT for review and comment. DOT  officials agreed with the findings and recommendations in the report and  offered technical corrections that we incorporated, as appropriate.  Regarding the recommendation to ensure that traffic records assessments  provide an in-depth evaluation that is complete and consistent, the  officials noted that NHTSA has begun several initiatives to identify  opportunities to improve the assessment process and provide the states  with a more effective assessment document.
We are sending copies of this report to the Secretary of Transportation  and interested congressional committees. The report is also available at no  charge on GAO’s Web site at http://www.gao.gov.
If you or your staff have any questions concerning this report, please  contact me on (202) 512-2834 or flemings@gao.gov. Contact points for our  offices of Congressional Relations and Public Affairs may be found on the  last page of this report. GAO staff who made major contributions to this  report are listed in appendix III.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: In response to your request, this report provides information on the status  of the quality of state traffic safety data systems. In particular, we sought  to identify (1) the extent to which state traffic safety data systems meet  National Highway Transportation Safety Administration’s (NHTSA)  performance measures for assessing the quality of data systems, and (2)  what progress states have made in improving traffic safety data systems  and what challenges remain.
To identify the extent of state traffic safety data systems meeting NHTSA’s  performance measures, we analyzed the most recent traffic records  assessments for each of the 50 states and the District of Columbia (D.C.).  A traffic records assessment is a state document that contains findings and  recommendations on the quality of a state’s traffic safety data systems,  among other things. Assessments are conducted or updated at least every  5 years as one of the eligibility requirements for Section 408 grant program  funding. At least three team members reviewed each assessment and  coded the extent to which a state’s six traffic safety data systems met each  of NHTSA’s six performance measures—timeliness, consistency,  completeness, accuracy, accessibility, and integration. After individual  team members independently coded the data quality of assigned state  traffic records assessments, the three member sub-group met to discuss  the coding categories and reached consensus on the final coding category  assignment for each performance measure. Independently coding, initial  unanimous agreement was reached 37 percent of the time amongst the  three coders before discussions to reach consensus. Across states, initial  unanimous agreement was as high as 58 percent for one state, but for two  states there was no unanimous agreement for any of the coding categories.  Within the performance measures there was also a range of initial  unanimous agreement. For vehicle information timeliness, individual  coders reached unanimity for 37 states, including D.C. (73 percent). The  lowest level of initial unanimity (14 percent, or seven states) occurred  within the injury surveillance system’s accuracy performance measure.
Throughout this document we use the term “coding category” to refer to  the extent to which a data system meets an individual performance  measure. We created broad categories based on information presented in  state traffic records assessments; these coding categories are not precise  measurements of the extent to which data systems met performance  measures. We assigned numbers to correlate with the coding categories  defined below:  0 – Did not meet or minimally met performance measure (i.e.,  negligible, 0 to 5 percent). The state did not meet or minimally met a  particular performance measure based on the available evidence. The state  clearly did not meet, or the state met the performance measure to a  negligible extent. For example, one state’s crash data timeliness was  described as, “At present, crash data entry is experiencing a 12-month  backlog. This is due to delays at every step in the process from initial  crash reporting through final data entry and the multi-step/multi-stop  process that is used in handling crash reports. The delays are having an  impact on highway safety analysis and decision making in the state.” Since  the criteria for crash timeliness is that the information should be available  within a time frame to be currently meaningful for effective analysis of the  state’s crash experience, preferably within 90 days, this performance  measure area was coded as a zero.  1 – Marginally met performance measure (i.e., slightly, to a limited  extent, greater than 5 to 50 percent). The state met the performance  measure at some level above “minimally,” but not to a significant extent.  The state met the performance measure to a slightly, or to a very limited  extent. For example, one state’s citation and adjudication data consistency  was described as, “Although there is a uniform traffic citation for [the  state], not all agencies use it in the same manner.  has opted to  use it differently than the rest of the state. Since  is a state with a  court administrator that oversees each court, there seems to be some  consistency in the way cases are adjudicated…  is recorded at the  courts is controlled so that each court records the same information.”  Since the criteria for citation consistency is that all jurisdictions should  use a uniform traffic citation form, and the information should be  uniformly reported throughout all enforcement jurisdictions, this  performance measure area was coded as a one.  2 – Generally met performance measure (i.e., significant extent, for  the most part, greater than 50 to 95 percent). For the most part, the state  met the performance measure, but with some limitations. For example,  one state’s vehicle data accuracy was described as, “…in transition. The  Department of Motor Vehicles has used Vehicle Identification Number  (VIN) Analysis Software to enhance accuracy, but the descriptive  information about vehicles was taken from registration and title  applications. Beginning in 2006, the Department of Motor Vehicles has  been entering the body style and descriptive information from VIN  decoding and has been upgrading the descriptions to VIN decoded entry  when re-titling vehicles.” The criteria for vehicle system accuracy includes  that the state should employ methods for collecting and maintaining  vehicle data that produces accurate data and should make use of current  technologies designed for these purposes; therefore, this performance  measure area was coded as a two.  3 – Completely met performance measure (i.e., fulfills or satisfies the  condition, greater than 95 to 100 percent). The state fully met all aspects of  the performance measure, and if any limitation was identified it was not  material in nature. For example, one state’s roadway data accessibility was  described as, “Data are accessed through the Roadway Information  Management System and Integrated Transportation Management System.  Various reports are produced on a daily basis for use both within the  Department of Transportation (DOT) and for use by consultants,  businesses and the general public.” Since the criteria for roadway  accessibility is that the information should be readily and easily available  to the principal users of these databases containing the roadway  information for both direct (automated) access and periodic outputs  (standard reports) from the files, this performance measure was coded as  a three.  9 – Unknown. By “unknown” we mean that no other categorization was  possible. This may be due to limited information preventing  categorization, or that such information is absent. For example, one state’s  roadway data integration was described as, “The integration of road and  crash files seems to be adequate for present uses within [the state’s  Department of Transportation].” This limited information did not directly  address the integration of roadway data. In another example, a state’s  injury surveillance data completeness and accuracy was described as,  “Data completeness and data accuracy were not able to be evaluated  during our assessment.” Due to the absent information, these performance  measure areas were coded as a nine.
The extent to which a state has met a performance measure is considered  a reflection of data system quality. Throughout this report, in instances  where a performance measure was coded as a zero or a one the  performance measure is considered not met, whereas, if a two or a three  was assigned the performance measure is considered met. After we  concluded the coding of the assessments, we conducted a series of  statistical analysis. Analysis included answering the following questions:    Overall frequency of each coding category (0, 1, 2, 3, 9).
Frequency of each coding category for each of the six data systems (0, 1, 2,  3, 9).
Frequency of each coding category for each of the six performance  measures (0, 1, 2, 3, 9).
Frequencies by measure and system (total of 36 sets of frequencies).
Sum “score” for each state (excluding the coding category 9).
Frequency of the coding category 9 in the new assessment format as  compared with the old format (which includes a section dedicated to  “Information Quality”).
Percent of states with one or more 9s and total percent of the time a 9 was  assigned.
The number of times a state scored a 3 or 2 (completely or substantially)  in each system. Provided as the number of states with zero 2s or 3s in each  system, one 2 or 3 in each system, etc.
The number of times a state scored a 0 or 1 (not met or marginally) in each  system. Provided as the number of states with zero 1s or 2s in each  system, one 0 or 1 in each system, etc.
The extent to which a state has met a performance measure is a reflection  of data system quality. In addition to our analysis of state traffic records  assessments, this objective was informed through documentary and  testimonial evidence gathered on site visits. We collected and reviewed  relevant advisories and guidance related to traffic safety. We also  interviewed federal, state, and local officials, data users, and other experts  to obtain perspectives on the quality of traffic safety data. However, we  did not factor these other information sources into our traffic records  assessment coding analysis.
To identify the progress states have made in improving traffic safety data  systems and to determine what challenges remain, we reviewed states’  progress in meeting performance measures reported to NHTSA and in  state documents, such as State Highway Safety Strategic Plans. We  conducted site visits to eight states: Georgia, Idaho, Maine, Minnesota,  North Carolina, Ohio, Texas, and Virginia. We selected these states based  on a number of factors, including NHTSA recommendations, fatality rates,  population, roadway ownership, prevalence of rural roads, and geographic  diversity. NHTSA officials also provided input on states that they believed  encompassed a wide range of traffic safety data system quality. During our  site visits we interviewed state officials to identify progress in improving  the quality of traffic safety data and associated systems. To identify state  challenges in improving data systems, we conducted a literature review of  past GAO work and other relevant studies. We also conducted in-depth  interviews with state officials responsible for data systems, and collectors  and users of state traffic safety data during our state site visits.  Additionally, we spoke with NHTSA, national industry associations  representing the different data systems, and experts in the field to inform  our analysis of the primary challenges states face, as well as to inform us  of state efforts to address these challenges. We compiled all the various  interviews and conducted an analysis to identify the most frequently cited  challenges.
section_title: Appendix II: Additional Data Analysis
paragraphs: The following table represents all values associated with our coding  analysis of traffic records assessments for 50 states and D.C. We  calculated scores for all 50 states and D.C. by adding the number of points  received by a state. The total number possible was calculated by  multiplying the number of systems (6) by the number of performance  measures (6) by the number of possible points available per measure (3).  This resulted in a maximum score of 108 points that states could receive  based on the quality of their traffic safety data systems.
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Sara Vermillion (Assistant  Director), Matt Cail (Analyst-in-Charge), Emily Eischen, Brandon Haller,  Delwen Jones, Catherine Kim, Kirsten Lauber, Hannah Laufe, Josh  Ormond, and Crystal Wesco made key contributions to this report.