The interstate commercial motor carrier industry, primarily the trucking industry, is an important part of the nation’s economy. Trucks transport over 11 billion tons of goods, or about 60 percent of the total domestic tonnage shipped. Buses also play an important role, transporting an estimated 860 million passengers in 2005. FMCSA estimates that there are 711,000 interstate commercial motor carriers, about 9 million trucks and buses, and about 10 million drivers. Most motor carriers are small; about 51 percent operate one vehicle, and another 31 percent operate two to four vehicles. Carrier operations vary widely in size, however, and some of the largest motor carriers operate upwards of 58,000 vehicles. Carriers continually enter and exit the industry. Since 1998, the industry has increased in size by an average of about 29,000 interstate carriers per year. In the United States, commercial motor carriers account for fewer than 5 percent of all highway crashes, but these crashes result in about 13 percent of all highway deaths, or about 5,500 of the approximately 43,000 highway fatalities that occur nationwide annually. In addition, on average, about 160,000 of the approximately 3.2 million highway injuries per year involve motor carriers. The fatality rate for trucks has generally decreased over the past 30 years but has been fairly stable since 2002. The fatality rate for buses decreased slightly from 1975 to 2005, but it has more annual variability than the fatality rate for trucks due to a much smaller total number of vehicle miles traveled. (See fig. 1.) In an attempt to reduce the number and severity of crashes involving large trucks, FMCSA was established by the Motor Carrier Safety Improvement Act of 1999. FMCSA assumed almost all of the responsibilities and personnel of the Federal Highway Administration’s Office of Motor Carriers. The agency’s primary mission is to reduce the number and severity of crashes involving large trucks and buses. It carries out this mission by (1) issuing, administering, and enforcing federal motor carrier safety regulations and hazardous materials regulations; (2) providing education and outreach for motor carriers and drivers on the safety regulations and hazardous materials regulations; (3) gathering and analyzing data on motor carriers, drivers, and vehicles; (4) developing information systems to improve the transfer of data; and (5) researching new methods and technologies to enhance motor carrier safety. FMCSA relies heavily on the results of compliance reviews to determine whether carriers are operating safely and, if not, to take enforcement action against them. (See fig. 2.) FMCSA conducts these on-site reviews to determine carriers’ compliance with safety regulations that address areas such as testing drivers for alcohol and drugs, insurance coverage, crashes, driver qualifications, driver hours of service, vehicle maintenance and inspections, and transportation of hazardous materials. Due to resource constraints, FMCSA and its state partners are able to conduct compliance reviews on only about 2 percent of the nation’s estimated 711,000 interstate motor carriers each year. It is FMCSA’s policy to target these reviews at carriers that have been assessed by SafeStat as having the highest risk of crashes, have been the subject of a safety-related complaint submitted to FMCSA, have been involved in a fatal accident, have requested an upgraded safety rating based on safety improvements, or have been assigned a safety rating of conditional following a previous compliance review. Based largely on the number and severity of violations that it identifies during compliance reviews, FMCSA assigns carriers safety ratings that determine whether they are allowed to continue operating. FMCSA can take a range of enforcement actions against carriers with violations, including issuing notices of violation informing carriers of identified violations and indicating that additional enforcement action may be taken if the violations are not corrected; issuing compliance orders directing carriers to perform certain actions that FMCSA considers necessary to bring the carrier into compliance with regulations; assessing fines for violations of the safety regulations; fines require carriers to pay a specific dollar amount to FMCSA; placing carriers or drivers out of service for unsatisfactory safety performance, failure to pay a fine, or imminently hazardous conditions or operations; revoking the operating authority of carriers for failure to carry the required amount of insurance coverage; pursuing criminal penalties in some instances when knowing and willful violations can be proved; and seeking injunctions from a court for violations of a final order such as an out-of-service order. FMCSA has 52 division offices that partner with the 56 recipients of its Motor Carrier Safety Assistance Program grants. FMCSA also funds and oversees enforcement activities, including compliance reviews, at the state level through this grant program. The program was appropriated $188 million, or about 38 percent, of FMCSA’s $501 million appropriation for fiscal year 2006. In fiscal year 2006, FMCSA conducted 9,719 compliance reviews, and its state partners conducted 5,463 compliance reviews. SafeStat assesses carriers’ risks relative to all other carriers based on safety indicators such as their crash rates and safety violations identified during roadside inspections and during prior compliance reviews. A carrier’s score is calculated on the basis of its performance in the following four safety evaluation areas: The accident area reflects a carrier’s crash history relative to other motor carriers based on data from states and MCMIS. The driver area reflects a carrier’s driver-related safety performance and compliance relative to other motor carriers based on driver violations identified during roadside inspections and compliance reviews. The vehicle area reflects a carrier’s vehicle-related safety performance and compliance relative to other motor carriers based on vehicle-related violations identified during roadside inspections and compliance reviews. The safety management area reflects the carrier’s safety management performance relative to other motor carriers based on safety-management-related violations (such as failing to implement a drug or alcohol testing program) and hazardous-materials-related violations identified during compliance reviews and on closed enforcement cases resulting from compliance reviews. A motor carrier’s score is based on the carrier’s relative ranking, indicated as a value, in each of the four safety evaluation areas. This value can range from 0 to 100 in each area, and any value of 75 or greater is considered deficient. Any value of less than 75 is not considered deficient and is not used in calculating a SafeStat score. FMCSA assigns categories to carriers ranging from A to H according to their performance in each of the safety evaluation areas. (See table 1.) Although a carrier may receive a value in any of the four safety evaluation areas, the carrier receives a SafeStat score only if it is deficient in two or more safety evaluation areas. The calculation used to determine a motor carrier’s SafeStat score is SafeStat score = 2 x accident value + 1.5 x driver value + vehicle value + safety management value As shown in the formula, the accident and driver areas have 2.0 and 1.5 times the weight, respectively, of the vehicle and safety management areas. FMCSA assigned more weight to these areas because accidents and driver violations correlate relatively better with future crash risk. In consultation with state transportation officials, insurance industry representatives, safety advocates, and the motor carrier industry, FMCSA used its expert judgment and professional knowledge to assign these weights, rather than determining them through a statistical approach, such as regression modeling. Based on the results of a compliance review, FMCSA assigns the carrier a safety rating of satisfactory, conditional, or unsatisfactory. The safety rating, which is distinct from a carrier’s SafeStat category, reflects FMCSA’s determination of a carrier’s fitness to operate safely. FMCSA issues out-of-service orders to carriers rated unsatisfactory, and these carriers are not allowed to resume operating until they make improvements that result in an upgraded safety rating. Carriers rated conditional are allowed to continue operating, but FMCSA aims to conduct follow-up compliance reviews on these carriers. FMCSA assigns safety ratings based on a carrier’s performance in six areas. (See table 2.) One area is the carrier’s accident rate, and the other five areas involve its compliance with regulations. The five regulation-based areas are (1) minimum insurance coverage and procedures for handling and evaluating accidents; (2) drug and alcohol use and testing, commercial driver’s license standards, and driver qualifications; (3) driver hours of service; (4) vehicle parts and accessories necessary for safe operation; inspection, repair, and maintenance of vehicles; and (5) transportation of hazardous materials. Regardless of a carrier’s safety rating, FMCSA can assess a fine against a carrier with violations, and it is more likely to assess higher fines when these violations are serious. FMCSA uses a tool to help it determine the dollar amounts of its fines. Federal law requires FMCSA to assess the maximum allowable fine against a carrier for each serious violation of federal motor carrier safety and commercial driver’s license laws if the carrier is found to have a pattern of such violations or a record of previously committing the same or a related serious violation. SafeStat identifies many carriers that pose high crash risks. However, modifications to FMCSA’s policy that carriers have to score among the worst 25 percent of carriers in two or more safety evaluation areas to receive high priority for a compliance review and focusing more on crash risk could result in the selection of carriers with a higher aggregate crash risk. FMCSA recognizes that SafeStat can be improved, and as part of its Comprehensive Safety Analysis 2010 reform initiative, which is aimed at improving its processes for identifying and dealing with unsafe carriers, the agency is considering replacing SafeStat with a new tool by 2010. FMCSA’s policy for prioritizing carriers for compliance reviews based on their SafeStat scores results in FMCSA’s conducting compliance reviews on carriers with a higher aggregate crash risk than carriers that are not selected. As a result, FMCSA’s prioritization policy has value as a method for targeting high-risk carriers. But changes to the policy could result in targeting carriers with an even higher aggregate crash risk. According to our analysis of SafeStat’s June 2004 categorization of carriers, the 4,989 carriers that received high priority for a compliance review (SafeStat categories A or B) had a higher aggregate crash risk (102 crashes per 1,000 vehicles in the 18 months following the SafeStat categorization) than the remaining 617,034 carriers (27 crashes per 1,000 vehicles). (See table 3.) However, the 2,464 carriers that scored among the worst 25 percent of carriers in the accident evaluation area alone (SafeStat category D) had a slightly higher aggregate crash risk (112 crashes per 1,000 vehicles) than did the carriers in SafeStat categories A or B. Furthermore, the 1,090 carriers that scored among the worst 10 percent and the 492 carriers that scored among the worst 5 percent of carriers in the accident area (and did not score among the worst 25 percent of carriers in any other area) had even higher aggregate rates of 148 and 213 crashes per 1,000 vehicles, respectively. Our analysis suggests that FMCSA’s targeting of high-risk carriers could be enhanced by giving high priority for a compliance review to carriers that score among the worst 25, 10, or 5 percent of carriers in the accident evaluation area alone. We recognize that giving such carriers high priority for a compliance review would increase FMCSA’s and the states’ compliance review workloads unless FMCSA were to make another change to its prioritization policy that resulted in removing the same number of carriers from the high-priority categories A and B. For example, if FMCSA had given high priority to the 492 carriers that scored among the worst 5 percent of carriers in the accident evaluation area in June 2004, it could have removed the 492 carriers in categories A or B with the lowest SafeStat score in order to hold its and the states’ compliance review workloads constant. The lowest-scoring carriers in categories A and B had an aggregate crash risk of 65 crashes per 1,000 vehicles, less than one-third the crash risk of the carriers that could have replaced them (214 crashes per 1,000 vehicles). We also found that carriers that scored among the worst 25 percent, 10 percent, or 5 percent of carriers in either the driver, vehicle, or safety management areas (and did not score among the worst 25 percent of carriers in any other area) had a lower aggregate crash risk than carriers in SafeStat categories A or B. Of these various groups of carriers with poor performance in a single area, the carriers that scored among the worst 10 percent of carriers in the driver area had the highest aggregate crash risk (70 crashes per 1,000 vehicles). In our June 2007 report, we estimated that FMCSA could improve SafeStat’s performance by about 9 percent by using a statistical regression model approach to weight the accident, driver, vehicle, and safety management evaluation areas instead of its current approach, which is based on expert judgment. Employing this approach would have allowed FMCSA to identify carriers with almost twice as many crashes in the following 18 months as those carriers identified under its current approach. We found that although the driver, vehicle, and safety management evaluation area scores are correlated with the future crash risk of a carrier, the accident evaluation area correlates the most with future crash risk and should be weighted more heavily than the current SafeStat formula weights this area. These results corroborate studies performed by the Volpe National Transportation Systems Center and Oak Ridge National Laboratory, the latter of which also employed statistical approaches. (See app. I for a discussion of these studies.) We believe that our regression model approach from our June 2007 report is preferable to the prioritization approach we developed in this report because it provides for a systematic assessment of the relative contributions of accidents and driver, vehicle, and safety management violations. That is, by its very nature, the regression model approach looks for the “best fit” in identifying the degree to which prior accidents and driver, vehicle, and safety management violations identify the likelihood of carriers having crashes in the future, compared with the current SafeStat approach and the prioritization approach we developed for this report, both of which use expert judgment to establish the relationship among the four evaluation areas. In addition, because the regression model could be run monthly—as is the current SafeStat model—any change in the degree to which accidents and driver, vehicle, and safety management violations better identify future crashes will be automatically considered as different weights are assigned to the four evaluation areas. This is not the case with the current SafeStat model, in which the evaluation area weights generally remain constant over time. Thus, the systematic assessment and the automatic updating of evaluation area weights using a regression model approach better ensure the targeting of carriers that pose high crash risks—both currently and in the future. We compared the performance of our regression model approach to the current SafeStat model and to two alternative approaches that employ the current SafeStat model approach (with the current weighting of evaluation areas) but give higher priority to some carriers in category D (carriers that scored among the worst 25 percent of carriers in only the accident evaluation area). The two alternatives were substituting carriers in the worst 5 percent of the accident evaluation area for carriers in SafeStat categories A and B with (1) the lowest accident area scores and (2) the lowest overall SafeStat numerical scores. The regression model approach performed better than the current SafeStat approach and at least as well as the alternatives discussed in this report, in terms of identifying carriers that experienced a higher aggregate crash rate or a greater number of crashes. (See table 4.) For example, the regression model approach identified carriers with an average of 111 crashes per 1,000 vehicles over an 18-month period compared with the current SafeStat approach that identified carriers for compliance reviews with an average of 102 crashes per 1,000 vehicles. The regression model approach also performed at least as well as the alternatives discussed in this report in terms of identifying carriers with the highest aggregate crash rate and much better than the alternatives in identifying carriers with the greatest number of crashes. Finally, the alternatives discussed in this report were superior to the results of FMCSA’s current prioritization policy in terms of identifying carriers with both a higher aggregate crash rate and a greater number of crashes. FMCSA officials told us that the agency plans to assess whether the approach developed in this report—giving high priority to carriers that perform very poorly in only the accident evaluation area (such as those that scored among the worst 5 percent)—would be an effective use of its resources. However, FMCSA officials expressed concern that adopting our regression model approach would reduce the effectiveness of FMCSA’s compliance review program by targeting many compliance reviews at carriers that, despite high crash rates, have good compliance records. FMCSA believes that compliance reviews of such carriers, compared with compliance reviews of carriers in SafeStat categories A or B (carriers that, by definition, have a history of noncompliance), have less potential to reduce accidents. FMCSA said that this is because compliance reviews are designed to reduce crashes by identifying safety violations that some carriers then correct, and compliance reviews of carriers with good compliance records but high crash rates have historically identified fewer serious violations than compliance reviews of carriers in SafeStat categories A and B. FMCSA officials told us that, as part of its Comprehensive Safety Analysis 2010 reform initiative, the agency is evaluating the potential for new ways to address motor carriers that are having crashes, but that it believes are not good candidates for the compliance review tool. (See the discussion on FMCSA’s Comprehensive Safety Analysis 2010 reform initiative in a subsequent section.) We agree with FMCSA that the use of our model could tilt enforcement heavily toward carriers with high crash rates and away from carriers with compliance problems. We believe that use of the model would enhance motor carrier safety, even if it resulted in FMCSA reviewing carriers with good compliance records. FMCSA’s mission—and the ultimate purpose of compliance reviews—is to reduce the number and severity of truck and bus crashes. As previously discussed, we found that while driver, vehicle, and safety management evaluation area scores are correlated with the future crash risk of a carrier, high crash rates are a stronger predictor of future crashes than is poor compliance with safety regulations. These facts suggest that FMCSA would improve motor carrier safety more by targeting carriers with high crash rates, even if they have better compliance records, than by targeting carriers in SafeStat categories A and B with significantly lower crash rates but with worse compliance records. The missing piece in the puzzle is that FMCSA does not have a good understanding of why some carriers, despite good compliance records, have high crash rates; how compliance reviews affect their crash rates; and what other approaches may be effective in reducing their crash rates. We believe that developing this understanding would be a natural outgrowth of implementing our regression model approach. FMCSA officials also said that placing more emphasis on the accident evaluation area would increase emphasis on the least reliable type of data used by SafeStat—crash data—and in so doing, it would increase the sensitivity of the results to crash data quality issues. However, our June 2007 report found that FMCSA has made a considerable effort to improve the reliability of crash data. That report also concluded that as FMCSA continues its efforts to have states improve crash data, any sensitivity of results from our regression model approach to crash data quality issues should diminish. FMCSA officials were also concerned that our issuing two reports on SafeStat within several months of each other could be interpreted as an indictment of SafeStat and of FMCSA’s responsiveness to our June 2007 report on this issue. This is not the case. SafeStat does a good job of identifying carriers that pose high crash risks. As we reported in June 2007, we found that SafeStat is nearly twice as effective (83 percent better than) as random selection in identifying carriers that pose high crash risks and, therefore, has value for improving safety. Nonetheless, we found that FMCSA’s policy for prioritizing compliance reviews could be improved by applying either our regression model approach or one of the prioritization approaches we developed in this report. While we believe that the regression model approach provides somewhat better safety results, we understand, as discussed in our June 2007 report, that it could require FMCSA to re-educate the motor carrier industry and others, such as safety advocates, insurers, and the public, about the new approach. We would prefer that FMCSA implement our recommendation that it use our regression model approach but adopting either our regression model approach or one of the prioritization approaches we developed in this report would, in our opinion, improve FMCSA’s targeting of high-risk carriers. The recommendation that we make in this report reflects this conclusion. Finally, FMCSA has been very helpful and responsive during both our—largely concurrent—reviews. For our June 2007 report, we assessed the quality of the data used by SafeStat and the degree to which the quality of the data affects SafeStat’s identification of high-risk carriers, and we identified actions FMCSA has taken to improve the quality of the data used by SafeStat. We found that crash data reported by the states from December 2001 through June 2004 have problems in terms of timeliness, accuracy, and completeness that potentially hinder FMCSA’s ability to identify high-risk carriers. Regarding timeliness, we found that including late-reported data had a small impact on SafeStat—had all crash data been reported within 90 days of when the crashes occurred, 182 of the carriers identified by SafeStat as highest risk would have been excluded (because other carriers had higher crash risks), and 481 carriers that were not originally designated as posing high crash risks would have scored high enough to be considered high risk, resulting in a net addition of 299 carriers (or 6 percent) to the original 4,989 carriers that the SafeStat model ranked as highest risk in June 2004. We were not able to quantify the effect of incomplete or inaccurate data on SafeStat’s ability to identify carriers that pose high crash risks, because doing so would have required us to gather crash records at the state level—an effort that was impractical. FMCSA has acted to improve the quality of SafeStat’s data by completing a comprehensive plan for data quality improvement, implementing an approach to correct inaccurate data, and providing grants to states for improving data quality, among other things. We could not quantify the effects of FMCSA’s efforts to improve the completeness or accuracy of the data for the same reason as just mentioned. (See app. II for a more detailed discussion of the quality of the data used by SafeStat.) As part of its Comprehensive Safety Analysis 2010, a reform initiative aimed at improving its processes for identifying and dealing with unsafe carriers and drivers, FMCSA is considering replacing SafeStat with a new tool by 2010. The new tool could take on greater importance in FMCSA’s safety oversight framework because the agency is considering using the tool’s assessments of carriers’ safety to determine whether carriers are fit to continue operating. In contrast, SafeStat’s primary use now is in prioritizing carriers for compliance reviews, and determinations of operational fitness are made only after the compliance reviews are completed. While the new tool may use some of the same data included in SafeStat, such as carriers’ crash rates and driver and vehicle violations identified during compliance reviews and roadside inspections, it may also consider a broader range of behavioral data related to crashes than does SafeStat. For example, the new tool may consider information from crash reports, such as whether driver fatigue, a lack of driver experience, a medical reason, a mechanical failure, shifting loads, or spilled or dropped cargo, were cited as causal or contributing factors. An FMCSA official told us that the agency is analyzing the relationship between these factors and crash rates to help it determine how the factors should be assessed and the relative weights to place on the factors. We believe that, compared with the expert-judgment-based approach that FMCSA used to select the weights for SafeStat’s evaluation areas, this analytical approach has the potential to better identify high-risk carriers. FMCSA manages its compliance reviews in a fashion that meets our standards for internal control, thereby promoting thoroughness and consistency in the reviews. For example, it records its policies and procedures related to compliance reviews in an operations manual. FMCSA also provides investigators with classroom and on-the-job training on how to plan for and conduct compliance reviews. In addition, it employs an information system that documents the results of compliance reviews and allows FMCSA and state managers to review the compliance reviews for thoroughness, accuracy, and consistency. FMCSA uses several approaches to monitor its compliance review program, including an agencywide review in 2002 that led to several changes in the program. FMCSA’s communication of its policies and procedures related to conducting compliance reviews meets our standards for internal control. These standards state that an organization’s policies and procedures should be recorded and communicated to management and others within the entity who need it and in a form (e.g., clearly written and provided as a paper or electronic manual) and within a time frame that enables them to carry out their responsibilities. FMCSA records and communicates its policies and procedures electronically through its “Field Operations Training Manual” (hereafter called the operations manual), which it provides to all federal and state investigators and their managers. The operations manual includes guidance on how to prepare for a compliance review. For example, it tells investigators that they must download and review a report that includes information on the carrier’s accidents, drivers, and inspections, and it explains how this information can help the investigator focus the compliance review. It also specifies the minimum number of driver and vehicle maintenance records to be examined and the minimum number of vehicle inspections to be conducted during a compliance review. FMCSA aims to update its operations manual twice a year. It posts updates to the operations manual that automatically download to investigators and managers when they connect to the Internet. In between these updates, FMCSA communicates policy changes by e-mail. In addition to the operations manual, FMCSA provides training to investigators on its policies and procedures related to compliance reviews. FMCSA policy requires that investigators successfully complete classroom training and examinations before they conduct a compliance review. The training covers the safety and hazardous materials regulations and software tools used during compliance reviews. According to FMCSA officials, investigators then receive on-the-job training, which allows them to accompany an experienced investigator during compliance reviews. This training lasts until managers decide that the trainees are ready to complete a compliance review on their own, typically after 3 to 6 months on the job. Investigators can also take additional classroom training on specialized topics throughout their careers. Furthermore, according to FMCSA officials, FMCSA’s division offices hold periodic and ad hoc meetings to train investigators about policy changes related to compliance reviews. In addition, in commenting on a draft of this report, FMCSA noted that it has an annual safety investigator certification process to ensure that only qualified personnel conduct compliance reviews. FMCSA’s documentation of compliance reviews meets our standards for internal control. These standards state that all transactions and other significant events should be clearly and promptly documented, and the documentation should be readily available for examination. This applies to the entire process or life cycle of a transaction or event from the initiation and authorization through its final classification in summary records. The standards also state that control activities, including reviews of information and system edit checks, should help to ensure that all transactions are completely and accurately recorded. FMCSA and state investigators use an information system to document the results of their compliance reviews, including information on crashes and any violations of the safety regulations that they identify. This documentation is readily available to FMCSA managers, who told us that they review it to help ensure completeness and accuracy. FMCSA officials told us that the information system also helps ensure thoroughness and consistency by prompting investigators to follow FMCSA’s policies and procedures, such as requirements to meet a minimum sample size. The information system also includes checks for consistency and reasonableness and prompts investigators when the information they enter appears to be inaccurate. An FMCSA manager told us that managers typically assess an investigator’s thoroughness by comparing the investigator’s rate of violations identified over the course of several compliance reviews with the average rate for investigators in their division office; a rate that is substantially below the average suggests insufficient thoroughness. Generally, FMCSA and state investigators and managers said they found the information system to be useful. FMCSA’s performance measurement and monitoring of compliance review activities meet our standards for internal control. These standards state that managers should compare actual performance to planned or expected results and analyze significant differences. Monitoring of internal controls should include policies and procedures for ensuring that the findings of audits and other reviews are promptly resolved. According to FMCSA and state managers and investigators, the managers review all compliance reviews in each division office and state to ensure thoroughness and consistency across investigators and across compliance reviews. The investigators we spoke with generally found these reviews to be helpful, and several investigators said that the reviews helped them learn policies and procedures and ultimately perform better compliance reviews. FMCSA and state managers told us that they also use monthly reports to track the performance of investigators using measures such as the numbers of reviews completed and the rates of violations found. Managers generally found that these reports provide useful information on investigators’ performance, and several managers said that they use the reports to help identify specific areas where an investigator needs additional coaching or training. However, several state managers said that monitoring of their investigators’ performance would be enhanced if they had access to FMCSA’s monthly report on their investigators; currently, states rely on their own custom reports. FMCSA told us that it plans to make its monthly report on state investigators available to state managers by October 2007. In addition to assessing the performance of individual investigators, FMCSA periodically assesses the performance of FMCSA division offices and state agencies, and it conducted an agencywide review of its compliance review program in 2002. According to officials at one of FMCSA’s service centers, the service centers lead triennial reviews of the compliance review and enforcement activities of each division office and its state partner. These reviews assess whether the division offices and state partners are following FMCSA policies and procedures, and they include an assessment of performance data for items such as number of compliance reviews conducted, rate of violations identified, and number of enforcement actions taken. The officials said that some reviews identify instances of deviations by division offices from FMCSA’s compliance review policies, but that only minor adjustments by the division offices are needed. The officials also said that the service centers compile best practices identified during the reviews and share these among the division offices and state partners. To ensure that concerns identified during the reviews are addressed, the officials said that the service centers monitor the quality of individual compliance reviews that lead to enforcement cases and the monthly reports on division office and state activities. The officials said that the service centers also check on responses to previously identified concerns during the triennial reviews. FMCSA’s agencywide review indicated that inconsistencies and bottlenecks in the compliance review process were reducing its efficiency and effectiveness, and FMCSA made several changes in 2003 aimed at improving compliance review policies, procedures, training, software, and supporting motor carrier data. Examples of problems identified and actions taken are as follows: FMCSA discouraged repeat visits to high-risk motor carriers that had received unsatisfactory ratings during their last compliance review within the past 12 months because the agency believed that not enough time had elapsed to show whether safety improvements had taken effect. FMCSA discouraged safety investigators from their earlier practice of favoring violations of drug and alcohol regulations over violations of hours-of-service regulations when they choose which violations to document for enforcement because crash data and FMCSA’s survey of its field staff suggest that compliance with hours-of-service regulations is more important for safety. FMCSA revised its operations manual to encourage FMCSA’s division offices to document the maximum number of areas of the regulations where major safety violations are discovered, rather than penalizing motor carriers for a few violations in a particular area at the expense of other areas. FMCSA’s review also concluded that most investigators were not following FMCSA’s policy requiring them to perform vehicle inspections as part of a compliance review if the carrier has not already received the required number of roadside vehicle inspections. FMCSA has since changed its policy so that inspecting a minimum number of vehicles is no longer a strict requirement—if an investigator is unable to inspect the minimum number of vehicles, he or she must explain why in the compliance review report. FMCSA told us that, as part of their review of individual compliance reviews, division office managers ensure that when compliance reviews have fewer than the minimum number of vehicle inspections, investigators provide adequate justification in their reports. We did not verify this statement because we did not have enough time or resources. We did, however, assess the extent to which compliance reviews included the minimum number of vehicle inspections. In fiscal year 2005, FMCSA and its state partners conducted 7,436 compliance reviews on carriers that had not already received the minimum number of vehicle inspections; of these, only 254 compliance reviews (3 percent) included the minimum number of vehicle inspections. FMCSA’s review also found that investigators considered inspections to be the one aspect of compliance reviews, other than licensing and insurance verification, that had the smallest effect on carriers’ safety performance. FMCSA’s review team recommended that FMCSA establish new criteria for conducting vehicle inspections during compliance reviews, and suggested that inspections could be made optional. In contrast, in 2002, the National Transportation Safety Board (the Safety Board) recommended that FMCSA require that all compliance reviews include vehicle inspections. The Safety Board based its recommendation on its belief that the vehicles that receive roadside inspections may be less likely to have violations than the vehicles that could be inspected during a compliance review. In July 2006, FMCSA responded that implementing this recommendation would be imprudent because it would divert attention from driver and other safety factors, and FMCSA’s recent study of the causes of large truck crashes indicates the importance of driver factors, such as driving too fast for conditions and driver fatigue. FMCSA has not changed its policy, but an FMCSA official told us that under the operational model that FMCSA has proposed for its Comprehensive Safety Analysis 2010 reform initiative, vehicle inspections during compliance reviews would be optional. FMCSA also told us that it is developing a policy that would allow investigators conducting compliance reviews to inspect vehicles that operate in intrastate commerce. FMCSA believes that this policy will increase the number of compliance reviews with the minimum number of vehicle inspections. Finally, FMCSA’s review found that although investigators generally sampled the number of carrier records required by FMCSA’s policies, the number of undersized samples of drivers’ work hour logs was a cause for concern. The review said that a lack of clarity in FMCSA’s requirements for how carriers must document drivers’ hours was likely resulting in some carriers having too few records to sample. FMCSA is working to clarify its documentation requirements, but it has not set a date for completing this task. From fiscal year 2001 through fiscal year 2006, each of the nine major applicable areas of the safety regulations was covered by most of the approximately 76,000 compliance reviews conducted by FMCSA and the states. (See table 5.) An FMCSA official told us that not every compliance review is required to cover all nine areas and cited the following reasons: Follow-up compliance reviews of carriers rated unsatisfactory or conditional are sometimes streamlined to cover only the area or areas of the regulations in which the carrier had violations. Commercial driver’s license standards and drug and alcohol use and testing regulations apply primarily to those carriers that operate one or more vehicles weighing over 26,000 pounds (gross vehicle weight rating), that haul hazardous material, or that transport more than 15 passengers. Minimum insurance coverage regulations apply only to for-hire carriers and private carriers of hazardous materials; they do not apply to private passenger and nonhazardous materials carriers. However, according to an FMCSA official, the area of these regulations that had the lowest rate of coverage—vehicle parts and accessories necessary for safe operation—is required for all compliance reviews except streamlined reviews that exclude this area. Vehicle inspections are supposed to be a key investigative technique for assessing compliance with this area, and the FMCSA official said that the lower rate of coverage for this area likely reflects the small number of vehicle inspections that FMCSA and the states conduct during compliance reviews. In addition to the safety regulations, compliance reviews of hazardous materials carriers, shippers, and cargo tank facilities must cover hazardous materials regulations. In fiscal years 2005 and 2006, FMCSA conducted about 6,000 compliance reviews of hazardous materials operators. Collectively, these compliance reviews covered between 40 percent and 80 percent of the various individual areas of these regulations. However, none of these compliance reviews was required to cover all areas of the hazardous materials regulations; the required areas vary with the type of operator. Because the categories that MCMIS uses to classify hazardous materials operators are different from the categories used to determine which areas of the regulations must be covered, we could not determine, for the different types of operators, the extent to which FMCSA’s compliance reviews covered the required areas. FMCSA placed many carriers rated unsatisfactory in fiscal year 2005 out of service and followed up with nearly all of the rest to determine whether they had improved. In addition, FMCSA monitors carriers to identify those that are violating out-of-service orders. However, it does not take additional action against many of the violators of out-of-service orders that it identifies. Furthermore, FMCSA does not assess the maximum fines against all of the serious violators that we believe the law requires, partly because FMCSA does not distinguish between carriers with a pattern of serious safety violations and those that repeat a serious violation. FMCSA followed up with 1,193 of 1,196 carriers (99.7 percent) that received a proposed safety rating of unsatisfactory following a compliance review that was completed in fiscal year 2005. FMCSA’s follow-up generally ensured that these carriers either made safety improvements that resulted in an upgraded final safety rating or—as required for carriers that also receive a final safety rating of unsatisfactory—were placed out of service. More specifically, FMCSA used the following approaches to follow up with these carriers: Follow-up compliance review. Based on such reviews, FMCSA upgraded the final safety ratings of 663 carriers (329 to satisfactory, and 334 to conditional). Assignment of a final rating of unsatisfactory and issuance of an out- of-service order. FMCSA assigned a final rating of unsatisfactory to 312 carriers and issued an out-of-service order to 309 (99 percent) of them. An FMCSA official told us that it did not issue an out-of-service order to 2 carriers because it could not locate them, and it did not issue an out-of- service order to another carrier because the carrier was still subject to an out-of-service order that FMCSA issued several years prior to the 2005 compliance review. Review of evidence of corrective action. Carriers can request an upgraded safety rating by submitting evidence of corrective action to FMCSA. Based on reviews of such evidence, FMCSA upgraded the final safety ratings of 217 carriers (23 to satisfactory, and 194 to conditional). Administrative review. Carriers that believe FMCSA made an error in assigning their proposed safety rating may request the agency to conduct an administrative review. Based on the administrative review, FMCSA upgraded the final safety rating of 1 carrier to conditional. FMCSA did not assign final safety ratings to the remaining 3 carriers. For 1 of these carriers, MCMIS indicates that the compliance review that resulted in the proposed rating of unsatisfactory did not identify any violations, even though carriers without violations are not supposed to receive a proposed unsatisfactory rating. For another of the carriers, MCMIS shows crashes, inspections, and a compliance review while also indicating that the carrier is inactive. FMCSA has been unable to locate the final carrier, and MCMIS indicates that the carrier is inactive. Unless FMCSA upgrades a proposed unsatisfactory safety rating or grants a carrier an extension, the agency is required under its policy to assign the carrier a final rating of unsatisfactory and to issue it an out-of-service order on the 46th day after the date of FMCSA’s notice of a proposed unsatisfactory rating for carriers of hazardous materials or passengers and on the 61st day for other types of carriers. Of the 309 out-of-service orders that FMCSA issued to carriers rated unsatisfactory following compliance reviews conducted in fiscal year 2005, 276 (89 percent) were issued on time, 28 (9 percent) were issued between 1 and 10 days late, and 5 (2 percent) were issued more than 10 days late. FMCSA also assigned final upgraded safety ratings within these time frames in 837 (95 percent) of the 881 cases in which it upgraded these ratings. FMCSA assigned 20 upgrades (2 percent) between 1 and 10 days late, and it assigned another 20 (2 percent) more than 10 days late. MCMIS did not have information on the timing of the other 4 upgrades. An FMCSA official told us that when an out-of-service order was issued more than 1 week late, the primary reason for the delay was that the responsible FMCSA division office had difficulty scheduling a follow-up compliance review and thus waited to issue the orders. The official said that other delays were caused by clerical errors; extended periods during which certain division offices operated without a person serving in the position with primary responsibility for ensuring that out-of-service orders are issued on time; a lack of complete compatibility between MCMIS and FMCSA’s enforcement database; and, in one service center whose policy is to personally serve out-of-service orders to carriers, insufficient advance notification by the service center to its division offices that an order was to be served. The official noted that the last two issues have been addressed and said that FMCSA plans to more closely monitor the timeliness of the issuance of out-of-service orders in all of FMCSA’s division offices. FMCSA uses two primary means to try to ensure that carriers that have been placed out of service do not continue to operate. First, FMCSA partners with states to help them suspend, revoke, or deny vehicle registration to carriers that have been placed out of service. FMCSA refers to these partnerships as the Performance and Registration Information Systems Management program (PRISM). PRISM links FMCSA databases with state motor vehicle registration systems and roadside inspection personnel to help identify vehicles operated by carriers that have been issued out-of-service orders. As of January 2007, 45 states had been awarded PRISM grants, and 27 states were operating with PRISM capabilities. FMCSA officials told us that some states have not applied for PRISM grants because they do not want to bear the costs that are not covered by the grants or they have not made the legislative changes required to implement PRISM. According to an FMCSA official, FMCSA has also begun working with PRISM states to enable them to receive automated notifications of carriers that have been placed out of service. PRISM can also identify carriers that attempt to register vehicles under a different carrier name, and FMCSA provided us with information on two out-of-service carriers that Connecticut, using PRISM, had caught trying to register vehicles by using a new company name. In addition, in commenting on a draft of this report, FMCSA said that during the first 6 months of fiscal year 2007, states that reported data to FMCSA indicated that at least 104 motor carriers had their state vehicle registrations suspended, revoked, or denied based on an FMCSA order to cease interstate operations. FMCSA and its state partners also monitor carriers for indicators—such as roadside inspections, moving violations, and crashes—that the carriers may be violating an out-of-service order. First, FMCSA recently began to require the state partners that receive Motor Carrier Safety Assistance Program grants to check during roadside inspections whether carriers are operating under revoked authority and to take enforcement action against any that are. Second, FMCSA visits some suspect carriers that it identifies by monitoring crash and inspection data to examine their records to determine whether they did indeed violate the order. FMCSA told us it is difficult for it to verify that such carriers were operating in violation of out-of-service orders because its resources do not allow it to visit each carrier or conduct roadside inspections on all vehicles, and we agree. In fiscal years 2005 and 2006, 677 of 1,741 carriers (39 percent) that were subject to an out-of-service order had a roadside inspection or crash; FMCSA cited only 36 of these 677 carriers for violating the out-of-service order. An FMCSA official told us that some of these carriers, such as carriers that were operating intrastate or leasing vehicles to other carriers, may not have been violating the out-of-service order. The official said that the agency did not have enough resources to determine whether each of the carriers was violating the out-of-service order. He also said that FMCSA recently completed a pilot program in which the agency cited obvious violators such as carriers that have a roadside inspection outside their home state. In commenting on a draft of this report, FMCSA said that it is developing new policies and procedures intended to establish a uniform national approach for follow-up, as well as additional enforcement action against motor carriers that have violated an out-of- service order. In 2006, the Safety Board assessed FMCSA’s approach to ensuring that carriers whose operating authority has been revoked do not operate and concluded that it was inadequate. The Safety Board recommended that FMCSA establish a program to address this issue. In response to this recommendation, FMCSA noted that, because the numbers of carriers that have been placed out of service or have had their operating authority revoked has significantly increased in recent years, it is difficult to ensure that these carriers do not continue to operate. An FMCSA official attributed this difficulty to FMCSA’s lack of resources to visit each carrier or conduct roadside inspections on all vehicles—the same reason FMCSA cites for not following up on all carriers that may be violating an out-of- service order. Despite this difficulty, FMCSA responded that it (1) is linking its licensing and insurance database to its primary carrier database to improve the ability of roadside inspection personnel in all states and registration offices in PRISM states to identify carriers that have had their operating authority revoked and (2) has directed division office managers to assess fines when data accessed during roadside inspections indicate that carriers were operating under revoked authority. In March 2007, the Safety Board said that FMCSA was making acceptable progress on the recommendation, but expressed concern that some states will choose not to implement PRISM and that, based on the program’s rate of implementation thus far, it will take too long to become fully operational in many other states. The Safety Board, therefore, encouraged FMCSA to implement PRISM more rapidly in all states. An FMCSA official told us that the agency is already making a concerted effort to encourage the 5 states without PRISM to adopt the program and the 18 PRISM states that do not yet have full PRISM capabilities to achieve them. FMCSA’s policy requires the agency to conduct follow-up compliance reviews on all carriers rated conditional and, over the last several years, the agency has reduced the number of such carriers needing review. After the Department of Transportation Inspector General reported in 1999 that FMCSA allowed motor carriers with less than satisfactory ratings to continue operations for extended periods of time, FMCSA began requiring follow-up compliance reviews on all carriers rated conditional. In fiscal years 2005 and 2006, respectively, FMCSA conducted 2,537 and 2,692 follow-up reviews of carriers rated conditional or unsatisfactory, exceeding its annual goal of 2,500 follow-up reviews. In addition, from fiscal year 2000 through fiscal year 2006, the number of carriers rated conditional that needed a follow-up review decreased from about 40,000 to about 30,000. While FMCSA has reduced the number of carriers rated conditional that need a follow-up review, it is difficult to assess the agency’s timeliness in conducting these reviews because FMCSA’s policy does not specify a time frame for following up on carriers with conditional safety ratings. The policy does discourage follow-up reviews within 12 months because FMCSA believes that more time is needed to show the effects of safety improvements. Yet the policy also gives FMCSA’s division office administrators the discretion to determine whether a follow-up review should be conducted within 12 months. Almost half of all carriers that received a conditional rating from fiscal year 2002 through fiscal year 2004 received a follow-up review within 12 months; however, because of the policy’s allowance for discretion, we could not determine how many, if any, of these follow-up reviews, occurred too soon. (See table 6.) In addition, because FMCSA does not specify a deadline for conducting follow-up reviews, we could not determine whether any of the reviews occurred too late. Our analysis of the timing of follow-up reviews shows that from fiscal year 2002 through fiscal year 2004, 66 percent of the carriers that received a conditional rating received a follow-up review within 24 months, while 7 percent received a follow-up review more than 24 months after they received their conditional rating. Another 27 percent of the carriers still needed a review as of September 2006. In 1999, the Safety Board recommended that FMCSA lower its threshold for rating a carrier unsatisfactory to include carriers with an unsatisfactory rating in either the driver or vehicle factor of the rating scheme. The Safety Board has classified this recommendation as one of its “most wanted” safety improvements since 2000. Although FMCSA has not yet decided whether it will implement this recommendation, it is developing a new rating methodology as part of its Comprehensive Safety Analysis 2010 reform initiative, and it plans to implement the methodology in 2010. As mentioned previously, the new methodology would base determinations of whether carriers are fit to continue operating on assessments made by the tool that FMCSA is developing to replace SafeStat, rather than on the results of compliance reviews. FMCSA believes that the new approach will enable the agency to assess the safety fitness of a larger share of the motor carrier industry. FMCSA is also considering determining the safety fitness of drivers, and applying interventions to those that it deems need them. FMCSA believes that the increased focus that this would bring to the safety of drivers is important because the results of its recent study on the causes of large truck crashes indicate that drivers of large trucks and other vehicles involved in truck crashes are 10 times more likely to be the cause of the crash than other factors, such as weather, road conditions, and vehicle performance. In addition, FMCSA is considering eliminating the conditional rating and using only two ratings—“continue to operate” and “unfit.” An FMCSA official told us that FMCSA may eliminate the conditional rating because the agency feels that the current satisfactory rating is being misinterpreted by some government agencies and members of the public that hire carriers as FMCSA’s seal of approval. The official said that the agency believes that the “continue to operate” rating, which would be given to all carriers that are allowed to continue to operate, is less likely to be viewed as a seal of approval than the satisfactory rating, which indicates a level of safety that is greater than the conditional rating that also allows carriers to continue operating. Depending on their safety performance, carriers or drivers allowed to continue operating could be subject to interventions, such as Web-based education, warning letters, requests for submission of documents, targeted roadside inspections, focused on-site reviews, comprehensive on-site reviews (similar to compliance reviews), and enforcement actions. From August 2006 through February 2007, data from MCMIS indicate that FMCSA performed compliance reviews on 1,136 of the 2,220 (51 percent) carriers that were covered by FMCSA’s mandatory compliance review policy. Under the Safe, Accountable, Flexible, Efficient Transportation Equity Act: A Legacy for Users, FMCSA is required to conduct compliance reviews on carriers rated in SafeStat categories A or B for 2 consecutive months. In response to this requirement, in June 2006, FMCSA implemented a policy requiring a compliance review within 6 months for any such carrier unless the carrier had received a compliance review within the previous 12 months. An FMCSA official told us that the agency did not have enough resources to conduct compliance reviews on all of the 2,220 carriers within the first 6-month period. In April 2007, FMCSA revised the policy because the agency believes that it required compliance reviews for some carriers that did not need them, leaving FMCSA with insufficient resources to conduct compliance reviews on other carriers that did need them. The carriers that did not need compliance reviews were those that had already had a compliance review and had corrected identified violations, but these violations continued to adversely affect their SafeStat rating because SafeStat penalizes carriers for violations regardless of whether they have been corrected. This unnecessary targeting drained resources, leaving FMCSA without the means to conduct compliance reviews of carriers that had never received such a review, but, in FMCSA’s view, should have received one because of current safety performance issues that led to their placement in SafeStat categories C, D, or E. The new policy requires compliance reviews within 6 months for carriers that have been in SafeStat categories A or B for 2 consecutive months and received their last compliance review 2 or more years ago (or have never received a compliance review). In addition, compliance reviews are recommended for carriers that have been in SafeStat categories A or B for 2 consecutive months and received their last compliance review more than 1 year ago but less than 2 years ago. FMCSA division offices can decide not to conduct a compliance review on such a carrier if (1) its SafeStat category changes to a category other than A or B or (2) its safety evaluation area values are based largely on prior compliance review violations that have been corrected or on accidents or inspections that occurred prior to the carrier’s last compliance review. We believe that these changes are consistent with the act’s requirement and give FMCSA appropriate discretion in allocating its compliance review resources. From October 2005 through October 2006, FMCSA reduced its backlog of enforcement cases that had been open for 6 months or more by about 70 percent (from 807 to 247). As the Department of Transportation Inspector General has noted, a large backlog of enforcement cases negatively affects the integrity of the enforcement process for two reasons. First, because FMCSA considers only closed enforcement cases when targeting motor carriers for a compliance review, high-risk motor carriers are less likely to be selected if they have an open enforcement case. Second, because FMCSA assesses smaller fines against carriers with open cases than against those with closed cases, it may not assess appropriate fine amounts against carriers with multiple enforcement cases (the number of prior enforcement cases is one of the criteria that FMCSA uses to determine fine amounts). FMCSA’s 2002 review of its compliance review program also found that delays in closing enforcement cases were negatively affecting the integrity of the agency’s enforcement process. An FMCSA official told us that in response to this review, the agency assigned a second attorney to work on enforcement cases. In 2005, we recommended that FMCSA establish a goal specifying how much it would like to reduce the enforcement backlog and by what date. In March 2007, FMCSA implemented this recommendation by establishing goals to (1) close, by the end of 2007, its backlog of 63 enforcement cases in its division offices that had been open for 270 days or more and (2) close, by August 31, 2007, its backlog of 14 cases pending before its Assistant Administrator for Enforcement for more than 18 months, without adding other cases to this backlog. FMCSA does not assess maximum fines against all of the serious violators that we believe the law requires. The law requires FMCSA to assess the maximum allowable fine for each serious violation by a carrier that is found (1) to have a pattern of committing such violations (pattern requirement) or (2) to have previously committed the same or a related serious violation (repeat requirement). The legislative history of this provision provides evidence that FMCSA must assess maximum fines in these two distinct situations. However, FMCSA’s policy on maximum fines does not fully meet these requirements. FMCSA enforces both requirements using what is known as the “three strikes rule,” applying the maximum allowable fine when it finds that a motor carrier has violated the same regulation three times within 6 years. FMCSA officials said they interpret both parts of the act’s requirements to refer to repeat violations, and because they believe that having two distinct policies on repeat violations would confuse motor carriers, FMCSA has chosen to address both requirements with its single three strikes policy. According to FMCSA officials, FMCSA developed the three strikes policy in response to a provision in the Motor Carrier Safety Act of 1984, which permitted FMCSA’s predecessor to assess a fine of up to $1,000 per offense (capped at $10,000) if the agency determined that “a serious pattern of safety violations” existed or had occurred. FMCSA officials told us that when Congress in 1999 enacted the current “pattern of violations” language in the Motor Carrier Safety Improvement Act, the agency interpreted it to be similar to the previous language and to mean three strikes. FMCSA’s interpretation does not carry out the statutory mandate to impose maximum fines in two different cases. In contrast to FMCSA, we read the statute’s use of the distinct terms “a pattern of violations” and “previously committed the same or a related violation” as requiring FMCSA to implement two distinct policies. A basic principle of statutory interpretation is that distinct terms should be read as having distinct meanings. In this case, the statute not only uses different language to refer to the violations for which maximum fines must be imposed, but it also sets them out separately and makes either type of violation subject to the maximum penalties. Therefore, one carrier may commit a variety of serious violations and another carrier may commit a serious violation that is the same as, or substantially similar to, a previous serious violation; the language on its face requires FMCSA to assess the maximum allowable fine in both situations—for a pattern of violations, as well as a repeat offense. FMCSA could define a pattern of serious violations in numerous ways that are consistent with the act’s pattern requirement. Our application of eight potential definitions shows that the number of carriers that would be subject to maximum fines depends greatly on the definition. (See table 7.) For example, a definition calling for two or more serious violations in each of at least four different regulatory areas during a compliance review would have made 38 carriers subject to maximum fines in fiscal year 2006. In contrast, a definition calling for one or more serious violations in each of at least three different regulatory areas would have made 1,529 carriers subject to maximum fines during that time. We also interpret the statutory language for the repeat requirement as calling for a “two strikes” rule as opposed to FMCSA’s three strikes rule. FMCSA’s interpretation imposes the maximum fine only after a carrier has twice previously committed a serious violation. The language of the statute does not allow FMCSA’s interpretation; rather it requires FMCSA to assess the maximum allowable fine for each serious violation against a carrier that has previously committed the same serious violation. In addition, in 2006, the Department of Transportation Inspector General found that FMCSA’s implementation of its three strikes rule had allowed many third strike violators to escape maximum fines. Specifically, of the 533 third strike violators of the hours of service or the drug and alcohol regulations between September 2000 and October 2004, 33 (6 percent) third strike violators were assessed the maximum fine. The Inspector General found that FMCSA did not consider many of these violators to be third strike violators because the agency, in keeping with its policy, did not count the carriers’ violations as strikes unless a violation resulted in the assessment of a fine. FMCSA does not always notify carriers of serious violations without fines and, therefore, FMCSA believes that counting such violations as strikes would violate the due process rights of carriers. The Inspector General agreed and recommended that FMCSA assess a no- dollar-amount fine or use another appropriate mechanism to legally notify a motor carrier of the violation and the policy that future violations will result in the maximum fine amount. An FMCSA official said that the agency is developing a policy designed to address this recommendation and plans to consider the related recommendation in this report as it develops the policy. FMCSA plans to implement the policy by June 2008. In fiscal years 2004 through 2006, there were more than four times as many carriers with a serious violation that constituted a second strike than there were carriers with a third strike. (See table 8.) For example, in fiscal year 2006, 1,320 carriers had a serious violation that constituted a second strike, whereas 280 carriers had a third strike. Carriers with a pattern of violations may also commit a second strike violation. For example, three of the seven carriers that had two or more serious violations in each of at least five different regulatory areas also had a second strike in fiscal year 2006. Were FMCSA to make policy changes along the lines discussed here, we believe that the new policies should address how to deal with carriers with serious violations that both are part of a pattern and repeat the same or similar previous violations. FMCSA’s policy for prioritizing carriers for compliance reviews based on their SafeStat scores furthers motor carrier safety because it targets many carriers that pose high crash risks and thus has value for reducing both the number and severity of motor carrier crashes. However, the policy does not always target the carriers that have the highest crash risks. Modifications to the policy that we identified could improve FMCSA’s targeting of high-risk carriers, thereby leading to compliance reviews that would have a greater potential to avoid crashes and their associated injuries and fatalities. Our June 2007 report found that a regression model approach would better identify carriers that pose high crash risks than does SafeStat, enabling FMCSA to better target its resources. We recommended in that report that FMCSA implement such an approach. However, if FMCSA does not implement this recommendation, the analysis presented in this report suggests an alternative approach that would also better target carriers that pose high crash risks. This approach would give high priority for compliance reviews to carriers with very poor scores (such as the worst 5 percent) in the accident safety evaluation area. While FMCSA follows up with most carriers with serious safety violations, it has not established a time frame for carriers rated conditional to receive a follow-up compliance review. As a result, many carriers with conditional ratings can continue to operate for 2 years or more without a follow-up compliance review, posing safety risks to themselves and the public. Finally, we found that FMCSA assesses maximum fines against carriers that twice repeat a serious violation. However, because of FMCSA’s interpretation of the statutory requirement to assess maximum fines against serious violators, many carriers that continue to accrue serious violations do not have the maximum fine assessed against them. Therefore, neither the statutory requirement nor FMCSA’s enforcement is as effective as possible in deterring unsafe practices and, as a result, additional accidents could occur. In our June 2007 report on the effectiveness of SafeStat, we recommended that FMCSA use a regression model approach to identify carriers that pose high crash risks rather than its expert judgment approach. Should the Secretary of Transportation decide not to implement that recommendation, we recommend that the Secretary of Transportation direct the FMCSA Administrator to take the following action: to improve FMCSA’s targeting of carriers that pose high crash risks, modify FMCSA’s policy for prioritizing compliance reviews so that carriers with very poor scores (such as the worst 5 percent) in the accident safety evaluation area will be selected for compliance reviews, regardless of their scores in the other areas. We also recommend that the Secretary of Transportation direct the FMCSA Administrator to take the following two actions: to help ensure that carriers rated conditional make safety improvements in a timely manner, establish a reasonable time frame within which FMCSA should conduct follow-up compliance reviews on such carriers and to meet the Motor Carrier Safety Improvement Act’s requirement to assess maximum fines and improve the deterrent effect of these fines, revise FMCSA’s related policy to include (1) a definition for a pattern of violations that is distinct from the repetition of the same or related violations and (2) a two strikes rule rather than a three strikes rule. We provided a draft of this report to the Department of Transportation for its review and comment. The department did not offer overall comments on the draft report. It said that it would assess the efficacy of the first recommendation, but it did not comment on the other recommendations. It offered several technical comments, which we incorporated where appropriate. As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies of this report to congressional committees and subcommittees with responsibilities for commercial motor vehicle safety issues; the Secretary of Transportation; the Administrator, FMCSA; and the Director, Office of Management and Budget. We also will make copies available to others upon request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov. If you have any questions about this report, please contact me at (202) 512-2834 or flemings@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Staff who made key contributions to this report are listed in appendix V. Several studies by the Volpe National Transportation Systems Center (Volpe), the Department of Transportation Office of Inspector General, the Oak Ridge National Laboratory (Oak Ridge), and others have assessed the predictive capability of the Motor Carrier Safety Status Measurement System (SafeStat) model and the data used by that model. In general, studies that assessed the predictive power of SafeStat offered suggestions to increase that power, and studies that assessed data quality found weaknesses in the data that the Federal Motor Carrier Safety Administration (FMCSA) relies upon. The studies we reviewed compared SafeStat with random selection to determine which does a better job of selecting carriers that pose high crash risks and assessed whether statistical approaches could improve that selection and whether carrier financial positions or driver convictions are associated with crash risk. In its 2004 and 1998 studies of the SafeStat model, Volpe analyzed retrospective data to determine how many crashes carriers in SafeStat categories A and B experienced over the following 18 months. The 2004 study used the carrier rankings from an application of the SafeStat model on March 21, 2001. Volpe then compared the SafeStat carrier safety ratings with state-reported data on crashes that occurred between March 22, 2001, and September 21, 2002, to assess the model’s performance. For each carrier, Volpe calculated a total number of crashes, weighted for time and severity, and then estimated a rate per 1,000 vehicles for comparing carriers in SafeStat categories A and B with the carriers in other SafeStat categories. The 1998 Volpe study used a similar methodology. Each study used a constrained subset of carriers rather than the full list contained in the Motor Carrier Management Information System (MCMIS). Both studies found that the crash rate for the carriers in SafeStat categories A and B was substantially higher than for the other carriers during the 18 months after the particular SafeStat run. On the basis of this finding, Volpe concluded that the SafeStat model worked. In response to a recommendation by the Department of Transportation Office of Inspector General, FMCSA contracted with Oak Ridge to independently review the SafeStat model. Oak Ridge assessed the SafeStat model’s performance and used the same data set (for March 21, 2001) provided by Volpe, which Volpe had used in its 2004 evaluation. Perhaps not surprisingly, Oak Ridge obtained a similar result for the weighted crash rate of carriers in SafeStat categories A and B over the 18-month follow-up period. Like the Volpe studies, the Oak Ridge study was constrained because it was based on a limited data set rather than the entire MCMIS data set. While SafeStat does better than simple random selection in identifying carriers that pose high crash risks, other methods can also be used. Oak Ridge extended Volpe’s analysis by applying regression models to identify carriers that pose high crash risks. Specifically, Oak Ridge applied a Poisson regression model and a negative binomial model using the safety evaluation area scores as independent variables to a weighted count of crashes that occurred in the 30 months before March 21, 2001. In addition, Oak Ridge applied the empirical Bayes method to the negative binomial regression model and assessed the variability of carrier crash counts by estimating confidence intervals. Oak Ridge found that the negative binomial model worked well at identifying carriers that pose high crash risks. However, the data set Oak Ridge had to use did not include any carriers with one reported crash in the 30 months before March 21, 2001. Because the data included only carriers with zero or two or more reported crashes, the distribution of crashes was truncated. Since the Oak Ridge regression model analysis did not cover carriers with safety evaluation area data and one reported crash, the findings from the study are limited in their generalizeability. However, other modeling analyses of crashes at intersections and on road segments have also found that the negative binomial regression model works well. In addition, our analysis, using a more recent and more comprehensive data set, supports the finding that the negative binomial regression model performs better than the SafeStat model. The studies carried out by other authors advocate the use of the empirical Bayes method in conjunction with a negative binomial regression model to estimate crash risk. Oak Ridge also applied this model to identify motor carriers that pose high crash risks. We applied this method to the 2004 SafeStat data and found that the empirical Bayes method best identified the carriers with the largest number of crashes in the 18 months after June 25, 2004. However, the crash rate per 1,000 vehicles was much lower than that for carriers in SafeStat categories A and B. We analyzed this result further and found that although the empirical Bayes method best identifies future crashes, it is not as effective as the SafeStat model or the negative binomial regression model in identifying carriers with the highest future crash rates. The carriers identified with the empirical Bayes method were invariably the largest carriers. This result is not especially useful from a regulatory perspective. Companies operating a large number of vehicles often have more crashes over a period of time than smaller companies. However, this does not mean that the larger company is necessarily violating more safety regulations or is less safe than the smaller company. For this reason, we do not advocate the use of the empirical Bayes method in conjunction with the negative binomial regression model as long as the method used to calculate the safety evaluation area values remains unchanged. If changes are made in how carriers are rated for safety, this method may in the future offer more promise than the negative binomial regression model alone. The quality of crash data is a long-standing problem that hinders FMCSA’s ability to accurately identify carriers that pose high crash risks. Despite the problems of late-reported crashes and incomplete and inaccurate data on crashes, the data were of sufficient quality for our use, which was to assess whether different approaches to categorizing carriers could lead to better identification of carriers that subsequently have high crash rates. Our reasoning is based on our use of the same data set to compare the crash risk of carriers in SafeStat categories A or B and of carriers that score among the worst 25, 10, or 5 percent in an individual safety evaluation area. Limitations in the data would apply equally to both results. FMCSA has undertaken a number of efforts to improve crash data quality. FMCSA’s guidance requires states to report all crashes to MCMIS within 90 days of their occurrence. Late reporting can cause SafeStat to miss some of the carriers that should have received a SafeStat score. Moreover, since SafeStat scoring involves a relative ranking of carriers, a carrier may receive a SafeStat score and have to undergo a compliance review because crash data for a higher risk carrier were reported late and not included in the calculation. Late reporting affected SafeStat’s ability to identify all high-risk carriers to a small degree—missing about 6 percent-—for the period that we studied. Late reporting of crashes by states also affected the safety rankings of more than 600 carriers, both positively and negatively. When SafeStat analyzed the 2004 data, which did not include the late-reported crashes, it identified 4,989 motor carriers as highest risk, meaning they received a category A or B ranking. With the addition of late-reported crashes, 481 carriers moved into the highest risk category, and 182 carriers dropped out of the highest risk category, resulting in a net increase of 299 carriers (6 percent) in the highest risk category. After the late-reported crashes were added, 481 carriers that originally received a category C, D, E, F, or G SafeStat rating received an A or B rating. These carriers would not originally have been given a high priority for a compliance review because the SafeStat calculation did not take into account all of their crashes. On the other hand, a number of carriers would have fared better if the late- reported crashes had been included in their score. Specifically, 182 carriers—or fewer than 4 percent of those ranked—fell from the A or B category into the C, D, E, F, or G category once the late-reported crashes were included. These carriers would have avoided a compliance review if all crashes had been reported on time. Overall, however, the vast majority of carriers (96 percent) were not negatively affected by late reporting. The timeliness of crash reporting seems to be improving. The median number of days it took states to report crashes to MCMIS dropped from 225 days in calendar year 2001 to 57 days in 2005 (the latest data available at the time of our analysis). In addition, the percentage of crashes reported by states within 90 days of occurrence has jumped from 32 percent in fiscal year 2000 to 89 percent in fiscal year 2006. (See fig. 3.) FMCSA uses a motor carrier identification number, which is unique to each carrier, as the primary means of linking inspections, crashes, and compliance reviews to motor carriers. Approximately 184,000 (75 percent) of the 244,000 crashes reported to MCMIS between December 2001 and June 2004 involved interstate carriers. Of these 184,000 crashes, nearly 24,000 (13 percent) were missing this identification number. As a result, FMCSA could not match these crashes to motor carriers or use data from them in SafeStat. In addition, the carrier identification number could not be matched to one listed in MCMIS for 15,000 (8 percent) other crashes that involved interstate carriers. Missing data or data that cannot be matched to carriers for nearly one quarter of the crashes for the period of our review potentially have a large impact on a motor carrier’s SafeStat score because SafeStat treats crashes as the most important source of information for assessing motor carrier crash risk. Theoretically, information exists to match crash records to motor carriers by other means, but such matching would require too much manual work to be practicable. We were not able to quantify the actual effect of the missing data and the data that could not be matched for MCMIS overall. To do so, we would have had to gather crash records at the state level—an effort that was impractical. For the same reason, we cannot quantify the effects of FMCSA’s efforts to improve the completeness of the data (discussed later). However, the University of Michigan Transportation Research Institute issued a series of reports analyzing the completeness of the data submitted to MCMIS by the states. One of the goals of the research was to determine the states’ crash reporting rates. Reporting rates varied greatly among the 14 states studied, ranging from 9 percent in New Mexico in 2003 to 83 percent in Missouri in 2005. It is not possible to draw wide-scale conclusions about whether states’ reporting rates are improving over time because only 2 of the states—Missouri and Ohio—were studied in multiple years. However, the reporting rates of these 2 states did improve. Missouri experienced a large improvement in its reporting rate, with 61 percent of eligible crashes reported in 2001, and 83 percent reported in 2005. Ohio’s improvement was more modest, increasing from 39 percent in 2000 to 43 percent in 2005. The University of Michigan Transportation Research Institute’s reports also identified a number of factors that may affect states’ reporting rates. One of the main factors affecting reporting rates is the reporting officer’s understanding of crash reporting requirements. The studies note that reporting rates are generally lower for less serious crashes and for crashes involving smaller vehicles, which may indicate that there is some confusion about which crashes are reportable. Some states, such as Missouri, aid the officer by explicitly listing reporting criteria on the police accident reporting form, while other states, such as Washington, leave it up to the officer to complete certain sections of the form if the crash is reportable, but the form includes no guidance on reportable crashes. Other states, such as North Carolina and Illinois, have taken this task out of officers’ hands and include all reporting elements on the police accident reporting form. Reportable crashes are then selected centrally by the state, and the required data are transmitted to MCMIS. Inaccurate data, such as information on nonqualifying crashes reported to FMCSA, potentially have a large impact on a motor carrier’s SafeStat score because SafeStat treats crashes as the most important source of information for assessing motor carrier crash risk. The University of Michigan Transportation Research Institute’s reports on state crash reporting show that, among the 14 states studied, incorrect reporting of crash data is widespread. This inaccuracy limits SafeStat’s ability to identify carriers that pose high crash risks. In the most recent reports, the researchers found that, in 2005, Ohio incorrectly reported 1,094 (22 percent) of the 5,037 cases it reported, and Louisiana incorrectly reported 137 (5 percent) of the 2,699 cases it reported. In Ohio, most of the incorrectly reported crashes did not qualify because they did not meet the crash severity threshold. In contrast, most of the incorrectly reported crashes in Louisiana did not qualify because they did not involve vehicles eligible for reporting. Other states studied by the institute had similar problems with reporting crashes that did not meet the criteria for reporting to MCMIS. The addition of these nonqualifying crashes could cause some carriers to exceed the minimum number of crashes required to receive a SafeStat rating and result in SafeStat’s mistakenly identifying carriers as posing high crash risks. Because each report focuses on reporting in one state in a particular year, it is not possible to identify the number of cases that have been incorrectly reported nationwide and, therefore, it is not possible to determine the impact of inaccurate reporting on SafeStat’s calculations. We also found examples of crashes that are reported to MCMIS but cannot be used by SafeStat because of data errors. Specifically, we found that the carrier’s identification number cannot be matched to an identification number in MCMIS in 8 percent of reported crashes. FMCSA cannot link these crashes to specific carriers without an accurate identification number and, therefore, cannot use these crashes in the SafeStat model to identify carriers that pose high crash risks. As noted in the University of Michigan Transportation Research Institute’s reports, states may be unintentionally submitting incorrect data to MCMIS because of difficulties in determining whether a crash meets the reporting criteria. For example, in Missouri, pickups are systematically excluded from MCMIS crash reporting, which may cause the state to miss some reportable crashes. This may occur because, in recent years, a number of pickups have been equipped with rear axles that may increase their weight above the reporting threshold and make crashes involving them eligible for reporting. There is no way for the state to determine which crashes involving pickups qualify for reporting without examining the characteristics of each vehicle. In this case, the number of omissions is likely to be relatively small, but this example demonstrates the difficulty states may face when identifying reportable crashes. In addition, in some states, the information contained in the police accident report may not be sufficient for the state to determine if a crash meets the accident severity threshold. It is generally straightforward to determine whether a fatality occurred as a result of a crash, but it may be difficult to determine whether an injured person was transported for medical attention or a vehicle was towed because of disabling damage. In some states, such as Illinois and New Jersey, an officer can indicate on the form if a vehicle was towed by checking a box, but there is no way to identify whether the reason for towing was disabling damage. It is likely that such uncertainty results in overreporting because some vehicles may be towed for other reasons. FMCSA has taken steps to try and improve the quality of crash data reporting. As we noted in November 2005, FMCSA has undertaken two major efforts to help states improve the quality of crash data. One program, the Safety Data Improvement Program, has provided funding to states to implement or expand activities designed to improve the completeness, timeliness, accuracy, and consistency of their data. FMCSA has also used a data quality rating system to rate and display ratings for the quality of states’ crash and inspection data. Because these ratings are public, this system creates an incentive for states to improve their data quality. To further improve these programs, FMCSA has awarded additional grants to several states and implemented our recommendations to (1) establish specific guidelines for assessing states’ requests for funding to support data improvement in order to better assess and prioritize the requests and (2) increase the usefulness of its state data quality map as a tool for monitoring and measuring commercial motor vehicle crash data by ensuring that the map adequately reflects the condition of the states’ commercial motor vehicle crash data. In February 2004, FMCSA implemented Data Q’s, an online system that allows for challenging and correcting erroneous crash or inspection data. Users of this system include motor carriers, the general public, state officials, and FMCSA. In addition, in response to a recent recommendation by the Department of Transportation Inspector General, FMCSA is planning to conduct a number of evaluations of the effectiveness of a training course on crash data collection that it will be providing to states by September 2008. While the quality of crash data is sufficient for use in assessing whether different approaches to categorizing carriers could lead to better identification of carriers that subsequently have high crash rates and has started to improve, commercial motor vehicle crash data continue to have some problems with timeliness, completeness, and accuracy. These problems have been well-documented in several studies, and FMCSA is taking steps to address the problems through studies of each state’s crash reporting system and grants to states to fund improvements. As a result, we are not making any recommendations in this area. Several studies have identified relationships between certain characteristics of motor carriers and drivers and their crash risks. Theses characteristics include carrier financial performance, carrier size, driver pay, and driver age. The studies we reviewed assessed whether financial performance or other characteristics of carriers, such as size, are associated with crash risk. Our 1991 study developed a model that linked changes in economic conditions to declining safety performance in the trucking industry. The study hypothesized that a decline in economic performance among motor carriers leads to a decline in safety performance in one or more of the following ways: (1) a lowering of the average quality of driver performance; (2) downward wage pressures encouraging driver noncompliance with safety regulations; (3) less management emphasis on safety practices; (4) deferred truck maintenance and replacement; and/or (5) the introduction of larger, heavier, multitrailer trucks. Using data on 537 carriers drawn from the Department of Transportation and the Interstate Commerce Commission, we found that seven financial ratios show promise as predictors of truck firms’ safety. For five of the seven financial variables we examined, firms in the weakest financial position had the highest subsequent accident rates. For example, weakness in any of three measures of profitability—return on equity, operating ratio, and net profit margin—was associated with subsequent safety problems as measured by accident rates. On behalf of FMCSA, a study carried out by Corsi, Barnard, and Gibney in 2002 examined how data on carriers’ financial performance correlate with a carrier’s safety rating following a compliance review. The authors selected motor carriers from MCMIS in December 2000 with complete data for the accident, driver, vehicle, and safety management safety evaluation areas. Using these data, the authors then matched a total of 700 carriers to company financial statements in the annual report database of the American Trucking Associations. The authors found that carriers that received satisfactory ratings following a compliance review performed better on two financial measures—operating ratio and return on assets— than carriers that received lower ratings. Two practical considerations limit the applicability of the findings from these two studies to SafeStat. First, the studies’ samples of 537 and 700 carriers, respectively, are not representative of the motor carriers that FMCSA oversees. For example, our sample included only the largest for- hire interstate carriers because these were the only carriers that were required to report financial information to the federal government. The carriers selected for the Corsi and others’ study were also not representative because a very small percentage of the carriers evaluated by the SafeStat model in June 2004 had scores for all four safety evaluation areas. About 2 percent had a score for the the safety management safety evaluation area, and of these, not all had complete data for the other three safety evaluation areas. Second, FMCSA does not receive annual financial statements from carriers and, according to an FMCSA official, it is unlikely that the agency could obtain the authority it would need to require financial statements from all carriers. In addition, because the relationships identified by our study are based on data and economic conditions that are almost 20 years old, the relationships would need to be reanalyzed within current conditions to determine whether they still exist. As part of its Comprehensive Safety Analysis 2010 reform initiative, discussed earlier in this report, FMCSA decided not to use financial data to help assess the safety risk of firms because of the limited availability of these data. A 1994 study by Moses and Savage found that crash rates decline as firm size increases; the largest 10 percent of firms have an accident rate that is one-third the rate of the smallest 10 percent of firms. Our 1991 study found that the smallest carriers, as a group, had an accident rate that exceeded the rate for all firms by 20 percent. The study by Moses and Savage also found that (1) private fleets that serve the needs of their parent companies, such as manufacturers and retailers, have accident rates that are about 20 percent lower than the rates of carriers that offer for-hire trucking; (2) carriers of hazardous materials have accident rates that are 22 percent higher than the rates of carriers that do not transport these goods; and (3) general freight carriers have accident rates that are 10 percent higher than the rates of other freight carriers. We believe that Moses and Savage’s findings are reasonable given their study’s design, data, and methodology, but because the findings are based on data and economic conditions that are about 15 to 20 years old, current data would need to be reanalyzed within current conditions to determine whether the findings are still valid. As mentioned above, our study shares this limitation and is further limited by an unrepresentative sample of motor carriers. An FMCSA official told us that the agency would not want to rely directly on data on the size of the carrier to assess safety risk because the agency believes that its data on indicators of carrier size, such as revenue, number of drivers, and number of power units, are not of sufficient quality. Similarly, the agency would not want to distinguish between private and for-hire carriers or between carriers that carry different types of freight because it does not believe that its data are sufficiently reliable. The studies we reviewed assessed whether driver characteristics— including convictions for traffic violations, age and experience, pay, or frequency of job changes—are associated with crash risk. A series of studies by Lantz and others examined the effect of incorporating conviction data from the state-run commercial driver license data system into the calculation of carriers’ safety management safety evaluation area scores. The studies found that the resulting driver conviction measure is weakly correlated with the crash-per-vehicle rate. However, the studies did not calculate new safety management safety evaluation area scores with the proposed driver conviction measure and then use the updated measure to estimate new SafeStat scores for carriers. FMCSA uses data on driver convictions to help target its roadside inspections, and it is considering using such data in the tool it is developing to replace SafeStat as part of its Comprehensive Safety Analysis 2010 reform initiative. Campbell’s 1991 study found that the risk of a fatal crash is significantly higher for younger truck drivers than for older drivers. Campbell used data from surveys of fatal crashes and large truck travel to calculate fatal involvement rates per mile driven by driver age. Overall, fatal involvement rates remained high through age 26. The fatal crash rates for drivers under 19 years of age were four times higher than the rate for all drivers, and the rates for drivers aged 19 to 20 years were six times higher. Our 1991 study found that younger, less experienced drivers posed greater-than-average accident risks. In particular, compared with drivers 40 to 49 years of age, drivers 21 to 39 years of age have 28 percent greater odds of accident involvement. Compared with those for drivers over 50 years of age, the odds of the youngest group of drivers having an accident are about 60 percent greater. The differences in accident risks between drivers with 0 to 13 years of experience, 14 to 20 years of experience, and 21 or more years of experience followed a very similar pattern. Although Campbell’s study provides only limited information about the quality of the data it used, we believe that its findings are reasonable given the study’s design and methodology, which relied on multiple kinds of analyses to substantiate a higher risk for younger drivers of large trucks. We believe that our 1991 findings are reasonable given our study’s design, data, and methodology. An FMCSA official told us that, at this time, the agency would not be able to use driver age in SafeStat or in a similar model because the agency does not have access to data on all drivers. FMCSA said that it is exploring the possibility of gaining broader access to data on drivers, which are maintained by the states, so that the agency can use the data to help assess the safety of drivers as part of its Comprehensive Safety Analysis 2010 reform initiative. Belzer and others’ 2002 study found that drivers with lower pay had higher crash rates. Because economic theory predicts that low pay levels are associated with poorer performing workers, the study hypothesized that low pay levels for drivers are associated with unsafe driving. The study found that for every 10 percent more in average driver compensation (mileage rate, unpaid time, anticipated annual raise, safety bonus, health insurance, and life insurance), the carriers experienced 9.2 percent fewer crashes. We believe that this finding is reasonable given the study’s design, data, and methodology. An FMCSA official told us that the agency could not use data on driver pay in SafeStat or in a similar model because such data are available only from studies or surveys that do not cover the full population of drivers. Staplin and others’ 2003 study for FMCSA found that drivers that average three or more jobs with different carriers each year have crash rates that are more than twice as high as drivers that average fewer job changes. Although the study authors acknowledge several limitations in the data used in study, we believe that the data and the analysis approach were sufficiently reliable to support the study’s finding of a relationship between the number of jobs and the number of crashes. An FMCSA official told us that, as for data on driver pay, the agency could not use data on the frequency of job changes in SafeStat or in a similar model because such data are available only from studies or surveys that do not cover the full population of drivers. To determine the extent to which FMCSA’s policy for prioritizing compliance reviews targets carriers that subsequently have high crash rates, we analyzed data from FMCSA’s MCMIS on the June 2004 SafeStat assessment of carriers and on the assessed carriers’ crashes in the 18 months following the SafeStat assessment. We selected June 2004 because this date enabled us to examine MCMIS data on actual crashes that occurred in the 18-month period from July 2004 through December 2005. We defined various groups of carriers for analysis, such as those in each SafeStat category, those to which FMCSA gave high priority (i.e., those in categories A or B), and those in the worst 5 or 10 percent of carriers in a particular safety evaluation area without being in the worst 25 percent of carriers in any other area. We then calculated the aggregate crash rate in the 18 months following the SafeStat assessment for each of these groups by dividing the total crashes experienced by all the carriers in a group during that time period by the total number of vehicles operated by those carriers, as reported on their motor carrier census form. We then compared crash rates among the various groups to determine whether there were any groups with substantially higher aggregate crash rates than the carriers in SafeStat categories A or B. We also talked to FMCSA officials about how FMCSA developed SafeStat, their views on other evaluations of SafeStat, and FMCSA’s plans to replace SafeStat with a new tool. In assessing how FMCSA ensures that its compliance reviews are completed thoroughly and consistently, we reviewed our report on internal control standards for the federal government. We identified key standards in the areas that we believe are critical to maintaining the thoroughness and consistency of compliance reviews, namely the recording and communication of policy to management and others, the clear documentation of processes, and the monitoring and reviewing of activities and findings. We assessed the extent to which FMCSA’s management of its compliance reviews is consistent with these internal control standards by interviewing FMCSA and state managers and investigators. We interviewed investigators who conduct compliance reviews and their managers in FMCSA’s headquarters office, as well as in 7 of FMCSA’s 52 field division offices that work with states, two of its four regional service centers that support division offices, and three state offices that partner with 3 of the FMCSA division offices in which we did our work. We also interviewed two safety investigators in each of the same 7 division offices. The division offices and states that we reviewed— California, Georgia, Illinois, New York, Ohio, Pennsylvania, and Texas— received 30 percent of all the of the grant funds that FMCSA awarded to the states in fiscal year 2005 (the latest year for which data were available) through its primary grant program, the Motor Carrier Safety Assistance Program. Because we chose the seven states judgmentally (representing the largest grantees), we cannot project our findings nationwide.Reviewing a larger number of grantees would not have been practical because of resource constraints. We gathered information on the recording and communication of policy from discussions with FMCSA officials, documents, and system software, including the electronic operations manual. We obtained information about how FMCSA documents the findings of compliance reviews through discussions with FMCSA officials and reviews of FMCSA documents. We obtained information on how FMCSA monitors and reviews the performance of its compliance reviews through discussions with FMCSA officials and reviews of FMCSA documents, including the 2002 report of FMCSA’s Compliance Review Work Group. The data assessments of the number of vehicles inspected during compliance reviews and the percentage of applicable areas of the regulations covered by compliance reviews since 2001 were provided to us by FMCSA. In assessing the extent to which FMCSA follows up with carriers with serious violations, we reviewed regulations directing how FMCSA should follow up and track these violators and analyzed data to determine if FMCSA had met these policies. Particularly, we examined FMCSA policies and discussed with FMCSA officials the agency’s policy to perform a follow-up compliance review on carriers in SafeStat categories A and B, its policy to place carriers rated unsatisfactory out of service, its policy to perform a follow-up compliance review on carriers with a conditional rating, and its reduction of its enforcement backlog. Additional analysis was performed—as of the end of each fiscal year from 2001 through 2006—using data from FMCSA’s MCMIS to determine the total number of carriers with a conditional rating that had not received a follow-up compliance review. We also used MCMIS to determine how many carriers with a conditional rating received a follow-up compliance review and how soon after the original compliance review the second review occurred. To assess FMCSA’s implementation of the statutory requirement to assess the maximum fine against any carrier with either a pattern of violations or previously committed violations, we compared FMCSA’s policy with the language of the act and held discussions with FMCSA officials. In addition, we assessed the number of carriers that would have been assessed the maximum fine under differing definitions of a pattern of violations. We also reviewed the report of the Department of Transportation Inspector General on the implementation of the policy and documents pertaining to FMCSA’s response to the Inspector General’s report. In determining the reliability of FMCSA’s data on compliance reviews, violations, and enforcement cases, we performed electronic testing for obvious errors in accuracy and completeness. As part of a recent evaluation of FMCSA’s enforcement programs, we interviewed officials from FMCSA’s data analysis office who are knowledgeable about the same data sources. We determined that the data were sufficiently reliable for the types of analysis we present in this report. To assess the extent to which the timeliness, completeness, and accuracy of MCMIS and state-reported crash data affect SafeStat’s performance, we carried out a series of analyses with the MCMIS master crash file, and the MCMIS census file, as well as surveying the literature to assess other studies’ findings on the quality of MCMIS data. To assess timeliness, we first measured how many days on average it was taking each state to report crashes to FMCSA by year for calendar years 2000 through 2005. We also recalculated SafeStat scores from June 25, 2004, to include crashes that had occurred more than 90 days previously but had not yet been reported to FMCSA by that date. We compared the number and rankings of carriers from the original SafeStat results with those obtained with the addition of late-reported crashes. In addition, we reviewed the University of Michigan Transportation Research Institute’s studies of state crash reporting to MCMIS to identify the impact of late reporting in individual states on MCMIS data quality. To assess completeness, we attempted to match all crash records in the MCMIS master crash file for crashes occurring between December 2001 and June 2004 to the list of motor carriers in the MCMIS census file. We used a variety of matching techniques to try and match the crash records without a carrier Department of Transportation number to carriers listed in the MCMIS census file. In addition, we reviewed the University of Michigan Transportation Research Institute’s studies of state crash reporting to MCMIS to identify the impact of incomplete crash reporting in individual states on MCMIS data quality. To assess accuracy, we reviewed an audit by the Inspector General that tested the accuracy of electronic data by comparing records selected in the sample to source paper documents. In addition, we reviewed the University of Michigan Transportation Research Institute’s studies of state crash reporting to MCMIS to identify the impact of incorrectly reported crashes in individual states on MCMIS data quality. We determined that the data reported to FMCSA for use in SafeStat— while not as timely, complete, or accurate as they could be—were of sufficient quality for our use. Through our analyses, we found that the data identify many carriers that pose high crash risks and are, therefore, useful for the purposes of this report. To understand what other researchers have found about how well SafeStat identifies motor carriers that pose high crash risks, we identified studies through a general literature review and by asking stakeholders and study authors to identify high-quality studies. The studies included in our review were (1) the 2004 study of SafeStat done by Oak Ridge National Laboratory, (2) the SafeStat effectiveness studies done by the Department of Transportation Inspector General and Volpe Institute, (3) the University of Michigan Transportation Research Institute’s studies of state crash reporting to FMCSA, and (4) the 2006 audit by the Department of Transportation Inspector General of data for new entrant carriers. We assessed the methodology used in each study and identified which findings are supported by rigorous analysis. We accomplished this analysis by relying on information presented in the studies and, where possible, discussing the studies with the authors. When the studies’ methodologies and analyses appeared reasonable, we used the findings from those studies in our analysis of SafeStat. We discussed with FMCSA and industry and safety stakeholders the SafeStat methodology issues and data quality issues raised by these studies. We also discussed the aptness of the respective methodological approaches with FMCSA. Finally, we reviewed FMCSA documentation on how SafeStat is constructed and assessments of SafeStat conducted by FMCSA. To identify studies on predictors of motor carrier and driver crash risk, we conducted a general literature review. We shared this preliminary list of studies with the members of the Transportation Research Board’s Committee on Truck and Bus Safety and requested them to identify additional relevant studies. We selected those studies that assessed a relationship between one or more motor carrier or driver characteristics and crash risk. Based on information presented in the selected studies, we assessed the methodology used in each study and report only those findings that were based on sound methodology and analysis. In addition to the individual named above, James Ratzenberger, Assistant Director; Carl Barden; Elizabeth Eisenstadt; David Goldstein; Ryan Gottschall; Laurie Hamilton; Eric Hudson; Bert Japikse; and Gregory Wilmoth made key contributions to this report.

Summary: The Federal Motor Carrier Safety Administration (FMCSA) has the primary federal responsibility for reducing crashes involving large trucks and buses. FMCSA uses its "SafeStat" tool to target carriers for reviews of their compliance with the agency's safety regulations based on their crash rates and safety violations. As requested, this study reports on (1) the extent to which FMCSA's policy for prioritizing compliance reviews targets carriers with a high risk of crashes, (2) how FMCSA ensures compliance reviews are thorough and consistent, and (3) the extent to which FMCSA follows up with carriers with serious safety violations. To complete this work, GAO reviewed FMCSA's regulations, policies, and safety data and contacted FMCSA officials in headquarters and nine field offices. By and large, FMCSA does a good job of identifying carriers that pose high crash risks for subsequent compliance reviews, ensuring the thoroughness and consistency of those reviews, and following up with high-risk carriers. FMCSA's policy for prioritizing compliance reviews targets many high-risk carriers but not other higher risk ones. Carriers must score among the worst 25 percent of carriers in at least two of SafeStat's four evaluation areas (accident, driver, vehicle, and safety management) to receive high priority for a compliance review. Using data from 2004, GAO found that 492 carriers that performed very poorly in only the accident evaluation area (i.e., those carriers that scored among the worst 5 percent of carriers in this area) subsequently had an aggregate crash rate that was more than twice as high as that of the 4,989 carriers to which FMCSA gave high priority. FMCSA told GAO that the agency plans to assess whether giving high priority to carriers that perform very poorly in only the accident evaluation area would be an effective use of its resources. FMCSA promotes thoroughness and consistency in its compliance reviews through its management processes, which meet GAO's standards for internal controls. For example, FMCSA uses an electronic manual to record and communicate its compliance review policies and procedures and teaches proper compliance review procedures through both classroom and on-the-job training. Furthermore, its investigators use an information system to document their compliance reviews, and its managers review these data, helping to ensure thoroughness and consistency between investigators. For the most part, FMCSA and state investigators cover the nine major applicable areas of the safety regulations (e.g., driver qualifications and vehicle condition) in 95 percent or more of compliance reviews, demonstrating thoroughness and consistency. FMCSA follows up with many carriers with serious safety violations, but it does not assess maximum fines against all of the serious violators that GAO believes the law requires. FMCSA followed up with more than 99 percent of the 1,196 carriers that received proposed unsatisfactory safety ratings from compliance reviews completed in fiscal year 2005, finding that 881 of these carriers made safety improvements and placing 309 others out of service. However, GAO found that FMCSA (1) does not assess maximum fines against carriers with a pattern of varied serious violations as GAO believes the law requires and (2) assesses maximum fines against carriers for the third instance of a violation, whereas GAO reads the statute as requiring FMCSA to assess the maximum fine for the second.