Motor Carrier Safety: Preliminary Information on the Federal Motor Carrier Safety Administration's Efforts to Identify and Follow Up with High-risk Carriers
Released: 2007-07-11T00:00:00

Letter

Background

In the United States, commercial motor carriers account for less than 5 percent of all highway crashes, but these crashes result in about 13 percent of all highway deaths, or about 5,500 of the approximately 43,000 nationwide highway fatalities that occur annually. In addition, about 160,000 of the approximately 3.2 million highway injuries per year involve motor carriers. While the fatality rate for trucks has generally decreased over the past 30 years, it has been fairly stable since 2002. (See fig. 1.) The fatality rate for buses decreased slightly from 1975 to 2005, but it has more annual variability than the fatality rate for trucks due to a much smaller total vehicle miles traveled.

FMCSA’s primary mission is to reduce the number and severity of crashes involving large trucks and buses. FMCSA relies heavily on the results of compliance reviews to determine whether carriers are operating safely and, if not, to take enforcement action against them. FMCSA conducts these on-site reviews to determine carriers’ compliance with safety regulations that address areas such as alcohol and drug testing of drivers, driver qualifications, driver hours of service, vehicle maintenance and inspections, and transportation of hazardous materials.

FMCSA uses a data-driven analysis model called SafeStat to assess carriers’ risks relative to all other carriers based on safety indicators, such as their crash rates and safety violations identified during roadside inspections and prior compliance reviews. A carrier’s score is calculated based on its performance in four safety evaluation areas: accidents and driver, vehicle, and safety management violations. (See fig. 2.)

SafeStat Identifies Many High-risk Carriers, but Enhancements Could Identify Carriers with Even Higher Risks

SafeStat identifies many carriers that pose a high risk for crashes and is about twice as effective (83 percent) as randomly selecting carriers for compliance reviews. As a result, it has value for improving motor carrier safety. However, two enhancements that we analyzed could lead to FMCSA identifying carriers that pose greater crash risks overall. These approaches entail giving more weight to crashes than the current SafeStat model does. FMCSA has concerns about these approaches, in part, because placing more emphasis on accidents would require it to place less emphasis on other types of problems. FMCSA recognizes that SafeStat can be improved, and as part of its Comprehensive Safety Analysis 2010 reform initiative—which is aimed at improving its processes for identifying and dealing with unsafe carriers and drivers—the agency is considering replacing SafeStat by 2010.

Using Either a Statistical Approach or Modifying Existing SafeStat Categorization Rules Could Improve Identification of High-risk Carriers

In June 2007, we reported that FMCSA could improve SafeStat’s ability to identify carriers that pose high crash risks if it applied a statistical approach, called the negative binomial regression model, to the four SafeStat safety evaluation areas instead of its current approach. We used this approach to determine whether systematic analyses of data through regression modeling offered improved results in identifying carriers that pose high crash risks over FMCSA’s model, which uses expert judgment and professional experience to apply weights to each of the safety evaluation areas. The negative binomial model results in a rank order listing of carriers by crash risk and the predicted number of crashes. This differs from SafeStat’s current approach, which gives the highest priority to carriers that are deficient in three or more safety evaluation areas or that score over a certain amount—SafeStat categories A and B. (See table 1.)

The other enhancement that we analyzed—the results of which are preliminary—utilized the existing SafeStat overall design but examined the effect of providing greater priority to carriers that scored among the worst 5 percent of carriers in the accident safety evaluation area (SafeStat category D). We chose this approach because we found that while the driver, vehicle, and safety management evaluation areas are correlated with the future crash risk of a carrier, the accident evaluation area correlates most with future crash risk. This approach would retain the overall SafeStat framework and categorization—categories A through G for carriers with safety problems—but would substitute carriers in category D (the accident category) for carriers in categories A and B that have either (1) lower overall SafeStat scores or (2) lower accident area scores.

We compared the performance of our regression model approach and placing greater weight on carriers that scored among the worst 5 percent of carriers in SafeStat category D to the current SafeStat model. The comparison showed that both these approaches performed better than the current SafeStat approach. (See table 2.) For example, the regression model approach identified carriers with an average of 111 crashes per 1,000 vehicles over an 18-month period compared with the current SafeStat approach, which identified carriers for compliance reviews with an average of 102 crashes per 1,000 vehicles. This 9 percent improvement would have enabled FMCSA to identify carriers with almost twice as many crashes in the following 18 months as those carriers identified in its current approach (19,580 v. 10,076). Placing greater emphasis on carriers in category D provided superior results to the current SafeStat approach both in terms of identifying carriers with higher crash rates (from 6 to 9 percent higher) and greater numbers of crashes (from about 600 to 800 more). In addition, the regression approach performed at least as well as placing greater emphasis on carriers in category D in terms of identifying carriers with the highest crash rates and much better in identifying carriers with the greatest number of crashes.

Because both the approaches that we analyzed would identify a larger number of carriers that pose high crash risks, FMCSA would choose the number of carriers to review based on the resources available to it, much as it currently does.

We believe that our statistically based regression model is preferable to placing greater weight on carriers in category D because it provides for a systematic assessment of the relative contributions of accidents and driver, vehicle, and safety management violations. We recommended that FMCSA adopt such an approach. By its very nature the regression approach looks for the “best fit” in identifying the degree to which prior accidents and driver, vehicle, and safety management violations identify the likelihood of carriers having crashes in the future, compared to the current SafeStat approach, in which the relationship among the four evaluation areas is based on expert judgment. In addition, because the regression model could be run monthly—as is the current SafeStat model—any change in the degree to which accidents and driver, vehicle, and safety management violations better identify future crashes will be automatically considered as different weights to the four evaluation areas are assigned. This is not the case with the current SafeStat model, in which the evaluation area weights generally remain constant over time.

FMCSA agreed that use of a negative binomial regression model looks promising but officials said that the agency believes that placing more emphasis on the accident area would be counterproductive. First, FMCSA is concerned that this would require placing correspondingly less emphasis on the types of problems the compliance review is designed to address so that crashes can be reduced (i.e., the lack of compliance with safety regulations related to drivers, vehicles, and safety management that is captured in the other evaluation areas). Along this line, FMCSA said that compliance reviews of carriers in SafeStat category D have historically resulted in fewer serious violations than compliance reviews of carriers in SafeStat category A or B. We agree with FMCSA that the use of the approaches that we are discussing here today could tilt enforcement heavily toward carriers with high crash rates and away from carriers with compliance issues. We disagree, however, that this would be counterproductive. We found that while driver, vehicle, and safety management evaluation area scores are correlated with the future crash risk of a carrier, high crash rates are a stronger predictor of future crashes than poor compliance with safety regulations. FMCSA’s mission—as well as the ultimate purpose of compliance reviews—is to reduce the number and severity of truck and bus crashes.

Second, FMCSA officials said that placing more emphasis on the accident evaluation area would increase emphasis on the least reliable type of data used by SafeStat—crash data—and in so doing, it would increase the sensitivity of the results to crash data quality issues. However, in June 2007 we reported that FMCSA has made considerable efforts to improve the reliability of crash data. The report also concluded that as FMCSA continues its efforts to have states improve crash data, any sensitivity of results from our statistically based model to crash data quality issues should diminish.

FMCSA Is Considering Replacing SafeStat with a New Tool by 2010

As part of its Comprehensive Safety Analysis 2010, a reform initiative aimed at improving its processes for identifying and dealing with unsafe carriers and drivers, FMCSA is considering replacing SafeStat with a new tool by 2010. The new tool could take on greater importance in FMCSA’s safety oversight framework because the agency is considering using the tool’s assessments of carriers’ safety to determine whether carriers are fit to continue operating. In contrast, SafeStat is primarily used now to prioritize carriers for compliance reviews, and determinations of operational fitness are made only after compliance reviews are completed. FMCSA also plans to develop a tool to assess the safety status of individual drivers, along with tools for dealing with unsafe drivers. Even though FMCSA is considering replacing SafeStat, we believe that implementing either of the approaches discussed in this statement would be worthwhile because it would be relatively easy to do and result in immediate safety benefits that could save lives.

FMCSA’s Management of Its Compliance Reviews Promotes Thoroughness and Consistency

Our preliminary assessment is that FMCSA manages its compliance reviews in a way that meets our standards for internal control, thereby promoting thoroughness and consistency in the reviews. It does so by establishing compliance review policies and procedures through an electronic manual and training, using an information system to document the results of its compliance reviews, and monitoring performance. We also found that compliance reviews cover most of the major areas of the agency’s safety regulations.

FMCSA Communicates Its Compliance Review Policies and Procedures through an Electronic Manual and Training

FMCSA’s communication of its policies and procedures related to conducting compliance reviews meets our standards for internal control. These standards state that an organization’s policies and procedures should be recorded and communicated to management and others within the entity who need it and in a form (that is, for example, clearly written and provided as a paper or electronic manual) and within a time frame that enables them to carry out their responsibilities. FMCSA records and communicates its policies and procedures electronically through its Field Operations Training Manual, which it provides to all federal and state investigators and their managers. The manual includes guidance on how to prepare for a compliance review (for example, by reviewing information on the carrier’s accidents, drivers, and inspections), and it explains how this information can help the investigator focus the compliance review. It also specifies the minimum number of driver and vehicle maintenance records to be examined and the minimum number of vehicle inspections to be conducted during a compliance review. FMCSA posts updates to the manual that automatically download to investigators and managers when they connect to the Internet. In addition to the manual, FMCSA provides classroom training to investigators and requires that investigators successfully complete that training and examinations before they conduct a compliance review. According to FMCSA officials, investigators then receive on-the-job training, in which they accompany an experienced investigator during compliance reviews. Investigators can also take additional classroom training on specialized topics throughout their careers.

FMCSA Investigators Use an Information System to Document the Results of Compliance Reviews

FMCSA’s documentation of compliance reviews meets our standards for internal control. These standards state that all transactions and other significant events should be clearly and promptly documented, and the documentation should be readily available for examination. FMCSA and state investigators use an information system to document the results of their compliance reviews, including information on crashes and any violations of the safety regulations that they identify. This documentation is readily available to FMCSA managers, who told us that they review it to help ensure completeness and accuracy. FMCSA officials told us that the information system also helps ensure thoroughness and consistency by prompting investigators to follow FMCSA’s policies and procedures, such as requirements to meet a minimum sample size. The information system also includes checks for consistency and reasonableness and prompts investigators when the information they enter appears to be inaccurate. FMCSA said managers may assess an investigator’s thoroughness by comparing the rate of violations the investigator identified over the course of several compliance reviews to the average rate for investigators in their division office; a rate that is substantially below the average suggests insufficient thoroughness.

FMCSA Monitors the Performance of Its Compliance Reviews and Has Taken Actions to Address Identified Issues

FMCSA’s performance measurement and monitoring of its compliance review activities meet our standards for internal control. These standards state that managers should compare actual performance to planned or expected results and analyze significant differences. According to FMCSA and state managers and investigators, the managers review all compliance reviews in each division office and state to ensure thoroughness and consistency across investigators and across compliance reviews. The investigators we spoke with generally found these reviews to be helpful, and several investigators said that the reviews helped them learn policies and procedures and ultimately perform better compliance reviews.

In addition to assessing the performance of individual investigators, FMCSA periodically assesses the performance of FMCSA division offices and state agencies and conducted an agencywide review of its compliance review program in 2002. According to officials at one of FMCSA’s service centers, the service centers lead triennial reviews of the compliance review and enforcement activities of each division office and its state partner. These reviews assess whether the division offices and state partners are following FMCSA policies and procedures, and they include an assessment of performance data for items such as the number of compliance reviews conducted, rate of violations identified, and number of enforcement actions taken. The officials said that some reviews identify instances in which division offices have deviated from FMCSA’s compliance review policies but that only minor adjustments by the division offices are needed. The officials also said that the service centers compile best practices identified during the reviews and share these among the division offices and state partners.

FMCSA’s review also concluded that most investigators were not following FMCSA’s policy requiring them to perform vehicle inspections as part of a compliance review if the carrier had not already received the required number of roadside vehicle inspections. Since conducting its 2002 review, FMCSA changed its policy so that inspecting a minimum number of vehicles is no longer a strict requirement—if an investigator is unable to inspect the minimum number of vehicles, he or she must explain why in the compliance review report.

Each of the Major Applicable Areas of the Safety Regulations Is Consistently Covered by Most Compliance Reviews

From fiscal year 2001 through fiscal year 2006, each of the nine major applicable areas of the safety regulations was consistently covered by most of the approximately 76,000 compliance reviews conducted by FMCSA and the states. (See table 3.) For the most part, 95 percent or more of the compliance reviews covered each major applicable area in the agency’s safety regulations.

An FMCSA official told us that not every compliance review is required to cover these nine areas. For example, follow-up compliance reviews of carriers rated unsatisfactory or conditional are sometimes streamlined to cover only the one or a few areas of the regulations in which the carrier had violations. As another example, minimum insurance coverage regulations apply only to for-hire carriers and private carriers of hazardous materials; they do not apply to private passenger and nonhazardous materials carriers.

However, according to an FMCSA official, the area of these regulations that had the lowest rate of coverage—vehicle parts and accessories necessary for safe operation—is required for all compliance reviews except streamlined reviews. Vehicle inspections are supposed to be a key investigative technique for assessing compliance with this area, and an FMCSA official said that the lower rate of coverage for the parts and accessories area likely reflects the small number of vehicle inspections that FMCSA and the states conduct during compliance reviews.

FMCSA Follows Up with Many Carriers with Serious Safety Violations but Does Not Assess Maximum Fines for All of the Violations Required by Law

Our preliminary assessment is that FMCSA placed many carriers rated unsatisfactory in fiscal year 2005 out of service and followed up with nearly all of the rest to determine whether they had improved. In addition, FMCSA monitors carriers to identify those that are violating out-of-service orders. However, it does not take additional action against many violators of out-of-service orders that it identifies. Furthermore, FMCSA does not assess maximum fines against all carriers, as we believe the law requires, partly because FMCSA does not distinguish between carriers with a pattern of serious safety violations and those that repeat a serious violation.

FMCSA Followed Up with Almost All Carriers That Received a Proposed Safety Rating of Unsatisfactory

FMCSA followed up with at least 1,189 of 1,196 carriers (99 percent) that received a proposed safety rating of unsatisfactory following compliance reviews completed in fiscal year 2005. These follow-ups resulted in either upgraded safety ratings or the carriers being placed out of service. Specifically, Based on follow-up compliance reviews, FMCSA upgraded the final safety ratings of 658 carriers (325 to satisfactory and 333 to conditional).

FMCSA assigned a final rating of unsatisfactory to 309 carriers. FMCSA issued out-of-service orders to 306 of these carriers. An FMCSA official told us that it did not issue out-of-service orders to the remaining three carriers either because the agency could not locate them or because the carrier was still subject to an out-of-service order that FMCSA issued several years prior to the 2005 compliance review.

After FMCSA reviewed evidence of corrective action submitted by carriers, it upgraded the final safety ratings of 214 carriers (23 to satisfactory and 191 to conditional).

Due to an error in assigning the proposed safety rating to one carrier, FMCSA upgraded its final safety rating to conditional.

For the remaining 14 carriers, FMCSA did not (1) provide us information on whether and how it followed up with 7 carriers in time for us to incorporate it in this statement and (2) respond to our request to clarify its follow-up approach for another 7 carriers in time for us to incorporate it in this statement.

Under its policies, FMCSA is generally required to assign the carrier a final rating of unsatisfactory and to issue it an out-of-service order after either 45 or 60 days, depending on the nature of the carrier’s business. Of the about 300 out-of-service orders that FMCSA issued to carriers rated unsatisfactory following compliance reviews conducted in fiscal year 2005, FMCSA told us that 89 percent were issued on time, 9 percent were issued between 1 and 10 days late, and 2 percent were issued more than 10 days late. We are working with FMCSA to verify these numbers. An FMCSA official told us that in the few instances where an out-of-service order was issued more than 1 week late, the primary reason for the delay was that the responsible FMCSA division office had difficulty scheduling follow-up compliance reviews and thus held off on issuing the orders.

FMCSA Monitors Carriers to Identify Those That Are Violating Out-of-Service Orders, but It Does Not Take Additional Action against Many of the Violators It Identifies

FMCSA uses two primary means to try to ensure that carriers that have been placed out of service do not continue to operate. First, FMCSA partners with states to help them suspend, revoke, or deny vehicle registration to carriers that have been placed out of service. FMCSA refers to these partnerships as the Performance and Registration Information Systems Management program (PRISM). PRISM links FMCSA databases with state motor vehicle registration systems and roadside inspection personnel to help identify vehicles operated by carriers that have been issued out-of-service orders. As of January 2007, 45 states had been awarded PRISM grants and 27 states were operating with PRISM capabilities.

Second, FMCSA monitors carriers for indicators—such as roadside inspections, moving violations, and crashes—that they may be violating an out-of-service order and visits some of the suspect carriers to examine their records to determine whether they did indeed violate the order. FMCSA told us it is difficult to detect carriers operating in violation of out- of-service orders because its resources do not allow it to visit each carrier or conduct roadside inspections on all vehicles, and we agree. In fiscal years 2005 and 2006, 768 of 1,996 carriers (38 percent) that were subject to an out-of-service order had a roadside inspection or crash; FMCSA cited only 26 of these 768 carriers for violating an out-of-service order. An FMCSA official told us that some of these carriers, such as carriers that were operating intrastate or that had leased its vehicles to other carriers, may not have been violating the out-of-service order. He said that FMCSA did not have enough resources to determine whether each of the carriers was violating an out-of-service order.

FMCSA Conducted Compliance Reviews on About Half of All High-risk Carriers That It Was Required to By Statute

From August 2006 through February 2007, FMCSA data indicate that the agency performed compliance reviews on 1,136 of the 2,220 (51 percent) carriers that were covered by its mandatory compliance review policy. The Safe, Accountable, Flexible, Efficient Transportation Equity Act: A Legacy for Users act requires that FMCSA conduct compliance reviews on carriers rated as SafeStat category A or B for 2 consecutive months. In response to this requirement, FMCSA implemented a policy in June 2006 requiring a compliance review within 6 months for any such carrier unless the carrier had received a compliance review within the previous 12 months. An FMCSA official told us that the agency did not have enough resources to conduct compliance reviews on all of the 2,220 carriers within 6 months.

In April 2007, FMCSA revised the policy because it believes that it required compliance reviews for some carriers that did not need them, leaving FMCSA with insufficient resources to conduct compliance reviews on other carriers that did need them. Specifically, FMCSA believes that carriers that had already had a compliance review were targeted unnecessarily after they had corrected identified violations, but these violations continued to adversely affect their SafeStat rating because SafeStat penalizes carriers for violations regardless of whether they have been corrected. The new policy requires compliance reviews within 6 months for carriers that have been in SafeStat category A or B for 2 consecutive months and received their last compliance 2 or more years ago (or have never received a compliance review) and offers some discretion to FMCSA division offices. For example, division offices can decide not to conduct a compliance review if its SafeStat score is based largely on violations that have been corrected or on accidents that occurred prior to the carrier’s last compliance review. We believe that these changes are consistent with the act’s requirement and give FMCSA appropriate discretion in allocating its compliance review resources.

FMCSA Does Not Assess Maximum Fines for All the Violations Required by Law

FMCSA does not assess the maximum fines against all carriers as we believe the law requires. The law requires FMCSA to assess the maximum allowable fine for each serious violation by a carrier that is found (1) to have committed a pattern of such violations (pattern requirement) or (2) to have previously committed the same or a related serious violation (repeat requirement). However, FMCSA’s policy on maximum fines does not fully meet these requirements. FMCSA enforces both requirements using what is known as the “three-strikes rule,” applying the maximum allowable fine when it finds that a motor carrier has violated the same regulation three times within a 6-year period. FMCSA officials said they interpret both parts of the act’s requirements to refer to repeat violations, and because they believe that having two distinct policies on repeat violations would confuse motor carriers, it has chosen to address both requirements with its single three-strikes policy.

FMCSA’s interpretation does not carry out the statutory mandate to impose maximum fines in two different cases. In contrast to FMCSA, we read the statute’s use of the distinct terms “a pattern of violations” and “previously committed the same or a related violation” as requiring FMCSA to implement two distinct policies. A basic principle of statutory interpretation is that distinct terms should be read as having distinct meanings. In this case, the statute not only uses different language to refer to the violations for which maximum fines must be imposed but also sets them out separately and makes either type of violation subject to the maximum penalties. Therefore, one carrier may commit a variety of serious violations and another carrier may commit the same or a substantially similar serious violation as a previous violation; the language on its face requires FMCSA to assess the maximum allowable fine in both situations—patterns of violations as well as repeat offenses.

FMCSA could define a pattern of serious violations in numerous ways that are consistent with the act’s pattern requirement. Our assessment of eight potential definitions shows that the number of carriers that would be subject to maximum fines depends greatly on the definition. (See table 4.) For example, a definition calling for two or more serious violations in each of at least four different regulatory areas during a compliance review would have made 38 carriers subject to maximum fines in fiscal year 2006. In contrast, a definition calling for one or more serious violations in each of at least three different regulatory areas would have made 1,529 carriers subject to maximum fines during that time.

We also interpret the statutory language for the repeat requirement as calling for a “two-strikes” rule as opposed to FMCSA’s three-strikes rule interpretation. FMCSA’s interpretation imposes the maximum fine only after a carrier has twice previously committed such violations. The language of the statute does not allow FMCSA’s interpretation; rather, it requires FMCSA to assess the maximum allowable fine for each serious violation against a carrier that has previously committed the same serious violation.

In fiscal years 2004 through 2006, more than four times as many carriers had a serious violation that constituted a second strike than carriers that had a third strike. (See table 5.) For example, in fiscal year 2006, 1,320 carriers had a serious violation that constituted a second strike, whereas 280 carriers had a third strike.

Carriers that commit a pattern of violations may also commit a second strike violation. For example, three of the seven carriers that had two or more serious violations in each of at least five different regulatory areas also had a second strike in fiscal year 2006. Were FMCSA to make policy changes along the lines discussed here, we believe that the new policies should address how to deal with carriers with serious violations that both are part of a pattern and repeat the same or similar previous violations.

Mr. Chairman, this concludes my prepared statement. I would be pleased to respond to any questions that you or other Members of the Subcommittee might have.

GAO Contacts and Staff Acknowledgement

For further information on this statement, please contact Susan Fleming at (202) 512-2834 or flemings@gao.gov. Individuals making key contributions to this testimony were David Goldstein, Eric Hudson, and James Ratzenberger.

Related GAO Products

Motor Carrier Safety: A Statistical Approach Will Better Identify Commercial Carriers That Pose High Crash Risks Than Does the Current Federal Approach. GAO-07-585. Washington, D.C.: June 11, 2007.

Unified Motor Carrier Fee System: Progress Made but Challenges to Implementing New System Remain. GAO-07-771R. Washington, D.C.: May 25, 2007.

Consumer Protection: Some Improvements in Federal Oversight of Household Goods Moving Industry Since 2001, but More Action Needed to Better Protect Individual Consumers. GAO-07-586. Washington, D.C.: May 16, 2007.

Transportation Security: DHS Efforts to Eliminate Redundant Background Check Investigations. GAO-07-756. Washington, D.C.: April 26, 2007.

Truck Safety: Share the Road Safely Pilot Initiative Showed Promise, but the Program’s Future Success Is Uncertain. GAO-06-916. Washington, D.C.: September 8, 2006.

Federal Motor Carrier Safety Administration: Education and Outreach Programs Target Safety and Consumer Issues, but Gaps in Planning and Evaluation Remain. GAO-06-103. Washington, D.C.: December 19, 2005.

Large Truck Safety: Federal Enforcement Efforts Have Been Stronger Since 2000, but Oversight of State Grants Needs Improvement. GAO-06- 156. Washington, D.C.: December 15, 2005.

Highway Safety: Further Opportunities Exist to Improve Data on Crashes Involving Commercial Motor Vehicles. GAO-06-102. Washington, D.C.: November 18, 2005.

This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.