The commercial motor-carrier industry is a vital part of the U.S. economy. As of December 2013, FMCSA estimates that there were more than 539,000 active motor carriers and 5.6 million commercial drivers operating in the United States that transport freight and move people, numbers that fluctuate over time due to the tens of thousands of carriers that enter and leave the market annually. Truck carriers are responsible for transporting approximately 13 billion tons of cargo annually, representing 67 percent of all cargo weight that was shipped in 2012. The United States’ commercial motor-carrier industry represents a range of businesses, including private and for-hire freight transportation, passenger carriers, and specialized transporters of hazardous materials. These carriers can be small and involve a single vehicle that is owned and operated by a single individual, or large corporations that own tens of thousands of vehicles. Similarly, commercial motor-vehicle operations differ greatly, including whether a carrier operates locally, regionally, or across the country (i.e., “over-the-road”), as well as the time of day when a driver operates. In addition, some carriers move goods as part of a single shipment to one destination (called “truckload” operations), while other carriers make numerous deliveries to many different destinations (called “less-than-truckload” operations). Private carriers run an internal trucking operation to support a primary business in another industry, such as a retail store chain, while for-hire carriers sell their trucking services on the open market. vehicle. FMCSA has reported that the most commonly cited driver-related factor in a fatal truck crash in 2012 was speeding. That same year, driver impairment, including by fatigue, was the fourth most commonly cited factor. Research indicates fatigue can lead to a state of diminished capacity, which can have ramifications for humans such as having more difficulty maintaining attention, becoming less communicative, and having reduced situational awareness. People are then at greater risk of committing errors in their work, which can ultimately lead to more crashes. Managing fatigue is particularly critical for tasks that require constant attention, such as driving a commercial vehicle. FMCSA is responsible for overseeing the large and diverse commercial motor-vehicle industry. To do so, FMCSA establishes safety standards for interstate motor carriers as well as intrastate hazardous-material carriers operating in the United States. To enforce compliance with these standards, FMCSA partners with state agencies to perform roadside inspections of vehicles and investigations of carriers. In fiscal year 2014, FMCSA had a budget of approximately $572 million and almost 1,100 FMCSA staff members located at headquarters, four regional service centers, and 52 division offices. Drivers of commercial motor vehicles in the United States have been subject to driving and working hour restrictions for almost 80 years. In 1937, the entity that previously carried out certain of FMCSA’s current functions—the Interstate Commerce Commission—adopted the first HOS rule. While the specific requirements in several of the provisions of that rule have changed over time, there are three general requirements that are still in place today: Daily off-duty period: Drivers must be off-duty—not working or driving—a minimum number of hours per day. Daily driving limit: Drivers can only drive a maximum number of hours per day. 60/70 hour on-duty limit: Drivers are restricted from driving when they reach a total of 60 or 70 hours of on-duty time over a rolling 7- or 8-day period, respectively. Because the 60/70 hour on-duty limit is a rolling period, drivers must calculate these hours by looking back over the prior 7- or 8-day period. For example, according to the schedule below, the driver has accumulated 68 hours in the 8-day rolling period from Thursday to Thursday, and 72 hours of on-duty time from Friday to Friday (see fig. 1). In this example, the driver worked under 70 hours (68 hours total) and is in compliance with the 60/70 hour on-duty limit from Thursday to Thursday, but not from Friday to Friday when he worked more than 70 hours (72 hours total) over an 8 day period. In 2003, FMCSA altered the HOS rule for drivers transporting freight in several significant ways: Created a 14-hour “driving window” that restricts driving beyond the 14th consecutive hour after a driver comes on-duty. Increased the daily driving limit (from 10 to 11 hours). Increased the off-duty period (from 8 to 10 hours). Collectively these provisions allow drivers to operate a commercial motor vehicle for 11 hours a day and be on-duty or work up to 14 hours per day, but also require drivers not to work or drive for at least 10 hours per day. FMCSA retained the 60/70 hour on duty limit, but allowed drivers transporting freight to “restart,” or begin at zero, their on-duty hours following an off-duty period of 34 hours or longer. Drivers are not required to take this 34-hour break, but may choose to do so to increase their scheduling flexibility, among other reasons. The provision, according to FMCSA, was adopted because it would provide drivers experiencing sleep loss or significant sleep debt with an opportunity to recover and, therefore, prevent a significant number of fatigue-related crashes. To understand the restart provision, consider the example depicted below in which a driver does not have any on-duty hours for Saturday and Sunday (48 hours). Prior to the restart provision, this driver would have reported working 72 hours from Friday through Friday and would be out of compliance with the HOS rule. Under the restart provision, the driver may now reset his/her total accumulated hours to zero on Monday because the driver was not driving or working over the weekend, a period greater than 34 hours. Therefore, while the driver was not in compliance with the 60/70 hour on-duty limit under the prior rule, he/she would be in compliance using the new restart provision, having accumulated only 60 hours for the 8 day period from Friday to Friday. (See figure 2). As previously discussed, in 2011 FMCSA made several significant changes to the HOS rule, including the restart provision. Specifically, the rule added the following provisions: Two-night provision: For the 34-hour off-duty period to count as a restart, it must include two nighttime periods from 1 a.m. to 5 a.m. 168-hour limit: Drivers can only take a 34-hour restart every 168 hours (once per week). 30-minute rest break: Drivers must take a 30-minute, off-duty break during the first 8 hours of their work day in order to continue to drive. According to FMCSA, the 2011 changes were made for several reasons. Specifically, the two-night requirement was intended to allow drivers more nighttime rest, which is more restorative than daytime rest because it better accords with the body’s circadian rhythms. The 168-hour limit was put into place to reduce fatigue by limiting the ability of drivers to work close to the maximum number of on-duty hours permitted on a continuing basis. For example, prior to the 2011 rule changes drivers who used a restart could work more than 80 hours in an average week. By limiting use of the restart to once every 168 hours, drivers could work no more than 70 hours in an average week. Lastly, the rest break requirement was designed to alleviate fatigue and fatigue-related performance degradation. FMCSA asserted that, while the 2011 rule would result in some economic costs, the changes would affect only a small percentage of drivers who regularly work long hours and that the safety and health benefits would outweigh any loss in productivity. Several organizations that represent motor carriers, however, have stated that the rule negatively impacted a larger portion of the trucking industry than was anticipated by FMCSA and that the rule had several unintended consequences, such as forcing a large amount of commercial vehicle traffic onto the roads during congested periods of the day. In July 2012, FMCSA was required by federal law to conduct a study examining the efficacy of the restart rule, which was to be completed by March of 2013. The study was to include statistically valid analysis and data on drivers in real world conditions affected by the maximum driving- time requirements. FMCSA was required to develop a methodology consistent with prior research examining the two-night provision that was conducted in a laboratory. Based on these requirements, FMCSA completed a field study (i.e., looking at drivers in real world conditions) that focused on the two-night provision, but did not examine the other two new HOS provisions: the 168-hour limit and the 30-minute rest break. To complete the field study, FMCSA contracted with Washington State University and Pulsar Informatics—a technology development and research company.amounts of data from 106 commercial vehicle drivers during two duty cycles that included two different restarts of at least 34 hours or longer. Data collected included information on sleeping and activity patterns, caffeine use, and three fatigue measures: Researchers from these institutions collected large the psychomotor vigilance test (PVT), which was the primary outcome measure of the study and measured a driver’s ability to react to visual stimuli, e.g., flashing lights, on a smart phone within a specific amount of time; the Karolinska Sleepiness Scale (KSS) that includes a subjective self- assessment of how tired a driver was at a specific point in time; and lane deviation, which measured lateral vehicle movements within a driving lane. Researchers used these data to compare fatigue levels for drivers taking a restart that included one-nighttime period versus drivers taking a restart that included two-or-more nighttime periods. Based on this evidence, the study concluded that a restart with two-nighttime breaks helps to mitigate fatigue and provides support of the efficacy of the new restart rule. The final report was submitted to FMCSA in September 2013 and issued publicly in January 2014. In December 2014—approximately 18 months after the HOS rule went into effect—congressional action resulted in the suspension of two provisions: the 168-hour limit and the two-night provision. These provisions are suspended until September 30, 2015, or FMCSA completes a new, naturalistic study—currently underway—examining the operational, safety, health, and fatigue impacts of the restart provisions in the HOS rule, whichever comes later. While similar in many respects to the prior field study, the new study is required to include more robust data and examine several issues not discussed as part of the prior field study. For example, both studies collected or will collect PVT data to assess driver fatigue. However, the new study will collect data over a significantly longer time period (approximately two weeks of data for each driver for the prior field study versus 5 months of data for the new study) and will examine crash and driver health data that were not considered as part of the field study. FMCSA is required to provide a copy of the new study to Congress by December 2015. Based on our prior work evaluating research programs, our internal expertise in research design, and established guidelines and reports for conducting research and program evaluations, we identified six generally accepted research standards that are critical for designing, analyzing, and reporting the results of scientific research. FMCSA’s 2014 field study followed three of these standards, including using measures that are valid and reliable, employing quality controls, and reporting results that are supported by data (see table 1). The field study partially met the standards of using a methodology that supports the study’s objectives and reporting conclusions that are linked to the results. The field study did not meet the standard of reporting methodological limitations and their impacts. The primary fatigue measure used in the field study is valid and reliable and supported by two secondary measures: According to our literature review and interviews with fatigue experts, the primary fatigue measure used in the field study—the PVT—is a reliable and valid measure of fatigue and is widely used in fatigue science research. While we identified limitations on the reliability and validity of the secondary measures used in the field study—the KSS and lane deviation—we also We also discussed their found they are widely used in fatigue science.use with FMCSA officials and determined that the KSS and lane deviation measures were used to support the primary findings from the PVT. Specifically, FMCSA officials stated, and we agree, that using multiple measures with different strengths and weaknesses can help to reinforce findings. In the case of the field study, each of these three measures independently supported the finding that drivers taking a one-night restart were more fatigued than those taking a two-or-more night restart. The field study included controls to identify data inconsistencies and errors: Specifically, the field study involved a visual data check that compared data for each driver across nine variables, including activity— as recorded by a wrist monitor, test results on the PVT and KSS, and GPS location. Researchers used this data check to identify any obvious errors and possible contradictions in the data, such as instances when the data might indicate a driver was coded as driving and sleeping simultaneously. Researchers also debriefed the study participants at the end of the study to discuss inconsistencies, mistakes, or missed entries as well as any outliers on the fatigue measures. This approach allowed researchers to account for incomplete or obviously biased data. Based on our review of the data and statistical coding, we believe that these quality controls were sufficient to ensure the data used in the field study are reliable. The results and analysis reported in the study are supported by the data: To assess whether the results and analysis were supported by the data, we replicated several aspects of the study. First, we acquired the raw data and the computer programs used in the field study to process the data and run statistical analyses. As a check on the reliability of the computer programs, we applied the programs to the data for one driver and verified that the results were consistent with numbers reported in the field study. Because the same computer program was applied to all drivers by the authors of the field study, this check provided assurance that the analysis for all drivers was reliable. Second, using data files provided by the authors, we replicated demographic information reported in the study, including average age, years of experience, and sex of the driver, among others. Similarly, using data files provided by the authors, we replicated the main study results. For example, our replication showed that the study accurately reported the number of reaction time errors or “lapses” on a PVT test for drivers following a one-night restart and a two- or-more night restart. The methodology used in the field study partially supports the study’s objectives: The field study examined drivers in a naturalistic environment, driving their normal routes, and collected fatigue related data in a way that was consistent with the study’s objectives. The study collected data on sleep patterns, fatigue, and driving behavior using wrist monitors, smartphones, and sensors attached to truck equipment. These methods did not require drivers to interact repeatedly with researchers or change their driving behaviors in response to study requirements, which otherwise might have biased measurements or prevented the results from applying to real world conditions. In addition, we determined the statistical method used to evaluate the study’s findings—mixed-effects regression modeling—was appropriate. Mixed effects modeling is an accepted method of analyzing longitudinal data, which, in this study, involved the same drivers taking the PVT at various times of day over several weeks. In addition, the models reasonably allowed for the possibility that the rule’s effects could vary according to the time of day and period of observation. However, we did find one area in which the methodology used did not support the study’s objectives. Specifically, the purpose of the study was to examine the efficacy of the 2011 HOS rule, which required drivers choosing to take a restart include two-nighttime periods (from 1 a.m. to 5 a.m.). This rule was expected to reduce fatigue for drivers that previously took restarts with one-nighttime period as part of their restart. Therefore, the study should have been designed to compare the ability of drivers to perform fatigue tests, e.g., the PVT, after a one-night or a two-night restart. However, the field study actually compared drivers with one-night restart breaks to those with two or more nights of rest. One concern with including drivers taking more than two nights rest is the possibility of biasing the results. For example, drivers with more rest could be less fatigued and have fewer lapses on a PVT test. Thus, including such drivers does not directly support the objective of the study to determine the impact of the changes to the restart provision from a one-night to a two-night restart. According to FMCSA officials, the decision to include drivers taking more-than-two night restarts was a judgment call made by researchers, and was likely due to the timeframe mandated by law for completing the study. Following the study’s issuance FMCSA officials stated that they conducted additional analysis and determined that the inclusion of drivers with a two-or-more night restart did not influence the results. We conducted a similar analysis and also found that the inclusion of drivers with two-or-more night restarts did not bias the results. Specifically, we found that the number of on-duty, PVT lapses for drivers who took a two- night restart was virtually the same as those taking a two-or-more night While we did not identify any bias, the decision to include drivers restart.with more than two-night restarts was not appropriate given the objectives of the study. The results of the field study support the report’s conclusions about fatigue, but conclusions about crash risk may be overstated: The study found a statistically significant difference across several measures between drivers with a one-night restart and those with a two-or-more night restart. Specifically, drivers with a one-night restart had more lapses on the PVT, rated themselves as more sleepy on the KSS, and had greater lane deviation than those with a two-or more night restart. FMCSA used these results to support two points in the conclusions section of the study: (1) the two-night provision mitigates fatigue, and (2) the two-night provision supports the efficacy of the restart rule and has implications for real-world driving performance, road safety, and crash risk. As discussed above, our review of relevant research and discussions with fatigue experts shows that the primary measure used in the field study—the PVT—is a reliable and valid measure of fatigue and is supported by the KSS and lane deviation, which are widely used in fatigue science. Therefore, the evidence supports FMCSA’s conclusion in the study that the two-night provision mitigates fatigue. Lane deviation, as defined in the field study, could mean lateral movement within a lane and did not imply crossing a line into another lane. test—an average difference of 0.3 lapses. In addition, FMCSA found a larger difference in the number of lapses between drivers taking a one- night versus a two-or-more night restart when operating at night—an average difference of 0.8 lapses per PVT test. FMCSA used these results to support its conclusions that this number of additional lapses, i.e., 0.8 more lapses, has “implications for real-world driving performance, road safety threat detection, evasive maneuvering and braking response speed and crash risk.” Further, FMCSA also provided a hypothetical example of the distance a truck could travel during a single lapse, noted that such inattention may mean the driver is not seeing critical information—such as road signs, traffic signals or other vehicles—and concluded that “an increase in lapses of attention increases crash risk.” The conclusion of the report—which implies a direct relationship between the study’s results and crashes—may overstate the findings and erroneously imply that this study provides evidence of reduced crash risk. As we discuss later in this report, while there is evidence that fatigue and crashes are generally related, we found little research that attempted to quantify this relationship using any of the fatigue measures in the field study. In particular, through our review of literature and conversations with fatigue experts, it was uncertain the extent to which, if any, a difference of 0.3 or 0.8 lapses on the PVT might be related to on-road safety outcomes.directly link results to crash risk due to challenges associated with According to FMCSA officials, its study does not performing such a study. Instead, FMCSA used lane deviation as a proxy measure for crashes and driving performance. In addition, FMCSA officials stated that, since fatigue is connected to crash risk, increases in fatigue, such as those shown in the field study, are associated with crash risk. While we agree that evidence generally supports that fatigue and crash risk are related, we are uncertain how fatigue differences of the size reported in the field study would be associated with crash risk. Thus, the safety implications and policy importance of the study’s estimated effects on fatigue may be overstated. The field study did not report all of its methodological limitations, including how these limitations might have affected the results: As with any research effort, FMCSA made methodological choices to achieve the goals of the study under the requirements outlined in MAP-21. For example, according to FMCSA, many of the decisions on recruiting drivers to participate were dictated by the short time frame under which FMCSA was required to complete the study—approximately 9 months— and the availability of resources. After considering several of these methodological choices that we reviewed, such as the sample size used, they appear to be reasonable given these constraints. However, by not reporting how certain decisions may have limited the study and impacted the results, FMCSA did not follow accepted research standards, as discussed below: The field study did not describe its driver recruitment process. According to FMCSA officials, participants in the study were recruited from three carriers that performed several different types of operations (i.e., local, regional, and over-the-road). One of these motor carriers was selected because of a prior relationship it had with FMCSA and the other two carriers were chosen after being identified as relatively safety conscious. A third-party data provider used logbook data from these carriers to identify drivers that met several criteria—including working near the maximum allowable weekly hours—which FMCSA used to identify specific operating locations that were likely to employ drivers meeting the study’s requirements. By not including information on the recruiting process in the field study, FMCSA’s study prevents readers from assessing potential bias in the collection of data and invites skepticism of the recruitment process. For example, FMCSA did not disclose that it recruited a substantial proportion of its sample of drivers with a one-night restart from a single, large U.S. city, according to one of the carriers selected to participate in the field study. Because these drivers likely operate in a similar environment, such a sample raises questions about whether these drivers’ measured fatigue levels after one-night restarts might have been related to other factors with that location. Such other factors might include characteristics of the local traffic or trucking operations. According to FMCSA officials, they recruited drivers in this manner because they needed to complete the work in a relatively short time frame—approximately 9 months—and had difficultly identifying drivers likely to take a one-night restart. While this method of recruiting drivers does not necessarily bias the study’s results, it has the potential to do so and should have been reported by FMCSA. The field study did not report the implications of its sample size. The field study collected data on a sample of drivers that produced statistically significant results. However, the sample size was not sufficient to address questions that have been raised about how the restart provision affects the diverse commercial-trucking industry. Several stakeholders we spoke with said that the restart provision likely has different effects across industry segments, including local, regional, and over-the-road drivers. These stakeholders said that the restart provision may more successfully reduce fatigue for over-the- road drivers who are away from home for weeks at a time than for drivers who operate locally and sleep in their own beds every night. We analyzed the study’s data to assess the implications of its sample size. We found that, while the field study included a diverse population of drivers from industry segments, its sample did not include enough drivers from each of these segments to estimate differences in PVT at all or to identify statistically significant differences.assessed the sample size that might have been required to estimate significant differences across industry segments. We found that the field study’s sample size was insufficient to estimate significant differences in the primary fatigue measure—the PVT—for each of In addition, we several segments of the industry.decisions on the sample size for the field study were made to accommodate the deadline for when it must be completed. Given the importance of understanding how the results of the field study might have varied by segment, FMCSA should have stated how the study’s sample size limited the generality of its findings and conclusions. According to FMCSA officials, The study did not report how a critical analytical decision might impact the results. As was discussed above, the field study compared drivers with one- versus two-or-more night restart breaks. According to accepted research standards, FMCSA should have reported why drivers taking a more than two-night restart were included in the study, even though the rule should have been designed to compare ability of drivers to perform fatigue tests, e.g., the PVT, after a one- night or a two-night restart. Furthermore, FMCSA should have reported the potential for this decision to bias the results and, in particular, to exaggerate the difference in fatigue between the two groups of drivers. According to FMCSA officials, researchers followed several standards when developing and conducting the field study, including Office of Management and Budget’s (OMB) guidelines on the use of independent peer review panels and the DOT Scientific Integrity Policy. The OMB guidelines FMCSA used require agencies to use a peer review process for important scientific information to enhance its quality and credibility. FMCSA officials reported using the guidelines to evaluate the research design prior to implementation as well as to assess the report findings. In addition, DOT’s Scientific Integrity Policy lays out nine elements that primarily detail the importance of communication, transparency, and integrity when using scientific information for decision making. While these standards address several important considerations, including enhancing credibility and transparency, they do not provide specific guidance on how to determine appropriate methods for designing, analyzing, and reporting the results of scientific research. Without such guidance, FMCSA research may not include critical elements, as occurred in the reporting of the field study. Without these elements, FMCSA leaves itself vulnerable to criticism over the integrity of its research—an important consideration, given the heightened profile of HOS regulations. FMCSA used several key assumptions to estimate the costs and benefits of the 2011 HOS rule. For example, FMCSA assumed that the rule would result in schedule adjustments for approximately 15 percent of drivers who work more than 65 hours per week on average. These assumed schedule adjustments drove FMCSA’s estimates of the economic costs and the health and safety benefits of the 2011 HOS rule. Our analysis of available data provides some insights into the potential effects of the 2011 HOS rule. For example, we found that about 12 percent of drivers in our dataset worked an average of over 65 hours per 8 day work week before the rule went into effect. The lack of nationally representative driver schedule data limits the ability of researchers to fully evaluate the rule’s impact. Although soon all carriers will be required to collect schedule data electronically and these data could be useful for evaluating the effects of this rule and in formulating future rules, current legal restrictions limit how these data can be used. As part of its rule-making process, FMCSA developed assumptions about the motor carrier industry and its operations to estimate the economic costs and the safety and health benefits of the 2011 HOS rule. For example, FMCSA assumed that drivers working more than 65 hours per week would incur the majority of schedule changes as a result of the rule. FMCSA estimated that such drivers made up approximately 15 percent of the driver workforce. FMCSA then made assumptions about how drivers would likely be affected by the rule to estimate a weekly per driver work time reduction of 0.05 hours (3 minutes) for drivers working an average of 60 hours per week; .38 hours (23 minutes) for drivers working an average of 70 hours per week; and 8.7 hours for drivers working an average of 80 hours per week. To estimate the economic costs of the rule, FMCSA calculated the cost of this lost work time and estimated $430 million in annual costs. Further, FMCSA assumed that reduced work time would increase a driver’s opportunity to sleep leading to safety and health benefits. FMCSA estimated the rule would save between 10 and 26 lives per year and result in $150 to $390 million in annual safety benefits. FMCSA also estimated between $70 million and $850 million in health benefits. (See figure 3). Analyzing available data can help provide some insight into the rule’s potential effects and the extent to which they aligned with FMCSA’s assumptions and estimates. Specifically, we conducted a comprehensive search for available data that might provide insight into the rule’s effects in the five main areas in which FMCSA made assumptions: (1) which drivers were affected, (2) how drivers were affected, (3) economic costs, (4) safety benefits, and (5) health benefits. The results from our analysis of schedule data from 16 for-hire carriers is consistent with FMCSA’s assumption that drivers working more than 65 hours per work week would be more likely to reduce their work hours. We also found support for FMCSA’s assumption that certain schedule changes could reduce driver fatigue and while total crashes, and crashes with injuries did not appear to change, crashes involving fatalities may have declined after the rule went into effect. However, we were unable to assess, due to a lack of data, FMCSA’s assumptions on economic and health effects. All of the data and analyses we used had limitations, including the extent to which the data are representative of the motor carrier industry and the extent to which we can attribute observed differences to the rule change. In addition, we could not fully analyze the possible effects of the 2011 HOS rule due to the relatively short time frame during which the rule was in place—approximately 18 months. (See table 2). Which drivers were affected: Our analysis of 2012 to 2014 driver schedule data from 16 for-hire motor carriers is consistent with FMCSA’s expectation that the percentage of drivers working the longest hours— over 65 hours per 8-day work week—decreased after the rule went into effect on July 1, 2013. Specifically, we found that the percentage of drivers in our dataset working over 65 hours per 8-day work week decreased from 12 percent immediately before the rule went into effect to 6 percent afterwards. Our analysis also found that the percentage of drivers working between 55 and 65 hours per 8-day work week decreased from 33 percent to 29 percent. In contrast, the percent of drivers working 35 to 55 hours per 8-day work week increased from 48 percent to 57 percent after the rule went into effect. (See figure 4). However, the driver schedule data used in our analysis are not representative of the entire motor carrier industry and do not include any information on drivers who work in some segments of the trucking industry, such as private motor carriers. Therefore, our findings cannot be generalized to the motor carrier industry as a whole. In addition, our interviews with representatives from 20 motor carriers suggest that a wide variety of motor carrier operations were potentially affected by the HOS rule. Specifically, 13 motor carriers we spoke with, including 10 for-hire carriers and three private carriers, stated they were affected by the rule. These carriers described themselves as truckload carriers, less-than-truckload carriers or a combination of both and employed drivers working both more than and less than 65 hours per week. In addition, seven motor carriers reported they were not affected by the HOS rule, including three for-hire carriers, three private carriers, and one carrier that used a truck on a part-time basis and did not fall into either category. How drivers were affected: Our analysis of schedule data from 16 for-hire carriers supports the assumption that some drivers may be working fewer hours and using restarts less often. Specifically, as shown above in figure 4, these schedule data show that fewer drivers were working more than 55 hours per 8-day work week after the rule went into effect in July 2013. We also estimated that drivers, on average, worked approximately 1 to 2 fewer hours per work week after the rule went into effect. In addition, the number of restarts drivers took per calendar week (168 hours) generally declined after the rule went into effect, including for drivers working more than 65 hours per 8-day work week (see fig. 5). We also found that drivers took approximately 6 percent fewer restarts per week, on average, after the rule went into effect. We also found evidence supporting a shift in traffic volume from night time hours to other parts of the day following the effective date of the HOS rule. These shifts cannot be directly attributed to the rule change and the data we analyzed from the Federal Highway Administration were limited to 14 states. Our analysis showed that commodity carrying, i.e., commercial, motor vehicle traffic volume increased by 2.6 percent between November and December 2012 and November and December 2013. However, as figure 6 shows, this increase was not consistent across the time of day. Specifically, commodity carrying motor vehicle traffic mostly increased during daytime hours (6 a.m. to 6 p.m.) and decreased during evening and nighttime hours (6 p.m. to 6 a.m.), relative to the increase in overall commodity carrying vehicle traffic. The largest differences from the growth in traffic volume occurred in the middle of the night (11 p.m. to 4 a.m.), when volume declined or remained flat despite an increase in commodity carrying vehicle traffic overall and in the mid- morning hours (7 a.m. to 10 a.m.) when volume grew faster than the overall growth rate. Consistent with our data analysis, representatives from 13 of the 20 motor carriers we interviewed reported a range of operational effects or changes they attributed partially or entirely to the 2011 HOS rule. The other seven motor carrier representatives did not report any operational effects due to the rule. For example, motor carriers reported a decrease in driving hours per driver (10 carriers), less frequent restart use (10 carriers), more driving during periods of traffic congestion (8 carriers), hiring more drivers (6 carriers), and an overall decrease in operational flexibility (10 carriers). Several of these operational changes, such as a decrease in driving hours per driver and subsequent hiring of more drivers to make up for reduced driver productivity, could lead to more commercial vehicles on the road. Economic Costs: We interviewed motor carriers and industry stakeholders about the economic impacts of the 2011 HOS rule but did not find a data source that would allow us to reliably analyze these impacts for the industry. Representatives of 11 out of the 20 motor carriers we interviewed mentioned negative economic impacts due to the 2011 HOS rule. Ten motor carriers reported a productivity loss, six reported a decrease in delivery timeliness, six reported less profitability, and four reported an increase in driver turnover. The other nine motor carriers we interviewed reported no negative economic effects. In addition, four of the eight drivers we spoke with reported a decrease in their income due to the 2011 HOS rule. These drivers reported that reduced work time—fewer or shorter work shifts—was the reason for their loss in income. The four drivers that reported no decrease in pay either did not have a schedule change or were working more hours per week to compensate for schedule changes due to the rule. Other industry stakeholders we interviewed, including 12 out of 13 associations representing segments of the motor carrier industry, shippers, and warehouse operators, also reported negative economic impacts on their respective industries due to the 2011 HOS rule, such as reduced productivity. For example, some industry stakeholders told us their members must hire more drivers to do the same amount of work and keep more inventory in stock to mitigate delays and delivery uncertainty. Safety Benefits: According to academic literature we reviewed, fatigue can negatively impact the ability to perform tasks. Fatigue, as a result of being awake for long periods of time or insufficient sleep, can have a negative impact on driver performance and can lead to safety critical events, including crashes. However, the literature is less clear on which HOS interventions (e.g., rest breaks and limiting driving and on-duty time) best minimize fatigue and reduce crash risk. For example, research has found that crash risk is higher when drivers start their shifts or after taking an extended break; but research also shows that drivers who do not take breaks have a higher crash risk than drivers who do take breaks. Our analysis suggests that drivers’ operating after the HOS rule went into effect on July 1, 2013, could have a lower risk of fatigue. To assess the extent to which the 2011 HOS rule might affect driver fatigue, we conducted two separate analyses using a biomathematical fatigue model. First, we used the model to compare hypothetical driver schedules that complied with two of the 2011 HOS rule provisions—the 168-hour limit and the two-night provision—to similar schedules that did not comply with the rule but were in compliance with the previous HOS rule.the rule went into effect would result in lower fatigue scores and, We found that some schedule changes that would be required after therefore, a lower risk of driver fatigue.schedules we modelled that had to change after July 1, 2013, in order to comply with the rule were: Maximum Allowed-Hours Day and Night schedules: 14-hour shifts over 5 consecutive days or nights. Drivers following a daytime schedule of 14-hour shifts would have to modify their schedules to comply with the 168-hour limit. Specifically, they could no longer take a restart with less than 168 hours (7 days) between when they began their last restart period and the beginning of the next restart period. To comply with the 168-hour limit, drivers following this daytime schedule must take 2 days off instead of one day off between work cycles. For night drivers working 14-hour shifts, working 5 consecutive days with one day off for a restart would not comply with either the 168-hour limit or the two-night provision. These drivers must also take 2 days off instead of one day off between work cycles. The longer restart period (2 days versus one day) taken by these hypothetical day and nighttime drivers working the maximum hours allowed each day results in lower fatigue scores and, therefore, lower risk of driver fatigue over the five day work cycle. Sixty- and 70-Hour Night Schedules: 10- to 12-hour shifts over six consecutive nights. Drivers working through the night but working less than the maximum allowed 14 hours each day would also have to change their schedules after July 1, 2013. Specifically, these drivers would have to take 2 days off between work cycles in order to comply with the two-night provision. Again, the longer restart period (2 days versus one day) results in the hypothetical driver having lower fatigue scores and, therefore, a lower risk of fatigue over the 6-day work cycle. Our second analysis using the biomathematical fatigue model estimated possible fatigue effects of the 2011 HOS rule based on information provided by motor carriers, industry stakeholders, and drivers we interviewed. We found that some of the real world schedule changes described by motor carriers resulted in a lower risk of driver fatigue, while other schedule changes showed a higher risk of driver fatigue. For example, a motor carrier told us that, to comply with the two-night provision, several of its nighttime drivers switched from a 6-day schedule to a 5-day schedule. Our analysis of this schedule change shows that drivers under this schedule have a lower risk of fatigue after the rule went into effect. In contrast, a different motor carrier told us that, to comply with the 168-hour limit, several of its nighttime drivers switched from a 4-day schedule with 3 days off to an alternating schedule of 5 days working and 2 days off and then 5 days working and 3 days off. Our analysis of this schedule change shows that these drivers have a higher risk of fatigue on certain days in their schedule after the rule went into effect. This may be due to more work time and less off-duty time for drivers following the schedule that complies with the 2011 HOS rule. We also collected data on crashes involving commercial trucks from 2008 to 2014. As shown in figure 7, the monthly incidence of truck crashes and crashes involving fatalities varies considerably over time. To assess whether the rule had any discernable effect on crashes, we conducted two types of analyses. We examined (1) whether the rule affected the number of monthly crashes and (2) whether the rule resulted in any shift in the time of day in which crashes occurred. Trend in crashes over time: Many factors influence the number of crashes over time. For example, as the economy grows, typically there will be more trucks on the highways moving freight and this would be expected to be correlated with a higher incidence of crashes, all other things being equal. We used a statistical model to examine how the rule’s implementation may have affected the number of monthly crashes, crashes with injuries, and fatal crashes, while controlling for other factors that affect the incidence of crashes, such as a proxy for trucking volume and seasonal variation. Our analysis showed a strong correlation between our measure of trucking volume and crashes—crashes occurred with more frequency when the industry was moving more freight—which is consistent with other previous work.found a positive statistical association between the implementation of the rule and the number of all crashes, suggesting that crashes rose—holding other factors constant—after the rule when into effect. However, our discussions with industry participants indicated that the winter of 2013 to 2014 was unusually harsh and made operations very difficult for the industry. These weather conditions may have been a contributing factor, irrespective of the rule change, to a short-term rise in crashes in December 2013 through February 2014. When we used statistical methods to control for these months in a second iteration of our analysis, we no longer found a statistically significant increase in the number of crashes after implementation of the rule. That is, the incidence of crashes would appear to not have changed due to the rule in this scenario. Our analysis of crashes with injuries and crashes with fatalities was similarly structured—we included variables to account for truck volume, seasonal variation, and in a second iteration of the model, the unusual 3 months of weather during the winter of 2013 to 2014. We found that the implementation of the rule was not associated with any change in the incidence of crashes with injuries, whether or not we statistically controlled for the winter of 2013 to 2014. However, our analysis does indicate that crashes involving fatalities may have become less frequent after the implementation of the HOS rule. In addition, our initial analysis Time of day shifts in crashes: Some stakeholders told us that the two- night provision of the HOS rule would potentially result in an increase in crashes involving commercial trucks during morning rush hours (i.e., 5 a.m. to 9 a.m.). They believed that some truckers who might have previously driven during the middle-of-the-night hours would be shifted to the early and mid-morning timeframe due to provisions of the new HOS rule. To test this possibility, we developed statistical models to examine whether, conditional on the number of crashes that occurred, there was a change in the likelihood of truck crashes occurring between 5 a.m. and 9 a.m. after the HOS rule went into effect. Our analysis found no statistically significant change in this likelihood associated with the implementation of the rule. An important caveat to our statistical analyses of crash data is that there are only 15 months of available data following the implementation of the rule. These limited data, which have non-trivial variation month to month, may not suffice to discern trends in the incidence of crashes. For example, as we explained above, our post-rule time frame covers only one winter—a time when crashes tend to rise—and we were told that this was an unusually difficult winter for the industry. These data limitations affect the ability to draw empirical inferences on the effects on crashes of the rule’s implementation. The 20 motor carriers we interviewed said they believed the rule had no noticeable effect or a negative impact on safety. Of the eight drivers we interviewed, one said the rule had no effect on safety, and seven said they had seen a negative impact on safety. For example, drivers mentioned that the time pressures they stated were associated with the 2011 HOS rule—particularly the 30 minute rest break—had resulted in more speeding. Drivers said that speeding occurs as drivers try to make deliveries or mileage goals before taking a 30-minute rest break. Safety groups we interviewed generally supported the 2011 HOS rule and thought it could improve safety for certain segments of the industry, but had concerns about ensuring compliance with the rule. Health Benefits: Academic literature we reviewed indicates that insufficient amounts of sleep have negative health impacts. These effects include obesity, diabetes, hypertension, and cardiovascular disease. However, we did not find a data source that would allow us to assess health impacts of the rule. According to FMCSA officials and other stakeholders we spoke with, it could take years to identify health impacts following an intervention, such as the HOS rule. In addition, representatives from 16 of the 20 motor carriers we interviewed said they believed the 2011 HOS rule had no noticeable health effects. The other four motor carriers said they either noticed negative health effects— due to increased stress or fatigue as drivers tried to make up for lost income, among other reasons—or had no way to measure health impacts. In addition, of the eight drivers we interviewed, six said their health was negatively impacted by the 2011 HOS rule, and two said there has been no impact. Those reporting a negative health impact cited, for example, being more stressed as they adjust to new schedules or try to make up for lost income. Safety groups we spoke with generally thought the 2011 HOS rule could improve driver health, but that health impacts would be difficult to attribute to the rule change. The ability of FMCSA and others to assess the effects of the 2011 HOS rule is impacted by the limited availability and representativeness of driver schedule data (i.e., records of drivers’ work hours). No organization, including FMCSA, collects or maintains a centralized database with representative driver schedule data that can be generalized to the entire motor-carrier industry. For example, while we obtained and analyzed drivers’ schedule data from a private research organization that includes over 15,000 usable driver records from 16 for-hire trucking companies, findings from that dataset cannot be generalized to the estimated 539,000 active motor carriers and 5.6 million drivers who operate in the United States. As a result it is not possible to determine the extent to which different types of commercial motor carriers, such as private carriers or drivers who own and operate their own vehicle, were affected by the rule because they are not included in the dataset. Similarly, FMCSA does not collect or use representative driver schedule data. Instead it uses roadside inspection, compliance review, and safety audit data as well as special studies to analyze motor carrier and drivers’ safety behaviors. Collecting representative schedule data is well recognized as a valuable analytical resource. Other DOT agencies that regulate the hours of service for transportation operators have demonstrated the importance of using representative schedule data. For example, the Federal Railroad Administration has collected logbook data from a representative sample of rail employees to examine fatigue and safety concerns since 2001. The Federal Aviation Administration also has access to electronic flight data on aircraft operations that is used to identify and analyze national trends, target agency resources, and identify and reduce or eliminate safety risks. In addition, FMCSA itself has cited the need to evaluate the actual impacts of its regulations, including the 2011 HOS rule, in order to meet executive requirements on regulatory review. We have also found that sufficient and appropriate data are critical for rulemaking, decision making, and assessing the effects of regulations and programs. As we have previously found, using data is especially critical for FMCSA, which has limited resources to oversee a large and diverse industry. Collecting these types of data has historically been difficult, but a recent statutory change to how driver schedule data will be collected provides a potential opportunity to do so. Many carriers use paper logbooks, a practice that makes collecting and analyzing schedule data administratively unmanageable. MAP-21 required FMCSA to develop a rule that will require motor carriers to stop using paper logbooks and instead, install electronic devices on individual vehicles to record and store drivers’ schedule data. This shift to electronic data will provide a potential source of representative data that could more easily be accessed and analyzed. However, FMCSA officials said that they do not currently plan to collect and use such data for research purposes because MAP-21 limits their use to the enforcement of laws. As we have shown, representative data on the motor carrier industry— specifically the number of drivers nationwide, the amount they drive, and how they use the restart provision—would provide critical information to FMCSA about the operations of the industry it oversees. Access to this type of data would also support data-driven rulemaking efforts and regulations. According to FMCSA officials, access to driver schedule data would enhance program evaluation, rulemaking, and the analysis of existing rules, including the HOS rule. We recognize, however, that there are concerns to collecting, storing, and analyzing these data, for example: Privacy: MAP-21 put protections in place to ensure driver privacy, including provisions to limit the harassment of drivers, protect personally identifiable information, and preserving the confidentiality of any personal data. Additionally, as we have previously found, collecting motor vehicle data raises concerns about sharing data with third parties, tracking drivers’ movements, and vehicles’ speed and location. Cost: FMCSA officials expressed concerns about the costs to the agency and industry to collect, store, and analyze representative schedule data. For example, FMCSA officials believe that obtaining, standardizing, and depersonalizing data for millions of drivers would pose substantial costs on the agency, motor carriers, and drivers. There may be ways of mitigating these privacy and cost concerns, but FMCSA has not examined the costs and benefits of collecting electronic driver-schedule data on a large-scale and currently does not plan to do so. For example, to address privacy concerns, information can be “de- identified” for the purposes of analyzing data. Specifically, the Federal Aviation Administration uses de-identified electronic flight data to identify and reduce or eliminate safety risks. In addition, our analysis of the effects of the HOS rule used schedule data from several thousand drivers, which we analyzed without any personally identifiable driver or motor carrier information. Cost concerns could potentially be addressed by collecting a nationally representative sample of the data—rather than from the entire universe of drivers. Sampling is a commonly used social science practice that allows researchers to apply conclusions drawn from a subset of a population to the entire population. Although FMCSA officials told us that they have no plans to examine the possibility of collecting, storing, and analyzing electronic driver schedule data because of the MAP-21 provision, they thought privacy concerns, for example, could be addressed through stripping raw data of drivers’ personally identifiable information before being shared with the agency or others. Until FMCSA evaluates the potential for collecting, storing, and analyzing data; the potential benefits of using the data; and privacy and cost concerns as well as the potential for FMCSA to mitigate them, the agency’s ability to use electronic schedule data in the future will remain unknown. The HOS rule has now been debated for over a decade with supporters and critics arguing over the extent to which it improves safety and health outcomes and negatively impacts the economy. FMCSA has attempted to quantify these effects through research efforts, but many continue to question the results. This is why it is critical for FMCSA to evaluate potential effects of rules, such as the HOS rule that has a heightened public profile, using established research standards and representative schedule data. FMCSA’s field study demonstrated that the agency did a reasonable job in designing the study and analyzing data. However, because the agency did not use guidance outlining specific standards for conducting and reporting the results of scientific research, the field study fell short in reporting several limitations and did not fully link the results to its overall conclusions. These shortcomings leave the agency open to criticism over the integrity of the study and invite skepticism about the results. Similarly, by not having access to and analyzing representative driver schedule data, FMCSA may continue to face challenges to the credibility of its rules. The lack of representative schedule data required FMCSA to make a number of assumptions about how the motor carrier industry would be affected by the HOS rule. Due to current data limitations, it is not possible to fully evaluate the rule’s impact. For example, our analysis suggests that the rule may have affected a larger population of drivers than FMCSA anticipated, but without representative data there is no way to be certain this is universally the case. Understanding the population of drivers affected by the HOS rule is critical for determining the associated economic costs and safety and health benefits. While electronically collected, representative schedule data that will soon be available to FMCSA would allow it to assess the impact of its rules, MAP-21 places limitations on the use of this data for purposes other than enforcing laws. Further, collecting, storing, and analyzing the data would require mitigating privacy and cost concerns. Given the potential value of such data for evaluating the impact of future regulatory rules, Congress may benefit from information on how the electronic data to be collected in response to the MAP-21 requirements could be extracted, stored, and analyzed and how privacy and cost concerns associated with the use of these data could be addressed. Such a study could help Congress determine whether limitations on the use of electronic schedule data for purposes other than enforcement could be amended in the future. Congress may wish to consider directing DOT to study and provide a report to Congress identifying approaches for extracting, storing, and analyzing electronically collected motor carrier drivers’ schedule data, including the potential benefits, privacy, and cost concerns, and options for how such concerns could be mitigated. To help ensure that FMCSA’s future studies follow generally accepted research standards, the Secretary of Transportation should direct the FMCSA administrator to adopt guidance outlining research standards for designing, analyzing, and reporting the results of scientific research. We provided a draft of this report to DOT for review and comment. DOT provided written comments, which are reprinted in appendix VIII. In its written comments, DOT stated that FMCSA agreed with our recommendation, although DOT also stated that FMCSA adhered to standard principles and practices of scientific research in conducting its January 2014 HOS study. As stated in this report, we identified generally accepted research standards that DOT followed; however, we also identified standards that it did not fully follow, namely, using a methodology that supports the study’s objectives, reporting conclusions that are linked to the study’s results, and reporting methodological limitations. We believe that FMCSA would strengthen its future research and enhance the integrity of future studies by adopting guidance outlining research standards for agency employees to follow. DOT also stated in its written comments that our report recognized several achievements associated with its 2011 HOS rule, specifically a decrease in the frequency of drivers using long work schedules, a lower risk of driver fatigue, and a reduction in the number of commercial vehicle crashes involving fatalities. As we state in this report, however, these findings cannot be directly attributed to the rule. In addition to the written comments, DOT also provided technical comments, which we incorporated as appropriate. We are sending copies of this report to the Administrator of FMCSA, the Secretary of the Department of Transportation, and interested congressional requesters. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov. If you or your staff has any questions about this report, please contact me at (202) 512-2834 or flemings@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix IX. Our objectives were to examine (1) the strengths and limitations of the Federal Motor Carrier Safety Administration’s (FMCSA) field study that examined whether the two-night provision reduces commercial driver fatigue, and (2) the key assumptions FMCSA used, at the time the 2011 hours of service (HOS) rule was promulgated, in estimating its effects and what is known about the economic, safety, and health effects of the rule. To assess the strengths and limitations of the field study, we drew on established guidelines and reports for assessing research and analysis, our reports on evaluating research programs, and our internal expertise in research design. We identified key standards used by professional associations, academics, and our prior work, with an emphasis on standards applicable to the design and methods used in the field study. (See table 3). Specifically, we identified six standards relating to (1) choosing evaluation measures that are valid and reliable; (2) using quality controls to identify data inconsistencies or errors; (3) reporting results and analysis that are supported by the data; (4) using a methodology, including statistical techniques, that support the objectives of the study; (5) reporting conclusions that are linked to the results; and (6) reporting all methodological limitations, including their potential impact on the results. We compared these standards to information included in the field study, statements made by FMCSA officials and researchers contracted by FMCSA to complete the study, and our analysis of study data. To assess the reliability and accuracy of data and results reported in the field study, we collected and reviewed data provided by the primary researchers. These data, collected between January and July 2013, included between 50 and 60 files for each of the 106 study participants, which the authors of the study used to build summary-level datasets for each driver. We also acquired the computer programs used by the field study’s authors to process the data and run statistical analyses. As a check on the reliability of both the computer programs and the data we received, we applied the programs to the data for one driver and verified that the results were consistent with numbers reported in the field study.In addition, we replicated results for 15 driver demographic variables, including age ranges, gender, experience, and type of operation, among others. For example, we confirmed that 100 men and 6 women participated, as the study reported. We also talked with the primary authors of the study about the steps they took to ensure the completeness and accuracy of the data, and checked the data for completeness and reasonableness. We determined that the data were sufficiently reliable for the purpose of our data analysis; in particular, we determined that we could, with sufficient reliability, conduct additional analysis of the data and align our results with those reported in the field study. We assessed the sensitivity of the study’s results to various alternative methodological choices. For example, we tested whether the results varied depending on whether the study included drivers having more than two nights of rest, as suggested by several stakeholders we interviewed. More discussion on this theory is included in the body of our report. those reported in the field study.our sensitivity testing and results, see appendix II. Databases consulted for these searches include Ei Compendex®, Embase®, EMCare®, MEDLINE®, NTIS: National Technical Information Service, PsycINFO, and Transport Research International Documentation, among others. selected these experts based on their numbers of published articles related to the fatigue measures used in the field study, as well as our prior work on rail’s HOS regulations, work that included opinions from several fatigue-science experts. To identify the data collection and analytical methods used in the field study we spoke with officials from FMCSA and individuals contracted to complete the study, including the primary researchers of the study. These officials provided information on the statistical analysis used in the field study, driver recruitment, and data reliability assessments. To better understand the recruitment process and data collection methods, we also interviewed officials from all three motor carriers that participated in the study. Finally, we spoke with numerous industry and safety stakeholders to get their views on the strengths and limitations of the field study. For a list of these stakeholders, see table 6 below. To identify key assumptions FMCSA used to estimate the effects of the 2011 HOS rule, a GAO economist, not involved in other aspects of our work and with substantial experience in cost-benefit analyses, reviewed the Regulatory Impact Analysis for the 2011 HOS rule. This expert identified an initial list of assumptions. We further categorized these assumptions into five areas: which drivers were affected, how drivers were affected, economic costs, safety benefits, and health benefits. FMCSA officials confirmed that our characterization of key assumptions was accurate. For more information on this process and the full list of FMCSA assumptions, see appendix III. To identify what is known about the possible economic, safety, and health effects of the 2011 HOS rule, we conducted a comprehensive search for existing databases with information on motor carrier operations. We conducted web searches of academic journals, as well as government and other stakeholder websites. We also used search engines and asked officials from FMCSA and other stakeholders for recommendations on databases we could use to analyze the effects of the rule. We categorized the databases we found by the type of data collected, source, whether that data was publicly or commercially available, and potential costs. Based on this information we identified three different, relevant data sources, which were used for a variety of purposes (see table 5). To assess the reliability of these databases, we reviewed documentation on data collection efforts and quality assurance processes, talked with knowledgeable officials about these data, and checked the data for completeness and reasonableness. We determined that the data were sufficiently reliable for the purpose of our data analysis, though we also identified limitations in each dataset. More information on our analysis of American Transportation Research Institute (ATRI) logbook data, Federal Highway Administration (FHWA) vehicle count data, and FMCSA crash data can be found in appendixes IV, VI, and VII, respectively. We spoke with multiple stakeholders, including organizations representing motor carriers, safety advocacy organizations, motor carrier companies, commercial vehicle drivers, and customers of the motor carrier industry (e.g., companies that ship freight) to understand how the 2011 HOS rule affected them and their members (see table 6). We selected stakeholders and shipping organizations to interview based on our prior work on commercial motor-carrier safety and recommendations from these stakeholders. We used FMCSA’s Motor Carrier Information Management System to select active, U.S.-based motor carrier companies subject to the hours of service rule. Our selection was based on three primary criteria: (1) operation type (for-hire versus private), (2) fleet size, and (3) cargo type (interstate versus intrastate hazardous material). Using these criteria we created six selection groups to identify target populations that included for-hire and private carriers and had a range of fleet sizes. We also created a seventh group that included only hazardous material carriers. We randomly selected 150 carrier companies from each of these groups for a total of 1,050 carrier companies across all seven groups. We then selected 3 to 5 carrier companies from each group using four additional selection criteria: (1) driver radius in miles (to identify carriers that drove a wide range of distances); (2) FMCSA region (for the purposes of ensuring geographic diversity); (3) HOS violations in the past 2 years (to identify carriers that might have experience with these violations); and (4) years in operation (to identify carriers with a range of experience in the industry). Motor carrier companies we spoke with also put us in contact with eight drivers whom we interviewed to understand how the 2011 HOS rule had impacted them. The information collected from these interviews with representatives of motor carriers and drivers is not generalizable to all motor carrier companies or drivers. To summarize information from our interviews with motor carriers, we systematically analyzed information they provided on how the 2011 HOS rule had impacted them. To do so, one analyst coded interview responses. A second analyst randomly selected three interviews to code. The results from the two analysts were compared to ensure that the coding method was consistently applied, which confirmed that the coding done by the first analyst was reliable. To identify possible fatigue effects on driver safety and health as well as the effectiveness of HOS rules in mitigating fatigue we conducted a literature review. Our literature review was also used to provide additional information on the biomathematical fatigue model we selected to analyze simulated and example driver schedules (see below). We identified specific articles for our literature review through searches conducted by a GAO librarian across bibliographic databases containing peer-reviewed materials, government reports, and conference papers.list of results by reviewing abstracts and focusing on articles published since 2008. This process identified 54 articles, most of which we used for background purposes or to provide additional context. We also identified several articles related to the effectiveness of HOS rules that were used We honed the to support our findings and required additional review. To select these articles we had two analysts code 10 of the 54 articles as either relating to effectiveness of HOS rules, relationship between fatigue and driver performance and safety, and/or relationship between fatigue and driver health. We compared the coding results and determined that the remaining coding could be reliably completed by a single analyst. Using this method we identified 15 articles related to the effectiveness of HOS rules. The methodologies and findings of these articles were reviewed by GAO’s technical experts (research methodologists and an economist) to ensure that the findings were well supported. To examine the potential effects of the 2011 HOS rule on driver fatigue, we used a biomathematical fatigue model of human alertness response to work and rest patterns. We obtained the Fatigue Audit InterDyne™ (FAID) model from InterDynamics Pty Ltd as the result of a competitive procurement. Our analysis using the FAID model involved two parts: 1. We developed schedules with different work periods (day and night) and different shift lengths (e.g. 14-hour shifts to simulate the maximum allowed daily on-duty hours).We then created different scenarios based on whether a schedule would need to change to comply with the two restart provisions in the 2011 HOS rule. For example, we used the FAID model to compare driver fatigue levels for a driver working 14-hour night shifts 5 days a week with one-night off (not in compliance with the two restart provisions in the 2011 HOS rule) to a driver working the same 14-hour night shifts 5 days a week with two-nights off (in compliance with the two restart provisions in the 2011 HOS rule). 2. We analyzed real world schedules described to us by motor carrier operators and drivers we interviewed regarding how they had changed schedules due to the new rule. For example, we used the FAID model to analyze fatigue levels of drivers for a motor carrier that told us their drivers had changed from six shifts per week (before the 2011 HOS rule went into effect) to five shifts per week (after the 2011 HOS rule went into effect) in order to be compliant with aspects of the rule. For a technical discussion of the scope, methodology, and additional results using the FAID model, see appendix V. As part of our review of the strengths and limitations of FMCSA’s field study that examined the two-nighttime provision of the 2011 hours of service (HOS) rule, we conducted two statistical analyses, as described below. First, we conducted sensitivity tests to assess how certain methodological decisions in the field study potentially affected its results. Second, we conducted a power analysis to assess the adequacy of the study’s sample size for estimating how the rule affected fatigue within each of several industry segments. During the course of our review, we asked stakeholders we interviewed to describe their general impressions of the data and statistical methods used to evaluate the rule. Stakeholders specifically questioned two methodological decisions: 1. The study analyzed data for drivers who had taken more than a two- night restart. Some stakeholders expressed concern that this decision did not conform with the purpose of the study to analyze the impact of changing the restart provision from requiring one-night of rest to exactly two nights. 2. The study collected data from what stakeholders characterized as a small sample of drivers, a sample that included local and regional drivers. Some stakeholders said that the study’s sample size of 106 drivers was not large enough to provide enough data to draw sound conclusions or be representative of the trucking industry. In addition, several stakeholders we spoke with said that the restart provision likely has different effects across industry segments, including local, regional, and long-haul or over-the-road drivers. To test the validity of these concerns, we acquired the source data and computer code used in the field study from the authors. We conducted sensitivity tests of these data that assessed whether the study’s results might have been different if the researchers had made different methodological choices. Our tests examined whether the results of the psychomotor vigilance tests (PVT) and the Karolinska Sleepiness Scale (KSS) would have been different had the study: 1) excluded drivers taking a two-or-more night restart and 2) allowed the outcomes to vary by the drivers’ industry segments, i.e., local, regional, or over-the-road. 𝑌𝑖𝑗 ~ N(𝐱𝐢𝐣𝛃 + 𝜇𝑗,𝜎𝑌2) 𝜇𝑗 ~ N(𝜇,𝜎𝜇2) xij = Condition = = Timingij Periodij = {I(Condition x Timingij x Periodij)} {1 night restart, 2+ night restart} {12:00 – 4:00 AM, 4:00 – 8:00 AM, … , 8:00 – 11:59 PM} {1st duty period, restart period, 2nd duty period} given by 𝑁 = ∑𝑁𝑗 Yij denoted the number of PVT lapses or the KSS score for driver j = 1, 2, … , J on measurement occasion i = 1, 2, … , N, with the total sample size . xij was a vector of indicator variables for each level of the Cartesian product of Condition, Timingij, and Periodij, similar to a design vector in an experimental study. Condition measured the number of nights in the restart period between the two duty periods in the study. Timingij and Periodij measured the time of day and stage of the study, respectively, when the PVT or KSS outcomes were measured.column vector of fixed coefficients. Since xij did not include an intercept 𝛃 was a and all covariates were categorical, 𝛃 denotes the cell means for each subgroup in xij. We increased the granularity of Condition in order to estimate the effect of interest, the contrast between mean fatigue after one-night versus two- night night restarts, by adding a separate category for three-or-more night restarts. The study estimated this overall contrast by taking equally weighted linear combinations of the estimated effects across subpopulations defined by Timingij x Periodij, using the “least-squares means” method implemented in SAS.order to minimize methodological changes other than those we evaluated. We used the same approach, in Our test found that including drivers who took more than a two-night restart in the treatment group did not affect the results, as shown by table 7. On-duty drivers taking only a two-night restart had 0.32 lapses more lapses per PVT bout, on average, than drivers taking a one-night restart. This estimate is statistically indistinguishable from the 0.33 lapses reported by the field study, which included drivers taking more than a two- night restart in the effect estimate. The results from the study and our tests are both statistically distinguishable from zero at the 0.10 level of confidence. Similarly, the KSS results reported in the field study for on- duty drivers are statistically indistinguishable to those we found when removing drivers taking more than two-night restarts from the treatment group. Our second sensitivity test estimated the difference in mean PVT and KSS outcomes between drivers with one-night restarts and two-or-more night restarts, separately by industry segments defined by local versus regional and over-the road operations. We attempted to estimate separate effects for the regional segment alone, but the study data did not include any regional drivers who took one-night restarts. As a result, we combined regional and over-the-road drivers, who may behave in similar ways such as driving longer distances and sleeping away from home with some regularity. Finally, we omitted effects for Periodij to simplify the model, since the estimated treatment effects did not vary when this variable was included or excluded. Thus, we estimated two alternatives: = xij Condition = Timingij = Operation = {I(Condition x Timingij x Operation)} {1 night restart, 2 night restart, 3+ nights restart} {12:00 – 4:00 AM, 4:00 – 8:00 AM, … , 8:00 – 11:59 PM } {Local, Regional or Over-the-Road} xij = Condition = Timingij = Operation = {I(Condition x Timingij x Operation)} {1 night restart, 2+ nights restart} {12:00 – 4:00 AM, 4:00 – 8:00 AM, … , 8:00 – 11:59 PM } {Local, Regional or Over-the-Road} Our test found that the field study did not sample enough drivers from each of these segments to estimate effects for each subgroup that were statistically distinguishable from zero (significant). We found that regional and over-the-road drivers had a significant difference of 0.42 PVT lapses and an insignificant difference of 0.13 KSS scores, on average, between drivers with a one-night and a two-or-more night restart (see table 8). In contrast, local drivers had insignificant differences of -0.15 PVT lapses and 0.26 KSS scores, on average. We found similar effects when comparing drivers with one-night versus two-night restarts. In sum, the variation in the results suggests that the 2011 HOS rule may have had variable effects on different segments of the industry. However, the relatively wide confidence intervals of these estimates, in part due to limited sample sizes within each industry segment, makes the exact degree of variation uncertain. To assess stakeholder concerns about the adequacy of the field study’s sample, we conducted a statistical power analysis to assess how many drivers and PVT measurements might be needed to estimate the HOS rule’s effects for each of several segments of the industry. In general, a power analysis estimates the probability that a study will conclude that an effect exists when, in fact, the effect does exist in the population of interest. In our application, we estimated the probability of identifying differences in mean PVT results for drivers taking a one-night versus two- night restart, separately for subpopulations of drivers operating in the local and regional or over-the-road industry segments and at various times of day. We found that the field study’s sample size was insufficient to estimate statistically significant differences in the primary fatigue measure—the PVT—for each of these industry segments and times. By collecting data from approximately 100 drivers, each taking 25 PVTs, the study would have had an approximately 5 percent or lower chance of identifying statistically significant differences as small as 0.3 PVT lapses for each subpopulation. This effect size is similar to what the study found across all time periods, using a sample of a similar size. In contrast, collecting data from approximately 4,000 to 6,000 drivers, each taking 200 PVTs, would have increased the chance of identifying significant differences for each subpopulation separately to approximately 40 to 60 percent. For an effect of 1 PVT lapse—slightly larger than what the study found for overnight observation periods—sample sizes of 800 drivers and 200 PVTs would have achieved approximately 60 to 80 percent power. These results illustrate the range of sample sizes that FMCSA would need to consider, subject to available resources for driver recruitment and study administration, in order to confidently detect effects of this size for each of several industry subpopulations. Below, we discuss the methods we used to conduct the power analysis and its results in more detail. The HOS study evaluated the effects of the rule on three measures of driver fatigue: PVT, KSS, and lane deviation. For the purpose of our power analysis, we focused on scores on the PVT due to the extensive scientific research that has validated this measure of fatigue. PVT scores consist of the number of lapses in attention a study participant experiences across a battery of tests. The study administered the PVT test at various times during the participation period, which spanned one- duty period, a restart period, and a second-duty period. Our power analysis focuses on the duty periods during which drivers would need to be alert in order to safely operate their commercial motor vehicle. Tables 9, 10, and 11 provide descriptive statistics about the study’s data collected during the duty periods from the 106 participating drivers. The number of PVTs varied across drivers from 8 to 53, with a median of 24.5 and a mean of 25.0. The study collected a total of 2,653 non-missing observations during the duty periods at the level of driver-tests, after excluding observations during the restart period. Across all observations, the mean number of PVT lapses was 1.8. PVT lapses had a skewed distribution, with a 90th quantile of 5, a 95th quantile of 8, and a 99th quantile of 18. (See table 9.) The study administered several hundred PVTs to drivers taking one, two, or three-or-more night restarts across various types of operations (e.g., local, regional, and over-the-road). (See table 10.) The study made fewer observations across subpopulations defined by restart nights and time period of observation, however, ranging from 44 to 274 tests per subpopulation (see table 11). This suggests that the three-way cross-classification of restart nights, operation type, and time period may produce limited sample sizes for each subpopulation. As we discuss above, the authors of the field study analyzed these data using a hierarchical linear model, with normally distributed random effects at the level of drivers but fixed effects otherwise. To analyze power, we used models similar to those we used above to estimate separate effects across industry segments. We used two separate methods to estimate power. These methods estimated power to detect effects for each of several subpopulations of drivers and for the overall population, respectively. Using the first method, we made distributional assumptions about the data generation process, informed by the field study’s results, and used Monte Carlo simulation methods and the actual model of the data to be estimated (from above). Using the second method, we made similar distributional assumptions, but we applied power equations for simplified versions of this model available in the statistical literature. Our analysis estimates power for estimating separate effects for each of 12 subpopulations defined by two industry segments (local versus regional/over-the-road) and six equally spaced observation periods (e.g., 12:00 AM to 4:00 AM). 𝑌�𝑖𝑗(𝑚) ~ 𝑁(𝐱�𝐢𝐣(𝐦)𝛃� + 𝜇�𝑗(𝑚),𝜎�𝑌2) 𝐱�𝐢𝐣(𝐦) ~ 𝑀𝑢𝑙𝑡𝑖𝑛𝑜𝑚𝑖𝑎𝑙(𝛉�) 𝜇�𝑗(𝑚) ~ 𝑁(𝜇̂,𝜎�𝜇2) For the mth simulated dataset, the estimates of interest are 𝛅(𝑚)T = [𝛽̂21(𝑚)− 𝛽̂11(𝑚),… ,𝛽̂2𝐾(𝑚) – 𝛽̂1𝐾(𝑚)], which, under the model, have entries equal to the difference in mean outcomes between drivers in subpopulation k having two versus one-night restarts. We use the set of M simulated values of δ to test composite hypotheses about effects for all subgroups, as well as multiple simple hypotheses about the effects for each subgroup. This implies that RTβ = δ = 0 under HLet VR = Var(RTβ) = RTVβR, where Vβ is the covariance matrix of β. For each simulated dataset, we test these hypotheses using the multivariate Wald test statistic, = 𝛃T𝐑𝐕𝐑−𝟏𝐑T𝛃 ~ 𝜒𝐾2 , with each simulation’s value equal to 𝐻(𝑚)= 𝛃�(𝑚)T 𝐑𝐕�𝐑(𝑚)−1 𝐑T𝛃�(𝑚). We reject H for simulation m if , the critical value of the chi-squared distribution with K 𝐻(𝑚)>𝜒𝐾,.952degrees of freedom, assuming α = .05. Since we assume that H) = E[I(Reject H in simulation m. We estimate power as �1−𝛽̂�= 1𝑀∑ I(𝐻(𝑚)>𝜒𝐾,.952 where I() is an indicator function ranging on {0,1}. that the inner product is the joint probability of rejecting all hypotheses. This is follows from the contrast weights in 𝐑𝐓, whose orthogonality implies independent comparisons. Sample and effect sizes similar to those in the field study provided low amounts of power to detect effects of -0.3 or -1 PVT lapses, as shown in table 12. The probability of detecting effects of this size for each of the 12 driver subpopulations was approximately 5 percent or less when sampling 100 drivers and administering 25 PVTs, using either a joint hypothesis test or multiple simple tests across subpopulations. Power increased to 0.11 when detecting an effect equal to -1 lapse using a joint test, but it remained approximately zero using multiple simple tests. These results confirm that the field study could not reliably estimate separate effects by type of driver and time of day. Our analysis found that increasing the field study’s sample size by several hundred additional drivers and PVTs would be necessary to achieve conventional levels of power, depending on the true effect size to be detected. Using a joint hypothesis test or multiple simple tests when the effect equaled -0.3 lapses, we found that about 6,000 drivers and 200 PVTs would be required to achieve power of 0.6 using a joint hypothesis test and 0.37 using multiple simple tests. To detect an effect of -1 lapse with power of approximately 0.6 to 0.8, samples of more than 800 drivers and 200 PVTs would be necessary. Since the field study reports effects in this range, the appropriate sample size would appear to fall between these bounds, subject to available resources for driver recruitment and study administration. Power generally increased more strongly with increases in the number of drivers, rather than the number of PVT tests. This result is consistent with the statistical literature on hierarchical linear models in which the marginal The moderately high effects of interest vary only across clusters.intraclass correlation of 0.68 in the study’s PVT data within drivers increases the required driver sample size to achieve a given level of power, all else being equal. However, the number of PVTs increases power more strongly when using multiple simple tests, because more PVTs increases the number of observations across time periods and therefore increases power for each test. To provide an alternative power estimate for the overall population of drivers, using less complex methods, we analyzed power for a simplified version of the model using results from the literature on hierarchical linear models. For this analysis, we calculated power for estimating one treatment effect 𝛿 as defined above, without assuming that the effect varied across subpopulations. This reduced the model to xij = {I(1 night restart), I(2+ night restart)}. We further assumed that the sample had the same (balanced) number of observations for each of the two treatment conditions. This simplified the model so that variance estimation, hypothesis testing, and power calculation became tractable using the closed-form methods developed by the literature on hierarchical linear models (largely by avoiding many covariates). The more realistic Monte Carlo analysis summarized above compensated for these simplifications, since using two different methods checked the robustness of results from either one. H0: 𝛽2 − 𝛽1 = 0 H1: 𝛽2 − 𝛽1≠0 E(𝑀𝑆𝑡𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡)=𝑁𝜎𝜇2+𝜎𝑌2+𝑁𝐽𝛿24 𝐹(1,𝐽−2,𝜆)=𝐸(𝑀𝑆𝑐𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛) similar to the Monte Carlo analysis above, and using various values for N We estimate F() using estimates of 𝜎𝜇2 and 𝜎𝑌2 from the prior study’s data, and J. We estimate power as (1−𝛽̂𝑘) = Pr(𝐹� > 𝐹.95) , where 𝐹.95 is the noncentrality parameter 𝜆= 𝜆̂, assuming α = .05. critical value of the F distribution with (1, J-2) degrees of freedom and Our analysis found that the field study had more power to detect overall effects than effects across multiple subgroups (see table 13). The field study’s approximate sample size of 100 drivers and an average of 25 PVTs had a 0.08 probability of detecting effects of -0.3 PVT lapses and a 0.42 probability of detecting effects of -1 PVT lapses for the overall population of drivers. Given these results, the study had low power to detect the overall effect of -0.33 lapses that it reported across all time periods, suggesting that the result may have been statistically unusual. Our analysis found that increasing the sample size to approximately 2,000 to 4,000 drivers and 200 PVTs would have been required to detect an effect of -0.3 PVT lapses with conventionally adequate levels of power. However, samples as small as 250 drivers and 25 PVTs would be sufficient to detect effects of -1 PVT lapse. As part of our review of the effects of the 2011 hours of service (HOS) rule, we were asked to identify the assumptions used by the Federal Motor Carrier Safety Administration (FMCSA) to estimate the costs and benefits of the rule. To do so, a GAO economist identified the key assumptions in the regulatory impact analysis (RIA) used by FMCSA to detail the costs and benefits of the 2011 HOS final rule. This expert developed an initial list of key assumptions after reading the RIA. We then categorized these assumptions into five groups: which drivers were affected, how drivers were affected, economic costs, safety benefits, and health benefits. We also had a sixth category for assumptions that did not clearly fit into one of the aforementioned categories. FMCSA officials reviewed our list of assumptions, offered some corrections, and ultimately agreed that the list of assumptions accurately represented the key assumptions used in the RIA. Below is a list of the assumptions we identified and FMCSA officials confirmed. These assumptions cover which drivers would be affected by the HOS rule and how they would be affected. The assumptions below also detail how operational changes (e.g. driver schedule changes) result in the economic costs and the safety and health benefits of the HOS rule that went into effect July 1, 2013. (See table 14). To identify possible effects of the 2011 hours of service (HOS) rule, including which drivers were affected and how those drivers were affected, we purchased 3 years of drivers’ schedule data from the American Transportation Research Institute (ATRI). The data contain information on daily on-duty hours for drivers that worked for 16 for-hire motor carriers from January 1, 2012, through December 31, 2014. Specifically, the dataset includes a driver identification number, the number of recorded on-duty hours for each driver for each day in the dataset, and the length of each restart the driver took. The 2012 dataset contains 57,096 unique driver records; the 2013 dataset contains 60,196 unique driver records, and the 2014 dataset contains 63,957 unique driver records. The data we used for our analysis of drivers’ schedule data are not representative of the motor carrier industry, and therefore, our findings based on these data cannot be generalized to all carriers in that population. The data we purchased was not filtered or processed by ATRI. We applied four data-quality filters to remove records that contained likely errors and to exclude drivers with minimal on-duty time that could potentially bias our results, specifically: We removed driver records with five or more null values in the on- duty-hours cells. This filter removed 40,622 records from the 2012 dataset, 42,496 records from the 2013 dataset, and 45,564 records from the 2014 dataset. We removed driver records with one or more instance of 18 or more hours of on-duty recorded in a single day. This filter removed 1,126 records from the 2012 dataset, 1,735 records from the 2013 dataset, and 1,520 records from the 2014 dataset. We removed driver records with less than 20 hours of total on-duty time logged during the year. This filter removed 32 records from the 2012 dataset, 34 records from the 2013 dataset, and 9 records from the 2014 dataset. We removed driver records that did not have a 0 value in the on-duty hours column on the day before a recorded restart with a value of 48 or greater. This filter removed 8 records from the 2012 dataset, 13 records from the 2013 dataset, and 61 records from the 2014 dataset. After applying these filters, there were 15,308 unique driver records in the 2012 dataset, 15,918 unique driver records in the 2013 dataset, and 16,803 unique driver records in the 2014 dataset. To compare our driver data to the Federal Motor Carrier Safety Administration’s (FMCSA) estimates of drivers’ weekly hours as well as to provide a meaningful denominator for hours worked (e.g. on-duty hours per work week), we defined and calculated a driver’s work week. As described below, we considered several specifications in our analysis, including how many days to include in a work week, how to calculate a work week, whether to use the driver or the work week as the unit of analysis that is compared over time, and how to categorize weekly on- duty time. As described in the background section, drivers may follow either a 70- hour over 8 days on-duty limit or a 60-hour over 7 days on-duty limit. According to ATRI, the majority of the drivers in the dataset we purchased likely follow the 70-hour on-duty limit. We calculated a work week using both 7 and 8 days to ensure we covered the range of possible work weeks. However, given that the majority of drivers in the dataset likely followed the 70-hour/8-day work week, we included these results in the body of this report. The number of days used to calculate a driver’s work week has an effect on the total and average hours worked. We used a fixed 7 day and fixed 8 day work week in the analysis presented below and in our report. Before making this decision, we considered two ways of calculating a driver’s work week: 1) a fixed work week and 2) a rolling work week. For drivers following the HOS rule, a work week is not based on a set week (e.g. Monday through Friday). Rather, drivers calculate their 60- or 70-hour on-duty limits over a rolling period of 7 or 8 consecutive days. To take this into account, we calculated a fixed work week and a rolling work week to approximate the weekly on- duty hours of drivers in our dataset. The fixed work week has 7 or 8 consecutive days that do not overlap. For example in a 90 day period, there are 12 7-day weeks and 11 8-day weeks. The rolling work week also has 7 or 8 consecutive days, but each subsequent week shifts the start of the week by only one day. Each rolling week overlaps with the previous week. We tested these two calculation methods on a subset of data (January 1 to March 31, 2013, and 2014), and we found that the method we used to calculate a work week did not change the results of our analysis. To examine the assumptions and estimates FMCSA made in its regulatory impact analysis (RIA) for the 2011 rule, we categorized our results using weekly on-duty hour categories similar to those included in that document.following parameters: FMCSA defined four categories of drivers with the “Moderate”—average weekly work time of 45 hours, includes weekly averages between 20 to 55 hours “High”—average weekly work time of 60 hours, includes weekly averages greater than 55 to 65 hours “Very High”—average weekly work time of 70 hours, includes weekly averages greater than 65 to 75 hours “Extreme”—average weekly work time of 80 hours, includes weekly averages greater than 75 hours In our analysis, we split the Moderate category into two groups: those working less than 35 hours per week and those working more than 35 hours per week. We did this split to determine whether the behavior changes made by those at the low end of the Moderate category differed from those at the high end of the Moderate category. As a result, our analysis uses five weekly on-duty categories. To analyze the data, we considered two different units of analysis: a driver and an individual work week. Specifically, for the time periods we examined, we determined the number of drivers whose average work weeks fell into each of the on-duty hour categories described above. For example, a driver that averaged 57 on-duty hours per 8-day work week would be placed in the greater than 55- to 65-hour on-duty category. We also analyzed the data by the number of work weeks that fell into each of the on-duty categories. For example, we identified 342,747 out of 959,529 total weeks with on-duty hours between 35 and 55 from January 1, 2012, to June 15, 2013 (see table 24 below). To analyze drivers’ use of restarts within our dataset, we followed the practice of both FMCSA and ATRI of excluding restarts longer than 72 hours (3 days). The rationale for this practice is that off-duty periods greater than 72 hours do not represent a normal or operational restart period by a commercial motor vehicle driver. While we provide descriptive information on all restarts in our dataset (see table below), we exclude restarts greater than 72.99 hours from our analysis of restart use by driver or driver groups (see tables 20, 21, and 25 below). We analyzed driver schedule data in two different ways. In the first section, we describe our analysis of the data by dividing the 3 years of data into: 1) seasonal datasets by creating 12 individual datasets composed of 75- to 90-day seasons and 2) before and after the rule datasets by creating two datasets of approximately 18 months before and after July 1, 2013. In the second section, we describe our statistical analysis of drivers’ schedule data using a subset of the full dataset that covered the entire time frame—2012 to 2014—but were limited to drivers for whom we had data in all 3 years. We divided each year of data into four seasons: winter, spring, summer, and fall. The spring, summer, and fall seasons were truncated to account for (1) the effective date of the 2011 HOS rule on July 1, 2013, and (2) the suspension of two of the restart provisions in the 2011 HOS rule. Specifically, data from June 16 to June 30 were removed to ensure that behavior changes in anticipation of the rule’s going into effect did not bias the dataset. Similarly, data from July 1 to July 14 were removed to account for drivers’ getting used to and understanding the rule. We also omitted data from December 16 through December 31 to account for the suspension of several restart provisions on December 16, 2014. To ensure we were comparing the same time periods in each season, all of these changes were applied across each year of data. (See table 15.) Dividing the data into seasons allowed us to isolate changes potentially due to seasonal variation and to show how drivers’ behavior—averaged over a relatively short period of time (75 to 90 days)—changed. For example, several motor carriers and drivers told us that bad weather in January and February 2014 disrupted operations for many motor carriers. By separating the datasets into seasons, we were able to ensure that changes we observed from one period to another were consistent across all seasons and were not only due to seasonal variation. We found that the percentage of drivers in our dataset working more than 55 hours per work week decreased after the 2011 HOS rule went into effect—between spring 2013 and summer 2013. In addition, our analysis suggests that the percentage of drivers working 55 or fewer hours per work week increased after the rule went into effect. Tables 16 through 22 show the results of our seasonal analysis of drivers’ schedule data using an 8-day work week and a 7-day work week. Consistent with our results using drivers as the unit of analysis, the overall percentage of weeks from our dataset in the more-than-55-hours- per-week categories decreased after the 2011 HOS rule went into effect. The overall percentage of weeks in the 55-or-fewer-hours categories increased after the rule went into effect. In both tables 18 and 19, the percentage of weeks with total on-duty hours greater than 65 hours per work week is higher than those found in tables 16 and 17. This finding suggests that while a driver may have an intense work schedule over one week, on average, drivers tend to have less intense work schedules over longer periods of time. Tables 18 and 19 show the total number of weeks that fell into our on-duty categories for drivers in our dataset using an 8- and 7-day work week. We found that restarts per driver remained relatively constant before and after the 2011 HOS rule went into effect for drivers in our dataset working between 35 to 75 hours per 8-day work week. We also found an increase of approximately one restart per driver for those working more than 75 hours per 8-day work week and a reduction of approximately one restart per driver for drivers working less than 35 hours per 8-day work week. Table 20 compares the restart use per driver across the on-duty hour categories. We also analyzed restart use per driver per calendar week to understand how often drivers in our dataset use a restart. With this specification, we found that restart use after the 2011 HOS rule went into effect appears to vary by drivers’ average weekly on-duty hours (see table 21). For drivers working on average more than 55 hours to 75 hours per 8-day work week, restart use per calendar week remained relatively constant before and after the 2011 HOS rule went into effect. For drivers working more than 75 hours per 8-day work week, restart use per calendar week appears to increase by almost 1. However, before the rule there were 69 drivers in this category and after the rule there were 15 drivers, therefore the change observed in Table 20 is based on a relatively small number of drivers. For drivers working 55 hours or less per 8-day work week, restart use per calendar week appears to decrease after the 2011 HOS rule went into effect. We analyzed the length of restart periods to understand how many off- duty hours drivers in our dataset took. We found that before the 2011 HOS rule went into effect, about 8 percent of the restarts taken by drivers in our dataset were between 34 and 37 hours (see table 22). As a percentage of all restarts taken in each season, restarts with lengths of 34 to 37 hours decreased after the rule went into effect. The percentage of restarts with a length between 45 to 72.99 hours increased after the rule went into effect. As our dataset did not contain the start and end times of a driver’s schedule or restarts, we are not able to analyze the extent to which there was a change in restarts with one- or two-night periods. For example, a driver working during daytime hours could take a 34 hour off- duty period that contains two nighttime periods (1 a.m. to 5 a.m.). In contrast, a driver working during nighttime hours may not have two nighttime periods (1 a.m. to 5 a.m.) when taking a 34-hour off-duty period. Therefore, we cannot determine the change in actual nights taken off-duty by looking at the length of a restart period on its own. Analysis of Data before and after the HOS Rule We also analyzed the data by dividing driver records into two periods: (1) before the rule went into effect—January 1, 2012, to June 15, 2013—and (2) after the rule went into effect—July 15, 2013, to December 15, 2014. As with the analysis by seasonal datasets, we applied filters to each year of data. To avoid double-counting drivers who appear across multiple years of data, we used the driver identification number to connect driver records with data in multiple years. The weekly on-duty hour average is calculated using the weeks that we have data for a driver. If a driver is only present in the 2012 data set, then that driver’s average weekly on- duty hours are calculated based on only 2012 data. If a driver is present in both the 2012 and 2013 datasets, then that driver’s average weekly on- duty hours are calculated using 2012 and 2013 (up to June 15, 2013) on- duty hours. The calculation for the post-rule dataset used the same method. We compared the number and percentage of drivers in our dataset by average weekly on-duty hours before the rule went into effect and after the rule went into effect. As with the seasonal analysis, the percentage of drivers in our dataset working more than 55 hours per work week decreased, and the percentage of drivers in our dataset working 55 hours or less per 8-day work week increased. While this analysis shows similar trends in the data, there are differences in the number and percentage of drivers in each weekly on-duty hour group. Specifically, in the seasonal analysis, before the rule went into effect (winter 2012 through spring 2013) the percentage of drivers in our dataset working more than 65 hours per 8-day work week ranges from about 9 to close to 12 percent. As shown in table 23, approximately 8 percent of drivers in our dataset before the rule went into effect are categorized as working more than 65 hours per 8-day work week on average. This difference is likely due to the time frame over which the average weekly on-duty hours are calculated. We also compared the number and percent of weeks before and after the rule went into effect categorized by total weekly on-duty hours (see table 24). Consistent with the seasonal analysis, the percent of 8-day work weeks with total on-duty hours of more than 65 also decreased. We also analyzed the length of restarts used by drivers’ average weekly on-duty time. We found that before the rule went into effect the percentage of 34- to 36.99-hour restarts taken by drivers in our dataset in each on-duty time category ranges from about 6 to 11-percent (see table 25). After the HOS rule went into effect, the relative distribution of restarts by length taken in each weekly on-duty category changed. Specifically, the percentage of restarts lasting 34- to 36.99- hours decreased for all driver categories. The percentage of 34- to 36.99-hour restarts taken by drivers in our dataset working the longest hours decreased from close to 11 percent to about 9 percent. The percentage of 34- to 36.99-hour restarts taken by the other driver groups saw larger decreases. As with our seasonal analysis of restart length, our results do not allow us to state that the number of night time periods in a restart period changed after the 2011 HOS rule went into effect. Statistical Analysis of Driver Schedule Data We also analyzed whether there were differences in how the same set of drivers behaved before and after the 2011 HOS rule went into effect. To do this analysis, we developed several statistical regression models of hours worked and restart use, in order to further account for seasonal variation and differences across drivers who do not change over time, such as carrier. Using these models, we estimated whether the differences in hours worked and restart use before and after the 2011 HOS rule went into effect were statistically distinguishable from zero. We applied these models to a sample of 6,934 drivers who met our filters for the entire period between January 1, 2012, and December 31, 2014. We estimated two sets of models, which made different assumptions about the distribution of the outcomes and the structure of the correlation of those outcomes within drivers over time. Both sets of models included season or month fixed effects and predicted hours worked per week; working 35 to 54, 55 to 64, or 65 or more hours per 8-day week; or the number of restarts per week. First, we estimated linear models with driver fixed effects (implemented through a within-transformation) and standard errors that were robust to arbitrary forms of heteroskedasticity and driver-level autocorrelation. These adjustments corrected for heteroskedasticity caused by using linear models to analyze binary outcomes and for autocorrelation caused by analyzing repeated outcomes within drivers over time. Second, we estimated generalized estimating equations (GEE) models, with outcomes within drivers having exchangeable or AR1 covariance structures. These covariance models were appropriate for the 3-year period, when the autocorrelation of driving schedules should be relatively consistent across days (exchangeable) or decline linearly (AR1). The models assumed the outcomes to be distributed normally, binomially, or negative binomially, depending on the scale, and used canonical link functions. We summarize the results of our analysis as ranges across models in the estimated contrast between time periods before and after the policy change. Most of our findings were robust to these plausible model assumptions. Drivers in our analysis sample worked approximately 1.1 to 2.5 fewer hours per 8-day week, on average, after the HOS rule was implemented than before, depending on model assumptions. Expressed as a proportion, the differences ranged from approximately 2.0 to 4.8 percent fewer hours. These differences were statistically distinguishable from zero at the 0.001 level. Drivers in our analysis sample were approximately 24 to 29 percent less likely to work 65 hours or more per 8-day work week after the HOS rule Similarly, drivers was implemented, depending on model assumptions.were approximately 12 percent more likely to work 35 to 54 hours per 8- day work week. All of these differences were statistically distinguishable from zero at the 0.001 level. The likelihood of working 55 to 64 hours per 8-day work week after the rule went into effect changed by less than 2 percent but the statistical significance and direction of the change varied according to model assumptions. Drivers took approximately 6.1 to 6.5 percent fewer restarts per 8-day week, on average, after the HOS rule was implemented than before, depending on model assumptions and adjusting for variation in driving intensity (as a measure of exposure). These differences for the overall analysis population of drivers were statistically distinguishable from zero at the 0.001 level. GAO was asked to assess the safety impacts of the 2011 hours of service (HOS) rule, which are predicated on reducing driver fatigue. To assess whether the rule change could result in less fatigued drivers and potentially fewer fatigue-related crashes, we used a biomathematical fatigue model—the Fatigue Audit InterDyne™ (FAID) model—to assess the risk of driver fatigue for schedules that comply with the 2011 HOS rule and similar schedules that do not. The FAID model provides a fatigue score based on the start and end time of a work shift. The higher the FAID score, the higher the risk of fatigue. A standard work week of 40 hours, Monday to Friday, 9 a.m. to 5 p.m. results in a peak FAID score of 41. In contrast, a 40-hour work week from 11 p.m. to 7 a.m. results in a peak FAID score of 97. Research indicates that scores above 80 are comparable to the fatigue-related impairment found in individuals with a blood alcohol level of over 0.05 percent, above the legal limit in many countries. those based on hypothetical driver schedules and those based on interviews with motor carriers, differ significantly from a typical 9 a.m. to 5 p.m. 40-hour work week. The examples described below, both The schedules we modelled below often involve working long hours—60 or more hours per week—and working overnight. As a result, many of the peak fatigue scores shown below are well above the range considered safe. The purpose of our analysis is to show the relative difference in peak fatigue scores for schedules in operation after the rule went into effect and those in operation before the HOS rule went into effect. A. Fletcher, N. Lamond, C. van den Heuvel, and D. Dawson, “Predication of performance during sleep deprivation and alcohol intoxication by a quantitative model of work-related fatigue,” Sleep Research Online, 5(2) (2003). The study found that a fatigue score of 80 is comparable to the impairment that would be observed in an individual with a blood alcohol concentration of 0.09 percent or greater. Two-night provision (two 1 a.m. to 5 a.m. periods in a restart) 168-hour limit (limits restarts to once every 168 hours) We also created scenarios based on interviews with motor carriers and drivers. These interview-based scenarios include the schedules interviewees reported using before the 2011 HOS rule went into effect and schedules interviewees said they started using in order to comply with the 2011 HOS rule. These schedules demonstrate real world adaptations to the 2011 HOS rule as described to us. Our analysis is not generalizable to the entire industry, but rather is intended to illustrate how HOS schedule changes can affect the risk of driver fatigue. Of the schedules we simulated, some did not require a change after the rule went into effect. For those that did require a schedule change, figures 8 through 11 below show the daily peak fatigue score when complying with the two-night provision and 168-hour limit in the 2011 HOS rule and the daily peak fatigue score for similar schedules that do not comply with one or both of those provisions in the 2011 HOS rule. Drivers can work up to 14 hours per day until reaching 70 hours—at which point they must either take a restart of at least 34 hours or remain off-duty until their consecutive 8-day sum of hours is less than 70. Therefore, we modelled a maximum daytime schedule with the following parameters: 14-hour shifts on-duty from 6 a.m. to 8 p.m. over five consecutive Under the previous HOS rule, a driver following this schedule would only need to take 34 hours off-duty to have a valid restart. However, after the rule went into effect and to comply with the 168-hour limit, a driver with this schedule must take 2 days off between work cycles. For example, the driver can work 14 hours between 6 a.m. and 8 p.m. on Monday through Friday, but must be off-duty Saturday and Sunday. This results in 58 hours off-duty. This schedule change—2 days off between work cycles instead of one day off—effectively lowers the average number of hours a driver can work. In terms of work time per 7-day period, the schedule that complies with the 168-hour limit averages 72 hours and the schedule that does not comply averages 82 hours. As shown in figure 8, the peak daily fatigue scores for the schedule that complies with the 2011 HOS rule (the schedule with 2 days off) are lower than the peak daily fatigue scores for the schedule that does not comply with the 2011 HOS rule. The peak daily fatigue scores for both of these schedules are high and above the levels generally considered safe on several days that are modelled. We also modelled a driver working the maximum allowed hours but during nighttime hours. The hypothetical schedule had the following parameters: 14-hour shifts on-duty from 8 p.m. to 10 a.m. over 5 consecutive days Under the previous HOS rule, the driver only had to take off 34 hours between the end of the work shift at 10 a.m. on Saturday and the start of the next work shift at 8 p.m. on Sunday, but this schedule violates both the two-night provision and the 168-hour limit because the off-duty period does not contain two periods from 1 a.m. to 5 a.m. and begins less than 168 hours after the prior restart period began. Thus, to comply with both the two-night provision and the168-hour limit in the 2011 HOS rule, a driver with this schedule must take 2 days off between work cycles. For example, the driver can work a 14-hour shift between 8 p.m. and 10 a.m. on Monday through Friday nights, but must be off-duty Saturday and Sunday nights to comply with the 2011 HOS rule. This results in an off- duty period of 58 hours. As shown in figure 9, the peak daily fatigue scores for the schedule used after the rule went into effect (the schedule with two days off) are lower than the peak daily fatigue scores for the schedule in use before the rule went into effect. The peak daily fatigue scores for both of these schedules are high, above the levels generally considered safe. Of the schedules we simulated, two additional hypothetical schedules needed to change to comply with the 2011 HOS rule. These hypothetical schedules had the following parameters: 10- to 12-hour on-duty shifts between 10 p.m. to 10 a.m. over 6 consecutive days (70 hours over a 6-day work cycle) 10-hour on-duty shifts from 12 a.m. to 10 a.m. over 6 consecutive days (60 hours over a 6-day work cycle) To comply with both the two-night provision and the168-hour limit in the 2011 HOS rule, drivers with these schedules must take 2 days off between work cycles instead of one day off as allowed under the prior HOS rule. For example, a driver can work a 10-hour shift between 12 a.m. and 10 a.m. on six consecutive nights but must be off-duty the next two nights. These two consecutive nights off allow this schedule to comply with the two-night provision, which requires two 1 a.m. to 5 a.m. periods in an off-duty period to qualify as a restart. As a result, a driver working 10-hour shifts on six consecutive nights (60 hours in a work cycle) must take an off-duty period of 52 hours. A driver working 10-to-12- hour shifts on six consecutive nights (70 hours in a work cycle) must take an off-duty period of 62 hours. As shown in figures 10 and 11, the peak daily fatigue scores for the schedules in use after the rule went into effect (the schedules with 2 days off) are lower than the peak daily fatigue scores for the schedules in use before the rule went into effect. The peak daily fatigue scores for the schedules shown in figures 10 and 11 are high, above the levels generally considered safe. We also used the FAID model to estimate the risk of driver fatigue based on schedule changes described in our interviews with an industry stakeholder, motor carriers, and drivers. Figures 12 through 15 below show the daily peak-fatigue score for the schedules interviewees adopted in order to comply with the 2011 HOS rule and the daily peak-fatigue score for the schedules they used before the 2011 HOS rule went into effect. Representatives of a motor carrier and an industry association we spoke with reported changing the schedules of drivers working over-night or in the early morning hours to ensure that drivers could continue to take the restart and comply with the two-night provision in the 2011 HOS rule. In the first example, drivers worked six consecutive10-hour shifts per week from 12:30 a.m. to 10:30 a.m. before the rule went into effect. Over 7 days, their schedules averaged 61 on-duty hours. These drivers generally began and ended their shifts at the same time each day. After the rule went into effect, the motor carrier reduced the number of shifts per driver from six consecutive shifts per week to five consecutive shifts per week. This change was made to ensure drivers would have two 1 a.m. to 5 a.m. periods during their off-duty period and could still count the off-duty time as a restart. In the second example, drivers worked 10-hour shifts from 3:00 a.m. to 1:00 p.m. over six consecutive days. Similar to the first example, drivers working this schedule averaged 61 hours on-duty over 7 days. To ensure drivers could take a restart that complied with the two- night provision, the motor carrier reduced the total number of shifts per driver from six consecutive days to five consecutive days. As shown in figures 12 and 13, the schedules that comply with the 2011 HOS rule have lower peak-fatigue scores for each day a driver works this schedule. The lower peak-fatigue scores suggest a lower risk of fatigue for a driver working according to the schedule complying with the 2011 HOS rule than the driver working according to the schedule that does not comply with the rule. The peak daily-fatigue scores for the schedules shown in figures 12 and 13 are high, above the levels generally considered safe on many of the days modelled. Representatives of other motor carriers we spoke with described schedule changes they made to ensure off-duty time complied with the 168-hour limit. For example, a representative of one motor carrier told us that a driver would typically work a long-haul route over 10 days. The driver would work between 10- and 14-hours per day for 5 days, reach the drop-off destination and go off-duty for 34 hours (a restart). Then the driver would drive home. Once home, the driver would take 3 days off which would count as a restart. To comply with the 168-hour limit and ensure the off-duty period would count as a restart, this schedule would have to change to ensure the off-duty periods began at least 7 days apart (168 hours). To do this, the driver could work 10- to 12-hour days for 6 days, take a 34-hour break on day 7, then drive home and take a shorter break, 2 days in this example. As shown in figure 14, complying with the 168-hour limit results in generally lower peak fatigue scores and therefore a lower risk of driver fatigue. In the second example, representatives of a motor carrier told us about short-haul drivers working overnight shifts whose schedules were changed to comply with the 168-hour limit. Before the rule went into effect, drivers could choose between two possible schedules. The first schedule (schedule A) had drivers working 12-hour shifts for 5 consecutive days, in our simulation from 8 p.m. to 8 a.m. Then drivers took the next 2 days off-duty and counted this time as a restart. The second schedule option (schedule B) had drivers working 13-hour shifts for 4 consecutive days, in our simulation from 8 p.m. to 9 a.m. These drivers then took the next 3 days off-duty and counted this time as a restart. Due to the 168-hour limit, the motor carrier switched drivers to the following schedule (schedule C): 12-hour shifts for 5 consecutive days with 2 days off followed by 12-hour shifts for 5 consecutive days with 3 days off. We compared the two schedules offered before the rule went into effect to the schedule offered after the rule was effective. As shown in figure 15, we found that schedule C, which complied with the 2011 HOS rule, had similar peak fatigue scores to Schedule A with the 5 days on- duty with 2 days off-duty schedule. Schedule C, the 2011 HOS compliant schedule, had slightly higher peak-fatigue scores than Schedule B, the 4 days on duty with 3-days off-duty schedule. This suggests that the schedule changes made by this carrier to comply with the 168-hour limit did not consistently lower the risk of fatigue faced by drivers working this schedule. As was discussed in the body of our report, on July 1, 2013, the Federal Motor Carrier Safety Administration (FMCSA) began to enforce the 2011 hours of service (HOS) rule that made several key changes to the number of hours commercial drivers can work and drive per day and week. Specifically, the new rule altered when and how commercial drivers are permitted to “restart” the maximum number of hours they can work over a 7- or 8-day period, and required drivers to take a 30-minute, off-duty break during shifts that last more than 8 hours. FMCSA understood that when it designed this rule, it would result in changes to drivers’ schedules. Specifically, FMCSA believed the rule would reduce driving time for drivers working 65 or more hours per week, but assumed that these hours would be shifted to other drivers or to other workdays rather than being eliminated altogether. While FMCSA asserted that total driving time for some individual drivers was likely to drop slightly due to the HOS rule, no attempt was made to quantify the effects of the rule on congestion. Stakeholders we spoke with were concerned that FMCSA did not adequately assess how the new HOS rule would impact traffic patterns, especially whether the rule would result in increased traffic congestion during morning hours, i.e., between 5 a.m. and 9 a.m. To evaluate changes in commercial vehicle traffic before and after the 2011 HOS rule went into effect on July 1, 2013, we collected and analyzed data from the Federal Highway Administration’s (FHWA) Weigh- in-Motion (WIM) and Classification database. This database includes information on non-commodity-carrying vehicles, including passenger vehicles, and commodity-carrying (i.e., commercial) motor vehicles through data sensors that are installed in roadways. These sensors allow states and FHWA to collect data on time and date, lane, speed, vehicle classification, and vehicle length, among other variables. Our analysis of this data was limited to two time periods before and after the rule went into effect. Specifically, we chose to evaluate data from November through December 2012 and November through December 2013 for the following reasons: We wanted to allow sufficient time for carriers to adapt to the new rule, including preparing for the change prior to July 1, 2013 and adapting to the change after July 1, 2013. We wanted to control for any potential seasonal effects within the commercial carrier industry by including the same months prior to and after the HOS rule went into effect. We wanted to control for unusual events that could have impacted the commercial motor vehicle industry. For example, we excluded the time period from January and February 2014, which according to motor carriers and drivers we spoke with, had a large number of winter storms that severely impacted the industry. We also excluded the end of October 2012 to avoid capturing potential impacts on the industry from Superstorm Sandy. We worked with FHWA officials to provide us with WIM data that fell into these timeframes. In order to ensure we were comparing data from similar locations between the two time periods we requested data only from stations that reported data for all days and hours between November through December 2012 and November through December 2013. Because not all stations reported data for every day or hour during these timeframes, our analysis was restricted to 324 stations from 14 states out of 987 total stations across the country (approximately 33 percent). (See figure 16). We also asked FHWA officials to filter the data for us into variables most useful for our analysis. For example, although WIM data include information on direction of travel and specific lane of travel, we were primarily interested in total vehicle counts at each station, to capture traffic volume. As a result we requested vehicle count totals for each station by month, day, and hour. In addition, FHWA grouped the data into two distinct categories: (1) non-commodity-carrying vehicles, and (2) commodity-carrying vehicles. We analyzed these data by combining individual station data and sorting them by hour of day before and after the HOS rule went into effect. We did this analysis for all days of the week, weekdays, and weekends to account for any change in traffic patterns caused by time of week. We also compared traffic patterns for non-commodity-carrying vehicles and commodity-carrying vehicles to determine whether one of those populations was behaving differently than the other and account for possible factors that might be influencing both groups, such as the economy. The Federal Motor Carrier Safety Administration (FMCSA) promulgated the 2011 hours of service (HOS) rule on the basis that it would improve safety due to a reduction in fatigue-related crashes. At the same time, however, some industry stakeholders, motor carriers, and drivers that we spoke with believe that the HOS rule has had some unintended effects that may actually lead to a decrease in safety, particularly because they anticipate increased congestion during certain early morning hours when roads tend to be congested. To analyze the safety effects of the 2011 HOS rule, we undertook two separate analyses to (1) assess whether there has been any apparent effect of the 2011 HOS rule on the numbers of motor carrier crashes and (2) whether the rule change appeared to affect the relative number of crashes that occurred after the rule’s implementation between 5 a.m. and 9 a.m. We used monthly crash data from the Motor Carrier Management Information System (MCMIS) from 2008 through 2014, supplied by FMCSA, to address the first question, and individual crash data from the same source to address the second. Table 26 shows information for each month from January 2008 to September 2014, on the numbers of total crashes involving motor carriers, as well as the number of crashes with injuries and the numbers of fatal crashes. As can be seen, the numbers of crashes show a good deal of variation over time, but there appears little by way of any patterned variation, or trend. For example, the monthly numbers of total crashes averaged 7,042, but ranged from a low of 5,593 in February 2009 to a high of 9,887 in January 2014. The monthly numbers of crashes with injuries and fatal crashes also showed a good deal of variation. Many potential factors can influence the number of crashes over time. For example, as the economy grows, it is reasonable to expect more goods and services are traded, resulting in more freight needing to be transported. Given the size of roads is unlikely to change much in the near term, more traffic may result in more congestion, and more congestion may result in more accidents. To assess whether the implementation of the 2011 HOS rule had an effect on crashes we developed two ordinary least squares regression models that controlled for trucking volume and seasonal variation, and in one case also controlled for the unusual winter weather in December 2013 to February 2014. (See table 27.) Model 1, fit separately for each of the three categories of crashes, regressed the numbers of crashes on 1) three quarterly dummy variables to assess for seasonal differences in the numbers of crashes across the four quarters of the year, 2) a dummy variable (denoted “Rule Period”) to contrast the numbers of crashes in the months following the rule change with the numbers of crashes in the months preceding it, and 3) a linear covariate measuring “trucking gross output,” which is a quarterly measure used as a proxy for the numbers of motor carriers on the road and at risk of crashing. The trucking gross output variable had a significant and positive relationship with the monthly numbers of all three groups of crashes (all crashes, injury crashes, and fatal crashes), and seasonal differences, as measured by quarter, were generally significant for total crashes and fatal crashes, but not for crashes with injuries. In model 1, the apparent effect of the rule change on crashes was significant and positive for total crashes, insignificant for crashes with injuries, and significant and negative for fatal crashes. However, our discussions with industry participants indicated that the winter from December 2013 through February 2014 was unusually harsh and made operations very difficult for the industry. These weather conditions may have been a contributing factor, irrespective of the rule change, to a short term rise in crashes during this time period. Because of this issue, we also tested an alternative iteration of this model: Model 2. Model 2 estimates all these same effects as the first model for each of the three groups of crashes but includes an additional dummy variable to account for any independent effect on the numbers of crashes due to the unusually disruptive winter weather from December 2013 to February 2014. Adding this dummy variable had no effect on our estimate of the effect of the rule change on crashes with injuries, which remained insignificant, or in fatal injuries, which remained significant and negative. It did, however, reduce the size of the estimated impact of the rule change for total crashes and rendered the effect of the rule change on total crashes insignificant. As noted above, we were told by stakeholders that the two-nighttime provision of the HOS rule would potentially result in an increased traffic involving large trucks between 5:00 a.m. and 9 a.m. Presumably, because of this provision, drivers taking a restart would have to wait until at least 5:00 a.m. to begin their day when roads are more congested. Using individual crash data from MCMIS, we also investigated whether, among all crashes, there was any effect of the rule change on the likelihood of crashes occurring between 5 a.m. and 9 a.m. versus any other time of the day. For each of the three sets of crashes, we first examined simple two-way cross-classifications (shown in table 28) and fit a simple bivariate logistic regression model that regressed the logarithm of the odds on crashes occurring between 5 a.m. and 9 a.m. on a dummy variable denoted as “Rule Period” that contrasted crashes that occurred before and after the rule change. Table 28 provides information on the numbers and percentages of total crashes, crashes with injuries, and fatal crashes that occurred between 5 a.m. and 9 a.m., and at all other times of the day, before and after the rule change. As can be seen, between 19 percent and 20 percent of crashes occurred between 5 a.m. and 9 a.m., both before and after the rule change. The value of the likelihood-ratio chi-square, shown at the base of each table, indicates that there was no statistically significant difference in the occurrence of crashes in the 5 a.m. to 9 a.m. time. Table 28. Numbers of Total Crashes, Crashes with Injuries, and Fatal Crashes That Occurred between 5 a.m. and 9 a.m. and at Other Times, before and after the Rule Change Time of crash (total crashes) Likelihood ratio chi-square = 0.014 with 1df, P - 0.91 Time of crash (crashes with injuries) Likelihood ratio chi-square = 0.637 with 1df, P - 0.43 Time of crash (fatal crashes) Also shown in table 28 is an alternative method of estimating the likelihood of crashes occurring between 5 a.m. and 9 a.m. by calculating the differences in the likelihoods of crashes in the 5 a.m. to 9 a.m. time frame and calculating odds and odds ratios, which are shown in the last two columns of table 28. The odds on crashes occurring between 5 a.m. and 9 a.m. are calculated by taking the number (or percentage) of crashes that occurred in that interval and dividing it by the number (or percentage) of crashes that occurred at other times. With respect to total crashes, for example, the odds on crashes occurring between 5 a.m. and 9 a.m. before the rule change were 90,769 ÷ 364,866 = .2488, and the odds of crashes occurring between 5 a.m. and 9 a.m. after the rule change were 22,880 ÷ 91,882 = .2490. While somewhat different and less traditional than percentages, the odds have a fairly direct and simple interpretation—in this case they imply that in both periods there were roughly 25 crashes in that interval for every 100 that occurred at all other times of the day. The odds of crashes occurring between 5 a.m. and 9 a.m. are very similar for crashes with injuries and fatal crashes. The odds ratios in the final column are calculated by taking the odds of crashes occurring in the designated time frame after the rule change and dividing by the odds of crashes at that time before the rule change. For total crashes, for example, the odds ratio is calculated as follows: (.2490 ÷ 0.2488) = 1.001. This ratio is only slightly different than 1, indicating that the likelihood of crashes occurring between 5 a.m. and 9 a.m. was nearly identical before and after the rule. The odds ratios for crashes with injuries and fatal crashes produce similar conclusions. One advantage of odds ratios is that unlike percentage differences, they can be adjusted using multivariate models (logistic regression models) so that they reflect the net effect of the rule change, after adjusting for other characteristics that might have differed, in this case, before and after the rule change. Table 29 shows that results of fitting bivariate (or unadjusted) logistic regression models (in the first row), and the multivariate regression models (in the remaining rows) to estimate the effect of the rule change on the likelihood of crashes occurring between 5 a.m. and 9 a.m. for total crashes, crashes with injuries, and fatal crashes. The bivariate, or unadjusted models, reproduce the same odds ratios we derived from the observed data in the two-way tables in table 28. We then fit multivariate models that regressed those same odds on the rule period variable while simultaneously controlling for a number of potentially confounding factors, including road, weather, and lighting conditions, trucking gross volume, and whether the crash occurred during a weekday or on the weekend. The coefficients for the multivariate model shown in table 29 are exponentiated odds ratios, which indicate the size and significance of the differences in the odds on crashes occurring between 5 a.m. and 9 a.m. as opposed to some other time of the day, both as a result of the rule change and as a result of the factors included in the model. The multivariate models reveal that a number of the control variables affected the likelihood of crashes occurring from 5 a.m. to 9 a.m., including different road, weather, and light conditions. Crashes were more likely to occur between 5 a.m. and 9 a.m., for example, when roads were icy rather than dry (by factors ranging from 2.2 for total crashes to 3.6 for fatal crashes), when conditions were foggy rather than clear (by factors ranging from 4.4 for total crashes to 5.0 for crashes with injuries and fatal crashes), and during dawn rather than daylight hours (by factors exceeding 40 in all cases). However, even with these factors controlled, the rule change appeared to have no significant effect on the likelihood of total crashes occurring between 5 a.m. and 9 a.m. Odds ratios in the multivariate models estimating the differences in the likelihood of crashes occurring before and after the rule change were very nearly 1.0, and ranged from 0.999 for total crashes to 1.025 for fatal crashes. As previously discussed, there are many potential factors that could influence the number of crashes in any given month, including weather and road conditions at the time of a crash. Our analyses take into account some of these factors by controlling or adjusting for seasonal differences and freight volume, but there are undoubtedly other potentially confounding factors, and we would have liked to have had a more direct measure of the numbers of motor carriers on the road in each month and at risk of crashing. Moreover, our dataset only included 15 data points, or months, since the rule went into effect, and neither the significant or insignificant changes we find in this short term are guaranteed in the longer term. Without additional data over a longer period of time, we are unable to robustly determine whether the HOS rule had an impact on crashes. In addition to the individual named above, H. Brandon Haller, Assistant Director; Amy Abramowitz; Sarah Arnett; Russell Burnett; Matthew Cook; Melinda Cordero; Leia Dickerson; Colin Fallon; David Hooper; Hannah Laufe; Ethan Levy; Grant Mallie; Joshua Ormond; Anna Maria Ortiz; Jerry Sandau; Doug Sloane; and Jeff Tessin made key contributions to this report.

Summary: FMCSA—within the Department of Transportation (DOT)—issues rules to address safety concerns of the motor carrier industry, including on truck drivers' HOS. In July 2013, FMCSA began to enforce three new provisions of its HOS rule. GAO was asked to review a 2014 FMCSA study on the rule, as well as the rule's assumptions and effects. This report (1) compares the study to generally accepted research standards, and (2) identifies the assumptions used to estimate the rule's costs and benefits and the rule's driver-operation, economic, safety, and health effects. GAO identified research standards that professional associations, academics, and GAO's prior work have used. GAO evaluated the 2014 FMCSA study against these standards. GAO also compared FMCSA's assumptions about how drivers would be affected by the HOS rule against actual drivers' schedule data from 16 for-hire carriers that cover the years 2012 through 2014. These data include information on over 15,000 drivers per year, but are not generalizable to the motor carrier industry as a whole. GAO found that the January 2014 study issued by the Federal Motor Carrier Safety Administration (FMCSA) to examine the efficacy of its hours of service (HOS) rule—a regulation that governs how many hours truck drivers transporting freight can work—followed most generally accepted research standards. However, FMCSA did not completely meet certain research standards such as reporting limitations and linking the conclusions to the results. For example, by not adhering to these standards, FMCSA's conclusion in the study about the extent to which crash risk is reduced by the HOS rule may be overstated. GAO found that FMCSA has not adopted guidance on the most appropriate methods for designing, analyzing, and reporting the results of scientific research. Without such guidance, FMCSA may be at risk for excluding critical elements in research it undertakes to evaluate the safety of its rules, leaving itself open to criticism. FMCSA made several assumptions and anticipated certain effects of the HOS rule in the regulatory impact analysis. Specifically, to estimate the economic costs of the rule, FMCSA assumed that some drivers would lose a certain amount of driving and on-duty time and then estimated the amount and cost of the work time lost. Further, FMCSA assumed that reduced work time could increase a driver's opportunity to sleep, leading to safety and health benefits. Assessing the effectiveness of the HOS rule is difficult because of the limited availability of representative driver schedule data (i.e., records of drivers' work hours). Nevertheless, GAO's analysis of a limited sample of available data provides some insight into the rule's effects and the extent to which they aligned with FMCSA's assumptions and estimates. For example, according to GAO's analysis, some drivers at a sample of 16 for-hire carriers who worked the longest hours (over 65 hours per work week) reduced their work hours after the rule went into effect, a finding consistent with FMCSA's assumptions that drivers working over 65 hours were more likely to be affected. However, GAO's analysis found that drivers who worked less than 65 hours per work week also changed their schedules after the rule went into effect, a result not anticipated by FMCSA. The ability of FMCSA and others to assess the effects of rules, such as the 2011 HOS rule, is impacted by the limited availability of representative driver schedule data. No organization collects or maintains a centralized database with such data that can be generalized to the motor carrier industry as a whole. Collecting schedule data has historically been difficult, but a recent statutory change that requires carriers to electronically record and store these data provides a potential data source for the future. However, before these data can be used for research purposes several challenges would have to be addressed. First, there are statutory limits on the use of these data for purposes other than enforcing motor carrier safety regulations. Additionally, privacy and cost concerns must be resolved before these data could be made available for analysis. According to FMCSA officials, they do not plan to study how to use these data in a way that will address privacy and cost concerns, in part, because of the statutory limits. Given the potential value of these data to future regulatory analysis, it may be important to provide Congress with information on how these data can be extracted, stored, and analyzed while addressing any privacy and cost concerns. GAO recommends that FMCSA adopt guidance outlining agency research standards. FMCSA agreed with GAO's recommendation. GAO also suggests that Congress consider directing DOT to study and report on how electronically collected driver schedule data can be extracted, stored, and analyzed in a way that addresses cost and privacy concerns.