Page 1 GAO-17-567R 2015 ECP Brake Rule

441 G St. N.W.

Washington, DC 20548

May 31, 2017

The Honorable John Thune

Chairman

Committee on Commerce, Science, and Transportation 

United States Senate

2015 Electronically Controlled Pneumatic Brake Rule: Comparison of DOT 

Forecasts for Selected Data Points for 2015 and 2016 to Preliminary Data for 

Those Years

Dear Mr. Chairman:

In recent years, freight rail shipments of crude oil—a flammable liquid—have increased in the 

United States. For example, according to the Energy Information Administration, the number of 

barrels of crude oil hauled by rail within the United States increased from 1.2 million in January 

2010 to 11.1 million in January 2017.1 If a train transporting such liquids derails—which can 

happen due to a number of reasons, including a broken rail, broken train wheels, or train 

handling—the contents, if released from the tank cars, can catch fire. 

In May 2015, the Department of Transportation (DOT) issued a final rule requiring, among other 

things, that freight railroads equip certain trains hauling flammable liquids, such as crude oil and 

ethanol, with electronically controlled pneumatic (ECP) brakes by 2021 and 2023, respectively.2

ECP brakes send an electric brake signal instantly and simultaneously to each individual car in 

a train, potentially allowing for faster brake application than on trains with conventional air 

brakes. If a derailment occurs, this faster brake application can reduce the likelihood of cars on 

the train derailing and releasing their contents.

In October 2016, as required by the Fixing America’s Surface Transportation Act (FAST Act),3

we reported on DOT’s ECP brake rule and the analyses DOT conducted in support of that rule, 

including its analysis of the rule’s potential costs and benefits.4 According to DOT, some of the 

 

1

While the general trend over this time period has been an increase in crude oil rail shipments, shipments decreased 

in calendar year 2016. According to the Energy Information Administration, the number of barrels of crude oil shipped 

by freight rail in the United States fell from 15.6 million barrels in January 2016 to 11.1 million in January 2017.

2

Trains transporting crude oil must equip with ECP brakes by January 2021; trains transporting ethanol must equip 

with ECP brakes by May 2023. 80 Fed. Reg. 26644, 26748 (May 8, 2015) (codified at 49 C.F.R. § 174.310).

3

Pub. L. No. 114-94, § 7311(a), 129 Stat. 1312, 1601 (2015).

4

GAO, Train Braking: DOT’s Rulemaking on Electronically Controlled Pneumatic Brakes Could Benefit from 

Additional Data and Transparency, GAO-17-122 (Washington, D.C.: Oct. 12, 2016).
Page 2 GAO-17-567R 2015 ECP Brake Rule

potential benefits from ECP brakes result from their ability to reduce the severity of 

derailments.5 As part of DOT’s analysis of potential benefits, DOT forecasted annual freight rail 

shipments of crude oil and ethanol, as well as the number of derailments and the severity of 

those derailments (based on the amount of product released and the number of injuries and 

deaths) over a 20-year period. We found that in forecasting the potential costs and benefits of 

the ECP brake rule, DOT used single point estimates for some variables, such as for annual rail 

shipments of crude oil and ethanol, instead of including a range of scenarios.6 For example, for 

every year in the 20 year forecast period, DOT forecasted a single value for rail shipments of 

crude oil and ethanol instead of a range of possible shipments. Because DOT relied on single 

point estimates, it did not take into account the potential uncertainty inherent in such forecasts, 

which can affect the estimated costs and benefits of the rule. For example, if the actual number 

of shipments of crude oil and ethanol are lower or higher than forecasted point estimates for any 

given year, the calculated benefits realized for ECP brakes may be different than forecasted.7

Considering a range of scenarios instead of using point estimates, can help mitigate uncertainty 

in forecasts. As a result, we recommended, among other things, that DOT take into account, in 

an updated analysis of the potential costs and benefits of the ECP brake rule conducted in 

response to a statutory requirement,8 the potential uncertainty in key variables and 

assumptions, discuss this uncertainty, and present ranges of possible forecasts.9

You asked us to compare selected single point data forecasts for shipments of crude oil and 

ethanol, the number of crude oil and ethanol derailments and associated derailment rate, the 

amount of product lost in such derailments and the related costs of cleanup, and the number of 

injuries and deaths sustained in such derailments in DOT’s ECP brake final rule to the actual 

values realized for those data points in 2015 and 2016. This report examines how DOT’s 

forecasts for 2015 and 2016 compare to actual values realized during those years for the 

following selected data points:

 

5

DOT’s analysis also considered other benefits, including business benefits to railroads such as reduced fuel 

consumption.

6

DOT solicited public input on a proposed ECP brake rule and used public comments to inform its final analysis and 

final rule. In addition, while DOT did not include a range of scenarios, DOT did acknowledge some uncertainties in 

some of the single point forecasts.

7

In addition, the calculated costs may be lower than forecasted in such a scenario because with lower than 

forecasted traffic, railroads may need to equip fewer trains with ECP brakes.

8

The FAST Act, enacted in December 2015, requires DOT to enter into an agreement with the National Academies of 

Sciences to test ECP brakes and, based on the results of this testing and our 2016 report, re-estimate the costs and 

benefits of the ECP brake requirement and update the regulatory impact analysis. Pub. L. No. 114-94, § 7311(b) and 

(c), 129 Stat. 1312, 1602. The Act requires the Secretary of DOT, by December 2017, either to determine the ECP 

brakes are justified based on the updated RIA or to repeal the ECP brakes requirement. Pub. L. No. 114-94, § 

7311(c)(2), 129 Stat. 1312, 1603.

9

We also recommended that DOT develop a plan to collect additional data from railroads on their experiences with 

ECP brakes (prior to 2015, some railroads had limited use of ECP brakes on certain routes) and publish additional 

information regarding computer-based modeling DOT used to support the 2015 ECP brake rule. DOT disagreed with 

our recommendations, stating that we did not have sufficient and appropriate evidence to justify our findings. We 

believe that the findings and recommendations in that report were well supported and justified, as discussed in that 

report. See GAO-17-122.
Page 3 GAO-17-567R 2015 ECP Brake Rule

• total annual rail shipments of crude oil and ethanol, as measured by carloads;

• the number of crude oil and ethanol freight train derailments and derailment rate;

• the amount of crude oil or ethanol released in such derailments and the related costs of 

such releases; and

• the number of injuries and deaths in such derailments.

To make these comparisons, we reviewed relevant documentation from DOT’s 2015 ECP brake 

rule, including the final rule and its supporting economic analysis to determine forecasts for

2015 and 2016 for the selected data points, and interviewed DOT officials regarding these data 

and how they collect them. We then compared those forecasts to preliminary or, when available, 

final data for 2015 and 2016. Forecasts by their nature involve uncertainty; therefore, it is 

expected that forecasts would not match actual data realized in these years. However, 

differences observed between forecasted and realized actual values may provide insight about 

the precision of the forecasted values.

For data on the number of carloads of crude oil and ethanol for 2015 rail shipments, we 

obtained data from DOT. To determine these shipments, DOT used 2015 Waybill Sample data 

from the Surface Transportation Board (STB).10 As of March 2017, STB had not completed its 

validation of the 2016 Waybill Sample data. As a result, DOT was unable to provide data on the 

number of carloads based on 2016 Waybill Sample data. In addition, given the potential for 

errors in data that have not been fully validated, we did not analyze and report on 2016 Waybill 

Sample data.

We also reviewed preliminary 2015 and 2016 data from DOT on the number of crude oil and 

ethanol derailments, the amount of crude oil or ethanol released in such derailments, the related 

costs of such releases, and the number of injuries and deaths in such derailments. Railroads 

are required to report data on derailments involving a release of hazardous materials to the 

Pipeline and Hazardous Materials Safety Administration (PHMSA) which tracks data on 

derailments in the Hazardous Materials Incident Database.11 PHMSA’s database tracks the type 

of incident, the type of hazardous material released, the amount of hazardous material released, 

among other things. Upon receiving an incident report from a railroad, DOT investigates it and 

may update information provided by railroads through the course of investigations. Railroads 

are also required to report information on certain train incidents such as derailments to the 

Federal Railroad Administration (FRA), which maintains these data in its Railroad Safety 

Information System.12

Due to limitations in DOT’s collection of data on derailments in these two databases and DOT’s 

validation of such data, 2015 and 2016 data on derailments and their effects are preliminary and 

subject to change. PHMSA’s database generally does not contain information on derailments 

when there is not a release of hazardous materials such as crude oil or ethanol. FRA’s 

database generally does not contain data on the hazardous material involved in a derailment if 

 

10The Surface Transportation Board (STB) requires railroads to compile a random sample of their waybills (a 

document that accompanies a shipment and describes the freight being transported) and submit them to STB. The 

sample is statistically weighted to be nationally representative of all rail freight movements that occur in the United 

States each year. 49 C.F.R. § 1244.2.

1149 C.F.R. § 171.16.

1249 C.F.R.§§ 225.9 and 225.11.
Page 4 GAO-17-567R 2015 ECP Brake Rule

there is no release. As a result, it is possible that some derailments of trains carrying crude oil or 

ethanol are not included in these databases. DOT, in order to address this limitation, cross 

references the two databases to validate the data contained in both. DOT also takes other steps 

to validate the data. For example, DOT searches media reports for any derailments not reported 

to either database. DOT has not yet completed cross referencing these databases for 2015 and 

2016 derailments.13 Furthermore, data on 2016 derailments may still be updated through the 

course of DOT investigations into derailments as well as updates from railroads, which have up 

to one year after a derailment occurs to report updated information to DOT.14

To review the reliability of the data provided by DOT, we reviewed documentation detailing data 

validation procedures, reviewed our recent reports using these data sources, and interviewed 

officials at DOT. We determined that these data are sufficiently reliable for the purposes of this 

report. However, because some 2015 and 2016 data are preliminary, it is difficult to be 

conclusive about the precision of DOT’s forecasts of the number (and rate) of derailments, the 

amount of crude oil or ethanol released in such derailments, and the number of injuries and 

deaths in such derailments.

We conducted this performance audit from February 2017 to May 2017 in accordance with 

generally accepted government auditing standards. Those standards require that we plan and 

perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that the evidence obtained 

provides a reasonable basis for our findings and conclusions based on our audit objectives.

Comparison of Rule’s 2015 and 2016 Forecasts for Selected Variables to 

Preliminary Data for Those Years

The 2015 and 2016 data we reviewed indicate that for those 2 years, DOT’s forecasted values 

for selected variables included in our review may be higher than values realized in those years 

based on preliminary data. Specifically, DOT’s 2015 forecast for the number of tank cars 

shipping crude oil and ethanol was higher than DOT’s estimated value for that year based on 

Waybill Sample data.15 In addition, preliminary data suggest that for each of the derailmentrelated forecasts we reviewed for 2015 and 2016, DOT’s forecasts appear to be higher than the 

railroads’ actual derailment experience. The forecasted number of crude oil and ethanol 

derailments (and derailment rate), the average amount of hazardous material lost in those 

derailments, and the number of injuries and deaths sustained were also higher than realized for 

both years.16 However, forecasts by their nature involve uncertainty; therefore, it is expected 

 

13According to DOT, it should complete this process for 2015 data by June 2017 and it is now trying to reconcile 

differing data on the number of gallons released in two derailments that occurred in 2015. In one instance, PHMSA’s 

database reports that 362,349 gallons were released while FRA’s database reports 379,549 gallons were released. In 

the other instance, PHMSA’s database reports that 1,000 gallons were released while FRA’s reports that 500 were 

released.

1449 C.F.R. § 171.16

15The 2015 value is estimated from a sample that is weighted to be nationally representative of all rail freight 

movements that occur in the United States each year.

16Because these data are preliminary, it is difficult to be conclusive about the precision of DOT’s forecasts of the 

number (and rate) of derailments, the amount of crude oil or ethanol lost in such derailments, and the number of 

injuries and deaths in such derailments.
Page 5 GAO-17-567R 2015 ECP Brake Rule

these forecasts would not be found to exactly match actual data realized in these years. In 

addition, as DOT forecasted cost and benefits of the rule over 20 years, in other years some 

actual data realized may be higher or lower than DOT forecasted. 

Freight Rail Shipments of Crude Oil and Ethanol

In the 2015 analysis supporting the ECP brake rule, DOT forecasted freight shipments for crude 

oil and ethanol from 2015 to 2034 using forecasted traffic data provided by the Railway Supply 

Institute, an industry association representing builders of tank cars and other entities.17 DOT’s 

forecast for 2015 was higher than the 2015 value calculated by DOT using Waybill Sample data 

(see table 1).18 For the reasons stated previously, we are unable to compare 2016 preliminary 

Waybill Sample data with DOT’s forecast. However, the Energy Information Administration 

creates estimates of freight rail shipments of crude oil and ethanol shipments (in terms of 

barrels and not the number of cars) based on Waybill Sample data it receives from the Surface 

Transportation Board which STB has not fully validated. According to the Energy Information 

Administration, in 2016 there were 373.1 million barrels of crude oil and ethanol shipped by 

freight rail in the U.S., down from 490.5 million in 2015.

Table 1: Comparison of Department of Transportation’s (DOT) Forecast of Crude Oil and 

Ethanol Shipments for 2015 and 2016 with Final Data When Available

DOT 

Forecast for

2015 2015 Final

DOT 

Forecast for 

2016 2016 Final

Shipments of crude oil and 

ethanol as measured by 

number of tank cars (in 

thousands)

1,119 1,014a 1,124 Not 

availableb

Source: GAO analysis of Department of Transportation data. | GAO-17-567R

a

As the Waybill Sample data are collected through a sample that is weighted to be nationally representative of all rail freight 

movements that occur in the United States each year, there should be a sampling error associated with this data point. However, 

according to DOT it did not calculate a sampling error but has confirmed that the Waybill Sample value has been appropriately

weighted to estimate actual carloads shipped.

b

As of March 2017, the Surface Transportation Board has not completed its validation of 2016 Waybill Sample data; as a result, 

DOT is not able to calculate data on 2016 shipments of crude oil and ethanol and given the potential for data errors in not fully 

validated data, we are unable to report on 2016 Waybill Sample data.

17Because data from the Railroad Supply Institute included data on traffic of crude oil and ethanol in Canada as well 

as the United States and DOT’s rule only requires ECP brakes on trains in the United States, DOT adjusted the

estimate to eliminate shipments in Canada.

18According to DOT, its forecast was based on data from the Energy Information Administration and the Railway 

Supply Institute and was not adjusted for a decline in oil prices that likely led to a reduction in shipments of crude oil 

and a related number of derailments in 2015 and 2016. Furthermore, according to DOT officials, the overall quality of 

tank cars used in service since 2015 has improved more than expected as old tank cars have been taken out of 

service. This improvement in fleet-wide tank car quality may have resulted in a lower quantity of crude oil spilled in 

derailments.
Page 6 GAO-17-567R 2015 ECP Brake Rule

DOT’s Derailment-Related Forecasts 

In the analysis supporting the 2015 ECP brake rule, DOT forecasted the number and rate of 

crude oil and ethanol rail derailments, the amount of crude oil and ethanol released in such 

derailments, the costs (e.g., cleanup and environmental) associated with those derailments, and 

the injuries and deaths sustained in such derailments, using historical data from a variety of 

sources:

• To forecast the number of derailments,19 DOT reviewed historic data from PHMSA’s

Hazardous Materials Incident Report database and FRA’s Railroad Safety Information

System on crude oil and ethanol freight derailments from 2009 through 2013. Based on

the historic data, DOT calculated a derailment rate based on the number of rail

shipments and applied that rate to its forecast for annual freight shipments of crude oil

and ethanol.

• To forecast the amount of crude oil or ethanol released in such derailments, DOT

reviewed historical data from 2006 to 2013 in PHMSA’s Hazardous Materials Incident

Report database and FRA’s Railroad Safety Information System to determine an

average amount of crude oil or ethanol released per derailment.

• To forecast the cost associated with those crude oil and ethanol releases, DOT assumed

a cost per gallon of $200 based on a review of literature and data from recent PHMSA

incident reports. That cost took into account cleanup costs as well as certain

environmental and other societal costs.

• To forecast the number of injuries and deaths sustained in crude oil and ethanol

derailments, DOT reviewed data for 2006 through 2014 and determined an injury rate.

DOT applied that rate to its forecast for annual crude oil and ethanol shipments from

2015 to 2034.

Where available, preliminary 2015 and 2016 data suggest that DOT’s forecasts were higher 

than the number of derailments and their resulting effects for those 2 years. (See table 2.) As 

previously noted, the 2015 and 2016 data we reviewed are preliminary and subject to change.

19DOT’s analysis did not include derailments that occur within rail yards where trains are only traveling at low speeds. 

According to DOT such derailments occur at low speeds, and the ECP brake requirement would do little to mitigate 

such derailments. Such derailments are not included in this analysis either.
Page 7 GAO-17-567R 2015 ECP Brake Rule

Table 2: The Department of Transportation’s (DOT) Forecast of Crude Oil and Ethanol 

Mainline Derailments for 2015 and 2016 Compared To Actual Derailments Based on 

Preliminary Data

DOT 

Forecast for 

2015

2015 

(Preliminary)a

DOT 

Forecast for 

2016

2016 

(Preliminary) a

Derailmentsb 11.9 9 11.95 4

Derailment rate (per thousand 

crude oil or ethanol tank cars)

.01064 .00888c 0.0164 Not availabled

Average gallons of crude oil 

or ethanol released per 

derailment

83,602 80,280 83,602 11,351

Range of gallons of crude oil 

or ethanol released in all 

derailments

N/Ae 0- 362,349 N/Ae 20-42,448

Total cost per gallon released 

in derailment

$200 Not availablef $200 Not availablef

Injuries sustained in 

derailments

3.61 1 3.63 0

Deaths sustained in 

derailments

0.26 0 0.26 0

Source: GAO analysis of Department of Transportation data. | GAO-17-567R

a

Because DOT has not completed validation of these data, they are preliminary at this time and subject to change. In addition, 2016 

data are also subject to change because railroads have up to one year after derailments to submit updated information (49 C.F.R. § 

171.16(c)) and DOT may also update information through the course of derailment investigations.

b

This only includes mainline derailments and not those that happened on rail yards for example because, according to DOT, such 

incidents occur at low speeds and the electronically controlled pneumatic brake rule would do little to mitigate them.

c

DOT calculated the derailment rate using the preliminary 2015 data on derailments and the 2015 value of shipments from the

Waybill Sample. As the Waybill Sample data are collected through a sample that is weighted to be nationally representative of all 

rail freight movements that occur in the United States each year, there should be a sampling error associated with this data point. 

However, according to DOT it did not calculate a sampling error but has confirmed that the Waybill Sample value has been 

appropriately weighted to estimate actual carloads shipped.

d

Because 2016 Waybill Sample data is not yet finalized, DOT is unable to calculate the derailment rate.

e

DOT did not forecast a range of crude oil or ethanol released per derailment in the supporting analysis and only forecasted an 

average.

.

f

According to DOT, derailment incident reports often do not reflect all costs—such as property damage or environmental damage. 

As a result, according to DOT, final data on the complete cost per gallon released in derailments are not available.
Page 8 GAO-17-567R 2015 ECP Brake Rule

Agency Comments

We provided a draft of this report to the Department of Transportation and Surface

Transportation Board for comment. DOT and the Surface Transportation Board provided 

technical comments that we incorporated as appropriated. 

 - - - - - 

We will send copies of this report to appropriate congressional committees, the Secretary of 

Transportation, and the Chair of the Surface Transportation Board. In addition, the report will be 

available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions concerning this report, please contact me at 

(202) 512-2834 or flemings@gao.gov. Contact points for our Offices of Congressional Relations 

and Public Affairs may be found on the last page of this report. Key contributors to this report 

were Sharon Silas (Assistant Director), Matt Rosenberg (Analyst in Charge), Namita BhatiaSabharwal, Dave Hooper, Christopher Jones, Delwen Jones, and SaraAnn Moessbauer.

Sincerely yours, 

Susan Fleming

Director, Physical Infrastructure Issues

(101694)
This is a work of the U.S. government and is not subject to copyright protection in the United States.

The published product may be reproduced and distributed in its entirety without further permission

from GAO. However, because this work may contain copyrighted images or other material,

permission from the copyright holder may be necessary if you wish to reproduce this material

separately.
The Government Accountability Office, the audit, evaluation, and investigative 

arm of Congress, exists to support Congress in meeting its constitutional 

responsibilities and to help improve the performance and accountability of the 

federal government for the American people. GAO examines the use of public 

funds; evaluates federal programs and policies; and provides analyses, 

recommendations, and other assistance to help Congress make informed 

oversight, policy, and funding decisions. GAO’s commitment to good government 

is reflected in its core values of accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no cost is 

through GAO’s website (http://www.gao.gov). Each weekday afternoon, GAO 

posts on its website newly released reports, testimony, and correspondence. To 

have GAO e-mail you a list of newly posted products, go to http://www.gao.gov

and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of production and 

distribution and depends on the number of pages in the publication and whether 

the publication is printed in color or black and white. Pricing and ordering 

information is posted on GAO’s website, http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537.

Orders may be paid for using American Express, Discover Card, MasterCard, 

Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube.

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.

Visit GAO on the web at www.gao.gov.

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-4400, 

U.S. Government Accountability Office, 441 G Street NW, Room 7125, 

Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

James-Christian Blockwood, Managing Director, spel@gao.gov, (202) 512-4707

U.S. Government Accountability Office, 441 G Street NW, Room 7814, 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Strategic Planning and 

External Liaison

Please Print on Recycled Paper.