The Impact of Participatory Safety

Rules Revision on Incident Rates,

Liability Claims, and Safety Culture

in the U.S. Railroad Industry

Office of Research

and Development

Washington, DC 20590

DOT/FRA/ORD-07/14 Final Report

July 2007

This document is available to the

public through the National Technical

Information Service, Springfield, VA 22161.

This document is also available on

the FRA Web site at www.fra.dot.gov.

U.S. Department

of Transportation

Federal Railroad

Administration

Conditions

Leading to

Rules

Revision

Management

Preparation

Program

Activities

Roll Out and

Communication

of New Safety

Rulebook

Management/

Workers

Change 

Behavior

Long-Term

Outcomes

Improved

safety culture

Fewer

incidents

Reductions in

injury-related

liability and

costs

High volume 

of rules

Commit to 

include labor in 

crafting rules

Change in

definition of a

safety rule

Broad 

participation in 

crafting rules

Commit to

new rules

Involve

workers

in safety

activities

Increase

compliance

How Participatory Safety Rules Revision Works
Notice 

This document is disseminated under the sponsorship of 

the Department of Transportation in the interest of 

information exchange. The United States Government 

assumes no liability for its contents or use thereof. 

Notice 

The United States Government does not endorse 

products or manufacturers. Trade or manufacturers’ 

names appear herein solely because they are 

considered essential to the objective of this report. 
REPORT DOCUMENTATION PAGE Form Approved 

 OMB No. 0704-0188 

Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302, and to the Office of Management and Budget, Paperwork Reduction Project (0704-0188), Washington, DC 20503. 

1. AGENCY USE ONLY (Leave blank) 2. REPORT DATE 3. REPORT TYPE AND DATES COVERED 

July 2007 Final Report 

4. TITLE AND SUBTITLE 5. FUNDING NUMBERS 

The Impact of Participatory Safety Rules Revision on Incident Rates, Liability Claims, and 

Safety Culture in the U.S. Railroad Industry R3116/RR304 

6. AUTHOR(S) 

Ranney, J., and Nelson, C.* 

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION 

U.S. Department of Transportation REPORT NUMBER 

Research and Special Programs Administration DOT-VNTSC-FRA-02-05 John A. Volpe National Transportation Systems Center 

Cambridge, MA 02142 

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSORING/MONITORING 

U.S. Department of Transportation AGENCY REPORT NUMBER 

Federal Railroad Administration 

DOT/FRA/ORD-07/14 1120 Vermont Ave., NW 

Washington, DC 20590 

11. SUPPLEMENTARY NOTES 

*Western Michigan University Evaluation Center, Kalamazoo, MI 49008 

COTR: Michael Coplen, Human Factors Program Manager 

12a. DISTRIBUTION/AVAILABILITY STATEMENT 12b. DISTRIBUTION CODE 

This document is available to the U.S. public through the National Technical Information 

Service, Springfield, VA 22161. This document is also available on the FRA Web site at 

www.fra.dot.gov. 

13. ABSTRACT (Maximum 200 words) 

The Federal Railroad Administration Human Factors Research and Development Program sponsored a lessons learned study to 

examine the impact of safety rules revision on safety culture, incident rates, and liability claims in the railroad industry. Safety 

rules revision identifies key rules that are universally enforceable and eliminates unnecessary and conflicting rules. The process 

also seeks to promote improvements in safety culture through labor-management collaboration by including a shift in primary 

responsibility for rules creation from management to front-line workers. 

In this study, the evaluation team reviewed relevant literature, interviewed key participants (management and labor) from 

transportation carriers that had undertaken safety rules revision, and analyzed relevant incident and injury data. Although 

outcome data were statistically inconclusive, a number of other indicators in this study suggested a positive benefit on carriers that 

used the process. Interviewees reported more enforceable safety rules, increased compliance, and overall improvements in 

several aspects of safety culture, such as labor-management relations. Moreover, some carriers reported significant reductions in 

the number of claims related to the Federal Employer’s Liability Act and the cost per claim. This report examines other potential 

benefits, challenges, and successful implementation strategies, as well as future directions and activities. 

14. SUBJECT TERMS 15. NUMBER OF PAGES 

Railroad safety culture, safety rules revision, safety rules consolidation, compliance, Federal 68 Employer’s Liability Act, FELA, incident rates, liability claims, lessons learned 

16. PRICE CODE 

17. SECURITY CLASSIFICATION 18. SECURITY CLASSIFICATION 19. SECURITY CLASSIFICATION 20. LIMITATION OF ABSTRACT 

 OF REPORT OF THIS PAGE OF ABSTRACT 

Unclassified Unclassified Unclassified

NSN 7540-01-280-5500 Standard Form 298 (Rev. 2-89) 

Prescribed by ANSI Std. 239-18 

298-102 
METRIC/ENGLISH CONVERSION FACTORS 

ENGLISH TO METRIC METRIC TO ENGLISH

LENGTH (APPROXIMATE) LENGTH (APPROXIMATE) 

1 inch (in) = 2.5 centimeters (cm) 1 millimeter (mm) = 0.04 inch (in) 

1 foot (ft) = 30 centimeters (cm) 1 centimeter (cm) = 0.4 inch (in) 

1 yard (yd) = 0.9 meter (m) 1 meter (m) = 3.3 feet (ft) 

1 mile (mi) = 1.6 kilometers (km) 1 meter (m) = 1.1 yards (yd) 

1 kilometer (km) = 0.6 mile (mi)

AREA (APPROXIMATE) AREA (APPROXIMATE) 

1 square inch (sq in, in2) = 6.5 square centimeters 

(cm2

) 

1 square centimeter (cm2

) = 0.16 square inch (sq in, in2) 

1 square foot (sq ft, ft2) = 0.09 square meter (m2) 1 square meter (m2) = 1.2 square yards (sq yd, 

yd2

) 

1 square yard (sq yd, yd2) = 0.8 square meter (m2) 1 square kilometer (km2) = 0.4 square mile (sq mi, mi2) 

1 square mile (sq mi, mi2) = 2.6 square kilometers 

(km2

) 

10,000 square meters (m2

) = 1 hectare (ha) = 2.5 acres 

1 acre = 0.4 hectare (he) = 4,000 square meters (m2) 

MASS - WEIGHT (APPROXIMATE) MASS - WEIGHT (APPROXIMATE) 

1 ounce (oz) = 28 grams (gm) 1 gram (gm) = 0.036 ounce (oz) 

1 pound (lb) = 0.45 kilogram (kg) 1 kilogram (kg) = 2.2 pounds (lb) 

1 short ton = 2,000 

pounds (lb) 

= 0.9 tonne (t) 1 tonne (t) =

=

1,000 kilograms (kg) 

1.1 short tons

VOLUME (APPROXIMATE) VOLUME (APPROXIMATE) 

1 teaspoon (tsp) = 5 milliliters (ml) 1 milliliter (ml) = 0.03 fluid ounce (fl oz) 

1 tablespoon (tbsp) = 15 milliliters (ml) 1 liter (l) = 2.1 pints (pt) 

1 fluid ounce (fl oz) = 30 milliliters (ml) 1 liter (l) = 1.06 quarts (qt) 

1 cup (c) = 0.24 liter (l) 1 liter (l) = 0.26 gallon (gal) 

1 pint (pt) = 0.47 liter (l) 

 1 quart (qt) = 0.96 liter (l) 

1 gallon (gal) = 3.8 liters (l) 

1 cubic foot (cu ft, ft3) = 0.03 cubic meter (m3) 1 cubic meter (m3) = 36 cubic feet (cu ft, ft3) 

1 cubic yard (cu yd, yd3) = 0.76 cubic meter (m3) 1 cubic meter (m3) = 1.3 cubic yards (cu yd, yd3)

TEMPERATURE (EXACT) TEMPERATURE (EXACT) 

[(x-32)(5/9)] °F = y °C [(9/5) y + 32] °C = x °F

QUICK INCH - CENTIMETER LENGTH CONVERSION

0 1 2 34

Inches

Centimeters 01 3 26 1 4 5 7 1 1098 1 12

5

3

QUICK FAHRENHEIT - CELSIUS TEMPERATURE CONVERSION

 -40° -22° -4° 14° 32° 50° 68° 86° 104° 122° 140° 158° 176° 194° 212°

 

°F

 °C -40° -30° -20° -10° 0° 10° 20° 30° 40° 50° 60° 70° 80° 90° 100°

For more exact and or other conversion factors, see NIST Miscellaneous Publication 286, Units of Weights and 

Measures. Price $2.50 SD Catalog No. C13 10286 Updated 6/17/98

ii
Table of Contents

List of Figures................................................................................................................................ iv 

List of Tables................................................................................................................................. iv 

Preface..............................................................................................................................................v 

Acknowledgments......................................................................................................................... vii 

Executive Summary.........................................................................................................................1 

1. Introduction.................................................................................................................................7 

2. Literature Review and Program Theory of Change....................................................................9 

2.1 Safety Culture and Its Linkage to Safety Outcomes............................................................9 

2.2 Rules, FELA, and Safety Culture in the Rail Industry ......................................................11 

2.3 The Program Theory of Change Behind Rules Revision...................................................14 

3. Methods.....................................................................................................................................19 

3.1 Interview Data....................................................................................................................19 

3.2 Injury Data .........................................................................................................................20 

4. Lessons from Rules Revision Efforts .......................................................................................23 

4.1 Context and Implementation..............................................................................................23 

4.2 Outcomes and Impacts.......................................................................................................28 

5. Conclusions and Recommendations.........................................................................................41 

5.1 Revised Program Theory ...................................................................................................41 

5.2 Program Impacts................................................................................................................42 

5.3 Implementing Safety Rules Revisions...............................................................................43 

5.4 Future Directions ...............................................................................................................44 

6. References................................................................................................................................47 

Appendix A. Statistical Analysis of Incident Rate Data...............................................................51 

Acronyms.......................................................................................................................................57 

iii
List of Figures 

Figure 1. Rules Revision Program Theory .................................................................................. 15 

Figure 2. CSXT and Industry Incident Rates Comparison, 1983-2002....................................... 35 

Figure 3. KCS and Industry Incident Rates Comparison, 1995-2002 ......................................... 37 

Figure 4. ACBL and Industry Incident Rates Comparison, 1999-2000 ...................................... 38 

Figure 5. Original and Revised Rules Revision Program Theory ............................................... 41 

List of Tables 

Table 1. Reduction in the Complexity and Number of Safety Rules............................................32

iv
Preface 

This report summarizes the lessons learned from the implementation of a new safety rules 

revision intervention in transportation carriers. The purpose of the study, sponsored by the 

Federal Railroad Administration (FRA) Human Factors Research and Development (R&D) 

Program, was to examine the impact of safety rules revision on safety culture, incident rates, and 

liability claims in the rail industry. To examine the impact and identify future activities in the 

safety rules revision area, the study examined lessons learned from transportation carriers that 

had undertaken safety rules revision. The study approach used data collected during interviews 

with management and labor employees in three railroads and one inland barge line. The data 

were content-analyzed to assess the impact on union-management relations, safety culture, 

liability claims, and other outcomes. The research team also examined injury rates at the carriers 

to judge the impact of the programs on safety. Given the preliminary and exploratory nature of 

this report, the data are used to suggest possible explanations and future activities. 

The work has been performed under an interagency agreement between the FRA’s Human 

Factors R&D Program and the Volpe National Transportation Systems Center’s Operator 

Performance and Safety Analysis Division. 

v

Acknowledgments 

The authors of this final report are Joyce Ranney, Volpe National Transportation Systems Center 

(Volpe Center), and Christopher Nelson, Western Michigan University Evaluation Center. Lori 

Wingate, Western Michigan University Evaluation Center; Mary Townsend, Volpe Center; and 

Kate Chang, EG&G Technical Services, Inc., provided invaluable insights and technical support. 

Michael Coplen of the Federal Railroad Administration (FRA) Office of Research and 

Development (RDV) provided valuable, ongoing guidance. Thomas Raslear, FRA RDV, 

provided additional insights. The work was performed as part of an interagency agreement 

between the Volpe Center and FRA. 

The authors express their gratitude to the safety and unionized operating employees of CSX 

Transportation (CSXT), Kansas City Southern (KCS), Canadian National/Illinois Central 

(CN/IC), and American Commercial Barge Lines (ACBL) who were interviewed in this study. 

The authors would like to thank Jessie Sampson, Teamsters, ACBL; Bob Keane, General 

Director Risk Management, CN/IC; Sandy Hall, Manager Operating Practices, and Paul 

Hillyard, Brotherhood of Railway Carmen, CSXT; and Thomas Leopold, General Director of 

Safety and Rules, and Fletcher Christian, United Transportation Union, KCS. Glenn Hotz, who 

led efforts as a senior safety executive at both ACBL and CSXT, provided perspective from 

experience with leading rules revisions at two companies. The authors would also like to 

acknowledge the insights gained from the Hile Group and Julie Hile regarding the technical 

principles underlying rules revision. Julie Hile also helped obtain access to the carriers that were 

involved in the study. 

Last, but not least, the authors wish to acknowledge E. Donald Sussman and Jordan Multer of 

the Volpe Center for their valuable contributions. 

vii

Executive Summary 

The purpose of this study, sponsored by the Federal Railroad Administration (FRA) Human 

Factors Research and Development Program, was to examine the impact of safety rules revision 

on safety culture, incident rates, and liability claims in the rail industry. To accomplish this and 

to identify future activities in the area, the authors examined lessons learned from transportation 

carriers that had undertaken safety rules revision. This report seeks to answer the following 

questions through a review of relevant literature, interviews with key participants (management 

and labor) in rules revisions at several freight carriers, and an analysis of incident data from the 

same organizations.

• How did freight carriers implement the safety rules revision process? What were 

some of the challenges these companies faced, and how did they address them? 

• What are the impacts of safety rules revisions on safety culture, liability, injury rates, 

and other related outcomes? 

• What enhancements to the current research methods would help the industry learn 

more about the costs and benefits of this intervention? 

• What areas concerning safety rules revision should be explored in the future?

Research suggests that safety culture is a significant contributing factor to variations in 

workplace safety outcomes. Accordingly, organizations in a number of industries have 

undertaken a variety of reforms aimed to improve the safety culture with the goal of reducing 

occupational injuries and deaths. In the railroad industry, more carriers are using the safety rules 

revision process to improve safety culture and obtain greater participation of front-line workers 

in the prevention of injuries. During safety rules revision (or rules consolidation), the primary 

responsibility for rules creation shifts from the management to teams of front-line workers, with 

management in a supporting role. To avoid some of the pitfalls of the old rulebooks, such as 

duplicative and conflicting safety rules and confusion about which rules applied to everyone 

versus which ones applied only to a given craft, the rules revision process uses a strict definition 

of a rule. It must be possible to comply with a rule 100 percent of the time, and a rule must 

describe the ONLY proper way to perform a work activity.1

 To minimize confusion, rules that 

are applicable for all employees are defined as core rules, whereas those that are appropriate for 

only members of a certain craft are craft-specific rules. Thus, rulemakers can identify key rules 

that are universally enforceable and eliminate unnecessary and conflicting rules. The rulebook 

format is also modified, so that the rules are easier to locate than in the older rulebooks, which 

often contained operating rules, training materials, and job aids throughout. Books created using 

the rules revision process are easier to understand and comply with because of the strict 

definition of a rule, increased clarity in the rulebook format, and the significant reduction in the 

number of rules. Rules revisions seek not only to reduce confusion due to poorly written 

rulebooks, but also to promote compliance and workforce participation in the rules and improve 

safety culture through the process of carrier-wide negotiation and collaboration on the 

development of the safety rules. 

 

1

 In addition to rules, recommended work practices also exist, which are suggested guidelines that do not have to be 

followed in every situation. 

1
Several pressures in the railroad industry prompted companies to create voluminous safety 

rulebooks. First, when employee behavior not governed by an existing rule led to an injury, 

management often wrote a new rule to prevent similar injuries. Second, railroad mergers during 

the 1980s and 1990s resulted in the combination of hundreds of overlapping, and sometimes 

conflicting, safety rules. Third, the Federal Employer’s Liability Act (FELA), which governs the 

handling of railroad worker injury compensation, requires plaintiffs to show employer fault to 

receive compensation.2

 In response, railroad management developed increasingly specific and 

numerous safety rules, which some members of the workforce contend led to confusion, the use 

of rulebooks as punitive tools for management, and rules proliferation to limit carrier liability.

Railroad management, labor, and FRA have expressed interest in safety rules revision to improve 

safety culture, prevent injuries, and reduce FELA-related losses in the industry. Rules revision 

builds trust between labor and management through broad workforce participation in the writing 

of rules and improves compliance because more ownership of the new rules exists. Carrier 

liability is reduced because the resulting rules are easier to understand and follow than the rules 

in use before the revision. Combining these benefits could lead to a reduction in injury-related 

costs, which could lead to a significant boost to the industry, as injury-related costs are 

considerable.3

 

Lessons Learned 

Analyses of incident rate data and interviews with key participants in safety rules revision efforts 

suggested a number of lessons learned. These lessons learned were placed in one of four 

categories: bottom-line benefits, other benefits, implementation strategies, and challenges. 

Bottom-Line Benefits 

Incident rate improvements. A preliminary analysis of incident data at three carriers suggested 

that safety rules revision (combined with other senior management activities to increase 

workforce participation in safety) had a positive impact on incident rates at one rail carrier 

(Kansas City Southern [KCS]), where a statistically significant (p < 0.01) improvement occurred 

in incident rates beginning in 1999.4 The management activities, which began before the actual 

rules revision effort, included a more developmental approach to rule violations and the hiring of 

a rules revision consultant to identify issues and build trust. Incident rate declines at the other 

two carriers could not be attributed with complete confidence to the process due to other changes 

that occurred at the same time, but available data did not fully rule out a positive impact on 

incident rates. During this study, it was not possible to assess the extent to which 

 

2

 When an incident occurs, blame is assigned to either workers or management. Employees may lose their jobs if it 

is determined they violated a rule, and the carrier may have to pay damages (in addition to medical costs) to an 

injured worker for the rest of the worker’s life if it is determined that management was acting in a neglectful 

manner. An unfortunate byproduct of the FELA insurance system is that rules have become central to the 

relationship between labor and management. 

3

 In its 2000 quarterly report to shareholders, Burlington Northern Santa Fe Corporation (BNSF) estimated that its 

personal-injury-related events were $207 million (BNSF, 2001). 

4

 Incident rates were calculated by taking the number of FRA reportable incidents (injuries, illnesses, deaths) per 

number of full-time equivalent workers per year. 

2
implementation effectiveness varied between different departments within the carriers, or 

whether a reduction existed in the specific types of incidents related to the rules in the new 

rulebook. In addition, given the data available, it was impossible to assess whether rules revision 

was more important than the management preparatory work or vice versa, since the evaluation 

team could not distinguish their relative effects. In this analysis, their combined effect was 

investigated, since the management of preparatory work and rules revision were both concerned 

with changing safety-related values, attitudes, and patterns of behavior. 

Reduced liability and injury costs. Safety executives at two carriers reported that the number of 

FELA claims and the cost per claim dropped significantly as a result of the effort. One executive 

suggested that FELA-related costs in his company decreased due to the increased clarity of the 

rulebooks and the decrease in the number of rules. 

Spillover effects in non-safety areas. One executive added that rules revision collaborations can 

sometimes create positive spillovers into productivity-related areas by creating important 

organizational precedents for collaborative decision processes. One respondent noted greater 

worker involvement in activities he/she previously considered management work (reengineering, 

cost-cutting activities). 

Other Benefits 

• Changes in values, attitudes, and patterns of behavior related to safety rules. The 

rules revision process shifted the primary responsibility for defining safety rules 

from management to the workforce. Respondents reported that the revised rules 

focused on significant safety issues and could be complied with 100 percent of the 

time. Compliance was often not possible with the previous rules. 

• Changes in the usability and number of safety rules. As expected, the rules revisions 

dramatically reduced the number of safety rules at the three carriers through a 

filtering process. The new book was important because it was a visible symbol that 

reached all employees and demonstrated a commitment to safety. The increased 

clarity also made the rules easier to follow, improved utilization, and better protected 

the workers. 

• Improvements in union-management relations. Most respondents reported 

improvements in union-management relationships. These workers reported 

increased trust in management commitment to making the workplace safer. One 

respondent reported no change, and another reported improvement in management 

relations with rank-and-file workers but deterioration in management relations with 

union leadership. 

• Perceived changes in compliance. Most respondents indicated that the rules revision 

process had brought greater compliance and greater safety consciousness. 

Implementation Strategies 

• Senior management preparation activities. Safety executives from three of the 

companies stated that senior management activities to increase the participation of 

the workforce in safety preceded the rules revisions.5

 The rules revisions occurred 

 

5

 The topic of senior management preparation was not discussed with Canadian National/Illinois Central (CN/IC). 

3
after senior management began to actively involve labor to make the safety culture 

more participative.

• Participation. Buy-in and participation at all levels of the company, from front-line 

worker to senior management, was deemed critical to successful implementation. 

Front-line worker involvement was central to the writing of the rules, and front-line 

supervisor commitment was especially important once the new rules were 

implemented.

• External help. An external consultant’s project facilitation was helpful. Since longtime workers and managers were entrenched in the status quo, an external, objective 

perspective was valuable and helped keep the process on track.

• Involvement of unions. All respondents noted the importance of having rank-and-file 

union members involved in the process. They differed in their opinions about the 

extent to which involvement of union officials was desirable/necessary. 

Challenges 

• Resistance to change among some groups. Several respondents noted pockets of 

resistance to the rules revisions, citing managers who feared that the process would 

amount to “giving the keys to the inmates” and labor members who suspected that 

the new rulebook would be used to hammer people, just like the old one. Most 

respondents said that companies must allow time to work out differences in opinion. 

Some suggested that individuals could facilitate the process by leading by example 

(e.g., embracing and following the new rules).

• Turnover on committees. Since committees included a mix of labor and 

management, it took time to develop trust. Working through difficult issues also 

took time and often required multiple meetings to resolve. When a high degree of 

turnover existed on committees, the process was especially slow. 

• Resource requirements. The process was time consuming and resource intensive. In 

addition to the costs of the employee time involved, most felt the need for an 

objective outside consultant. These costs could make the process prohibitive for the 

short lines and railroads with scarce resources. Railroads should not undertake this 

program unless they plan to see it through and sustain it long term. 

In short, these findings are encouraging but not conclusive. Ultimately, a final evaluation of the 

rules revision process will depend upon how stakeholders weigh each of the various outcomes. 

Future Direction and Activities 

• Reduction in injury-related liability and costs. Safety executives from two carriers 

reported reductions in the number and cost of FELA claims as a result of rules 

revision. This could not be substantiated due to the lack of data from the carriers, 

but it is worth pursuing in the future given the significant costs associated with 

FELA claims in the industry (Section 2). The impact of safety rules revision on 

other injury-related costs should be studied more closely as well. 

• Senior management involvement. Safety executives from three carriers discussed 

senior management preparatory work that included labor to make the safety culture 

4
more participative. In this study, it was not clear whether rules revision was more 

important than the management preparatory work or vice versa, since the evaluation 

team could not distinguish their relative effects. Since positive results were found, 

especially at KCS, while analyzing their combined effect, future work will look more 

closely at the role of senior management in supporting rules revision and other safety 

interventions. In addition, management involvement after the preparatory activities 

should be considered to address rules revision sustainability. For instance, how were 

senior managers kept involved when a change at the top occurred? Several studies 

indicated the importance of senior management to safety, as well as the bottom line. 

The United Kingdom’s Railway Safety organization developed a strategic safety 

management orientation program for senior managers to address issues raised about 

the role of senior management in the aftermath of the Clapham accident (Nelson, 

2002). Analyzing other ways management can engage with labor to improve safety 

could prove beneficial. 

• Role of union officials. Respondents from one carrier noted that union officials 

resisted rules revision. Future activity will explore the role of union officials in rules 

revisions, the source of their reservations, and their relations with rank-and-file 

workers and management; both of whom tended to be uniformly enthusiastic about 

rules revision. 

• Injury data impact assessment at different levels. Core safety rules apply across the 

carrier, and craft-specific safety rules apply within a particular craft. Although the 

impact assessment provided in this report took the carrier as the unit of analysis, the 

extent and quality of implementation may vary by department. Thus, future research 

activities should look for variations in implementation across departments or crafts to 

test whether the safety outcome trends differ. Further, it is possible that a reduction 

occurred only in the types of incidents that specifically relate to the rules in the new 

rulebooks, rather than a generalized incident rate impact. Future activities should 

examine injuries related to the new rules and test for variations in safety outcomes. 

The department management or representing unions could use information about 

differences across crafts and types of rules in their efforts to improve safety. 

• Measure of compliance. The interview data obtained in this study stated that 

compliance with safety rules improved as a result of the process. In the future, it will 

be helpful to obtain company records of safety rule violations, as a proxy for 

compliance, to confirm the interview data. The analysis could determine whether 

rule violations decreased for all rules, or only for the core rules. A general decrease 

in rule violations could be related to improvements in safety culture. 

• Sustainability. At least one of the carriers studied has since added a considerable 

number of rules to its safety rulebook (coincident with the departure of the senior 

executive who sponsored the rules revision effort), raising the question of rules 

revision sustainability. Future activity might track rulebooks and safety practices in 

carriers that have undertaken rules revisions several years after the end of the process 

to assess the durability of the reforms. In particular, does the pattern of participation 

and rule drafting continue? Are core and craft-specific rules still 100 percent 

enforceable and not in conflict with other rules? Do the crafts still own their own 

rules? Finally, are subsequent increases in the number and type of rules related to 

increases in incident rates? 

5
6

• Resource intensiveness. Most respondents mentioned that the process was time 

consuming and required the use of an external consultant. Future activity should 

investigate refinements to the process that make it less time intensive and costly 

while achieving the same level of consensus. For instance, the findings on the 

importance of leadership suggest that leadership coaching might be a cost-effective 

way to achieve some safety improvements. Reductions in resource requirements 

would be particularly essential for the smaller and more resource-constrained 

railroads. Future research should also study the return on investment for this 

intervention. 

In conclusion, this study finds rules revision worthy of further consideration. Despite the lack of 

conclusive objective data, the interviewees agreed that safety rules revision had an overall 

positive benefit. Moreover, the outcome data, while inconclusive, suggests a number of possible 

benefits worth further exploration. The potential savings associated with a possible reduction in 

FELA claims alone merit future investigation. 
1. Introduction 

The proliferation of safety rules at U.S. railroad companies in recent years appears to hinder 

healthy safety cultures.1

 The sheer volume of these often overlapping rules contributes to poor 

compliance due to confusion and disagreement about which rules are to be followed. When 

combined with the existing regime of fault-based injury liability laws that govern the industry,2

rules often become the focus of worker-management conflict rather than tools for 

communication about safety hazards and solutions. In response to this problem, several railroad 

carriers developed and refined a rules revision process that they claimed increased the value and 

clarity of safety rulebooks and improved the safety culture in which the rules were embedded. 

Proponents suggested that the improvement of safety culture through this intervention led to 

decreases in injury rates. This study focuses on this intervention, known as rules revision (or 

rules consolidation), and includes the following elements: senior management sponsorship, 

shifting primary responsibility for defining rules from management to the workforce, broad 

participation in the crafting of the new rulebook, a strict definition of what is and is not a rule, 

and an easy-to-understand book format. In this report, whenever the term rules revision is 

included, it refers to the process that includes these elements. Whenever the term rules is used, it 

refers to safety rules. 

Rules revision seeks to change patterns of behavior within the organization. The primary 

responsibility for determining the worth of rules is transferred from management to the 

workforce. This shift is thought to increase the workforce’s ownership of the rules, how they 

value the rules, and their willingness to be governed by them. A premise of the intervention is 

that the shift in ownership, as well as the strict guiding principle for defining rules, and the 

improved book format lead to increased trust among workers and management, a more usable 

rulebook, increased compliance, and a resulting decline in injuries. 

FRA funded this study to assess the impact of rules revision as an instrument of safety 

improvement. The Volpe National Transportation Systems Center (Volpe Center), with the 

assistance of the Western Michigan University Evaluation Center, prepared this report and 

summarized lessons learned by freight carriers that have undertaken these reforms. In addition 

to assessing the impact of rules revision on safety outcomes, the report summarized lessons 

learned about successful implementation. The following key questions drove the research: 

• How did freight carriers implement the safety rules revision process? What were 

some of the challenges these companies faced, and how did they address them? 

• What impacts have safety rules revisions had on safety culture, liability, injury rates, 

and other related outcomes? 

 

1

 The definition of safety culture used for this study is taken from the United Kingdom’s Advisory Committee on the 

Safety of Nuclear Installations (ACSNI) Human Factors Study Group in 1993: “The safety culture of an 

organization is the product of individual and group values, attitudes, competencies, and patterns of behaviour that 

determine the commitment to, and the style and proficiency of, an organization’s health and safety programmes.” 

2

 One study suggests that FELA–the governing liability law in the rail industry–perpetuates an acrimonious 

environment between management and employees when it comes to safety rules (Reinach, 2001). Moreover, an 

FRA-sponsored safety culture assessment reported that the current disciplinary process “encourages false reporting 

and pressure not to report injuries” (Evans Planning Group, 1998). Section 2 provides more details on FELA and its 

relationship to safety culture. 

7 
• What enhancements to current research methods would help the industry learn more 

about the costs and benefits of this intervention? 

• What future directions concerning safety rules revision would be useful? 

Data for the study came from a series of interviews with safety executives, workers, and union 

officials at three railroads and one barge line (see Section 3). Injury rates from three of the four 

carriers were also examined to assess the impact. 

The findings reported herein might help rail carriers to (1) decide whether to undertake rules 

revision, (2) determine how to most effectively implement such a revision, (3) assess the impact 

of revision processes on labor-management relations and other facets of organizational culture, 

and (4) assess the potential effectiveness in reducing injuries and liability claims. In addition, 

the findings will help FRA to identify concerns that are important to the industry in the area of 

safety culture. FRA has a long-term goal of sponsoring a workshop where promising safety 

culture interventions will be shared with the industry. Information from this study will be used 

to help determine if safety rules revision should be included in that workshop. 

The report is organized into five sections. Section 2 reviews the link between safety culture, 

rules revision, and safety outcomes. It also outlines the process by which rules revision is 

thought to shift responsibility for the rules from management to the workforce, resulting 

ultimately in safety outcomes. Section 3 provides an overview of the data collection methods 

used in this study. Section 4 summarizes key findings from the analysis of interview and 

incident data. This summary includes the context for the efforts, how rules revision processes 

were implemented, and a preliminary assessment of whether the rules revision process has 

succeeded in improving carriers’ safety cultures, injury rates, injury-related costs, and other 

outcomes. Section 5 provides conclusions and recommendations. 

Throughout this report, readers should bear in mind that the findings presented herein are based 

on a small sample size and retrospective recollections of participants who had participated in the 

rules revision process. Thus, the findings are suggestive and exploratory rather than definitive. 

Having said that, the findings provide a baseline for discussions about the rules revision 

process’s potential for effectiveness in the rail industry and for future research and evaluation 

activity on the topic. Section 5 provides suggestions for future research and evaluation. 

8 
2. Literature Review and Program Theory of Change 

It is reasonable to ask whether an intervention’s design addresses key drivers of safety success. 

Such knowledge can provide useful information in diagnosing reasons for program failure and 

provide a roadmap for program improvements. Proponents of rules revision believe that the key 

point of leverage is improvements in safety culture, which will lead to increased compliance with 

the rules and a reduction in injuries. This section presents a brief review of literature on (1) the 

relationship between safety culture and safety outcomes, (2) the link between safety rules and 

safety culture in the rail industry, and (3) precisely how the rules revision process is supposed to 

leverage these connections into safety improvements (the theory of change).3

 

2.1 Safety Culture and Its Linkage to Safety Outcomes 

A growing body of research has shown safety culture to be associated with positive safety 

outcomes. A study of 50 chemical plants with 6,000 employees demonstrated the relationship 

between positive safety culture and lower accident rates using a survey (Carder and Ragan, 

2003). This study took place over 10 years and included extensive psychometric testing to verify 

the results. Ostrom, et al. (1993) conducted a study that found that an organization’s shared 

beliefs and attitudes about safety, as measured by a survey, were found to differentially affect 

safety performance in different departments, according to accident rates. This study, conducted 

with a Department of Energy contractor, used a second survey based on the indigenous norms 

and values of the resident organization. Hofmann and Stetzer (1996) conducted a third study and 

found that safety culture, measured using a survey, and unsafe behaviors, measured by random 

safety audits, were associated with actual accidents. Data were collected from 21 teams and 22 

individuals in a midwestern chemical processing plant. A fourth study, conducted with four 

railroads, found that the business units with the most positive safety cultures, as measured by a 

survey, were those units with the best safety performance (Bailey and Petersen, 1989). Yet 

another study that provides evidence of the relationship between positive safety culture and 

lower accident rates used case analysis methods. This case study, examining the positive safety 

culture of an aircraft carrier, found that where loss of life was expected due to the complex, 

inherently hazardous nature of the work, a minimal number of unsafe events were found 

(Rochlin, LaPorte, et al., 1987) The study described the patterns of behavior and values that 

encourage this proactive, vigilant safety culture. 

In a random sample of 97 Canadian manufacturing plants, Simard and Marchand (1997) found a 

positive relationship between safety culture and compliance with safety rules. This study, 

involving 1,061 workgroups, also found that the supervisor’s propensity to engage in 

participatory management of safety was important, reinforcing the idea that increased ownership 

relates to compliance. 

While the studies previously mentioned are concerned with industrial, transportation, and 

military organizations, the linkage between safety culture and injury rates receives indirect 

support from a study conducted in the service industry. Naumann and Bennett (2000) found that, 

in a study of banks, improvements in one aspect of organizational culture–procedural justice 

 

3

 A more extensive review of safety culture literature appears in the “Baseline Data Collection Summary Report: 

Amtrak Safety Rules Revision Process” (Ranney, J. and K. Chang, 2001). 

9 
culture4

–were associated with increases in the frequency of helping behaviors. While their study 

does not focus on safety, it provides evidence that improvements in organizational culture can 

increase employees’ willingness to engage in behaviors that contribute to the achievement of 

group goals. Examples of safety-related helping behaviors might include participation in safety 

activities, such as safety committees. 

The values and patterns of behavior associated with safety rules can best be understood in an 

historical context, beginning with a discussion of the labor-management relationship. Railroads, 

like other transportation modes, derive their organizational structure from a military model and 

therefore have a strong command and control hierarchy. Aviation, maritime, and railroads all 

demonstrate this pattern, with aviation being influenced by the U.S. Air Force, maritime by the 

U.S. Navy, and railroads by the U.S. Army. This military influence encourages an authoritarian 

management style that permeates the entire enterprise, including this study’s specific interest 

area of safety rules. At the same time, the unions in the transportation industry are also strong 

and commanding in their own way. Transportation unions are powerful for two reasons: 

• First, highly skilled workers, such as pilots, air traffic controllers, captains of ships, 

and locomotive engineers, take many years to become experienced. If they choose to 

strike, they are difficult to replace. 

• Second, the same unions represent the same job categories across the industry, which 

means that a locomotive engineer working in any railroad in any part of the country 

is represented by the same union. As a result, a union issue in one geographical area 

could be felt throughout the country. Organizational tension is created on safety 

rules and other issues when management’s military style meets labor’s highly skilled 

power style. 

The historical trends in the scientific paradigms that underlie safety analysis affect safety rules. 

Hale (2000) suggests that the current interest in safety culture is part of a broader paradigm shift 

in the safety discipline away from a blame the worker, rule-based approach and toward a systems 

view of the causes of incidents, including organizational explanations. A century ago, he writes, 

the field was primarily focused on designing-out injuries through technology. This focus led to 

significant advances with technologies such as “personal protective equipment, no-go areas 

around dangerous processes and bursting disks and bunds for tanks.”5

 According to Hale, the 

human factor age, characterized by an increasing focus on the individual contribution to 

accidents, followed the engineering age. Significant improvements were made and continue to 

be made, related to operator performance. The engineering and human factors ages shared an 

emphasis on eliminating, as far as possible, the need for unnecessary human intervention through 

automation. If human intervention could not be eliminated, the approaches sought to reduce 

harm by restricting individuals’ choices by specifying rigid rules. Hale writes: 

Ultimately, we get a rule for everything and safety is seen as something which requires no 

thinking any longer, but simply good training, a prodigious memory, a large safety manual 

or computer to refer to, and an iron discipline.6

 

 

4

 Procedural justice culture involves perceptions about how a group is treated as a whole. 5

 Hale (2000), p. 6. 

6

 Ibid., p. 7-8. 

10 
These approaches have been succeeded by the belief that safety analysis must look at 

organizational, not just individual, factors. 

2.2 Rules, FELA, and Safety Culture in the Rail Industry 

Existing safety practice in the rail industry has traditionally reflected the engineering and human 

factor ages with the emphasis on reducing discretion through rules (2000). Indeed, it is not 

uncommon for rail carriers’ safety rulebooks to have more than 500 rules.7 For the most part, 

the rules are extremely detailed. They are often duplicative and unclear. Reactive rulemakin

mergers and acquisitions; liability and FELA; and rules, safety culture, and safety outcomes 

contributed to the proliferation of rules in the rail industry. 

g; 

 

2.2.1 Reactive Rulemaking 

Industry officials noted that safety rules were frequently written in response to particular 

accidents or near misses. The motivation for writing rules appeared to be connected to a desire 

that similar accidents never happen again. Supervisors often reported that having an employee 

killed, injured, or even experienced a near miss could be a life-changing experience–one that 

supervisors never wanted to live through again. 

In short, a primary reason for the proliferation of rules in the rail industry is the desire to prevent 

accidents. A classic study of bureaucratic organization once observed that compassion spawns 

red tape (Kaufman, 1977). 

2.2.2 Mergers and Acquisitions 

The large number of mergers and acquisitions in the railroad industry contributed to the 

proliferation of safety rules in railroad companies. A review of Coleman’s “The Family Tree of 

North American Railroads” suggested that 170-200 mergers and acquisitions occurred between 

1865 and 1995 in the United States (Coleman, 2001). Coleman’s analysis was limited to Class I, 

former Class I, or major regional lines and, therefore, should be taken as a conservative number. 

Many of the railroads had too many predecessors to track. This food chain-like dynamic 

produced the Class I railroads. To highlight a few Class I examples between 1996 and the 

present: Southern Pacific acquired Denver & Rio Grande Western and Saint Louis Southwestern 

in February 1997. Union Pacific then acquired Southern Pacific in 1997. In 1999, a portion of 

Conrail merged with CSX Transportation (CSXT), and another portion merged with Norfolk 

Southern. 

The need to combine rulebooks occurred every time as each railroad had a pre-existing set of 

practices and historical circumstances reflected in their rulebooks. Most companies did 

relatively little to address the inevitable duplication and inconsistencies that resulted from the 

merging process. 

7

 See Table 1 for an account of the number of rules in the carriers studied both before and after the rules revision 

process. 

11 
2.2.3 Liability and FELA 

Existing liability laws for the rail industry provide a clear economic incentive for a proliferation 

of rules. FELA, which governs worker compensation for injuries, is best understood in contrast 

to the worker’s compensation system that governs most other industries. 

Worker’s compensation is a no-fault system that bases the cost of premiums on the injury rates 

of the industry. Both employers and employees pay into the system, and employees receive 

benefits when a claim is approved. Essentially a contract exists between employers and workers 

whereby employers lose a number of previously available legal defenses against liability, 

including contributory negligence on the part of the worker and the claim that workers 

voluntarily assume risk in taking their jobs. In exchange, the worker’s compensation system 

regularizes employers’ exposure to liability by linking compensation to a benefits schedule that 

increases as a function of the injury’s severity. For their part, workers accept ceilings on 

employer liability in exchange for the avoidance of lengthy litigation and delays in the receipt of 

benefits (General Accounting Office, 1996). 

FELA retains the kernel of a fault-based system in that employees retain the right to sue 

employers and employers retain access to the defense of contributory negligence on the part of 

employees. Employee compensation is determined after an apportioning of fault between 

employer and employee.8 

The FELA system encourages the proliferation of safety rules because the existence of a safety 

rule, and evidence that the employee failed to follow the rule, can be an effective employer 

defense against liability claims. Thus, companies may view the proliferation of rules as a way to 

minimize exposure to liability in a fault-based system in which such exposure can vary 

considerably over time. The financial incentives for rulemaking are intensified by the fact that, 

under FELA, employers must place into escrow the amount equal to the claim an injured 

employee makes. 

2.2.4 Rules, Safety Culture, and Safety Outcomes 

The proliferation of rules, combined with the fault-based nature of FELA, tends to foster an 

adversarial relationship between railroad management and labor, in which each party wants to 

blame the other. As a result, full and complete investigation of an accident or injury might not 

occur, due to incentives for both the employee and railroad management to avoid revealing any 

facts or circumstances that would indicate their responsibility for the accident or injury. 

When a reportable safety incident occurs, a worker’s immediate concern, other than the extent of 

his injury, is his job. If the accident investigation team determines that he has broken rules, he 

 

8

 Over the years, the FELA system has adopted a number of administrative procedures that streamline the claims 

process, rendering it similar to the no-fault worker’s compensation system. For most claims, the employee and the 

railroad’s representatives negotiate directly and reach a settlement. Approximately 70 percent of claims are settled 

this way. Another 10 percent are settled after the employee retains legal counsel but before a suit is filed. The 

remaining 20 percent of injury cases are litigated in a traditional adversarial process; of these, only 5 percent reach 

the point of a jury verdict (Reinach and Gertler, 2001). 

12 
may be let go.9

 No matter what, the outcome of an injury is typically a negative mark in an 

employment record. Indeed, some unions are unwilling to participate in investigations since part 

of the goal is to assign blame and potentially mete out disciplinary action. Thus, this reporting 

system discourages the airing of important safety-related information, inhibits the creation of a 

cooperative working relationship between labor and management, and may hinder efforts to 

improve safety in the workplace. 

Previous research supports this view of the rules and FELA in safety culture. For instance, a 

study conducted by Elling (1991) on worker perceptions found that British railway workers 

perceived safety rules negatively. In particular, this included the following: 

• 80 percent of respondents considered that the rules were mainly concerned with 

pinning blame. 

• 79 percent of respondents thought there were too many rules, 12 percent thought 

there were too few; only 9 percent had no complaints on this score. 

• 77 percent of respondents found them conflicting, 95 percent of respondents thought 

that work could not be finished on time if the rules were all followed. 

• 85 percent of respondents found it hard to find what they wanted in the rulebook. 

• 70 percent of respondents found them too complex and hard to read when they had 

found them. 

• 71 percent of respondents thought there was little motivation given to follow rules. 

“Not one [of the respondents] could remember ever having referred to the rules in a practical 

work situation in the last six months.” Similarly, a study of railroad yard employees found that 

employees perceived that management used the safety rules against employees, especially in 

cases of injury, to protect the financial interests of the railroad (Reinach and Gertler, 2001). The 

authors of this study suggested that FELA contributed to an adversarial relationship between 

workers and management. 

The FELA adversarial safety culture has economic and safety implications. The search 

continues for ways to reduce injury costs. In its 2000 quarterly report to shareholders, BNSF 

estimated its personal injury-related events were $207 million (BNSF, 2001). Though injury 

costs were lower that year than during the previous 2 years, they were still a significant burden 

and contributed to annual decreased net income for the railroad. In 1994, the Transportation 

Research Board estimated that job-related injuries cost Class I railroads more than $1 billion 

annually (Reinach and Gertler, 2001). 

Views of the costs of FELA and its impact on safety are divided. Unionized workers believe the 

cost of potential FELA awards can be used as leverage to ensure that management eliminates 

unsafe conditions. Management, believing that the FELA compensation concept is out of date, 

focuses primarily on the safety costs incurred by damaged equipment (Reinach and Gertler, 

 

9

 Generally, after an accident, the supervisor submits a form to a central reporting office detailing the incident. 

This form often includes the supervisor’s report, the employee’s report (including a legal release), and a portion 

to be filled out by a medical treatment facility. If warranted, a committee, which may include a union 

representative, is formed to investigate the accident. 

13 
2001). In either case, it appears that improving rule compliance would reduce the number of 

injuries to the satisfaction of labor and lower liability costs to the satisfaction of management. 

2.3 The Program Theory of Change Behind Rules Revision 

The proliferation of rules, combined with FELA’s liability regime, has created disincentives for 

open communication about safety issues, cooperative efforts to address safety problems, and 

compliance with rules. Safety rules revision represents one type of intervention designed to 

address challenges in safety culture and, ultimately, improve bottom-line safety performance. 

Given the evidence that improvements in safety culture are an important component in the theory 

linking rules revisions to reductions injury rates, the question becomes: How can rules revisions 

change safety cultures? 

This section describes the rules revision process in greater detail, as well as the mechanisms by 

which it seeks to change safety culture and safety outcomes.10 The section begins with a general 

statement of the mechanisms rules revision uses to leverage changes in safety culture, 

specifically which values and patterns of behavior it seeks to change. 

The revision process that is the focus of this study views formal safety rules as an essential part 

of a carrier’s safety culture. The process includes broad workforce participation in the decision 

about what rules are worthy of being rules, determination of which rules cover all employees and 

which ones are craft specific, and what language will be used in the new rules. The conventional 

approach to safety rules revision has been to have the safety department commission a technical 

writer to rewrite them with direction from management. This conventional approach results in a 

new rulebook, but it generally does not include broad participation from the workforce. In 

contrast, the rules revision process that is the focus of this study sees the rewriting of the 

rulebook as an occasion for improving the safety culture. The advocates posit that rules will be 

effective only if “values, attitudes, competencies and patterns of behavior” support them 

(ACSNI, 1993) that directly change the blame game. Carroll further suggested that other safety 

process management practices must support to impact outcomes. Carroll noted that that 

removing the blame game and increasing compliance allowed for more robust safety 

management practices to function effectively, such as improved root cause analysis and 

communication between engineers and operators (Carroll, 2000). 

The desired result of rules revision is to make the rules more credible and hence worthy of being 

used by the workforce for governance and standard setting. The process changes the “values, 

attitude, competencies and patterns of behavior” by shifting the primary responsibility for 

 

10 The description of the rules revision program theory is based on discussions with the Hile Group, the primary 

provider of consulting services in the safety rules revision process. The Hile Group has implemented this approach 

at four railroads (BNSF, CSXT, KCS, and CN/IC) and one inland waterway carrier (American Commercial Barge 

Lines (ACBL)). Attempts to describe the theory of change behind programs and policies are often called program 

theories (Chen, 1990; Chen and Rossi, 1992). Program theories can aid in program design, modifications, and 

evaluations in a number of ways. First, they provide a shared road map for the implementation of an intervention. 

Such road maps provide useful benchmarks that allow program designers and stakeholders to chart implementation 

progress. Second, such program road maps help evaluators and stakeholders discern the extent to which any 

findings of no impact might be the result of implementation failure versus inherent flaws in program design. 

Finally, program theories aid in program improvement by identifying strong and weak points in the flow of events 

that link program inputs and processes with policy outcomes. Indeed, one often finds that some parts of an 

intervention work better than others. 

14 
drafting rules from management to workers and by revising or eliminating rules that are not 

meaningful to the workforce. When rules are conflicting and voluminous, one can easily defend 

noncompliance to oneself and to one’s peers. When only a few rules exist that one’s peers have 

drafted personally, common sense and peer pressure encourage compliance. Rules revision, in 

short, is a process of “streamlining rules into the culture.”11 

Figure 1 illustrates the linkages.12 On the far left side of the diagram lie the conditions that lead 

to a need for rules revision. Next to these conditions are the activities in the safety rules revision 

process itself. The third box depicts factors related to compliance with the new rules, including 

sense of ownership, communication, and cooperation among various parts of the organization 

undergoing rules revision. Finally, on the far right side of the diagram are the program’s longterm goals for outcome measures.13

• Safety 

culture 

• Injury rates 

High volume 

of rules due 

to: 

• Injury 

avoidance 

• Mergers 

• FELA

• Change in 

roles and 

responsibilities 

• Core teams 

• Satellite teams 

• Change in 

definition of a 

safety rule 

• Reduction in 

safety rules 

• Broad 

participation 

Conditions 

Leading to 

Rules 

Revision 

Program 

Activities 

Roll Out and 

Communication 

of New Safety 

Rulebook 

Long-Term 

Outcomes 

• Commitment and 

ownership from 

front-line 

workers, safety 

professionals, 

human 

resources, legal 

• Implementation/ 

communication of 

new rulebook

Figure 1. Rules Revision Program Theory 

The third box in Figure 1 is important because it emphasizes that stakeholder commitment, 

ownership, and communication are essential components. The process employed by the Hile 

Group features the following strategies for developing such commitment, ownership, and open 

communication: 

• Management, employees, and unions are involved in all aspects of the process. 

• A Core Team is created to oversee the work of craft-specific Satellite Teams, draft 

rules and non-rules materials (e.g., recommended work practices), disseminate 

information, incorporate feedback, and promote the cultural shift. 

• Craft-specific Satellite Teams draft craft-specific rules and non-rules materials, 

disseminate information, incorporate feedback, and promote the cultural shift. 

 

11 Hile Group interview. 

12 The program theory presented here was revised as a result of what was learned in this study. Section 5 presents 

the revised program theory. 

13 Like most program theories, this diagram is an oversimplification. Indeed, one could provide a more indepth 

model of the relationship among these outcomes. For instance, it may be that an interaction exists between safety 

culture and incidence of at-risk behaviors. Moreover, changes in both safety culture and at-risk behaviors/conditions 

might cause changes in injury rates. 

15 
• All crafts, functions, constituencies, and possible attitudes regarding the process are 

represented on the Core and Satellite Teams. 

• Consultants are involved in project design, facilitation, provision of safety 

performance expertise, and writing support. 

• An explicit implementation/communication phase is included after the new rulebook 

is written to communicate the new rules to all employees (the consultant supports but 

does not provide it). 

The use of Core and Satellite Teams increases the ownership and accountability for the rules by 

the workforce and specific unions. Core Teams write core rules, which are defined as practices 

across all crafts that can be complied with 100 percent of the time and that represent the only 

way to perform a work activity. Satellite Teams, comprised of union representatives from the 

major crafts (Operations, Mechanical, Track Engineering, Clerical), write craft-specific rules that 

should be complied with 100 percent of the time for a specific craft. The Core and Satellite 

Teams clearly distinguished rules from recommended work practices. Recommended work 

practices, as the name implies, are to be followed at the worker’s discretion, making allowances 

for the particular circumstances in which workers find themselves. Given the crisis-andresponse origin of many safety rules generated under the FELA regime, many of the rules in the 

older safety rulebooks do not apply to all workers in all situations. Therefore, reclassifying the 

rules as recommended work practices increases the focus on the real rules. 

Proponents of rules revision note that another important change is the language and tone of the 

rules. When management writes the rules, they are written from a “thou shalt” perspective. 

When the workforce writes the rules that govern themselves, they are written from a more 

introspective “we will” perspective. Changing from “thou shalt” to “we will” reflects a change 

in values and attitudes from the traditional top-down approach for rules and safety activities to 

one that is participatory instead. Proponents contend that this change in language was one reason 

why compliance was easier to obtain. 

Below is an example of an old rule exemplifying the “thou shalt” attitude and how it was 

changed to a rule with more of a “we will” attitude on the part of the worker. 

• Old Rule for Welding-Fire Protection: 

Use only a friction lighter, stationary pilot light, or some other source of suitable ignition to 

light torch. Do not light torch with matches, cigarette, or cigarette lighter (Illinois Central 

Railroad System, 1998). 

• New Rule for Welding-Fire Protection: 

Use only approved flint strikers to light welding torch (CN/IC, 2002). 

Previous research provides some validation of the linkage between participatory decisionmaking 

by the workforce and rule compliance. Several studies have found that safety compliance is 

higher when workgroups have some power and influence over the decisions that affect their 

safety. Grote and Kunzler (1996) and Grote et al. (1997) found in their study of chemical and 

transportation companies that the integration of safety in day-to-day operations was easier in 

organizations whose primary task was defined in terms of the quantity, quality, and safety of 

16 
production, as opposed to organizations whose primary task was only defined in terms of 

quantity and quality. Rules revision gave significant control to workgroups over safety in their 

work environment because it allowed them to rewrite the safety rules used on a day-to-day basis. 

Other studies point to the critical role that first-line supervisors play in safety rule compliance.14 

In their research on workgroup propensity to comply with safety rules, Simard and Marchand 

(1997) found that micro-organization factors at the front line are the primary determinants of 

compliance. Research was conducted using 1,061 workgroups from a random sample of 97 

manufacturing plants. Results from this study suggest that a cooperative workgroup-supervisor 

relationship is the most important variable, but the supervisor’s propensity to engage in 

participatory management of safety was also important. Other findings suggested that the frontline supervisor may act like a safety culture engineer by motivating workers to conform to safety 

rules rather than imposing rules on them. Simard and Marchand (1997) found that compliance 

was higher when supervisors had some control over the safety of their people. Naumann and 

Bennett (2000) found, in their study of 220 employees at 40 different locations of 2 banks, that 

safe behavior and perceptions of safety are largely dependent on supervisors. This study found 

that workgroups whose supervisors applied the same policies and procedures to all employees 

perceived more fairness than workgroups whose supervisors applied policies and procedures less 

systematically. 

Finally, other studies point to the importance of senior management in fostering compliance. In 

a study on compliance with operating rules in the rail industry, Coplen (1999) found that senior 

management unintentionally encouraged middle management to value productivity over 

protection by offering productivity bonuses and other incentives. Findings by Ranney (1996) 

linked senior management leadership in managing team-based organizations to bottom-line 

results. 

In sum, proponents believe that rules revision only improves safety culture and outcomes when 

all the groups in an organization work together effectively to encourage the change in the pattern 

of behaviors and values in which safety rules are embedded. The combination of players, 

decisions, dialogue, and new rulebook are necessary to change the system in which the safety 

rules are embedded. 

 

14 These studies are interesting because supervisor activity is not directly emphasized in the Hile Group’s approach. 

17 

3. Methods 

Two sources of data were collected from carriers that completed rules revisions: interviews and 

incident data. Incident data allowed the evaluation team to examine the impact of rules revision 

on bottom-line safety outcomes. The interview data, by contrast, allowed the team to examine 

the reasons for undertaking rules revisions, key implementation issues, and a range of possible 

outcomes of the intervention. 

3.1 Interview Data 

Eleven people were interviewed from three railroads and one in-land barge carrier. Respondents 

included five senior safety officials and six workers/union members. A barge carrier was 

included because it was transportation-related, previously had a large number of safety rules, and 

used a similar rules revision process. One of the railroads (CN/IC) was in the process of 

conducting its rules revision process when the interviews occurred, so the data collected from 

CN/IC primarily detailed the reasons it initiated rules revision and the way it had structured its 

effort. 

During the first phase of data collection in fall 2001, the evaluation team interviewed safety 

executives at the four companies. For the second phase of data collection in winter 2002, the 

evaluation team interviewed five union workers at these same companies. The team conducted 

followup interviews with safety executives at the companies during the spring and summer of 

2002 to confirm interpretations of the data and explore issues that arose during the initial data 

analysis. At that time, the Hile Group provided an analysis of the number of rules at each carrier 

before and after rules revision. 

The evaluation team obtained the names of the freight carriers’ safety executives from the 

consultant who facilitated the processes in these carriers.15 The team sent each executive a letter 

explaining the research project and requesting an interview to generate lessons learned from 

revision of railroad safety rules.16 The questions asked during the interviews reflected the 

following differences: CN/IC was beginning its rules revision process, CSXT and KCS had 

completed its interventions, and ACBL was the barge line. The evaluation team asked safety 

executives from the carriers that had completed rules revision the following types of questions: 

• What were your reasons for undertaking the rules revision project? 

• To what extent was the project successful? 

• What factors helped/hindered the drafting and implementation of the new rulebook? 

• How were the unions involved, and what were the strengths and weaknesses of their 

involvement? 

• What was the role of the rules revision consultant with your project? 

• What advice do you have for other companies seeking to do a rules revision? 

• What quantitative impacts did you see as a result of rules revision? 

• What unanticipated benefits or liabilities did you experience as a result of the effort? 

 

15 The Hile Group was the only consulting firm providing services in the area of participative rules revisions at that 

time. 

16 The safety executive at a fifth carrier could not be contacted by the evaluation team and, therefore, was not 

included in the sample. 

19 
The interview with the CN/IC executive was limited to why the company was engaging in 

rules revision and how it was structuring the effort. 

The evaluation team asked the safety executives to provide names and contact information for 1 

to 5 individuals who could provide a union perspective on rules revision (12 were identified, 6 

were interviewed). Most of these individuals were union workers; some were local union 

representatives.17 All interviewees were intimately involved in the rules revision process. 

Interview questions with the workers who had completed rules revision included the following 

types of questions: 

• What job do you hold in your carrier and union? 

• From your point of view, what aspects of rules revision were successful/not 

successful? 

• What factors helped/hindered the rules revision process? 

• How would you describe the impact of rules revision on the relationships between 

management and labor? 

• What advice do you have for other companies seeking to conduct a rules revision? 

The interview with the CN/IC employee focused mostly on aspects of the project startup that 

were working successfully from the worker’s point of view. 

All interviews were conducted over the phone and lasted approximately 1 hour. Typically, there 

were two interviewers (a lead interviewer and a note taker); the conversations were not recorded. 

Written notes from the interview were transcribed by the note-taker and reviewed by three 

members of the evaluation team. 

In order to ensure the validity of the conclusions drawn from the interview data, followup 

interviews with safety executives were conducted to verify interpretations generated by the initial 

round of qualitative data analysis. In addition, where possible, the evaluation team asked the 

same question of more than one respondent from each firm to triangulate interpretations. 

Finally, four members of the evaluation team checked the interpretations of the qualitative data 

to ensure reliability. 

The small sample size in this study limits the generalizability of the findings. As noted above, 

respondents were chosen to target individuals who had either led or participated in projects as 

members of Core and/or Satellite Teams. This gives the evaluation team a reasonable degree of 

confidence in the external validity of the findings. Practical constraints limited the evaluation 

team’s ability to obtain nominations for interviewees from union representatives. 

3.2 Injury Data 

While injury data alone were not obtainable, railroad incident data, consisting of FRA reportable 

injuries, illnesses, and fatalities to employees on duty (EOD), were available. Incident rates were 

calculated by multiplying yearly FRA reportable incidents (injuries, illnesses, deaths) by 200,000 

and dividing the product by actual employee hours worked. Incident and employee hours 

worked data (which includes all employees in all departments) were taken from Railroad Safety 

 

17 In one case, the name of the safety director at a unionized facility was given to the team. During an interview, 

this director provided the name of a front-line union worker, whom the team interviewed later. 

20 
Statistics (FRA, 1997-2002) and Accident/Incident Bulletin (FRA, 1983-1996). Glenn Hotz, 

Vice President for Safety and Health at ACBL at the time the study was initiated, provided 

ACBL incident data. 

For all companies, incident rate trends were compared with comparable industry-level data. For 

the rail firms these data came from the sources noted above. ACBL data were two-digit standard 

industrial code time series data, downloaded from the Bureau of Labor Statistics Web site.18 

 

18 http://www.bls.gov. 

21 

4. Lessons from Rules Revision Efforts 

After setting out the context of the safety rules revision process and exploring the ways in which 

it is supposed to operate, this report now turns to a review of the interview and outcome data, 

including incident rates. 

4.1 Context and Implementation 

The information presented in this section is based on interviews with stakeholders of U.S. rail 

and barge carriers that had undertaken rules revisions. The section begins with a brief discussion 

of the context for the efforts, including the reasons these carriers undertook rules revisions. 

Next, the carriers’ experiences with drafting new rulebooks and putting them into effect are 

examined. The section concludes with a special focus on the role of unions in the process and 

other observations about the implementation of rules revisions. 

4.1.1 Reasons for Undertaking a Safety Rules Revision 

Safety professionals from each of the carriers independently decided to initiate the rules revision 

process and sought assistance from the Hile Group. CSXT was the first carrier to try the 

intervention, so the CSXT safety professionals were instrumental in creating the process with the 

Hile Group. Safety professional respondents from all of the carriers stated that informal sharing 

occurred between carriers to determine how to make safety rules more effective. 

When respondents were asked to identify their company’s principal reasons for undertaking a 

rules revision, the reasons given were generally congruent with those of advocates of the rules 

revision process. These reasons centered on the complexity of high-volume rulebooks, 

uncertainty about which rules applied where, low compliance levels, and lack of communication 

and trust. 

Nearly all respondents noted that the old rulebooks simply had too many rules. “The men were 

getting more books than they could carry,” observed one conductor. The number of rules 

appeared to impede the integration of the rules into the day-to-day practices of line workers. If 

too many rules existed, workers could not remember them all. More importantly, the old 

rulebooks did not allow workers and the managers who enforced the rules to distinguish between 

the most and least important rules. In addition, many rules were not applicable to all workers. A 

former union local chairman described the state of the rules: 

There was a rule that said, ‘Don’t pull broken glass from a window.’ Does 

that really need to be a rule? Those types of rules disappeared right away. 

There’s not very many core rules, but they are ones that everyone needs to 

adhere to; they’re not craft-specific. People started to realize there was lots 

of redundancy, lots of rules that didn’t need to be in there. The new rules 

are more precise, more straightforward. 

The rules revision process, according to one safety executive, provided the company with a way 

to distinguish rules, which must be followed under all circumstances, from work procedures, 

which are only advisory in nature: “If it is a rule, it has to contain a ‘must’ or ‘must not.’ 

Otherwise, it is a work procedure, but is not the only safe way to do something. A rule is the 

only safe way.” 

23 
Interview responses also reflected changes in the process of rulemaking. Indeed, many spoke—

often without prompting—of the organizational processes that generated the old rulebooks. The 

old rules, according to these respondents, grew out of a reactive, and seemingly unreflective, 

process by which rules were formulated in response to specific, isolated events. A former frontline worker spoke of a rule requiring that cans be opened with can openers. The rule, on this 

worker’s account, arose after an employee had cut himself trying to open a can of food with a 

pocketknife. As one worker observed: “The old rules—we called them CYA [cover your ass] 

rules—were typically made in reaction to incidents. If someone got injured, they made up a rule 

about it.” 

While stories about can-opener rules are humorous, many of the respondents indicated that even 

seemingly arcane rules could be used as instruments of punishment rather than tools for pursuing 

safety improvements. Another part of the ACSNI definition for safety culture introduced earlier 

states, “Organizations with a positive safety culture are characterized by communications 

founded on mutual trust” (1993). Rules, as described by these respondents, are not founded on 

mutual trust. In a similar vein, a safety executive noted: “Our company’s culture was that the 

safety rulebook was used as a ‘gotcha tool.’ If you do something wrong and get hurt, I’m going 

to find a rule to nail you with—rather than using the rulebook as a safety tool to prevent people 

from getting hurt.” 

These responses to questions about the reasons for undertaking rules revisions generated three 

observations. First, the interview data appeared to support the claim that the blame game 

fostered by FELA was related to problems in carriers’ safety cultures. Indeed, respondents 

reported that the rulebooks, prior to rules revisions, were used primarily to assign or avoid 

blame, not as part of an ongoing attempt to work together to improve safety. Second, the 

interview data appeared to support the claim, embedded in the rules revision program theory, that 

a link existed between the content and pattern of use of safety rulebooks and organizational 

safety culture. This, in turn, supported the notion that improvements in safety culture may be 

facilitated by changes in the content and pattern of use of safety rulebooks. Third, the terms and 

concepts the rules revision consultant used were remarkably consistent to those used by 

respondents during the interviews. This suggested a considerable level of understanding and 

acceptance of the intervention by the employees, an important indicator of the degree and quality 

of implementation. 

Throughout, one must remember that all respondents came from carriers that had previously 

decided to undertake rules revisions. Thus, the evaluation team can make no claim that this 

diagnosis of safety problems resonates industry-wide. In addition, it is important to note that the 

interviewers did not ask respondents directly about the connection between FELA provisions and 

their carrier’s safety rulebooks and cultures. 

4.1.2 The Startup Phase 

If poor safety culture is an important cause of poor safety performance, the process of reform 

must try to leverage culture change. The rules revision process, as discussed in Section 3, seeks 

to improve the culture through a participatory process. How did this work? Who participated? 

What factors helped and hindered this process? 

24 
Most respondents emphasized the inherent difficulty of rules revisions. Some indicated that the 

suspicion, distrust, and conflict generated by the existing safety culture and rulebook structure 

could continue to cast a shadow over the rules revision process. Representatives of crafts, 

according to one safety executive, were concerned that revising and repealing rules might lead to 

more injuries, while management representatives expressed the fear that the process amounted to 

“giving the keys to the inmates.” A respondent indicated that worker representatives had a hard 

time getting past the suspicion that the new rules would be used to hammer people, just like the 

old ones. At least one respondent noted resistance from company attorneys, who continued to 

view the rulebook as a way to indemnify the company against liability for accidents. One 

worker involved in a rules revision process was succinct: “This is not an easy thing to do.” 

Another finding that emerged from the interview data is the importance of management 

commitment (including resources) to the process. In all four cases, the revision process took a 

minimum of 6 months. Most said that the time was required to allow the necessary consensus 

building among management and the numerous crafts. 

Several safety professionals explained that before the start of the rules revision, management 

activities existed to help prepare for the effort, such as discussions addressing issues of distrust 

and suspicion on the part of labor and management. Two of the carriers’ senior management 

became active sponsors of the project, and in another carrier, they were committed but less 

visible.19 Senior management from one of the carriers where visible management were 

committed wrote a value statement that embraced a more participative management style in all 

areas of the company, not just safety. This company held employee meetings and distributed 

videotapes and written materials to communicate the message. Rather than the traditional 

punitive approach, responses to rules violations became more developmental; employees were 

trained and coached, rather than disciplined. This company also had safety committees visit 

other stations to conduct risk assessments designed to improve their ability to assess risk, raise 

their personal awareness of safety, and give the assessed stations feedback on hazards. These 

efforts paved the way for the rules revision, and the new safety rulebook was a highly visible 

way to demonstrate the commitment to participatory culture on the part of management. 

Another carrier had the rules revision consultant work with all of the safety committees before 

the rules revision initiative started to help till the soil by raising and addressing concerns. 

4.1.3 The Drafting Process 

Because the process allowed the carriers to identify issues and open up communication channels, 

most respondents pointed to a number of strategies that allowed the carriers to address, and 

sometimes even overcome, the issues identified. Most agreed that buy-in at all levels of the 

company is a successful outcome of the rules revision process. In particular, respondents noted 

the importance of getting front-line workers involved in the process. As one worker observed, 

“There are a lot of crafts [at the worksite]. We tried to get someone from each area to be 

involved.” Worker involvement appeared to be especially helpful in distinguishing core, 

company-wide safety rules from craft-specific rules and work practices. The worker previously 

quoted went on to observe that, “It’s a lot better for someone who actually performs the work to 

analyze a task and work up a procedure to make employees safe.” Having workers create and 

 

19 CN/IC data were insufficient for commenting on this topic. 

25 
review the rules helped to ensure that the new rules were reasonable and practicable. One 

worker explained: 

There was a rule that you had to wear leggings that covered your leg from 

your knee to your shoe, in certain hot-work areas. Some welders and 

fitters pointed out that if something fell into the top of the legging, it 

would burn you even worse than if you didn’t have the leggings. So that 

got changed... It benefits the company to listen to the employees. A lot of 

these guys have a great deal of sense about them. 

This example illustrates how the rules revision process, by opening up channels of 

communication, tapped into hitherto unused (or perhaps underused) knowledge about safety 

embedded with the workforce. 

Most respondents underscored the importance of commitment and buy-in from top company 

management. According to one worker, such a commitment can help keep the process on track 

even in the face of resistance from detractors. One respondent, however, said he wished his 

company had provided even stronger leadership. 

Respondents emphasized that leadership in the process could not be purely top-down in nature. 

Along with seeking buy-in from front-line workers, reformers should include a steering 

committee that represents a wide variety of interests. As one safety executive noted: 

We established a high-level steering committee—broad-based in 

membership. We identified possible barriers—law, medical, risk 

management, senior level managers from all crafts, union leaders—and 

included them on the steering committee. Then it became easy to cascade 

the efforts down to the ranks and ask for involvement. 

Bringing management and workers together on teams, however, can present its own problems. 

Meetings can often become venues for airing old grievances and giving voice to long-standing 

sources of conflict. One worker stated that carriers must be willing to allow a certain amount of 

venting during such meetings if the process is to succeed: “One guy threw a book through a 

wall; others threw chairs.” Most respondents indicated that such conflicts could be tempered by 

continuing dialogue and the development of a (sometimes new) language for discussing these 

issues. One safety executive said that his company was able to get around this problem by 

casting their discussion of new rules in moral terms, which centered on an emerging consensus 

that all members of the company had a responsibility to see that workers did not get hurt. 

Another safety executive surmised that it is important to let workers and managers vent at first 

but then emphasized that these problems are in the past and that it is time to move forward and 

fix them. 

At least one worker cautioned that sending replacements when a regular member could not 

attend committees and working groups would further slow the process. 

People would come to several meetings, then they couldn’t make one, so 

they’d send a replacement. It takes time for those new people to 

understand—they’d want to bring up the same points that the group had 

already gone over. 

26 
This worker also saw some potential advantages in sending replacements. 

After awhile, I realized it [sending replacements] broadens the base of 

commitment, gives you new ideas. You want them to be quick, but there 

are months between meetings and you need to get everyone on the same 

page first. [It is okay to send replacements, but] replacements have to be 

brought up to speed, and you have to allow time to bring them back 

around. 

Finally, several respondents highlighted the importance of having outside leadership and 

assistance from the consultant. According to one safety executive, the consultant provided not 

only writing, organizational, and technical skills, but also a totally unbiased voice. This person 

continued: 

We’re all a product of where we were programmed. It’s very hard to be 

neutral. I was very slanted on some things and not on others. Railroads 

are so entrenched that they need someone from the outside to be neutral. 

4.1.4 The Roll-Out Phase 

Conventionally, one would expect the managers to roll out finished rulebooks to the entire 

company with much fanfare. In this case, the marketing process actually began during drafting, 

since the rules revision process was participative. Workforce involvement in rules rewriting and 

employee familiarity with new rulebooks greatly facilitated the formal rulebook roll-out phase. 

The fact that interviewees had considerably less to say about the roll-out phase than the drafting 

phase suggests that the roll-out process was less problematic, since broad participation was 

achieved during the drafting phase. 

Several respondents noted that, even after the consensus building during the drafting phase, 

significant pockets of resistance to the new safety rulebooks still remained. Some respondents 

noted that resisters always exist in any change in policy and programs. One worker attributed 

this to long-standing personal relationships that leave some managers unwilling to confront 

workers they have supervised for decades. Some workers viewed safety measures as an 

inconvenience and, he added, would retaliate against new rules by slowing work processes. 

Another worker expressed a similar sentiment: 

Some people still have jaded perspectives about safety. You’ll find a lot 

of people who have been employed here for 20-30 years have a different 

outlook on safety and management. They’ve been around for a long time, 

when things were managed more poorly and safety wasn’t as important. 

They say, ‘Why do I have to wear a respirator? I didn’t wear one for the 

first 28 years, why now?’ 

One safety executive added that, “The further away from headquarters, the less consistency there 

is. You’ve got to communicate the content and intent.” 

The importance of communication was echoed by a number of respondents. Among the 

communication techniques they reported using were feedback sessions with workers, brochures 

mailed to employees’ homes, and reviews of rules at safety meetings. One safety executive said 

27 
that his company reviewed a rule at each of its weekly safety meetings; when they ran out of 

rules, they started over again. At least one respondent noted that the consultant played an 

important role by helping to draft company communiqués about the new safety rulebook. 

Leadership by example provided another vehicle for communication of new rules across work 

sites. One conductor gave the following account of a rule that met resistance from workers. 

There was one major change with the new rules—implementation of a red 

zone for employee protection when moving between railroad equipment. 

That was a major change. The members participated—it was up to us to 

set an example and use it [the new rule] and it caught on. I was concerned 

because it was a completely different—a new signal. I didn’t like the idea 

of replacing an existing signal. I felt we should use the old signal. I didn’t 

like it. But a different signal is not going to make or break this rulebook. 

This comment illustrates the importance of having a core group of employees who are 

champions of the new rulebook and are willing to put it into practice even though they may 

disagree with certain parts of it. 

4.1.5 The Role of Union Officials 

Given the importance of unions discussed earlier, respondents were asked to describe and assess 

the involvement of rank-and-file, as well as officials, in the rules revision processes. Union 

rank-and-file held the majority of positions on the Core and Satellite Teams at each of the 

carriers examined. At one company, general chairmen from the unions were asked to nominate 

people to be involved in the process. Management selected members of the teams from the list 

of nominations. 

Some disagreement existed among respondents about the desirability or necessity of involving 

union officials in the process. A worker at one company—a union rank-and-file employee—said 

that union officials had resisted the rules revision and had sought to derail it. When asked what 

advice he would give to other railroads, he noted that a union official “can really put a stinker in 

the middle of this thing” and create a number of ancillary issues that Core and Satellite Teams 

must address. He added that the official’s message had little resonance with the rank-and-file. 

Indeed, the safety executive of another company noted that the fact that rank-and-file members 

were a part of the drafting process neutralized objections from union leaders. “It’s hard to argue 

with recommendations from your own people,” he observed. 

4.2 Outcomes and Impacts 

As discussed in Section 2, the intended outcome of the rules revision process was to increase 

compliance and reduce injury rates. The theory of change and literature review suggested a 

number of intermediate outcomes underlying the rules revision process that were thought to pave 

the way for reductions in injury rates. This section examines seven outcomes: 

• Changes in values, attitudes, and patterns of behavior related to safety rules 

• Changes in the usability and number of safety rules 

• Changes in union-management relations 

28 
• Perceived changes in compliance 

• Changes in incident rates 

• Changes in injury-related liability and costs 

• Spillover effects in non-safety areas

4.2.1 Changes in Values, Attitudes, and Patterns of Behavior Related to Safety Rules 

The interview data suggested that the rules revision process generated a significant change in the 

values, attitudes, competencies, and patterns of behavior related to safety rules–as envisioned by 

the program theory discussed above. This fundamental change in how rules were valued and 

used was shown in the following quote from a safety professional: 

A main barrier at [our] carrier was defining the definition of a safety rule. 

All rules had been created by management and dictated to the workforce. 

Usually they had been created at the time of a catastrophe. The workforce 

was asked to comply with rules that didn’t make sense. Some were 80 

years old when we had different equipment. 

Most respondents described similar changes, including a shift from a solely management-driven 

rulemaking process to one that emphasized the role of the workforce in deciding what rules were 

worthy of being real rules and each craft’s responsibility for their own rulebook. The fact that 

the process encouraged consensus meant that extensive concurrence existed about every rule in 

the book, which greatly increased the book’s credibility. Another change mentioned was a move 

from confusing criteria about what went into a rulebook to a strict definition of a rule and what 

could be included in a rulebook. 

A worker described the core rules versus suggested work practices: 

(Core) Rules applies to all departments; they’re safety-related, critical to 

safety, enforceable, compliable. If they’re less than that, they might be 

‘departmental procedures,’ which are departmental specific and also safety 

related. There are also ‘suggested work procedures’ which are not 

enforceable. 

Another pattern that changed was the type of language used in the rules. The language 

changed from a “thou shalt” to a “we will” style of writing. Below is an example, shared 

earlier in this report, of an old rule exemplifying the “thou shalt” attitude and how it was 

changed to a rule that assumed more of a “we will” attitude on the part of the worker. 

• Old Rule for Welding-Fire Protection: 

Use only a friction lighter, stationary pilot light, or some other source of suitable ignition 

to light torch. Do not light torch with matches, cigarette, or cigarette lighter (Illinois 

Central Railroad System, 1998). 

29 
30 

• New Rule for Welding-Fire Protection: 

Use only approved flint strikers to light welding torch (Canadian National/Illinois 

Central Railroad, 2002). 

The new rule sounds more like a rule that coworkers would write for themselves. It is 

succinct and focuses on what should be done to perform the job safely.20 

In this next example, the new rule was written with a “we will” attitude, is explicit about 

what is prohibited, and was written to cover a wide range of equipment. 

• Old Rule for Tool Use: 

Employee will use handle, which is standard for the type of jack being operated. Use of 

a bar or handle which does not fit is prohibited. Use of jack handles that have been 

welded is prohibited (Illinois Central Railroad System, 1998). 

• New Rule for Tool Use: 

Use approved tools, equipment, and materials for the purpose(s) intended. Unauthorized 

modifications, overrides to safety devices, and removal of safety guards are prohibited 

(CN/IC, 2002). 

This final example contrasts an old rule that is overly prescriptive with the new rule that is 

simpler to follow. 

• Old Rule for Fork Lift Use: 

Employees are prohibited from: 

− Riding on any machine except when specifically designed for that purpose. 

− Riding on coupling devices or tongues between tractors or trucks. 

− Riding up or down, standing or working from a platform on a forklift truck or 

similar device unless the platform is suitably designed, properly secured, and 

protected by handrails (Illinois Central Railroad System, 1998). 

• New Rule for Fork Lift Use (partial list): 

When operating forklifts: 

− Face the direction of movement with loads. Back loaded forklift down 

ramps/inclines or if view is obstructed. 

− Keep body inside moving vehicle. 

 

20 A question could be raised about whether the rule in this example is written in a way that can easily accommodate 

changes in technology. Rules should empower the workforce, but also be flexible enough to accommodate 

technological advances. How would a shift in technology to electric starters be handled? Would the entire 

consensus process need to be repeated to accommodate this technological improvement? 
− Do not transport passengers. 

− Space forks so that load is fully supported and so that the forks are fully under 

the load (CN/IC, 2002). 

4.2.2 Changes in the Usability and Number of Safety Rules 

One of the most noticeable improvements was the increased understandability of the rulebooks 

due to the reduction in the number and complexity of the safety rules. The collaborative process 

and new definition of a rule helped streamline the books. This also led to increased concurrence 

about the worthiness of the rules that were included and increased the focus on what was 

important. Additionally, the front-line employees benefited from having fewer rules to 

remember. Although having fewer rules is not a success measure in and of itself, when 

combined with the improvements in understandability and concurrence, it provides some insight 

into the program’s effectiveness. 

Table 1 illustrates the reduction in the complexity and number of safety rules for three carriers 

for which data were available during this study. It was difficult to precisely compare the number 

of rules before and after because the old rulebooks had duplicative and conflicting safety rules 

and confusion about which rules applied to everyone and which ones applied only to a given 

craft. In addition, operating rules, training material, and job aids were interspersed throughout 

the rulebooks. To obtain a relatively consistent picture, the evaluation team asked the Hile 

Group to hand count the rules in the original versus new rulebooks for all the carriers. In each 

case, the safety rules before the revision process numbered in the hundreds and were reduced to 

less than 30 core rules. The number of rules should be taken here as an imprecise measure of the 

improved readability, and usability, of the rulebooks, but not as a direct measure of safety. 

31 
Table 1. Reduction in the Complexity and Number of Safety Rules 21 

Company Before Revision After Revision 

ACBL 400 safety and operating rules, job 

aids, training information 

(confusion about which rules 

applied to all employees versus a 

specific craft) 

24 core safety rules (applied to 

everyone) and 101 craft-specific 

rules and recommended work 

practices 

CSXT 900 safety and operating rules, job 

aids, training information 

(confusion about which rules 

applied to all employees versus a 

specific craft) 

17 core safety rules (applied to 

everyone) and craft-specific rules 

for transportation (19), mechanical 

(88), track engineering (number 

not available, but estimated to be 

similar to mechanical), and clerical 

(number not available) 

KCS 742 safety and operating rules, job 

aids, training information 

(confusion about which rules 

applied to all employees versus a 

specific craft) 

17 core rules (applied to everyone) 

and craft-specific rules for 

transportation (93), mechanical 

(242), track engineering (227), and 

clerical (98) 

A respondent from CSXT stated that the company has since added 300-400 new rules, following 

a change in senior management in 1995. Interestingly, the ownership for the rules at CSXT has 

not shifted back to management, “The amount of craft participation has been upheld. There is 

strong involvement in the Core Safety Committee.” 

4.2.3 Changes in Union-Management Relations 

Most respondents reported improvements in union-management relations. Worker respondents 

were asked to rank union-management relations on a scale of 1 to 5 (with 1–very hostile and 5–

very cooperative) both before and after the rules revision. Of the six who provided ratings,22

four reported ratings that were higher after the rules revision (they indicated improvements of 1, 

 

21 As mentioned earlier, CN/IC was initiating rules revision when the initial interviews were conducted and therefore 

were not included in the impact analysis. In fall 2002 when the Hile Group provided the before and after 

comparison of the number of rules, CN/IC had completed its rulebook. CN/IC initially had “1,360 safety and 

operating rules, job aids, training information (some craft-specific and some for general use)” before and “18 core 

safety rules (applied to everyone), and craft-specific rules for transportation (62), mechanical (259), track 

engineering (216), and clerical (133)” after. 22 Only six respondents provided ratings because this question was added to the interview protocol in the middle of 

the interview process. 

32 
2, 3, and 4 points). A fifth respondent reported no change in union-management relations (a 

rating of 3 both before and after). A sixth said that, while relations between management and 

rank-and-file union members had improved from a 2 to a 4, relations between management and 

union officials had deteriorated from a 2 to a 1. 

One worker who reported an improvement in union-management relations observed that the 

rules revision process “helped the relationship between management and the union. It showed 

employees that management was willing to listen.” A worker from another company suggested 

that the rules revision process also helped to draw the unions into a more constructive dialogue 

on safety: 

There is a lot more safety involvement by the unions. Local reps work 

with management on safety at the local level. Local unions didn’t 

participate before—maybe they’d show up for [the] meeting and that was 

it. There was like a fence you didn’t cross. The book process served to 

draw both sides in. Now you have more involvement. 

Another worker observed that management and workers appeared to have placed greater 

emphasis on worker safety since the rules revision. They reported: 

Five years ago, safety wasn’t a big deal. We’ve accomplished a lot. 

These people [management] are genuinely interested in safety. Rules 

weren’t their main priority, but that has changed... I see that men at the 

superintendent level care about safety. For example, we have a lot of 

switches that are notoriously hard to throw. There is a regular program to 

oil and maintain those switches. If there is a lot of rain or a flood, they are 

out there working on it. It will be fixed. Before, if something needed 

fixing, it might not get fixed at all if they didn’t consider it bad enough. 

Now if I turn something in [report something that might be unsafe], it gets 

fixed right away. 

It’s been a complete turnaround... There had been safety committees 

going on for several years. Not much was being accomplished. Only a 

few men were participating, now there’s more participation and more is 

getting accomplished. [Now] everything is leading toward our #1 goal, 

which is to be the safest railroad in the industry. 

Reports that management and employees were more likely, after the rules revision process, to 

engage in open communication about safety was more directly relevant to the issue of 

management and labor relationships. One safety executive observed that: “The consolidation 

was an outstanding success. It helped develop trust. Craft employees know they have had a 

hand in putting together the rules.” 

Another safety executive took the point further: 

With the new culture, it is easier to confront people engaged in unsafe 

behavior, since interveners can cite rules in which line workers’ peers had 

a part in creating. Before people were doing crazy things like jumping off 

elevated surfaces, wearing short sleeve shirts. After, 90 percent of people 

33 
were complying with rules; they were wearing [personal protective 

equipment (PPE)]; supervisors were more educated. They weren’t 100 

percent safe, but much improved. 

4.2.4 Changes in Compliance 

Respondents were asked to comment on the extent to which the rules revision process led to 

changes in compliance with rules, the degree to which there was a focus on safety, and other 

factors related to safety culture. 

Generally, agreement existed among respondents that the rules revision process had been 

associated with improvements in rules compliance. Safety executives reported that they 

observed fewer at-risk behaviors among employees. As one worker noted: 

It seems like more people are abiding by the rules. For example, with 

PPE: there is a chart in the book about PPE. You don’t see people 

without their hardhats, glasses, safety shoes. I never see people without it. 

I see 100 percent participation. If not, you approach them in a friendly 

manner, and don’t be critical. 

One executive cautioned against attributing too much causal influence to the rules revision 

process specifically. This respondent noted that commitment by management to worker 

participation was a necessary precursor to culture change. Rules revisions can be a useful 

vehicle for improving safety culture but not without a prior commitment to collaborative 

decision processes. 

While the interview data on rule compliance was encouraging, the evaluation team was not able 

to confirm the compliance improvement independently. It would have been helpful to have 

company records related to safety rule violations, as a proxy indicator of compliance, to confirm 

the interview data. If companies do not keep such data as a part of their FELA records, a 

secondary analysis of the injury data could be performed to determine if any rules were violated 

when injuries occurred. The additional data could then be used to examine whether rule 

violations decreased only for the rules that became core rules or if there was a more generalized 

effect. It would be helpful to know if safety culture improvements led to a decrease in rule 

violations across the board. 

4.2.5 Changes in Incident Rates 

One of the most important indicators of success for any safety intervention is the extent to which 

it leads to reductions in incidents. While the safety executives interviewed said they undertook 

the process primarily to improve their rulebooks and obtain more participation from the 

workforce in safety governance as outlined previously, they did report that incident rates had 

improved as a result of the rules revision process. In order to assess the rules revisions’ impacts 

on injury rates, the evaluation team gathered data on the incident rates at the carriers who had 

completed the process: CSXT, KCS, and ACBL. The incident rates, presented in Figures 2, 3, 

and 4, may be interpreted as the number of employees injured in a year per 100 employees 

working full-time. 

34 
In each case, time series data for the treatment group was compared with similar data for the 

remainder of the industry, including both Class I and short lines (excluding the treatment firm). 

This analysis allowed evaluators to assess whether any observed changes in the treatment firms 

might have happened without the intervention (Shadish, Cook et al., 2002). 

CSXT 

Figure 2 shows that incident rates at CSXT had been generally improving since the mid-1980s. 

Thus, the key question is whether the rules revision led to additional safety improvements. 

Visual inspection of Figure 2 suggests that a sharp drop occurred in injury rates between 1989 

and 1990. Given that the formal rules revision process did not begin until 1991, it seems 

unlikely that the rules revision per se caused the change in incident rates. Indepth interviews 

with company officials, however, indicated that rules revision was preceded by a preparatory 

process that included extensive union-management collaboration designed to increase worker 

involvement in safety and emphasize management commitment to safety (see Section 4.1.1). If 

the evaluation team takes these preparatory activities as the beginning of the intervention, then 

evidence exists that rules revision preceded the drop in injury rates. Statistical analysis suggests 

that this immediate drop was statistically significant (see Table A-2). 

12

1983 1985 1987 1989 1991 1993 1995 1997 1999 2001 2003

Incident 

Rate

Year

10

 8

 6

 4

 2

 0

Management

Preparation, 

1989-1990

Conrail 

Merger,

1998-1999

Rules 

Revision,

1991-1993

Senior

 Management

Change, 1995

CSX-T

Industry 

Figure 2. CSXT and Industry Incident Rates Comparison, 1983-2002 

Thus, it appeared that rules revision, combined with management preparatory activities, led to 

additional safety improvements–at least in the short-run. Examination of the industry series 

suggested, however, that while improvements at CSXT preceded similar improvements in the 

industry, they were not unique. Indeed, statistical analysis suggested that the pre-/postdifference at CSXT was indistinguishable from that in the industry at large (p = 0.60). Thus, 

while evidence existed of a possible positive impact on incident rates at CSXT, the fact that these 

improvements were mirrored–if belatedly–by the remainder of the industry raised the possibility 

that they were caused by something other than rules revision. 

35 
The evaluation team noted that the increase in the rate of safety improvement during the early 

1990s was followed by a gradual leveling off, with rates on the rise again by the late 1990s. This 

addressed the question of the sustainability of any rules revision impact. One interpretation of 

the reversal is that it reflects a general wearing off of the intervention’s impact. Alternatively, it 

might reflect the intervention’s lack of resiliency to changes in the organizational and task 

environments. It could indicate that management thought that safety was under control and 

became distracted by the leadership change at the company in 1995. As one respondent 

observed, 

We lost a senior leader in [1995] and it had tremendous impact. With new 

leadership, the rulebook came under fire. We didn’t have time to develop 

deep roots to the change effort and were in the middle of implementation 

when leadership changed. The new senior leader said the rulebook looked 

like a cartoon. There was a reversion back to the pre-[1989] status quo to 

a certain extent, but not completely. 

Industry experts have stated that safety is a constant struggle. After the leadership change, 

CSXT merged with Conrail in 1999. The addition of Conrail’s rules to CSXT safety rulebooks 

and the reduction in clarity of organizational norms may have fueled an increase in incident 

rates. Other exogenous factors that might have contributed were technology, reduced employee 

expertise due to an influx of new hires and training. Finally, the reverse in rates might reflect 

some sort of floor effect. That is, the effort costs of reducing injuries might be much greater at 

lower incident rates, creating barriers to further reductions. While available data were not 

conclusive, it was clear that resiliency of the rules revision intervention is an issue that bears 

further investigation. 

KCS

Similar data were gathered for KCS. Once again, interview data suggested that the intervention 

began not with the formal rules revision (2000), but with a number of preparatory management 

activities in 1999. These earlier activities were designed to improve trust between management 

and workers and featured changes by management to make the rules violation reporting process 

more developmental and less punitive. In addition, the rules revision consultant was hired to 

work with all the safety committees to identify issues, build trust, and obtain agreement from 

labor to rewrite the safety rules in a participatory manner. 

As in Figure 3, the preparatory management activities were associated with a statistically 

significant increase in the rate of improvement (p < 0.001). Unlike CSXT, the post-intervention 

improvements at KCS were not mirrored in the remainder of the industry. Indeed, the difference 

in pre-/post-incident rate changes between KCS and the remainder of the industry was found to 

be statistically significant,23 lending credence to the claim that the rules revision and 

management preparatory work at KCS (rather than some other factor common to the entire 

industry) caused the improvements. 

 

23 Scientific convention generally adopts a significance level of 0.05 as the criterion for deciding if an event is 

statistically significant. Given the minimal KCS data available, and the exploratory nature of the study, 0.07 was 

considered worthy of consideration because of the potential benefits a positive result could have for the railroads. 

36 
New 

Technology,

Mid 2001

Management

Preparation, 

1999-2000

Rules

Revision,

 2000-2001

12

1995 1997 1999 2001 2003 2005

Incident 

Rate

Year

10

 8

 6

 4

 2

 0

KCS

Industry 

Figure 3. KCS and Industry Incident Rates Comparison, 1995-2002 

These positive findings were limited by the fact that relatively few post-intervention observations 

existed. The challenge was how to sustain safety improvements in the face of various exogenous 

factors; indeed, an increase occurred in incident rates during 2002. One KCS safety executive 

explained that this increase might correspond with the introduction of a new car control system 

that led to delays and fatigue issues, as well as other technological changes. Once again, this 

raised questions about the resiliency of the intervention to changes in organizational and task 

environment. 

ACBL

ACBL had management preparatory activities in 1994, including collaborative sessions where 

managers and workers established core values, goals, and objectives. As with CSXT, evidence 

existed that incident rates were already improving before the rules revision. Unlike CSXT, no 

statistical evidence existed that the rules revision process, combined with the management 

preparatory activities, was followed by any additional improvements in safety outcomes (see 

Figure 4). In fact, a slight, though statistically insignificant, reduction occurred in the rate of 

safety improvement (p = 0.13). A lack of pre-/post-difference might be viewed as positive had 

an increase in incident rates occurred in the remainder of the industry. While a slight, though 

statistically insignificant, increase occurred in the industry (p = 0.36), the increase at ACBL was 

larger and statistically different from the industry increase (p = 0.10). In short, no evidence 

existed of any pre-/post-incident rate changes associated with rules revision at ACBL. 

The null findings might be due, in part, to confounding of the data as a result of the four mergers 

and acquisitions between 1996 and 2000. 24 Therefore, when one reviews the ACBL data, it is a 

combination of ACBL plus four additional companies. While independent pre-merger/ 

 

24 The mergers and acquisitions involved ContiCarriers and Terminals, Inc., in 1996, National Marine, Inc., in 1998, 

Peavy Barge Line in 2000, and Ultrapetrol in 2000. 

37 
38 

acquisition injury data were not available for the other carriers, the safety professional 

interviewed stated that the strong safety culture that ACBL had achieved, due in part to the rules 

revision, helped to smooth out the merger process and helped the other companies to lower their 

injury rates as they joined. For instance, “National Marine had an injury rate of 11 before they 

joined ACBL and dropped to 4 the year they merged (1998).” The respondent stated that if the 

mergers and acquisitions had not occurred, the slope change at ACBL might have been steeper. 

Unfortunately, the data did not allow the evaluation team to test these claims empirically. 

Incident 

Rate

Year

Management 

Preparation, 

1994-1995

Rules 

Revision,

1995-1996

4 Mergers,

1996-2000

 0

 2

 4

 6

 8

 10

 12

 14

 16

ACBL

Industry 

1990 1992 1994 1996 1998 2000 2002

Figure 4. ACBL and Industry Incident Rates Comparison, 1999-2000 

Department-Level Rates 

It is possible that incident rates measured at the department level could show significant 

pre-/post-impacts. One rail safety executive mentioned variation in implementation at the 

department level: “Mechanical had good sharing with one session a week. Not as good as 

[Track] Engineering. Clerical was okay. Transportation–it was hard to get them to spend time.” 

When asked about their injury rate, a rail mechanical worker also indicated a potential difference 

by department. He seemed to track only his department’s injury rate. “There was a 40-50 

percent reduction in Mechanical. We’re at about 1.82 for car maintenance (train yards, repair 

shops locomotive service not attended to by locomotive operations) and .85 for locomotive 

operations (located in Georgia, New York).” 
Incident Rates and Specific Rules 

Due to limitations in the data set, the analysis only tested for a reduction in total incidents. It is 

possible that the incident decreases observed were primarily due to a reduction in incidents that 

specifically related to the new core rules. It would be useful to test whether only the injuries 

related to the new rules went down or whether all injuries were reduced due to a generalized 

effect of improved safety culture. Section 4.2.4 expressed a similar sentiment, to look more 

closely at the relationship between the outcome measures and the specific rules. 

Summary of Incident Rate Findings 

To summarize the results, a preliminary analysis of incident rate data found possible evidence of 

a positive impact at CSXT and KCS and no evidence of impact at ACBL. The impacts were 

described as possible at CSXT because of questions about whether the improvements observed 

could have been caused by factors that helped not only CSXT, but also the entire industry, and at 

KCS because of the paucity of post-intervention data. In addition, as noted above, the findings 

for CSXT and KCS were highly sensitive to the assumption that the intervention began with 

management preparatory activities. The ACBL findings were subject to uncertainty because 

while rates did not decline, they did not increase in spite of several mergers after the rules 

revision. Thus, while some evidence existed that rules revision holds promise as a mechanism 

for reducing incident rates, these findings were subject to considerable uncertainty. 

During this study, it was not possible to determine the extent to which departmental 

implementation effectiveness impacted departmental injury rates or whether rules that were 

included in the new rulebook impacted rule-related injury rates. Additional data and analysis 

will be required to distinguish these relative effects. In addition, given the data available, it was 

impossible to assess whether rules revision was more important than the management 

preparatory work or vice versa, since the evaluation team could not distinguish their relative 

effects. In this analysis, their combined effect was investigated, since both were concerned with 

changing safety-related values, attitudes, and patterns of behavior. 

4.2.6 Changes in Injury-Related Liability and Costs 

Two carriers noted that rules revision decreased the number of claims and the claims payouts. 

One of the respondents reported that she was told, “Safety is contributing to the bottom line 

because of the drop.” Another person mentioned claims decreasing both in payout and number: 

“We also saw our claims go down significantly... The claims ratio dropped, but also the cost per 

claim dropped by 45 percent between [1995] and today. Claims data is very closely held so there 

is no written documentation that can be given out.” 

The evaluation team was not able to obtain documentation to substantiate these statements. , 

Inclusion of the information, however, seemed warranted given the exploratory nature of the 

study. The third carrier that completed the effort did not mention this outcome. The fourth 

carrier did not yet have any outcome data. 

In addition to the reduction in claims, another FELA-related impact concerns the reduction in the 

number of rules. One respondent pointed out a legal advantage to having fewer rules. 

39 
... We always included risk management and legal in the Satellite and 

Core Teams. Didn’t want them to feel in the end that the rulebook was not 

defensible in court. The reason it is considered more defensible is because 

there are fewer rules. With fewer rules it is possible to do more indepth 

training. With employees participating there is more agreement on what is 

important to enforce in all situations on the side of labor and management. 

In addition the ‘preferred work practices’ category puts responsibility for 

determining and using the ‘preferred work practices’ on the shoulders of 

both management and labor. If a rule cannot be enforced due to a facility 

or other reason a set of preferred work practices are identified and anyone 

can use them. 

4.2.7 Spillover Effects in Non-Safety Areas 

One executive added that collaboration on rules revisions can sometimes create positive 

spillovers in non-safety areas by creating important organizational precedents for collaborative 

decision processes. One respondent stated that after the rules revision project occurred, joint 

management-labor groups came together to implement improvements in several areas other than 

safety, such as reengineering and other cost-cutting projects. This respondent cited workforce 

involvement in efforts to improve such productivity-related issues as traffic flow, turn time, cost 

reductions, and fuel efficiency as examples of non-safety areas addressed by the same 

collaborative relationships created for rules revision. Previously, the labor management 

relationship at this carrier had been strained, and collaborative work on productivity issues did 

not occur often. 

40 
5. Conclusions and Recommendations 

More than a mechanism for reducing paperwork burdens and administrative complexity, the 

rules revision process seeks to develop consensus around core and craft safety rules and, in so 

doing, improve rules compliance and safety culture, thus reducing injury rates. This section 

presents a revised version of the program theory, evaluates key program impacts, summarizes 

and discusses key findings on the implementation of rules revisions, and provides suggestions for 

further work. 

5.1 Revised Program Theory 

Revisions to the program theory presented in Section 2 are provided to show a more complete 

and informative picture of the process. Figure 5 shows the original program theory above the 

revised one to facilitate comparison. Boxes in the revised program theory that were added or 

modified have bold outlines. 

Original Program Theory 

Figure 5. Original and Revised Rules Revision Program Theory 

Revised Program Theory 

• Change in 

roles and responsibilities 

• Core teams 

• Satellite 

teams 

• Change in 

definition of a 

safety rule 

• Reduction in 

safety rules 

• Broad 

participation

High volume 

of rules due 

to: 

• Injury 

avoidance 

• Mergers 

• FELA 

Conditions 

Leading to 

Rules 

Revision 

Program 

Activities 

Roll Out and 

Communication 

of New Safety 

Rulebook 

Long-Term 

Outcomes 

• Safety culture 

• Injury rates 

(departmental 

versus total; 

related to 

specific rules) 

• Reductions in 

injury-related 

liability and 

costs 

Management 

Preparation 

• Commitment 

to 

participatory 

culture 

• Inclusion of 

labor

Management/ 

Workers 

Change 

Behavior 

• Involvement 

of workers in 

safety 

activities 

• Compliance 

• Problem 

solving 

between 

management 

and workers

• Commitment and 

ownership from 

front-line 

workers, safety 

professionals, 

human 

resources, legal 

• Implementation/ 

communication of 

new rulebook 

Conditions 

Leading to 

Rules 

Revision 

Program 

Activities 

Roll Out and 

Communication 

of New Safety 

Rulebook

Long-Term 

Outcomes 

• Safety 

culture 

• Injury rates 

High volume 

of rules due 

to: 

• Injury 

avoidance 

• Mergers 

• FELA 

• Change in 

roles and 

responsibilities 

• Core teams 

• Satellite teams 

• Change in 

definition of a 

safety rule 

• Reduction in 

safety rules 

• Broad 

participation 

• Commitment and 

ownership from 

front-line 

workers, safety 

professionals, 

human 

resources, legal 

• Implementation/ 

communication of 

new rulebook

41 
Several safety executives said that senior management became active and visible in their 

commitment to building a participatory culture before the rules revision process began, and the 

inclusion of labor was important. 25 Therefore, a step was added to the program theory diagram 

to account for management preparation. 

Another box was added to the program theory to address changes in management and worker 

behaviors following the roll out of the new safety rulebook. Interview data suggested that the 

level of increased involvement in safety activities, such as safety committees and peer-to-peer 

feedback of at-risk behavior, occurred, which may be necessary before changes in injury rates 

can be observed. Compliance and problem solving between management and workers are other 

outcomes that were included in this box in the program theory. 

Changes were made to the final box in the program theory. The injury rate outcome was revised 

to allow for the possibility that the injury impact might be observed at the department level 

and/or that the injury impact might be observed at the specific rule level. Another bullet was 

added to show that reductions in injury-related liability and costs are perceived to be a long-term 

outcome of the rules revision process. 

5.2 Program Impacts 

Data presented in this report suggest a number of benefits associated with rules revision. 

Evidence on bottom-line outcomes was inconclusive. Evidence on other outcomes, however, 

was more clearly positive. 

5.2.1 Bottom Line Benefits 

• Incident rate improvements. A preliminary analysis of incident data at three carriers 

suggested that safety rules revision (combined with other senior management activities to 

increase workforce participation in safety) had a positive impact on incident rates at one 

rail carrier (KCS), where a statistically significant (p < 0.01) improvement occurred in 

incident rates beginning in 1999. The management activities, which began before the 

actual rules revision effort, included a more developmental approach to rule violations 

and the hiring of a rules revision consultant to identify issues and build trust. Incident 

rate declines at the other two carriers could not be attributed with complete confidence to 

the process because other changes occurred at the same time, but available data do not 

fully rule out a positive impact on incident rates. During this study, it was not possible to 

assess the extent to which implementation effectiveness varied between different 

departments within the carriers or whether a reduction existed in the specific types of 

incidents related to the rules in the new rulebook. In addition, given the data available, it 

was impossible to assess whether rules revision was more important than the 

management preparatory work or vice versa, since the evaluation team could not 

distinguish their relative effects. In this analysis, their combined effect was investigated, 

since the management preparatory work and rules revision are both concerned with 

changing safety-related values, attitudes, and patterns of behavior. 

 

25 This was not discussed with CN/IC. 

42 
• Reduced liability and injury costs. Safety executives at two carriers reported that the 

number of FELA claims and the cost per claim dropped significantly as a result of the 

effort. One executive suggested that FELA-related costs for his company decreased due 

to the increased clarity of the rulebooks and the decrease in the number of rules. 

• Spillover effects in non-safety areas. One executive added that collaboration on rules 

revisions can sometimes create positive spillovers into productivity-related areas by 

creating important organizational precedents for collaborative decision processes. One 

respondent noted an increase in worker involvement on activities they previously 

considered management work (reengineering, cost-cutting activities). 

5.2.2 Other Benefits 

• Changes in values, attitudes, and patterns of behavior related to safety rules. The rules 

revision process shifted the primary responsibility for defining safety rules from the 

management to the workforce. Respondents reported that the revised rules focused on 

significant safety issues and could be complied with 100 percent of the time. Compliance 

was often not possible with the previous rules. 

• Changes in the usability and number of safety rules. As expected, rules revisions 

dramatically reduced the number of safety rules through the filtering process employed 

by the three carriers studied. The new book was important because it was a visible 

symbol that reached all employees and demonstrated a commitment to safety. The 

increased clarity made the rules easier to follow, improved utilization, and better 

protected the workers. 

• Improvements in union-management relations. Most respondents reported improvements 

in union-management relationships. These workers reported increased trust that 

management was committed to making the workplace safer. One respondent reported no 

change, and another reported improvement in management relations with rank-and-file 

workers but deterioration in management relations with union leadership. 

• Perceived changes in compliance. Most respondents indicated that the rules revision 

process had brought greater compliance and greater safety consciousness. 

5.3 Implementing Safety Rules Revisions 

The evaluation found a remarkable degree of consistency in the way that respondents understood 

and described the process and the reasons for undertaking it. Most noted that their previous 

safety rulebooks were developed unreflectively and reactively in response to particular safety 

incidents. The end results were extensive, and cumbersome rulebooks were viewed by 

employers as insurance against liability for accidents and viewed by workers as hammers used to 

punish workers. It appears that these rulebooks both reflected and promoted unhealthy safety 

cultures, which featured poor communication, unclear expectations, and lack of trust. While 

most respondents did not speak of the legal climate, it seems likely that the FELA provisions 

described in Section 2 contributed to these complex rulebooks and undesirable safety cultures. 

Interviews suggested a number of promising strategies for implementing rules revision. 

43 
• Senior management preparation activities. Safety executives from three of the 

companies stated that the rules revisions were preceded by senior management 

activities to increase the participation of the workforce in safety.26 The rules 

revisions occurred after senior management had started actively to involve labor in 

an attempt to make the safety culture more participative. 

• Participation. Buy-in and participation at all levels of the company, from front-line 

workers to senior management, was critical to successful implementation. Front-line 

worker involvement was central to the writing of the rules, and front-line supervisor 

commitment was especially important once the new rules were implemented. 

• External help. Project facilitation by the external consultant was found to be helpful. 

Since long-time workers and managers were entrenched in the status quo, an 

external, objective perspective was valuable and helped keep the process on track. 

• Involvement of unions. All respondents noted the importance of having rank-and-file 

union members involved in the process. They differed in their opinions about the 

extent to which involvement by union officials was desirable/necessary. 

Respondents also identified a number of challenges. 

• Resistance to change among some groups. Several respondents noted pockets of 

resistance to the rules revisions. They cited managers who feared that the process 

would amount to “giving the keys to the inmates” and labor members who suspected 

that the new rulebook would be used to hammer people, just like the old one. Most 

respondents said that companies must allow time for differences in opinion to be 

worked out. Some suggested that individuals could facilitate the process by leading 

by example (e.g., embracing and following the new rules). 

• Turnover on committees. Since committees included a mix of labor and 

management, it took time to develop trust. Working through difficult issues also 

took time and often occurred over multiple meetings. When a high degree of 

turnover occurred on committees, the process was especially slow. 

• Resource requirements. The process was time consuming and resource intensive. In 

addition to the costs of the employee time involved, most felt the need for an 

objective outside consultant. These costs could make the process prohibitive for the 

short lines and railroads with scarce resources. Railroads should not undertake this 

program unless they plan to see it through and sustain it long term. 

5.4 Future Directions 

The research and evaluation activities in this study were exploratory but indicate the importance 

of organizational factors on safety. The following recommendations for further activities build 

on the findings presented in this report. 

• Reduction in injury-related liability and costs. Safety executives from two carriers 

reported reductions in the number and cost of FELA claims as a result of rules 

revision. This could not be substantiated due to the lack of data from the carriers but 

is worth pursuing in the future, given the significant costs associated with FELA 

 

26 The topic of senior management preparation was not discussed with CN/IC. 

44 
claims in the industry (Section 2). The impact of safety rules revision on other 

injury-related costs should be studied more closely as well. 

• Senior management involvement. Safety executives from three carriers discussed 

senior management preparatory work that involved including labor to make the 

safety culture more participative. In this study, it was not clear whether rules 

revision was more important than the management preparatory work or vice versa, 

since the evaluation team could not distinguish their relative effects. Since positive 

results were found, especially at KCS, while analyzing their combined effect, future 

work should more closely examine the role of senior management in supporting rules 

revision and other safety interventions. In addition, management involvement after 

the preparatory activities must be considered to address sustainability. For instance, 

how do you keep senior managers involved when a change at the very top occurs? 

Several studies have pointed out the importance of senior management to safety, as 

well as the bottom line. The United Kingdom’s Railway Safety organization 

developed a strategic safety management orientation program for senior managers to 

address issues raised about the role of senior management in the aftermath of the 

Clapham accident (Nelson, 2002). Analyzing other ways management can engage 

with labor to improve safety could prove to be very beneficial. 

• Role of union officials. Respondents from one carrier noted that union officials 

resisted the rules revision. Future activity might explore the role of union officials in 

rules revisions, the source of their reservations, and their relations with rank-and-file 

workers and management; both of whom tended to be uniformly enthusiastic about 

rules revision. 

• Injury data impact assessment at different levels. Core safety rules apply across the 

carrier, and craft-specific safety rules apply within a particular craft. Although the 

impact assessment provided in this report took the carrier as the unit of analysis, the 

extent and quality of implementation may vary by department. Thus, future research 

activities should look for variations in implementation across departments or crafts to 

test whether the safety outcome trends differ. Further, it is possible that a reduction 

occurred only in the types of incidents that specifically relate to the rules in the new 

rulebooks, rather than a generalized incident rate impact. Future activities should 

examine injuries related to the new rules and test for variations in safety outcomes. 

The department management or representing unions could use this information to 

improve safety. 

• Measure of compliance. The interview data obtained in this study stated that 

compliance with safety rules improved as a result of the process. In the future, it 

would be helpful to obtain company records of safety rule violations, as a proxy for 

compliance, to confirm the interview data. The analysis could determine whether 

rule violations decreased for all rules or only for the core rules. A general decrease 

in rule violations could be related to improvements in safety culture. 

• Sustainability. At least one of the carriers studied has since added a considerable 

number of rules to its safety rulebook (this was coincident with the departure of the 

senior executive who sponsored the rules revision effort), raising the question of the 

sustainability of rules revisions. Future activity might track rulebooks and safety 

practices in carriers that have undertaken rules revisions several years after the end 

45 
of the process to assess the durability of the reforms in these carriers. In particular, 

does the pattern of workforce participation and rule drafting continue? Are core and 

craft rules still 100 percent able to be complied with and not in conflict with other 

rules? Do crafts still own their own rules? Finally, are subsequent increases in the 

number and type of rules related to increases in incident rates? 27

• Resource intensiveness. Most respondents mentioned that the process was time 

consuming and required the use of an external consultant. Future activity should 

investigate refinements to the process that make it less time intensive and costly 

while achieving the same level of consensus. For instance, the findings on the 

importance of leadership suggest that leadership coaching might be a cost-effective 

way to achieve some safety improvements. Reductions in resource requirements 

would be particularly essential for the smaller and more resource-constrained 

railroads. Future research should also study the return on investment for this 

intervention. 

In conclusion, this study found rules revision worthy of future study. Despite the lack of 

conclusive quantitative data, the interviewees agreed that safety rules revision had an overall 

positive benefit. Moreover, the outcome data, while inconclusive, suggested a number of 

possible benefits worth further exploration. The potential savings associated with a possible 

reduction in FELA claims alone merit future investigation.

 

27 More generally, the renascence of rules proliferation might allow for a removed treatment analysis, in which 

evaluators test not only whether the advent of the intervention leads to improvements in outcomes, but also whether 

withdrawal of the intervention leads to deterioration of outcomes. Such designs increase evaluators’ ability to 

establish whether an intervention is both a necessary and a sufficient condition of the observed outcomes (Shadish, 

2002). 

46 
6. References 

ACSNI Study Group on Human Factors. (1993). Third Report, Organising for Safety. London: 

Health and Safety Commission. 

Ashenfelter, O. (1978). Estimating the effect of training programs on earnings. Review of 

Economics and Statistics. 

Bailey, C.W., and D. Petersen. (1989). Using perception surveys to assess safety system 

effectiveness. Professional Safety: 22-26. 

Burlington Northern Santa Fe Corporation. (2001). Q 3 Filing–Third Quarter Filing: Securities 

and Exchange Commission. 2002. 

Canadian National/Illinois Central Railroad. (2002). L.I.F.E.: Live Injury-Free Everyday. 

Card, D. and D. Sullivan. (1988). Measuring the effect of subsidized training programs on 

movements in and out of employment. Econometrica 56(3): 497-530. 

Carder, B. and P.W. Ragan. (2003). A survey-based system for safety measurement and 

improvement. Journal of Safety Research 34(2): 157-165. 

Carroll, J.S. (2000). Beyond Compliance. MIT/CCPS Workshop, Cambridge, MA, MIT Sloan 

School of Management. 

Chen, H. (1990). Theory-Driven Evaluations. Newbury Park, CA: Sage Publications. 

Chen, H., and P. Rossi, eds. (1992). Using Theory to Improve Program and Policy Evaluations.

New York: Greenwood Press. 

Coleman, C.D. (2001). W&H Main Yards: The Family Tree of North American Railroads. 

Available at http://www.spikesys.com/Trains/fmly_tre.html. 

Coplen, M.K. (1999). Compliance with Railroad Operating Rules and Corporate Culture 

Influences: Results of a Focus Group and Structured Interviews. Washington, DC: Federal 

Railroad Administration. 

Elling, M.G.M. (1991). Veiligheidsvoorschriften in de industrie (Safety Rules in Industry). 

Faculty of Philosophy and Social Sciences Publication WMW No. 8. Netherlands, University of 

Twente. 

Evans Planning Group. (1998). Safety Culture Review. Washington, DC: Federal Railroad 

Administration. 

Federal Railroad Administration. (1983-1996). Accident/Incident Bulletin. Washington, DC: 

Federal Railroad Administration. 

Federal Railroad Administration. (1997-2002). Railroad safety statistics: Annual report. 

Washington, DC: Federal Railroad Administration. 

47 
General Accounting Office. (1996). Federal Employer's Liability Act: Issues Associated with 

Changing How Railroad Work-Related Injuries Are Compensated. Washington, DC. 

Greene, W. (2000). Econometric Analysis. Upper Saddle River, NJ: Prentice Hall. 

Grote, G. and C. Kunzler. (1996). Safety culture and its reflections in job and organizational 

design: Total safety management. International Journal of Environment and Pollution 6: 618-

631. 

Grote, G., C. Kunzler, et al. (1997). Understanding safety culture in organizations-The concept 

of Total Safety Management and its practical use in audit instruments. Advances in Safety and 

Reliability-Proceedings of the ESREL '97 International Conference on Safety and Reliability. C. 

Guedes Soares. Oxford: Elsevier: 71-78. 

Hale, A. (2000). Railway safety management: The challenge of the new millennium. Safety 

Science Monitor 4(1 Article 1): 1-15. 

Hofmann, D.A. and A. Stetzer. (1996). A cross-level investigation of factors influencing unsafe 

behaviors and accidents. Personnel Psychology 49(2): 307-339. 

Illinois Central Railroad System. (1998). Safety Awareness. 

Kaufman, H. (1977). Red Tape: Its Origins, Uses, and Abuses. Washington, DC: The 

Brookings Institution. 

Kmenta, J. (1986). Elements of Econometrics. New York: Macmillan. 

McDowall, D., R. McCleary, et al. (1980). Interrupted Time Series Analysis. Beverly Hills, 

CA: Sage Publications. 

Naumann, S.E. and N. Bennett. (2000). A case for procedural justice climate: Development 

and test of a multilevel model. Academy of Management Journal 43(5): 881-889. 

Nelson, A. (2002). Developing the competency of managers in the field of safety management. 

Tokyo, Japan: International Railway Safety Conference. 

Ostrom, L.C., C. Wilhelmsen, et al. (1993). Assessing safety culture. Nuclear Safety 34(2): 

163-172. 

Ranney, J. (1996). Unleashing the Power of Corporate Teaming. Lexington, MA, Mercer 

Management Consulting: 1-9. 

Ranney, J., and K. Chang. (2001). Baseline Data Collection Summary Report: Amtrak Safety 

Rules Revision Process–Safer Amtrak from Effective Teams. Cambridge, MA: Federal Railroad 

Administration. 

Reinach, S., and J. Gertler. (2001). An Examination of Railroad Yard Worker Safety.

Washington, DC: Federal Railroad Administration. 

Rochlin, G., T.R. LaPorte, et al. (1987). The Self-Designing High-Reliability Organization: 

Aircraft Carrier Flight Operations at Sea. Naval War College Review, Autumn: 76-91. 

48 
Shadish, W., T. Cook, et al. (2002). Experimental and Quasi-experimental Design for 

Generalized Causal Inference. New York: Houghton Mifflin. 

Simard, M. and A. Marchand. (1997). Workgroups' propensity to comply with safety rules: 

The influence of macro-micro organizational factors. Ergonomics 40(2): 172-188. 

49 

Appendix A. Statistical Analysis of Incident Rate Data 

Formal statistical analysis, which provided a more precise characterization of the magnitude of 

the effects, supplemented visual inspection of treatment and comparison group incident rates 

series (see Section 4.2.5). In keeping with the use of the control series quasi-experimental 

design, the analysis sought to estimate the difference in pre-/post-differences between the 

treatment and comparison groups (Shadish, Cook, et al., 2002). 

The statistical analysis proceeded in three steps. First, interrupted times series models were 

estimated against the treatment series only. This provided an assessment of whether rules 

revision, combined with management preparatory activities, was a sufficient condition of 

reductions in incident rates. Causal attribution, however, also requires demonstration that rules 

revision, combined with management preparatory activities, was a necessary condition of pre-

/post-differences. Thus, the second stage in the analysis involved estimating the same 

interrupted time series models against the comparison group series. Finally, a pooled regimeswitching regression model was estimated in order to statistically compare the estimated pre-

/post-differences in the treatment series with the estimated pre-/post-differences in the 

comparison series (e.g., Ashenfelter, 1978; Card and Sullivan, 1988). All models were estimated 

using robust regression (iteratively reweighted least squares), which downweights outlying 

observations.28 

A.1 Modeling the Treatment Series 

The most important decision in modeling the treatment series was placement of the point of 

intervention. In three cases (CSXT, KCS, and ACBL), qualitative data suggested that the 

intervention began not with the formal rules revision process but with various preparatory 

management activities that preceded the formal rules revision process (see Section 4.1.1). Thus, 

the point of intervention was the preparatory activities.29 Since the idea behind interrupted time 

series analysis is to test the null hypothesis of no difference between pre- and post-intervention 

series, the remaining features of the model specifications were tailored to each case’s data (see 

McDowall, McCleary et al., 1980).30 

The models for KCS and ACBL were quite simple given that the paucity of observations 

precluded more complex models. The distribution of rates in these cases suggested a Gaussian 

stochastic process. Since inspection of the time series revealed no structural breaks, the pre-

/post-differences were modeled as spline regressions ( e.g., Kmenta, 1986; Greene, 2000), which 

include slope shifts but no intercept shifts. Equation 1 gives the formal statistical model for KCS 

and ACBL: 

RATEt TIMEt TPOST tt += ββ + β + ε 10 2 (1)

 

28 The robust regressions were estimated using STATA version 7, with Huber weights and biweights. 

29 In all three cases, placing the point of intervention at the beginning of formal rules revision produced null 

findings. 

30 Model specification in the KCS model was also limited by the small number observations. 

51 
where TIMEt represents the pre-intervention slope, TPOSTt the difference between the pre- and 

post-intervention slope, and εt an independently and identically distributed disturbance term. 

Inspection of the CSXT series, by contrast, suggested a nonlinear process both in the pre- and 

post-intervention series. Moreover, the distribution of the rates suggested a log-normal 

stochastic process. Thus, the log of incident rates was estimated as a function of the following 

five parameters: (1) the incident rate immediately preceding the intervention (β0),31 (2) the rate 

of decline and acceleration/reversion before the intervention (β1 and β2), (3) the immediate 

change in rate (intercept shift parameter) just after the intervention (β3), (4) the immediate 

change in the rate of decline (slope shift parameters) in incident rate (β4), and (5) the rate at 

which incident rates reverted toward the pre-intervention mean (β5). Equation 2 gives the formal 

model:32

t t t t TPOSTi POSTT t RATE += ββ TIME + β TIME + β INTERV + β + β + ε 2

3 4 5

2

10 2 )log( (2)

A.2 Modeling the Comparison Series 

Assessing the causal impact of the intervention requires a demonstration that the intervention 

was a necessary condition of any observed pre-/post-differences. Thus, the second step in the 

analysis involved estimating the same intervention model against incident rate data from the 

comparison groups. For each case, the placement of the intervention was the same as in the 

treatment series. This provided a test of whether any pre-/post-changes in the treatment series 

might have occurred without the intervention (i.e., whether the treatment was necessary 

condition of pre-/post-changes). 

A.3 Estimating the Differences in Pre-/Post-Differences 

In the final step of the analysis, estimates from the treatment and comparison group impact 

models were compared to assess whether any changes in the treatment group were truly unique. 

Visual comparison of the coefficient estimates for each group was followed by estimation of a 

pooled model in a regime-switching regression. Here, the treatment and comparison group series 

were pooled. The models shown in Equations 1 and 2 were modified to include (1) a dummy 

variable distinguishing treatment from comparison group data and (2) interactions between the 

dummy variable and each of the remaining parameters. The coefficients on the dummyinteraction terms represent the difference between the two groups and allow the evaluation team 

to statistically test the null hypothesis of no difference between the coefficient estimates for each 

group. Pooling data in switching regressions requires the assumption of homogeneous variance 

between the regimes (Kmenta, 1986). Fortunately, for each of the three cases, F-tests for the 

homogeneity of variance provided support for this assumption (see Table A-1). 

 

31 The time variables were centered at the period immediately preceding the intervention. Thus, the intercept may be 

interpreted as the predicted level immediately preceding the intervention. 32 A fully linear model was also estimated. The nature of the time series, however, makes this model less credible 

than the nonlinear model. 

52 
Table A-1. Test of the Homogeneity of Variances in Treatment 

and Comparison Group Series 

Standard Deviation

Firm Treatment Series Comparison Series 

Test of Homogeneity 

of Standard 

Deviation (p-value) 

CSXT 2.8 2.1 0.22 

KCS 0.3 0.5 0.39 

ACBL 2.0 2.4 0.99 

A.4 Findings 

The statistical analysis confirmed findings from visual inspection of the graphs. The following 

provides key statistical findings for each of the three cases. 

CSXT. Table A-2 provides findings for the analysis of CSXT. Values in the first column 

provide estimates for the CSXT series only, while values in the second column provide estimates 

for the comparison series. Estimates of the difference-in-difference are provided in the rightmost 

column. Since incident rates were modeled in logs, the regression coefficients may be 

interpreted as percentage changes. 

The parameters of greatest interest are those for INTERV (the immediate drop in rates), TPOST 

(the change in slope after the intervention), and T2

POST (the change in the rate of 

acceleration/reversion in the rate of change). Visual inspection of columns I and II confirms the 

finding from the graphical analysis of an immediate drop in rates coincident with management 

preparatory activities at CSXT, but not in the comparison series. The difference between CSXT 

and the comparison group barely reaches conventional standards of statistical significance (p = 

0.10), as shown in column III; however, it reflects the fact that safety improvements at CSXT 

preceded those in the comparison group. The coefficient on TPOST also confirms the finding 

that, in spite of an earlier rate decline in CSXT than in the industry, the eventual rates of postintervention decline in the two groups were nearly identical. Again, the absence of a clear 

difference is illustrated by the p-value in column III (0.60). Finally, the statistical analysis 

confirms the finding of a reversion in the CSXT series. Indeed, the coefficient on the T2

POST

variable is 0.01 for CSXT and -0.02 for the industry comparison group.33 In sum, while it is 

clear that the decline in the CSXT series preceded that in the industry, no clear evidence exists 

that the decline was unique to CSXT.34 

KCS. As noted above, both the nature of the time series and the paucity of observations 

suggested a linear model in the KCS case. Thus, the coefficient of primary interest in this 

analysis focuses on the TPOST variable. The results reported in Table A-3 show that, while KCS 

 

33 The small magnitude of the coefficients reflects that the variable is in squared units. 

34 High collinearity likely inflated the standard errors on the difference-in-difference estimates on the β4 and β5

coefficients, increasing the likelihood of Type II error (improperly failing to reject the null hypothesis). 

53 
experienced a statistically significant increase in the rate of safety improvement (β = -0.51, p < 

0.001), the industry comparison series showed no change (p = 0.19). The estimate in column III 

confirms that the difference in pre-/post-differences between the KCS and industry series (β = -

0.76) is statistically significant (p = 0.07). In short, while the KCS series showed an increase in 

the rate of improvement after the intervention, the remainder of the industry did not. 

Table A-2. Coefficient Estimates for CSXT and Industry (Logged Rates)#

Variable 

I 

CSXT (β) 

(p-values) 

II 

Industry (β) 

(p-values) 

ΙΙΙ 

β CSXT - β Industry

(p-value) 

Intercept (β0) 1.35* 

(0.09) 

0.92 

(0.19) 

0.43 

(0.39) 

TIME (β1) -0.16 

(0.27) 

-0.24* 

(0.10) 

0.08 

(0.24) 

TIME2

 (β2) 0.01 

(0.50) 

0.02 

(0.14) 

-0.01 

(0.76) 

INTERV (β3) -0.05 

(0.79) 

0.32** 

(0.02) 

-0.37* 

(0.10) 

TPOST (β4) -0.29** 

(0.03) 

-0.27*** 

(0.005) 

-0.02 

(0.60) 

T2

POST (β5) 0.01 

(0.67) 

-0.02 

(0.19) 

0.03*** 

(0.006) 

#

Pre-intervention series runs from 1983-1989; post-intervention series runs from 1990-2002. 

*p < 0.10 

**p < 0.05 

***p < 0.01 

Examination of the coefficients on the TIME variable (pre-intervention slope) show that a 

statistically discernible difference existed between KCS and the rest of the industry in the rate of 

change before the intervention. On the one hand, this questions the use of the remainder of the 

industry as a comparison group. On the other hand, the fact that the KCS slope was less 

favorable before the intervention only strengthens the case for a positive impact.35 

 

35 High collinearity might imply that the p-values on the difference-in-difference estimates on β2 are biased upward. 

54 
Table A-3. Coefficient Estimates for KCS and Industry#

Variable 

I 

KCS (β) 

(p-values) 

II 

Industry (β) 

(p-values) 

ΙΙΙ 

β (KCS) - 

β (Industry) 

(p-value) 

Intercept (β0) 3.47*** 

(<0.001) 

3.39*** 

(<0.001) 

0.08 

(0.65) 

ΤΙΜΕ (β1) 0.15*** 

(0.001) 

-0.21 

(0.12) 

0.36** 

(0.05) 

TPOST (β2) -0.51*** 

(<0.001) 

0.25 

(0.19) 

-0.76* 

(0.07) 

#

Pre-intervention series runs from 1995-1998; post-intervention series runs from 1999-2002. 

*p < 0.10 

**p < 0.05 

***p < 0.01 

ACBL. The analysis of ACBL and its matched industry is identical to the analysis of KCS data. 

Table A-4 presents the results. Unlike the other two cases, no strong evidence exists of a pre-

/post-difference, either in ACBL (p = 0.13) or in the remainder of the industry (p = 0.36).36 

Examination of the regression coefficients suggests that, if anything, ACBL and the industry 

experienced a slight decrease in the rate of improvement (β = 0.51), with the difference between 

the two being statistically significant (p = 0.10). Thus, no indication exists that rules revision 

had a positive impact on incident rates and slight evidence of a negative impact in this case, 

which should be regarded with cautiousness as outlined in the report. 

Table A-4. Coefficient Estimates for ACBL and Industry#

Variable 

I 

ACBL (β) 

(p-values) 

II 

Industry (β) 

(p-values) 

ΙΙΙ 

β ACBL - β Industry

(p-value) 

Intercept (β0) 4.87*** 

(<0.001) 

11.21*** 

(<0.001) 

-6.34*** 

(0.005) 

ΤΙΜΕ (β1) -0.65*** 

(0.01) 

-0.59*** 

(0.009) 

-0.06 

(0.23) 

TPOST (β2) 0.40 

(0.22) 

0.03 

(0.93) 

0.37 

(0.13) 

#

Pre-intervention series runs from 1990-1994; post-intervention series runs from 1995-2002. 

*p < 0.10 

**p < 0.05 

***p < 0.01 

 

36 High collinearity might imply that the p-values on the difference-in-difference estimates on β2 are biased upward. 

55 

Acronyms 

ACBL American Commercial Barge Lines 

ACSNI Advisory Committee on the Safety of Nuclear Installations 

BNSF Burlington Northern Santa Fe Corporation 

CN/IC Canadian National/Illinois Central 

CSXT CSX Transportation 

EOD employees on duty 

FELA Federal Employer’s Liability Act 

FRA Federal Railroad Administration 

KCS Kansas City Southern 

PPE personal protective equipment 

Volpe Center Volpe National Transportation Systems Center 

57