Notice

This document is disseminated under the sponsorship of 

the Department of Transportation in the interest of 

information exchange. The United States Government 

assumes no liability for its contents or use thereof.

Notice

The United States Government does not endorse products 

or manufacturers. Trade or manufacturers’ names appear 

herein solely because they are considered essential to the 

objective of this report.
i 

REPORT DOCUMENTATION PAGE Form Approved

OMB No. 0704-0188

Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, 

searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments 

regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to Washington 

Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302, and to 

the Office of Management and Budget, Paperwork Reduction Project (0702-0288), Washington, D.C. 20503

1. AGENCY USE ONLY (Leave blank) 2. REPORT DATE 3. REPORT TYPE AND DATES COVERED

December 2009 Technical Report

4. TITLE AND SUBTITLE

An Evaluation of the Switching Operations Fatality Analysis 

2010 Working Group’s Processes

5. FUNDING NUMBERS

HG384/RR04

DTFR53-07-D00004-TO008

6. AUTHOR(S)

Deborah Bonnet, Michael Coplen, Michael Quinn Patton, Joyce Ranney 

& Juna Snow

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)

U.S. Department of Transportation/Research and Innovative Technology 

Administration/John A. Volpe National Transportation Systems 

Center, 55 Broadway, Cambridge, MA 02142-1093

Fulcrum Corporation, 2731 Prosperity Avenue, Suite B, Fairfax, VA 

22031-4335

8. PERFORMING ORGANIZATION 

REPORT NUMBERS

9. SPONSORING/MONITORING AGENGY NAME(S) AND ADDRESS(ES)

U.S. Department of Transportation

Federal Railroad Administration

Office of Research and Development

1200 New Jersey Avenue, SE

Washington, DC 20590

10. SPONSORING/MONITORING AGENCY 

REPORT NUMBER

DOT/FRA/ORD-09/23

11. SUPPLEMENTARY NOTES

Program Manager: Michael Coplen, Federal Railroad Administration

Task Monitor: Stephanie Morrow, John A. Volpe National Transportation Systems Center

12a. DISTRIBUTION/AVAILABILITY STATEMENT

This document is available to the public through the FRA Web site.

12b. DISTRIBUTION CODE

13. ABSTRACT

The Switching Operations Fatality Analysis (SOFA) Working Group was formed to analyze the factors contributing to 

fatalities in switching operations. The 2010 Working Group invited an independent team of evaluators to assess the 

thoroughness of the SOFA methodology with the goal of promoting the utilization of SOFA findings. This report, 

prepared by the evaluation team, describes the SOFA 2010 tools and processes and concludes that they are both 

systematic and rigorous. The railroad industry is urged to give careful consideration to the findings of the SOFA 

Working Group in the interest of improving rail yard safety in the years ahead.

14. SUBJECT TERMS

Evaluation, SOFA, Fatality, Rail

15. NUMBER OF PAGES

21

16. PRICE CODE

17. SECURITY CLASSIFICATION

Unclassified

18. SECURITY CLASSIFICATION 

OF THIS PAGE

Unclassified

19. SECURITY CLASSIFICATION 

OF ABSTRACT

Unclassified

20. LIMITATION OF 

ABSTRACT

NSN 7540-01-280-5500 Standard Form 298 (Rec. 2-89)

Prescribed by ANSI/NISO Std. 

239.18

298-102
ii

METRIC/ENGLISH CONVERSION FACTORS 

ENGLISH TO METRIC METRIC TO ENGLISH

LENGTH (APPROXIMATE) LENGTH (APPROXIMATE)

1 inch (in) = 2.5 centimeters (cm) 1 millimeter (mm) = 0.04 inch (in)

1 foot (ft) = 30 centimeters (cm) 1 centimeter (cm) = 0.4 inch (in)

1 yard (yd) = 0.9 meter (m) 1 meter (m) = 3.3 feet (ft)

1 mile (mi) = 1.6 kilometers (km) 1 meter (m) = 1.1 yards (yd)

1 kilometer (km) = 0.6 mile (mi)

AREA (APPROXIMATE) AREA (APPROXIMATE)

1 square inch (sq in, in2) = 6.5 square centimeters 

(cm2

) 

1 square centimeter (cm2

) = 0.16 square inch (sq in, in2) 

1 square foot (sq ft, ft2) = 0.09 square meter (m2) 1 square meter (m2) = 1.2 square yards (sq yd, 

yd2

) 

1 square yard (sq yd, yd2) = 0.8 square meter (m2) 1 square kilometer (km2) = 0.4 square mile (sq mi, mi2) 

1 square mile (sq mi, mi2) = 2.6 square kilometers 

(km2

) 

10,000 square meters (m2

) = 1 hectare (ha) = 2.5 acres

1 acre = 0.4 hectare (he) = 4,000 square meters (m2) 

MASS - WEIGHT (APPROXIMATE) MASS - WEIGHT (APPROXIMATE)

1 ounce (oz) = 28 grams (gm) 1 gram (gm) = 0.036 ounce (oz)

1 pound (lb) = 0.45 kilogram (kg) 1 kilogram (kg) = 2.2 pounds (lb)

1 short ton = 2,000 

pounds (lb)

= 0.9 tonne (t) 1 tonne (t) =

=

1,000 kilograms (kg)

1.1 short tons

VOLUME (APPROXIMATE) VOLUME (APPROXIMATE)

1 teaspoon (tsp) = 5 milliliters (ml) 1 milliliter (ml) = 0.03 fluid ounce (fl oz)

1 tablespoon (tbsp) = 15 milliliters (ml) 1 liter (l) = 2.1 pints (pt)

1 fluid ounce (fl oz) = 30 milliliters (ml) 1 liter (l) = 1.06 quarts (qt)

1 cup (c) = 0.24 liter (l) 1 liter (l) = 0.26 gallon (gal)

1 pint (pt) = 0.47 liter (l)

1 quart (qt) = 0.96 liter (l)

1 gallon (gal) = 3.8 liters (l)

1 cubic foot (cu ft, ft3) = 0.03 cubic meter (m3) 1 cubic meter (m3) = 36 cubic feet (cu ft, ft3) 

1 cubic yard (cu yd, yd3) = 0.76 cubic meter (m3) 1 cubic meter (m3) = 1.3 cubic yards (cu yd, yd3)

TEMPERATURE (EXACT) TEMPERATURE (EXACT)

[(x-32)(5/9)] °F = y °C [(9/5) y + 32] °C = x °F

QUICK INCH - CENTIMETER LENGTH CONVERSION

0 1 2 3 4 5

Inches

Centimeters 0 1 2 3 4 5 6 987 1110 1312

QUICK FAHRENHEIT - CELSIUS TEMPERATURE CONVERSIO

 -40° -22° -4° 14° 32° 50° 68° 86° 104° 122° 140° 158° 176° 194° 212°

 

°F

 °C -40° -30° -20° -10° 0° 10° 20° 30° 40° 50° 60° 70° 80° 90° 100°

For more exact and or other conversion factors, see NIST Miscellaneous Publication 286, Units of Weights and 

Measures. Price $2.50 SD Catalog No. C13 10286 Updated 6/17/98
iii

ACKNOWLEDGEMENTS

The authors extend gratitude to the SOFA Working Group Chair, Joseph Gallant, who requested 

and facilitated this review, and to the current SOFA Working Group members for their open and 

enthusiastic participation in this evaluation: Stephen Bruno, William Browder, Robert Butcher, 

Gary Devall, Ty Dragoo, John T. Little, David Mears, Phillip Qualy, Thomas Streicher, and 

Robert Svob, Jr. In addition, we thank database analyst Gary Connors and past SOFA Working 

Group members for their valuable insights: Danny Boyles, David Brickey, George Last, Thomas 

Perkovich, David Skinner, and John Smullen. 

The Federal Railroad Administration funded this evaluation through a contract with Fulcrum 

Corporation and an interagency agreement with the Volpe National Transportation Systems 

Center.
iv

Contents

Acknowledgements ........................................................................................................................ iii

Executive Summary ........................................................................................................................ v 

1. Introduction ......................................................................................................................... 1 

1.1 Background ...................................................................................................................... 1

 1.1.1 SOFA 1999 .............................................................................................................. 1 

 1.1.2 SOFA 2004 .............................................................................................................. 2 

 1.1.3 Ongoing Efforts to Promote SOFA Recommendations ........................................... 3 

1.2 SOFA 2010 ...................................................................................................................... 3

2. SOFA Process Evaluation ................................................................................................... 5 

 2.1 The SOFA Working Group Is Appropriately Constituted ............................................... 5 

2.2 The SOFA Working Group Is Cohesive and Resistant to Political Influences ............... 6

2.3 SOFA 2010 Is Explicitly Designed for Utilization .......................................................... 7

2.4 Information Analyzed by the Working Group Was Complete, Sound, and Varied ........ 7

2.5 SOFA Case Analyses Are Meticulous ............................................................................. 8

2.6 The Working Group Reaches Consensus on Key Conclusions about Each Case ........... 9

2.7 Aggregation of SOFA Case Data Was Rigorous ............................................................. 9

2.8 The SOFA Working Group Practices Continuous Improvement .................................... 9

2.9 The Process Evaluation Was Thorough and Independent ............................................... 9

3. Conclusions ....................................................................................................................... 11 

4. References ......................................................................................................................... 12 

Appendix A ................................................................................................................................... 13

Appendix B ................................................................................................................................... 14
v 

Executive Summary 

This report documents the evaluation of the processes used by the Switching Operations Fatality 

Analysis (SOFA) 2010 Working Group to analyze the 54 fatalities in switching operations that 

occurred from January 2004 to November 2009. Using accepted professional standards for 

evaluation, the evaluation team concluded that the SOFA analyses are systematic, rigorous, 

comprehensive, and objective. Furthermore, the findings of the SOFA 2010 Working Group are 

valid, significant, and worthy of the railroad industry’s thoughtful attention and bold response. 

These conclusions are based on the following observations: 

1. The SOFA Working Group is appropriately constituted. 

2. The SOFA Working Group is cohesive and resistant to political influences. 

3. SOFA 2010 is explicitly designed for utilization. 

4. Information analyzed by the Working Group was complete, sound, and varied. 

5. SOFA case analyses are meticulous. 

6. The Working Group reaches consensus on key conclusions about each case. 

7. Aggregation of SOFA case data was rigorous. 

8. The SOFA Working Group practices continuous improvement. 

9. The process evaluation was thorough and independent. 
1 

1. Introduction

Since early 2009, the SOFA Working Group has been preparing for its third major report, to be 

released in 2010. The Working Group has applied its tools and processes for analyzing train yard 

tragedies, refined over its 12 years of experience, to the 54 fatalities that occurred since January 

2004. At the same time, the Working Group has worked diligently to ensure widespread and 

effective use of its 2010 findings to achieve the goal of zero switching deaths. The group has 

reviewed successes and disappointments from its past efforts, called upon evaluation 

professionals for guidance, and used principles of utilization-focused evaluation (Patton, 2008) 

to inform its work. 

The evaluation team has examined SOFA against accepted professional standards for evaluation 

(Joint Committee on Standards for Educational Evaluation, 1994; American Evaluation 

Association, 2004). We have found the Working Group’s methods appropriate and its findings 

valid. We offer this report, first, to highlight the credibility of SOFA findings, and second, to 

document how SOFA works for the benefit of other similar initiatives that may draw on SOFA’s 

lessons.

1.1 Background

The SOFA Working Group first convened in February 1998 in response to a letter from a senior 

official of the Federal Railroad Administration (FRA) to industry leaders. The letter expressed 

concern that during the prior 6 years, 66 train and engine service employees had been fatally 

injured in incidents other than major train collisions and called for a task force consisting of 

representatives from labor, management, and FRA to find a way to prevent these tragic 

occurrences. The letter went on to say: 

“The Team will conduct a detailed fact finding and review and analysis of these 

incidents to determine whether trends or patterns can be found, identify best 

practices, and, if possible, formulate recommendations for the entire industry 

based on the findings. . . . The findings and recommendations from this team are 

neither intended to be used in a rulemaking process nor to otherwise lead to 

formal action by FRA. Rather, railroads will be able to evaluate the team’s 

findings and recommendations with respect to their individual operating 

requirements. . . .” 

1.1.1 SOFA 1999

The SOFA Working Group met almost monthly for 20 months to analyze the 76 fatalities that 

involved railroad employees engaged in switching operations from January 1, 1992, through July 

1, 1998. As the work proceeded, the group learned that: 

• The summary reports prepared by FRA investigators alone were not sufficient; to 

understand the underlying causal factors required reviewing entire case files, including 

photographs of the site and statements of eye witnesses. 

• To find trends and patterns across cases, the Working Group needed a codified database 

of standardized information. One was developed and continues to evolve. 
2 

• Although FRA investigation reports tried to establish a single probable cause, fatalities 

more often resulted from the coming together of a complex set of factors. The SOFA 

process turned to possible contributing factors rather than probable cause and 

accommodated more than one factor per case. 

• Shortly after beginning its evaluation process, the SOFA Working Group accepted 

Human Factors support offered by the Office of Research and Development at FRA 

and the Volpe National Transportation Systems Center. This assistance proved valuable 

to the development of systematic and rigorous processes for analyzing individual cases 

and synthesizing across cases. 

The SOFA Report: Findings and Recommendations of the SOFA Working Group, released in 

October 1999, identified five operating recommendations, later shortened to 5 Lifesavers: 

1. Secure equipment before action is taken.

2. Protect employees against moving equipment. 

3. Discuss safety at the beginning of a job or when a project changes. 

4. Communicate before action is taken.

5. Mentor less experienced employees to perform service safely.

The report offered additional recommendations to the industry concerning: 

• Unexpected train movement 

• Crew resource management

Finally, the report recommended the following to the SOFA Working Group and FRA: 

• Review of severe injuries

• Maintenance of the SOFA matrix

• Computer support for fatality investigation 

• Continued review and monitoring of switching fatalities

• Team-oriented approach to switching fatality investigation 

All of the last set of recommendations were carried out, including the publication of SOFA’s 

Severe Injuries to Train and Engine Service Employees: Data Description and Injury 

Characteristics, published in July 2001. 

1.1.2 SOFA 2004

The SOFA Working Group continued to meet from time to time. Findings and 

Recommendations of the SOFA Working Group: August 2004 Update was based on the 

expanding database that added to the original 76 switching fatalities another 48 that occurred 

between July 1998 and December 2003. The five operating recommendations (5 Lifesavers) 

were further validated. The 2004 report also identified 10 Special Switching Hazards: 
3 

• Close clearance

• Struck by main-line trains

• Free rolling rail cars

• Employee tripping, slipping, falling 

• Unsecured cars

• Unexpected movement of rail cars

• Equipment issues 

• Struck by motor vehicle or loading device 

• Drugs and alcohol 

1.1.3 Ongoing Efforts to Promote SOFA Recommendations

In addition to reviewing fatalities and issuing reports, the SOFA Working Group and others have 

used several means to educate and motivate the railroad industry—from carrier leaders to 

workers at the ballast level—to take the necessary actions to ensure safety in switching 

operations. For a time, the FRA Administrator held monthly conference calls with carrier, labor, 

and FRA leaders to hear what steps they had taken. The Working Group members took every 

opportunity to speak publicly about SOFA and to reinforce its messages privately. It examined 

and encouraged further inquiry into promising safety technologies. A goal of zero switching 

fatalities was announced. A memorandum, Best Practices Guidelines for Implementing 

Operating Recommendations (March 2000), was issued to encourage education and a positive, 

judicious approach to implementation and to discourage use of the operating recommendations 

as a basis for discipline. There were occasional safety alerts when new hazards were identified. 

A SOFA video was produced and distributed, and a Web site was created. The railroad industry 

took the lead in creating Crew Resource Management, a generic training program for train and 

engine employees. The 5 Lifesavers was printed and distributed nationwide on hats, pens, walletsize cards, refrigerator magnets, stickers, and switch-list covers. 

The Volpe Center has kept the SOFA Working Group and others current on trends in switching 

operations fatalities by issuing quarterly reports of the number of deaths that appear to be related 

to SOFA’s 1999 and 2004 findings.1

1.2 SOFA 2010

 The Third Quarter 2009 report concluded that fatalities 

related to operating recommendations have dropped substantially since SOFA began, although 

hazard-related fatalities have not.

The SOFA Working Group was reconvened in January 2009 with the charge to analyze 

switching operation fatalities that occurred after December 2003, add them to the database of 

fatalities between 1992 and 2003, and issue an updated report in early 2010. As of November 1, 

2009, there were 54 new fatalities, bringing the total to 178—an average of 10 per year. 

 

1 “Switching Fatality and Severe Injury Updates” are compiled on the basis of investigation reports, without the 

benefit of the SOFA Working Group’s analyses, and should therefore be regarded as preliminary.
4 

Despite extensive efforts and evident successes, the 2010 Working Group was dissatisfied with 

SOFA’s effectiveness. The goal of zero fatalities had not been achieved. There was a sense that 

attention to the 5 LifeSavers and 10 Special Switching Hazards had waned since the fanfare 

surrounding their introduction. The group was particularly concerned that SOFA’s 

recommendations had been lifted up as special rules—whose violation is often treated as 

especially egregious—rather than as an encouragement to focus on education and collaborative 

problem-solving. The hope of achieving deep systemic shifts toward organizational cultures 

where safety takes precedence over productivity had not been realized. To a person, Working 

Group labor representatives said that, as a result of SOFA participation, they had become far 

more attuned to safety issues and had substantially changed their behaviors when back on the 

job. Yet, how to meaningfully extend their personal awakenings to colleagues throughout the 

industry remained unclear. 

“A chart is not going to wake anyone up. I’m not saying to show pictures of 

corpses, but just talk about how bad it could be and that it is your choice. These 

images and discussions here helped me change. Because of what I learned here, it 

woke me up.” 

“How many cases have we had that people ran with a pin lifter? I did that for 

years, but I didn’t know that was a SOFA recommendation. Sure, maybe someone 

told me not to do it, but until I served here and read the cases, I didn’t realize the 

importance that this recommendation is followed.” 

With these disappointments in mind, the Working Group invited a professional evaluator to its 

February 2009 meeting to facilitate conversation about how to attain greater and more lasting 

impact with the 2010 report. The group engaged in a logic modeling process to clarify the 

outcomes it hoped to achieve (see Appendix A). This exercise led to three important conclusions: 

• SOFA had been fairly successful in getting the message of railroad safety out, but the 

findings and recommendations were too seldom used effectively.

• The credibility of SOFA’s processes and findings would be key to subsequent use and 

implementation of the 2010 report. 

• Effective utilization of the findings would depend not only on the report itself, but also 

on the various ways of engaging stakeholders, and planning for utilization needed to 

begin well in advance of the report’s release. 

The present report is one element of the emerging utilization strategy for SOFA 2010.
5 

2. SOFA Process Evaluation

The SOFA 2010 Working Group submitted itself to the scrutiny of five seasoned evaluation 

professionals, the authors of this report (see Appendix B).2

We concluded that the SOFA analyses are systematic, rigorous, comprehensive, and objective. 

The findings are valid, significant, and worthy of the railroad industry’s thoughtful attention 

and bold response. We base these claims on the following:

The evaluation team was first formed 

for two purposes: (1) to advise the SOFA Working Group on ways to encourage effective use of 

SOFA findings and (2) to assist in the preparation of the 2010 SOFA report. As our work with 

the Working Group evolved, two additional purposes emerged. It became clear that SOFA 

represents an exceptionally systematic and comprehensive model—one that may well serve as an 

exemplar for other initiatives in the railroad industry and even beyond. Thus, this process 

evaluation report was also conceived as a way (3) to highlight the credibility of SOFA findings, 

and (4) to document how SOFA works for the benefit of other similar initiatives. 

1. The SOFA Working Group is appropriately constituted. 

2. The SOFA Working Group is cohesive and resistant to political influences. 

3. SOFA 2010 is explicitly designed for utilization. 

4. Information analyzed by the Working Group was complete, sound, and varied. 

5. SOFA case analyses are meticulous. 

6. The Working Group reaches consensus on key conclusions about each case. 

7. Aggregation of SOFA case data was rigorous. 

8. The SOFA Working Group practices continuous improvement. 

9. The process evaluation was thorough and independent. 

The balance of this report elaborates upon each of these conclusions. 

2.1 The SOFA Working Group Is Appropriately Constituted

The 2010 SOFA Working Group consists of 11 members representing carrier management, 

labor, and FRA. Members are appointed by their respective organizations:

• Association of American Railroads (one member), 

• American Short Line and Regional Railroad Association (ASLRRA) (two members),3

• Brotherhood of Locomotive Engineers and Trainmen (three members), 

• United Transportation Union (three members), and

• Federal Railroad Administration (two members). 

 

2 The evaluation team and its methods are described later in Section 2.9.

3 Two members of the ASLRRA rotate attendance at SOFA meetings. 
6 

Collectively, the Working Group brings 359 years of railroad experience and ranges in age from 

26 to 65. Two members were on the 1999 and 2004 SOFA Working Groups and one additional 

member participated in 2004, thereby bringing continuity to the process. 

Members are selected not only for their experience but also for analytic, communication, and 

teamwork skills—important factors in their collective success as a working group. 

2.2 The SOFA Working Group Is Cohesive and Resistant to Political Influences

The SOFA Working Group is highly cohesive, and its members are adamant in claiming this as 

instrumental to SOFA’s success. A shared commitment to the importance of the work—saving 

lives—has pulled the group through some taxing conflicts. The Chairman’s skillful leadership 

has helped cultivate productive group norms of persistent questioning and spirited debate and

balanced by mutual respect and ego containment. Other unwritten rules include: 

• Working Group members leave their sector allegiances at the door. Getting to the truth 

of each case trumps protecting turf. 

• All SOFA deliberations are private. What is said in the SOFA meeting room stays in 

the SOFA meeting room. 

• SOFA members enjoy the backing of the organizations they represent. Their leaders 

respect the confidential and collaborative nature of the work and refrain from 

interfering with the group’s deliberations and conclusions. 

Most notably, SOFA has succeeded where many others have failed by overcoming the 

traditionally adversarial relations among labor, management, and regulators, deeply ingrained in 

the railroad industry, to achieve quality collaborative work: 

“It’s a good, collaborative group. The operations work well. . . . We are dealing 

with real people, and everyone is allowed to express their opinion. It is an 

effective process, and it does a very good job of getting to the root causes and 

contributing factors of these accidents. I was surprised to see how much 

information was provided by the field investigators. They do very comprehensive 

work that gives us a chance to do a real evaluation based on the facts, which have 

survived through the whole investigative process.” 

The Working Group seemed pleased by an evaluation team member’s mention that it took all 

morning of the first day of observation for her to tell who was who—that is, to distinguish 

among the management, labor, and government representatives; and furthermore, it was based on 

the attendees’ stories, not how they looked or spoke, the opinions they expressed, or the positions 

they took. 

Yet another indicator of the SOFA Working Group’s camaraderie is that six former members, 

now retired, traveled at their own expense to the September 2009 meeting to reunite with their 

colleagues. While there, they donated several hours to participate in utilization planning for 

SOFA 2010. 
7 

2.3 SOFA 2010 Is Explicitly Designed for Utilization

As mentioned previously, SOFA 2010’s second meeting focused on utilization of the report that 

would be released a year hence. The logic modeling exercise aided in reflecting on past practices 

and bringing use to the forefront. Throughout the year, the Working Group drew in experts to 

aid in thoughtful reflection on what worked before, what did not, and how things have changed 

since 1999. 

A pivotal realization occurred at the September 2009 meeting. Amid concerns that past SOFA 

recommendations had been instituted as rules, an alert observer noted that recommendations 

were written in a way that read like rules: “At the beginning of each tour of duty, all crew 

members will meet and discuss all safety matters and work to be accomplished. Additional 

briefings will be held any time work changes are made and when necessary to protect their 

safety during their performance of service.”4

This insight contributed to the decision to engage the report’s intended users in interpretation and 

decisions for action, thereby deepening their understanding of the implications of the findings. 

Planning turned toward involving labor, management, and government in a collaborative 

examination of the data beyond the confines of the SOFA Working Group.

 Even when translated into the more user-friendly 5 

Lifesavers, SOFA recommendations can still be read as rules: “Discuss safety at the beginning of 

a job or when a project changes.” 

5

Among other advantages, this utilization-focused approach would recognize that solutions are 

best formulated in context, and acknowledge that the best fix for a given problem would likely 

vary from carrier to carrier and site to site. 

 

2.4 Information Analyzed by the Working Group Was Complete, Sound,

and Varied

By “complete,” we mean that all fatalities within SOFA’s scope are analyzed.

As one would expect, a SOFA case begins with a fatality; specifically, the accidental death of an 

on-duty train or engine employee while performing switching operations. All railroad-related 

deaths are reported immediately to the National Response Center, which alerts the Team Leader 

of FRA’s Accident Analysis Branch (who is also the SOFA Chairman). Cases are assigned to 

one of FRA’s eight regions for investigation. The region appoints one or more FRA investigators 

to go to the site as soon as possible to begin work. Teams are preferred; in the case of a 

switching operation death, the SOFA Working Group prefers that the Inspector in Charge be an 

Operating Practices Inspector or another inspector trained in human factors. 

Investigation protocols are spelled out precisely and clearly in FRA’s 101-page Accident 

Investigation Guidelines: General Compliance Manual, Part IV, Chapter 9 (2006). The process 

is complex and thorough. FRA investigators interview virtually everyone with firsthand 

knowledge of the case. They examine the site and equipment involved and consult records, 

photos, maps, etc. When necessary, FRA investigators may subpoena witnesses, require the 

 

4 Recommendation 3.

5 As of this writing, details of this plan are still in development.
8 

production of records, exhibits, and other evidence, administer oaths, and take testimony. A 

given case may involve collaborating with local law enforcement and emergency response 

authorities, state and local investigators, the National Transportation Safety Board, and/or the 

Occupational Safety and Health Administration. 

Investigation reporting forms are long and detailed. They involve narratives as well as checklists 

and fill-in-the-blank items and conclude with the investigation team’s conclusions regarding 

probable cause and contributing factors. Supporting documentation is attached.

Investigation Fatality Memoranda are submitted to Regional Review Groups, which may require 

additional information or corrections before forwarding the complete report to the Accident 

Analysis Branch at FRA Headquarters, whose Team Leader then determines which cases are 

within SOFA’s scope. In advance of SOFA meetings, all members are provided the Investigation 

Fatality Memorandum. Supporting documentation (sometimes as thick as 6 inches) is brought to 

the meeting. 

Veteran SOFA members have noticed a marked improvement in the quality of investigation 

reports over time. This is due, at least in part, to improvements in the Manual recommended by 

SOFA in 1999 and 2004, response to the SOFA recommendation to conduct investigations in 

teams, and regional-level training provided by the Accident Analysis Branch Team Leader.

2.5 SOFA Case Analyses Are Meticulous

Although all SOFA Working Group members read all Fatality Memoranda in advance, one 

member is assigned to dig deeply into each case, examining the entire case file. The presenter 

completes a data sheet corresponding to the SOFA database fields for basic facts such as time of 

day of the occurrence and the deceased’s age, length of service, and date of last formal safety 

training. He or she sketches the accident site to show the location of tracks, relevant equipment, 

workers, and other pertinent elements for use in a formal presentation. After briefly summarizing 

the case for the Working Group, the presenter highlights particularly relevant supporting 

documentation not found in the summary narrative and then responds to questions. 

At that point, the group dissembles into impromptu small group discussions. These subgroups 

often request supporting documents, such as photos, which are then passed around to the other 

groups. Eventually the Working Group reconvenes as a committee of the whole for further 

discussion and debate. They share theories, alternative explanations, and hunches. By the end, 

most group members are very familiar with the details of the case. 

This year, Google Earth has proven a useful tool. The aerial view of the accident site was 

instrumental to understanding at least one case. Other sources outside the case file are 

occasionally consulted, but only rarely and with the group’s permission. 

The team frequently conducts integrity checks of the SOFA database to assure accuracy and 

completeness of the data. Whereas past SOFA Working Groups have used spreadsheets, the 

2010 Working Group employed a former FRA operations researcher to create a well-designed 

Microsoft Access database, complete with tabs and pulldown menus. The group uses an LCD

projector to display the forms as they are completed. This helps keep the conversation on track 

and makes it possible to look up similar cases from the past to ensure consistency. The database 
9 

consultant plays the additional role of reminding the group of definitions, criteria for possible 

contributing factors, and so on, thereby contributing to the integrity of the database as well as the 

focus of conversation. 

The evaluation team was impressed with the Working Group’s tenacity. The case analysis 

process is exhaustive, penetrating, and self-reflective. For any given case, some group members 

have experienced or witnessed similar situations. They can picture the circumstances clearly, and 

often ask out loud, “What would I have done? Have I ever made that mistake?” and are visibly 

shaken when the answer is “yes.”

2.6 The Working Group Reaches Consensus on Key Conclusions about 

Each Case

The most important database elements are also those requiring the most judgment. Possible 

contributing factors and external circumstances are assigned only with unanimous agreement. 

Case abstracts of about three sentences, which are published, are wordsmithed by consensus. 

Each case analysis is given as much time and attention as necessary to reach consensus. Some 

are completed in 1 hour; others take most of a day. The group’s commitment to give each case its 

full attention and to adhere to the consensus process takes precedence over any concerns about 

time or efficiency. A SOFA cycle begins with an inventory of several years’ of cases, and the 

group meets for 3 days each month until it is done. 

2.7 Aggregation of SOFA Case Data Was Rigorous

The SOFA database is thoroughly cleaned. Quantitative analyses are appropriate to the data and 

the research questions. Qualitative analyses, which venture beyond the raw numbers to capture 

important themes by drawing on the Working Group’s experience and judgment, are 

nevertheless well grounded in the data; evaluation team members expert in qualitative methods 

monitored for this as they observed and led the discussions about SOFA’s overarching findings. 

2.8 The SOFA Working Group Practices Continuous Improvement

The Working Group is to be commended for its commitment to ongoing improvement. SOFA 

1999 did good work, but SOFA 2004 did better, and SOFA 2010 better still. For example, on the 

basis of the lessons learned in the aftermath of the 1999 report, the SOFA Working Group issued 

implementation guidelines. When case codes were found inadequate, they were revised and prior 

cases were recoded. When spreadsheets became unwieldy, the SOFA Working Group converted 

to a database. When investigation reports were found incomplete, FRA rewrote the manual and 

retrained the investigators. When the Working Group needed help in getting its findings used, it

called in experts in evaluation utilization and invited the present process evaluation. More 

examples abound. 

2.9 The Process Evaluation Was Thorough and Independent

The evaluation was performed by five seasoned evaluation professionals (see Appendix B for 

biographical information). The evaluators were independent of the SOFA Working Group and 

used established professional standards for evaluation to assess the SOFA processes. 
10 

The evaluation team members attended between two and four SOFA 2010 meetings each. 

Altogether we observed more than one dozen case analyses and led four full-group discussions 

covering, such topics as SOFA’s theories of change, efforts to motivate and support use of earlier 

reports, what has worked toward that end, and barriers to implementing SOFA recommendations 

in particular and railroad safety in general. One of those discussions included six retired SOFA 

members as well as the 2010 Working Group. We have read past SOFA reports, the FRA 

Accident Investigation Guidelines, and various other documents pertaining to SOFA’s history. 

In addition, we conducted individual face-to-face interviews with 10 of the 11 Working Group 

members and their database management consultant, providing them opportunity to voice 

reservations and concerns anonymously. We have been briefed by the SOFA Chair on several 

occasions, conferred with the database management consultant, and examined the database itself. 

The evaluation team found the Working Group fully cooperative, candid, and responsive to 

requests for documents and information. 
11 

3. Conclusions

The evaluation team reiterates: the SOFA analyses are systematic, rigorous, comprehensive, and 

objective. The findings are valid, significant, and worthy of the railroad industry’s thoughtful 

attention and bold response. 

We look forward with optimism to the impact of SOFA 2010’s utilization-focused efforts. 
12 

4. References

American Evaluation Association. (2004). Guiding Principles for Evaluators. 

http://www.eval.org/Publications/GuidingPrinciples.asp. 

Findings and Recommendations of the SOFA Working Group, October 

1999. http://www.fra.dot.gov/pdf/sofa_rep.pdf. 

Findings and Recommendations of the SOFA Working Group, Appendix – Volume II, August 

2000. http://www.fra.dot.gov/pdf/safety/sofa/sofa_vol2.pdf. 

Findings and Recommendations of the SOFA Working Group: August 2004 Update, August 

2004. http://www.fra.dot.gov/us/content/1506. 

FRA Accident Investigation Guidelines: General Compliance Manual, Part IV, Chapter 9, 

February 2006. http://www.fra.dot.gov/downloads/Safety/accinvguide06.pdf. 

Joint Committee on Standards for Educational Evaluation. (1994). The Program Evaluation 

Standards. Thousand Oaks, CA: Sage. 

Patton, M. Q. (2008). Utilization-Focused Evaluation (4th ed). Thousand Oaks, CA: Sage. 

Severe Injuries to Train and Engine Service Employees: Data Description and Injury 

Characteristics, July 2001. http://www.fra.dot.gov/pdf/safety/sofa/SOFA_Injury.pdf. 
13 

APPENDIX A 

SOFA Working Group Process Logic Model

Stakeholders collaborate to change RR practice Historically challenging context: Rules-driven culture

•Study 

utilization of 

past 

information 

dissemination 

& adoption

•Review 

requirements

•Analyze 

switching 

fatality data

•Identify 

contributing 

factors 

•Develop 

factors-based 

recommendatio

ns (Findings) 

•Disseminate 

Findings

•Train yard & 

engine service 

workers

•RR partners: 

Funders, 

carriers, 

associations & 

management

•Labor unions

•FRA & 

regulatory 

agencies

•RR/Labor 

understand 

need to adopt 

Findings

•Partners see 

need to 

collaborate on 

adoption of 

Findings (how 

to break down 

barriers) 

•Stakeholders 

see need for 

voluntary 

processes

•RR/Labor 

adopt/follow 

Findings

•Partners help 

adopt Findings 

(actually break 

down barriers)

•Reduce reactive 

mechanisms

•Model for 

voluntary 

programs is 

adopted by 

other programs

•Zero employee 

switching 

fatalities

•Positive, rather 

than punitive, 

relationships

•Utilization 

model

• Multiple data 

sources

• Partners

• Cooperation & 

collaboration 

• Time

• Clear purpose

• Experiential, 

technical, & 

practical 

knowledge (400 

years 

combined)

• Analysis skills

• Technical 

support

Train yard & 

engine service 

employee 

fatalities; 

productivity 

demands; 

disjointed 

stakeholder 

groups

Save lives of 

train yard & 

engine service 

employees to 

measure & 

demonstrate 

effective 

adoption of 

safe practices 

through 

sharing 

resources & 

lessons learned

Focus – Question – Observe & Ask – Analyze & Interpret – Facilitate & Recommend: Implementation Plan & Buy-In Strategies

Logic Model Template ©2003 UW Extension, http://www.uwex.edu/ces/pdande/; adapted by InnovatEd Consulting
14 

APPENDIX B 

The Evaluation Team

Deborah Bonnet is Director of Evaluation for Fulcrum Corporation, an engineering firm based in 

the Washington, D.C., area. She has conducted more than 120 studies in her 35-year evaluation 

career, in most cases as the principal investigator. She earned an M.S. in human factors from 

Virginia Tech and an M.B.A. from Indiana University and has held various leadership positions in 

the American Evaluation Association.6

Michael Coplen is Senior Evaluator and Manager of Culture and Safety Performance Studies for the 

Federal Railroad Administration, Office of Research and Development, Human Factors Program. He 

began his career as a railroad brakeman and engineer and later earned an M.A. in organizational 

behavior from the University of Nebraska. His 15 years as an evaluation practitioner include 

membership on the first SOFA Working Group. He was awarded the 2009 Alva and Gunnar Myrdal 

Government Award by the American Evaluation Association for successful evaluation use in FRA 

and promotion of evaluation methods in the Federal Government. 

Michael Quinn Patton is an independent evaluation consultant, former president of the American 

Evaluation Association, and author of five major books on evaluation, including Utilization-Focused 

Evaluation. After receiving his Ph.D. in sociology from the University of Wisconsin, he served for 

many years on the faculties of the University of Minnesota and the Union Institute. He has worked 

with organizations and programs at the international, national, state, and local levels, and with 

philanthropic, not-for-profit, private sector, and government programs as a generalist working across 

the full range of efforts at improving human effectiveness and results. 

Joyce Ranney is a senior program evaluator in the Human Factors Research and System 

Applications Center of Innovation at the Volpe National Transportation Systems Center. In the past 

several years, she has conducted five major multiyear evaluation studies in the railroad industry 

showing significant bottom-line improvements in safety and safety culture. She holds a B.S. in 

speech from Southern Illinois University and a Ph.D. from the University of California at Los 

Angeles in organization behavior.

Juna Snow is Principal Consultant of InnovatEd Consulting and a subcontractor with the Volpe 

Center. With more than 15 years of experience in educational development, research, and evaluation, 

she serves on the faculty of the University of Illinois at Urbana-Champaign, where she earned an 

M.S. in ecology and a Ph.D. in education. She chairs the Research on Evaluation Special Interest 

Group of the American Educational Research Association. 

 

6 http://www.eval.org