DOT/FAA/AM-17/7

Office of Aerospace Medicine

Washington, DC 20591

Line Operations Safety 

Assessments (LOSA) in 

Maintenance and Ramp 

Environments

Lynn Crayton1

Carla Hackworth1

Carrie A. Roberts2

S. Janine King2

1

Civil Aerospace Medical Institute

Federal Aviation Administration

Oklahoma City, OK 73125

2

Cherokee CRC, LLC

Oklahoma City, OK 73125

January 2017

Final Report
NOTICE

This document is disseminated under the sponsorship of 

the U.S. Department of Transportation in the interest of 

information exchange. The United States Government 

assumes no liability for the contents thereof.

___________

This publication and all Office of Aerospace Medicine 

technical reports are available in full-text from the 

Civil Aerospace Medical Institute’s publications website: 

http://www.faa.gov/go/oamtechreports
i

Technical Report Documentation Page

1. Report No. 2. Government Accession No. 3. Recipient's Catalog No.

DOT/FAA/AM-17/7

4. Title and Subtitle 5. Report Date

Line Operations Safety Assessments (LOSA) in Maintenance and Ramp 

Environments 

January 2017

6. Performing Organization Code

7. Author(s) 8. Performing Organization Report No.

1

Crayton L, 1Hackworth C, 2Roberts CA, 2King SJ

9. Performing Organization Name and Address 10. Work Unit No. (TRAIS)

1

FAA Civil Aerospace Medical Institute

P.O. Box 25082

Oklahoma City, OK 73125

2

Cherokee CRC, LLC

P.O. Box 25082

Oklahoma City, OK 73125

11. Contract or Grant No.

12. Sponsoring Agency name and Address 13. Type of Report and Period Covered

Office of Aerospace Medicine

Federal Aviation Administration

800 Independence Ave., S.W.

Washington, DC 20591 14. Sponsoring Agency Code

15. Supplemental Notes

Work was accomplished under approved task AM-B-HRR-521

16. Abstract

Data-driven modifications to risk management are needed in maintenance and ramp environments to assist 

the Federal Aviation Administration (FAA) in the development of adequate tools/systems that increase 

safety by aiding the identification of hazards and managing the associated risks. Proactive and predictive 

approaches, as opposed to reactive changes based on post-accident/event investigation, align with the 

principles of risk management and fundamental concepts of Safety Management Systems (SMSs). A Line

Operations Safety Assessment (LOSA) is a formal process that uses trained observers to monitor normal 

operations of a company’s activities and record their observations. The findings are not intended to be used 

punitively but are meant to provide a “diagnostic snapshot” of the positives and the areas in need of 

improvement within a company. The FAA encourages airlines to conduct LOSAs in the interest of safety 

improvement; however, it does not mandate, approve, or monitor LOSA programs. We distributed a 

questionnaire to evaluate the utility of LOSA programs across the industry and identify lessons learned and 

best practices for implementing a program. We examined Ramp LOSA and Maintenance LOSA programs 

that either had been implemented or were being implemented at airlines; maintenance, repair, and overhaul 

service companies; and other aviation service organizations. LOSA questionnaire results increase the FAA’s 

knowledge of LOSA activity and successful LOSA programs within the aviation community. Our results 

show the progress rate of phase completion for the most recent LOSA programs overall was higher for Ramp 

than for Mx respondents. All Ramp respondents reported completion of all phases of their organization’s 

LOSA program. Mx respondents reported varied states of completion for the phases. Ramp respondents 

represented organizations with more active observers conducting between 200 and 11,000 observations 

during the implementation period; however, Maintenance LOSA organizations had fewer observers by 

comparison and had conducted between 5 and 30 observations during the implementation period. 

17. Key Words 18. Distribution Statement

Safety Management System (SMS), Line Operations 

Safety Assessment (LOSA), Threat and Error 

Management (TEM)

Document is available to the public through the Internet:

http://www.faa.gov/go/oamtechreports/

19. Security Classif. (of this report) 20. Security Classif. (of this page) 21. No. of Pages 22. Price

Unclassified Unclassified 46

Form DOT F 1700.7 (8-72) Reproduction of completed page authorized
ii

ACKNOWLEDGMENTS

Research reported in this paper was conducted under the Flight Deck Program Directive/Level of 

Effort Agreement between the Federal Aviation Administration Human Factors Division (ANG-C1) and 

the Aerospace Human Factors Research Division (AAM-500) of the Civil Aerospace Medical Institute. A 

special thank you is extended to those who were generous with their time and experience and responded 

to the LOSA questionnaire.

The authors gratefully acknowledge Dr. Brenda Wenzel for the tremendous support she gave to the 

design and development of the data collection instrument used to conduct this study.
iii

CONTENTS

List of Tables.........................................................................................................................................iv

List of Figures ........................................................................................................................................v

List of Acronyms...................................................................................................................................vi

Executive Summary .............................................................................................................................vii

Introduction ............................................................................................................................................1

Background ............................................................................................................................................2

LOSA Development...........................................................................................................................2

Managing and Understanding Error...................................................................................................3

Method....................................................................................................................................................3

Instrument...........................................................................................................................................3

Procedure............................................................................................................................................5

Participants.........................................................................................................................................5

Results....................................................................................................................................................6

Preparation..........................................................................................................................................6

Development ....................................................................................................................................10

Training ............................................................................................................................................11

Implementation.................................................................................................................................13

Reporting and Feedback...................................................................................................................14

Background ......................................................................................................................................15

Discussion ............................................................................................................................................17

Preparation........................................................................................................................................17

Development ....................................................................................................................................17

Training ............................................................................................................................................18

Implementation.................................................................................................................................19

Reporting and Feedback...................................................................................................................19

Conclusion............................................................................................................................................19

References............................................................................................................................................19

Appendix: LOSA Best Practices Questionnaire.....................................................................................1
iv

LIST OF TABLES

Table 1. LOSA Best Practices Questionnaire Sections with Descriptions and Sample Questions.................4

Table 2. 11 Major Steps to Implement a LOSA Program...............................................................................5

Table 3. Roles Served by Respondents in LOSA Programs...........................................................................6

Table 4. Multi-Level and Multi-Strategy Marketing Effort for LOSA Programs...........................................8

Table 5. Methods Used to Introduce LOSA Program to the Workforce.........................................................9

Table 6. Methods Used for LOSA Implementation Team Member Selection..............................................10

Table 7. Recommended LOSA Training ......................................................................................................12

Table 8. Active Safety Programs ..................................................................................................................16
v

LIST OF FIGURES

Figure 1. Approaches to safety management. .................................................................................................1

Figure 2. Factors considered in the LOSA program Readiness Assessment. .................................................7

Figure 3. Support sought for LOSA program across organizational groups...................................................7

Figure 4. Initial support for LOSA program across groups. ...........................................................................8

Figure 5. Change in level of support/acceptance for LOSA program across groups......................................8

Figure 6. Reported challenges in gaining support for LOSA program from senior management. .................9

Figure 7. Personnel and departments represented on the most recent program team. ..................................11

Figure 8. Main focus for collecting LOSA data in your organization. .........................................................13

Figure 9. Data collection format in use by the LOSA observers. .................................................................14

Figure 10. Personnel or departments informed of LOSA data collections results........................................15

Figure 11. Time since most recent LOSA program was introduced.............................................................15

Figure 12. Progress of most recent LOSA program......................................................................................16
vi

LIST OF ACRONYMS

CRM..................................................................................................................Crew Resource Management

FAA ............................................................................................................ Federal Aviation Administration

ICAO.............................................................................................International Civil Aviation Organization 

LOSA..................................................................................................... Line Operations Safety Assessment

Mx...............................................................................................................................................Maintenance

SMS ..................................................................................................................Safety Management Systems

TEM................................................................................................................Threat and Error Management
vii

EXECUTIVE SUMMARY

Data-driven modifications to risk management are needed in maintenance and ramp environments to 

assist the Federal Aviation Administration (FAA) in the development of adequate tools/systems that 

increase safety by aiding the identification of hazards and managing the associated risks. Proactive and 

predictive approaches, as opposed to reactive changes based on post-accident/event investigation, align 

with the principles of risk management and fundamental concepts of Safety Management Systems 

(SMSs).

In 1994 researchers at the University of Texas-Austin developed a Crew Resource Management 

(CRM) audit methodology for normal operations in the cockpit for Delta Airlines (Klinect, Murray, 

Merritt, & Helmreich, 2003). This area of research fostered the birth of the Threat and Error Management 

(TEM) model and development of the Line Operations Safety Audit (Helmreich, Wilhelm, Klinect, & 

Merritt, 2001). A Line Operations Safety Assessment (LOSA) is a formal process that uses trained 

observers to monitor normal operations of a company’s activities and record their observations. The 

findings are not intended to be used punitively but are meant to provide a “diagnostic snapshot” of the 

positives and the areas in need of improvement within a company (FAA, 2006; Ma et al., 2011).

The success of LOSA in the flight deck area led the FAA to sponsor a Maintenance and Ramp Line 

Operations Safety Assessment project in October 2008 (Ma & Rankin, 2012). The Maintenance LOSA 

referred to Line Operations Safety Assessment, believed to be softer than the term “Audit.” The FAA 

joined forces with Airlines for America (A4A, formerly Air Transport Association) as part of the 

Maintenance and Ramp Human Factors Taskforce committee. Tremendous time and effort spanning over 

three years produced observation data collection instruments, electronic databases, LOSA procedures, and 

training. These products are currently available to the public on the FAA’s Aviation Maintenance Human 

Factors website http://www.faa.gov/about/initiatives/maintenance_hf/.

Incipient knowledge was gained with support from Boeing, United, Jet Blue, and UPS airlines in beta 

testing and vetting the LOSA products and communications. Boeing continues to support this effort 

through promotional outreach to the aviation community. 

The FAA encourages airlines to conduct LOSAs in the interest of safety improvement; however, it 

does not mandate, approve, or monitor LOSA programs (FAA, 2006). The FAA developed the Aviation 

Safety Information Analysis and Sharing (ASIAS) system to promote the open exchange of safety data 

such as LOSA among users, but the system does not collect information specific to LOSA programs. We 

distributed a questionnaire to evaluate the utility of LOSA programs across the industry and identify 

lessons learned and best practices for implementing a program. We examined Ramp LOSA and 

Maintenance LOSA programs that either had been implemented or were being implemented at airlines; 

maintenance, repair, and overhaul service companies; and other aviation service organizations. We used 

the 11 major steps of implementing a LOSA program identified in the Implementation Guideline for 

Maintenance Line Operations Safety Assessment and Ramp LOSA Programs (Ma & Rankin, 2012) as a 

basis for the questionnaire. The questions were intended to extract lessons learned and challenges the 

participants experienced during the implementation of their LOSA programs. 

LOSA questionnaire results increase the FAA’s knowledge of LOSA activity and successful LOSA 

programs within the aviation community. Our results show the progress rate of phase completion for the 

most recent LOSA programs overall was higher for Ramp than for Mx respondents. All Ramp 
viii

respondents reported completion of all phases of their organization’s LOSA program. In contrast, Mx 

respondents reported varied states of completion for the phases. Ramp respondents represented 

organizations with more active observers conducting between 200 and 11,000 observations during the 

implementation period, while Maintenance LOSA organizations had fewer observers by comparison and 

had conducted between 5 and 30 observations during the implementation period. Therefore, conclusions 

from the questionnaire are limited by these factors; however, the feedback from the respondents is of 

value as they are users of the LOSA program and described their experiences.

As noted previously, there is not a common database used by participating carriers or companies to 

share information. Thus, the most optimal method for identifying challenges, best practices, and lessons 

learned is to solicit feedback from users. The small sample limits the results; nonetheless, the information 

is vital as companies explore using proactive, voluntary programs as part of their SMSs. The challenges 

noted can be shared with the airline maintenance community to assist as other companies consider 

implementing a LOSA program. 
1

LINE OPERATIONS SAFETY ASSESSMENTS (LOSA) IN 

MAINTENANCE AND RAMP ENVIRONMENTS

INTRODUCTION

Safety is a primary concern in aviation. Traditionally, air operators relied on the investigation of 

accidents and serious incidents for data collection and analysis of safety data to manage and react to 

safety concerns (reactive). Today, they have adopted a new approach that explores the organizations’

normal operations and environment to detect emerging safety risks (predictive). This new approach also

attempts to mitigate risks by identifying threats and managing errors before, rather than after, an 

accident/incident occurs (proactive) (ICAO, 2002). 

The Line Operations Safety Assessment (LOSA) utilizes trained observers to monitor and record 

company activities during normal and uncommon operations. Organizations that examine operations 

during both normal and eventful situations are better poised to gain the full picture of their day to day 

business activities and identify areas of success and where safety may be compromised. Such auditing has 

the potential to evaluate an organization (including its systems, processes, and personnel), ascertain the 

validity and reliability of its safety information, and consequently assess its internal control (Ma et al., 

2011). In 2012, Ma and Rankin furthered research by identifying implementation steps throughout daily 

operations. These major steps were utilized as a basis to evaluate LOSA programs across the industry. 

Figure 1 provides a description of how organizations can respond and act to maintain safety within their 

organization. This figure contrasts reactive, proactive, and predictive approaches to safety management. 

There has been confusion distinguishing between the “proactive” and “predictive” approaches. In this 

study, the difference between the two is that the proactive approach focuses on conditions that are known 

to be hazardous now, whereas, the predictive approach focuses on predicting present conditions that may 

potentially become hazards in the future. The basis for organizational processes can be located on the 

Federal Aviation Administration (FAA) Programs and Initiatives for SMS

website: http://www.faa.gov/about/initiatives/sms/explained/basis/

Figure 1. Approaches to safety management.
2

A Line Operations Safety Assessment (LOSA) is a proactive and predictive approach to safety 

management and is aligned with the functional components of a Safety Management System (SMS; Ma et 

al., 2011): Safety Policy, Safety Risk Management, Safety Assurance, and Safety Promotion. Component 

explanations are located on the FAA Programs and Initiatives for SMS website: 

https://www.faa.gov/about/initiatives/sms/explained/components/. Initially developed for use in the flight 

deck environment, LOSA has expanded to the ramp and maintenance environments (Ma et al., 2011). A 

LOSA program is an organizational tool used to assess and monitor the day-to-day operations of these 

environments by examining the operating environment and documenting positive behaviors and areas in 

need of improvement within a company. The goal is to identify best practices and to uncover threats and 

errors that often go unnoticed, but are precursors to accidents and incidents (ICAO, 2002; Klinect, 

Murray, Merritt, & Helmreich, 2003). During a LOSA observation, safety threats are logged. The 

observer’s record details related to threats and errors as well as how the observed behaviors might be 

associated with negative outcomes (Ma et al., 2011). The LOSA process documents potential safety risks

so that errors can be identified and managed. In addition, the process encourages members of an 

organization to move from reacting to events to uncovering hazards that could lead to future problems 

(https://www.faa.gov/about/initiatives/sms/explained/components/). Additionally, LOSAs assist with 

identifying examples of superior performance that can be reinforced and used as models for training 

(ICAO, 2002).

LOSA’s acceptance is widespread throughout the aviation community; however, development and 

implementation of a LOSA program requires a number of considerations (Ma & Rankin, 2012). In the 

interest of promoting the use of LOSA, the FAA sponsors a website with free downloads of promotional 

materials, training modules, LOSA forms, and databases with reporting functions for storing LOSA 

observation data. The materials are located on the FAA Human Factors in Aviation Maintenance 

website: https://www.faa.gov/about/initiatives/maintenance_hf/.

This report is meant to equip readers with details about the steps that some organizations have taken 

to implement a Maintenance LOSA or Ramp LOSA program. We gathered information about the 

program implementation processes from these companies via a questionnaire based on the 11 major steps 

to implementing a LOSA program, detailed by Ma and Rankin (2012). 

BACKGROUND

LOSA Development

LOSA emerged from the evolution of Cockpit/Crew Resource Management (CRM) and the 

development of Threat and Error Management (TEM) (Helmreich, Klinect, & Wilhelm, 1999). Within the 

realm of aviation, CRM is the concept of reducing human error by training crews to make better use of 

human resources in the operational environment (Helmreich, Merritt, & Wilheim, 1999). TEM is based 

on the premise that human errors cannot be completely eliminated and involves the management of 

threats (events or errors that occur independently of the flight crew) and errors (human actions or 

inactions that result in unintended consequences) (Merritt & Klinect, 2006) . By identifying potential 

threats/error vulnerabilities within a system, proactive measures can be taken to reduce the chances of 

errors occurring and minimize safety risks. Both CRM and TEM concepts encourage the idea that human 

error is inevitable but that measures can be taken to reduce the safety risks it poses (Helmreich, Klinect, & 

Wilhelm, 1999).
3

Managing and Understanding Error

In 1994, a collaborative partnership was formed between Delta Airlines and the University of TexasAustin while engaged in a Human Factors Research Project to develop an audit methodology for 

assessing CRM training transfer in real-time operations (Klinect, Murray, Merritt, & Helmreich, 2003). 

Initially, researchers designed observation forms to record CRM behaviors in the cockpit. However, in 

efforts to capture the full complexity of a flight, researchers adopted a TEM perspective and expanded the 

forms to record threats and errors, how they were managed, and the outcomes of errors. Research on 

CRM behaviors progressed with the understanding that, in order to assess “normal” behaviors on the line 

rather than “angel performance” (ideal behavior due to the awareness of being observed), crew members 

that were observed had to believe there would be a guarantee of confidentiality and non-punitive 

repercussions to errors (Merritt & Klinect, 2006). Ultimately, this led to the creation of Flight LOSA. 

In October 2008, the FAA sponsored a Maintenance and Ramp Line Operations Safety Assessment 

project with the goal of extending the LOSA methodology to aviation maintenance and ramp 

environments (Ma & Rankin, 2012). The FAA joined forces with Airlines for America (formerly Air 

Transport Association) to form a Maintenance and Ramp Human Factors Taskforce committee to lead 

and oversee this effort. By October 2011, the committee had produced data collection instruments such as 

observation forms for maintenance and ramp operations, threat codes and error codes, electronic 

databases, LOSA procedures, and training for maintenance and ramp operations. These products have 

undergone a series of beta tests and revisions with support from the air carriers United, JetBlue, and UPS, 

including program performance upgrades to the electronic databases from Access to SQL: 

http://www.faa.gov/about/initiatives/maintenance_hf/losa/. Materials such as training, forms, and other 

publications are located on the FAA Human Factors in Aviation Maintenance website: 

https://www.faa.gov/about/initiatives/maintenance_hf/.

Following the release of the supporting documentation necessary for a Maintenance and Ramp LOSA 

program, we asked participants to describe the steps that their organizations have taken to implement a 

Maintenance LOSA or Ramp LOSA program. We solicited feedback from Maintenance (Mx) and Ramp 

LOSA users for a comparison of programs during the implementation process. The goal was to assess the 

progress of the responding organizations’ Maintenance LOSA or Ramp LOSA programs.

METHOD

Instrument

Drawing from the descriptions provided by Ma and Rankin (2012), we used the major steps of 

implementing a LOSA program identified in the Implementation Guideline for Maintenance Line 

Operations Safety Assessment and Ramp LOSA Programs as the broad outline for development of the 

LOSA questionnaire. We organized the steps under five questionnaire headings: Preparation, 

Development, Training, Implementation, and Reporting and Feedback (Table 1).
4

Table 1. LOSA Best Practices Questionnaire Sections with Descriptions and Sample Questions

Section Description Sample Questions

Preparation Questions relate to assessment of the 

organization’s readiness for the program 

and obtaining support from senior 

management, the labor union, and the 

workforce. 

• How supportive, initially, were 

Maintenance/Ramp workers of the 

most recent LOSA program in your 

organization?

• What type of information was required 

by senior management to decide on 

support for the most recent LOSA 

program in your organization?

Development Questions relate to organizing a program 

team and building the program 

infrastructure (i.e., database, training, 

and publicity of the program).

• How did your organization select 

members for the most recent LOSA 

program team?

• In your organization, what resources 

were used as framework for the most 

recent LOSA program infrastructure,

i.e., database, forms, training, and 

publicity? 

Training Questions relate to training LOSA 

observers, reviewers, quality controllers, 

and program administrators. 

• Which groups received FAA LOSA 

training in your organization?

• Which FAA training modules were in 

use for the most recent LOSA program 

in your organization?

Implementation Questions relate to collecting data and 

populating and maintaining a LOSA 

database.

• How many observations were 

conducted during the implementation 

period?

• Was the FAA LOSA database software 

in use “as is” (no modifications)?

Reporting and 

Feedback

Questions relate to compiling reports 

from the collected data and informing 

management and the workforce of the 

results. 

• What types of reports were frequently 

generated using the LOSA database 

tool?

• What personnel or departments did 

your organization inform of the results 

for the most recent LOSA data 

collections? 

The five sections of the LOSA questionnaire broadly encompass Ma and Rankin’s 11 major steps to 

program implementation. Ma and Rankin’s steps 4 and 5 address aspects of a LOSA program are not 

directly part of the initial implementation; therefore, we partially address these steps in the Development 

and Background sections of the LOSA questionnaire (Table 2). We discuss the LOSA Questionnaire 

sections in more detail in the Results sections of this report.
5

Table 2. 11 Major Steps to Implement a LOSA Program

Steps Major Steps to Implement a LOSA Program LOSA Questionnaire Section

1. Obtain senior management’s buy-in. If approval is given, 

then take the following steps

Preparation

2. Form an implementation team Development

3. Market Maintenance and/or Ramp LOSA programs Preparation

4. Integrate with existing safety programs/SMS Background* 

5. Develop LOSA infrastructure, including three parallel 

activities: Adapt/customize LOSA database, Conduct 

train-the-trainer training, Establish and maintain a virtual 

LOSA website

Development*

6. Adapt/customize and conduct observer training Training

7. Collect data Implementation

8. Validate data Implementation

9. Populate and maintain database Implementation

10. Analyze data and compile a report Reporting and Feedback

11. Provide feedback to employees Reporting and Feedback

*Partially addressed by the questionnaire.

Procedure

In July 2014, we invited 33 people to participate in taking the Mx and Ramp LOSA questionnaire.

The names were gathered from a database of Mx and Ramp LOSA contacts (LOSA beta testers, aviation 

professionals who had requested LOSA information, and those known by us to have had involvement 

with a LOSA program). The questionnaire was distributed electronically via email to the participants. 

Questions consisted of forced-choice, open-ended, and combination items. We used question branching 

(skip logic) to create a customized path through the questionnaire based on the participant’s responses to 

certain questions. The questionnaire was available online for three consecutive weeks. We estimated it 

would take 15-25 minutes for respondents to complete the questionnaire. A weekly email was sent to nonresponders to request their participation in the questionnaire. The completed questionnaires were 

submitted directly to a contractor for removal of any identifiable information before the data were 

provided to the FAA. Participants were instructed to consider the LOSA program from their current or 

former Mx and Ramp organization when reporting their most recent involvement.

Participants

Questionnaire respondents included aviation professionals from commercial and cargo airlines; 

helicopter service providers; and maintenance, repair, and overhaul service companies who had 

involvement with planning, implementing, or participating in a Ramp or Maintenance LOSA program. A 

total of 13 questionnaires were submitted. Ten participants indicated current involvement in a LOSA 

program and three had previous LOSA program experience. All participants had current or previous 

experience in different types of LOSA programs including only Ramp, only Maintenance, a combination 

of both Ramp and Mx, or only Flight LOSA programs. Given the purpose of the questionnaire was to 
6

compare Ramp and Mx LOSA, the respondent with only Flight LOSA experience was removed from the 

remainder of the report. The remaining 12 respondents had current or previous involvement with only

Ramp, only Mx, or both Ramp and Mx LOSA programs. Most respondents reported serving multiple 

roles in their organization’s LOSA program, although one did not respond (see Table 3).

Table 3. Roles Served by Respondents in LOSA Programs

Ramp Maintenance Roles

4 4 Program Manager

3 1 Database Administrator

4 2 Data Analyst

2 1 Observer

3 2 Senior Management

1 1 Frontline Manger

2 3 Quality Assurance Representative

0 1 Operations Representative

2 3 Training Representative

2 3 Safety Representative

0 1 Labor Union Representative

0 1 Employee Group Representative

5 6 Total Respondents (N=11)

Frequency Count sums to greater than the Number of Respondents (n) due to multiple 

responses [mark all that apply].

RESULTS

We describe the results in accordance with LOSA program stages, partitioning into five distinct

sections of the LOSA questionnaire: Preparation, Development, Training, Implementation, and Reporting 

and Feedback. Additionally, we provide information about the maturity of LOSA programs and other 

existing safety programs in Mx and Ramp organizations. Collectively, the results from each section

provide insight about whether the LOSA programs adhere to Ma and Rankin’s steps to LOSA program 

implementation and are described below.

Preparation

The Preparation section of the Mx and Ramp LOSA questionnaire coincides with Ma and Rankin’s 

steps 1 and 3 (see Table 2): Obtain senior management’s buy-in and market Maintenance and/or Ramp 

LOSA programs. Ma and Rankin proposed that a readiness check should be conducted prior to 

organizations instituting a LOSA program. Recognizing the key elements noted as critical for readiness,

we assessed organizational readiness for a LOSA program through the completeness of the Readiness 

Assessment and by examining support for LOSA programs across organizational levels (i.e., senior 

management, labor union, labor force, labor/employee groups; step 1). Participants were asked to indicate 

the degree to which a series of factors were considered in their organization’s Readiness Assessment on a 

5-point scale from Not at all to Completely. As shown in Figure 2, the majority of Ramp respondents 

reported that each factor was Very or Completely considered in their organization’s Readiness 

Assessment; however, fewer Mx respondents reported that the factors were considered within their 

organization.
7

Figure 2. Factors considered in the LOSA program Readiness Assessment.

When asked about support for the LOSA program, most respondents agreed that support was sought 

from the various levels or groups within the organization and all reported that support was sought from 

senior management. Support for employee groups was the only visible difference in that Mx respondents 

indicated that program support was not sought from employee groups. However, the reason for this is 

unknown and we should be cautious drawing any conclusions from such a small sample. Figure 3

displays the support sought from each group. 

Figure 3. Support sought for LOSA program across organizational groups.

For each of the respective groups, we asked about their initial support for the program and if the 

support/acceptance changed as the program progressed. When asked about senior management’s support, 

the majority felt that they were very or completely supportive initially (Figure 4), and that their support 

either did not change or increased over time (Figure 5). Union/labor groups, workers, and employee 

groups were viewed as less supportive initially, but the majority of these group respondents felt that 

support/acceptance of the program increased over time.

66.7 66.7

83.3 83.3

100.0

66.7

100.0

16.7

66.7

50.0 50.0

66.7

50.0 50.0

Potential

financial

benefits

Program

performance

metrics

Required

resources

Potential

safety

benefits

Other safety

programs

success

Familiarity

with SMS

concept

Existence of 

“just culture”

% Very/Completely Considered

Ramp Mx

100.0

80.0 80.0

60.0

100.0

83.3

66.7

0.0

Senior Management Union/Labor

Group

Mx/Ramp Workers Employee

Groups

% Support Sought 

Ramp Mx
8

Figure 4. Initial support for LOSA program across groups.

Figure 5. Change in level of support/acceptance for LOSA program across groups.

Gaining support from senior management is critical to instituting a successful LOSA program. Once 

that is established, communicating the benefits and purpose to the union and workforce must follow. 

Education and marketing the program are key steps. Ma and Rankin proposed using a multi-layer and

multi-strategy approach to market the LOSA program within the organization to lessen the resistance that 

often accompanies change in the work environment. They suggested using different marketing strategies

and methods based on audience type (Table 4). 

Table 4. Multi-Level and Multi-Strategy Marketing Effort for LOSA Programs

Audience Strategy and Methods

Middle Management Face-to-face meeting, brochure, project website

Labor union and employee group Face-to-face meeting, project website

Frontline employees Standardized posters, project website, letter/reminder from 

the implementation team

Business partners Face-to-face meeting, project website

We considered marketing to mean an introduction of the LOSA program across the workforce. To 

find out if organizations with a LOSA program are using Ma and Rankin’s recommended approach, we 

asked participants about the method used to introduce the LOSA program to the workforce in their 

100.0

25.0 25.0

33.3

83.3

40.0

50.0

0.0

Senior Management Union/Labor Group Mx/Ramp Workers Employee Groups

% Initial Support

Ramp Very/Completely Supportive

Mx Very/Completely Supportive

40.0

100.0

75.0

33.3

50.0

60.0

100.0

Senior Management Union/Labor Group Mx/Ramp Workers Employee Groups

% Change in Support

Ramp Somewhat/

Much More

Mx Somewhat/

Much More
9

organization (step 3). As indicated in Table 5, some respondents reported that face-to-face meetings were 

the most frequently used method for introducing the LOSA program to the workforce. See item B13 in 

Appendix A.

Table 5. Methods Used to Introduce LOSA Program to the Workforce

Frequency Count Methods % of Respondents

Ramp Mx Ramp % Mx %

1 4 Posters 20.0 67.0

1 3 Brochures 20.0 50.0

5 5 Face-to-face meetings 100.0 83.0

1 3 Company website 20.0 50.0

2 4 Emails 40.0 67.0

4 1 Memos 80.0 17.0

0 1 Other 00.0 17.0

5 6 Total Respondents

Frequency Count sums to greater than the Number of Respondents (n) due to multiple responses [mark all that apply].

Challenges during preparation. Some respondents reported challenges in gaining support from 

senior management, union/labor groups, and maintenance/ramp workers (Figure 6). While no Ramp 

respondents reported challenges in gaining support from senior management, 50% of Mx respondents 

reported challenges with senior management. Some challenges cited in gaining support from senior 

management included concern about the effect on daily maintenance operations, overtime coverage, 

determining appropriate observation targets, skepticism until the benefits were produced, support for 

required resources, and convincing management that LOSA is a good investment despite the necessary 

man-hours needed to carry out the observations in the field.

Figure 6. Reported challenges in gaining support for LOSA program from senior management.

0.0

100.0

0.0

50.0

33.3

16.7

Yes No Do not know

Ramp Mx
10

Most respondents reported challenges in gaining support from union/labor groups. One respondent

indicated dislike of the peer-to-peer concept and felt it was reporting on others and was not that person’s

job. Another reported the union was concerned that information gathered by the observers would identify 

specific employees and results would be used for punitive purposes. Anonymity and volunteerism were 

also concerns. 

Most Ramp respondents (75%) reported challenges in gaining support from workers while only 25% 

of Mx respondents reported challenges. Examples of challenges included LOSA being used as a 

disciplinary tool, observers being labeled as “snitches,” and the perception that the LOSA program is

focused on the individual performing the task rather than the task itself. 

Development

The Development section of the Mx and Ramp LOSA questionnaire coincides with Ma and Rankin’s 

steps 2 and 5 (see Table 2): Form an implementation team and Develop LOSA infrastructure. We asked 

participants about the existence and formation of a LOSA program implementation team in their 

organization (step 2). All Ramp respondents and most Mx respondents (83.3%) reported that their 

organization formed a team responsible for conducting and coordinating the LOSA program processes. Of 

those, all Ramp and most Mx respondents were also members of their organization’s LOSA 

implementation team. Multiple methods were employed for implementation team member selection

(Table 6). See item C3 in Appendix A.

Table 6. Methods Used for LOSA Implementation Team Member Selection

Frequency Count Methods

Ramp Maintenance

4 2 Accepted volunteers

4 0 Assigned by management

0 2 Applied and qualified

2 2 Nominated by the union

3 2 From the department that oversees 

safety programs

5 2 Total Respondents

Frequency Count sums to greater than the Number of Respondents (n) due to multiple responses 

[mark all that apply].

To understand the makeup of organizations’ LOSA implementation teams, we asked which personnel 

and departments internal to the organization were represented on the team. All Ramp and Mx respondents

reported that the personnel from their safety departments were members of their program teams.

Additionally, several of them reported that senior management, frontline managers, training and quality 

assurance departments, and union/labor groups were represented on the teams (Figure 7). 
11

Figure 7. Personnel and departments represented on the most recent program team.

We considered construction of a LOSA database, preparation of training, and publication of LOSA 

information for the workforce as tasks associated with the development of LOSA infrastructure (step 5). 

We assessed the development of LOSA program infrastructure in organizations by asking what resources 

were used as framework for the most recent LOSA program. The top resources reported by all 

respondents were existing materials from a LOSA program within their organization and FAA materials

for LOSA.

Challenges during development. When we asked about challenges/issues in organizing the LOSA 

implementation team, one respondent indicated that their organization experiences challenges with 

coordination and initially, responsibilities of the team, but that trial and error and past audit experience 

helped to smooth things out. Participants reported challenges or issues in constructing a LOSA database. 

Some challenges included interfacing FAA LOSA software with existing company IT platforms, and 

obtaining approval to install the SQL-based software. One challenge noted in developing LOSA training

was ensuring high quality training in organizations with large numbers of observers or trainers, or both. 

Training

The Training section of the LOSA questionnaire coincides with Ma and Rankin’s step 6 (Table 2):

Adapt/customize and conduct observer training. Ma and Rankin provided five recommended LOSA 

training components to be adapted to suit an organization’s safety needs and environment. The FAA

LOSA training modules are a part of the key contents of the LOSA observer training recommended by 

Ma and Rankin (Table 7). We assessed the utilization of FAA LOSA training modules in organizations 

instituting a LOSA program.

60.0 60.0

100.0

20.0

80.0

40.0

0.0 0.0

75.0 75.0

100.0

75.0

50.0 50.0

0.0 0.0

Senior

mgmt

Frontline

mgrs

Safety

personnel

Training

Dept

Quality

Assurance

Dept

Union/

Labor Group

Other

(specify

below)

Do not

know

Ramp Mx

*represented on most recent program team
12

Table 7. Recommended LOSA Training

FAA LOSA 

Training Modules

Specific LOSA 

Training 

Components

Key Contents Duration and Format

• Threat and Error 

Management 

(TEM) and 

LOSA

1.LOSA Awareness 

Training

• Maintenance LOSA and 

Ramp LOSA 

background

• TEM

• Brief ROI

Less than 1 hour, inperson training or 

Computer-based Training 

(CBT)

• Threat and Error 

Management 

(TEM) and 

LOSA

• LOSA Mx

Operations Base 

and Line or 

LOSA Ramp 

Operations

• Mx Data Entry or 

Ramp Data Entry

• Admin Data 

Entry

2.LOSA Train-theTrainer

• LOSA Awareness 

training (full)

• TEM

• LOSA products

• Implementation process

• Forms/Scenarios 

(Maintenance LOSA 

and/or Ramp LOSA)

• Database

• Reporting

16-18 hours, in-person 

training with hands-on 

observation and database 

practice and “teach-back” 

session

• Threat and Error 

Management 

(TEM) and 

LOSA

• LOSA Mx 

Operations Base 

and Line or 

LOSA Ramp 

Operations

• Mx Data Entry or 

Ramp Data Entry

• Admin Data 

Entry

3.LOSA Observer 

Training

• LOSA Awareness 

training (full)

• TEM

• LOSA products

• Implementation process

• Forms/Scenarios 

(Maintenance LOSA 

and/or Ramp LOSA)

• Database

• Observation protocol

16 hours, in person 

training with hands-on 

observation and database 

practice

• Mx Data Entry or 

Ramp Data Entry

• Admin Data 

Entry

4.LOSA Database 

Training

• Set up LOSA database

• Maintain LOSA 

database

• Customization

2 hours, in person training 

with hands-on database 

practice

• N/A 5.LOSA Reporting 

Training

• Reporting 1-2 hours, in person 

training with hands-on 

database practice

The training consisted of modules covering Maintenance Operations Base and Line, Ramp 

Operations, Threat and Error Management (TEM) and LOSA, Considerations for Implementing LOSA, 

Maintenance/Ramp Data Entry, and Administrative Data Entry. Respondents were asked to mark all of 

the FAA training modules used for their most recent program by selecting “used as is” or ”modified for 
13

use” for modules relevant to their program. Results indicated that 60% of Mx respondents and 20% of 

Ramp respondents used FAA training materials for their organization’s LOSA program. 

Overall results show that all FAA training module topics, except LOSA Ramp Operations training,

were modified for use by the organization instituting a LOSA program. While all Ramp respondents 

reported that new training was created within their organization, only 60% of Mx respondents reported 

that their organization created new training. Some new training reported by an Mx respondent included 

the incorporation of soft skills training or customization for their specific program (both Mx and Ramp).

Respondents indicated that new training was created for organizational groups ranging from LOSA 

quality controllers to Senior Management; however, they indicated that the training was primarily created 

to aid LOSA observers and LOSA trainers.

Challenges during training. When we asked about challenges/issues with adapting/customizing and 

conducting LOSA training in their organizations, one respondent indicated that training is widespread and 

quality control measures are needed to ensure consistency of the message. There was also a reported need 

to develop and customize training specific to the organization as well as a consideration for computerbased training versus in-person training. Lastly, there were challenges in finding times for training that 

were suitable for staff and the training provider. 

Implementation 

The Implementation section of the questionnaire coincides with Ma and Rankin’s steps 7-9: collect 

data, validate data, and populate and maintain the database. We assessed the focus and frequency of data 

collection and how it is recorded in LOSA programs. When asked about the focus for collecting LOSA 

data, the majority of Ramp respondents reported the main focus of data collection was the entire 

operation, while responses were split equally among Mx respondents who reported that entire operation, 

identified/known problems, specific aircraft types, or specific facility was the main focus (Figure 8).

Figure 8. Main focus for collecting LOSA data in your organization.

The FAA offers LOSA data collection instruments in paper and electronic formats. To learn about the 

format used to collect LOSA data in organizations, we asked what data collection format was used by 

organizational LOSA observers to collect their data. All respondents for both Ramp and Mx reported use 

of newly created electronic forms, while 66.7% of Ramp respondents and 50% of Mx respondents also 

indicated that their organization created new paper forms (Figure 9).
14

Figure 9. Data collection format in use by the LOSA observers.

Respondents were asked how many observers were active during the Implementation period and how 

many observations were conducted during that period. Results show that Ramp respondents representing 

organizations with more active observers reported that their organizations conducted between 200 and 

11,000 observations during the implementation period, while Mx organizations had fewer observers, by 

comparison, reporting that their organization had conducted between 5 and 30 observations during the 

implementation period. For those that responded, we found that Ramp programs rotated their observers, 

while observers in Mx programs were not rotated.

Challenges during implementation. When asked about challenges/issues with collecting LOSA 

data, respondents noted the inability to perform the number of required observations and difficulty getting 

permission from technicians to be observed. No challenges with populating and maintaining the database 

were reported by Ramp participants; however, Mx participants reported challenges with networking the 

FAA LOSA database tool with multiple facilities.

Reporting and Feedback

The Reporting and Feedback section of the LOSA questionnaire coincides with Ma and Rankin’s 

steps 10 and 11: Analyze data and compile a report, and Provide feedback to employees. We assessed the 

utility of the FAA Maintenance and Ramp LOSA Database software for generating reports in 

organizations with a LOSA program (step 10). Sixty-percent of Ramp respondents and 50% of Mx 

respondents reported that their organizations used the FAA’s LOSA database software tool for generating 

LOSA reports. Respondents who did not use the database software tool for generating reports reported

developing an in-house program for communicating and generating reports. 

The personnel or departments informed of the results of LOSA data collections (step 11) included 

senior management, frontline managers, safety personnel, the training department, maintenance/ramp 

workers, and labor unions. All Ramp respondents agreed that frontline managers, safety personnel, and 

the training department were informed and most agreed that senior management and ramp workers were 

also informed (Figure 10). Mx respondents’ percentages were equally dispersed across senior 

management, frontline managers, safety personnel, maintenance workers, and labor unions with fewer 

indicating that the training department was informed.

33.3

0.0

66.7

0.0 0.0

100.0

0.0

50.0 50.0

0.0 0.0

100.0

Paper - FAA

'as is'

Paper -

Adapted FAA

Paper -

Created new

Electronic -

FAA 'as is'

Electronic -

Adapted FAA

Electronic -

Created new

Ramp Mx
15

Figure 10. Personnel or departments informed of LOSA data collections results.

The questionnaire assessed the method respondents’ organizations used for providing feedback to 

senior management, frontline managers, and maintenance/ramp workers. All respondents reported that 

face-to-face meetings were the most widely used method for reporting LOSA results to these groups. 

However, when reporting LOSA results to Ramp workers, face-to-face meetings percentages dipped to 

60%. Provision of LOSA results to these groups via a written report on overall findings was reported by 

at least half of all respondents for both Ramp and Maintenance. Participants were not specifically asked 

about challenges they may have experienced during the Reporting and Feedback phase; however, the 

majority reported that they were asked to share additional information about best practices and lessons 

learned, which are summarized in the Discussion section of this report.

Background

The Background section of the LOSA Questionnaire coincides with Ma and Rankin’s step 4: 

Integrate with Existing Safety Programs/SMS. This section of the questionnaire collects additional 

details about the participants and their organizations’ LOSA programs that are factors in identifying best 

practices for LOSA implementation. Information about the participants, such as their roles in the LOSA 

program, was discussed in the Methods section of this report (Participants). When asked how long ago 

the most recent LOSA program was introduced into their organization, the majority of Ramp respondents 

reported more than 3 years, while Mx respondents had a broader range of tenure, as shown in Figure 11. 

Figure 11. Time since most recent LOSA program was introduced. 

80.0

100.0 100.0 100.0

80.0

60.0

0.0

66.7 66.7 66.7

33.33

66.67 66.7

33.3

Senior

mgmt.

Frontline

managers

Safety

personnel

Training

dept.

Mx/Ramp

workers

Labor Union Other

Ramp Mx

0.0

16.7

33.3

50.0

0.0

16.7 16.7 16.7

33.3

16.7

6 mos

or less

More than 6 mos,

less than 1 yr

More than 1 yr,

less than 3 yr

More than 3 yr Do not know

Ramp Mx
16

Participants were asked the status of the most recent LOSA program across five stages: preparation, 

development, training, implementation, and reporting. The majority of respondents noted their most 

recent program was currently active; however, some reported that their most recent Mx LOSA program 

had gone dormant prior to completing all five phases of LOSA implementation.

When asked about the progress of the most recent LOSA program, all Ramp respondents reported 

completion of all phases of their organization’s LOSA program. In contrast, Mx respondents reported 

varied states of completion for the phases (Figure 12).

Figure 12. Progress of most recent LOSA program.

All Ramp and Mx respondents reported that their respective organizations had other active safety 

programs in addition to LOSA. Table 8 shows safety programs that were active in their organizations in 

conjunction with their LOSA program. Respondents who selected “Other” went on to list Flight 

Operational Quality Assurance, Internal Safety Reporting System, Aviation Safety Action Plan groups,

and Event Review committees as the other active safety programs in use in their organizations. Thus, 

many Mx and Ramp companies are using LOSA to complement other safety programs. This speaks to the 

maturity of the SMS approach used by many of the companies that responded and choose to participate in 

launching a LOSA for Ramp and Maintenance. See item H5 in Appendix A.

Table 8. Active Safety Programs

Frequency Count Programs % of Respondents

Ramp Maintenance Ramp Maintenance

3 3 Aviation Safety Action Program (ASAP) 60.0 50.0

3 5 Aviation Safety Reporting System (ASRS) 60.0 83.3

3 4 Continuing Analysis and Surveillance 

System (CASS)

60.0 66.7

1 5 Maintenance Error Decision Aid (MEDA) 20.0 83.3

0 1 Ramp Error Decision Aid (REDA) 0.0 16.7

1 2 Other (Specify below) 20.0 33.3

1 0 None of the above 20.0 0.0

0 0 Do not know 0.0 0.0

5 6 Total Respondents

Frequency Count sums to greater than the Number of Respondents (n) due to multiple responses [mark all that apply].

100.0 100.0 100.0 100.0 100.0

83.3

66.7

83.3

50.0

33.3

Preparation

Completed

Development

Completed

Training

Completed

Implementation

Completed

Reporting

Completed

 Ramp Mx
17

DISCUSSION

In the present study, we set out to evaluate the utility of Maintenance and Ramp LOSA programs 

across the industry and to identify best practices for implementing a LOSA program. Given the voluntary 

nature of LOSA, the number of established programs and companies that use FAA LOSA products is 

unknown, and there is not a common database used by participating carriers or companies to share 

information. We solicited feedback from Mx and Ramp LOSA users for a comparison of programs during 

the implementation process. We expected to find that adherence to Ma and Rankin’s implementation 

steps may surface as a best practice in companies that have implemented a LOSA program. The results of 

the LOSA questionnaire support our hypothesis. We found this to be true in all phases of program 

development. Again, we acknowledge that the number of questionnaire respondents was small. 

Ma and Rankin discuss the 11 major steps within three phases of program implementation:

Preparation, LOSA Infrastructure Development, and Active Implementation. They discuss a fourth phase, 

Continuous LOSA Application, which includes additional steps and activities for companies wanting to 

continue LOSA efforts beyond implementation. Our questionnaire addressed the major steps within five 

phases of program implementation. We discuss the challenges of each phase, based on the results of the 

questionnaire, in the following paragraphs. 

Preparation

Challenges in gaining support from senior management for Maintenance LOSA programs stemmed

from concern about the effect the program may have on company operations and personnel usage. To 

lessen these concerns, LOSA implementation teams should determine the focus of the program, including 

the size and timing of implementation. Ma and Rankin suggested that companies should start with a trial 

implementation that focuses on a specific area or operation such as an identified problem area or recent 

organizational change. A trial implementation would allow LOSA teams and senior management to get a 

glimpse of the program process and procedures while limiting operational personnel usage and 

disturbance.

Challenges in gaining support from other organizational groups such as union/labor groups and 

Ramp/Maintenance workers came from concern that the program may result in disciplinary actions taken 

against workers who made errors and/or performed improper procedures. This issue can be challenging as 

LOSA is introduced. Adherence to the nonpunitive foundation of LOSA is critical to the program’s 

success. The commitment to the philosophy of threat and error management and LOSA must be 

understood and supported by senior management prior to implementation. Effective training increases 

knowledge about the program and lets the workforce know ahead of time what to expect and what is 

expected from them. We believe that the FAA’s evolving Compliance Philosophy and industry 

understanding of the term “Just Culture” will increase the acceptance of peer-to-peer observation systems 

like LOSA.

Development

The development phase consists of organizing the implementation team and building the program 

infrastructure, including database construction and development/customization of training. The 

participants reported three challenges during the development phase: 1. interfacing the LOSA software 

with company IT platforms, 2. preparing LOSA training, 3. developing the LOSA program team. 
18

To avoid the first two challenges, we suggest consulting with your company’s IT and Training 

departments after you have assessed readiness for the LOSA program and obtained support from senior 

management. Review the FAA LOSA software specifications and discuss them with your IT department. 

Obtain information and collaborate with personnel regarding company IT restrictions, platforms, and 

training methods/techniques. Gathering this information early during the implementation process will 

alleviate challenges you may encounter during the development and training phases. We caution that a 

database should be designed for simplicity and compatibility with existing IT systems. Avoid a complex 

solution to a straight forward requirement. Solutions are available with some common commercial off the 

shelf data tools.

The third challenge participants reported was with developing the LOSA program team, including 

coordination of team members and determining team responsibilities. The coordination of team members 

may be more of a challenge for large organizations than for smaller ones, considering the greater number 

of employees and the greater scope of the LOSA program. 

For large organizations, we suggest following Ma and Rankin’s recommendations identified in the 

Implementation Guideline for Maintenance and Ramp LOSA programs. They suggested the LOSA 

implementation team should include 8-10 key members, both internal and external to the organization,

and recommended three important characteristics of the LOSA implementation team:

1. Internal members should at least include representatives from the company’s safety, 

operations and training departments, senior management, and personnel designated for data 

analysis and LOSA program manager functions.

2. External members are industry representatives.

3. All members should be willing participants and supporters of safety and continuous 

improvements.

For small organizations, the program team may include only one or two members due to limited 

resources. A LOSA program can be developed by a team with as few as two or three members with each 

member serving multiple roles. For example, the company’s Maintenance Safety Manager may be the 

LOSA Program Manager whose role is to deliver LOSA training and conduct LOSA observations. Data 

analysts may also be involved with database development and implementation.

Training

Participants reported challenges they encountered during the training phase of the program, including

training development, delivery method, schedule, and training consistency throughout the organization. 

Training department involvement during the implementation process may help to reduce challenges that

may occur during the training phase of the implementation process. Early discussions with your training 

department can shed light on training aspects to consider as you plan training for your LOSA program. 

For instance, you should consider the audience (the LOSA users, their roles in the program and 

knowledge of LOSA and LOSA software) when selecting the type of training to provide. Once you know 

the audience, review the FAA LOSA training modules to determine whether they are suitable for your 

training needs, or if you need to customize the training. You should contact your organization’s training 

department for assistance with training development and to determine the best delivery method based on 

the type of training and audience size.
19

Implementation

The implementation phase entails collecting LOSA data and populating and maintaining a LOSA 

database. Organizations implementing a LOSA program must determine both the number of observers to 

be assigned and the target number of observations. Depending on the focus of your LOSA program, it 

may take trial and error to find the most efficient approach for coordinating and conducting LOSA 

observations. Ma and Rankin (2012) recommended coordinating a team of two or three observers for each 

aircraft service turnaround depending on the type of aircraft and activities to be observed. It is critical to 

scale implementation starting with small test groups before proceeding to significant implementation. 

Keeping everyone on the LOSA team informed is important for successful implementation.

Reporting and Feedback 

The present study focused on the process of establishing a Maintenance and/or Ramp LOSA program. 

The goal was to collect best practices and lessons learned from LOSA programs that completed all or any 

phases of the implementation process. The Reporting and Feedback phase of the process involves 

compiling reports from the collected data and informing management and employees of the results. 

Compiling reports and providing feedback to employees are the final steps of the implementation process. 

We were interested in knowing whether or not LOSA programs completed all of Ma and Rankin’s 11 

steps of the implementation process. Although the LOSA questionnaire did not ask participants 

specifically about challenges they may have experienced during the Reporting and Feedback phase, all

Ramp respondents reported completion of all phases of their organization’s LOSA program. Mx 

respondents reported varied states of completion for the phases.

CONCLUSION

The results are limited by the small sample; nonetheless, the feedback and information we obtained 

are vital as companies explore using proactive, voluntary programs as part of their safety management 

systems. As noted previously, there is not a common database used by participating carriers or companies 

to share information. Thus, the most optimal method for identifying challenges, best practices, and 

lessons learned is to solicit feedback from users. A LOSA program will not survive without committing to 

educating all levels of the organization on the process, ensuring that the core principals are maintained, 

and persevering through the challenges of initiating a new safety management approach. The challenges 

noted, lessons learned, and success stories can be shared with the airline maintenance community to assist 

as other companies consider implementing a LOSA program.

REFERENCES

Federal Aviation Administration (2006). Advisory Circular: Line Operations Safety Audits (AC No: 120-

90): Federal Aviation Administration, Washington, DC.

Federal Aviation Administration. (2014, March). Human Factors in Aviation Maintenance. Retrieved 

from http://www.faa.gov/about/initiatives/maintenance_hf/

Federal Aviation Administration. (2014, August). Line Operations Safety Assessments (LOSA). 

Retrieved from http://www.faa.gov/about/initiatives/maintenance_hf/losa/
20

Federal Aviation Administration. (2014, August). Safety Management System Basis. Retrieved 

from http://www.faa.gov/about/initiatives/sms/explained/basis/

Federal Aviation Administration. (2014, August). Safety Management System Components. Retrieved 

from https://www.faa.gov/about/initiatives/sms/explained/components/

Helmreich, R.L., Klinect, J.R., & Wilhelm, J.A. (1999). Models of threat, error, and CRM in flight 

operations. In Proceedings of the Tenth International Symposium on Aviation Psychology (pp. 677-

682). Columbus, OH: The Ohio State University.

Helmreich, R.L., Merritt, A.C., & Wilhelm, J.A. (1999). The evolution of Crew Resource Management 

training in commercial aviation. International Journal of Aviation Psychology, 9(1), 19-32.

Helmreich, R.L., Wilhelm, J.A., Klinect, J.R., & Merritt, A.C. (2001). Culture, error and Crew Resource 

Management. In E. Salas, C.A. Bowers and E. Edens (Eds.), Improving teamwork in organizations: 

applications of resource management training. (pp. 305-331). Erlbaum, Mahwah, NJ.

International Civil Aviation Organization. (2002). Line Operations Safety Audit (Doc 9803 AN/761).

Montreal, Canada.

Klinect, J., Murray, P., Merritt, A., & Helmreich, R. (2003). Line operations safety audit (LOSA): 

Definition and operating characteristics. Paper presented at the 12th International Symposium on 

Aviation Psychology, Dayton, Ohio.

Ma, J., Pedigo, M., Blackwell, L., Gildea, K., Holcomb, K., Hackworth, C., & Hiles, J. (2011). The Line 

Operations Safety Audit Program: Transitioning from Flight Operations to Maintenance and Ramp 

Operations. Report DOT/FAA/AM-11/15. Washington, D.C.: Federal Aviation Administration,

Office of Aviation Medicine.

Ma, J. & Rankin, W. (2012). Implementation Guideline for Maintenance Line Operations Safety 

Assessment (M-LOSA) and Ramp LOSA (R-LOSA) Programs. Report DOT/FAA/AM-12/9. 

Washington, D.C.: Federal Aviation Administration, Office of Aviation Medicine.

Merritt, A. & Klinect, J. (2006). Defensive flying for pilots: An introduction to threat and error 

management. The University of Texas Human Factors Research Project. The LOSA Collaborative.
A-1

APPENDIX: LOSA BEST PRACTICES QUESTIONNAIRE

2014 LOSA Best Practices Questionnaire

This questionnaire focuses on the best practices and lessons learned during the process of establishing a 

Maintenance and/or Ramp Line Operations Safety Assessment (LOSA) program. It is intended for 

organizations that are in the process of implementing a Maintenance and/or Ramp LOSA program, 

whether or not the program is ongoing.

You will be asked how your organization approached instituting the program, in particular, the 

challenges/issues that were encountered, how they were addressed, and why some solutions worked and 

others did not. The questions are partitioned into the following program stages; each is described in terms 

of the associated tasks:

• Preparation - assess the organization’s readiness for the program, provide awareness training 

and gain support from senior management, the labor union, and the workforce.

• Development - organize a program team and build the program infrastructure, including database 

construction, customizing and scheduling training, and distributing program information to the 

workforce.

• Training - provide training to actual users of the LOSA database software tool, i.e., observers, 

reviewers, quality controllers, and database administrators.

• Implementation - collect data, and populate and maintain database.

• Reporting and Feedback - produce reports from the collected data and provide the results to 

management and the workforce.

Your thoroughness in completing the questionnaire is appreciated and will contribute to aviation 

safety and the flying public. 

Note: There are questions written with double verbs like are/were and is/was to capture lessons learned 

from your current and past experiences with LOSA programs. As a reminder, your input will be saved if 

you need to exit and return to complete the questionnaire. You will need your password to log in as 

before. 

Instructions: Read each question carefully before responding. Some questions require a response to 

skip items not relevant to your involvement with a LOSA program.

1. Are you currently involved with a LOSA program? (required)

 Yes (If yes, skip to item 3)

 No

2. Were you previously involved with a LOSA program (current and former organizations)? (required)

 Yes

 No (If no, routed out of the questionnaire)
A-2

3. Which types of LOSA programs have you been involved in (current and former organizations)? [mark 

all that apply] (required)

 Ramp LOSA 

 Maintenance LOSA 

 Flight LOSA (If Flight LOSA is the only selection, routed out of the questionnaire)

 Do not know (If Do not know is the only selection, routed out of the questionnaire)

4. Which type of LOSA program were you most recently involved in? (required)

 Ramp LOSA

 Maintenance LOSA

 Flight LOSA (If Flight LOSA, routed out of the questionnaire)

Section A. The following questions focus on the program that you were most recently involved with.

A1. What is the status of the most recent program? (required)

 Not yet launched (If Not yet launched, skip to item A4)

 Currently active (If Currently active, skip to item A4)

 Dormant (If Dormant, skip to item A4)

 Terminated

 Do not know (If Do not know, skip to item A4)

A2. Approximately how long did the most recent program last?

 6 months or less

 More than 6 months, but less than 1 year

 More than 1 year, but less than 3 years

 More than 3 years

 Do not know

A3. Why was the most recent program terminated? [mark all that apply]

 Lack of funding

 Insufficient manpower

 High workload demands by the organization

 Resistance from the workforce

 Lack of support from management

 Lack of computers for data entry

 Inadequate training

 Collected data unused

 Return-on-investment (ROI) not justified

 Other (specify below)

 Do not know
A-3

A3a. Other reason(s) the most recent program was terminated:

A4. Approximately how long ago was the most recent program introduced in your organization?

 6 months or less

 More than 6 months, but less than 1 year

 More than 1 year, but less than 3 years

 More than 3 years

 Do not know

A5. What is/was the progress of the most recent program in your organization? (required)

Completed Underway Not 

started

Do not 

know

Preparation - assess the organization's readiness for the 

program, provide awareness training and gain support from 

senior management, the labor union, and the workforce.

   

Development - organize a program team to coordinate the 

program, which includes building the program infrastructure, 

constructing the database, customizing and scheduling 

training, and distributing program information to the 

workforce.

   

Training - provide training to LOSA observers, reviewers, 

quality controllers, and database administrators.    

Implementation - collect data, and populate and maintain 

database.    

Reporting and Feedback - produce reports from collected data 

and provide the results to management and the workforce.    
A-4

Section B. The following questions address lessons learned during the Preparation stage which 

typically involves an assessment to determine the readiness of your organization for a LOSA 

program and obtaining support for the program. (Display section B if A5 Preparation = Completed OR Underway)

B1. Are/were the following considered in the readiness assessment of your organization’s most recent

program?

Not at 

all

Somewhat Moderately Very Completely Do not 

know

Projected annual cost      

Potential financial benefits      

Program performance metrics      

Required resources      

Potential safety benefits      

Success of other safety programs      

Familiarity with the Safety 

Management System (SMS) concept      

Existence of a “just culture”      

Other (specify below)      

B1a. Other information considered in assessing your organization’s readiness: (Display if B1 Other (specify 

below)=Somewhat, Moderately, Very, or Completely)

B2. Are/were there challenges/issues in assessing the readiness of your organization for the most 

recent program? (required)

 Yes (describe below)

 No (If No, skip to item B6)

 Do not know (If Do not know, skip to item B6)

B3. List challenges/issues in assessing organizational readiness:

B4. Did any attempts to overcome a challenge/issue in assessing organizational readiness fail? (required)

 Yes (describe below)

 No (If No, skip to item B6)

 Do not know (If No, skip to item B6)

B5. List examples of attempts that failed:
A-5

B6. Is/was support for the most recent program sought from the following groups in your organization? 

[mark all that apply] (required)

 Senior Management

 Union/Labor Group

 Maintenance/Ramp Workers

 Employee Groups

 Other (specify below)

 Do not know (If Do not know, skip to item B13)

B6a. Other groups that need/needed to support the most recent program: (If B6a is displayed, skip to item B13)

B7. How supportive initially were the following of the most recent program in your organization?

Not at all Somewhat Moderately Very Completely Do not 

know

Senior Management      

Union/Labor Group      

Maintenance/Ramp Workers      

Employee Groups      

B8. Did their level of support/acceptance change as the most recent program progressed in your 

organization?

Much 

less

Somewhat 

less

No 

change

Somewhat 

more

Much 

more

Do not 

know

Senior Management      

Union/Labor Group      

Maintenance/Ramp Workers      

Employee Groups      
A-6

B9. What type of information is/was required by senior management to decide on support for the most 

recent program in your organization? [mark all that apply] (Display if B6=senior management)

 Projected annual cost

 Financial benefits

 Safety benefits

 Program performance metrics

 Timing of implementation

 LOSA training schedule

 Required resources

 LOSA philosophy

 Union/labor group's buy in

 Expected return on investment (ROI)

 Regulatory requirements

 Other (specify below)

 Do not know

B9a. Other information required by senior management:

B10. Are/were there challenges/issues in gaining support from {selected choice B6} for the most 

recent program? (required) (Loop and Merge - Display if B6=senior management, union/labor group, maintenance/ramp 

workers, or employee groups)

 Yes (specify below)

 No (If No, skip to item B13)

 Do not know (If Do not know, skip to item B13)

B11. List challenges/issues in gaining support for the program:

B12. Did any attempts to overcome a challenge/issue in gaining support from {selected choice B6} fail? 

(required) (Loop and Merge - Display if B6=senior management, union/labor group, maintenance/ramp workers, or employee 

groups)

 Yes (describe below)

 No (If No, skip to item B13)

 Do not know (If Do not know, skip to item B13)

B12a. List examples of attempts that failed:
A-7

B13. Which method of introducing the most recent program to the workforce is/was in use in your 

organization? [mark all that apply]

 Posters

 Brochures

 Face-to-face meetings

 Company website

 Emails

 Memos

 Other (specify below)

 Do not know

B13a. Other method of introducing the most recent program:

Section C. The following questions address lessons learned during the Development stage of the 

LOSA program which typically involves organizing a program team and building the program 

infrastructure (i.e., database, training, and publicity of the program). (Display section C if A5 Development = 

Completed OR Underway)

C1. Did/does your organization have a team responsible for conducting and coordinating the most 

recent program processes? (required)

 Yes

 No (If No, skip to item C9)

C2. Are/were you a member of the most recent program team in your organization? (required)

 Yes

 No (If No, skip to item D)

C3. How did your organization select members for the most recent program team? [mark all that apply]

 Accepted volunteers

 Assigned by management

 Applied and qualified

 Nominated by the union

 From the department that oversees safety programs

 Other (specify below)

 Do not know

C3a. Other means used to select most recent program team members:
A-8

C4. Which personnel and departments in your organization are/were represented on the most 

recent program team? [mark all that apply]

 Senior management

 Frontline managers

 Safety personnel

 Training Department

 Quality Assurance Department

 Union/Labor Group

 Other (specify below)

 Do not know

C4a. Other personnel and departments represented on the most recent program team:

C5. Are/were there challenges/issues in organizing the program team? (required)

 Yes (describe below)

 No (If No, skip to item C9)

 Do not know (If Do not know, skip to item C9)

C6. List challenges/issues in organizing the program team:

C7. Did any attempts to overcome a challenge/issue in organizing the program team fail? (required)

 Yes (describe below)

 No (If No, skip to item C9)

 Do not know (If Do not know, skip to item C9)

C8. List examples of attempts that failed:

C9. With which program infrastructure task(s) are/were you involved? [mark all that apply] (required)

 Constructing the LOSA database

 Preparing LOSA training

 Publicizing the program information to the workforce

 Other (specify below) 

 None of the above (If None of the above, skip to Section D)

C9a. Other program infrastructure tasks you were involved in:
A-9

C10. In your organization, what resources were used as framework for the most recent program 

infrastructure i.e., database, forms, training, and publicity? [mark all that apply]

 Existing Flight, Ramp, or Maintenance LOSA program in your organization

 Existing Flight, Ramp, or Maintenance LOSA program in another organization

 FAA materials for Ramp LOSA

 FAA materials for Maintenance LOSA

 FAA Maintenance and Ramp LOSA database software

 Airline for America (A4A) consultations and materials

 Other (specify below)

C10a. Other sources that served as the framework for the most recent program infrastructure:

C11. Are/were there challenges/issues in {selected choice C9}? (required) (Loop and Merge - Display if 

C9=constructing the LOSA database, Preparing LOSA training, or Publicizing the program information to the workforce)

 Yes (describe below)

 No (If No, skip to item C14)

 Do not know (If Do not know, skip to item C14)

C12. List challenges/issues in {selected choice C9}: 

C13. Did any attempts to overcome a challenge/issue in {selected choice C9} fail? (required) (Loop and 

Merge - Display if C9=constructing the LOSA database, Preparing LOSA training, or Publicizing the program information to the 

workforce)

 Yes (describe below)

 No

 Do not know

C13a. List examples of attempts that failed:

C14. Approximately how long has/had the team worked on the most recent program in preparation for 

data collection?

 6 months or less

 More than 6 months, but less than 1 year

 More than 1 year, but less than 3 years

 More than 3 years

 Do not know
A-10

Section D. The following questions address lessons learned during the Training stage of the LOSA 

program which typically involves training LOSA observers, reviewers, quality controllers, and 

program administrators. (Display section D if A5 Training = Completed OR Underway)

D1. Is/was the FAA most recent training in use at your organization? (required)

 Yes

 No (If No, skip to item D4)

 Do not know (If Do not know, skip to item D4)

D2. Which of the following groups received FAA most recent training in your organization? [mark all 

that apply]

 Program managers

 Database administrators

 LOSA reviewers

 LOSA observers

 Quality controllers

 Senior management

 Maintenance/Ramp workers

 Union/Labor groups

 LOSA trainers

 Other (specify below)

 Do not know)

D2a. Other groups that received FAA most recent training:

D3. Which of the following FAA training modules are/were in use for the most recent program in your 

organization?

Used 'as 

is'

Modified for use Do not know

LOSA Maintenance Operations Base and Line   

LOSA Ramp Operations   

Threat and Error Management (TEM) and LOSA   

Considerations for Implementing Maintenance LOSA   

Maintenance Data Entry   

Ramp Data Entry   

Admin Data Entry   
A-11

D4. Is/was there other existing most recent training in use in your organization}? (required)

 Yes (describe below)

 No

 Do not know

D4a. Other existing most recent training in use in your organization:

D5. Is/was any new most recent training created in your organization? (required)

 Yes (describe below)

 No (If No, skip to item D7)

 Do not know (If Do not know, skip to item D7)

D5a. Other existing most recent training in use in your organization:

D6. Which of the following groups is/was the new most recent training created for in your organization? 

[mark all that apply] (required)

 Senior management

 Maintenance/Ramp workers

 LOSA trainers

 Database administrators

 LOSA observers

 LOSA reviewers

 LOSA quality controllers

 Other (specify below)

D6a. Other group(s) that new most recent training is/was created for: 

D7. Are/were there challenges/issues with most recent training in your organization? (required)

 Yes (describe below)

 No (If No, skip to Section E)

 Do not know (If Do not know, skip to Section E)

D8. List challenges/issues with most recent training:

D9. Did any attempts to overcome a challenge/issue with most recent training fail? (required)

 Yes (describe below)

 No

 Do not know
A-12

D9a. List examples of attempts that failed:

Section E. The following questions address lessons learned during the Implementation stage of the 

LOSA program which typically involves collecting data and populating and maintaining the 

database. (Display section E if A5 Implementation = Completed OR Underway)

E1. Is/was an implementation schedule developed for the most recent in your organization?

 Yes

 No (explain below)

 Do not know

E1a. Reason(s) why there is/was not an implementation schedule developed for the most recent program:

E2. What is/was the main focus for collecting most recent data in your organization?

 Entire operation

 Identified/known problems

 Specific types of tasks

 Specific aircraft types

 Specific facility

 Particular time of the day

 Following an operational change

 Other (specify below)

E2a. Other main focus for collecting most recent data:

E3. Are/were there challenges/issues with the focus of the most recent data collection in your 

organization? (required)

 Yes (describe below)

 No (If No, skip to item E6)

 Do not know (If Do not know, skip to item E6)

E4. List challenges/issues with the focus of the most recent data collection:

E5. Did any attempt to overcome a challenge/issue with the focus of the most recent data collection fail? 

(required)

 Yes (describe below)

 No

 Do not know

E5a. List examples of attempts that failed:
A-13

E6. What data collection format is/was in use by the most recent observers in your organization? 

[mark all that apply]

Used FAA form 'as is' Adapted FAA form Created new form Do not know

Paper    

Electronic    

E7. Approximately how many most recent observers are/were active in your organization?

[Enter a whole number. If you do not know, then enter DK.]

E8. How many most recent observations are/were conducted during the implementation period?

[Enter a whole number. If you do not know, then enter DK.]

E9. Are/were the most recent observers rotated in your organization? (required)

 Yes

 No (If No, skip to item E11)

 Do not know (If Do not know, skip to item E11)

E10. How often are/were the most recent observers rotated?

 Less than weekly

 Weekly

 Bi-weekly

 Monthly

 Other (specify below)

 Do not know

E10a. Other:

E11. How are/were the most recent observers selected? [mark all that apply]

 Volunteered

 Assigned by management

 Applied and qualified

 Nominated by the union

 From the department that oversees safety programs

 Other (specify below)

E11a. Other means of selecting the most recent observers:
A-14

E12. Are/were there challenges/issues with collecting most recent data in your organization? (required)

 Yes (describe below)

 No (If No, skip to item E15)

 Do not know (If Do not know, skip to item E15)

E13. List challenges/issues with collecting most recent data:

E14. Did any attempt to overcome a challenge/issue with collecting most recent data fail? (required)

 Yes (describe below)

 No

 Do not know

E14a. List examples of attempts that failed:

E15. Is/was the FAA most recent database software in use ‘as is’ (no modifications)?

 Yes (If Yes, skip to item E17)

 No

 Do not know

E16. Was the FAA most recent database software modified/customized for use?

 Yes

 No (If No, skip to Section F)

 Do not know (If Do not know, skip to Section F)

E17. Are/were there challenges/issues with populating the most recent database in your organization? 

(required)

 Yes (describe below)

 No (If No, skip to item E20)

 Do not know (If Do not know, skip to item E20)

E18. List challenges/issues with populating the database:

E19. Did any attempt to overcome a challenge/issue with populating the most recent database fail? 

(required)

 Yes (describe below)

 No (If No, skip to item E20)

 Do not know (If Do not know, skip to item E20)

E19a. List examples of attempts that failed:
A-15

E20. Are/were there challenges/issues with maintaining the most recent database in your organization? 

(required)

 Yes (describe below)

 No (If No, skip to Section F)

 Do not know (If Do not know, skip to Section F)

E21. List challenges/issues with maintaining the database:

E22. Did any attempt to overcome a challenge/issue with maintaining the most recent database fail? 

(required)

 Yes (describe below)

 No (If No, skip to Section F)

 Do not know (If Do not know, skip to Section F)

E22a. List examples of attempts that failed:

Section F. The following questions address the lessons learned during the Reporting and Feedback

stage of the LOSA program which typically involves compiling reports from the collected data and 

informing management and the workforce of the results. (Display section F if A5 Reporting and Feedback = 

Completed OR Underway)

F1. Does/Did your organization inform its workforce of the results from most recent data collections? 

(required)

 Yes

 No (If No, skip to item F4)

 Do not know (If Do not know, skip to item F4)

F2. What personnel or departments does/did your organization inform of the results for most recent data 

collections? [mark all that apply]

 Senior management

 Frontline managers

 Safety personnel

 Training Department

 Maintenance/Ramp workers

 Labor Union

 Other (specify below)

 Do not know

F2a. Other personnel or departments that are/were informed of the most recent results:
A-16

F3. What communication channel does/did your organization use to inform {selected choice F2} of 

the most recent results? [mark all that apply] (Loop and Merge - Display if F2=senior management, frontline managers, or 

maintenance/ramp workers)

 Face-to-face meeting

 Written report on overall findings

 Company letter

 Company website

 Newsletter

 Other (specify below)

 Do not know

F3a. Other communication channel(s) used to inform {selected choice F2} of the most recent results:

F4. Is/was the most recent database tool used to generate reports in your organization?

 Yes

 No (explain below)

 Do not know (If Do not know, skip to Section G)

F4a. Reason(s) the database tool is/was not used to generate reports: (If F4a is displayed, skip to Section G)

F5. Which of the following types of reports are/were frequently generated using the most recent database 

tool? [mark all that apply]

 General

 Summary

 Error Outcome

 Effectively Managed

 Demographics

 Other (specify below)

 Do not know

F5a. Other types of reports frequently generated:

Section G. Additional information about best practices and lessons learned during the process of 

establishing a LOSA program.

G1. Are there lessons learned that would contribute to a successful most recent program?

 Yes (describe below)

 No

G1a. List lessons learned from your involvement in the most recent program:
A-17

Section H. Background Information.

H1. What is/was your role in the most recent program? [mark all that apply]

 Program Manager

 Database Administrator

 Data Analyst

 Observer

 Senior management

 Frontline Manager

 Quality Assurance Representative

 Operations Representative

 Training Representative

 Safety Representative

 Labor Union Representative

 Employee Group Representative

 Other (specify below)

H1a. Other role in the most recent program:

H2. Are/were personnel external to your organization involved in any of the stages of the most recent

program?

Yes No

Preparation  

Development  

Training  

Implementation  

Reporting and Feedback  

H3. What type of organization is/was your organization?

 Passenger airline

 Cargo airline

 Maintenance repair

 Ground handling

 Other (specify below)

H3a. Other type of airline:
A-18

H4. Does/did your organization have other active safety programs?

 Yes

 No (skip to end of the block)

 Do not know

H5. Which other safety programs are/were active in your organization? [mark all that apply]

 Aviation Safety Action Program (ASAP)

 Aviation Safety Reporting System (ASRS)

 Continuing Analysis and Surveillance System (CASS)

 Maintenance Error Decision Aid (MEDA)

 Ramp Error Decision Aid (REDA)

 Other (specify below)

 None of the above

 Do not know

H5a. Other safety programs active in your organization: