Advisory 

Circular

 

Subject: LINE OPERATIONS SAFETY 

AUDITS 

Date: 4/27/06

Initiated by: AFS-230 

AC No: 120-90 

1. PURPOSE. This AC provides the rationale and procedure for conducting a Line Operations 

Safety Audit (LOSA) at an airline. 

2. APPLICABILITY. A LOSA is a voluntary safety program. While the Federal Aviation 

Administration (FAA) encourages airlines to voluntarily conduct LOSA programs in the interest 

of safety improvement, a LOSA does not entail any requirement for FAA approval, acceptance, 

or monitoring. While an airline may elect to share the results of a LOSA with the FAA, there is 

no requirement to do so. This AC is intended to provide informational guidance on LOSA to 

airline personnel in flight safety, flight training, flight operations, and other interested industry 

parties. 

3. CONTENTS. 

a. Introduction. This AC introduces the LOSA process and distinguishes it from other 

proactive safety data programs such as Flight Operational Quality Assurance (FOQA) and the 

Aviation Safety Action Program (ASAP). 

b. Data Collection. This AC also lists the various types of data collected via a LOSA and 

shows how they contribute to an airline’s safety management system. Next, the personnel 

involved in a LOSA and their respective roles are described. 

c. Implementation. This AC offers a detailed step-by-step guide to the LOSA process from 

initial planning to the final report. It concludes with the 10 operating characteristics that define a 

LOSA and which ensure the integrity of the process. Under the guidance of this AC, an airline 

wanting to implement a LOSA should abide by all 10 characteristics. The appendices in this AC 

describe the Threat and Error Management (TEM) model upon which the LOSA method was 

based (Appendix 1); provide more detail on the training and standardizing of LOSA observers 

(Appendix 2); provide a sample observation form (Appendix 3); a sample letter to the pilots 

(Appendix 4); and sample vignettes for training (Appendix 5). 
AC 120-90 4/27/06 

Page 2 Par 4 

4. DEFINITION OF A LOSA.

a. A LOSA is a formal process that requires expert and highly trained observers to ride the 

jumpseat during regularly scheduled flights to collect safety-related data on environmental 

conditions, operational complexity, and flightcrew performance. Confidential data collection 

and non-jeopardy assurance for pilots are fundamental to the process. 

b. Using a medical metaphor, a LOSA is similar to a patient’s annual physical examination. 

People have comprehensive checkups in the hope of detecting serious health issues before they 

become consequential. A set of diagnostic measures, such as blood pressure, cholesterol, and 

liver function, flag potential health concerns, which in turn suggest needed changes to the 

patient’s current lifestyle. A LOSA is built upon the same proactive notion. It provides a 

diagnostic snapshot of strengths and weaknesses that an airline can use to bolster the “health” of 

its safety margins and prevent degradation. 

c. LOSA is distinct from - but complementary to - other proactive safety programs such as 

electronic data acquisition systems (e.g., FOQA), and voluntary reporting systems (e.g., ASAP). 

However, these programs have two major conceptual differences. 

(1) First, FOQA and ASAP rely on outcomes to generate data. For FOQA, it is flight 

parameter exceedances; for ASAP, it is adverse events that crews report. By contrast, LOSA 

samples all activities in normal operations. In these regularly scheduled flights, there may be 

some reportable events, but there will also be some near-events and, importantly, a majority of 

well-managed, successful flights. LOSA provides a unique opportunity to study the flight 

management process, both successful and unsuccessful, by noting the problems crews encounter 

on the line and how they manage them. 

(2) The second major difference is the perspective taken by each program. With its focus 

on electronic data acquisition downloaded directly from the aircraft, FOQA can be said to have 

the “airplane perspective.” ASAP provides the “pilot perspective” by using pilots’ voluntary 

disclosure and self-reporting of events. ASAP reports provide insight into why events occur as 

seen from the crew’s perspective. By contrast, LOSA provides a “neutral, third-party 

perspective” in that LOSA observers record contextual and flightcrew data on every phase of 

flight, regardless of the outcome. All three perspectives provide useful data to an airline’s safety 

management system. 

(3) A third, more pragmatic difference between the programs relates to logistics. FOQA 

and ASAP are continuous programs; i.e., they are set up to collect data on a daily basis. A 

LOSA is more project-based. The full LOSA process, from advance planning and observer 

selection and training, to data collection, analyses and final report, can take between 6 and 

12 months. A LOSA is recommended every 3 years. Despite these differences, data from one 

program can be cross-referenced and used to guide data collection in another. For example, 

ASAP reports may highlight a problem with departures at a particular airport. This information 

can be fed to the LOSA steering committee, which can then target more observations out of that 

airport to understand the magnitude and specifics of the problem. As another example, a LOSA 

may identify a high incidence of unstable approaches, leading to a review of procedures and the 
4/27/06 AC 120-90 

Par 4 Page 3 

specification of new approach parameters. FOQA data can track adherence to the new 

specifications in the interim period leading up to the next LOSA. 

5. IMPORTANCE OF A LOSA. A LOSA provides unique data about an airline’s defenses 

and vulnerabilities. As explained in paragraph 4, a LOSA does not replace other safety-data 

sources such as FOQA or ASAP. Instead, it complements these programs and extends the reach 

of an airline’s safety management system. The data collected during a LOSA can impact almost 

every department in an airline, as the following examples show. The data collected during a 

LOSA can help an airline: 

a. Identify Threats in the Airline’s Operating Environment. Observers note events in the 

operational environment (e.g., adverse weather, airport conditions, air traffic control (ATC) 

clearances, terrain, and traffic congestion, and how they are managed by flightcrews). Highprevalence threats and/or threats with higher mismanagement rates can be prioritized for further 

investigation; lower mismanagement rates signify areas of strength. For example, understanding 

the extent to which certain airports or ATC practices pose a problem for flightcrews, and 

capturing the strategies flightcrews adopt to deal with them, can lead an airline to develop special 

procedures or advisories to help its pilots manage the known threat. 

b. Identify Threats from within the Airline’s Operations. Observers note events arising 

from within the airline’s own operations and how they are managed (e.g., operational time 

pressure, dispatch errors, aircraft malfunction/minimum equipment list (MEL) items, and 

problems with ground, ramp, maintenance, and cabin personnel). A high number of threats 

arising from dispatch or cabin might signal that these departments require attention, or that 

intergroup cooperation with pilots needs to be improved, or that procedures are inconsistent 

across departments. As above, prevalence and management rates provide cues for prioritizing 

action. 

c. Assess the Degree of Transference of Training to the Line. Data provided by 

Advanced Qualification Programs (AQP), Line Operation Evaluations (LOE), and LineOperational Flight Training (LOFT) can provide insight on whether training concepts are 

learned, but not whether they are actually practiced on the line. A LOSA provides that 

operational information, which can be reviewed from a training perspective to understand which 

areas of training, if any, are not transferring successfully to the line. 

d. Check the Quality and Usability of Procedures. A LOSA provides insights about 

potential problems with procedures. For example, if 5 percent of observed crews make a callout 

error during descent/approach/land, there may be a problem with those crews. However, if 

50 percent of observed crews make the same error, then the evidence suggests a problem with 

the callout procedure. Procedures can be ill-timed, over-long, confusing, and/or compete for the 

pilots’ attention with other more important activities. A LOSA will locate problematic 

procedures and policies via poor adherence rates. A LOSA can also identify the extent of 

procedural deviations across fleets. 
AC 120-90 4/27/06 

Page 4 Par 5 

e. Identify Design Problems in the Human/Machine Interface. A LOSA captures aircraft 

handling and automation errors on different fleets that can highlight systemic flaws in design, 

interface, or adaptation. The rate at which certain errors go undetected and become 

consequential can also indicate potential design vulnerabilities. An airline can feed these LOSA 

findings back to the aircraft manufacturers, as well as writing standard operating procedures 

(SOP) to circumvent the flaws. 

f. Understand Pilots’ Shortcuts and Workarounds. With experience comes expertise; 

pilots learn ways to save time and be more efficient. These techniques are rarely seen in a line 

check, when performance is usually done “by the book.” A LOSA provides a principled manner 

by which an airline can capture collective expertise from within the pilot group, and then share 

that information with all its pilots through formal airline communication channels. Using LOSA, 

false expertise — the adoption of a shortcut or workaround that is flawed in its safety 

assumptions — can also be identified and remedied. 

g. Assess Safety Margins. Threats and errors that are mismanaged can result in undesired 

aircraft states, if sufficiently serious. Vertical and lateral deviations and unstable approaches are 

examples of undesired aircraft states, also known as accident and incident precursors. A LOSA 

provides data about the prevalence and management of these incident and accident precursors. 

Thus, an airline acquires data about how close it is operating to the edge of the safety envelope, 

without crossing the boundary into an incident or accident. 

h. Provide a Baseline for Organizational Change. LOSA results provide baseline and 

outcome measurement data against which organizational interventions can be measured. Using 

the medical metaphor, this would be akin to the patient deciding to cut out fried foods upon 

learning of a high cholesterol count. The next checkup reveals, in quantifiable form, whether 

this strategy has been effective in reducing cholesterol or whether other actions are necessary. 

Similarly, a followup LOSA provides a new set of results that will show whether the 

organizational changes were effective in reducing certain threats, errors, and/or undesired states. 

i. Provide a Rationale for Allocation of Resources. Because LOSA results highlight both 

the strengths and weaknesses in an organization, the results provide a data-driven rationale for 

prioritizing and allocating scarce organizational resources toward interventions. 

6. INVOLVEMENT IN A LOSA. When first exploring whether or not to conduct a LOSA, it 

is advisable to gather representatives from all departments that may be potentially involved, 

including flight operations, training, flight standards, the safety department, and the pilot group. 

a. Departments. The flight operations and training departments typically know first-hand 

what is and is not working well. These departments often have specific areas that they would 

like the LOSA to focus on. Possibly the most important reason for their involvement is that, 

ultimately, many of the problem areas that are identified during a LOSA should be addressed by 

these departments. They will also be the recipients of the potential benefits derived from the 

LOSA. If these departments do not support LOSA, then there will be resistance to the findings; 

however, if these departments are part of the process, there will be a sense of ownership, and 

they will be invested in the results. 
4/27/06 AC 120-90 

Par 6 Page 5 

b. Pilots’ Association or Group. 

(1) The importance of having the pilots involved with and supporting the LOSA cannot 

be overstated. If the line pilots are convinced that their association or pilot group supports 

LOSA, they will be more willing to accept the presence of observers on their flight decks. 

Additionally, if pilots believe this process is beneficial to them and to safety, they will be 

forthcoming and candid with their views and safety concerns. On the other hand, if the pilots 

view LOSA as a management tool to “spy on their cockpits” and they respond with “angel” 

performance rather than typical performance, then the results will not be fruitful. 

(2) Hence, where airlines have a formal pilots’ association, leaders of the association 

should be involved in the LOSA process from the beginning. If no formal pilots’ association 

exists, pilot representatives should still be included. The pilots’ association or group can also 

help disseminate the results of the LOSA and inform the pilots as to the company’s plans as a 

result of the LOSA. 

c. LOSA Coordinator and Steering Committee. Because buy-in and support is crucial, 

consideration should be given to forming a LOSA steering committee drawn from the various 

departments and the pilots’ association. The LOSA steering committee and/or the LOSA 

coordinator have many tasks and logistical responsibilities, including: 

(1) Publicize the upcoming LOSA in pilot newsletters to build awareness. 

(2) Distribute a letter to all line pilots explaining the purpose of the LOSA (Appendix 3 

has a sample letter). 

(3) Decide the size and focus of the LOSA. 

(4) Select the observers, organize their schedules, and set up observer training. 

(5) Organize a secure site for collection of the observation forms and subsequent data 

analysis. 

NOTE: All of these tasks are discussed in more detail in the “How to” 

section of this document. 

d. Observers. 

(1) LOSA observers should be carefully selected to ensure the integrity of the LOSA 

process. LOSA observers should be familiar with the airline’s procedures and operations; the 

line pilots should also respect them. The observer team can include a small number of non-pilots 

as long as they can anticipate and understand flightcrew tasks and their surrounding operational 

context. However, the majority of the team should be active pilots from that airline.
AC 120-90 4/27/06 

Page 6 Par 6 

(2) In airlines that operate more than one fleet, observers should be scheduled to observe 

across fleets other than their own. This adds value to the process in that the observer looks more 

at the “big picture” rather than the fine detail. For similar reasons, experience has shown that 

using a small percentage of external observers drawn from pilots with LOSA experience at other 

airlines adds value in the form of a “control group” for observations. External observers—

typically 10 to 20 percent of the total observer team—will normally attend the airline LOSA 

training and will need to familiarize themselves with airline procedures. They are particularly 

useful in picking up systemic and organizational threats to which airline staff are often “blind” 

due to familiarity. 

(3) A LOSA observer should be like a “fly on the wall,” able to occupy the cockpit 

jumpseat and capture data without being obtrusive or interfering with the crew’s performance. 

This involves creating an environment where the crew almost does not realize they are being 

observed. LOSA observers will observe errors and undesired aircraft states as part of their 

observations; however, they should only interrupt if they perceive the safety of the flight to be 

seriously and immediately endangered. A helpful rule of thumb is to ask observers to think of 

themselves as a guest riding in the jumpseat of another airline. This seems to help distance the 

observer from the flightcrew while still being able to politely point out safety concerns if they 

come about. 

(4) LOSA observers should be scheduled in the status of “passenger,” “supernumerary,” 

or “observer” only (i.e., they should not be scheduled as a member of the legal operating crew). 

This assists both the crew and the observer to understand that LOSA observers are data 

collectors only, not evaluators there to critique crews. Observers do not have any responsibility 

for the operation. 

e. Line Pilots. A LOSA cannot succeed without the full and candid cooperation of the line 

pilots, and there can be no cooperation without trust. Line pilots should be informed in advance 

about the purpose and planned implementation of a LOSA. They should receive a letter cosigned by credible representatives of both management and the pilot organization that assures 

them of the confidential and nonjeopardy status of LOSA data. The letter should also include a 

disclaimer giving all pilots the choice of declining a jumpseat observer at their discretion. Only 

by building in these guarantees and safeguards will the line pilots feel sufficiently comfortable to 

act normally in the cockpit in the presence of a LOSA observer. A final assurance should be an 

inhouse publication of a summary of LOSA results along with an outline of initial actions and 

proposed changes. 

f. Data Analyst and Report Writer. The data analyst should have knowledge of the 

airline’s flight operations as well as database management and data analysis skills. However, an 

airline might choose a third-party analyst if expertise is not available inhouse or if line pilots 

have expressed reservations about the integrity of the LOSA implementation or objectivity of the 

final report. The data analyst and report writer work together to prepare a report of the findings 

to be presented to management and pilots. 
4/27/06 AC 120-90 

Par 7 Page 7 

7. WHEN TO CONDUCT A LOSA. 

a. There are several factors to consider when scheduling a LOSA. Given all the personnel 

involved, a LOSA should be scheduled to fit with other operational priorities. For example, is 

there a particular time in the year when more observers will be available? Is there a better time 

for the scheduling department to roster these people? Also, is there a particular time that is more 

interesting from a safety or operational perspective? Some examples: bad-weather season, peak 

traffic season, after the introduction of an operational change such as new aircraft, altered routes, 

or a merger. 

b. A LOSA should not be implemented immediately after a major incident or accident. The 

airline will be in a heightened state of awareness at this time and pilots will be overly sensitive to 

observation; hence, the chances of getting normal data will be diminished. At a minimum, 

airlines should wait at least 1 year after a major safety event before scheduling a LOSA. Once 

an airline has done a LOSA, a critical question is when to do the next one. LOSA data provide a 

baseline against which to measure improvements. A realistic timeframe to review LOSA results, 

develop and implement action plans, and monitor results is 3 years. Hence, to measure the 

effectiveness of organizational changes, a repeat LOSA every 3 years is recommended. 

8. HOW TO IMPLEMENT A LOSA. This section presents a step-by-step guide to 

implementing a LOSA. Broadly speaking, there are steps associated with getting good-quality 

data from observers (data collection), and steps associated with ensuring that accurate and 

meaningful data are given to management and line pilots (data analysis and feedback). An 

airline can conduct its own LOSA by observing the following steps: 

a. Data Collection. 

(1) Form a LOSA steering committee and appoint a LOSA coordinator. 

(2) Gather Information and LOSA Resources from other Airlines and Industry 

Groups. Before conducting a LOSA for the first time, the committee and coordinator should 

seek out information from other airlines that have already conducted a LOSA. Other airlines 

may be able to share observer selection and training techniques, observation forms, scheduling 

tips, and other logistical aids. 

(3) Publicize LOSA within the Airline and Send a Letter to the Line Pilots. A first 

task is advance publicity via company publications to build line pilot awareness and acceptance 

of the upcoming LOSA. Next, the coordinator organizes and distributes a letter to all pilots 

explaining the purpose of the LOSA. This letter specifies the purpose of the audit, the fact that 

all observations are of a non-jeopardy nature, and that all data will be kept strictly confidential. 

The letter is signed by the highest level of management within flight operations, with the 

endorsement of other relevant personnel such as chief pilots and pilots’ representatives 

(Appendix 4 has an example letter). The letter of announcement should precede the line audit by 

at least 1 month, with a followup alert 1 week before starting observations. LOSA observers 

should have copies of the signed letter to show crewmembers in case questions arise. 
AC 120-90 4/27/06 

Page 8 Par 8 

(4) Decide the Focus of the LOSA. The LOSA steering committee decides the focus of 

LOSA. One option is to sample broadly across the entire operation—this would be an effective 

strategy for a first LOSA. Alternately, the LOSA steering committee can focus on problems that 

have been identified by other data sources, such as FOQA and ASAP. This approach would 

schedule LOSA observations on particular routes, in certain regions, or into particular airports 

that have been identified as problematic. The committee can also focus a LOSA on a new fleet 

or other recent organizational changes. 

(5) Decide the Number of Observations. 

(a) Most airlines will find it cost effective to conduct a LOSA on a sample of their 

operation—the question is how big a sample? As a general guideline for a full LOSA, match the 

number of observations per fleet to the relative number of departures per day. For example, if 

30 percent of departures occur on Fleet A, then approximately 30 percent of the LOSA 

observations should occur on Fleet A. Within each fleet, try to sample as many different crews 

as possible and, as a rule, conduct 50 or more observations per fleet. Below that number, there is 

the risk of not accurately capturing a representative sample. 

(b) Modify the guideline slightly when focusing on a particular operation or region. 

For example, to sample international flights into a particular subcontinent, then regardless of 

what percentage they constitute of the airline’s daily departures, still schedule at least 

50 observations to ensure a good sample. 

(6) Create an Observation Form. The observation form should be based on a 

conceptual framework that captures multiple aspects of normal operations, including the 

operating environment and flightcrew performance. It should provide categories and codes to 

streamline observations and save the observer’s time, but it should also require a written 

description of the flight that captures the full context. Appendix 3 has an example of an 

observation form based upon the TEM model. 

(7) Select observers. 

(a) The observer team should have representatives from flight operations, training, 

safety, and the pilots’ association. Some airlines employ a selection procedure whereby 

management and the pilots’ association each put forth a list of acceptable observers, and then 

those who appear on both lists are selected. 

(b) The number of observers needed depends on the size of the audit and the 

observers’ workload. There is substantial work involved in completing an observation form and 

providing a detail-rich narrative for each flight; therefore, the recommended number of 

observations is 10 to 15 per observer, depending on routes and schedules. Hence, a LOSA that 

plans 150 domestic observations requires at least 10 observers, while a 300-observation LOSA 

that includes international flights requires 20 to 25 observers. 
4/27/06 AC 120-90 

Par 8 Page 9 

(8) Train observers. 

(a) LOSA observers should be educated about the purpose and rationale of LOSA, 

and trained in the use of the observation form. LOSA observer training typically takes 2 to 

3 days. To assist in the design of the training, members of the steering committee may want to 

attend LOSA observer training at another airline first or, if possible, attend an industrysponsored LOSA conference for guidance. 

(b) Observers should practice with scripted scenarios or videos until they are 

confident they can use the observation form correctly. At this point, they can be dispatched to 

the line; however, it is recommended that observers be brought back in after one or two flights to 

discuss their observations, correct any misperceptions, and coach them on areas that require 

clarification. Appendix 2 provides more detail on the objectives and content of observer 

training. Appendix 2 also addresses the standardization of observers and LOSA data. 

(9) Schedule Observations. Plan no more than two observations per observer per day to 

allow sufficient time to complete the observation form and write a rich narrative. Schedule 

observers across fleets regardless of their type rating to encourage a more general, cross-fleet 

perspective of flightcrew performance. Build some flexibility into the schedule to allow for the 

unexpected. Finally, do not let the observations continue indefinitely—schedule all observations 

within a 1- to 3-month period if possible, or else the impact of LOSA will be lost. 

(10) Decide on a Data Repository. 

(a) The LOSA coordinator organizes a secure site for the data and oversees the 

receipt of the observation forms. The coordinator should be able to protect the identity of the 

observers and the observed to ensure complete confidentiality and nonjeopardy conditions. 

Under no circumstances should it be possible to connect individuals with particular observations. 

(b) The observations can be kept inhouse if data management and analysis expertise 

is available and if data security can be assured. Alternately, the data can be sent to a trusted third 

party who will assume responsibility for data collection, cleaning, and analysis. The decision 

will depend on airline resources and pilot trust issues. 

(11) Provide Logistical Support. Give the observers the name of a contact person, 

most likely the LOSA coordinator, who can be reached if there are any problems with scheduling 

or data collection. 

b. Data Analysis and Feedback. 

(1) Verify the Data. Convene a meeting of “local experts”—airline personnel familiar 

with the operation of each fleet (possibly fleet managers or member of the steering committee, 

but not any of the observers). The group’s task is to review and verify the observations against 

current manuals, policies, and procedures. For example, an observer might log a procedural 

error for failure to make an approach callout when in fact there is no written procedure in the 

airline’s flight operations manual. The data verification group would delete this particular 
AC 120-90 4/27/06 

Page 10 Par 8 

“error” from the database. This step is a data integrity check in that it ensures that events are 

correctly recorded in line with each fleet’s procedures and policies. It also builds ownership in 

the results and dispels any later criticism that the coding was not an accurate representation of 

the airline’s operations. 

(2) Analyze Data. 

(a) LOSA data reveal strengths and vulnerabilities in an airline’s operations. The 

data analyst should investigate the prevalence and management of different events and errors. 

Although certain types of comparisons will seem obvious, many analyses can and should be 

based upon hunches and theories derived from local knowledge of operations. If the analyst 

knows how fleets and operations are managed, comparisons that reflect this structure can be 

made. If the analyst knows the kinds of information that might be useful to training, safety, or to 

domestic or international flight operations, results can be tailored to these particular aspects of 

the operation. Feedback from various airline stakeholders is critical during this stage of 

preparing the report. The analyst should not hesitate to distribute early drafts to key people 

within the airline familiar with LOSA to crossverify the results. This not only helps validate 

derived trends, but it gives other airline personnel ownership of the report. 

(b) Patterns will emerge as the data are analyzed. Certain errors occur more 

frequently than others, certain airports or events emerge as more problematic than others, certain 

SOPs are routinely ignored or modified, and certain maneuvers pose greater difficulty for 

adherence than others. These events and practices form the basis of suggested targets for 

enhancement. 

(3) Prepare Report. The last stage of a LOSA is a written report that presents the 

overall findings of the audit. With a large database like the one generated from a LOSA, it is 

easy to fall into the trap of trying to present everything. The report should be concise and 

present only the most significant trends from the data. Along with the results, the report should 

provide an initial list of targets for enhancement. Targets need to be action-focused and datadriven. Some example targets that might emerge from a LOSA include: 

(a) Reduce the number of unstabilized approaches. 

(b) Streamline predeparture checklists. 

(c) Reduce SOP crossverification errors. 

(d) Understand automation errors on the new fleet. 

(e) Investigate conditions at airports X and Y. 

(f) Improve management of adverse weather threats. 

(g) Investigate high rate of MEL items on the ZZ fleet. 
4/27/06 AC 120-90 

Par 8 Page 11 

(h) Reduce dispatch errors at the hub. 

(i) Develop an international flight operations guide. 

(j) Develop a module on intentional noncompliance errors for captain upgrade 

training. 

(4) Brief Management. 

(a) The LOSA report should be presented to management in operations, training, 

standards, safety, and possibly other departments depending on the results. For example, 

representatives from ramp, maintenance, dispatch, and cabin may want to hear how their work is 

perceived from the pilots’ perspective, particularly if it is problematic. A briefing to the pilots’ 

association or group, as applicable, is also recommended. 

(b) Once the various departments are briefed on the report, they will likely want to 

investigate the data more deeply themselves. The data should be available in aggregated form 

for them to review. Some flight narratives will also be of interest, hence the prerequisite 

insistence on deidentifying the observations. 

(5) Brief Line Pilots. 

(a) Line pilots should also be informed of the significant results in the LOSA report. 

To sustain the pilots’ interest in the LOSA project, make an announcement at the end of the data 

collection phase that the LOSA observations have been completed, stating how many and on 

what fleets, and advise when the pilots can expect to see the results. 

(b) When the report is ready, the highlights should be presented to the pilots, either 

as one LOSA debriefing event or spread over time in the airline newsletter or other safety 

periodical. Pilots will want to know what changes will be undertaken as a result of the LOSA. 

(6) Monitor Safety Change Process. 

(a) Historically, organizational safety changes within airlines have been driven by 

accident/incident investigation and intuition. Today, airlines must deal proactively with accident 

and incident precursors. To be successful, the safety change process must be data-driven. 

Measurement of daily operations is fundamental because unless an organization uses systematic 

measurement, the perspective it has on the strengths and weaknesses of its operations is largely 

based on anecdote and opinion. 

(b) A LOSA provides specific and quantified results. To take full advantage of this 

specificity, the targets for enhancement that arise from the data analysis should go through a 

formal safety change process to produce improvement. A formal safety change process provides 

a principled approach to target limited resources and helps the airline avoid “turf” issues by 

clearly defining and prioritizing the issues that impact flight operations. The basic steps of a 

safety change process are: 
AC 120-90 4/27/06 

Page 12 Par 8 

1 Measurement (with LOSA) to obtain the targets. 

2 Detailed analysis of targeted issues. 

3 List of potential changes for improvement. 

4 Risk analysis and prioritization of changes. 

5 Selection and funding of changes. 

6 Implementation of changes. 

7 Time for changes to stabilize. 

8 Remeasurement. 

9. USE OF LOSA DATA.

a. A well-conducted and well-analyzed LOSA identifies strengths and vulnerabilities in an 

airline’s operations. It provides this information in a quantifiable form against which targets can 

be specified and improvements can be measured. The following example briefly illustrates the 

step-by-step integration of LOSA data into the safety change process. 

(1) An airline’s LOSA results indicate that 16 percent of observed flights involved an 

unstable approach. Because observations were scheduled across the operation, and the number 

of observations exceeded 50 per fleet, the LOSA committee is confident that the percentage is an 

accurate representation of operations as a whole. 

(2) Following management briefings and extensive discussion, a specific target for 

improvement is created to “reduce the number of unstabilized approaches by 50 percent; i.e., 

reduce the number of unstable approaches from 16 percent to 8 percent or fewer of all landings.” 

(3) An action committee is formed for unstabilized approaches. They formalize the 

parameters and definition of an unstable approach, they review existing procedures and training, 

and they introduce changes in all relevant areas. 

(4) A repeat LOSA is conducted 3 years after the first LOSA. The data, once aggregated 

and analyzed, show the new rate of unstable approaches to be 12 percent. The airline concludes 

that changes made to the operation were successful in reducing the rate of unstabilized 

approaches from 16 percent to 12 percent, an improvement of 25 percent. Upon reviewing the 

results of the second LOSA, the airline recommits to its original target of reducing the unstable 

approach rate to 8 percent or lower, and continues to focus efforts in this area. 

b. Depending on the sophistication of an airline’s safety management system, and the extent 

to which different safety programs within the airline are premised on the same conceptual 
4/27/06 AC 120-90 

Par 9 Page 13 

framework, data from a LOSA can be cross-referenced with data from the ASAP and/or FOQA 

programs, if the airline operates one or both of those voluntary programs. Each data source 

provides unique yet complementary evidence of the airline’s safety status. In the above example, 

the airline might track unstable approaches through its FOQA program using new flight 

parameters decided by the action committee and then implemented into procedures and training. 

To see if pilots are incurring problems with the new procedure, the FOQA aircraft data can be 

cross-referenced with ASAP reports of events resulting from unstable approaches. This way, the 

airline does not have to wait until the next LOSA to learn if its interventions are being 

successful. 

c. LOSA data are useful in another way. LOSA presents a broad view of operations; a 

repeat LOSA can maintain that broad focus. For example, did the changes that were introduced 

after the first LOSA improve results in one area, only to cause problems in another? Checklist 

adherence may have improved, but did error detection—the superordinate goal of improving 

checklist adherence—actually improve or is the new adherence simply cosmetic? 

10. SUMMARY: THE 10 OPERATING CHARACTERISTICS OF LOSA. 

a. Ten operating characteristics define and summarize the LOSA process. It is 

recommended that a LOSA observe all 10 characteristics to ensure the integrity of the LOSA 

process and the quality of the final product. 

b. The 10 Operating Characteristics Are: 

(1) Jumpseat Observations During Normal Flight Operations. LOSA observations 

are limited to regularly scheduled flights. Line checks, initial line indoctrination or other training 

flights are off-limits due to the extra level of stress put on pilots during this type of situation. 

Having another observer onboard adds to an already high stress level, thus providing an 

unrealistic picture of performance. In order for the data to be representative of normal 

operations, LOSA observations should be collected on regular and routine flights. 

(2) Joint Management/Pilots’ Association Sponsorship. In order for LOSA to 

succeed as a viable safety project, there needs to be support not only from the management side 

but also from the pilots. The joint sponsorship provides a “check and balance” for the project to 

ensure that change, as necessary, will be made as a result of LOSA data. When considering 

whether to conduct a LOSA, the first question to be asked by airline management is whether the 

pilots’ association (or pilot group representatives) endorses the project. If the answer is “No,” 

the project should not be initiated until endorsement is obtained. 

(3) Voluntary Crew Participation. Maintaining the integrity of LOSA within an airline 

and the industry as a whole is extremely important for long-term success. To accomplish this 

goal, all LOSA observations are collected with voluntary crew participation. Before conducting 

LOSA observations, observers should first ask the flightcrew for permission to be observed. If 

the crew declines, the observer takes another flight with no questions asked. If an airline 

conducting a LOSA has an unreasonably high number of declines, this should serve as an 

indicator that there are critical “trust” issues to be resolved.
AC 120-90 4/27/06 

Page 14 Par 10 

(4) Deidentified, Confidential, and Nondisciplinary Data Collection. LOSA 

observers are required to not record names, flight numbers, dates, or any other information that 

can identify a crew or individual. The purpose of LOSA is to collect safety data, not to punish 

pilots. Airlines cannot allow themselves to squander a unique opportunity to gain insight into 

their operations by having pilots fearful that a LOSA observation could be used against them for 

disciplinary reasons. If a LOSA observation is ever used for disciplinary reasons, the credibility 

of the entire safety program may be irreparably compromised. 

(5) Targeted Observation Form. The LOSA observation form in Appendix 3 is 

predicated on the TEM framework described in Appendix 1. However, other conceptual 

frameworks can be used for LOSA data collection. Whatever framework is used, it should 

generate meaningful data on a variety of topics, including what the crews did well, what they did 

poorly, and how they managed each phase of flight. A narrative written by the observer should 

have sufficient detail to allow others to understand the flight and all its events. The observers 

need to describe the environmental conditions and events surrounding the pilots’ behavior so that 

the crews’ performance can be understood in full context. 

(6) Trained and Calibrated Observers. Primarily, pilots conduct LOSA. Observation 

teams will typically include line pilots, instructor pilots, safety pilots, management pilots, and 

representatives of the pilots’ safety committee. It is critical to select observers that are respected 

and trusted within the airline to ensure line acceptance of LOSA. After observers are selected, 

they are trained and calibrated in the LOSA methodology, including the use of the LOSA 

observation form. Observers’ training in the concepts and methodology of LOSA will ensure 

that the observations will be conducted in the most standardized manner (see Appendix 2). 

(7) Trusted Data Repository. In order to maintain confidentiality, airlines should have 

a trusted data repository. This site can be in-house, such as that used for other confidential data, 

or it can be offsite. The goal is that no individual observations will be misplaced or improperly 

disseminated through the airline. 

(8) Data Verification. Data-driven programs like LOSA require quality data 

management procedures and consistency checks. For LOSA, these checks are done at dataverification roundtables. A roundtable consists of three or four department and pilots’ 

association representatives who review all the raw data for possible inaccuracies. The end 

product is a database that is validated for consistency and accuracy according to the airline’s 

standards and manuals before any statistical analysis is performed. 

(9) Targets for Enhancement. The final product of a LOSA is the data-derived targets 

for enhancement based on emergent patterns in the data. It is then up to the airline to develop an 

action plan based on these targets, using experts from within the airline to analyze the targets and 

implement appropriate change strategies. 
4/27/06 AC 120-90 

Par 10 Page 15 (and 16) 

(10) Feedback Results to the Line Pilots. In order to ensure long-term success of 

LOSA, airlines should communicate the results back to the line pilots. Pilots will want to see not 

only the results of the audit, but also management’s plan for improvement.

ORIGINAL SIGNED BY 

CAROL E.GILES (for) 

James J. Ballough 

Director, Flight Standards Service 

4/27/06 AC 120-90 

 Appendix 1 

 Page 1 

APPENDIX 1. THREAT AND ERROR MANAGEMENT 

1. Introduction. 

The Threat and Error Management (TEM) model is a conceptual framework for understanding 

operational performance in complex environments. Originally created to capture the flightcrew’s 

task in commercial aviation, the model is generic and can be applied to numerous work 

situations. The added value that TEM brings to other performance models is that it focuses 

simultaneously on the operating environment and the humans working in that environment. 

Because the model captures ongoing performance in its “natural” or normal operating context, 

the resulting description is realistic, dynamic, and holistic. Because the model can also quantify 

the specifics of the environment and the effectiveness of performance in that environment, it is 

also highly diagnostic1

. 

There are several ways of using the TEM model, from focusing on a single event (as is the case 

with accident/incident analysis) to understanding systemic patterns in a large set of events (as 

with Line Operations Safety Audits (LOSA)). As a training tool, TEM can help individuals 

clarify their performance needs and vulnerabilities; and as part of a safety management system, 

TEM can help an organization measure and improve the effectiveness of its organizational 

defenses and safeguards. 

2. The Model. 

This section defines and provides examples of the various components of the TEM model. 

a. Threats. 

A threat is defined as an event or error that occurs outside the influence of the flightcrew 

(i.e., it was not caused by the crew), increases the operational complexity of a flight, and requires 

crew attention and management if safety margins are to be maintained. 

There are threats from the environment—adverse weather, airport conditions, terrain, traffic, 

and air traffic control (ATC)—and threats emanating from within the airline—aircraft 

malfunctions and master equipment list (MEL) items, problems, interruptions, or errors from 

dispatch, cabin, ground, maintenance, and the ramp. The crew, for example, may anticipate 

threats by briefing a thunderstorm in advance; or they may be unexpected, occurring suddenly 

and without warning such as in flight aircraft malfunctions. Some threats are easily resolved and 

quickly dismissed from the crew’s workload, while other threats require greater attention and 

management. A mismanaged threat is defined as a threat that is linked to or induces flightcrew 

error. 

b. Errors. 

 

1

 The rest of this appendix will talk about TEM exclusively in the aircraft piloting environment so as to provide 

specific examples and situations. However, it is important to remember that the TEM framework can be applied to 

other high-risk jobs. To do so requires task and situational analyses by subject matter experts to develop the 

relevant threat and error taxonomies. 
AC 120-90 4/27/06 

Appendix 1 

Page 2 

Crew error is defined as action or inaction that leads to a deviation from crew or 

organizational intentions or expectations. Errors in the operational context tend to reduce the 

margin of safety and increase the probability of adverse events. 

Broadly speaking, there are handling errors (flight controls, automation), procedural errors 

(checklists, briefings, callouts) and communication errors (with ATC, ground, or pilot-to-pilot). 

See the error management worksheet in the sample observation form, Appendix 3, for a more 

complete list of errors. 

Understanding how the error was managed is as important, if not more important, than 

understanding the prevalence of different types of error. It is of interest then if and when the 

error was detected and by whom, as well as the response(s) upon detecting the error, and the 

outcome of the error. As with threats, some errors are quickly detected and resolved, leading to 

an inconsequential outcome, while others go undetected or are mismanaged. A mismanaged 

error is defined as an error that is linked to or induces additional error or an undesired aircraft 

state. 

c. Undesired Aircraft States. 

An UAS is defined as a position, condition, or attitude of an aircraft that clearly reduces 

safety margins and is a result of actions by the flightcrew. It is a safety-compromising state that 

results from ineffective error management. Examples include unstable approaches, lateral 

deviations, firm landings, and proceeding towards wrong taxiway/runway (more examples are 

noted on the observation form in Appendix 3). Events such as equipment malfunctions or ATC 

command errors can also place the aircraft in a compromised position, but these would be 

considered threats. 

As with errors, UASs can be managed effectively, returning the aircraft to safe flight, or the 

crew action or inaction can induce an additional error, incident, or accident. 

d. Threat and Error Countermeasures. 

A description of a flight is not complete without noting what the crew was doing to anticipate 

threats and avoid errors, as well as managing those that occurred. The following crew behaviors 

are considered threat and error countermeasures: 

• Planning countermeasures—planning, preparation, briefings, contingency management—

are essential for managing anticipated and unexpected threats 

• Execution countermeasures—monitor/cross-check, taxiway/runway management, 

workload, and automation management—are essential for error detection and error 

response 

• Review/modify countermeasures—evaluation of plans, inquiry—are essential for 

managing the changing conditions of a flight 
4/27/06 AC 120-90 

 Appendix 1 

 Page 3 

In addition to crew behaviors, TEM countermeasures also include equipment and procedural 

countermeasures. Warning systems such as ground proximity warning systems (GPWS) and 

weather alerts can be considered threat countermeasures, just as checklists and well-written 

procedures provide the means for error avoidance and error detection. 

In sum, the TEM model captures the dynamic activity that is a flightcrew planning and 

executing a flight in real time and under real conditions. The use of the model is that it can be 

applied proactively or reactively, at the individual, organizational, and/or systemic levels. 

3. Practical Applications of the TEM Model. 

a. TEM as a Training Tool. 

TEM has been found to be an effective tool for human factors training programs at a number 

of airlines throughout the world2. TEM training emphasizes the value of threat anticipation and 

management, error avoidance, and error detection and recovery. The model allows pilots to 

analyze their own performance strengths and vulnerabilities. 

TEM concepts can be trained effectively in the classroom in the absence of LOSA. 

However, TEM training can be enhanced if an airline has also conducted a LOSA. The LOSA 

results can help shape the training curriculum, and pilots can discuss the findings during training. 

Pilots are always interested in and respond well to data derived from their own operation. 

It is important to clarify that TEM is not crew resource management (CRM) and should not 

be considered a replacement for it. TEM and CRM refer to overlapping but not equivalent 

activities. CRM refers specifically to activities conducted by the crew to optimize performance. 

These activities include threat and error countermeasures such as briefing, contingency planning, 

and monitor/cross-checking, but they also include higher-order concepts such as leadership and 

establishing open communication in the cockpit. Similarly, TEM includes crew 

countermeasures, but it also encompasses equipment, procedural, and regulatory 

countermeasures. 

As a training tool, TEM can help individuals clarify their performance needs and 

vulnerabilities from a different perspective. Hence, threat and error management concepts could 

be introduced and explored as one component of CRM training. 

b. TEM as a Reporting Tool for Incidents. 

Reporting forms structured to the TEM framework instruct the pilots to describe the event at 

the level of threats and errors. The TEM format prompts pilots to report information about the 

threats that were present, the errors they may have made, how well the event was managed, and 

how the event may have been avoided or handled better. Preliminary work has shown that even 

pilots who have not had training in the TEM model are able to complete the reporting form, a 

fact that speaks to the intuitive nature of the TEM framework. 

 

2

 The International Civil Aviation Organization (ICAO) has adopted the TEM model in its Human Factors Training 

Manual (ICAO Document 9683, 2002) 
AC 120-90 4/27/06 

Appendix 1 

Page 4 

In the ASAP environment, TEM can “go inside the pilot’s head” in a way that LOSA as an 

objective observational tool cannot do and most assuredly does not want to do. With ASAP, 

pilots can report personal or historical factors that contributed to the event – information that is 

not privy to an observer. With LOSA, the benefit is that observers may detect threats and errors 

that the crews themselves do not detect. This is one example of how LOSA and ASAP data can 

complement each other at the system level. 

c. TEM as a Systematic Observation Tool. 

The TEM model was first conceived in conjunction with the development of LOSA; hence, 

its original application was as an observation tool. As of the publication of this advisory circular 

(AC), feasibility studies are currently underway to explore the transfer of the methodology to 

airline flight dispatch and air traffic control (ATC). In 2002, one major airline began exploring 

an adapted version of LOSA called Dispatch Operations Safety Audit (DOSA). Early results 

demonstrated that such a transfer of methodology is possible and could ultimately provide a 

360-degree perspective on the interaction between pilots and dispatchers. In addition, ICAO has 

instituted a formal group of ATC subject matter experts from across the world to develop the 

Normal Operations Safety Survey (NOSS), a formal protocol to observe normal operations in 

ATC, based on the TEM model and LOSA methodology. 

d. TEM as a Reactive Analysis Tool for Accidents and Incidents. 

TEM can be used as an analysis tool to understand rare events, such as accidents and serious 

incidents. For example, the International Air Transport Association (IATA) Safety Committee 

has adopted the TEM model as an analysis framework for its incident review meetings, based on 

its ease of use and utility of the extracted data. 

e. TEM as a Proactive Analysis Tool. 

When TEM is used as the framework for safety data collection, a wealth of information can 

be extracted. An airline can use the data to understand patterns at the organizational level. The 

data can also be collected across the industry and analyzed for systemic trends. 

An analysis based on TEM can: 

• Quantify those aspects of the working environment that can pose a problem for the 

efficiency or safety of the operation (threat prevalence) 

• Quantify the management of those threats as either effective or ineffective (threat 

management)
4/27/06 AC 120-90 

 Appendix 1 

 Page 5 (and 6) 

• Recognize high rates of threat prevalence and mismanagement as systemic vulnerabilities 

• Codify and quantify the errors that crews commit (error prevalence) 

• Codify and quantify the error management process from diagnosis to response and 

outcome (error management) 

• Recognize high rates of error prevalence and error mismanagement as systemic flaws in 

procedures, policies, training, aircraft design, and or interagency coordination, and 

• Locate strengths as well as vulnerabilities in organizational safeguards 

4. Conclusion.

The TEM model is intuitive, practical, and versatile. More and more airlines are realizing the use 

of TEM, as exemplified in the following quote from an airline manager: 

“Since our LOSA, we have introduced a 1-day TEM course for our pilots. The 

overwhelming response has been positive, and there’s a recognition among pilots 

that in TEM, the academic community is using language we understand and feel. 

Additionally we are using TEM as a debriefing tool for training events. Once 

again this makes sense to the pilots, and the common reaction is one of dawning; 

penny-dropping; eureka!” 

4/27/06 AC 120-90 

 Appendix 2 

 Page 1 

APPENDIX 2. TRAINING AND STANDARDIZING LOSA OBSERVERS 

This appendix details the training and standardizing of Line Operations Safety Audits (LOSA) 

observers. Some of this material appears throughout the advisory circular (AC); this appendix 

draws that information together and provides more in-depth information. 

1. Observer Training. 

Observer training typically requires 2 to 3 days of classroom training with a followup session 

after one or two line observations. There are five topics that need to be covered in observer 

training: 

• LOSA rationale and etiquette 

• Company policies and procedures 

• Observation form 

• Threat and Error Management (TEM) concepts 

• Narratives

a. LOSA Rationale and Etiquette. 

Observers will likely have a rudimentary understanding of LOSA when selected for the 

project; however, they will need to fully understand the safety rationale for conducting a LOSA 

at their airline. A “big picture” perspective will help observers understand the “why” of LOSA 

and will underscore the importance of their role in the LOSA process. Also, the observers will 

be ambassadors for LOSA while observing on the line, and it is important that they be able to 

explain the process fully, to answer any questions that the line pilots may have, and allay any 

fears or concerns. 

Specifically, the observers need to understand the safety rationale for normal operations 

monitoring—a discussion of proactive vs. reactive safety strategies is recommended. The 

observers also need to know how the data collected from the LOSA will be used to understand 

strengths and weaknesses in the operations. An overview of the whole process from 

observations to data cleaning and analysis, to the diagnostic report and the development of 

targets for enhancement is recommended. 

Of course, the observers will also need to know the “how” of LOSA, specifically the 

etiquette associated with being a LOSA observer. An observer needs to learn how to approach a 

crew, how to ask permission to observe a flight, how to walk away so that the crew can discuss 

it, and to accept without question any crew’s decision to deny access to the cockpit. The 

observer should also carry a copy of the letter of endorsement jointly signed by management and 

pilots’ association to show any interested crew. 

On the jumpseat, the observer’s behavior is best summarized as “a fly on the wall.” The 

observer needs to be unobtrusive, yet responsive to any queries the crew may have. The 

observer does not complete the observation form on the jumpseat—this would be a disconcerting 

distraction to the crew. Instead, the observer is encouraged to carry a small pocket notebook or 
AC 120-90 4/27/06 

Appendix 2 

Page 2 

legal pad to jot down minimalist notes, just enough to jolt the memory when outside the cockpit 

and completing the full observation form. 

LOSA observers should be trained to accept their role as observers, not evaluators—they are 

not check airmen. LOSA observers will observe errors and undesired aircraft states as part of 

their observations; however, they should only interrupt if they perceive the safety of the flight to 

be seriously and immediately endangered. A helpful rule of thumb is to ask observers to think of 

themselves as a guest riding in the jumpseat of another airline. This seems to help distance the 

observer from the flightcrew while still being able to politely point out safety concerns if they 

come about. 

The LOSA observer is the only person in the whole LOSA process who has access to crew 

identities. It is essential, therefore, that observers are reminded throughout the training of their 

responsibilities in this regard. Confidentiality is paramount and observed crew behavior should 

not be discussed with anyone—not even other observers. Experience has shown that at the end 

of a flight, crews will often ask the observer to “debrief” their performance. In these 

circumstances, it is essential that the observer politely decline the invitation. This emphasizes 

the concept that the observer is not there to evaluate the crew, merely to record events. 

LOSA observers should act in an unobtrusive and consistent manner so that line pilots have a 

similarly positive experience of LOSA, which in turn will favorably affect their receptivity to the 

final results and outcomes. 

b. Company Policies and Procedures. 

Observers need to be current with company policies and procedures so as to observe 

procedural adherence on the line and detect any deviations. Selecting active line pilots from the 

airline is one way to ensure this. Spending some time in the classroom reviewing procedures 

across fleets allows all observers to get “up to speed” on the fleets they will be observing. All 

observers should also be encouraged to review the manuals as homework. 

c. Observation Form. 

This subsection refers to the mechanics of correctly completing and submitting a LOSA 

observation form. During the training, the observers work to develop the needed competencies 

as defined by the observation form. Hence, observers should see the observation form as soon as 

possible after the training begins so that they have a clear sense of what is expected of them. 

With the observation form in hand, the observers can be led through the various sections, and 

then practice using case studies (see next subsection). 

If the observation form is software-based, time should be spent ensuring that all observers 

have the necessary computer skills to open the form, enter and edit data, and submit the 

observation. Mastering these skills in the classroom will avoid potential loss of data later in the 

LOSA due to computer error. Observers should have the name of a contact person in case of 

computer problems. 
4/27/06 AC 120-90 

 Appendix 2 

 Page 3 

d. TEM Concepts. 

If the airline is using TEM as the basis of its LOSA, observers need to receive training in the 

framework. Specifically, observers need to be able to define, distinguish, and identify threats, 

errors, and undesired aircraft states. This is best achieved with a mixture of lecture, case studies, 

and review. Lecture material should include multiple examples of each type of threat, error, and 

undesired aircraft state, and case studies can take the form of scripted vignettes and/or actual 

accident and incident report excerpts. The distinction between threats, errors, and undesired 

aircraft states becomes clear with practice and observers are usually able to correctly distinguish 

examples of all three categories in 2 days or fewer of classroom involvement. Some training 

examples are provided in Appendix 5. 

e. Narratives. 

Observers need training in writing the flight narrative. If they understand the concepts 

underlying the observation form as well as the diagnostic rationale for conducting a LOSA, the 

observers will realize that a good-quality narrative is imperative. The observation form should 

contain several prompts to help the observer provide sufficient detail and observers should be 

encouraged to “overwrite” the flight rather than provide too little detail. In particular, observers 

need to record events that happened, such as threats, errors, or undesired aircraft states, the 

context in which they happened, and the crew’s response and management of the event. 

Observers are selected because they are experts at understanding flight operations and this 

expertise is best expressed in detailed narratives. As long as the observer provides a detailed 

narrative of the flight, any coding oversights can be remedied later in the data-cleaning process. 

f. Training Objectives. 

In sum, at the end of LOSA observer training, an observer should be able to: 

(1) Knowledgeably and confidently explain the rationale and process for conducting a 

LOSA at the airline; 

(2) Enact the LOSA observer etiquette in a professional and consistent manner; 

(3) Demonstrate knowledge of company policies and procedures; 

(4) Use the observation form accurately and comprehensively; 

(5) Understand the theoretical framework of the observation form. If the tool is based on 

TEM concepts, the observer should be able to define, distinguish, and identify threats, errors, and 

undesired aircraft states; and 

(6) Write detailed and comprehensive flight narratives from which others will be able to 

understand the full context of the flight and related events.
AC 120-90 4/27/06 

Appendix 2 

Page 4 

Members of the LOSA steering committee may want to attend LOSA observer training at 

another airline or attend an industry-sponsored LOSA conference for further guidance before 

designing the training. 

2. Observer Standardization. 

Standardization refers to the need to be sure that flight details are recorded in a systematic and 

consistent fashion. In LOSA, standardization is a multi-step process that involves standardizing 

the observers and conducting followup data-cleaning and coding of completed observations. 

The first step in any observer standardization is good-quality training. To be sure observers 

understand the concepts, group discussions are encouraged. Focusing on the finer points of the 

model and the observation form, these discussions help calibrate the observers to a common 

standard. A test can be administered at the end of the training to be sure that all observers have 

grasped the necessary knowledge and can demonstrate the required competencies as specified by 

the training objectives. If observers complete this test satisfactorily, they can be released to the 

line to complete one or two “trial” observations. 

The LOSA project coordinator or trainer should schedule time with each observer to discuss their 

trial observations. If the observer does a good job, as evidenced particularly by the quality of the 

narrative, the observations can be retained and used in the LOSA, and the observer can be sent 

back out to complete their observations. If the quality of the narrative is poor (e.g., lacking 

sufficient detail), the trainer can work with the observer to help draw out missing information. If 

the observer has forgotten details and cannot recreate the flight, the observation should be 

discarded, and the observer sent out to complete another trial observation. It is the LOSA 

coordinator’s decision to drop any observer from the observation team if that person fails to meet 

the required standard. For this reason, it can be a good idea to initially recruit and train more 

observers than needed, to allow for attrition, illness, and scheduling conflicts. 

a. Data Standardization. There are several supplementary techniques that ensure goodquality standardized data are used in a LOSA. 

First, observers are not asked to evaluate performance, but simply to observe it. From a data 

standpoint, this is the distinction between subjective judgment and objective observation. For 

example, observers are asked to note threats without any subjective judgment—if there is a 

thunderstorm, record it; if there is an aircraft malfunction or ground maintenance problem, 

record it. It is the same for errors and undesired aircraft states. Observers also note the crew’s 

response to the threats, errors, and undesired aircraft states and the outcome. The observer is not 

assessing crew performance or providing a subjective evaluation—the observer is telling the 

story of the flight. 
4/27/06 AC 120-90 

 Appendix 2 

 Page 5 (and 6) 

Second, the data analyst always checks observation forms submitted by observers against the 

narrative. Observers are expert at describing a flight—they are not necessarily expert at 

assigning codes to the various threats, errors, etc., especially if it is their first time as a LOSA 

observer. While training is given in the codes, nonetheless it is realistic to accept that observers 

won’t necessarily retain this information perfectly. The narrative is the “fail-safe” in the system 

in that it allows the analyst to read the events of the flight and match them to the observer’s 

codes. Because the analyst is the expert with the codes, he or she can add any codes that were 

missed and recode anything that might not be quite correct. Hence, a good-quality narrative is 

the ultimate key to standardized data. Observers provide a comprehensive narrative, and the 

analyst ensures consistent and accurate coding. 

A third step in standardizing the LOSA data prior to analysis involves verifying the data with 

a team of local experts—airline personnel familiar with the operation of each fleet (possibly fleet 

managers or representation of the airline’s operations, member of the steering committee, but not 

any of the observers). The group’s task is to review and verify the observations against current 

manuals, policies, and procedures. For example, an observer might log a procedural error for 

failure to make an approach callout when, in fact, there is no written procedure in the airline’s 

flight operations manual. The “error” would then be deleted from the database. The data 

verification group acts as a check on the analyst’s coding, ensuring events are correctly recorded 

in line with each fleet’s procedures and policies. It also builds ownership in the results and 

dispels any later criticism that the coding was not accurate. 

In sum, there are several methods that ensure that LOSA data are consistently and accurately 

recorded: 

• Observers are trained, calibrated, tested, and recalibrated 

• Objective observation, not subjective evaluation, is the basis and outcome of the 

observation 

• The narrative is stressed as key to high-quality data 

• The data analyst applies consistent coding to the observations 

• The data verification group checks the analyst’s coding against specific procedures 

Following the above steps will ensure reliability and validity of the data analyzed from LOSA 

observations. 

4/27/06 AC 120-90 

 Appendix 3 

 Page 1 

APPENDIX 3. SAMPLE LOSA OBSERVATION FORM 
AC 120-90 4/27/06 

Appendix 3 

Page 2 

APPENDIX 3. SAMPLE LOSA OBSERVATION FORM (Continued) 
4/27/06 AC 120-90 

 Appendix 3 

 Page 3 

APPENDIX 3. SAMPLE LOSA OBSERVATION FORM (Continued)
AC 120-90 4/27/06 

Appendix 3 

Page 4 

APPENDIX 3. SAMPLE LOSA OBSERVATION FORM (Continued) 
4/27/06 AC 120-90 

 Appendix 3 

 Page 5 

APPENDIX 3. SAMPLE LOSA OBSERVATION FORM (Continued)
AC 120-90 4/27/06 

Appendix 3 

Page 6 

APPENDIX 3. SAMPLE LOSA OBSERVATION FORM (Continued)
4/27/06 AC 120-90 

 Appendix 3 

 Page 7 (and 8) 

APPENDIX 3. SAMPLE LOSA OBSERVATION FORM (Continued) 

4/27/06 AC 120-90 

 Appendix 4 

 Page 1 (and 2) 

APPENDIX 4. SAMPLE LETTER TO THE LINE PILOTS 

To: All XX Airline Pilots 

From: (Name) Senior Director, Flight Operations 

 (Name) Director, Flight Training and Standards 

 (Name) Director, Flight Safety and Quality Assurance 

 (Name) Chairman, Pilots’ association Safety Committee 

Subject: Line Operations Safety Audit (LOSA) 

Beginning mid-October and continuing for approximately five weeks, this airline will 

conduct a Line Operations Safety Audit (LOSA). LOSA observations are no-jeopardy events, 

and all data are confidential and de-identified. LOSA data go directly to the XX Research 

Program for data entry and analysis. 

We will use our own active pilots to conduct cockpit jumpseat observations. Be assured that 

these observations are not checkrides. Although some LOSA observers may be check airmen, 

they are not there to critique your performance - their mission is to be an unobtrusive observer 

and to fill out data collection forms after the flight is completed. 

The ultimate customer of the audit is you, the line pilot. The audit should help us identify 

problem areas so that we can correct them and make your job easier. Did you ever see a 

procedure that could be done better, but didn’t feel like you had a way to feed that idea into the 

system for possible change? Are some procedures better than others as far as helping avoid, trap 

and mitigate errors? LOSA should help us identify the strengths and weaknesses of our crew 

procedures, and with that information, management is committed to making necessary changes 

to continually improve the way that we do business. 

In short, we’re doing a LOSA so that we can improve the system to better support you. After 

the audit is completed, we’re committed to telling you how it went, and how we plan to make 

improvements. 

Please extend your usual professional courtesies to the LOSA observation team, and thank 

you for your unfailing cooperation. 

Sincerely, 

(Name), Senior Director, Flight Operations 

(Name), Director, Flight Training and Standards 

(Name), Director, Flight Safety and Quality Assurance 

(Name), Chairman, Pilots’ association Safety Committee 

4/27/06 AC 120-90 

 Appendix 5 

 Page 1 (and 2) 

APPENDIX 5. SAMPLE VIGNETTES FOR OBSERVER TRAINING 

Case Study One – Read the following and list all threats, errors, and undesired aircraft states as 

well as the flightcrew response and outcome. 

Predeparture / Taxi-out – The Captain requested an extra 5000 pounds of fuel to be loaded. After ground 

confirmed the fueling, the First Officer alerted the Captain that they were 2000 pounds off. The Captain radioed 

back to ground and the missing 2000 pounds was loaded. 

Solution – One threat, no errors or undesired aircraft states 

Threat #1 – Ground Handling Threat – The ground crew failed to load the all 5000 lbs of requested fuel – 

The flightcrew response was the First Officer detecting and correcting the shortage. The outcome was 

inconsequential. 

Case Study Two – Read the following and list all threats, errors, and undesired aircraft states as 

well as the flightcrew response and outcome. 

Descent / Approach - While descending through 18000 feet, the First Officer performed the entire descent checklist 

from memory, which is against standard operating procedures. The Captain noticed the First Officer doing it but 

chose to ignore it. In the end, everything was set correctly. 

Solution – No threats, one error and no undesired aircraft states 

Error #1 – Checklist Error – The FO performed the descent checklist from memory. The flightcrew 

response was the captain detecting the error but failing to correct (ignored). The outcome was 

inconsequential 

Case Study Three – Read the following and list all threats, errors, and undesired aircraft states 

as the flightcrew response and outcome. 

Descent / Approach / Land – During a 30-degree bank on a visual approach, the Captain unwillingly allowed the 

aircraft to get 15 knots below minimum maneuvering speed. While it should have been detected sooner, the low 

speed was eventually pointed out by the First Officer. The Captain said thanks and immediately increased the 

speed. 

Solution – No threats, one error and one undesired aircraft state 

Error #1 – Aircraft Handling Error – The Captain allowed the speed to decay during a bank turn. No one 

detected the decay before it dropped below minimum maneuvering speed. Therefore, the error was 

undetected and linked to an undesired aircraft state. 

Undesired Aircraft State #1 - As soon as the speed went below minimum maneuvering speed, the aircraft 

was in an undesired aircraft state that was detected by the First Officer and corrected by the Captain. The 

final undesired aircraft state outcome was inconsequential.