Flightcrew Member Training Hours Requirement 

Review Aviation Rulemaking Committee

Report from the THRR ARC

May 23, 2011
THRR ARC Final Report i

TABLE OF CONTENTS

Table of Contents........................................................................................................................... i

Executive Summary.....................................................................................................................iii

1.0 Recommendations.................................................................................................................. 1

1.1 Issue 1................................................................................................................................... 1

1.2 Issue 2................................................................................................................................... 2

1.3 Issue 3................................................................................................................................... 3

1.4 Issue 4................................................................................................................................... 4

1.5 Issue 5................................................................................................................................... 4

1.6 Issue 6................................................................................................................................... 4

1.7 Issue 7................................................................................................................................... 5

2.0 Background ............................................................................................................................ 6

2.1 ARC Composition................................................................................................................ 6

2.2 Clarification of Scope........................................................................................................... 6

2.3 Methodology ........................................................................................................................ 7

2.4 Competency-Based Training................................................................................................ 7

2.5 Not Considered................................................................................................................... 10

3.0 Rationale ............................................................................................................................... 11

3.1 Issue 1 Rationale ................................................................................................................ 11

3.2 Issue 2 Rationale ................................................................................................................ 17

3.3 Issue 3 Rationale ................................................................................................................ 22

3.4 Issue 4 Rationale ................................................................................................................ 26

3.5 Issue 5 Rationale ................................................................................................................ 28

3.6 Issue 6 Rationale ................................................................................................................ 29
THRR ARC Final Report ii

3.7 Issue 7 Rationale ................................................................................................................ 31

Appendix A—THRR ARC Members and Support Staff..................................................... A–1

THRR ARC Members and Affiliation ................................................................................... A–1

Support Staff .......................................................................................................................... A–2

Appendix B—Acronyms...........................................................................................................B–1

Appendix C—Point Grading Example .................................................................................. C–1

Sample Grading Scale .............................................................................................................C–1

Joint Aviation Authority 5-Point Grading Example ...............................................................C–2

Transport Canada 4-Point Grading Scale................................................................................C–4

FlightSafety 4–Point Grading Scale........................................................................................C–6

Appendix D—Applying Issue 3’s Recommendations........................................................... D–1

Implementing the Early Evaluation Concept (Short-Cycle) .................................................. D–1

Appendix E—Dissenting Statement........................................................................................E–1

Appendix F—Charter...............................................................................................................F–1
THRR ARC Final Report iii

EXECUTIVE SUMMARY

The Federal Aviation Administration (FAA) chartered the Flightcrew Member Training 

Hours Requirement Review (THRR) Aviation Rulemaking Committee (ARC) to develop 

the report required by Section 209 of the Airline Safety and FAA Extension Act of 2010 

(Pub. L. 111–216). The THRR ARC charter appears in its entirety in Appendix F.

The THRR ARC was given three tasks, which are described below.

I. To assess and make recommendations concerning—

(1) The best methods and optimal time needed for flightcrew members of Title 14, 

Code of Federal Regulations (14 CFR) part 121 and 135 air carriers to master aircraft 

systems, maneuvers, procedures, takeoffs and landings, and flightcrew coordination;

(2) The initial and recurrent testing requirements for pilots, including the rigor and 

consistency of testing programs such as check rides;

(3) The optimal length of time between training events for such flightcrew members, 

including recurrent training events;

(4) The best methods to reliably evaluate a flightcrew member’s mastery of aircraft 

systems, maneuvers, procedures, takeoffs and landings, and crew coordination;

(5) Classroom instruction requirements governing curriculum content and hours of 

instruction;

(6) The best methods to allow specific academic training courses to be credited toward 

the total flight hours required to receive an airline transport pilot (ATP) certificate; 

and

(7) Crew leadership training.

II. To consider the scalability of recommendations to address the needs of small businesses.

III. To develop regulatory recommendations as may be required.

During early deliberations, the THRR ARC recognized that the aviation training paradigm 

needed to shift to achieve substantial changes to training/checking, and by extension, safety. 

This need is clearly illustrated in a report of the International Air Transport Association’s 

(IATA) Training and Qualification Initiative:

―For decades, the content of flightcrew training [programs] has remained 

unchanged according to regulation whilst aircraft design and reliability have 

evolved dramatically. Regulatory prescriptions for flightcrew training and 

checking are based on events, which are now highly improbable in aircraft 
THRR ARC Final Report iv

designed to meet modern standards. Training [programs] are consequently 

burdened with items, which do little to mitigate real risks or enhance safety.‖

1

Understanding it was necessary to provide methods to move beyond traditional prescriptive 

training if more effective learning and assessment are to occur, the THRR ARC investigated the 

field of learning systems design. Ultimately, the ARC accepted the need to move towards 

competency-based training, utilizing evidence (data and analysis) to develop the best training.

Today, the optimum method to accomplish this would be through the group of methodologies

referred to as Instructional Systems Design (ISD).

ISD methodologies are based upon analysis of the workforce, the equipment to be operated, and 

the operating environment. Variables must be considered, including the number of pilots, 

fleet size, and the diversity of fleet and staffing levels. Once the principles of ISD are utilized, 

training programs can be developed that address the common cohort of the individual operators 

and allow some flexibility for meeting the training needs of the individual.

The THRR ARC has concluded the FAA-approved training program known as Advanced 

Qualification Program (AQP) provides the best current method for training, assessing, and 

validating flightcrew members for commercial air carrier operations. Due to practical 

limitations, AQP is not a readily achievable, or in some cases even a possible training solution 

for all operators.

AQP has provided the industry with principles that have proven effective using the following 

three pillars:

1. Competency-Based Training. This principle relies on training that is delivered and 

evaluated based upon the amount of training each individual needs to ―master‖ the 

required tasks. Competency may be achieved at different rates for different people, and 

the amount of elapsed time between training events that competency is maintained also 

varies individually and must be considered. Competency-based training varies from 

prescriptive training because it recognizes one size cannot fit all.

2. Train to Proficiency. Inherent to any competency-based training program is the principle 

of ―train to proficiency‖. Train to proficiency provides for the variations of individual 

learning rates. It also allows for additional exposure to a task until the pilot is proficient, 

and competency is achieved. Training to proficiency drives the quantity of training 

required to gain competency

3. Data Collection and Feedback. This is possibly the most important element to ensure the 

effectiveness of this approach. This is where the skill and knowledge of the individual 

and flightcrew can be assessed, and the health of the training program is monitored. The 

data element provides a continuous feedback loop allowing for rapid adjustments when 

performance indicators warrant action and helps determine the correct intervals between 

training events.

 

1

 IATA Training and Qualification Initiative (ITQI) Report 2009 Fall Edition
THRR ARC Final Report v

In the competency-based training approach, the amount of time required to master an element is 

not fixed. Instead, it is determined by how long it takes for the individual to reach and/or 

maintain competency with regard to knowledge, skills, and attitudes (KSA). These KSAs are 

established through detailed task analysis of the aircraft and environment in which the pilot will 

be operating. Likewise, the evaluation and validation criteria are based on a demonstration of 

mastery of individual and flightcrew KSAs. 

The THRR ARC was challenged by the need to provide scalable options that acknowledge the 

vast diversity of air carrier size and scope that exists in the industry, particularly between the 

large scheduled airlines of part 121 and the small, on-demand operators of part 135. Any 

―universal‖ training recommendations made by the ARC need to be appropriate for the smallest 

single-pilot, single-aircraft operator and the largest air carriers with fleets comprising hundreds 

of large jet aircraft flown by thousands of pilots.

The THRR ARC evaluated the seven issues to determine the optimal training solutions, but 

recognized that a prescriptive regulatory minimum must be maintained in order to accommodate 

the needs of the smallest air carriers that lack the resources and data pool necessary for the 

development of an ISD-driven training program. The ARC did believe the minimum regulatory 

standard should be reevaluated to determine air carriers’ ability to produce an acceptable level of 

safety. In light of the many recommendations made by the ARC, the FAA should alter the 

minimum standard for smaller air carriers where prudent to improve pilot training and also 

provide mechanisms within the regulations, policy, and guidance that enable and encourage the 

smaller air carriers to adopt ISD-driven methods where able.

The THRR ARC used the basic principles of ISD to address Issues 1 through 5. Issues 6 and 7 

were the subject of separate FAA-established ARCs. The ARC reviewed presentations by 

subject matter experts (SME) from those ARCs and used that information as the foundation for 

its response to those issues.
THRR ARC Final Report 1

1.0 RECOMMENDATIONS

1.1 ISSUE 1

The best methods and optimal time needed for flightcrew members of Title 14 Code of Federal 

Regulations (CFR) part 121 and 135 air carriers to master aircraft systems, maneuvers, 

procedures, takeoffs and landings, and crew coordination. (See section 3.1 on page 11 for 

the rationale.)

1.1.1 RECOMMENDATIONS

The THRR ARC recommends—

 That a process-based training development system, such as ISD, be used to develop a 

training course tailored to meet the requirements of each operator.

 That scenario-based training be used as often as possible throughout the curriculum.

 That as a minimum, each flightcrew member receives annual recurrent training without 

the option to replace the training event with a checking event, as currently permitted.

 That instructors and evaluators attend annual recurrent training that includes methods for 

ensuring consistent grading of training and validation events.

 That the identifiable pilot record shall only record satisfactory/unsatisfactory completion.

 That a multipoint grading scale will be used to assess and enhance the training program.

 That a new ARC be established to identify the methodology and data necessary to support 

a process-based training development system for those operators who do not currently 

use such a system or are without access to sufficient data to maintain such a system (that 

is, small air carriers). Examples of data that should be considered include:

o Aircraft data monitoring programs, such as flight data monitoring (FDM) or Flight 

Operations Quality Assurance (FOQA);

o Aviation Safety Action Program (ASAP);

o Operator implemented safety reporting systems;

o Safety Management System (SMS) and/or Internal Evaluation Program information;

o Line Operations Safety Audit (LOSA);

o Pilot survey and/or critique information; and
THRR ARC Final Report 2

o Any other available regulator or industry information such as Information for 

Operators (InFO), Safety Alert for Operators (SAFO), and Flight Standards Board 

(FSB) reports. 

 That an ARC be established to evaluate and clarify the rules governing the relationship 

between part 121 and part 135 air carriers and 14 CFR part 142 training centers.

1.2 ISSUE 2

The initial and recurrent testing requirements for pilots, including the rigor and consistency of 

testing programs such as check rides. (See section 3.2 on page 17 for the rationale.)

1.2.1 RECOMMENDATIONS

The THRR ARC recommends—

 That when a second in command (SIC) is required, the SIC shall be held to the same 

minimum performance standard as the pilot in command (PIC).

 That flight proficiency be validated by using a maneuvers list developed by the 

air carrier, who must use an ISD approach to job task analysis. The maneuvers list should

be based on the air carrier’s specific equipment and operation and may be supplemented 

by any FAA or industry requirements.

 That training, validation, and evaluation of flight proficiency in multi-pilot operations 

include pilot monitoring (PM) skills.

 That the line check program employed in part 121 is suitable for those operations.

 That for part 135 operations, the adoption of line check rule changes be made, as 

proposed by the Part 135/125 ARC.

2

.

 

2

Federal Docket No. FAA-2002-13923
THRR ARC Final Report 3

1.3 ISSUE 3

The optimal length of time between training events for such flightcrew members, including 

recurrent training events. (See section 3.3 on page 22 for the rationale.)

1.3.1 RECOMMENDATIONS

 That early evaluation (short-cycle) should be required for:

o Extended pilot absence from flight operations requiring requalification training that

exceeds recurrent training but does not require initial training.

o Unsatisfactory performance during an assessment.

 That under specified conditions, the evaluation interval may be extended if the air carrier 

can demonstrate operational characteristics contained in one or more of the following 

seven options.

o Option 1: Implementation of a First Look Program— Special Purpose Operational 

Training (SPOT) Check; a 1-month interval extension for a § 121.441 or § 135.297 

flight evaluation.

o Option 2: Implementation of Line Oriented Flight Training Program; a 1-month 

interval extension for a § 121.441 or § 135.297 flight evaluation.

o Option 3: Implementation of an Enhanced Training Data Collection and Feedback 

Program; a 1-month interval extension for a § 121.441 or § 135.297 flight evaluation.

o Option 4: Implementation of an SMS; a 1-month interval extension for a § 121.441 

or § 135.297 flight evaluation.

o Option 5: Implementation of FOQA or FDM International Civil Aviation 

Organization (ICAO) program; a 1-month interval extension for a § 121.441 or 

§ 135.297 flight evaluation.

o Option 6: Enhanced Management Oversight program; a 1-month interval extension 

for a § 121.441 or § 135.297 flight evaluation.

o Option 7: Demonstration of Individual Pilot Operational Tempo; a 1-month interval 

extension for a § 121.441 or § 135.297 flight evaluation.

 That the evaluation interval is not extended beyond 12 months, unless the air carrier has 

developed an AQP program in accordance with part 121, subpart Y.
THRR ARC Final Report 4

1.4 ISSUE 4

The best methods to reliably evaluate a flightcrew member’s mastery of aircraft systems, 

maneuvers, procedures, takeoffs and landings, and crew coordination. (See section 3.4 on

page 26 for the rationale.)

1.4.1 RECOMMENDATIONS

The THRR ARC recommends—

 That flightcrew member mastery be evaluated through the use of an appropriate 

flight simulator training device (FSTD).

 That the flight proficiency be verified using scenario-based evaluation.

1.5 ISSUE 5

Classroom instruction requirements governing curriculum content and hours of instruction. 

(See section 3.5 on page 28 for the rationale.)

1.5.1 RECOMMENDATION

The THRR ARC members concluded the same process and procedures for Issue 1 apply to 

classroom training. Therefore, the recommendations for Issue 5 are included in those for Issue 1.

(See section 3.1 on page 11 for the rationale.)

1.6 ISSUE 6

The best methods to allow specific academic training courses to be credited toward the total 

flight-hours required to receive an airline transport pilot certificate. (See page 29 for 

the rationale.)

1.6.1 RECOMMENDATIONS

The THRR ARC recommends—

 That the FAA implement the First Officer Qualification (FOQ) ARC’s recommendation to

establish an aeronautical experience credit system for an ATP certificate with SIC 

privileges in operations conducted under part 121. One member of the THRR ARC 

dissented from this recommendation.

 That an aeronautical experience credit system not be allowed for experience requirements 

for any ATP certificate for a PIC at a part 121 or part 135 air carrier.
THRR ARC Final Report 5

 That the use of an aeronautical experience credit system not be used for any part 135 

flight-hour requirements.

1.7 ISSUE 7

Crew leadership training. (See page 31 for the rationale.)

1.7.1 RECOMMENDATION

The THRR ARC recommends—

 That the FAA implement the Flight Crewmember Mentoring, Leadership, and 

Professional Development (MLP) ARC recommendation to require leadership training 

for the PIC. An ISD process could potentially be used in lieu of the specific hour 

requirement in the MLP ARC’s recommendation.

 That leadership training be integrated into appropriate training events through the 

ISD process.
Flightcrew Member THRR ARC Report 6

2.0 BACKGROUND

2.1 ARC COMPOSITION

The THRR ARC was composed of subject matter experts from 13 organizations (see 

Appendix A for a complete list). Collectively, the ARC members brought significant levels of 

experience to the deliberations in air carrier operations; the development, implementation, and 

management of pilot training and qualification programs; and the establishment of pilot training 

and qualification standards at the domestic and international level. The ARC also had access to 

representatives from the FAA to answer any specific regulatory or guidance issues that arose 

during deliberations.

2.2 CLARIFICATION OF SCOPE

While the THRR ARC’s tasking was to address ―flightcrew members,‖ which by definition 

includes flight engineers and navigators, the ARC deliberations and response only address the 

flight deck crew (that is, pilots). This adjustment to the scope was made because the language 

from the legislation and charter implied that the ARC’s focus is intended to be on pilot training. 

Although flight engineers and navigators are included in the definition of flightcrew member, 

their actual cohort is extremely small compared to pilots. Thus, the ARC felt that the overall 

impact on safety could better be served by spending its limited time and resources to address

pilot training, the the principle area of concern.

In general, the THRR ARC members believe the existing regulatory minimum requirements 

have served the industry well, improvements are possible, and in some instances, necessary.

One of the biggest challenges for the THRR ARC was addressing parts 121 and 135 

simultaneously. The difference in operating philosophies sometimes made it difficult to find a 

common solution. Ultimately, the ARC chose to recommend the optimal solution for each issue, 

which often required adjusting standards to the best practice of the more robust guidance. In 

several instances, it was difficult for the ARC members to envision how the FAA would apply 

the THRR ARC recommendations to many part 135 air carriers, particularly those classified as 

single-pilot or single-PIC operators. Where practical, this report notes the difficulty in applying 

certain recommendations to part 135 air carriers and suggests that the FAA undertake separate 

rulemaking, ideally via an ARC, to address concerns the THRR ARC was unable to consider.

With regard to the challenges faced when attempting to address the varied operations that exist 

under part 135, the FAA convened the Part 135/125 ARC in 2005, which was specifically tasked 

with reviewing and recommending changes to part 135. These changes included a complete 

review of the training and checking requirements of those operators, with consideration given to 

the small business aspects prevalent in that industry segment. The THRR ARC did not have 

sufficient time to review and comment on the extensive recommendations made by that ARC,

and therefore has no specific comment on them. The THRR ARC does believe the FAA should 

review those recommendations in conjunction with its own, as appropriate, when considering 

new training standards for part 135.
Flightcrew Member THRR ARC Report 7

2.3 METHODOLOGY

The bulk of the THRR ARC report provides guidance on Issues 1 through 5. These issues

address the level of training, validation, and evaluation needed to ensure a pilot in 

part 1213or 135 flight operations has mastery of the KSAs to operate in the complex air carrier 

arena, and that the intervals between recurrent training events do not compromise safety. While 

the ARC tasking requires distinct responses to each issue, these five issues share enough 

commonality to allow deliberations and analysis to be conducted in three groupings:

 Training Courseware,

 Intervals of Training, and

 Evaluation and Validation.

Accordingly, the THRR ARC developed three subcommittees, one for each group. Each 

subcommittee was then assigned the specific issues(s) most closely associated with its area. The 

issues were divided as follows:

 Training Courseware: Issue 1 and Issue 5

 Intervals of Training: Issue 3

 Evaluation and Validation: Issues 2 and Issue 4

Issues 6 and 7 were the subject of two separate ARCs: the FOQ ARC and MLP ARC. While the 

THRR ARC charter requires independent deliberation and response, the ARC membership 

agreed that it was prudent to be familiar with and build on the previous ARCs’ work. The 

THRR ARC received a series of briefs on the FOQ and MLP ARCs’ work and used that as a 

foundation to develop its responses.

2.4 COMPETENCY-BASED TRAINING

Following extensive discussion on the optimal method to train and evaluate pilots, the 

THRR ARC members agreed upon a training system established on a train-to-proficiency 

standard. Currently, training is prescriptive in the specific instruction delivered and assessed,

and in the hours required for each training event. This training system provides air carriers with 

little flexibility to adjust to the identified needs of an individual pilot or group of similarly 

situated pilots. The train-to-proficiency philosophy recognizes that the optimal time required to 

master a particular task can vary, and permits tailoring the training to the recipient. This 

philosophy is the over-arching element for how the THRR ARC approached its discussions and 

subsequent recommendations.

 

3

Discussion of Part 121 was restricted to the regulations current as of May 5, 2011.
Flightcrew Member THRR ARC Report 8

Currently, this philosophy is best associated with the ISD concept, which is explained in further 

detail in Section 2.4.1. Although relying heavily upon the current use of ISD, the THRR ARC 

does not intend to restrict the use of newer concepts for training program design that may arise in 

the future. To the contrary, the ARC members decided not to focus on a singular method of 

program design. The goal is to encourage air carriers to develop training programs that produce

pilots who are ―trained-to-proficiency.‖ Today, that output is best captured by the ISD process, 

so it was used by the ARC as a model system.

The ISD curriculums are designed to provide a level of training that will allow a normally 

progressing pilot of average skills in a target group to complete all training successfully in the 

planned hours and days. Pilots following these curriculums are continually assessed on all tasks 

each day in each phase of training. At the completion of each phase, a pilot’s training is 

validated and his or her performance must meet standards. If at any time a pilot’s performance is 

assessed as not meeting the standards for a task during any curriculum period of training or phase 

validation, the pilot will be remediated to standards before progressing.

During the remediation process, a pilot may be provided additional training periods for that 

phase of training. Remedial training will continue as long as the pilot is making reasonable 

progress. When a pilot fails to make progress at a reasonable rate, he or she should be

considered to have failed the curriculum.

At the end of the curriculum for a pilot who has met standards for all tasks and validations,

the pilot must then be evaluated. The evaluation should be a scripted operational 

scenario-based event, and must be developed to allow for the evaluation of decision-making, 

leadership, crew resource management (CRM), and threat and error management (TEM)

skills. If a pilot fails the evaluation, he will be remediated to standards and then re-evaluated. 

If the pilot fails the evaluation a second time, the pilot should be considered to have failed 

the curriculum.

2.4.1 ISD PROCESS

ISD is a systematic process of developing training curriculum. It is used and applied in many 

different industries and work settings. ISD encompasses adult learning theory, educational 

psychology, human factors engineering, and cognitive psychology.

The ISD process involves the following steps:

 Analysis:

o Develop a job or task list.

o Analyze the list to determine its essential skill and knowledge requirements.

o Determine which skills and knowledge requirements must be trained and/or tested.

o Develop objectives, with conditions and standards that define acceptable 

performance.
Flightcrew Member THRR ARC Report 9

 Design:

o Allocate the objectives defined in the analysis step to the appropriate curricula.

o Sequence and group the objectives within a curriculum for optimum training 

and evaluation.

o Develop an evaluation strategy, and develop test items for the objectives.

 Development:

o Develop instructional materials (for example, lesson plans, handouts, videos, and 

tests) for the curriculum developed in the design phase.

o Determine the appropriate use of available technologies.

o Train instructors to use the new curriculum materials.

o Test the training materials by having the instructors teach a small group of students 

and modify the materials if necessary.

 Implementation:

o Deliver the curriculum to the students.

o Collect instructor and student critiques during and after the delivery process.

o Collect student performance data.

 Evaluation:

o Perform continual analysis of performance data, and use the results from that 

analysis to modify the course.

o The following are possible sources of data for course evaluation:

 FOQA,

 ASAP,

 LOSA,

 Pilot critique,

 Any available data such as: InFO, SAFO, and FSB reports.
Flightcrew Member THRR ARC Report 10

In the ISD model, evaluation takes place throughout the development process. While performing 

the five steps, the developer will constantly be evaluating the process and resulting materials. It 

is a method of quality control to ensure the final result is as appropriate to the learning situation 

as possible.

Annual recurrent training should be established by performing risk analysis to determine the 

critical events that need to be trained. In addition to establishing event criticality, event currency 

must also be determined. Currency applies to events flightcrew members perform on a regular 

basis; annual training may not be necessary. The ISD process will be used to determine event 

criticality and currency.

Flightcrew members should then be trained on critical events and any other FAA or 

industry-required events on an annual basis. The current regulations allow for a 

―checking event‖ to be substituted for annual training. The THRR ARC believes this 

practice is detrimental to pilot performance and that annual training should be required.

2.5 NOT CONSIDERED

The following training and evaluation events were considered by the THRR ARC, but were 

deemed outside the scope of its charter:

 Transportation Security Administration training requirements (Title 49, Code of 

Federal Regulations (49 CFR), part 1500)

 Hazardous materials training (49 CFR, part 172)
Flightcrew Member THRR ARC Report 11

3.0 RATIONALE

3.1 ISSUE 1 RATIONALE

The best methods and optimal time needed for flightcrew members of Title 14 Code of Federal 

Regulations (CFR) part 121 and 135 air carriers to master aircraft systems, maneuvers, 

procedures, takeoffs and landings, and crew coordination.

3.1.1 SECTION INTRODUCTION

A grasp of the evolution of flight training is integral to this response. Traditional pilot training 

consists of several days or weeks of classroom lecture. The trainee then receives some form of 

procedures training in an aircraft or FSTD, followed by a checking event.

The following quote, taken from a report of the IATA Training and Qualification Initiative4,

helps illustrate the direction of this evolutionary process, a direction that will lead to higher 

levels of training quality if it is followed:

―In addition to the wealth of accident and incident reports, flight data collection and 

analysis offers the possibility to tailor training programs to meet real risks. The aim is to 

identify and train the real skills required to operate, whilst addressing any threats 

presented by the evidence collected. If we fail to make a substantial change to the 

philosophy of airline pilot training, we will face an increasing gap between training and 

reality. It is time to create an effective bridge between safety data and the development 

of training solutions. At the same time, we must embed robust methodologies for the 

handling of real events and the development of key pilot skills, both technical and 

non-technical, into those training solutions.”

The use of technology to meet training needs in all professional fields is well grasped and is no 

less available to aviation training. Exploiting technology to match the correct training tool to a 

specific training objective improves overall training quality and reduces the cost for learning and 

increased availability of training to a broader base of air carrier pilots.

The industry has a wealth of safety-related evidence at its disposal, including accident and 

incident reports and LOSA, FOQA, and ASAP data. In addition to safety data, the industry also 

has training data. Analyzing this data through a formalized SMS process provides an 

opportunity to influence and advance the philosophy of flightcrew training, by weighing risks 

and determining which situations are more likely to happen. There is strong evidence these 

programs have contributed significantly to the reduction in aircraft accident and incidents.

 

4

 IATA Training and Qualification Initiative (ITQI) Report 2009 Fall Edition
Flightcrew Member THRR ARC Report 12

At this time, these programs are generally limited to large air carriers with fleets and pilot 

populations large enough to produce a viable data sample.5 Smaller operators need access to this 

type of data in order to properly conduct the ISD evaluation process.

Ultimately, the success of any training program is highly dependent on the competency of its

instructors. Historically, there has not been an appropriate level of criticality for 

instructor/evaluator training within the industry, and often this training is a onetime event. 

Instructor/evaluator training is a critical part of the process and will also be developed using a 

process-based training development system, such as ISD.

To fully develop a response to this issue, the following discussion and recommendations will 

focus on three areas:

1. Pilot Training,

2. Grading, and

3. Instructor/Evaluator Training.

3.1.2 PILOT TRAINING

Background

Considering the variability and scope of part 121 and part 135 flight operations, it is 

impractical to create a prescriptive list of tasks and optimal time that adequately address 

all certificate holder training and qualification requirements. Typical variables include 

the following:

 Experience of the flightcrew member;

 Training resources available/utilized;

 Scope of the certificate holder’s operation (that is, long haul international, short haul 

domestic, single pilot operation, and associated threats);

 Type and complexity of the aircraft;

 Number of pilots with respect to the number of available data points; and

 Fleet mix, including variants.

 

5

 It is possible that creative solutions could allow smaller air carriers (and to some degree, training centers) to 

create a sufficient population to support these programs or variations thereof by combining data across multiple 

air carrier sources. The THRR ARC believes the value imparted by these data programs is sufficient to pursue these 

unique solutions. One example of a multi-carrier data collaboration program is the Medallion Program that has 

seen much success in Alaska.
Flightcrew Member THRR ARC Report 13

The ARC endorses an analytical approach (ISD) to training development characterized by the 

use of a systematic process of analysis, design, development, implementation, and evaluation to 

define the best training methods tailored to meet the requirements of each operator. The 

optimum time required to complete training will be determined through the process.

The ISD process should lead to a strategy of training pilots to established standards. Once a pilot 

has demonstrated the ability to perform a maneuver or task to standard within that training cycle, 

there is no longer any need to assess that maneuver or task, and that pilot is considered to have 

―mastered‖ it. It is not necessary to evaluate each and every task a pilot must perform. The 

evaluation recommendations are further discussed in section 3.1.4.

The ISD process should apply to all FAA-required training curriculums. The recurrent training 

curriculum is established through risk assessment to determine task criticality and establish the 

appropriate annual training requirements.

Historically, the Federal Aviation Regulations have allowed air carriers the option to substitute a 

checking event for a training event. When this option is exercised by the air carrier, pilots are 

deprived of the opportunity to improve their skills and learn new skills through the interaction 

with a highly trained instructor pilot.

Scenario-based training incorporates profiles that include standard operating procedures (SOP),

normal and abnormal emergency checklists, aircraft performance, CRM, and TEM. Profiles 

should contain an operationally relevant real-world-like situation or scenario, reflective of any 

special operational authorizations the operator has been granted. These profiles should be used 

to focus the students on a particular subject matter.

All training should integrate the use of current and future advanced technologies and 

flight training equipment, including full flight simulators. Air carriers should utilize a suite of 

equipment matched on the basis of analysis to the training requirements at any given stage of a 

curriculum. Judicious analysis determines the appropriate type and level of FSTD to use.

The training process should integrate aircraft systems; training, including normal, abnormal, and 

emergency procedures; CRM; and TEM. Most of these training topics have typically been 

considered classroom training, but training has been proven to be most effective when conducted 

with a combination of classroom training, computer-based training, training devices, simulators

or aircraft, and other technologies.

It is important to have qualified flightcrew members occupy each seat during training and 

checking events. When an air carrier conducts its own training and checking, the best practice 

for flightcrew pairing is PIC and SIC.

If logistics do not allow the pairing of a PIC with an SIC for training or checking, the following 

are acceptable alternatives:

 Same position (PIC/PIC) or (SIC/SIC), as long as the flightcrew members have been 

specifically trained in the tasks that are ―seat dependent.‖ For example, if an SIC is being 

used as a substitute for the PIC, that SIC must be trained in the specific PIC 

seat-dependent tasks.
Flightcrew Member THRR ARC Report 14

 PIC or SIC and air carrier certificate holder qualified flightcrew member.

After lengthy discussions, it became apparent to the THRR ARC members the current 

regulations do not adequately address many issues between air carriers and part 142-approved 

training centers. Some of these issues include, but are not limited to:

 Check airman/instructor training,

 Training program development,

 Training program maintenance/revision,

 Flightcrew member pairing,

 Multiple training providers used by one operator, and

 Multi-operator concurrent training (same training program, different air carrier using).

Conclusion

The best training methods will be determined through the use of an ISD process. The same 

process will determine the optimal time. Scenario-based training applied in conjunction with the

most appropriate training device provides for a highly effective learning experience.

Recommendations

The THRR ARC recommends—

 That a process-based training development system, such as ISD, be used to develop a 

training course tailored to meet the requirements of each operator.

 That scenario-based training be used as often as possible throughout the curriculum.

 That as a minimum, each flightcrew member receives annual recurrent training without 

the option to replace the training event with a checking event as currently permitted.

 That a new ARC be established to identify the methodology and data necessary to support 

a process-based training development system for those operators who do not currently 

use such a system or are without access to sufficient data to maintain such a system (that 

is, small air carriers). Examples of data that should be considered include:

o Aircraft data monitoring programs, such as flight data monitoring (FDM) or FOQA;

o ASAP;

o Operator implemented safety reporting systems;

o SMS and/or Internal Evaluation Program information;
Flightcrew Member THRR ARC Report 15

o LOSA;

o Pilot survey and/or critique information; and

o Any other available regulator or industry information such as InFO, SAFO, FSB, 

Flight Safety Foundation, and Air Charter Safety Foundation reports.

 That an ARC be established to evaluate and clarify the rules governing the relationship 

between parts 121 and 135 air carriers and part 142 training centers.

3.1.3 GRADING

Background

The practice of pass/fail and satisfactory/unsatisfactory grading may have been influenced by 

certain FAA forms and job aids used by air carriers to develop their own proficiency check 

grading system, which remains valid for the purpose of assessing overall performance of the 

flightcrew member.

Mutli-point grading scales for tasks and maneuvers have shown to be better for purposes of data 

analysis. This granularity provides better assessment of the pilot’s training progress and the 

strengths and weaknesses, and trends of the program over time (examples of 4- and 5-point 

grading scales are shown in Appendix C). Instructors and evaluators should make comments 

about the flightcrew member’s progress through the training, validation, and evaluation process. 

The grades and comments should only be used to further feed the ISD process and should not, 

under any circumstances, be maintained in the flightcrew member’s permanent record. 

Maintaining the grades and comments in the flightcrew member’s permanent record would have 

disastrous results in the integrity of the training program.

Conclusion

The training program should include a grading scale. It should provide sufficient granularity or 

detail to be used in the ISD evaluation process.

Recommendations

The THRR ARC recommends—

 That the identifiable pilot record shall only record satisfactory/unsatisfactory completion.

 That a multi-point grading scale will be used to assess and enhance the training program.

3.1.4 INSTRUCTOR/EVALUATOR TRAINING

Background

The THRR ARC recognizes that effective validation of pilot skills can only be accomplished by 

individuals who are specifically trained to conduct instruction and validation. In addition to the 

training requirements currently applicable for initial designation as an evaluator, annual recurrent 
Flightcrew Member THRR ARC Report 16

training should be required. This training should include operator-specific ground instruction 

and observation of the evaluator conducting validation events for the certificate holder in a

line-oriented scenario.

Instructor/evaluator training is a critical part of the process and will also be developed using ISD. 

The instructor/evaluator training should have a continued professional development requirement. 

Currently this training is a onetime event. Calibration of the instructor/evaluator needs to 

provide for consistent grading across the instructor staff. This is important for effective 

program evaluation.

Conclusion

The creditability of any training program is only as good as the personnel who are entrusted to 

conduct the training, validation, and evaluation. To be effective in the role, it is essential specific 

focused training be provided and the currency be maintained through required recurrent training.

Recommendations

The THRR ARC recommends—

 That instructors and evaluators attend annual recurrent training that includes methods for 

ensuring consistent grading of training and validation events.
Flightcrew Member THRR ARC Report 17

3.2 ISSUE 2 RATIONALE

The initial and recurrent testing requirements for pilots, including the rigor and consistency of 

testing programs such as check rides.

3.2.1 SECTION INTRODUCTION

The evaluation of a pilot’s KSAs involves the observation of the pilot’s performance during a set 

of tasks (maneuvers) that were developed using an ISD process. This observation measures the 

pilot’s performance against a set of defined standards of performance. For multi-crew 

operations, it is important to also measure the performance of all flightcrew member positions 

during this evaluation. In response to this issue, the following discussion and recommendations 

will focus on five areas:

1. Flight Proficiency Standards;

2. Validating the Task;

3. Flightcrew Composition for Validation;

4. Validation of PM Skills; and

5. Line Checks.

3.2.2 FLIGHT PROFICIENCY STANDARDS

Background

The current flight proficiency standards used when evaluating a pilot is based on their grade of 

pilot certificate (that is, commercial or ATP practical test standard (PTS)). This approach is 

problematic in that it can allow different demonstrated levels of proficiency between the PIC and 

SIC, particularly in the case of turbojet aircraft. Although the standards of performance defined 

by these two FAA PTS sources are similar when compared task by task, some variation does 

exist. The use of multiple FAA sources for pilot performance may not be the ideal solution.

Conclusion

One level of proficiency for air carrier pilots can best be served by holding all pilots to a 

common standard of performance, regardless of duty position and operating regulation.

Recommendation

The THRR ARC recommends—

 That when an SIC is required, the SIC shall be held to the same minimum performance 

standard as the PIC .
Flightcrew Member THRR ARC Report 18

3.2.3 VALIDATING THE TASK

Background

The current evaluations of air carrier pilots’ flight proficiency are based on a list of tasks 

(maneuvers) that may be defined by multiple regulatory sources: FAA Practical Test Standards 

or part 121, subpart H; part 135; and FAA guidance materials. To accurately validate the skills 

of a pilot conducting air carrier operations, the validation should be tailored to the operational 

environment in which the pilot operates. For example, validation of instrument approach 

procedure skills should be based on those instrument procedures used by the air carrier in the 

actual conditions in which it uses them. This will allow operators to focus on areas that need 

special emphasis.

Although all air carriers essentially perform the same basic flight tasks—ground operations, 

takeoffs, instrument approaches, and landings—the required list of maneuvers to which a pilot 

must be trained, checked, and tested both initially and annually, will be unique to each specific 

air carrier.

CONCLUSION

The operational analysis using an ISD approach to training requirements will determine the tasks 

for each air carrier.

RECOMMENDATION

The THRR ARC recommends—

 That flight proficiency be validated by using a maneuvers list developed by the air 

carrier, who must use an ISD approach to job task analysis. The maneuvers list should be 

based on the air carrier’s specific equipment and operation, and may be supplemented by 

any FAA or industry requirements.

3.2.3 VALIDATION OF PILOT MONITORING SKILLS

Background

Typically, flight operations involve PIC and SIC performing both pilot flying (PF) and 

PM duties.

Each flightcrew member must carefully monitor the aircraft’s flight path and systems, and 

actively cross-check the actions of each other. Effective flightcrew member monitoring and 

cross-checking can be the last line of defense; when a flightcrew member can detect an error or 

unsafe act, this may break the chain of events leading to an accident scenario. Conversely, when 

this layer of defense is absent, the error may go undetected, leading to adverse safety 

consequences.
Flightcrew Member THRR ARC Report 19

A strict adherence to procedures associated with each flightcrew member position is essential. 

Observation is needed during normal and abnormal procedures in order to check the proper 

division of duties between the PF and PM.

Conclusion

Each pilot should demonstrate PM duties sufficient to determine compliance with and knowledge 

of aircraft procedures and company SOPs, including normal and abnormal procedures.

Recommendations

The THRR ARC recommends—

 That training, validation, and evaluation of flight proficiency in multi-pilot operations 

include PM skills.

3.2.4 LINE CHECKS

Background

As currently regulated, no certificate holder may use any person as a PIC of an aircraft unless, 

within the preceding 12 calendar months, that person has passed a line check in which he or she 

satisfactorily performs the duties and responsibilities of a PIC in one of the types of airplanes he

or she is to fly. The flight check shall be given by an approved check airman or by an 

FAA inspector. The line check shall consist of at least one flight over a typical part of the 

certificate holder’s route, over a foreign or Federal airway, or over a direct route, and shall 

include takeoffs and landings at one or more representative airports.

While operation of the aircraft is not insignificant, the purpose of the line check is not to 

determine competency to safely operate the specific aircraft type. The objective is instead to 

observe the PIC perform the duties and responsibilities associated with the conduct of a revenue 

flight. Another benefit of the line check is for a check airman or an FAA inspector to observe, 

evaluate, and provide data to the in-flight operations of a certificate holder within the total 

operational environment of the air transportation system.

Line checks may be conducted by a check airman or FAA inspector occupying the approved 

observer’s seat in the cockpit (jumpseat) or, if the check airman is fully line qualified, from 

either pilot seat while also serving as a required flightcrew member.

The line check program for part 121 operations continues to be effective and appropriate and no 

recommendations for change are proposed.
Flightcrew Member THRR ARC Report 20

When conducting line checks in part 135 on-demand air carrier operations, there are significant 

constraints on the opportunity for a check airman or FAA inspector to observe actual line 

operations. Most aircraft used in on-demand operations do not have approved observer seat and the 

check airman or FAA inspector must sit in the passenger compartment. In general, observations 

from the passenger compartment are not adequate for monitoring cockpit commutations and 

flightcrew member interaction. The short notice scheduling and lack of published routes make the 

scheduling of the line check very difficult. The majority of the line checks conducted in on-demand 

air carrier operations are flights dispatched for the sole purpose of accomplishing that check and are 

not conducted in revenue service with passengers.

Unlike scheduled air carrier operations where these checks are conducted during revenue 

operations, the on-demand operator must bear the total cost of the check. To counter the economic 

disadvantages, the on-demand operator will attempt to comply with the regulatory requirements 

with the minimum flight time possible. Line checks conducted in the aircraft under these 

circumstances do not provide the check airman or inspector with many of the opportunities to 

comply with the intent of the regulation, such as operations in high altitude airspace. In many 

instances, operators are not authorized adequate numbers of check airmen and therefore must rely 

on FAA inspectors to conduct some line checks. Due to limited FAA resources, many 

on-demand operators are having difficulty in scheduling qualified FAA inspectors to conduct the 

line checks.

In 2003, the FAA chartered the Part 135/125 ARC to recommend changes to those operations.6 

In preparation for that ARC, the FAA solicited public comments. It received more comments 

from the public regarding line check concerns than any other issue raised for the ARC. The 

Part 135/125 ARC achieved full consensus on recommendations that would allow for greater 

flexibility in scheduling the line check, less dependence on FAA resources, and more 

importantly, would also encourage conducting line checks as part of line operations. The final 

recommendations from the ARC were to: provide for an alternative means of compliance

(MOC) for the initial line check in the form of initial operating experience (IOE); extend the 

authority for line check airmen to similar aircraft for which they may not be qualified; and 

provide for an alternative MOC for recurrent line checks by using a simulator-based Line 

Operational Program.

Conclusion

The line check program for part 121 operations continues to be effective and appropriate.

To achieve a more effective and efficient line check program for part 135 operators, the line 

check program should be conducted more like those conducted in part 121 scheduled air carrier 

operations with regard to evaluation of the flightcrew in more realistic operational environments.

The following recommendations would allow for greater flexibility in the scheduling of the 

check, less dependence on FAA resources and more importantly, to encourage the conduct of 

line check as part of line operations (For information about implementing these 

recommendations, see page 20A).

 

6

 See 68 FR 5488 and FAA Order 1110.135.
Flightcrew Member THRR ARC Report 20A

For operations using aircraft that do not have an approved observer’s seat and do not have 

FAA-approved line check airmen, the operator may accomplish the PIC line check by any of the 

following methods:

 A flight in actual revenue service using a contract line check airman from another 

operator, or

 A Line Operational Simulation (LOS) conducted in a Full Flight Simulator using a 

contract check airman from a 14 CFR part 142 training center or an FAA Inspector.

For operations using aircraft that do have an approved observer’s seat and do not have FAA 

approved Line Check Airmen, the operator may accomplish the PIC line check by any of the 

following methods:

 A flight in actual revenue service using a contract line check airman from another 

operator or an FAA inspector occupying the observer seat, or

 An LOS conducted in a Full Flight Simulator using a contract check airman from a 

part 142 training center or an FAA Inspector.

For operations using aircraft that do have an approved observer’s seat and do have 

FAA-approved line check airmen, the operator may accomplish the PIC line check by any of the 

following methods:

 A flight in actual revenue service using a company line check airman or an FAA inspector 

occupying the observer seat.

 A flight in actual revenue service using a company line check airman occupying a pilot 

seat and performing the duties as a SIC.

 Using a company line check airman who is qualified in one of the aircraft operating by 

the certificate holder but not qualified on the specific aircraft on which the PIC is being 

checked. This check is conducted from the observer seat. The purpose of the line check 

is to determine the PIC is competent to operate safely within the ATC system. While 

operation of the aircraft is not insignificant, the purpose of the line check is not to 

determine competency to safely operate the specific aircraft type. For this reason, it is 

intended that a check airman employed by the certificate holder can conduct a line check 

in any aircraft family. For example, a check airman in a Gulfstream V is authorized to 

conduct a line check in a Hawker 800, even if the check airman is not type rated nor 

qualified to operate the Hawker 800. 

 An LOS conducted in a Full Flight Simulator using a contract check airman from a part 

142 training center or an FAA Inspector on an alternating rotation with the aircraft.
Flightcrew Member THRR ARC Report 21

Recommendations

The THRR ARC recommends–

 That the line check program employed in part 121 is suitable for those operations.

 That for part 135 operations, the adoption of line check rule changes be made, as 

proposed by the Part 135/125 ARC. 
Flightcrew Member THRR ARC Report 22

3.3 ISSUE 3 RATIONALE

The optimal length of time between training events for such flightcrew members, including 

recurrent training events.

3.3.1 SECTION INTRODUCTION

It is the opinion of the THRR ARC that existing regulatory training intervals are adequate to 

ensure pilots maintain adequate KSAs necessary to safely carry out flightcrew member duties in 

commercial flight operations. However, the ARC did consider change of intervals for 

evaluation. The remaining portion of this section’s discussion deals only with the modification 

of evaluation intervals.

Although there are a number of factors that may influence the appropriate length between 

evaluation events for an individual pilot or all pilots flying for an operator, the ARC believes 

existing regulatory intervals for these events must be reduced in cases where a pilot reenters the 

operator’s training program after an extended absence, or where they have demonstrated 

unsatisfactory performance during an evaluation. This concept is described as a short-cycle

(described below).

The THRR ARC also examined whether the FAA should consider the merits of extending certain 

evaluation event intervals for operators by demonstrating one or more attributes described later 

in this section.

Operators will have to consider how to allocate available resources to adequately demonstrate to 

the FAA any or all of the characteristics listed above to apply for extending their assessment 

intervals. The more of these characteristics the operator is able to demonstrate, the greater 

assessment interval the FAA may grant, up to a maximum interval of 1 year.

Background

Frequency of evaluations center around the knowledge and tasks necessary to maintain the 

requisite higher cognitive skills needed to handle normal, non-normal, or emergency conditions 

during flight operations. For the purpose of this discussion, these higher cognitive skills are 

referred to as the KSAs needed to handle normal, non-normal, or emergency conditions.

The ARC did not consider optimum intervals for operators evaluating under an AQP since 

intervals for those operators are derived from an ISD analysis. These recommendations for 

optimum intervals are only appropriate for operators training under part 135, subpart G, and/or 

part 121, subpart O.

The ARC currently does not believe there is sufficient justification for extending any interval 

beyond 12 months. An operator who wishes to extend intervals beyond 12 months should 

develop and implement an AQP as currently permitted under part 121, subpart Y.
Flightcrew Member THRR ARC Report 23

EARLY EVALUATION (SHORT-CYCLE)

The THRR ARC believes there are specific cases where existing regulatory evaluation intervals 

are not appropriate when a pilot had an extended absence from the operator’s training program, 

or where a pilot has demonstrated unsatisfactory performance during an evaluation. In these 

cases, the ARC has determined the pilot should be subject to a ―short-cycle‖ where their interval 

is shortened to one-half of the evaluation interval for the event that was unsatisfactory.

The specific methods to carry out the short-cycle concept and develop regulatory requirements 

are complex. The THRR ARC did not have time to consider all implementation issues. 

Therefore, the ARC recommends the FAA consider various options for implementing these 

concepts and publish options for public comment prior to rulemaking.

EXTENDING EVALUATION INTERVALS

The THRR ARC believes there are cases where an air carrier may be granted an extension to an 

existing regulatory evaluation interval in cases where the air carrier demonstrates certain 

operational characteristics. The ARC considered seven operational characteristics that the FAA 

could consider single or cumulative in incrementally increasing the evaluation interval up to a 

maximum of 12 months.

Note: The THRR ARC recommends any carrier wishing to extended evaluation intervals 

beyond 12 months should develop and implement AQP.

The THRR ARC considered a total of seven characteristics that could be used by an air carrier to 

justify a 1-month interval extension. An air carrier may be entitled to one or more months of 

interval extension based on the number of characteristics it demonstrates to the FAA.

Three of these characteristics were identified from the regulatory exemption process the FAA 

has previously used to grant operators regulatory relief from PIC proficiency check requirements 

of up to 12 months. This exemption, called the ―Single-Visit-Exemption,‖ has been granted to 

operators while they developed an AQP. The THRR ARC believes provisions from this 

exemption may be considered in granting interval extensions:

 Implementation of a First Look Program—SPOT Check; a 1-month interval extension for 

a § 121.441 or § 135.297 flight evaluation.

 Implementation of a line oriented flight training (LOFT) Program; a 1-month interval 

extension for a § 121.441 or § 135.297 flight evaluation.

 Implementation of an Enhanced Training Data Collection and Feedback Program; a 

1-month interval extension for a § 121.441 or § 135.297 flight evaluation.

Four other characteristics were identified, such as appropriate management staffing and oversight 

of flight operations that exceed minimum baseline regulations. Numerous air carriers realize the 

benefit(s) of ―over and above‖ processes, procedures, and programs to enhance the 

company’s safety and thus operational performance. As a result, the THRR ARC believes 
Flightcrew Member THRR ARC Report 24

operators who employ voluntary safety systems or oversee their flight operations with full-time 

management oversight, or demonstrate frequent flight operations may also be justified in 

extending their training and assessment intervals. The four voluntary safety or enhanced 

management characteristics are—

1. Implementation of an SMS; a 1-month interval extension for a § 121.441 or § 135.297 

flight evaluation.

2. Implementation of FOQA or FDM (ICAO) program; a 1-month interval extension for a 

§ 121.441 or § 135.297 flight evaluation.

Note: In considering these two characteristics, the THRR ARC realizes that most 

voluntary and mandatory safety programs are created for large part 121 air carriers. The 

approval process and operator and inspector guidance is not available to the part 135 

industry. Due to this, numerous part 135 air carriers have had to adapt ICAO-based 

safety systems to meet the requirements to operate as an international carrier compliant 

with ICAO. Since the United States also recognizes ICAO, an ICAO-based system 

should be considered to be equivalent when no specific U.S. regulation is available.

3. Enhanced Management Oversight program; a 1-month interval extension for a § 121.441 

or § 135.297 flight evaluation.

4. Demonstration of Individual Pilot Operational Tempo; a 1-month interval extension for a 

§ 121.441 or § 135.297 flight evaluation.

The THRR ARC has defined these seven characteristics into options that carriers can use to 

extend evaluation intervals up to 12 months. Suggestions for implementing these options are 

contained in Appendix D.

Recommendations

The THRR ARC recommends—

 That early evaluation (short-cycle) should be required for:

o Extended pilot absence from flight operations requiring requalification training that 

exceeds recurrent training but does not require initial training.

o Unsatisfactory performance during an assessment.

 That under specified conditions, the evaluation interval may be extended if the air carrier 

can demonstrated operational characteristics contained in one or more the following 

seven options.

o Option 1: Implementation of a First Look Program—SPOT Check; a 1-month 

interval extension for a § 121.441 or § 135.297 flight evaluation.
Flightcrew Member THRR ARC Report 25

o Option 2: Implementation of Line Oriented Flight Training Program; a 1-month 

interval extension for a § 121.441 or § 135.297 flight evaluation.

o Option 3: Implementation of an Enhanced Training Data Collection and Feedback 

Program; a 1-month interval extension for a § 121.441 or § 135.297 flight evaluation.

o Option 4: Implementation of an SMS; a 1-month interval extension for a § 121.441 

or § 135.297 flight evaluation.

o Option 5: Implementation of FOQA or FDM (ICAO) program; a 1-month interval 

extension for a § 121.441 or § 135.297 flight evaluation.

o Option 6: Enhanced Management Oversight program; a 1-month interval extension 

for a § 121.441 or § 135.297 flight evaluation.

o Option 7: Demonstration of Individual Pilot Operational Tempo; a 1-month interval 

extension for a § 121.441 or § 135.297 flight evaluation.

 That the evaluation interval is not extended beyond 12 months, unless the air carrier has 

developed an AQP program in accordance with part 121, subpart Y.
Flightcrew Member THRR ARC Report 26

3.4 ISSUE 4 RATIONALE

The best methods to reliably evaluate a flightcrew member’s mastery of aircraft systems, 

maneuvers, procedures, takeoffs and landings, and crew coordination.

3.4.1 SECTION INTRODUCTION

The evaluation of a pilot’s KSAs involves observation of the pilot’s performance during a set of 

maneuvers that were developed using an ISD process. This observation measures the pilot’s 

performance against a set of defined standards of performance. For multi-crew operations, it is 

important to measure the performance of all flightcrew member positions during this evaluation. 

In response to this issue, discussion and recommendations focus on two areas:

 The use of scenario-based evaluations.

 The use of FSTD equipment in the evaluation process.

Background

SCENARIO-BASED EVALUATION

At the end of the curriculum and for a pilot who has met standards for all tasks and validations,

the pilot must then be evaluated. The evaluation should be a scripted operational 

scenario-based event and should in addition to evaluating skills associated with accomplishing 

selected tasks, must be developed to allow for the evaluation of decision making, leadership, 

CRM, and TEM skills. If a pilot fails the evaluation, he will be remediated to standards and then 

reevaluated. If the pilot fails the evaluation, a second time it should be considered the pilot has 

failed the curriculum.

Evaluation profiles should contain an operationally relevant real-world-like situation or scenario, 

reflective of any special operational authorizations the operator has been granted, that is used to 

focus the flightcrew members into a particular subject matter.

THE USE OF FSTD EQUIPMENT IN THE EVALUATION PROCESS

Parts 121 and 135 permit certificate holders to use simulators for varying amounts of the 

training, testing, and checking required by the FAA. Use of simulators is a voluntary alternative 

to training and checking in the airplane. The only required use of an FSTD in the current 

regulations is in the windshear requirements of § 121.409(d). Using FSTD rather than aircraft 

allows for more in-depth checking in a safer environment, including the practice of critical 

emergency procedures using a broad range of scenarios to replicate virtually any possible 

situation, in every conceivable weather and environmental condition. FSTDs also provide other 

benefits, such as reducing noise, air pollution, and air traffic congestion, and conserving 

petroleum resources. This proposal addresses concerns raised by the National Transportation 

Safety Board Recommendations A–94–191 through A–94–194, which state that part 121 flight 

training and checking should be required in FSTDs wherever possible.
Flightcrew Member THRR ARC Report 27

Conclusion

The ARC strongly believes better pilot performance will result from realistic training and 

validation of pilot skills. Therefore, pilot validation events should be accomplished in 

line-oriented events in flight simulators (when simulation is available). The events should be 

structured to ensure all pilot skill tasks are validated, but flexibility should be permitted in how 

those tasks are validated. This flexibility should include the ability to validate training over 

multiple sessions rather than one check-ride event. In addition, the line-oriented validation 

should focus on operator specific requirements, specific operational challenges faced by the 

operator, or issues identified in the operator’s safety management program.

Recommendations

The THRR ARC recommends—

 That flightcrew member mastery is best evaluated by the use of an appropriate FSTD.

 That the flight proficiency be accomplished by using scenario-based evaluation.
Flightcrew Member THRR ARC Report 28

3.5 ISSUE 5 RATIONALE

Classroom instruction requirements governing curriculum content and hours of instruction. 

As discussed in the rationale for Issue 1, classroom training is integral to overall pilot training. 

Therefore, the THRR ARC members concluded the same process and procedures discussed in 

section 3.1 apply to classroom training. For information about the background, conclusion, and 

recommendations, see Section 3.1 on page 28.
Flightcrew Member THRR ARC Report 29

3.6 ISSUE 6 RATIONALE

The best methods to allow specific academic training courses to be credited toward the total 

fligh- hours required to receive an airline transport pilot certificate.

3.6.1 SECTION INTRODUCTION

Background

Today, part 121 requires an ATP certificate for pilots acting as PIC and a commercial certificate 

for SIC pilots. With the passage of Pub. L. 111-216, a pilot serving as SIC for a part 121 

air carrier will require an ATP certificate. This legislation allows the FAA to consider using 

specific academic training courses towards the ATP flight hour requirements. Under part 135, 

PICs are required to hold an ATP certificate for operations in turbojet airplanes having a 

passenger seating configuration of 10 or more. An ATP certificate is also required for a PIC 

conducting commuter operations in any multi-engine airplane.

7

 An SIC in those operations is

required to hold a commercial certificate.

The THRR ARC believes the existing ATP requirements are appropriate for obtaining that 

certificate when the pilot will be acting as the PIC of an aircraft (where FAA regulations require 

an ATP certificate for the aircraft and type of operation). For an ATP certificate required for 

pilots acting as a SIC under part 121 regulations, the ARC recommends a system providing 

academic credit, with limitations. The ARC’s position is that any pilot serving as PIC at a

part 121 air carrier must meet the minimum regulatory standard for full ATP certification, and no 

academic credit would be provided for a pilot seeking PIC privileges.

The FAA chartered an ARC to address Section 217 of Pub. L. 111-216, the FOQ ARC. The 

FOQ ARC considered what criteria would permit academic training and quality of experience to 

substitute for the existing aeronautical experience requirements required to obtain an ATP

certificate necessary to exercise SIC privileges under part 121. The FOQ ARC developed an 

aeronautical experience credit system that could be used to meet ATP aeronautical experience 

requirements, but restricted the applicability of that system to the issuance of an ATP certificate 

limited to SIC privileges in part 121 operations. In reviewing the recommendations drafted by 

the FOQ ARC, the THRR ARC believes the aeronautical experience credit system proposed is 

valid for exercising SIC privileges under part 121.

In addition to the recommendations proposed by the FOQ ARC, the THRR believes the 

following factors must be considered by the FAA:

 There is a need for any aeronautical experience credit system to be reconciled, where 

possible, with ICAO licensing standards and recommended practices and initiatives.

 

7

 See § 135.243 (a).
Flightcrew Member THRR ARC Report 30

 The ARC does not support applying an academic training credit system beyond the 

certification for ATP to act as a part 121 SIC. In the opinion of the THRR members, a 

pilot that will be upgrading from an SIC to PIC at a part 121 air carrier would need to 

meet the existing regulatory requirements for ATP. This means an SIC upgrading to PIC 

would need to meet the minimum total flight hours requirement for an ATP certificate of 

1,500 hours, as well as other specific flight hour requirements for cross-country, night, 

and instrument flying experience.8

 The recommendations submitted by FOQ ARC are specific to the certification 

requirements for a pilot who will perform SIC duties at an air carrier operating under 

part 121. The requirements for a pilot that will perform SIC duties at a part 135 

air carrier are not addressed. It is the view of the THRR ARC the certification 

requirements for pilots of part 135 air carriers are appropriate to the operations they 

conduct. The ARC therefore does not recommend any changes to the part 135 

pilot certification and experience requirements.

Conclusion

The THRR ARC concludes there is value and benefit to providing limited academic training and 

experience credits for those pilots who are required to obtain an ATP certificate in order to 

exercise SIC privileges in part 121 operations.

Recommendations

The THRR ARC recommends—

 That the FAA implement the FOQ ARC’s recommendation to establish an aeronautical 

experience credit system for an ATP certificate with SIC privileges in operations 

conducted under part 121. One member of the THRR ARC dissented from this 

recommendation.9

 That an aeronautical experience credit system not be allowed for experience 

requirements for any ATP certificate for a PIC at a part 121 or part 135 air carrier.

 That the use of an aeronautical experience credit system not be used for any part 135 

flight-hour requirements.

 

8

See § 61.159.

9 Refer to Appendix E for CAPA’s Dissent Statement submitted to the FOQ ARC.
Flightcrew Member THRR ARC Report 31

3.7 ISSUE 7 RATIONALE

Crew leadership training. 

3.7.1 SECTION INTRODUCTION

Background

There is required training for a pilot to serve as a PIC. This training largely consists of the 

performance of operational tasks and responsibilities specific to the duty station, and does not 

necessarily provide education to the PIC on his or her leadership role. CRM training, which is 

now required for all air carriers, contains some elements of the desired leadership training. It is

not designed to aid the PIC in assuming a leadership role in the aircraft and the air carrier as the 

training envisioned by the THRR ARC would.

In accordance with § 206 of Pub. L. 111-216, the FAA convened the MLP ARC to provide 

recommendations for flightcrew member mentoring, leadership, and professional development 

for part 121 air carriers. While the tasking of the MLP ARC’s deliberations and 

recommendations exceed that of the THRR’s scope, their analysis of leadership training was 

reviewed by the THRR ARC members. With some modifications, the THRR ARC concurs with 

their recommendation to require leadership training for the PIC. The ARC has made no 

assessment of the other recommendations of the MLP ARC. Additionally, while the MLP ARC 

deliberations were limited to part 121 operators, the THRR ARC is tasked with considering 

leadership training for part 135 operators as well.

The THRR ARC suggests an ISD process could potentially be used in lieu of the specific 

hour requirement recommended by the MLP ARC. The ISD method of training permits the 

air carrier to determine, based upon data and experiences, the optimal training times for pilots. 

Establishing a mandatory fixed-minimum runs contrary to the principles of training the 

THRR ARC endorses.

Further, due to the wide variety of operations and the size and scope of various air carriers, 

including those operating under part 135, a specific hour requirement could ultimately have a 

negative learning impact on the pilots. For instance in a small operation, with only a few aircraft 

and PICs, mandating a minimum of 32 hours of training, as proposed by the MLP ARC, could 

prove difficult to execute.

The THRR ARC believes a facilitated discussion among those attending the course is a key 

component to the success of this particular aspect of training. The ARC concurs with the 

MLP ARC that leadership training is not well suited to distance learning due to the value of 

the interactive group dynamic among multiple pilots. Additional items each air carrier may 

deem necessary to introduce in a leadership and command course may be suitable for 

distance learning.
Flightcrew Member THRR ARC Report 32

The FAA should, when describing how an air carrier with a small number of PICs would 

implement this training, permit flexible solutions in which PICs from multiple, similarly situated 

air carriers (that is, similar aircraft and scope/type of operations) could pool their PICs into a 

class size that would then provide the interactive group discussion dynamics envisioned.

The leadership training should be developed as a training event separate from the existing PIC 

upgrade transition syllabus. It is recommended that each air carrier develop their course to 

incorporate air carrier-specific course material. The leadership training should be divided into 

two segments. The initial segment would be completed prior to upgrade training in order to 

cover the leadership modules of the course. The second segment would need to be completed 

between 6 and 18 months after completion of the operating experience/IOE and should 

incorporate lessons learned during the new captain’s initial experiences as a captain. It should 

also reinforce the concepts covered in the initial leadership and command course.

Conclusion

Providing specific training to the PIC on his or her leadership role will increase the safety and 

professionalism of air carrier operations.

Recommendations

The THRR ARC recommends—

 That the FAA implement the MLP ARC recommendation to require leadership training 

for the PIC. An ISD process could potentially be used in lieu of the specific hour 

requirement in the MLP ARC’s recommendation.

 That leadership training be integrated into appropriate training events through the 

ISD process.
Flightcrew Member THRR ARC Report A–1

APPENDIX A—THRR ARC MEMBERS AND SUPPORT STAFF

THRR ARC MEMBERS AND AFFILIATION

Ms. Jacqueline Rosser, Co-Chair, National Air Transportation Association (NATA)

Capt. Randy Hamilton, Co-Chair, Regional Airline Association (RAA)

Mr. Timothy Beglau, Designated Federal Official, FAA

AERO-MICRONESIA

Mr. Michael Quinn

AIR LINE PILOTS ASSOCIATION, INTERNATIONAL

Mr. Bill Edmunds

Capt. Chuck Hogeman

AIR TRANSPORT ASSOCIATION

Capt. John Dudley

Capt. Dave Lynn

AIRBUS AMERICAS

Capt. Rudy Canto

ATLANTIC SOUTHEAST AIRLINES

Capt. Eric Carlson

Capt. Patrick Greene

THE BOEING COMPANY

Capt. Terry McVenes

Capt. Tom Pryde

BOMBARDIER AEROSPACE

Mr. Robert Freeman
Flightcrew Member THRR ARC Report A–2

CAE SIMUFLITE

Mr. William Campbell

Mr. Steve Hall

THE COALITION OF AIRLINE PILOTS ASSOCIATIONS

Capt. Gregory Bowen

FLIGHTSAFETY INTERNATIONAL

Mr. Rick Bedard

Mr. John Hinchey

KEY AIR

Capt. Knut Finnevolden

NATA

Mr. Dave Hewitt

RAA

Mr. Steve Briner

SUPPORT STAFF

FAA

Ms. Catherine Burnett

Mr. Christopher MacWhorter

Mr. Jeffrey Schroeder

Mr. Christopher Skully

Mr. Peter Tokarz

PAI CONSULTING

Ms. Kelly Akhund

Ms. Sarah O’Brien
THRR ARC Final Report B–1

APPENDIX B—ACRONYMS

14 CFR Title 14, Code of Federal Regulations

AC advisory circular

AQP Advanced Qualification Program

ARC Aviation Rulemaking Committee

ASAP Aviation Safety Action Program

ATP airline transport pilot

CRM crew resource management

FAA Federal Aviation Administration

FDM flight data monitoring

FOQ First Officer Qualification 

FOQA Flight Operations Quality Assurance

FSB Flight Standards Board

FSTD flight simulator training device

IATA International Air Transport Association

ICAO International Civil Aviation Organization

ILS instrument landing system

InFO Information for Operators

IOE initial operating experience

ISD Instructional Systems Design

ITQI IATA Training and Qualification Initiative

KSA knowledge, skills, and attitude

LOFT Line Oriented Flight Training

LOSA Line Operations Safety Audit

MLP Mentoring, Leadership, and Professional Development
THRR ARC Final Report B–2

NATA National Air Transportation Association

OPTEMPO operational tempo

PF pilot flying

PIC pilot in command

PM pilot monitoring

POI principal operations inspector

PTS practical test standard

RAA Regional Airline Association

SAFO Safety Alert for Operators

SIC second in command

SME subject matter expert

SMS Safety Management Systems

SOP standard operating procedures

SPOT Special Purpose Operational Training

TEM Threat and error management

THRR Training Hours Requirement Review
THRR ARC Final Report C–1

APPENDIX C—POINT GRADING EXAMPLE

SAMPLE GRADING SCALE

5–Excellent 

Performance remains well within the prescribed performance 

standards. Individual performance, management, and CRM skills are 

exemplary. 

4–Standard 

Student performance remains within the prescribed performance 

standards. Individual performance meets expectations. CRM skills 

are effective. 

3–Satisfactory 

Deviations occur from the performance standards that are recognized 

and corrected in a timely manner. Individual performance 

meets expectations. CRM skills are effective. 

REASON CODE Required 

2–Below Standard 

Deviations occur from the performance standards that are not recognized 

and corrected in a timely manner. Individual performance does not 

jeopardize the airplane. CRM skills are not completely effective. 

REASON CODE Required 

Standard Scale: 

Reason Code 

Required Generates Rollover Repeat

5–Excellent NO NO NO

4–Standard NO NO NO

3–Satisfactory YES NO NO

2–Below Standard YES YES YES

1–Unsatisfactory YES YES YES

N/A–Not Accomplished NO YES _
THRR ARC Final Report C–2

1–Unsatisfactory 

Major deviations from the performance standards occur that are not 

recognized or corrected. Individual performance could result in a 

hull loss or loss of life. CRM skills are not effective. 

REASON CODE Required 

0–Not Attempted 

For tasks not accomplished due to time constraints or equipment malfunctions, Aviation-savvy 

training systems will rollover the task(s) to the next training event unless it appears in the 

following lesson. If it is the last training day prior to the check, all training events must be 

signed off. If it is a simulator-related issue, provide some comment indicating that there was a 

simulator/device-related issue. 
THRR ARC Final Report C–3

REASON CODES

Technical 

PY Proficiency Lack of psychomotor (stick and rudder) skills.

PD Procedure Deviation in the execution of regulations and/or operator 

procedures. The intention is correct but the execution is flawed. 

A task is forgotten or skipped.

NC Intentional 

Noncompliance

Willful deviation from regulations and/or operator procedures.

AT Automation Lack of knowledge or skill in automation use.

Proficiency Errors

 Manual handling

 Lateral/vertical deviations

 Wrong airplane configuration

 Ground navigation

 Systems knowledge

 Radios/instruments

Automation

 Autopilot

 Mode control panel

 Flight Management System

 Autothrottle

 Electronic Flight Bag

 Display management

 Information management

 Inappropriate level

Procedural Errors

 Checklists

 Briefings

 Callouts

 Cross-checking

 Limitations

 Documentation

 Improper initiation

 Not completed

 Other procedural

Intentional Noncompliance

 Violations of SOP or regulations
THRR ARC Final Report C–4

Non-technical (CRM/TEM)

CM Communication Miscommunication, misinterpretation, or failure to communicate or 

solicit pertinent information. Failure to communicate the plan or 

set the tone for crew coordination.

SA Situational 

Awareness

Failure to monitor or perceive elements in the environment, the 

comprehension of their meaning, and the projection of their status 

in the near future.

DM Decision Making An error that is not standardized by regulations or operator 

procedures and that unnecessarily compromises safety. Failure to 

develop a plan.

WL Workload 

Management

Absence of actions/strategies that allow for timely and appropriate 

completion of all operational tasks. Failure to prioritize tasks 

and/or manage responsibilities.

Communication

 Pilot-to-pilot communication.

 Crew to external communication.

 Failure to communicate and 

cooperate with crew members and 

other staff in sharing information.

 Improper radio phraseology.

 Doesn’t verbalize action.

Decision Making

 Fails to establish goals and 

monitor outcomes.

 Jumps to conclusions.

 Extreme maneuvers on approach.

 Choosing to fly into adverse weather.

 Information management.

Situational Awareness

 Fails to monitor/perceive 

elements in environment and 

project them into the future.

 Inability to actively monitor and 

evaluate the operation.

Workload Management

 Fails to slow things down and 

control the pace by making time.

 Fails to prioritize tasks.

 Absence of strategies for task 

completion.

 Absence of actions for task 

completion.
THRR ARC Final Report C–1

REPEATS AND FINAL RATINGS 

Provided that there is time left in the training session, repeats allow the instructor to perform 

tasks that are not satisfactory and capture a grade for this additional task. If the grade is a 3 or 

higher, the item will not roll into the next scheduled session and repeats will not be required. 

Select the number of times you repeated the task, not the total number of times the 

task was accomplished. 

GRADING ACCURATELY 

The grades given in all courses will be evaluated on a monthly basis or when there are 

curriculum revisions driven by customer/program requests. These data are crucial in 

determining where deficiencies in curriculum exist. To make valid changes, the data must 

be reliable. Instructor observation skills of student performances are critical in determining all 

types of student performance. Even though the focus is on lower levels of performance, good 

performance averages will be used to determine proficiency levels. Inflated grades will skew the 

placement of where these gates in training occur. 
THRR ARC Final Report C–2

JOINT AVIATION AUTHORITY 5-POINT GRADING EXAMPLE10

Grade Description

5 The ideal performance under existing conditions. Anticipates and adapts easily to 

changing or unusual flight situations.

4 The aim of the exercise is safely achieved with very few minor variations from 

ideal. Performance shows smooth control of aircraft.

3 The aim of the exercise is safely achieved with frequent minor but no major 

variations from the ideal.

2 The aim of the exercise is safely achieved. Performance includes not more than 

one major variation from the ideal and may include frequent minor variations from 

the ideal.

1 The aim of the exercise is safely achieved in a rough manner. Performance includes 

more than one major variation from the ideal and indicates a level of skill or 

knowledge, which results in a marginally acceptable performance.

0 Any one of the following will result in an assessment of fail:

 The aim of the exercise is not completed.

 There is an insufficient level of knowledge to ensure safety.

 The aim of the exercise is completed, but at the expense of using unsafe 

airmanship and/or handling errors.

 Dangerous aircraft handling, requiring assistance from the examiner.

 Tolerances specified in the flight test standards are exceeded.

Written remarks are required when awarding a flight test exercise a mark of 2 or less. The 

remarks should be clear and concise, and in the case of an exercise assessed as—

 Grades 1 and 2 reflect the major variation(s) from the acceptable performance for the 

exercise as outlined in the appropriate flight test standards; or

 Grade 0 reflects the appropriate item or items that result in an assessment of fail as listed 

in the grading scale section of the flight test standard(s).

 

10 Joint Aviation Authority. Flight Examiners Manual: Aeroplane and Helicopter, version 17, May 2011.
THRR ARC Final Report C–3

During a flight test, it is sometimes difficult to write clear and concise remarks. It is 

recommended that examiners use notes made during the flight test to complete a final copy of 

the flight test report. This provides the examiner with the opportunity of referencing the 

appropriate flight test standards while writing final comments.

Was Performance Acceptable?

YES

Were There Errors?

Yes

Were the Errors Major?

Yes

How Many Errors?

More Than 1

Rating: 1

Only 1

Rating: 2

No

No

No

Rating: 0

Rating: 5

How Many Minor Errors?

Frequent

Rating: 3

Few

Rating: 4
THRR ARC Final Report C–4

TRANSPORT CANADA 4-POINT GRADING SCALE11

When applying the 4-point scale, award the mark that best describes the weakest element(s) 

applicable to the candidate’s performance. Remarks to support mark awards of 1 or 2 must 

link to a safety issue, a qualification standard (performance criteria), or an approved technique 

or procedure.

4 Above 

standard

Performance remains well 

within the qualification 

standards and flight 

management skills are 

excellent.

 Performance is ideal under existing 

conditions.

 Aircraft handling is smooth and precise.

 Technical skills and knowledge exceed the 

required level of competency.

 Behavior indicates continuous and highly 

accurate situational awareness.

 Flight management skills are excellent.

 Safety of the flight is assured. Risk is 

well mitigated.

3 Standard Minor deviations occur 

from the qualification 

standards and 

performance remains 

within prescribed limits.

 Performance meets the recognized standard,

yet may include deviations that do not 

detract from the overall performance.

 Aircraft handling is positive and within 

specified limits.

 Technical skills and knowledge meet the 

required level of competency.

 Behavior indicates that situational 

awareness is maintained.

 Flight management skills are effective.

 Safety of the flight is maintained. Risk is 

acceptably mitigated.

 

11 Transport Canada. TP 14277 – Pilot Examiner Manual, 3rd ed., April 2010.
THRR ARC Final Report C–5

2 Basic 

standard

Major deviations from the 

qualification standards 

occur, which may include

momentary excursions 

beyond prescribed limits,

but these are recognized 

and corrected in a 

timely manner.

 Performance includes deviations that detract 

from the overall performance, but are 

recognized and corrected within an 

acceptable time frame.

 Aircraft handling is performed with limited 

proficiency and/or includes momentary 

deviations from specified limits.

 Technical skills and knowledge reveal 

limited technical proficiency and/or depth 

of knowledge.

 Behavior indicates lapses in situational 

awareness that are identified and corrected 

by the pilot/crew.

 Flight management skills are effective but 

slightly below standard. Where applicable, 

some items are only addressed when 

challenged or prompted by other 

crewmembers.

 Safety of the flight is not compromised. 

Risk is poorly mitigated.

1 Below 

standard

Unacceptable deviations 

from the qualification 

standards occur, which 

may include excursions 

beyond prescribed limits 

that are not recognized or 

corrected in a timely 

manner

 Performance includes deviations that 

adversely affect the overall performance, are 

repeated, have excessive amplitude, or for 

which recognition and correction are 

excessively slow or nonexistent, or the aim 

of the task was not achieved.

 Aircraft handling is rough or includes 

uncorrected or excessive deviations from 

specified limits.

 Technical skills and knowledge reveal 

unacceptable levels of technical proficiency 

and/or depth of knowledge.

 Behavior indicates lapses in situational 

awareness that are not identified or 

corrected by the pilot/flightcrew.
THRR ARC Final Report C–6

 Flight management skills are ineffective, 

unless continuously challenged or prompted 

by other flightcrew members.

 Safety of the flight is compromised. Risk is 

unacceptably mitigated.

FLIGHTSAFETY 4–POINT GRADING SCALE12

1–Proficient (to PTS Standard)

2–Normal progress (anticipate proficient within normal course structure)

3–Needs additional training (simulator or ground school beyond course structure)

4–Unsatisfactory

TASKS/TOPICS THAT ARE NOT REQUIRED TO BE TRAINED TO PROFICIENCY:

T–Trained procedure only

D–Discussed task/topic

C–Complete (LOFT Only)

FlightSafety encourages instructor remarks by task or by training session. For grades of 1 and 2, 

these remarks are usually the location of training and a summary of the scenario; grades of 

3 and 4 require remarks addressing each objective standard in the PTS not met, as well as any 

target areas for emphasis.

 

12 FlightSafety International. Learning Center Operations Manual, Revision 5.2, 22 March 2011, Section 5.11
THRR ARC Final Report D–1

APPENDIX D—APPLYING ISSUE 3’S RECOMMENDATIONS

IMPLEMENTING THE EARLY EVALUATION CONCEPT (SHORT-CYCLE)

The ARC recommends that if a pilot has had an extended absence from flight duties requiring 

additional training beyond the normal recurrent training (but less than the training required for 

initial qualification) then that pilot shall be subject to a short cycle for the appropriate evaluation.

The ARC also recommends that if a pilot demonstrates unsatisfactory performance while 

completing one of the following events, he or she shall be subject to a short-cycle for the 

respective event:

 Part 135

o Recurrent Ground Training (either written or oral test) § 135.351(b).

o Competency Check § 135.297(b).

o Instrument Proficiency check § 135.297.

o Line check § 135.299.

o ATP Certificate Practical Test § 61.153(g).

o Aircraft Type Rating Practical Test § 61.63(d).

 Part 121

o Recurrent Training–Ground § 121.427(b)(1) oral and/or written test.

o Proficiency Check § 21.441(a).

o Proficiency Training §§ 121.441(a) and 121.409.

o Line check § 121.440.

o LOFT §§ 121.441(a) and 121.409.

o ATP Practical Test § 61.153(g).

o Aircraft Type Rating Practical Test § 61.63(d).

Pilots who have demonstrated unsatisfactory performance must receive the necessary training to 

achieve performance standards. Once this training is completed, pilots must be reevaluated 

before being allowed to resume flight duties. After resuming flight duties, the pilot will return 

for a short-cycle of the event deemed unsatisfactory that is one-half the normal interval stipulated 

in the operator’s training program. Once completing the short-cycled evaluation satisfactorily, 
THRR ARC Final Report D–2

the pilot returns to the normal interval approved for the operator. If unsatisfactory performance 

is again demonstrated during the short-cycled training and KSA assessment, then the pilot is 

subject to another short cycle interval.

In cases where the training and assessment interval is an odd number, the interval is one-half

rounded up to an even number.

 Example A: An operator uses a 12-month recurrent training and assessment interval for 

the competency check required under § 135.293(b). A pilot who demonstrates 

unsatisfactory performance during KSA assessment would be required to complete 

recurrent training and assessment 6 months after the unsuccessful KSA assessment and 

necessary remedial training. If during the short-cycled recurrent training the pilot does 

not require any additional training beyond the normal events and satisfactorily 

demonstrates performance standards, then they would return to the normal 

12-month interval.

 Example B: An operator is allowed by the FAA to use a 9-month training and assessment 

interval for a proficiency check (§ 121.441(a)). A pilot who undergoes recurrent training 

but demonstrates unsatisfactory performance during KSA assessment would be required 

to complete recurrent training and assessment 5 months after the unsuccessful KSA 

assessment and necessary remedial training (one-half of 9 months would be 4.5 months,

rounded up to 5 months). If during the short-cycled recurrent training the pilot does not 

require any additional training beyond the normal events and satisfactorily demonstrates 

performance standards, then they would return to the normal 9-month interval.

The existing ―base-month‖ concept, where training may be scheduled 1 month prior to the base 

month or 1 month after the base month, should apply in these provisions.

IMPLEMENTING EVALUATION EXTENSIONS

The ARC recommends that the FAA offer one of seven options (either individually or combined) 

to operators who wish to extend training and assessment intervals by implementing enhanced 

safety or training programs. The FAA should consider the provisions for each option listed 

below in considering an extended interval.

The options include the following:

Option 1 Requirements: Implementation of a First Look Program—SPOT Check; 

a 1- month interval extension for a § 121.441 or § 135.297 flight evaluation.

Prior to annual training there will be an initial simulator session consisting of a 2-hour SPOT 

session. The simulator instructor conducting the period will complete a blind (deidentified) data 

collection form on each flightcrew member. Four fixed ―first look‖ maneuvers will be identified 

on an aircraft specific basis by the director of training for FAA approval.

Four fixed maneuvers should be selected on the following criteria:

 At least one maneuver will be flown with a diminished aircraft system.
THRR ARC Final Report D–3

 At least one maneuver will be flown with diminished avionics.

 At least one maneuver will be flown engine out.

 At least one approach/missed approach will be flown manually (without autopilot).

A sampling of all other maneuvers in this Appendix (called ―variable maneuvers‖) will be 

selected by the simulator instructor, along with any other aircraft specific training items as 

determined by the director of training.

A complete evaluation utilizing the concepts of fixed/variable maneuvers will be administered 

during the SPOT simulator session. The evaluation itself will be for a total of 2 hours (with 

PF duties split about 1.5 hours for the PIC and .5 hours for the SIC) and conducted in the 

flightcrew concept mode. Flightcrew members are accountable for and individually evaluated on 

all respective PF and PM duties, as applicable during the entire period. The instructor 

completing the SPOT evaluation will complete a blind data collection on each 

flightcrew member as described later.

A minimum of eight maneuvers will be completed during the check: six of these events must be 

completed with the PIC flying (four fixed, two variable) and two of the fixed maneuvers with the 

SIC flying.

Proficiency Check maneuvers—Sample of maneuvers—

 PIC PF; SIC PM—

o Fixed:

 Instrument landing system (ILS) with flight control failure,

 Raw data ILS,

 Takeoff with an engine failure, and

 Full procedure non-precision manually flown approach,

o Variable:

 Aborted takeoff, and

 ILS to lowest minimums.

 SIC PF; PIC PM—

o Fixed:

 Full procedure missed approach, and

 ILS with engine-out.
THRR ARC Final Report D–4

For all tasks, maneuvers, or procedures included on first look evaluations, the certificate holder 

will identify the performance standards to be employed. Current FAA practical test advisory 

materials and pertinent flight standardization board reports for a given type of aircraft will be 

employed for guidance in developing specifications for performance standards. The 

specifications for performance standards will be submitted to the air carrier’s principal 

operations inspector (POI) for approval.

Each event on the SPOT check will be individually graded based on the following criteria:

 For successful completion of an event, SPOT check data collection will not require 

retraining and retesting. Two unsatisfactory items will be retrained during the remainder 

of recurrent training:

o Event flown to standards, and

o Event unsatisfactory.

 Level of flightcrew member coordination displayed—

o Flightcrew member coordination met standards, and

o Flightcrew member coordination was unsatisfactory.

 Data will be aggregated for a report to the FAA with 2 points for each event as described 

above: 1 for successful completion and 1 for flightcrew member coordination, resulting 

in 16 points available for each SPOT check.

Data will be collected and analyzed during the training until all pilots have cycled through a 

SPOT evaluation. If, at the end of the period, the SPOT program indicates an aggregate average 

of greater than 90 percent ones (1), the time between checking may be extended for 1 month, up 

to the 12-month maximum. Approval for a 1-month extension is determined by the POI upon 

review of data. Collection and analysis of data will be continued and the results will be 

presented to the certificate holder’s POI annually to determine if the checking extension should 

be reapproved.

Option 2 Requirements: Implementation of LOFT Program; a 1-month interval 

extension for a § 121.441 or § 135.297 flight evaluation.

Air carrier training programs must include a 2-hour, full flightcrew member complement 

simulator period dedicated to LOFT, as outlined in the line operational simulation advisory 

circular (AC) 120–35C. This LOFT session will be conducted separately from the normal 

simulator training program. The LOFT scenario will be constructed to focus on human factors 

skills as well as other specific training objectives appropriate to the air carrier.
THRR ARC Final Report D–5

For all tasks, maneuvers, or procedures included in LOFT scenarios, the certificate holder will 

identify the performance standards to be employed. Current FAA practical test advisory 

materials and pertinent flight standardization board reports for a given type of aircraft will be 

employed for guidance in developing specifications for performance standards. The 

specifications for performance standards will be submitted to the POI for approval.

LOFT will be a no-jeopardy event. The period cannot result in an unsatisfactory grade, although 

additional LOFT may be given if deemed necessary. The LOFT will be videotaped and 

reviewed at the end of the period to facilitate the debriefing session.

Instructors conducting the LOFT will complete one blind data form on the entire flightcrew:

 Successful completion of LOFT:

o All LOFT events met standards, or

o Event was unsatisfactory.

 Level of CRM Displayed:

o CRM met standards, or

o CRM was unsatisfactory.

Because LOFT and CRM are key elements in the training program, a LOFT working group 

will be assembled each year consisting of pilot representatives from each aircraft type and 

coordinated by the air carrier’s director of training and CRM working group. The team will 

determine the annual CRM theme and design LOFT scenarios that complement and reinforce 

that theme. Recurrent CRM training will use the theme as the main topic of classroom 

discussion, and aircraft types will tailor the basic scenario for specific use LOFT. LOFT 

design will adhere to AC 120–35, as amended, for general guidance and comply with the 

following requirements.

 Annual LOFT scenarios and the CRM theme must always be complementary.

 LOFT scenarios (and the CRM theme) must be varied and updated annually.

 Identical LOFT scenarios cannot be used in consecutive training sessions for a 

given pilot.

 Flightcrew member substitution stated in AC 120–35C for recurrent LOFT will 

be utilized.
THRR ARC Final Report D–6

Data will be collected and analyzed during the training until all pilots have cycled through a 

LOFT scenario. If, at the end of the period, the LOFT program indicates an aggregate average 

greater than 90 percent ones (1), the time between checking may be extended for 1-month, up to 

the 12-month maximum. Collection and analysis of data will continue and the result will be 

presented to the certificate holder’s POI annually to determine if the checking extension should 

be reapproved.

Option 3: Implementation of an Enhanced Training Data Collection and Feedback 

Program; a 1-month interval extension for a § 121.441 or § 135.297 flight 

evaluation.

Each pilot is required to attend training required under 14 CFR at least annually. Annual training 

for each pilot will be a combination of distributed training and onsite training that must be 

accomplished during the pilot’s 3-month eligibility period (base month, plus or minus 1 month). 

This will include:

 Annual ground school instruction, which includes mandated training under

parts 121 and 135 in recurrent school subject lessons.

 Annual simulator flight training to proficiency for each flightcrew member.

Annual simulator events will be conducted to the maximum extent possible in a 

flightcrew member-oriented session, with a flightcrew consisting of a PIC and SIC.

For all tasks, maneuvers, or procedures included in proficiency evaluations, the certificate holder 

will identify the performance standards to be employed. Current FAA practical test advisory 

materials and pertinent flight standardization board reports for a given type of aircraft will be 

employed for guidance in developing specifications for performance standards. The 

specifications for performance standards will be submitted to the POI for approval.

The certificate holder will develop a proficiency rating scale for the training sessions from 

1–5 for approval by their POI.

Examples of these include:

 Met standards on first attempt.

 Failed to meet standards, debriefed, no additional simulator training required.

 Failed to meet standards on first attempt. Met standards on second attempt after debrief 

and retraining, additional program hours not required.

 Training to proficiency required additional time over and above program requirements.

 Unsatisfactory, unable to meet standards.

Each instructor will complete a data sheet on completion of each training date evaluating each 

task or maneuver. Each evaluator will grade each task or maneuver based on the above criteria.
THRR ARC Final Report D–7

Data on pilot performance conducted under this part shall be captured by the certificate holder 

and reported to the FAA POI. Performance evaluations will be accomplished in accordance with 

the following categories and provisions above.

 By aircraft type (according to the criteria in AC 120–53).

 By flightcrew position (PIC or SIC). Each flightcrew member’s data shall be reported as 

a single identifiable record. Appropriate identifiers will be used to distinguish flightcrew 

position in order to permit tracking individual performance, where possible, through

first-look evaluation (where appropriate) using proficiency training and evaluation. The 

certificate holder will report additional training sessions beyond the normally 

programmed training for any flightcrew member.

 By individual task, maneuver, or procedure, as applicable.

 By type of evaluation, as follows:

o First-look (if applicable),

o Proficiency training, and

o Proficiency evaluation.

 Rating scale data shall be obtained for each task, maneuver, or procedure.

 For items requiring multiple attempts to acquire proficiency, the number of repetitions 

required. Rating scale data shall be obtained for each repetition of a proficiency task, 

maneuver, or procedure.

The certificate holder shall provide the FAA with deidentified raw data on a monthly basis, in 

accordance with POI requirement.

The monthly report will:

 Document the certificate holder’s own analysis of data acquired under this part.

 During the second and subsequent years of data collection, incorporate comparisons of 

data between equivalent periods of the preceding years.

 Identify any trends, common problem areas, or potential deficiencies.

 Include a description of corrective steps taken or warranted.

Implementation of this program, when approved by the certificate holder’s POI, may be used to 

increase the period between evaluations by 1 month, period not to exceed 1 year.
THRR ARC Final Report D–8

Option 4: Implementation of an SMS; a 1-month interval extension for a § 121.441 

or § 135.297 flight evaluation.

If the air carrier has implemented an SMS (FAA or ICAO recognized), a 1-month interval 

extension for a §§ 121.441 or 135.293 flight evaluation is approvable.

Option 5: Implementation of FOQA or FDM (ICAO) program; a 1-month interval 

extension for a § 121.441 or § 135.297 flight evaluation.

If the air carrier has a FOQA or FDM (ICAO) program in place, a 1-month interval extension for 

a §§ 121.441 or 135.297 flight evaluation is approvable.

Option 6: Enhanced Management Oversight program; a 1-month interval 

extension for a § 121.441 or § 135.297 flight evaluation.

If a full 14 CFR part 119 qualified management team (including safety manager for part 135 

air carrier) is managing more than 75 percent of its work days and actually occupying a cockpit 

for less than 25 percent of work days, a 1-month interval extension for a §§ 121.441 or 135.297 

flight evaluation is approvable.

Option 7: Demonstration of Individual Pilot Operational Tempo; a 1-month 

interval extension for a § 121.441 or § 135.297 flight evaluation.

For the purpose of this proposal, operational tempo (OPTEMPO) is a term used to 

describe operational attributes that impact a flightcrew member’s KSAs with respect to 

aircraft operations.

Studies indicate that skill and knowledge decay is directly related to the frequency of use or 

practice of a given skill. Several studies cited by Ms. Monica Martinussen and 

Mr. David R. Hunter, authors of Aviation Psychology and Human Factors, substantiate this as 

referenced below.

 Skill decay is defined as a loss or decay of trained or acquired skills or knowledge after 

periods of nonuse. The inverse is intuitive: as trained/acquired skills or knowledge are 

exercised; abilities improve, or at least skill decay is prevented.

 Difficulties with skill decay are exacerbated by the current generation of cockpit 

automation, which tends to place pilots in a passive, monitoring mode.

 Some skills decay faster than others. Memories and cognitive skills tend to be lost much 

faster than motor skills.

 Basic visual flight skills remain intact for up to 36 months, while skills relating to 

instrument flight procedures fall below acceptable levels within 12 months for about half 

the pilots in the study.
THRR ARC Final Report D–9

This led to the supposition that frequency of activity by a pilot in critical phases of flight should 

be considered when determining the optimum training interval between recurrent training events. 

The THRR ARC recommends giving a 1-month assessment interval extension for each of the 

following OPTEMPO thresholds, not to exceed a 12-month interval for any individual pilot:

 180 or more take-offs and landings within the previous 6 months, and

 36 instrument approaches in actual instrument meteorological conditions within the 

previous 6 months.

The operator must establish a tracking system to substantiate that their pilots meet 

these thresholds.

This assessment interval credit recognizes that pilots who fly more frequently than others should 

sustain their motor and cognitive skills better than those pilots who fly less often. However, any 

pilot that fails an assessment would be required to enter a ―short-cycle‖ as described earlier due 

to unsatisfactory assessment. After satisfactorily completing the short-cycle requirements, the 

pilot may return to the previous extended interval based on OPTEMPO factors if he or she meets 

the prescribed thresholds.

Options 1-7 may be used only if the following requirements are met:

 The pilot successfully passed the previous two proficiency checks.

 The pilot is single-airplane (fleet) qualified.

 The pilot is a full-time employee.

 An FAA-approved training program for the air carrier is used.

In all cases, data collection and analysis requirements necessary to comply with the above 

safety programs listed above must substantiate that extending training intervals beyond current 

regulatory requirements is appropriate.
THRR ARC Final Report E–1

APPENDIX E—DISSENTING STATEMENT

CAPA was also a member of the FOQ ARC and submitted the following dissenting statement to 

specific recommendations made by that ARC. To the extent that the THRR ARC discussed and 

made recommendations based upon the FOQ ARC’s recommendations for an academic 

credit system, CAPA also dissented to the THRR ARC’s recommendations. CAPA provided a 

copy of its prior statement to serve as its dissenting opinion for the THRR ARC recommendation 

related to academic credit.
THRR ARC Final Report E–2
THRR ARC Final Report E–3
THRR ARC Final Report E–4
THRR ARC Final Report E–5
THRR ARC Final Report E–6
THRR ARC Final Report E–7
THRR ARC Final Report E–8
THRR ARC Final Report E–9
THRR ARC Final Report E–10
THRR ARC Final Report F–1

APPENDIX F—CHARTER
THRR ARC Final Report F–2
THRR ARC Final Report F–3
THRR ARC Final Report F–4