# Flight

c rew M ember Training Hours Requirement Review Aviation Rulemaking Committee Report from the THRR ARC

May 23, 2011THRR ARC Final Report i



## TABLE OF CONTENTS

Table of Contents

................................ ................................ ................................ ........................... i
Executive Summary

................................ ................................ ................................ ..................... iii

1.0 Recommendations

................................ ................................ ................................ .................. 1

1.1 Issue 1

................................ ................................ ................................ ................................ ... 1

1.2 Issue 2

................................ ................................ ................................ ................................ ... 2

1.3 Issue 3

................................ ................................ ................................ ................................ ... 3

1.4 Issue 4

................................ ................................ ................................ ................................ ... 4

1.5 Issue 5

................................ ................................ ................................ ................................ ... 4

1.6 Issue 6

................................ ................................ ................................ ................................ ... 4

1.7 Issue 7

................................ ................................ ................................ ................................ ... 5

2.0 Background

................................ ................................ ................................ ............................ 6
2.1 ARC Composition

................................ ................................ ................................ ................ 6

2.2 Clarification of Scope

................................ ................................ ................................ ........... 6

2.3 Methodology

................................ ................................ ................................ ........................ 7

2.4 Competency

- Based Training

................................ ................................ ................................ 7

2.5 Not Considered

................................ ................................ ................................ ................... 10

3.0 Rationale

................................ ................................ ................................ ............................... 11

3.1 Issue 1 Rationale

................................ ................................ ................................ ................ 11

3.2 Issue 2 Rationale

................................ ................................ ................................ ................ 17

3.3 Issue 3 Rationale

................................ ................................ ................................ ................ 22

3.4 Issue 4 Rationale

................................ ................................ ................................ ................ 26

3.5 Issue 5 Rationale

................................ ................................ ................................ ................ 28

3.6 Issue 6 Rationale

................................ ................................ ................................ ................ 29THRR ARC Final Report ii

3.7 Issue 7 Rationale

................................ ................................ ................................ ................ 31

Appendix A

— THRR ARC Members an d Support Staff

................................ ..................... A

– 1

THRR ARC Members and Affiliation

................................ ................................ ................... A

– 1

Support Staff

................................ ................................ ................................ .......................... A

– 2

Appendix B

—Acronyms

................................ ................................ ................................ ........... B

– 1
Appendix C

— Point Grading Example

................................ ................................ .................. C

– 1
Sample Grading Scale

................................ ................................ ................................ ............. C

– 1

Joint Aviation Authority 5

- Point Grading Example

................................ ............................... C

– 2

Transport Canada 4

- Point Grading Scale

................................ ................................ ................ C

– 4

FlightSafety 4

– Point Grading Scale

................................ ................................ ........................ C

– 6

Appendix D

—

Applying Issue 3’s Recommendations

................................ ........................... D

– 1
Implementing the Early Evaluation Conc ept (Short

- Cycle)

................................ .................. D

– 1

Appendix E

— Dissenting Statement

................................ ................................ ........................ E

– 1
Appendix F

— Charter

................................ ................................ ................................ ............... F

– 1THRR ARC Final Report iii



## EXECUTIVE SUMMARY

The Federal Aviation Administration ( FAA ) chartered the Flight c rew M ember Training Hours Requirement Review (THRR) Aviation Rulemaking Committee (ARC) to develop the report required by Secti on 209 of the Airline Safety and FAA Extension Act of 2010 (P ub

. L. 111

– 216). The THRR ARC charter appears in its entirety in Appendix F

.
The THRR ARC was given three tasks, which are described below.

I. To assess and make recommendations concerning

—
(1) The best methods and optimal time needed for flightcrew members of Title 14, Code of Federal Regulations ( 14 CFR ) part 121 and 135 air carriers to master aircraft systems, maneuvers, procedures, takeoffs and landings, and flightcrew coordination;

(2) The i nit ial and recurrent testing requirements for pilots, including the rigor and consistency of testing programs such as check rides;

(3) The optimal length of time between training events for such flightcrew members, including recurrent training events;

(4) The best methods to reliably evaluate a flightcrew member ’ s mastery of aircraft systems, maneuvers, procedures, takeoffs and landings, and crew coordination;

(5) Classroom instruction requirements governing curriculum content and hours of instruction;

(6) The best methods to allow specific academic training courses to be credited toward the total flight hours required to receive an airline transport pilot (ATP) certificate; and

(7) Crew leadership training.

II. To consider the scalability of recommendations to add ress the needs of small businesses

.

III. To develop regulatory recommendations as may be required

. During e arly deliberations

, the THRR ARC recognized that the aviation training paradigm needed to shift to achieve substantial changes to training /checking , and b y extension, safety. This need is clearly illustrated in a report of the International Air Transport Association’s (IATA) Training and Qualification Initiative :

― For decades, the content of flightcrew training [programs] has remained unchanged according t o regulation whilst aircraft design and reliability have evolved dramatically. Regulatory prescriptions for flightcrew training and checking are based on events, which are now highly improbable in aircraftTHRR ARC Final Report iv

designed to meet modern standards. Training [pro grams] are consequently burdened with items, which do little to mitigate real risks or enhance safety. ‖ 1

Understanding it was necessary to provide methods to move beyond traditional prescriptive training if more effective learning and assessment a re to occ ur , t he THRR ARC investigated the field of learning systems design. Ultimately

, the ARC accepted the need to move towards competency

- based training , utilizing evidence (data and analysis) to develop the best t raining

. Today, t he optimum method to accompl ish this would be through the group of methodologies referred to as I nstructional S ystem s Design (ISD)

.
ISD methodologies are based upon analysis of the workforce, the equipment to be operated

, and the operating environment . V ariables must be considered, including the number of pilots

, fleet size , and the diversity of fleet and staffing levels. Once the principles of ISD are utilized, training programs can be developed that address the common cohort of the individual operators and allow some flexibility f or meeting the training needs of the individual.

The THRR ARC has concluded the FAA

- approved training program known as Advanced Qualification Program (AQP) provides the best current method for training, assessing

, and validating flightcrew members for comm ercial air carrier operations. D ue to practical limitations, AQP is not a readily achievable, or in some cases even a possible training solution for all operators.
AQP has provided the industry with principles that have proven effective using the followin g three pillars:

1. Competency

- Based Training

. This principle relies on t raining that is delivered and evaluated based upon the amount of training each individual needs to ―master‖ the required tasks

. Competency may be achieve d at different rates for differ ent people

, and the amount of elapsed time between training events that competency is maintained also varies individually and must be considered. Competency

- based training varies from prescriptive training because it recognizes one size cannot fit all.

2. Tr ain to Proficiency

. Inherent to any competency

- based training program is the principle of ―train to proficiency‖

. Train to proficiency provides for the variations of individual learning rates. It also allows for additional exposure to a task until the p ilot is proficient, and competency is achieved. Training to proficiency drives the quantity of training required to gain competency

3. Data Collection and Feedback

. This is p ossibly the most important element to ensure the effectiveness of this approach . T his is where the skill and knowledge of the individual and flight crew can be assessed

, and the health of the training program is monitored. The data element provides a continuous feedback loop allowing for rapid adjustments when performance indicators war rant action and helps determine the correct intervals between training events

.

1 IATA Training and Qualification Initiative (ITQI) Report 2009 Fall EditionTHRR ARC Final Report v

In the competency

- based training approach, the amount of time required to master an element is not fixed . Instead, it is determined by how long it takes for the individual to r each and/or maintain competency with regard to knowledge, skills

, and attitudes (KSA)

. These KSAs are established through detailed task analysis of the aircraft and environment in which the pilot will be operating. Likewise, the evaluation and validation criteria are based on a demonstration of mastery of individual and flight crew KSAs

.
The THRR ARC was challenged by the need to provide scalable options that acknowledge the vast diversity of air carrier size and scope that exists in the industry, partic ularly between the large scheduled airlines of p art 121 and the small

, on

- demand operators of p art 135. Any ―universal‖ training recommendations made by the ARC need to be appropriate for the smallest single

- pilot, single

- aircraft operator and the largest air carriers with fleets comprising hundreds of large jet aircraft flown by thousands of pilots.

T he THRR ARC evaluated the seven issues to determine the optimal training solutions, but recognized that a prescriptive regulatory minimum must be maintained in order to accommodate the needs of the smallest air carriers that lack the resources and data pool necessary for the development of an ISD

- driven training program

. T he ARC did believe the minimum regulatory standard should be reevaluated to determine ai r carriers’ ability to produce an acceptable level of safety. In light of the many recommendations made by the ARC, the FAA should alter the minimum standard for smaller air carriers where prudent to improve pilot training and also provide mechanisms with in the regulations, policy

, and guidance that enable and encourage the smaller air carriers to adopt ISD

- driven methods where able.

The THRR ARC used the basic principles of ISD to address Issues 1 through 5. Issues 6 and 7 were the subject of separate FA A

- established ARCs. The ARC reviewed presentations by subject matter experts ( SME ) from those ARCs and used that information as the foundation for its response to those issues.THRR ARC Final Report 1



## 1.0 RECOMMENDATIONS

1.1 I SSUE 

### 1

The best methods and optimal time needed f or flightcrew members of Title 14 Code of Federal Regulations (CFR) part 121 and 135 air carriers to master aircraft systems, maneuvers, procedures, takeoffs and landings, and crew coordination

. (See section 3.1 on page 11 for the rationale.)

1.1.1 R ECOM MENDATIONS

The THRR ARC recommends

—

 T h at a process

- based training development system

, such as ISD

, be used to develop a training course tailored to meet the requirements of each operator

.

 That scenario

- based training be used as often as possible throughout the curriculum.

 T hat as a minimum, each flightcrew member receive s annual recurrent training without the option to replace the training event with a checking event

, as currently permitted

.

 T hat i nstructors and evaluators attend annual recurrent training t hat includes methods for ensuring consistent grading of training and validation events

.

 That t he identifiable pilot record shall only record satisfactory/unsatisfactory completion.

 That a multipoint grading scale will be used to assess and enhance the trai ning program.

 Th at a new ARC be established to identify the methodology and data necessary to support
a process

- based training development system for those operators who do not currently use such a system or are without access to sufficient data to maintai n such a system (that is, small air carriers). Examples of data that should be considered include:

o Aircraft data monitoring programs, such as flight data monitoring (FDM) or Flight Operations Quality Assurance ( FOQA ) ;

o Aviation Safety Action Program ( ASAP ) ;

o Operator implemented safety reporting systems;

o S afety M anagement S ystem (S MS ) and/or Internal Evaluation Program information;

o Line Operations Safety Audit ( LOSA ) ;

o Pilot survey and/or critique information; andTHRR ARC Final Report 2

o Any other available regulator or industry inf ormation such as Information for Operators (InFO), Safety Alert for Operators (SAFO), and Flight Standards Board (FSB) reports

.

 Th at an ARC be established to evaluate and clarify the rules governing the relationship between part 121 and part 135 air carri ers and 14 CFR part 142 training centers.



### 1.2 I SSUE 

### 2

The i nitial and recurrent testing requirements for pilots, including the rigor and consistency of testing programs such as check rides

. (See section 3.2 on page 17 for the rationale.)

1.2.1 R ECOMMEND ATION S

The THRR ARC recommends

—

 That when a s econd in c ommand ( SIC ) is required, the SIC shall be held to the same minimum performance standard as the p ilot in command ( PIC )

.

 That flight proficiency be validated by using a maneuvers list developed by the

a ir carrier

, who must us e an ISD approach to job task analysis. The maneuvers list should be based on the air carrier’s specific equipment and operation and may be supplemented by any FAA or industry requirements.

 That training, validation, and evaluation of flight proficiency in multi

- pilot operations include pilot monitoring (PM) skills.

 That t he line check program employed in part 121 is suitable for those operations.

 That f or part 135 operations, the adoption of line check rule changes be made, as propo sed by the P art 1 3 5/ 1 2 5 ARC

. 2

.

2 Federal Docket No. FAA

- 2002

- 13923THRR ARC Final Report 3



### 1.3 I SSUE 

### 3

The optimal length of time between training events for such flightcrew members, including recurrent training events

. (See section 3.3 on page 22 for the rationale.)

1.3.1 R ECOMMENDATION S

 That early evaluation ( short

- cycle) should be required for:

o Extended pilot absence from flight operations requiring requalification training that exceeds recurrent training but does not require initial training.

o Unsatisfactory performance during an assessment.

 That under specifi ed conditions, the evaluation interval may be extended if the air carrier can demonstrate operational characteristics contained in one or more of the following seven options.

o Option 1: Implementation of a First Look Program

— Special Purpose Operational Tr aining (SPOT) Check; a 1

- month interval extension for a § 121.441 or § 135.297 flight evaluation.

o Option 2: Implementation of Line Oriented Flight Training Program; a 1

- month interval extension for a § 121.441 or § 135.297 flight evaluation.

o Option 3: Im plementation of an Enhanced Training Data Collection and Feedback Program; a 1

- month interval extension for a § 121.441 or § 135.297 flight evaluation.

o Option 4: Implementation of a n SMS; a 1

- month interval extension for a § 121.441

or § 135.297 flight ev aluation.

o Option 5: Implementation of FOQA or FDM International Civil Aviation Organization (ICAO) program; a 1

- month interval extension for a § 121.441 or § 135.297 flight evaluation.

o Option 6: Enhanced Management Oversight program; a 1

- month interval e xtension for a § 121.441 or § 135.297 flight evaluation.

o Option 7: Demonstration of Individual Pilot Operational Tempo; a 1

- month interval extension for a § 121.441 or § 135.297 flight evaluation.

 That the evaluation interval is not extended beyond 12 mon ths, unless the air carrier has developed an AQP program in accordance with part 121, subpart Y.THRR ARC Final Report 4



### 1.4 I SSUE 

### 4

The best methods to reliably evaluate a flightcrew member ’ s mastery of aircraft systems, maneuvers, procedures, takeoffs and landings, and crew co ordination

. (See section 3.4 on
page 26 for the rationale.)

1.4.1 R ECOMMENDATION S

The THRR ARC recommends

—

 That flightcrew member mastery be evaluated through the use of an appropriate flight simulator training device ( FSTD )

.

 T hat t he flight proficiency be verified using scenario

- based evaluation.



### 1.5 I SSUE 

### 5

Classroom instruction requirements governing curriculum content and hours of instruction

. (See section 3.5 on page 28 for the rationale.)

1.5.1 R ECOMMENDATION

T he THRR ARC members concluded the sa me process and procedures for Issue 1 apply to classroom training. Therefore, the recommendations for Issue 5 are included in those for Issue 1. (See section 3.1 on page 11 for the rationale.)



### 1.6 I SSUE 

### 6

The best methods to allow specific academic trai ning courses to be credited toward the total flight

- hours required to receive an airline transport pilot certificate

. (See page 29 for the rationale.)

1.6.1 R ECOMMENDATION S

The THRR ARC recommends

—

 That the FAA implement the First Officer Qualification ( FOQ) ARC’s recommendation to establis h an aeronautical experience credit system for an ATP certificate with SIC privileges in operations conducted under part 121

. One member of the THRR ARC dissented from this recommendation.

 That an aeronautical experien ce credit system not be allowed for experience requirements for any ATP certificate for a PIC at a part 121 or part 135 air carrier

.THRR ARC Final Report 5

 That the use of an aeronautical experience credit system not be used for any part 135 flight

- hour requirements

.



### 1.7 I SSUE 

### 7

Crew leadership training. (See page 31 for the rationale.)

1.7.1 R ECOMMENDATION

The THRR ARC recommends

—

 Th at the FAA implement the Flight C rew m ember Mentoring, Leadership, and Professional Development (MLP) ARC recommendation to require leadership tra ining for the PIC

. A n ISD process could potentially be used in lieu of the specific hour requirement in the MLP ARC ’s recommendation

.

 That l eadership training be integrated into appropriate training events through the ISD process.Flightcrew Member THRR ARC Report 6



## 2.0 BACKGROUND

2.1 ARC C OMPOSITION

The THRR ARC was composed of subject matter experts from 13 organizations (see Appendix A for a complete list)

. Collectively

, the ARC members brought significant levels of experience to the deliberations in air carrier operations; the deve lopment, implementation, and management of pilot training and qualification programs; and the establishment of pilot training and qualification standards at the domestic and international level. The ARC also had access to representatives from the FAA to a nswer any specific regulatory o r guidance issues that arose during deliberations.



### 2.2 C LARIFICATION OF 

### S COPE

While the THRR ARC’s tasking was to address ―flightcrew members

, ‖ which by definition include s flight engineers and navigators, the ARC deliberati ons and response only address the flight deck crew ( that is

, pilots )

. This adjustment to the scope was made because the language from the legislation and charter implied that the ARC’s focus is intended to be on pilot training. Although flight engineers and navigators are included in the definition of flightcrew member, their actual cohort is extremely small compared to pilots . Thus, the ARC felt that the overall impact on safety could better be served by spending its limited time and resources to addres s pilot training, the the principle area of concern

. In general, the THRR ARC members believe the existing regulatory minimum requirements have served the industry well, improvements are possible, and in some instances, necessary.

One of the biggest challe nges for the THRR ARC was addressing part s 121 and 135 simultaneously. The difference in operating philosophies sometimes made it difficult to find a common solution. Ultimately, the ARC chose to recommend the optimal solution for each issue, which often required adjusting standards to the best practice of the more robust guidance. In several instances, it was difficult for the ARC members to envision how the FAA would apply the THRR ARC recommendations to many part 135 air carriers, particularly those c lassified as

s ingle

- p ilot or s ingle

- PIC operators. Where practical

, this report notes the difficulty in applying certain recommendations to part 135 air carriers and suggest s that the FAA undertake separate rulemaking, ideally via an ARC, to address conce rns the THRR ARC was unable to consider.

With regard to the challenges faced when attempting to address the varied operations that exist under p art 135, the FAA convened the Part 135/125 ARC in 2005

, which was specifically tasked with reviewing and recomme nding changes to p art 135 . These changes include d a complete review of the training and checking requirements of those operators

, with consideration given to the small business aspects prevalent in that industry segment. The THRR ARC did not have suffici ent time to review and comment on the extensive recommendations made by that ARC

, and therefore has no specific comment on them. The THRR ARC does believe the FAA should review those recommendations in conjunction with its own , as appropriate, when consid ering new training standards for p art 135.Flightcrew Member THRR ARC Report 7



### 2.3 M ETHODOLOGY

The bulk of the THRR ARC report provides guidance on I ssues 1 through 5 . These issues address the level of training, validation

, and evaluation needed to ensure a pilot in
p art 121 3 or 135 flight operations has mastery of the KSAs to operate in the complex air carrier

arena , and that the intervals between recurrent training events do not compromise safety. While the ARC tasking requires distinct responses to each issue, these five issues share en ough commonality to allow deliberations and analysis to be conducted in three groupings:

 Training Courseware

,

 Intervals of Training , and

 Evaluation and Validation

. Accordingly, the THRR ARC developed three subcommittees, one for each group. Each subcommit tee was then assigned the specific issues(s) most closely associated with its area

. The issues were divided as follows :

 Training Courseware: Issue 1 and Issue 5

 Intervals of Training: Issue 3

 Evaluation and Validation: Issues 2 and Issue 4

Issues 6 and 7 were the subject of two separate ARCs : the FOQ ARC and MLP ARC

. While the THRR ARC c harter requires independent deliberation and response, the ARC membership agreed that it was prudent to be familiar with and build on the previous ARCs’ work. The THR R ARC received a series of briefs on the FOQ and MLP ARC s ’ work and used that as a foundation to develop its responses.



### 2.4 C OMPETENCY 

### - B ASED 

### T RAINING

Following extensive discussion on the optimal method to train and evaluate pilots, the THRR ARC members a greed upon a training system established on a train

- to

- proficiency standard. Currently, training is prescriptive in the specific instruction delivered and assessed

, and in the hours required for each training event. This training system provides air carr iers with little flexibility to adjust to the identified needs of an individual pilot or group of similarly situated pilots. The train

- to

- proficiency philosophy recognizes that the optimal time required to master a particular task can vary

, and permits ta iloring the training to the recipient. T his philosophy is the over

- arching element for how the THRR ARC approached its discussions and subsequent recommendations.

3 Discussion of Part 121 was restricted to the regulations current a s of May 5, 2011.Flightcrew Member THRR ARC Report 8

Currently, this philosophy is best associated with the ISD concept

, which is explained in fu rther detail in Section 2.4.1

. Although relying heavily upon the current use of ISD

, the THRR ARC does not intend to restrict the use of newer concepts for training program design that may arise in the future. To the contrary, the ARC members decided not to focus on a singular method of program design . The goal is to encourage air carriers to develop training programs that produce pilot s who are ―trained

- to

- proficiency

. ‖ Today, that output is best captured by the ISD process

, so it was used by the ARC a s a model system.

The ISD curriculums are designed to provide a level of training that will allow a normally progressing pilot of average skills in a target group to complete all training successfully in the planned hours and days. Pilots following these curriculums are continually assessed on all tasks each day in each phase of training. At the completion of each phase

, a pilot ’ s training is validated and his or her performanc e must meet standards. If at any time a pilot’s performance is assessed as not meeting the standards for a task during any curriculum period of training or phase validation

, the pilot will be remediated to standards before progressing.

During the remediation process

, a pilot may be provided additional training periods for that phase of training. Remedial training will continue as long as the pilot is making reasonable progress. When a pilot fails to make progress at a reasonable rate

, he or she should be considered to have failed the curriculum.

At the end of the curriculum for a p ilot who has met standards for all tasks and validations

, the pilot must then be e valuated. The e valuation should be a scripted operational scenario

- based event , and must be developed to allow for the evaluation of decision

- making, leadership, crew resour ce management ( CRM )

, and threat and error management ( TEM ) skills. If a pilot fails the e valuation

, he will be remediated to standards and then r e

- e valuated. If the pilot fails the e valuation a second time

, the pilot should be considered to have failed the curriculum.

2.4.1 ISD P ROCESS

ISD is a systematic process of developing training curriculum. It is used and applied in many different industries and work settings. ISD encompasses adult learning theory, educational psychology, human factors engineer ing, and cognitive psychology.

The ISD process involves the following steps:

 Analysis:

o Develop a job or task list

.

o Analyze the list to determine its essential skill and knowledge requirements

.

o Determine which skills and knowledge requirements must be train ed and/or tested

.

o Develop objectives, with conditions and standards that define acceptable performance

.Flightcrew Member THRR ARC Report 9

 Design:

o Allocate the objectives defined in the analysis step to the appropriate curricula

.

o Sequence and group the objectives within a curriculum for opt imum training and evaluation

.

o Develop an evaluation strategy, and develop test items for the objectives

.

 Development:

o Develop instructional materials ( for example, lesson plans, handouts, videos, and tests) for the curriculum developed in the design phase

.

o Determine the appropriate use of available technologies

.

o Train instructors to use the new curriculum materials

.

o Test the training materials by having the instructors teach a small group of students and modify the m aterials if necessary

.

 Implementation:

o De liver the curriculum to the students

.

o Collect instructor and student critiques during and after the delivery process

.

o Collect student performance data

.

 Evaluation:

o Perform c ontinual analysis of performance data

, and use the results from that analysis to mo dify the course

.

o The following are possible sources of data for course evaluation :

 FOQA

,

 ASAP

,

 LOSA

,

 Pilot critique

,

 Any available data such as : I n FO, SAFO, and FSB reports

.Flightcrew Member THRR ARC Report 10

In t he ISD model

, evaluation takes place throughout the development process. Whi le performing the five steps , the developer will constantly be evaluating the process and resulting materials. It is a method of quality control to ensure the final result is as appropriate to the learning situation as possible.

Annual recurrent training should be established by performing risk analysis to determine the critical events that need to be trained. In addition to establishing event criticality, event currency must also be determined. Currency applies to events flightcrew members perform on a regular basis ; annual training may not be necessary. The ISD process will be used to determine event criticality and currency.

Flightcrew members should then be trained on critical events and any other FAA or industry

- required events on an annual basis. The current regulations allow for a ―checking event‖ to be substituted for annual training. The THRR ARC believes this practice is detrimental to pilot performance and that a nnual training should be required.



### 2. 5 N OT 

### C ONSIDERED

The following training and evaluation events were considered by the THRR ARC , but were deemed outside the scope of its charter:

 Transportation Security Administration t raining requirements ( Title 49, Code of Federal Regulations ( 49 CFR ) , part 1500)

 Hazardous m aterials t raining (49 CFR, part 172 )Flightcrew Member THRR ARC Report 11



## 3.0 RATIONAL E

3.1 I SSUE 

### 1 R ATIONAL E

The best methods and optimal time needed for flightcrew members of Title 14 Code of Federal Regulations (CFR) part 121 and 135 air carriers to master aircraft systems, maneuvers, procedures, takeoffs a nd landings, and crew coordination.

3.1.1 S ECTION I NTRODUCTION

A grasp of the evolution of flight training is integral to this response

. Traditional pilot training consist s of several days or weeks of classroom lecture. The trainee then receive s some fo rm of procedures training in an aircraft or FSTD , followed by a checking event

.

T he following quote

, taken from a report of the IATA Training and Qualification Initiative 4

, helps illustrate the direction of this evolutionary process , a direction that will lead to higher level s of training quality if it is followed :

― In addition to the wealth of accident and incident reports, flight data collection and analysis offers the possibility to tailor training programs to meet real risks. The aim is to identify and train the real skills required to operate, whilst addressing any threats presented by the evidence collected. If we fail to make a substantial change to the philosophy of airline pilot training, we will face an increasing gap between training and reality

. It is time to create an effective bridge between safety data and the development of training solutions. At the same time, we must embed robust methodologies for the handling of real events and the development of key pilot skills, both technical and

non

- technical, into those training solutions

. ”

The use of technology to meet training needs in all professional fields is well grasped and is no less available to aviation training. Exploiting technology to match the correct training tool to a specific train ing objective improves overall training quality and reduces the cost for learning and increased availability of training to a broader base of air carrier pilots.

The industry has a wealth of safety

- related evidence at its disposal, including accident and i ncident reports and LOSA, FOQA, and ASAP data

. In addition to safety data, the industry also has training data. Analyzing this data through a formalized SMS process provides an opportunity to influence and advance the philosophy of flightcrew training, b y weighing risks and determining which situations are more likely to happen. There is strong evidence these programs have contributed significantly to the reduction in aircraft accident and incidents.

4 IATA Training and Qualification Initiative (I TQI) Report 2009 Fall EditionFlightcrew Member THRR ARC Report 12

At this time, these programs are generally limited to large air carriers with fleets and pilot populations large enough to produce a viable data sample. 5 Smaller operators need access to this type of data in order to properly conduct the ISD evaluation process.

U ltimately

, the success of any training program is highly dependent on the competency of it s instructors. Historically

, there has not been an appropriate level of criticality for instructor/evaluator training within the industry , and often this training is a onetime event. Instructor/evaluator traini ng is a critical part of the process and will also be developed using a
process

- based training development system

, such as ISD.

To fully develop a response to this issue, the following discussion and recommendations will focus on three areas:

1. Pilot Trainin g ,
2. Grading , and

3. Instructor/ Evaluator Training

.

3.1. 2 P ILOT T RAINING
Background Considering the variability and scope of part 121 and part 135 flight operations, it is impractical to create a prescriptive list of tasks and optimal time that adequately addr ess all certificate holder training and qualification requirements. Typical variables include the following:

 Experience of the flightcrew member;

 Training resources available/utilized;

 Scope of the certificate holder ’ s operation (that is, long haul intern ational, short haul domestic, single pilot operation, and associated threats);

 Type and complexity of the aircraft;

 Number of pilots with respect to the number of available data points ; and

 Fleet mix, including variants.

5 It is possible that creative solutions could allow smaller air carriers (and to some degree

, training centers) to create a sufficient population to support these programs or variations thereof by combining data across multi ple air carrier sources. The THRR ARC believes the value imparted by these data programs is sufficient to pursue these unique solutions. One example of a multi

- carrier data collaboration program is the Medallion Program that has seen much success in Alas ka.Flightcrew Member THRR ARC Report 13

The ARC endorses an analytical appr oach ( ISD ) to training development characterized by the use of a systematic process of analysis, design, development, implementation

, and evaluation to define the best training methods tailored to meet the requirements of each operator. The optimum time r equired to complete training will be determined through the process.

T he ISD process should lead to a strategy of training pilots to established standards . Once a pilot has demonstrated the ability to perform a maneuver or task to standard within that tra ining cycle

, there is no longer any need to assess that maneuver or task , and that pilot is considered to have ―mastered‖ it . It is no t necessary to evaluate each and every task a pilot must perform. The evaluation recommendations are further discussed i n section 3

. 1.4.

The ISD process should apply to all FAA

- required training curriculums . The recurrent training curriculum is established through risk assessment to determine task criticality and establish the appropriate annual training requirements.

Hist orically, the Federal Aviation Regulations have allowed air carriers the option to substitute a checking event for a training event. When this option is exercised by the air carrier, pilots are deprived of the opportunity to improve their skills and learn new skills through the interaction with a highly trained instructor pilot.

Scenario

- based training incorporates profiles that include standard operating procedures ( SO P)

, normal and abnormal emergency checklists, aircraft performance, CRM

, and TEM

. Profi les should contain an operationally relevant real

- world

- like situation or scenario , reflective of any special operational authorizations the operator has been granted

. These profiles should be used to focus the students on a particular subject matter.

All training should integrate the use of current and future advanced technologies and flight training equipment, including full flight simulators. Air carriers should utilize a suite of equipment matched on the basis of analysis to the training requirements at any given stage of a curriculum. Judicious analysis determines the appropriate type and level of FSTD to use.

The training process should integrate aircraft systems ; training

, including normal, abnormal, and emergency procedures ; CRM ; and TEM . Most of these training topics have typically been considered classroom training , but training has been proven to be most effective when conducted with a combination of classroom training , computer

- based training, training devices, simulator s or aircraft, and othe r technologies.

I t is important to have qualified flight crew members occupy each seat during training and checking events. When an air carrier conducts its own training and checking, the best practice for flight crew pairing is PIC and SIC

. If logistics do not allow the pairing of a PIC with an SIC for training or checking, the following are acceptable alternatives:

 Same position (PIC/PIC) or (SIC/SIC), as long as the flight crew members have been specifically trained in the tasks that are ―seat dependent

. ‖ For example, if a n SIC is being used as a substitute for the PIC, that SIC must be trained in the specific PIC
seat

- depend e nt tasks.Flightcrew Member THRR ARC Report 14

 PIC or SIC and air carrier certificate holder qualified flight crew member

. After lengthy discussions, it became apparent t o the THRR ARC members the current regulations do not adequately address many issues between air carriers and part 142

- approved training centers. Some of these issues include, but are not limited to:

 C heck airman/instructor training

,

 T raining program deve lopment

,

 T raining program maintenance/revision

,

 F lightcrew member pairing

,

 M ultiple training providers used by one operator

, and

 M ulti

- operator concurrent training (same training program, different air carrier using).

Conclusion The best training methods w ill be determined through the use of an ISD process. The same process will determine the optimal time. Scenario

- based training applied in conjunction with the most appropriate training device provides for a highly effective learning experience.

Recommend ations

The THRR ARC recommends

—

 That a process

- based training development system

, such as ISD , be used to develop a training course tailored to meet the requirements of each operator

.

 That scenario

- based training be used as often as possible throughout the curriculum.

 That as a minimum, each flightcrew member receive s annual recurrent training without the option to replace the training event with a checking event as currently permitted

.

 That a new ARC be established to identify the methodology and data nece ssary to support a process

- based training development system for those operators who do not currently use such a system or are without access to sufficient data to maintain such a system (that is, small air carriers). Examples of data that should be consi dered include:

o Aircraft data monitoring programs, such as flight data monitoring (FDM) or FOQA ;

o ASAP ;

o Operator implemented safety reporting systems ;

o SMS and/or Internal Evaluation Program information ;Flightcrew Member THRR ARC Report 15

o LOSA ;

o Pilot survey and/or critique information ; and

o Any other available regulator or industry information such as I n FO, SAFO, FSB, Flight Safety Foundation , and Air Charter Safety Foundation reports

.

 Th at an ARC be establish ed to evaluate and clarify the rules governing the relationship between parts 121 and 1 35 air carriers and part 142 training centers.

3.1. 3 G RADING
Background The practice of p ass/ f

ail and s atisfactory/ u nsatisfactory grading may have been influenced by certain FAA f

orms and job aids used by air carriers to develop their own proficiency chec k grading system, which remains valid for the purpose of assessing overall performance of the flightcrew member.

Mutli

- point grading scales for tasks and maneuvers have shown to be better for purpose s of data analysis. This g ranularity provides better ass essment of the pilot’s training progress and the strengths and weaknesses, and trends of the program over time (examples of 4

- and 5

- point grading scale s are shown in Appendix C ). Instructors and evaluators should make comments about the flightcrew member ’s progress through the training, validation

, and evaluation process. The grades and comments should only be used to further feed the ISD process and should not

, under any circumstances, be maintained in the flightcrew member’s permanent record. Maintain ing the grades and comments in the flightcrew member’s permanent record would have disastrous results in the integrity of the training program.

Conclusion The training progra m should include a grading scale

. It should provide sufficient granularity or det ail to be used in the ISD evaluation process.

Recommendations

The THRR ARC recommends

—

 Th at th e identifiable pilot record shall only record satisfactory/unsatisfactory completion.

 That a multi

- point grading scale will be used to assess and enhance the trai ning program.

3.1. 4 I NSTRUCTOR / E VALUATOR T RAINING
Background

The THRR ARC recognizes that effective validation of pilot skills can only be accomplished by individuals who are specifically trained to conduct instruction and validation. I n addition to the training requirements currently applicable for initial designation as an evaluator, annual recurrentFlightcrew Member THRR ARC Report 16

training should be required. This training should include operator

- specific ground instruction and observation of the evaluator conducting validation even ts for the certificate holder in a line

- oriented scenario.

Instructor/evaluator training is a critical part of the process and will also be developed using ISD. The instructor/evaluator training should have a continued professional development requirement

. Currently this training is a onetime event. Calibration of the instructor/evaluator needs to provide for consistent grading across the instructor staff. This is important for effective
program evaluation

.
Conclusion The creditability of any training p rogram is only as good as the personnel who are entrusted to conduct the training, validation

, and evaluation. To be effective in the role

, it is essential specific focused training be provided and the currency be maintained through required recurrent tra ining.

Recommendations

The THRR ARC recommends

—

 That i nstructors and evaluators attend annual recurrent training that includes methods for ensuring consistent grading of training and validation events.Flightcrew Member THRR ARC Report 17



### 3.2 I SSUE 

### 2 R ATIONALE

The initial and recurrent test ing requirements for pilots, including the rigor and consistency of testing programs such as check rides.

3.2.1 S ECTION I NTRODUCTION

The evaluation of a pilot’s KSAs involves the observation of the pilot’s performance during a set of tasks (maneuvers) tha t were developed using an ISD process

. This observation measures the pilot’s performance against a set of defined standards of performance. For multi

- crew operations, it is important to also measure the performance of all flight crew member positions duri ng this evaluation. In response to this issue, t he following discussion and recommendations will focus on five areas:

1. Flight Proficiency Standards ;

2. Validating the Task ;

3. Flightc rew Composition for Validation ;

4. Validation of PM Skills ; and

5. Line Checks

.

3.2. 2 F LIGHT P ROFICIENCY S TANDARDS

Background The current flight proficiency standards used when evaluating a pilot is based on their grade of pilot certificate ( that is

, c ommercial or ATP practical test standard (PTS) ) . This approach is problematic in that i t can allow different demonstrated levels of proficiency between the PIC and SIC , particularly in the case of turbojet aircraft

. Although the standards of performance defined by these two FAA PTS sources are similar when compared task by task, some variat ion does exist. The use of multiple FAA sources for pilot performance may not be the ideal solution.

Conclusion One level of proficiency for air carrier pilots can best be served by holding all pilots to a common standard of performance , regardless of dut y position and operating regulation

.
Recommendation

The THRR ARC recommends

—

 That when an SIC is required, the SIC shall be held to the same minimum performance standard as the PIC

.Flightcrew Member THRR ARC Report 18

3.2. 3 V ALIDATING THE T ASK

Bac kground

The current evaluations of air carr ier pilots’ flight proficiency are based on a list of tasks (maneuvers) that may be defined by multiple regulatory sources: FAA Practical Test Standards

or part 121

, subpart H ; part 135 ; and FAA guidance materials. To accurately validate the skills of a pilot conducting air carrier operations, the validation should be tailored to the operational environment in which the pilot operates. For example, validation of instrument approach procedure skills should be based on those instrument procedures used by t he air carrier in the actual conditions in which it use s them. This will allow operators to focus on areas that need special emphasis.

Although all air carriers essentially perform the same basic flight tasks

— ground operations, takeoffs, instrument approa ches

, and landings

— the required list of maneuvers to which a pilot must be trained, checked

, and tested both initially and annually, will be unique to each specific air carrier.

C ONCLUSION

The operational analysis using an ISD approach to training requirem ents will determine the tasks for each air carrier.

R ECOMMENDATION

The THRR ARC recommends

—

 T hat flight proficiency be validated by using a maneuvers list developed by the air carrier

, who must use an ISD approach to job task analysis. The maneuvers list should be based on the air carrier’s specific equipment and operation, and may be supplemented by any FAA or industry requirements.

3.2. 3 V ALIDATION OF P ILOT M ONITORING S KILLS

Background Typically, flight operations involve PIC and SIC performing both pil ot flying ( PF ) and PM duties.

E ach flightcrew member must carefully monitor the aircraft’s flight path and systems, and actively cross

- check the actions of each other. Effective flight crew member monitoring and

cross

- checking can be the last line of defen se; when a flight crew member can detect an error or unsafe act, this may break the chain of events leading to an accident scenario. Conversely, when this layer of defense is absent

, the error may go undetected, leading to adverse safety
consequences.Flightcrew Member THRR ARC Report 19

A st rict adherence to procedures associated with each flight crew member position is essential. O bservation is needed during normal and abnormal procedures in order to c heck the proper division of duties between the PF and PM

.
Conclusion Each pilot should dem onstrate PM duties sufficient to determine compliance with and knowledge of aircraft procedures and company SOPs

, including normal and abnormal procedures.

Recommendations

The THRR ARC recommends

—

 That training, validation, and evaluation of flight profici ency in multi

- pilot operations include PM skills.

3.2. 4 L INE C HECKS

Background As currently regulated, no certificate holder may use any person as a PIC of an aircraft unless, within the preceding 12 calendar months, that person has passed a line check in which he or she satisfactorily performs the duties and responsibilities of a PIC in one of the types of airplanes he

or she is to fly. The flight check shall be given by an approved check airman or by an FAA inspector. The line check shall consist of at least one flight over a typical part of the certificate holder’s route, over a foreign or Federal airway, or over a direct route

, and shall include takeoffs and landings at one or more representative airports.

While operation of the aircraft is not insign ificant, the purpose of the line check is not to determine competency to safely operate the specific aircraft type. The objective is instead to observe the PIC perform the duties and responsibilities associated with the conduct of a revenue flight. Anoth er benefit of the line check is for a check airman or an FAA inspector to observe

, evaluate , and provide data to the in

- flight operations of a certificate holder within the total operational environment of the air transportation system.

Line checks may be conducted by a check airman or FAA inspector occupying the approved observer’s seat in the cockpit (jumpseat) or, if the check airman is fully line qualified, from either pilot seat while also serving as a required flightcrew member.

The line check program for part 121 operations continues to be effective and appropriate a nd no recommendations for change are proposed.Flightcrew Member THRR ARC Report 20

When conducting line checks in p art 135 on

- demand air carrier operations, there are significant constraints on the opportunity for a check ai rman or FAA inspector to observe actual line operations. Most aircraft used in on

- demand operations do not have approved observer seat and the check airman or FAA inspector must sit in the passenger compartment. In general, observations from the passenge r compartment are not adequate for monitoring cockpit commutations and flight crew member interaction. The short notice scheduling and lack of published routes make the scheduling of the line check very difficult. The majority of the line checks conducted in on

- demand air carrier operations are flights dispatched for the sole purpose of accomplishing that check and are not conducted in revenue service with passengers

. Unlike scheduled air carrier operations where these checks are conducted during revenue

o perations, the on

- demand operator must bear the total cost of the check. To counter the economic disadvantages, the on

- demand operator will attempt to comply with the regulatory requirements with the minimum flight time possible. Line checks conducted in the aircraft under these circumstances do not provide the check airman or inspector with many of the opportunities to comply with the intent of the regulation, such as operations in high altitude airspace. I n many instances, operators are not authorized adequate numbers of check airmen and therefore must rely on FAA inspectors to conduct some line checks. Due to limited FAA resources, many

on

- demand operators are having difficulty in scheduling qualified FAA inspectors to conduct the line checks.

In 20 03 , the FAA chartered the P art 135/125 ARC to recommend changes to those operations. 6
In preparation for that ARC

, the FAA solicited public comments. It received more comments from the public regarding line check concerns than any other issue raised for th e ARC. The Part 135/125 ARC achieved f ull consensus on recommendations that would allow for greater flexibility in scheduling the line check, less dependence on FAA resources

, and more importantly, would also encourage conduct ing line checks as part of li ne operations. The final recommendations from the ARC were to : provid e for an alternative means of compliance (MOC) for the initial line check in the form of initial operating experience ( IOE ) ; exten d the authority for line check airmen to similar aircra ft for which they may not be qualified ; and provid e for an alternative MOC for recurrent line checks by using a simulator

- based Line O perational Program.

Conclusion The line check program for part 121 operations continues to be effective and appropriate.

T o achieve a more effective and efficient line check program for p art 135 operators , the line check program should be conducted more like those conducted in p art 121 scheduled air carrier operation s with regard to evaluation of the flight crew in more realis tic operational environments

.
The following recommendations would allow for greater flexibility in the scheduling of the check, less dependence on FAA resources and more importantly, to encourage the conduct of line check as part of line operations (For in formation about implementing these recommendations, s ee p age 20A ).

6 See 68 FR 5488 and FAA Order 1110.135.Flightcrew Member THRR ARC Report 20A

For operations using aircraft that do not have an approved observer’s seat and do not have FAA

- approved line check airmen, the operator may accomplish the PIC line check by any of the following methods:

 A flight in actual revenue service using a contract l ine check airman from another operator, or

 A Line Operational Simulation (LOS) conducted in a Full Flight Simulator using a contract check airman from a 14 CFR part 142 training center or an FAA Inspector.

For operations using aircraft that do have an appr oved observer’s seat and do not have FAA approved Line Check Airmen, the operator may accomplish the PIC line check by any of the following methods:

 A flight in actual revenue service using a contract line check airman from another operator or an FAA inspe ctor occupying the observer seat, or

 An LOS conducted in a Full Flight Simulator using a contract check airman from a part 142 training center or an FAA Inspector.
For operations using aircraft that do have an approved observer’s seat and do have FAA

- appro ved line check airmen, the operator may accomplish the PIC line check by any of the following methods:

 A flight in actual revenue service using a company line check airman or an FAA inspector occupying the observer seat.

 A flight in actual revenue service using a company line check airman occupying a pilot seat and performing the duties as a SIC.

 Using a company line check airman who is qualified in one of the aircraft operating by the certificate holder but not qualified on the specific aircraft on which t he PIC is being checked. This check is conducted from the observer seat. The purpose of the line check is to determine the PIC is competent to operate safely within the ATC system. While operation of the aircraft is not insignificant, the purpose of the line check is not to determine competency to safely operate the specific aircraft type. For this reason, it is intended that a check airman employed by the certificate holder can conduct a line check in any aircraft family. For example, a check airman i n a Gulfstream V is authorized to conduct a line check in a Hawker 800, even if the check airman is not type rated nor qualified to operate the Hawker 800.

 An LOS conducted in a Full Flight Simulator using a contract check airman from a part 142 training center or an FAA Inspector on an alternating rotation with the aircraft.Flightcrew Member THRR ARC Report 21

Recommendations

The THRR ARC recommends

–

 Th at th e line check program employed in part 121 is suitable for those operations.



###  That f or part 135 operations, the adoption of line check rule c hanges be made, as proposed by the P art 135/125 ARC.Flightcrew Member THRR ARC Report 22



### 3.3 I SSUE 

### 3 R ATIONALE

The optimal length of time between training events for such flightcrew members, including recurrent training events.

3.3.1 S ECTION I NTRODUCTION

It is the opinion of the THRR AR C that existing regulatory training intervals are adequate to ensure pilots maintain adequate KSAs necessary to safely carry out flightcrew member duties in commercial flight operations. However, the ARC did consider change of intervals for evaluation. T he remaining portion of this section’s discussion deals only with the modification of evaluation intervals.

Although there are a number of factors that may influence the appropriate length between evaluation events for an individual pilot or all pilots fly ing for an operator, the ARC believes existing regulatory intervals for these events must be reduced in cases where a pilot reenters the operator’s training program after an extended absence, or where they have demonstrated unsatisfactory performance durin g an evaluation. This concept is described as a short

- cycle (described below).

The THRR ARC also examined whether the FAA should consider the merits of extending certain evaluation event intervals for operators by demonstrating one or more attributes desc ribed later in this section.

Operators will have to consider how to allocate available resources to adequately demonstrate to the FAA any or all of the characteristics listed above to apply for extending their assessment intervals. The more of these chara cteristics the operator is able to demonstrate, the greater assessment interval the FAA may grant, up to a maximum interval of 1 year.
Background Frequency of evaluations center around the knowledge and tasks necessary to maintain the requisite higher cogn itive skills needed to handle normal, non

- normal, or emergency conditions during flight operations. For the purpose of this discussion, these higher cognitive skills are referred to as the KSAs needed to handle normal, non

- normal, or emergency conditions.

The ARC did not consider optimum intervals for operators evaluating under an AQP since intervals for those operators are derived from an ISD analysis. These recommendations for optimum intervals are only appropriate for operators training under part 135, subpart G, and/or part 121, subpart O.

The ARC currently does not believe there is sufficient justification for extending any interval beyond 12 months. An operator who wishes to extend intervals beyond 12 months should develop and implement an AQP as cu rrently permitted under part 121, subpart Y.Flightcrew Member THRR ARC Report 23

E ARLY E VALUATION (S HORT - C YCLE )

The THRR ARC believes there are specific cases where existing regulatory evaluation intervals are not appropriate when a pilot had an extended absence from the operator’s training program, or where a pilot has demonstrated unsatisfactory performance during an evaluation. In these cases, the ARC has determined the pilot should be subject to a ―short

- cycle‖ where their interval is shortened to one

- half of the evaluation interval for the event that was unsatisfactory.

The specific methods to carry out the short

- cycle concept and develop regulatory requirements are complex. The THRR ARC did not have time to consider all implementation issues. The refore, the ARC recommends the FAA cons ider various options for implementing these concepts and publish options for public comment prior to rulemaking.

E XTENDING E VALUATION I NTERVALS

The THRR ARC believes there are cases where a n air carrier may be granted an extension to an existing regulatory evaluation interval in cases where the air carrier demonstrates certain operational characteristics. The ARC considered seven operational characteristics that the FAA could consider single or cumulative in incrementally increasing the evaluation interval up to a maximum of 12 months.

N ote : The THRR ARC recommends any carrier wishing to extended evaluation intervals beyond 12 months should develop and implement AQP.

The THRR ARC considered a total of seven characteristics that could be used by a n air carr ier to justify a 1

- month interval extension. An air carrier may be entitled to one or more months of interval extension based on the number of characteristics it demonstrate s to the FAA.

Three of these characteristics were identified from the regulatory e xemption process the FAA has previously used to grant operators regulatory relief from PIC proficiency check requirements of up to 12 months. This exemption

, called the ―Single

- Visit

- Exemption

, ‖ has been granted to operators while they developed an AQP. The THRR ARC believes provisions from this exemption may be considered in granting interval extensions:

 Implementation of a First Look Program

— SPOT Check; a 1

- month interval extension for

a § 121.441 or § 135.297 flight evaluation.

 Implementation of a line oriented flight training ( LOFT ) Program; a 1

- month interval extension for a § 121.441 or § 135.297 flight evaluation.

 Implementation of an Enhanced Training Data Collection and Feedback Program; a 1

- month interval extension for a § 121.441 or § 135.297 fl ight evaluation.

Four other characteristics were identified

, such as appropriate management staffing and oversight of flight operations that exceed minimum baseline regulations. Numerous air carriers realize the benefit(s) of ―over and above‖ processes, p rocedure s , and programs to enhance the company’s safety and thus operational performance. As a result, the THRR ARC believesFlightcrew Member THRR ARC Report 24

operators who employ voluntary safety systems or oversee their flight operations with full

- time management oversight, or demonstra te frequent flight operations may also be justified in extending their training and assessment intervals. The four voluntary safety or enhanced management characteristics are

—

1. Implementation of a n SMS ; a 1

- month interval extension for a § 121.441 or § 135 .297 flight evaluation.

2. Implementation of FOQA or FDM (ICAO) program; a 1

- month interval extension for a § 121.441 or § 135.297 flight evaluation.

Note: In considering these two characteristics

, the THRR ARC realizes that most voluntary and mandatory safe ty programs are created for large part 121 air carriers. The approval process and operator and inspector guidance is not available to the part 135 industry. Due to this, numerous part 135 air carriers have had to adapt ICAO

- based safety systems to meet t he requirements to operate as an international carrier compliant with ICAO. Since the United States also recognizes ICAO, an ICAO

- based system should be considered to be equivalent when no specific U.S. regulation is available.

3. Enhanced Management Oversig ht program; a 1

- month interval extension for a § 121.441

or § 135.297 flight evaluation.

4. Demonstration of Individual Pilot Operational Tempo; a 1

- month interval extension for a § 121.441 or § 135.297 flight evaluation.

The THRR ARC has defined these seven characteristics into options that carriers can use to extend evaluation intervals up to 12 months. Suggestions for implementing these options are contained in Appendix D.

Recommendations

The THRR ARC recommends

—

 That early evaluation (short

- cycle) should be required for:

o Extended pilot absence from flight operations requiring requalification training that exceeds recurrent training but does not require initial training.

o Unsatisfactory performance during an assessment.

 That under specified conditions, the e valuation interval may be extended if the air carrier can demonstrated operational characteristics contained in one or more the following

seven options.

o Option 1: Implementation of a First Look Program

— SPOT Check; a 1

- month interval extension for a § 121. 441 or § 135.297 flight evaluation.Flightcrew Member THRR ARC Report 25

o Option 2: Implementation of Line Oriented Flight Training Program; a 1

- month interval extension for a § 121.441 or § 135.297 flight evaluation.

o Option 3: Implementation of an Enhanced Training Data Collection and Feedb ack Program; a 1

- month interval extension for a § 121.441 or § 135.297 flight evaluation.

o Option 4: Implementation of a n SMS ; a 1

- month interval extension for a § 121.441

or § 135.297 flight evaluation.

o Option 5: Implementation of FOQA or FDM (ICAO) prog ram; a 1

- month interval extension for a § 121.441 or § 135.297 flight evaluation.

o Option 6: Enhanced Management Oversight program; a 1

- month interval extension for a § 121.441 or § 135.297 flight evaluation.

o Option 7: Demonstration of Individual Pilot Op erational Tempo; a 1

- month interval extension for a § 121.441 or § 135.297 flight evaluation.

 That the evaluation interval is not extended beyond 12 months, unless the air carrier has developed an AQP program in accordance with part 121

, subpart Y.Flightcrew Member THRR ARC Report 26



### 3.4 I SSUE 

### 4 R ATIONALE

The best methods to reliably evaluate a flightcrew member’s mastery of aircraft systems, maneuvers, procedures, takeoffs and landings, and crew coordination.

3.4.1 S ECTION I NTRODUCTION

The evaluation of a pilot’s KSAs involves observation of the pilot’s performance during a set of maneuvers that were developed using an ISD process. This observation measures the pilot’s performance against a set of defined standards of performance. For multi

- crew operations, it is important to measure the performance of all flightcrew member positions during this evaluation. In response to this issue, discussion and recommendations focus on t wo areas:

 The use of scenario

- based evaluations

.

 The use of FSTD equipment in the evaluation process

. Background
S C ENARIO - B

ASED E VALUATION

At the end of the curriculum and for a pilot who has met standards for all tasks and validations

, the pilot must then be evaluated. The evaluation should be a scripted operational scenario

- based event and should in addition to eval uating skills associated with accomplishing selected tasks, must be developed to allow for the evaluation of decision making, leadership, CRM, and TEM skills. If a pilot fails the evaluation, he will be remediated to standards and then reevaluated. If th e pilot fails the evaluation

, a second time it should be considered the pilot has failed the curriculum.

Evaluation profiles should contain an operationally relevant real

- world

- like situation or scenario, reflective of any special operational authorization s the operator has been granted, that is used to focus the flight crew members into a particular subject matter.

T HE USE OF FSTD EQUIPMENT IN THE EVA LUATION PROCESS

P art s 121 and 135 permit certificate holders to use simulators for varying amounts of the
tr aining, testing, and checking required by the FAA. Use of simulators is a voluntary alternative to training and checking in the airplane. The only required use of a n FSTD in the current regulations is in the windshear requirements of § 121.409(d). Using FSTD rather than aircraft allows for more in

- depth checking in a safer environment, including the practice of critical emergency procedures using a broad range of scenarios to replicate virtually any possible situation, in every conceivable weather and en vironmental condition. FSTD s also provide other benefits

, such as reducing noise, air pollution, and air traffic congestion, and conserving petroleum resources. This proposal addresses concerns raised by the National Transportation Safety Board Recommend ations A

– 94

– 191 through A

– 94

– 194, which state that part 121 flight training and checking should be required in FSTD s wherever possible

.Flightcrew Member THRR ARC Report 27

Conclusion The ARC strongly believes better pilot performance will result from realistic training and validation of pilo t skills. Therefore, pilot validation events should be accomplished in line

- oriented events in flight simulators (when simulation is available). The events should be structured to ensure all pilot skill tasks are validated, but flexibility should be perm itted in how those tasks are validated. This flexibility should include the ability to validate training over multiple sessions rather than one check

- ride event. In addition, the line

- oriented validation should focus on operator specific requirements, sp ecific operational challenges faced by the operator, or issues identified in the operator’s safety management program.

Recommendations

The THRR ARC recommends

—

 That flightcrew member mastery is best evaluated by the use of a n appropriate FSTD

.

 Th at the fli ght proficiency be accomplished by using scenario

- based evaluation.Flightcrew Member THRR ARC Report 28



### 3.5 I SSUE 

### 5 R ATIONALE

Classroom instruction requirements governing curriculum content and hours of instruction.

As discussed in the rationale for Issue 1 , classroom training is integra l to overall pilot training. The refore, the THRR ARC members concluded the same process and procedures discussed in section 3.1 apply to classroom training. For information about the background, conclusion, and recommendations, see Section 3.1 on page 28

.Flightcrew Member THRR ARC Report 29



### 3.6 I SSUE 

### 6 R ATIONALE

The best methods to allow specific academic training courses to be credited toward the total fligh

- hours required to receive an airline transport pilot certificate.

3.6.1 S ECTION I NTRODUCTION

Background Today, part 121 requires a n ATP certificate for pilots acting as PIC and a c ommercial certificate for SIC pilots. With the passage of P ub

. L

. 111

- 216, a pilot serving as SIC for a part 121 air carrier will require an ATP certificate

. This legislation allows the FAA to consider us ing specific academic training courses towards the ATP flight hour requirements. Under part 135, PICs are required to hold an ATP certificate for operations in turbojet airplanes having a passenger seating configuration of 10 or more. An ATP certificate is also required for a PIC conducting commuter operations in any multi

- engine airplane

. 7 An SIC in those operations is required to hold a c ommercial certificate.

The THRR ARC believes the existing ATP requirements are appropriate for obtaining that certif icate when the pilot will be acting as the PIC of an aircraft (where FAA regulations require an ATP certificate for the aircraft and type of operation). For an ATP certificate required for pilots acting as a SIC under part 121 regulations, the ARC recomme nds a system providing academic credit, with limitations. T he ARC ’s position is that any pilot serving as PIC at a part 121 air carrier must meet the minimum regulatory standard for full ATP certification

, and no academic credit would be provided for a pi lot seeking PIC privileges.

The FAA chartered an ARC to address Section 217 of P ub

. L

. 111

- 216 , the FOQ ARC

. Th e FOQ ARC considered what criteria would permit academic training and quality of experience to substitute for the existing aeronautical experien ce requirements required to obtain an ATP certificate necessary to exercise SIC privileges under part 121. The FOQ ARC developed an aeronautical experience credit system that could be used to meet ATP aeronautical experience requirements

, but restricted t he applicability of that system to the issuance of an ATP certificate limited to SIC privileges in part 121 operations. In reviewing the recommendations drafted by the FOQ ARC, the THRR ARC believes the aeronautical experience credit system proposed is

va lid for exercising SIC privileges under part 121.

I n addition to the recommendations proposed by the FOQ ARC, the THRR believes the following factors must be considered by the FAA :

 There is a need for any aeronautical experience credit system to be reconci led, where possible, with ICAO licensing standards and recommended practices and initiatives.

7 See § 135.243 (a).Flightcrew Member THRR ARC Report 30

 T he ARC does not support applying an academic training credit system beyond the certification for ATP to act as a part 121 SIC. In the opinion of the THRR member s, a pilot that will be upgrading from an SIC to PIC at a part 121 air carrier would need to
meet the existing regulatory requirements for ATP. This means a n SIC upgrading to PIC would need to meet the minimum total flight hours requirement for an ATP cer tificate of 1,500 hours, as well as other specific flight hour requirements for cross

- country, night, and instrument flying experience. 8

 The recommendations submitted by FOQ ARC are specific to the certification requirements for a pilot who will perform SI C duties at an air carrier operating under part 121. The requirements for a pilot that will perform SIC duties at a part 135 air carrier are not addressed. It is the view of the THRR ARC the certification requirements for pilots of part 135 air carriers are appropriate to the operations they conduct. The ARC therefore do es not recommend any changes to the part 135 pilot certification and experience requirements.

Conclusion

The THRR ARC concludes there is value and benefit to providing limited academic tr aining and experience credits for those pilots who are required to obtain an ATP certificate in order to exercise SIC privileges in part 121 operations.

Recommendations

The THRR ARC recommends

—

 That the FAA implement the FOQ ARC ’s recommendation to establi sh an aeronautical experience credit system for an ATP certificate with SIC privileges in operations conducted under part 121. One member of the THRR ARC dissented from this recommendation. 9

 Th at an aeronautical experience credit system not be allowed for experience requirements for any ATP certificate for a PIC at a part 121 or part 135 air carrier.

 That the use of an aeronautical experience credit system not be used for any part 135 flight

- hour requirements

.

8 See § 61.159.

9 Refer to Appendix E for CAPA ’s Dissent Statement submitted to the FOQ ARC.Flightcrew Member THRR ARC Report 31



### 3.7 I SSUE 

### 7 R ATIONALE

C rew leadership train ing.

3.7.1 S ECTION I NTRODUCTION

Background There is required training f or a pilot to serve as a PIC . This training largely consists of the performance of operational tasks and responsibilities specific to the duty station

, and does not necessarily prov ide education to the PIC on his or her leadership role. CRM training, which is now required for all air carriers, contains some elements of the desired leadership training . It is not designed to aid the PIC in assuming a leadership role in the aircraft a nd the air carrier as the training envisioned by the THRR ARC would.

In accordance with § 206 of P ub

. L

. 111

- 216, the FAA convened the MLP ARC to provide recommendations for flightcrew member mentoring, leadership

, and professional development for part 121 air carriers. While the tasking of the MLP A RC’s deliberations and recommendations exceed that of the THRR’s scope, their analysis of leadership training was reviewed by the THRR ARC members. With some modifications, the THRR ARC concur s with their reco mmendation to require leadership training for the PIC. The ARC has made no assessment of the other recommendations of the MLP ARC. Additionally, while the MLP ARC deliberations were limited to part 121 operators, the THRR A RC is tasked with considering l eadership training for part 135 operators as well.

The THRR ARC suggests an ISD process could potentially be used in lieu of the specific hour requirement recommended by the MLP ARC. The ISD method of training permits the air carrier to determine, based u pon data and experiences, the optimal training times for pilots. E stablish ing a mandatory fixed

- minimum runs contrary to the principles of training the THRR ARC endorses.

Further, due to the wide variety of operations and the size and scope of various air carriers, including those operating under part 135, a specific hour requirement could ultimately have a negative learning impact on the pilots. For instance in a small operation, with only a few aircraft and PICs, mandating a minimum of 32 hours of train ing, as proposed by the MLP ARC, could prove difficult to execute.

The THRR ARC believes a facilitated discussion among those attending the course is a key component to the success of this particular aspect of training. The ARC concur s with the MLP ARC th at leadership training is not well suited to distance learning due to the value of the interactive group dynamic among multiple pilots. Additional items each air carrier may deem necessary to introduce in a leadership and command course may be suitable fo r distance learning.Flightcrew Member THRR ARC Report 32

The FAA should, when describing how an air carrier with a small number of PICs would implement this training, permit flexible solutions in which PICs from multiple, similarly situated air carriers ( that is , similar aircraft and scope/t ype of operations) could pool their PICs into a class size that would then provide the interactive group discussion dynamics envisioned.

The leadership training should be developed as a training event separate from the existing PIC upgrade transition sylla bus. It is recommended that each air carrier develop their course to incorporate air carrier

- specific course material. The leadership training should be divided into two segments. The initial segment would be completed prior to upgrade training in order to cover the leadership modules of the course. The second segment would need to be completed between 6 and 18 months after completion of the operating experience/IOE and should incorporate lessons learned during the new captain’s initial experiences as a captain . It should also reinforce the concepts covered in the initial leadership and command course.

Conclusion Providing specific training to the PIC on his or her leadership role will increase the safety and professionalism of air carrier operations.

R ecommendations

The THRR ARC recommends

—

 That the FAA implement the MLP ARC recommendation to require leadership training for the PIC

. A n ISD process could potentially be used in lieu of the specific hour requirement in the MLP ARC ’s recommendation

.

 That l eadership training be integrated into appropriate training events through the ISD process

.Flightcrew Member THRR ARC Report A

– 1 

## APPENDIX A

— THRR ARC MEMBERS AND SUPPORT STAFF

THRR ARC M EMBERS AND 

### A FFILIATION

Ms. Jacqueline Rosser, Co

- Chair , National Air Transportation Association (NATA) Capt

. Randy Hamilton, Co

- Chair , Regional Airline Association (RAA)

Mr. Timothy Beglau, Designated Federal Official, FAA

A ERO - M ICRONESIA

Mr. Michael Quinn

A IR L INE P ILOTS A SSOCIATION , I NTERNATIONAL

Mr. Bill Edmunds Capt. Chuck Hogeman

A IR T RANSPORT A SSOCIATION

Capt

. John Dudley Capt. Dave Lynn

A IRBUS A MERICAS

Capt. Rudy Canto

A TLANTIC S OUTHEAST A IRLINES

Capt. Eric Carlson Capt. Patrick Greene

T HE B OEING C OMPANY

Capt. Terry McVenes Capt. Tom Pryde

B OMBARDIER A EROSPACE

Mr. Robert FreemanFlightcrew Member THRR ARC Report A

– 2 CAE S IMUFLITE

Mr. Willi am Campbell

Mr. Steve Hall

T HE C OALITION OF A IRLINE P ILOTS A SSOCIATIONS

Capt. Gregory Bowen

F LIGHT S AFETY I NTERNATIONAL

Mr. Rick Bedard

Mr. John Hinchey

K EY A IR

Capt. Knut Finnevolden

NATA

Mr. Dave Hewitt

RAA

Mr. Steve Briner



### S UPPORT 

### S TAFF

FAA

Ms. Catherine Burnett

Mr. Christopher MacWhorter

Mr. Jeffrey Schroeder

Mr. Christopher Skully

Mr. Peter Tokarz

PAI C ONSULTING

Ms. Kelly Akhund

Ms. Sarah O’BrienTHRR ARC Final Report B

– 1



## APPENDIX B

— ACRONYMS

14 CFR Title 14, Code of Federal Regulations

AC advisory circular

AQP Advanced Qualif ication Program

ARC Aviation Rulemaking Committee

ASAP Aviation Safety Action Program

ATP airline transport pilot

CRM crew resource management

FAA Federal Aviation Administration

FDM flight data monitoring

FOQ First Officer Qualification

FOQA Flight Opera tions Quality Assurance

FSB Flight Standards Board

FSTD flight simulator training device

IATA International Air Transport Association

ICAO International Civil Aviation Organization

ILS instrument landing system

I n FO Information for Operators

IOE initial o perating experience

ISD Instructional Systems Design

ITQI IATA Training and Qualification Initiative

KSA knowledge, skills, and attitude

LOFT Line Oriented Flight Training

LOSA Line Operations Safety Audit

MLP Mentoring, Leadership, and Professional Develo pmentTHRR ARC Final Report B

– 2

NATA National Air Transportation Association

OPTEMPO operational tempo

PF pilot flying

PIC pilot in command

PM pilot monitoring

POI principal operations inspector

PTS practical test standard

RAA Regional Airline Association

SAFO Safety Alert for Oper ators

SIC second in command

SME subject matter expert

SMS Safety Management Systems

SOP standard operating procedures

SPOT Special Purpose Operational Training

TEM Threat and error management

THRR Training Hours Requirement ReviewTHRR ARC Final Report C

– 1



## APPENDIX C

— P OINT G RADING E XAMPLE

S AMPLE 

### G RADING 

### S CALE

5

– Excellent

Performance remains well within the prescribed performance standards. Individual performance, management

, and CRM skills are exemplary.

4

– Standard Student performance remains within the prescribed performance standards. Individual performance meets expectations. CRM skills

are effective

.

3

– Satisfactory

Deviations occur from the performance standards that are recognized and corrected in a timely manner. Individual performance
meets expectations. CRM skills are effective.

REASO N CODE Required

2

– Below Standard

Deviations occur from the performance standards that are not recognized and corrected in a timely manner. Individual performance does not jeopardize the airplane. CRM skills are not completely effective.

REASON CODE R equired Standard Scale: Reason Code Required Generates Rollover Repeat

5

– Excellent NO NO NO

4

– Standard NO NO NO

3

– Satisfactory YES NO NO

2

– Below Standard YES YES YES

1

– Unsatisfactory YES YES YES

N/A

– Not Accomplished NO Y ES

_THRR ARC Final Report C

– 2

1

– Unsatisfactory Major deviations from the performance standards occur that are not recognized or corrected. Individual performance could result in a hull loss or loss of life. CRM skills are not effective.

REASON CODE Required

0

– Not Attempte d

For tasks not accomplished due to time constraints or equipment malfunctions, Aviation

- savvy training systems will rollover the task(s) to the next training event unless it appears in the following lesson. If it is the last training day prior to the ch eck, all training events must be signed off. If it is a simulator

- related issue, provide some comment indicating that there was a simulator/ device

- related issue.THRR ARC Final Report C

– 3

R EASON C ODES

Technical

PY Proficiency Lack of psychomotor (stick and rudder) skills.

PD Procedure Deviation in the execution of regulations and/or operator procedures. The intention is correct but the execution is flawed. A task is forgotten or skipped.

NC Intentional Noncompliance Willful deviation from regulations and/or operator procedur es.

AT Automation Lack of knowledge or skill in automation use.

Proficiency Errors

 Manual handling

 Lateral/vertical deviations

 Wrong airplane configuration

 Ground n avigation

 Systems knowledge

 Radios/ i nstruments

Automation

 Autopilot

 Mode control panel

 Fl ight Management System

 Autothrottle

 Electronic Flight Bag

 Display management

 Information management

 Inappropriate level Procedural Errors

 Checklists

 Briefings

 Callouts

 Cross

- checking

 Limitations

 Documentation

 Improper initiation

 Not completed

 Other procedu ral

Intentional Noncompliance

 Violations of SOP or regulationsTHRR ARC Final Report C

– 4

Non

- technical (CRM/TEM)

CM Communication Miscommunication, misinterpretation, or failure to communicate or solicit pertinent information. Failure to communicate the plan or set the tone fo r crew coordination.

SA Situational Awareness Failure to monitor or perceive elements in the environment, the

comprehension of their meaning, and the projection of their status in the near future.

DM Decision Making An error that is not standardized by r egulations or operator procedures and that unnecessarily compromises safety. Failure to develop a plan.

WL Workload Management Absence of actions/strategies that allow for timely and appropriate

completion of all operational tasks. Failure to prioritize tasks and/or manage responsibilities.

Communication

 Pilot

- to

- pilot communication

.

 Crew to external communication

.

 Failure to communicate and cooperate with crew members and other staff in sharing information

.

 Improper radio phraseology

.

 Doesn’t verbali ze action

.
Decision Making

 Fails to establish goals and monitor outcomes

.

 Jumps to conclusions

.

 Extreme maneuvers on approach

.

 Choosing to fly into adverse weather.

 Information management

. Situational Awareness

 Fail s to monitor/perceive elements in environ ment and project them into the future.

 Inability to actively monitor and evaluate the operation.

Workload Management

 Fails to slow things down and control the pace by making time.

 Fails to prioritize tasks.

 Absence of strategies for task completion.

 Absenc e of actions for task completion.THRR ARC Final Report C

– 1

R EPEATS AND F INAL R ATINGS

Provided that there is time left in the training session, repeats allow the instructor to perform tasks that are not satisfactory and capture a grade for this additional task

. If the grade is a 3 or higher, the item will not roll into the next scheduled session and repeats will not be required. Select the number of times you repeated the task, not the total number of times the task was accomplished.

G RADING A CCURATELY

The grades given in all courses will be evaluated on a monthly basis or when there are curriculum revisions driven by customer/program requests. These data are crucial in determining where deficiencies in curriculum exist. To make valid changes, the data must be reliable. Ins tructor observation skills of student performance s are critical in determining all types of student performance. Even though the focus is on lower levels of performance, good performance averages will be used to determin e proficiency levels. I nflated grad es will skew the placement of where these gates in training occur.THRR ARC Final Report C

– 2



### J OINT 

### A VIATION 

### A UTHORITY 

### 5

- P OINT 

### G RADING 

### E XAMPLE 10

Grade Description

5 The ideal performance under existing conditions. Anticipates and adapts easily to changing or unusual flight situati ons.

4 The a im of the exercise is safely achieved with very few minor variations from ideal. Performance shows smooth control of aircraft.

3 The a im of the exercise is safely achieved with frequent minor but no major variations from the ideal.

2 The a i m of the exercise is safely achieved. Performance includes not more than

one major variation from the ideal and may include frequent minor variations from the ideal.

1 The a im of the exercise is safely achieved in a rough manner. Performance includes

mo re than one major variation from the ideal and indicates a level of skill or knowledge, which results in a marginally acceptable performance.

0 Any one of the following will result in an assessment of fail:

 The a im of the exercise is not completed

.

 There is an i nsufficient level of knowledge to ensure safety.

 The a im of the exercise is completed

, but at the expense of using unsafe airmanship and/or handling errors.

 Dangerous aircraft handling

, requiring assistance from the examiner.

 Tolerances specified in the flight test standards are exceeded.

Written remarks are required when awarding a flight test exercise a mark of 2 or less. The remarks should be clear and concise

, and in the case of an exercise assessed as

—

 Grades 1 and 2 reflect the major variatio n(s) from the a cceptable p erformance for the exercise as outlined in the appropriate flight test standards; or

 Grade 0 reflect s the appropriate item or items that result in an assessment of fail as listed in the g rading s cale section of the flight test s ta ndard(s).

10 Joint Aviation Authority. Flight Examiners Manual: Aeroplane and Helicopter, version 17, M ay 2011.THRR ARC Final Report C

– 3

During a flight test, it is sometimes difficult to write clear and concise remarks. It is recommended that examiners use notes made during the flight test to complete a final copy of the f light t est r eport. This provides the examiner with the o pportunity of referencing the appropriate flight test standards while writing final comments.

Was Performance Acceptable?

YES

Were There Errors?

Yes

Were the Errors Major?

Yes

How Many Errors?

More Than 1
Rating: 1 Only 1
Rating: 2 No

No

No Rating: 0

Rating: 5

How Many Minor Errors?

Frequent

Rating: 3 Few

Rating: 4THRR ARC Final Report C

– 4



### T RANSPORT 

### C ANADA 

### 4

- P OINT 

### G RADING 

### S CALE 11

When applying the 4

- point scale, award the mark that best describes the weakest element(s)
ap plicable to the candidate’s performance. Remarks to support mark awards of 1 or 2 must link to a safety issue, a qualification standard (performance criteria), or an approved technique

or procedure.

4 Above

s tandard Performance remains well

within the qua lification standards and flight management skills are excellent.  Performance is ideal under existing conditions.

 Aircraft handling is smooth and precise

.

 Technical skills and knowledge exceed the required level of competency.

 Behavior indicates continuous and highly accurate situational awareness.

 Flight management skills are excellent.

 Safety of the flight is assured. Risk is well mitigated.

3 Standard Minor deviations occur

from the qualification standards and performance remains within prescribed limit s.  Performance meets the recognized standard

, yet may include deviations that do not detract from the overall performance.

 Aircraft handling is positive and within specified limits.

 Technical skills and knowledge meet the required level of competency.

 Beha vior indicates that situational

awareness is maintained.

 Flight management skills are effective.

 Safety of the flight is maintained. Risk is acceptably mitigated.

11 Transport Canada. TP 14277

– Pilot Examiner Manual , 3 rd ed., April 2010.THRR ARC Final Report C

– 5

2 Basic

s tandard Major deviations from the

qualification standards occur, which may include momentary excursions beyond prescribed limits

, but these are recognized and corrected in a timely manner

.  Performance includes deviations that detract from the overall performance, but are recognized and corrected within an acceptable time frame.

 Aircraft handling is performed with limited proficiency and/or includes momentary deviations from specified limits.

 Technical skills and knowledge reveal limited technical proficiency and/or depth of knowledge.

 Behavior indicates lapses in situational awareness th at are identified and corrected by the pilot/crew.

 Flight management skills are effective but slightly below standard. Where applicable, some items are only addressed when challenged or prompted by other crewmembers.

 Safety of the flight is not compromise d. Risk is poorly mitigated.

1 Below

s tandard Unacceptable deviations

from the qualification standards occur, which may include excursions beyond prescribed limits that are not recognized or corrected in a timely

manner  Performance includes deviations th at adversely affect the overall performance, are repeated, have excessive amplitude, or for which recognition and correction are excessively slow or nonexistent, or the aim of the task was not achieved.

 Aircraft handling is rough or includes uncorrected or excessive deviations from specified limits.

 Technical skills and knowledge reveal unacceptable levels of technical proficiency and/or depth of knowledge.

 Behavior indicates lapses in situational awareness that are not identified or corrected by the pilot/ flight crew.THRR ARC Final Report C

– 6  Flight management skills are ineffective, unless continuously challenged or prompted by other flight crew members.

 Safety of the flight is compromised. Risk is unacceptably mitigated.



### F LIGHT 

### S AFETY 

### 4

– P OINT 

### G RADING 

### S CALE 12

1

– Proficient (to PTS S tandard)

2

– Normal p rogress (anticipate proficient within normal course structure)

3

– Needs a dditional t raining ( s imulator or g round s chool beyond course structure)

4

– Unsatisfactory

T ASKS /T OPICS THAT ARE NOT R EQUIRED TO BE TRAINE D TO PROFICIENCY :

T

– Trained p rocedure only

D

– Discussed task/topic

C

– Complete (LOFT Only)

FlightSafety encourage s instructor remarks by task or by training session. For grades of 1 and 2

, these remarks are usually the location of training and a summary of the scenario; grades of 3 and 4 require remarks addressing each objective standard in the PTS not met , as well as any target areas for emphasis.

12 FlightSafety International

. Learning Center Operations Manual, Revision 5.2, 22 March 2011, Section 5.11THRR ARC Final Report D

– 1



## A PPENDIX D

— APPLYING ISSUE 3’S RECOMMENDATIONS

I MPLEMENTING THE 

### E ARLY 

### E VALUATION 

### C ONCEPT 

### (S HORT 

### - C YCLE 

### )

The ARC recommends that if a pilot h as had an extended absence from flight duties requiring additional training beyond the normal recurrent training (but less than the training required for initial qualification) then t hat pilot shall be subject to a short cycle for the appropriate evaluatio n.

The ARC also recommends that i f a pilot demonstrates unsatisfactory performance while completing one of the following events, he or she shall be subject to a short

- cycle for the respective event:

 P art 135

o Recurrent Ground Training (either written or ora l test) § 135.351(b).

o Competency Check § 135.297(b).

o Instrument Proficiency check § 135.297.

o Line check § 135.299.

o ATP Certificate Practical Test § 61.153(g).

o Aircraft Type Rating Practical Test § 61.63(d).

 P art 121

o Recurrent Training

– Ground § 121.427(b)(1 ) oral and/or written test.

o Proficiency Check § 21.441(a).

o Proficiency Training §§ 121.441(a) and 121.409.

o Line check § 121.440.

o LOFT §§ 121.441(a) and 121.409.

o ATP Practical Test § 61.153(g).

o Aircraft Type Rating Practical Test § 61.63(d).

P ilot s who ha ve demonstrated unsatisfactory performance must receive the necessary training to achieve performance standards. Once this training is completed, pilots must be reevaluated before being allowed to resume flight duties. After resuming flight duties, the pil ot will return for a short

- cycle of the event deemed unsatisfactory that is o ne

- half the normal interval stipulated in the operator’s training program. Once completing the short

- cycled evaluation satisfactorily,THRR ARC Final Report D

– 2

the pilot returns to the normal interval ap proved for the operator. If unsatisfactory performance is again demonstrated during the short

- cycled training and KSA assessment, then the pilot is subject to another short cycle interval.

In cases where the training and assessment interval is an odd numb er, the interval is one

- half rounded up to an even number.

 Example A: An operator uses a 12

- month recurrent training and assessment interval for the competency check required under § 135.293(b). A pilot who demonstrates unsatisfactory performance during KSA assessment would be required to complete recurrent training and assessment 6 months after the unsuccessful KSA assessment and necessary remedial training. If during the short

- cycled recurrent training the pilot does not require any additional training beyond the normal events and satisfactorily demonstrates performance standards, then they would return to the normal 12

- month interval.

 Example B: An operator is allowed by the FAA to use a 9

- month training and assessment interval for a proficiency check (§ 121.441(a)). A pilot who undergoes recurrent training but demonstrates unsatisfactory performance during KSA assessment would be required to complete recurrent training and assessment 5 months after the unsuccessful KSA assessment and necessary remedi al training (one

- half of 9 months would be 4.5 months

, rounded up to 5 months). If during the short

- cycled recurrent training the pilot does not require any additional training beyond the normal events and satisfactorily demonstrates performance standards , then they would return to the normal 9

- month interval.
The existing ―base

- month‖ concept, where training may be scheduled 1 month prior to the base month or 1 month after the base month

, should apply in these provisions.

I MPLEMENTING E VALUATION E XTENSION S

The ARC recommends that the FAA offer one of seven options (either individually or combined) to operators who wish to extend training and assessment intervals by implementing enhanced safety or training programs. The FAA should consider the provisions f or each option listed below in considering an extended interval.

The options include the following:
Option 1 Requirements: Implementation of a First Look Program

— SPOT Check; a 1

- month interval extension for a § 121.441 or § 135.297 flight evaluation.

Pri or to annual training there will be an initial simulator session consisting of a 2

- hour SPOT session. The s imulator instructor conducting the period will complete a blind (deidentified) data collection form on each flightcrew member. Four fixed ― f irst l ook‖ maneuvers will be identified on an aircraft specific basis by the d irector of t raining for FAA approval.

Four fixed maneuvers should be selected on the following criteria:

 At least one maneuver will be flown with a diminished aircraft system

.THRR ARC Final Report D

– 3

 At least one maneuver will be flown with diminished avionics

.

 At least one maneuver will be flown engine out

.

 At least one approach/missed approach will be flown manually (without autopilot).

A sampling of all other maneuvers in this Appendix (called ―variable man euvers‖) will be selected by the simulator instructor, along with any other aircraft specific training items as determined by the d irector of t raining.

A complete evaluation utilizing the concepts of fixed/variable maneuvers will be administered during the SPOT simulator session. The evaluation itself will be for a total of 2 hours (with PF duties split about 1.5 hours for the PIC and .5 hours for the SIC) and conducted in the flight crew concept mode. Flightcrew members are accountable for and individuall y evaluated on all respective PF and PM duties, as applicable during the entire period. The instructor completing the SPOT evaluation will complete a blind data collection on each flight crew member as described later.

A minimum of eight maneuvers will be completed during the check: six of these events must be completed with the PIC flying (four fixed, two variable) and two of the fixed maneuvers with the SIC flying.

Proficiency Check maneuvers

— Sample of maneuvers

—

 PIC PF; SIC PM

—

o Fixed :

 I nstrument landing system (ILS) with flight control failure,

 Raw data ILS

,

 Takeoff with an engine failure, and

 Full procedure non

- precision manually flown approach,

o Variable :

 Aborted takeoff, and

 ILS to lowest minimums

.

 SIC PF; PIC PM

—

o Fixed :

 Full procedure missed approach, and

 ILS with engine

- out.THRR ARC Final Report D

– 4

For all tasks, maneuvers, or procedures included on first look evaluations, the certificate holder will identify the performance standards to be employed. Current FAA practical test advisory materials and pertinent flight standar dization board reports for a given type of aircraft will be employed for guidance in developing specifications for performance standards. The specifications for performance standards will be submitted to the air carrier’s principal operations inspector ( P OI ) for approval.

Each event on the SPOT check will be individually graded based on the following criteria:

 For successful completion of an event, SPOT check data collection will not require retraining and retesting. Two unsatisfactory items will be retra ined during the remainder of recurrent training:

o Event flown to s tandards , and

o Event u nsatisfactory

.

 Level of f

lightcrew m ember c oordination d isplayed

—

o Flightcrew m ember c oordination met standards, and

o Flightcrew m ember c oordination was unsatisfactory.

 Dat a will be aggregated for a report to the FAA with 2 points for each event as described above : 1 for successful completion and 1 for f lightc rew m ember c oordination

, resulting in 16 points available for each SPOT check.

Data will be collected and analyzed d uring the training until all pilots have cycled thr o u gh a SPOT evaluation. If, at the end of the period , t he SPOT program indicates an aggregate average of greater than 90 percent ones (1), the time between checking may be extended for 1 month , up to the 12

- month maximum. Approval for a 1

- month extension is determined by the POI upon review of data. Collection and analysis of data will be continued and the results will be presented to the certificate holder’s POI annually to determine if the checking ext ension should be reapproved.
Option 2 Requirements: Implementation of LOFT Program; a 1

- month interval extension for a § 121.441 or § 135.297 flight evaluation. Air carrier training programs must include a 2

- hour, full flightcrew member complement simulat or period dedicated to LOFT, as outlined in the l ine o perational simulation advisory circular (AC) 120

– 35 C . This LOFT session will be conducted separately from the normal simulator training program. The LOFT scenario will be constructed to focus on human factors skills as well as other specific training objectives appropriate to the air carrier.THRR ARC Final Report D

– 5

For all tasks, maneuvers, or procedures included i n LOFT scenarios, the certificate holder will identify the performance standards to be employed. Current FAA pr actical test advisory materials and pertinent flight standardization board reports for a given type of aircraft will be employed for guidance in developing specifications for performance standards. The specifications for performance standards will be subm itted to the POI for approval.

LOFT will be a no

-

jeopardy event. The period cannot result in an unsatisfactory grade, although additional LOFT may be given if deemed necessary. The LOFT will be videotaped and reviewed at the end of the period to facilita te the debriefing session.

Instructors conducting the LOFT will complete one blind data form on the entire flight crew:

 Successful completion of LOFT :

o All LOFT events met standards , or

o Event was u nsatisfactory

.

 Level of CRM Displayed :

o CRM met standards , or

o CRM was unsatisfactory

. Because LOFT and CRM are key elements in the training program, a LOFT working group will be assembled each year consisting of pilot representatives from each aircraft type and coordinated by the a ir c arrier’s d irector of t raining an d CRM working group. The team will determine the annual CRM t heme and design LOFT scenarios that complement and reinforce that theme. Recurrent CRM training will use the theme as the main topic of classroom discussion, and aircraft types will tailor the basic scenario for specific use LOFT. LOFT design will adhere to AC 120

– 35, as amended, for general guidance and comply with the following requirements.

 Annual LOFT scenarios and the CRM theme must always be complementary.

 LOFT scenarios (and the CRM them e) must be varied and updated annually.

 Identical LOFT scenarios cannot be used in consecutive training sessions for a given pilot.

 Flightc rew member substitution stated in AC 120

– 35C for recurrent LOFT will be utilized.THRR ARC Final Report D

– 6

Data will be collected and analyzed during the training until all pilots have cycled thr ough a LOFT scenario. If, at the end of the period, the LOFT program indicates an aggregate average greater than 90 percent ones (1), the time between checking may be extended for 1

- month, up to the 12

- month maximum. Collection and analysis of data will continue and the result will be presented to the certificate holder’s POI annually to determine if the checking extension should be reapproved.
Option 3: Implementation of an Enhanced Training Data Coll ection and Feedback Program; a 1

- month interval extension for a § 121.441 or § 135.297 flight evaluation. Each pilot is required to attend training required under 14 CFR at least annually. Annual training for each pilot will be a combination of distribute d training and onsite training that must be accomplished during the pilot’s 3

- month eligibility period (base month, plus or minus 1 month). This will include:

 Annual ground school instruction

, which includes mandated training under part s 121 and 135 in re current school subject lessons

.

 Annual simulator flight training to proficiency for each flightcrew member.

Annual simulator events will be conducted to the maximum extent possible in a flight crew member

- oriented session, with a flightcrew consisting of a PIC and SIC.

For all tasks, maneuvers, or procedures included i n proficiency evaluations, the certificate holder will identify the performance standards to be employed. Current FAA practical t est advisory materials and pertinent flight standardization boa rd reports for a given type of aircraft will be employed for guidance in developing specifications for performance standards. The specifications for performance standards will be submitted to the POI for approval.

The certificate holder will develop a pro ficiency rating scale for the training sessions from 1

– 5 for approval by their POI.

Example s of these include :

 Met standards on first attempt.

 Failed to meet standards, debriefed, no additional simulator training required.

 Failed to meet standards on firs t attempt. Met standards on second attempt after debrief and retraining, additional program hours not required.

 Training to proficiency required additional time over and above program requirements.

 Unsatisfactory

, u nable to meet standards.

Each instructor will complete a data sheet on completion of each training date evaluating each task or maneuver. Each evaluator will grade each task or maneuver based on the above criteria.THRR ARC Final Report D

– 7

Data on pilot performance conducted under this part shall be captured by the cer tificate holder and reported to the FAA POI. Performance evaluations will be accomplished in accordance with the following categories and provisions above.

 By aircraft type (according to the criteria in AC 120

– 53)

.

 By flight crew position (PIC or SIC). Ea ch flightcrew member’s data shall be reported as a single identifiable record

. A ppropriate identifiers will be used to distinguish flight crew position in order to permit tracking individual performance, where possible, through first

- look evaluation (where appropriate) using proficiency training and evaluation. The certificate holder will report additional training sessions beyond the normally programmed training for any flight crew member.

 By individual task, maneuver, or procedure, as applicable.

 By type of evaluation, as follows:

o First

- look (if applicable)

,

o Proficiency t raining , and

o Proficiency e valuation.

 Rating s cale data shall be obtained for each task, maneuver, or procedure.

 For items requiring multiple attempts to acquire proficiency, the number of repetitions required. Rating s cale data shall be obtained for each repetition of a proficiency task, maneuver, or procedure.

The certificate holder shall provide the FAA with deidentified raw data on a monthly basis

, in accordance with POI requirement.

Th e monthly report will:

 Document the certificate holder’s own analysis of data acquired under this part

.

 During the second and subsequent years of data collection, incorporate comparisons of data between equivalent periods of the preceding years

.

 Identify a ny trends, common problem areas

, or potential deficiencies

.

 Include a description of corrective steps taken or warranted.

Implement ation of this program, when approved by the certificate holder’s POI, may be used to increase the period between evaluations by 1 month , period not to exceed 1 year

.THRR ARC Final Report D

– 8

Option 4: Implementation of a n SMS ; a 1

- month interval extension for a § 121.441 or § 135.297 flight evaluation. If the air carrier has implemented an SMS (FAA or ICAO recognized )

, a 1

- month interval extension for a §§ 121.441 or 135.293 flight evaluation is approvable.
Option 5: Implementation of FOQA or FDM (ICAO) program; a 1

- month interval extension for a § 121.441 or § 135.297 flight evaluation. If the air carrier has a FOQA or FDM (ICAO) program in place, a 1

- month interval extension for a §§ 121.441 or 135.297 flight evaluation is approvable

. Option 6: Enhanced Management Oversight program; a 1

- month interval extension for a § 121.441 or § 135.297 flight evaluation.

If a full 14 CFR part 119 qualified manage ment team (including safety manager for part 135 air carrier) is managing more than 75 percent of its work days and actually occupying a cockpit for less than 25 percent of work days, a 1

- month interval extension for a §§ 121.441 or 135.297 flight evaluati on is approvable.
Option 7: Demonstration of Individual Pilot Operational Tempo; a 1

- month interval extension for a § 121.441 or § 135.297 flight evaluation.

F or the purpose of this proposal

, operational tempo (OPTEMPO) is a term used to describe operatio nal attributes that impact a flight crew member’s KSAs with respect to aircraft operations.

Studies indicate that skill and knowledge decay is directly related to the frequency of use or practice of a given skill. Several studies cited by Ms. Monica Martin ussen and Mr. David R. Hunter, authors of Aviation Psychology and Human Factors , substantiate this as referenced below.

 Skill decay is defined as a loss or decay of trained or acquired skills or knowledge after periods of nonuse. T he inverse is intuitive : as trained/acquired skills or knowledge are exercised ; abilities improve

, or at least skill decay is prevented.

 Difficulties with skill decay are exacerbated by the current generation of cockpit automation , which tends to place pilots in a passive, monit oring mode.

 Some skills decay faster than others. Memories and cognitive skills tend to be lost much faster than motor skills.

 Basic visual flight skills remain intact for up to 36 months

, while skills relating to instrument flight procedures fall below a cceptable levels within 12 months for about half the pilots in the study.THRR ARC Final Report D

– 9

This led to the supposition that frequency of activity by a pilot in critical phases of flight should be considered when determining the optimum training interval between recurrent t raining events. The THRR ARC recommends giving a 1

- month assessment interval extension for each of the following OPTEMPO thresholds, not to exceed a 12

- month interval for any individual pilot:

 180 or more take

- offs and landings within the previous 6 month s, and

 36 instrument approaches in actual instrument meteorological conditions within the previous 6 months.

The operator must establish a tracking system to substantiate that their pilots meet these thresholds.

This assessment interval credit recognizes t hat pilots who fly more frequently than others should sustain their motor and cognitive skills better than those pilots who fly less often. However, any pilot that fails an assessment would be required to enter a ―short

- cycle‖ as described earlier due
to u nsatisfactory a ssessment. After satisfactorily completing the short

- cycle requirements, the pilot may return to the previous extended interval based on OPTEMPO factors if he or she meets the prescribed thresholds.

Options 1

- 7 may be used only if the foll owing requirements are met:

 The pilot successfully passed the previous two proficiency checks

.

 The pilot is single

- airplane (fleet) qualified

.

 The pilot is a full

- time employee

.

 A n FAA

- approved training program for the air carrier is used.

In all cases, da ta collection and analysis requirements necessary to comply with the above safety programs listed above must substantiate that extending training intervals beyond current regulatory requirements is appropriate.THRR ARC Final Report E

– 1



## APPENDIX E

— DISSENT ING STATEMENT

CAPA was a ls o a member of the FOQ ARC and submitted the following dissenting statement to specific recommendations made by that ARC. To the extent that the THRR ARC discussed and made recommendations based upon the FOQ ARC’s recommendations for an academic credit sys tem, CAPA also dissented to the THRR ARC’s recommendations. CAPA provid ed a copy of its prior statement to serve as its dissenting opinion for the THRR ARC recommendation related to academic credit

.THRR ARC Final Report E

– 2THRR ARC Final Report E

– 3THRR ARC Final Report E

– 4THRR ARC Final Report E

– 5THRR ARC Final Report E

– 6THRR ARC Final Report E

– 7THRR ARC Final Report E

– 8THRR ARC Final Report E

– 9THRR ARC Final Report E

– 10THRR ARC Final Report F

– 1



## APPENDIX F

— CHARTERTHRR ARC Final Report F

– 2THRR ARC Final Report F

– 3THRR ARC Final Report F

– 4