# Human Factors

– HWG

Final Report

June 15

, 2004

HF

- HWG

Human Factors

- Harmonization Working Group
Flight Crew Error / Flight Crew Performance

Considerations in the
Flight Deck Certification Process

Federal Aviation Administration

– USA
European Aviatio n Safety Agency

– Europe

Document
Title Final Report

Version number 2. 7 Number of pages 18 plus 7 Appendices
Status Final version

Authors

Editor s

– Dr. P. Emmerson

, Dr. R.C. Graeber BAE Systems The Boeing Company Concurred by

Dr. R.C. Graeber

–

Co

- Chair The Boeing Company

Mr. D. Ronceray

–

Co

- Chair Airbus IndustrieHFHWG Final Report Final Version, 2.7 June 15 , 2004

2 TABLE OF CONTENTS



### Item Description Recommended Action

1 Working Group Report Provided for TAEIG Consideration

2 Appendix 1

- Text of Proposed Advisory Material Provided to TAEIG for Consideration

3 Appendix 2

- HF HWG Process Provided to TAEIG for supporting

information

4 Appendix 3

- Top 33 Deficiencies Identified by HF HWG Provided to TAEIG for supporting

information

5 Appendix 4

- Membership, Meetings Provided to TAEI G for supporting

information

6 Appendix 5

– Description of Relevant Regulations Provided to TAEIG for supporting

information

7 Appendix 6

–List of Typical

Human Errors Provided to TAEIG for supporting

information

8 Appendix 7

– FAA Position Paper Provi ded to TAEIG for supporting

informationHFHWG Final Report Final Version, 2.7 June 15 , 2004

3



### 1. BACKGROUND:

S AFETY I SSUE A DDRESSED /S TATEMENT OF THE P ROBLEM (1) What prompted this rulemaking activity (e.g., accident, accident investigation, NTSB recommendation, new technology, service history, etc.)? What focused
our attention on the issue?

Recent statistics (Boeing, 2003) indicate that there were 139 commercial jet aircraft

accidents world

- wide between 1993 and 2002. In 67% of these, flight crew error was cited as a major factor by the investigatin g authority. These statistics remain consistent across accident categories that have been recently examined, such as controlled flight into terrain, loss of control, etc.

For current generation aircraft the world

- wide rate for hull

- loss/fatal accidents ov er the last 10 years has remained static at about 1.5 accidents per million departures (Boeing 2003). Without a decrease in this rate, a substantial increase in the number of flights over the coming

decades will result in an increase in hull losses. Reduc ing crew error , in addition to the implementation of advanced safety technology such as TAWS, provides the greatest opportunity to accomplish a rate decrease. At the same time, the vulnerability of the aviation system to flight crew error may increase in t he future due to a number of factors, including increased traffic density and the growing diversity of cultures operating aircraft worldwide.

In order to develop effective strategies for reducing flight crew error it is important to look

behind the label of “error” to find the contributing factors that have led crews to make errors, miss the detection of errors, or recover late from errors that have safety consequences. Contributing factors can often be

-- and have been

-- identified from design, qualifi cation, operations (e.g., procedures, maintenance), and other areas. Such an analysis led both the 1996 FAA Human Factors Team report and the JAA Human Factors Steering Group to recommend that measures be taken to address one set of factors contributing t o flight crew error, specifically the design and certification of transport category flight decks. As a result, the Harmonization Working Group on Flight Crew Error/Flight Crew Performance Considerations in the Flight Deck Certification Process (i.e., the Human Factors HWG) was

established in the fall of 1999 and tasked as follows:

Step 1 . Review relevant existing material (FAR/JAR 25 regulations, advisory material, policy, and related references) and make recommendations about what regulatory standards

a nd/or advisory material should be updated or developed to consistently address design

- related flight crew performance vulnerabilities, and prevention and management (detection, tolerance, and recovery) of flight crew error. This review should be accomplish

ed in the context of both the Type Certification and Supplemental type Certification processes. Step 2. Based on results of the Step 1 review, recommend new advisory material to address design

- related vulnerabilities of flight crew performance and the man agement of flight

crew error. Step 3. Recommend (or plan for the development of) new regulatory material to address design

- related vulnerabilities of flight crew performance and the management of flight crew error. If rulemaking is not recommended, provid e reasons and propose non

rulemaking alternatives. Step 4. Recommend an implementation plan for products of Steps 1

- 3, and develop Terms of Reference for fulfilling the plan. Step 5. During accomplishment of these steps, identify implications for qualifi cation and operations for communication to appropriate groups.HFHWG Final Report Final Version, 2.7 June 15 , 2004

4 At the same time, the European Joint Airworthiness Authorities issued an Interim Human Factors Policy (JAA INT POL 25/14) for certificating flight deck systems. It also supported forming the HF HWG with the expectation that its output should cover the scope and intent of the Interim Policy in such a way that would eventually enable a harmonized solution to be put in place and replace it.

(2) What is the underlying safety issue to be addresse d in this proposal?

Detailed analyses of past incidents and accidents have shown that the majority are related to crew performance failures in the form of a series of errors, sometimes in combination with system failure, which led to severe safety conse quences. In some cases the analysis of contributing factors has shown that the equipment was inadequately designed to support flight crew performance in a manner that would enable continued safe flight. This proposal addresses the contribution of flight de ck equipment design to the occurrence of those flight crew errors that can significantly impair safe operations in commercial transport aircraft.

(3) What is the underlying safety rationale for the requirement?

This requirement is intended to help to p revent the certification of unsafe design features that may lead flight crews to make errors, not detect errors, or recover too late from errors that may have serious safety consequences. Current requirements address the design of various features of speci fic flight deck equipment in a manner intended to assure that the crew can operate that particular device safely; however, there is no guidance or existing requirement that addresses the prevention and management of flight crew error in a comprehensive or integrated manner across the various flight deck components. The proposed requirement addresses in a comprehensive fashion those design characteristics that affect the occurrence and management of flight crew error. It applies across all flight deck equipm ent that provides information to the flight crew or that the crew uses to control the operation of the airplane.

(4) Why should the requirement exist?

Human error cannot be entirely prevented, nor is it predictable. While effective training can help m itigate some of the threat, there must be a Part 25 requirement to address it responsibly within the design process, especially when novel features are proposed. Such a requirement must recognize that crew performance is by its very nature an integrated re sponse to a particular operational situation. As a result, flight crew error typically involves interaction with more than one piece of equipment. Consequently, in order to address the safety risk resulting from flight crew error designers must undertake a n integrated approach to minimize its occurrence and manage its effects. A list of typical human errors relevant to flight deck performance is given in Appendix 6.

C URRENT S TANDARDS OR M EANS TO A DDRESS (1) What are the current regulations relative to this subject? (Include both the FAR’s and JAR’s.)

The proposed rule is intended to define requirements that extend or generalize those in the regulations listed below, however it is not proposed that any of these regulations be amended

or deleted. A more det ailed description of the regulations is given in Appendix 5.

14 CFR 25: 14 CFR 25.671(a) Pilot compartment (as amended 4/30/1965).

14 CFR 25.771(a), (c) and (e) Pilot compartment (as amended 4/30/1965).HFHWG Final Report Final Version, 2.7 June 15 , 2004

5 14 CFR 25.777 Cockpit controls (as amended 12/1/1 978).

14 CFR 25.779 Motion and effect of cockpit controls (as amended 8/20/1990).

14 CFR 25.1301 Function and installation (10/3/1964).

25.1309 (a) and (c) Equipment, systems, and installations (as amended 9/1/1977).

14 CFR 25.1523 Minimum flight c rew (as amended 5/28/1965).

14 CFR 25 Appendix D Criteria for determining minimum flight crew (as amended 5/28/1965).

EASA CS 25:

CS 25.1301 Function and installation.

25.1309 Equipment, systems, and installations.

(2) How have the regulations been app lied? (What are the current means of compliance?) If there are differences between the FAR and JAR, what are they and how has each been applied? (Include a discussion of any advisory material that currently exists).

Individually and collectively the rule s itemized above do not fully address design requirements relating to flight crew performance and flight crew error under all expected operating conditions or for all equipment and systems with which flight crews interact. Specifically:

• § 25.671(a) applies only to flight controls.

• §25.771(a) addresses only avoidance of unreasonable concentration and fatigue and does not address other causes of flight crew error and performance degradation.

• §25.771(c) only requires that means to control the airplane must be functionally equivalent for each pilot position, and does not set criteria for these means.

• §25.777 defines requirements only for controls and only with respect to their locations and certain physical attributes, and does not comprehensively address contro l characteristics affecting all the aspects of flight crew error and performance that the proposed rule does.

• §25.779 defines requirements for the motion and effect of controls, and does not comprehensively address control characteristics affecting all the aspects of flight crew error and performance that the proposed rule does.

• §25.1301 is very general, and is not consistently applied to address the safety issues covered by the proposed rule.

• §25.1309(a) applies only to required equipment, and because it i s very general it has not been consistently applied to address the safety issues covered by the proposed rule.

• §25.1309(c) defines requirements that are applicable only in the event of equipment failure, while the requirements of the proposed rule apply to all expected operating

conditions.

• §25.1523 and Appendix D address workload only with respect to establishing the minimum flight crew, in contrast to the proposed rule which encompasses workloadHFHWG Final Report Final Version, 2.7 June 15 , 2004

6 issues that may not be sufficient to affect the size or comp osition of the minimum flight

crew.

Recent and current practice has been that compliance with requirements itemized above has followed the general guidance found in FAA ACs 20

- 88A, 25

- 7A, 25

- 11, and ACs dealing with specific types of equipment and system s, or with the guidance in JAA Advisory Circulars ACJ 25.1329. There is no specific, comprehensive guidance available defining acceptable means of compliance for the rules identified above. In addition, advances in crew interface and systems technologies have outpaced both the existing FAA and JAA/EASA regulations and guidance. Flight crew interface related issue papers and interim policy guidance have been used to fill these gaps in the regulatory and acceptable means of compliance material.

JAA Interim

Policy is used by EASA. Assessment of the flight deck to ascertain whether novel features are present is conducted in accordance with IR 21A.16B and if necessary the certification team raises a Special Condition as specified in the JAA Interim Policy. The Special Condition is being used in several certification program s.

(3) What has occurred since those regulations were adopted that has caused us to conclude that additional or revised regulations are necessary? Why are those regulations now inadequate?

As noted above, advances in crew interface and systems technologies have presented issues not addressed in the current regulations and advisory material. JAA sought to address this by issuing JAA INT POL 25/14, “Human Factors Aspects of Flight Deck Desig n.” This Interim Policy, and the Special Condition that results from its general application on European certification programs, has created a de

- facto lack of harmonization.

In order to comply with ER for airworthiness, the EASA needs to modify CS

- 25. T he consequence is at least to incorporate the JAA INT POL 25/14 or equivalent into CS 25.

I F NO REGULATIONS CUR RENTLY EXIST : (1) What means, if any, have been used in the past to ensure that this safety issue is addressed? Has the FAA relied on issue pap ers? Special Conditions? Policy statements? Certification action items? Has the JAA relied on Certification Review Items? Interim Policy? If so, reproduce the applicable text
from these items that is relative to this issue.

The JAA published an inte rim policy INT/POL 25/14 on 15 March 2001. This interim policy proposes a generic special condition, and guidelines for acceptable means of compliance to deal with the Human factors aspects of flight deck design when novel features are introduced.

The INT POL states that:

“If novel features are present, the Team shoul d raise a Special Condition as follows: a) The design of the integrated Flight Deck Interface must adequately address the foreseeable performance, capability and limitations of the Flight Cre w. b) More specifically the Authority must be satisfied with the following aspects of the Flight Deck Interface design: i) ease of operation [including automation] ii) the effects of crew errors in managing the aircraft systems, including the potential for error, the possible severity of the consequences, and the provision for recognition and recovery from error iii) task sharing and distribution of workload between crew members during normal and abnormal operation iv) the adequacy of feedback, including cl ear and unambiguous: • presentation of information • representation of system condition by display of system statusHFHWG Final Report Final Version, 2.7 June 15 , 2004

7 • indication of failure cases, including aircraft status • indication when crew input is not accepted or followed by the system

• indication

of prolonged or severe compensatory action by a system when such action could adversely affect aircraft safety.”

The policy has been transferred to the EASA.

(2) Why are those means inadequate? Why is rulemaking considered necessary (i.e., do we need a

general standard instead of addressing the issue on a case

- by

case basis?)

Regulations on a case

- by

- case basis have been used in the past to respond to safety concerns and accident investigation boards’ recommendations. This is a reactive approach. In or der to address novel issues raised by new technologies and new operating environments, a proactive approach is needed because neither regulators nor industry can predict what those will be in

the future.



### 2. DISCUSSION of PROPOSAL

A. S ECTION

- BY
- S ECTIO N D ESCRIPTION OF P ROPOSED A CTION (1) What is the proposed action? Is the proposed action to introduce a new regulation, revise the existing regulation, or to take some other action?

The proposed action is to publish a new regulation and advisory material that provides guidance for this and certain existing rules, together with some recommended tasking for other groups to address deficiencies identified by this HWG in the following areas:

• Avionics

• Icing

• AFM

In proposing this action, the HF HWG also reque sts a Phase 4 review of the draft NPRM package and proposed AC material prior to publication, as per the Fast Track process.

(2) If regulatory action is proposed, what is the text of the proposed regulation?



### FAR/CS§ 25.1302 Installed Systems and Equipm ent for Use by the Flight Crew

This section applies to installed equipment intended for the flight crewmembers’ use in the operation of the airplane from their normally seated position on the flight deck. This installed equipment must be shown, individu ally and in combination with other such equipment, to be designed such that qualified flight crewmembers trained in its use can safely perform their tasks associated with the intended function by meeting the following requirements:

(a) Flight deck contro ls must be installed and information necessary to accomplish these tasks must be provided

. (b) The flight deck controls and information intended for the flight crew use must:

i. Be presented in a clear and unambiguous form, at resolution and precision

approp riate to the task, andHFHWG Final Report Final Version, 2.7 June 15 , 2004

8 ii. Be accessible and usable by the flight crew in a manner consistent with the urgency, frequency, and duration of their tasks, and

iii. Enable flight crew awareness, if awareness is required for safe operation, of the

effects on the aircraf t or systems resulting from flight crew actions.

(c) Operationally

- relevant behavior of the installed equipment must be:

i. Predictable and unambiguous

, and

ii. Designed to enable the fligh t crew to intervene in a manner app ropriate to the

task.

(d) To the extent practicable, the installed equipment must enable the flight crew to manage errors resulting from flight crew interaction with the equipment that can be reasonably expected in service, assuming flight crews acting in good faith. This subparagrap h does not apply to skill

- related errors associated with manual control of the airplane.

(3) If this text changes current regulations, what change does it make? For each change: (a) What is the reason for the change? (b) What is the effect of the change?

This proposed text adds a new paragraph to existing regulations. The proposal does not replace or modify any text that curren tly exists in the regulations. The overall reason for adding this paragraph is as follows: 14 CF R Part 25 / CS

- 25 contains regu lations for design of flight deck equipment that are system

- specific (e.g. 25. 777, 1321, 1329, 1543 etc.), generally applicable (e.g. 25.1301(a), 1309(c), 771(a)), and for establishing minimum flight crew in 25.1523 a nd Appendix D. The proposed 14 CFR /CS 25.1302 augments the currently existing generally applicable rules by adding more explicit requirements for design attributes related to flight crew performance, including avoidance and management of flight crew error s . In addition, other ways to avoid an d manage flight crew error are regulated through the rules that govern licensing and qualification of pilots and aircraft operations (e.g. Parts 61, 91, 121, 135 etc. for FAA and JAR FCL and JAR Ops for JAA/EASA) . Taken toge ther, the proposed regulation

a nd existing rules as mentioned above represent complementary approaches to provide a high degree of safety.

This complementary approach is important and is based on recognition that equipment design, training/licensing/qualification, and operations/proced ures each provide a safety contribution to risk mitigation and that an appropriate balance among them is needed. In the past there have been cases where design characteristics known to contribute to error were accepted because

of the rationale that traini ng or procedures would mitigate that risk. However, we now know that often can be inappropriate. Similarly, it would not be appropriate to require the equipment design to provide all risk mitigation, because of the unintended consequences. For example, if a pilot misunderstands a controller's clearance, that does not mean we should mandate datalink or some other design solution as a Part 25 requirement. In the current regulations, some error mitigations are required of the equipment as part of the opera ting rule requirements (TAWS, etc), not airworthiness requirements

- and that is appropriate. The HWG did not find a safety case that would justify putting all of the error management into design requirements (and mandate it through Part 25).

As stated

, a proper balance is needed among design approval requirements in the minimum airworthiness standards embodied in Part 25, and the requirements for training/ licensing/ qualification and operations/procedures. The proposed regulation is written and scoped with the intention of achieving that appropriate balance.HFHWG Final Report Final Version, 2.7 June 15 , 2004

9 Change by change discussion:

Introductory Sentence :

This sentence state s the requirement that the provisions of this paragraph apply to each item of installed equipment that is intended for the flight crew’s use in operation of the aircraft from their normally seated positions in the flight deck.

“ Intended for the flight crew’s use in the operation of the aircraft from their normally seated position ” means that the intended function of the inst alled equipment includes use by the flight crew in the operation of the aircraft; e.g., a display that provides information for the flight crew members to navigate. The phrase “flight crewmembers” is intended to include any or all individuals making up th e minimum flight crew as determined for compliance with §25.1523. The phrase “from their normally seated position” means the flight crewmembers are seated at their normal duty stations for operating the aircraft. It is intended to limit the scope of this requirement so that it does not address systems or equipment that are not used while performing their duties in normal and non

- normal conditions in the operation of the aircraft. For example, this paragraph is not intended to apply to items such as certa in circuit breakers or maintenance controls intended for use by the maintenance crew (or by the flight crew when they are not operating the aircraft).

Rationale for the change : The sentences are intended to describe the scope of applicability

of this new

paragraph and its requirements. The rationale for each element of this sentence is

included in the discussion above.

Second sentence : The phrase “it must be shown” means that the applicant is required to provide sufficient evidence to support complianc e determinations for each of the requirements in the proposed rule. This language “must be shown” is included for harmonization with JAA/EASA rulemaking objectives. It is not intended to require a showing of compliance beyond what would be required accor ding to § 21.21(b). Accordingly, the extent of demonstrations, tests or data needed to show compliance with the proposed rule is not expected to entail more extensive or onerous efforts for items that are simple or similar to previously approved equipment and installations

.

The phrase “individually and in combination with other such equipment” means that the requirements of this paragraph must be met when the equipment is installed on the flight deck with other equipment and that it must not cause other equipment to be unable to comply with these requirements. For example, a display must not be designed such that it provides inconsistent or conflicting information relative to other installed equipment.

In addition, the provisions of this paragraph assu me a qualified flight crew trained in the use of the installed equipment. This means a flight crew that is allowed to fly the airplane by meeting the qualification requirements in the operating rules for the relevant authority (in the US, the FAA)

– so th e design must meet these requirements with qualified flight crewmembers allowed to operate the airplane. If type design or supplemental type design approval is sought before a training program is accepted, the applicant should document any novel, complex, or highly integrated design features and assumptions made during design that have the potential to affect training time or flight crew procedures. The rule and associated material are written assuming that either these design features and assumptions, or knowledge of a training program (proposed or in the process of being developed) will be coordinated with the appropriate operational approval authorities when judging the adequacy of the design.HFHWG Final Report Final Version, 2.7 June 15 , 2004

10 The phrase that states “the equipment be designed so that t he flight crew can safely perform the tasks associated with the intended function of the equipment” applies in both normal and

non

- normal conditions. Tasks intended for performance under non

- normal conditions are generally those prescribed by non

- normal ( including emergency) flight crew procedures. The phrase “safely perform the tasks” is intended to describe the safety objective that the equipment design enables the flight crew to perform the tasks with sufficient accuracy and in a timely manner, and wit

hout unduly interfering with other required tasks. The phrase “tasks associated with the intended function” is intended to characterize either tasks required to operate the equipment or tasks for which the equipment’s intended function provides support.

Rationale for the change : The sentences are intended to describe the scope of applicability of this new paragraph and its provisions. The rationale for each element of this sentence is

included in the discussion above.

25.1302 (a) : This sentence requi res that, for any flight deck equipment identified in the lead sentences of the proposed rule, appropriate controls must be installed and the necessary information must be provided. In addition, the controls and information displays must be

sufficient to a ccomplish the tasks of the flight crew.

Rationale for the change: Based on a review of the deficiencie s in the existing regulations,

t he HF HWG concluded that it is necessary to explicitly require that the information and

control needs associated with th e tasks of the flight crew be met. Although this may seem obvious, it is necessary to be explicit because this requirement is not included in the existing regulations.

25.1302 (b) This subparagraph addresses the requirements for the flight deck controls and information that are necessary and appropriate to accomplish the flight crew tasks, as determined for compliance with subparagraph (a). The intent is to assure that the design of the information presentation and controls is usable by the flight crew. In particular, this subparagraph seeks to reduce the occurrence of design

- induced flight crew errors by imposing design requirements on flight deck information presentation and controls. Subparagraphs (1) through (3) specify these design requirements.

Specifically, the proposed 25.1302(b) (1) requires that the information and controls be provided in a clear and unambiguous form, at resolution and precision appropriate to the task. For information, “clear and unambiguous” means that it can be perceived c orrectly (e.g., is legible) and that the information can be correctly understood in the context of the flight crew tasks, and supports the flight crew’s ability to carry out those tasks. For controls, the requirement for “clear and unambiguous” presentatio n means that the crew must be able to use them appropriately to achieve the intended function of the equipment without contributing to crew confusion or errors that could degrade safe operation.

The general intent is to foster the design of equipment c ontrols whose operation is intuitive, consistent with the effects on the parameters or states they affect, and compatible with the operation of other controls in the flight deck. “Resolution and precision appropriate to the

task” means that the informatio n or control must be provided or operate at a level of detail and accuracy appropriate to the accuracy in accomplishing the task expected of the flight crew. Insufficient resolution or accuracy would mean that the flight crew could not perform the task

ad equately, and excessive resolution has the potential to make a task too difficult due to readability issues, or by implying that a task must be accomplished more accurately that it actually must.

Subparagraph (2) requires that the information and contro ls be accessible and usable by the flight crew in a manner consistent with the urgency, frequency, and duration of their tasks.HFHWG Final Report Final Version, 2.7 June 15 , 2004

11 This means, for example, that controls used more frequently or urgently must be readily accessed (e.g., take fewer steps or act ions to perform the control function). Controls needed less frequently or urgently may be acceptable if they are less accessible. It also means that controls used less frequently or urgently should not interfere with controls that are used more urgently o r frequently.

Subparagraph (3) requires that the equipment presents information so that the flight crew can be aware of the effects on the aircraft or systems that result from flight crew actions, if that awareness is required for safe operation. The inte nt of this requirement is to assure that flight deck equipment provides feedback to the flight crew about system or aircraft states that result from flight crew actions, so that the flight crew can detect their own errors.

Rationale for the change : The requirements in 1302(b) are intended to address the following deficiencies of existing rules:

§ 25.771 (a) addresses flight crew interface requirements for controls, but does not include criteria for information presentation.

§ 25.777 (a) addresses flight crew interface requirements for controls, but only with respect to where they are located.

§§ 25.777(b) and 25.779 address direction of motion and actuation for controls but do not encompass new types of controls such as cursor control devices. These rule s also do not encompass, for example, control functions incorporated into displays via menus, which have been shown to have a potential for significant accessibility issues.

§ 25.1523 and Appendix D address workload and task performance, but only with respe ct to determining minimum crew. These issues are known to have significant safety implication beyond the size of the minimum crew, for example with respect to flight crew error.

Design requirements about information and controls are necessary:

• to properly

support flight crew in planning their tasks,

• to make available to the flight crew appropriate, effective means to carry

- out planned actions, and

• to enable the flight crew to have appropriate feedback information about the effects of

their actions on the aircraft.HFHWG Final Report Final Version, 2.7 June 15 , 2004

12 25.1302 (c) This subparagraph requires that the equipment must be designed so that installed equipment behavior that is operationally relevant to the flight crew’s tasks is: (1) Predictable and unambiguous, and (2) Designed to enable the fligh t crew to intervene in a manner appropriate to the task (and intended function). It is intended to define requirements for system behavio r that could cause or contribute to flight crew error or otherwise prevent the flight crew from being able to properly perform their tasks. The phrase “operationally

- relevant behavior” is intended to convey what the equipment does from the flight crew’s perspective and perception that is necessary for crew awareness of the system operation or planning, or necessary to ope rate the system. This is intended to distinguish it from the logic within the system design, much of which the flight crew does not know or need to know and should be transparent to them.

Subparagraph (1) is intend ed to describe that the behavio r needs to be such that the flight crew can know what the system is doing and why. “Predictable and unambiguous” means that a qualified flight crew can retain enough information about what the system will do under

foreseeable circumstances as a result of crew act ion or as a result of changing situation such that they can safely operate the systems. It is necessary for this behavior to be unambiguous because the pilot actions may have different effects on the airplane from different states or in

different circumsta

nces.

Subparagraph (2) requires that the design must be such that the flight crew will be able to take some action, or to change or alter an input to the system as appropriate to the task.

Rationale for the change: Improved technologies have increased safety and performance, but service experience has shown that flight crew confusion can result from equipment behavior (especially from automated systems) that is excessively complex or dependent on logical

states or mode transitions that are not understoo d or expected by the flight crew. Such design

characteristics have been determined to contribute to incidents and accidents.

25.1302 (d ) This subparagraph addresses the reality that even well

- trained, proficient flight crews using well

- designed systems will make errors, so the equipment must be designed to enable the flight crew to manage errors that result from their interaction with the installed equipment. Errors “resulting from flight crew interaction with the equipment” means those

errors in some way attributable to or related to the design of the controls, information, or equipment behavior (e.g., indications and controls that are complex and inconsistent with each other or other systems on the flight deck, or a procedure that is inconsistent with the design of the equipment) are considered to be within the scope of this regulatory and advisory material.

“Managing errors” means that the design of the equipment must enable the flight crew to detect and/or recover from the errors resulting from fli ght crew interaction with the equipment, or ensure that effects of such flight crew errors on the airplane functions or capabilities are evident to the flight crew and continued safe flight and landing is possible, or discourage such flight crew errors by using switch guards, interlocks, confirmation actions, or other effective means, or preclude the effects of errors through system logic, redundant, robust, or fault tolerant system design.

The requirement to manage errors applies to those errors that can b e reasonably expected in service from qualified and trained flight crews. The term “reasonably expected in service” means those errors that have been seen in service with similar or comparable equipment or which can be projected to occur based on general experience and knowledge of human performance capabilities and limitations related to the use of controls, information, or system logic of the type being assessed.

The statement “This subparagraph does not apply to skill

- related errors associated with manu al control of the airplane” is intended to exclude errors resulting from flight crew proficiency inHFHWG Final Report Final Version, 2.7 June 15 , 2004

13 control of flight path and attitude with the primary roll, pitch, yaw and thrust controls, and which are related to the design of the flight control systems . These issues are considered to be adequately addressed by existing rules such as 14 CFR 25 Sub

- part B and § 25.671(a).

It is not intended that the design be required to compensate for deficiencies in flight crew training or currency/recency of experience , assuming at least the minimum requirements for flight crew certification for the intended operation based on certification of flight crewmembers (per 14 CFR or JAR Ops or JAR FCL, for the US and Europe, respectively) as

discussed above.

This requiremen

t is intended to exclude management of errors as a result of decisions, acts, or omissions by the flight crew that are not in good faith. This exclusion is intended to avoid imposing requirements on the design to accommodate errors that are committed with malicious or purely contrary intent. The proposed rule is not intended to require the applicant

to consider errors that are a result of acts of violence or threats of violence.

This “good faith” exclusion is also intended to avoid imposing requirements on the design to accommodate errors that are due to obvious disregard for safety by a flight crewmember.. However, it is recognized that errors committed intentionally may still be in good faith and could be contributed to by design characteristics under cer tain circumstances; for example, a poorly designed procedure that is not compatible with the controls or information provided to the flight crew.

Requiring that errors be manageable only to the extent practicable is intended to address both

economic and operational practicability. The intent is to avoid imposing requirements without considering the economic feasibility and commensurate safety benefit.

In addition, it is intended to address operational practicability, to avoid introducing error
management into the design that make operation of the flight deck or airplane more difficult (for example too many guards or interlocks on means to shut down an engine, when in some instances the flight crew is required to do this rapidly) or that take away authorit y or means for the flight crew to intervene or carry out an action when it is their responsibility and best good faith judgment.

Rationale for the change: Service experience has shown that errors will occur, even with proficient, properly trained, and w ell rested flight crews operating well

- designed systems. Such errors have been found to have contributed to transport accidents and incidents. Therefore, managing the errors that do occur (that are a result of flight crew interaction with the equipment t hat can be reasonably expected in service) is an important safety objective addressed by this subparagraph. Even though the scope of applicability of this material is limited to errors for which there is a contribution from, or relationship to, design, th is rule paragraph is expected to result in design changes that contribute to safety (such as "undo" functions in certain designs, among others.)

B. A LTERNATIVES C ONSIDERED (1) What actions did the working group consider other than the action proposed? Explain alternative ideas and dissenting opinions.

After the initial phases of the HFHWG task to identify deficiencies in the regulations, TAEIG requested that HFHWG develop not only recommendations, but also regulatory and guidance

material. Section 4 of this report addresses development of the advisory material.

Several regulatory approaches were considered to address the highest priority deficiencies established during the initial phases of the tasking. Each of the following approaches to rules

we re debated and considered. The final version of the rule recommended in this report contains elements of the JAA Interim Policy, the “error” rule approach, and the “designHFHWG Final Report Final Version, 2.7 June 15 , 2004

14 attributes” approach. Significant elements of the “error” approach that were not u sed in the final recommendation are explicitly included in the recommended draft AC, and use the last paragraph of the final recommended rule to provide a regulatory basis.

Pros and Cons for various approaches to new or modified regulations are shown belo w:

Adopting JAA Interim policy

Pros:

• The JAA Interim Policy is already published and is being applied.

• It addresses a broad set of design considerations.

Cons:

• Does not establish or describe a minimu m level of safety and so would require

considerable re

- drafting to become a rule.

• It is w

ritten as a policy, to be invoked for novel items as a CRI, not as an AC/AMC with MOC, etc.

• The scope of a rule based on the JAA IntPol is difficult to establish.

• The JAA IntPol addresses issues beyond Part 25 (mainten ance and cabin crew functions).

• Methods of compliance have been defined only to a limited extent and at a general

level.

• It is invoked for novel items only.

The “rule” part of the Interim Policy lists several important considerations to be used as part

of the process of evaluating or analyzing the acceptability of a design. The language itself does not establish or describe a minimum level of safety and so would require considerable

re drafting to become a rule. However, many of the considerations are embo died in the final

recommended version of the new rule.

A new AC/ACJ based on existing regulations, with no new regulations or changes

Pros:

• A number of rules currently in P art 25 address the problem of flight deck design from both prescriptive and genera lly applicable perspectives, as well as governing how to establish the minimum flight crew. Most do have not guidance, and a new AC

would address that.

• Providing missing guidance for these rules could improve the ability of regulators to interpret and enf orce them.

• 25.1523 and appendix D in particular have much of the language needed to address error in terms of avoidance, recognition, and recovery.

Cons:

• 25.1523 and Appendix D are limited to the purpose of establishing minimum flight

crew.

• Remaining rules are seen by regulators as too general, and therefore too arguable and easily contested by a determined applicant attempting to defend a “poor” design. For example, there are no standard interpretations for “intended function” (25.1301) in this context, n or for “unreasonable concentration or fatigue” (25.771(a)).

• On the whole, this approach was not seen by regulators to be sufficiently targeted at

the identified deficiencies to be effective and enforceable

.

Changes to multiple existing rulesHFHWG Final Report Final Version, 2.7 June 15 , 2004

15 Pros:

• W ould require no new rules

• S ystem specific rules are generally easier to interpret due to narrower context

Cons:

• Would be necessary

to make changes to nearly every system specific rule

• Many existing rule are prescriptive, so effect would be limited to those syst ems covered.

• The effect of adding specific HF guidance to general rules on the ir ori ginal intent and meaning were unknown.

• Some general rules are limited in purpose, e.g. 25.1523 addresses workload only with respect to establishing the minimum crew.

• Syste m specific rules would be unlikely to be able to address new interface technologies, or systems that do not exist today.

• Changes to multiple existing rules would be likely to lead to inconsistent approaches for different systems.

New rule with flight crew error as a central safety objective:

This approach was pursued diligently for over a year by HF HWG, developed through a

multitude of versions

, and debated and refined considerably. This approach was an attempt to address avoidance, recognition and de tection, recovery, mitigation of effects, and making effects of errors apparent to the flight crew.

Pros:

• This approach would directly addresses the findings of many safety studies by addressing error explicitly

• A new rule explicitly addressing flight cre w error would be global, intended to focus on the result of bad design and thus provide complete coverage.

• This approach explicitly addresses the deficiency related to design

- related error and tasking of this group

– explaining the rationale for the rule w ould be more straightforward.

• The objective and logic of rule are explicitly stated.

• This approach allows multiple ways of achieving compliance and the corresponding safety objective.

• Some boundaries for scope of rule are addressed explicitly.

• Simplifies 2 5.1309(c) by removing second sentence related to flight crew error after

a failure.

• Apparent broad coverage.

Cons:

• Has face validity in that it addresses in name (error) the result of design inadequacies.

• It appears to regulate crew behavior rather than de sign.

• Part of the rule on avoidance of error required the applicant to prove a negative (i.e., that errors cannot occur).

• The WG was unable to develop language to appropriately limit scope.

• pushed the designer to consider the flight crew as an error genera tor

• It seems to invite probabilistic approaches to compliance, that are agreed not to have a basis or methodology.

• It could be interpreted as open ended; boundaries must be explicitly described.

• It may involve excessive use of subjective opinion for compli ance determination

• Ties to AC/AMC & coverage of topics (automation, integration, pilots characteristics) are not completely clear (except for error section)

• MoCs may be difficult or impossible to define completelyHFHWG Final Report Final Version, 2.7 June 15 , 2004

16 • Some applicants may attempt to use analys is as primary MoC and may not result in better design

New rule emphasizing Design Attributes known to Address Flight Crew Error

Pros:

• It addresses design characteristics that lead to error rather than error itself, placing it more directly within the int ent of Part 25, as well as potentially allowing for more focused discussions between the applicant and authority).

• It narrows the focus to certain aspects of design characteristics, allowing more

focused discussions.

• It has explicit ties to the flight crew tasks.

• It is potentially easier to tie to methods of compliance.

• It allows a more direct regulatory basis for sections such as integration and

automation.

Cons:

• The list of characteristics may not be complete, thus leaving “holes” that the error

based ru le would cover.

• Being based on design characteristics may not result in applicant taking a better, more structured, approach to the design process.

Dissenting Opinions There were no Minority Positions reported within the HF HWG

, and all members agreed wit h

the content of

the proposed regulation FAR/CS§ 25.1302

.



### 3. COSTS AND OTHER ISSUES THAT MUST BE CONSIDERED

C OSTS A

SSOCIATED WITH THE P ROPOSAL (1) Who would be affected by the proposed change? How? (Identify the parties that would be materially affect ed by the rule change

– airplane manufacturers, airplane operators, etc.)

Data not currently available

O THER I SSUES (1) Will small businesses be affected?

Small businesses may be affected if they do not already have a Human Factors capability

within t hem (and many do not) when designing new products/components for the user interface on the flight deck or when developing components in the role of a sub

- contractor as well as when demonstrating compliance of their product to the new requirement. They will either be required to develop the capability wi thin the company or contract relevant expertise to fulfill the requirements of the new rule

.

Companies engaged in the modification of the flight decks of older aircraft that subsequently require an STC will be required to conform to the new rule if the modifications made are significant and hence will also require access to human factors capability. (2) Will the proposed rule require affected parties to do any new or additional record keeping ? If so, expl ain.

Some extra record keeping will be required outlining the human factors design, development and testing process. Additional record keeping will also be needed to report on the associatedHFHWG Final Report Final Version, 2.7 June 15 , 2004

17 activity to demonstrate compliance with the new requirement (jus tification of the proposed evaluations, scenarios, etc, as well as compilation of the results to arrive at the final compliance statement). The extra record keeping required would be no different in its general nature, though, from that required for the de sign, development and testing of any new aircraft component, but the volume of work needed for an agreed certification document will certainly be considerably bigger than that needed for an internal manufacturer document. (3) Will the proposed rule create any unnecessary obstacles to the foreign
commerce of the United States

-- i.e., create barriers to international trade?

Data not currently available

(4) Will the proposed rule result in spending by State, local, or tribal governments, or by the private sector, that will be $100 million or more in one year?

Data not currently available



### 4. ADVISORY MATERIAL

A. I S EXISTING FAA OR JAA

ADVISORY MATERIAL A DEQUATE ? I S THE EXISTING FAA AND JAA

ADVISORY MATERIAL HA RMONIZED ?

Part of the basis for the forma tion of the FAA/JAA HF HWG w as that there was, in reality, very little advisory material to provide a possible means of compliance for the human factors aspects of a design for even the existing regulations. Any new proposed regulation related to

the huma n factors aspects of a design, as it affected the cockpit flight crew in the performance

of their duties , would

also need appropriate advisory material. The extent of the deficiency, in this aspect of the advisory material, was unknown.

As explained in Appendix 2, the HF HWG applied a cross

- basis analysis of the FAA and JAA regulatory and advisory material to determine their adequacy to address the flight crew error and performance vulnerabilities of a product or system's certifi cation within the flight deck. A

two

- pronged analysis looked at the actual regulatory and advisory material to evaluate its adequacy in addressing the flight crew error and performance vulnerabilities of the design. These two approaches were based on the development a nd validatio n of both a theory

- based and experien

ce

- based processes and topics. The theory

- based approach was accomplis hed by HWG subgroup B. The HWG simultaneously looked at a large collection of accident and

incident data to determine if their occurrence indicated inadequa cies in the advisory material. This experience

- based

approach was accomplished by HWG sub group C.

Each sub group conducted their respective analysis. During the acco mplishment of this process, sub group B identified 33 design characteristics of t he flight crew interface of a product /system /flight deck. These characteristics (shown in Appendix 3) were identified by

first review

ing a construct of basic human

- system input , action, and response model. They, and subseque ntly, sub group C as well, con ducted their analysis to identify if the advisory ma terial itself, or the accidents /incidents data, indicated a deficiency reflected in one of the 33 identified flight crew interface design characteristics.

As with the analysis of the regulations, trace ability of the analysis a nd analytical conclusions

were used throughout the HWG's analysis phase of the advisory material. This traceability

resulted in approximately 250 deficiencies being identified in the combined regulatory and advisory material, which

related to the flig

ht crew interface of a design. These deficiencies

served as the framework for the rest of the investigation and the final recommendations.HFHWG Final Report Final Version, 2.7 June 15 , 2004

18 Eventually, the HWG had reached a point where the analysis was complete. Both regulatory

and advisory material was now evaluated by the full HF HWG to determine which if any of the regulations or advisory m aterial was deemed deficient. This was accomplished by each member independently rating the deficienc y as to the magnitude of its in adequa cy re lative to certification, thereby filtering the r esults of the analysis

.

Using several analytical and statistical tools the HF HWG identified just 33 deficiencies that could be ranked high enough to form the basis for any recommended c hanges to the advisor y material (or regulations). At the highest summary level the HF HWG concluded that:

• While some rules are not written perfectly, all are adequate

-- if only marginally so.

• The rules are generally in place to legally define acceptability of the flight cre w interface issues. However, there was not guidance as to what was ( or is) an acceptable means to comply with this regulation subpart

.

• As noted elsewhere in this report, this analysis concluded that a single new rule sub paragraph was needed that would s et the overall flight crew interface a nd error tolerance requirement of a product/system/flight deck

.

• More AC/ACJ's were identified as devoid of recommended information for defining a means to determine if the flight crew interface of a product/system/fli ght deck was adequat e. A proposed single new AC/AMC , focused on the human performance aspects linked to a design, would indeed be needed.

The draft HF AC/AMC would be structured to focus on the deficiencies as they were identified. These could be groupe d in the following domains:

• Evaluation process

• Intended function and associated tasks

• Flight crew error management

• Controls

• Displays

• Automation

• Interface integration

Some of the principle deficiencies would be addressed by constructing a data package o f other industry teams. The deficiency topics that were addressed in this way were:

• Aircraft flight manual inclusion of the automation philosophy

• Review of the AC/ACJ addressing icing to insure adequate addressing of the specific flight crew interface to icing and associated systems.

• Certain aspects of displays and displays as controls

• Alerting

As far as harmonization is concerned, nothing is harmonized at the time of writing this report.

B. I F NOT , WHAT ADVISORY MATER IAL SHOULD BE ADOPTE D ? S HOUL D THE EXISTING MATER IAL BE

REVISED , OR SHOULD NEW MATER IAL BE PROVIDED ?

Human factors design guidelines already existing within Advisory Circulars are somewhat fragmented and, taken as a whole, do not present a unified, coherent approach to the human

fact ors engineering of the modern flight deck. Analysis of the human factors content of the advisory material reviewed as part of the analytical process showed its content to be specific to the application area with which the particular AC was concerned. As a direct result, any new advisory material should consider wider issues surrounding the human engineering of theHFHWG Final Report Final Version, 2.7 June 15 , 2004

19 flight deck, such as presenting a coherent, consistent interface to the pilots wherever possible. As such, the advisory material must describe general design principles that apply to all interfaces not just to interfaces with specific items of equipment.

The advisory material to complement the proposed rule should contain information on the generic principles of human centered design, for ex ample providing guidance upon such issues and the basic tenets of display design, design of minor controls and implementation of automated systems, etc. It should also include aspects missing in their entirety from previous ACs, such as a description of t he characteristics of the piloting population and methods for the human centered testing of flight deck equipment at the various stages of the design and development process. In accordance with the requirements outlined by the Aviation Rulemaking Advisory

Committee, any advisory material developed should also specifically address techniques for the minimization and assessment of design

- related flight crew error.

To help avoid potential conflicts in guidance with material in other advisory circulars, desi gn issues that are associated with specific items of flight deck equipment already covered in separate, existing ACs, should not be included in the proposed Human Factors AC/ACJ.

C. I NSERT THE TEXT OF TH E PROPOSED ADVISORY MATERIAL HERE ( OR ATTACH ), OR SUMMARIZE

THE INFORMATION IT W ILL CONTAIN , AND INDICATE WHAT F ORM IT WILL BE IN ( E

. G ., A DVISORY C IRCULAR , A DVISORY C IRCULAR

– J OINT , POLICY STATEMENT , FAA O RDER , ETC .)

T he HWG found several inadequacies in the current regulations and advisory material. As also

noted in section

2 , the HWG developed a single new rule that, when integrated with the existing rules, will allow the human factors aspects of a d esign to be better addressed. Further it developed a draft Human Factors AC/A MC to provide the firs t ever guidance for compliance with several existing rules as we ll as the proposed new rule. The pr oposed draft of the

AC/AMC is attac hed to this report as Appendix 1

.

There were no Minority Position reports within the HF HWG regarding the proposed

AC/AMC

, and all members agreed with its content

. However, the FAA subsequently requested that this report include a Position Paper which is provided a s Appendix 7

. The Position Paper was introduced on Apri l 16, 2004, following the final HWG meeting in February 2 004 during

which all comments on the 15 th draft were dispositioned as they had been for all previous

drafts (see Appendix 2)

. Therefore, the Position Paper has not been reviewed, d iscussed, or agreed to by the Working G roup . Its inclusion here is intended as information for the reader.