Human Factors: Status of Efforts to Integrate Research on Human Factors Into FAA's Activities
Released: 1996-06-27T00:00:00

Letter

Background

The study of human factors examines how humans interact with machines and other people (pilots, air traffic controllers, or design and acquisition personnel) and determines whether procedures and regulations take into account human abilities and limitations. Identifying chances for human error can reduce the need for later replacing or modifying equipment and procedures. Human factors affect the operation of all of FAA’s functions, including research, the acquisition of equipment, and safety. FAA’s work on human factors focuses on such issues as whether equipment is designed to enhance operators’ performance and minimize errors and whether the procedures used by air traffic controllers promote safe operations. For example, much of the information conveyed to pilots by air traffic controllers has been standardized to minimize the possibility of misunderstanding. (See app. I for a more complete definition of human factors and examples of how human factors have affected safety in specific situations.)

The Aviation Safety Research Act of 1988 directed FAA to augment its research on human factors and coordinate its work with that of NASA and Defense because the Congress believed that FAA did not have sufficient expertise in all areas of human factors. A report by the Office of Technology Assessment, cited in the House report on the act as the basis for the legislation, recommended that FAA allocate resources for developing its regulatory support staffs’ expertise in human factors and establish a focal point for human factors within the agency. In addition, the Congress has indicated through the budget process that research on human factors should be a priority in FAA’s overall research program. Figure 1 compares the congressional appropriations for FAA’s research on human factors with FAA’s funding requests.

FAA’s Organization for Human Factors

Key aspects of FAA’s human factors organization are the 1993 policy, the position of Chief Scientist, and the guidance on considering human factors in the acquisition process. The 1993 policy prescribes the roles and responsibilities of FAA’s assistant and associate administrators and program directors, as well as of the Human Factors Coordinating Committee (HFCC), including its chair, the Chief Scientist. The Chief Scientist also manages the Human Factors Division, which is housed in FAA’s Office of Aviation Research. (Fig. 2 illustrates the location of the Human Factors Division within FAA’s organizational structure.) On April 1, 1996, FAA changed its acquisition process and method of incorporating the consideration of human factors into that process. The creation of an Office of System Safety in 1995 may further affect the organizational structure for human factors.

Human Factors Policy

In October 1993, FAA issued an order for incorporating and coordinating the consideration of human factors throughout the agency. Under the order, assistant and associate administrators and program directors are responsible for, among other things, establishing formal procedures to ensure the systematic consideration of human factors within their organizations. However, FAA’s order does not prescribe the (1) methods for considering human factors, (2) minimal standards for incorporating human factors, or (3) requirements for seeking guidance on human factors from specialists that the administrators and directors are to follow. FAA officials in the three units where we held discussions—research and acquisitions, regulation and certification, and air traffic services—indicated that they have not fully established formal procedures for incorporating the consideration of human factors in their activities.

FAA created the Human Factors Coordinating Committee in 1989 to facilitate the agency’s work on human factors and enhance the use of information on human factors. However, according to the Chief Scientist, the committee is not a decision-making body, even though its members are designated by the agency’s assistant and associate administrators and program directors. Instead, the Chief Scientist said, the committee is primarily a forum for exchanging information. As the committee’s chair, the Chief Scientist carries out most of the committee’s responsibilities.

Chief Scientist

In addition to chairing the Human Factors Coordinating Committee, the Chief Scientist heads the Human Factors Division. This division is housed within the headquarters Office of Aviation Research, under the Associate Administrator for Research and Acquisitions. Among other things, the Human Factors Division develops policies on human factors that promote the productivity and safety of the national airspace system. The division is staffed by seven professional human factors specialists—six full-time and one part-time.

According to its mission statement, the Human Factors Division seeks to provide scientific and technical support for FAA’s research on human factors in civil aviation and its applications in the agency’s programs for acquisitions, regulation and certification, and air traffic services. However, we found that the Human Factors Division’s ability to provide this support depends on the extent to which the associate administrators and program directors use the division. FAA does not require the division—or any other unit with scientific and technical expertise in human factors—to review the quality of the work on human factors performed by other FAA units or contractors. FAA does not require its administrators to seek guidance from human factors specialists, such as those in the Human Factors Division. Although the scope of our audit did not include a detailed examination of the application of human factors in acquisitions, we have previously found inadequate technical oversight in FAA’s management of acquisitions. For example, in a previous review of FAA’s modernization program, we found that not following the technical principles of the human factors discipline in designing equipment delayed some projects. Instead of relying on the discipline’s objective criteria for measuring the performance of alternative designs, FAA consulted users’ preferences, only to find that its efforts were misdirected because different groups of users had different preferences.

Incorporating the Consideration of Human Factors in Acquisitions and Operations

Recent legislative and organizational changes may affect how the formal consideration of human factors is incorporated in the acquisition process and may strengthen the application of human factors in operations, such as safety.

Acquisitions and Human Factors

Several offices under the Associate Administrator for Research and Acquisitions are responsible for developing and acquiring new systems, such as air traffic control equipment. According to staff in the Human Factors Division, applying considerations of human factors increases a product’s or a process’s performance and efficiency while decreasing developmental, operational, and maintenance costs over the lifetime of the product or process.

To develop and deploy equipment more efficiently, in 1995, FAA adopted a new management approach that relies on integrated product teams,whose members include end-users, contractors, and all other parties responsible for developing or procuring new equipment or processes. As a first step in ensuring that human factors are considered in acquisitions, the Human Factors Division developed a requirement in FAA’s 1993 acquisition policy that all new acquisition projects include a human factors plan. Such a plan was to (1) describe how considerations of human factors should be applied and (2) document how a piece of equipment or a process should perform when operated as expected by the end-users.

However, on April 1, 1996, in response to new legislation exempting FAA from most federal procurement statutes, FAA implemented the Federal Aviation Administration Acquisition Management System, which superseded FAA’s 1993 acquisition policy. According to the initial guidance provided for this new system, human factors may be formally considered at an earlier stage in the acquisition process than previously, but this early consideration is not required. Furthermore, the extent to which human factors should be considered is not specified in the system’s guidance, nor is a separate plan for human factors required. There is no requirement for integrated product teams to obtain recommendations from human factors specialists.

Operations and Human Factors

According to some FAA human factors specialists, considering human factors is key to improving the safety of aviation operations. In 1990, the FAA Administrator testified before the Congress that the agency’s objective in aviation safety is zero accidents. The following year, the Administrator testified that human error was the most serious impediment to FAA’s achieving that goal. He said that FAA planned to accentuate its consideration of human factors in all of its programs, from training to procurement.

To help reach its goal of zero accidents in aviation operations, FAA, in 1995, created a staff Office for System Safety. This office is headed by the Assistant Administrator for System Safety, who reports directly to the FAA Administrator. The objective of this office is to proactively determine potential sources of accidents and prevent them from occurring. The Assistant Administrator for this office has indicated that human factors will be an important part of his office’s work.

Although the Human Factors Division administers FAA’s research on human factors, some of which is directly concerned with safety, its staff are not involved in some applications of human factors to safety. For example, the Office of Regulation and Certification—responsible for aircraft certification, safety inspections, and flight operational safety—plans to strengthen its emphasis on human factors by hiring at least one specialist, rather than rely on the specialists in the Human Factors Division. According to the Associate Administrator for Regulation and Certification, the specialists in the Human Factors Division do not have the expertise needed to apply considerations of human factors to developing requirements for regulation and certification.

FAA’s Research on Human Factors

The Human Factors Division is responsible for identifying aviation-related issues in research on human factors and for allocating and coordinating FAA’s resources for internal and external research on human factors.

Identifying Research Issues

To identify aviation-related issues in research on human factors, the Human Factors Division consults with FAA units and other members of the aviation community. To develop its initial objectives for research on human factors, FAA participated in a task force in April 1989, sponsored by the Air Transport Association of America. This task force identified a number of significant research topics, which FAA incorporated into the National Plan for Civil Aviation Human Factors. This plan—developed by the Human Factors Division in conjunction with the Department of Defense, NASA, industry, and academia—includes a framework that categorizes research on the basis of five priorities, or “thrusts,” and provides guidelines for initiating and managing research on human factors in aviation. (See app. II for a description of each priority and a listing of the ongoing projects under each.) Besides participating in the task force, the Human Factors Division has worked with the aviation community to develop research issues by participating in conferences and workshops. In comparing FAA’s processes to the aviation community’s, we found that FAA not only looks to the aviation community but the aviation community also often looks to FAA to focus attention on particular research issues. For example, FAA sponsored a national conference in 1995 on the challenge of approaching zero accidents.

In addition, the Human Factors Division identifies research issues that the aviation community may not. For example, by managing the research sponsored by FAA units, the Human Factors Division is able to identify research needs that may apply to other FAA units and the aviation community as a whole.

According to the Assistant Administrator, the newly created Office of System Safety will proactively seek to identify safety issues that may indicate the need for additional research on human factors. For example, this office has assumed responsibility from the Office of Aviation Safety for an ongoing project to develop methods for extracting information on human factors from FAA’s existing sources of data. However, according to the Assistant Administrator, this office has not yet developed a research agenda. While staff from the office have met with personnel from the Human Factors Division, no joint activities have been established and no plans have been developed for interactions between the two units.

Although the Human Factors Division identifies FAA’s needs for research on human factors, at least one operating unit is also independently identifying and executing its own research needs. The Office of Regulation and Certification identifies research issues on the basis of its needs and determines what organization will conduct the research. Specifically, the Associate Administrator for Regulation and Certification has established a Human Factors Task Force to review existing literature; obtain information from avionics manufacturers, operators, and industry technical groups; and conduct simulations. The task force was not chartered to initiate research; however, it may make recommendations leading to research on human factors. The Human Factors Division was involved neither in determining the need for the task force nor in planning its work. The possibility exists that the task force’s recommendations could lead the Office of Regulation and Certification to initiate research duplicating the work of the Human Factors Division. Thus, FAA would be deprived of the opportunity to leverage resources for research.

Allocating and Coordinating Resources for Research

Although the Human Factors Division is primarily responsible for allocating and coordinating FAA’s resources for internal and external research on human factors, FAA’s other units are not required to coordinate their research with the division, whether their research is performed internally, by the units themselves, or externally, through interagency agreements or through contractors.

Internal Allocation and Coordination

Starting in 1995, the Office of Aviation Research made the Human Factors Division responsible for allocating most of the agency’s Research, Engineering, and Development funds for research on human factors—nearly $28 million. In fiscal year 1995, the Human Factors Division funded research projects in support of FAA’s acquisition ($5 million), regulation and certification ($12.5 million), and air traffic services ($10.5 million) programs.

The Human Factors Division has also assumed the responsibility for funding contracts or grants for research on human factors at entities such as FAA’s Civil Aeromedical Institute (CAMI) located in Oklahoma City, FAA’s Technical Center near Atlantic City, NASA, the Department of Transportation’s Volpe Transportation Center, and other institutions.Previously, when its research on human factors was funded solely by its operating units, FAA provided no centralized planning for and oversight of its core research on human factors. Now that the Human Factors Division is coordinating FAA’s funding for research (conducted by CAMI, FAA’s Technical Center, NASA, the Volpe Transportation Center, and other institutions), it is constructing a combined database of ongoing research projects, which should give greater visibility to FAA’s research on human factors and permit closer monitoring of the research projects that the agency has funded. As a part of its research administration, the Human Factors Division also monitors whether scientific and technical principles are being applied to the research it funds.

Some FAA units may not be coordinating their research on human factors with the Human Factors Division. For example, some integrated product teams may be conducting such research through contractors, but FAA has no mechanism to ensure that the information developed by a private contractor for one team is made available to another contractor addressing similar issues for another team. Thus, because the FAA units that sponsor their own research on human factors are not required to coordinate their work with that of other units or to inform the Human Factors Division about their research, the possibility of duplication exists.

External Coordination

The Human Factors Division has memoranda of agreement or understanding with NASA and the Department of Defense. According to officials in both the Human Factors Division and NASA, a beneficial result of their coordination is that NASA has not duplicated research being conducted by the division. In addition, the Human Factors Division contracts with NASA to conduct some of its research on human factors in areas where NASA has more experience and/or expertise.

FAA also contracts with the Department of Defense to conduct research on human factors. While much of Defense’s research is specific to defense needs, Defense officials indicated that using the framework articulated in the National Plan for Civil Aviation Human Factors will enable the Department to better coordinate its research on human factors with FAA’s work in similar areas.

Conclusions

The organizational structure for FAA’s work on human factors is still evolving. Therefore, it is too soon to evaluate the effectiveness of the agency’s procedures for incorporating the consideration of human factors throughout FAA and for monitoring the quality of the agency’s work on human factors. Nonetheless, we have found that some FAA units are not coordinating their research with the Human Factors Division, although this division is, currently, primarily responsible for allocating and coordinating FAA’s resources for internal and external research on human factors. Without agencywide coordination of the research on human factors, the potential for duplication exists and the opportunity to leverage the agency’s research dollars by combining related projects is diminished.

Recommendation

To reduce the possibility of duplication and maximize the opportunity to leverage resources for research on human factors, we recommend that the Secretary of Transportation direct the Administrator, FAA, to ensure that all units within FAA coordinate their research through the agency’s Human Factors Division.

Agency Comments

We provided copies of a draft of this report to the Department of Transportation (DOT) and FAA. We met with officials from the Office of the Secretary of Transportation, including the Chief of the Audit Liaison Division, and FAA officials, including the Special Assistant to the Associate Administrator for Regulation and Certification and the Chief Scientist and Technical Advisor on Human Factors, who generally agreed with the report’s findings and recommendation. They provided us with information clarifying FAA’s formal consideration of human factors in the agency’s new acquisition process; we incorporated this information into the text as necessary. According to the Office of Regulation and Certification, the possibility that its Task Force on Human Factors would recommend research duplicating the work of the Human Factors Division is minimal because the research might be administered by the Human Factors Division. However, the Human Factors Division is concerned that, without adequate coordination, the task force could initiate future research that might duplicate the division’s work. FAA indicated that the Office of Regulation and Certification is taking steps to hire a human factors specialist whose first duty will be to develop, in conjunction with the Human Factors Division, a documented process for coordinating research. Unless FAA ensures that research will be administered through the Human Factors Division or until the agency establishes a documented process for coordinating research, we continue to believe that the possibility of duplication exists. DOT expressed concern about our discussion of FAA’s practice of not reviewing the quality of the agency’s work on human factors, noting that quality is difficult to assess. While we agree that assessing quality is difficult, we continue to believe that scientific and technical standards are available for assessing the quality of the agency’s work on human factors. We further believe that adherence to such standards is important to ensure the usefulness of the work’s results.

Scope and Methodology

To determine how FAA has incorporated the consideration of human factors into its research, acquisition, and safety programs, we examined FAA’s organizational structure and reviewed FAA’s policy orders, formal guidance, and strategies for compiling and applying information on human factors. We interviewed FAA officials in the research and acquisitions, regulation and certification, and air traffic services units, but we did not discuss the consideration of human factors in the airports and civil aviation security units because of time constraints. To determine the processes that FAA uses to identify issues in aviation-related research on human factors and compare these processes to those of the aviation community, we reviewed FAA’s plans and research abstracts, interviewed agency officials, and contacted members of the aviation community. To determine how FAA allocates and coordinates resources internally and externally, we interviewed FAA, NASA, and Defense officials and other members of the aviation community and reviewed the legislative requirements for these activities. Because FAA’s work on human factors was not centralized, we relied on data from the Human Factors Division on activities in the Research, Engineering, and Development budget. However, we were not able to obtain similar information for the work on human factors supported through other FAA accounts because such information is not available. We conducted our review from September 1995 through June 1996 in accordance with generally accepted government auditing standards.

We are sending copies of this report to the Secretary of Transportation and the FAA Administrator. We will also make copies available to others on request. Please call me at (202) 512-3650 if you or your staff have any questions about this report.

Definition of Human Factors

The human factors discipline is a scientific and technical approach for designing, operating, and maintaining systems. The goal of this approach is to improve the efficiency and reliability of systems by enhancing the integration of these systems’ components. These components generally consist of the facilities and equipment, rules and regulations, human operators, and environment (physical, economic, political, and social) in which they operate. Thus, the human factors discipline tries to optimize the interactions between the components of a system.

To achieve its goal, the human factors discipline relies on research that combines human sciences and systems engineering. In aviation, the application of human factors research focuses on the complex connections between (1) the members of the flight crew, (2) the flight crew and the aircraft they pilot, (3) the flight crew and the air traffic controllers, (4) the air traffic controllers and their equipment, and (5) the rules, regulations, laws, and standard operating procedures that govern aviation operations. Table I.1 illustrates human factors issues in selected aviation incidents.

Type(s) of human factors issue(s)

Specific potential human factors issue(s)

The crew was not familiar with sophisticated new flight control equipment requiring accurate interpretation and operation.

The crew could have (1) misinterpreted the vertical speed/flight path angle display on the flight control computer or (2) entered the wrong data.

Communication and coordination between the captain and the first officer could have been poor.

Both the captain and first officer might have had limited experience with this type of aircraft.

A last-moment air traffic control approach procedure might have distracted the crew’s attention from the aircraft’s position in relation to the airport and to the altitude/descent rate.

The pilot disengaged two computerized safety features: an autothrottle and an alpha floor protection function.

The pilot did not fully understand the safety features’ functions: The autothrottle maintains a specified speed and the alpha floor protection function prevents the engine from stalling.

An antivibration clamp on an engine-mounted hydraulic tube was missing.

A maintenance technician forgot to install the clamp.

FAA did not initially determine the systems’ operational requirements.

FAA should have established performance baselines for the systems being developed.

The pilot did not achieve a satisfactory level of performance, despite remedial training.

The pilot may not have possessed the skills needed to become competent, despite training.

The pilot and first officer had little experience flying together, and the first officer may not have known that the pilot’s skills were inadequate. (Table notes on next page)

French Transport Ministry officials, as quoted in an article appearing in Aviation Week and Space Technology (Jan. 3, 1994). We did not verify the accuracy of the facts presented in this article.

Steven M. Casey, Set Phasers on Stun and Other True Tales of Design, Technology, and Human Error, Santa Barbara: Aegean Publishing Co., 1993.

R. Curtis Graeber and David A. Marx, “Reducing Human Error in Aircraft Maintenance Operations,” Seattle: Boeing Commercial Airplane Group (Presented at the Flight Safety Foundation’s 46th Annual International Air Safety Seminar, Nov. 8-11, 1993).

Former FAA contractor.

American Eagle officials, as quoted in an article in U.S.A. TODAY (Sept. 27, 1995). We did not verify the accuracy of the facts presented in this article.

Human Factors Research Areas and Ongoing Research Projects

FAA’s framework for research on human factors is organized into five broad areas: (1) human-centered automation, (2) information management and display, (3) selection and training, (4) human performance assessment, and (5) bioaeronautics.

Human-Centered Automation

Human-centered automation research focuses on the role of the operator and the effects of using automation to assist humans in accomplishing their assigned tasks with greater safety and efficiency. The research in this area is designed to identify and apply knowledge of the relative strengths and limitations of humans in an automated environment. It investigates the implications of computer-based technology for the design, evaluation, and certification of controls, displays, and advanced systems.

Information Management and Display

Research conducted under this area seeks to improve safety and performance by addressing the presentation and transfer of information among components in the national airspace system (NAS), including controllers’ workstations, the flight deck, operational and airway facilities, and all the interfaces in between.

Selection and Training

The National Airspace System’s efficiency and effectiveness are enhanced through research to understand the relationship between human abilities and the performance of aviation tasks; to enhance the measures and methods for predicting future job/task performance; to develop a scientific basis for designing training programs, devices, and aids; to define criteria for assessing future training requirements; and to identify new ways for selecting aviation system personnel. The recipients of research findings on selection and training are flight crews, air traffic controllers, airways facilities systems management personnel, aircraft maintenance technicians, airport security personnel, and others in the aviation community who contribute to safety and efficiency through staffing and training decisions.

Areas of Ongoing Research

Selection, Training, Certification, and Staffing of ATC Personnel

Model Advanced Qualification Program (AQP) (ATCS/PTS)

Human Performance Assessment

Research in this area is designed to improve the understanding of human performance capabilities and limitations in aviation and the means to measure them. Individuals’ cognitive and interpersonal skills, teams’ characteristics, and organizational factors directly shape the safety and efficiency of aviation operations. This research will provide information to improve safety and productivity through better equipment design, training, and system performance.

Areas of Ongoing Research

Automated Analysis of Machine Measured Performance

Human Performance in Inspection

Basic Scientific Information on Factors Impacting Controller Performance

Pilot-ATC Communication: Identification of Human Factors Associated With Effective Transfer of Information

Crew Resource Management (CRM) in Aircraft Maintenance and Inspection

Air Crew Performance Measurement

Assessing Automation Impacts on Controller/Sector Performance and

Aviation System Safety Monitoring

Organizational and Environmental Factors Affecting Controller

Basic Scientific Knowledge of Human Performance Factors

Models of Aeronautical Decision-Making

Color Vision Deficiency and Use of Advanced Color-Coded Displays

Assessment of ATCs Crew Performance: Development and Validation

Readiness to Perform (RTP) Test Validation

Glare Vision Testing in the Certification of Pilots

Human Factors of Performance and Pilot Aging

Assessing Automated ATC Systems Through the Use of NAS Data

Organizational Impact of New Technologies on Airway Facilities

Human Factors Considerations in the Use of Nondestructive Test (NDT)

CAMI Cabin Safety Database

Shiftwork in Controllers of Varying Age

Factors in Aircraft Accident Rates (Utilizing the Consolidated Database)

Bioaeronautics

This area, which focuses on the bioengineering, biomedicine, and biochemistry associated with performance and safety, seeks to enhance personal performance and safety by maximizing the health and physiological integrity of crews and passengers.

Major Contributors to This Report

Resources, Community, and Economic Developmnent Division, Washington, D.C.

Atlanta Field Office

Veronica O. Mayhand The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.