Next Generation Air Transportation System: FAA and NASA Have Improved Human Factors Research Coordination, but Stronger Leadership Needed
Released: 2010-08-06T00:00:00

Letter

Background

Human factors is a discipline concerned with, among other things, designing products that are efficient for people to use. As such, human factors combines features of many disciplines, including psychology, engineering, anthropology, sociology, and linguistics. Human factors R&D focuses on people as they interact with the design of products. The goal of human factors is to minimize the potential for design-induced error by ensuring that the equipment is suitable for the users and their environment. The human factors discipline can be described as having two components: human factors research, which seeks to acquire information, and human factors engineering, which seeks to apply the information gained from research to equipment, systems, software, and training, among other things.

Recognizing the importance of human factors considerations, FAA issued a Human Factors Policy Order in 1993 that requires human factors issues to be integrated into the planning and execution of all FAA activities associated with system acquisitions and operations. FAA offers several guidance documents on implementing human factors considerations, which, FAA officials told us, helped aviation stakeholders, such as contractors and research institutions, meet the requirement. For example, officials with the MITRE Corporation told us that—in collaboration with FAA, airlines, and others—they researched human factors issues in the development of the Automatic Dependent Surveillance-Broadcast System, which is an information-reporting technology that, when used in conjunction with other navigation technologies, is expected to enable more precise information about aircraft position. MITRE collected human factors data on how pilots use the broadcast system, collaborated with human factors engineers, and asked human factors personnel to observe pilots’ in-flight interaction with the system while it was being tested.

FAA has several offices that are tasked with ensuring that FAA programs integrate human factors issues. FAA’s Human Factors Research and Engineering Group (HFREG) is responsible for conducting the human factors R&D for NextGen, with the program director serving as the principal advisor to the FAA Administrator on human factors issues. HFREG is divided into three R&D areas: (1) Flight Deck/Aviation Maintenance/System Integration, which develops human performance information that the agency uses in fulfilling its regulatory responsibility and provides to the aviation industry for use in designing and operating aircraft and training pilots and maintenance personnel; (2) Air Traffic Control/Technical Operations, which researches human factors issues with respect to the roles of air traffic controllers, air traffic managers, and maintenance technicians; and (3) general Human Factors Research and Engineering, which attempts to ensure that the incorporation of human factors engineering is explicit, timely, systematic, comprehensive, efficient, and effective. In fiscal year 2009, HFREG conducted dozens of R&D activities including the following: Mitigating fatigue in flight operations. Collecting data on fatigue variables (such as sleep patterns, alertness, and mood) to develop better fatigue-mitigating duty and rest schedules, and outline limits of acceptable performance and flight safety.

Improving pilots’ visual approaches through perceptual training. Investigating the skills pilots need in order to effectively conduct a visual approach, and developing training and performance metrics that will improve training and evaluation of pilots on visual approach tasks.

Assessing safety risks. Calculating the safety risks of an error occurring in relation to the amount of time a controller spends on a task.

In addition, FAA has assigned human factors experts to several offices involved in the development of new systems and in the oversight of aircraft operation and maintenance in order to ensure that human factors issues are addressed. FAA has established chief systems engineers to focus on agencywide, cross-cutting technical and operational issues pertaining to NextGen. Because of the scope of NextGen, FAA contracted with Volpe to provide a chief system engineer for human factors to identify and help the agency better ensure that human factors issues are integrated into the development of NextGen aviation systems. As a result of the observations and recommendations of that Volpe expert, FAA has designated a new position for human factors integration lead and assigned that position to FAA’s System Engineering and Safety organization.

NASA has two units primarily responsible for ensuring human factors consideration in aviation: the Airspace Systems Program and the Aviation Safety Program, both within its Aeronautics Research Mission Directorate. The Airspace Systems Program is the unit chiefly responsible for NASA’s input into NextGen. The primary research role for the Airspace Systems Program is to contribute to the operations of the airspace system by developing concepts, capabilities, and technologies for high-capacity, efficient, and safe airspace systems. The Aviation Safety Program is dedicated to improving the safety of current and future aircraft operating in the national airspace system. The research focus is on the way aircraft are designed, built, operated, and maintained. Scientists and engineers in this program develop concepts and tools to address aircraft aging and durability, among other areas.

FAA and NASA have each invested about $121 million in human factors R&D from fiscal year 2004 to fiscal year 2009 (see fig. 1). Starting in fiscal year 2005, NASA adjusted the size of its human factors research staff by reassigning some staff to other programs and reducing the contractor and academic technical support for human factors R&D. NASA reorganized its aeronautical research plan to focus on what it calls “fundamental research,” which takes a technology to a point where it can be further matured by manufacturers and eventually integrated into new aircraft or engine designs. FAA’s investment in human factors R&D is increasing, along with additional appropriations for overall research development, though overall R&D appears to be increasing at a higher rate (see fig. 2). NASA takes the lead in both identifying human factors concepts that need to be implemented to support a particular technology or system and developing the human factors engineering models and algorithms. NASA then works with FAA on testing the new concept and hands off the responsibility to FAA to make the concept operational. NASA officials told us that it generally takes a concept 5 to 7 years to become operational after NASA transfers responsibility to FAA. Furthermore, in June 2010, NASA officials informed us of a new Integrated Systems Research Program that is to focus on maturing and integrating NextGen technologies into operational systems. The program began in fiscal year 2010 at a funding level of $62.4 million.

NextGen is a major transformation of the aviation system that will have significant implications for human factors considerations. NextGen will transform aviation procedures and the design of the aviation system and introduce new technologies that pose dramatic changes to the roles and responsibilities of both air traffic controllers and pilots and change the way they interface with their systems. According to FAA, under NextGen, a satellite-based system would guide all phases of a flight, including climb, cruise, descent, and taxi. Instead of monitoring aircraft movements using ground-based radar and transmitting voice flight instructions to aircraft, air traffic controllers would primarily monitor automated systems and intervene when anomalies and emergencies occur. As a result, FAA and NASA need to research the human factors considerations associated with the new roles of both flight crew and air traffic management staff, and incorporate the results into the implementation of the new system. In addition, FAA and NASA will have to identify and develop the training necessary for these changing roles, including the time frame before NextGen is fully realized, when some aircraft will be equipped with NextGen systems and others will not.

FAA and NASA structure their NextGen human factors R&D according to a planned three-phase implementation of the NextGen system to align and prevent duplication of NextGen R&D efforts. FAA—which is ultimately in charge of implementing NextGen—is mainly responsible for the R&D to help address near-term implementation (2009-2013), which addresses the day-to-day promotion of the safe and efficient operation of the current aviation system and the implementation of some NextGen systems, and midterm implementation (2012-2018), which consists of leveraging existing aircraft capabilities and introducing new aircraft capabilities to establish a foundation for a longer-term evolution of the aviation system. Within FAA, the Air Traffic Organization is responsible for implementing near- and midterm improvements in coordination with other FAA lines of business. Within the Air Traffic Organization, several offices have different roles in the development of NextGen. For example, within the NextGen and Operations Planning Office, the NextGen Integration and Implementation (NGII) office is tasked with monitoring the progress of NextGen development and implementation and facilitating necessary coordination. These offices are also responsible for ensuring that human factors R&D conducted by HFREG is integrated into NextGen. NASA is responsible for conducting research to help address far-term implementation (2018-2025). As researchers better define system concepts, NASA officials inform FAA officials about research results and FAA officials then use the results to further develop the system. Figure 3 shows the key FAA and NASA organizations involved in human factors activities.

Although FAA and NASA Are Generally Coordinating Their Human Factors R&D, Key Areas Could Benefit from Increased Attention

Participation in Advisory and Coordinating Bodies and Other Efforts Help FAA and NASA Align Their Human Factors R&D

FAA and NASA officials take advantage of a number of existing mechanisms to coordinate their human factors R&D efforts. First, they use the Research, Engineering, and Development Advisory Committee (REDAC), which advises on FAA’s research, engineering, and development activities with experts from industry, academia, and other government agencies. REDAC was established in 1989 to advise the FAA Administrator on research and development needs in human factors, air traffic services, airport technology, aircraft safety, and environmental issues. According to officials from both agencies, their collaboration on REDAC helps to coordinate human factors R&D efforts. One of the REDAC subcommittees is devoted to human factors, and according to officials with HFREG and NGII, has provided important perspectives on research management and coordination among agencies, including human factors R&D. Several REDAC subcommittees have held meetings at NASA to facilitate its participation and ensure that REDAC is briefed on relevant NASA human factors projects as well as FAA’s human factors R&D efforts. NASA officials also use REDAC to brief FAA officials on their human factors R&D efforts as well.

In 2007, FAA and NASA took steps to better coordinate their human factors efforts as a direct result of REDAC’s influence. The REDAC human factors subcommittee recommended that FAA and NASA exchange information about their human factors R&D efforts to better facilitate research coordination, which FAA and NASA did. In addition, in 2009, the subcommittee noted that while the agencies had improved coordination of human factors R&D, they could further improve coordination of FAA and NASA human factors R&D related to the NextGen Controller Efficiency Program. In response, officials with HFREG and NGII told us that they now review NASA human factors research announcements to determine their applicability for FAA NextGen R&D. NASA proposals encompass research that includes human factors issues as part of the proposed work.

In addition, FAA and NASA take advantage of existing forums, meetings, and interagency agreements to coordinate their human factors R&D efforts. Officials with HFREG and NGII told us that FAA and NASA exchange R&D results through reports, presentations, and joint panel discussions at various seminars and professional conferences, including the annual Human Factors and Ergonomics Society conference. FAA officials added that they also attend NASA’s technical interchange meetings to share ideas, learn of NASA’s human factors research efforts, and coordinate research projects. FAA also exchanges R&D planning documentation with NASA annually and as needed to facilitate human factors R&D coordination activities.

The agencies also have undertaken specific efforts to coordinate human factors R&D related to NextGen. FAA established research transition teams to address research gaps and coordinate research between FAA and NASA related to the primary NextGen systems. In September 2008, we reported that FAA and NASA established four research transition teams to outline how the two agencies will jointly develop research requirements. These teams help FAA and NASA identify R&D needed to implement NextGen and ensure that the research is not only conducted but effectively transitioned to the implementing agency. FAA is to provide requirements for users of the technologies, while NASA is to conduct the research and provide an understanding of the engineering rationale for design decisions. According to FAA, these research transition teams facilitate coordination and transition of new technologies and concepts related to NextGen, including human factors components. For example, FAA and NASA are using the research transition teams to coordinate human factors research on the roles and responsibilities of air traffic controllers and pilots, as well as their information needs and procedures, among other issues.

In addition, over the past several years, FAA and NASA officials have established memorandums and interagency agreements that allow the agencies to collaborate on research projects and coordinate human factors R&D related to NextGen. The agreements include reimbursable interagency agreements between HFREG and NASA to leverage resources. According to interagency agreements and FAA officials, leveraging activities include: researching, modeling, and testing the advanced technologies, automation, and services and capabilities that are required for successful implementation of NextGen with particular emphasis on the issues associated with the NextGen flight deck, allowing collaborative research to develop NextGen data communications, human factors collision avoidance requirements, aircraft merging and spacing separation assurance systems, and guidance for use of NextGen synthetic vision systems, enhanced flight vision systems, and advanced cockpit vision technologies, and developing models, simulations, and demonstrations that will quantify efficiencies and benefits for the included programs, and evaluate the operational feasibility of concepts.

HFREG has approved or initiated 35 human factors research activities in partnership with NASA, universities, and private corporations. Supporting flight deck human factors efforts for NextGen, HFREG has approved or initiated 22 NextGen human factors research activities. FAA funds the activities and plans to budget $45 million for them between fiscal year 2009 and fiscal year 2011. In addition, HFREG has approved or initiated 13 NextGen air traffic control human factors research activities. NASA, the Volpe National Transportation Systems Center, and academic and private research facilities and institutions are conducting much of the research, with the goal of providing scientific and technical information to support development of NextGen-related standards, procedures, training, policy and other guidance as well as human factors assessments of NextGen technologies and procedures. The research includes projects related to NextGen communication systems, automation and human roles and responsibilities, risk and error management, decision making, aircraft separation assurance and collision avoidance, ground operations, aircraft trajectory management, instrument procedures, personnel training and qualifications, and single pilot operations. NASA officials have agreed to consult HFREG officials about their NextGen human and automation roles and responsibility research and inform them about the research. In addition, FAA signed two 5-year interagency agreements with NASA in 2009 to provide NASA up to $19 million in funding for human factors research projects covering both flight deck and air traffic control issues.

FAA Has Not Developed a Cross-Agency Coordination Plan

While FAA and NASA officials have taken many steps to coordinate their human factors R&D, JPDO issued a report in April 2008 that raised concerns regarding FAA and NASA coordination of human factors R&D for supporting NextGen. Specifically, JPDO reported that there was no cross-agency plan for identifying and addressing priority NextGen human factors issues and recommended that FAA, in cooperation with NASA, develop such a plan. JPDO recommended that FAA initiate an effort across agencies, industry, and academia to develop a cross-agency plan for NextGen human factors R&D that establishes focus areas for human factors research and development; inventories existing capabilities and laboratories for conducting human factors R&D; capitalizes on past and current human factors research and, where appropriate, reorients it; and ensures that the agencies perform the appropriate human factors R&D during the initial phases of NextGen.

HFREG developed a human factors R&D portfolio in 2009 as part of its effort to improve cross-agency coordination of NextGen human factors R&D. Officials added that the portfolio is the beginning of their attempt to meet JPDO’s recommendation to develop a cross-agency human factors research plan. The portfolio lists and describes all past, ongoing, and planned NextGen human factors R&D projects. HFREG officials stated that the portfolio demonstrates the extent to which FAA and NASA human factors R&D efforts are aligned, and described the portfolio as a repository of NextGen human factors R&D. They added that the portfolio is intended to assist NextGen researchers in developing concepts, establishing requirements, identifying research gaps, and determining additional research and engineering considerations.

FAA’s human factors portfolio is a good step toward better coordinating human factors R&D, but does not currently satisfy JPDO’s cross-agency plan recommendation. Our review of the FAA portfolio indicates that it is a listing and description of R&D projects and results, but not a cross- agency plan with features characteristic of plans, such as role definitions, goals, and time frames. Likewise, the DOT Inspector General reported in April 2010 that FAA has not developed a cross-agency research plan to identify and address how NextGen will affect the roles of controllers and pilots and help ensure that new concepts and technologies can be safely implemented. The Inspector General observed that such a plan would establish an agreed-upon set of initial focus areas for research, provide inventories of existing facilities for research, and capitalize on past and current research because both NASA and FAA conduct human factors work specifically for air traffic management.

A cross-agency plan could help better ensure that FAA and NASA follow key collaboration practices. We have previously reported that federal agencies must effectively collaborate in order to deliver results more efficiently and in a way that is consistent with their multiple demands and limited resources. We identified several practices that could enhance and sustain collaboration efforts, including agreeing on roles and responsibilities, establishing mutually reinforcing or joint strategies, and establishing compatible policies, procedures, and other means to operate across agency boundaries, among other things. A cross-agency coordinating plan that establishes an agreed-upon set of initial focus areas for research, inventories existing facilities for research, and capitalizes on past and current research would help FAA and NASA more closely follow key practices for enhancing and sustaining collaboration.

Experts’ Opinions on Sufficiency of Human Factors Coordination Efforts Vary

Our panel of nine human factors experts had mixed views about FAA’s and NASA’s efforts to improve coordination of their human factors R&D efforts. While some experts told us that the steps the agencies have taken in response to JPDO and REDAC recommendations are sufficient, others suggested that FAA and NASA could do more to improve their human factors coordination. Similarly, officials representing two aviation associations had mixed views regarding coordination; one association stated that NASA and FAA are well coordinated, while another stated that FAA and NASA need to provide more clarity and consensus on their coordination plans.

Four of the nine experts stated that FAA and NASA were coordinating well on human factors research related to NextGen and did not suggest further actions the agencies could take to better coordinate research. However, five experts stated that FAA and NASA could better coordinate human factors research. They suggested hosting additional human factors conferences to improve coordination, and prioritizing coordination of NextGen human factors research. More specifically, two experts told us that while the agencies have held conferences and research workshops (as previously discussed), they have not held conferences specifically devoted to human factors research for supporting NextGen. According to FAA officials, hosting such conferences is very expensive, so HFREG tries to leverage hosting sessions at external conferences and annual meetings. For example, FAA officials sponsored a session on human factors issues related to NextGen at the Human Factors and Ergonomics Society’s Aerospace Systems Technical Group meeting in May 2008 and plan to hold another similar session at this year’s annual meeting in September.

FAA and NASA Have Several Ongoing Human Factors R&D Efforts That Support NextGen, but a Majority of Experts Suggest FAA and NASA Adopt Additional Measures, Including Strengthening Human Factors Leadership

FAA and NASA Have Undertaken a Variety of Human Factors R&D Efforts and Designated Funding to Support NextGen

FAA and NASA have created and shared planning documents for how the agency will incorporate human factors R&D into NextGen. As previously noted, FAA has taken steps to standardize the way it integrates human factors considerations into all aviation projects. To this end, FAA developed a NextGen Human System Integration Roadmap to identify and address human factors R&D needs for supporting NextGen in particular. In addition, as previously discussed, FAA created the Human Factors Portfolio, which lists and describes all past, ongoing, and planned NextGen human factors R&D projects. According to FAA, the portfolio was intended to identify potential gaps and unfunded R&D needs across midterm and potential far-term operational improvements for NextGen. Although we find it currently lacking as a coordination tool, it does enumerate the NextGen projects that are under way, which could be useful in terms of monitoring the efforts of other stakeholders.

In addition, HFREG officials told us that FAA has a range of human factors R&D initiatives that support NextGen. FAA not only conducts focus groups and interviews with a panel of human factors experts, but also conducts live simulations and field trials to evaluate system and human performance in different scenarios. For example, FAA conducted human simulations with pilots and air traffic controllers in fiscal year 2008 and planned further simulations for its High Density Airport Capacity and Efficiency Improvement Project in fiscal year 2009. The agency also conducts field surveys and interviews of operational personnel that are extensively used to address major NextGen and other aviation human factors issues that have an impact on the workforce. For example, FAA plans to conduct a survey to assess the degree of fatigue in the controller workforce.

NASA also has human factors research efforts that support NextGen. Officials told us that NASA experiments with early concept technologies that will involve human interaction, thereby fully leveraging the strengths and mitigating the weaknesses of both the human and automated components. NASA staff then conduct simulations to test human compatibility and subsequently help FAA develop the technologies that prove themselves capable of supporting NextGen.

Over the last 2 years, FAA has also dedicated financial resources specifically to incorporating human factors R&D into NextGen. Prior to fiscal year 2008, FAA used funding from its overall human factors R&D budget for NextGen projects, one of various types of human factors R&D; however, since fiscal year 2008, FAA has had a specific human factors research and development budget for NextGen. To incorporate human factors issues into NextGen, for example, conducting additional human simulations and field trials, FAA invested $25.5 million in human factors R&D specifically dedicated to NextGen from fiscal year 2008 through fiscal year 2010, and has requested additional funding for fiscal year 2011 to fiscal year 2013. NASA officials told us that NASA conducts applied human factors research across its Aviation Safety and Airspace Systems programs and does not have a specific line item budget for NextGen. According to these officials, this research addresses human factors considerations for new concepts and technologies applicable to NextGen. In addition, NASA’s Aeronautics Research Mission Directorate programs were realigned in 2006, causing difficulty in assessing funding trends across several years of similar research activities.

Experts Cited Suggestions Regarding Human Factors Efforts Supporting NextGen

For the most part, aviation human factors experts we interviewed stated that FAA’s and NASA’s human factors R&D efforts adequately support NextGen. For example, experts commended FAA and NASA for appropriately conducting human factors R&D according to the three-phase implementation structure for NextGen systems. As previously mentioned, FAA is mainly responsible for R&D to support near-term implementation and midterm implementation, while NASA conducts much of the research to address far-term implementation. One expert also told us that FAA, in response to REDAC input, has developed a good method for understanding likely human performance. NASA also has modeled NextGen systems to predict how beneficial NextGen systems will be to users. However, a majority of experts offered suggestions for further incorporating human factors issues into NextGen. Experts specifically identified the following suggestions: Better ensure that human factors issues are fully integrated throughout design and development of NextGen systems. Human factors must be considered and integrated throughout the design and development of aviation systems. Failure to fully consider human factors issues at all stages can increase costs and delay projects. Six of nine experts and a senior official at the Volpe National Transportation Systems Center were concerned that NextGen developers may not be adequately considering human factors R&D throughout the entire NextGen planning and implementation process. FAA has not fully integrated human factors considerations into the development of some aviation systems. For example, FAA did not fully address human factors considerations in developing the En Route Automation Modernization (ERAM) system, which FAA plans to complete by 2010. According to the National Air Traffic Controllers Association (NATCA), air traffic controllers involved in initial operations capabilities tests at an air traffic control center in Salt Lake City have come across significant problems with using the system. According to NATCA, controllers have found the new formats cumbersome, confusing, and difficult to navigate, thus indicating that FAA did not adequately involve those who operate the system (controllers) in the early phases of system development. As a result, to better ensure optimal performance of ERAM, FAA will have to address these human factors issues before it deploys the new system. This could increase the costs or delay the implementation of other components of NextGen, such as the previously mentioned Automatic Dependent Surveillance-Broadcast System, since the operation of numerous NextGen components will depend on this new system. FAA officials within the En Route Automation Modernization office agreed with NATCA’s views on the new system and added that the simulation capabilities of its Technical Center in Atlantic City, New Jersey, where the agency conducts human factors testing, were not robust enough to capture all of the problems subsequently identified by controllers. In May 2010, however, FAA announced the building of an Aviation Research and Technology Park near FAA’s Technical Center to provide a central location for partners in academia, industry, and other state and federal government agencies to work on NextGen. According to FAA, the park is being built with no direct cost to FAA and has amassed $3.5 million in grant funding. In June 2010, FAA issued a task order to MITRE Corporation to conduct a programmatic review of the ERAM problem and make an assessment of what circumstances led to the current delay, among other things. The MITRE Corporation is expected to issue a final report on October 1, 2010.

Similarly, in reviewing the development of the Operational and Supportability Implementation System, the Department of Transportation’s Inspector General reported that FAA identified a number of significant human factors concerns with the system, such as inadequately addressing weather information. The Inspector General concluded that system developers did not adequately consider human factors research throughout design and development, thereby contributing to the delay of the system’s implementation. Similarly, as noted in a report we issued in 2005, FAA’s failure to provide adequate attention to human factors issues when implementing the Standard Terminal Automation Replacement System resulted in schedule slips and a significant cost increase of $500 million. As noted, however, since fiscal year 2008, FAA has designated funding solely for human factors R&D supporting NextGen. It remains to be seen if FAA’s added emphasis on human factors research and engineering will better ensure that human factors issues are fully integrated into the development of future NextGen components.

Ensuring the mitigation of human factors issues also involves oversight of contractors. HFREG officials told us that they do not track vendors to make sure they are considering human factors R&D issues in their development, as this is a responsibility of the program managers who lead procurement efforts for FAA systems. However, once contracts are awarded, contractors are supposed to follow the contract specifications, which can include human factors system performance requirements. HFREG officials told us that in the past they collaborated with program office human factors coordinators to assess outside vendors’ compliance with human factors issues; they found that the contractors were not in compliance in all aspects, particularly human factors. In April 2010, the Department of Transportation’s Inspector General also expressed concern about FAA’s ineffective oversight of a contractor in developing NextGen systems, adding that NextGen implementation will require significant contract oversight. Furthermore, FAA’s post-implementation review of the Advanced Technologies and Oceanic Procedures system concluded that FAA and the contractors who developed the system did not, from a human factors perspective, develop the system to meet FAA’s needs. The post-implementation review recommended that for future systems, FAA should ensure that it articulates to contractors in unambiguous terms the human factors-related characteristics that the proposed system must meet. According to the Chief Scientist for NextGen and Operations Planning, a contractor developing an aviation system may have implemented human factors designs that were originally flawed or may have had a flawed methodology for incorporating human factors issues into system development. FAA program offices and contractors often support the incorporation of human factors consideration in a system by convening a panel of controllers and obtaining their feedback. Such a method may result in the controllers providing information regarding their preferences instead of information regarding the usability of the system to the controller panel. An alternative method may be to conduct a modeling effort that analyzes data on human performance for certain components of the system. HFREG officials also noted that under the best of circumstances, all major and most human factors issues should be identified and mitigated during system development, making it unusual for additional problems to arise when a system is being implemented. To address this issue, experts stated FAA should ensure system developers consider human factors in all phases of the development of aviation systems (as required by the Human Factors Policy Order). Having oversight of system developers (including contractors) that develop NextGen systems to make sure they adhere to FAA’s Human Factors Policy Order would significantly reduce the possibility of expensive and untimely delays. FAA has taken action to improve its oversight of contractors. For example, in its June 2010 letter to MITRE, FAA requested an assessment of the ERAM contractor’s program management procedures and practices as part of an overall review of the program.

Improve collaboration of human factors efforts across FAA departments. Collaboration within FAA departments is important to ensure that aviation systems are designed and developed with agency input from human factors researchers. Several experts we interviewed stated that system development projects with a human factors research component take place in different departments and offices at FAA, and that those developing the systems do not always collaborate. While HFREG provides R&D and engineering support, HFREG officials told us that there is no requirement for program offices or developers to consult with HFREG. HFREG conducted a post-implementation review of the Advanced Technologies and Oceanic Procedures that implied that system managers did not properly consider human factor issues. This suggests that the system managers either did not consult human factors stakeholders (including HFREG) or did not fully address their human factors issues through a collaborative working relationship. As a result, the post-implementation review concluded that from a human factors perspective, the system that was implemented in the field was not the system FAA had asked for. FAA’s experience in developing the Advanced Technologies and Oceanic Procedures is an indication of what can happen when system developers fail to collaborate with human factors specialists and develop a comprehensive human factors program. To improve collaboration, HFREG officials also told us that the Chief Scientist of the NextGen and Operations Planning unit sponsored a technical interchange meeting in January 2010 to better ensure that all FAA units involved in NextGen development are aware of the need to fully consider human factors in their work. The Chief Scientist plans to host another technical interchange meeting on July 29, 2010.

Strong Leadership Would Improve Consideration of Human Factors Issues

A majority of the experts we interviewed agree that strong leadership is needed to provide adequate consideration of human factors issues within NextGen. Furthermore, a September 2008 National Academy of Public Administration’s report identified leadership as the single most important element of success for large-scale systems integration efforts like NextGen. That report highlighted leadership as a NextGen implementation challenge. The critical impact of human factors issues on NextGen indicates that human factors issues require strong leadership to ensure they are a priority for NextGen. FAA has not prioritized consistently staffing the top two leadership positions within FAA that are formally responsible for human factors R&D. Specifically, the Chief Systems Engineer for Human Factors position has been vacant since the previous chief retired in January 2010. Moreover, FAA did not assign a permanent program director of HFREG for 16 months, from January 2009 until FAA filled the position in June 2010.

The leadership void was the issue most frequently identified by the nine experts. Seven of nine experts we interviewed told us that the lack of leadership within FAA is a significant challenge in ensuring that human factors R&D supports NextGen. Although a majority of the experts were concerned that the leadership void could have prevented human factors issues from being fully considered for NextGen, subsequently delaying the implementation of a system, none could identify any specific examples. Nevertheless, FAA officials emphasized the importance of both positions. FAA officials told us that the Chief System Engineer position could be pivotal in integrating and maximizing the effectiveness of human factors in support of NextGen and is thus critical to prioritizing NextGen research and resources within FAA. JPDO officials we interviewed stressed that the program director of HFREG is the single most important position needed to ensure that the necessary human factors R&D is conducted and that the results are integrated into the development of NextGen systems.

According to FAA officials, FAA has not had a chance to fill the position of Chief System Engineer—which FAA now refers to as the human factors integration lead—because of a hiring freeze and uncertainty as to which unit to put the position. FAA has resolved those issues and plans to begin the process for filling the position. Officials cautioned, however, that it may take a long time to find a qualified candidate with the right human factors expertise and other relevant skill sets. Nonetheless, FAA would like to fill the position by the close of fiscal year 2010. FAA officials also told us that it took a long time to fill the position of program director for HFREG, in part because of the long process of completing required personnel administrative procedures. The new program director of HFREG was formerly the acting program director and had been in that position since the previous program director left. The assignment to program director involved a change in the position classification that involved several time-consuming administrative procedures to address, according to HFREG officials and an FAA senior executive.

Experts also told us in filling these positions that the new leaders should have adequate authority to make sure that human factors issues are considered (particularly early in system development) and prioritized during all phases of NextGen development. These positions currently lack the authority to ensure that human factors issues are addressed early and throughout the NextGen system development process. Such authority could mitigate the need to redesign these systems after implementation has begun, which can cause delays and add costs. For example, as previously discussed, it has been found that FAA’s human factors plans have not adequately addressed how humans will use newly developed NextGen weather information. One of the experts we consulted who has worked extensively with FAA on human factors R&D told us that a program director of the HFREG or Chief System Engineer who has adequate authority could have reviewed the weather information to ensure that human factors were fully integrated into that and other NextGen systems. However, in filling the position of program director of HFREG, FAA did not authorize the new program director with additional authority to review NextGen programs and ensure that human factors issues are addressed. HFREG officials told us that FAA is conducting a review of distribution among HFREG, service units, and other offices for responsibility and authority to conduct human factors activities to better serve the human factors needs of NextGen.

Conclusions

Human factors research must be incorporated into NextGen to ensure that controllers, pilots, and other aviation system users can operate NextGen in a safe and efficient manner. To this end, FAA and NASA have pursued a wide range of efforts to incorporate human factors R&D into NextGen. However, these and future efforts will require a sustained focus not only across agencies but from the beginning to the end of the long process of developing a complex system like NextGen. Some suggest that FAA can meet this challenge by incorporating two elements into its human factors R&D efforts: a cross-agency plan developed in cooperation with NASA to identify, prioritize, and coordinate NextGen human factors issues, and strong and consistent leadership with the authority to not only prioritize human factors issues but ensure that they are taken into account throughout NextGen.

Recommendations for Executive Action

We recommend that the Secretary of Transportation direct the FAA Administrator to take the following two actions: create a cross-agency human factors coordination plan in cooperation with NASA, as JPDO has previously recommended, that establishes an agreed-upon set of initial focus areas for research, inventories existing facilities for research, and capitalizes on past and current research of all NextGen issues, and assign a high priority to filling the vacancy of human factors integration lead and structure that position and the program director of HFREG position in a manner that provides the authority to ensure that human factors research and development is coordinated, considered, and prioritized in all phases of NextGen development.

Agency Comments

We provided a draft of this report to the Department of Transportation and NASA for review and comment. NASA had no comments. DOT agreed to consider the recommendations and provided technical clarifications, which we incorporated into the report as appropriate.

We are sending copies of this report to the Secretary of Transportation, FAA, NASA, and interested congressional committees. The report is also available at no charge on GAO’s Web site at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-2834 or dillinghamg@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix II.

Appendix I: Scope and Methodology

In response to your request, this report provides information on the status of the Department of Transportation’s Federal Aviation Administration’s (FAA) and National Aeronautics and Space Administration’s (NASA) efforts to incorporate human factors issues into the Next Generation Air Transportation System (NextGen). In particular, we sought to identify the extent to which (1) FAA’s and NASA’s human factors research and development (R&D) is coordinated, and (2) FAA’s and NASA’s human factors R&D supports NextGen.

In determining the extent to which FAA’s and NASA’s human factors R&D is coordinated, we obtained and analyzed information provided by FAA and NASA officials on mechanisms in place to align human factors R&D efforts. We asked FAA and NASA officials to describe the mechanisms that are in place to coordinate the agency’s human factors R&D. We assessed the information FAA and NASA officials provided us regarding their coordination mechanisms by comparing those efforts with recommendations issued by the Joint Planning and Development Office (JPDO)—an interagency organization responsible for planning NextGen. In 2008, JPDO issued a cross-agency gap analysis that found FAA and NASA lacked a cross-agency plan for identifying and addressing priority NextGen human factors issues. We also assessed FAA’s and NASA’s coordination efforts by summarizing the views of nine external aviation human factors experts who reviewed and assessed FAA’s and NASA’s coordination mechanisms. See our discussion below for more detail regarding the nine aviation human factors experts. We also obtained the views of several aviation industry officials, including officials from the Aerospace Industries Association, Air Transport Association, Air Line Pilots Association, MITRE Corporation, National Air Traffic Controller Association, JPDO, Volpe National Transportation Systems Center, and the Boeing Corporation. We also reviewed relevant reports issued by GAO, the Inspector General of the Department of Transportation, and the National Academy of Public Administration.

In determining the extent to which FAA’s and NASA’s human factors R&D supports NextGen, we obtained relevant planning documents from FAA and NASA and had FAA and NASA officials provide us with detailed descriptions of their human factors R&D efforts. We provided this information and other related planning documents to nine aviation human factors experts and representatives from three aviation industry associations and asked them about their views on the extent to which FAA’s and NASA’s human factors research supports NextGen. The experts provided suggestions that FAA and NASA could adopt to better incorporate human factors issues in developing NextGen, and we reported the suggestions that a majority of experts recommended FAA and NASA adopt. In addition, we obtained the views of several aviation industry officials identified above.

In assessing FAA and NASA human factors R&D coordination and human factors R&D supporting NextGen, we summarized the views of nine aviation human factors experts. We took several steps to identify potential aviation human factors experts. First, we identified experts in human factors R&D that GAO had consulted in the past. We then asked cognizant FAA and NASA officials responsible for and knowledgeable about aviation-related human factors R&D to recommend experts in aviation- related human factors R&D. In addition, we conducted comprehensive Internet searches for human factors aviation experts. Finally, we asked experts identified in the first four steps to recommend other human factors aviation experts. Taking these steps enabled us to identify 25 potential experts.

To make our final expert selection, we narrowed our selection of the 25 potential experts based on the following criteria: knowledge of aviation-related human factors research as determined by published research, such as human factors research related to aviation development, and knowledge of NextGen planning and implementation needs as determined by research, published work, and participation in NextGen seminars, conferences, and workshops.

Applying the criteria listed above to the 25 potential experts resulted in a final selection of 11 experts who have significant knowledge in both aviation-related human factors R&D and more specifically human factors R&D pertaining to NextGen. We obtained and synthesized responses from 9 of the 11 aviation human factors experts. The experts we obtained responses from are listed in table 1. We interviewed an additional selected expert prior to finalizing our methodology and incorporated the expert’s views where appropriate in this report.

Appendix II: GAO Contact and Staff Acknowledgments

Staff Acknowledgments

In addition to the contact above, other key contributors to this report were Ed Laughlin, Assistant Director; Samer Abbas; Bert Japikse; Richard Hung; Michael Mgebroff; Tina Paek; and Amy Rosewarne.