Next Generation Air Transportation System: FAA and NASA Have Improved Human Factors Research Coordination, but Stronger Leadership Needed
Published: 2010-08-06T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: To address challenges to the aviation industry's economic health and safety, the Federal Aviation Administration (FAA) is collaborating with the National Aeronautics and Space Administration (NASA) and other federal partners to plan and implement the Next Generation Air Transportation System (NextGen). NextGen will transform the current radar-based air traffic control system into a satellite-based system. Pilot and air traffic controller roles and responsibilities are expected to become more automated, thereby requiring an understanding of human factors, which studies how humans' abilities, characteristics, and limitations interact with the design of the equipment they use, environments in which they function, and jobs they perform. FAA and NASA are tasked with incorporating human factors issues into NextGen. As requested, this report discusses the extent to which FAA's and NASA's human factors research (1) is coordinated and (2) supports NextGen. To address these issues, GAO reviewed coordination mechanisms and planning documents and synthesized the views of nine aviation human factors experts.
section_title: What GAO Found
paragraphs: While FAA and NASA officials are coordinating their NextGen human factors research efforts in a variety of ways, they lack a cross-agency human factors plan for coordination. FAA and NASA have participated in research advisory committees and interagency research transition teams, signed interagency agreements, and held cross-agency meetings and conferences focused on human factors issues. FAA also created a human factors portfolio to identify and address priority human factors issues but not a cross-agency human factors coordination research plan in cooperation with NASA, as previously recommended by FAA's Joint Planning and Development Office (JPDO)--an interagency organization responsible for planning NextGen. As a result, FAA has not established an agreed-upon set of initial focus areas for research that identifies and capitalizes on past and current research and establishes focus areas for human factors research and development, among other things. The experts GAO contacted generally agreed that FAA's and NASA's human factors research efforts adequately support NextGen, but made several suggestions, including enhancing human factors research leadership, for further incorporating human factors issues into NextGen systems. FAA and NASA have undertaken a variety of human factors efforts to support NextGen, including, among other things, creating planning documents detailing how human factors research will be incorporated into NextGen and dedicating financial resources specifically to NextGen human factors research. While the human factors experts GAO interviewed stated that these efforts support NextGen, a majority offered the following suggestions for further integrating human factors issues into NextGen: (1) Better ensure that human factors issues are fully integrated throughout the development of NextGen systems. FAA did not do this in the development of past systems, a fact that led to schedule slippages and cost increases. (2) Improve collaboration of human factors efforts within FAA departments. (3) Establish strong leadership. A 2008 National Academy of Public Administration's report identified leadership as the single most important element of success for large-scale systems integration efforts like NextGen. FAA has not prioritized consistently staffing the top two human factors positions. Specifically, the position of the Chief Systems Engineer for Human Factors (now referred to as the human factors integration lead) has been vacant since January 2010. Moreover, FAA did not have a permanent program director of its Human Factors Research and Engineering Group from January 2009 until June 2010. These two positions currently lack the authority to ensure that human factors issues are addressed early and throughout the NextGen system development process to prevent the need to redesign these systems after implementation, which can cause delays and add costs. As a result, FAA may lack consistent leadership with the sufficient authority to not only prioritize human factors issues but ensure that human factors issues are addressed throughout NextGen. FAA should (1) create a coordination plan and (2) give priority to filling vacant leadership positions and provide the positions with authority for prioritizing human factors. FAA agreed to consider the recommendations.

section_title: Letter
paragraphs: The aviation industry is critical to the nation’s economic health and safety,  accounting for over $1 trillion in economic activity annually and handling  about 50,000 flights per day while also maintaining a high level of safety.  The Department of Transportation’s (DOT) Federal Aviation  Administration (FAA) and the aviation industry, however, face several  challenges to the industry’s economic health and safety, including  increases in demand and congestion. To meet these challenges, FAA is  taking the lead in transforming the current air traffic control system to a  new system—the Next Generation Air Transportation System (NextGen).  Whereas the current system is radar-based, NextGen is a more automated,  aircraft-centric, satellite-based system. FAA is charged with implementing  NextGen by 2025 with the cooperation and collaboration of the National  Aeronautics and Space Administration (NASA) and other federal partners.
Under NextGen, pilots and air traffic controllers will rely to a greater  extent on automation, changing their roles and responsibilities in ways  that will require a full understanding of what are known as human factors  issues. Human factors research and development (R&D) studies how  humans’ abilities, characteristics, and limitations interact with the design  of the equipment they use, the environments in which they function, and  the jobs they perform. FAA and NASA—the primary agencies responsible  for integrating human factors issues into NextGen—must ensure that  NextGen explores human factors issues so that controllers, pilots, and  others will operate NextGen components in a safe and efficient manner.
Without adequate consideration of human factors issues, the  implementation of NextGen could be delayed.
You requested that we study FAA’s and NASA’s efforts with regard to  human factors related to NextGen. This report provides information on the  extent to which (1) FAA’s and NASA’s human factors R&D efforts are  coordinated and (2) FAA’s and NASA’s human factors R&D efforts support  NextGen. To provide information on human factors R&D coordination, we  obtained and analyzed information provided by FAA and NASA officials on  mechanisms in place to coordinate human factors R&D. We assessed these  coordination efforts by comparing them with recommendations issued by  FAA’s Joint Planning and Development Office (JPDO)—an interagency  organization responsible for long-term NextGen planning efforts. We also  obtained information from nine aviation human factors experts who  reviewed and assessed FAA’s and NASA’s process for coordinating their  human factors R&D. All the experts have been involved in aviation human  factors R&D specifically directed at NextGen systems. To provide  information on the extent to which the human factors efforts support  NextGen, we reviewed relevant planning documents from FAA and NASA  and asked FAA and NASA officials in interviews to describe their human  factors R&D efforts in detail. We provided these planning documents and  information from the interviews to the nine aviation human factors experts,  as well as representatives from three aviation industry associations, and  obtained and synthesized their views regarding the extent to which FAA’s  and NASA’s human factors R&D efforts support NextGen. In addition, we  obtained views from officials representing the Aerospace Industries  Association, Air Transport Association, Air Line Pilots Association, MITRE  Corporation, National Air Traffic Controller Association, Volpe National  Transportation Systems Center, JPDO, and the Boeing Corporation.
We performed our work from August 2009 to August 2010 in accordance  with generally accepted government auditing standards. Those standards  require that we plan and perform the audit to obtain sufficient and  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
subsections: section_title: Background
paragraphs: Human factors is a discipline concerned with, among other things,  designing products that are efficient for people to use. As such, human  factors combines features of many disciplines, including psychology,  engineering, anthropology, sociology, and linguistics. Human factors R&D  focuses on people as they interact with the design of products. The goal of  human factors is to minimize the potential for design-induced error by  ensuring that the equipment is suitable for the users and their  environment. The human factors discipline can be described as having two  components: human factors research, which seeks to acquire information,  and human factors engineering, which seeks to apply the information  gained from research to equipment, systems, software, and training,  among other things.
Recognizing the importance of human factors considerations, FAA issued  a Human Factors Policy Order in 1993 that requires human factors issues  to be integrated into the planning and execution of all FAA activities  associated with system acquisitions and operations. FAA offers several  guidance documents on implementing human factors considerations,  which, FAA officials told us, helped aviation stakeholders, such as  contractors and research institutions, meet the requirement. For example,  officials with the MITRE Corporation told us that—in collaboration with  FAA, airlines, and others—they researched human factors issues in the  development of the Automatic Dependent Surveillance-Broadcast System,  which is an information-reporting technology that, when used in  conjunction with other navigation technologies, is expected to enable  more precise information about aircraft position. MITRE collected human  factors data on how pilots use the broadcast system, collaborated with  human factors engineers, and asked human factors personnel to observe  pilots’ in-flight interaction with the system while it was being tested.
FAA has several offices that are tasked with ensuring that FAA programs  integrate human factors issues. FAA’s Human Factors Research and  Engineering Group (HFREG) is responsible for conducting the human  factors R&D for NextGen, with the program director serving as the  principal advisor to the FAA Administrator on human factors issues.  HFREG is divided into three R&D areas: (1) Flight Deck/Aviation  Maintenance/System Integration, which develops human performance  information that the agency uses in fulfilling its regulatory responsibility  and provides to the aviation industry for use in designing and operating  aircraft and training pilots and maintenance personnel; (2) Air Traffic  Control/Technical Operations, which researches human factors issues  with respect to the roles of air traffic controllers, air traffic managers, and  maintenance technicians; and (3) general Human Factors Research and  Engineering, which attempts to ensure that the incorporation of human  factors engineering is explicit, timely, systematic, comprehensive,  efficient, and effective. In fiscal year 2009, HFREG conducted dozens of  R&D activities including the following:    Mitigating fatigue in flight operations. Collecting data on fatigue  variables (such as sleep patterns, alertness, and mood) to develop better  fatigue-mitigating duty and rest schedules, and outline limits of acceptable  performance and flight safety.
Improving pilots’ visual approaches through perceptual training.  Investigating the skills pilots need in order to effectively conduct a visual  approach, and developing training and performance metrics that will  improve training and evaluation of pilots on visual approach tasks.
Assessing safety risks. Calculating the safety risks of an error occurring in  relation to the amount of time a controller spends on a task.
In addition, FAA has assigned human factors experts to several offices  involved in the development of new systems and in the oversight of  aircraft operation and maintenance in order to ensure that human factors  issues are addressed. FAA has established chief systems engineers to  focus on agencywide, cross-cutting technical and operational issues  pertaining to NextGen. Because of the scope of NextGen, FAA contracted  with Volpe to provide a chief system engineer for human factors to identify  and help the agency better ensure that human factors issues are integrated  into the development of NextGen aviation systems. As a result of the  observations and recommendations of that Volpe expert, FAA has  designated a new position for human factors integration lead and assigned  that position to FAA’s System Engineering and Safety organization.
NASA has two units primarily responsible for ensuring human factors  consideration in aviation: the Airspace Systems Program and the Aviation  Safety Program, both within its Aeronautics Research Mission Directorate.  The Airspace Systems Program is the unit chiefly responsible for NASA’s  input into NextGen. The primary research role for the Airspace Systems  Program is to contribute to the operations of the airspace system by  developing concepts, capabilities, and technologies for high-capacity,  efficient, and safe airspace systems. The Aviation Safety Program is  dedicated to improving the safety of current and future aircraft operating  in the national airspace system. The research focus is on the way aircraft  are designed, built, operated, and maintained. Scientists and engineers in  this program develop concepts and tools to address aircraft aging and  durability, among other areas.
FAA and NASA have each invested about $121 million in human factors  R&D from fiscal year 2004 to fiscal year 2009 (see fig. 1). Starting in fiscal  year 2005, NASA adjusted the size of its human factors research staff by  reassigning some staff to other programs and reducing the contractor and  academic technical support for human factors R&D. NASA reorganized its  aeronautical research plan to focus on what it calls “fundamental  research,” which takes a technology to a point where it can be further  matured by manufacturers and eventually integrated into new aircraft or  engine designs. FAA’s investment in human factors R&D is increasing,  along with additional appropriations for overall research development,  though overall R&D appears to be increasing at a higher rate (see fig. 2).  NASA takes the lead in both identifying human factors concepts that need  to be implemented to support a particular technology or system and  developing the human factors engineering models and algorithms. NASA  then works with FAA on testing the new concept and hands off the  responsibility to FAA to make the concept operational. NASA officials told  us that it generally takes a concept 5 to 7 years to become operational  after NASA transfers responsibility to FAA. Furthermore, in June 2010,  NASA officials informed us of a new Integrated Systems Research  Program that is to focus on maturing and integrating NextGen  technologies into operational systems. The program began in fiscal year  2010 at a funding level of $62.4 million.
NextGen is a major transformation of the aviation system that will have  significant implications for human factors considerations. NextGen will  transform aviation procedures and the design of the aviation system and  introduce new technologies that pose dramatic changes to the roles and  responsibilities of both air traffic controllers and pilots and change the  way they interface with their systems. According to FAA, under NextGen,  a satellite-based system would guide all phases of a flight, including climb,  cruise, descent, and taxi. Instead of monitoring aircraft movements using  ground-based radar and transmitting voice flight instructions to aircraft,  air traffic controllers would primarily monitor automated systems and  intervene when anomalies and emergencies occur. As a result, FAA and  NASA need to research the human factors considerations associated with  the new roles of both flight crew and air traffic management staff, and  incorporate the results into the implementation of the new system. In  addition, FAA and NASA will have to identify and develop the training  necessary for these changing roles, including the time frame before  NextGen is fully realized, when some aircraft will be equipped with  NextGen systems and others will not.
FAA and NASA structure their NextGen human factors R&D according to  a planned three-phase implementation of the NextGen system to align and  prevent duplication of NextGen R&D efforts. FAA—which is ultimately in  charge of implementing NextGen—is mainly responsible for the R&D to  help address near-term implementation (2009-2013), which addresses the  day-to-day promotion of the safe and efficient operation of the current  aviation system and the implementation of some NextGen systems, and  midterm implementation (2012-2018), which consists of leveraging existing  aircraft capabilities and introducing new aircraft capabilities to establish a  foundation for a longer-term evolution of the aviation system. Within FAA,  the Air Traffic Organization is responsible for implementing near- and  midterm improvements in coordination with other FAA lines of business.  Within the Air Traffic Organization, several offices have different roles in  the development of NextGen. For example, within the NextGen and  Operations Planning Office, the NextGen Integration and Implementation  (NGII) office is tasked with monitoring the progress of NextGen  development and implementation and facilitating necessary coordination.  These offices are also responsible for ensuring that human factors R&D  conducted by HFREG is integrated into NextGen. NASA is responsible for  conducting research to help address far-term implementation (2018-2025).  As researchers better define system concepts, NASA officials inform FAA  officials about research results and FAA officials then use the results to  further develop the system. Figure 3 shows the key FAA and NASA  organizations involved in human factors activities.
section_title: Although FAA and NASA Are Generally Coordinating Their Human Factors R&D, Key Areas Could Benefit from Increased Attention
subsections: section_title: Participation in Advisory and Coordinating Bodies and Other Efforts Help FAA and NASA Align Their Human Factors R&D
paragraphs: FAA and NASA officials take advantage of a number of existing mechanisms  to coordinate their human factors R&D efforts. First, they use the Research,  Engineering, and Development Advisory Committee (REDAC), which  advises on FAA’s research, engineering, and development activities with  experts from industry, academia, and other government agencies. REDAC  was established in 1989 to advise the FAA Administrator on research and  development needs in human factors, air traffic services, airport technology,  aircraft safety, and environmental issues. According to officials from both  agencies, their collaboration on REDAC helps to coordinate human factors  R&D efforts. One of the REDAC subcommittees is devoted to human  factors, and according to officials with HFREG and NGII, has provided  important perspectives on research management and coordination among  agencies, including human factors R&D. Several REDAC subcommittees  have held meetings at NASA to facilitate its participation and ensure that  REDAC is briefed on relevant NASA human factors projects as well as  FAA’s human factors R&D efforts. NASA officials also use REDAC to brief  FAA officials on their human factors R&D efforts as well.
In 2007, FAA and NASA took steps to better coordinate their human  factors efforts as a direct result of REDAC’s influence. The REDAC human  factors subcommittee recommended that FAA and NASA exchange  information about their human factors R&D efforts to better facilitate  research coordination, which FAA and NASA did. In addition, in 2009, the  subcommittee noted that while the agencies had improved coordination of  human factors R&D, they could further improve coordination of FAA and  NASA human factors R&D related to the NextGen Controller Efficiency  Program. In response, officials with HFREG and NGII told us that they  now review NASA human factors research announcements to determine  their applicability for FAA NextGen R&D. NASA proposals encompass  research that includes human factors issues as part of the proposed work.
In addition, FAA and NASA take advantage of existing forums, meetings,  and interagency agreements to coordinate their human factors R&D  efforts. Officials with HFREG and NGII told us that FAA and NASA  exchange R&D results through reports, presentations, and joint panel  discussions at various seminars and professional conferences, including  the annual Human Factors and Ergonomics Society conference. FAA  officials added that they also attend NASA’s technical interchange  meetings to share ideas, learn of NASA’s human factors research efforts,  and coordinate research projects. FAA also exchanges R&D planning  documentation with NASA annually and as needed to facilitate human  factors R&D coordination activities.
The agencies also have undertaken specific efforts to coordinate human  factors R&D related to NextGen. FAA established research transition  teams to address research gaps and coordinate research between FAA and  NASA related to the primary NextGen systems. In September 2008, we  reported that FAA and NASA established four research transition teams  to outline how the two agencies will jointly develop research  requirements. These teams help FAA and NASA identify R&D needed to  implement NextGen and ensure that the research is not only conducted  but effectively transitioned to the implementing agency. FAA is to provide  requirements for users of the technologies, while NASA is to conduct the  research and provide an understanding of the engineering rationale for  design decisions. According to FAA, these research transition teams  facilitate coordination and transition of new technologies and concepts  related to NextGen, including human factors components. For example,  FAA and NASA are using the research transition teams to coordinate  human factors research on the roles and responsibilities of air traffic  controllers and pilots, as well as their information needs and procedures,  among other issues.
In addition, over the past several years, FAA and NASA officials have  established memorandums and interagency agreements that allow the  agencies to collaborate on research projects and coordinate human factors  R&D related to NextGen. The agreements include reimbursable  interagency agreements between HFREG and NASA to leverage resources.  According to interagency agreements and FAA officials, leveraging  activities include:  researching, modeling, and testing the advanced technologies, automation,  and services and capabilities that are required for successful  implementation of NextGen with particular emphasis on the issues  associated with the NextGen flight deck,     allowing collaborative research to develop NextGen data communications,  human factors collision avoidance requirements, aircraft merging and  spacing separation assurance systems, and guidance for use of NextGen  synthetic vision systems, enhanced flight vision systems, and advanced  cockpit vision technologies, and    developing models, simulations, and demonstrations that will quantify  efficiencies and benefits for the included programs, and evaluate the  operational feasibility of concepts.
HFREG has approved or initiated 35 human factors research activities in  partnership with NASA, universities, and private corporations. Supporting  flight deck human factors efforts for NextGen, HFREG has approved or  initiated 22 NextGen human factors research activities. FAA funds the  activities and plans to budget $45 million for them between fiscal year  2009 and fiscal year 2011. In addition, HFREG has approved or initiated 13  NextGen air traffic control human factors research activities. NASA, the  Volpe National Transportation Systems Center, and academic and private  research facilities and institutions are conducting much of the research,  with the goal of providing scientific and technical information to support  development of NextGen-related standards, procedures, training, policy  and other guidance as well as human factors assessments of NextGen  technologies and procedures. The research includes projects related to  NextGen communication systems, automation and human roles and  responsibilities, risk and error management, decision making, aircraft  separation assurance and collision avoidance, ground operations, aircraft  trajectory management, instrument procedures, personnel training and  qualifications, and single pilot operations. NASA officials have agreed to  consult HFREG officials about their NextGen human and automation roles  and responsibility research and inform them about the research. In  addition, FAA signed two 5-year interagency agreements with NASA in  2009 to provide NASA up to $19 million in funding for human factors  research projects covering both flight deck and air traffic control issues.
section_title: FAA Has Not Developed a Cross-Agency Coordination Plan
paragraphs: While FAA and NASA officials have taken many steps to coordinate their  human factors R&D, JPDO issued a report in April 2008 that raised  concerns regarding FAA and NASA coordination of human factors R&D  for supporting NextGen. Specifically, JPDO reported that there was no  cross-agency plan for identifying and addressing priority NextGen human  factors issues and recommended that FAA, in cooperation with NASA,  develop such a plan. JPDO recommended that FAA initiate an effort across  agencies, industry, and academia to develop a cross-agency plan for  NextGen human factors R&D that    establishes focus areas for human factors research and development;  inventories existing capabilities and laboratories for conducting human  factors R&D;    capitalizes on past and current human factors research and, where  appropriate, reorients it; and    ensures that the agencies perform the appropriate human factors R&D  during the initial phases of NextGen.
HFREG developed a human factors R&D portfolio in 2009 as part of its  effort to improve cross-agency coordination of NextGen human factors  R&D. Officials added that the portfolio is the beginning of their attempt to  meet JPDO’s recommendation to develop a cross-agency human factors  research plan. The portfolio lists and describes all past, ongoing, and  planned NextGen human factors R&D projects. HFREG officials stated  that the portfolio demonstrates the extent to which FAA and NASA human  factors R&D efforts are aligned, and described the portfolio as a repository  of NextGen human factors R&D. They added that the portfolio is intended  to assist NextGen researchers in developing concepts, establishing  requirements, identifying research gaps, and determining additional  research and engineering considerations.
FAA’s human factors portfolio is a good step toward better coordinating  human factors R&D, but does not currently satisfy JPDO’s cross-agency  plan recommendation. Our review of the FAA portfolio indicates that it is  a listing and description of R&D projects and results, but not a cross- agency plan with features characteristic of plans, such as role definitions,  goals, and time frames. Likewise, the DOT Inspector General reported in  April 2010 that FAA has not developed a cross-agency research plan to  identify and address how NextGen will affect the roles of controllers and  pilots and help ensure that new concepts and technologies can be safely  implemented. The Inspector General observed that such a plan would  establish an agreed-upon set of initial focus areas for research, provide  inventories of existing facilities for research, and capitalize on past and  current research because both NASA and FAA conduct human factors  work specifically for air traffic management.
A cross-agency plan could help better ensure that FAA and NASA follow  key collaboration practices. We have previously reported that federal  agencies must effectively collaborate in order to deliver results more  efficiently and in a way that is consistent with their multiple demands and  limited resources. We identified several practices that could enhance and  sustain collaboration efforts, including agreeing on roles and  responsibilities, establishing mutually reinforcing or joint strategies, and  establishing compatible policies, procedures, and other means to operate  across agency boundaries, among other things. A cross-agency  coordinating plan that establishes an agreed-upon set of initial focus areas  for research, inventories existing facilities for research, and capitalizes on  past and current research would help FAA and NASA more closely follow  key practices for enhancing and sustaining collaboration.
section_title: Experts’ Opinions on Sufficiency of Human Factors Coordination Efforts Vary
paragraphs: Our panel of nine human factors experts had mixed views about FAA’s  and NASA’s efforts to improve coordination of their human factors R&D  efforts. While some experts told us that the steps the agencies have taken  in response to JPDO and REDAC recommendations are sufficient, others  suggested that FAA and NASA could do more to improve their human  factors coordination. Similarly, officials representing two aviation  associations had mixed views regarding coordination; one association  stated that NASA and FAA are well coordinated, while another stated that  FAA and NASA need to provide more clarity and consensus on their  coordination plans.
Four of the nine experts stated that FAA and NASA were coordinating well  on human factors research related to NextGen and did not suggest further  actions the agencies could take to better coordinate research. However,  five experts stated that FAA and NASA could better coordinate human  factors research. They suggested hosting additional human factors  conferences to improve coordination, and prioritizing coordination of  NextGen human factors research. More specifically, two experts told us  that while the agencies have held conferences and research workshops (as  previously discussed), they have not held conferences specifically devoted  to human factors research for supporting NextGen. According to FAA  officials, hosting such conferences is very expensive, so HFREG tries to  leverage hosting sessions at external conferences and annual meetings.  For example, FAA officials sponsored a session on human factors issues  related to NextGen at the Human Factors and Ergonomics Society’s  Aerospace Systems Technical Group meeting in May 2008 and plan to hold  another similar session at this year’s annual meeting in September.
section_title: FAA and NASA Have Several Ongoing Human Factors R&D Efforts That Support NextGen, but a Majority of Experts Suggest FAA and NASA Adopt Additional Measures, Including Strengthening Human Factors Leadership
subsections: section_title: FAA and NASA Have Undertaken a Variety of Human Factors R&D Efforts and Designated Funding to Support NextGen
paragraphs: FAA and NASA have created and shared planning documents for how the  agency will incorporate human factors R&D into NextGen. As previously  noted, FAA has taken steps to standardize the way it integrates human  factors considerations into all aviation projects. To this end, FAA  developed a NextGen Human System Integration Roadmap to identify and  address human factors R&D needs for supporting NextGen in particular.  In addition, as previously discussed, FAA created the Human Factors  Portfolio, which lists and describes all past, ongoing, and planned  NextGen human factors R&D projects. According to FAA, the portfolio  was intended to identify potential gaps and unfunded R&D needs across  midterm and potential far-term operational improvements for NextGen.  Although we find it currently lacking as a coordination tool, it does  enumerate the NextGen projects that are under way, which could be  useful in terms of monitoring the efforts of other stakeholders.
In addition, HFREG officials told us that FAA has a range of human factors  R&D initiatives that support NextGen. FAA not only conducts focus  groups and interviews with a panel of human factors experts, but also  conducts live simulations and field trials to evaluate system and human  performance in different scenarios. For example, FAA conducted human  simulations with pilots and air traffic controllers in fiscal year 2008 and  planned further simulations for its High Density Airport Capacity and  Efficiency Improvement Project in fiscal year 2009. The agency also  conducts field surveys and interviews of operational personnel that are  extensively used to address major NextGen and other aviation human  factors issues that have an impact on the workforce. For example, FAA  plans to conduct a survey to assess the degree of fatigue in the controller  workforce.
NASA also has human factors research efforts that support NextGen.  Officials told us that NASA experiments with early concept technologies  that will involve human interaction, thereby fully leveraging the strengths  and mitigating the weaknesses of both the human and automated  components. NASA staff then conduct simulations to test human  compatibility and subsequently help FAA develop the technologies that  prove themselves capable of supporting NextGen.
Over the last 2 years, FAA has also dedicated financial resources  specifically to incorporating human factors R&D into NextGen. Prior to  fiscal year 2008, FAA used funding from its overall human factors R&D  budget for NextGen projects, one of various types of human factors R&D;  however, since fiscal year 2008, FAA has had a specific human factors  research and development budget for NextGen. To incorporate human  factors issues into NextGen, for example, conducting additional human  simulations and field trials, FAA invested $25.5 million in human factors  R&D specifically dedicated to NextGen from fiscal year 2008 through fiscal  year 2010, and has requested additional funding for fiscal year 2011 to  fiscal year 2013. NASA officials told us that NASA conducts applied human  factors research across its Aviation Safety and Airspace Systems programs  and does not have a specific line item budget for NextGen. According to  these officials, this research addresses human factors considerations for  new concepts and technologies applicable to NextGen. In addition,  NASA’s Aeronautics Research Mission Directorate programs were  realigned in 2006, causing difficulty in assessing funding trends across  several years of similar research activities.
section_title: Experts Cited Suggestions Regarding Human Factors Efforts Supporting NextGen
paragraphs: For the most part, aviation human factors experts we interviewed stated  that FAA’s and NASA’s human factors R&D efforts adequately support  NextGen. For example, experts commended FAA and NASA for  appropriately conducting human factors R&D according to the three-phase  implementation structure for NextGen systems. As previously mentioned,  FAA is mainly responsible for R&D to support near-term implementation  and midterm implementation, while NASA conducts much of the research  to address far-term implementation. One expert also told us that FAA, in  response to REDAC input, has developed a good method for  understanding likely human performance. NASA also has modeled  NextGen systems to predict how beneficial NextGen systems will be to  users. However, a majority of experts offered suggestions for further  incorporating human factors issues into NextGen. Experts specifically  identified the following suggestions:  Better ensure that human factors issues are fully integrated  throughout design and development of NextGen systems. Human  factors must be considered and integrated throughout the design and  development of aviation systems. Failure to fully consider human factors  issues at all stages can increase costs and delay projects. Six of nine  experts and a senior official at the Volpe National Transportation Systems  Center were concerned that NextGen developers may not be adequately  considering human factors R&D throughout the entire NextGen planning  and implementation process. FAA has not fully integrated human factors  considerations into the development of some aviation systems. For  example, FAA did not fully address human factors considerations in  developing the En Route Automation Modernization (ERAM) system,  which FAA plans to complete by 2010. According to the National Air  Traffic Controllers Association (NATCA), air traffic controllers involved in  initial operations capabilities tests at an air traffic control center in Salt  Lake City have come across significant problems with using the system.  According to NATCA, controllers have found the new formats  cumbersome, confusing, and difficult to navigate, thus indicating that FAA  did not adequately involve those who operate the system (controllers) in  the early phases of system development. As a result, to better ensure  optimal performance of ERAM, FAA will have to address these human  factors issues before it deploys the new system. This could increase the  costs or delay the implementation of other components of NextGen, such  as the previously mentioned Automatic Dependent Surveillance-Broadcast  System, since the operation of numerous NextGen components will  depend on this new system. FAA officials within the En Route Automation  Modernization office agreed with NATCA’s views on the new system and  added that the simulation capabilities of its Technical Center in Atlantic  City, New Jersey, where the agency conducts human factors testing, were  not robust enough to capture all of the problems subsequently identified  by controllers. In May 2010, however, FAA announced the building of an  Aviation Research and Technology Park near FAA’s Technical Center to  provide a central location for partners in academia, industry, and other  state and federal government agencies to work on NextGen. According to  FAA, the park is being built with no direct cost to FAA and has amassed  $3.5 million in grant funding. In June 2010, FAA issued a task order to  MITRE Corporation to conduct a programmatic review of the ERAM  problem and make an assessment of what circumstances led to the current  delay, among other things. The MITRE Corporation is expected to issue a  final report on October 1, 2010.
Similarly, in reviewing the development of the Operational and  Supportability Implementation System, the Department of  Transportation’s Inspector General reported that FAA identified a number  of significant human factors concerns with the system, such as  inadequately addressing weather information.  The Inspector General  concluded that system developers did not adequately consider human  factors research throughout design and development, thereby contributing  to the delay of the system’s implementation. Similarly, as noted in a report  we issued in 2005, FAA’s failure to provide adequate attention to human  factors issues when implementing the Standard Terminal Automation  Replacement System resulted in schedule slips and a significant cost  increase of $500 million. As noted, however, since fiscal year 2008, FAA  has designated funding solely for human factors R&D supporting NextGen.  It remains to be seen if FAA’s added emphasis on human factors research  and engineering will better ensure that human factors issues are fully  integrated into the development of future NextGen components.
Ensuring the mitigation of human factors issues also involves oversight of  contractors. HFREG officials told us that they do not track vendors to  make sure they are considering human factors R&D issues in their  development, as this is a responsibility of the program managers who lead  procurement efforts for FAA systems. However, once contracts are  awarded, contractors are supposed to follow the contract specifications,  which can include human factors system performance requirements.  HFREG officials told us that in the past they collaborated with program  office human factors coordinators to assess outside vendors’ compliance  with human factors issues; they found that the contractors were not in  compliance in all aspects, particularly human factors. In April 2010, the  Department of Transportation’s Inspector General also expressed concern  about FAA’s ineffective oversight of a contractor in developing NextGen  systems, adding that NextGen implementation will require significant  contract oversight. Furthermore, FAA’s post-implementation review of  the Advanced Technologies and Oceanic Procedures system concluded  that FAA and the contractors who developed the system did not, from a  human factors perspective, develop the system to meet FAA’s needs. The  post-implementation review recommended that for future systems, FAA  should ensure that it articulates to contractors in unambiguous terms the  human factors-related characteristics that the proposed system must meet.  According to the Chief Scientist for NextGen and Operations Planning, a  contractor developing an aviation system may have implemented human  factors designs that were originally flawed or may have had a flawed  methodology for incorporating human factors issues into system  development. FAA program offices and contractors often support the  incorporation of human factors consideration in a system by convening a  panel of controllers and obtaining their feedback. Such a method may  result in the controllers providing information regarding their preferences  instead of information regarding the usability of the system to the  controller panel. An alternative method may be to conduct a modeling  effort that analyzes data on human performance for certain components of  the system. HFREG officials also noted that under the best of  circumstances, all major and most human factors issues should be  identified and mitigated during system development, making it unusual for  additional problems to arise when a system is being implemented. To  address this issue, experts stated FAA should ensure system developers  consider human factors in all phases of the development of aviation  systems (as required by the Human Factors Policy Order). Having  oversight of system developers (including contractors) that develop  NextGen systems to make sure they adhere to FAA’s Human Factors  Policy Order would significantly reduce the possibility of expensive and  untimely delays. FAA has taken action to improve its oversight of  contractors. For example, in its June 2010 letter to MITRE, FAA requested  an assessment of the ERAM contractor’s program management procedures  and practices as part of an overall review of the program.
Improve collaboration of human factors efforts across FAA  departments. Collaboration within FAA departments is important to  ensure that aviation systems are designed and developed with agency  input from human factors researchers. Several experts we interviewed  stated that system development projects with a human factors research  component take place in different departments and offices at FAA, and  that those developing the systems do not always collaborate. While  HFREG provides R&D and engineering support, HFREG officials told us  that there is no requirement for program offices or developers to consult  with HFREG. HFREG conducted a post-implementation review of the  Advanced Technologies and Oceanic Procedures that implied that system  managers did not properly consider human factor issues. This suggests  that the system managers either did not consult human factors  stakeholders (including HFREG) or did not fully address their human  factors issues through a collaborative working relationship. As a result,  the post-implementation review concluded that from a human factors  perspective, the system that was implemented in the field was not the  system FAA had asked for. FAA’s experience in developing the Advanced  Technologies and Oceanic Procedures is an indication of what can happen  when system developers fail to collaborate with human factors specialists  and develop a comprehensive human factors program. To improve  collaboration, HFREG officials also told us that the Chief Scientist of the  NextGen and Operations Planning unit sponsored a technical interchange  meeting in January 2010 to better ensure that all FAA units involved in  NextGen development are aware of the need to fully consider human  factors in their work. The Chief Scientist plans to host another technical  interchange meeting on July 29, 2010.
section_title: Strong Leadership Would Improve Consideration of Human Factors Issues
paragraphs: A majority of the experts we interviewed agree that strong leadership is  needed to provide adequate consideration of human factors issues within  NextGen. Furthermore, a September 2008 National Academy of Public  Administration’s report identified leadership as the single most important  element of success for large-scale systems integration efforts like  NextGen. That report highlighted leadership as a NextGen implementation  challenge. The critical impact of human factors issues on NextGen  indicates that human factors issues require strong leadership to ensure  they are a priority for NextGen. FAA has not prioritized consistently  staffing the top two leadership positions within FAA that are formally  responsible for human factors R&D. Specifically, the Chief Systems  Engineer for Human Factors position has been vacant since the previous  chief retired in January 2010. Moreover, FAA did not assign a permanent  program director of HFREG for 16 months, from January 2009 until FAA  filled the position in June 2010.
The leadership void was the issue most frequently identified by the nine  experts. Seven of nine experts we interviewed told us that the lack of  leadership within FAA is a significant challenge in ensuring that human  factors R&D supports NextGen. Although a majority of the experts were  concerned that the leadership void could have prevented human factors  issues from being fully considered for NextGen, subsequently delaying the  implementation of a system, none could identify any specific examples.  Nevertheless, FAA officials emphasized the importance of both positions.  FAA officials told us that the Chief System Engineer position could be  pivotal in integrating and maximizing the effectiveness of human factors in  support of NextGen and is thus critical to prioritizing NextGen research  and resources within FAA. JPDO officials we interviewed stressed that the  program director of HFREG is the single most important position needed  to ensure that the necessary human factors R&D is conducted and that the  results are integrated into the development of NextGen systems.
According to FAA officials, FAA has not had a chance to fill the position of  Chief System Engineer—which FAA now refers to as the human factors  integration lead—because of a hiring freeze and uncertainty as to which  unit to put the position. FAA has resolved those issues and plans to begin  the process for filling the position. Officials cautioned, however, that it  may take a long time to find a qualified candidate with the right human  factors expertise and other relevant skill sets. Nonetheless, FAA would  like to fill the position by the close of fiscal year 2010. FAA officials also  told us that it took a long time to fill the position of program director for  HFREG, in part because of the long process of completing required  personnel administrative procedures. The new program director of  HFREG was formerly the acting program director and had been in that  position since the previous program director left. The assignment to  program director involved a change in the position classification that  involved several time-consuming administrative procedures to address,  according to HFREG officials and an FAA senior executive.
Experts also told us in filling these positions that the new leaders should  have adequate authority to make sure that human factors issues are  considered (particularly early in system development) and prioritized  during all phases of NextGen development. These positions currently lack  the authority to ensure that human factors issues are addressed early and  throughout the NextGen system development process. Such authority  could mitigate the need to redesign these systems after implementation  has begun, which can cause delays and add costs. For example, as  previously discussed, it has been found that FAA’s human factors plans  have not adequately addressed how humans will use newly developed  NextGen weather information. One of the experts we consulted who has  worked extensively with FAA on human factors R&D told us that a  program director of the HFREG or Chief System Engineer who has  adequate authority could have reviewed the weather information to ensure  that human factors were fully integrated into that and other NextGen  systems. However, in filling the position of program director of HFREG,  FAA did not authorize the new program director with additional authority  to review NextGen programs and ensure that human factors issues are  addressed. HFREG officials told us that FAA is conducting a review of  distribution among HFREG, service units, and other offices for  responsibility and authority to conduct human factors activities to better  serve the human factors needs of NextGen.
section_title: Conclusions
paragraphs: Human factors research must be incorporated into NextGen to ensure that  controllers, pilots, and other aviation system users can operate NextGen in  a safe and efficient manner. To this end, FAA and NASA have pursued a  wide range of efforts to incorporate human factors R&D into NextGen.  However, these and future efforts will require a sustained focus not only  across agencies but from the beginning to the end of the long process of  developing a complex system like NextGen. Some suggest that FAA can  meet this challenge by incorporating two elements into its human factors  R&D efforts:    a cross-agency plan developed in cooperation with NASA to identify,  prioritize, and coordinate NextGen human factors issues, and    strong and consistent leadership with the authority to not only prioritize  human factors issues but ensure that they are taken into account  throughout NextGen.
section_title: Recommendations for Executive Action
paragraphs: We recommend that the Secretary of Transportation direct the FAA  Administrator to take the following two actions:    create a cross-agency human factors coordination plan in cooperation  with NASA, as JPDO has previously recommended, that establishes an  agreed-upon set of initial focus areas for research, inventories existing  facilities for research, and capitalizes on past and current research of all  NextGen issues, and    assign a high priority to filling the vacancy of human factors integration  lead and structure that position and the program director of HFREG  position in a manner that provides the authority to ensure that human  factors research and development is coordinated, considered, and  prioritized in all phases of NextGen development.
section_title: Agency Comments
paragraphs: We provided a draft of this report to the Department of Transportation and  NASA for review and comment. NASA had no comments.  DOT agreed to  consider the recommendations and provided technical clarifications,  which we incorporated into the report as appropriate.
We are sending copies of this report to the Secretary of Transportation,  FAA, NASA, and interested congressional committees. The report is also  available at no charge on GAO’s Web site at http://www.gao.gov.
If you or your staff have any questions about this report, please contact me  at (202) 512-2834 or dillinghamg@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made major contributions to this report  are listed in appendix II.
section_title: Appendix I: Scope and Methodology
paragraphs: In response to your request, this report provides information on the status  of the Department of Transportation’s Federal Aviation Administration’s  (FAA) and National Aeronautics and Space Administration’s (NASA)  efforts to incorporate human factors issues into the Next Generation Air  Transportation System (NextGen). In particular, we sought to identify the  extent to which (1) FAA’s and NASA’s human factors research and  development (R&D) is coordinated, and (2) FAA’s and NASA’s human  factors R&D supports NextGen.
In determining the extent to which FAA’s and NASA’s human factors R&D  is coordinated, we obtained and analyzed information provided by FAA  and NASA officials on mechanisms in place to align human factors R&D  efforts. We asked FAA and NASA officials to describe the mechanisms that  are in place to coordinate the agency’s human factors R&D. We assessed  the information FAA and NASA officials provided us regarding their  coordination mechanisms by comparing those efforts with  recommendations issued by the Joint Planning and Development Office  (JPDO)—an interagency organization responsible for planning NextGen.  In 2008, JPDO issued a cross-agency gap analysis that found FAA and  NASA lacked a cross-agency plan for identifying and addressing priority  NextGen human factors issues. We also assessed FAA’s and NASA’s  coordination efforts by summarizing the views of nine external aviation  human factors experts who reviewed and assessed FAA’s and NASA’s  coordination mechanisms. See our discussion below for more detail  regarding the nine aviation human factors experts. We also obtained the  views of several aviation industry officials, including officials from the  Aerospace Industries Association, Air Transport Association, Air Line  Pilots Association, MITRE Corporation, National Air Traffic Controller  Association, JPDO, Volpe National Transportation Systems Center, and the  Boeing Corporation. We also reviewed relevant reports issued by GAO,  the Inspector General of the Department of Transportation, and the  National Academy of Public Administration.
In determining the extent to which FAA’s and NASA’s human factors R&D  supports NextGen, we obtained relevant planning documents from FAA  and NASA and had FAA and NASA officials provide us with detailed  descriptions of their human factors R&D efforts. We provided this  information and other related planning documents to nine aviation human  factors experts and representatives from three aviation industry  associations and asked them about their views on the extent to which  FAA’s and NASA’s human factors research supports NextGen. The experts  provided suggestions that FAA and NASA could adopt to better  incorporate human factors issues in developing NextGen, and we reported  the suggestions that a majority of experts recommended FAA and NASA  adopt. In addition, we obtained the views of several aviation industry  officials identified above.
In assessing FAA and NASA human factors R&D coordination and human  factors R&D supporting NextGen, we summarized the views of nine  aviation human factors experts. We took several steps to identify potential  aviation human factors experts. First, we identified experts in human  factors R&D that GAO had consulted in the past. We then asked cognizant  FAA and NASA officials responsible for and knowledgeable about  aviation-related human factors R&D to recommend experts in aviation- related human factors R&D. In addition, we conducted comprehensive  Internet searches for human factors aviation experts. Finally, we asked  experts identified in the first four steps to recommend other human  factors aviation experts. Taking these steps enabled us to identify 25  potential experts.
To make our final expert selection, we narrowed our selection of the 25  potential experts based on the following criteria:    knowledge of aviation-related human factors research as determined by  published research, such as human factors research related to aviation  development, and    knowledge of NextGen planning and implementation needs as determined  by research, published work, and participation in NextGen seminars,  conferences, and workshops.
Applying the criteria listed above to the 25 potential experts resulted in a  final selection of 11 experts who have significant knowledge in both  aviation-related human factors R&D and more specifically human factors  R&D pertaining to NextGen. We obtained and synthesized responses from  9 of the 11 aviation human factors experts. The experts we obtained  responses from are listed in table 1. We interviewed an additional selected  expert prior to finalizing our methodology and incorporated the expert’s  views where appropriate in this report.
section_title: Appendix II: GAO Contact and Staff Acknowledgments
subsections: section_title: Staff Acknowledgments
paragraphs: In addition to the contact above, other key contributors to this report were  Ed Laughlin, Assistant Director; Samer Abbas; Bert Japikse; Richard Hung;  Michael Mgebroff; Tina Paek; and Amy Rosewarne.