Aviation Safety: Status of Recommendations to Improve FAA's Certification and Approval Processes
Published: 2013-10-30T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Among the agency's responsibilities for aviation safety, FAA issues certificates for new aircraft and parts and grants approvals for changes to air operations and aircraft. In 2010, GAO made recommendations to improve FAA's certification and approval processes. Subsequently, the Act required FAA to work with industry to assess the certification process and address some of the findings in GAO's report. In July 2013, FAA issued reports on its efforts, including those in response to committee recommendations and FAA's implementation plans. This testimony addresses FAA's responses to the recommendations made by GAO in 2010 and the two joint FAA-industry committees concerning (1) the certification and approval processes and (2) the consistency of regulatory interpretation. It also discusses future challenges facing FAA's certification and approval processes.
This statement is based in part on GAO's 2010 report. More detailed information on the objectives, scope, and methodology for that work can be found in that report. In addition, for this statement, GAO interviewed industry representatives, reviewed the methodologies used to develop the committees' recommendations, and assessed the recommendations and FAA's planned responses to those recommendations in terms of whether they were relevant, clear, actionable, and feasible.
GAO is not making any new recommendations in this testimony.
section_title: What GAO Found
paragraphs: In 2010, GAO reported that industry stakeholders and experts believed that the Federal Aviation Administration's (FAA) certification and approval processes contribute positively to the safety of the national airspace system. However, stakeholders and experts also noted that negative certification and approval experiences--such as duplication of approvals--although infrequent, can result in delays that industry says are costly. GAO made two recommendations requiring, among other things, that FAA develop a continuous evaluative process and a method to track submission approvals. FAA addressed one recommendation and partially addressed the other. An FAA-industry committee established in response to the FAA Modernization and Reform Act of 2012 (the Act) made six recommendations to improve the certification and approval processes, including establishing a performance measurement process. In response to recommendations from the certification process committee, FAA developed an implementation plan with 14 initiatives, but the initiatives do not contain some elements essential to a performance measurement process, such as performance measures. Without performance measures, FAA will be unable to evaluate current and future programs.
GAO also reported in 2010 that variation in FAA's interpretation of standards for certification and approval decisions is a long-standing problem. A second FAA-industry committee, established in response to the Act, made recommendations concerning the consistency of regulatory interpretation. FAA reported that it is determining the feasibility of implementing the recommendations and expected to develop an action plan by December 2013. Further, FAA reported it would measure implementation, but not outcomes; measuring outcomes helps to understand if the action is having the intended effect.
Among the challenges facing FAA, its certification and approval workload is expected to grow due to the introduction of new technologies and materials and expected progress in the deployment of the Next Generation Air Transportation System. Having efficient and consistent certification and approval processes would allow FAA to better use its resources to meet these increasing workload demands and better ensure aviation safety in an era of limited resources.

section_title: Letter
paragraphs: I am pleased to be here today to discuss the Federal Aviation  Administration’s (FAA) certification process. FAA is responsible for  aviation safety, in part, by issuing certificates for new air operators, new  aircraft, and aircraft parts and equipment, as well as by granting  approvals for such things as changes to air operations and aircraft. FAA  issues certificates and approvals based on its evaluation of aviation  industry submissions against standards set forth in federal aviation  regulations and related FAA guidance. In 2010, we found that variation in  FAA’s interpretation of standards for certification and approval decisions  was a long-standing issue. While we found that the processes for  certification and approval are viewed by the aviation industry as generally  working well, the industry believes process inefficiencies have negatively  affected it. We made recommendations to address some of these  inefficiencies. The FAA Modernization and Reform Act of 2012 (the Act)  required FAA to work with industry to assess and recommend  improvements to the certification and approval processes (in Section 312)  and to establish an advisory group to address the findings in our report  related to consistency of regulatory interpretation (in Section 313). In July  2013, FAA issued reports on these issues, including recommendations  and implementation plans.
My statement today discusses FAA’s responses to the recommendations  we made in our 2010 report and the recommendations by the two industry  committees that FAA established in response to the Act concerning (1)  FAA’s certification and approval processes and (2) FAA’s consistency of  regulatory interpretations. It also discusses challenges to making further  improvements to the certification and approval processes. This statement   For that report, we convened a panel  is based in part on our 2010 report.of aviation industry and other experts. The panel included FAA senior  managers; officials representing large and small air carriers, aircraft and  aerospace product manufacturers, aviation services firms, repair stations,  and aviation consultants; and academicians specializing in aviation and  organization theory. We also interviewed aviation trade groups and  certificate and approval holders of various sizes that represented a broad  range of aviation industry sectors—including air carriers, repair stations,  and manufacturers. More detailed information on our objectives, scope,  and methodology for that work can be found in the report. In addition, in  preparing for this hearing, in October 2013 we interviewed selected  industry officials representing aircraft and parts manufacturers, airlines,  and repair stations and reviewed the two July 2013 reports prepared by  FAA and the industry committees it established to respond the Act. The  FAA-industry reports contain recommendations to FAA, information on  the method used to develop the recommendations, and FAA’s response  to the recommendations. We reviewed the methodologies used to  develop the committees’ recommendations using best practices. We also  assessed whether the committees considered the feasibility of the  recommendations in developing them. In addition, we reviewed the  recommendations and FAA’s responses to the recommendations  contained in the two reports.  We assessed the recommendations and  FAA’s planned responses to those recommendations, including 14 FAA  initiatives, in terms of whether they were relevant, clear, and actionable  using relevant criteria.report, to identify challenges.
We reviewed prior GAO work, including our 2010  We conducted the work on which this statement is based in accordance  with generally accepted government auditing standards. Those standards  require that we plan and perform the audit to obtain sufficient, appropriate  evidence to provide a reasonable basis for our findings and conclusions  based on our audit objectives. We believe that the evidence obtained  provides a reasonable basis for our findings and conclusions based on  our audit objectives.
subsections: section_title: Background
paragraphs: FAA’s Aircraft Certification Service (Aircraft Certification) and Flight  Standards Service (Flight Standards) issue certificates and approvals for  the operators and aviation products used in the national airspace system  based on standards set forth in federal aviation regulations. FAA  inspectors and engineers working in Aircraft Certification and Flight  Standards interpret and implement the regulations governing certificates  and approvals via FAA policies and guidance, such as orders, notices,  and advisory circulars. (See fig. 1.)
Aircraft Certification’s approximately 950 engineers and inspectors in 42  field offices issue approvals to the designers and manufacturers of aircraft  and aircraft engines, propellers, parts, and equipment. Since 2005,  Aircraft Certification has used project sequencing to prioritize certification  submissions on the basis of available resources. Projects are evaluated  against several criteria, including safety attributes and their impact on the  air transportation system. Figure 2 outlines the key phases in Aircraft  Certification’s approval process.
In Flight Standards, approximately 4,000 inspectors issue certificates  allowing individuals and entities to operate in the National Airspace  System (NAS). These include certificates to commercial air carriers,  operators of smaller commercial aircraft, repair stations, and pilot schools  and training centers. Flight Standards also issues approvals for programs,  such as training. Flight Standards field office managers in over 100 field  offices use the Certification Services Oversight Process to initiate  certification projects within their offices. Delays occur when FAA wait-lists  certification submissions because it does not have the resources to begin  work on them. Once FAA determines that it has the resources to oversee  an additional new certificate holder, accepted projects are processed on a  first-in, first-out basis within each office. Figure 3 illustrates the key steps  in the Flight Standards certification process.
Responsibility for the continued operational safety of the NAS is shared  by Aircraft Certification and Flight Standards, which oversee certificate  holders, monitor operators’ and air agencies’ operation and maintenance  of aircraft, and oversee designees and delegated organizations (known  as organization designation authorizations or ODA).
section_title: FAA’s Certification and Approval Processes
paragraphs: In 2010, we reported that many of FAA’s certification and approval  processes contribute positively to the safety of the NAS, according to  industry stakeholders and experts.and approval processes work well most of the time because of FAA’s  long-standing collaboration with industry, flexibility within the processes,  and committed, competent FAA staff. Industry stakeholders and experts  noted that negative certification and approval experiences, such as  duplication of approvals, although infrequent, can result in costly delays  for them, which can disproportionately affect smaller operators. We made  two recommendations to improve the efficiency of the certification and  approval processes.  FAA addressed one recommendation and partially  addressed the other. We found that while FAA had taken actions to  improve the efficiency of its certification and approval processes, it lacked  outcome-based performance measures and a continuous evaluative  process to determine if these actions were having the intended effects. To  address these issues, we recommended that FAA develop a continuous  evaluative process and use it to create measurable performance goals for  the actions, track performance toward those goals, and determine  appropriate process changes. To the extent that this evaluation of agency  actions identifies effective practices, we further recommended that FAA  consider instituting those practices agency wide, i.e., in Aircraft  Certification and Flight Standards. In response to our recommendation,  FAA implemented new metrics that provide the ability to track process  performance and product conformity to standards. These metrics would  allow FAA to set measurable performance goals necessary to determine  the effectiveness of the certification and approval processes and assist  FAA in deciding on necessary and appropriate actions to address  systemic issues that could negatively impact agency processes and their  outcomes. These actions addressed the intent of our recommendation.  We also recommended that FAA develop and implement a process in   They also noted that the certification  Flight Standards to track how long certification and approval submissions  are wait-listed, the reasons for wait-listing them, and the factors that  eventually allowed initiation of the certification process. As of October  2013, FAA had partially addressed this recommendation by altering the  software in its Flight Standards’ Certification Service Oversight Process  database to designate when certification submissions are wait-listed. The  database now tracks how long certification submissions are wait-listed.  As a result, FAA now has the capability to track how long certification  submissions are wait-listed and reallocate resources, if appropriate, to  better meet demand.
In April 2012, as required by Section 312 of the Act, FAA established the  Aircraft Certification Process Review and Reform Aviation Rulemaking  Committee (certification process committee). Its role is to make  recommendations to the director of FAA’s Aircraft Certification Service to  streamline and reengineer the certification process. The committee  considered guidance and current certification issues—including methods  for enhancing the use of delegation and the training of FAA staff in safety  management systems—and assessed the certification process. It  developed six recommendations, which called for FAA to develop comprehensive implementation plans for certification process  improvement initiatives, including measuring the effectiveness of the  implementation and benefits of improvements as well as developing a  means to track and monitor initiatives and programs; continue to improve the effectiveness of delegation programs; develop an integrated, overarching vision of the future state for  certification procedures; update Part 21 certification procedures to reflect a systems approach develop and implement a comprehensive change management plan  to prepare the workforce for its new responsibilities in a systems  safety approach to certification and oversight; and   review continued operational safety and rulemaking processes and  implement reforms to improve efficiency.
We found these recommendations to be relevant, clear, and actionable.  In response to the committee’s recommendations, FAA developed a plan  that includes 14 initiatives to implement the committee’s  recommendations and publicly reported the plan in July 2013.
We believe that the committee took a reasonable approach in assessing  FAA’s aircraft certification process and developing recommendations by  assessing the status of previous recommendations from 19 reports  related to the certification process, reviewing certification guidance and  processes as well as major initiatives, and reviewing other areas that it  believed required consideration when making recommendations for  improving efficiencies in the certification process. FAA has many  initiatives and programs underway that it believes will respond to the  committee’s recommendations to improve efficiency and reduce costs  related to certifications. For example, FAA and two industry groups had  already developed an ODA action plan to address the effectiveness of the  ODA process. We found these initiatives were generally relevant to the  recommendations and clear and measurable. However, FAA’s initiatives  and programs to implement the recommendations do not contain some of  the elements essential to a performance measurement process.   For  example, the certification process committee recommended that FAA  develop an integrated roadmap and vision for certification process  reforms, including an integrated overarching vision of the future state for  certification procedures. While FAA has outlined a vision in AIR: 2018, it  has not yet developed a roadmap. FAA is planning to roll out its roadmap,  which is to include information on major change initiatives and a scaled  change management process, concurrently with or following  implementation of many of its certification process improvement  initiatives. This calls into question FAA’s ability to use the roadmap to  guide the initiatives.
FAA has developed milestones for each initiative and deployed a tracking  system to track and monitor the implementation of all certification-related  initiatives. However, FAA has not yet developed performance measures  to track the success of most of the initiatives and programs. The agency  plans to develop these measures of effectiveness after it has  implemented its initiatives. Without early performance measures, FAA will  not be able to gather the appropriate data to evaluate the success of  current and future initiatives and programs. In addition, in response to the  certification process committee’s recommendation to review rulemaking  processes and implement reforms to improve efficiency, FAA plans to  expedite the rulemaking process by implementing a new rulemaking  prioritization model. However, this model will have no effect on the  duration of the rulemaking process since it only prioritizes potential  rulemaking projects for submission to the rulemaking process and makes  no changes to the rulemaking process per se.
section_title: Consistency of Regulatory Interpretation
paragraphs: In 2010, we reported that variation in FAA’s interpretation of standards for  certification and approval decisions is a long-standing issue that can  result in delays and higher costs for industry. For example, a 1996 study  found that, for air carriers and other operators, FAA’s regulations are  often ambiguous; subject to variation in interpretation by FAA inspectors,  supervisors, or policy managers; and in need of simplification and  consistent implementation.officials we interviewed for our 2010 report indicated that although  variation in decisions is a long-standing, widespread problem, it has rarely  led to serious certification and approval process problems, and experts on  our panel generally noted that serious problems occur less than 10   Experts on our panel and most industry  percent of the time. Nonetheless, when such occasions occur, experts on  our panel ranked inconsistent interpretation of regulations, which can lead  to variation in decisions, as the most significant problem for Flight  Standards and as the second most significant problem for Aircraft  Certification. Panelists’ concerns about variation in decisions included  instances in which approvals are reevaluated and sometimes revised or  revoked in FAA jurisdictions other than those in which they were originally  granted. Such situations can result in delays and higher costs for industry  but also may catch legitimate safety concerns. According to industry  stakeholders we spoke with, variation in FAA’s interpretation of standards  for certification and approval decisions is a result of factors related to  performance-based regulations, which allow for multiple avenues of  compliance, and the use of professional judgment by FAA staff. FAA’s  Deputy Associate Administrator for Aviation Safety and union officials  representing FAA inspectors and engineers acknowledged that variation  in certification and approval decisions occurs and that FAA has taken  actions to address the issue, including the establishment of a quality  management system to standardize processes across offices.
A second FAA-industry committee—the Consistency of Regulatory  Interpretation Aviation Rulemaking Committee (regulatory consistency  committee)—established to respond to Section 313 of the Act, identified  three root causes of inconsistent interpretation of regulations—(1) unclear  regulatory requirements; (2) inadequate and nonstandard FAA and  industry training in developing regulations, applying standards, and  resolving disputes; and (3) a culture that includes a general reluctance by  both industry and FAA to work issues of inconsistent regulatory  application through to a final resolution and a “fear of retribution.” The root  causes are consistent with issues raised in our 2010 review and those  raised by industry during that review. To address the root causes, the  committee made six recommendations to promote clearer regulations and  guidance, more standardized application of rules, a consolidation and  cross-reference of guidance and rules, and improved communication  between FAA and industry. In priority order, those recommendations  called for developing a single master source for guidance organized by Title 14  of the Code of Federal Regulations (which covers commercial  aviation); developing instructions for FAA staff with policy development  responsibilities;  reviewing FAA and industry training priorities and curriculums; setting up a board to provide clarification to industry and FAA on  improving the clarity in final rules issued by FAA; and creating a communications center to act as a central clearinghouse to  assist FAA staff with queries about interpretation of regulations.
We found that the committee took a reasonable approach in identifying  these root causes and developing its recommendations. It compiled and  reviewed case studies involving issues of regulatory application, obtained  additional information by surveying industry stakeholders, and reviewed  FAA regulatory guidance material. The recommendations are relevant to  the root causes, actionable, and clear. The committee also considered  the feasibility of the recommendations by identifying modifications to  existing efforts and programs and prioritizing the recommendations.
FAA reported on July 19, 2013, that it is determining the feasibility of  implementing these recommendations. The agency told us that it  expected to develop an action plan to address the recommendations and  metrics to measure implementation by December 2013. We note that  measuring implementation may provide useful information, however, FAA  is not intending to measure outcomes. Measuring outcomes can help in  understanding if an action is having the intended effect.
section_title: Challenges Moving Forward
paragraphs: FAA’s certification and approval processes generally work well. However,  when the certification and approval processes do not work well, the result  can be costly for industry and FAA. Inconsistent interpretation of  regulations can lead to rework by FAA and industry. Likewise, inefficient  processes can require extra time and resources. FAA faces challenges in  implementing the committees’ recommendations and further improving its  certification and approval processes. FAA’s certification and approval  workload is expected to grow over the next 10 years because of activities  such as the introduction of new technologies and materials, such as  composite materials used in airplanes, according to one industry  committee report. Additional work will be needed to establish new  means of compliance and establish new standards. In addition, FAA’s  certification and approval workload is likely to increase substantially as  the Next Generation Air Transportation System (NextGen) progresses  and operators will need to install additional equipment on their aircraft to  take full advantage of NextGen capabilities. Having certification and  approval processes that work well will allow FAA to better meet these  increasing workload demands and better ensure aviation safety in an era  of limited resources.
To its credit, FAA has taken steps toward improving the efficiency of its  certification and approval processes. It will be critical for FAA to follow  through with its plans for implementing the key recommendations to  achieve the intended efficiencies and streamlining. However, making  fundamental changes to the certification and approval processes can  require a cultural change by its workforce and resistance to change can  cause delays. Some improvements to the processes, such as those  requiring new rulemakings, will likely take years to implement and,  therefore, will require a sustained commitment as well as congressional  oversight.
Chairman LoBiondo, Ranking Member Larsen, and members of the  Subcommittee, this concludes my prepared statement. I would be  pleased to answer any questions at this time.
section_title: GAO Contact and Staff Acknowledgements
paragraphs: For further information on this testimony, please contact Gerald L.  Dillingham, Ph.D., at (202) 512-2834 or dillinghamg@gao.gov. In  addition, contact points for our Offices of Congressional Relations and  Public Affairs may be found on the last page of this statement. Individuals   making key contributions to this testimony include Teresa Spisak  (Assistant Director), Pamela Vines, Melissa Bodeau, David Hooper, Sara  Ann Moessbauer, Josh Ormond, and Jessica Wintfeld.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.