FAA’s Aircraft Certification Service (Aircraft Certification) and Flight Standards Service (Flight Standards) issue certificates and approvals for the operators and aviation products used in the national airspace system based on standards set forth in federal aviation regulations. FAA inspectors and engineers working in Aircraft Certification and Flight Standards interpret and implement the regulations governing certificates and approvals via FAA policies and guidance, such as orders, notices, and advisory circulars. (See fig. 1.) Aircraft Certification’s approximately 950 engineers and inspectors in 42 field offices issue approvals to the designers and manufacturers of aircraft and aircraft engines, propellers, parts, and equipment. Since 2005, Aircraft Certification has used project sequencing to prioritize certification submissions on the basis of available resources. Projects are evaluated against several criteria, including safety attributes and their impact on the air transportation system. Figure 2 outlines the key phases in Aircraft Certification’s approval process. In Flight Standards, approximately 4,000 inspectors issue certificates allowing individuals and entities to operate in the National Airspace System (NAS). These include certificates to commercial air carriers, operators of smaller commercial aircraft, repair stations, and pilot schools and training centers. Flight Standards also issues approvals for programs, such as training. Flight Standards field office managers in over 100 field offices use the Certification Services Oversight Process to initiate certification projects within their offices. Delays occur when FAA wait-lists certification submissions because it does not have the resources to begin work on them. Once FAA determines that it has the resources to oversee an additional new certificate holder, accepted projects are processed on a first-in, first-out basis within each office. Figure 3 illustrates the key steps in the Flight Standards certification process. Responsibility for the continued operational safety of the NAS is shared by Aircraft Certification and Flight Standards, which oversee certificate holders, monitor operators’ and air agencies’ operation and maintenance of aircraft, and oversee designees and delegated organizations (known as organization designation authorizations or ODA). In 2010, we reported that many of FAA’s certification and approval processes contribute positively to the safety of the NAS, according to industry stakeholders and experts.and approval processes work well most of the time because of FAA’s long-standing collaboration with industry, flexibility within the processes, and committed, competent FAA staff. Industry stakeholders and experts noted that negative certification and approval experiences, such as duplication of approvals, although infrequent, can result in costly delays for them, which can disproportionately affect smaller operators. We made two recommendations to improve the efficiency of the certification and approval processes. FAA addressed one recommendation and partially addressed the other. We found that while FAA had taken actions to improve the efficiency of its certification and approval processes, it lacked outcome-based performance measures and a continuous evaluative process to determine if these actions were having the intended effects. To address these issues, we recommended that FAA develop a continuous evaluative process and use it to create measurable performance goals for the actions, track performance toward those goals, and determine appropriate process changes. To the extent that this evaluation of agency actions identifies effective practices, we further recommended that FAA consider instituting those practices agency wide, i.e., in Aircraft Certification and Flight Standards. In response to our recommendation, FAA implemented new metrics that provide the ability to track process performance and product conformity to standards. These metrics would allow FAA to set measurable performance goals necessary to determine the effectiveness of the certification and approval processes and assist FAA in deciding on necessary and appropriate actions to address systemic issues that could negatively impact agency processes and their outcomes. These actions addressed the intent of our recommendation. We also recommended that FAA develop and implement a process in They also noted that the certification Flight Standards to track how long certification and approval submissions are wait-listed, the reasons for wait-listing them, and the factors that eventually allowed initiation of the certification process. As of October 2013, FAA had partially addressed this recommendation by altering the software in its Flight Standards’ Certification Service Oversight Process database to designate when certification submissions are wait-listed. The database now tracks how long certification submissions are wait-listed. As a result, FAA now has the capability to track how long certification submissions are wait-listed and reallocate resources, if appropriate, to better meet demand. In April 2012, as required by Section 312 of the Act, FAA established the Aircraft Certification Process Review and Reform Aviation Rulemaking Committee (certification process committee). Its role is to make recommendations to the director of FAA’s Aircraft Certification Service to streamline and reengineer the certification process. The committee considered guidance and current certification issues—including methods for enhancing the use of delegation and the training of FAA staff in safety management systems—and assessed the certification process. It developed six recommendations, which called for FAA to develop comprehensive implementation plans for certification process improvement initiatives, including measuring the effectiveness of the implementation and benefits of improvements as well as developing a means to track and monitor initiatives and programs; continue to improve the effectiveness of delegation programs; develop an integrated, overarching vision of the future state for certification procedures; update Part 21 certification procedures to reflect a systems approach develop and implement a comprehensive change management plan to prepare the workforce for its new responsibilities in a systems safety approach to certification and oversight; and review continued operational safety and rulemaking processes and implement reforms to improve efficiency. We found these recommendations to be relevant, clear, and actionable. In response to the committee’s recommendations, FAA developed a plan that includes 14 initiatives to implement the committee’s recommendations and publicly reported the plan in July 2013. We believe that the committee took a reasonable approach in assessing FAA’s aircraft certification process and developing recommendations by assessing the status of previous recommendations from 19 reports related to the certification process, reviewing certification guidance and processes as well as major initiatives, and reviewing other areas that it believed required consideration when making recommendations for improving efficiencies in the certification process. FAA has many initiatives and programs underway that it believes will respond to the committee’s recommendations to improve efficiency and reduce costs related to certifications. For example, FAA and two industry groups had already developed an ODA action plan to address the effectiveness of the ODA process. We found these initiatives were generally relevant to the recommendations and clear and measurable. However, FAA’s initiatives and programs to implement the recommendations do not contain some of the elements essential to a performance measurement process. For example, the certification process committee recommended that FAA develop an integrated roadmap and vision for certification process reforms, including an integrated overarching vision of the future state for certification procedures. While FAA has outlined a vision in AIR: 2018, it has not yet developed a roadmap. FAA is planning to roll out its roadmap, which is to include information on major change initiatives and a scaled change management process, concurrently with or following implementation of many of its certification process improvement initiatives. This calls into question FAA’s ability to use the roadmap to guide the initiatives. FAA has developed milestones for each initiative and deployed a tracking system to track and monitor the implementation of all certification-related initiatives. However, FAA has not yet developed performance measures to track the success of most of the initiatives and programs. The agency plans to develop these measures of effectiveness after it has implemented its initiatives. Without early performance measures, FAA will not be able to gather the appropriate data to evaluate the success of current and future initiatives and programs. In addition, in response to the certification process committee’s recommendation to review rulemaking processes and implement reforms to improve efficiency, FAA plans to expedite the rulemaking process by implementing a new rulemaking prioritization model. However, this model will have no effect on the duration of the rulemaking process since it only prioritizes potential rulemaking projects for submission to the rulemaking process and makes no changes to the rulemaking process per se. In 2010, we reported that variation in FAA’s interpretation of standards for certification and approval decisions is a long-standing issue that can result in delays and higher costs for industry. For example, a 1996 study found that, for air carriers and other operators, FAA’s regulations are often ambiguous; subject to variation in interpretation by FAA inspectors, supervisors, or policy managers; and in need of simplification and consistent implementation.officials we interviewed for our 2010 report indicated that although variation in decisions is a long-standing, widespread problem, it has rarely led to serious certification and approval process problems, and experts on our panel generally noted that serious problems occur less than 10 Experts on our panel and most industry percent of the time. Nonetheless, when such occasions occur, experts on our panel ranked inconsistent interpretation of regulations, which can lead to variation in decisions, as the most significant problem for Flight Standards and as the second most significant problem for Aircraft Certification. Panelists’ concerns about variation in decisions included instances in which approvals are reevaluated and sometimes revised or revoked in FAA jurisdictions other than those in which they were originally granted. Such situations can result in delays and higher costs for industry but also may catch legitimate safety concerns. According to industry stakeholders we spoke with, variation in FAA’s interpretation of standards for certification and approval decisions is a result of factors related to performance-based regulations, which allow for multiple avenues of compliance, and the use of professional judgment by FAA staff. FAA’s Deputy Associate Administrator for Aviation Safety and union officials representing FAA inspectors and engineers acknowledged that variation in certification and approval decisions occurs and that FAA has taken actions to address the issue, including the establishment of a quality management system to standardize processes across offices.