Aviation Safety: Certification and Approval Processes Are Generally Viewed as Working Well, but Better Evaluative Information Needed to Improve Efficiency
Released: 2010-10-29T00:00:00

Letter

Background

Located in FAA’s Office of Aviation Safety, the Aircraft Certification Service (Aircraft Certification) and Flight Standards Service (Flight Standards) issue certificates and approvals for the operators and aviation products used in the national airspace system based on standards set forth in federal aviation regulations. FAA inspectors and engineers working in Aircraft Certification and Flight Standards interpret and implement the regulations governing certificates and approvals via FAA policies and guidance, such as orders, notices, and advisory circulars.

Aircraft Certification

Aircraft Certification’s approximately 950 engineers and inspectors in 38 field offices issue approvals to the designers and manufacturers of aircraft and aircraft engines, propellers, parts, and equipment, including the avionics and other equipment required for the Next Generation Air Transportation System (NextGen)—a federal effort to transform the U.S. national airspace system from a ground-based system of air traffic control to a satellite-based system of air traffic management. These approvals are issued in three areas: (1) design—including type certificates for new aircraft, engine, or propeller designs, amended type certificates (issued only to the type certificate holder) for derivative models, and supplemental type certificates for major changes to existing designs by either the type certificate holder or someone other than the original type certificate holder; (2) production—including production certificates, which certify a manufacturer’s ability to build an aircraft, engine, or propeller in accordance with an FAA-approved design, and parts manufacturer approvals for spare and replacement parts; and (3) flight approval—original airworthiness certificates and approvals for newly manufactured aircraft, engines, propellers, and parts. Aircraft Certification, along with Flight Standards, provides a safety performance management system intended to assure the continued operational safety of all aircraft operating in the national airspace system and of U.S.-built aircraft operating anywhere in the world. Aircraft Certification is also responsible for the appointment and oversight of designees and delegated organizations that play a critical role in acting on behalf of FAA to perform many certification and approval activities, such as the issuance of design and airworthiness approvals for aircraft parts.

Since 2005, Aircraft Certification has used project sequencing to prioritize certification submissions on the basis of available resources. Projects are evaluated against several criteria, including safety attributes and their impact on the air transportation system. In fiscal year 2009, Aircraft Certification issued 4,248 design approvals, 2,971 production approvals, and 508 airworthiness certificates. Figure 1 shows the Aircraft Certification approvals issued for fiscal years 2005 through 2009. As of June 2010, according to FAA, Aircraft Certification had a backlog of 47 projects. (According to a senior FAA official, the number of approvals decreased from fiscal year 2006 to fiscal year 2007 because Aircraft Certification implemented a new data collection system in fiscal year 2007 that improved data collection definitions and processes.) Figure 2 contains key information about Aircraft Certification’s organization, and figure 3 indicates key phases in Aircraft Certification’s product approvals process.

Flight Standards

Flight Standards’ nearly 4,000 inspectors issue certificates allowing individuals and entities to operate in the national airspace system. Flight Standards also issues approvals for programs, such as training and minimum equipment lists. Flight Standards field office managers in over 100 field offices use the Certification Services Oversight Process to initiate certification projects within their offices. According to FAA, the field offices are also assisted by a headquarters-based office that provides experts on specific aircraft and airlines. Accepted projects are processed on a first-in, first-out basis within each office once FAA determines that it has the resources to oversee an additional new certificate holder. Flight Standards issued 599 air operator and air agency certificates in fiscal year 2009. These include certificates to commercial air carriers under 14 C.F.R. part 121, operators of smaller commercial aircraft under 14 C.F.R. part 135, repair stations under 14 C.F.R. part 145, and pilot schools and training centers under 14 C.F.R. parts 141 and 142, respectively. According to its Director, Flight Standards also issues over 6,000 approvals daily. Figure 4 shows the number of air operator and air agency certificates issued by Flight Standards in fiscal years 2005 through 2009.

FAA officials noted that certification projects within and among the categories of air operators and air agencies require various amounts of FAA resources. For example, FAA indicated that an agricultural operator certification requires fewer FAA resources than a repair station certification. Additionally, certifications of small commercial aircraft operations that are single pilot, single plane require a different set of resources than operations that are dual pilot and/or fly more aircraft. As of July 2010, Flight Standards had 1,142 certifications in process and a backlog of 489 applications. According to an FAA official, Flight Standards has more wait-listed applications than Aircraft Certification because it receives numerous requests for certificates, and its certifications are substantially different in nature from those issued by Aircraft Certification.

Flight Standards is also responsible for assuring the continued operational safety of the national airspace system by overseeing certificate holders, monitoring (along with Aircraft Certification) operators’ and air agencies’ operation and maintenance of aircraft, and overseeing designees and delegated organizations. Flight Standards inspectors were tasked with overseeing 13,089 air operators and air agencies, such as repair stations, as of March 2010. Unless assigned to a large commercial air carrier issued a certificate under part 121, a Flight Standards inspector is typically responsible for overseeing several entities that often perform different or several functions within the system—including transporting passengers, repairing aircraft, and training pilots.

Figures 5 and 6 contain key information about Flight Standards’ organization and certification process for air operators and air agencies.

Extent of Variation in Interpretation Is Unknown but Potentially Stems from Factors Related to Performance-Based Regulations and FAA’s Processes

Extent of Variation in FAA’s Interpretation of Standards for Certification and Approval Decisions Is Unknown, but Stakeholders and Experts Indicate That Serious Problems Occur Infrequently

Studies we reviewed and aviation stakeholders and experts we spoke with indicated that variation in FAA’s interpretation of standards for certification and approval decisions is a long-standing issue that affects both Aircraft Certification and Flight Standards, but the extent of the problem has not been quantified in the industry as a whole. Inconsistent or variant FAA interpretations have been noted in studies published over the last 14 years. A 1996 study by Booz Allen & Hamilton, conducted at the request of the FAA Administrator to assess challenges to the agency’s regulatory and certification practices, reported that, for air carriers and other operators, the agency’s regulations are often ambiguous; subject to variation in interpretation by FAA inspectors, supervisors, and policy managers; and in need of simplification and consistent implementation. A 1999 task force, convened at the request of the FAA Administrator to assess FAA’s certification process, found that the agency’s requirements for the various approvals—such as type certificates and supplemental type certificates—varied substantially because of differences in standards and inconsistent application of those standards by different FAA field offices. While FAA has put measures in place since these two reports appeared, a 2008 Independent Review Team, which was commissioned by the Secretary of Transportation to assess FAA’s safety culture and approach to safety management, found that a wide degree of variation in “regulatory ideology” among FAA staff continues to create the likelihood of wide variation in decisions within and among field offices.

Industry officials and experts representing a broad range of large and small aviation businesses told us that variation in interpretation and subsequent decisions occurs in both Aircraft Certification and Flight Standards, but we found no evidence that quantified the extent of the problem in the industry as a whole. Specifically, 10 of the 13 industry group and individual company representatives we interviewed said that they or members of their organization experienced variation in FAA’s certification and approval decisions on similar submissions; the remaining 3 industry representatives did not raise variation in interpretations and decisions as an issue. For example, an official from one air carrier told us that variation in decisions occurs regularly when obtaining approvals from Flight Standards district offices, especially when dealing with inspectors who are newly hired or replacing a previous inspector. He explained that new inspectors often task air carriers to make changes to previously obtained minimum equipment lists or conformity approvals for an aircraft. The official further noted that inspector assignments often change for reasons such as transfers, promotions, or retirement and that four different principal operations inspectors were assigned to his company during the past 18 months.

Experts on our panel and most industry officials we interviewed indicated that, though variation in decisions is a long-standing, widespread problem, it has rarely led to serious certification and approval process problems. Experts on our panel generally noted that serious problems with the certification and approval processes occur less than 10 percent of the time. However, when we asked them to rank certification and approval process problems we summarized from their discussion, they chose inconsistent interpretation of regulations, which can lead to variation in decisions, as the most significant problem for Flight Standards and as the second most significant problem for Aircraft Certification. Panelists’ concerns about variation in decisions included instances in which approvals are reevaluated and sometimes revised or revoked in FAA jurisdictions other than those in which they were originally granted. Industry officials we interviewed, though most had experienced it, did not mention the frequency with which variation in decisions occurred. However, 8 of the 13 said that their experiences with FAA’s certification and approval processes were generally free of problems compared with 3 who said they regularly experienced problems with the process.

FAA’s Deputy Associate Administrator for Aviation Safety and union officials representing FAA inspectors and engineers acknowledged that variation in certification and approval decisions occurs. The Deputy Associate Administrator noted that variation in interpretation and certification and approval decisions occurs in both Aircraft Certification and Flight Standards. He acknowledged that a nonstandardized process for approvals exists and has been a challenge for, and a long-term criticism of, the agency. Furthermore, he explained that efforts were being made to address the issue, including the establishment of (1) an Office of Aviation Safety quality management system (QMS) to standardize processes across Aircraft Certification and Flight Standards, (2) a process for industry to dispute FAA decisions, and (3) standardization offices within Aircraft Certification directorates. The first two efforts are discussed in greater detail later in this report.

Industry Stakeholders Noted That Variation in Decisions Occurs as a Consequence of Performance-Based Regulations and FAA’s Exercise of Professional Judgment

Variation in FAA’s interpretation of standards and certification and approval decisions occurs as a result of factors related to performance- based regulations and the use of professional judgment by FAA inspectors and engineers, according to industry stakeholders. FAA uses performance- based regulations, which identify a desired outcome and are flexible about how the outcome is achieved. For example, performance-based regulations on aircraft braking would establish minimum braking distances for aircraft but would not call for a particular material in the brake pads or a specific braking system design. According to officials in FAA’s rulemaking office, about 20 percent of FAA’s regulations are performance-based. Performance-based regulations, which are issued governmentwide, provide a number of benefits, according to literature on the regulatory process. By focusing on outcomes, for example, performance-based regulations give firms flexibility in achieving the stated level of performance; such regulations can accommodate technological change in ways that prescriptive regulations that focus on a specific technology generally cannot. For those certifications and approvals that relate to performance-based regulations, variation in decisions is a consequence of such regulations, according to one air carrier, since performance-based regulations allow the applicant multiple avenues to comply with regulations and broader discretion by FAA staff in making certification and approval decisions. According to senior FAA officials, performance-based regulations allow innovation and flexibility while setting a specific safety standard. The officials added that the benefits of performance-based regulations outweigh the potential for erroneous interpretation by an individual inspector or engineer. While agreeing with this statement, a panel member pointed out that the potential for erroneous interpretation also entails a risk of inconsistent decisions.

In addition, FAA oversees a large, diverse industry, and its certification and approval processes rely, in part, on FAA staffs’ exercise of professional judgment in the unique situations they encounter. In the opinion of senior FAA officials, some differences among inspectors may be due to situation-specific factors that industry stakeholders may not be aware of. According to officials from Flight Standards, because differences may exist among regions and district offices, operators changing locations may encounter these differences.

Key Stakeholders and Experts Said the Certification and Approval Processes Generally Work Well, but When They Do Not, It Can Be Costly for Industry

Stakeholders and Experts Said the Certification and Approval Processes Contribute to System Safety and Work Well Most of the Time

Many industry stakeholders and experts stated that FAA’s certification and approval processes contribute positively to the safety of the national airspace system. For example, industry stakeholders who participated in our expert panel ranked the office’s safety culture and record as the greatest strength of Flight Standards’ certification and approval processes and the third greatest strength of Aircraft Certification’s processes.

Industry stakeholders and experts also noted that the certification and approval processes work well most of the time because of FAA’s long- standing collaboration with industry, flexibility within the processes, and committed, competent FAA staff. In most instances, stakeholders and experts said, when industry seeks certifications and approvals, its experiences with FAA’s processes are positive. For example, two aviation manufacturers and an industry trade association with over 400,000 members noted that most of their experiences or their members’ experiences were positive. Seventeen of 19 panelists indicated positive or very positive experiences with Aircraft Certification, and 9 of 19 panelists indicated positive experiences with Flight Standards. Panelists ranked FAA’s collaboration with applicants highly—as the second greatest strength of both Aircraft Certification and Flight Standards. In addition, representatives of two trade associations representing over 190 aviation companies said that the processes provide flexibility for a large, diverse industry. Additionally, panelists ranked FAA inspectors’ and engineers’ expertise as the greatest strength of Aircraft Certification and the third greatest strength of Flight Standards, while officials from two industry trade groups cited the inspectors’ and engineers’ competence and high level of expertise.

Industry Stakeholders Said Negative Experiences Are Infrequent but Can Cause Costly Delays

Industry stakeholders and experts noted that negative certification and approval experiences, although infrequent, can result in costly delays for them, which can disproportionately affect smaller operators. While industry stakeholders indicated that negative experiences occur in dealings with both Aircraft Certification and Flight Standards, experts on our panel noted that negative experiences are more likely to occur with Flight Standards than with Aircraft Certification. For example, three experts noted that, overall, industry’s experience in obtaining certifications and approvals from Flight Standards has been negative or very negative, while no experts thought industry’s experience with Aircraft Certification was negative. The panelists indicated that negative experiences occur during the processing of certifications and approvals and as applicants wait for FAA resources to become available to commence their certification or approval projects. For example, an aviation industry representative reported that his company incurred a delay of over 5 years and millions of dollars in costs when it attempted to obtain approvals from Aircraft Certification and Flight Standards field offices. Another industry representative indicated that it abandoned an effort to obtain an operating certification after spending $1.2 million and never receiving an explanation from FAA as to why the company’s application was stalled. One panelist indicated that the negative experiences focus more on administrative aspects of the certification and approval processes and not on safety-related items.

The processing of original certifications and approvals in Aircraft Certification and Flight Standards involves progressing through a schedule of steps or phases. Responsibilities of both FAA and the applicant are delineated. However, even with this framework in place, industry stakeholders noted that the time it takes to obtain certifications and approvals can differ from one FAA field office to another because of differences in office resources and expertise. In some cases, delays may be avoided when FAA directs the applicant to apply at a different field office.

Nevertheless, applicants who must apply to offices with fewer resources can experience costly delays in obtaining certifications or approvals.

Delays also occur when FAA wait-lists certification submissions because it does not have the resources to begin work on them. Aircraft Certification meets weekly to review certification project submissions. If it determines that a submission is to be wait-listed, the applicant is sent a 90-day delay letter and if, after the initial 90 days, the submission is still wait-listed, the applicant is sent another letter. Additionally, Aircraft Certification staff and managers periodically contact applicants to advise them of the status of their submissions. Flight Standards also notifies applicants when their certification submissions are wait-listed, and Flight Standards staff are encouraged to communicate with applicants regularly about the status of their submissions. However, according to an FAA notice, staff are advised not to provide an estimate of when an applicant’s submission might be processed. While Aircraft Certification tracks in a national database how long individual submissions are wait-listed, Flight Standards does not. Without data on how long submissions are wait-listed, Flight Standards cannot assess the extent of wait-listing delays or reallocate resources to better meet demand. Further, industry stakeholders face uncertainty with respect to any plans or investments that depend on obtaining a certificate in a timely manner.

Industry stakeholders have also raised concerns about the effects of inefficiencies in the certification and approval processes on the implementation of NextGen. As NextGen progresses, operators will need to install additional equipment on their aircraft to take full advantage of NextGen capabilities, and FAA’s certification and approval workload is likely to increase substantially. According to our October 2009 testimony on NextGen, airlines and manufacturers said that FAA’s certification processes take too long and impose costs on industry that discourage them from investing in NextGen equipment. We reported that this inefficiency in FAA’s processes constitutes a challenge to delivering NextGen benefits to stakeholders and that streamlining FAA’s processes will be essential for the timely implementation of NextGen. FAA is working to address the certification issues that may impede the adoption and acceleration of NextGen capabilities. Flight Standards has identified NextGen-dedicated staff in each of its regional offices to support the review and approval of NextGen capabilities within each region. Aircraft Certification has created a team of experts from different offices to coordinate NextGen approvals and identify specialists in Aircraft Certification offices with significant NextGen activity. FAA also plans a number of other actions to facilitate the certification and approval of NextGen-related technology, including new procedures and criteria for prioritizing certifications, updating policy and guidance, developing additional communication mechanisms, and developing training for inspectors and engineers. Since many of these actions have either just been implemented or have not yet been completed, it is too early to tell whether they will increase the efficiency of FAA’s certification and approval processes and reduce unanticipated delays and costs for the industry.

Industry Stakeholders and Experts Told Us That the Efficiency of FAA’s Processes Is Hampered by Several Shortcomings; FAA Has Taken Some Actions to Remedy Them

Industry stakeholders also noted that the efficiency of the certification and approval processes was hampered by a lack of sufficient staff resources to carry out certifications and approvals and a lack of effective communication mechanisms for explaining the intent of the regulations to both FAA staff and industry. The stakeholders said that these inefficiencies have resulted in costly delays for them.

Stakeholders and experts said that, at some FAA offices, delays in obtaining certifications and approvals were due to heavy staff workloads, a lack of staff, or a lack of staff with the appropriate expertise. Staff and managers at one FAA field office told us that in the past a lack of staff had contributed to delays in completing certifications. The relative priority of certifications and approvals within FAA’s overall workload also affects the availability of staff to process certifications and approvals. According to FAA, its highest priority is overseeing the continued operational safety of the people and products already operating within the national airspace system, but the same staff who provide this oversight are also tasked with the lower-priority task of processing new certifications and approvals. Additionally, Flight Standards field staff we contacted said that the system under which their pay grades are established and maintained provides a disincentive for inspectors to perform certification work because the system allocates no credit toward retention of their pay grades for doing certification work. Flight Standards headquarters officials pointed out that there is an incentive for field office inspectors to perform initial certifications because once certificated the new entities add points to an inspector’s complexity calculation, which is used to determine his or her pay grade.

FAA has addressed staff resource issues by increasing the number of inspectors and engineers. Over the past 3 years, FAA has steadily increased its hiring of Aircraft Certification engineers and Flight Standards inspectors, thereby reducing the risk of certification delays. According to agency data, FAA’s hiring efforts since fiscal year 2007 have resulted in an 8.8 percent increase in the number of Aircraft Certification engineers and a 9.4 percent increase in the number of Flight Standards inspectors on board. FAA hired 106 engineers in Aircraft Certification and 696 inspectors in Flight Standards from the beginning of fiscal year 2007 to March 15, 2010. FAA also hired 89 inspectors in Aircraft Certification from fiscal year 2007 through August 2010. In addition, Flight Standards headquarters staff are available to assist field staff with the certification of part 121 air carriers—an average of 35 of these staff were available for this assistance annually from 2005 through 2009, and they helped with 16 certification projects.

Furthermore, FAA delegates many certification activities to individuals and organizations (called designees) to better leverage its resources. As we previously reported, FAA’s designees perform more than 90 percent of FAA’s certification activities. We have reported that designees generally conduct routine certification functions, such as approvals of aircraft technologies that the agency and designees already have experience with, allowing FAA staff to focus on new and complex aircraft designs or design changes. Panelists ranked the expanded use of designees second and fifth, respectively, among actions that we summarized from their discussions that would have the most positive impact on improving Aircraft Certification’s and Flight Standards’ certification and approval processes. FAA is increasing organizational delegations under its organization designation authorization (ODA) program and expects the ODA program will allow more effective use of its resources over time.

Stakeholders pointed to a lack of effective communication mechanisms as another problem with the certification and approval processes, especially deficiencies in the guidance FAA issues and a lack of additional communication mechanisms for sharing information on the interpretation of regulations. Stakeholders said that the lack of effective communication mechanisms can lead to costly delays when, for example, methods or guidance for complying with regulations is not clear. Stakeholders and experts had several issues with the FAA guidance that interprets the regulations and provides supplemental information to the industry. Stakeholders said there are sometimes discrepancies between the guidance and the regulations. For example, one stakeholder reported informing an FAA training course instructor that a particular piece of guidance contradicted the regulations. The instructor agreed that the contradiction existed but told the stakeholder that FAA teaches to the guidance, not the regulations. One employee group representing some FAA inspectors was concerned that not all guidance has been included in an online system that FAA has established to consolidate regulations, policy, and guidance. FAA acknowledged that it is working to further standardize and simplify the online guidance in the Flight Standards information management system.

Stakeholders also identified a lack of opportunities for sharing information about the interpretation of regulations and guidance. An industry expert noted that FAA lacks a culture that fosters communication and discussion among peer groups. Moreover, an industry group with over 300 aviation company members suggested that FAA should support and promote more agencywide and industrywide information sharing in less formal, less structured ways to enhance communication. Finally, according to an official of an employee group representing some FAA inspectors, because their workloads tend to be heavy, inspectors are less able to communicate with the companies they oversee, and the reduced level of communication contributes to variation in the interpretation of regulations. FAA officials disagreed with these assertions and indicated that FAA staff participate in numerous committees and conferences, share methods of compliance in technical areas via forums with stakeholders, and communicate resolutions to problems in various formats, such as by placing legal decisions online.

FAA Has Taken Other Actions That Might Address Process Shortcomings but Lacks Performance Data to Assess the Actions’ Effectiveness

Other FAA actions could identify and potentially address some of the shortcomings in the agency’s certification and approval processes as follows: In 2004, FAA’s Office of Aviation Safety introduced QMS, which is intended to ensure that processes are being followed and improved and to provide a methodology to standardize processes. QMS is expected to help ensure that processes are followed by providing a means for staff to report nonconformance with FAA procedures or processes and was established as part of the office’s effort to achieve certification by the International Organization for Standardization (ISO). Any employee can submit a report and check the status of an issue that has been reported. From October 2008 to March 2009, approximately 900 reports were submitted, and 46 internal audits were completed. For example, in July 2009, an FAA staffer noted that a required paragraph on aging aircraft inspection and records review was missing from a certificate holder’s operations specifications. The issue was resolved and closed in August 2009 when the missing paragraph was issued to the certificate holder. Some FAA staff told us that QMS has helped improve the processes because it requires management action to respond to report submissions.

To provide industry stakeholders with a mechanism for appealing certification and other decisions, the Office of Aviation Safety implemented the Consistency and Standardization Initiative (CSI) in 2004. Appeals must begin at the field office level and can eventually be taken to FAA headquarters. CSI requires that FAA staff document their safety decisions and that stakeholders support their positions with specific documentation. Within Aircraft Certification and Flight Standards, CSI cases at each appeal level are expected to be processed within 30 working days. The total length of the CSI process depends on how many levels of appeal the stakeholder chooses. Aircraft Certification has had over 20 CSI cases, and Flight Standards has had over 300. Most CSI cases in Aircraft Certification involved clarification of a policy or an approved means of complying with a regulation, while most of those submitted to Flight Standards involved policy or method clarification, as well as scheduling issues, such as delays in addressing a stakeholder’s certification, approval, or other issue. The large discrepancy between the number of cases filed for the two services, according to FAA officials, may be due to the fact that Aircraft Certification decisions are the result of highly interactive, deliberative processes, which are not typical in granting approvals in Flight Standards, where an inspector might find the need to hand down a decision without prolonged discussion or deliberation. Stakeholders told us that CSI lacks agencywide buy-in and can leave stakeholders who use the program potentially open to retribution from FAA staff. However, others noted that CSI is beneficial because it requires industry stakeholders to use the regulations as a basis for their complaints, which often leads to resolution. According to one of our panelists, inconsistencies occur when FAA does not start with the regulations as the basis for decisions.

Although QMS and CSI are positive steps toward identifying ways to make the certification and approval processes more efficient, FAA does not know whether the programs are achieving their stated goals because it has not established performance measures for determining program accomplishments. One of the goals for QMS is to reduce inconsistencies and increase standardization. A QMS database documents the reports submitted and, through information in these reports, FAA says it has identified instances of nonconformance and initiated corrective action to prevent recurrence; revised orders to ensure they are consistent with actual practice; and improved its processes to collect feedback from stakeholders and take action on trends. However, FAA does not know whether its actions have reduced inconsistencies because its measures describe the agency’s output—for example, number of audits conducted— rather than any outcomes related to reductions in process inconsistencies. FAA officials described CSI goals as promoting early resolution of disagreements and consistency and fairness in applying FAA regulations and policies. They provided us with data on the number of CSI cases in both Aircraft Certification and Flight Standards, the types of complaints, and the percentage of resolutions that upheld FAA’s original decision, but as with the overall QMS program, we could find no evidence that FAA has instituted CSI performance measures that would allow it to determine progress toward program outcomes, such as consistency and fairness in applying regulations and policies. Outcome-based performance measures would also allow QMS and CSI program managers to determine where to better target program resources to improve performance.

Conclusions

FAA has taken actions to address variation in decisions and inefficiency in its certification and approval processes, although the agency does not have outcome-based performance measures and a continuous evaluative process to determine if these actions are having their intended effects. Because the number of certification and approval applications is likely to increase for NextGen technologies, achieving more efficiency in these processes will help FAA better manage this increased workload, as well as its current workload. In addition, while both Aircraft Certification and Flight Standards notify applicants whether resources are available to begin their projects, Flight Standards does not monitor how long applicants are wait-listed and is therefore unaware how long projects are wait-listed and unable to reallocate resources to better meet demand for certification services.

Recommendations for Executive Action

To ensure that FAA actions contribute to more consistent decisions and more efficient certification and approval processes, we recommend that the Secretary of Transportation direct the Administrator of FAA to take the following two actions:

Determine the effectiveness of actions to improve the certification and approval processes by developing a continuous evaluative process and use it to create measurable performance goals for the actions, track performance toward those goals, and determine appropriate process changes. To the extent that this evaluation of agency actions identifies effective practices, consider instituting those practices agency wide.

Develop and implement a process in Flight Standards to track how long certification and approval submissions are wait-listed, the reasons for wait-listing them, and the factors that eventually allowed initiation of the certification process. Use the data generated from this process to assess the extent of wait-listing delays and to reallocate resources, as appropriate, to better meet demand.

Agency Comments

We provided a copy of a draft of this report to the Department of Transportation (DOT) for its review and comment. DOT provided technical comments, which we incorporated as appropriate.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 21 days from the report date. At that time, we will send copies to the appropriate congressional committees, the Secretary of Transportation, the Administrator of FAA, and other interested parties. The report also will be available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staff members have any questions or would like to discuss this work, please contact me at (202) 512-2834 or dillinghamg@gao.gov. Contact points for our offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

This report provides information on the Federal Aviation Administration’s (FAA) processes for granting certifications and approvals to air operators, air agencies such as repair stations, and designers and manufacturers of aircraft and aircraft components. It describes the processes and discusses (1) the extent of variation in FAA’s interpretation of standards with regard to the agency’s certification and approval decisions and (2) key stakeholder and expert views on how well the certification and approval processes work. To address these objectives, we reviewed relevant studies, reports, and FAA documents and processes; convened a panel of aviation industry and other experts; and interviewed aviation industry members, an expert, and FAA officials. We did not address FAA processes for issuing certifications to individuals, such as pilots and mechanics.

Expert Panel

We contracted with the National Academy of Sciences (the Academy) to convene a panel on FAA’s certification and approval processes on December 16, 2009. The panel was selected with the goal of obtaining a balance of perspectives and included FAA senior managers; officials representing large and small air carriers, aircraft and aerospace product manufacturers, aviation services firms, repair stations, geospatial firms, and aviation consultants; and academics specializing in aviation and organization theory. (See table 1.)

In the first session, FAA and industry officials presented their organizations’ perspectives on these processes and responded to questions. The presenters then departed and did not participate in the remaining sessions. In the next three discussion sessions, the panelists— led by a moderator—shared their views on various aspects of FAA’s certification and approval processes. After the first two discussion sessions, panelists voted in response to questions posed by GAO. (See app. II for the questions and responses.) The views expressed by the panelists were their own and do not necessarily represent the views of GAO or the Academy. We shared a copy of an earlier draft of this report with all of the presenters and panelists for their review and to ensure that we correctly captured information from their discussions and, on the basis of their comments, made technical corrections to the draft as necessary.

Industry Interviews

We interviewed aviation industry certificate and approval holders, trade groups, an industry expert, officials of unions that represent FAA inspectors and engineers, and FAA staff in Aircraft Certification and Flight Standards (see table 2). The industry and trade groups were selected to provide a range of large and small companies and a variety of industry sectors (e.g., aircraft and parts manufacturers, air carriers, and repair stations). The interviews were conducted to gain an understanding of the extent of variation in FAA’s certification and approval decisions and interviewees’ views on FAA’s certification and approval processes. The FAA interviews provided an understanding of the key aspects of FAA’s certification and approval processes, information on data collection and analysis related to the processes, and current and planned process improvement efforts. In addition to using information from the individual interviews, as relevant throughout the report, we analyzed the content of the interviews to identify and quantify the key issues raised by the interviewees.

Appendix II: Summary of Responses from GAO Expert Panel

This appendix summarizes the responses the panelists provided to questions we posed at the close of their discussion sessions. The response options were based on the contents of their discussions. To develop the rankings in questions 1, 2, and 12, we asked the panelists, in a series of three questions, to vote for the option he or she believed was the first, second, and third greatest, most significant, or most positive. To rank order the items listed for these questions, we assigned three points to the option identified as greatest, most significant, or most positive; two points to the second greatest, most significant, or most positive; and one point to the third greatest, most significant, or most positive option. We then summed the weighted values for each option and ranked the options from the highest number of points to the lowest. 1. What is the greatest strength of the certification and approval processes?

2. What is the most significant problem with the certification and approval processes?

3. What leading factor has contributed to problems with the certification and approval processes?

Leading factor in process problems FAA’s prioritization system for managing certifications and approvals FAA’s rulemaking process and development of guidance (e.g., amount of time required to develop or change regulations, etc.)

Culture of FAA (e.g., stove-piping, resistance to change, etc.)

Organizational structure of FAA (e.g., decentralization, varying procedures among local offices, etc.)

4. How often do serious problems occur each year with the certification and approval processes?

5. Overall, how positive or negative do you think industry’s experience has been in obtaining certifications and approvals from Aircraft Certification and Flight Standards?

6. How would you assess the overall impact of the certification and approval processes on the safety of the national airspace system?

7. Overall, how would you characterize efforts to improve the certification and approval processes?

8. Overall, how would you characterize efforts to prioritize certifications and approvals?

9. Overall, how would you characterize efforts to improve dispute resolution through the Consistency and Standardization Initiative (CSI)?

10. Regarding efforts to improve dispute resolution through CSI, what is the key factor hindering the progress of efforts?

11. What should be done to mitigate the effects of this factor?

FAA should establish support for efforts FAA should improve data collection and analysis related to efforts Do not believe efforts are ineffective Do not know/no basis to judge This response option was not available to the panelists.

12. What action will have the most positive impact on improving the certification and approval processes?

Expand use of designees/organization designation authorizations (ODA)

Expand use of designees/organization designation authorizations (ODA)

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Gerald L. Dillingham, Ph.D., (202) 512-2834 or dillinghamg@gao.gov.

Staff Acknowledgments

In addition to the contact named above, Teresa Spisak (Assistant Director), Sharon Dyer, Bess Eisenstadt, Amy Frazier, Brandon Haller, Dave Hooper, Michael Silver, and Pamela Vines made key contributions to this report.