Aviation Safety: Additional Oversight Planning by FAA Could Enhance Safety Risk Management
Released: 2014-06-25T00:00:00

Letter

Background

When fully implemented, SMS provides a continuous approach to managing safety risk, which FAA expects will improve aviation safety. SMS is not an additional safety program that is distinct from existing activities that accomplish an entity’s safety mission, such as quality management, quality assurance, or similar activities. Rather, SMS provides a set of decision-making processes and procedures to plan, organize, direct, and control business activities in a manner that enhances safety and ensures compliance with regulatory standards. According to FAA, the overarching goal of SMS is to improve safety by helping ensure that the outcomes of any management or system activity incorporate informed, risk-based decision making.

SMS consists of four key components: (1) safety policy, (2) safety risk management, (3) safety assurance, and (4) safety promotion. (See fig.1.) Together, these four components are intended to provide a systematic approach to achieving acceptable risk levels. FAA provides its personnel with guidance on the principles underpinning these components in its official orders and other internal FAA guidance. To the industry, FAA currently provides SMS guidance via advisory circulars, an SMS newsletter, focus groups, and a dedicated page for the SMS program office on the FAA website.

FAA is undertaking the transition to SMS in coordination with the international aviation community, working with ICAO to adopt applicable global standards for safety management. ICAO first mandated SMS worldwide for air traffic service providers in 2001. ICAO later specified that member states should mandate SMS implementation for airports, air carriers, and others by 2009. FAA began SMS implementation in 2005, but FAA officials informed ICAO that the agency and industry would not be able to meet the 2009 deadline. The United States filed a “difference” with ICAO—indicating that it does not yet completely comply with the standard—with the understanding that implementation is under way and that FAA is in the midst of a rulemaking to require SMS for commercial air carriers and certificated airports. There have been other actions within the United States to encourage SMS implementation. For instance, in 2007, NTSB recommended that FAA require all commercial air carriers to establish an SMS and in 2011 added SMS for all modes of transportation to its Most Wanted List. NTSB identified SMS as one of the most critical changes needed to reduce the number of accidents and save lives.

FAA’s implementation of SMS will affect how the agency oversees the aviation industry. Historically, FAA oversight of airlines, airports, and other regulated entities has involved oversight of such things as operations and maintenance. Once SMS requirements for industry are in place, FAA will continue this oversight, but will also apply SMS principles to its processes for oversight. Specifically, the agency will provide oversight of the safety management systems of service providers, such as commercial air carriers and certificated airports, to help ensure that they are managing safety within their operations through SMS.

Internally, FAA has directed the Air Traffic Organization, Airports Organization, Aviation Safety Organization (Aviation Safety), the Office of Commercial Space Transportation, the Next Generation Air Transportation System (NextGen) Office, and the Office of Security and Hazardous Materials Safety to implement SMS. Within Aviation Safety, the Aircraft Certification Service (Aircraft Certification), and the Flight Standards Service (Flight Standards), among others, are also implementing SMS.

Oversight of the aviation industry’s implementation of SMS will be conducted by FAA’s inspector workforce, some of whom have been involved in SMS pilot programs. The inspector workforce includes approximately 4,000 Flight Standards aviation safety inspectors around the country who oversee certificate holders, including commercial air carriers and repair stations, among others. Approximately 950 aircraft certification engineers and inspectors are responsible for overseeing firms that design and manufacture aircraft, aircraft engines, propellers, and other parts and equipment. The nation’s certificated airports are overseen by approximately 40 airport certification inspectors assigned to regional offices.

In 2009, FAA issued an Advance Notice of Proposed Rulemaking (ANPRM) to solicit comments on establishing a regulatory framework to require SMS for various sectors of the aviation industry, including commercial air carriers, repair stations, and design and manufacturing firms. FAA received public comments on the ANPRM but withdrew it following a 2010 congressional mandate for FAA to issue a rule requiring air carriers to implement SMS. FAA subsequently issued a Notice of Proposed Rulemaking (NPRM) in November 2010 that would require SMS for commercial air carriers. FAA simultaneously developed a proposed rule for certificated airports and issued an NPRM in October 2010 that would also require SMS at those facilities. The status of rulemaking efforts for repair stations and design and manufacturing firms are discussed below.

FAA’s Internal Implementation of SMS Continues, While Aviation Industry Segments Are in Various Stages of Voluntary Implementation

FAA Organizations Continue SMS Implementation

Five FAA organizations are proceeding with SMS implementation. A sixth organization, Air Traffic, completed implementation in 2010. Figure 2 shows the FAA organizations that are implementing SMS as well as industry segments included in this report that these selected organizations oversee.

The Air Traffic Organization completed SMS implementation in 2010 and now uses SMS-based processes to identify hazards, enact mitigations, and assess the extent to which the mitigations are working. Since we last reported on their progress in 2012, the other organizations—Aviation Safety, the Airports Organization, the NextGen Office, and the Office of Commercial Space Transportation—have continued SMS implementation by developing SMS guidance and establishing internal SMS procedures, and an additional organization, the Office of Security and Hazardous Materials Safety has begun SMS implementation.

Aviation Safety—including two of its components, Flight Standards and Aircraft Certification—are developing or updating SMS guidance and plans, according to FAA officials. For example, Flight Standards is developing for an SMS-based oversight system for commercial air carriers.

The Airports Organization established procedures for conducting safety risk management for certain airport actions that require FAA approval.

At the time of our last report, the NextGen Office and the Office of Commercial Space Transportation were implementing SMS in line with FAA’s strategic goal in its 2009-2013 Flight Plan to implement SMS policy in all appropriate FAA organizations. In May 2013, FAA issued a revised order that directed these two organizations to implement SMS. In addition, an FAA order directed another organization, the Office of Security and Hazardous Materials Safety, to implement SMS. That organization is updating its inspections and investigations manual to be more consistent with SMS principles and is coordinating with other FAA organizations’ SMS efforts to identify hazards in the transport of hazardous materials.

Although Not Currently Required, Some Aviation Stakeholders Are Voluntarily Implementing SMS

Voluntary implementation of SMS varies by industry segment, while FAA continues rulemaking to require SMS implementation by commercial air carriers and certificated airports. Beginning in 2007, FAA initiated various pilot projects for commercial air carriers, certificated airports, repair stations, and design and manufacturing firms, among others, to encourage voluntary SMS implementation and to guide rulemaking activities. The pilot projects varied among FAA organizations. The Airports Organization and Aircraft Certification pilot projects were conducted for fixed periods of time, while the Flight Standards pilot project, which began in 2007, is ongoing. Flight Standards officials said that the pilot project for commercial air carriers would end after the final rule is published and that they hope to transition the pilot project for the remaining certificate holders (e.g., repair stations) into a more formal program. Participant selection for the pilots also varied. FAA selected the participants for the Aircraft Certification pilot for design and manufacturing firms, while the Flight Standards and Airports Organization pilot project participants volunteered to participate. The pilot projects are more fully described in forthcoming sections of this report.

While most commercial air carriers—the first industry segment for which SMS implementation will be mandatory—are moving forward with implementation, a small proportion of certificated airports, repair stations, and design and manufacturing firms have begun SMS implementation. For example, less than 1 percent of the more than 4,700 repair stations are currently implementing SMS. These three industry segments, according to FAA officials and industry stakeholders, may be waiting for additional guidance or rules, which may not be finalized for a number of years, if the time frames for finalizing the air carrier and airport SMS rules are any indication.

Commercial Air Carriers

Seventy-seven of the 83 U.S. commercial air carriers are at varying levels of SMS implementation as FAA nears completion of the final SMS rule. Flight Standards developed four SMS implementation levels to benchmark the progress of commercial air carriers (see table 1 below for additional details on the Flight Standards SMS implementation levels). In general, Flight Standards officials work with commercial air carriers to determine their SMS implementation level. Flight Standards reports the SMS level for each participating stakeholder in its monthly SMS newsletter. According to the March 2014 SMS newsletter, of the 77 commercial air carriers implementing SMS, the vast majority of them (58) were at SMS level one or two, 10 were at level three, and 9 were at the final continuous improvement level.

The Airline Safety and Federal Aviation Administration Extension Act of 2010 stipulated that FAA issue a final SMS rule for commercial air carriers. In November of 2010, FAA published an NPRM that would require all commercial air carriers to implement an SMS that meets the requirements of the new regulation and is acceptable to FAA within 3 years of the effective date of the final rule. According to FAA, it did not meet the statutory deadline of August 1, 2012, to issue the final rule due to the lengthy internal DOT review process and complexity of the required benefit-cost analysis. The DOT Significant Rulemaking Report for June 2014 indicated that FAA anticipates publishing the final rule in September 2014. (See fig. 3.) Commercial air carriers that have already begun to implement SMS must ensure that their efforts comply with the requirements of the final rule.

While FAA’s rulemaking to require SMS at all 545 certificated airports has been delayed, FAA officials estimated that 9 of the nation’s largest certificated airports were voluntarily implementing SMS as of April 2014. Between 2008 and 2012, the Airports Organization conducted three voluntary SMS pilot studies to provide airports opportunities to gain knowledge about SMS and provide feedback to FAA in its rulemaking efforts. The pilot studies also allowed certificated airports to share their SMS implementation practices with other airports. Thirty-one certificated airports of various sizes participated in at least one of the pilot studies. Twenty-one of the 22 airports in the first pilot study and 1 airport that did not participate in any of the pilot studies received a grant to develop an SMS plan. To further encourage SMS implementation, in August 2013, the Airports Organization published guidance that provides some certificated airports access to federal funds for certain SMS activities, including making SMS management software eligible for funding through the Airport Improvement Program. FAA officials indicated that another goal of the guidance was to ensure that airports would be aware of the basic elements required for SMS implementation plans and manuals.

FAA began the SMS rulemaking process for certificated airports in July 2008 and in October 2010 published an NPRM to require that all certificated airports develop and maintain an SMS that is approved by FAA. According to FAA officials, the nature of the comments received on the NPRM led them to significantly modify the proposed rule, and provide another period for public comment. As a result, in December 2012, FAA announced that it would amend the next scheduled step in the rulemaking process and issue a supplemental notice of proposed rulemaking (SNPRM), which was originally scheduled for publication in December 2013, rather than publish a final rule. FAA is considering changes in the proposed rule’s applicability and to some proposed requirements, including SMS implementation options for various sizes of certificated airports. FAA has not yet issued the SNPRM, but in June 2014 FAA estimated that the SNPRM would be published for public comment in October 2014. With FAA’s decision to issue an SNPRM, it is likely that required SMS implementation for certificated airports will not occur for some time. Figure 4 summarizes key dates related to SMS rulemaking for certificated airports, as indicated in the DOT Significant Rulemaking Report for June 2014.

Few repair stations are currently implementing SMS, and FAA has not yet determined if it will conduct rulemaking to require SMS at repair stations. As of March 2014, just 15 repair stations (of more than 4,700) were implementing SMS through the Flight Standards SMS pilot project, according to an FAA newsletter. As it does with air carriers, Flight Standards uses the four SMS implementation levels to benchmark and determine the progress of repair station SMS implementation. Thirteen were at SMS level one or two. Our interviews with 5 repair stations that participated in the pilot program suggested that there may be fewer than 14 repair stations currently implementing SMS because representatives we interviewed from 4 of the repair stations indicated that they had either ceased or never actually initiated their SMS efforts. Among the reasons these repair station representatives cited for not moving forward included a lack of FAA support. For example, representatives from one repair station told us that FAA did not respond to their questions on SMS policy changes and did not provide feedback on the gap analysis they performed. FAA officials indicated that there are reasons that more repair stations may want to implement SMS. Flight Standards officials said that once the commercial air carrier SMS rule is finalized, repair stations might want to implement their own SMS so that it could be recognized by the commercial air carriers they may contract with. A senior FAA official noted that because the commercial air carrier SMS rule will include identifying risks posed by repair stations, it would have an impact on repair station operations. Additionally, as other countries complete SMS rulemaking, some repair stations may want to implement SMS for international business reasons.

Design and Manufacturing Firms

Several design and manufacturing firms were implementing SMS as of April 2014, while an FAA rulemaking committee studies SMS incorporation into the design and manufacturing environment of approximately 3,000 firms. Aircraft Certification conducted an SMS pilot project for design and manufacturing firms from 2010 to 2012 with 11 participants. The participants were selected to represent the diversity of the industry in terms of size and types of products designed and/or manufactured. According to an FAA official, approximately 6 pilot project participants were continuing SMS implementation as of April 2014. In our discussions with several design and manufacturing firms that participated in the pilot, we found that 2—Honeywell and Boeing—were planning to complete SMS implementation in 2015. Honeywell representatives indicated that they expect to complete SMS implementation by mid-2015. Boeing representatives said their company was implementing an SMS as defined to ICAO standards and also expect to complete SMS implementation by mid-2015.

In October 2012, FAA chartered a 2-year aviation rulemaking committee (ARC) to, among other things, study SMS incorporation into the design and manufacturing firm environment. According to a January 2014 FAA plan, once the committee has completed its work, FAA will begin a rulemaking process that will result in a change to the current rule for certificating design and manufacturing firms. The plan anticipates the release of an NPRM by January 2016 and publication of a revised final rule by June 2017 to require, among other things, design and manufacturing firms to implement SMS. However, an FAA official stated in April 2014 that FAA had not yet determined when an SMS requirement might be proposed for design and manufacturing firms.

FAA and Stakeholders Cited Rulemaking, Data Issues, and Stakeholder Uncertainty as Key Challenges to Implementing SMS

FAA Cited Challenges in Completing SMS Rulemaking

FAA officials stated that a primary challenge in developing SMS rules for commercial air carriers and certificated airports is developing the benefit- cost analyses. FAA officials explained that estimating the expected benefits for SMS rules for air carriers and airports relies on analysis of the economic cost of past accidents that might not have occurred had SMS been in place. However, determining which past accidents might not have occurred based on SMS implementation either by air carriers or airports or both is complex and somewhat subjective, and this determination can be affected by other safety improvements that have occurred in recent years or that are being proposed. For example, if subsequent to a particular accident, some other safety improvements have been implemented that would have reduced its likelihood, only a portion of the value of the damage and harm related to that accident can be included in the current benefit calculation for future SMS since the likelihood of the same accident occurring has already been somewhat mitigated. Additionally, an airport industry trade organization expressed concerns in its public comments on the airport NPRM that the proposed rule incorrectly assumed that certain accidents would be mitigated by an airport SMS because those occurrences happened on airport grounds but in areas controlled by air carriers. The trade organization reviewed 53 of the 89 accidents included in the FAA analysis. In the trade organization’s view, 37 of these 53 accidents had little or limited connection to safety actions that an airport might take within its SMS. The likelihood of the accident or incident occurring could therefore be mitigated by the air carrier’s SMS implementation. Some accidents might also be avoided based on joint implementation of SMS systems by both the air carriers and airports. In such cases, expected benefits have to be apportioned across the benefit-cost analyses of the two separate SMS systems—that is, the SMS for air carriers and the SMS for airports—otherwise expected benefits would be double counted. In addition, a number of comments on the proposed rules for both the air carrier and airports questioned the estimated costs imposed on aviation stakeholders related to the implementation of the SMS systems. FAA, as part of its rulemaking process, revisited the benefit-cost analysis after reviewing public comments on the proposed rules.

Another challenge FAA faces in implementing SMS rules is planning for how data collected for proactive safety efforts will be used. Over the years, we have identified numerous challenges related to FAA data and made a number of recommendations, some of which FAA has addressed, to improve the use of data that FAA collects. One recommendation of particular significance to SMS implementation, however, has yet to be fully implemented. In 2010, we recommended that FAA develop a comprehensive plan that addresses how data fits into FAA’s implementation of a proactive approach to safety oversight. We indicated that the plan should fully describe the relevant data challenges, analytical approaches to be used, staffing requirements, and the efforts to address them. Because SMS relies on data for maximum benefit, we concluded that a comprehensive data plan by FAA would help expand its capability to improve aviation oversight. DOT concurred with this recommendation, and although FAA has taken actions such as implementing data quality controls consistent with our standards for data quality, it has not yet completed the recommended plan. For example, it has not outlined how it will analyze or use data gathered under SMS, assessed the staffing requirements it will need to conduct the data analysis, and indicated a timeline to complete a plan. GAO believes this recommendation remains valid.

Aviation Stakeholders Cited Uncertainty over FAA Oversight, Implementation Costs, Data Protection Issues, and Final Rule Requirements among Their Key Challenges

SMS implementation challenges have been identified during FAA’s SMS pilot projects and the rulemaking processes for commercial air carriers and certificated airports. We also identified some of these challenges in our prior SMS work. We selected five frequently cited challenges to discuss with aviation stakeholders: data sharing and protection, resource constraints, implementation costs, FAA oversight, and collaboration with other aviation stakeholders. Our discussions with 20 selected aviation stakeholders that participated in at least one FAA SMS pilot project confirmed that these challenges still exist for some stakeholders. Based on these discussions, we found that a majority of the selected stakeholders identified uncertainties about FAA’s oversight plans (17 stakeholders); uncertainties about SMS implementation costs (14 stakeholders); and data protection (16 stakeholders) as challenges. In addition, we found that uncertainty about the FAA’s final rule requirements raised some additional concerns with the stakeholders we interviewed, and half of the stakeholders said that of the four SMS components, the safety risk management is the most challenging to implement because of the time and analyses required to identify and mitigate risks.

Uncertainty about FAA’s SMS Oversight Plans

Seventeen stakeholders identified their uncertainty about FAA’s plans and preparation for oversight of industry’s SMS implementation as a challenge, with 12 identifying it as a great or very great challenge. Specifically, they reported a lack of information about FAA’s intended oversight plans, training for FAA inspectors responsible for oversight, and the potential for inconsistent regulatory interpretation because FAA has not shared that information with stakeholders. Just as the move to SMS will change how stakeholders manage risk, FAA’s oversight approach will also need to undergo some changes. (See table 2.) According to FAA, while its inspectors will continue to check for compliance with safety regulations and retain enforcement authority, oversight will also include reviewing the SMS guidance and records and ensuring that processes for identifying and mitigating risks are being carried out.

Internal control standards, an FAA rulemaking committee, and the ICAO Safety Management Manual all state the need to establish plans to ensure that goals and objectives are met. Additionally, we have previously found that communicating internal control efforts on a timely basis to internal and external stakeholders, such as FAA inspectors and aviation industry stakeholders, helps ensure that effective oversight can take place. A 2009 SMS aviation rulemaking committee recommended that FAA ensure that sufficient planning, policy and guidance, and workforce training are in place prior to SMS implementation to accommodate efficient, timely, objective and consistent oversight. Internal control standards also state that personnel should be provided with the right training and tools to perform their duties, and the ICAO Safety Management Manual notes the significance of training to successful SMS implementation and indicates that priority for training must be given to personnel involved in implementation or oversight of SMS. While acknowledging that its oversight role will be changing, FAA has not yet completed plans for overseeing industry’s implementation of SMS, including plans to develop guidance and train inspectors about their SMS oversight functions and responsibilities. Nine of the aviation industry stakeholders and three industry trade groups we interviewed expressed concerns that FAA inspectors will not be prepared to oversee industry implementation of SMS. These concerns related primarily to a lack of SMS training for FAA inspectors. FAA officials told us that they plan to finalize SMS rules before addressing oversight issues.

Based on the NPRM, commercial air carriers would have 3 years after the effective date of the final rule to comply with SMS implementation requirements. When the final rule for commercial air carriers becomes effective, FAA inspectors will assume oversight responsibility to ensure that commercial air carriers comply with the final rule. Included with this responsibility is approving commercial air carriers’ SMS plans. According to Flight Standards data, as of November 2013, over 2,700 of approximately 3,900 of their inspectors had completed initial training in SMS activities. This includes inspectors who oversee commercial air carriers. However, even though the publication of the final SMS rule for air carriers is expected before the end of the year—currently planned for September 2014—Flight Standards has not yet established guidance for inspectors for overseeing commercial air carriers’ implementation of SMS or updated its inspector training program to incorporate such guidance. In addition, even though a large number of inspectors have received initial SMS training, stakeholders remain concerned about inspectors’ knowledge of SMS as well as the potential for inconsistent interpretation of SMS requirements.

At the same time, the 77 commercial air carriers that have been voluntarily implementing SMS will need to review their SMS to ensure compliance with the final rule and may request FAA guidance during that process to determine whether and what modifications need to be made. Given that most commercial air carriers already have started implementing SMS, the demand for inspector guidance or assistance may come quickly. For example, inspectors might be asked to review a commercial air carrier’s SMS to ensure that all required components are included. Without taking proactive preparations for oversight before requiring SMS, FAA could find itself with oversight responsibilities for which it is not fully prepared. Consequently, the intended benefits of implementing this proactive approach to managing risk could be limited.

For other industry segments, notably airports and design and manufacturing firms, that are likely to be next to face an SMS requirement, FAA has taken initial steps to train inspectors working in those areas. In 2008, the Airports Organization provided 3 days of basic SMS instruction to airport safety inspectors during the organization’s annual recurrent training. Officials in the Airports Organization acknowledged that, should FAA finalize an SMS rule for certificated airports, up-to-date training will need to be developed and delivered to the approximately 40 airport safety inspectors, including training in detailed inspection procedures for oversight of an airport’s SMS. Likewise, Aircraft Certification plans for some of its staff to review the Flight Standards initial SMS training course to determine what type of SMS training is necessary for some of its estimated 950 aircraft certification engineers and inspectors.

In addition to training, six aviation stakeholders we interviewed expressed concerns about potentially inconsistent regulatory and other interpretations by FAA inspectors. This concern was identified by smaller stakeholders we interviewed as well as those that hold multiple certificates overseen by different FAA organizations. For example, one stakeholder we interviewed indicated that the company holds certificates for multiple facilities in multiple FAA districts, and is concerned that inspectors from different districts may have different interpretations of the SMS rules once they are implemented. We found in 2010 that ensuring consistency in regulatory interpretations has been a long-standing issue for FAA. This was also identified in the SMS ARC final report as an issue that FAA should address prior to SMS implementation. In 2012, FAA formed another aviation rulemaking committee to address consistency in regulatory interpretation by Flight Standards and Aircraft Certification. The committee made six recommendations to improve the consistency of regulatory interpretation and improve communication with industry. In July 2013, FAA provided a preliminary implementation plan to address the recommendations and indicated a detailed implementation plan with milestones would be developed, but provided no time frames for when that detailed plan would be released.

Similarly, two stakeholders that operate both domestically and overseas expressed concerns about whether FAA’s SMS regulations will be consistent with the ICAO framework that the regulations are designed to meet and harmonized with the SMS requirements of other countries. These stakeholders are concerned that potential inconsistencies could pose problems if different standards exist in different countries. For example, one repair station with overseas operations we spoke to said that any SMS they may be required to implement to meet FAA regulations must also comply with the SMS requirements in other countries where this repair station operates. The need for international acceptance of a service provider’s SMS was also identified as an issue for FAA to address in the SMS ARC’s final report in 2010.

Uncertainty about SMS Implementation Costs

Aviation stakeholders we interviewed largely had not conducted cost assessments of their SMS efforts (only 5 of 20 had done so), and the difficulty in assessing these costs could present a challenge to those considering voluntary implementation of SMS. Fourteen of the 20 stakeholders told us that identifying and assessing SMS costs was a challenge. Further complicating cost estimates, three stakeholders indicated that implementation costs are spread throughout the organization and that isolating those specifically related to SMS would be difficult. For example, one stakeholder indicated that because the computer systems it purchased to support SMS also support other operational objectives, it has not attempted to separate out SMS costs. In addition, seven stakeholders said that the absence of a final rule specifying SMS requirements made assessing costs difficult.

Though stakeholders we spoke with were generally unable to determine the cost of implementing SMS, they were able to identify some types of related expenditures. We asked each of them about specific costs related to SMS implementation, and they reported incurring the following types of costs:

14 trained employees on SMS or SMS concepts,

10 incurred recordkeeping costs,

10 purchased computer systems, and

7 hired new employees.

Other costs identified by stakeholders included those related to safety promotion and mitigating hazards identified through SMS implementation.

Concerns about Collecting and Sharing SMS Data

Industry stakeholders we spoke with had concerns about sharing and protecting their safety data; 16 of the 20 stakeholders we interviewed identified this as a challenge, with 7 identifying this challenge as great or very great. In March 2010, the SMS ARC final report identified this as an issue and recommended that prior to the promulgation of an SMS rule, protections be put in place to ensure that safety information and proprietary data are protected from disclosure and use for other purposes, such as enforcement actions.

The 2012 FAA Modernization and Reform Act (the 2012 Act) included a provision that placed a limitation on the disclosure of safety information under the federal Freedom of Information Act (FOIA) for information gathered for the purposes of developing and implementing an SMS. However, information provided by or to publicly owned airports is also subject to state FOIA laws, which are not covered under the protections in the 2012 Act. Commercial air carriers, repair stations, and design and manufacturing firms are privately owned and not directly subject to state FOIA laws. However, any data airports collect and any data shared with airports could, according to FAA officials and industry experts, be subject to state FOIA laws, because most certificated airports in the U.S. are owned by a public entity such as a state, city, port, or other local or regional government body. According to officials in the Airports Organization, a federal legislative resolution to override state laws is not a feasible option. These officials reported that in some locations, airports have begun to work with state legislative bodies to address the issue of data protection and disclosure.

Without legal protections to prevent data disclosure, stakeholders told us they feel at risk in a variety of potential scenarios related to SMS implementation.

Litigation for damage (real or perceived) caused by incidents disclosed through SMS—Eight stakeholders we spoke with, as well as the 2010 SMS ARC, indicated that protections are needed not only to prevent the disclosure of safety data through the state FOIA process, but also from disclosure as part of litigation. For example, a certificated airport must meet the regulatory requirements of 14 C.F.R. Part 139. One standard included in Part 139 is that holes in airport pavement must not exceed 3 inches in depth. An airport representative we met with indicated that if a hole less than 3 inches deep is noted through a reporting system implemented under SMS, the airport may postpone that repair until a later time as it is within the maximum depth allowed under the regulations and the risk is acceptable. Additionally, it would not likely negatively impact the operations of a large aircraft on the runway. However, it could pose a hazard for smaller general aviation aircraft. If the airport does not make that repair and the hole factors into an accident involving a general aviation aircraft, the extent of the airport’s responsibility and legal liability is uncertain and may become the subject of a lawsuit. Because the intent of SMS is to correct hazardous conditions before an accident occurs, stakeholders want protections that address their vulnerability to potential litigation.

Misinterpretation of safety data by a third party—If safety data is released to the public under a state FOIA, the aviation industry is exposed to potential misinterpretation, misuse, and further dissemination of the data by third parties. For example, one stakeholder expressed concern that safety data could be obtained, reviewed, analyzed, or disseminated by a third party without the proper context, including how and why the data were collected and the limitations of their use. Without the proper context, this stakeholder stated that dissemination would lead to misinterpretation, which may then be further disseminated by others such as policy makers or the media.

Enforcement actions by FAA or other regulatory bodies—As we discussed earlier, the safety risk management component of SMS includes systems that allow individuals to identify potential safety hazards, assess the risks arising from those hazards, and develop mitigation plans to reduce or eliminate those risks. ICAO guidance indicates that in an SMS environment, procedures should be in place to ensure information obtained under SMS will not be used for enforcement actions. However, four stakeholders expressed concern that data about potential hazards could be used beyond SMS. For example, a repair station’s SMS would address identified hazards and, if necessary, develop a mitigation plan to address those hazards. However, if an identified hazard also constitutes a regulatory violation, the company might be subject to FAA enforcement action. The SMS ARC recommended in 2010 that FAA should establish a policy or regulation that provides limits on enforcement actions applicable to information that is identified or produced by SMS. Currently, FAA foregoes civil penalty actions when violations are promptly disclosed to FAA under a voluntary disclosure program, subject to some limitations. FAA believes that the open sharing of apparent violations and a cooperative approach to solving problems will enhance and promote aviation safety. Current FAA policy covers commercial air carriers, repair stations, and design and manufacturing firms, but not certificated airports. It is unclear if FAA will extend this coverage to information provided as part of SMS efforts not currently covered.

In 2012, we also found the protection of data to be a concern and recommended that FAA consider strategies to address concerns that may adversely affect data collection and data sharing—essential to realizing the benefits of SMS. FAA began discussions with airports and airport associations in 2013 to determine what type of SMS data should be considered for protection. FAA also received and considered comments from airports and airport organizations about data collection, protection, and sharing in response to the NPRM for airports. We concluded in our September 2012 report that the success of SMS relies heavily on the sharing of safety data and that without appropriate protections, the willingness of the aviation industry to share safety data will likely be jeopardized. Although protections exist for safety data from commercial air carriers, FAA has determined that a similar legislative approach is not feasible for data from certificated airports. However, according to FAA, some airports are working to address data protections under state laws through outreach to state agencies and legislators.

Uncertainty about Final Rule Requirements

Four of the stakeholders we spoke with felt that delays in the rulemaking process were detrimental to their implementation of SMS while others were confident that their SMS implementation would meet forthcoming requirements. Representatives from two industry stakeholders we spoke with told us that they did not want to move forward in implementing SMS until rulemaking is complete, largely because they first want to know what the requirements will be. They explained that they were concerned about the time and resources required to rework the SMS should it not meet the final rule requirements.

Fourteen of the stakeholders we interviewed were concerned that the SMS requirement would not be scalable and flexible, that is, one which is sufficient to apply to a broad range of organizations from small operators to large ones with multiple facilities and certifications. Some of the concerns were based on the stakeholders’ views that FAA tends to fashion one-size-fits-all regulations or regulations that focus on one subset of a stakeholder group. For example, stakeholders cited a concern that FAA will establish regulatory requirements that are achievable for larger operators but may require more significant efforts by small operators. The NPRM for the commercial air carrier SMS requirement states that the proposed regulation is designed to be performance-based. FAA explained that because the regulation would be performance-based, it would allow commercial air carriers to comply through a variety of methods (suggesting flexibility) and to accommodate a variety of business sizes and models (suggesting scalability). The NPRM for the SMS requirement at certificated airports states that FAA envisions SMS as an adaptable and scalable system. FAA explained that an SMS could be developed by an organization to meet its unique operating environment. Accordingly, FAA stated in the NPRM that it would prescribe only the general framework for an SMS.

Several stakeholders also expressed concerns about how SMS regulations would affect their current efforts to improve safety. For example, one design and manufacturing firm we spoke with reported that it began using system safety techniques similar to those used in SMS in the 1990s. Although this stakeholder participated in an FAA pilot, the firm has developed an SMS approach in house based on the ICAO model. Fourteen stakeholders reported that prior to implementing SMS they had undertaken their own proactive safety or quality assurance efforts. FAA’s SMS ARC recommended in 2010 that SMS regulations allow for the incorporation of existing safety management efforts already in place in order to prevent duplicative safety efforts. FAA indicated that such programs could be used to build an operational SMS. Other stakeholders expressed concern that the content of the final rule may require them to adopt practices that may not improve safety in their company while imposing additional burdens. For example, one stakeholder hoped it would not be required to document every hazard identified and mitigated, stating that such documentation did not necessarily improve safety.

Safety Risk Management Challenge

We also asked all 20 industry stakeholders which of the four components of SMS would be the most challenging to implement, and more stakeholders (10) cited safety risk management than any other component. As previously discussed, safety risk management is designed to examine a company’s operational functions and operational environment to identify hazards and analyze associated risks. The intent of this process is to focus on the areas of greatest risk from a safety perspective and on mitigation, taking into account such factors as complexity and operational scope. Specifically, stakeholders indicated that conducting a risk analysis can be time consuming. A robust analysis includes identifying a wide range of risks, and determining which would require mitigation—a process that some stakeholders were concerned may keep stakeholder staff away from other duties. A number of stakeholders also noted the difficulty in identifying risks in some cases, including industry changes that may have an impact on risk.

Stakeholders Identified Additional Actions FAA Could Take to Improve SMS Effectiveness and Address Implementation Challenges

Although 12 of the 20 aviation stakeholders we interviewed are continuing with their SMS implementation, they identified actions FAA could take to improve SMS effectiveness and help address the challenges of implementation. These suggestions target SMS training, guidance, and collaboration and communication.

SMS Training for Industry

Although we did not ask specifically about any internal training programs developed by the 20 aviation stakeholders we spoke with, four stakeholders cited some difficulties with obtaining needed SMS training for their staff, including allocating time and resources for the training and finding training that was specific to their segment of the industry. Further, a 2012 study by the Transportation Research Board of FAA’s airport pilot studies found that pilot participants identified time restrictions and funding as obstacles to developing an SMS training plan for airport personnel. ICAO recommends that civil aviation authorities, like FAA, facilitate the SMS education or training of its stakeholders where feasible or appropriate. To that end, stakeholders suggested an action that FAA could take to mitigate stakeholder difficulties in this area. Specifically, two stakeholders we interviewed felt that allowing their staffs to attend the same training as FAA personnel could be beneficial in addressing differences in regulatory interpretation and for increasing industry’s understanding of FAA’s definitions and concepts as they relate to SMS. According to FAA officials, they have considered the training needs of industry stakeholders and provided some training through the pilot projects.

SMS Guidance

Although, as required by ICAO, FAA has provided, as well as updated, guidance to help stakeholders to implement SMS, some stakeholders saw a need for additional guidance and updates. For example, one stakeholder noted that FAA should create one risk matrix to be used by all organizations. Two stakeholders suggested standardizing SMS definitions across FAA. Another stakeholder suggested that FAA create a single document on the SMS framework. FAA has already responded to these concerns with an April 2012 order designed to establish common language standards for some SMS terminology; the order also provides a single risk matrix for the agency. Stakeholders also suggested that FAA develop additional guidance to aid them in documenting work processes, assessing organizational hazards, and estimating SMS costs. FAA plans to disseminate additional SMS guidance for air carriers and airports when final rules for these industry segments are published. The final rule for air carriers is expected to be published in September 2014.

Collaboration and Communication

Fourteen of the 20 stakeholder representatives we interviewed were pleased with their collaboration and communication with FAA during the pilot projects; six stakeholders, however, noted that FAA could take additional actions to improve collaboration and communication in line with ICAO’s guidance for SMS. This guidance recommends establishing an appropriate communication platform to facilitate SMS implementation, particularly for SMS requirements and guidance material. FAA has used numerous methods to communicate about and encourage collaboration regarding SMS. These methods include the SMS newsletter, SMS focus groups, participation in the Safety Management International Collaboration Group, and the various SMS pilot projects. However, some stakeholders suggested that additional opportunities exist for the agency to share lessons learned. For example, one stakeholder was not aware of any FAA communications on its progress regarding the data protection and sharing issue, and three others thought that FAA needed to encourage the industry to share lessons learned through existing forums. Two stakeholders suggested using the Commercial Aviation Safety Team as an example of, and the Aviation Safety Information Analysis and Sharing system as a potential tool for, improved SMS collaboration between FAA and commercial air carriers. Two stakeholders also recommended broadening current FAA collaboration and communication efforts by opening membership in SMS focus groups and other working groups to additional industry members, such as foreign air carriers; increasing the number of conferences and meetings where information-sharing can take place; and increasing FAA attendance at these forums. The Airports Organization noted that it continually updates its SMS website, and Flight Standards indicated that it is allowing its most significant forum with the aviation industry, InfoShare, to focus more on SMS implementation and management.

Conclusions

Maintaining a high level of safety in the U.S. aviation industry is a shared responsibility among FAA, air carriers, airports, and other stakeholders. FAA continues to implement SMS internally, in accordance with ICAO requirements, and industry stakeholders such as commercial airlines, certificated airports and others have begun to voluntarily adopt SMS in advance of regulations requiring its implementation. The successful implementation of SMS, with its proactive, risk-based approach to maintaining aviation safety, could help ensure the continued safety of the U.S. aviation system. However, the lack of a comprehensive plan for using data, which we recommended in 2010, may limit the effectiveness of SMS once it is implemented. Although FAA has taken steps to implement data quality controls that meet our standards, it has not disclosed its plans for using data collected or to ensure that it has the requisite staffing to make use of the data to be collected from commercial air carriers, certificated airports, and others. We continue to believe that the success of SMS relies on data for maximum benefit and that such a data plan will strengthen FAA’s capability to improve aviation oversight. In addition, the SMS ARC recommended in March 2010 that, among other things, FAA must ensure sufficient planning, guidance, and workforce training are in place prior to SMS implementation. Without providing the necessary training and guidance to its inspector workforce, FAA may not be adequately prepared to ensure the benefits of SMS as industry sectors are required to implement it. Although FAA continues to encourage voluntary implementation of SMS, it also must be prepared to exercise its oversight functions, once final rules requiring SMS implementation are in place, to ensure that industry is developing and utilizing processes to identify, document, and mitigate safety risks. This includes providing clarification and guidance to air carriers, airports and others as they develop their own SMS’s once final rules for each industry segment are in place.

Recommendation for Executive Action

To maximize the effectiveness and potential benefits of SMS implementation, we recommend that the Secretary of Transportation direct the FAA Administrator to take the following action:

Develop a plan to provide oversight of industry implementation of SMS, a plan that includes providing guidance and training to the relevant FAA inspectors by the time final SMS rules for industry sectors (commercial air carriers, certificated airports, repair stations, design and manufacturing firms) are published.

Agency Comments

We provided DOT with a draft of this report for review and comment. DOT provided technical corrections and clarifications, which we incorporated as appropriate.

We are sending copies of this report to the appropriate congressional committees, the Secretary of Transportation, the FAA Administrator, and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or any of your staff members have any questions about this report, please contact me on (202) 512-2834 or at dillinghamg@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix II.

Appendix I: Objectives, Scope, and Methodology

Our objective was to review the implementation of safety management systems (SMS) in the U.S. aviation industry and provide an update on SMS implementation within the Federal Aviation Administration (FAA). To do so, we addressed the following questions: 1. What is the status of SMS implementation within FAA and for key segments of the aviation industry, including certificated airports, commercial air carriers, repair stations, and design and manufacturing firms? 2. What are the key challenges FAA and the aviation industry face in implementing SMS? 3. What additional actions do aviation stakeholders believe FAA could take to improve SMS implementation and potential effectiveness?

To assess the status of SMS implementation within FAA and for key segments of the aviation industry, we reviewed FAA orders and advisory circulars on SMS and reports on its SMS pilot projects for commercial air carriers, certificated airports, repair stations, and design and manufacturing firms and from the SMS rulemaking projects for certificated airports and commercial air carriers. We also reviewed FAA information on industry SMS implementation status. In addition, we interviewed FAA officials and industry trade group representatives. To update the status of FAA’s implementation of SMS, we reviewed FAA guidance, implementation plans, and other documents published or revised since our 2012 report and interviewed FAA officials and trade group representatives.

To determine the key challenges FAA and the aviation industry face in implementing SMS as well as additional actions FAA and other stakeholders may take to improve the implementation and potential effectiveness of SMS, we interviewed officials from FAA, FAA employee, and industry groups. To obtain an international perspective on the challenges and additional actions, we interviewed representatives from foreign aviation authorities, specifically the European Aviation Safety Agency, Transport Canada, and the Civil Aviation Safety Authority of Australia. In addition, we also interviewed two Canadian airlines and one Canadian airport that have implemented SMS. Table 3 lists industry and international interviewees.

To obtain the industry stakeholder perspective for each of our objectives, we collected and analyzed information through structured interviews of a total of 20 industry stakeholders—5 airlines, 5 airports, 5 repair stations, and 5 design and manufacturing firms—that participated in FAA’s SMS pilot projects. We selected the certificated airports based on their hub size, geographic location, and whether we interviewed them for our prior study. We did not include certificated airports that we interviewed from that study. We chose the commercial air carriers based on carrier type (mainline or regional; cargo or charter), the level of SMS implementation they had reached by the time of our interview, and whether we interviewed them for the prior study; two of the air carriers we selected were interviewed for our 2012 study. For the repair stations, we choose five firms based on level of SMS implementation and the aircraft category (e.g., transport, commuter, or acrobatic) to which they provide services. We chose this selection factor as a result of research completed by the Center for Aviation Research, which found that SMS compliance for repair stations could be based on aircraft category. For design and manufacturing firms, we chose five firms based on the extent of SMS implementation, recommendations from industry stakeholders, and types of products they design or manufacture, or both. We conducted all stakeholder interviews with a standardized data collection instrument to maintain consistency across the interviews. To ensure that our interview questions were clear and reliable, we conducted two pretests with knowledgeable individuals and refined the interview questions based on those results. Because these 20 stakeholders comprise a non- representative sample, the results from these interviews cannot be projected to the universe of these industry segments. Table 4 lists these selected industry stakeholders.

We conducted this performance audit from May 2013 to June 2014 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions, based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Heather MacLeod (Assistant Director), Amy Abramowitz, James Geibel, David Hooper, Christopher Jones, Delwen Jones, Brooke Leary, Josh Ormond, Pamela Vines, and Elizabeth Wood made key contributions to this report.

Related GAO Products

Aviation: Status of DOT’s Actions to Address the Future of Aviation Advisory Committee’s Recommendations. GAO-13-657. Washington, D.C.: July 25, 2013.

Aviation Safety: Status of Recommendations to Improve FAA’s Certification and Approval Processes. GAO-14-142T. Washington, D.C.: October 30, 2013.

Department of Transportation: Key Issues and Management Challenges, 2013. GAO-13-402T. Washington, D.C.: March 14, 2013.

General Aviation Safety: Additional FAA Efforts Could Help Identify and Mitigate Safety Risk. GAO-13-36. Washington, D.C.: October 4, 2012.

Aviation Safety: Additional FAA Efforts Could Enhance Safety Risk Management. GAO-12-898. Washington, D.C.: September 12, 2012.

Aviation Safety: FAA Is Taking Steps to Improve Data, but Challenges for Managing Safety Risks Remain. GAO-12-660T. Washington, D.C.: April 25, 2012.

Aviation Safety: Enhanced Oversight and Improved Availability of Risk- Based Data Could Further Improve Safety. GAO-12-24. Washington, D.C.: October 5, 2011.

Aviation Safety: Certification and Approval Processes Are Generally Viewed as Working Well, but Better Evaluative Information Needed to Improve Efficiency. GAO-11-14. Washington, D.C.: October 7, 2010.

Aviation Safety: Improved Data Quality and Analysis Capabilities Are Needed as FAA Plans a Risk-Based Approach to Safety Oversight. GAO-10-414. Washington, D.C.: May 6, 2010.