Aviation Rulemaking: Further Reform Is Needed to Address Long-standing Problems
Released: 2001-07-12T00:00:00

Letter

Introduction

The Federal Aviation Administration (FAA) is responsible for developing, administering, enforcing, and revising an effective, enforceable set of aviation safety regulations that enhance aviation safety and security and promote the efficient use of airspace. Generally, a regulation is an agency statement that is designed to implement, interpret, or prescribe law or policy or to describe procedural requirements. The process by which FAA and other federal agencies develop regulations is called rulemaking.

FAA’s rulemaking activities encompass all of the agency’s areas of responsibility, including air traffic control, aviation security, and commercial space transportation. FAA must address both long-standing and emerging issues in its rulemaking efforts. For example, questions about the safety of aging aircraft and the adequacy of flight duty rest requirements for airline pilots have been debated for decades. In contrast, the issues of fire safety standards for cargo compartments and the transport of oxygen generators emerged after the Valujet crash outside of Miami in May 1996.

Rulemaking can be a complex and time-consuming process, and the Congress expressed its concerns about the speed of FAA’s rulemaking in 1996, when it enacted legislation that established time frames for steps in the process. While some rules may need to be developed quickly to address safety issues or guide the use of new technologies, rules must be carefully considered before being finalized because they can have a significant impact on individuals, industries, the economy, and the environment. Figure 4 provides a case study of FAA’s efforts to address a complex, long- standing aviation safety issue by creating a rule to regulate flight duty and rest requirements for flight crew members.

Federal Rulemaking Process Involves Common Steps and Documents

Rulemaking involves three stages of agency activity. First, an agency identifies a need for rulemaking. Second, it initiates the rulemaking process, develops a proposed rule, and publishes it for public comment. After a public comment period, the agency finalizes the rule by considering the comments received and drafting and publishing the final rule. Figure 5 provides an overview of the process as it applies to FAA.

A rulemaking issue may be identified internally or externally. For example, FAA staff may find that changes in aviation technology or operations or the emergence of a safety problem warrant rulemaking. Alternatively, the public or the aviation industry may petition the agency to develop a new rule or provide an exemption from existing rules. At the beginning of fiscal year 2001, FAA was responding to 57 petitions for rulemaking and 415 petitions for exemptions while reviewing 84 recommendations by its advisory committee—the Aviation Rulemaking Advisory Committee (ARAC). In addition, the Congress, the President, or the Secretary of the Department of Transportation (DOT) may direct FAA to develop a rule, or the National Transportation Safety Board (NTSB) may issue a safety recommendation. After a rulemaking issue is identified, an agency must consider the issue in light of its resources and other rulemaking issues that may be equally compelling. Some rulemaking issues may require study and analysis before an agency’s management can decide whether to initiate the rulemaking process and devote resources to developing a proposed rule.

Once an agency has decided to initiate rulemaking, the basic process for developing and issuing regulations is spelled out in section 553 of the Administrative Procedure Act of 1946 (APA). Most federal agencies, including FAA, use notice and comment rulemaking. Once rulemaking is initiated, agencies generally must develop and publish a proposed rule or “notice of proposed rulemaking” in the Federal Register. A public comment period follows, during which interested persons have the opportunity to provide “written data, views, or arguments.” After the comment period ends, the agency finalizes the rule by reviewing the comments, revising the rule as necessary, and publishing the final rule in the Federal Register at least 30 days before it becomes effective. Most rules are later incorporated into the Code of Federal Regulations (CFR).

For the remainder of this report, we will use the term “rulemaking” to refer to the notice and comment process by which FAA’s rules are developed and codified in the CFR.

Rules vary in importance, complexity, and impact. Under Executive Order 12866, federal agencies and the Office of Management and Budget (OMB) categorize proposed and final rules in terms of their potential impact on the economy and the industry affected. Executive Order 12866 defines a regulatory action as “significant” if it has an annual impact on the economy of $100 million or more; adversely affects the economy in a material way (in terms of productivity, competition, jobs, environment, public health or safety, or state, local, or tribal governments or communities); creates a serious inconsistency or interferes with another agency’s materially changes the budgetary impact of entitlements, grants, user fees, or loan programs or the rights and obligations of recipients thereof; or raises novel legal or policy issues arising out of legal mandates, the President’s priorities, or the principles set forth in the order.

Since 1996, significant rulemaking entries have constituted about half of all of FAA’s rulemaking entries in the Unified Agenda, a semiannual report of federal regulatory activities. Figure 6 shows the total number of FAA’s rulemaking entries and the number of significant rulemaking entries listed in the October Unified Agendas from 1995 through 2000.

Significant rules often take longer to issue than nonsignificant rules. They may require extensive regulatory analyses of the potential economic, social, and environmental impacts of one or more alternatives. These analyses may take months to complete and are needed to ensure that the projected economic impact has been correctly quantified and that the costs the rule will impose on the affected industry and individuals are justified.

Significant rules typically require more levels of review than nonsignificant rules. Executive Order 12866 requires that OMB review agencies’ proposed and final significant rules before they are published in the Federal Register. Moreover, clearances for proposed and final rules may be required at the departmental level for those agencies that are part of a cabinet-level department.

To reduce this burden, the Federal Aviation Reauthorization Act of 1996 grants rulemaking authority directly to the Administrator, except that the Administrator may not issue a proposed or final rule without obtaining the Secretary’s approval if that rule is significant as defined by statute. The Wendell H. Ford Aviation Investment and Reform Act for the 21st Century narrowed the scope of rules that would be considered to be significant, setting the threshold for economic significance to $250,000,000 and eliminating inconsistency and interference with other agencies’ actions and material changes to budgetary impact of entitlements, grants, user fees, or loan programs and recipients’ rights and obligations as criteria.

Nevertheless, agencies that report to the Office of the Secretary of Transportation (OST), including FAA, have also been required by the Secretary to submit for review all rules deemed significant under the executive order as well as rules that OST has indicated are to be considered to be “significant” under supplemental guidelines. These additional criteria increase the number of rules for which agencies within DOT are expected to complete regulatory analyses. For example, FAA published a significant rule in April 2000 that limited the number of commercial air tours permitted in the Grand Canyon. While the rule was not considered a significant regulatory action under Executive Order 12866, and would not have been significant under the statute, it was considered significant under the Department’s supplemental guidelines because the rulemaking had a potentially substantial economic impact on Native American tribes. Specifically, the rule was expected to have a significantly adverse impact on the Hualapai Tribe’s economic development and self-sufficiency, since the trive relied on income from air tour operations and tourist dollars brought to the reservation by the air tours.

The additional analyses and reviews required for significant rules are incorporated into the basic process that all federal agencies use for rulemaking: developing a proposed rule, releasing the proposed rule for public comment, and developing a final rule.

Various offices within FAA conduct the required analyses and reviews of rulemaking documents, as shown in table 1. In the early stages of rulemaking, each rule is the responsibility of a program office with technical expertise in a specific area. This office develops the initial rulemaking documents, as indicated in table 1. Depending on the content of the rule, the program office may be a staff office, like the Office of Chief Counsel, that also has the additional responsibility of reviewing all significant rules. Alternatively, it may be an office with responsibility for a technical area, such as the Office of Civil Aviation Security Policy and Planning. Each of these offices has managers who can become involved in the rulemaking process by reviewing the work of its representatives on a rulemaking team. Generally, FAA’s rulemaking teams consist of representatives from the program office, the Office of Rulemaking, the Office of Aviation Policy and Plans, and the Office of the Chief Counsel. In addition to significant and nonsignificant rulemaking, the staff in these offices also work on other projects, including airworthiness directives, airspace actions, and responses to petitions and exemptions.

Prior Studies of Federal Rulemaking Have Found Similar and Persistent Problems

The ultimate goal of the federal rulemaking process is to develop and issue a quality rule in a timely and efficient manner. Time is of particular importance when safety is at stake or when the pace of technological development exceeds the pace of rulemaking. Many of the problems federal agencies face in developing and publishing rules are long-standing and similar across agencies, and they have been cited in studies and discussions of the process since at least the 1970s. For example, a Senate study in July 1977 cited deficiencies in decisionmaking, planning, and priority-setting by top management as causes of delay in federal rulemaking. In July 2000, DOT’s Office of the Inspector General (OIG) reviewed the Department’s rulemaking process and found that the Department had taken as long as 12 years to issue significant rules. The OIG attributed the lack of timeliness of the Department’s rulemaking partly to a lack of timely decisionmaking and prioritization.

Studies specifically targeting the efficiency of FAA’s rulemaking process over almost 40 years have also identified similar problems. Figure 7 provides a list of key studies on FAA’s rulemaking process.

The central findings of the most recent study of FAA’s rulemaking process, published in 1997, echoed the findings of past studies. For this report, we grouped the problems identified by the 1997 study into three areas: management involvement, administration of the rulemaking process, and human capital.

Problems With Management Involvement, Process Administration, and Human Capital Impaired FAA’s Rulemaking

In terms of management involvement, FAA’s 1997 study of its rulemaking process found that problems related to shifting priorities, the timing of management involvement, and the willingness of management to delegate authority all caused delays. Inconsistent and changing priorities among FAA offices caused false starts, delays in the process, and wasted resources. Inadequate or ill-timed involvement by FAA’s senior management hindered the agency’s ability to make timely decisions. As a result, rule drafters frequently worked without adequate direction or buy-in from policymakers, causing extensive queuing, delays, and rework. The reluctance of FAA’s rulemaking management to delegate authority caused problems in internal coordination and accountability and created extensive layers of review that delayed the rulemaking process. Rulemaking projects were also often delayed because no one was held accountable for keeping projects on schedule. The lack of coordination resulted in “finger-pointing” as to why problems remained unsolved. FAA’s 1997 study identified similar concerns with the timeliness of rulemaking efforts by FAA’s industry advisory committee. For example, the committee had too many projects, some of which were duplicative or overlapping. A lack of coordination and accountability between FAA and the committee also impaired the effectiveness of the advisory committee.

In terms of process administration, FAA’s 1997 study found that confusion concerning the roles and responsibilities of rulemaking participants at FAA created difficulties in determining who had responsibility for what actions, led to breakdowns in coordination and communication, and resulted in inadequate supervision. Multiple information systems also hampered coordination and led to inaccurate tracking records and databases, as well as to information that was hard to access (e.g., archives of decisions made). Without reliable records, FAA often could not pinpoint where problems and backlogs occurred. Moreover, even when it did identify weaknesses, it lacked systems with which to evaluate and improve the process.

In terms of human capital management, the 1997 study found that FAA had not established systems for selecting and training personnel involved in rulemaking. Rulemaking teams at FAA typically did not observe project schedules, which they regarded as unrealistically optimistic. Measures of timeliness were not consistently used to measure and evaluate the performance of rulemaking participants. FAA’s rulemaking process lacked a system for consistently tying incentives and rewards to specific measures of performance.

Concerns About the Efficiency of FAA’s Rulemaking Prompted Congressional Action and the Latest FAA Review

Responding to concerns about the efficiency of FAA’s rulemaking process and in particular the time required for departmental review by OST, the Congress enacted legislation in 1996 designed to speed FAA’s efforts to develop and publish final rules. The Federal Aviation Reauthorization Act of 1996 amended section 106 of title 49 U.S.C. to establish a 16-month time limit for FAA’s finalization of rules after the close of the public comment period and a 45-day requirement for OST’s review of FAA’s significant proposed and final rules (see ch. 2). (The act also established a 24-month time limit for finalization of rules after publication of an advanced notice of proposed rulemaking, a request for information that FAA may issue in developing a proposed rule. Because this notice is not always issued, we did not use it as a measure in our analysis.)

In response, FAA reviewed its rulemaking process, established its own suggested time frames for completing steps in the process (see ch. 3), and identified potential improvements to its process in the general areas of management involvement, process administration, and human capital management. These improvements are discussed in chapter 3.

Objectives, Scope, and Methodology

The Chairman of the Subcommittee on Aviation, House Committee on Transportation and Infrastructure, asked us to review FAA’s rulemaking process to determine whether FAA could improve the efficiency of its rulemaking process. Specifically, we addressed three main questions in our review: What are the time frames for FAA’s rulemaking, including the time FAA took to initiate the rulemaking process in response to statutory requirements and safety recommendations and, once begun, to develop and publish significant rules?

What were the effects of FAA’s 1998 reforms on its process and on its time frames for completing rulemaking?

How effective were FAA’s reform efforts in addressing the factors that affect the pace of the rulemaking process?

To determine the time frames for FAA’s rulemaking, we created a database of proposed and final rules that constituted the agency’s significant rulemaking workload from fiscal year 1995 through fiscal year 2000. We focused our analysis on 76 significant rulemaking actions identified by FAA in the semiannual editions of the Unified Agenda or identified in our search of the Federal Register. This consisted of rulemaking actions that had either been published for public comment or were initiated but had not yet been published for public comment. The initiation dates and dates of published actions for the 76 rules are provided in appendix I. These rules constituted most (about 83 percent) of FAA’s significant rule workload and were more likely to be complex and/or the subject of controversy and potential delay. Our database contained data obtained from FAA’s Integrated Rulemaking Management Information System and from our review of proposed and final rules published in the Federal Register. In creating our database, to determine the dates that rulemaking projects were initiated, we used the dates recorded in FAA’s information system. For the dates of the publication of proposed and final rules, we used the dates of publication in the Federal Register. To determine the extent to which FAA’s rulemaking met statutory time frames, we compiled information from our database of rulemaking actions and applied standards established by the Congress in 1996.

To determine the effects of FAA’s 1998 reforms on the agency’s rulemaking process, we reviewed the 1997 report on FAA’s rulemaking process and discussed the 1998 reforms with FAA staff and management from the working team that participated in the study. We discussed rulemaking reforms with rulemaking officials from several other federal regulatory agencies the Animal and Plant Health Inspection Service (APHIS), the Environmental Protection Agency (EPA), the Federal Highway Administration (FHWA), the Federal Motor Carrier Safety Administration (FMCSA), the Food and Drug Administration (FDA), the National Highway Transportation Safety Administration (NHTSA), and the Nuclear Regulatory Commission (NRC) to identify what steps they had taken to improve their rulemaking processes and to discuss their efforts to improve rulemaking. We selected these agencies because they had developed significant rules that were potentially technically complex and have an impact on public safety (e.g., regulation of nuclear power, environmental concerns, and food safety). We also compared FAA’s time frames for responding to public comment and finalizing significant rules with that of other federal regulatory agencies by collecting data from the Federal Register on the time spent processing significant rules by APHIS, EPA, FDA, and NHTSA. To determine the extent to which FAA’s rulemaking met FAA’s suggested time frames for steps in the process before and after the reforms, we compiled information from our database of rulemaking actions and applied it to the time frames suggested in FAA’s rulemaking guidance. We also reviewed the number of significant rules FAA published before and after implementing its reforms as a measure of improvement in the rulemaking process.

To determine the effectiveness of FAA’s reform efforts in addressing the factors that affect the pace of the rulemaking process management involvement, process administration, and human capital management we considered case studies of specific rules, as well as the views of rulemaking officials and other stakeholders in the rulemaking process, including representatives of NTSB, OST, and OMB . We also surveyed 134 FAA employees who had served as rulemaking team members on significant rules listed in FAA’s Unified Agendas since the beginning of fiscal year 1994. We chose these employees for our survey because these staff had recent experience and were likely to be familiar with changes in the reformed process. We mailed a survey to rulemaking staff to obtain their views on the status of the rulemaking process and the impact of rulemaking reforms. We received 109 responses (a response rate of about 81 percent). A copy of the survey instrument that summarizes the responses we received is provided in appendix II. We supplemented our survey results with semistructured interviews of rulemaking team members involved in four rulemaking projects. For our semistructured interviews, we asked a series of questions designed to elicit staff members’ views on the results of the reform efforts and suggestions for improving the process.

We conducted our work from April 2000 through March 2001 in accordance with generally accepted government auditing standards.

FAA Took Varying Times to Complete Steps of the Rulemaking Process, Meeting Legislative Requirements in One-Half or Less of Cases

The time FAA took to formally initiate rulemaking in response to a congressional mandate or a recommendation by the National Transportation Safety Board (NTSB) varied widely. Between fiscal year 1995 and fiscal year 2000, FAA initiated most rulemaking efforts in response to mandates and safety recommendations within 2 years, but some were initiated many years later. Once FAA formally initiates rulemaking, the time it takes to complete the process depends on many factors, including the complexity of the issue. FAA finalized and published in the Federal Register 29 significant rules over the 6-year period from fiscal year 1995 through fiscal year 2000. It took a median of about 2 ½ years to proceed from formal initiation through publication of the final rule, ranging from less than 1 year to almost 15 years. Twenty percent of these final rules took 10 years or more to complete. During this same time period, departmental review, one step in the process for both proposed and final rules, took a median time of about 4 months. FAA’s median pace for finalizing a rule after the close of the public comment period about 15 months was comparable to that of four other federal agencies. However, FAA met the 16-month statutory requirement for finalizing a rule after the close of the public comment period in less than half of the cases since the legislation was passed and other mandated time limits in only 2 of 7 cases.

FAA Initiated Most Rulemaking in Response to Mandates and Safety Recommendations Within 2 Years

FAA initiated most rulemaking actions in response to safety recommendations from the NTSB and mandates from the Congress within 2 years. Of the 76 significant rulemaking actions we reviewed, 32 rulemaking actions (or about 42 percent) were the subject of a congressional mandate or recommendation by the NTSB. While congressional mandates may require that FAA take rulemaking actions, NTSB’s recommendations do not. However, FAA is required to respond formally to the recommendation and specify what action is or is not being taken and why. As shown in figure 8, FAA formally initiated about 60 percent of mandated rulemaking actions and about one-third of NTSB’s recommendations within 6 months.

However, FAA sometimes took many years to respond to a mandate or recommendation. For example, figure 8 also shows that in one-fourth of the mandated cases and one-third of the recommendations we examined, FAA took more than 5 years to initiate rulemaking. Figure 9 provides a case study of a rulemaking issue with safety implications aviation child safety seats—in which more than 7 years passed between NTSB’s recommendation and FAA’s initiation of the rulemaking process. In this case, the delay occurred because of policy-related disagreements between FAA and NTSB. After receiving NTSB’s recommendation to require child safety seats on aircraft, FAA studied the issue. It issued a related technical order and rule but decided not to pursue rulemaking to require child safety seats on aircraft. In part, its decision was based on a study it presented to the Congress that concluded that if child safety seats were required on aircraft, passenger diversion to other transportation modes could cause a net increase in fatalities. FAA eventually changed its policy position and initiated rulemaking after the White House Commission on Aviation Safety recommended that FAA make child-restraint systems mandatory on aircraft.

In contrast to the lengthy period of time that sometimes occurs between NTSB’s recommendations and FAA’s initiation of rulemaking, FAA responded within 1 month to an NTSB recommendation in 1999 to require flight data recorders on Boeing 737 aircraft. Figure 10 provides a case study of this rulemaking effort.

Time FAA Took to Complete Steps in the Rulemaking Process Varied

For significant rules published during the 6-year period from fiscal year 1995 through fiscal year 2000, FAA took a median time of about 2 ½ years to proceed from the formal initiation of the rulemaking process to the publication of the final rule in the Federal Register. This time period ranged from less than 1 year to almost 15 years. Six of the 29 final rules (or 20 percent) took 10 years or more to complete. FAA took a median time of about 20 months to proceed from initiating the process to proposing the rule for public comment. It took a median time of about 15 months to finalize the rule after the close of the public comment period.

The time taken for one step of the rulemaking process that occurs in FAA’s development of both proposed and final rules departmental review and approval—has been of particular concern to the Congress. In the Federal Aviation Reauthorization Act of 1996, the Congress addressed its concern by establishing a time frame for this step. The act requires the Secretary of DOT to review proposed and final significant rules and respond to FAA, either by approving them or by returning them to FAA with comments, within 45 days after receiving them.

While FAA’s information system tracked the date of OST’s approval of some significant rules, it did not track the date of OST’s response to FAA’s transmittals of significant rules when it sent them back to FAA with comments rather than approving them. We were therefore unable to measure the extent to which the Department had met the 45-day requirement set forth in the 1996 act. FAA rulemaking officials said that they did manually track this information for individual rules and planned to incorporate this capability into the next upgrade of the information system. FAA’s information system did contain the dates that some rules were submitted by FAA to OST and the dates of OST’s final approval. We used these dates to measure the time it took for OST to approve FAA’s significant proposed and final rules from fiscal year 1997, when the legislation went into effect, through fiscal year 2000. Overall, for both proposed and final rules, the median time OST took to approve the rules (including review, comment, and FAA’s response, if any) was 4.1 months (124 days). Measuring proposed and final rules separately, we found that the median time OST took to approve proposed rules was 4.7 months (140 days), while the median time OST took to approve final rules was 2.3 months (69 days). In chapter 4, we discuss the views of departmental and FAA staff on issues that impact the time required for departmental approval.

In a more recent effort to reduce delays related to OST’s review, on April 5, 2000, the Wendell H. Ford Aviation Investment and Reform Act for the 21st Century amended title 106 of 49 U.S.C. by raising the dollar threshold required for secretarial approval and eliminating several criteria that triggered departmental review of significant rules. The Congress included this language to, among other things, streamline FAA’s rulemaking process by reducing the number of significant rules that had to be submitted for departmental review and approval. Because the legislation preempts DOT’s Order 2100.5 (which defines what rules FAA and other DOT modal administrations are to submit to OST for review, as discussed in ch. 1), FAA is required to submit to OST only those significant rules that meet the criteria defined in the act. At the time of our review, FAA and OST had not yet implemented the provisions of the act. As a result, the number of FAA’s significant rules that met the criteria for OST review had not been reduced.

Time FAA Took to Finalize Rules After the Comment Period Was Comparable to That of Four Other Agencies

Although we did not compare the time frame of FAA’s entire rulemaking process to that of other agencies, we did find that the time FAA took to finalize rules after the close of the public comment period was comparable to that of four other federal agencies. We selected four regulatory agencies—APHIS, EPA, FDA, and NHTSA and compared the time they took to finalize rules from fiscal year 1995 through fiscal year 2000. The results are presented in figure 11. The figure shows that, except for APHIS, which finalized all of its significant rules within 2 years of the close of the public comment period, agencies generally finalized between two-thirds and three-fourths of their significant rules within 24 months of the close of the public comment period.

FAA Met Legislative Requirements in Half or Less of Its Rulemaking Efforts

The Federal Aviation Reauthorization Act of 1996 established a 16-month time frame for FAA’s finalization of rules after the close of the public comment period. From October 1996 through March 2001, FAA met this deadline in 7 of 18 cases by either publishing a final rule in the Federal Register or taking other final action within 16 months of the close of the public comment period. Figure 12 provides a case study of FAA’s rulemaking to prohibit the transportation of discharged or unfilled oxygen generators in aircraft. This effort exceeded the congressional time frame by about 11 months. (See app. III for a complete list of rules subject to the act’s time frames.)

The Congress has also mandated time frames for steps in FAA’s rulemaking on specific issues. The agency did not meet many of these legislated time frames. Specifically, of the 20 congressionally mandated rules that were part of FAA’s workload between fiscal year 1995 and fiscal year 2000, 7 included a time frame for agency action. FAA met the time frame in only 2 cases, both of which called for initiating the rulemaking process by a certain date. Appendix IV provides additional information regarding the current status of the seven rules with congressionally mandated time frames. Figure 13 provides a case study of FAA’s proposed rule to revise procedures for aircraft registry to assist drug enforcement efforts that exceeded a specific legislative mandate by more than 10 years.

FAA’s Reforms Addressed Long-standing Rulemaking Problems but Did Not Reduce Rulemaking Times

To respond to congressional concerns about the timeliness of its rulemaking process and address long-standing problems (see ch. 1), FAA began implementing reform initiatives in January 1998 to improve the process in two of the three central areas we have identified: management involvement and process administration. FAA considered but did not implement most initiatives to improve human capital management. Other agencies have also implemented reforms to address similar types of problems. FAA’s median times to proceed from initiation of rulemaking through the release of the proposed rule for public comment and to finalize the rule after the close of the public comment period did not improve after FAA implemented its 1998 reforms. Despite FAA’s reforms, the time taken for departmental review and approval of FAA’s significant rules was not reduced. In addition, fewer rules were published while proposed and final rules remained in the rulemaking process for longer periods of time.

FAA’s 1998 Efforts to Improve Rulemaking Addressed Long- standing Problems

FAA began implementing reform initiatives in January 1998 to improve its rulemaking process in two of the three central areas we have identified: management involvement and process administration. FAA considered but did not implement most initiatives to improve human capital management.

FAA Took Steps to Improve Management Involvement in Process

“With the direct involvement of senior-level management in the rulemaking process, I anticipate a dynamic rulemaking program that more directly meets the safety and technology challenges of a rapidly evolving aviation industry.”

In particular, to address long-standing concerns about delays that occurred during departmental review and approval of its significant rules (see ch. 2), FAA included a representative from OST on its rulemaking steering committee and management council, hoping that improved coordination would reduce the time taken for OST’s review. Table 2 shows the members and duties of FAA’s steering committee and management council.

To formalize the new process and provide consistent and comprehensive guidance to rulemaking staff and management, FAA also developed a new rulemaking manual. Among other things, this manual suggested time frames for steps in the rulemaking process and established a system for the steering committee to follow in prioritizing rulemaking projects, as shown in table 3.

Finally, to maximize the efficient use of employees’ and management’s time, FAA planned to limit reviews to those that added value and to delegate more responsibility for rulemaking decisions to rulemaking teams. Prior to the reforms, both nonsignificant and significant rules went through multiple layers of internal review. This practice stemmed more from agency protocol than from necessary oversight. For example, a team member’s decision could pass through sequential reviews by his or her immediate managers, office directors, associate administrators, and the Office of the Administrator. FAA proposed eliminating intermediate manager and director-level review and approval for both nonsignificant and significant rules so that rules could pass directly from teams to associate administrators. In doing so, FAA hoped to use available resources more efficiently, improve team members’ morale, and reduce delays. However, the agency stopped short of eliminating review and approval of significant rules by associate administrators, as was recommended in studies of FAA’s rulemaking in 1988, 1996, and 1997. According to officials from the Office of Rulemaking, the revised process was designed to enable the management council to delegate coordination and approval of nonsignificant rules to managers below the associate administrator level and the reform was intended to allow teams to act with the full knowledge of their respective associate administrator’s position on important issues.

FAA Took Steps to Administer the Process More Efficiently

To address problems in administering the rulemaking process, FAA implemented a series of reforms. These reforms were primarily designed to clarify the extent and limitations of each team member’s roles and responsibilities, to improve the monitoring of rules and the management of rulemaking documents throughout the process, and to ensure ongoing evaluation of the process. Given the potential complexity of rulemaking issues, inconsistent and unclear lines of responsibility between policy, technical, legal, and economic reviews have historically slowed the rulemaking process. In its reform, FAA documented in the rulemaking manual the roles and responsibilities for each member of the rulemaking team. Specific appendixes in this manual detail the purpose, intent, and limitations of legal and economic reviews.

FAA also created a new system for monitoring rule status and document management, the Integrated Rulemaking Management Information System, which was designed to increase the use of automation in the rulemaking process. According to FAA’s Office of Rulemaking, its new system consolidated the functions of the existing rulemaking tracking and document management systems. The new system was designed to also provide access to a regulatory guidance library and the DOT’s Docket Management System.

Finally, FAA developed rulemaking quality standards and established a continuous improvement team and a quality team to ensure ongoing evaluation of the rulemaking process, monitor the quality of rulemaking documents, and provide recommendations on potential improvements to the process. FAA’s rulemaking quality standards are documented in an appendix to its rulemaking manual, Rulemaking Quality Standard and Guide. The guide offers practical tips, provides techniques, and suggests references and examples for rulemaking writers. FAA’s continuous improvement team—envisioned as a staff-level team—was established to review the evaluations from rulemaking teams in order to provide recommendations to the rulemaking management council on improvements to the process to be incorporated into the rulemaking manual. Similarly, the role of the rulemaking quality team—envisioned as a management-level team— was to continually monitor and improve the quality of rulemaking documents and provide recommendations to the rulemaking management council on improvements to the process. These two teams were consolidated in 1999 because FAA management concluded that the two functions were difficult to separate and that both functions would benefit from both staff- and management-level participation.

FAA Considered but Did Not Fully Implement Recommendations to Improve Human Capital Management

To promote accountability in the rulemaking process, the working team for the 1997 study recommended a number of human capital management strategies to improve the training, evaluation, and rewarding of rulemaking staff. The team recommended that FAA provide orientation training on the new rulemaking process to all staff involved in rulemaking efforts. It also recommended skills assessment and additional ongoing training on functional skill development, conflict resolution, facilitation and consensus-based decisionmaking, project management, and team leader training. To measure efficiency and reward performance more consistently, the team recommended that FAA establish performance measures in the areas of rule-processing times, rule quality, and rulemaking productivity, as well as systems for performance evaluation. It also recommended that FAA develop a guide to clarify to supervisors the conditions to consider when granting rewards for good performance in rulemaking and to specify possible rewards. As discussed in chapter 4, FAA considered but did not take steps to formally implement these recommendations related to performance evaluation and rewards. According to the Office of Rulemaking, the staff resources needed to develop and implement these initiatives were not available because rulemaking staff and management were fully occupied with the day-to-day management of the rulemaking process. As a result, FAA relied on existing training and rewards systems.

Other Federal Agencies Have Undertaken Reforms to Address Similar Problems

During our investigation, we also discussed rulemaking reform with rulemaking officials from several other federal regulatory agencies whose rules involved public safety to identify what steps they had taken to improve their rulemaking processes. Although an evaluation of the effectiveness of the reforms undertaken by other regulatory agencies was beyond the scope of this review, the results of our discussions with the other rulemaking officials showed that the reforms other agencies have proposed or implemented are in some cases similar to those proposed by FAA, and they generally address the same types of problems faced by FAA.

For example, officials at several agencies we talked with considered management involvement a crucial element of an efficient rulemaking process. They used a variety of ways to improve management involvement, including the use of senior management councils and rulemaking coordinators. For example, EPA told us they established a regulatory policy council of senior management, as well as regulatory coordinators across EPA to manage priorities and resources. Other approaches cited by regulatory agencies we contacted included the use of agency ombudsmen and “senior champions.” Officials at the FMCSA said a regulatory ombudsman outside of agency program offices is responsible for moving rules through the process and tracking rules against established milestones. According to FMCSA officials, the ombudsman has the authority to resolve disagreements affecting timely processing; ensure sufficient staffing to meet statutory and internal deadlines; and represent FMCSA in discussions about individual rulemakings with other organizations, including OST, OMB, and other federal agencies. Rulemaking officials at FDA told us they assign a “senior champion” from the agency’s program offices to be responsible for scheduling rulemaking actions and ensuring that timely actions are taken. FDA officials said that the senior champion concept improves accountability by establishing a single point of responsibility. To reduce layers of internal review, other federal agencies have taken steps to delegate authority by limiting the amount of sequential review that takes place. For example, FDA officials said they limit a program office’s concurrence procedures and sign-off requirements to include only necessary staff. Rulemaking officials of the APHIS said that, in April 1999, they began limiting all staff organizations’ reviews of regulatory packages to 2 weeks. Finally, senior managers at the EPA said they provide flexibility to associate and regional administrators to determine what procedures to follow on a rule-by-rule basis, allowing managers more autonomy to tailor procedures to fit different needs.

To better administer the rulemaking process, other federal agencies have developed automated tracking systems to monitor the progress of regulations under development, established evaluation systems for learning about delays in the process, and initiated appropriate actions to overcome internal delays. For example, FDA uses a tracking system to monitor the progress of all regulatory documents, which helps expedite the internal clearance process for regulations under development. FDA officials said that the tracking system has saved FDA time in processing regulations but had not estimated the amount of time saved.

In the area of human capital management, other federal agencies cited a number of initiatives for training and performance measurement and evaluation. For example, to provide regulation writers with the training necessary to adequately prepare draft regulations, officials from APHIS encourage rulemaking staff to attend available courses and conferences or advisory committee meetings on the relevant subjects. The officials also encourage staff to seek technical support in drafting regulations and said that agencies could encourage, through incentives, technical staff to provide technical assistance to regulation writers. In addition, FDA officials suggested using a mentor program for new staff or existing staff to encourage them to consult with experienced regulation writers. Other agencies have established quality standards for their rulemaking to measure the performance of their rulemaking processes. For example, EPA measures the quality of regulatory documents and holds senior managers accountable for ensuring that regulatory actions meet the definition of a quality action. When program offices at EPA are unable to demonstrate that they can develop quality actions, fewer rulemaking actions will be assigned to them. According to EPA regulatory officials, this is an incentive for senior managers to develop quality rules. At FMCSA, officials said they had a formal structure of accountability of rulemaking products and dates in performance agreements that involve the head of the agency down to division directors. These performance agreements have specific rulemakings that include dates for which the staff is held accountable. In addition, FMCSA has a supplemental statement to the performance agreement for every staff member for rulemaking work products and dates.

Despite Reforms, FAA’s Rulemaking Times Did Not Improve

The median time FAA took to proceed from formal initiation of rulemaking through publication of the final rule increased from about 30 months in the 3-year period prior to the reform (fiscal years 1995 to 1997) to 38 months in the 3-year period following the reform (fiscal years 1998 to 2000). FAA’s median times for proceeding from initiation through release of the proposed rule for public comment and for proceeding from the close of the public comment period through publication of the final rule both increased by more than 3 months after the reforms. Specifically, the median time FAA took to proceed from initiation through the release of the proposed rule for public comment increased from 16.5 months in the 3-year period prior to the reforms to 20.4 months in the 3-year period following the reforms. The median time FAA took to finalize the rule after the close of the public comment period increased from 14 months to 16.3 months during the same time periods, as shown in figure 14.

Rulemaking Reforms Did Not Reduce the Time Taken for Departmental Review and Approval of FAA’s Significant Rules

The time OST took to review and approve rules did not improve after FAA reformed its rulemaking process in 1998. Overall, for both proposed and final rules, the median time OST took to approve rules (including review, comment, and FAA’s response, if any) increased from about 125 days before FAA’s reforms to about 130 days after the reforms, an increase of about 5 days after the reforms. Measuring the proposed and final rules separately, we found that the median time taken for OST’s approval of proposed rules increased by 2 days, while the median time taken for OST’s approval of final rules decreased by 1 day, as shown in figure 15.

The Number of Significant Rules Published Declined as Proposed and Final Rules Remained in the Process Longer

Since 1998, FAA has published fewer rules. As shown in figure 14, FAA finalized 18 significant final rules in the 3-year period prior to implementing its reform. In the 3-year period following the reform, FAA finalized only 11 significant final rules. FAA rulemaking officials attributed the change in productivity of significant rules to the agency’s efforts to classify more rulemakings as nonsignificant and, thus, to decrease levels of evaluation and review within FAA, as well as to eliminate review by the Department and OMB. However, the number of nonsignificant rules the agency published from 1995 to 2000 do not reflect this. For example, FAA published almost 50 nonsignificant proposed and final rules each year in 1995 and 1996, as compared to less than 30 nonsignificant proposed and final rules each year in 1999 and 2000.

In the years since FAA’s reforms, the median time that initiated significant rulemaking projects had remained in the process without being released for public comment (the proposed rule stage) increased  by more than 4 years from the end of fiscal year 1997 to the end of the fiscal year 2000. At the same time, the median time that FAA’s unpublished significant final rulemaking projects remained in the process after going through the public comment period also increased, by about 5 months. This is shown in figure 16.

FAA’s Suggested Time Frames Often Not Met

As part of its rulemaking reform in January 1998, FAA established its own time frames for developing and publishing proposed and final rules, as shown in figure 17.

Although these time frames were established as a part of FAA’s reforms and were, thus, not an applicable standard for rulemaking efforts prior to the reforms, we compared processing times for the 3-year period preceding FAA’s reforms to processing times for the 3-year period following FAA’s reforms to measure the extent of the change. The percentage of FAA’s proposed rules that proceeded from initiation through release for public comment within FAA’s suggested time frames dropped from 47 percent prior to the reforms to 19 percent after the reforms. The percentage of rules that proceeded from the close of the public comment period to publication as a final rule within FAA’s suggested time frames dropped from 39 percent to 36 percent in the same time periods. Overall, FAA did not meet time frames suggested in its rule making guidance for more than half of its proposed and final rules published, as shown in figure 18.

Incomplete Implementation of FAA's Reforms Impaired Efforts to Address Long-standing Rulemaking Problems

Despite the reforms FAA made to its rulemaking process, many of the problems that have historically impeded the efficiency of rulemaking at FAA continued. Our survey of FAA rulemaking staff showed that less than 20 percent agreed that FAA has made the changes necessary to improve the rulemaking process. In addition, only about 20 percent of the staff surveyed agreed that the rulemaking process has become more efficient and effective in the last 2 years. (A copy of the survey is provided in app. II.) Our interviews with FAA rulemaking staff and management and our observations of specific rulemaking projects supported the staff's perception and confirmed that problems in the three central areas of management involvement, the administration of the rulemaking process (process administration), and human capital continued to slow the process.

Problems Related to Management Involvement Continued to Slow the Process

Problems related to three general areas of management involvement continued to slow the process. Multiple, shifting priorities made it difficult to allocate resources effectively and often disrupted the timing of the rulemaking process. Too often, policy issues were not resolved in a timely manner. Finally, multiple layers of review continued to contribute to delays.

Numerous and Shifting Priorities Created Staffing Resources Problems That Slowed the Rulemaking Process

An excessive number of rulemaking priorities continued to impair the efficiency of the process. The number of projects on FAA's top priority list grew from 35 in February 1998, when FAA established the priority list after implementing its reforms, to 46 in April 2000. At that time, the Associate Administrator for Regulation and Certification said it was critical to shorten that list to a more manageable number. However, the number of top rulemaking priorities continued to increase, to 49 rules by March 2001. According to the Director of the Office of Rulemaking, the maximum number of rulemaking projects that can be effectively managed is about 30 to 35 projects. Rulemaking officials cited external and internal pressures to add rules to its priority list, noting that the agency's priorities change due to external influences such as accidents, NTSB recommendations and congressional actions and mandates. Internally, they attributed the growth in the number of priority rulemaking projects in part to a lack of commitment to the reformed process of some participants and to what they described as “parochial” views of priorities that resulted in efforts to circumvent the decisions of the rulemaking steering committee. For example, officials said that some program offices circumvented the approval process for adding rulemaking projects to the top priority list by adding projects to their own short-term incentive plans, creating pressure on the steering committee to add the rules to the top priority list. Our survey of rulemaking staff showed that less than one-third (29 percent) of the staff agreed that senior managers supported the steering committee's decisions regarding priorities.

Not only were too many rules given top priority, but changes in the relative ranking of “top” priorities created problems in managing staffing resources, thereby increasing the processing time for significant rules. Eighty-three percent of the survey respondents agreed that changing priorities in the rulemaking process caused delays in the process. Team members said they were frequently pulled off of top-priority rules to work on other projects that their management considered higher priority. They noted that these disruptions created delays.

It is important to note that, while some of the causes of shifting priorities stem from the current rulemaking process and can be changed, others relate to events that FAA cannot control. For example, new safety issues may emerge whenever there is an aviation accident. In addition, rulemaking efforts in progress that are related to issues such as safety threats can be overtaken by new events that then drive the agency's priorities. While the agency can monitor the effects of outside situations, there is little it can do to control them. Figure 19 provides an illustration of the impact of events on FAA's development of a proposed rule on aviation security.

Although FAA cannot prevent unexpected events from influencing its rulemaking priorities, FAA's system for prioritizing rules established as part of its 1998 reform lacks explicit criteria to guide rulemaking management in establishing and ranking the priority of projects and assigning available resources. While FAA's rulemaking manual describes factors that must be considered in prioritizing rulemaking projects—such as the legislative time frames established by the Congress and projects initiated in response to special commissions and the NTSB—the policy does not define how these criteria should be ranked in order of importance.

Identifying and ranking the agency's top rulemaking priorities is important because FAA's “A” list includes the Administrator's priorities as well as other top priority rules sponsored by different offices. The FAA Administrator's set of rulemaking priorities constitute about half of the “A” list of projects that are actively worked on. For example, in July 2000, 21 of the agency's 45 top priority rules were on the Administrator's list. Yet FAA's policy for determining rulemaking priorities does not establish the relative importance of the different factors that rulemaking managers must consider in determining the priority of rules within the “A” list of top priority projects. Without clear criteria for determining the rules' relative ranking and consensus among all offices involved in rulemaking, rulemaking managers may have difficulty in objectively determining, for example, whether legislated time frames take precedence over the Administrator's priorities or the safety recommendations of the NTSB. Thus, a final ranking is de facto left to the steering committee, which is made up of managers whose priorities are tied to the functions of their individual offices. One result is that managers from different offices may be more likely to allocate their staff resources on an ad-hoc, short-term basis, rather than in a strategic fashion to complete the agency's highest priority rules.

During our review, the Office of Rulemaking suggested that one way of allocating staff resources to ensure that top priority rules are completed is to “dedicate” team members to work on rules until they are completed. This approach was recommended in previous reviews of FAA's rulemaking process and is used in FAA's acquisitions of air traffic control equipment. According to the Office of Rulemaking, if this approach was put into place, managers of offices involved in rulemaking activities, including the offices involved in legal, technical, and economic analyses as well as the Office of Rulemaking, would ensure that rulemaking team members worked only on the highest priority rule by dedicating their staffs to that project. FAA successfully used this approach to develop its “commuter rule” in 1995, as shown in figure 20.

The Office of Rulemaking, which also cited changing priorities as an ongoing problem, noted that a continuing lack of realism in prioritizing fostered a sense of overload on the part of rulemaking staff. Our survey of rulemaking staff showed that only about 17 percent agreed that the amount of work was reasonable, allowing team members to produce high-quality products and services. Furthermore, less than one-third (about 30 percent) agreed that management from their office provided sufficient staff and resources to support and promote improvement in the rulemaking process.

Management Difficulties in Resolving Complex Policy Issues Continued to Slow the Rulemaking Process

On some rulemaking projects, the time FAA management took to resolve complex policy issues added years to the overall time taken to complete the rule. Based on findings that sequential review and decisionmaking by management late in the process had previously caused problems such as extensive backlogs, rework, and delays, FAA intended in its reform to promote a proactive management approach in which policy decisions would be made early in the process. However, only about 28 percent of the rulemaking staff we surveyed agreed that senior management focused on the prevention of problems rather than on the correction of problems, and only 11 percent of the staff surveyed agreed that sequential processing does not impact the timeliness of the rulemaking process. Figure 21 provides a case study of a rulemaking effort related to Flight Operational Quality Assurance (FOQA) Programs in which management's inability to resolve difficult policy issues early in the process contributed significantly to the overall time the rule has been in the process. This rule has taken years to develop because of complex policy issues that at the time of our review still had not been resolved. The policy issues concern the waiving of enforcement actions for violations discovered through FOQA data voluntarily provided by airlines. Agency officials said that the reason this issue has been so difficult to resolve is that the rule could set a precedent that would affect other regulatory agencies' enforcement efforts, and it therefore has ramifications beyond the Department's efforts to improve aviation safety. As a result, they considered their rulemaking efforts to be a management success.

Multiple Levels of Management Review Continued to Slow the Process

Delays in the process caused by multiple layers of review within the agency continued despite FAA's reform efforts because the reduction in layers of review and the level of employee empowerment envisioned in FAA's reform did not materialize. In reviewing approved project records, we found that delegation of authority beyond the director's office had not been achieved in spite of FAA's plans to do so. (FAA's plans are detailed in ch. 3.) For example, in reviewing projects approved since the reform was implemented in January 1998, we found that in five of six projects not only did the directors of team members' offices review and approve team members' decisions, but so did their immediate managers and other managers. In 1997, FAA had concluded that multiple layers of review fostered a lack of accountability in the rulemaking process and that this, in turn, led to milestones that were unrealistic or not observed because final responsibility for the project was unclear.

Our survey of rulemaking team members showed that few (4 percent) agreed that layers of review did not interfere with the timely processing of rules. As noted above, only 11 percent agreed that sequential processing does not impact the time required to complete the rulemaking process. Finally, a minority of the respondents agreed they had the ability to establish realistic schedules; 36 percent of the survey respondents agreed that rulemaking teams set realistic schedules, and 19 percent of rulemaking staff agreed that rulemaking teams have sufficient control over the rulemaking process to set realistic milestones.

Senior rulemaking managers at FAA said that there was a fine line between employee empowerment and the need for adequate oversight, particularly for rules that were likely to have a significant economic or other impact on the aviation industry. They said that FAA's reform was not intended to eliminate managers from decisionmaking in the rulemaking process or give rulemaking teams total independence, noting that the primary focus of the rulemaking reform effort was to reduce the levels of review for nonsignificant rules. According to officials from the Office of Rulemaking, review and approval of certain nonsignificant rules that would harmonize certification requirements for passenger aircraft established by the Joint Aviation Authorities (FAA's European counterpart) have already been delegated below the level of associate administrators.

Our discussions with rulemaking staff revealed a variety of reasons why they strongly disagreed that rulemaking teams had enough control over the process to set realistic milestones. Staff noted that internal management decisions to change rulemaking priorities before a project was completed and external reviews by OST caused process delays and were beyond their control. Less than 2 percent of the rulemaking team members agreed that departmental reviews improved the timeliness of the rulemaking process, and less than 15 percent agreed that departmental reviews improved the quality of rulemaking. Figure 22 shows the impact of coordination with OST on FAA's time frames in its rulemaking efforts to revise regulations governing the standards for aircraft repair stations.

FAA's internal review process reflects the lack of empowerment as well. Officials from OST and the Office of Rulemaking said that the requirement for numerous layers of review reflects FAA's hierarchical management structure and that the lack of empowerment is embodied in FAA's “grid sheet” for signing off on a proposed rule. The grid sheet can involve 20 different signatures, each indicating a different layer of review. Moreover, they said that these extensive reviews can reduce accountability, noting that, because FAA requires a lot of signatures, rulemaking documents are sometimes passed through the process without FAA officials reading them. Figure 23 illustrates the multiple layers of review that occurred in reaching team concurrence for a proposed rule to require that emergency medical equipment be carried aboard certain passenger aircraft.

DOT officials said that the time needed for their review and approval of FAA's significant rules can be lengthy if FAA's position is not thoroughly evaluated in terms of departmental policy early in the rulemaking process. DOT officials also cited lack of coordination among FAA's program offices and a lack of empowerment and accountability of rulemaking teams as problems that continued to contribute to delays in the process. They said that departmental review served a valuable role in ensuring that OMB's concerns were adequately addressed and noted, for example, that OST's efforts to coordinate proposed changes were hindered when FAA staff did not have the authority to make the suggested changes.

Problems in Process Administration Continued to Impair Rulemaking

Problems in three central areas related to the administration of the rulemaking process continued to contribute to delays. Significant confusion persisted regarding the roles and responsibilities of rulemaking team members. Information systems lacked complete, accurate, or current data and were inconsistently used. Finally, key elements of a continuous improvement program to identify and correct problems in the process were not in place.

Confusion Remained Regarding the Roles and Responsibilities of Participants

Although FAA attempted to address confusion over roles and responsibilities in its reforms, our survey indicated that only about 40 percent of the individuals we surveyed agreed that the “roles and responsibilities are clearly understood.” In addition, less than half (47 percent) of the survey respondents agreed that “roles and responsibilities are clearly established.”

The effort by the Office of Rulemaking to define roles and responsibilities for rulemaking participants in its rulemaking manual did not appear to have eliminated confusion. As we indicated in chapter 3, the manual describes the specific roles of legal and economic reviewers. According to FAA's guidance, legal reviews should focus on the legal authority for the action proposed, compliance of the proposal with applicable laws, and whether the requirements being imposed are stated with sufficient clarity and justification to be enforced and defended in court, if need be. Economic reviews should estimate the costs and benefits of a proposed or final rulemaking. However, rulemaking management said that legal reviews continued, in some cases, to focus on nonlegal issues and that the scope of economic reviews could potentially be reduced. Senior legal staff involved in the rulemaking process noted that FAA's Chief Counsel is a political appointee whose role as advisor to the Administrator can result in the office's involvement in policy issues, as well as assessments of the quality of analyses conducted to support rules.

Information Systems Consistently Track Only “A” List Rules and Are Not Consistently Used

In September 2000, we reported on the importance of information technology resources for federal agencies to gather and share information, and FAA officials cited the development of a rulemaking management information system as a major element of its rulemaking reforms. According to FAA's Office of Rulemaking, its new automated system consolidated the functions of the existing project-tracking and document- management systems. FAA's tracking of its 24 “A” list significant rules on this system has established data on rulemaking times for specific steps that should help it to monitor the rulemaking process. However, the small number of rules that it consistently tracks and a lack of agencywide implementation has made the system less useful than it could potentially be.

Because FAA used the project-tracking portion of the automated system only for its “A” list of priority projects, including 24 significant rules, the system was missing complete and accurate data for many of the remainder of FAA's significant rulemaking projects. FAA rulemaking officials said that they did not have the resources available to complete, correct, or update records of rules that were not being actively worked on from the agency's “A” list of rules, citing resource limitations. However, since previously initiated rulemaking projects may be shifted onto the “A” list, historical data could be useful for measuring the performance of the rulemaking process over time.

FAA rulemaking officials also noted that FAA's rulemaking policy allows teams to select milestones on a case-by-case basis. However, continuing to consider some milestones in the system voluntary may result in a lack of consistent and comparable information on rules. Without complete, accurate, and consistent data on all FAA's rulemaking projects, FAA managers will not be able to use the information system to its fullest capacity—to measure the time elapsed between specific steps in the process to identify where and to what extent delays occur over time. Since the rulemaking process can take years to complete, a longer-term management perspective on the performance of the process is essential.

FAA agreed that additional performance and statistical measures should be incorporated into the reporting system to enhance its ability to manage the process and said it had begun making changes to the system.

The document management portion of the automated system was limited in its usefulness because it had not been fully implemented across all offices involved in rulemaking. FAA's technology plan called for an “automation champion” to lead the initiative across all of the affected offices. However, according to the Office of Rulemaking, FAA had not designated a champion or developed a plan or goals for an integrated system outside the Office of Rulemaking. As a result, offices outside of the Office of Rulemaking had not fully implemented the new system. Although all rulemaking team members received initial training on the new system, only 26 percent of the respondents to our survey agreed that rulemaking team members were provided with training when new technologies and tools were introduced. After the initial training, we found that the system was not effectively implemented outside of the Office of Rulemaking.

We reviewed the rulemaking documents in the system for four significant safety-related rules and found that since FAA's reforms in 1998, only 1 of 27 rulemaking staff outside of the Office of Rulemaking on the 4 rulemaking teams had used the automated system. This staff person used the system only twice, on the same day in February 1998. Although officials from the Office of Rulemaking said that the new system was available to all rulemaking staff, only about 23 percent of the survey respondents agreed that their coworkers used FAA's automated capabilities to record rulemaking actions. Individuals from the Office of the Chief Counsel said that they either did not have access to the automated system or that their computers were not capable of using the rulemaking software. While economists in the Office of Policy and Plans with whom we spoke had access to the system, they said that the software was too cumbersome. One economist said that he preferred to develop rulemaking documents that were inaccessible to change by other team members in order to maintain the integrity of his work product.

FAA's Implementation of Continuous Improvement Efforts Was Incomplete

Despite explicit efforts in the rulemaking reform to establish systems to evaluate the new process and establish quality standards and guidance, FAA had not fully implemented a continuous improvement or quality review program. The concept of continuous improvement is embodied in quality management principles as well as the Government Performance and Results Act. Continuous improvement efforts are essential for identifying problems in the rulemaking process. However, continuous improvement and quality management teams established in FAA's reforms reported problems in attempting to implement review systems.

In the fall of 1999, members of the continuous improvement team expressed concerns about the purpose and authority of the team related to management's participation in establishing and supporting the evaluation function. To improve the effectiveness of the system, FAA combined the teams to include both staff and management. Despite the reorganization, little substantive work had been done in the area of process improvement at the time of our review. For example, project teams are to complete a “lessons learned” evaluation after publication of a proposed rule to document practices and procedures that worked well, identify problem areas, and determine opportunities to improve the entire rulemaking process. However, since the process was reformed in January 1998 through fiscal year 1999, we found that FAA had not documented any evaluations.

The quality and continuous improvement teams were also expected to review sample rulemaking documents during the progress of a selected project, perform periodic quality assurance reviews with selected rulemaking teams, and make recommendations to the management council regarding their findings. However, none of these quality review functions had been accomplished. No evaluations or recommendations had been documented, and the rulemaking manual had not been updated since its publication in December 1998. In discussing the issue at a steering committee meeting, members of the management council attributed the lack of implementation of process improvement efforts to an inadequate level of organizational commitment to the reformed process. Rulemaking officials said that, although the continuous improvement team met on a regular basis to discuss lessons learned, the team had not documented the results of their discussions. They said they planned to incorporate the ability to document lessons learned in the next version of the management information system and that they were updating the rulemaking manual. They also said that another team, made up of managers from key offices, has met monthly and sometimes weekly to implement and improve the reformed process.

Human Capital Management Initiatives for Rulemaking Activities Were Overlooked

Human capital management initiatives focusing on training, performance measurement and evaluation, and rewards for rulemaking efficiency and quality work were generally not implemented at the time of our review. According to the National Performance Review, which made recommendations regarding federal agencies' rulemaking processes in 1993, proper training, performance measurement, and performance incentives are needed to ensure that the agency officials involved in regulatory activities work as effectively as possible. We reported on the importance of training, performance measures, and performance incentives as key elements of an effective human capital strategy in September 2000. In preparing its 1997 report, FAA's working team recommended a series of human capital management initiatives to help rulemaking participants adjust to the revised process and foster change throughout FAA. These areas included training and skills assessment as well as performance measurement, evaluation, and rewards for rulemaking participants.

Recurring Rulemaking Training Was Limited to the Office of Rulemaking Staff

Although FAA's reform plan called for orientation training on the new rulemaking process and ongoing training in a wide range of areas for all staff involved in rulemaking, rulemaking participants outside the Office of Rulemaking generally received training only on the information system software and an introduction to the new process. A formal program for continuing the training of all rulemaking team members in the areas of functional skill development, conflict resolution, facilitation of and consensus-based decisionmaking, project management, and team-leader training was not implemented. About 50 percent of the staff surveyed agreed that they received the training they needed to perform their jobs. Similarly, although FAA's reforms called for the analysis of the skills needed to function in the revised rulemaking process and to establish a mentoring program, the Office of Rulemaking had not conducted a formal analysis, and we found no evidence of such an analysis in the other offices involved in rulemaking. Only the Office of Rulemaking had established a mentoring program. Representatives from the Office of Policy and Plans and the Office of the Chief Counsel said that they had recurring training programs, but they agreed that these programs did not include a formal segment devoted to training to support the rulemaking process, as envisioned by the reform.

Lack of Rulemaking Performance Measurement, Evaluation, and Rewards Hinders Effectiveness of Reforms

As we reported in January 2000, a key element of human capital management is the use of performance management systems, including pay and other incentives, to link performance to results. However, in the area of rulemaking, FAA has not consistently done so for rulemaking staff and management. Although FAA's reform effort included recommendations to measure and evaluate team and individual team member performance and to develop an associated rewards system, these human capital management efforts were not implemented on a consistent, agencywide basis. According to rulemaking officials, the staff resources needed to develop and implement these initiatives were not available because rulemaking staff and management were fully occupied with the day-to-day management of the rulemaking process. As noted above, we found evidence that some individual senior managers' performance evaluations included rulemaking projects specific to their program areas. The Government Performance and Results Act of 1993 requires agencies to pursue performance-based management including results-oriented goal setting and performance measurement. Although the act gives agencies the impetus for tailoring their human capital systems to their specific missions and objectives, it is up to agencies, like FAA, to follow through on the opportunity. FAA implemented an agencywide effort to link performance with rewards in April 2000. FAA's new core compensation plan provides for pay increases tied to performance and individual contributions. Despite the opportunities provided by the new compensation system, as well as personnel reforms enacted in 1996 to provide FAA with greater flexibility in human capital management, FAA management has not established systems to measure and reward performance in rulemaking based on the quality or timeliness of the process.

One measure of rulemaking performance is the time taken to complete steps in the process to develop and issue a rule. To implement rulemaking reforms, senior managers involved in FAA's rulemaking agreed that process milestones were appropriate measures of rulemaking performance. However, results from our survey of rulemaking staff indicate that, while slightly more than one-half (51 percent) agreed that milestones are used to assess the overall performance of teams, team members did not believe that using milestones is an accepted or acceptable means of measuring performance. For example, less than one half of the respondents (about 48 percent) agreed that senior management holds team members accountable when teams do not meet milestones. Only 20 percent agreed that senior management is held accountable when teams do not meet milestones. Less than 20 percent agreed that rulemaking teams have sufficient control over the rulemaking process to set realistic milestones. Only 36 percent of the staff agreed that teams set realistic schedules. Only 8 percent agreed that their offices provide incentives based on the milestones of the rulemaking process. Officials in the Office of Rulemaking suggested that one method to provide agencywide incentives for timely rulemaking would be to include a goal for the agency's timely rulemaking in the short-term incentive plans for all senior managers involved in rulemaking.

The Office of Rulemaking did not develop a separate rulemaking award system as recommended by the working team. They said rulemaking awards were given based on the preexisting agency award system in which individuals and teams are recognized for outstanding performance on various projects. Although about 70 percent of the staff surveyed agreed that management from their offices provides an environment that “supports my involvement, contributions, and teamwork on the rulemaking team,” few rulemaking staff that responded to our survey agreed that teamwork is rewarded. Specifically, only 28 percent of rulemaking staff agreed with the statement “I am appropriately rewarded for teamwork in the rulemaking process (e.g., performance ratings, cash awards, certificates, or public recognition).”

Conclusions

FAA's reforms of its rulemaking process have not fully addressed the long- standing problems that can lead to unnecessary delays because the initiatives have either not been fully implemented or their implementation has been impaired by a lack of management commitment and support. Management's attention to factors critical to achieving desired results— establishing baseline data, priorities, a plan for addressing root causes, and an evaluation system to measure the agency's progress—would facilitate effective implementation of the reform initiatives begun in 1998.

FAA's management committees that were established as a part of the reform are a step in the right direction in FAA's efforts to improve management involvement, encourage timely resolution of policy issues, and reduce layers of review. Clarifying staff and management's roles in the process and including performance expectations, measures, evaluations, and rewards based on these roles is an essential step in establishing a performance system for rulemaking that emphasizes accountability and results. The system must hold staff and managers accountable for producing timely, quality rules that are needed to improve aviation safety and security. Equally essential are automated information systems to monitor the performance of the individuals and offices in the process and provide information to continually evaluate and improve rulemaking. A performance management system is a key element of an effective human capital strategy that is the best, and perhaps the only, means of obtaining the needed level of commitment and support from FAA management and staff. FAA's new Core Compensation Plan that provides for pay increases tied to performance and individual contributions offers the agency an opportunity to establish new systems for performance measurement, evaluation, and rewards based on timeliness and quality in rulemaking for all offices involved in the process. Finally, the Wendell H. Ford Aviation Investment and Reform Act for the 21st Century provides an as yet unrealized opportunity for FAA to reduce the number of rules that must go through one of the levels of review—the review by the Office of the Secretary of Transportation. Adhering to the provisions of the act could reduce the processing time for selected significant rules that meet the criteria established in the act.

Recommendations

To improve the efficiency of its rulemaking process and reap the maximum benefits from its rulemaking reform efforts, we recommend that the Secretary of Transportation direct the FAA Administrator to take steps to improve management involvement in the rulemaking process by reducing the number of top-priority projects to a manageable number over time by limiting the number of projects added until existing projects are completed and establishing criteria for ranking the highest priority rules so that the lowest ranked of these priority rules may be tabled if necessary to allow sufficient resources to be applied to emerging, higher-priority projects; providing resources sufficient for rulemaking teams to meet the agency's suggested time frames. One approach, suggested by the Office of Rulemaking, is to prototype the use of dedicated rulemaking teams by assigning staff for the duration of rulemaking projects. This approach would give the teams the ability to focus their efforts and manage projects to completion; holding managers at the director and associate administrator level accountable for making and supporting policy decisions as early as possible in the rulemaking process; and empowering team members by giving them the authority to coordinate with their associate administrators so that they can represent the associate administrator's policies, thus eliminating the need for the separate step of associate administrator's review and approval; empowering team members by permitting them to set their own schedules and deadlines; and holding staff and management accountable for ensuring that schedules are realistic.

In addition, the Secretary of Transportation should direct the FAA Administrator to take steps to improve administration of the rulemaking process by clearly communicating the roles and responsibilities of program and support staff on rulemaking teams and holding team members and their managers accountable for limiting their reviews to established criteria; ensuring that information systems used for rulemaking tracking and coordination contain current, complete, and accurate data on the status of all significant rulemaking projects, including the time elapsed between FAA's transmission of rules to OST and the receipt of OST's comments or approval; and implementing elements of its proposed continuous improvement program and using the resulting information to identify problems in the process and potential solutions.

Finally, the Secretary of Transportation should direct the FAA Administrator to take steps to improve human capital management of the rulemaking process by establishing a human capital management strategy for offices involved in rulemaking that includes providing training and support to all participants that promotes use of the agency's automated information system and collaborative, team- based decisionmaking skills, and assessing the skills of rulemaking staff and developing targeted training to better enable them to fulfill their rulemaking roles; and establishing and implementing performance measures based on expectations, evaluations, and incentives that promote timely, quality rules. One approach suggested by the Office of Rulemaking would be to include a goal for the agency's timely rulemaking in the short-term incentive plans for all senior managers involved in rulemaking.

In addition, we recommend that the Secretary revise departmental policies to make them consistent with the provisions of the Wendell H. Ford Aviation Investment and Reform Act for the 21st Century and reduce the number of FAA's significant rules subject to its review.

Agency Comments

We provided a draft of this report to the Office of the Secretary of the Department of Transportation and FAA for their review and comment. In following discussions, departmental and FAA officials indicated that they agreed with a number of the draft report's recommendations.

For example, they said that FAA will take steps to ensure that the rulemaking tracking system is completely accurate and up-to-date, and includes all appropriate tracking milestones. Furthermore, they agreed that FAA will use its continuous improvement program to identify potential process improvements and will hold senior management accountable for providing policy input as early as possible in the rulemaking process. These officials also indicated that some of the draft report's recommendations will require further consideration, and that a specific response to each of the report's recommendations will be provided in the Department's response to the final report.

FAA officials provided technical comments, which we incorporated into the report. The Department also provided written comments on the report, which discussed four main points about the results of the review. The full text of FAA's written comments is provided in appendix V, along with our detailed response to these comments.