The FAA’s mission is to provide a safe and efficient national aerospace system. FAA’s key aviation functions include regulating compliance with civil aviation safety standards and air commerce, operating the national air traffic management system, and assisting in the development of airports. The achievement of FAA’s mission is dependent in large part on the skills and expertise of its workforce. FAA consists of nearly 50,000 people, organized into 5 lines of business and several staff offices. Its workforce provides aviation services including air traffic control, maintenance of air traffic control equipment, and certification of aircraft, airline operations and pilots. FAA’s human resource management office is responsible for managing agencywide implementation of personnel reform and providing policy and guidance to regional human resource management divisions that manage the implementation of personnel reform within their areas of responsibility. In September 1993, the National Performance Review concluded that federal budget, procurement, and personnel rules prevented FAA from reacting quickly to the needs of the air traffic control system for new and more efficient equipment and flexibilities for attracting and hiring staff. In May 1994, building on these concerns, Congress directed the Secretary of Transportation to undertake a study of management, regulatory, and legislative reforms that would enable FAA to provide better air traffic control services without changing FAA’s basic organizational structure. The resulting FAA report to Congress, issued in August 1995, concluded that the most effective internal reform would be to exempt FAA from most federal personnel rules and procedures. In reporting on FAA’s request for these exemptions in October 1995, we concluded that, if the Congress decided to provide FAA with new personnel authority, the agency could be used to test changes before they were applied governmentwide. At that time, we emphasized the importance of establishing goals prior to the application of the new authority, noting that an evaluation of FAA’s efforts after some experience had been obtained would be important for determining the success of the effort and its governmentwide applicability. On November 15, 1995, Congress, in making appropriations for the Department of Transportation, directed the FAA Administrator to develop and implement a new personnel management system. The law exempted FAA from most provisions of title 5 of the United States Code and other federal personnel laws. The law required that FAA’s new personnel management system address the unique demands of the agency’s workforce, and, at a minimum, provide greater flexibility in the compensation, hiring, training and location of personnel. Subsequent legislation reinstated title 5 requirements related to labor-management relations, and the Federal Aviation Reauthorization Act of 1996 placed additional requirements on FAA by requiring that any changes made to FAA’s personnel management system be negotiated with the agency’s unions. Accordingly, compensation levels became subject to negotiations with employee unions. On April 1, 1996, FAA introduced its new personnel management system. In January 2001, we designated strategic human capital management as a governmentwide high-risk area.As our January 2001 High-Risk Series and Performance and Accountability Series reports make clear, serious human capital shortfalls are eroding the ability of many agencies, and threatening the ability of others, to economically, efficiently, and effectively perform their missions. In 2002, our studies of human capital management in the federal government identified a variety of elements—critical success factors and practices for effective implementation of flexibilities—that are important for consideration of federal human capital management efforts. For example, systems to gather and analyze data, performance goals and measures, linkage between human capital management goals and program goals of the organization, and accountability are among the elements that we have identified as essential for effective strategic human capital management. Appendix III provides an overview of our March 2002 model for strategic human capital management and key practices for federal agencies’ effective use of human capital flexibilities we identified in December 2002. Many of these elements relate directly to weaknesses we have identified in our recent reviews of FAA. For example, in July 2001, we reported that a lack of performance measurement, evaluation, and rewards hindered the effectiveness of rulemaking reforms. In October 2001, we reported that the overall effectiveness of FAA’s training for air traffic controllers was uncertain and that FAA had not measured productivity gains from changes in controllers’ duties. We reported in June 2002 on FAA’s difficulties in acquiring and developing staff to meet agency needs through air traffic control workforce planning. Most recently, we reported in October 2002 on the inability of air traffic control management to determine the impact of new relocation policies because of a lack of baseline data. Once exempted from most provisions of title 5, FAA initiated a broad set of personnel changes. For the purposes of this report, we grouped them into the areas of compensation and performance management, workforce management, and labor and employee relations. Figure 2 shows some of the major initiatives in each area, as well as whether they required exemptions from title 5 personnel rules. FAA required exemption from title 5 rules in order to implement its new, broadbanded pay structure. Before obtaining that exemption, FAA paid its employees according to the General Schedule (GS) pay system mandated by title 5. In its 1995 report to Congress, FAA stated that the GS pay system—which rewarded employees for their length of service, rather than for their competencies, skills, or accomplishments—resulted in multiple levels of supervisors at the same grade level and pay range, an inability to grant pay increases until statutorily mandated time or experience requirements were satisfied, and the administrative burden of administering about 35 special GS pay rates that were exceptions to regular pay ranges. The Office of Personnel Management (OPM) echoed these concerns in an April 2002 report. OPM concluded that the GS system’s narrow pay ranges, time-based pay progression rules and across-the-board delivery of annual increases was not effective in promoting performance- based pay. Once exempted from these provisions of title 5, FAA replaced the traditional grade and step pay system with a broadbanded pay structure that provides for a wider range of pay and greater managerial flexibility to attract, retain, and reward employees. The new pay band system includes plans tailored to specific employee segments: a core compensation plan for the majority of nonunion employees and negotiated versions of the core compensation pay plan for employees represented by unions; a unique pay plan for air traffic controllers and air traffic managers; and an executive pay plan for nonpolitical executives, managers, and some senior professionals. To illustrate the pay band system, under core compensation, the GS 15- grade pay schedule and step pay increases were replaced with a system in which employees are placed in a pay band under nine job categories including a specialized category that comprises eight specialized occupations. Each career category contains two to five pay bands. Each pay band represents a minimum and a maximum range of pay. For example, the base pay for a band “D” clerical support employee is at least $23,600 but no more than $35,400. Figure 3 shows the distribution of pay bands for career level job categories under core compensation. (For a more detailed comparison of the GS system and core compensation plan, see app. IV.) In its 1995 report to Congress, FAA reported that the federal performance management system under title 5 limited the ability of agency managers to reward their best employees. After being exempted from this system, FAA incorporated performance management elements into the new compensation system to encourage results-oriented behavior and to recognize and reward performing employees via permanent annual salary increases. For example, under its core compensation plan, all employees are eligible for a permanent pay increase, called an organizational success increase, based on the Administrator’s assessment of the extent to which the entire agency has achieved its annual agency goals. In addition, notably high-performing individuals may receive an additional permanent pay increase, called a superior contribution increase, based on supervisory recommendation. FAA has criteria for awarding superior contribution increases. These criteria include collaboration, customer service and impact on organizational success. Additional criteria may be used by some lines of business and staff offices because of their unique needs. FAA is not required to grant cost of living allowances or locality pay increases but elected to continue providing these pay adjustments, which are generally applicable to the federal pay system. FAA’s 1995 report to Congress also stated that the federal performance management system limited the ability of agency managers to deal with unacceptable performance. FAA’s legislative exemption from title 5 enabled the agency to establish its new performance management system. According to human resource management officials, this system focuses on human capital development by helping to make individual employees aware of their roles and responsibilities in helping the agency achieve its program goals and provides ongoing feedback and written evaluations to improve individual employee performance. The new performance management system incorporates a variety of feedback approaches in addition to traditional supervisor-to-employee feedback, including performance plans that discuss managers’ and employees’ agreements regarding job expectations and feedback from the employee to the supervisor. At the end of the performance evaluation cycle, employees receive a narrative performance summary instead of a year-end rating that defines employees’ performance in specific categories. The performance summary reflects an assessment of achievements based on outcomes and expectations, while professional competencies such as collaboration and customer service are elements of the new compensation system. As a result, the performance management system is not directly linked to pay for performance elements of FAA’s new compensation system. While FAA’s program documentation described union involvement and the use of employee focus groups in the development of the system, FAA did not systematically validate the final version of the performance management system with all employees before beginning implementation in 2002. Human resource officials said they planned to validate the new system by obtaining employee input through an employee attitude survey in 2003 and through continuing negotiations with employee unions and that these would allow for continuing refinements. Some of FAA’s workforce management reform initiatives required exemption from title 5 while others did not. For example, FAA’s workforce planning initiative did not require an exemption from title 5. On the other hand, changes in procedures governing hiring and locating staff, as well as some training initiatives, such as fee-for-service training programs, did require exemptions from title 5. In requesting exemption from title 5 requirements governing hiring and locating staff in 1995, FAA cited inefficiencies of working through OPM to hire and geographically place qualified staff at key facilities or to reassign employees in response to changing needs. According to an FAA staffing task force, the agency had lost highly qualified candidates because managers could not fill jobs in a timely manner. FAA estimated that it took an average of 6-8 months to bring a new hire onboard from outside the federal service using OPM as a hiring source and that it took, on average, 60 days to permanently fill a position internally. FAA also considered OPM allocations for executive positions excessively rigid, as any increases to the allocation provided had to be supported by the Department of Transportation and approved by OPM. Moreover, FAA stated that the temporary internal movement process (from one FAA location to another), also governed by OPM regulation, was equally inflexible because it limited the duration of temporary assignments, and imposed onerous processing requirements. The movement process required paperwork to be processed every 120 days and could require up to seven separate personnel actions for a 2-year temporary assignment. FAA’s 1995 request for flexibilities in the area of training was based on perceived redundancies and inefficiencies in its training programs. According to an FAA personnel reform training task force report in 1996, centralized agency training programs required by title 5 provided standard training that did not always address specific business needs. FAA also requested exemption from title 5 in order to have flexibility to provide unfunded or partially funded moves of employees to locations where they and their skills are most needed. According to FAA Air Traffic Services and human resource management officials, FAA historically interpreted title 5 rules as a requirement to fully reimburse all Permanent Change of Station (PCS) moves since the agency considered all such moves to be in the interest of the federal government. After Congress provided FAA with its new flexibilities, FAA developed a new framework for workforce planning to guide executive, occupational, and managerial/supervisory workforce planning. This did not require an exemption from title 5. With regard to hiring, FAA used its exemption from title 5 to establish hiring policies that allow FAA to hire applicants directly from outside the government and from other federal agencies without going through OPM. To do so, FAA established three hiring approaches: (1) using centralized registers, (2) announcing vacancies, and (3) authorizing on-the-spot hiring. According to FAA human resource management officials, the agency also used its exemption from title 5 to streamline staffing by decreasing the number of appointment types from 14 to 2 (temporary and permanent) and hiring authorities from approximately 500 to 1. FAA also established a flexible system for adjusting the number of executive positions in response to shifting agency priorities. This new system allows the Administrator to establish new executive positions and reassign and select the top management team. In the area of training, FAA (1) delegated responsibility for managing training funds and programs to its lines of business, (2) allowed users to select training from multiple providers, (3) created fee-for-service training programs, and (4) provided broader authority to fund degree programs for employees. The latter two initiatives required exemptions from title 5. Another area of workforce management for which FAA used its exemption from title 5 requirements was relocating employees. As part of its reform, FAA delegated the authority to determine eligibility for and amount of benefits to each line of business and provided three PCS funding options: (1) full PCS reimbursement, (2) fixed relocation payments, and (3) unfunded moves. As before reform, if the move is in the interest of the government, FAA will fully reimburse the individual for costs associated with the move. Under the new PCS rules, if FAA determines that it will derive some benefit from a move, even though the move is not in the interest of the government, the agency may offer a fixed relocation payment of up to $25,000. If a move is not in the interest of the government and FAA does not determine that it will derive some benefit from the move, there is no basis for offering PCS funding. However, as a result of FAA’s personnel reform, employees may choose to make unfunded moves at their own expense for personal reasons, to gain experience needed for professional advancement, or for promotion. FAA was ultimately not exempted from title 5 requirements governing labor-management relations. As part of its overall reform effort, it undertook several initiatives in the area of labor relations that did not require exemption from title 5. For example, FAA and its unions established a new forum—the National Labor Management Partnership Council—for union representatives and senior management to exchange information and ideas. To improve overall employee relations, FAA also established a new forum for nonunion employees to facilitate communications between employees and FAA management that also did not require an exemption from title 5. Similarly, in consultation with union and nonunion employee groups, FAA developed a new policy promoting a Model Work Environment to create and maintain an effective working environment for its employees by managing diversity and practicing equal employment opportunity and affirmative action. In addition, on July 1, 1998, FAA established an Accountability Board to standardize procedures to insure management's uniform and effective handling of sexual harassment allegations and related misconduct of a sexual nature. In July 2000, the scope of the Board was expanded to include harassment and other misconduct that creates or may create an intimidating, hostile or offensive work environment based on race, color, religion, gender, sexual orientation, national origin, age and disability. The establishment of the Board did not require exemption from title 5. FAA required exemption from title 5 to establish the Guaranteed Fair Treatment Program, an alternative dispute resolution method in which a three-person review panel adjudicates employee grievances. FAA intended the new program to be the only method by which employees not covered by a union agreement could seek administrative reviews of grievances and to replace the traditional approach under title 5 rules involving the Merit Systems Protection Board. (As discussed later, FAA was later required by Congress to reinstate the traditional title 5 process and now offers employees the choice of the two processes for resolving disputes.) While FAA has completed many of the initiatives that required changes to policy and procedures, it has not yet completed implementation of some of the more complex elements of the personnel reform it began in 1996, specifically compensation and performance management systems and workforce planning initiatives (see fig. 4). FAA officials said that the diversity of skills and duties of FAA’s workforce as well as negotiations with unions that represented a large number of employees has slowed somewhat the pace and extent of implementation of compensation and performance management initiatives. According to the Assistant Administrator for Human Resource Management, FAA’s implementation strategy was to establish a broad policy framework and then focus incrementally on individual elements of reform to eventually achieve full implementation. Between April 1996 and October 1998, for certain workforce management and labor and employee relations initiatives, FAA defined the new flexibilities available through agencywide “corporate” policies and then empowered the individual lines of business to adapt and make use of the new tools as appropriate. According to human resource management officials, these initiatives helped FAA “jump-start” its reform effort, while other reform initiatives, such as compensation, required varying incremental degrees of development because of the diverse characteristics of FAA’s workforce. Human resource management officials said other initiatives, such as workforce planning, were considered to be of a lower priority in terms of implementation. As of September 30, 2002, FAA had fully implemented its broadbanded compensation plans, including the performance incentive increases, for about three-quarters of the agency’s workforce. About 8,000 nonunion employees are paid under the core compensation plan, all senior executives (about 180) are paid under the executive compensation plan, and about another 9,000 employees represented by three of FAA’s nine unions are paid under negotiated versions of the core compensation plan. Because the performance incentive elements of the new system were not incorporated until late 2001, fiscal year 2002 will be the first year in which all employees under core compensation experience a full cycle with all the elements of its reformed compensation system fully in place. In addition, more than 19,000 air traffic controllers are paid according to a specialized, negotiated pay plan that includes pay banding and superior contribution increases. The remainder of FAA’s workforce (about 13,000), most notably those union employees whose union has not reached a new agreement with FAA, continues to be compensated under the traditional GS grade and step system under title 5 rules, as shown in figure 5. The implementation of FAA’s new performance management system has not yet been completed for most FAA employees. In 1995, prior to its reform effort and in response to new performance management regulations issued by OPM, FAA decided to establish a separate way of managing performance. At this time, it uncoupled its performance management system from its compensation system, based performance appraisals on a two-tiered evaluation (“meets expectations” or “does not meet expectations”) of employees’ performance against performance standards, provided for year-end summary ratings, and established supplemental criteria (such as making a significant contribution to the efficiency, economy, or improvement of government operations) to use as a basis for merit pay. In 1999, as part of its reform effort, FAA began development of a new performance management system. This new system consists of a narrative evaluation of employees’ performance against performance standards combined with feedback and coaching. The new performance management system does not provide for a year-end summary rating or a basis for merit pay. Instead, the new compensation system includes criteria that are separate and distinct from the performance management system (such as collaboration, customer service, and impact on organizational success) for awarding merit-based pay raises, which are called superior contribution increases. FAA implemented this new performance management system on October 1, 2001, for the Office of Human Resource Management, the Office of Regions and Center Operations, and the Regulation and Certification line of business. Since October 2001, additional staff in a variety of FAA organizations have been placed under the new performance management system, bringing the total under the system to about 20 percent of FAA’s total workforce. As with the compensation system, the new performance management system must be included in the negotiated agreements with FAA’s employee unions. FAA implemented most workforce management initiatives in 1996 by defining the flexibilities available through agencywide “corporate” policies and empowering the individual lines of business to adapt and make use of the new tools as appropriate for their staff. The individual lines of business adapted agencywide policies detailing the flexibilities available for hiring, training, and relocating employees by issuing parallel policies to guide their respective workforces and address any applications unique to their staff. While FAA established similar agencywide corporate policies and guidance for developing workforce plans for three staff levels— executive, managerial and supervisory, and occupational—this initiative is still under way. FAA began its executive workforce planning in November 2000. Development of Individual Development Plans for executives—the final element of the executive workforce planning effort— was originally scheduled to be finalized in August 2001 but was still under way at the time of our review. FAA has not yet initiated its managerial and supervisory workforce planning effort. This effort is set to begin in fiscal year 2003. FAA’s occupational workforce planning, which was originally scheduled to be completed in September of 2001, was still under way at the time of our review. Human resource management officials said that four of the five lines of business—Airports, Air Traffic Services, Regulation and Certification, and Research and Acquisitions—had completed their occupational workforce plans, and the remaining line of business— Commercial Space Transportation—was still developing a plan. FAA announced a series of agencywide policies governing labor and employee relations in 1996 that established the National Labor Management Partnership Council, the National Employees Forum, the Guaranteed Fair Treatment Program, and a policy promoting a Model Work Environment. FAA required less time to develop and implement these changes because comparable labor and employee representative groups were already in place prior to the reform effort and FAA had existing appeal processes and workplace improvement policies that served as a basis for the Guaranteed Fair Treatment Program and Model Work Environment. The variety of skills and areas of technical expertise represented in FAA’s workforce has affected the implementation of the agency’s new compensation plan. For example, the agency has a unique pay plan for air traffic controllers in the field based on the complexity of the facility, while FAA’s new core compensation plan is based on the duties and responsibilities of 16 different types of positions (ranging from students to pilots to physicians). The schedule for implementing changes in compensation and performance management has been dictated, in part, by the timing of negotiations with employee unions and the ability of FAA and its unions to reach agreement on the new systems. For example, because FAA’s contract with the National Air Traffic Controllers Association (NATCA), the organization representing FAA’s largest group of unionized employees, had expired, management had to negotiate a new agreement in 1998 before it had completed development of its new core compensation pay plan. While the air traffic pay plan, like the core compensation plan, is intended to include annual pay increases based on individuals’ performance, these performance-based increases have not been implemented as intended due to an unresolved dispute between NATCA and FAA management over the details of implementation. As a result, the air traffic pay plan distributed annual performance-based incentive pay equally among all union members for fiscal years 1999, 2000, and 2001, unlike the core compensation plan developed for the rest of the agency in which only higher performing individuals may receive performance-based incentive pay. At the time of our review, FAA and the air traffic controllers union had not yet determined how fiscal year 2002 and future years’ incentive pay increases would be allotted. According to human resource management officials, the new core compensation has not been negotiated for union employees that represent about 30 percent of FAA’s total workforce, and the need to negotiate the incorporation of compensation and performance management initiatives into union contracts has increased the length of time needed to negotiate some contracts. For example, before 1996, FAA and the Professional Airways System Specialists union took from 3 to 14 months to negotiate an agreement, but the negotiation time more than doubled to 29 months for the latest agreement. FAA and the National Association of Air Traffic Specialists have been attempting to negotiate a new contract since 1997, and the parties had not yet reached agreement at the time of our review. Labor relations officials attributed increases in negotiation times to the expanded scope of contract negotiations, which now includes negotiating compensation that historically was not negotiated. The performance management system has also not yet been implemented for most of the unionized segments of the agency’s workforce. According to FAA officials, 2,324 union employees in FAA’s Office of Regions and Center Operations and Office of Public Affairs, representing only about 5 percent of FAA’s total workforce represented by unions, were under the new system at the time of our review. FAA had little or no data on the effects of many of the reform initiatives. Human resource management officials cited positive effects of the reform initiatives in the areas of compensation and workforce management, while in the area of labor and employee relations, labor management officials provided a limited amount of data suggesting that labor relations had not improved. Managers and employees with whom we spoke in our interview effort generally cited less positive views on the effects of reform initiatives. FAA had not systematically collected or analyzed data to determine whether the new compensation system had achieved its objective of increasing the agency’s ability to attract and retain employees. Human resource management officials said the new compensation system had achieved this objective. They said the initiative had made the agency more competitive in hiring because FAA can now offer higher starting salaries within the wider-range of pay afforded by the pay bands. In addition, air traffic officials we spoke with said that the air traffic control pay plan has made it easier to staff hard-to-fill positions at busier air traffic facilities. They noted, however, that they did not have a definition for hard-to-fill positions and had not tracked the extent to which positions they might consider hard to fill had been filled more or less quickly since the new pay plan was instituted. In contrast, many FAA managers and employees we interviewed were critical of the new compensation system. Nearly two-thirds of those responding to our structured interview (110 of 176) disagreed or strongly disagreed that the new pay system is fair to all employees. While we did not attempt to evaluate the concerns raised during interviews, we did find some evidence that helps explain these perceptions of unfairness. For example, concerns about air traffic controller pay disparities are supported by a Department of Transportation Inspector General report. This report found that FAA’s initial implementation of the new compensation system led to inequities in pay between air traffic managers, supervisors, and specialists in field facilities, who are covered by the air traffic pay plan that FAA negotiated with NATCA in October 1998, and a much smaller group of air traffic managers and supervisors in regional and headquarters locations, who (together with other FAA managers and employees) are covered by the new core compensation plan. Because of differences between the two plans, managers and employees transferring from regional and headquarters locations to field facilities were not eligible for the same pay increases as those who were already assigned to field facilities in October 1998. To address this situation, FAA issued new guidance in July 2001 that established consistent rules for setting pay when employees move within and among the various pay systems in FAA, including movements between field positions and positions in regional offices and headquarters. Even so, perceptions of unfairness persist. According to the President of the FAA Conference Manager’s Association (FAACMA), the new guidance created the perception among some managers and employees of a financial disincentive for air traffic controllers to move from field facilities to regional offices or headquarters to gain supervisory and managerial experience. Further, the FAACMA President, as well as some controllers with whom we spoke, stated that such a move would result in a significant loss of pay—generally about $10,000 to $20,000. “Because of pay discrepancies,” one regional air traffic manager said, “we can’t get highly paid employees to move over to management positions.” Human resource management officials said that, while some field employees who move to positions in regional offices or headquarters would see a pay reduction of $10,000 or $20,000, not all such moves would result in such a pay reduction. According to our review of FAA’s July 29, 2001, guidance, an unfair disparity in pay between air traffic controllers would be created only when managers and employees were paid above or below established pay bands. At our request, FAA analyzed the salaries of its air traffic control staff and determined that 327, or fewer than 2 percent, of about 20,000 controllers (including supervisors, managers, and employees) were paid above current pay band maximums. (FAA’s analysis did not identify any staff being paid below established pay band minimums for their positions.) When we compared the distribution of 2002 base pay for all air traffic controllers in field facilities and in regional and headquarters facilities, we found that the regional and headquarters controllers are generally paid less under core compensation than the field controllers are paid under the air traffic pay plan. As shown in figure 6, the percentage of controllers paid between $100,000 and $130,000 is smaller in the regions and in headquarters than in the field. This is consistent with FAA’s goal of providing higher levels of pay to controllers in an operational environment. In addition, the percentage of controllers paid between $60,000 and $80,000 is greater in the regions and in headquarters than in the field. According to human resource management officials, the pay rates of many field employees and supervisors can be accommodated within the pay ranges of regional office and headquarters management positions, as shown above. Thus, they said that pay discrepancies should not affect the ability to entice field employees to move into management positions. However, it is understandable that some air traffic managers and controllers perceive a financial disincentive for moving from the field to a regional office or headquarters because, although the range of pay under both systems is comparable, the number of higher paid positions is greater in the field than in the regional offices or headquarters. To the extent that these perceptions persist, FAA may find it more difficult to place its most experienced air traffic managers in regional offices and headquarters. However, this disparity is consistent with FAA’s goal of basing pay on the operational environment and is explicitly stated in FAA’s July 2001 pay plan for air traffic managers and controllers. A general perception of unfairness regarding FAA’s new compensation system has led to increased unionization among FAA employees outside of the air traffic services line of business as well as within it, according to both internal and external sources. FAA human resource officials said that considerable unionization began before such systems as core compensation were implemented and that most concerns cited during unionization efforts were of uncertainty and loss of guarantees, not of unfairness. However, the introduction of the pay system corresponded with an acceleration in the increase in employees seeking union representation after FAA began its reform effort. For example, employees represented by unions (as a percentage of FAA’s total workforce) increased from 63 percent prior to the reform in 1995 to 66 percent in 1998 and to 79 percent by 2001. FAA labor relations officials and FAA spokespersons for new unions at FAA told us that a perceived inequity regarding pay was the prime reason new unions were formed. A 1999 study by the National Academy of Public Administration (NAPA) also found that real and perceived inequities in levels of pay were “major contributors to the view among a growing number of employees that you must belong to a union to get your fair share.” A more recent FAA study in 2001 likewise acknowledged that the new pay system “may be one possible explanation” for the increase in unionization. Between 1998—when FAA began testing and implementing its new pay system—and 2001, the number of employees choosing representation by unions increased nearly 20 percent (from about 32,800 to more than 38,800 employees). Figure 7 shows the number of FAA employees represented by unions from 1991 through 2001. Because FAA had not completed a full appraisal cycle for staff under its new performance management system at the time of our review, FAA had little data, and we were not able to obtain the views of managers and employees on the effects of the new system. We noted that FAA’s performance management approach does not use a multi-tiered rating system to rate performance. We have previously raised concerns that such approaches may not provide enough meaningful information and dispersion in ratings to recognize and reward top performers, help everyone attain their maximum potential, and deal with poor performers. According to human resource management officials, the compensation system provides a means of recognizing and rewarding top performers through separate assessments not directly linked to performance assessments under the performance management system. The measurable element related to performance management is the number of employees that receive superior contribution increases under FAA’s new compensation system. About 20 percent of employees are to receive the highest superior contribution increases (1.8 percent addition to base pay) and 45 percent are to receive the next highest level of superior contribution increases (0.6 percent increase in base pay). Whereas human resource management officials provided some limited data to support their views that reform initiatives had improved the agency’s flexibility in hiring and relocating employees, the managers we spoke with were less likely to see positive results. According to human resource management officials, FAA’s use of the new hiring flexibilities, though restricted by hiring freezes, has reduced external hiring times from an average of 6 months to as little as 6 weeks. They said the examples they provided for air marshal hiring were intended to provide an illustration that the policies allow positions to be filled quickly, even in the case of large recruitment efforts. However, the Department of Transportation’s Office of Inspector General, when reviewing FAA’s personnel reform in 1998, questioned FAA’s ability to support this assertion in the absence of data. (See fig. 8.) Throughout our review, we asked FAA officials from both the human resource management office and lines of business for any documentation or data to support the reduction in hiring times and they were unable to provide any such data. At the close of our review, however, human resource management officials cited some limited data resulting from the Federal Air Marshal Program. According to FAA human resource management officials, the program following the terrorists’ attacks on September 11, 2001, was one of the largest recruitment efforts ever undertaken by FAA. (FAA received and processed more than 200,000 applications.) According to FAA officials, it would not have been possible to fill the air marshal positions in the numbers and time frames required without the flexibilities available under FAA’s personnel system. They provided data reflecting a sample of approximately 1,000 candidates for the air marshal positions. Of those candidates hired, about 30 percent (140) were hired and placed within 6 weeks. In total, 70 percent (333) were hired and placed within 8 weeks. In contrast to the positive views of human resource management officials, FAA managers had less positive views on the effects of hiring reforms, while employees, who are less involved in the hiring process, had mixed views. Among the 46 managers we interviewed, only about a third (15) agreed or strongly agreed that the initiatives have improved the ability of their line of business or staff office to fill job vacancies. Furthermore, only 12 of the 46 managers believed the speed of hiring has improved. These opinions, while not necessarily representative of all FAA managers today, are similar to the views expressed by FAA managers in 1998. According to a survey FAA conducted then, 34 percent of managers responding said that FAA’s streamlined staffing procedures had made it easier to fill vacancies in their organization, and 32 percent said the speed of hiring had improved. Human resource management officials also said that new policies governing the relocation of employees had given managers more flexibility in relocating employees and employees more flexibility in making career decisions. Under these new policies, FAA may provide fixed relocation payments as well as full funding for PCS moves, and it allows unfunded moves, which were not allowed under FAA's prior policy. Figure 9 shows that the majority of moves between field offices for managers from fiscal year 1999 through 2001 (the only years and type of moves for which data were available) were unfunded. In contrast to the positive views of FAA human resource management officials, FAACMA representatives raised concerns about the impact of the new policies in the air traffic services line of business, suggesting that they might have unintended consequences, including a reduction in the number of qualified applicants, a reduction in the diversity of potential applicant pools and subsequent discrimination in filling positions, and a negative impact on employee morale if fluctuations in the annual funding for relocation payments led to disparities in the payments for comparable moves over time. Air traffic officials said they were still reviewing these concerns and planned to comment in the near future. Although FAA’s Office of Labor Relations did not have historical, agencywide data to quantify an increase in grievances, FAA labor management officials said the number of grievances filed at the national level by employees represented by unions had increased and this increase was a sign that the initiatives had not achieved the reform objective of establishing a collaborative labor-management relations environment that would minimize the traditional adversarial relationship. They said that the number of grievances filed began to increase following personnel reform changes the agency had made. For example, they noted that in 1999, the core compensation plan was implemented and grievances increased. However, human resource officials said that grievances by union employees could not have pertained to implementation of the compensation pilot because the pilot test only applied to nonunion employees, not to union employees. The Office of Labor Relations implemented a new system for tracking grievance data in October 2001 and began systematically collecting information on the sources (such as headquarters, regions, and unions) and subject (such as compensation, use of leave, and discipline) of grievances filed across the agency. While limited data suggested that FAA’s introduction of an alternative dispute resolution program for employees not represented by unions did reduce the processing times for resolving appeals, employees’ reactions to the new system suggest that many employees did not see this initiative as an improvement. FAA introduced its internal alternative dispute resolution approach—the Guaranteed Fair Treatment Program—in April 1996 in an effort to streamline the appeals process. This approach met with resistance from employees and led Congress, in 2000, to reinstate the traditional title 5 process that uses the Merit Systems Protection Board. As a result, FAA now offers employees the choice of using either the guaranteed fair treatment program or the traditional title 5 process. The only data human resource officials were able to provide on appeals dated back to fiscal year 1997. Although these data are old, they indicated that for fiscal year 1997, appeals went through the guaranteed fair treatment process more quickly (5 to 7 months) than through the Protection Board process (10 months). Even so, the Deputy Assistant Administrator for Labor Relations said that employees, who have been able to choose between the two processes, have generally not chosen to use the guaranteed fair treatment process. He said that one reason employees have not used the guaranteed fair treatment process is because its potential benefits, such as the employee’s right to help select the arbitrator, have not been effectively communicated to them. In addition, according to the Deputy Assistant Administrator, both FAA managers and union leaders have complained about having to pay the cost of the arbitrator, while employees have complained about having to pay their own legal fees for attorneys regardless of the outcome of the appeal. FAA reimburses an employee’s legal fees if the employee wins his/her appeal when using the Protection Board's process. Most FAA managers and employees we interviewed said that labor and employee relations had changed in the last 5 years. For example, 130 of the 176 managers and employees we interviewed agreed or strongly agreed that labor-management relations had changed in the last 5 years. Of those 130, 75 said that labor-management relations had declined. Similarly, 130 of the 176 managers and employees we interviewed said employee morale had changed in the last 5 years, and of those 130, 99 said that employee morale had declined. While employees’ perceptions regarding the changes in labor and employee relations cannot be linked directly to FAA’s personnel reform, some employees cited specific reform initiatives, such as compensation and the Model Work Environment established to improve employee relations, when discussing the decline of labor-management relations and morale. Union representatives for three of FAA’s nine unions said that a complaint filed against FAA by the Federal Labor Relations Authority (FLRA) in March 2001 had reduced collaboration between labor and management. FLRA charged FAA with bargaining in bad faith because it had refused to sign an agreement negotiated with the American Federation of State, County and Municipal Employees, a union that represents employees at FAA headquarters. FAA management did not sign the agreement and submitted it instead to the Office of Management and Budget (OMB) for review. OMB subsequently disapproved some portions of the contract. Following an investigation of the circumstances, FLRA directed FAA management and the union to sign and implement the contract. However, in September 2002, an administrative law judge recommended that FLRA dismiss the union's complaint, finding that FAA clearly gave notice to the union of the OMB approval condition and that the union agreed to that condition. In the area of employee relations, FAA provided us with some data that may support the views of FAA officials that the Model Work Environment has had a positive effect. A recent decline in the number of equal employment opportunity (EEO) complaints may, to an unknown extent, reflect the effects of FAA’s Model Work Environment. These complaints are concerns expressed by employees about legally prohibited discrimination on the basis of race, color, religion, sex, national origin, age, or handicap. An analysis by FAA’s Office of Civil Rights of data it had collected on the number and types of formal EEO complaints showed that while such complaints increased in the years immediately following the implementation of the Model Work Environment in 1996, they began to decline 3 years later. As figure 10 shows, the number of EEO complaints increased from 412 in 1996 to 635 in 1998 and then declined to 485 in 2001. About three-quarters of the FAA managers and employees we interviewed (134 of 176) agreed or strongly agreed that they understood the goals of the Model Work Environment. These goals include reflecting diversity and eliminating discrimination and harassment in the workplace, which are common causes of equal employment opportunity complaints. While some employees cited positive effects of the program, other employees were skeptical of its impact. Figure 11 illustrates FAA employees’ divergent views on the Model Work Environment. Even though the decrease in the number of EEO complaints cannot be directly linked to the Model Work Environment initiative, the availability of data and analysis on EEO complaints could provide one objective basis for FAA to discuss the effects and assess the efficacy of this policy and address the concerns of those employees who view its impact less positively. FAA’s lack of empirical data on the reform effort’s effects is one indication that it has not fully incorporated elements that we and others have identified as important to effective human capital management into its reform effort. Systems to gather and analyze relevant data provide a basis against which performance goals and measures can be applied. FAA human resource management officials said that the agency should have spent more time to develop baseline data and performance measures before implementing the broad range of reforms but that establishing these elements was a complex and difficult task. They said FAA was under significant pressure to rapidly implement reforms and that one impact of FAA’s incremental approach to implementing the reforms was that baseline measures tended to change as more people were brought under the reformed systems. FAA also has not gone far enough in establishing linkage between reform goals and program goals of the organization, another element we have identified as important to effective human capital management. We found that the lack of these elements has been pointed out repeatedly in evaluations of FAA’s human capital reform effort, but FAA has not developed specific steps and time frames by which these elements will be established and used for evaluation. Incorporation of these elements could also help FAA build accountability into its human capital management. The lack of baseline and comparative data for analysis and the lack of performance goals and measures has made it difficult to objectively evaluate the effects or success of FAA’s reform effort. Systems to gather and analyze relevant data provide a basis against which performance goals and measures can be applied. FAA human resource management officials agreed that the agency should have spent more time to develop baseline data and performance measures before implementing the broad range of reforms but said that establishing measures and goals and reaching consensus on their use was a complex and difficult task with which all federal agencies struggle. They said the agency was under significant pressure to rapidly implement reforms, and that one impact of FAA’s incremental approach to implementing the reforms was that baseline measures tended to change as more people were brought under the reformed systems. Human resource management officials also said that, while FAA has not systematically collected data and analyzed results to identify the benefits of all of the reform initiatives, the Office of Human Resource Management has taken a number of steps since 1998 to increase evaluation and measurement of some human resource management activities and outputs. Actions they cited (in addition to the previously discussed evaluations of compensation implementation) included meeting with consultants, human resource managers and intergovernmental groups and providing briefings to FAA management. While we were in the final stages of our review, they prepared, in response to our request, an informal report that described the type of measures they were planning to, or had recently begun to apply as part of a “Balanced Scorecard” approach to assessing human resource management activities. The measures in the scorecard approach are based on existing sources of data—customer surveys conducted by the Department of Transportation and FAA employee attitude surveys—as well as new data related to the hiring process, such as the “Time to Fill” (a vacancy) questionnaire, results from employment selection feedback questionnaires, a survey for new recruits and, since December 1999, a separation survey for employees leaving the agency. Human resource officials said they had been “strategically refining” the employee attitude survey since 1995 to address key human capital issues, such as clarity of performance expectations and workforce planning. Our work on strategic human capital management in the federal government has found that many federal agencies have difficulties in defining goals and measures and developing and using performance information to evaluate the effectiveness of human capital management efforts but that high-performing organizations do so. In cases where evaluations show that sufficient progress is not being made, high- performing organizations use data to identify opportunities for improvement. Similarly, the National Association of Public Administration (NAPA) has reported the need for performance data, goals, and evaluation to determine progress, make midcourse corrections, and assign accountability for achieving the desired outcomes in federal human capital management efforts. NAPA reported that, in the absence of such systematic evaluation information, the human capital management process will be driven by anecdotal information that may, or may not, reflect the condition of human capital management in the organization. Elements we have identified as facilitating the success of improvement initiatives include establishing clear goals and objectives for the improvement initiative, concrete management improvement steps that will be taken, key milestones that will be used to track the implementation status, and cost and performance data that will be used to gauge overall progress. In addition to the lack of performance data, the performance goals and measures for personnel reform in FAA’s human resource management and strategic mission plans are qualitative and do not consistently lend themselves to measurement or assessment, as they are not specific, measurable, and time-based. For example, the goal related to reform in FAA’s 1999 human resource management strategic plan is to “ensure that FAA has the right people doing the right work at the right time at the right cost” and has the following measures associated with it: increased flexibility to pay competitive salaries; increased ability to attract and retain high performers; increased managerial flexibility to assign, locate, and manage the performance of employees more effectively; and decreased hire cycle time. This goal and its associated measures do not lend themselves to specific, quantitative, and time-based evaluation. For example, while “decrease hire cycle time” implies that hire cycle time will be measured as part of evaluating the achievement of this goal, it does not establish a quantitative basis for assessment or specify a period of assessment. A more specific, quantitative, and time-based measure might be to “decrease median or average hire cycle time by September 2003 by X percent (from median or average cycle time for fiscal year 2002) for Y percent of all new hires.” We reported on FAA’s weaknesses in developing and using performance information in our report on the results of governmentwide surveys of performance management issues in May 2001. In that report, we found that FAA managers we surveyed reported they did not consistently use performance measures or data and that FAA was worse than the rest of the federal government on multiple aspects of performance measurement and the use of performance information. For example, we found that the agency was statistically significantly lower than the rest of the government in the percentage of managers who reported that they had outcome, customer service, or quality performance measures; and in the percentage of managers who reported that they used performance information to set program priorities, allocate resources, adopt new approaches, or coordinate program efforts. At the time of our review, human resource management officials were still in the process of developing baseline data, performance goals and measures and were still working to identify potential linkages between its human capital management reforms and program goals of the organization. The types of data and measures proposed by human resource management officials are comparable to those that have been historically suggested— many of them since FAA initiated development of its personnel reform in 1995—and their implementation is an important effort. However, the balanced scorecard measurement approach proposed by human resource management officials focuses primarily on the work environment and processes within the Office of Human Resource Management and the hiring process rather than on the many other human capital management reform initiatives being implemented across the agency. According to FAA human resource management officials, the office had been working for more than a year to expand the scope of the scorecard to incorporate measures with wider implications for all of FAA in response to discussions with human resource managers and based on information from FAA customers and employees. Table 1 provides an overview of the balanced scorecard measures proposed by the human resource management office, highlighting those that focus on the activities and output of the Office of Human Resource Management. An expanded overview of these performance measures that includes areas of measurement and proposed data sources is provided in appendix V. Clearly linking an agency’s overall human capital management strategy to its program goals is another element we have identified as key to effective human capital management. In a 1997 review of FAA’s personnel reform, the Volpe National Transportation Systems Center highlighted this issue of linkage, as shown in figure 12. While FAA has taken some steps to link its human capital reform initiatives to its program goals, these steps do not go far enough to help the agency measure the reform’s success. Specifically, FAA incorporated various aspects of personnel reform into its 1999 strategic human resource management plan, which stated that performance measurement was to focus on attaining organization goals but did not establish the measures with which to do so. Similarly, FAA’s 2001 strategic plan, prepared under the Government Results and Performance Act, includes a goal for the agency to “fundamentally change the way it operates by implementing personnel reform” but does not explicitly link this goal for personnel reform to organizational program goals of aviation safety and system efficiency. Human resource management officials said that organizational and individual incentive goals established under the compensation system explicitly linked individual performance to agency goals including safety and system efficiency and that the standards for performance under FAA’s new performance management system directly reflect agency and organizational programmatic goals. Nonetheless, linkage between FAA’s personnel reform goals and the agency’s programmatic goals continues to be weakened by a lack of specific, quantitative, and time-based measures and goals. FAA’s lack of relevant data, analysis, and performance goals and measures has been repeatedly articulated since 1995 by other internal and external reviews of the reform effort. While these reviews have called for FAA to incorporate these elements into its reform effort, and several recent studies have also highlighted the issue of linkage, FAA has not established and carried out a plan with specific steps and time frames for doing so. A chronology of these studies is provided in table 2. Several of these studies also attributed problems related to a lack of ownership for the reform effort or a lack of accountability for implementation or results. For example, in 1999, the National Academy of Public Administration identified the lack of ownership for personnel reform as a challenge that must be resolved. (See fig. 13.) As shown in figure 14, a 1998 departmental review found that FAA had not clearly established accountability for implementation of the reform initiatives. According to the most recent assessment of the status of FAA’s personnel reform, published by a consultant in September 2002 and shown in figure 15, a lack of ownership and inconsistent support for personnel reform by FAA’s executive management team has impaired reform implementation efforts. Our work on effective human capital management at federal agencies has found that building accountability into an agency’s human capital approach is important to the effective use of human capital flexibilities. Furthermore, we have found that in high performing organizations, managers are held accountable for achieving strategic goals, and clearly defined performance expectations are in place to hold employees and teams at all levels accountable. Establishing systems for gathering performance data and incorporating specific, time-based performance measures and goals that are linked to the agency’s program goals into the reform effort would improve the agency’s ability to set more meaningful strategic goals for its human capital reform effort and more clearly defined performance expectations for its human capital management. Together, this would help the agency build accountability into the reform effort and its overall human capital management approach. Congress granted FAA flexibilities in its human capital management so that the agency could more effectively manage its workforce and achieve its mission. Yet, more than 7 years after the agency received broad exemptions from laws governing federal civilian personnel management, it is not clear whether and to what extent these flexibilities have helped FAA to do so. It is clear that FAA has faced significant challenges in implementing its human capital reform initiatives and evaluating the success of its effort. Challenges, including implementing reform initiatives throughout its workforce with a wide range of skills and negotiating agreements with employee unions, reflect difficulties that may be faced by other federal agencies that seek to implement human capital management flexibilities. FAA is not able to determine the effectiveness of its human capital reform initiatives because it has not incorporated key elements of effective human capital management into its effort thus far. While FAA has established preliminary linkages between its reform goals and the agency’s program goals, the lack of explicit linkage will make it difficult to assess the effects of the reform initiatives on the program goals of the organization even after data, measurable goals, and performance measures for human capital management efforts are established. FAA has acknowledged the importance of establishing these elements. It has repeatedly said that it is working to collect and analyze data and develop performance goals and measures, but it has not completed these critical tasks, nor has it established specific steps and time frames by which it will do so. As FAA moves forward, a more strategic approach to its reform effort would allow it to better evaluate the effects of its reform initiatives, use the evaluations as a basis for any strategic improvements to its human capital management approach, and hold agency leadership accountable for the results of its human capital management efforts. Doing so would also enable the agency to share its results with other federal agencies and Congress. In order to acquire the information needed to make more informed strategic human capital decisions and better ensure that FAA’s personnel reforms achieve their intended results in a timely fashion, we recommend that the Secretary of Transportation direct the FAA Administrator to develop empirical data and establish specific, measurable, time-based goals and performance measures related to these goals; and use them to evaluate the effects of the reforms on the agency’s human capital management, programs, and mission so that the agency can make any needed improvements. Developing these evaluation tools is particularly urgent for those initiatives, such as FAA’s new compensation system for air traffic employees, for which possible negative effects have been raised by employees; and FAA’s new performance management system. define and describe explicit linkages between human capital management reform initiatives and program goals of the organization. establish time frames by which data will be collected and analyzed and by which goals, performance measures, and explicit linkage will be established and used to evaluate the success of the reform initiatives and hold agency leadership accountable for the results of its human capital management efforts. We provided a draft of this report to the Department of Transportation for its review and met with Department of Transportation officials, including FAA's Assistant Administrator for Human Resource Management, to obtain their comments. The department officials generally agreed with the report's recommendations and indicated that the findings presented in the audit report would be useful as FAA moves forward with its human capital reforms. They also noted issues in three areas. First, these officials emphasized that implementing a new human capital system within an existing workforce presented FAA with a significant challenge, given the size of FAA's workforce, the large unionized population, and the variety of occupations and functions within the agency. Second, while these officials agreed that establishing more definitive measures and baseline data, as identified in our recommendations, are important in determining the effectiveness of the new human capital programs, they stated that they have been making significant progress in developing those measures. Third, in responding to our concern that FAA is not able to determine the effectiveness of its human capital reform initiatives because it has not incorporated key elements of effective human capital management into its effort, these officials told us that FAA used the results of its pilot testing and phased implementation approach to modify systems to ensure effectiveness before full implementation and that subsequent assessments were conducted to determine whether the programs were accomplishing the intended goals. They said that FAA already has substantial information to indicate that its new programs and initiatives are on the right track and should be effective in meeting the reform effort’s intent. As examples, they referred to reviews by NAPA and the consulting firm Deloitte & Touche, which they said had characterized FAA’s human capital reforms as “state- of-the-art.” The officials stated that FAA’s design process had been characterized in the NAPA review as yielding high-quality policies, and FAA’s reform effort had been characterized in the NAPA review as heading in the right direction and as “a change management issue that is unparalleled in the federal sector.” They further stated that Deloitte & Touche’s review had found that the guiding principles and objectives of FAA’s personnel reform were sound, and that some programs have already been largely successful, such as streamlined recruitment and staffing processes. Notwithstanding the characterizations in these assessments, both NAPA and Deloitte & Touche raised concerns about issues we found in our review, particularly FAA’s lack of baseline data and specific performance measures to assess the effectiveness of its reform effort and establish a basis for continuous improvement. Department officials also said that FAA’s new human capital system is consistent with the President’s Management Agenda and the Administration’s Human Capital Plan, and that other federal officials have touted the types of programs FAA developed and implemented as the wave of the future for the rest of the federal government. FAA emphasized that its agency is unique among federal agencies in implementing a performance-based and market-based pay system applicable to both nonunion and union employees, which clearly links annual pay adjustments to key agency programs and to individual employee performance and contributions. We agree that other federal agencies considering human capital reform may find FAA’s programs and experiences useful to consider, as FAA was granted human capital flexibilities in 1995 and has been working since to implement its human capital reform effort. In fact, we feel that this increases the importance of FAA’s efforts to effectively evaluate its reform. However, based on our prior work on human capital management, we found in our review that FAA’s efforts to link its human capital reform initiatives to its program goals do not go far enough to help the agency measure the reform’s success and that linkage between FAA’s personnel reform goals and the agency’s programmatic goals continues to be weakened by a lack of specific, quantitative, and time-based measures and goals. FAA also provided technical clarifications, which we included in the report where appropriate. We are sending copies of this report to the Administrator, Federal Aviation Administration. We also will make copies available to others upon request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov. If you or your staff have any questions about this report, please call me at (202) 512-3650. Key contacts and major contributors to this report are listed in appendix VII. To determine the human capital changes that FAA initiated after being granted broad flexibilities in 1995 and the extent to which these reform initiatives required exemptions from title 5, we reviewed federal personnel management requirements under title 5, agency documents identifying personnel reform initiatives, and reports by OPM on personnel management flexibilities already available under title 5. We also discussed the changes with officials from FAA’s Office of the Assistant Administrator for Human Resource Management and OPM. To determine the status of implementation of FAA’s personnel reform and factors that have affected reform implementation, we collected and analyzed internal and external evaluations—including those conducted by the Department of Transportation’s Office of Inspector General and NAPA—of different aspects of FAA’s personnel reform and the available data on the results. We also discussed the status of, and barriers to, implementation of personnel reform initiatives with FAA human resource management officials and representatives from the lines of business. To determine the views of FAA managers and employees on the effects of FAA’s personnel reform initiatives we conducted a series of structured interviews with 176 randomly selected FAA managers and employees. Our structured interview included questions about how the agency manages its employees, compensation and performance management, and labor and employee relations. We discussed the design of these questions with officials from FAA and with representatives from FAA’s five largest unions—the National Air Traffic Controllers Association (about 19,500 members), the Professional Airways Systems Specialists (about 11,600 members), the National Association of Air Traffic Specialists (about 2,300 members), the American Federation of State, County and Municipal Employees (about 2,000 members), and the American Federation of Government Employees (about 1,500 members). We then pre-tested the structured interview with managers and employees in FAA’s Southern Region and made appropriate revisions. To maximize our chances of obtaining the views of managers and employees across the different segments of FAA’s workforce, we applied a judgmental stratification to our random sample (therefore, it may not be representative of the actual composition of FAA’s workforce): 25 percent managers, and 75 percent employees; and 60 percent Air Traffic Services/air traffic control staff, 40 percent from the rest of FAA. In addition, for non-headquarters respondents, we selected 70 percent of our respondents from field facilities and 30 percent from regional offices. Our respondents were randomly selected from electronic lists of names provided by FAA. We conducted our structured interviews at FAA headquarters in Washington, D.C.; FAA’s Mike Monroney Aeronautical Center in Oklahoma City, Oklahoma; and field facilities and regional offices in six of FAA’s nine geographic regions, including offices in the immediate vicinity of Anchorage, Alaska; Atlanta, Georgia; Chicago, Illinois; Dallas, Texas; Los Angeles, California; and New York City, New York. Field facilities we visited included air traffic control towers, en route centers, automated flight service stations, terminal radar approach control centers, airports district offices, and flight standards district offices. A total of 176 FAA staff participated in our survey throughout the months of May, June, July, and August 2002. The information obtained through this survey pertains to only these 176 respondents and cannot be generalized to any other population. However, because we selected interview respondents at random, we have increased the chances of capturing the breadth of opinions across the agency. A copy of our structured interview and the summary results for our close-ended questions is provided in appendix II. To augment the views and opinions collected from the structured interviews, we also obtained the views of FAA senior managers or representatives of all five lines of business and representatives of employees’ associations. To determine the extent to which FAA management and employees’ views were supported by data, we examined the results from FAA’s employee attitude surveys conducted between 1997 and 2000, as well as other internal surveys of executives, managers, and supervisors related to various aspects of FAA’s personnel reform effort. In addition, we collected available data from FAA’s Office of Human Resource Management and Office of Civil Rights. To determine how FAA’s experiences compared with our findings from our human capital management work at other agencies, we reviewed our human capital management audit work that focused on federal agencies’ efforts to implement improvement initiatives and human capital flexibilities, as well as work conducted by other organizations involved in assessing federal agencies’ reform efforts including OPM and NAPA, and we compared our findings on FAA’s experiences with these findings. We conducted our work in accordance with generally accepted government auditing standards from November 2001 through October 2002. STRUCTURED INTERVIEW FORM AND SELECTED RESULTS ATC (A)/Non-ATC (NA): ________ Regional Office (RO)/Field Office (FO): ________ Manager or Supervisor (MS)/Employee (E): ________ Location (AT, AN, CH, DA, NY, LA)/ID Number:________ Thank you for meeting with us today. We work with the General Accounting Office in Washington DC and we’ve been asked by Congress to see how the personnel reforms at FAA are going. One of the ways we’re doing that is by asking FAA employees like you to tell us about how the personnel reforms here are affecting you, your ability to perform your work, and your unit’s ability to achieve its mission- in both good and bad ways. We’re going to several different regions and talking to many employees- we selected your name at random and appreciate your willingness to talk to us. I will be asking you a series of questions, some you can answer from a range of standard responses like strongly agree to strongly disagree, and others will give you an opportunity to provide a little bit more specific information. ____ will be taking notes to be sure that we capture all you have to say. Your responses are confidential- we won’t report your name with anything you say here and we’ll report our results as a summary of what everyone tells us. The entire interview should take about 30 minutes. Do you have any questions before we begin? (Personnel reform includes: hiring, training, compensation, performance management, labor and employee relations, among others.) 1. To start with, how knowledgeable are you about FAA’s personnel reform efforts? Which reform(s) do you know most about? Least about? The first set of questions I’ll ask deal with how FAA manages its workforce. For the next question and others throughout the survey, I’d like you to use this scale for your answer- please tell me if you strongly agree (SA), agree (A), disagree (D) or strongly disagree (SD). 2. I receive the training I need to do my job effectively. ____ 3. Could you give me more detail about that? How has the fact that the lines of business are responsible for funding and managing training affected the amount and/or quality of training you’ve received? 4. What, if any, training do you need that your line of business (or staff office) has not provided? 5. Using the scale again: The ability of my line of business (or staff office) to efficiently and effectively fill job vacancies has improved in the last 5 years. Office 6. Has the speed of hiring improved? How? {Pause for response} Has the quality of candidates improved? How? The next set of questions I’ll ask deal with the new pay and performance management systems including the new pay bands and pay-for-performance system, as well as, the “meets/does not meet standards” performance rating system implemented in 1996. 7. Using the scale again: I am better off under the new pay band system than under the grade and step pay system. 8. Using the scale again: I think the new pay system is fair to all employees. Non-ATC 9. What do you like about the new pay system? Is there anything you don’t like about the new pay system? 10. Using the scale again: Separating cash awards from performance appraisals has made the appraisal process more fair. 11. Using the scale again: The way my most recent formal performance appraisal was handled gave me useful information for improving my performance. Non-ATC 12. Using the scale again: Awards and recognition more appropriately reflect employees’ performance today than 5 years ago. 13. Using the scale again: FAA’s process for promotion better targets qualified people now than it did 5 years ago. Non-ATC 14. To what extent have managers and supervisors become more (or less) accountable for achieving agency goals in the last 5 years? The next set of questions I’ll ask deal with labor relations and employee relations. 15. Using the scale again: FAA’s employee unions have had a positive impact on implementing personnel reform in the agency. 16. Do you think labor-management relations have changed in the last 5 years? (As applicable) How? What specifically has driven this change? 17. Do you think employee morale has changed in the last 5 years? (As applicable) How? What specifically has driven this change? 18. Using the scale again: FAA employees had sufficient opportunities to provide input for personnel reform policies and initiatives before they were finalized and implemented. ____ 19. Using the scale again: I understand the goals of the Model Work Environment. 20. Please describe the effect Model Work Environment has had (if any) on employee morale. 21. Using the scale again: I have received sufficient and timely information on personnel reform changes that affect my job. ____ 22. What personnel reforms have been particularly well communicated? Which method(s) of communication work(s) best or would work best? Worst? The last set of questions deal with the overall result of personnel reforms at FAA. 23. Using the scale again: Personnel reform has made FAA a better place to work. 24. Using the scale again: Personnel reform will make FAA a better place to work. 25. What are the 3 most positive outcomes of FAA’s personnel reform efforts? 26. What are the 3 most negative outcomes of FAA’s personnel reform efforts? 27. What kinds of comments have you heard about the personnel reforms we’ve been discussing today from your co-workers? 28. Do you have any suggestions for improving FAA’s implementation of personnel reform? Okay, although we won’t be using your name with this information, I would like to ask just a couple of questions about your position here. 29. How long have you worked at FAA? AFGE AFSCME NAGE NAATS NATCA PAACE PASS Thank you very much for answering my questions today. We really appreciate your time, and the feedback we get from you will help FAA and Congress make future decisions about personnel reform. In March 2002, we issued a model for strategic human capital management that incorporates lessons learned in our reviews of other agencies’ human capital management practices, as well as our own experiences. The model identifies eight critical success factors and highlights some of the steps agencies can take to make progress in managing human capital strategically. These eight factors, shown in figure 16, are organized in pairs to correspond with the four governmentwide high-risk human capital challenges that our work has shown are undermining agency effectiveness. In November 2002, we issued a report that identified six key practices for federal agencies’ effective use of human capital flexibilities that incorporate the concepts and critical factors of our model. Based on our interviews with human resource management directors from across the federal government, we identified the following key practices that agencies should implement to use human capital flexibilities effectively, as shown in Figure 17. Blank cells indicate no old grade equivalent to new pay band for manager levels. (proposed data sources) Customer perceptions regarding human resource management office consultation & staff expertise (employee surveys) Customer perceptions regarding personnel reform Human resource management office & line of business human capital management efforts (employee surveys) Human resource management office spending (budget & accounting data) Knowledge transfer Improved line of business processes & practices Meet unique needs (human resources office reporting system) Human resource management office labor distribution (cost accounting system) (employee surveys) Consolidation in bargaining units Partnership Council meeting attendance (Personnel management evaluations, human resource management office policy training evaluations) Timeliness of automated/nonautomated selections Percentage of voluntary & involuntary attrition (data on grievances and unauthorized labor practices) (Selecting official interview data) Human Resource Management Office employee perceptions regarding communication Performance rewarded (employee surveys) (employee surveys) Identify and close skill gaps (skills/training assessment for human resources office) (skills/training assessment for Human Resource Management Office) (human resource management office information system audit) Timeliness and responsiveness to internal Human Resource Management Office requests (employee survey) (human resource management office reporting system) Percentage of personnel, compensation, and benefits funding spent on training (budget & accounting data) In addition to those individuals named above, William Doherty, Michele Fejfar, David Hooper, Jason Schwartz, E. Jerry Seigler, Margaret Skiba, Tina Smith, Alwynne Wilbur, and Kristy Williams made key contributions to this report. The General Accounting Office, the investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full- text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics. Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to www.gao.gov and select “Subscribe to GAO Mailing Lists” under “Order GAO Products” heading.

Summary: In 1996, the Federal Aviation Administration (FAA) undertook a human capital reform effort under one of the most flexible human capital management environments in the federal government, including broad exemptions from title 5 laws governing federal civilian personnel management. GAO was asked (1) to examine the changes FAA initiated in its reform effort, including whether they required an exemption from title 5 and their implementation status; (2) determine the effects of the reform effort according to available data and the views of FAA officials, managers, and employees; and (3) assess the extent to which FAA's reform effort incorporated elements that are important to effective human capital management. In 1996, FAA initiated human capital reform initiatives in three broad areas, some of which required exemption from title 5, and some of which have been fully implemented. FAA has not yet completed implementation of some key initiatives. For example, FAA's new compensation system remains unimplemented for about one-quarter of the agency's workforce--those staff whose unions have not reached agreements with FAA. FAA's need to implement initiatives among a workforce with a wide range of skills and to negotiate changes with multiple unions were among factors that affected the pace and extent of reform implementation. FAA had little data with which to assess the effects of its reform effort. While FAA human capital officials cited positive effects of FAA's reform effort, the views of managers and employees GAO interviewed were generally less positive. FAA's lack of empirical data on the effects of its human capital initiatives is one indication that it has not fully incorporated elements that are important to effective human capital management into its overall reform effort. These elements include data collection and analysis, performance goals and measures, and linkage of reform goals to program goals. FAA human resource management officials said that the agency should have spent more time to develop baseline data and performance measures before implementing the broad range of reforms but that establishing these elements was a complex and difficult task. FAA has also not gone far enough to establish linkage between reform goals and overall program goals of the organization. GAO found that the lack of these elements has been pointed out repeatedly in evaluations of FAA's human capital reform effort, but FAA has not developed specific steps and time frames by which these elements will be established and used for evaluation. Incorporation of these elements could also help FAA build accountability into its human capital management.