FAA MUST IMPROVE ITS CONTROLLER 

TRAINING METRICS TO HELP IDENTIFY 

PROGRAM NEEDS

Federal Aviation Administration

Report Number: AV-2011-072

Date Issued: March 30, 2011
U.S. Department of

Memorandum

Transportation

Office of the Secretary

of Transportation

Office of Inspector General

Subject: INFORMATION: FAA Must Improve Its 

Controller Training Metrics To Help Identify 

Program Needs

Federal Aviation Administration

Report Number AV-2011-072

Date: March 30, 2011

From: Jeffrey B. Guzzetti

Assistant Inspector General

 for Aviation and Special Program Audits

Reply to 

Attn. of: JA-10

To: Federal Aviation Administrator

The Federal Aviation Administration (FAA) plans to hire and train nearly 

11,000 new air traffic controllers through fiscal year (FY) 2019 to replace the 

large numbers of those now retiring. As FAA begins training this influx of new 

hires, it must have accurate metrics on their training progress to ensure that key air 

traffic control facilities have enough controllers for safe and efficient operations.

In 2009, at the request of Representative Jerry F. Costello, then Chairman of the 

House Subcommittee on Aviation, we reviewed training failures among newly 

hired air traffic controllers.

1

 During that review, we found that FAA’s reported 

training failure rate was not accurate and that FAA’s primary source of training 

failure data, the National Training Database (NTD),

2

This review follows up on our 2009 audit. Our audit objectives were to 

(1) evaluate FAA’s actions to improve its system for tracking the training progress 

of newly hired controllers and (2) review FAA’s metrics for measuring and 

reporting the effectiveness of its controller training program. As a part of the 

evaluation, we examined FAA’s methodology for calculating the training attrition 

and completion rates of developmental (new) controllers. We conducted this 

contained outdated and 

inaccurate data. Both are critical metrics for managing this important program. 

 1 OIG Report Number AV-2009-059, “Training Failures Among Newly Hired Air Traffic Controllers,” June 8, 2009. 

OIG reports are available on our website: www.oig.dot.gov. 2 The NTD is FAA’s primary tool for national oversight of the controller training program. FAA uses the NTD to 

track the progress of developmental controllers at air traffic facilities. Originally designed to track the training time 

for developmental controllers, the database was expanded to capture data relating to the various categories of 

training attrition, including training failures.
2

review between November 2009 and January 2011 in accordance with government 

auditing standards prescribed by the Comptroller General of the United States. 

Exhibit A details our scope and methodology.

RESULTS IN BRIEF

FAA has improved its tracking process for new controller training over the last 

year. The Agency has taken a number of corrective actions to address the 

problems we identified in 2009 that contributed to an inaccurate training failure 

rate reported for newly hired controllers. These actions include establishing a new 

Quality Assurance Group in March 2010 to oversee NTD data collection, 

separately tracking new hires and veteran controllers, distinguishing between 

training failures and other types of attrition in the NTD, and issuing guidance to 

enforce data accuracy. While these actions are positive steps towards improving 

how it tracks the training progress of newly hired controllers, FAA continues to 

face challenges in identifying training program needs and measuring the overall 

success of the training program.

FAA’s metrics for measuring the effectiveness of the controller training program 

do not provide a complete picture because they include controllers who have not 

completed their initial training. For example, if there are 100 controllers in the 

training program and 9 of those controllers fail or leave, FAA reports an attrition 

rate of 9 percent. This produces unrealistic results because some of the remaining 

91 in-progress controllers may also leave the program at a later time. Eventually, 

all controllers in training will either be certified or leave the program, but because 

FAA includes all in-progress controllers in its attrition rate, it dilutes the 

program’s actual loss and completion rates. As a result, FAA cannot rely on these

data to make appropriate and timely adjustments to the program. When we 

assessed the number of controllers who successfully completed training against 

those who did not, we found a significantly higher attrition rate of 21 percent for 

newly hired controllers. This presents a very different picture of success 

compared to the 9-percent attrition rate reported by FAA for FY 2009.

Accurate training data are necessary so that FAA can adequately prepare new hires 

to replace retiring veteran controllers, assign new hires to the appropriate level and 

type of facility, and adjust its training program when warranted. Our

recommendation focuses on steps FAA should take to measure and present a more 

complete picture of the effectiveness of its air traffic controller training program. 

BACKGROUND

New air traffic controllers must complete an arduous training program that 

includes learning the basic concepts of air traffic control at the FAA Academy, 

followed by extensive facility training at their assigned location. Those controllers 
3

who are unable to pass the training process are either (1) transferred within their 

assigned facilities to a new area of operation, (2) transferred to a less complex 

facility to begin the training process again, or (3) terminated from employment 

with FAA. While certification times for individual controllers may vary, FAA’s 

goal is to have terminal candidates—who manage air traffic in the vicinity of 

airports—complete the training process in 2 years, and en route candidates—who

manage high-altitude traffic—in 3 years.

FAA HAS IMPROVED ITS TRACKING PROCESS FOR NEW 

CONTROLLER TRAINING OVER THE LAST YEAR

FAA has taken a number of actions in response to our June 2009 report to improve 

its tracking process for newly hired controllers in training. For example, we

recommended that FAA develop a process for conducting periodic reviews to 

ensure that air traffic facilities complied with NTD data entry requirements and 

identify factors that could indicate trends or causes of training failures. In March 

2010, the Technical Training Organization3

We also recommended that FAA develop a comprehensive definition of “training 

failure” and other types of training attrition and consistently apply these 

definitions at the 316 air traffic facilities and FAA Headquarters. This is 

important because we found FAA’s classification process for training attrition—

either due to failure or facility transfer—was too subjective. Individual managers 

exercised wide leeway in how they classified attrition, often citing reasons other 

than training failure for an employee’s transfer or termination. This made it 

impossible to ensure that the data facility training managers entered in the NTD

accurately reflected why new controllers did not successfully complete their

training. FAA’s actions to address this recommendation included:

established the Quality Assurance and 

Reporting Group to conduct periodic reviews of NTD data collection. The group

is also responsible for maintaining complete records on all aspects of new 

controller training, including the dedicated tracking of newly hired and veteran 

controllers’ training progress. This was a good step toward improving the 

accuracy and oversight of training data within the NTD.

• adding a new table of definitions to the NTD that distinguish between training 

failures and other types of training attrition and 

• issuing a memorandum in March 2010 to FAA’s Vice Presidents of both 

Terminal and En Route and Oceanic Services requesting that NTD entries be 

kept up to date (updated every 14 days) and more reliable (by using the NTD’s 

data error checker).

 3 FAA Air Traffic Organization, Technical Training has oversight of the air traffic controller training program.
4

FAA’s actions were sufficient for us to close the recommendations from our prior 

report. However, FAA must now ensure that the training managers at 

Headquarters and all air traffic control facilities consistently implement these 

actions to ensure they accurately report new controller training data.

FAA’S METRICS FOR MEASURING EFFECTIVENESS DO NOT 

PROVIDE A COMPLETE PICTURE OF THE CONTROLLER 

TRAINING PROGRAM

While FAA has improved its tracking process, the current metrics for measuring 

the effectiveness of its controller training program do not provide a complete 

picture of program success because they include controllers who have not 

completed their initial training. As a result, FAA is not reporting an attrition rate 

that it can rely on to make warranted adjustments to the training program. During 

our review, we used alternate metrics that focused on the annual output of the 

training program. These showed a significantly higher FY 2009 attrition rate for 

new controller training than what FAA reported. 

FAA’s Method Understates Actual Controller Attrition 

FAA measures the effectiveness of its controller training program by calculating

the percentage of training attrition for the fiscal year in which the new controllers

start their initial training. For example, according to FAA, as of March 2010, the 

attrition rate for new controllers who started their initial training in FY 2009, or 

the “Class Hired in FY 2009,” was 9 percent, as depicted in figure 1 below. 

Figure 1. FAA’s New Controller Training Data for “Class Hired in FY 2009”

Successes 4%

(71 Controllers)

Attritions 9%

(136 

Controllers)

In-Progress 87%

(1,387

Controllers)

Source: FAA
5

However, figure 1 also shows that the success rate was only 4 percent while 

87 percent of the controllers were still completing their initial training, which can 

take 2 to 3 years. While FAA’s metrics show a lower reported attrition rate for 

new controllers, this methodology will not give a complete picture of the 

effectiveness of the training program until all 1,387 new, in-progress controllers 

complete their initial training. For the 9-percent attrition rate to be realized, FAA 

is assuming that all of these controllers will succeed. This is an unrealistic 

assumption given that, historically, it is not uncommon for newly hired controllers 

to leave after more than 2 years in training. 

As more new hires complete their initial training, the attrition rate has historically 

increased. For example, we examined FAA’s attrition data for all new controllers 

hired in FY 2007. In December 2007, FAA reported that the attrition rate for 

those new controllers was approximately 7 percent. In March 2010, more than

2 years later, FAA reported the attrition rate for the same “Class Hired in

FY 2007” controllers was 24 percent (see figure 2). By including in-progress 

controllers in the training output metrics, FAA is diluting the attrition and 

completion rates until most of the controllers have completed training. However, 

waiting 2 years for most controllers to complete the training would not allow FAA 

to make adjustments to the program to boost controller training success. 

Figure 2. FAA’s New Controller Training Data for “Class Hired in FY 2007”

as of March 2010

Our analysis shows that a better metric for measuring the effectiveness of the 

training program would be to focus on how many controllers complete their 

training or leave the program during a given period of time, regardless of when 

they were hired. We used this metric during our review and found that it gives a 

more complete picture of the training program results because it captures the 

actual success of the program over an extended period of time and eliminates the 

Successes-60%

(1,097 

Controllers)

Attritions- 24%

(435 

Controllers)

In-Progress-16%

(295 Controllers)

Source: FAA
6

uncertainty of in-progress controllers. Officials from Raytheon who manage the 

controller training program and our senior statistician also concur that this is a 

more realistic and comprehensive method for measuring the effectiveness of new 

controller training and that it is misleading to include in-progress controllers in the 

attrition rate calculations. 

Using this approach, we analyzed the success rate of all new controllers who 

completed their initial training over the past 3 years, regardless of when they 

began their training. We grouped the controllers by the fiscal year they ended 

training and then identified whether they ended the training successfully or 

unsuccessfully. Our analysis showed that the attrition rate for the controllers who

ended their initial training in FY 2009 was 21 percent, and their success rate was

79 percent (see figure 3 below). These metrics show an attrition rate more than 

double the 9-percent FY 2009 attrition rate reported by FAA and are comparable 

to the attrition rates for FY 2008 and FY 2010, which were 31 percent and 

22 percent, respectively. By excluding in-progress controllers, our rate reflects the 

actual percentage of controllers who have been certified or left the training 

program. These attrition rates are a more complete and accurate measure of the 

effectiveness of the controller training program because they will not change as 

more controllers complete their training. After we discussed our results with FAA 

program officials, they began calculating completion and attrition rates without inprogress controllers in addition to their current calculations. 

Figure 3. OIG Analysis of FAA’s Training Data

(FY 2008 - November 2009)

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

FY 2008 FY 2009 FY 2010

Attritions Successes

69%

31%

79%

21%

78%

22%

Source: FAA Status of Controllers Ending Training in a Fiscal Year
7

CONCLUSION

The safety of the National Airspace System relies on having a fully staffed, welltrained air traffic controller workforce. To maintain the controller workforce in 

light of recent increases in controller retirements, FAA faces tremendous 

challenges in executing its plans to train nearly 11,000 new controllers over the 

next 10 years. Having accurate metrics to measure and report on the effectiveness 

of its controller training program should be a critical component of that plan. 

However, FAA’s current metrics do not portray an accurate picture of the 

program’s success. If the actual number of new controllers who do not complete 

initial training is greater than FAA anticipates, the Agency runs the risk of not 

maintaining a sufficient number of fully trained controllers at some of the most 

complex air traffic control facilities in the National Airspace System.

RECOMMENDATION

To help identify controller training program needs and provide a more accurate 

picture of its training program success, FAA needs to modify its training metrics 

by removing controllers still in training from its attrition and completion 

calculations. Specifically, we recommend that FAA replace its current training 

metrics with metrics that focus on how many controllers complete their training or 

leave the program during a given period of time.

AGENCY COMMENTS AND OFFICE OF INSPECTOR GENERAL 

RESPONSE 

We provided FAA with our draft report on February 25, 2011, and received its 

response on March 17, 2011. FAA concurred with our recommendation and has 

taken acceptable corrective actions. FAA’s entire response is included at the 

appendix to this report.

ACTIONS REQUIRED

In accordance with Department of Transportation Order 8000.1C, we are closing 

the recommendation. We appreciate the courtesies and cooperation of FAA 

representatives during this audit. If you have any questions concerning this report, 

please contact me at (202) 366-0500 or Bob Romich, Program Director, at (202) 

366-6478.

#

cc: Anthony Williams, AAE-001

Martin Gertel, M-100 
8

Exhibit A. Scope and Methodology

EXHIBIT A. SCOPE AND METHODOLOGY

We conducted this performance audit from November 2009 to January 2011 in 

accordance with generally accepted government auditing standards. Those 

standards require that we plan and perform the audit to obtain sufficient, 

appropriate evidence to provide a reasonable basis for our findings and 

conclusions based on our audit objectives. We believe that the evidence obtained 

during this audit provides a reasonable basis for our findings and conclusions, 

based on our audit objectives. The following scope and methodology were used in 

conducting this review. During this audit we visited FAA Headquarters.

To evaluate the actions taken by FAA to improve the tracking of the training 

progress of newly hired controllers, we evaluated the current accuracy of the data 

contained in FAA’s National Training Database (NTD). We compared the total 

number of new controllers in training for FY 2009 that was recorded in the NTD 

against the same population contained in the Federal Personnel Payroll System 

(FPPS). Additionally, we interviewed FAA personnel from the Office of 

Technical Training and Development to determine the extent of FAA’s 

compliance with the recommendations in our June 2009 audit report, “Training

Failures Among Newly Hired Air Traffic Controllers.” 

Finally, we evaluated FAA’s methodology for calculating controller training 

metrics. We conducted interviews with managers from the Office of Technical 

Training and Development to discuss the accuracy and validity of FAA’s 

controller training attrition rate, as an indicator of the effectiveness of the new 

controller training program. We also developed an alternative metrics calculation 

(calculated by dividing unsuccessful controllers by all completions which means 

we do not include new controllers whose completion status is still pending) and 

presented both methodologies to the OIG statistician for an analysis of the 

statistical validity of both metrics calculations. 
9

Exhibit B. Major Contributors to This Report

EXHIBIT B. MAJOR CONTRIBUTORS TO THIS REPORT

Name Title 

Daniel Raville Program Director 

Robert Romich Project Manager 

Erik Phillips Senior Analyst 

Petra Swartzlander Statistician

Benjamin Huddle Analyst

Mi Hwa Button Analyst

My Phuong Le Analyst

Andrea Nossaman Writer/Editor
10

Appendix. Agency Comments

APPENDIX. AGENCY COMMENTS 

Federal Aviation 

Administration 

Memorandum

Date: March 17, 2011

To: Jeffery B. Guzzetti, Assistant Inspector General for Aviation and Special Program 

Audits 

From: Clay Foushee, Director, Audit and Evaluations, AAE-1

Subject: OIG Draft Report: FAA Must Improve Its Controller Training Metrics To Help 

Identify Program Needs

The FAA has successfully produced new controllers at the volume and speed described in the

Controller Workforce Plan and has replaced about 25 percent of the workforce with a new

generation of fully qualified controllers while maintaining high standards for certification. To

help gauge if controllers are certifying at the right place and at the right time, FAA focused on

developing solid and representative metrics. We continue to improve the processes for tracking

and measuring training effectiveness to help ensure that facilities across the nation will have

enough qualified controllers for the safe and efficient operation of the most complex airspace

system in the world.

In 2010, the Office of Technical Training established a new Quality Assurance and Reporting

group to oversee training metrics and evaluate how well both individual and organizational

competency developed for the technical workforce, with a focus on cost, performance and

people. We aggressively pursued data to establish and maintain measurable and meaningful

metrics to assess performance. As a result, there are now nearly 70 reportable indicators to

evaluate training performance for FAA’s technical workforce. Forty of these measurements

provide the agency with specific feedback on controller training.

In order to provide a comprehensive indicator of training effectiveness, FAA uses multiple

metrics. In addition to the indicators detailed in the OIG report, FAA also measures the number

of Certified Professional Controller (CPC) completions, the number of controller developmentals

in training, the number of controller developmentals who complete various stages of training, the

time to CPC certification, the on-time CPC completion rate and average on-the-job training

hours. The combination of these indicators provides a more holistic perspective of controller

training effectiveness than any single metric.

In addition to monthly metrics used by the training organization, there are Quarterly Business

Review discussions with leadership from other air traffic control service units, such as En Route
11

Appendix. Agency Comments

and Terminal, to share the latest performance indicators. These reviews focus on strategies to

improve the agency’s training effectiveness and how the training organization can better support

air traffic technical training needs. These interactions with the service organizations have been

helpful in identifying the potential for FAA to continue to improve its methods for measuring

training effectiveness during the hiring surge.

The following is provided in response to the OIG’s recommendation:

OIG Recommendation: To help identify controller training program needs and provide a more

accurate picture of its training program success, FAA needs to modify its training metrics by

removing controllers still in training from its attrition and completion calculations. Specifically,

we recommend that FAA replace its current training metrics with metrics that focus on how

many controllers complete their training or leave the program during a given period of time.

FAA Response: Concur. The FAA needs to provide an accurate assessment of its training

program success, which FAA has implemented with the combination of metrics described above.

FAA recognizes the potential to further improve these metrics, and agrees that the standard

recommended by the OIG report provides a useful addition to its suite of training effectiveness

metrics. Prior to the issuance of this response, FAA incorporated this “Air Traffic Facility

Unsuccessful Completion Rate (at completion)” indicator into the metrics program. However,

measures that include controllers still in training in our attrition rates serve specific workforce

planning projection and hiring purposes, and will continue to be generated as part of FAA's

comprehensive suite of metrics.

The FAA currently tracks training attrition over longer time periods in order to discern both

short- and long-term trends. Monitoring attrition by the fiscal year in which training began is a

valid indicator that provides feedback on each fiscal-year class, but useful data is also generated

by following this metric through until everyone hired in a given year completes training

successfully or fails. Until then, we include those controllers still in training, but monitor the

attrition rates over a 12-month time period to more accurately assess trends. We also expand this

metric to assess training progress over a 24-month period to assess both completions and attrition

on a biannual basis to help us determine if current hiring is sufficient to maintain adequate

controller staffing levels.

We go even further and use hiring, staffing and training effectiveness data to measure longer

term projections in controller staffing. These indicators provide high value in our overall

training metrics suite, which are used to convey a complete picture of the overall controller

training program effectiveness. The FAA intends to continue to use each of these metrics, along

with the metric recommended by OIG to measure both workforce planning and training

effectiveness. Actions pursuant to this recommendation are considered complete.