Office of Inspector General

Audit Report

WEAKNESSES IN PROGRAM AND 

CONTRACT MANAGEMENT CONTRIBUTE 

TO ERAM DELAYS AND PUT OTHER 

NEXTGEN INITIATIVES AT RISK 

Federal Aviation Administration

Report Number: AV-2012-179

Date Issued: September 13, 2012
U.S. Department of

Memorandum

Transportation

Office of the Secretary

of Transportation

Office of Inspector General

Subject: ACTION: Weaknesses in Program and Contract 

Management Contribute to ERAM Delays and Put 

Other NextGen Initiatives at Risk 

Federal Aviation Administration

Report No. AV-2012-179

Date: September 13, 2012

From: Jeffrey B. Guzzetti

Assistant Inspector General 

 for Aviation and Special Program Audits

Mary Kay Langan-Feirson

Assistant Inspector General 

 for Acquisition and Procurement Audits

Reply to 

Attn. of: JA-10, JA-60

To: Acting Federal Aviation Administrator

Since 2002, the Federal Aviation Administration (FAA) has been developing the 

$2.1 billion En Route Automation Modernization (ERAM) program to replace and 

significantly enhance the existing hardware and software at facilities that manage 

high-altitude air traffic. ERAM is a foundational component of FAA’s Next 

Generation Air Transportation System (NextGen) and is critical to meeting FAA’s 

goals for increasing airspace capacity and reducing flight delays. We have testified 

on several occasions before Congress that delays in implementing ERAM could 

significantly impact the cost and pace of NextGen1

 1 OIG Testimony Numbers CC-2010-048, “Challenges in Meeting FAA’s Long-Term Goals for the Next Generation 

Air Transportation System,” April 21, 2010, CC-2011-016, “Actions Needed To Meet FAA’s Long-Term Goals for 

NextGen,” February 16, 2011, and CC-2011-036, “Progress and Challenges in Developing and Transitioning to the 

Next Generation Air Transportation System,” October 5, 2011. OIG reports and testimonies are available on our 

Web site: 

implementation. FAA planned 

to deploy ERAM to the Nation’s 20 en route facilities by the end of 2010. 

However, due to software problems that were identified early on at its two key test 

sites in Salt Lake City, UT, and Seattle, WA, FAA has delayed ERAM’s schedule 

beyond original completion dates. 

http://www.oig.dot.gov.
2

Given the importance of ERAM for transforming the National Airspace System 

(NAS), the Chairman and Ranking Member of the House Committee on 

Appropriations, Subcommittee on Transportation, Housing, and Urban 

Development and Related Agencies, requested that we assess FAA’s progress with 

implementing ERAM. Accordingly, our audit objectives were to (1) evaluate 

FAA’s progress in correcting ERAM’s persistent software problems; (2) evaluate 

whether the ERAM contract is designed and administered effectively to manage 

costs and achieve desired outcomes; and (3) identify the risks that ERAM’s delays 

pose to FAA’s plans to implement critical NextGen initiatives. 

We conducted this audit in accordance with generally accepted Government 

auditing standards. Exhibit A details our scope and methodology. Exhibit B lists 

the entities we visited or contacted.

RESULTS IN BRIEF

FAA’s multibillion dollar ERAM program has experienced software problems that 

have impacted the system’s ability to safely manage and separate aircraft. As a 

result of the delays at the key sites, FAA now projects that ERAM will be almost 

4 years behind schedule, with an uncertain final completion date. If problems 

persist, cost increases could reach in excess of $500 million and interfere with 

program execution. FAA’s problems in advancing ERAM are attributable to a 

number of fundamental program management weaknesses that have impeded the 

Agency’s ability to effectively implement ERAM and effectively manage other 

major acquisitions. These weaknesses include (1) setting an unrealistic schedule, 

(2) allowing ERAM to successfully pass Government Acceptance2

ERAM’s problems are also traceable to weaknesses in its contract, which is not 

structured or administered to effectively manage costs and achieve desired 

program outcomes. Due to insufficient acquisition planning, FAA did not fully 

adopt best practices that would have permitted more effective contract 

management when designing the ERAM contract structure. For example, FAA 

structured ERAM as a traditional, large-scale contract with enormous contract 

even though 

testing at the Agency’s Technical Center was limited and could not replicate actual 

field conditions, (3) ignoring early warning signs of trouble, such as an

unexpectedly high number of problem reports, and (4) a lack of attention to 

identify, communicate, and fix ERAM’s problems. This was compounded by a 

management culture that was slow to fully acknowledge the extent of ERAM’s 

problems or communicate them to senior FAA management. 

 2 Government acceptance of ERAM by the FAA Technical Center requires meeting specific established criteria. These 

criteria include successfully completing developmental testing activities per the Statement of Work, listing all 

problem trouble reports, demonstrating that all contractual requirements are satisfied, and completing both functional 

and physical configuration audits. After Government Acceptance is reached, FAA assumes all costs, including fixing 

any additional problems discovered. 
3

tasks that span several years instead of using modular contracting, which would 

divide the contract into manageable segments for better control. In addition, FAA 

has still not fully finalized the costs for 16 of 57 contract tasks,3

ERAM’s delays pose significant risks to FAA’s plans to implement critical 

NextGen initiatives because of complex interdependencies between ERAM and 

other key systems needed to advance NextGen. Continued problems will affect the 

pace of critical efforts such as Data Communications (DataComm), System Wide 

Information Management (SWIM), and Automatic Dependent Surveillance -

Broadcast (ADS-B). ERAM delays will also impact FAA’s ability to develop 

other NextGen-related software enhancements and other improvements, such as 

trajectory-based operations

yet the Agency 

authorized the contractor to begin work on them—a practice that gives the 

contractor little incentive to control costs. Additionally, FAA has awarded the 

contractor over $150 million in cost incentives despite cost overruns, delays, and 

software problems. Problems with the ERAM contract were further compounded 

by weaknesses in FAA’s acquisition workforce and poor contract management 

practices. These include (1) high contracting officer (CO) turnover and heavy 

reliance on support service contractors, (2) poorly organized and incomplete 

contract files, and (3) limited reviews of vendor invoices that failed to detect 

unallowable charges, including nearly $69,000 in fraudulent travel expenses.

4

We are making a series of recommendations to improve ERAM’s contract 

structure and oversight and reduce the associated risks to future NextGen related 

programs. 

and the development and transition to a common 

automation platform for FAA air traffic facilities. In addition, ERAM’s schedule 

delays and corresponding cost growth have forced FAA to reallocate millions in 

funds from other capital NextGen programs. Despite the significant program risks 

and unresolved issues—and that FAA plans to allocate more than $500 million to 

align and integrate NextGen-critical initiatives with ERAM—FAA has not fully 

assessed the impact of these delays on other programs’ costs and schedules. 

BACKGROUND

ERAM is a key NextGen enabling program for processing high-altitude air traffic5

 3 As of January 17, 2012.

flight information across the NAS. It replaces HOST, the legacy en route 

automation system, which consists of a 40-year-old computer hardware and

4 Trajectory-based operations focus on more precisely managing aircraft from departure to arrival with the benefits of 

reduced fuel consumption, lower operating costs, and reduced emissions. 5 En route airspace is high-altitude traffic typically above 10,000 feet, where aircraft reach their cruising altitudes and 

fly as direct a route as possible between their departure and destination points. 
4

software system, plus a backup, and more than 800 computer display workstations

(see figure 1) at 20 of FAA’s Air Route Traffic Control Centers (ARTCCs).6

In 2002, FAA awarded ERAM as a sole

source contract to Lockheed Martin. The 

ERAM contract is a hybrid of multiple 

contract types, including fixed-price

incentive, cost-plus-fixed-fee, cost-plusincentive-fee, and time-and-materials.

However, its software development 

portion relies primarily on a cost-plusincentive-fee (CPIF) contract type, which 

requires the Government to shoulder much 

of the program’s cost risk. The ERAM 

contract offers several types of fees to 

motivate the contractor to effectively 

manage costs, schedule, and performance. 

These fees include: (1) cost incentive fees 

for delivering contractual items below predefined cost targets, (2) schedule incentive 

fees for achieving important schedule milestones, and (3) performance incentive 

fees for meeting specific performance criteria.

Initially, FAA planned to implement both ERAM hardware and software (release 

1) beginning in fiscal year 2009 at all centers nationwide and reach operational use 

by December 2010. Software release 1 was intended to replicate the functionality 

of the current HOST system and add a few new capabilities. Software releases 2 

and 3, which would provide additional capabilities, were to become available to 

operational sites in September 2009 and September 2010, respectively, concurrent 

with release 1. FAA plans to develop a series of additional software releases 

through 2020 to enhance ERAM capabilities and support planned NextGen 

initiatives. 

In June 2005, we reported on the risks facing the ERAM program and made a 

number of recommendations, including that FAA maximize the use of fixed price 

agreements, withhold incentive payments to the contractor until it met 

Government criteria, and defer work on software development for future 

capabilities that had yet to be defined or priced.7

 6 ARTCCs, also referred to as en route centers or centers, control high-altitude air traffic nationwide. 7 OIG Report Number AV-2005-066, “FAA’s En Route Modernization Program Is On Schedule but Steps Can Be 

Taken to Reduce Future Risks,” June 29, 2005. 

Figure 1. ERAM Air Traffic 

Controller Workstation

Source: FAA
5

FAA HAS NOT FULLY RESOLVED CRITICAL ERAM SOFTWARE 

ISSUES, CAUSING SIGNIFICANT COST OVERRUNS AND 

DELAYS AND EXPOSING FUNDAMENTAL PROGRAM 

MANAGEMENT WEAKNESSES

FAA has yet to fully resolve critical software-related issues that impact ERAM’s 

ability to separate and control aircraft. These continuing problems raise concerns 

about elements of ERAM’s design, as well as the system’s backup architecture. As 

a result, ERAM’s nationwide implementation has been significantly delayed, 

resulting in increased costs. FAA now projects that ERAM will be almost 4 years

behind schedule, with an uncertain final completion date. If problems persist, cost 

increases could reach in excess of $500 million. ERAM’s problems are directly 

traceable to a number of underlying program management deficiencies that 

impede the Agency’s ability to implement ERAM and effectively manage other 

large-scale acquisitions.

Software Problems With ERAM’s Core Capability for Managing Traffic 

Persist 

FAA has been using ERAM to continuously control live traffic at Salt Lake City 

since October 2010 and at Seattle since December 2010. More recently, Denver, 

Minneapolis, and Albuquerque centers have also begun using ERAM on a fulltime basis. However, FAA has continued to identify significant software problems 

related to functions that are critical to safely separating and managing air traffic. 

These include errors that tag flight data to the wrong aircraft, incorrect display of 

flight information to controllers, and problems with aircraft hand-offs between 

controllers within a facility and between facilities with adjacent airspace. (See 

figure 2 for a data error example.)
6

Because of these problems, controllers at these two key test sites have been forced 

to rely on a large number of workarounds,8

To address these persistent problems with ERAM, FAA developed a series of 

corrective action plans, revised schedules, and milestones. The Agency also

responded with numerous ERAM software builds to correct the problems and 

spent an average of nearly $16 million per month from January through June 2011 

on these efforts. However, the total cost to complete ERAM remains uncertain. 

Table 1 chronicles ERAM’s progress and setbacks since March 2010.

temporary fixes that increase their 

workload and distract their attention away from controlling traffic. For example, if 

a data block—aircraft and flight plan information displayed to the controller—is 

paired to the wrong aircraft, a controller must manually re-enter the flight 

information for one or more aircraft. This cumbersome process increases the risk 

for data entry errors and, more importantly, takes the controller’s focus away from 

the primary task of managing and separating aircraft. FAA continues to work on 

these problems and to limit the number of workarounds.

 8 A workaround is a method or series of steps to correct or manage an ERAM software deficiency or faulty capability. 

It must be executed each time the problem occurs.

Figure 2. Example of Incorrect Data on ERAM Controller Screen

Source: FAA (textboxes added by OIG) 

Event Description: Two aircraft (tagged as AAL415 & QXE349) merge 

on the screen at different altitudes and vectors. ERAM then produces a 

“ghost” target TFC6253 (a false target). This series of non-conflicting 

events activates a conflict alert for the controller’s attention. 

Controller’s Concern: There is no reason for ERAM to cause an active 

conflict alert in any one of these scenarios; consequently, this is a 

significant distraction to the controller.
7

Table 1. Chronology of Significant ERAM Events, March 2010–

January 2012

Timeframe Significant ERAM events

Mar–Dec

2010

• FAA places moratorium on new ERAM software builds to focus on fixing the 

numerous problems affecting air traffic management and system stability. 

• ERAM achieves continuous operations at Salt Lake City, beginning on October 19.

• FAA achieves continuous operations at key test sites and conducts preliminary 

Independent Operational Assessment (IOA), a prerequisite for continuing 

deployment at additional sites. 

• Between October and December, Salt Lake City and Seattle both experience critical 

ERAM system failures caused by software problems. Seattle falls back to the legacy 

system, pending an emergency ERAM software build.

Jan–Aug

2011

• FAA’s IOA team finds that ERAM is “not operationally ready for national 

deployment.” The team determines that there are 17 hazards that must be fixed or 

mitigated before ERAM is ready for deployment to additional sites. 

• Despite the team’s warning, FAA declares prematurely that ERAM is ready for 

further deployment. FAA develops an action plan to fix or mitigate the identified 

hazards and complete initial operations at three new sites. However, FAA again 

postpones using ERAM at the new sites—even on a limited basis—due to delays 

delivering new software. 

• FAA and the National Air Traffic Controllers Association (NATCA) establish working 

groups to improve ERAM problem analysis, prioritization, and implementation. The 

workgroups call for a halt to FAA’s plans to deploy ERAM at new sites due to 

concerns about ERAM’s ability to maintain key test site operations. FAA cancels 

plans to begin operations at three new sites planned for April 2011. 

• NATCA and FAA program officials agree on a definition of ERAM’s “core 

functionality” and develop a plan to address 117 issues before restarting limited 

operations and deploying at new sites. FAA and Lockheed Martin develop and begin 

implementing software builds to address the problems.

Sept–Nov 

2011

• FAA revises the implementation schedule again and plans to begin operational use 

of ERAM at 6 new sites in December 2011—with a total of 11 planned for FY12. 

• On November 21, Salt Lake City suffers another critical software failure and falls 

back to the legacy backup system for several hours. FAA suspends an ongoing IOA, 

and transitions key test sites to an earlier ERAM software version, pending fixes. 

• Testing of an emergency software build reveals a previously undiscovered, critical 

problem, present in the software for at least 1 year. This problem could have caused 

critical failures in one or more new sites preparing to use ERAM. 

Dec 2011 –

Jan 2012

• On December 22, FAA completes IOA re-assessment, which allowed Denver, 

Albuquerque, and Minneapolis to begin using ERAM on a limited basis in late 

December. The additional three sites planned for December were rescheduled for 

January 2012. 

Source: OIG analysis
8

Between December 2011 and April 2012, after continuing to fix ongoing software 

problems, ERAM began limited operations9

ERAM’s persistent problems raise questions about whether elements of the 

system’s overall design—particularly functions related to tracking aircraft and 

displaying information to controllers—could be contributing to these problems. 

Our work on FAA’s Standard Terminal Automation Replacement System 

(STARS),

at seven additional sites—Denver, 

Minneapolis, Albuquerque, Chicago, Oakland, Los Angeles, and Houston. FAA 

program officials decided that the Agency should “pause” to assess the new sites’ 

progress toward continuous operations for the remainder of fiscal year 2012 before 

continuing deployment to other sites. This has proven to be a prudent decision in 

that Seattle experienced two new critical system failures in late March and early 

April 2012. FAA has now shifted the deployment dates for the four remaining 

fiscal year 2012 sites—New York, Indianapolis, Kansas City, and Miami—to 

fiscal year 2013. 

10 which uses the same aircraft tracking software (tracker)11 as ERAM, 

found similar problems, such as data tags detaching and not connecting with the 

correct aircraft. After we discussed these similarities with Lockheed Martin and 

FAA program officials, the Agency tasked MITRE12

ERAM’s repeated software failures and the recent discovery of a previously 

unidentified serious hidden software defect—one that could have caused a critical 

failure at one or more of the new operational sites—calls into question FAA’s 

plans to remove the legacy backup system. ERAM’s primary and backup 

channels

to examine the current 

ERAM tracker’s performance parameters and accuracy. MITRE’s interim report 

has identified additional potential issues and plans to complete its initial 

assessment in the summer of 2012. Lockheed Martin reports that it has made nine 

changes to the tracker. These tracker issues could impact FAA’s ability to 

integrate new satellite-based surveillance systems, such as ADS-B, with radar 

information for use by controllers. The performance of the ERAM tracker—and its 

integration with satellite based systems—is an important watch item for NextGen. 

13

 9 Limited operations are the initial operational use of ERAM to control live air traffic. It begins with 4- to 8-hour runs 

during weekday mid-shifts, progresses to 48-hour runs on weekends, and then transitions to weekdays and longer 

periods. Limited operations end after continuous operations are achieved, with no intent to fall back to legacy 

systems.

suffered multiple failures during live operations—such as both 

channels going down simultaneously and one channel taking down the other. Each 

10 FAA and Department of Defense air traffic controllers use STARS to control terminal traffic. The terminal 

environment controls aircraft taxiing, including departures and arrivals, up to 50 miles within an airport’s vicinity. 11 The tracker is a complex software algorithm that uses surveillance sources, such as radar or ADS-B, to display 

information on an air traffic automation system display. 12 MITRE Corporation manages a Federally Funded Research and Development Center (FFRDC) for the FAA known 

as the Center for Advanced Aviation System Development (CAASD). CAASD is a unique organization that assists 

FAA with scientific research and analysis. 13 The ERAM system operates with two identical and interlinked data paths, or channels, for processing and 

distributing flight and radar data used by the air traffic controllers. Each path provides full functionality. However, 

one is designated as primary for facilities’ general use, and the other is used as a backup for the primary.
9

failure resulted in a brief loss of air traffic control. While transitioning from the 

legacy system to ERAM, the Agency relied on a separate, dissimilar hardware and 

software system, known as the Enhanced Backup Surveillance (EBUS) system.

However, once ERAM is fully operational, FAA planned to remove EBUS, 

thereby leaving ERAM with a single backup channel. Given ERAM’s history of 

repeated system failures, many controllers, technical operations personnel, and 

testing personnel have argued that FAA should modify and retain EBUS as an 

additional backup mechanism, at least until ERAM is more mature. FAA’s 

December 2011 Independent Operational Assessment (IOA)14

ERAM’s Problems Have Caused Schedule Delays and Cost Overruns

report also 

highlighted the need for a dissimilar backup system. In response, FAA’s ERAM 

program office assessed requirements, made changes, and has deployed 

modifications needed to retain EBUS after the legacy system is decommissioned. 

FAA has stated that it is continuing to assess long-term alternatives for a 

dissimilar backup system.

ERAM’s continued problems and FAA’s difficulty in resolving them have resulted 

in significant schedule delays and cost overruns. In June 2011, FAA senior 

management decided to continue the program and rebaselined ERAM, estimating

that the cost to complete the program would increase by an additional 

$330 million. FAA is now planning to declare the system fully functional and 

operationally ready at the national level in 2014—a slip of almost 4 years 

(44 months) from the original schedule. However, FAA’s plans remain fluid and 

continue to change. 

Since rebaselining ERAM, FAA continued to identify significant software-related 

problems. Our work and MITRE’s ERAM assessment suggest that if delays 

continue, cost growth could be in excess of $500 million with completion delayed 

until 2016. ERAM will continue to face substantial risk for cost growth, schedule 

delays, and performance shortfalls as FAA fully deploys the system, especially to 

more complex sites. In particular, Initial Operating Capability (IOC) only marks 

the milestone where controllers begin to use the system on a limited basis to 

manage traffic; therefore, significant risk and work still lie ahead before full 

deployment. The ERAM Director of Program Operations stated that significant 

additional software problems are likely as more sites begin to use the system. In its 

report, MITRE also cautioned that additional time and resources could be required 

when implementation moves to larger sites.

For example, ERAM’s performance at sites such as those in Chicago and Los 

Angeles will be a driving factor for potential future delays and cost overruns. 

 14 FAA Office of Independent Safety Assessment conducted an IOA for the ERAM system, which was conducted at 

Salt Lake City Air Route Traffic Control Center (ARTCC) (ZLC) and Seattle ARTCC (ZSE) and issued its final 

report on March 18, 2011.
10

These facilities manage airspace that is divided into smaller and more heavily 

congested sectors, which could exacerbate existing problems and make 

workarounds more complicated. 

FAA’s decision to continue concurrent software development and introduce new 

capabilities, while attempting to fix problems, has further compounded difficulties 

in implementing ERAM. In 2005, we proposed that FAA focus on its primary 

objective of providing a replacement for the HOST legacy system, and wait to 

pursue additional capabilities in future developments.15

ERAM’s delays will also result in sustaining the HOST legacy system longer than 

planned, forcing FAA to incur additional costs to maintain two different 

automation systems (HOST and ERAM). HOST was originally envisioned to be 

decommissioned in December 2010, but FAA will need to maintain the system 

until at least 2014. According to FAA, costs for maintaining the HOST system 

averaged $1.4 million per month during the period of December 2010 to May 

2011. In addition, as a result of retirements and normal personnel turnover, many 

facilities face challenges in staffing maintenance technicians for HOST. As a 

result, FAA extended legacy system training at the FAA Training Academy 

beyond its original plan. FAA Training Academy officials stated that experienced 

maintenance technicians are retiring and they are having difficulty finding 

qualified instructors. Recent progress at the two key sites, Salt Lake City and 

Seattle, has allowed FAA to decommission both legacy systems at these sites. 

However, FAA and 

Lockheed Martin have continued to add new capabilities—such as those to 

support more advanced NextGen requirements—while simultaneously fixing core 

functionality issues. This overlapping software development and deployment 

increases the likelihood of introducing problems into subsequent software builds 

or of re-introducing previous problems. To mitigate these problems, FAA is 

implementing plans to more thoroughly evaluate software releases before 

deploying them.

ERAM’s Problems Are Attributable to Fundamental Breakdowns in 

Program Management 

FAA’s cost, schedule, and performance problems with ERAM are attributable to 

breakdowns in its program management, execution, and oversight. Up until 

December 2009, and well after Government Acceptance, FAA reported that 

ERAM was on track, or ahead of schedule, even though there were already serious 

problems with the program. Our work shows that FAA did not establish effective 

controls or useful milestones during ERAM’s planning and deployment stages to 

address problems and ignored early warning signs such as a higher than expected 

incidence of problem reports. As a result, when significant problems occurred, 

 15 OIG Report Number AV-2005-066, “FAA’s En Route Modernization Program Is On Schedule But Steps Can Be 

Taken To Reduce Future Risks,” June 29, 2005.
11

FAA was not well positioned to address them. The ERAM experience represents 

important lessons learned for FAA as it develops and implements even more 

complex software intensive systems for NextGen that are expected to 

fundamentally change the way air traffic is managed. For example:

• FAA underestimated the complexity of implementing ERAM and ignored 

early warning signs of trouble. FAA and its contractor significantly 

underestimated the difficulty in fielding ERAM and were overly optimistic that 

they could fully field all 20 sites in just over 1 year. Specifically, FAA and 

Lockheed Martin falsely assumed that fielding ERAM as a one-to-one 

replacement for HOST—a system that has evolved over the 40 years of its 

operational life—would be manageable. FAA also dismissed or ignored early 

warning signs of trouble during site deployment, working instead towards the 

goal of reaching milestones early. For example, ERAM achieved the Site 

Acceptance milestone at the Salt Lake City site in April 2008—8 months early, 

but with a higher than expected number of problem reports. As a result, FAA 

has stopped implementing monthly software builds twice in order to fix 

specific problems with the system, which has delayed implementation.

• FAA did not adequately test ERAM at the Technical Center prior to 

accepting the system for the Government and releasing the software to the 

key test sites. FAA allowed ERAM to successfully pass Government 

Acceptance even though testing at the Agency’s Technical Center was limited 

and did not replicate actual field conditions. Government Acceptance is 

critically important because it is the point where the Government assumes full 

responsibility for paying for and fixing any new problems that are discovered. 

FAA management did not account for testing limitations at the Technical 

Center, and as a result, the Agency lacked a full understanding of the maturity 

and stability of the software after testing.16

• FAA did not set realistic expectations regarding what would be required

to implement ERAM. The ERAM program office did not clearly 

communicate that the initial software would be relatively immature and was 

Further, FAA did not sufficiently 

test the tracker, which could be a contributing factor to ERAM’s problems to 

date. Also, robust testing with live traffic and active controllers at one or more 

of FAA’s facilities was not a prerequisite for Government Acceptance. As a 

result, the software left the Technical Center and was released to the key test 

sites with a significant, and undetermined, series of defects. FAA recognizes 

this problem, and among other things, is planning to improve the quality of 

testing by introducing a new test model and adjusting times to implement the 

test process. 

 16 For example, FAA tested and simulated the basic ERAM operating software but did not fully test the interface 

requirements for linking and processing radars, different automation platforms, or multiple airspace sectors.
12

not ready for operational use, and that site personnel and controllers would be

expected to further test, identify problems, and evaluate fixes to the software.

FAA’s failure to set realistic expectations for facility management and staff at 

the key sites negatively impacted user confidence in the system. Further, the 

use of operational runs at key test sites for additional problem identification 

and resolution was extremely labor intensive and adversely affected the site’s 

confidence in the maturity, stability, and operational suitability of the system. 

Moreover, FAA failed to convey the fact that ERAM would differ significantly 

in both appearance and function from HOST. As a result, controllers expected 

a system that would look and act like the legacy system, rather than a 

completely new system with a wide range of new capabilities. This failure to 

set clear expectations contributed to the problems the key test sites experienced 

when operational testing began, and also contributed to an unexpectedly high 

number of software problem trouble reports.

• FAA used ineffective milestones for measuring progress with ERAM. 

FAA’s key milestones for measuring progress with ERAM, such as Initial 

Operating Capability (IOC), do not accurately portray the current progress of 

the program. For example, FAA identified IOC at the two key sites as an 

important program milestone and the pivot point for further deployment. 

However, this benchmark has not proven an effective indicator of progress 

because the key sites experienced multiple failures after the milestone was 

achieved. IOC meant that the system was only ready for very limited control of 

live air traffic. Moreover, after achieving this status, the key sites began a 

measured transition from limited operations (running for a period of 4–48 

hours during off-peak times), to extended operations (running for 48 hours for 

1 to 2 weeks, 24 hours a day), and eventually to continuous operations. Yet, the 

success of this transition was site specific, as it depended on individual site 

requirements and the resolution of site-specific software problems before the 

system became fully operational. Moreover, the process of successfully 

achieving continuous operations is also site specific and requires a significant 

amount of time to achieve. Therefore, the use of IOC for tracking progress 

with ERAM gave FAA decision makers a false sense of confidence in the 

maturity of the system when in reality, much work and time still remained at 

the key sites and beyond. 

For example, on April 14, 2012, FAA declared IOC at Houston center on the 

first ERAM software release that enabled controllers to use ADS-B. However, 

software problems identified in testing have prevented the use of ERAM 

during daytime and peak operation periods and will likely result in delays in 

using ERAM for full-time operations. The problems may also delay the 

opportunity for more robust testing of ADS-B with ERAM. Houston center is 

currently using the legacy system to control air traffic that are ADS-B 

equipped and operate in the Gulf of Mexico servicing oil platforms. 
13

FAA’s use of vague, confusing, and ineffective milestones could become a 

larger concern as FAA begins tracking the progress and implementation of 

multiple interdependent programs. These include SWIM and DataComm, 

which are necessary to achieve NextGen-related capabilities. 

• FAA’s Acquisition Management System (AMS) does not adequately 

establish criteria for Government Acceptance. FAA’s AMS does not 

provide specific guidance to assist program managers in accepting large 

software-intensive programs such as ERAM from a contractor. Our discussions 

with FAA officials found that Government Acceptance is defined not by AMS 

but rather by the individual program manager for each acquisition and its 

corresponding contract. As a result, one FAA program’s definition of 

Government Acceptance can likely be inconsistent with another program’s 

definition. In the case of ERAM, Government Acceptance was declared before 

the system was adequately tested with other air traffic systems and in an 

operational environment. As a result, FAA had little understanding of what it 

would take to deploy this complex new system. 

• Problems with FAA’s management culture contributed to ERAM delays. 

ERAM’s problems indicate a management culture that did not fully 

acknowledge the extent of ERAM’s problems or effectively communicate 

problems to senior FAA management. Our discussions with FAA officials and 

staff at air traffic facilities revealed that staff and managers routinely did not 

share bad news about ERAM with FAA senior management and that senior 

program officials at the headquarters level actively withheld and suppressed 

bad news from being reported to higher levels. Further, site personnel stated 

that they felt pressure from the program office to maintain schedules and were 

uncertain of the program office’s commitments to fix discovered problems. 

These concerns were exacerbated by poor communication with the field 

regarding new software releases, discovery of new problems, and the 

recurrence of old problems that had already been fixed. Consequently, trust 

between program management and the sites deteriorated during critical periods 

of ERAM’s deployment. In response, FAA took actions to address many of 

these problems by appointing a new Director of Program Operations and 

ERAM Program Manager. Our discussions with FAA staff at several facilities 

found a consensus that the program office’s efforts have improved the 

effectiveness of decision making authority, oversight, and communications.
14

ERAM’S CONTRACT IS NOT STRUCTURED OR ADMINISTERED 

TO EFFECTIVELY MANAGE COSTS AND ACHIEVE DESIRED 

OUTCOMES 

ERAM’s problems are also attributable to weaknesses with the ERAM contract, 

which is not structured or administered to effectively manage costs and achieve 

desired program outcomes. Due to insufficient acquisition planning, FAA did not 

fully adopt best practices when designing ERAM’s contract structure that would 

have permitted more effective contract management. In addition, weaknesses in 

FAA’s acquisition workforce and poor contract management practices have led to 

insufficient oversight of the multibillion-dollar ERAM contract.

ERAM’s Contract Structure Reduced FAA’s Ability To Effectively 

Manage the Contract 

FAA did not manage ERAM’s cost, schedule, or performance effectively because 

it did not fully adopt best practices for information technology (IT) acquisitions 

when designing ERAM’s contract structure (see table 2). 

Table 2. Weaknesses in ERAM’s Contract Structure 

Best practices for structuring contracts Weaknesses in ERAM’s contract structure

Modular contracting should be used to 

divide major systems acquisitions into 

manageable contract tasks completed every 

6–12 months.

FAA did not divide ERAM into manageable 

contract segments, and it develops software 

releases concurrently, which increases 

interface and inoperability risks.

Contract Line Item Numbers (CLINs) and 

Contract Subline Item Numbers 

(SubCLINs)—with clear cost, schedule, and 

performance objectives—should be used to 

fund separate deliverables and integral parts 

of deliverables for major acquisitions.17

ERAM’s CLINs were too large, covered too 

long a time span, and were not divided 

sufficiently into SubCLINs to manage costs, 

schedule, and performance.

Scope, costs, and contract terms should be 

definitized (or finalized) in a timely manner.

FAA has not always definitized scope and 

costs in a timely manner.

Incentives should be designed to motivate 

the contractor to achieve schedule, cost and 

performance goals. These incentives should 

be awarded regularly to offer continuous 

motivation to the contractor.

FAA paid the contractor over $150M in cost 

incentives, despite software problems, delays, 

and cost overruns. Incentive fees were not 

tied to predetermined goals that are evaluated 

at regular intervals.

Source: OIG analysis

 17 A CLIN is a specific unit of work to be performed by the contractor as a pay item. SubCLINs are used to facilitate 

payment, delivery tracking, contract funds accounting, or other management purposes. Each SubCLIN has its own 

delivery schedule, period of performance or delivery date, and unit or total single price.
15

FAA Did Not Design the ERAM Contract Using Manageable Contract 

Segments

FAA did not divide ERAM into manageable contract segments—an approach 

known as modular contracting—when designing the program. However, modular 

contracting has been recommended by Federal guidance for over a decade18

While FAA partially applied modular concepts to ERAM’s software development, 

it did not design a modular contract structure with which to manage it. 

Specifically, ERAM software is built using spiral development, a “build a little, 

test a little” approach that involves incremental phases of code design. However, 

the ERAM contract does not include these incremental contract objectives and 

related cost targets or incentives, so it is difficult for the ERAM program office 

and the contractor to understand when deliverables are due and at what agreedupon cost. The lack of a modular approach can also create interface and 

interoperability risks and require more oversight of the contractor. For example, 

FAA is developing multiple software releases at once to reduce development time, 

rather than completing, implementing, and troubleshooting one software release 

before it initiates another. 

and is 

an industry best practice to reduce operational risks and better control costs. This 

approach involves dividing the work for large IT contracts into discrete, 

contractual segments with firm, short-term performance, cost, and schedule 

objectives. 

FAA acquisition officials stated that the ERAM contract was not structured in a 

more modular fashion because the task of replacing the existing HOST system was 

too monumental and complex to be delivered in the short increments required by 

modular contracting (release cycles of 6 to 12 months). FAA also said that the 

modular contracting concepts strongly recommended in 201019

Although FAA has independent procurement authority to develop its own 

acquisition process,

by the former U.S. 

Chief Information Officer (CIO) for all agencies could not have applied to the 

ERAM contract because it was awarded in 2002, prior to the CIO’s 

recommendation. 

20

 18 The Clinger-Cohen Act of 1996 states Federal agencies should use modular contracting for IT acquisitions to the 

maximum extent possible.

modular contracting has been recommended for Federal 

agencies for over 14 years. There are a number of Federal requirements and best 

practices intended to help Federal agencies manage their IT investments more 

effectively. Many of these contain modular contracting concepts that are useful in 

managing aspects of IT acquisitions structured like ERAM, such as: 

19 Office of Management and Budget (OMB) plan by Vivek Kundra, former U.S. Chief Information Officer, “25 Point 

Implementation Plan to Reform Federal Information Technology Management,” December 9, 2010. 20 In DOT’s Fiscal Year 1996 Appropriations Act, Congress provided FAA with broad authority to develop its own 

acquisition process which relieved the Agency from Federal acquisition laws or regulations. FAA established its 

AMS, a set of policies and guidance designed to address the unique needs of the Agency.
16

• The Clinger-Cohen Act of 1996 required Federal agencies to use modular 

contracting for IT acquisitions to the maximum extent possible.

• The Raines Rules, guidance enacted by OMB in 1996 under the Clinger-Cohen 

Act, specify that major information systems investments should be 

implemented in phased, successive segments.

• The Federal Acquisition Regulations (FAR) was amended in 1998 to require 

that Federal agencies other than FAA use modular contracting to the maximum 

extent possible.

• The General Services Administration implemented a “Guide for Modular 

Contracting” in 1998, which the Department of Defense has incorporated into 

its IT acquisition guidance. 

More recently, in 2010 the former U.S. CIO issued a 25-point plan for reforming 

Federal IT program management. The plan recommends that agencies only 

approve funding for major IT programs that use a modular approach since it has 

been shown to increase success and reduce risk. With regard to the development 

process, the plan also recommends that agencies lock down current software 

releases and push noncritical functionality to future releases rather than developing 

multiple releases at once.

FAA’s Contract Structure Prevented Effective Cost Tracking for ERAM 

Deliverables 

FAA’s large-scale contract structure makes it difficult to account for the individual 

factors that have driven cost overruns at ERAM test sites. FAA divided ERAM 

into CLINs that span several years. For example, FAA designed a single, large 

CLIN that now contains almost 9 years and over $1 billion of work related to 

ERAM’s software release 1. FAA included all design, development, testing, and 

implementation work in the same CLIN, rather than establishing separate CLINs 

and SubCLINs for each of these phases. FAA also did not establish separate 

CLINs for individual en route sites, along with applicable cost targets, milestones, 

and incentives. Consequently, when FAA began implementing ERAM software 

release 1 at its test site at Salt Lake City in 2008, the program began to experience 

problems that the Agency had difficulty managing because “implementation” was 

all covered under the single, large CLIN. This CLIN structure exacerbated the 

problems that led to an almost 4-year slip to the original ERAM schedule and a 

cost overrun that could exceed $500 million. 

FAA has since notified OMB that it will manage future ERAM software releases 

by breaking out periodic cost goals and frequent milestones (i.e., use smaller 

CLINs and SubCLINs). According to FAA’s ERAM Program Manager, future 

software builds can be much smaller than release 1 because the program has 
17

significantly improved its software core capabilities. This will allow FAA to track 

costs and manage the program more effectively. 

If FAA restructures the ERAM contract using smaller CLINs and sufficient 

SubCLINs beginning with release 4, we estimate that at least $157 million in 

funds could be put to better use by allowing FAA to track costs and manage the 

program’s schedule and performance more effectively.21

FAA Does Not Finalize ERAM’s Scope, Cost, and Performance Periods for 

CLINs in a Timely Manner

FAA awarded ERAM as a letter contract in December 2002, which allows the 

contractor to start work before FAA finalizes the project costs, schedule, scope of 

work, and contract terms (known as contract definitization). However, this 

contract practice increases risks and gives the contractor little incentive to control 

costs until work is definitized. FAA’s AMS does not require a specific timeframe 

for definitization. However, the ERAM contract established a timeframe for 

definitizing initial CLINs and establishing terms, such as scope and cost. In 

comparison, the FAR, which FAA does not follow, allows no more than 180 days 

for letter contract definitization.22 FAA did not meet its own contract terms or the 

FAR benchmark of 180 days for definitizing ERAM. FAA is still working to 

definitize 16 out of 57 CLINs for the contract,23

The ERAM Contract’s Cost Incentives Were Ineffective at Controlling 

Costs

even though the contractor has 

been authorized to work on them. For example, FAA initially definitized one of 

these CLINs 25 days after the date required in the contract and 48 days after 

FAR’s 180-day benchmark. However, FAA has since modified this CLIN 

45 times—increasing target costs by $328 million. 

After Government Acceptance at the key site, FAA continued to pay the 

contractor incentives, including over $150 million in cost incentives for meeting 

target costs, even though ERAM began to experience software problems, schedule 

slips, and a cost overrun of as much as $500 million. FAA’s practice of 

continually increasing target costs as ERAM’s work requirements grew and 

software problems increased negated the contractor’s incentive to manage contract 

costs. 

 21 Software release 4 is a follow-on to release 3, which focuses on adding NextGen functionality such as DataComm 

and will be funded by various NextGen programs. Funds could be used more efficiently if management took actions 

to implement and complete the recommendations. Our estimate of $157 million in funds put to better use is the lower 

end of FAA’s estimated cost range for release 4 ($157 million). We obtained these figures from FAA’s current 

ERAM CO. 22 FAR 16.603-2(c), “Letter Contracts.” 23 As of January 17, 2012.
18

In addition, FAA does not finalize incentive fees until the end of a CLIN’s 

performance period, which could span many years. This practice does not offer the 

contractor an immediate or continuous incentive. Instead, smaller incentives could 

be awarded at shorter intervals, such as bi-annually, to effectively motivate the 

contractor. 

Since FAA has had difficulty establishing cost targets due to ERAM’s changing 

requirements and software problems, award fees might provide a more effective 

cost management incentive. Award fees could be tied to meeting and exceeding 

performance goals that contribute to program success, such as achieving 

operational readiness at remaining facilities within a target date.24 OMB guidance 

states that agencies should consider using an award fee with performance 

measures designed to encourage effective cost management when it is not feasible 

to determine objective incentive fee cost targets.25

Weaknesses in ERAM’s Acquisition Workforce and Poor Contract 

Management Practices Have Led to Insufficient Contract Oversight 

Weaknesses in FAA’s acquisition workforce and poor contract management 

practices have led to insufficient oversight of the multibillion-dollar ERAM 

contract (see table 3).

 24 According to FAA’s AMS, award fees should be designed to motivate the contractor by offering additional profit for 

excellent performance in key areas that support the Government’s desired acquisition outcomes. 25 OMB guidance requires that agencies consider an incentive for performance-based acquisitions that is most likely to 

motivate efficient and economical performance and that this incentive fee should be used when a cost target can be 

pre-determined and a formula can be used to adjust the negotiated fee over an established cost range. OMB guidance 

also stipulates that an award fee with performance measures designed to encourage economical performance should 

be considered when it is not feasible to determine objective cost targets to motivate the contractor through incentive 

fees.
19

Table 3. Weaknesses in ERAM’s Contracting Staff and Management 

Practices

Requirement or best practice for contract 

management

Weaknesses in ERAM’s contract 

management

Contracting office staff should be consistent 

and have institutional knowledge of the 

contract.

ERAM has experienced high CO turnover. 

Contracting staff rely heavily on support 

contractors. 

Contract file should contain an organized 

record of all contractual actions.

ERAM’s contract files are disorganized, 

incomplete, and not centrally located. 

When reviewing invoices, the contracting 

officer’s technical representative (COTR) 

should require supporting documentation to 

prevent unallowable costs.

The COTR only performs basic checks of 

invoices and does not require supporting 

documentation for expenses. As a result, FAA 

did not detect nearly $69,000 in unallowable 

travel costs.

Program operations field managers 

(POFM), regional staff who oversee contractor 

performance at facilities nationwide, should be 

trained on monitoring contractor performance 

and be given guidance on the contract.

ERAM’s POFMs lacked training and 

guidance, increasing the risk that they could 

assign tasks that exceed contract scope and 

fail to detect performance problems.

Source: OIG analysis

High Acquisition Workforce Turnover Has Resulted in a Lack of 

Institutional Knowledge Needed for Effective Contract Oversight

ERAM has had eight COs in the past 9 years. FAA explained that it is not unusual 

for contracts as lengthy as ERAM to experience high turnover in contracting staff. 

However, because of this high turnover, ERAM’s COs lack the institutional 

knowledge needed to successfully administer the complex contract. The 

Government Accountability Office (GAO) reported in October 2011 that a 

consistent and stable contracting staff is a critical factor for successful major 

systems acquisitions.26 The effects of high turnover are exacerbated by incomplete 

and poorly organized contract files. At the time of our review, ERAM’s contract 

files did not contain a complete history of all contractual actions, as required by 

AMS guidance.27

 26 GAO Report Number GAO-12-7, “Critical Factors Underlying Successful Major Acquisitions,” October 21, 2011.

For example, FAA could not provide sufficient support for about 

$28 million in performance incentive fees paid to the contractor. The file also 

existed in three separate locations—one physical file and two virtual files. This 

makes it difficult for newer staff to readily access and understand the contract’s 

complete history. A well-maintained contract file also provides a trail of 

supporting documentation in the event of litigation, audits, or congressional 

inquiries. 

27 AMS states basic contract files containing records of all contractual actions should be maintained by the organization 

or person administering the contract. In 2006—about 4 years after FAA awarded its ERAM contract—the Agency 

required that COs use a contract file checklist of essential contracting documents and clearances intended to support 

the complete history of an acquisition from pre-award through contract closeout.
20

In addition, COs rely heavily on contract support staff. For example, during our 

review, COs could neither answer our questions about the contract nor provide all 

requested contract documents. Instead, they either referred us back to former COs 

or to FAA’s support service contractors to provide documentation. We also found 

that these support contractors write contract modifications and assist in contract 

administration—tasks which are closely associated with inherently governmental 

functions. 

The program’s COTR also heavily relies on FAA’s support service contractors for 

contract administration. For example, the COTR delegated the task of maintaining 

the contract correspondence file to the support service contractors, even though the 

COTR’s official designation letter indicates that the task cannot be delegated. In 

addition, the COTR said that, because of the size of the ERAM contract, he is not 

able to perform full reviews of contractor invoices or validate whether the 

contractor actually performs the work billed. FAA’s new ERAM Program 

Manager stated that the COTR—rather than performing full reviews—currently 

“spot checks” all invoices to ensure that hours, charges, and details reflect actual 

work performed. The COTR also assesses whether the work billed seems 

reasonable. However, even though the ERAM contract allows FAA to request 

supporting documentation, the Agency still does not require the contractor to 

routinely provide supporting documentation for expenses, such as travel vouchers 

or receipts. Consequently, FAA’s invoice review procedures were not sufficient to 

identify some unallowable charges. 

Prior audits have detected weaknesses in Lockheed Martin’s travel expense 

policies and identified instances of unallowable ERAM travel costs. For example, 

a 2006 internal audit found that a Lockheed Martin employee working on ERAM 

had been paid lodging expenses for 2 days he stayed at home.28 In addition, 

Lockheed Martin recently self-reported that a former software engineer submitted 

almost $69,000 in fraudulent travel expenses between February 2008 and 

December 2010.29

FAA also relied on POFMs, who are primarily systems engineers, to oversee the 

contractor’s performance at the Agency’s 20 nationwide en route centers where 

ERAM software is being implemented. Yet, the majority of POFMs we spoke to 

had not been given training as technical officer’s representatives (TOR) or 

guidance on the contract. For example, at one facility, the POFM had reported 

Despite these findings, FAA still does not require supporting 

documentation for all expenses. ERAM’s ongoing history of unallowable travel 

costs and FAA’s insufficient invoice review procedures creates the potential that 

FAA may have unknowingly paid other unallowable expenses.

 28 In response, FAA’s then-CO began requiring the contractor to submit supporting documentation to verify billed 

costs. However, Lockheed Martin did not comply with the CO’s request, as FAA did not enforce it. The following 

ERAM CO decided to retract this supporting documentation requirement in 2008—around the time FAA began 

incurring what became nearly $69,000 in fraudulent travel expenses. 29 The sum includes only direct costs and excludes indirect costs.
21

problems with contractor performance, but he did not properly document them, so 

FAA did not take action. Many POFMs we spoke to also reported that they had 

not seen the ERAM contract’s Statement of Work nor been given guidance on 

what they are allowed to assign the contractor, creating the potential that POFMs 

could assign tasks that exceed the contract’s scope and budget. 

FAA has taken actions to address some of these acquisition weaknesses. For 

example, in response to OIG audit findings on FAA’s acquisition workforce,30

FAA recently required all TORs—a term that also applies to POFMs—to train and 

certify as COTRs.31

FAA Has Not Used Contract Management Tools Effectively

In addition, FAA has appointed a new Acquisition Executive 

to oversee the Agency’s Office of Acquisition and Business.

We also found that FAA has not correctly implemented management tools for 

ERAM, which are intended to improve the management of costs, schedule, 

performance, and risk (see table 4).

Table 4. Ineffective Use of Management Tools

Effective use of management tools

Ineffective use of ERAM’s management 

tools

Earned Value Management (EVM) systems 

must include all work to completion for 

accurate forecasts of schedule and cost 

trends.

ERAM’s EVM system does not include all 

work, so forecasts are inaccurate and do not 

help detect problems with the program.

Integrated Baseline Reviews (IBRs) should 

be performed early to ensure adequate 

planning. 

FAA did not complete IBRs for four of its five 

largest contract modifications—each 

exceeding $100M.

Risk management process should provide 

early detection of risks.

Problems were discovered at key sites in 

June 2009, yet ERAM’s risk management 

process did not detect significant risks until 

almost 2 years later. 

Source: OIG analysis

• EVM. FAA incorrectly implemented ERAM’s EVM system,32

 30 OIG Report Number ZA-2011-148, “FAA’s Policies and Plans Are Insufficient to Ensure an Adequate and Effective 

Acquisition Workforce,” August 3, 2011.

a management 

tool intended to forecast performance trends and help managers identify cost 

31 In April 2011, FAA eliminated the terms “Task Order Representative” from its AMS. All persons appointed by the 

CO to support contract administration are now designated COTRs. 32 To set up an EVM system, an integrated cost and schedule baseline is developed by time-phasing budget resources 

for defined work. As work is performed and measured against the baseline, the corresponding budget value is 

“earned.” Using this earned value metric, cost and schedule variances are identified and analyzed to forecast whether 

the program will meet desired budget and schedule goals at completion.
22

and schedule problems early. According to national EVM standards,33 EVM 

systems should compare performance against a baseline, which should include 

all authorized work for the program. FAA program officials stated that the 

EVM baseline for ERAM was based on the contract’s baseline and work 

breakdown structure,34

• IBRs. FAA did not complete timely IBRs for ERAM. Both OMB and AMS 

require IBRs, which are contract management tools intended to improve 

program performance. Specifically, an IBR is an evaluation of a program’s 

baseline plan to determine whether all program requirements have been 

addressed, risks have been identified, mitigation plans are in place, and 

resources are sufficient to complete the work. However, we found that FAA 

did not conduct its initial IBR for ERAM until 337 days after contract award. 

Although AMS includes some high-level policy on IBRs, it does not specify a 

timeframe for conducting IBRs. FAA also lacked guidance at the time of 

ERAM’s contract award on how to implement AMS IBR policy. In contrast, 

guidance from other agencies, such as DOD, requires IBRs within 180 days of 

contract award. We also found that FAA did not complete IBRs—and thus did 

not complete adequate planning and risk assessments—for four out of five of 

its largest contract modifications, each exceeding $100 million. DOD’s 

acquisition regulations,

rather than those for the overall program, as required by 

EVM standards. Despite incorrect implementation, FAA maintains that EVM 

was an accurate management tool for ERAM until the program began 

experiencing significant software problems after Government Acceptance at 

the initial key test site. However, these software problems required significant 

additional work, which FAA never accounted for in the EVM baseline. 

Without an accurate baseline that includes all required work, ERAM’s EVM 

system has not identified significant problems with the program. For example, 

although ERAM is almost 4 years behind and may be as much as $500 million 

over budget, FAA’s March 2011 EVM report stated that “all ERAM 

milestones to date have been achieved on or ahead of schedule, while meeting 

cost targets.” Three months after this EVM report, FAA rebaselined ERAM 

due to cost overruns and delays. FAA stated that it is planning to revise 

ERAM’s EVM baseline to incorporate all of the program’s work requirements. 

35

 33 OMB and AMS require major programs like ERAM to base their EVM systems on American National Standard 

Institute (ANSI)/Electronic Industries Alliance (EIA) Standard-748-A. The ANSI/EIA-748 guidelines describe the

attributes of an effectively integrated cost, schedule, and technical performance management system.

on the other hand, require IBRs within 180 days of 

major modifications. In 2008, FAA developed a “Program Level Integrated 

Baseline Review Guide,” and its 2010 update requires program offices to 

conduct IBRs for sole source contracts, such as ERAM, prior to definitization. 

34 A work breakdown structure (WBS) is a hierarchically structured grouping of project elements that organizes and 

defines the total scope of the project. Each descending level is an increasingly detailed definition of a project 

component. Project components may be projects (a product-oriented WBS) or tasks (a task-oriented WBS). 35 Defense Federal Acquisition Regulation Supplement (DFARS) is DOD’s implementation and supplementation of the 

FAR.
23

However, the guidance does not specify a timeframe to submit IBRs for major 

contract modifications, so FAA can still approve major contract modifications 

without sufficient planning.

• Risk Management Process. FAA did not sufficiently identify ERAM’s risks 

early in the program, even though FAA developed comprehensive risk 

management process guidance in 2002 that would have applied to ERAM. In 

addition, DOD’s risk guidance states that risk management should start as early 

as possible to avoid the greatly increased costs of addressing risks later. 

Significant problems were discovered at key test sites in June 2009,36

ERAM’s risks contributed to significant cost and schedule problems that required 

the FAA Administrator to consider whether to terminate ERAM, in accordance 

with Public Law.

yet 

FAA’s risk management process for ERAM did not detect significant risks 

until January 2011—almost 2 years later. Additionally, FAA had initially 

identified only four “medium” risks and one “low” risk, despite ERAM’s 

considerable cost overruns, schedule delays, and software problems. FAA’s 

new project manager developed a June 2011 risk assessment that provided a 

more accurate portrayal of ERAM risks; it identified 28 active risk areas 

including 12 “high” risks. FAA reports indicate that it has addressed some of 

these risks. Specifically, FAA has reduced the number of ERAM’s risks from 

28 to 16, as of October 2011.

37 Ultimately, FAA decided not to terminate, in part because the 

Agency had already planned to rebaseline the program.38

 36 FAA had received even earlier warnings that it would need to carefully monitor risks for ERAM, as GAO 

highlighted FAA’s air traffic control modernization programs as high-risk for 14 years, from 1995 to 2009.

In June 2011, FAA’s 

Joint Resources Council (JRC)—a body responsible for making authorization and 

funding decisions—approved rebaselining the ERAM program. As a result, the 

program’s cost variance now exceeds its original baseline by more than 

15 percent, and the schedule is delayed by 48.9 percent. However, further 

implementation delays could place the Agency in the position of once again 

having to make a determination about program termination. Specifically, the law 

requires the FAA Administrator to terminate programs funded from Facilities and 

Equipment appropriations with cost or schedule variances of 50 percent or more, 

unless the Administrator decides to continue the program and submits the basis for 

its decision to the House and Senate. ERAM’s continuing delays may soon 

increase the program’s schedule variance above this 50 percent threshold. 

37 Public Law 104-264, “Federal Aviation Reauthorization Act of 1996,” October 9, 1996—Requires the FAA 

Administrator to consider terminating any major acquisition—specifically, those funded from Facilities and 

Equipment appropriations—that has cost or schedule variances of over 10 percent or fails to achieve at least 90 

percent of the performance goals established for the program. It also requires the FAA Administrator to terminate 

programs that have cost or schedule variances of over 50 percent or fails to achieve at least 50 percent of the 

performance goals established for the program, unless the Agency decides to continue the program and submits the 

basis for its decision to the House and Senate to continue the program. 38 JRC Meeting Minutes, dated March 16, 2011.
24

CONTINUED PROBLEMS WITH ERAM POSE RISKS TO 

NEXTGEN INITIATIVES

Continued problems with ERAM will have a cascading effect on FAA’s NextGen 

efforts and the Agency’s capital account now and well into the foreseeable future. 

ERAM is a critical step on the path to NextGen and is the bridge for future 

NextGen automation capabilities for controllers. ERAM has complex 

interdependencies with key NextGen initiatives that are required for 

fundamentally changing the way air traffic is managed in the United States. 

Problems with ERAM will impact many aspects of NextGen, including FAA’s 

ability to deliver new Performance-Based Navigation routes and transition to the 

common automation platform envisioned for future NextGen capabilities.

Despite the significant program risks and unresolved issues, FAA has not 

conducted a detailed assessment of ERAM’s interdependencies or impact on other 

programs’ costs and schedules. Our analysis shows, however, that three of the 

largest and most complex NextGen transformational programs are dependent on 

the successful implementation of ERAM to meet their performance parameters. 

These programs—ADS-B, DataComm, and SWIM—are allocating more than 

$500 million specifically to integrate with ERAM. The following table details 

individual program interdependencies to ERAM:

Table 5. ERAM Interdependencies With Key NextGen Programs

Program Description ERAM Interdependencies

Automatic Dependent Surveillance-Broadcast (ADS-B)

Uses aircraft avionics and ground-based systems 

to provide information on aircraft location to pilots 

and traffic controllers. 

FAA plans to provide the ERAM program $74M to 

display ADS-B data for use by controllers in the highaltitude environment. 

Data Communications (DataComm)

Provides two-way data communication between 

controllers, automation platforms, and flight crews. 

DataComm is intended to supplement rather than 

replace voice communications in all phases of 

flight. 

FAA plans to provide the ERAM program with as 

much as $400M to develop an interface that provides 

controller-pilot message processing and displays 

information to controllers in the en route centers. 

System-Wide Information Management (SWIM)

Provides a more agile exchange of information 

through a secure, NAS-wide information web that 

will connect FAA systems and improve interaction 

with other agencies, air navigation service 

providers, and airspace users. 

FAA planned to provide the ERAM program with as 

much as $117.7M (for SWIM Segment 1 only) to 

modernize and enhance its flight data processing 

and external interfaces with terminal air traffic control 

and the Traffic Flow Management systems. However, 

the SWIM Program Office terminated Segment 1 

early and has only provided $41 million to ERAM.

Source: OIG analysis of FAA documents 
25 

$11.63

$18.91

$13.14

$10.43

$26.27

$13.76 $15.69

$4.13

$5.44

$8.46 $12.26

$8.75

$4.98

$7.34

0

5

10

15

20

25

30

35

40

Jan Feb Mar Apr May Jun Monthly average

Dollars inmillions

Funded from ERAMprogram Funded from other NEXTGEN programs

The importance of ERAM to NextGen execution is illustrated by the fact that FAA 

is investing hundreds of millions of dollars on ERAM to interface with other 

systems that are critical to achieve new NextGen capabilities. For example, a key 

component for realizing benefits from ADS-B is displaying satellite-based 

information on air traffic controller displays. Between January and June 2011, the 

ERAM Program Office spent about $7 million a month in addition to ERAM 

program costs for work to integrate other systems, such as ADS-B, SWIM, and 

DataComm, into ERAM. The following figure shows monthly expenditures on 

ERAM and the work to integrate the transformational systems with the new 

system for managing high-altitude air traffic. 

Figure 3. ERAM Monthly Expenditures Funded by ERAM and Other 

NextGen Programs for the First Half of 2011 

Source: FAA ERAM program office

In addition to these three transformational programs, delays with ERAM will 

impede other NextGen efforts, including: 

 The implementation of FAA’s new Performance-Based Navigation routes and 

procedures that allow aircraft to fly more flexible routes, based on aircraft 

avionics and satellite-based navigation. New performance-based navigation

routes are an important stepping stone for near-term NextGen initiatives and

boosting capacity at already congested airports. New automated systems for

controllers, such as ERAM, are key to maximizing the benefits of new routes.
26

• Trajectory-based operations, an important NextGen capability, focus on the 

management of aircraft through all phases of flight. This capability is expected 

to predict the path of each aircraft in time and space to facilitate the transition 

from today’s ground-based radar to more accurate satellite-based systems that 

will result in better strategic management of the entire NAS and reduce fuel 

consumption by the airlines and aircraft emissions. Progress with ERAM is 

important with trajectory-based operations because FAA plans to begin to 

implementation of this capability in the high-altitude environment.

• Future software enhancements for new NextGen capabilities estimated to cost

close to $1 billion through fiscal year 2017, including a flexible and dynamic 

airspace that will allow controllers to shift airspace segments to other 

controllers, based on weather and traffic pattern changes. However, FAA must 

fix core capabilities for managing aircraft before the new capabilities can be 

implemented.

• Common automation platform, which will combine both terminal and en route 

operations into a common automation system. Currently, FAA operates and

maintains a diverse system of automation systems with unique displays, 

software, and hardware. FAA believes that a common automation platform will

reduce costs, improve air traffic and airspace management, and allow the 

Agency to consolidate and realign its facilities. The problems with ERAM are

one reason why FAA cannot determine when it can begin to develop and 

transition to a common automation platform.

Schedule delays and corresponding cost growth with ERAM have forced FAA to 

reprogram funds from other FAA capital programs. According to Agency officials 

responsible for capital planning and budgeting, FAA thus far has reallocated funds 

from the NextGen solution sets (development efforts for NextGen capabilities and 

procedures), tower replacement, electrical power systems for air traffic control 

facilities, and planned technical improvements to communications and oceanic 

automation systems. As we have noted in our previous work, continuing cost 

growth with ERAM, especially in the current budget environment, will crowd out 

other capital programs. 

CONCLUSION

ERAM is a critical component of FAA’s efforts to modernize the National 

Airspace System and transition to NextGen. While FAA has taken steps to 

improve program management, significant challenges and risks remain to 

successfully complete the program. In addition, FAA’s problems with ERAM 

have revealed weaknesses with the contract vehicle, highlighting the need to 

improve the management of the contract as the program moves forward. Sustained 

comprehensive actions will be required to manage ERAM and its contract, achieve 
27

full implementation of ERAM at all facilities, and gain user confidence in the 

system. Ultimately, achieving NextGen’s goal of more efficient airspace for the 

future will depend on FAA’s ability to effectively manage within cost and 

schedule large-scale acquisitions such as ERAM to support its NextGen portfolio. 

Until FAA improves its acquisition, program, and contract management practices, 

the Agency will likely find itself repeating its past and current missteps in future 

large-scale software-intensive NextGen acquisitions, putting the future of 

NextGen at risk. 

RECOMMENDATIONS

To reduce risk with further implementation of ERAM and to address 

programmatic management weaknesses, we recommend that the ERAM program 

office: 

1. Develop a mitigation plan to address ERAM’s core capabilities problems 

at all 20 ERAM sites before deploying new capabilities. 

2. Evaluate available options and take action to deploy an additional backup 

for ERAM until the system has become significantly more mature.

To improve ERAM’s contract structure and oversight, we recommend that FAA’s 

Acquisition Executive:

3. Revise the Contract Line Item Number (CLIN) structure to more 

effectively track ERAM costs. This should include establishing 

subordinate CLINs, cost targets, and incentives to better achieve program 

objectives, beginning with software release 4. 

4. Include a requirement in the Acquisition Management System (AMS) to 

definitize CLINs in a reasonable time period, such as FAR’s 180-day 

benchmark. Ensure that future ERAM CLINs are definitized according to 

the new requirement.

5. Design incentives to better achieve desired program outcomes. For 

example, offer incentives over shorter intervals, such as bi-annually, to 

effectively motivate the contractor.

6. Review AMS requirements for a contracts file list and contract 

maintenance procedures to verify that they are adequate. In addition, 

develop a process to verify that major contract files are reviewed by 

FAA’s National Acquisition Evaluation Program for compliance with 

AMS policy and best practices for contract management.
28

7. Develop a formal process for ERAM invoice reviews that requires 

supporting documentation, such as travel vouchers and hotel receipts.

8. Update the performance measurement baseline for ERAM’s earned value 

management system to include all remaining work on the ERAM contract, 

including planned work that has not yet been priced and work performed 

by the Government.

9. Develop procedures in FAA’s “Program Level Integrated Baseline Review 

Guide” to verify that integrated baseline reviews meet the requirements 

and to establish a time frame for conducting integrated baseline reviews 

after executing major contract modifications. 

10. Complete the comprehensive risk management guidance that FAA is 

currently developing, to more effectively manage acquisition risks.

To reduce risk to future NextGen related programs, we recommend that FAA:

11. Assess current testing capabilities and limitations at FAA’s Technical 

Center and develop corrective action plans to more robustly test future 

complex software-intense air traffic systems.

12. Require complex software-intensive systems (that are interdependent on 

other systems, such as ERAM) to be successfully tested in a live, 

operational environment, at one or more FAA air traffic facilities, prior to

Government Acceptance.

13. Revise AMS to better define key milestones, such as Government 

Acceptance and initial operating capability, so that milestones are clear 

measures of progress for managing major acquisitions.
29

AGENCY COMMENTS AND OFFICE OF INSPECTOR GENERAL 

RESPONSE 

We provided FAA with our draft report on June 13, 2012, and received its formal 

response on July 24, 2012. FAA’s response is included in its entirety as an 

appendix to this report. In its response, FAA states that it has made substantial 

progress with ERAM, and Agency management is confident that the program can 

stay within the revised cost and schedule baseline. We agree that ERAM is on 

stronger footing now than when we began our review. We attribute this to 

sustained management attention by FAA leadership as well as focused risk 

management and close work with controllers. Throughout our review, we 

communicated our views to FAA officials on actions needed to reduce risk and 

strengthen contract oversight, and the Agency took steps to address our concerns. 

Our draft report acknowledged that FAA has made strides toward improving the 

way it tracks ERAM deliverable costs. For example, we reported that FAA 

indicated that it will manage future software releases by breaking out periodic cost 

goals into more discrete contractual segments and establishing frequent 

milestones. In May 2012, in response to our concerns, FAA modified the ERAM 

contract to begin definitizing its software implementation effort and to break large 

software development efforts into smaller contract segments. This will allow FAA 

to better manage its software releases. 

Notwithstanding FAA’s risk mitigation efforts and the use of the system at more 

facilities, considerable risk still lies ahead for completing ERAM within the 

revised baseline. For example, as we have stated previously, when FAA fields 

ERAM at large sites, such as New York and Washington—which are more 

complex than any of the previous locations—it will likely identify new problems, 

raising the risk that program costs will grow. Therefore, it is uncertain how much 

ERAM will ultimately cost, how long it will take, what capabilities will be 

delivered, and what trade-offs will be needed to complete deployment. The ERAM 

experience represents important lessons learned for FAA as it develops and 

implements even more complex software-intensive systems envisioned for 

NextGen. 

In responding to our recommendations, FAA concurred with 12 of our 

13 recommendations and partially concurred with 1. Based on FAA’s response, 

we believe the Agency met the intent of nine recommendations, but we are 

requesting additional information, revised responses, and/or targeted completion 

dates for recommendations 1, 7, 8, and 12, as detailed below. All 

recommendations will remain open pending completion of planned actions. 

FAA concurred with recommendation 1, to develop a mitigation plan addressing 

ERAM’s core capabilities problems at the 20 sites, and asked that we close the 
30

recommendation. However, FAA continues to add new capabilities while fixing 

problems identified at the sites. As we noted in our report, this has compounded

problems with ERAM’s implementation—an issue that FAA recently experienced 

at Houston center, which hindered efforts to use the system on a full-time basis. 

Therefore, we request that FAA provide us with its plans to mitigate risks and 

related milestones for deploying ERAM to the remaining sites, such as New York.

FAA concurred with recommendation 7, stating that controls are in place to ensure 

that its “spot-checking” approach for reviewing invoices meets the criteria for 

balancing the effort of the reviews with the risks for potentially fraudulent 

charges. However, the intent of our recommendation was to ensure that FAA’s 

invoice “spot checking” reviews include reviewing supporting documentation for 

travel costs, such as travel vouchers and hotel receipts. To mitigate the incidence 

of additional fraudulent travel charges billed to the ERAM contract, we are 

requesting that FAA clarify whether its “spot-checking” approach will include a 

review of supporting documentation for billed travel costs, such as travel vouchers 

and hotel receipts.

FAA partially concurred with recommendation 8. By September 30, 2012, FAA 

plans to restructure its EVM measurement baseline to adopt a product oriented 

approach that is aligned to program milestones, such as achieving IOC at a 

specific en route site. This will improve the reliability of measurements of cost and 

schedule variances impacting the program. In addition, although FAA agrees that 

planned future work should be included in the EVM measurement baseline, as 

required by FAA’s acquisition guidance, it has not yet determined whether or how 

it plans to implement this requirement for ERAM. For example, FAA is unsure 

whether to include planned work in the EVM system for ERAM when 

programmatic work is unstable or programmatic requirements and milestones are 

not mature. OMB guidance incorporates national EVM standards that state EVM 

systems should compare performance against a baseline that includes all 

authorized work for the program. Accordingly, the recommendation is unresolved 

pending FAA’s determination of whether it intends to include all planned 

authorized work in the EVM measurement baseline. 

FAA concurred with recommendation 12, but its response does not meet the intent 

of the recommendation. FAA states that AMS prescribes an approach that supports 

the “notion” that complex capabilities must be successfully tested in a live, 

operational environment at one or more facilities before Government Acceptance. 

However, as we note, live operational testing did not occur with ERAM before the 

Government accepted the system and assumed the responsibility to fix ERAM’s 

problems. This was partly due to the fact that the Government Acceptance 

milestone is inconsistently defined and interpreted by individual program 

managers, as we reported. Therefore, we are asking FAA to reconsider its 

response and provide more specific information on its planned actions and target 
31

completion dates to not only require, but also ensure that complex softwareintensive systems are tested in a live operational environment at one or more FAA 

air traffic facilities prior to Government Acceptance. 

ACTIONS REQUIRED 

We consider recommendations 2, 3, 4, 5, 6, 9, 10, 11, and 13 resolved but open 

pending the completion of planned actions. For recommendations 1, 7, 8, and 12, 

we request that FAA provide target milestones, additional information, and/or 

reconsider its response, as specifically detailed above. In accordance with 

Department of Transportation Order 8000.1C, we request that FAA provide us this 

additional information within 30 days of this report.

We appreciate the courtesies and cooperation of Department of Transportation 

representatives during this audit. If you have any questions concerning this report, 

please call Jeff Guzzetti, Assistant Inspector General for Aviation and Special 

Programs, at (202) 366-0500, or Mary Kay Langan-Feirson, Assistant Inspector 

General for Acquisition and Procurement Audits, at (202) 366-5225. 

#

cc: FAA Audit Liaison, AAE-100

OST Audit Liaison, M-1
32

Exhibit A. Scope and Methodology

EXHIBIT A. SCOPE AND METHODOLOGY

We conducted this performance audit between September 2010 and June 2012 in 

accordance with generally accepted Government auditing standards as prescribed 

by the Comptroller General of the United States. Those standards require that we 

plan and perform the audit to obtain sufficient, appropriate evidence to provide a 

reasonable basis for our findings and conclusions based on our audit objectives. 

We believe that the evidence obtained provides a reasonable basis for our findings 

and conclusions based on our audit objectives. 

We conducted onsite interviews with management officials, technical operations 

personnel, air traffic controllers, contract oversight staff, and contractor staff at 

FAA’s two key test sites, as well as at five selected en route centers that were 

planned to begin initial operations in fiscal year 2011. We visited the FAA 

Technical Center in Atlantic City, NJ, to examine software testing, discuss the 

Center’s testing limitations, and interviewed the manager of ERAM’s contract 

oversight staff. We interviewed FAA program officials at FAA Headquarters, 

National Air Traffic Controllers Association (NATCA) union officials, 

Professional Aviation Safety Specialists (PASS) union officials, and officials from 

Lockheed Martin and Raytheon to discuss FAA’s implementation of ERAM. We 

interviewed ERAM contracting staff and support contractors to discuss contract 

oversight. We met with training officials at FAA Headquarters and FAA Training 

Academy in Oklahoma City, OK. We interviewed FAA’s senior officials, Air 

Traffic Organization officials, Joint Planning and Development Office officials, 

and NextGen Integration and Implementation officials to discuss the impact of 

ERAM delays on NextGen initiatives. We spoke with the Air Traffic 

Organization’s Office of Safety about its ERAM assessment. Finally, we met with 

MITRE officials to discuss its ERAM assessment and determine how FAA should 

align its future investments as its move forward with NextGen. Exhibit B lists all 

organizations we contacted during this audit. 

In addition to conducting interviews, we also reviewed relevant policies and 

guidance from the Office of Management and Budget, the Government

Accountability Office, FAA’s Acquisition Management System, and the 

Department of Defense, as well as other applicable laws and regulations. We 

collected and analyzed program data, FAA’s National Airspace System and 

NextGen Enterprise Architecture documents, cost and schedule projections for 

ERAM and other NextGen programs, problem trouble reports, the ERAM 

contract, and other pertinent documents and records. Our assessment of the ERAM 

contract included reviews of contract terms, contract modifications, contract costs, 

incentive fees, and pre-award contract documents. We also reviewed FAA’s use of 

program management tools, such as earned value management and integrated 

baseline reviews. Further, we reviewed all 81 bi-weekly invoices for onsite 
33

Exhibit A. Scope and Methodology

maintenance support to determine if FAA required enough supporting 

documentation to detect unallowable costs. 
34

Exhibit B. Organizations Visited or Contacted

EXHIBIT B. ORGANIZATIONS VISITED OR CONTACTED

FAA Facilities

FAA Headquarters

Denver Air Route Traffic Control Center

Salt Lake City Air Route Traffic Control Center

Seattle Air Route Traffic Control Center

Minneapolis Air Route Traffic Control Center

Albuquerque Air Route Traffic Control Center

Houston Air Route Traffic Control Center

Los Angeles Air Route Traffic Control Center

Chicago Air Route Traffic Control Center

Oakland Air Route Traffic Control Center

FAA Training Academy at the Mike Monroney Aeronautical Center 

FAA Accounts Payable Office at the Mike Monroney Aeronautical Center

FAA William J. Hughes Technical Center

Other Organizations

Lockheed Martin Information Systems & Global Solutions

Raytheon Network Centric Systems

MITRE Corporation

National Air Traffic Controllers Association (NATCA)

Professional Aviation Safety Specialists (PASS)

CEXEC, Inc.

TASC, Inc.

Evans Incorporated
35

Exhibit C. Major Contributors to This Report

EXHIBIT C. MAJOR CONTRIBUTORS TO THIS REPORT 

Office of Aviation and Special Program Audits 

Barry DeWeese Program Director 

Joseph Hance Project Manager 

Sean Woods Senior Auditor (Lead)

Won Kim Senior Auditor

Vicki Smith Analyst

Arthur Shantz Technical Advisor

Audre Azuolas Writer-Editor

Office of Acquisition and Procurement Audits 

Terry Letko Program Director

Dory Dillard-Christian Project Manager

Stacie Seaborne Senior Analyst

Jill Cottonaro Senior Analyst

Amanda Watson Senior Auditor

Rachel Miller Senior Auditor

David Lahey Auditor

Meghann Noon Auditor

Patti Lehman Auditor

Christina Lee Analyst and Writer-Editor
36

Appendix. Agency Comments

APPENDIX. AGENCY COMMENTS

Federal Aviation

Administration

Memorandum

Date: July 24, 2012

To: Jeffrey B. Guzzetti, Director, Assistant Inspector General for Aviation and 

Special Program Audits

From: H. Clayton Foushee, Director, Office of Audit and Evaluation, AAE-1

Subject: FAA’s Response to the Office of Inspector General Draft Report on En Route 

Automation Modernization (ERAM) program 

The Federal Aviation Administration (FAA) has made substantial progress on the ERAM 

program beyond that conveyed in the Office of Inspector General (OIG) draft report. The 

FAA has already implemented several of the OIG’s recommendations, and the FAA 

disagrees with the draft report’s statements that the program is likely to incur over $150M 

in costs above its new baseline and delays into 2016. Moreover, the FAA finds it 

disconcerting that the OIG draft report failed to recognize or acknowledge the substantial 

recent progress made by the agency to place the ERAM program back on a strong 

footing, and some of the OIG’s findings are based upon out of date information.

The original ERAM program baseline has been modified to include a $330M cost 

variance and a three-year, eight-month schedule variance. However, the program was 

re-baselined in June 2011 and is currently operating within that new baseline. The last 

site Operational Readiness Date (ORD) milestone shifted from December 2010 to August 

2014. 

Although the agency will continue to adjust deployment dates for individual sites within 

the approved baseline, there is a high degree of confidence in the program’s final 

completion date. As anticipated, since re-baselining the FAA has also continued to 

identify software related issues that affect ERAM performance, particularly as new 

capabilities are implemented, and more sites are brought on-line. The FAA anticipates, 

however, that software fixes will be implemented within the boundaries of the current 

approved baseline.
37

Appendix. Agency Comments

The OIG report should recognize a number of major deployment milestones that have 

now been accomplished, including: 

• Achieving Initial Operating Capability (IOC) at seven more sites, bringing the current 

total to nine sites.

• Achieving ORD at two of those nine sites, and achieving continuous operations at 

another three (meaning they are operating on ERAM with no planned fall-back to the 

legacy HOST system), while the remainder continue to work through the progression 

of longer and longer operational runs toward continuous operations.

• Declaring IOC on the first ERAM software release enabled with Automatic 

Dependent Surveillance Broadcast (ADS-B), allowing for operational use of both 

ERAM and a key NextGen program in Houston.

• Decommissioning of the legacy HOST system at Seattle and Salt Lake Centers.

The growing level of ERAM-enabled operations has led to multiple instances where 

nearly one half of the nation's air traffic was being served by ERAM-based air traffic 

control procedures. Since December of 2011, the system has accumulated over 2,600 

hours of operations for newly IOC'd sites other than Seattle and Salt Lake, across a range 

of varying airspace needs and traffic volumes. The program is well positioned to 

continue to activate sites within the current budget and schedule as planned into FY13.

The ERAM acquisition team now maintains firm oversight and control of the program. 

Through key management changes of personnel overseeing the project, including the 

appointment of a new Contracting Officer in October 2011, FAA has reinforced its ability 

to oversee and administer all contract activities. Over the last six months, the team has 

negotiated and definitized a number of significant undefinitized contract actions valued at 

over $374M. Portions of the negotiations included establishing a more modular contract 

structure while incentivizing performance. Contract provisions have been established to 

track performance and formally measure the results. The associated performance 

incentives are detailed in the contractually established release plan. This plan delineates 

all of the activities required to meet the contract schedule and program requirements.

The shared obligations of such activities force both the FAA and the contractor to track 

progress and limit change, and all changes now have a directly traceable impact on the 

software release plan and ability to achieve each milestone. The milestones are directly 

tied to the contract incentives and the ability of the contractor to earn the negotiated fee 

for performance. These activities have occurred in conjunction with the planning and 

developing of acquisitions and documents in support of future software releases and 

various Contractor Depot Level Support (CDLS) activities.

Beginning in early 2011, the Program Office has undertaken a series of management 

initiatives that are also helping to get the program back on track. This includes 

addressing strategic, structural, process, personnel, and incentive aspects of the program’s 

overall approach. Specific examples in each of these areas are highlighted below, and 

have been implemented in the past 18 months. These improvements are reinforced here 
38

Appendix. Agency Comments

to emphasize the differences between where the program currently stands and many of 

the challenges highlighted in the report:

- Strategy: The program has addressed strategic changes that create a formula for 

success. This new strategic direction is driven by the following program artifacts:

the In-Service Decision (ISD) Action Plan, the ERAM Process Improvement Plan, 

the updated Office of Management and Budget Improvement plan, and the artifacts 

used in conjunction with the re-baselining of ERAM by the Joint Resources Council.

In an effort to enhance the focus on operational needs and improve organizational 

integration with 2nd Level Support, the work to transition Program Office ownership 

to 2nd Level has been accelerated (began in October 2011). Furthermore, a revised 

waterfall schedule was developed to allow the time and attention necessary to 

efficiently and effectively address facility needs. Based on the outcomes highlighted 

earlier in this document (i.e. accumulation of IOC and ORD milestones), the FAA 

feels confident this strategic direction is the correct one.

- Structure: The Program Office redefined the specific program structure with a rebaseline of funding and schedule. New program governance was put in place in early 

2011 which included the labor unions (Article 48 working group), a steering 

committee and regular Program Management Reviews that include a cross-section of 

key program stakeholders. There were also several process improvements made 

within the program’s communications and management. For example, there was a 

realignment of personnel for a more streamlined approach and new communication 

channels established to clarify decision authority across organizational boundaries. In 

addition, the Program Office and Article 48 working group standardized procedures 

for how sites transition from IOC to continuous operations on ERAM.

- Process: ERAM has implemented a series of process improvements across all 

aspects of the system lifecycle, including in the areas of release management, 

software build packaging, test, deployment, and issues management. This work has 

improved the quality defect rate of software coming out of test, the effective 

collaboration with Union partners, the flow of information to the field facilities, and 

the ability to integrate complex data needed for planning. Specific examples of these 

changes include:

o The use of the National User Team to provide Air Traffic (AT) perspective 

and collaboration on the design of software fixes.

o The use of AT Subject Matter Experts to support site software activation 

activities, training, and comparable local planning processes.

o Improved communications evidenced by revised briefing packages and 

increased presence of support personnel at the site, to more effectively 

manage the release of software to the facilities.

- People: The program has optimized its resourcing strategy to align with both 

ERAM’s strategic goals as well as with the operational vision of the FAA. This 

began in early 2011, with the appointment of a new Director of Program Operations. 

Since then, the FAA has institutionalized its new Program Management Office,
39

Appendix. Agency Comments

created this fiscal year. Additional personnel changes in the past year include the 

assignment of a new program manager, a new manager of Air Traffic Manager 

Programs, and a new Director of Air Traffic Systems. New working relationships 

with the National Air Traffic Controllers Association and the Professional Airway 

Systems Specialists (via collaborative work groups initiated in 2011), as well as with 

Lockheed Martin (who has also introduced new resources in key positions in the past 

18 months) have improved coordination and integration of the team.

- Rewards: New rewards on the program include a range of opportunities. In working 

with Lockheed Martin, the new contract provides new incentives based on the 

successful achievement of future milestones (described in more detail below). More 

informally, communications in response to achievement of milestones, all-hands 

workforce meetings (where milestones are highlighted and hard-work 

acknowledged), and leadership recognition of positive achievements all incentivize 

program stakeholders. 

RECOMMENDATIONS AND RESPONSES

Recommendation 1: Develop a mitigation plan to address ERAM’s core capabilities 

problems at all 20 ERAM sites before deploying new capabilities.

FAA Response: Concur. The ERAM program has already completed this plan. 

Release 2 software builds--EAB0300 (July 15, 2011), EAB1100 (August 18, 2011), 

EAB1200 (December 5, 2011) and EAB1300 (April 24, 2012)--have been deployed to 

address identified core functionality issues. Since the time that “core functionality” 

issues were identified in spring of 2011, the program has achieved the following 

milestones that demonstrate core functionality has been substantially addressed within the 

system:

• The two program key sites, ZSE (Seattle, WA) and ZLC (Salt Lake City, UT), have 

achieved their ORD, and the legacy HOST system, has been decommissioned.

• Seven new sites after the key sites have achieved IOC.

• ZDV (Denver, CO), ZMP (Minneapolis, MN), and ZAB (Albuquerque, NM) have 

begun use of ERAM for continuous operations.

• ZHU (Houston, TX) has successfully demonstrated the operational use of ADS-B 

capability in ERAM through limited operational runs.

Depending upon the definition of “new capabilities,” many planned ERAM software 

release 3 functions augment and strengthen core functions (i.e. ADS-B integration should 

improve surveillance and radar processing accuracy, among other examples). More 

specifically, with release 3, the program has already implemented new functionality to 

include ADS-B and System Wide Information Management and has begun development 

on release 4 to support other core programs. As such, adding “new capabilities” to future 

software builds will in many cases act to strengthen ERAM core functionality 

capabilities. While this approach may introduce software complexities, the agency does 

not believe that these complexities constrain its ability to address core functionality in 

ERAM. Evidence of this can be seen in the following examples:
40

Appendix. Agency Comments

• Since activating the release 3 software baseline for software development, test, and 

deployment activities, an additional three sites have entered continuous operations 

and two sites have entered ORD (as noted above). This was achieved in conjunction 

with the pursuit of additional NextGen capability.

• In addition, the instances of problems identified as a part of site testing on a “prebuild” basis has dropped significantly through release 3. The number of problems 

found1

o EAB1300 (made operationally available April 24, 2012) - 61 total (34 Critical, 27 

highs of interest) 

during site test processes for the last three software builds appear below, 

spanning both release 2 and release 3:

o EAC1003 (made operationally available June 7, 2012) - 42 total (16 Critical, 26 

highs of interest) 

o EAC1100 (planned to be made operationally available June 9, 2012) - 29 total (15 

Critical, 14 High of interest)

Accordingly, the FAA requests that this recommendation be closed.

Recommendation 2: Evaluate available options and take action to deploy an additional 

backup for ERAM until the system has become significantly more mature.

FAA Response: Concur. As of May 2012, modifications to the Enhanced Back Up 

System (EBUS) have been deployed and EBUS will remain operational until ERAM’s 

operational stability (validated and measured through quantitative data on system 

availability, failure rates of system components, and the like) validates that the system is 

sufficiently stable2

. The ERAM Program Office and 2nd Level Engineering organizations 

are currently assessing the costs and benefits of operating EBUS beyond FY12 and will 

make a decision on the need for sustainment of EBUS, or alternatives and associated 

costs, by the end of FY12.

Recommendation 3: Revise the Contract Line Item Number (CLIN) structure to more 

effectively track ERAM costs. This should include establishing subordinate CLINs, cost 

targets, and incentives to better achieve program objectives, beginning with software 

release 4.

FAA Response: Concur. The recent negotiation/definitization of the “ERAM FY12/13 

Site Waterfall Proposal,” completed in May 2012, included the establishment of a 

contract structure that designates efforts under the software development CLIN into SubCLINs segmented by software releases. Subsequent releases will be assigned 

sequentially by Sub-CLIN. Cost targets and incentives are accordingly established and 

tracked at the Sub-CLIN and individual software release level, as are the associated cost 

 1 Note these are issues found and resolved during the test phase of each build. They do not represent a count of issues 

found during operational use of ERAM. 2 EBUS was originally deployed between 2004 and 2006 as a back-up to ERAM. More recent modifications allow for 

continued maintenance of EBUS beyond the decommissioning of the legacy HOST system.
41

Appendix. Agency Comments

accounts. This recommendation is now fully implemented, and the FAA requests that it 

be closed.

Recommendation 4: Include a requirement in AMS to definitize CLINs in a reasonable 

time period, such as FAR’s 180-day benchmark. Ensure that future ERAM CLINs are 

definitized according to the new requirement.

FAA Response: Concur. The FAA will review current Acquisition Management 

System (AMS) language regarding definitizing changes, and revise accordingly to ensure 

it requires action within a reasonable time period. This review will be completed by 

September 30, 2012.

Since December of 2011, the ERAM acquisition team has completed negotiations, 

definitized, or is in the process of definitizing, all undefinitized contract actions, with the 

exception of datacom S1P1 proposal preparation (which is in progress) and 

approximately $70K of legacy miscellaneous efforts falling under three CLINs. 

Specifically, En-Route Information Display System (ERIDS) Option 3 (definitized 

February 15, 2012), ERIDS Option 4, Legacy Display System Replacement (DSR) CDLS 

(definitized June 7, 2012) and the FY12/13 Site Waterfall efforts (definitized May 23, 

2012), having a combined value in excess of $374M, spanning over 25 CLINs/subCLINs, are now definitized. Acquisition schedules not exceeding 180 days are in 

development for future work including release 4 activities (R.4.1 Air Borne Re-Route / 

Ground Interval Management and R.4.2 Data COMM S1P1). 

Recommendation 5: Design incentives to better achieve desired program outcomes. 

For example, offer incentives over shorter intervals, such as bi-annually, to effectively 

motivate the contractor.

FAA Response: Concur. The contract CLIN structure was revised as of the completion 

of the contract renegotiation in May 2012, to incentivize performance based upon 

milestones specific to operational software releases. The work on redesigning these 

incentives began in March of 2011 and was resolved with the negotiation of the new 

contract. A significant portion of what was previously a one-time fee incentive has been 

allocated to five performance targets for each operational release (starting with software 

release EAC1200). Each of these five performance targets has mandatory and challenge 

success criteria that must be achieved for the partial or complete award of an incentive 

fee. The performance-based incentive milestones for each operational software release 

are assigned a specific contract designation by Sub-CLIN. This more segmented and 

modular approach will facilitate specific tracking and verification of the milestones by 

the Technical Officer, Contracting Office (CO) and the Quality Reliability Officer. The 

specific measures are tied to a contractually established release plan that identifies the 

milestone dates as well as the supporting detailed activities and their requisite schedule 

dates. FAA requests that this recommendation be closed.

Recommendation 6: Review AMS requirements for a contracts file list and contract 

maintenance procedures to verify that they are adequate. In addition, develop a process 
42

Appendix. Agency Comments

to verify that major contract files are reviewed by FAA’s National Acquisition Evaluation 

Program for compliance with AMS policy and best practices for contract management.

FAA Response: Concur. The FAA will review published AMS contract file checklists 

and maintenance procedures for adequacy and applicability. This review will be 

completed by September 30, 2012. Additionally, the FAA’s National Acquisition 

Evaluation Program will continue to include major contract files in its annual sampling 

for onsite reviews. 

The FAA AMS required checklists have been included and completed for all ERAM files 

and contract modifications since the issuance of Modification 90 in December 2010. 

Assurance that procedures are being followed is being facilitated by review of all 

modifications to the contract. Such reviews include, but are not limited to, the ERAM 

CO's, Contracting Officer Technical Representative, Contracts Manager and FAA legal 

counsel. The FAA has also initiated a Contract Administrative Review process in which 

major contracts are briefed by the CO in detail to a FAA contract management panel, 

including the Director of Contracts. ERAM Contract number DTFA01-03-C-00015 was 

the first briefing under this FAA initiative. The briefing was successfully accomplished 

on February 24, 2012. This approach will be repeated for future contract ERAM contract 

modifications as well. Based upon actions already taken, the FAA requests that this 

recommendation be closed.

Recommendation 7: Develop a formal process for ERAM invoice reviews that requires 

supporting document documentation such as travel vouchers and hotel receipts.

FAA Response: Concur. Controls continue to be in place ensuring the “spot-checking” 

approach meets the requisite criteria for balancing a reasonable effort of detailed invoice 

reviews with the risks these reviews mitigate – potentially incorrect or fraudulent charges 

to the FAA. FAA’s best-practices include a combination of training and lessons-learned 

activities to help educate contracting staff about the nature of invoice reviews. In 

addition to adhering to and imposing compliance with FAA Cost Principles and AMS 

clause 3.2.4-5 “Allowable Cost and Payment,” the ERAM team retains financial 

spreadsheets and follows practices for processing vouchers/invoices consistent with FAA 

procurement guidance section T3.10.1, appendix D.7. Based upon the availability of this 

guidance, and in following these two standards, the FAA believes that the OIG 

recommended formal process is in place and is being followed. Thus, the FAA requests 

that this recommendation be closed.

Recommendation 8: Update the performance measurement baseline for ERAM’s 

earned value management system to include all remaining work on the ERAM contract, 

including planned work that has not yet been priced and work performed by the 

Government.

FAA Response: Partially Concur. The ERAM program is expanding its existing Earned 

Value Management (EVM) approach to be a program-wide performance reporting tool 

rather than solely focusing on the prime vendor activities. Non-Prime Cost Performance 
43

Appendix. Agency Comments

Reports (CPR) have been generated starting in April 2012. These CPR will be integrated 

with the prime vendor reports which have been restructured to follow a product oriented 

Work Breakdown Structure better aligning to program milestones. This approach 

provides a range of benefits including more holistic reporting of progress against 

milestones and more accurate insight into anticipated cost or schedule risk against those 

milestones. In some cases, the FAA agrees that adding planned, future work that has not 

yet been fully priced would benefit from inclusion in the baseline. However, the business 

rules for addressing this should include considerations such as the maturity of the 

associated requirements, maturity of milestones and commitments made within and 

outside the agency, and the stability of the work program itself so as to not introduce 

unnecessary variability or volatility in the EVM reporting. Because the longer-range 

future requirements are not fully developed, the FAA is reluctant to fully concur with 

regard to the inclusion of all ERAM-related contract work into the baseline. The FAA 

will develop the initial implementation of this performance measurement baseline and an 

associated proposal for how to include planned future work into this EVM system by 

September 30, 2012.

Recommendation 9: Develop procedures in FAA’s “Program Level Integrated Baseline 

Review Guide” to verify that integrated baseline reviews meet the requirements and to 

establish a time frame for conducting integrated baseline reviews after executing major 

contract modifications. 

FAA Response: Concur. The FAA EVM Focal Point began an initiative in October

2011 to track and ensure Integrated Baseline Reviews (IBR) are conducted on major 

programs in accordance with the AMS Policy and the IBR Guide. The AMS Policy 

Section 4.16 EVM requires the conduct of IBR. The FAA’s Program Level IBR Guide 

contains the procedures in section 2.0 used to verify that integrated baseline reviews meet 

the requirements as stated in AMS Section 4.16 EVM. Section 2.0 Program Level IBR 

Process provides the necessary steps for IBRs, they include the preparation of an IBR 

Plan, the approval of the IBR Plan, IBR preparation and conduct, the preparation of 

findings, the development and approval of an action plan, and the closure of the actions 

contained in the plan. Each step identifies the responsible office and the process step 

description. The IBR Guide also provides the time frame for conducting the IBR for 

major contracts, and if it is not conducted prior to contract award, it must be conducted 

within 90 to 180 days of contract award or program baseline establishment. Currently, 

the Guide does not specifically state that an IBR is required after executing major 

contract modifications. The IBR Guide will be revised to make conducting an IBR after 

the execution of a major contract modification a requirement by September 30, 2012.

Recommendation 10: Complete the comprehensive risk management guidance that 

FAA is currently developing, to more effectively manage acquisition risks.

FAA Response: Concur. While the FAA agrees that comprehensive risk management 

guidance is needed to effectively manage acquisition risks, the risk management toolkit 

referenced in this recommendation was completed in 2006. The risk management toolkit 

was based on the Risk Management section of the Systems Engineering Manual
44

Appendix. Agency Comments

completed in 2002. In a recent update to FAA Acquisition System Toolset, the link 

published inadvertently directed users to a track changes version of the toolkit rather than 

the final version. If the OIG used this previous link, it may have appeared that the 

guidance was not completed. Accordingly, the FAA requests that this recommendation 

be closed. 

Recommendation 11: Assess current testing capabilities and limitations at FAA’s 

Technical Center and develop corrective action plans to more robustly test future 

complex software-intense air traffic systems.

FAA Response: Concur. The FAA shares OIG’s perspective that it is prudent to 

continually assess capabilities and limitations and develop corrective actions to improve 

upon our processes and capabilities for all aspects of testing. This is accomplished 

through the engagement of cross organizational teams to identify test and laboratory 

requirements and new processes to support program objectives. As issues arise, 

laboratory managers work with the laboratory users to determine how to improve current 

testing capabilities and take corrective action as needed.

As part of the current ERAM regression testing conducted at the William J. Hughes 

Technical Center, ERAM is regularly connected with the following live interfaces: En 

Route, Oceanic, Terminal, Surveillance, Weather, and Traffic Management. This testing 

is conducted prior to deployment of an ERAM software release. Additionally, in support 

of the International Civil Aviation Organization 2012 program, the International Upgrade 

to Flight Plan Processing, ERAM has established live connections to both Canada and 

Mexico. These live connections were established in 2010 to support the testing which 

commenced in late 2011 and continues through 2012.

In addition to the increased use of live interfaces, the Technical Center has improved its 

simulation capabilities. Starting in early 2011 and continuing through 2012, simulation 

hardware has been upgraded to allow the simulation of any or all surveillance capabilities 

(i.e., radars, ADS-B, etc.) used at air traffic control sites. The number and complexity of 

the simulation scenarios have been improved to allow two ERAM systems and three 

Terminal systems to be interfaced and run simultaneously. Another example of improved 

testing capabilities is the incorporation of a shadow mode capability which has been 

established with several field sites. This effort started in late 2009 and the Standard 

Terminal Automation Replacement System test team continued to enhance the capability 

in 2010 to support deployment of a software system release to Philadelphia (PHL). This 

capability enables the sites’ live radar and flight data information to be utilized at the 

Technical Center over FAA Telecommunications Infrastructure / Bandwidth Manager 

(currently used for the Terminal Automation Modernization Replacement test program). 

As a result of the increased simulation capabilities, the number of lab runs has increased 

for ERAM testing. The increased demand for lab resources has necessitated the 

installation of a fifth ERAM testbed to support current ERAM and future NextGen 

testing needs (i.e., DataComm). This new ERAM laboratory is expected to be available 

in January 2013 and will have the same capabilities as the other ERAM labs with 
45

Appendix. Agency Comments

connections to the various National Airspace System (NAS) laboratories and live 

interfaces throughout the Technical Center. 

The Technical Center continues to enhance its network infrastructure and connectivity to 

other laboratory capabilities and assets internally and externally, using a Live Virtual 

Constructive Environment for distributed simulation and testing activities to support 

NextGen. A recent example of this capability expansion includes network connectivity to 

the Florida Test Bed, National Aeronautics and Space Administration North Texas 

Facility, Department of Defense (DoD) Research and Engineering Network (DREN), and 

Boeing.

The Technical Center is also exploring tools that could provide the basis for a System of 

Systems Assessment Platform (SoSAP) for NextGen. SoSAP is a simulation architecture 

similar to those used by the DOD for concept exploration, requirements definition, and 

testing complex software-intensive systems. A robust SoSAP would support large scale 

system testing by using software agents and emulated systems with the system under test 

to evaluate system behaviors and interactions inherent within complex systems. SoSAP 

could support multiple phases of the AMS and Ideas to In-Service Process. A 

demonstration of an early SoSAP prototype, leveraging DOD experience, was provided 

to the Technical Center Director in May 2012.

The Technical Center has also improved their testing processes, with all of the NextGen 

Organization’s (ANG) Technical Center test organizations and the Laboratory Services 

Division certified to the International Organization for Standardization (ISO) 9001:2008 

quality management standard. The ISO certification provides consistent best practices, 

standards, and procedures across the ANG test and laboratory organizations. As an 

example of process improvement, the Technical Center has increased the use of FAA 

field specialists in the design and execution of system tests.

Additionally, the Air Traffic Systems (ATS) organization, within the Program 

Management Office (AJM), has begun an assessment of the test capabilities currently at 

its disposal. This assessment is envisioned to include a review of test assets and 

processes, gaps in capabilities needed to support known future NextGen programs, and 

best-practices from other programs that could be expanded or institutionalized. 

Stakeholders for this assessment would include personnel from AJM, ANG, and various 

aspects of the operation that currently support test (including facility and operational 

service unit personnel). This work is planned to produce a cost-benefit analysis (i.e. 

business case), requirements, and associated work plan, to close the gaps and implement 

an enhanced, future-state test capability. ATS plans to deliver this business case, 

requirements, and work plan by the end of FY13.

Recommendation 12: Require complex software-intensive systems (that are 

interdependent on other systems, such as ERAM) to be successfully tested in a live, 

operational environment, at one or more FAA air traffic facilities, prior to Government 

Acceptance.
46

Appendix. Agency Comments

FAA Response: Concur. The approach prescribed within the AMS currently meets this 

recommendation and fully supports the notion that complex capabilities must be 

successfully tested in a live, operational environment at one or more FAA facilities prior 

to full Government Acceptance (GA). Completion and full closure of GA is an iterative 

process over the lifecycle of a program. Some level of GA is required in the program’s 

initial implementation to enable the use of the system or service in a live, operational 

environment (i.e. before it can enter the NAS). GA of any system or service is driven by 

the ability of that system or service to meet the required specifications developed as part 

of the contract between the government and the vendor. 

Controls exist within the AMS to address the OIG’s recommendation at incremental 

stages. These include the following: 

1. Factory Acceptance Testing Milestone - the hardware and software acceptance 

of the functional specification requirements from the prime contractor;

2. Interface Acceptance Milestone - integration and testing utilizing live interfaces; 

and 

3. Site/System Acceptance Test Milestone - testing required at the site to meet the 

IOC program milestone. 

Beyond ERAM, this approach to GA has and will continue to guide the implementation 

of the FAA’s other systems. This is prescribed in the AMS and is aligned to the OIG’s 

recommendation. The FAA requests that this recommendation be closed.

Recommendation 13: Revise the Acquisition Management System to better define key 

milestones, such as Government Acceptance and initial operating capability, so that 

milestones are clear measures of progress for managing major acquisitions.

FAA Response: Concur. Each of the key milestones noted (GA, IOC, and in-service 

decision) are defined by the current AMS. In their current form, the definition of GA, 

IOC, and in-service decision represent critical and specific milestones each program must 

proceed through. However, the FAA recognizes that, in light of the practices on 

programs such as ERAM, the agency can review the definitions in the AMS to further 

establish clear criteria for entrance and exit at the referenced key milestones. The FAA 

has started this review and anticipates that it will be completed by October 30, 2012.