Office of Inspector General

Audit Report

FAA’S ACQUISITION STRATEGY FOR 

TERMINAL MODERNIZATION IS AT RISK FOR 

COST INCREASES, SCHEDULE DELAYS, AND 

PERFORMANCE SHORTFALLS 

Federal Aviation Administration

Report Number: AV-2013-097

Date Issued: May 29, 2013
U.S. Department of

Memorandum

Transportation

Office of the Secretary

of Transportation

Office of Inspector General

Subject: ACTION: FAA’s Acquisition Strategy for 

Terminal Modernization Is at Risk for Cost 

Increases, Schedule Delays, and Performance 

Shortfalls

Federal Aviation Administration

Report Number AV-2013-097

Date: May 29, 2013

From: Jeffrey B. Guzzetti

Assistant Inspector General

 for Aviation Audits

Reply to 

Attn. of: JA-10

To: Federal Aviation Administrator

The Federal Aviation Administration (FAA) operates more than 200 terminal 

radar approach control (TRACON) air traffic control facilities. In 1996, FAA 

began a plan to modernize and standardize the terminal automation systems that 

controllers rely on to manage traffic within a 50-mile radius of airports by 

implementing the Standard Terminal Automation Replacement System (STARS) 

throughout the National Airspace System (NAS). However, FAA has experienced 

significant cost growth and schedule delays with STARS and needed an interim 

solution to satisfy immediate modernization needs. From 1998 to 2003, FAA 

updated existing software and hardware for automation systems at sites yet to be 

converted to STARS. These systems were referred to as the Common Automated 

Radar Terminal System (CARTS).

By 2004, FAA was operating two distinct systems at different TRACONs—either 

CARTS or STARS. In the same year, FAA decided to revise its terminal 

automation approach through a three-phased acquisition strategy designed to 

reduce costs by reevaluating alternatives and incorporating lessons learned from 

earlier phases. Renamed the Terminal Automation Modernization/Replacement

(TAMR) program, this effort is now necessary both to replace aging equipment 

and achieve FAA’s goals to enhance capacity and reduce delays through the Next 

Generation Air Transportation System (NextGen).

Since 2003, we have reported on FAA’s progress and challenges with terminal 

modernization and have noted that changing requirements significantly contribute 
2

to schedule delays and cost growth.1 Given FAA’s past issues with terminal 

modernization and the importance of FAA’s TAMR program in meeting the 

Agency’s NextGen goals, we initiated this audit to evaluate FAA’s current 

terminal modernization effort. Specifically, our audit focuses on the current phase 

of TAMR—TAMR Phase 3, Segment 1—which aims to replace CARTS with an 

updated version of STARS at 11 sites, some of which are the largest and most 

important TRACONs in the NAS. Our audit objectives were to assess (1) risks 

related to developing and implementing software and hardware requirements for 

Phase 3, Segment 1 and (2) whether FAA’s final investment decision resulted in a 

reliable cost and schedule baseline. 

We conducted this audit in accordance with generally accepted Government 

auditing standards. Exhibit A details our scope and methodology, and exhibit B 

provides a list of the organizations we visited or contacted. 

BACKGROUND

In the 1970s and 1980s, FAA implemented its first Automated Radar Terminal 

systems (ARTS) at TRACONs, which displayed radar feeds to help controllers 

control takeoffs and landings, the most critical phases of flights. FAA’s efforts to 

modernize these systems since 1996 are illustrated in the following timeline 

(figure 1).

Figure 1. Timeline of FAA’s Terminal Modernization Efforts

FAA’s TAMR effort is designed to modernize air traffic control facilities to 

handle increased traffic and will support NextGen initiatives, such as the use of a 

satellite-based air traffic surveillance program, Automatic Dependent 

 1 OIG Report Number AV-2003-058, “FAA Needs To Reevaluate STARS Costs and Consider Other Alternatives,” 

September 9, 2003; OIG Report Number AV-2005-016, “Report on Terminal Modernization: FAA Needs To Address 

Its Small, Medium, and Large Sites Based on Cost, Time, and Capability,” November 23, 2004; OIG Report Number 

AV-2005-061, “Status of FAA`s Major Acquisitions: Cost Growth and Schedule Delays Continue to Stall Air Traffic 

Modernization,” May 26, 2005. OIG reports are available on our Web site at http://www.oig.dot.gov.

1996

FAA awards $953M 

contract to Raytheon 

to develop and 

deploy STARS at 

172 TRACONS and 

towers

Due to cost overruns 

and delays, CARTS 

developed and 

deployed as interim 

solution for an 

estimated $239M

Phase 2 complete: 

STARS deployed 

at 5 new sites and 

CARTS upgraded 

at 4 sites

1998-2003 2004 2007 2008 2010 2011 2012

FAA initiates 

3-phased 

approach to 

terminal

modernization

Phase 1 

complete: 

STARS 

deployed at 

47th site

FAA splits Phase 

3 into 2 segments 

and decides to 

deploy STARS at 

remaining CARTS 

to promote single 

automation 

platform

Seg 1 – FAA 

decides STARS 

will replace 

CARTS at 11 

large TRACONS 

at cost of $438M

Seg 2 – FAA 

decides to 

deploy STARS 

at 97 small 

sites that were 

modernized 

over 10 yrs ago 

for $462M

STARS CARTS TAMR
3

Surveillance-Broadcast (ADS-B). FAA is currently in Phase 3 of TAMR. For this

phase, FAA originally planned to replace all 108 of the remaining CARTS 

systems, but in April 2010 the Agency broke up the phase into 2 segments. The 

first segment focused on short-term (11 TRACONs with large traffic volumes) 

planning objectives, while the second segment focused on mid-term 

(97 TRACONs with small traffic volumes) planning objectives. In September 

2010, FAA decided to deploy STARS as the terminal automation system to best 

promote FAA’s goal for a single terminal automation platform. On December 21, 

2011, FAA completed its final investment decision for Segment 1, pushing 

estimated completion to 2017. On September 19, 2012, FAA completed its final 

investment decision for Segment 2.

FAA’s capital investment programs, such as TAMR, generally follow the 

acquisition lifecycle set forth in the Agency’s Acquisition Management System 

(AMS), which establishes policy and guidance for all aspects of FAA’s acquisition 

process. Programs are generally baselined at the final investment decision 

milestone when the Joint Resources Council (JRC)2 approves the cost and 

schedule parameters, the specific performance requirements, and expected benefits 

for a program. Prior to the final investment decision, AMS requires the program 

office to manage the investment by developing a business case along with 

supporting documentation (e.g., implementation strategy and planning document, 

program requirements document) to support the baseline decision. FAA’s 

Investment Planning and Analysis office must validate the supporting information 

to ensure the business case provides a credible picture of the investment and its 

impact on the NAS. The office also evaluates and assesses the cost, benefits, risks, 

schedule, and economics of a particular initiative. FAA considers this effort 

crucial for reducing the risks of a program’s investment decision.

RESULTS IN BRIEF

FAA faces significant risks in developing and implementing the technical 

requirements for its current terminal modernization effort. Specifically, FAA has 

yet to identify and finalize all the software and hardware requirements needed to 

successfully replace CARTS with STARS at the 11 large TRACONs. FAA must 

first modernize STARS hardware to meet the current capabilities of CARTS, but 

has encountered problems such as keyboard failures that have resulted in delays. 

Furthermore, while FAA anticipated the need for developing 94 software and 

hardware requirements (referred to as “gaps”) that are critical for successfully 

replacing CARTS with STARS, at least 5 more gaps have been identified that will 

require extensive software development and testing. FAA anticipates identifying 

more gaps once it begins transitioning to STARS at the highest-volume 

 2 The JRC is FAA’s investment review board. It makes corporate-level resource decisions, including authorization and 

funding for new investment programs. 
4

TRACONs, which could further delay implementation. At the same time, because

full STARS capability at the 11 sites is still years away, FAA is continuing to add 

new capabilities to CARTS at select facilities to support air traffic operations. The 

longer FAA must maintain and update CARTS at these sites, the greater the 

implementation risk because FAA will have to add the same new capabilities to 

STARS to maintain functionality at the 11 sites.

FAA’s final investment decision did not result in a reliable schedule and cost 

baseline for implementing STARS at the 11 large TRACONs, putting TAMR at 

risk of further schedule delays and cost growth. For example, FAA’s approved

schedule to deploy STARS by 2017 lacks key deployment milestones and 

completion dates and was not evaluated for risk per AMS requirements. Moreover, 

in July 2012, 6 months after its final investment decision, TAMR program 

officials approved a different schedule with the STARS contractor that requires

the contractor to deploy STARS systems more than a year earlier than the baseline

schedule. This timeframe ignores repeated concerns that FAA’s software 

development and testing efforts would likely require additional time. As a result, it 

remains unclear how and when FAA will accomplish all the tasks necessary to 

implement STARS at the 11 sites. FAA’s current cost estimates are similarly 

unreliable because FAA omitted major program cost elements from the cost 

baseline approved during its final investment decision. For example, even though 

the TAMR schedule extends through 2017, FAA’s approved cost baseline of 

$438 million only covers the effort through 2015, and it remains unclear how the 

additional 2 years will be funded. The Agency also excluded anticipated technical 

refresh and modernization costs, estimated at $270 million, as well as longer-term, 

undetermined costs to continue maintaining and upgrading CARTS longer than 

originally planned. Finally, we found that the work products used to support the 

TAMR business case contained inconsistent information or lacked key details, 

further undermining FAA’s ability to know the true costs and timeline of TAMR

and its impact to the NAS. 

We are making recommendations to FAA to address challenges and risks to its 

terminal modernization efforts.

FAA FACES CONSIDERABLE RISKS IN DEVELOPING AND 

IMPLEMENTING SOFTWARE AND HARDWARE REQUIREMENTS 

FAA faces significant risks with its current terminal modernization effort because 

it has not yet finalized the technical requirements for the complex transition from 

CARTS to STARS. According to FAA, to successfully execute its acquisition 

strategy and implement STARS at the 11 large sites, the Agency must develop, 

test, and deploy upgrades to STARS hardware and software before it can fully 

replace the current CARTS systems. In the meantime, FAA must also continue to 
5

maintain and sustain CARTS at the 11 facilities until the STARS replacement is 

accomplished. Without finalized requirements for these efforts, the Agency is at 

risk of increased software development costs and implementation delays.

FAA Must Address Problems With STARS Modernization Before It 

Can Meet the Existing CARTS Functionality 

FAA has yet to identify and finalize all the software and hardware requirements 

needed to successfully replace CARTS with STARS at the 11 TRACONs. This 

lack of firm requirements is a particular concern given the complex and 

simultaneous efforts needed for this transition. First, FAA must modernize STARS 

hardware (i.e., computer processors, monitors, operating systems, and the dual 

display architecture), which will involve extensive testing to determine whether 

the hardware is fit for use.

At the same time, FAA must update STARS software to obtain the enhanced 

functionality required to support the 11 large TRACONs. For example, one 

enhancement would enable STARS to support more than 8 remote towers—an 

important upgrade given that 6 of the 11 sites have more than 8 remote towers. 

Another enhancement would reduce STARS recovery time from power failures 

from 6 minutes to 60 seconds. In accordance with its NextGen plans, FAA is also 

planning to deploy new ADS-B functionality at all STARS sites, which will 

require further operational testing and evaluation activities. 

FAA considers the greatest risks to STARS development to be software 

development and acceptance. In particular, after upgrading STARS hardware and 

software, FAA will then need to incorporate at least 100 new functions, referred to 

as “gaps,” into STARS to mirror the capabilities that CARTS currently provides.

In modernizing and upgrading STARS for use at the 11 TRACONs, FAA initially

identified the need to develop and implement 94 gaps in order for STARS to 

perform the same functionality as CARTS. These gaps equate to about 80,000 

lines of software code and require extensive development and testing before they 

can be accepted by FAA. However, according to an FAA analysis of past STARS 

software development, it takes on average 16 months for a software build with

about 13,000 lines of code to be developed, tested, and accepted by FAA. 

Consequently, FAA’s plan to deliver the 80,000 lines of software code at the first 

key site by September 2014—only 33 months after the program was baselined—is 

a high-risk endeavor. These risks will be further exacerbated if FAA follows

through with its contract plans to achieve operational capability at the first key site 

by February 2013.

Some of the needed gaps in STARS are critical for safely managing aircraft (see 

table 1). For example, automatic proximity alerts will help controllers maintain 

safe separation distances between aircraft in the terminal environment. However, 
6

the proximity alerts will require extensive software development that will be timeconsuming to install because of their impact on STARS operational software that 

supports existing capabilities. 

Table 1. Key Gaps (Requirements) Needed To Upgrade STARS 

Gap Item Operational/Technical Impact

Automatic Proximity 

Alerts

Added software will improve situational awareness by 

automating a function to alert controllers of loss of separation.

Automatic Update of 

Altimeter Settings 

Added software will allow controllers to determine proper 

altitude of aircraft.

Modernized Computer 

Data Recovery 

Added hardware and software will allow continuous recording 

of data without manual intervention.

Number of Remote 

Towers 

Added hardware will allow the use of STARS at TRACONs 

with more than eight remote towers.

Airspace Violator Alert Added software will allow controllers to identify potential 

airspace violators.

Source: FAA.

Furthermore, FAA has not yet determined all the site-specific requirements needed 

to provide STARS with the necessary capabilities and anticipates discovering 

additional gaps as site surveys and deployment continue. For example, FAA 

identified 5 new gaps in July 2011 after it submitted the original 94 gaps to the 

contractor for cost proposal development. The five new gaps were generated 

during site surveys conducted at the Segment 1 sites. While FAA officials do not 

consider these gaps as having a significant impact on STARS performance, the 

Agency has yet to complete site surveys that may uncover more gaps. Until FAA 

determines whether new gaps exist and how they will be implemented, the Agency 

will not know the extent of the cost and schedule risks that the resulting 

requirements may present to successfully implementing STARS. 

In addition, MITRE3 has reported that the more traffic volume that exists at a site, 

the more complex the transition to STARS will likely be, increasing the chance for 

additional requirements that will take added time to complete. This is a particular 

concern because 7 of the 11 Segment 1 TRACONs are the largest in traffic volume 

for 2011, as shown in table 2. 

 3 MITRE Corporation manages FAA’s federally funded research and development center.
7

Table 2. Traffic Volume for CARTS Facilities Slated for STARS in 

Segment 1

Facility Traffic Volume (2011) Rank

Southern California TRACON 1,985,000 1

New York TRACON 1,894,000 2

Potomac TRACON 1,492,000 3

Northern California TRACON 1,464,000 4

Chicago TRACON 1,243,000 5

Atlanta TRACON 1,223,000 6

Dallas Fort-Worth TRACON 1,120,000 7

Denver TRACON 886,000 10

Minneapolis TRACON 552,000 18

St. Louis TRACON 319,000 26

Louisville TRACON 212,000 50

Source: FAA Operations Network.

FAA’s risks related to its still-evolving requirements are highlighted in several 

unanticipated technical issues that FAA has already reported, which may impact 

milestones. For example, in the midst of hardware and software modernization 

activities, keyboards for STARS equipment have been failing at higher than 

normal rates, and the Agency is reviewing various options if the STARS 

contractor cannot identify a fix. This has resulted in delays in installing the dual 

display architecture for aiding the transition from CARTS to STARS at the initial 

key site. In addition, a key software build that incorporates needed STARS 

enhancements has required more updates, and FAA has expanded the number of 

operational test sites initially planned, resulting in more delays. FAA is actively 

addressing these challenges; however, the potential exists for more unanticipated 

delays associated with the program’s still-evolving requirements.

FAA Is Adding New Capabilities to CARTS, Which May Further Impact 

the STARS Transition

As FAA works to upgrade STARS, the Agency is also managing CARTS systems 

with differing capabilities across the NAS, further complicating terminal 

modernization efforts. Because STARS implementation at the 11 sites is still years 

away, FAA is continuing to add new functionality to CARTS at select facilities to 

support air traffic operations. For example, FAA is currently upgrading CARTS 

hardware at the New York TRACON to support ADS-B. The Agency has also 

added proximity alert technology at Minneapolis and St. Louis TRACONs and is 
8

planning to implement this technology at the other sites that will be replaced with 

STARS, with the exception of Dallas-Fort Worth. 

These simultaneous upgrades to CARTS pose several challenges to FAA. In a 

September 2011 report, MITRE stated—and we agree—that the more operational 

enhancements FAA continues to add to the CARTS system, the more difficult it 

will be to gain user support for the transition from CARTS to STARS, because 

CARTS will already be providing all the capabilities that controllers need. In 

addition, the longer FAA is forced to maintain and update CARTS at these sites, 

the greater the risk for new STARS requirements, as more software development 

will likely be required to ensure STARS systems meet the functionality for the 

11 sites that the CARTS systems already provide.

FAA’S FINAL INVESTMENT DECISION DID NOT RESULT IN A 

RELIABLE COST AND SCHEDULE FOR IMPLEMENTING STARS

FAA held a final investment decision for the 11 sites in December 2011—after 

postponing it four times since March 2010—and approved a schedule baseline 

through 2017 and a cost baseline of $438 million. However, several factors limit 

the reliability of this baseline. First, the approved baseline schedule lacks 

completion dates such as detailed milestones for software development and 

deployment activities and a complete risk assessment per AMS guidelines. In 

addition, the approved schedule is in conflict with the deployment schedule 

negotiated with the contractor. Second, cost elements for developing STARS 

hardware and software and sustaining CARTS were omitted from FAA’s TAMR 

business case. In addition, program funding approved in the cost baseline was only 

budgeted through 2015, even though the schedule baseline has a 2017 completion 

date. FAA also used incomplete work products that did not comply with AMS as 

its basis for approving the program. Because FAA approved the TAMR program 

with unreliable schedule and cost data and incomplete supporting work products, 

considerable risks and uncertainty exists regarding when the Agency will complete 

this effort and at what cost.

FAA’s Approved Baseline Schedule Lacks Completion Dates and 

Milestones and Conflicts With the Contract Schedule, Which

Introduces High-Level Risk

FAA’s approved baseline schedule lacks detailed milestones, limiting its reliability 

as an indicator of the true timeline for successfully implementing STARS at the 

11 large TRACONs. Originally slated to proceed until 2015, the approved TAMR 

schedule was revised on the date of the final investment decision meeting

(December 21, 2011), to extend completion time of the 11 sites initial operating 
9

capability (IOC)4 to 2016, and full operational capability to 2017. This was done 

because major concerns were raised within FAA after several independent 

evaluations of TAMR’s detailed working schedules highlighted major risks with 

the contractor’s ability to complete IOC for all 11 sites by 2015. For example, 

MITRE reported in September 2011 that FAA’s software development efforts 

would likely require additional time for operational testing and acceptance. The 

same report also identified that FAA officials in the TAMR program office and 

FAA Technical Center anticipated significant risk regarding implementation of 

new hardware elements for STARS. MITRE also recommended in December 

2011 that FAA address software deployment risks by allotting more time to 

specific deployment milestones in the approved baseline schedule.

5

However, the new schedule that FAA senior management both proposed and 

approved at the final investment decision meeting did not undergo a risk 

assessment, as required by AMS, to ensure it was feasible, and lacked completion 

dates for 8 sites as well as key milestones for hardware and software development, 

testing, and deployment activities at all 11 sites (see exhibit C). Establishing these 

milestones is important because 129 remote air traffic control facilities are 

associated with these 11 TRACONs, and STARS must be deployed to those sites 

as well. 

Moreover, despite the concerns raised by MITRE and other FAA officials, TAMR 

program officials approved a new schedule with the STARS contractor in July 

2012 that conflicts with the baseline schedule FAA senior management approved 

at the December 2011 final investment decision. FAA’s actions raise major 

concerns about its schedule’s reliability, particularly regarding the dates for 

achieving IOC for all 11 sites. For example, FAA directed the contractor to 

complete IOC a year and a half earlier for the first site than approved in the final 

investment decision baseline. This re-introduces the high level of risk FAA 

originally identified during its independent evaluations of TAMR’s detailed 

working schedules, because it does not allow the time for software development 

and testing that FAA senior managers previously allocated.

According to AMS, schedule risks result from the likelihood that program actions 

may not be accomplished in the amount of time planned. However, a detailed 

program schedule that identifies each milestone and the critical steps to achieving 

it are necessary to identify and mitigate schedule risks. FAA officials state they 

performed sufficient risk assessment activities and met the intent of AMS by 

adding the 2 years to its approved schedule. However, FAA’s actions to direct the 

 4 Initial Operating Capability (IOC) is the milestone where controllers begin to use the system on a limited basis to 

manage traffic. 5 MITRE Briefing: “TAMR Program Phase 3 Interview Report,” dated September 2011, and MITRE Briefing: “TAMR 

Program Phase 3 Segment 1 Schedule Analysis,” dated December 2011.
10

contractor to complete IOC a year and a half earlier than the approved schedule 

raises concerns about whether FAA sufficiently addressed the schedule risks (see 

table 3).

Table 3. FAA’s Approved Baseline Schedule Conflicts With the 

Contract Schedule for Implementing the 11 Sites

Facility

IOC 

Baselined 

December 2011

IOC 

Contract 

July 2012

1. Dallas Fort-Worth TRACON September 2014 February 2013

2. Northern California 

TRACON TBD October 2013

3. Atlanta TRACON TBD February 2014

4. Southern California 

TRACON TBD June 2014

5. Minneapolis TRACON October 2015 August 2014

6. Potomac TRACON TBD October 2014

7. Denver TRACON TBD February 2015

8. St. Louis TRACON TBD April 2015

9. Louisville TRACON TBD June 2015

10. Chicago TRACON TBD June 2015

11. New York TRACON October 2016 October 2015

Source: FAA’s Acquisition Program Schedule Baseline for TAMR Phase 3, Segment 1, and STARS Contract Mod 79.

IOC - Initial Operating Capability is defined as site using TAMR Phase 3 hardware and software components 

operationally to control air traffic over a 2-4 hour period, typically on a mid-shift.

TBD - To be determined.

FAA’s Omission of Key Cost Elements in the TAMR Business Case

Masks the True Costs To Implement STARS and Increases the Risk of 

Further Cost Growth

FAA’s current cost estimates for TAMR are similarly unreliable because FAA 

omitted key program cost elements from the cost baseline and business case 

approved in December 2011. According to AMS, the business case analysis at the 

final investment decision must present updated cost, schedule, benefit, and risk 

information for an alternative generated from an initial investment analysis. 

However, FAA’s business case information was never updated. Additionally, FAA 

did not include an analysis to determine whether TAMR achieved the required

80-percent confidence level for success, and a cost/benefit analysis to justify the 
11

expected benefits associated with this acquisition. Instead, FAA performed a cost 

effectiveness analysis6 comparing two alternatives: 

• Replacing all 11 TAMR Phase 3, Segment 1 sites with STARS at a cost of 

$462 million (Preferred Alternative)

• Maintaining the Segment 1 CARTS sites and replacing the 52 STARS 

Phase 1 sites with CARTS (Base Case) at a cost of $731 million

FAA chose the Preferred Alternative since the analysis indicated it was more cost 

effective and a better overall value.7 However, this amount excludes a major 

known cost factor—technical refresh and modernization costs (estimated to cost

$270 million between 2013 and 2017) to support computer hardware and software 

upgrades—even though some of these are key enhancements required to address 

the gap requirements. FAA stated that these were excluded because they were part 

of a separate cost baseline. However, several technical refresh activities are 

prerequisites to deploying STARS at the 11 Segment 1 sites. 

In addition, the TAMR business case did not account for unknown, longer-term 

costs, such as maintaining and upgrading CARTS longer than originally planned at 

sites until they receive STARS. For example, FAA spent $16 million on CARTS 

upgrades at the New York TRACON because STARS was not ready for 

deployment and the TRACON needed to meet near-term ADS-B requirements. 

The original CARTS contract, awarded in March 1999, was slated to expire on 

September 2011. However, the CARTS contract had to be extended twice—first to 

March 2012 at a cost of $2 million, and then to March 2013 at a cost of 

$11 million—primarily due to delays in investment decisions for TAMR. After 

this contract expires, FAA anticipates awarding a 6-year CARTS contract focusing 

on equipment, software, maintenance, and engineering service; however, the costs 

to sustain CARTS in the long-term remain undetermined. 

FAA’s Cost Baseline Does Not Include Funds To Support the Last 

2 Years of Implementation or Address Funding Shortfalls

FAA’s cost baseline to support STARS deployment through fiscal year 2015 is 

$438 million. However, funds will be required to support FAA’s terminal 

modernization goals beyond 2015, as the Agency plans to complete implementing 

initial STARS capabilities in 2016 and full STARS capabilities in 2017 for the 

11 large TRACONs. FAA has yet to determine the required additional funding

needed to support the remaining 2 years of TAMR implementation.

 6 TAMR Phase 3, Segment 1 did not perform a traditional benefits analysis and relied on a cost effectiveness study. As 

stated in its business case analysis, "a cost effectiveness analysis is effective whenever it is unnecessary or impractical 

to consider the dollar value of the benefits provided by the alternative under consideration.”

7 FAA stated that it substituted a cost effectiveness analysis in alignment with Office of Management and Budget 

(OMB) guidance. However, the OMB document clearly states that its guidance should not supersede agency 

requirements.
12

Even before FAA extended the completion date for the 11 sites out by 2 years, it 

was already facing a $115 million funding shortfall. Table 4 shows FAA’s 

approved baseline cost and funding plan and shortfall for the TAMR program at 

the time of the final investment decision. 

Table 4. Funding Shortfall To Implement the 11 Sites

TAMR Phase 3 Segment 1 Cost Baseline ($ in Millions)

Cost Element Prior FY 11 FY 12 FY13 FY14 FY15 Total

TAMR Phase 3 Segment 1 $11.1 $49.9 $140.2 $143.1 $76.5 $17.2 $438.0

Capital Investment Plan $38.0 $59.9 $103.3 $74.7 $32.2 $14.8 $322.9

Delta: CIP less Baseline $26.9 $10.0 ($36.9) ($68.4) ($44.3) ($2.4) ($115.1)

Remaining Balance $26.9 $36.9 $0.0 ($68.4) ($112.7) ($115.1)

Source: TAMR JRC Final Investment Decision, December 21, 2011.

In comparison with earlier planning estimates included in its Capital Investment 

Plan, FAA’s cost baseline for TAMR results in shortfalls of about $68 million in 

2013, $44 million in 2014, and $2 million in 2015. However, officials from FAA’s 

budget office state that they have now eliminated the shortfall by reprogramming 

and readjusting funds from other capital programs. We caution that funding 

program shortfalls from capital budgets that are projected to be flat in the near 

future will negatively impact FAA’s long-term ability to sustain and modernize 

existing systems and develop new systems.

FAA’s Supporting Documents for the Final Investment Decision Did 

Not Adhere to AMS

Despite approving the TAMR plan for the 11 sites in December 2011, FAA did 

not provide a credible and complete picture of its investment and its impact to the 

NAS. We found that work products used to support the TAMR business case 

contained inconsistent information or lacked key details, further underscoring that 

FAA did not perform complete validation and verification activities before 

deciding to move forward. According to FAA’s AMS policy for acquisitions, an 

investment analysis plan must be well conceived, low risk, well documented, and 

well understood. FAA is required to validate and verify its business case and other 

supporting work products that are the basis for the recommendation to approve or 

decline moving forward with a program prior to the final investment decision. 

Table 5 highlights specific problems we identified with work products FAA used 

to support the TAMR final investment decision.
13

Table 5. Incomplete Work Products Used for TAMR’s Final 

Investment Decision 

Work Product AMS Requirements Problems Identified 

Implementation 

Strategy and 

Planning 

Document 

(ISPD)

• Defines overall implementation 

strategy and planning for the 

investment program. Conveys 

the most critical, relevant, and 

meaningful information about 

how a program will be 

implemented and managed.

• The ISPD should be finalized 

before it is submitted to the 

JRC, contain a program 

schedule based on a work 

breakdown structure, and be 

consistent with the schedule in 

the business case analysis.

The ISPD included in the TAMR final investment

decision did not contain a schedule with a work 

breakdown structure—and indicated that one would not 

be available until the final schedule is approved and 

authorized. 

A few days prior to the final investment decision 

meeting, FAA included a schedule in the ISPD as an 

addendum. However, the schedule conflicts with the 

schedules presented in both the acquisition program 

baseline and the business case analysis.

Program 

Requirements 

Document

• When presented at the final 

investment decision, must 

clearly define the operational 

framework and requirements 

the program must achieve.

Despite AMS requirements, FAA opted not to produce 

a program requirements document for the TAMR final 

investment decision because it initially indicated the 

program would primarily be a technical refresh effort. 

However, the effort has since become significantly 

more complex. 

FAA later indicated that in lieu of a requirements 

document, it would update the TAMR system-level 

specification document to include any new 

requirements. However, this specification document is 

not an effective substitute because it is much less 

specific, pertinent, and focused. 

Independent 

Evaluation 

Review (IER)

• Prior to a final investment 

decision, FAA’s Investment 

Planning and Analysis office 

validates investments by 

assessing the cost, benefits, 

risks, and schedule for a 

program. The group presents 

its findings to FAA senior 

management in the form of an 

IER.

The IER document contained two conflicting 

implementation schedules. Additionally the IER office 

appears to have verified the preliminary rather than the 

approved baseline schedule referenced in the 

acquisition program baseline. 

The preliminary schedule, referenced in the business 

case analysis, did not include the 2 years added to the 

approved baseline schedule. The extension of several 

baseline milestones in the approved schedule will likely 

result in additional costs over the $438 million 

approved.

Acquisition 

Program 

Baseline

• Identifies the binding 

agreement between FAA 

management and the program 

office concerning the cost, 

schedule, and capabilities that 

the program will provide for a 

major acquisition.

FAA did not clearly demonstrate that it performed a full 

analysis on the schedule identified in the document or 

the cost to implement the program. 

Source: OIG analysis and FAA Acquisition Management System
14

Ultimately, FAA’s work products supporting the TAMR business case lacked 

clarity because the information presented was not finalized, well documented, or 

validated. FAA officials indicated they were under pressure to make a decision on 

TAMR so that other interdependent NextGen programs could move forward with 

their schedule priorities. However, without clear analysis supporting a 

justification, decision makers cannot be assured that FAA’s current 

implementation strategy is necessarily the most effective option. 

CONCLUSION

Successfully modernizing terminal automation is critical to ensuring safety in the 

NAS, enhancing capacity, and minimizing delays along with realizing future 

NextGen capabilities. FAA’s success depends on finalizing requirements, 

establishing and following comprehensive and realistic schedules, and setting clear 

cost targets for all aspects of the effort. Until FAA does so, its terminal automation 

modernization project remains at risk of uncontrolled cost increases, unanticipated 

schedule delays, and ineffective systems that do not provide the capabilities 

needed for a safe and modern NAS.

RECOMMENDATIONS

To improve FAA’s effectiveness in achieving terminal modernization, we 

recommend the Agency:

1. Develop a requirements document for TAMR Phase 3 Segment 1 to ensure 

the operational and technical requirements for the 11 large TRACONs are 

specific, pertinent, and focused on requirements needed to transition 

CARTS to STARS.

2. Develop and implement a formal process to effectively manage, budget for, 

and incorporate new requirements (gaps) as they arise in the terminal 

modernization acquisition strategy. 

3. Develop a comprehensive approved baseline schedule for TAMR Phase 3, 

Segment 1 that includes: (a) Initial Operating Capability and Operational 

Readiness dates for each of the 11 sites that STARS will replace; 

(b) timeframes for testing and validating new software and hardware 

requirements to support STARS deployment; and (c) software testing to the 

maximum extent possible to ensure products are suitable for deployment.

4. Update, verify, and validate the accurate and complete cost, schedule, and 

benefits for TAMR Phase 3, Segment 1, as prescribed by AMS.
15

AGENCY COMMENTS AND OFFICE OF INSPECTOR GENERAL 

RESPONSE 

We provided a draft of this report to FAA on March 1, 2013, and received its 

response on April 23, 2013, which is included as an appendix to this report. FAA 

concurred with recommendations 2, 3, and 4, and partially concurred with 

recommendation 1. FAA’s proposed actions for recommendations 1, 2, and 4 meet 

the intent of our recommendations and include reasonable timeframes for 

implementation. Although FAA partially concurred with recommendation 1, 

indicating that it would not be beneficial to create a new requirements document 

for the 11 large TRACONs at this time, the Agency proposed an alternative course 

of action that meets the intent of our recommendation. However, FAA’s response 

to recommendation 3 does not address our concern, and we are requesting 

additional information, as detailed below.

FAA acknowledged that timeframes for TAMR baseline schedule activities may 

be adjusted due to unanticipated risk factors, such as local needs for site 

adaptation. However, we remain concerned with the Agency’s position that it 

already maintains a working schedule for the TAMR program that includes these 

timeframes. As our report indicates, independent evaluations of TAMR’s working

schedule highlighted major concerns regarding the contractor’s ability to complete 

initial operating capability (IOC) for all 11 sites by 2015. For example, MITRE 

reported in September 2011 that FAA’s software development efforts for STARS 

would likely require additional time for operational testing and acceptance. To 

address this concern, FAA senior management approved a baseline schedule that 

pushed the completion of IOC for all 11 sites to 2016. However, this baseline

schedule lacks details on the specific timeframes for testing and validating 

requirements. Accordingly, we are requesting that the Agency update the approved 

baseline schedule with the timeframes for testing and validating new software and 

hardware requirements. This will provide assurance that the program’s official 

baseline schedule has allotted adequate time for operational testing and 

acceptance, and reduce overall confusion regarding discrepancies between FAA’s 

baseline and working schedules. We consider recommendation 3 open and 

unresolved pending receipt of this information.

ACTIONS REQUIRED 

FAA’s planned actions for recommendations 1, 2, and 4 are responsive and we 

consider these recommendations resolved but open pending completion of the 

planned actions. For recommendation 3, we are requesting that the Agency 

provide additional information, as detailed above. In accordance with Department 

of Transportation Order 8000.1C, we request that FAA provide this information 

within 30 calendar days.
16

We appreciate the courtesies and cooperation of FAA representatives during this 

audit. If you have any questions concerning this report, please call me at (202) 

366-0500 or Kevin Dorsey, Program Director, at (202) 366-1518.

#

cc: FAA Deputy Administrator

FAA Chief of Staff

DOT Audit Liaison, M-1

FAA Audit Liaison, AAE-100
17

Exhibit A. Scope and Methodology

EXHIBIT A. SCOPE AND METHODOLOGY

We conducted this performance audit from March 2011 through March 2013 in 

accordance with generally accepted Government auditing standards. Those 

standards require that we plan and perform the audit to obtain sufficient, 

appropriate evidence to provide a reasonable basis for our findings and 

conclusions based on our audit objectives. We believe that the evidence obtained 

provides a reasonable basis for our findings and conclusions based on our audit 

objectives.

To determine whether FAA’s acquisition strategy for terminal modernization 

effectively addresses technological and operational risks, we focused our review 

on two key issues: (1) risks related to developing and implementing software and 

hardware requirements, and (2) whether FAA’s final investment decision resulted 

in a reliable cost and schedule baseline.

We analyzed key planning and funding documents for the TAMR program. We 

reviewed various documents received from the FAA’s TAMR Program Office and 

the FAA’s William J. Hughes Technical Center in Atlantic City, NJ. These 

documents include: TAMR planning documents, investment decision documents,

status briefings, requirements documents, schedule proposals, FAA budget 

documents, and NextGen planning documents. We also analyzed contractor

documentation relating to STARS and CARTS, and MITRE industry reports 

associated with TAMR.

We interviewed TAMR program office officials at FAA Headquarters and the 

Technical Field Operational Support group at the FAA William J. Hughes 

Technical Center in Atlantic City, NJ. We also interviewed various FAA 

departments including: Terminal Services-Safety and Operations Support,

Terminal Services-Logistics Management, Investment and Planning Analysis 

Office, and Acquisition and Business (Acquisition Executive). We also met with 

contracting officials from Raytheon Company and Lockheed Martin Corporation.

We also consulted with OIG’s Offices of Legal Counsel and Acquisition and 

Procurement Audits to obtain their insight on FAA’s financing arrangement with 

the contractor for TAMR activities.

We conducted site visits at TRACONs slated for STARS in TAMR Phase 3,

Segment 1: Potomac, Atlanta, Dallas, and Northern California. We also conducted 

site visits at Air Traffic Control Towers (ATCT)/TRACONs slated for TAMR 

Phase 3, Segment 2: Austin, TX, Waterloo, IA and Atlantic City, NJ. We also 

visited an ATCT/TRACON that currently uses STARS in Philadelphia, PA.
18

Exhibit B. Organizations Visited or Contacted

EXHIBIT B. ORGANIZATIONS VISITED OR CONTACTED

Federal Aviation Administration (FAA)

Headquarters

Terminal Automation Modernization-Replacement Washington, DC

Program Office

Investment & Planning Analysis (IP&A) Office Washington, DC

Acquisition and Business Office Washington, DC

Terminal Services Washington, DC

Sites 

William J. Hughes Technical Center Atlantic City, NJ

Philadelphia ATCT/TRACON Philadelphia, PA

Atlanta TRACON Peachtree City, GA

Potomac TRACON Warrenton, VA

Northern California TRACON Mather, CA

Dallas TRACON Dallas-Ft Worth, TX

Austin, TX ATCT/TRACON Austin, TX

Waterloo, IA ATCT/TRACON Waterloo, IA

Atlantic City, NJ ATCT/TRACON Atlantic City, NJ

Industry

Raytheon Company Washington, DC

Lockheed Martin Corporation Washington, DC

MITRE Corporation McLean, VA
19

Exhibit C. Program Schedule for TAMR Phase 3, Segment 1

EXHIBIT C. PROGRAM SCHEDULE FOR TAMR PHASE 3,

SEGMENT 1 

Facility IOC ORD

Dallas Fort-Worth TRACON September 2014 May 2015

Northern California TRACON TBD TBD

Atlanta TRACON TBD TBD

Southern California TRACON TBD TBD

Minneapolis TRACON October 2015 TBD

Potomac TRACON TBD TBD

Denver TRACON TBD TBD

St. Louis TRACON TBD TBD

Louisville TRACON TBD TBD

Chicago TRACON TBD TBD

New York TRACON October 2016 October 2017

Source: FAA’s Acquisition Program Schedule Baseline for TAMR Phase 3, Segment 1.

IOC - Initial Operating Capability is defined as site using TAMR Phase 3 hardware and software components 

operationally to control air traffic over a 2-4 hour period, typically on a mid-shift.

ORD - Operational Readiness Date.

TBD - To be determined.
20

Exhibit D. Major Contributors to This Report

EXHIBIT D. MAJOR CONTRIBUTORS TO THIS REPORT 

Name Title

Kevin Dorsey Program Director 

Arnett Sanders Project Manager

Constance Hardy Senior Analyst

Katrina Knight Senior Auditor

Kiesha Henson Auditor

Jennifer Hoffman Analyst

Amy Berks Senior Counsel

Andrea Nossaman Senior Writer-Editor

Audre Azuolas Writer-Editor
21

Appendix. Agency Comments

APPENDIX. AGENCY COMMENTS

Federal Aviation

Administration

Memorandum

Date: April 23, 2013

To: Jeffrey B. Guzzetti, Director, Assistant Inspector General for Aviation and

Special Program Audits

From: H. Clayton Foushee, Director, Office of Audit and Evaluation, AAE-1

Subject: Federal Aviation Administration’s (FAA) Response to Office of Inspector 

General (OIG) Draft Report: FAA’s Acquisition Strategy for Terminal 

Modernization

Air Traffic Control modernization is providing continuous improvement using well 

established processes for transitioning to new systems and methods that will enable the 

National Airspace System (NAS) to handle increasing traffic density while further improving 

aviation safety. Presently, the FAA is operating both Standard Terminal Automation 

Replacement System (STARS) and Common Automated Radar Terminal System (CARTS) 

depending on the location. This increases the complexity of operations, training, and raises 

costs as compared to operating a single system. Further, completing the transition to a 

unified system using STARS is necessary to achieve the long term operational benefits 

envisioned under NextGen. 

The FAA has already completed considerable progress with the implementation of STARS 

under the first phases of the program, and is now focusing much of its effort for this program 

on replacing the aging CARTS system at the 11 most complex Terminal Radar Approach 

Controls (TRACONS). While the OIG report presents a reasonable summary of the risks 

associated with the remaining efforts, it is important to realize that the risks discussed were 

derived from technical reports performed by contractors for the FAA, such as MITRE, and 

from discussions with FAA employees, and review of FAA documents. The key here is that 

the risks are known, and are being managed. For the handful of issues that continue to be 

identified, such as site specific “gaps,” these too are both typical and anticipated.

Finally, the OIG draft report identified a number of minor procedural issues relating to the 

fulfillment of several individual elements of FAA’s Acquisition Management System 

(AMS). While some schedules could have been better synchronized earlier in the process, 

some analyses could have been better documented, and there are still questions relating to 
22

Appendix. Agency Comments

out-year budget uncertainties, these issues do not modify the overall requirement for the 

system or the need to proceed with implementation. 

FAA has carefully reviewed the information in the report and will ensure that it 

accommodates recommended actions to the fullest extent practicable and has effective 

systems and processes for identifying and managing risks as it completes its efforts on 

Terminal Automation Management/Replacement (TAMR). As described more fully in 

response to the recommendations, FAA has recognized opportunities to improve risk 

management processes, by tightening up overall and site specific requirements management, 

better synchronizing scheduling activities, and incorporating other lessons learned from other 

NAS modernization projects, such as En Route Automation Modernization (ERAM).

RECOMMENDATIONS AND RESPONSES

Recommendation 1: Develop a requirements document for TAMR Phase 3 Segment 1 to 

ensure the operational and technical requirements for the 11 large TRACONs are specific, 

pertinent, and focused on requirements needed to transition CARTS to STARS.

FAA Response: Partially Concur. The FAA recognizes the importance of complete and up 

to date requirements documentation for system acquisition. It has reviewed the 

documentation available in the combined requirements document for TAMR Phase 3 

Segments 1 and 2 in arriving at its Joint Resources Council (JRC) Final Investment Decision 

(FID) and has determined that there is no demonstrable benefit to be gained by creating a 

separate or new document for TAMR Phase 3 Segment 1 requirements. As the system has 

been implemented over time, the program maintains an overall system specification/ 

requirements document for STARS System and Subsystem Specification (SSS). 

The agency has determined that its needs can be addressed sufficiently through the 

configuration and the requirements management processes to ensure that specific and 

focused requirements are iteratively incorporated into existing documentation. As part of 

this requirements management process, the Agency established a process for managing site 

specific requirements. As the specific site surveys and planning activities continue and are 

completed at each of the 11 CARTS TRACON sites in the program baseline, potential 

requirements changes are evaluated and administered through the Engineering Change 

Proposal (ECP) Working Group. The ECP’s role is to ensure a consistent and measured 

transition from CARTS to STARS at each site. 

The FAA is currently improving upon the existing process to formalize the requirements 

management process (currently under development, as described in response to 

Recommendation 2 below), which will simultaneously maintain the STARS SSS through 

the program baseline. The resulting requirements document, reflecting the operational and 

technical requirements for the 11 large TRACONs, will be finalized by December 31, 2016 

(in conjunction with the last site Initial Operating Capability (IOC) Acquisition Program 

Baseline (APB) milestone). As part of Configuration Management process for the program, 

the requirements document will be updated iteratively up to the last site deployment. At the 

completion of deployment, the requirements document will represent the fielded ‘as-is’ 

system.
23

Appendix. Agency Comments

Recommendation 2: Develop and implement a formal process to effectively manage, 

budget for, and incorporate new requirements (gaps) as they arise in the terminal 

modernization acquisition strategy.

FAA Response: Concur. The FAA is addressing this process through two approaches: 

1. Within the Air Traffic Organization (ATO), Mission Support (AJV), the Program 

Management Office, and Terminal Services are developing new processes and 

standards to ensure: 1) the enterprise standards for operational requirements; 2) the 

needs of individual service units; and 3) that the implementation requirements of 

specific programs and systems are consistent and compatible. AJV is leading the 

development of a requirements management process standard for the ATO, which it 

plans to complete by the end of Fiscal Year 2013. 

2. The TAMR program is creating local process standards for formally managing 

requirements changes. Software is the responsibility of the Requirements Tiger Team 

(RTT), which validates proposed changes to the TAMR baseline and translates them 

into specific contractual requirements as a formal work plan for the contractor. The 

RTT also ensures that all contract requirements are formally introduced via ECPs to 

the STARS SSS. The ECP Working Group, working as a collaborative forum 

between the FAA and its vendor partner, is accountable for the review and disposition 

process to support the thorough and expeditious evaluation, disposition, and 

implementation of STARS hardware, firmware, and related Commercially Available 

Software ECPs. This work is planned to be completed by July 31, 2013.

Recommendation 3: Develop a comprehensive approved baseline schedule for TAMR 

Phase 3, Segment 1 that includes: (a) Initial Operating Capability and Operational 

Readiness dates for each of the 11 sites that STARS will replace; (b) timeframes for testing 

and validating new software and hardware requirements to support STARS deployment; 

and (c) software testing to the maximum extent possible to ensure products are suitable for 

deployment.

FAA Response to 3 (a): Concur. The FAA uses the working schedule for the TAMR 

program as the implementation mechanism for the APB. This includes IOC and Operational 

Readiness Date targets for each of the 11 large TRACON sites that STARS will replace. 

This approach, in keeping with the AMS best-practices and guidance, also builds upon 

lessons-learned from the ERAM program and other large-scale acquisitions. The target 

timeframes for these activities may adjust within the confines of the APB milestones 

established at FID to account for either unplanned or unanticipated risk factors. These 

include variables such as local needs for site adaptation, site readiness, the availability of site 

resources, and any other NAS changes that may have emerged in the interim. 

Per AMS requirements, the program will undergo an Integrated Baseline Review (IBR) in 

May of 2013 and will employ the recommendations from this review to further enhance 

project management processes. The IBR results and recommendation implementation results 

will be provided to the OIG as part of our request to close this recommendation by 

September 30, 2013.
24

Appendix. Agency Comments

FAA Response to 3 (b): Concur. The program’s working schedule maintains this 

information, and the schedule for this work is governed through periodic Program 

Management Reviews (PMR) and other forums within the TAMR program office. TAMR 

schedule files that demonstrate timeframes for testing and validating new software and 

hardware requirements to support STARS deployment are located on a central Knowledge 

Sharing Network and maintained by the program office. Given that the program’s working 

schedule maintains these data, and that the use of the working schedule to manage this 

information is in alignment with AMS guidance, the FAA requests that this request be 

closed.

FAA Response to 3 (c): Concur. In the area of software testing, the TAMR program is also 

incorporating lessons learned from the ERAM program. The test strategy for TAMR 

includes a range of structured events with entrance and exit criteria for deploying new 

software to TRACONs. This includes: a) early user involvement events involving controller 

system testing prior to government acceptance; b) informal risk mitigation testing involving 

the program office and vendor prior to software delivery at the William J. Hughes Technical 

Center test facility; and c) formal operational test and evaluation including software bug 

fixes, engineering change verification, regression tests, and operational evaluation by users 

prior to software delivery to the sites. The TAMR program office is in the process of 

codifying this process in standard documentation, which it plans to complete by September 

30, 2013.

Recommendation 4: Update, verify, and validate the accurate and complete cost, 

schedule, and benefits for TAMR Phase 3, Segment 1, as prescribed by AMS.

FAA Response: Concur. The FAA is guided by the AMS for all of its acquisition programs, 

including TAMR. There are two key controls that serve to verify and validate the accuracy 

and completeness of the programs cost performance baseline. These controls are subsequent 

to the program’s FID and implementation of the program, which is ongoing. The two key 

controls include:

1. Earned-Value Management (EVM) – TAMR is implementing a formal change 

control board for all program changes that impact either individual components of the 

TAMR program or the TAMR portfolio (across phases and segments) as a whole. 

This was implemented in February 2013, and will be documented in a Change 

Management Plan and will include the development of standard operating procedures 

to guide its execution. This Change Management Plan will be completed by 

September 30, 2013.

2. Post Implementation Review (PIR) – the PIR is used by the FAA to validate benefits 

accrual and to answer the following questions:

o Did the FAA get what it asked for?

o Is the investment program providing the service customer needs?

o Are there any lessons learned about the acquisition management process?
25

Appendix. Agency Comments

This review will begin by March 31, 2014, and a report is anticipated approximately 

six months after the work begins. The high-level TAMR Phase 3 Segment 1 PIR 

strategy is as follows:

o Stringent monitoring of the integrated program schedule and EVM provides 

the means to measure and evaluate program adherence to cost and schedule 

baselines during the solution implementation phase; and, 

o Examine actual cost, schedule, benefits/improvement, and mission outcome 

performance against stated baseline goals.