HOMELAND 

DEFENSE

Continued Actions 

Needed to Improve 

Management of Air 

Sovereignty Alert 

Operations

Report to Congressional Requesters

January 2012

GAO-12-311

United States Government Accountability Office

GAO
United States Government Accountability Office

Highlights of GAO-12-311, a report to 

congressional requesters 

January 2012

HOMELAND DEFENSE

Continued Actions Needed to Improve Management 

of Air Sovereignty Alert Operations

Why GAO Did This Study

In the 11 years since September 11, 

2001, the U.S. government has put 

forth extensive efforts to protect the 

nation’s aviation sector and airspace.

These efforts include air sovereignty 

alert (ASA) operations, for which the 

Air Force provides personnel and fully

fueled, fully armed aircraft sitting on 

constant alert at 18 sites across the 

United States. In 2009, GAO found 

shortcomings in the Department of 

Defense’s (DOD) management of ASA 

operations, leading to a number of 

GAO recommendations. For this 

report, GAO examined the extent to 

which (1) the Air Force has 

implemented GAO’s 2009 

recommendations, (2) DOD has 

implemented a risk-based

management approach for ASA 

operations, and (3) the Air Force has

accurately identified expenditures for 

ASA operations. To do so, GAO 

analyzed relevant strategies, planning 

documents, guidance, and expenditure 

data; and interviewed North American 

Aerospace Defense Command 

(

What GAO Recommends

NORAD), Air Force, National Guard 

Bureau, and other DOD officials.

Congress may wish to consider 

requiring the Air Force to fully 

implement GAO’s 2009 

recommendations. In addition, GAO

recommends that DOD improve its risk 

management of ASA operations and 

improve the Air Force’s ability to 

accurately identify ASA expenditures.

DOD fully or partially agreed with all of

GAO’s recommendations. 

 

What GAO Found

The Air Force has not fully implemented the recommendations from GAO’s 2009 

report. With regard to GAO’s recommendation that the military services should 

formally assign ASA duties to the units that consistently conduct them and 

ensure that the readiness of those units is fully assessed, the Air Force did so. 

However, the National Guard Bureau is considering reversing that action 

because it believes that the recommendation can be better addressed through 

the Air Force’s standard deployment process. The Air Force has also not 

established a timetable to implement ASA as a steady-state mission; has not 

developed and implemented a plan to recapitalize the aging fighter aircraft that 

conduct ASA operations before the end of their service lives; and, when ASA 

units are deployed to support other ongoing operations, the Air Force continues 

to identify replacement units to perform the ASA mission on an ad hoc basis. All 

of the above were related to recommendations GAO made to the Air Force in its 

2009 report. Separately, GAO found considerable confusion about the 

capabilities associated with ASA operations in part because, in September 2011, 

NORAD stopped using the term “air sovereignty alert” and created a new term, 

“aerospace control alert” (ACA), without clearly defining ACA or the missions that 

are now included within it. 

DOD has taken a series of actions for ASA operations that are consistent with a 

risk-based management approach. However, several key actions have yet to be 

taken that would enable the department to better balance risk and costs. Riskbased management includes conducting routine risk assessments that evaluate 

threats, vulnerabilities, and criticality of assets, as recommended in GAO’s 2009 

report, and selecting between alternative courses of action to mitigate risk and 

make decisions about allocating resources. Although threats to the nation’s air 

sovereignty continue to emerge and evolve, GAO found that DOD is unable to 

measure the extent to which ASA helps to achieve the department’s homelanddefense goal of securing the United States from direct attack because DOD has 

not established performance measures. NORAD has not conducted routine risk 

assessments of ASA operations. DOD has also yet to conduct a cost-benefit 

analysis for two of the three alternatives to current ASA operations that GAO 

evaluated. Adopting a more-rigorous risk-based management approach—

including balancing risk and costs—would help policymakers within DOD and 

elsewhere more effectively allocate finite DOD resources.

Weak internal controls limit the ability of the Air Force and National Guard 

Bureau to accurately identify ASA expenditures. GAO analyzed the fiscal year 

2010 expenditure information that the Air Force and National Guard Bureau

submitted to Congress along with DOD’s fiscal year 2012 budget justification and 

found the reported expenditures of more than $246 million to be inaccurate. For 

example, GAO found that the Air Force overstated ASA flying-hour expenditures 

by at least $22 million and included expenditures related to national specialsecurity events, which are not part of ASA operations. GAO found that the Air 

Force’s ability to identify ASA expenditures is limited by unclear roles and 

responsibilities for programming and budgeting and a lack of guidance on 

defining and tracking ASA expenditures. These types of internal controls are 

important to ensuring basic accountability, maintaining funds control, and 

preventing fraud and abuse. 

View GAO-12-311. For more information, 

contact Brian J. Lepore at (202) 512-4523 or 

leporeb@gao.gov. 
Page i GAO-12-311 Homeland Defense 

Letter 1 

Background 5

The Air Force Has Not Fully Implemented the Recommendations 

from Our 2009 Report, and Mission Confusion Persists 8

DOD Has Not Fully Implemented a Risk-Based Management 

Approach That Balances Risk and Costs for ASA Operations 12

Weak Internal Controls Limit the Ability of the Air Force and 

National Guard Bureau to Accurately Identify ASA Expenditures 19

Conclusions 24

Matter for Congressional Consideration 24

Recommendations for Executive Action 24

Agency Comments and Our Evaluation 25

Appendix I Scope and Methodology 31

Appendix II Air Force’s Implementation of Our 2009 Recommendations as of 

November 2011 33

Appendix III Comments from the Department of Defense 35

Appendix IV GAO Contacts and Staff Acknowledgments 39

Related GAO Products 40

Table

Table 1: Nonexhaustive GAO-Developed List of Potential Measures 

of Effectiveness and Measures of Performance for ASA 

Operations 14

Figures

Figure 1: GAO’s Risk-Based Management Framework 6

Contents
Page ii GAO-12-311 Homeland Defense 

Figure 2: National Infrastructure Protection Plan’s RiskManagement Framework 7

Figure 3: Air Force’s Implementation of Our 2009 

Recommendations as of November 2011 9

Figure 4: Air Sovereignty Alert (ASA) versus Aerospace Control 

Alert (ACA) 11

Abbreviations

ACA Aerospace Control Alert

ASA Air Sovereignty Alert

DOD Department of Defense

NORAD North American Aerospace Defense Command

OMB Office of Management and Budget

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-12-311 Homeland Defense 

United States Government Accountability Office 

Washington, DC 20548

January 31, 2012

Congressional Requesters

Since the September 11, 2001, terrorist attacks, the U.S. government has 

taken extensive efforts to protect the aviation sector and airspace over 

the United States—including the establishment of restricted flight zones; 

installation of hardened cockpit doors; enhanced screening of airline 

passengers and baggage before flights; increased use of air marshals on 

selected flights; the vigilance of civilian passengers and flight attendants 

aboard commercial aircraft; and having fully fueled, fully armed jets ready 

to scramble at any time in response to airborne threats. As a result of 

these efforts, the Transportation Security Administration believes that a 

hijacking scenario over the United States remains a possibility although it 

is less likely to occur today. Other threats to the nation’s air sovereignty 

continue to emerge and evolve.1 As a part of the federal government’s 

overall efforts to protect U.S. airspace, the Department of Defense (DOD) 

performs Operation Noble Eagle, which consists of several operations, 

including the maintenance of fighter aircraft and trained personnel on alert 

24 hours a day, 365 days a year, at 18 alert sites across the United 

States.2

As a component of Operation Noble Eagle, ASA operations are 

conducted by several DOD components under the command of North 

Since its inception in 2003, these efforts have been commonly 

referred to as air sovereignty alert (ASA) operations. The ASA sites serve 

to deter, respond to, and if necessary, defeat airborne threats over the 

United States and Canada. In effect, these fighter jets can be the last line 

of multiple layers of defense.

 

1

Airborne threats to the nation can come from a range of vehicles (e.g., commercial, 

general aviation, cargo, ultra light, and military aircraft and cruise missiles) and modes of 

attack (e.g., detonating explosives on board, hijacking, crashing the aircraft, and launching 

of weapons).

2

Operation Noble Eagle, which was initiated after the terrorist attacks of September 11, 

2001, is DOD’s umbrella homeland defense operation for North America and Hawaii. 

There are a variety of missions that could be conducted underneath this umbrella 

operation. For example, North American Aerospace Defense Command (NORAD) could 

conduct irregular air patrols above major metropolitan areas and critical infrastructure 

facilities as well as maintain an alert force of fighter, tanker, and control aircraft. Aircraft 

sorties and alert commitments are based on a tiered response system. As threat levels 

intensify, the number of aircraft on alert and on patrol increases.

 
 

Page 2 GAO-12-311 Homeland Defense 

American Aerospace Defense Command (NORAD) for the airspace over 

North America, which includes Alaska and the continental United States, 

and under the command of U.S. Pacific Command for the airspace over 

the state of Hawaii and U.S. territories in the Pacific. The Joint Chiefs of 

Staff have directed the Air Force (among other components) to provide 

support for ASA operations. Generally, Air National Guard aircraft and 

personnel are used for alert, since the National Guard Bureau believes 

that ASA operations are consistent with the Air National Guard’s 

homeland mission. Both the Air Force’s Air Combat Command and the 

National Guard Bureau3

In January 2009, we issued a report that identified a number of 

shortcomings in DOD’s management of ASA operations.

believe that Air National Guard units can conduct 

alert duty with less effect on overseas missions and at a lower cost than 

active-duty Air Force units. Consequently, Air National Guard units are on 

alert at 17 of the 18 ASA sites. In addition, units at all 18 ASA sites (i.e., 

both Air National Guard and active-duty personnel) are dual-tasked to 

conduct both expeditionary missions, such as Operation Enduring 

Freedom in Afghanistan, and ASA operations.

4

 

3

The National Guard Bureau is a joint function within the DOD that, among other things, 

participates with staffs from the Department of Army and the Department of Air Force in 

the formulation, development, and coordination of all programs, policies, concepts, and 

plans pertaining to or affecting the National Guard, the Army National Guard of the United 

States, and the Air National Guard of the United States. The Air National Guard of the 

United States is a reserve component of the U.S. Air Force.

First, we found 

that NORAD faced difficulty determining the appropriate levels and types 

of units, personnel, and aircraft for ASA operations. We also found that, 

after 8 years of operations, the Air Force had not yet established ASA as 

a steady-state mission, which would include integrating ASA operations 

into the Air Force’s planning, programming, and budgeting cycle. Partly 

resulting from the absence of a steady-state mission, NORAD had only 

partially assessed the readiness of units that consistently conduct ASA 

operations to ensure that those units were organized, trained, and 

equipped to perform ASA operations. In addition, we found that the Air 

Force faced significant challenges replacing or extending the service life 

of aging fighter aircraft, to the extent that, if aircraft were not replaced, 11 

4

GAO, Homeland Defense: Actions Needed to Improve Management of Air Sovereignty 

Alert Operations to Protect U.S. Airspace, GAO-09-184 (Washington, D.C.: Jan. 27, 

2009). In addition, GAO testified on this topic in April 2009, GAO, Homeland Defense: 

Actions Needed to Address Management of Air Sovereignty Alert Operations to Protect 

U.S. Airspace, GAO-09-612T (Washington, D.C.: Apr. 22, 2009).
 

Page 3 GAO-12-311 Homeland Defense 

of the 18 ASA sites could be without aircraft by 2020. On the basis of 

those findings, we identified a number of actions that DOD needed to take 

to improve management of ASA operations. Specifically, we 

recommended that DOD should conduct routine risk assessments as part 

of a risk-based management approach, implement ASA as a steady-state 

mission, and develop plans to address future challenges.

GAO and other federal agencies have long advocated the use of a riskbased management approach to enhance preparedness against potential 

terrorist threats.5

On the basis of your request that we review DOD’s actions on our 

January 2009 recommendations, other issues associated with ASA 

operations, and the Air Force’s ability to sustain those capabilities, this 

report examines the extent to which (1) the Air Force has implemented 

the recommendations made to the service in our 2009 report, (2) DOD 

has implemented a risk-based management approach for ASA 

operations—including our 2009 recommendation for routine risk 

assessments—that balances risk with cost, and (3) the Air Force 

accurately identified expenditures for its ASA operations. We are also 

issuing a classified version of this report. That version includes a 

classified appendix of our analysis of NORAD’s ASA basing model.

Risk management is a systematic, analytical process to 

determine the likelihood that a threat will harm physical assets or 

individuals and then to identify actions to reduce risk and mitigate the 

consequences of an attack. The principles of risk management 

acknowledge that while risk generally cannot be eliminated, enhancing 

protection from known or potential threats can serve to reduce risk. Key 

elements of risk management include assessing threats, vulnerabilities, 

and criticality of assets and selecting between alternative courses of 

action to mitigate risk. As we have previously reported, risk management 

can help policymakers make informed decisions, take actions, and 

prioritize resource investments under conditions of uncertainty. For 

additional information, see the Related GAO Products list at the end of 

this report.

 

5

GAO, Homeland Security: Key Elements of a Risk Management Approach, 

GAO-02-150T (Washington, D.C.: Oct. 12, 2001); Intelligence, Surveillance, and 

Reconnaissance: DOD Needs a Strategic, Risk-Based Approach to Enhance Its Maritime 

Domain Awareness, GAO-11-621 (Washington, D.C.: June 20, 2011); Joint Strike Fighter: 

Strong Risk Management Principles Essential as Program Enters Most Challenging 

Phase, GAO-09-711T (Washington, D.C.: May 20, 2009); DOD Instruction, 3020.45, 

Defense Critical Infrastructure Program (DCIP) Management (Apr. 21, 2008). 
 

Page 4 GAO-12-311 Homeland Defense 

To determine the extent to which the Air Force has implemented the 

recommendations from our 2009 report, we reviewed ASA-related 

guidance, plans, and other documents from the Office of the Assistant 

Secretary of Defense for Homeland Defense and Americas’ Security 

Affairs, Air Force’s homeland defense office, Air Force’s Air Combat 

Command, National Guard Bureau, and 3 of the 18 ASA units. We also 

interviewed officials from these organizations. To determine the extent to 

which DOD has adopted a risk-based management approach to balance 

risk and costs for ASA operations, we evaluated documents and 

interviewed officials from NORAD, the Office of the Assistant Secretary of 

Homeland Defense and Americas’ Security Affairs, the Air Force, and the 

National Guard Bureau. On the basis of our analyses of these documents 

and discussions, we compared actions that DOD has taken to three of the 

five phases within GAO’s risk-based management framework.6 To 

determine the extent to which the Air Force was able to accurately identify 

ASA expenditures, we analyzed the fiscal year 2010 expenditure 

information the Air Force included in ASA budget justification materials 

provided to Congress in support of the fiscal year 2012 budget and 

documented the steps taken to identify those expenditures through 

interviews, e-mails, and related documentation. We also compared the 

practices used by the Air Force to prepare the ASA budget justification 

against best practices for producing cost estimates.7

 

6

We could not evaluate the extent to which DOD implemented the last two phases—

management selection and implementation and monitoring—because (1) DOD had not 

fully implemented the first three phases of GAO’s risk-based management framework, 

which are essential for the latter two phases, and (2) DOD was still in the decision-making 

stage for the alternatives that we identified.

To determine the 

expenditures of each ASA site, we developed a survey that we sent out to 

the Air National Guard and active-duty Air Force ASA units to gather 

fiscal year 2009 and 2010 expenditures. We were provided information 

from 18 of the 19 ASA sites that incurred expenditures for ASA 

7

GAO, GAO Cost Estimating and Assessment Guide: Best Practices for Developing and 

Managing Capital Program Costs, GAO-09-3SP (Washington, D.C.: March 2009).
 

Page 5 GAO-12-311 Homeland Defense 

operations.8 Of the 18 sites that provided us information, 3 of the sites 

provided us incomplete information.9

We conducted this performance audit from March 2011 to January 2012 

in accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that 

the evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives. Additional information on our 

scope and methodology appears in appendix I.

ASA operations consist of ground operations that take place before 

fighter aircraft take off, including such activities as maintaining the fighter 

aircraft. They also include those activities that may take place after a unit 

receives an alert from NORAD, but before the aircraft are airborne. For 

example, pilots and maintenance personnel may rush from their nearby 

lodging facility to the alert aircraft facility, where maintenance personnel 

conduct final preparations while the pilots sit in their aircraft awaiting 

further instruction (battle station). Alternatively, pilots may taxi the aircraft 

to the end of the runway and await further instruction (runway alert) or 

take off in response to the alert (scramble).

Once aircraft take off, an Air National Guard pilot converts from Title 32 

status under the command and control of the state governor to federal 

 

8

There are 18 steady-state ASA sites. However, in fiscal year 2010, McEntire Joint 

National Guard Base, South Carolina, constructed and upgraded facilities since it was 

going to take on alert duty from Shaw Air Force Base, South Carolina, at the beginning of 

fiscal year 2011. Therefore, we included McEntire Joint National Guard Base’s fiscal year 

2010 expenditures in our survey efforts. Budget officials from Elmendorf Air Force Base, 

Alaska, told us that they were unable to provide us accurate ASA expenditure information 

because the base does not track ASA expenditures separately from its other mission 

expenditures.

9

Budget officials from Shaw Air Force Base, South Carolina; Homestead Air Force Base, 

Florida; and Hickam Air Force Base, Hawaii, were unable to provide us information for 

some aspects of the operation, such as flying hour or personnel expenditures.

Background

ASA Operations
 

Page 6 GAO-12-311 Homeland Defense 

Title 10 status under the command and control of NORAD.10 Within limits, 

NORAD can increase personnel, aircraft, and the number of ASA sites 

based on changes in the threat conditions.

GAO has previously reported that a risk-based management approach 

helps policymakers make informed decisions and prioritize resource 

investments. Risk management is a widely endorsed strategy for helping 

decision makers allocate finite resources and take action under conditions 

of uncertainty. We have previously recommended a five-phase risk-based 

management approach as shown in figure 1.

Figure 1: GAO’s Risk-Based Management Framework

The framework is dynamic and new information can be entered at any 

phase. The framework can be used to inform agency officials and 

 

10Title 32 and Title 10 refer to sections of the United States Code. 

Risk-Based Management 

Approach
 

Page 7 GAO-12-311 Homeland Defense 

decision makers of the basic components of a risk-based management 

approach or can be used as a stand-alone guide. The risk-based 

management approach, as outlined above is designed to be flexible in 

that the approach may be applied at various organizational levels from a 

department or a multiagency organization down to specific projects or 

operations, such as ASA operations.

As we previously reported, because there is no one uniformly accepted 

approach to risk management, terms and activities may differ across 

applications. For example, the Department of Homeland Security’s 2009 

National Infrastructure Protection Plan identifies a six-phase riskmanagement framework as shown in figure 2.11

Figure 2: National Infrastructure Protection Plan’s Risk-Management Framework

The overall difference 

between this framework and GAO’s is that the National Infrastructure 

Protection Plan’s framework splits GAO’s risk-assessment phase into two 

separate phases.

According to the 2009 National Infrastructure Protection Plan, the core 

criteria for risk assessments include information needed to understand 

and address each of the following components of risk—threat, 

vulnerability, and consequence. Any approach that omits the substance of 

the steps may result in resources that are not targeted to the highest 

 

11Department of Homeland Security, National Infrastructure Protection Plan (2009). 
 

Page 8 GAO-12-311 Homeland Defense 

security needs. In addition, failing to monitor the implementation of 

countermeasures, including those implemented by other agencies, may 

result in a misallocation of resources. Similarly, failing to conduct routine 

or periodic assessments of programs or operations could result in missed 

opportunities to increase their efficiency and effectiveness.

DOD, itself, has identified risk management, in this context, as a process 

of systematically identifying, assessing, and managing risks arising from 

operational factors and making decisions that balance cost with mission 

benefits. For example, DOD’s defense critical infrastructure program is a 

risk-management program that seeks to ensure the availability of certain 

identified assets critical to DOD missions. Activities include the 

identification, assessment, and security enhancement of assets essential 

for executing the National Military Strategy. The Defense Logistics 

Agency similarly established its enterprise risk-management approach to 

identify, assess, communicate, and manage risk in a cost-effective 

manner.

The Air Force has fully implemented one recommendation from our 2009 

report, but has not fully implemented the remaining four 

recommendations. In addition, confusion over the definition and scope of 

the mission perpetuates confusion about the capabilities and 

expenditures associated with the mission.

As of November 2011, the Air Force had fully implemented one of the five 

recommendations from our 2009 report12

 

12

and taken partial steps to 

implement the remaining four recommendations, as shown in figure 3 

below. We made the following recommendations: (1) to formally assign 

ASA duties to the appropriate units; (2) to implement ASA as a steadystate mission; (3) to establish a timetable for implementation of the 

preceding recommendation; (4) to develop and implement a plan to 

GAO-09-184. 

The Air Force Has 

Not Fully 

Implemented the 

Recommendations 

from Our 2009 Report, 

and Mission 

Confusion Persists

Air Force Has Taken 

Partial Steps to Implement 

Recommendations
a

The Air Force issues mission designed operational capabilities statements that identify the unit’s mission(s) and related requirements 

(e.g., type and number of personnel). The unit’s readiness is based on these requirements.

b

5th generation aircraft, such as the F-22 and F-35, have stealth characteristics, fused sensor data, and advanced radars.

Page 9 GAO-12-311 Homeland Defense

address any projected capability gaps; and (5) to develop and implement a 

formal method to replace deploying units.

Figure 3: Air Force’s Implementation of Our 2009 Recommendations as of November 2011

Source: GAO analysis of DOD information.

Recommendation Grade Actions taken

We recommend that the Secretary of Defense direct the 

military services with units that consistently conduct ASA 

operations to formally assign ASA duties to these units and 

then ensure that the readiness of these units is fully 

assessed, to include personnel, training, equipment, and 

ability to respond to an alert.

We recommend that the Secretary of Defense direct the 

Secretary of the Air Force to implement ASA as a steadystate mission according to NORAD, DOD, and Air Force 

guidance by: 

(1) incorporating ASA operations within the Air Force 

submissions for the 6-year Future Years Defense Program;

(2) updating the Air Force homeland defense policy, 

homeland operations doctrine, and concept of operations to 

incorporate and define the roles and responsibilities for ASA 

operations; and

(3) updating and implementing the ASA program action 

directive. 

We recommend that the Secretary of Defense direct the 

Secretary of the Air Force to establish a timetable to 

implement ASA as a steady-state mission [by taking the 

actions identified in the preceding recommendation]. 

We recommend that the Secretary of Defense direct the 

Secretary of the Air Force to develop and implement a plan 

to address any projected capability gaps in ASA units due to 

the expected end of the useful service lives of their F-15 and 

F-16 fighter aircraft. 

We recommend that the Secretary of Defense direct the 

Secretary of the Air Force to develop and implement a formal 

method to replace deploying units that still provides unit 

commanders flexibility to coordinate replacements. 

Interactivity instructions

Click on a recommendation to see the actions taken. Click on the “x” to clear. See appendix II for the non-interactive, printer-friendly version. 

Recommended action complete

Recommended action underway

0 50 100

0 50 100

0 50 100

0 50 100

0 50 100
 

Page 10 GAO-12-311 Homeland Defense 

We continue to believe that the prior recommendations have merit and 

that by not fully addressing the recommendations, DOD could leave the 

long-term sustainability of ASA operations at risk. Since the 2009 report 

was issued, some issues have arisen that might have been averted had 

the Air Force implemented the recommendations. For example, since the 

Air Force canceled the ASA program action directive that clearly identified 

the service’s Air Combat Command as responsible for programming for

ASA operations, the National Guard Bureau assumed programming and 

budgeting for its ASA sites to ensure that its units that were conducting 

ASA operations were funded. This in turn has led to the National Guard 

Bureau having to fund emergent expenses with funds that were

programmed for other purposes. For example, according to National 

Guard Bureau officials, when an ASA site is undergoing certain 

unplanned construction, the National Guard Bureau must fund the 

transfer of ASA personnel and equipment to a separate base.

Further complicating DOD’s management approach to ASA operations, 

we found considerable confusion about the capabilities and expenditures 

associated with ASA operations. In September 2011, NORAD stopped 

using the term “air sovereignty alert” (ASA) and created a new term, 

“aerospace control alert” (ACA) without clearly defining ACA or the 

capabilities that are now included within it. When we asked NORAD 

officials which capabilities are included under ACA, they told us that ACA 

includes—but is not limited to—DOD’s traditional scope of ASA 

operations (i.e., the dedicated fighter aircraft and personnel at 18 steadystate alert sites across the United States), as well as a morecomprehensive list of capabilities used in their aerospace control 

mission.

13

 

13NORAD has three overall missions: aerospace warning, aerospace control, and 

maritime warning. According to NORAD’s most-recent chartering agreement, aerospace 

control includes providing surveillance and exercising operational control of the airspace 

of Canada and the United States.

As figure 4 shows, according to NORAD, ACA capabilities 

include air defense and air sovereignty operations personnel and 

equipment at command and control centers at NORAD and elsewhere, 

ground-based defense systems, air patrols, airborne radar, and air-to-air 

refuelers—in addition to ASA fighters sitting on ground alert. However, Air 

Force and National Guard Bureau officials told us that while they were 

uncertain as to the definition of ACA, they believe the new term is 

Undefined New 

Terminology Regarding 

ASA Operations 

Perpetuates Confusion 

about the Mission
Page 11 GAO-12-311 Homeland Defense

interchangeable with ASA (i.e., the dedicated fighter aircraft and personnel 

at 18 steady-state alert sites across the United States).

Figure 4: Air Sovereignty Alert (ASA) versus Aerospace Control Alert (ACA)

Source: NORAD and DOD.

Western Air Defense Sector 

(WADS) and Eastern Air 

Defense Sector (EADS)

Continental NORAD Region 

Command’s (CONR) and 

Alaskan NORAD Region 

Command’s (ANR) Air 

Operations Centers

NORAD / NORTHCOM 

Command and Control Center

Airborne radar Ground-based defense system Joint Air Defense Operations 

Center (JADOC)

Refuelers Air patrols

Fighters on ground alert

Fighters on ground alert

Air Sovereignty Alert Aerospace Control Alert

Aerospace Control is the combination of air sovereignty mission (i.e., know and control what is flying in U.S. and 

Canadian airspace) and air defense mission (i.e. measures designed to defend against attacking enemy aircraft or 

missiles). According to NORAD officials, ACA is the capabilities (i.e., personnel and equipment) that are on some 

type of alert to support the broader mission.

Interactivity instructions: Click on a photo thumbnail to view larger.
 

Page 12 GAO-12-311 Homeland Defense 

While NORAD’s new terminology may better reflect its overall aerospacecontrol mission, without a clear definition of what capabilities and 

expenditures are related to ACA—compared to ASA—Congress will lack 

a complete picture of the funding and other resources needed for DOD to 

conduct ASA operations (i.e., the dedicated fighter aircraft and personnel 

at 18 steady-state alert sites across the United States). DOD is required 

by law to submit a consolidated budget justification document annually 

that includes all funding requirements for ASA operations. As will be 

discussed in more detail later in this report, even before the change in 

terminology, the Air Force and National Guard Bureau faced difficulties 

providing Congress with complete and accurate information regarding 

expenditures associated with ASA operations. During our review, officials 

from NORAD, the Office of the Assistant Secretary of Defense for 

Homeland Defense and Americas’ Security Affairs, the Joint Chiefs of 

Staff, U.S. Pacific Command, the Air Force, and the National Guard 

Bureau could not consistently define what capabilities and expenditures 

should be considered part of ASA. Until DOD defines ACA; identifies the 

specific capabilities that support it—including personnel and equipment; 

and distinguishes the place of ASA within that mission for the purposes of 

providing consistent information in response to their budget-justification 

requirement, Congress, NORAD, and other entities within DOD could be 

making important homeland security decisions based on inconsistent 

information.

DOD has taken a series of actions for ASA operations that are consistent 

with a risk-based management approach; however, several key actions 

have yet to be taken to fully implement a risk-based management 

approach that would enable the department to better balance risk and 

costs.

According to our prior work, an ongoing risk-based management 

approach is a best practice that enhances an organization’s decision 

making, including determining operational requirements, and helps to 

guide the use of limited resources. We have previously recommended a 

five-phase approach to a risk-management framework, including: 

strategic goals and objectives, risk assessment, evaluation of 

alternatives, management selection, and implementation and monitoring.

DOD Has Not Fully 

Implemented a RiskBased Management 

Approach That 

Balances Risk and 

Costs for ASA 

Operations

Risk Management Helps 

Balance Risk and Costs
 

Page 13 GAO-12-311 Homeland Defense 

Risk management is a widely endorsed strategy for decision makers to 

allocate finite resources and take action under conditions of uncertainty.

For example, the 2009 National Infrastructure Protection Plan states that 

its risk management framework integrates and coordinates strategies, 

capabilities, and governance to enable risk-informed decision making 

related to the nation’s critical infrastructure and key resources. This 

framework is applicable to threats such as natural disasters, manmade 

safety hazards, and terrorism, although different information and 

methodologies may be used to understand each.

DOD’s overarching homeland defense goal is to secure the United States 

from direct attack, but DOD is unable to measure the extent to which ASA 

operations help to achieve that goal. In its 2005 Strategy for Homeland 

Defense and Civil Support, DOD linked homeland defense with several 

objectives associated with ASA operations—including deterring, 

detecting, and if necessary, defeating airborne threats.14 While these 

actions are initial steps in the first phase of a risk-based management 

approach, NORAD has not identified performance measures that could 

identify the extent to which its ASA operations are progressing toward 

DOD’s homeland defense goal and objectives.15

 

14DOD, Strategy for Homeland Defense and Civil Support (Washington, D.C.: March 

2005).

While U.S. Pacific 

Command has established one type of performance measure identified in 

Joint Chiefs of Staff doctrine on joint planning (measures of 

effectiveness), the command has not established the other type of 

performance measure (measures of performance). U.S. Pacific 

Command officials also recognized that the command’s performance 

measures may not be comprehensive since its single ASA site does not 

deal with the same operational tempo as the other 17 ASA sites. 

According to Joint Chiefs of Staff doctrine on planning, continuous 

evaluation of operations against measures of performance and measures 

of effectiveness to assess the component’s progress toward 

accomplishing a task or achieving an objective is a key component of 

DOD’s planning process. Measures of effectiveness are criteria used to 

assess changes in system behavior, capability, or operational 

environment that are tied to measuring the attainment of an end state or 

15For purposes of this report, performance measures refer to both measures of 

effectiveness and measures of performance.

DOD Has Established and 

Linked Its Homeland

Defense Strategic Goal to 

Its ASA Objectives, but 

Has Not Established 

Performance Measures
 

Page 14 GAO-12-311 Homeland Defense 

an objective, or creation of an effect. Measures of performance are 

criteria to assess friendly actions that are tied to measuring task 

accomplishment. To provide examples of performance measures that 

could be applied to ASA operations, we created potential measures of 

effectiveness and measures of performance after analyzing NORAD and 

DOD planning documents and talking with NORAD and DOD officials, 

including ASA units. Table 1 lists some of the potential measures we 

developed. NORAD and DOD officials agreed that these examples could 

be a viable start for a more comprehensive list of performance measures 

to evaluate the effectiveness and performance of ASA operations.

Table 1: Nonexhaustive GAO-Developed List of Potential Measures of Effectiveness 

and Measures of Performance for ASA Operations

Measures of effectiveness Measures of performance

Number of track-of-interest infractions 

within the Special Flight Restricted Area 

within a certain period of time. 

Number (or percentage) of times ASA 

aircraft launch within Operation Noble 

Eagle requirements. 

Number and type of aircraft declared to be 

of interest within an established Temporary 

Flight Restricted Area within a certain 

period of time. 

Number (or percentage) of times ASA 

aircraft launch within NORAD established 

requirements. 

Number of track-of-interest aircraft that 

divert after certain tactics, techniques, and 

procedures are performed. 

Number (or percentage) of times ASA pilots 

are able to accurately identify and 

communicate track-of-interest aircraft tail 

number to command and control entities 

within 5 minutes. 

Total number of commercial and generalaviation aircraft declared to be a track of 

interest within a certain metropolitan area 

within a certain period of time. 

Number of Alert Force Evaluations failed 

each year. 

Source: GAO analysis of DOD data.

Measuring the status of tasks, effects, and objectives becomes the basis 

for reports to senior commanders and civilian leaders on the progress of 

the operation. NORAD could then advise the President and the Secretary 

of Defense accordingly and adjust operations as required. When we 

discussed the lack of performance measures with DOD officials, they 

acknowledged that such information would be helpful yet did not exist for 

ASA operations.

DOD officials pointed out that NORAD evaluates each ASA unit generally 

every 20 months. As we previously reported, NORAD’s unit assessments 

evaluate the quality of alert aircraft, to include the overall condition of the 

aircraft, and a unit’s ability to respond to different air sovereignty 
 

Page 15 GAO-12-311 Homeland Defense 

scenarios.16 While these assessments evaluate the equipment and 

operations of a single ASA unit against a planned scenario, the individual 

assessments do not provide NORAD or DOD situational awareness of 

actual ASA operations that occur every day (as opposed to planned 

evaluations); do not allow either NORAD or DOD to identify trends across 

ASA units; and do not provide NORAD or DOD criteria with which to 

evaluate the effectiveness and performance of ASA operations against 

DOD’s overall homeland-defense goal. Without ASA performance 

measures, NORAD and DOD are limited in their ability to assess ASA 

operations.17

Although we have previously recommended that NORAD should conduct 

routine risk assessments of ASA operations18

Although NORAD has not conducted routine risk assessments, it took 

steps toward implementing our 2009 recommendation by developing a 

computer model to evaluate current ASA basing locations. However, we 

found that the model had certain limitations that limit its ability to provide 

NORAD commanders, DOD decision makers, and Congress risk-based 

information. Specifically, it did not include a prioritized list of metropolitan 

areas and critical infrastructure locations that NORAD should protect. The 

—the second phase of a 

risk-based management approach—we found that it has not implemented 

this recommendation. Specifically, our 2009 ASA report stated that by 

undertaking routine risk assessments, NORAD could better evaluate the 

extent to which previous threats have been mitigated by DOD or other 

government agencies; better evaluate current and emerging threats to 

determine which ones require the most urgent attention; and determine 

operational requirements to address changing conditions. Moreover, 

routine risk assessments could help NORAD evaluate alternatives to 

current operations, especially in a resource-restricted environment. 

Further, such assessments could enhance NORAD’s ability to determine 

and apply the appropriate level and type of resources—including units, 

personnel, and aircraft—to ASA operations.

 

16GAO-09-184. 

17According to DOD’s joint doctrine on planning, an assessment with performance 

measures should answer two questions: (1) Is the Joint Force Commander doing things 

right? and (2) Is the Joint Force Commander doing the right things? 

18GAO-09-184. 

NORAD Has Not 

Conducted Routine Risk 

Assessments, and Its ASA 

Model Has Limitations
 

Page 16 GAO-12-311 Homeland Defense 

2009 National Infrastructure Protection Plan states that identifying and 

prioritizing key assets to protect, such as metropolitan areas and critical 

infrastructure, is a key component for risk management. According to 

NORAD officials, the model omitted this information because DOD had 

not issued additional guidance since the initial lists were distributed after 

the September 11, 2001, terrorist attacks. The model also did not 

incorporate data that are associated with risk: threat, vulnerability, and 

consequence because NORAD designed the model to assess the ability 

of its aircraft to respond to a single type of threat regardless of its location 

in the United States.

We also found some limitations with the assumptions and reliability of the 

input data used by NORAD’s computer model.19 NORAD officials agreed 

with our observations and agreed to take action to address them. Further, 

we found that while NORAD officials regularly run queries using the 

model, they do not generate reports documenting the results of their 

analyses. GAO’s Standards for Internal Control in the Federal 

Government recommends ongoing monitoring, in addition to separate 

evaluations, to assess the quality of performance over time and ensure 

that the findings of audits and other reviews are promptly resolved.20

 

19Specific information about the model’s inputs and areas for improvement are provided in 

the classified version of this report.

 A 

report that documents the model’s analyses may allow analysts to 

routinely identify trends over a period of time so that commanders could 

determine the extent to which their operations are effective or to which a 

threat might be evolving. We believe that NORAD’s computer model, if 

properly designed, could sufficiently address our 2009 recommendation 

to conduct routine risk assessments, as well as allow NORAD to assess 

the specifics of detailed ASA scenarios. These areas of improvement are 

consistent with DOD methodology used in other DOD risk-management 

efforts, such as force protection and defense critical infrastructure, as well 

as the U.S. Coast Guard’s and the Transportation Security 

Administration’s risk-based efforts. Similarly, The National Strategy for the 

Physical Protection of Critical Infrastructures and Key Assets states that 

responsible authorities using models and simulations can evaluate the 

risk associated with particular vulnerabilities more accurately and 

20GAO, Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1

(Washington, D.C.: November 1999).
 

Page 17 GAO-12-311 Homeland Defense 

subsequently make more-informed protection decisions.21 Without 

conducting routine risk assessments or maintaining a risk-based 

computer model that includes key components identified above, DOD will 

be unable to identify and assess the level of risk that it is taking; evaluate 

its ASA posture as the threat evolves or is mitigated by other U.S. 

government efforts; and make risk-based decisions, especially in a 

constrained fiscal environment. In addition, without documenting the 

results of risk assessments or the analyses that are completed using data 

from the computer model, DOD will be limited in its ability to identify 

trends in threats, vulnerabilities, and potential consequence over time.

We identified three alternatives to DOD’s current ASA operations that the 

department has considered since 2009, and DOD has analyzed costs and 

benefits of these alternatives to varying extents. During this review, we 

found that DOD has considered the following three alternatives to its 

current ASA posture: establish a new ASA site to support the President 

while in Chicago; acquire additional capabilities to address a perceived 

threat; and modify the current number and location of ASA sites. Of these 

three alternatives, we determined that DOD conducted a cost-benefit 

analysis for one of the alternatives, but had not conducted a cost-benefit 

analysis on the other two alternatives at the time of our review. According 

to GAO’s risk-based management framework, weighing costs and 

benefits is a critical component of risk management when choosing 

among alternative security measures because it links the benefits of 

alternatives to the costs associated with implementing and maintaining 

them. The actions that DOD should take in the “risk-assessment” phase 

should be the basis for identifying and considering alternatives. Capital 

investments of federal expenditures generally require a cost-benefit or 

cost-effectiveness approach.22

 

21White House, The National Strategy for the Physical Protection of Critical Infrastructures 

and Key Assets (February 2003).

We analyzed the three alternatives to ASA 

operations that DOD officials told us they had considered since 2009:

22Office of Management and Budget (OMB) circulars A-11 (sect. 7), A-94, and A-130 

generally apply to capital investments. For example, OMB Circular A-94 outlines 

procedures for cost-benefit analysis that are suggested for use in the internal planning of 

executive-branch agencies, but are mandatory in certain contexts, such as for analyses 

submitted to OMB in support of budget programs pursuant to OMB Circular A-11. 

DOD Has Considered 

Costs and Benefits of One 

of Three Alternatives to 

ASA Operations
 

Page 18 GAO-12-311 Homeland Defense 

• Establish a new ASA site to support the President while in Chicago. 

DOD evaluated the possibility of establishing a new ASA site in the 

Chicago metropolitan area to support the President’s visits to his 

hometown. In evaluating this option, DOD identified several sites that 

were considered and analyzed potential costs for each of these sites.

In addition, DOD analyzed the benefits, such as cost savings, and 

drawbacks associated with the various sites and concluded that the 

cost outweighed the benefit of an additional ASA site at this location.

• Acquire capabilities to address a perceived threat. NORAD wants 

DOD to acquire a number of capabilities for ASA operations to 

address a capability gap that NORAD believes exists based on its 

ASA computer model. While NORAD has identified some of the 

potential costs associated with one of the new capabilities, it did not 

include all of the costs associated with that capability; nor did it 

include costs associated with other capabilities that could be acquired.

However, since NORAD requested these additional capabilities 

through DOD’s Joint Capabilities Integration and Development 

System process, a cost-benefit analysis is not scheduled to occur until 

NORAD’s proposal has been assessed and approved as part of that 

process.23

• Modify the number and location of ASA sites. In 2010, NORAD 

analyzed its ASA basing strategy to identify whether it could change 

the number and location of ASA sites without affecting its ability to 

defend the homeland from aviation-related attack. NORAD’s analysis 

determined that the probability of success of an attack increased as 

the number of ASA sites decreased. The analysis also identified a 

“point of diminishing returns” below which the number of ASA sites 

could not be further reduced without assuming an unacceptable level 

of risk. However, the analysis did not identify potential cost savings 

that could result from eliminating a given number of sites. Should 

NORAD, DOD, or Congress consider modifying the number and 

location of ASA sites in the future, without an analysis that balances 

both risk and costs, decision makers will be unable to make fully 

informed decisions about whether the potential cost savings (or 

increase) warrants the corresponding increase (or decrease) in risk.

 

23The Joint Capabilities Integration and Development System analysis process is a 

requirements-generation system intended to prioritize and ensure that the most-essential 

needs of the warfighter are met. According to this process, when a material solution is 

recommended to mitigate a capability gap, an analysis of alternatives is generally 

conducted that evaluates the performance, operational effectiveness, operational 

suitability, and the estimated costs of an alternative to meet a mission capability. 
 

Page 19 GAO-12-311 Homeland Defense 

We could not evaluate the extent to which DOD implemented the last two 

phases—management selection, and implementation and monitoring—

because (1) DOD had not fully implemented the first three phases of 

GAO’s risk-based management framework, which are essential for the 

last two phases, and (2) DOD was still in the managerial-selection phase 

for the alternatives that we identified.

While we are encouraged that DOD has examined some alternatives to 

its current ASA operations, adopting a more-rigorous risk-based 

management approach—including balancing risk and costs—would help 

decision makers within DOD and elsewhere more-effectively allocate 

finite DOD resources.

Internal control weaknesses limit the Air Force’s ability to accurately 

identify expenditures with regard to Air Force management of ASA 

operations. Without accurate expenditure information, Congress, 

NORAD, and DOD will not have an accurate basis for decision making 

and realistic budget formulation.

We analyzed the fiscal year 2010 expenditure information the Air Force 

and National Guard Bureau submitted to Congress24 and found that it did 

not accurately identify all ASA expenditures.25

 

24In 2008, Congress mandated that for fiscal year 2010 and each year thereafter, the 

Secretary of Defense submit to the President for consideration by the President for 

inclusion with the budget materials submitted to Congress a consolidated budget 

justification that covers all programs and activities of the ASA mission of the Air Force, 

including military-personnel expenditures; flying hours; and any other associated ASA 

mission expenditures of the Air Force and the Air National Guard. 

According to the Air 

Force’s submission to Congress, more than $246 million was expended 

on ASA operations in fiscal year 2010. While the Air Force included all of 

the mandated funding requirements categories in its budget-justification 

25For the purpose of this review, we evaluated the ASA fiscal year 2010 expenditures 

included in the fiscal year 2012 budget-justification documents because at the time of our 

review it was the most-recent budget justification reflecting resources spent on ASA 

operations. 

Weak Internal 

Controls Limit the 

Ability of the Air 

Force and National 

Guard Bureau to 

Accurately Identify 

ASA Expenditures

Air Force and National 

Guard Bureau Officials 

Have Not Accurately 

Identified ASA 

Expenditures
 

Page 20 GAO-12-311 Homeland Defense 

document, we found three areas in which the expenditures reported by 

the Air Force were inaccurate.

First, we found that the Air Force overstated its ASA flying-hour 

expenditures by at least $22 million. According to Air Force officials, they 

mistakenly used a placeholder for the ASA flying-hour expenditure of 

approximately $57.6 million, instead of using the actual flying-hour 

expenditure data. During the course of our review, the Air Force provided 

us with actual flying-hour data, which showed that the service expended 

$35.6 million—38 percent less than reported.

Second, the ASA budget-justification document included expenditures 

that were not associated with ASA operations. For example, it included 

flying-hour and personnel expenditures related to non-ASA homelanddefense missions, such as air patrols protecting the President and 

national special-security events. Hence, it is likely that the Air Force 

further overstated its ASA flying-hour expenditures beyond the $22 million 

discussed above.

Third, the budget justification document the Air Force provided to 

Congress did not include some expenditures that are related to ASA. 

While the budget justification included one military construction project for 

a munitions storage unit at the Toledo, Ohio, ASA site, it did not include 

any other expenditures associated with constructing, maintaining, and 

upgrading ASA facilities in fiscal year 2010. For example, when asked if 

the officials included the construction expenditures related to establishing 

an ASA site at McEntire Joint National Guard Base, South Carolina, in 

fiscal year 2010, such as constructing hangers for the alert fighter jets 

and upgrading crew quarters, the officials stated that these expenditures 

were not included in the ASA budget justification. However, we asked 

budget officials at each site about ASA expenditures for fiscal year 2010 

in a survey. According to their responses and additional information from 

command officials, ASA sites as a whole spent at least $3.8 million in 

facilities construction, maintenance, and equipment that was not included 

in the budget justification.

When asked to identify the expenditures included in the budget 

justification and the process by which the ASA expenditures were 

compiled, the Air Force and National Guard Bureau could not answer 

many of our questions regarding which detailed expenditures were 

included under the expenditure totals listed in the budget justification. 

Since the Air Force and National Guard Bureau were unable to accurately 

identify the expenditures to conduct ASA operations, we conducted a 
 

Page 21 GAO-12-311 Homeland Defense 

survey of all ASA sites in an effort to identify approximate expenditures 

for fiscal year 2010. According to the information obtained from this 

survey, the expenditures for each unit to conduct ASA operations during 

fiscal year 2010 ranged from $2 million to $8 million, with the average site 

expenditure being approximately $5 million.

In addition, Air Force and National Guard Bureau officials told us they did 

not have documentation or guidance outlining how the ASA expenditures 

were defined, identified, and compiled for the budget justification. DOD is 

required by various statutes to improve its financial-management 

processes, controls, and systems to ensure that complete, reliable, 

consistent, and timely information is prepared and responsive to the 

financial information needs of agency management and oversight 

bodies.26 Additionally, GAO’s Standards for Internal Control in the Federal 

Government cite the importance of developing detailed internal-controls 

policies, procedures, and practices to ensure basic accountability, 

maintain funds control,27 and prevent fraud and abuse.28 Such efforts 

should include clearly defined roles and responsibilities and accurate and 

complete guidance on defining and tracking ASA expenditures. Without 

adequate internal controls in place, the Air Force’s ability to accurately 

identify ASA expenditures will be limited.

The Air Force and National Guard Bureau lack clearly defined 

programming and budgeting roles and responsibilities for ASA operations, 

which in turn limited the Air Force’s ability to identify ASA expenditures. 

We found that the Air Force and National Guard Bureau have not clearly 

defined programming and budgeting roles and responsibilities since the 

Air Force canceled its 2003 ASA program action directive in 2009. The 

directive required the service’s Air Combat Command to program for ASA 

operations. As a result, Air Combat Command programmed and 

 

26These statutes include the Chief Financial Officers Act of 1990, the Government 

Management Reform Act of 1994, the Federal Financial Management Improvement Act of 

1996, and various annual authorization and appropriation act provisions. 

27Funds controls are controls over the use of public funds, including (1) using public funds 

efficiently and effectively and for the purposes and within the time frames and amounts 

prescribed by law; (2) making payments to the right parties in the correct amount within 

allowable time frames and recouping any improper payments; and (3) accurately 

recording and reporting on transactions and use of public funds.

28GAO/AIMD-00-21.3.1. 

Unclear ASA Roles and 

Responsibilities 

Contributed to 

Weaknesses in the 

Identification of ASA 

Expenditures
 

Page 22 GAO-12-311 Homeland Defense 

budgeted for ASA operations between 2003 and 2009. However, since 

the directive was canceled and no additional guidance was issued, Air 

Combat Command stopped programming, planning, and budgeting for 

ASA operations at Air National Guard sites. National Guard Bureau 

officials told us that they consequently assumed these responsibilities.

However, Bureau officials have questions about the bureau’s authority to 

fund and conduct ASA operations in the manner in which they are 

currently conducted. We have requested the Air Force’s official position 

regarding whether the Air National Guard should be conducting this 

mission under a Title 10 or Title 32 status. As of January 2012, the Air 

Force had not responded to our inquiry. We also found other internalcontrol weaknesses, such as a lack of oversight of funds expended for 

ASA operations. According to a National Guard Bureau internal-review 

report, a lack of oversight and other internal-control weaknesses, such as 

poor guidance, led to improper compensation at several ASA sites that 

are operated by Air National Guard units.29 Without clearly defined roles 

and responsibilities, questions about who is responsible for programming, 

planning, and budgeting will remain; and the ability to accurately identify 

ASA expenditures will be limited. In addition, the Air Force and National 

Guard Bureau may remain vulnerable to improper use of limited 

resources.

Air Force and National Guard Bureau financial-management officials told 

us that they did not accurately capture the ASA expenditures because 

they lacked guidance that defines ASA operations and the process to 

track the associated expenditures. First, as mentioned previously in the 

report, there is no agreed-upon definition of ASA operations among 

NORAD and DOD components that contribute to ASA operations.

Consequently, the financial-management guidance does not clearly 

define the expenditures that should be associated with ASA operations.

Instead, the guidance that is provided to the Air Force by the Office of the 

Under Secretary of Defense (Comptroller) contains broad categories that 

do not clearly state what specific expenditures should or should not be 

included in the budget exhibit.30

 

29According to the National Guard Bureau, federal authorities are working to recover the 

improper compensation. 

For example, the guidance associated 

30The Office of the Under Secretary of Defense (Comptroller) could not find the original 

ASA budget-justification guidance and template so it recreated the template for the 

purposes of our audit.

The Air Force and National 

Guard Bureau’s Lack of 

Guidance on Defining and 

Tracking ASA 

Expenditures Limited Its 

Ability to Accurately 

Identify ASA Expenditures
 

Page 23 GAO-12-311 Homeland Defense 

with identifying ASA military-personnel expenditures states that Air Force 

officials should provide military-personnel costs associated with the 

military full-time equivalents supporting the ASA mission. Since the 

guidance does not clearly identify the type or group of military personnel 

who are associated with ASA, financial managers could respond to this 

reporting requirement with a range of different personnel, such as 

personnel that work at NORAD’s air-defense sectors and DOD’s refueling 

units since they assist in ASA operations, but are not necessarily included 

within the 18 alert sites.

Second, Air Force and National Guard Bureau financial-management 

officials told us that they had not issued any guidance explaining the 

process for tracking ASA expenditures at the local level. According to Air 

Force guidance, financial managers and budget officers at all levels must 

make sure that operating personnel are made aware of the importance of 

properly identifying and capturing all costs associated with emergency 

and special programs, like ASA, because only accurate cost data can 

provide the documentation to meet reporting requirements. Guidance to 

the local level on the process of tracking ASA expenditures would be 

important because Air Force and National Guard Bureau officials told us 

that they develop the ASA budget justification using the information that 

the financial managers at the ASA sites enter into the service’s 

accounting systems. While the National Guard Bureau’s annual financialmanagement guidance does not include any information on tracking ASA 

expenditures, we found that it does contain specific financialmanagement guidance for a similar mission. Specifically, the Air National 

Guard’s counterdrug mission is similar to the ASA mission in that they 

both involve Guard personnel supporting a federal mission and have 

specific accounting codes dedicated to tracking their expenditures. When 

we asked about this disparity, National Guard Bureau financialmanagement officials acknowledged that there was not a good reason for 

having not included more guidance about tracking ASA expenditures and 

told us that the bureau would address this in the next version of the 

guidance. However, a new version of the National Guard Bureau’s 

financial-management guidance was subsequently issued and it did not 

include guidance on tracking ASA expenditures. National Guard Bureau 

financial-management officials stated that changes to the guidance will be 

driven by a number of factors, including appropriations law, questions 

from their comptrollers in the field, and our findings. Without clear 

guidance on defining and tracking ASA expenditures, unit officials will be 

unable to provide Air Force and National Guard Bureau officials accurate 

and consistent ASA expenditure information.
 

Page 24 GAO-12-311 Homeland Defense 

Since the September 11, 2001, terrorist attacks, the U.S. government has 

taken extensive efforts to protect the airspace over the United States, 

aviation threat profiles have changed, and fiscal resources have become 

more constrained. Nevertheless, NORAD and DOD have not fully 

evaluated how to most-effectively manage ASA operations in this current 

environment. Until the Air Force fully implements the recommendations 

identified in our January 2009 report, including establishing ASA as a 

steady-state mission by updating Air Force homeland-defense policy and 

doctrine, concerns about the long-term sustainability of ASA operations 

will remain. Additionally, until DOD defines ACA; identifies the specific 

capabilities that support it, including personnel and equipment; and 

distinguishes the place of ASA within that mission for the purposes of 

meeting Congress’s budget-justification requirement, Congress, NORAD, 

and other entities within DOD could be making important homelandsecurity decisions based on different definitions and a resulting variance 

in understanding of what is required to accomplish that mission.

Further, in this environment, it is critical that DOD makes basing, 

acquisition, and other decisions about ASA operations on the basis of 

risk-management principles that balance risk and costs. These principles 

include establishing performance measures; identifying and prioritizing 

locations that should be protected; utilizing accurate, comprehensive, and 

documented risk assessments; and conducting cost-benefit analyses for 

alternatives. With a more-rigorous, risk-based management approach, 

DOD and congressional decision makers would be in a better position to 

effectively allocate resources. Finally, unless the Air Force addresses 

internal-control weaknesses that limit its ability to identify ASA 

expenditures, Congress, NORAD, and DOD will not have an accurate 

basis for investment decision making and realistic budget formulation.

In order to ensure that the Air Force is taking action that addresses the 

long-term sustainability of ASA operations, Congress may wish to 

consider requiring the Secretary of the Air Force to fully implement the 

remaining actions identified in our 2009 report within a time period that 

Congress believes most prudent.

To address these issues, we recommend that the Secretary of Defense 

take the following seven actions. 

In order that all DOD components may better understand the mission they 

have been asked to perform, we recommend the Secretary of Defense 

Conclusions

Matter for 

Congressional 

Consideration 

Recommendations for 

Executive Action
 

Page 25 GAO-12-311 Homeland Defense 

direct the Assistant Secretary of Defense for Homeland Defense and 

Americas’ Security Affairs, in coordination with the Joint Chiefs of Staff, to 

clearly define ASA operations and its relationship to NORAD’s ACA 

operations.

In order to implement a more-complete risk-based management approach 

that balances risk and costs for ASA operations, we recommend the 

Secretary of Defense take the following five actions:

• Direct the Joint Chiefs of Staff, in coordination with the U.S. element 

of NORAD and U.S. Pacific Command, to develop performance 

measures for ASA operations and then use these measures to 

evaluate the mission and make adjustments, as warranted, on the 

basis of the performance-measure results.

• Direct the Office of the Assistant Secretary of Defense for Homeland 

Defense and Americas’ Security Affairs, in coordination with the Joint 

Chiefs of Staff, to issue updated guidance, which includes a prioritized 

list of metropolitan areas and critical infrastructure that NORAD is 

supposed to protect.

• Direct the U.S. element of NORAD to update its ASA computer model 

to address identified areas for improvement to include incorporating 

the elements of risk—threat, vulnerability, and consequence.

• Direct the U.S. element of NORAD to document the results of its risk 

assessments so that NORAD and DOD can identify trends over time.

• Identify the appropriate DOD entities that should conduct a 

comprehensive cost-benefit analysis of NORAD’s ASA basing 

strategy and then have those entities conduct such an analysis.

In order to accurately identify ASA expenditures and address other 

internal control weaknesses, we recommend that the Secretary of 

Defense direct the Secretary of the Air Force and the Director of the 

National Guard Bureau to issue guidance that

• defines ASA programmatic and budgeting roles and responsibilities;

• defines all expenditures that should be identified as ASA expenditures 

in financial-management systems; and

• identifies the proper procedures to track ASA expenditures in their 

financial-management systems.

In written comments on a draft of this report, DOD stated that it concurred 

with one of our recommendations and partially concurred with the other 

six recommendations. However, DOD did not consistently identify 

specific actions that it would take to implement the recommendations and 

Agency Comments 

and Our Evaluation 
 

Page 26 GAO-12-311 Homeland Defense 

it did not identify time frames for implementing the recommendations. 

DOD’s comments are reprinted in appendix III. The Department of 

Homeland Security and the Federal Bureau of Investigation also reviewed 

a draft of this report but did not provide any comments. 

DOD concurred with our recommendation to clearly define ASA 

operations and its relationship to NORAD’s ACA operations and stated 

that all organizations involved in the air sovereignty mission would use 

standard terminology and that the new terminology would be reflected in 

subsequent plans, execute orders, and DOD publications. 

DOD partially concurred with our recommendation to develop 

performance measures for ASA operations, but did not identify any 

specific actions that it would take to implement the recommendation. In 

its comments, DOD stated that it recognizes the merit of, and routinely 

employs performance measures that are incorporated into military service 

training requirements. However, NORAD officials told us during our 

review that the command had not developed any performance measures 

for ASA operations. Consequently, we believe that the development and 

use of ASA-related performance measures would improve DOD’s ability 

to assess the overall effectiveness of its ASA operations and enhance the 

performance measures to which DOD referred to in its comments.

DOD partially agreed with our recommendation that it issue updated 

guidance, which includes a prioritized list of metropolitan areas and 

critical infrastructure that NORAD is supposed to protect. DOD stated that 

the current guidance on prioritized metropolitan areas and critical 

infrastructure was produced by the National Security Staff and the criteria 

used to produce it are still valid. However, this guidance has not been 

updated since shortly after September 11, 2001, and DOD has produced 

other similar lists within the department that it uses in some contexts. For 

these reasons, we continue to believe that DOD should consider updating 

existing guidance or developing new guidance concerning a prioritized list 

of metropolitan areas and critical infrastructure locations. We anticipate

that NORAD and DOD could seek additional guidance from the National 

Security Staff if they believe the Staff’s guidance needs to be reviewed or 

updated. Such actions would better ensure that key national security 

organizations have consistent and up to date expectations. 

DOD partially concurred with our recommendation to direct the U.S. 

element of NORAD to update its ASA computer model to address 

identified areas for improvement to include incorporating the elements of 

risk—threat, vulnerability, and consequence. In its comments, DOD 

stated that NORAD determines mission risk by evaluating its computer 
 

Page 27 GAO-12-311 Homeland Defense 

model results (based on vulnerability, threat capability, and targets of 

consequence) and comparing the results to intelligence assessments of 

threat intent. However, our analysis of the model found that it did not 

incorporate threat, vulnerability, and consequence (or criticality) data

because NORAD designed the model to assess the ability of its aircraft to 

respond to a single type of threat regardless of its location in the United 

States. DOD’s comments also stated that NORAD would explore 

improvements to the ASA model in support of NORAD/NORTHCOM 

decision making as targets and threats evolve and capabilities improve. 

We continue to believe it appropriate that NORAD’s future improvements 

to the model should include the three components that DOD has longconsidered essential in risk assessments: threat, vulnerability, and 

consequence (or criticality). Such actions would be consistent with DOD’s 

other efforts to protect its people, property, and systems from attacks. We 

also expect that future improvements to the model will include the specific 

areas we identified in the classified version of this report. 

DOD partially concurred with the recommendation to direct the U.S. 

element of NORAD to document the results of its risk assessments so 

that NORAD and DOD can identify trends over time. In its response, 

DOD identified several NORAD studies or assessments—including a 

2009 Collateral Damage Study and the 2010 ASA Basing Study—that are 

subject to trend analysis. We believe that NORAD’s efforts to study or 

analyze different aspects of its missions are positive actions. However, 

while NORAD issued a one-time report to Congress in March 2011 that 

identified the results of its 2010 ASA Basing Study, a one-time study is 

not a trend analysis and the command has not documented the results of 

its analyses of the ASA model since that time. We continue to believe 

that DOD should document over time the results of analyses stemming 

from the ASA basing model itself. This would provide a valuable 

reference of the model’s inputs and outputs over time and improve the 

ability of current and future leaders to identify and address evolving 

threats within U.S. airspace. 

DOD partially concurred with our recommendation that the Secretary of 

Defense should identify the appropriate DOD entities that should conduct 

a comprehensive cost-benefit analysis of NORAD’s ASA basing strategy 

and then have those entities conduct such an analysis. The response 

stated that multiple cost-benefit analyses have been conducted on 

NORAD’s ASA basing strategy by NORAD, OSD Cost Assessment and 

Program Evaluation, the Joint Staff, the Department of Air Force, and the 

Office of the Assistant Secretary of Defense for Homeland Defense and 

Americas’ Security Affairs. However, when we asked NORAD and DOD 

officials, including some of those listed above, for the cost-benefit 
 

Page 28 GAO-12-311 Homeland Defense 

analysis of its basing strategy (i.e. potential cost savings that could result 

from eliminating a given number of sites), the officials told us that such an 

assessment had not been completed. Further, when we asked which 

organization would be responsible for completing such studies, the 

officials provided us conflicting views of whose responsibility it would be. 

Nonetheless, DOD’s comments also recognize that such cost-benefit 

analyses are critical for NORAD’s force posturing. As such, we believe 

that NORAD and DOD’s ability to demonstrate its use of these costbenefit analyses in future decision making efforts will help to ensure 

effective mission capability consistent with the current fiscal environment.

DOD partially concurred with our recommendation to direct the Secretary 

of the Air Force and the Director of the National Guard Bureau to issue 

guidance that addresses internal control weaknesses that kept the Air 

Force from providing accurate ASA expenditure information to Congress. 

Such weaknesses included unclear roles and responsibilities; lack of 

guidance that identified expenditures that should be identified as ASA 

expenditures in financial-management systems; and proper procedures to 

track ASA expenditures in their financial-management systems. In its 

response, DOD stated that it believes that programmatic and budgetary 

roles are clearly understood and well defined in DOD’s planning, 

programming, budgeting, and execution process. However, the Air 

Force’s inability to provide Congress an accurate reporting of resources 

used for ASA operations indicates that internal control weaknesses exist 

that have not been mitigated by DOD’s standard process. For example, 

officials from the Air Force’s homeland defense office, the National Guard 

Bureau, and the Air Force’s Combat Command could not provide us any 

documentation of which organization is responsible for programming and 

budgeting for ASA operations since the Air Force canceled its ASA 

program action directive in 2009.

As a result of these weakness, the Air Force provided Congress 

inaccurate information about its fiscal year 2010 ASA expenditures—

ranging from overstating its flying hour costs by at least 38 percent to 

understating its facilities construction, maintenance, and equipment 

expenditures. DOD’s response did not indicate that it would issue 

guidance to address these internal control weaknesses. Given the current 

fiscal environment, DOD should identify and take specific actions 

immediately to address our findings so that Congress, NORAD, and DOD 

will have an accurate basis for decision making and realistic budget 

formulation. 
 

Page 29 GAO-12-311 Homeland Defense 

We are sending copies of this report to appropriate congressional 

committees, the Secretary of Defense, the Commander of NORAD, the 

Secretary of the Air Force, the Commissioner of Customs and Border 

Protection, the Director of the Federal Bureau of Investigation, the

Administrator of the Transportation Security Administration, the 

Commandant of the U.S. Coast Guard, and the Director of OMB. In 

addition, the report is available at no charge on the GAO website at 

http://www.gao.gov.

If you or your staff have any questions about this report, please contact 

me at (202) 512-4523 or leporeb@gao.gov. Contact points for our Offices 

of Congressional Relations and Public Affairs may be found on the last 

page of this report. GAO staff who made key contributions to this report 

are listed in appendix IV. 

Brian J. Lepore

Director, Defense Capabilities and Management
 

Page 30 GAO-12-311 Homeland Defense 

List of Requesters

The Honorable Adam Smith

Ranking Member

Committee on Armed Services

House of Representatives

The Honorable J. Randy Forbes

Chairman

The Honorable Madeleine Z. Bordallo

Ranking Member

Subcommittee on Readiness

Committee on Armed Forces

House of Representatives

The Honorable Roscoe G. Bartlett

Chairman

The Honorable Silvestre Reyes

Ranking Member

Subcommittee on Tactical Air and Land Forces

Committee on Armed Forces

House of Representatives
Appendix I: Scope and Methodology

Page 31 GAO-12-311 Homeland Defense 

To determine the extent to which the Air Force has implemented the 

recommendations from our 2009 report, we interviewed officials from the 

Assistant Secretary of Defense for Homeland Defense and Americas’ 

Security Affairs, the Air Force’s homeland-defense office, the Air Force’s 

Air Combat Command, the National Guard Bureau, and three air 

sovereignty alert (ASA) units. We gathered information about steps taken 

related to the recommendations and obtained supporting documentation, 

when possible. In those cases where the Air Force had taken action, we 

asked the officials about the effect that the action had on their operations.

In those cases where the Air Force had not taken action, we asked the 

officials to identify the rationale for lack of action and the plan that they 

had in place—including time frames—to implement the 

recommendations.

To determine the extent to which the Department of Defense (DOD) has 

adopted a risk-based management approach to balance risk and costs for 

ASA operations, we reviewed and analyzed applicable documentation 

and interviewed relevant officials. Specifically, we reviewed prior GAO 

reports that recommended organizations use a risk-based management 

approach to manage risk and costs.1

 

1

GAO, Joint Strike Fighter: Strong Risk Management Principles Essential as Program 

Enters Most Challenging Phase, 

We reviewed documents and 

interviewed officials from the Office of the Assistant Secretary of Defense 

for Homeland Defense and Americas’ Security Affairs, the Air Force, and 

the North American Aerospace Defense Command (NORAD) to 

determine the extent to which NORAD has followed a risk-based 

management approach for ASA operations. We reviewed NORAD’s ASA 

assessments and compared them to the elements required for a risk 

assessment. We reviewed acquisition documentation to determine 

whether the command considered the cost and benefits when considering 

alternatives for ASA operations. During our discussions with officials, we 

asked how and whether they measure the performance of ASA 

operations and whether they consider cost for ASA operations. To help us 

GAO-09-711T (Washington, D.C.: May 20, 2009); 

Intelligence, Surveillance, and Reconnaissance: DOD Needs a Strategic, Risk-Based 

Approach to Enhance Its Maritime Domain Awareness, GAO-11-621 (Washington, D.C.: 

June 20, 2011); Defense Acquisitions: Managing Risk to Achieve Better Outcomes, 

GAO-10-374T (Washington, D.C.: May 20, 2009); Aviation Security: DHS and TSA Have 

Researched, Developed and Begun Deploying Passenger Checkpoint Screening 

Technologies, but Continue to Face Challenges, GAO-10-128 (Washington, D.C.: Oct. 7, 

2009); and Risk Management: Further Refinements Needed to Assess Risks and 

Prioritize Protective Measures at Ports and Other Critical Infrastructure, GAO-06-91

(Washington, D.C. Dec. 15, 2005). 

Appendix I: Scope and Methodology
Appendix I: Scope and Methodology

Page 32 GAO-12-311 Homeland Defense 

better understand whether DOD incorporates risk into ASA operations, 

we reviewed documentation on how other agencies incorporate risk into 

their missions. We then reviewed documentation on NORAD’s Operation 

Noble Eagle Fighter Basing Location Model and analyzed the model’s 

inputs and assumptions.2

In order to determine the extent to which the Air Force was able to 

accurately identify ASA expenditures, we analyzed the fiscal year 2010 

expenditure information the Air Force included in its fiscal year 2012 ASA 

budget-justification document. We documented the steps taken to identify 

and compile the ASA expenditures through interviews, e-mails, and 

related documentation. We then compared the information and practices 

used by the Air Force to prepare the ASA budget justification against 

cost-evaluation best practices consistently applied by organizations 

throughout the federal government. Specifically, we evaluated the budget 

justification to determine whether it included all expenditures associated 

with ASA operations;

We interviewed officials from NORAD on how it 

operationalizes the model in support of ASA operations.

3

 

2

The Operation Noble Eagle Fighter Basing Location Model is a Monte Carlo simulation 

that creates events and evaluates how effective the response will be, given a predefined 

fighter posture and location. 

whether it contained few, if any, mistakes; and 

whether the underlying documentation and expenditure calculations were 

well documented so that they could be easily repeated or updated. To 

determine the expenditures of each ASA site we developed a survey, 

which we sent out to the Air National Guard and active-duty ASA units to 

gather fiscal year 2009 and 2010 expenditures. We developed the 

expenditure survey by gathering ASA operations-requirement 

documentation that outlined the personnel, operational, and facilities 

requirements for conducting ASA operations. In order to ensure greater 

reliability in the expenditure information we gathered from the ASA sites, 

we instructed the sites to separate non-ASA Operation Noble Eagle 

expenditures from ASA expenditures; include all of the ASA construction 

and maintenance expenditures; and to remove duplications in their 

responses. We also consulted with two Air National Guard and two active 

duty ASA sites to verify the accuracy and completeness of the survey’s 

instructions, parameters, and expenditure categories. We found the data 

gathered in this survey to be sufficiently reliable for our purposes.

3

We defined ASA operations as the fully fueled, fully armed aircraft and trained personnel 

on alert 24 hours a day, 365 days a year, and the infrastructure, equipment, and supplies 

required to maintain them at the 18 ASA sites across the United States. 
Appendix II: Air Force’s Implementation of Our 

2009 Recommendations as of November 2011 

Page 33 GAO-12-311 Homeland Defense 

Recommendation Actions taken

We recommend that the Secretary of 

Defense direct the military services with 

units that consistently conduct Air 

Sovereignty Alert (ASA) operations to 

formally assign ASA duties to these units 

and then ensure that the readiness of 

these units is fully assessed, to include 

personnel, training, equipment, and ability 

to respond to an alert.

Fully Implemented: The National Guard Bureau, in coordination with Air Combat 

Command, initially implemented this recommendation by placing ASA operations on the 

units’ secondary mission statements.a

However, North American Aerospace Defense Command (NORAD) does not believe that 

the National Guard Bureau should remove the reference to ASA operations from the 

units’ secondary mission statements. NORAD officials told us that they do not support the 

bureau’s position because NORAD experienced some challenges when it placed units 

that do not normally sit alert on alert for the 10th anniversary of the September 11, 2001, 

terrorist attacks. While our assessment gives the Air Force credit for having taken action, 

we are also concerned that removing the reference to ASA operations from the units’ 

secondary mission statements could have adverse effects, such as the units not reporting 

the extent to which they are able to conduct both their ASA operations and overseas 

deployments in their appropriate readiness systems. It is too early to determine the 

consequences of the bureau’s decision, although a bureau official acknowledged that the 

problems we identified in our 2009 report could reoccur if this new approach with NORAD 

was not fully implemented.

However, the National Guard Bureau told us that it 

does not believe that the Secretary of Defense has formally assigned ASA operations to 

the Air National Guard units. As of November 2011, bureau officials told us that they were 

taking action to remove the reference to ASA operations from these statements because 

they now believe that they could better address the findings identified in the 2009 report 

through the Air Force’s standard deployment process.

We recommend that the Secretary of 

Defense direct the Secretary of the Air 

Force to implement ASA as a steady-state 

mission according to NORAD, Department 

of Defense (DOD), and Air Force guidance 

by

(1) incorporating ASA operations within the 

Air Force submissions for the 6-year 

Future Years Defense Program;

(2) updating the Air Force homeland 

defense policy, homeland operations 

doctrine, and concept of operations to 

incorporate and define the roles and 

responsibilities for ASA operations; and

(3) updating and implementing the ASA 

program action directive. 

Partially Implemented: The Air Force has taken one of three steps to implement ASA as 

a steady-state mission. Specifically:

(1) The National Guard Bureau has incorporated ASA operations in its fiscal year 2013 

Future Years Defense Program.

(2) The Air Force had not updated any of the identified documents. Consequently, overall 

roles and responsibilities remain undefined.

(3) Rather than updating and implementing the ASA program action directive, the Air 

Force cancelled the program action directive. This leaves the Air Force and the National 

Guard Bureau without clear roles and responsibilities for operational planning, 

programming, and budgeting for ASA operations, which GAO has identified as an internal 

control weakness.

We recommend that the Secretary of 

Defense direct the Secretary of the Air 

Force to establish a timetable to implement 

ASA as a steady-state mission [by taking 

the actions identified in the preceding 

recommendation]. 

Not Implemented: The Air Force has not established a timetable to implement ASA as a 

steady-state mission.

Appendix II: Air Force’s Implementation of 

Our 2009 Recommendations as of November 

2011 
Appendix II: Air Force’s Implementation of Our 

2009 Recommendations as of November 2011 

Page 34 GAO-12-311 Homeland Defense 

Recommendation Actions taken

We recommend that the Secretary of 

Defense direct the Secretary of the Air 

Force to develop and implement a plan to 

address any projected capability gaps in 

ASA units due to the expected end of the 

useful service lives of their F-15 and F-16 

fighter aircraft. 

Minimally Implemented: The Air Force has taken some action that could mitigate 

potential challenges, if proactively managed, but has not developed a single plan to 

manage the recapitalization of the aircraft that conduct ASA operations. Specifically, the 

service has initiated efforts to test the effective service life of its remaining F-15 and F-16 

fleets. In addition, the service has requested funding to update both platforms with radar 

and communication equipment that will support ASA operations. The Air Force has also 

identified the extent to which it could cover its alert sites if either the F-15 or F-16 fleet 

were unavailable.

However, the potential grounding of ASA aircraft—such as the grounding of F-15s during 

2007 and F-22s during 2011, the ongoing delays in the F-35 program, and the 

unspecified fielding locations for the F-35 aircraft could pose challenges in the future. 

Further, the Air Force could have challenges if the retirement of the current fleet of F-15s 

and F-16s are not managed in a manner that considers the effect on ASA operations. For 

example, if the Air Force were to decrease the number of older models of F-16s in the 

inventory, this could affect ASA operations since these aircraft are flown by units that 

conduct these operations. Also, if the Air Force retires the fleet of F-15s and F-16s with 

the longest remaining effective service lives before those with shorter service lives, the 

service could experience a shortage in fighter aircraft available for ASA operations before 

the 5th-generation aircraft are available.

We recommend that the Secretary of 

Defense direct the Secretary of the Air 

Force to develop and implement a formal 

method to replace deploying units that still 

provides unit commanders flexibility to 

coordinate replacements. 

b

Minimally Implemented: In response to this recommendation, DOD noted that it has a 

global force-management system that it uses to formally manage its forces around the 

world. However, the approach that DOD uses to backfill personnel and units for ASA 

operations is still managed on an ad hoc basis.

NORAD and the Air Force have developed some planning documents, such as a plan for 

backfilling ASA units on a large scale. For example, one of the planning documents 

identifies potential units that could conduct ASA operations at bases that are equipped 

with F-15s if those aircraft were grounded for a period of time as they were in 2008. In 

addition, Air Combat Command and the National Guard Bureau developed a 

memorandum of understanding that identifies a process by which Air National Guard 

units will be scheduled to support non-ASA Operation Noble Eagle missions, such as 

national special-security events, throughout the year, if called upon. The agreement 

identifies a process by which Air National Guard units will be identified and scheduled to 

support these missions, if called upon, and the time frame during which each unit will be 

ready to support these missions. It also identifies Air Combat Command as the command 

that will be responsible for providing funds to those units when called upon.

However, the Air Force has not developed similar Operation Noble Eagle planning 

documents to replace ASA units that deploy to support overseas operations or units that 

are taken off alert for a short period of time when they fail an alert force evaluation. 

Consequently, the National Guard Bureau, in coordination with Air Combat Command, 

must try to find personnel and equipment that are available to support these units. In 

addition, since it is unclear who is responsible for funding these backfill efforts, the 

National Guard Bureau must use funds that were programmed for other purposes to 

cover these costs.

Source: GAO analysis of DOD information.

a

The Air Force issues mission designed operational capabilities statements that identify the unit’s 

mission(s) and related requirements (e.g., type and number of personnel). The unit’s readiness is 

based on these requirements. 

b

Fifth-generation aircraft, such as the F-22 and F-35, have stealth characteristics, fused sensor data, 

and advanced radars. 
Appendix III: Comments from the Department 

of Defense

Page 35 GAO-12-311 Homeland Defense 

Appendix III: Comments from the 

Department of Defense

Note: GAO received 

Department of Defense 

comments on January 27, 

2012.
Appendix III: Comments from the Department 

of Defense

Page 36 GAO-12-311 Homeland Defense 
Appendix III: Comments from the Department 

of Defense

Page 37 GAO-12-311 Homeland Defense 
Appendix III: Comments from the Department 

of Defense

Page 38 GAO-12-311 Homeland Defense 
Appendix IV: GAO Contacts and Staff 

Acknowledgments

Page 39 GAO-12-311 Homeland Defense 

Brian J. Lepore, (202) 512-4523 or leporeb@gao.gov

In addition to the contact named above, the following individuals made 

significant contributions to this report: Davi M. D’Agostino, Director 

(retired); Joseph Kirschbaum, Assistant Director; James D. Ashley; 

Tommy Baril; Mark Braza; Susan C. Ditto; Brandon Jones; David F. 

Keefer; Charles W. Perdue; Karen A. Richey; Jeff M. Tessin; Amie M. 

Steele; Amanda G. Weldon; and Michael Willems.

Appendix IV: GAO Contacts and Staff 

Acknowledgments 

GAO Contacts

Staff 

Acknowledgments
Related GAO Products

Page 40 GAO-12-311 Homeland Defense 

Homeland Defense: Actions Needed to Improve Management of Air 

Sovereignty Alert Operations to Protect U.S. Airspace. GAO-09-184. 

Washington, D.C.: January 27, 2009.

Homeland Defense: Actions Needed to Address Management of Air 

Sovereignty Alert Operations to Protect U.S. Airspace. GAO-09-612T. 

Washington, D.C.: April 22, 2009.

Tactical Aircraft: Air Force Fighter Reports Generally Addressed 

Congressional Mandates, but Reflected Dated Plans and Guidance, and 

Limited Analyses. GAO-11-323R. Washington, D.C.: February 24, 2011

Tactical Aircraft: DOD’s Ability to Meet Future Requirements Is Uncertain, 

with Key Analyses Needed to Inform Upcoming Investment Decisions. 

GAO-10-789. Washington, D.C.: July 29, 2010.

Aviation Security: TSA Has Made Progress, but Additional Efforts Are 

Needed to Improve Security. GAO-11-938T. Washington, D.C.: 

September 16, 2011.

Aviation Security: TSA’s Revised Cost Comparison Provides a More 

Reasonable Basis for Comparing the Costs of Private-Sector and TSA 

Screeners. GAO-11-375R. Washington, D.C.: March 4, 2011.

Transportation Security: Comprehensive Risk Assessments and Stronger 

Internal Controls Needed to Help Inform TSA Resource Allocation. 

GAO-09-492. Washington, D.C.: March 27, 2009.

Intelligence, Surveillance, and Reconnaissance: DOD Needs a Strategic, 

Risk-Based Approach to Enhance Its Maritime Domain Awareness. 

GAO-11-621. Washington, D.C.: June 20, 2011.

Defense Management: Comprehensive Cost Information and Analysis of 

Alternatives Needed to Assess Military Posture in Asia. GAO-11-316. 

Washington, D.C.: May 25, 2011.

Defense Critical Infrastructure: Actions Needed to Improve the 

Identification and Management of Electrical Power Risks and 

Vulnerabilities to DOD Critical Assets. GAO-10-147. Washington, D.C.: 

October 23, 2009.

Related GAO Products

Homeland Defense

DOD Tactical Aircraft

Aviation Security

Risk Management
Related GAO Products

Page 41 GAO-12-311 Homeland Defense 

Risk Management: Strengthening the Use of Risk Management Principles 

in Homeland Security. GAO-08-904T. Washington, D.C.: June 25, 2008.

Highlights of a Forum Convened by the Comptroller General of the United 

States: Strengthening the Use of Risk Management Principles in 

Homeland Security. GAO-08-627SP. Washington, D.C.: April 15, 2008.

Homeland Security: Applying Risk Management Principles to Guide 

Federal Investments. GAO-07-386T. Washington, D.C.: February 7, 

2007.

Risk Management: Further Refinements Needed to Assess Risks and 

Prioritize Protective Measures at Ports and Other Critical Infrastructure. 

GAO-06-91. Washington, D.C.: December 15, 2005.

Department of Defense: Financial Management Improvement and Audit 

Readiness Efforts Continue to Evolve. GAO-10-1059T. Washington, D.C.: 

September 29, 2010.

Financial Management

(351594)
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (www.gao.gov). Each weekday afternoon, 

GAO posts on its website newly released reports, testimony, and 

correspondence. To have GAO e-mail you a list of newly posted products, 

go to www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube. 

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. 

Visit GAO on the web at www.gao.gov. 

Contact:

Website: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Please Print on Recycled Paper.