INTELLIGENCE 

COMMUNITY

Analysis of 

Alternatives Approach 

for a New Site 

Reflects Most 

Characteristics of a 

High-Quality Process

Report to the Chairman, Subcommittee 

on Military Construction, Veterans 

Affairs, and Related Agencies, 

Committee on Appropriations, 

U.S. Senate

July 2017

GAO-17-643

United States Government Accountability Office
United States Government Accountability Office

Highlights of GAO-17-643, a report to the 

Chairman, Subcommittee on Military 

Construction, Veterans Affairs, and Related 

Agencies, Committee on Appropriations, U.S. 

Senate

July 2017

INTELLIGENCE COMMUNITY

Analysis of Alternatives Approach for a New Site 

Reflects Most Characteristics of a High-Quality 

Process 

What GAO Found

In 2012, the National Geospatial-Intelligence Agency (NGA) began an analysis of 

alternatives (AOA) process to evaluate potential sites for its new NGA Campus 

West (NGA West) using key evaluation factors related to mission, security, 

development and sustainability, schedule, cost, and environment. NGA’s process 

included levels of analysis and considerations to select the agency-preferred 

alternative from an original list of 186 potential sites, subsequently narrowed to 

the final four alternative sites (see figure). The process culminated in the June 

2016 selection of the agency-preferred alternative, the St. Louis City site. 

Map of Alternatives for National Geospatial-Intelligence Agency (NGA) West Sites

NGA’s process for selecting a site for the new NGA West campus substantially 

met three of the four characteristics of a high-quality, reliable AOA process. 

Specifically, NGA’s process substantially met the characteristics that 

demonstrate a well-documented, comprehensive, and unbiased AOA process. It

partially met the credibility characteristic, in part because it did not fully include 

information on the risks and sensitivities to cost estimates. NGA officials stated 

that there was no comprehensive DOD guidance to inform its AOA process, and

although NGA’s AOA process is complete, NGA plans to develop full cost 

estimates as part of construction, planning, and design. In September 2016,

GAO recommended that DOD develop guidance for the use of AOA best 

practices for certain types of military construction decisions. While DOD did not 

concur and the recommendation remains open, GAO continues to believe such 

guidance would help ensure that future AOA processes are reliable and would 

result in decisions that best meet mission needs.

View GAO-17-643. For more information, 

contact Brian Lepore at (202) 512-4523 or 

leporeb@gao.gov.

Why GAO Did This Study

NGA, a defense agency and element 

of the Intelligence Community, 

provides geospatial intelligence to 

military and intelligence operations to 

support national security priorities. It 

currently operates out of two primary 

facilities—its headquarters in 

Springfield, Virginia, and its NGA West 

campus in St. Louis, Missouri. In 2012, 

NGA determined that a new location 

for its NGA West facilities was 

necessary to meet security standards 

and better support its national security 

mission. NGA estimates that the 

construction of the new campus will 

cost about $945 million.

GAO was asked to evaluate the AOA 

process that NGA used to select the 

site for its new campus. This report (1) 

describes the process NGA used, 

including the key factors it considered 

and (2) evaluates the extent to which 

the AOA process met best practices for 

such analyses. 

GAO visited the existing NGA West

campus and the final four alternative 

sites that were considered, analyzed 

and assessed reports and information 

that document NGA’s AOA process for 

selecting the site, and interviewed 

relevant officials about the process. 

GAO evaluated NGA’s process against 

best practices identified by GAO as 

characteristics of a high-quality, 

reliable AOA process.

What GAO Recommends

GAO is not making recommendations 

to NGA. In commenting on a draft of 

this report, NGA expressed concerns 

about GAO’s assessment of NGA’s 

estimates of cost risks and 

sensitivities. GAO continues to believe 

its assessment accurately reflects 

NGA’s process.
Page i GAO-17-643 Intelligence Community

Letter 1

Background 6

NGA’s Site Selection Process Began in 2012, and Potential Sites 

Were Evaluated Based on Mission, Security, Environment, 

Cost, and Schedule Considerations 10

NGA’s AOA Process for Selecting the New NGA West Site 

Substantially Met Three and Partially Met One Characteristic of 

a High-Quality, Reliable AOA Process but Lacked Important 

Cost Information 20

Agency Comments and Our Evaluation 28

Appendix I Analysis of Alternatives (AOA) 22 Best Practices with 

GAO’s Evaluation of National Geospatial-Intelligence 

Agency (NGA) AOA Process and Scores 31

Appendix II Comments from the National Geospatial-Intelligence Agency 39

Appendix III GAO Contact and Staff Acknowledgments 41

Related GAO Products 42

Tables

Table 1: National Geospatial-Intelligence Agency (NGA) Site 

Evaluation Results and Agency-Preferred Alternative 19

Table 2: Summary of GAO’s 22 Analysis of Alternatives (AOA) 

Best Practices Grouped into Four Characteristics, with 

GAO’s Scores for the National Geospatial-Intelligence 

Agency (NGA) AOA Process for the Site Selection of NGA 

West 21

Table 3: Evaluation of the National Geospatial-Intelligence Agency 

(NGA) Analysis of Alternatives (AOA) Process against 

GAO’s Best Practices 31

Contents
Page ii GAO-17-643 Intelligence Community

Figures

Figure 1: Geographic Locations of the Four Final Site Alternatives 

for the New National Geospatial-Intelligence Agency 

(NGA) West 12

Figure 2: Key Events and Decisions for the National GeospatialIntelligence Agency (NGA) West Site Selection Process 14

Abbreviations List

AOA analysis of alternatives

DOE Department of Energy 

DOD Department of Defense

NGA National Geospatial-Intelligence Agency

NGA East National Geospatial-Intelligence Agency Campus 

East

NGA West National Geospatial-Intelligence Agency Campus 

West

PMO Program Management Office

Corps of Engineers U.S. Army Corps of Engineers

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-17-643 Intelligence Community

441 G St. N.W.

Washington, DC 20548

July 28, 2017

The Honorable Jerry Moran

Chairman

Subcommittee on Military Construction, 

Veterans Affairs, and Related Agencies

Committee on Appropriations

United States Senate

Dear Mr. Chairman:

The National Geospatial-Intelligence Agency (NGA), a defense agency 

and member of the Intelligence Community, provides geospatial 

intelligence to the U.S. military, intelligence operations and analysis, and 

homeland defense in support of national security priorities, such as 

counterterrorism and counter-proliferation. NGA conducts the majority of 

its work in two locations—its Springfield, Virginia, headquarters, known as 

NGA Campus East (NGA East), and NGA Campus West (NGA West) 

facilities in St. Louis and Arnold, Missouri.1 In 2012, NGA determined that 

it needed a new NGA West facility in order to better support its national 

security mission and address physical facility and security challenges of 

the current St. Louis campus, among other reasons. Preliminary 

estimates put the project’s construction cost at roughly $945 million, and it 

is designated a mega-project by the U.S. Army Corps of Engineers 

 1

According to officials, the data center in Arnold, Missouri, and the campus in St. Louis are 

collectively referred to as NGA West. Because only the St. Louis facilities are to be moved 

to a new site, this report’s use of “NGA West” refers only to the St. Louis campus. NGA 

employees are also stationed at other locations with military, diplomatic, and other 

partners, both domestically and overseas.

Letter
Page 2 GAO-17-643 Intelligence Community

(Corps of Engineers).2 According to NGA officials as of May 2017, 

construction of NGA West is expected to begin in summer 2019.

As part of the 2005 Base Realignment and Closure process, NGA closed 

some of its facilities in the Washington, D.C., metropolitan area and 

consolidated to a new headquarters facility on Fort Belvoir in Springfield, 

Virginia. NGA East was originally planned to cost $1.1 billion but 

ultimately cost over $2.5 billion after new requirements were added that 

were identified as essential to mission operability.3

You asked us to evaluate the analysis of alternatives (AOA) process that 

NGA used to select the site for its new NGA West. Our objectives were to 

(1) describe the overall site selection and AOA process NGA used, 

including the key factors it considered and (2) evaluate the extent to 

which NGA’s AOA process met best practices for such analyses.

To describe the process and considerations that informed NGA’s site 

selection for its new NGA West, we assessed documents and other 

information and interviewed officials from NGA and other DOD 

organizations, including the Corps of Engineers and the Office of the 

Undersecretary of Defense for Acquisition, Technology and Logistics. In 

addition, we visited the existing NGA West in St. Louis, Missouri in order 

to observe the condition of the current facilities, and we visited the four 

final alternative sites in Missouri and Illinois. We reviewed documentation 

 2

The Corps of Engineers developed this cost estimate (in 2014 dollars) as part of the 

environmental impact statement, which identifies economic, environmental, and other 

impacts of the project. This estimate was completed as part of the site selection process 

prior to the development of facility design and structural plans, and it was not provided as 

an official estimate of construction costs, according to NGA and Corps of Engineers 

officials. Corps of Engineers mega projects are those identified to be of large dollar value 

(in excess of $200 million), long duration, high complexity, and flexible pricing, among 

other attributes. According to NGA and Corps of Engineers officials, full construction cost 

estimates for the project are currently under development. Additionally, any estimate for 

facility construction costs would likely be for unclassified construction costs only and 

would not include costs for relocation, technology, or intelligence-related costs identified in 

classified budget documents. 

3

In 2013 we reported that numerous Base Realignment and Closure military construction 

projects underestimated costs, in part due to reported issues with the Cost of Base 

Realignment Actions Model for comparing costs and savings for base closure and 

realignment recommendations. See GAO, Military Bases: Opportunities Exist to Improve 

Future Base Realignment and Closure Rounds, GAO-13-149 (Washington, D.C.: Mar. 7, 

2013). We made several recommendations to improve the cost estimating process, and 

while DOD has taken action in response to some recommendations, as of May 2017 none 

had yet been fully implemented.
Page 3 GAO-17-643 Intelligence Community

that described the condition of the current facilities, and we examined 

economic and qualitative analysis conducted in 2010–2012 that informed 

the decision to build a new NGA West on a new site.4

We further reviewed NGA plans, processes, and internal and public 

reports outlining the site selection and evaluation process to understand 

the requirements, criteria, and other factors NGA considered in selecting 

the site for the new NGA West, from 2012 through its final public decision 

in June 2016. These documents include two site location studies 

conducted during 2012 through 2014 that identified available sites and 

minimum requirements; internal briefing and process documents 

regarding each stage of the agency’s process to select its preferred 

alternative, which outlined the evaluation criteria as well as how each 

alternative was scored and evaluated against those criteria; and 

preliminary master plans for each site. We reviewed NGA and Corps of 

Engineers cost and schedule assessments; environmental analyses for 

the project, including the publicly released draft and final environmental 

impact statements and NGA’s record of decision;5 documents and 

memoranda outlining inputs and decisions regarding the agency’s 

preferred alternative; and relevant policy and guidance documents 

including the 2015 NGA Strategy, and DOD instructions for economic 

analysis and acquisition of real property.6

To determine the extent to which NGA’s AOA process for the selection of 

the site for its new NGA West aligns with best practices for such 

analyses, we reviewed all data and documentation developed by or for 

 4

Documents include seismic and condition assessments of the current facilities, security 

and threat analysis, economic analysis, and other qualitative analysis. Additionally, NGA 

refers to the project as “Next NGA West,” or N2W, but for the purposes of this report we 

will refer to it as NGA West.

5

U.S. Army Corps of Engineers, Draft Environmental Impact Statement for the Next NGA 

West Campus in the Greater St. Louis Metropolitan Area (Kansas City, MO: October 

2015) and Final Environmental Impact Statement for the Next NGA West Campus in the 

Greater St. Louis Metropolitan Area (Kansas City, MO: April 1, 2016); NGA, Record of 

Decision Next NGA West Campus in the Greater St. Louis Metropolitan Area (June 2, 

2016).

6

NGA, 2015 NGA Strategy (available at 

https://www.nga.mil/About/NGAStrategy/Pages/default.aspx); DOD Instruction 4165.71, 

Real Property Acquisition (Jan. 6, 2005); and DOD Instruction 7041.3, Economic Analysis 

for Decision-Making (Nov. 7, 1995) and its subsequent reissue DOD Instruction 7041.03, 

Economic Analysis for Decision-Making (Sept. 9, 2015) (hereinafter cited as DOD 

Instruction 7041.03 (Sep. 9, 2015)).
Page 4 GAO-17-643 Intelligence Community

DOD, including NGA and the Corps of Engineers, as part of NGA’s site 

selection and evaluation process (which we refer to as its “AOA 

process”).7 We discussed these data and this documentation with officials 

from NGA and the Corps of Engineers, and with other DOD officials. We 

refer to this collection of data and documentation to support NGA’s AOA 

process as NGA’s “AOA body of work.” After collecting available data, 

documentation, and other information, we evaluated NGA’s AOA body of 

work against our 22 AOA best practices.8

To evaluate NGA’s AOA process against these best practices, we took 

the following steps: (1) two GAO analysts independently examined the 

AOA information received from NGA, providing a score for each of the 22 

best practices; (2) a third GAO analyst adjudicated any differences 

between the two analysts’ initial scoring; (3) a GAO specialist on AOA 

best practices, independent of the audit team, reviewed the team’s 

adjudicated AOA documentation and scores, cross-checking the scores 

 7

As part of our previous work assessing AOA processes for intelligence facilities, we 

evaluated NGA’s process that led to the determination to build a new government-owned 

facility. Accordingly, this report does not re-evaluate NGA’s decision to build a new facility, 

instead focusing on the subsequent site selection process to determine the agency’s 

preferred alternative for the new NGA West site. However, to the extent those prior 

documents are referenced or reproduced in materials used in the site selection process, 

or used as an input to the site selection analysis, we considered the information as part of 

our analysis.

8

These best practices were published in GAO, Amphibious Combat Vehicle: Some 

Acquisition Activities Demonstrate Best Practices; Attainment of Amphibious Capability to 

Be Determined, GAO-16-22 (Washington, D.C.: Oct. 28, 2015), and more recently in Joint 

Intelligence Analysis Complex: DOD Partially Used Best Practices for Analyzing 

Alternatives and Should Do So Fully for Future Military Construction Decisions, 

GAO-16-853 (Washington, D.C.: Sept. 30, 2016) and the previously published classified 

version of GAO-16-853. An earlier version of the best practices was published in GAO, 

DOE and NNSA Project Management: Analysis of Alternatives Could Be Improved by 

Incorporating Best Practices, GAO-15-37 (Washington, D.C.: Dec. 11, 2014). Explanation 

and application of the AOA best practices, including the definitions of each of the best 

practices and the applicability of the best practices to DOD military construction and 

acquisition policies and procedures, can be found in GAO-16-853.
Page 5 GAO-17-643 Intelligence Community

and all of the analyses for consistency.9 The resulting scores for 

individual best practices were used to identify the averaged, overall score 

for the four summary characteristics of a high-quality, reliable AOA 

process—that it should be well-documented, comprehensive, unbiased, 

and credible.10 Next, we shared our draft analysis with NGA, asking the 

agency to provide technical comments and any additional documentation 

or other information that might affect our assessment. We then 

incorporated these additional comments to ensure that our analysis 

included all available information. Finally, we applied the same 

methodology and scoring process explained above to revise our initial 

analysis based on NGA’s technical comments and any other additional 

evidence we received, as appropriate.

While applying our AOA process best practices to NGA’s AOA body of 

work, we assessed the reasonableness of the information we collected. 

Examining NGA’s process for site selection against our AOA best 

practices allowed us to assess the strengths and limitations of the 

agency’s process. Our best practices were not used to determine whether 

NGA had made the correct decision on the location for its new campus or 

whether the department would have arrived at a different conclusion had 

it more fully conformed to the best practices. Rather, we used our best 

practices to assess the degree to which NGA can provide reasonable 

assurance that its process met each of the four characteristics of a highquality, reliable AOA process. Based on discussions with NGA and the 

Corps of Engineers, we also determined that the NGA data we used to 

understand NGA’s site selection process were sufficiently reliable for our 

purposes of describing and understanding NGA’s process and rationale 

for choosing the St. Louis City site as the new location.

 9

The process for scoring the alignment with best practices consists of qualitative and 

quantitative categories. The qualitative categories are as follows—Fully Met: NGA 

provided complete evidence that satisfies the elements of the best practice; Substantially 

Met: NGA provided evidence that satisfies a large portion of the elements of the best 

practice; Partially Met: NGA provided evidence that satisfies about half of the elements of 

the best practice; Minimally Met: NGA provided evidence that satisfies a small portion of 

the elements of the best practice; and Not Met: NGA provided no evidence that satisfies 

any of the elements of the best practice. The corresponding quantitative categories are as 

follows: Not Met = 1, Minimally Met = 2, Partially Met = 3, Substantially Met = 4, and Fully 

Met = 5. 

10The average score for each characteristic corresponds to one of five qualitative 

categories as follows: Not Met = 1.0 to 1.4, Minimally Met = 1.5 to 2.4, Partially Met = 2.5 

to 3.4, Substantially Met = 3.5 to 4.4, and Fully Met = 4.5 to 5.0.
Page 6 GAO-17-643 Intelligence Community

We conducted this performance audit from September 2016 to July 2017 

in accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that 

the evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives.

The existing NGA West campus consists of 15 facilities on 27 acres. 

Some of the buildings’ original construction dates back to the early 1800s, 

and 22 acres of the site are on the National Register of Historic Places, 

according to NGA documents. In 2009 through 2010, NGA contracted 

with an independent firm to assess the condition of the existing NGA 

West. This assessment gave the facilities an overall condition rating of 

“poor,” generally because of the insufficiency of the anti-terrorism and 

force protection measures, the average age of the structures, numerous 

code and accessibility shortfalls, and lack of seismic protection.11

Near the end of the completion of the NGA East headquarters 

consolidation in 2011, NGA focused its attention on the need to improve 

the operational capacity, security requirements, and modernization of its 

NGA West facilities. From approximately 2009 through 2012, NGA 

conducted a series of evaluations to inform its efforts to modernize NGA 

West.12 These analyses included a condition assessment of the existing 

facilities; an economic analysis of alternatives to evaluate the options of 

building a new facility (“build new”), fully renovating the existing facilities 

(“modernize”), or remaining in the current facilities with minimum essential 

repairs (“status quo”); and a qualitative analysis of non-cost 

considerations for the build new, modernize, and status quo options 

identified in the economic analysis.

 11Frontenac Engineering and Ross & Baruzzini, Inc., Facilities Condition Assessment: 

Second Street Complex—St. Louis, MO (May 13, 2010). A study in 1996 also found that 

the seismic protection of many of the NGA West buildings was not adequate per 

contemporaneous building codes.

12As mentioned above, we previously evaluated NGA’s process that led to the 

determination to build a new government-owned facility for NGA West. 

Background

Decision to Build a New 

NGA West
Page 7 GAO-17-643 Intelligence Community

In 2012 NGA determined that a new NGA West would best meet the 

agency’s mission and resource needs. After examining the options of 

renovating its current facility, leasing, or building a new governmentowned facility, NGA determined that building a new, government-owned 

facility was the preferred option. NGA officials stated that they are in the 

process of soliciting design-build proposals and that the final design-build 

contract is planned for award near the end of fiscal year 2018. 

Construction is expected to begin approximately in the summer of 2019.

We identified 22 best practices for an AOA process in October 2015, 

based on government and private-sector guidance and input from 

subject-matter experts.13 Many federal and industry guides have 

described approaches to analyses of alternatives; however, there was no 

single set of practices for the AOA process that was broadly recognized 

by both government and private-sector entities. We developed these best 

practices by (1) compiling and reviewing commonly mentioned AOA 

policies and guidance used by different government and private-sector 

entities and (2) incorporating experts’ comments on a draft set of 

practices to develop a final set of practices. The 22 best practices are 

grouped into four characteristics that describe a high-quality, reliable AOA 

process and can be used to evaluate whether an AOA process meets the 

characteristics of well-documented, comprehensive, unbiased, and 

credible. These practices can be applied to AOA processes for a broad 

range of capability areas, projects, and programs, including military 

construction projects and decision-making processes, in which an 

alternative must be selected from a set of possible options.

In September 2016, we recommended that DOD develop guidance that 

requires the use of AOA best practices, including those practices we 

identified, when conducting AOA processes for certain types of military 

construction decisions. DOD did not concur with this recommendation 

and disagreed that these best practices apply to military construction 

decision-making processes.14 We continue to believe that this 

recommendation is valid and that the principles demonstrated by the best 

practices—and the practices themselves—draw from related DOD and 

other practices.

 13These 22 best practices and their definitions were originally published in GAO-16-22 

and are based on previously published best practices.

14GAO-16-853. As of June 2017, this recommendation has not been implemented.

Best Practices for an 

Analysis of Alternatives 

Process
Page 8 GAO-17-643 Intelligence Community

Our best practices also parallel those found in DOD and Air Force 

guidance on military construction and analysis for decision making.15 For 

example, according to an Air Force instruction governing the planning and 

programming of military construction projects, one of the required 

planning actions is to evaluate alternative solutions.16 According to a DOD 

directive pertaining to military construction, DOD must monitor the 

execution of its military construction program to ensure—among other 

things—that the program is accomplished in the most cost-effective 

way.17 This guidance for cost effectiveness aligns with our AOA best 

practice Develop Life-cycle Cost Estimates, which focuses on providing 

decision makers with the information they need to assess the costeffectiveness of alternatives. Further, DOD Instruction 7041.03, on 

economic analysis for decision making, contains numerous cost 

estimating principles and procedures that align with those called for in our 

AOA best practices.18 As we reported in 2016, these policy documents 

and instructions align with the general intent of our best practices, and 

there are many similarities between our best practices and the 

department’s guidance.

Additionally, in our previous work reviewing AOA process for other 

national security facilities, agencies generally concurred with our 

recommendations to consider including our best practices in future 

guidance. For example, in 2014 we assessed three National Nuclear 

Security Administration construction projects and found each project’s 

AOA partially met our best practices for conducting an AOA process. The 

 15Examples of sources used to develop the 22 best practices include Office of Aerospace 

Studies, U.S. Air Force Materiel Command, Analysis of Alternatives (AOA) Handbook: A 

Practical Guide to Analyses of Alternatives (June 2013); Defense Acquisition University, 

The Defense Acquisition Guidebook (accessed Feb. 13, 2017 at

https://www.dau.mil/tools/dag); DOD Instruction 5000.02, Operation of the Defense 

Acquisition System (Jan. 7, 2015); and U.S. Army, Army Cost Benefit Analysis Guide, 3rd 

Ed (Apr. 24, 2013). For more information on the development of our AOA best practices, 

including the congruence of these best practices with DOD and military policies, see 

GAO-16-853. 

16Air Force Instruction 32-1021, Planning and Programming Military Construction 

(MILCON) Projects, (Feb. 25, 2016). 

17DOD Directive 4270.5, Military Construction (Feb. 12, 2005). 

18DOD Instruction 7041.03 (Sept. 9, 2015). 
Page 9 GAO-17-643 Intelligence Community

Department of Energy agreed with our recommendation and has begun 

implementation.19

 19In GAO-15-37, we recommended that the Department of Energy (DOE) update its 

project management requirements to incorporate our best practices, and DOE agreed. As 

of June 2017, the recommendation had not been implemented, but DOE has updated a 

relevant departmental order to state that it will be consistent with our best practices and 

has committed to incorporating these best practices in forthcoming guidance expected to 

be complete in fiscal year 2017. For a list of reports related to this topic, see the Related 

GAO Products page at the end of this report.
Page 10 GAO-17-643 Intelligence Community

NGA launched its search for a new NGA West site in 2012 with a site 

location study conducted by an outside real estate firm, and it concluded 

the search with the issuance of a record of decision in June 2016. The 

site location study included a check for existing federal sites that could 

accommodate NGA West’s workforce and mission.20 This search resulted 

in a total of 186 sites being identified initially as possible options; the list 

was narrowed to 6 sites in the St. Louis metropolitan area for further 

 20Jones Lang LaSalle, National Geospatial-Intelligence Agency West Facilities 

Modernization Site Location Study Final Report (June 17, 2013). NGA attempted to 

identify federal sites, including outside the St. Louis metropolitan area, but due to general 

lack of availability, and legal requirements regarding closure or realignment of certain 

installations in accordance with section 2687 of Title 10 of the United States Code, as well 

as the existing workforce that resides in the area, the site selection focused on the greater 

St. Louis area, according to officials. 

NGA’s Site Selection 

Process Began in 

2012, and Potential 

Sites Were Evaluated 

Based on Mission, 

Security, 

Environment, Cost, 

and Schedule 

Considerations

Site Selection Process 

Began in 2012 and 

Concluded with the June 

2016 Record of Decision
Page 11 GAO-17-643 Intelligence Community

study.21 During preliminary master planning, 4 of the 6 sites identified by 

the site location studies were determined to be suitable for further 

analysis to select the agency’s preferred alternative.22 Three of these 

sites are in the Missouri cities of Fenton, Mehlville, and St. Louis, and one 

is in St. Clair County, Illinois, near Scott Air Force Base. See figure 1 for 

the geographic distribution of the 4 sites.

 21After the 186 sites were identified, a filtering process against minimum criteria resulted 

in 31 for further study, and sufficient information was received for 22 to be evaluated 

against specific criteria, resulting in 6 final sites. The steps to narrow the 186 sites to 6 

were as follows: (1) Establish site minimum requirements including minimum acreage and 

delineated areas; (2) Issue a short general request for information for the sites that 

appeared to meet the site minimums; (3) Issue a more detailed request for information on 

the narrowed list of 31 sites; and (4) Perform consensus evaluation on the 22 sites that 

responded to the detailed request for information based on the evaluation factors. A 

second site location study using the same methodology was conducted by the Corps of 

Engineers in 2014 in order to validate the original results, given the amount of time that 

had elapsed, and to ensure that input from the public was included. The second study 

evaluated 13 site locations—the original 6 from the first study and 7 additional sites—to 

determine which properties were viable and available for NGA West, ultimately validating 

the original 6 sites for further analysis.

22NGA determined that two sites were no longer viable for NGA West. One could not 

accommodate mission requirements without greatly exceeding the high-rise construction 

limit. A second became unsuitable for the project following the sale of a portion of the site. 
Page 12 GAO-17-643 Intelligence Community

Figure 1: Geographic Locations of the Four Final Site Alternatives for the New National Geospatial-Intelligence Agency (NGA) 

West
Page 13 GAO-17-643 Intelligence Community

The subsequent site selection process included an environmental impact 

statement as required by the National Environmental Policy Act of 1969, 

analysis of NGA and the Corps of Engineers’ compliance with related 

DOD policies and other federal laws and requirements, preliminary 

master planning conducted by the Corps of Engineers, and a site 

evaluation process initiated by the NGA West Program Management 

Office (PMO), which is responsible for managing the NGA West project.23

To select the final site from the four alternatives, NGA initiated a site 

evaluation process in August 2015 that was led by the NGA West PMO. 

This process involved various teams of experts analyzing the sites and 

evaluating them against defined criteria to identify the advantages and 

disadvantages of each site.

Figure 2 provides an overview of the key elements and milestones of 

NGA’s site selection process, beginning with its earlier decision to build 

and concluding with its 2016 selection of the new site and issuance of its 

record of decision.

 23Under the National Environmental Policy Act of 1969, Pub. L. No. 91-190 (1970), 

codified at 42 U.S.C. §§ 4321-4347, agencies evaluate the likely environmental effects of 

projects they are proposing using an environmental assessment or, if the projects likely 

would significantly affect the environment, a more detailed environmental impact 

statement. If, however, the agency determines that activities of a proposed project fall 

within a category of activities the agency has already determined has no significant 

environmental impact—called a categorical exclusion—then the agency generally need 

not prepare an environmental assessment or impact statement. The key elements and 

documents required when conducting an environmental impact statement include notice of 

intent in the Federal Register; draft and final environmental impact statements with 

opportunities for public comment; identification of the agency-preferred alternative (if there 

is one); and a record of decision. Additionally, according to officials from NGA and the 

Office of the Undersecretary of Defense for Acquisition, Technology and Logistics, NGA 

complied with DOD instructions on real property acquisition and economic analysis in its 

decision to build a new NGA West and select a new site. 
Page 14 GAO-17-643 Intelligence Community

Figure 2: Key Events and Decisions for the National Geospatial-Intelligence Agency (NGA) West Site Selection Process

According to NGA officials, there was no NGA or DOD policy or set of 

practices to comprehensively guide NGA’s site selection and AOA 

process. As a result, NGA relied on various DOD policies and 

instructions, other federal guidance, and industry standards. It 

incorporated these practices into the site selection process to ensure that 

it complied with federal requirements and industry practice to develop its 
Page 15 GAO-17-643 Intelligence Community

AOA process, according to NGA and Corps of Engineers officials.24

Additionally, NGA officials stated that our AOA best practices would have 

been helpful in planning the site selection process for NGA West, but the 

process began in 2012, and our 22 best practices were not published until 

October 2015.

At the outset of the site evaluation process in August 2015, the PMO set 

forth broad sets of criteria to use in analyzing the four alternatives. These 

broad sets of criteria, referred to as “evaluation factors,” were mission, 

security, development and sustainability, schedule, cost, and 

environment. In addition, each site was assessed to ensure that it 

complied with key laws, regulations, and directives. The PMO divided the 

analysis of the evaluation factors among NGA and Corps of Engineers 

teams.

The mission, security, and development and sustainability factors were 

assigned to two NGA evaluation teams of subject-matter experts—the 

“mission evaluation team” and “security, infrastructure and schedule 

 24According to NGA officials, given existing DOD guidance that applies to NGA 

operations, military construction, and program oversight, and because NGA consists of 

three main facilities and does not regularly conduct large-scale construction projects, NGA 

does not have separate guidance for AOA for construction. NGA identified various federal 

and DOD guidance documents and regulations that generally applied to NGA’s site 

selection process, including General Services Administration, Site Selection Guide; DOD 

Instruction 4165.71 Real Property Acquisition, and DOD Instruction 7041.3 Economic 

Analysis (the latter was updated in September 2015 as DOD Instruction 7041.03).

Additionally, although DOD did not concur with our 2016 recommendation to develop 

guidance requiring the use of our AOA best practices, NGA officials stated that such 

guidance would have been helpful to their site selection process.

Potential Sites Were 

Assessed Based on Key 

Factors, and NGA and the 

Corps of Engineers 

Conducted Additional 

Analysis in Order to Reach 

a Final Determination

Potential Sites Were Assessed 

Based on Mission, Security, 

Environment, Cost, and 

Schedule Factors
Page 16 GAO-17-643 Intelligence Community

evaluation team” (referred to here as security evaluation team).25 Each of 

these teams used its expertise to develop “sub-factors” to assess the 

advantages and disadvantages of each site. For example:

• The mission evaluation team developed 10 mission-related subfactors based on the PMO guidance, NGA’s mission, and the strategic 

goals outlined in the 2015 NGA Strategy.

26 The mission-related subfactors focused largely on elements pertaining to NGA’s workforce 

and partnerships, such as the sites’ proximity to the existing 

workforce, their distance from NGA’s Arnold facility, and the likelihood 

that the sites would attract mission partners to create a “GEOINT 

Valley.”27

• The security evaluation team developed 13 sub-factors related to 

security and infrastructure based on PMO guidance, DOD and other 

federal security and energy requirements, threat analysis, and other 

subject-matter expertise. Examples of the sub-factors include a 500-

foot setback, perimeter security elements, sustainable characteristics, 

and infrastructure resilience.

Separate evaluations of cost, schedule, and environmental considerations 

were conducted by the Corps of Engineers in its role as construction 

agent as part of the environmental impact analysis. In addition, NGA and 

the Corps of Engineers conducted an assessment of relevant laws and 

regulations.

The PMO integrated these analyses and provided an additional layer of 

review to each of the evaluation factors, in some cases adjusting them. 

 25According to the PMO’s documentation of the process, the original scope of the security 

evaluation team was to also evaluate schedule and environmental considerations, but 

because the Corps of Engineers was evaluating these elements as part of the National 

Environmental Policy Act analysis and in its role as NGA’s construction agent for the 

project, the security evaluation team focused its analysis on the security and development 

and sustainability evaluations. 

26The four strategic goals of “Our People, Our Partners, Our Profession, and Our Value”

focus on various aspects of the NGA workforce, expanding partnerships and engagement, 

excelling at NGA’s craft, and NGA’s customers. 

27In support of NGA’s strategic goal of “Our Partners,” NGA has identified a desire to 

create a regional GEOINT—or, geospatial intelligence—community of various 

government, industry, and academic partners in relation to NGA West, and this informed 

the criteria at all levels of analysis. Specifically, the mission evaluation team and PMO 

analysis expressed this desire to attract mission partners colloquially as creating a 

“GEOINT Valley,” and the site selection team and director used “Team GEOINT” to 

express this aim. 
Page 17 GAO-17-643 Intelligence Community

For instance, the PMO reorganized the 10 mission-related sub-factors for 

its review. Specifically, while the mission evaluation team focused the 

sub-factors largely on NGA’s strategic goals related to workforce and 

partnerships, the PMO’s analysis reorganized those same mission-related 

sub-factors by how they supported all four of the 2015 strategic goals. 

The PMO listed under three “strategic effects”—“Create GEOINT Valley,”

“Enhance Operations,” and “Attract and Sustain the Workforce”—all of the 

sub-factors related to that strategic effect. The PMO re-analyzed the sites 

by weighting those strategic effects and sub-factors that were linked to 

multiple strategic goals higher than those that were linked to fewer such 

goals. The PMO also adjusted some of the sub-factors used in the 

evaluation for security and for development and sustainability.

The PMO’s additional analysis did not change the overall outcome of the 

evaluation of the sites; rather, it validated the mission evaluation team’s 

conclusion and generally supported all but one of the overall findings of 

the other analyses.28 At the conclusion of the PMO’s analysis in 

December 2015, the PMO’s conclusion was that no one site had emerged 

as a clear preferred alternative.

Because the master planning and site evaluation process concluded that 

all four sites—Fenton, Mehlville, St. Louis City, and St. Clair—could meet 

the overall requirements and that no single site held substantial 

advantage over another, the NGA Director requested additional analysis 

with refined criteria to more clearly differentiate among the final four sites. 

Consequently, in January 2016 NGA initiated a new site selection team—

consisting of NGA and Corps of Engineers personnel who had previously 

been involved in various stages of the process—to reassess the sites 

against refined criteria and perspectives in order to determine the agencypreferred alternative.29

The site selection team carried forward five of the six original evaluation 

criteria from the start of the site evaluation process, as well as compliance 

 28Out of the seven broad evaluation factors, the only area in which the PMO did not agree 

with the previous teams’ assessments was the security evaluation team’s conclusion that 

the St. Clair site was slightly more advantageous regarding the development and 

sustainability criterion. In this instance the PMO’s analysis concluded that all of the sites 

were equal. 

29The PMO identified one site—Fenton—as not having sufficient advantages to be carried 

forward for further consideration, given its high cost, schedule risk, and security concerns, 

but the site selection team began its process with all four final sites.

NGA Director Requested 

Additional Analysis in Order to 

Reach a Decision
Page 18 GAO-17-643 Intelligence Community

with federal law, policy, and other regulations, to develop its six “refined 

criteria.”30 In reviewing these refined criteria, the site selection team

determined that cost and schedule accounted for the greatest differences 

among the sites. The team therefore used the cost and schedule 

assessments completed as part of the PMO process to narrow the sites, 

concluding that because the Mehlville and Fenton sites were the most 

expensive and posed the greatest schedule risk they should be 

eliminated from final consideration.

The site selection team then focused its analysis on the final two sites—

St. Clair and St. Louis City—to inform the Director’s selection. The team 

used the following six refined criteria to evaluate the sites: (1) cost, (2) 

schedule, (3) security, (4) mission efficiency and expansion, (5) 

applicability of and compliance with federal policies, executive orders, and 

federal initiatives; and (6) environmental considerations. The team 

proposed narrowing the relevant sub-criteria to those that provided the 

greatest differentiation among the sites, according to officials on the team. 

For example, the security criterion was narrowed to include 3 of the 

original 13 security and infrastructure evaluation sub-factors, and the 

adjusted “mission efficiency and expansion” criterion included one of the 

mission evaluation team’s 10 original mission sub-factors.

Subsequently, the NGA Director provided additional direction, including 

adding a review of potential support from Scott Air Force Base, based on 

the support NGA East receives from being located at Ft. Belvoir, as well 

as ensuring that the security-related sub-factors carried over from prior 

analyses were consistently defined. Additionally, the director added 2 

sub-criteria to the mission-related criterion to ensure that the site 

evaluation continued in terms of NGA’s strategic goals of partnership and 

people:

1. “Team GEOINT,” which refers to NGA’s current and future 

partnerships with academic, public, and private sector partners, and 

which parallels the “GEOINT Valley” element evaluated by the 

mission evaluation team and PMO.

 30The PMO had previously concluded that there was no difference in the seventh area of 

criteria—development and sustainability—and the site selection team did not include this 

in its refined criteria.
Page 19 GAO-17-643 Intelligence Community

2. “Team NGA,” which refers to the potential effects of workforce 

recruitment and retention that were also analyzed in the mission 

evaluation team and PMO analyses.

According to NGA officials, while certain sub-factors or criteria were 

adjusted to provide further layers of analysis, the most important factors 

were always seen as mission and security. Additionally, NGA and Corps 

of Engineers officials said that adding these two sub-criteria expanded the 

analysis of the mission-related criteria to resemble the scope of the

PMO’s analysis and incorporated the NGA Director’s mission and vision 

perspective.

Finally, the NGA Director determined the weighting of the final criteria to 

evaluate the last two sites, the site selection team provided input on 

which of the sites was more advantageous with respect to each criterion, 

and in March 2016 this information was used to inform the NGA Director’s 

selection of the agency-preferred alternative. The weighting and final 

decisions are shown in table 1.31

Table 1: National Geospatial-Intelligence Agency (NGA) Site Evaluation Results and Agency-Preferred Alternative 

Criterion (Evaluation Factor) Weight More Advantageous Site

Mission Efficiency and Flexibility High St. Louis City

Security High St. Clair

Environmental Impact Medium St. Louis City

Laws, Executive Orders, Regulations, 

Policies

Medium St. Louis City

Schedule Low Slight advantage – St. Clair

Cost Low Slight advantage – St. Louis City

Director’s Selection of Agency’s Preferred 

Alternative

St. Louis City

Source: GAO analysis of NGA information. | GAO-17-643

Note: “Weight” refers to the level of precedence each criterion was given in evaluation of the sites. 

Specifically, those criteria with “high” weights would bear more heavily on the final decision than 

those with a “medium” or “low” weight.

 31Simultaneous with the conclusion of the site selection team’s analysis, the St. Clair and 

St. Louis City sites each provided updated information that was included in the director’s 

cost and schedule analysis, but according to officials this did not substantially affect the 

overall findings.
Page 20 GAO-17-643 Intelligence Community

The NGA Director selected the St. Louis City site as the agency-preferred 

alternative. It was identified in the publication of the final environmental 

impact statement and finalized with the issuance of the record of decision 

in June 2016.

We compared NGA’s AOA process for selecting a site for the new NGA 

West campus to our AOA best practices and determined that NGA’s 

process substantially met three and partially met one characteristic of a 

high-quality, reliable AOA process. Although NGA’s AOA process 

substantially met most of the characteristics, we did find areas where the 

process could have been strengthened if NGA had more fully 

incorporated the AOA best practices. See table 2 for a summary of our 

assessment and appendix I for additional details on our scoring of NGA’s 

alignment with each of the 22 best practices.

NGA’s AOA Process 

for Selecting the New 

NGA West Site 

Substantially Met 

Three and Partially 

Met One 

Characteristic of a 

High-Quality, Reliable 

AOA Process but 

Lacked Important 

Cost Information

Site Selection Process 

Substantially Met Three of 

the Four Characteristics 

for a High-Quality, Reliable 

AOA Process but Could 

Have Been Strengthened
Page 21 GAO-17-643 Intelligence Community

Table 2: Summary of GAO’s 22 Analysis of Alternatives (AOA) Best Practices Grouped into Four Characteristics, with GAO’s 

Scores for the National Geospatial-Intelligence Agency (NGA) AOA Process for the Site Selection of NGA West

Characteristic Best Practice

GAO Scoring of NGA AOA 

Process

Well-documented: The AOA 

process is thoroughly described in a 

single document, including all source 

data, clearly detailed methodologies, 

calculations and results, and 

selection criteria are explained.

12. Identify significant risks and 

mitigation strategies. 

14. Tie benefits/effectiveness to 

mission need.

18. Document AOA process in a 

single document. 

19. Document assumptions and 

constraints. 

3 – Partially Met

4 – Substantially Met

3 – Partially Met

4 – Substantially Met

Overall Assessmenta (Well-documented): Substantially Met Average Score: 3.5 

Comprehensive: The AOA process 

ensures that the mission need is 

defined in a way to allow for a robust 

set of alternatives, that no 

alternatives are omitted, and that 

each alternative is examined 

thoroughly for the project’s entire life 

cycle.

1. Define mission need. 

3. Develop AOA timeframe. 

8. Develop list of alternatives. 

11. Assess alternatives’ viability. 

15. Develop life-cycle cost estimates. 

5 – Fully Met

4 – Substantially Met

5 – Fully Met

4 – Substantially Met 

3 – Partially Met 

Overall Assessmenta (Comprehensive): Substantially Met Average Score: 4.2 

Unbiased: The AOA process does 

not have a predisposition toward one 

alternative over another and is based 

on traceable and verified information.

2. Define functional requirements. 

4. Establish AOA team. 

6. Weight selection criteria. 

7. Develop AOA process plan. 

13. Determine and quantify benefits and effectiveness. 

20. Ensure AOA process is impartial. 

22. Compare alternatives 

5 – Fully Met

5 – Fully Met

3 – Partially Met

5 – Fully Met

3 – Partially Met

3 – Partially Met

2 – Minimally Met
Page 22 GAO-17-643 Intelligence Community

Characteristic Best Practice

GAO Scoring of NGA AOA 

Process

Overall Assessmenta (Unbiased): Substantially Met Average Score: 3.71

Credible: The AOA process 

thoroughly discusses the limitations 

of the analyses resulting from the 

uncertainty that surrounds both the 

data and assumptions made for each 

alternative.

5. Define selection criteria. 

9. Describe alternatives. 

10. Include baseline alternative. 

16. Include a confidence interval or range for life-cycle 

cost estimates. 

17. Perform sensitivity analysis. 

21. Perform independent review. 

4 – Substantially Met

5 – Fully Met

3 – Partially Met

2 – Minimally Met

2 – Minimally Met

3 – Partially Met

Overall Assessmenta (Credible): Partially Met Average Score: 3.16 

Source: GAO analysis of NGA information. | GAO-17-643

a

We determined the overall assessment rating by assigning each individual best practice rating a 

qualitative and a quantitative score: Not Met = 1, Minimally Met = 2, Partially Met = 3, Substantially 

Met = 4, and Fully Met = 5. Then we took the average of the individual assessment ratings to 

determine the overall rating for each of the four characteristics. The resulting average becomes the 

overall assessment as follows: Not Met = 1.0 to 1.4, Minimally Met = 1.5 to 2.4, Partially Met = 2.5 to 

3.4, Substantially Met = 3.5 to 4.4, and Fully Met = 4.5 to 5.0.

NGA’s AOA process for selecting a site for the new NGA West 

substantially met the well-documented characteristic of a high-quality, 

reliable AOA process, although we did find areas for improvement.

For example, NGA’s AOA body of work demonstrated that the 

assumptions and constraints for each alternative for the site selection 

process were documented. NGA West’s Prospective Sites Master Plan

included a set of overall assumptions that guided the preliminary planning 

process and provided specific assumptions and constraints for each 

alternative. Specifically, the plan identified various assumptions and 

constraints for the four final sites, such as calculations of the site 

boundaries, the estimated number of parking spaces, the square footage 

of the buildings and estimates of the building’s height, site utilities, and 

environmental constraints, among other things. In one instance, the plan 

documented the assumption that if the Mehlville site were to be used, all 

utilities would need to be removed from within the property line and 

existing buildings, parking lots, and roads would have to be demolished.

In another example, the Corps of Engineers conducted a schedule and 

negotiation risk assessment and recorded scores for each site and some 

mitigation strategies for specific issues. The assessment documented 

risks to meeting the site acquisition schedule with the St. Louis site 

because, among other reasons, the site needed environmental cleanup 

Well-documented 

Characteristic: Substantially 

Met
Page 23 GAO-17-643 Intelligence Community

that was expected to take six months. The Fenton site had high 

negotiation risks, in part because the asking price of the site was 

significantly higher than the appraised value. However, NGA did not 

provide information on other risks, such as technical feasibility and 

resource risks, and did not rank the risks or provide over-arching 

mitigation strategies for each alternative. According to the best practice, 

not documenting the risks and related mitigation strategies for each 

alternative prevents decision makers from performing a meaningful tradeoff analysis, which is necessary to select an alternative to be 

recommended.

NGA’s AOA process for selecting a site for the new NGA West 

substantially met the comprehensiveness characteristic of a high-quality 

AOA process, but although it had strengths, we identified some 

limitations.

NGA’s AOA process considered a diverse range of alternatives to meet 

the mission need and conducted market surveillance and market 

research to develop as many alternative solutions as possible. According 

to our best practices, an AOA process that encompasses numerous and 

diverse alternatives ensures that the study provides a broad view of the 

issue and guards against biases to the AOA process. Specifically, NGA’s 

AOA process included a site location study that provided a summary of 

the thorough analysis that NGA conducted to identify potential site 

locations for the new NGA West campus. The study relied on information 

from local real estate market databases and input from the local real 

estate community, multiple municipal officials and organizations, and the 

public to identify an original set of 186 possible sites and narrow that list 

to a final 6 for further analysis.

However, although the NGA body of work provides evidence that the 

Corps of Engineers developed initial cost estimates that compared each 

alternative using different cost categories, NGA’s AOA process did not 

include life-cycle cost estimates for the final 4 sites.32 NGA officials chose 

not to analyze total construction and other facility sustainment costs, 

because they assumed that since the sites were in the same geographic 

 32A life-cycle cost estimate including all costs from the inception of the project through 

design, development, deployment, operation, maintenance, and disposal should be 

developed for each alternative.

Comprehensive Characteristic: 

Substantially Met
Page 24 GAO-17-643 Intelligence Community

area, construction and operating costs would be similar.33 However, the 

estimates did not include sufficient details regarding all of the costs 

examined—specifically, how the cost estimates were developed for 

information technology trunk line costs. NGA stated that this best practice 

had limited application to its AOA process because it had determined that 

variation in the life-cycle cost estimates based on the location of the four 

sites—all in the St. Louis metropolitan area—was negligible. NGA officials 

also stated that the lack of final project design details constrained their 

ability to develop full life-cycle cost estimates. However, without estimates 

for full life-cycle costs, decision makers may not have a complete picture 

of the costs for each alternative and may have difficulty comparing the 

alternatives, because comparisons may not be based on accurate and 

complete information. NGA and Corps of Engineers officials said that they 

are in the process of developing full life-cycle cost estimates for the 

construction and design of the new NGA West campus, for the agencypreferred alternative.

NGA’s AOA process for selecting a site for the new NGA West 

substantially met the characteristic of an unbiased AOA process, although 

we did identify some limitations.

NGA’s AOA body of work demonstrated that NGA had developed 

functional requirements based on the mission need without a 

predetermined solution and that the requirements were realistic, 

organized, and clear. For example, NGA’s AOA body of work provided 

facilities requirements and specifically listed 11 site location and campus 

requirements that were tied to mission needs, including requiring a facility 

that will support future changes to mission requirements and allow for 

continuity of NGA operations. NGA’s AOA body of work also identified 

physical requirements for the new NGA West campus, for example, that 

the new facility must have at least 800,000 gross square feet and a 500-

foot security buffer, and it must allow for a possible expansion in the 

future.

 33According to Corps of Engineers officials, NGA did not ask them to conduct more longterm cost analysis or to develop life-cycle cost estimates. NGA officials stated that 

consideration of the estimated costs was restricted to those where variation was identified 

and could be reasonably defined. NGA officials focused on the costs they identified as 

being the most variable: land acquisition, site development, transportation improvements, 

and costs for an information technology trunk line. 

Unbiased Characteristic: 

Substantially Met
Page 25 GAO-17-643 Intelligence Community

However, although the NGA AOA body of work demonstrated a thorough 

comparison of the alternatives throughout the site evaluation process, it 

did not provide evidence that net present value was used to compare or 

differentiate among the alternatives, nor did it provide a rationale for why 

net present value could not be used.34 NGA officials acknowledged that 

they did not compare the alternatives using net present value. They 

stated that they had normalized some of the costs but that it was not 

necessary to normalize all costs, because the estimates were all done 

during the same time period. According to our best practice, if net present 

value is not used to compare the alternatives, then the AOA team should 

document the reason why and explain and describe the other methods 

applied. Additionally, comparing items that have been discounted or 

normalized with net present value allows for time series comparisons, 

since alternatives may have different life cycles or different costs and 

benefits.35

NGA’s AOA process for selecting the site for the new NGA West campus 

partially met the credible characteristic for an agency’s AOA process. 

Although NGA’s AOA process had strengths, it also had limitations, such 

as lacking important information related to cost risks and sensitivity 

analyses for both cost and benefits identified.

NGA’s AOA body of work described the alternatives in sufficient detail to 

allow for robust analysis. Specifically, it provided descriptions of each of 

the alternatives at varying levels of detail. For example, the first site 

location study provided descriptions of the top 6 potential sites, including 

information on size, the sites’ strengths and weaknesses, and any 

acquisition or development issues. The NGA AOA body of work also 

provided evidence that site master planning was conducted to provide 

additional details on the physical and environmental attributes of each 

site, as well as constraints and benefits. For example, the NGA West 

Prospective Sites Master Plan described the Mehlville site as having 

landscape features such as mature trees, waterways, areas of steep 

 34Net present value is a financial calculation that accounts for the time value of money by 

determining the present value of future savings minus up-front investment costs over a 

specific period of time.

35DOD Instruction 7041.03 on economic analysis directs that, when comparing 

alternatives, the costs and benefits should be compared and ranked according to net 

present value, and the instruction provides a specific formula to calculate it.

Site Selection Process 

Only Partially Met the 

Characteristic for a 

Credible AOA Because It 

Lacked Important 

Information on Cost Risks 

and Sensitivities
Page 26 GAO-17-643 Intelligence Community

topography, options for public transportation, bike-friendly streets, and 

existing utility infrastructure.

However, NGA did not fully include key information on either the risk or 

the uncertainty related to cost estimates or the sensitivity to the costs and 

benefits identified as part of its AOA process. For example, the NGA body 

of work did not include a confidence interval or range for the cost 

estimates for each viable alternative in order to document the level of risk 

associated with the estimate. NGA’s AOA body of work documented the 

estimated alternatives’ initial costs and included contingency costs across 

all four alternatives. Corps of Engineers officials told us that they had 

developed a 30 percent design and 5 percent construction contingency 

cost factor across the four alternatives to account for cost risks in those 

areas. However, the NGA AOA body of work did not provide evidence of 

a confidence interval or range for the costs provided. NGA acknowledged 

that while its AOA body of work did not identify the risk associated with 

specific cost elements for each alternative, it did provide a “level of 

confidence,” because the methodology behind the cost components in the 

estimate implied a high level of confidence. Although we agree that NGA 

did provide a contingency factor for the site development costs and 

provided cost estimates for all four viable alternatives, NGA did not 

develop a confidence interval or risk range for those estimates. NGA’s 

cost estimates were used as a determining factor in the final decision 

among the four alternatives. However, without understanding the cost risk 

and the uncertainty of those costs as outlined in the best practice, a 

decision maker might be unable to make an informed decision.

Additionally, the NGA AOA body of work did not demonstrate that NGA 

had conducted a sensitivity analysis for the cost and benefit and 

effectiveness estimates for each alternative in order to examine how 

changes in key assumptions would affect the cost and benefit estimates. 

The NGA AOA body of work documented that some sensitivity analysis or 

level of risk was analyzed as part of the schedule analysis, and NGA 

officials stated that the project considered how different values and 

variables affect each other during the criteria and evaluation analysis. 

However, the NGA AOA body of work did not document the sensitivity of 

cost and benefit estimates to changes in key assumptions for each 

alternative, and a sensitivity analysis was not applied to the initial cost 

estimates or benefit assumptions that were used to make the final site 

selection.

NGA officials stated that this best practice has limited application to its 

AOA process, because the lack of variables between sites constrained 
Page 27 GAO-17-643 Intelligence Community

their ability to develop full life-cycle cost estimates and complete a 

sensitivity analysis. NGA officials stated that their sensitivity analysis was 

limited to those considerations that were measurable and sensitive to 

change—predominantly schedule risk associated with land acquisition 

activities. Further, NGA officials explained that because all the site 

alternatives were located within the St. Louis metropolitan area, any 

variations in conditions would have equal effect. Although we agree that 

NGA did conduct a sensitivity analysis for schedule risks, NGA neither 

documented how the schedule sensitivity affected its cost or benefit 

estimates nor performed a sensitivity analysis for the various assumptions 

used to develop the cost or benefit for each alternative. According to the 

DOD instruction on economic analysis, a sensitivity analysis is a “what-if”

exercise that should be performed to test the conclusions and 

assumptions of changes in cost and benefit variables and should always 

be performed when the results of the economic analysis do not clearly 

favor any one alternative. 36 According to our best practice, not 

conducting a sensitivity analysis to identify the uncertainties associated 

with different assumptions increases the chances that an AOA team will 

recommend an alternative without understanding the full effects of costs, 

which could lead to cost and schedule overruns.

Although NGA’s AOA process did not reflect all of the characteristics of a 

high-quality process, we are not making recommendations in this report, 

in part because NGA plans to conduct additional cost analysis and in part 

because we made an applicable recommendation to DOD in 2016. 

Specifically, although NGA’s AOA process is complete, NGA and Corps 

of Engineers officials said that they are developing full life-cycle cost 

estimates for the construction and design of the new NGA West campus 

and that these estimates will include many elements from our best 

practices. Further, we continue to believe that our September 2016 

recommendation that DOD develop guidance requiring the use of AOA 

best practices for certain military construction decisions could help ensure 

that future AOA processes conducted by DOD agencies like NGA are 

reliable and that agencies identify a preferred alternative that best meets 

mission needs.37 While DOD did not concur with our recommendation, as 

we reported in 2016, our best practices are based on longstanding, 

fundamental tenets of sound decision making and economic analysis. 

Additionally, our best practices align with many DOD and military policies, 

 36DOD Instruction 7041.03 (Sep. 9, 2015).

37GAO-16-853. 
Page 28 GAO-17-643 Intelligence Community

directives, and other guidance pertaining to military construction. Further, 

during this review NGA officials stated that DOD did not have a set of 

best practices for conducting an AOA to help NGA make decisions 

regarding its military construction project, and that our AOA best practices 

would have been helpful had they been published prior to the start of 

NGA’s site selection process in 2012. Accordingly, we continue to believe 

our prior recommendation is relevant and that unless DOD has guidance 

directing that certain military construction AOA processes be conducted in 

accordance with identified best practices, it may not be providing 

Congress with complete information to inform its oversight of DOD’s 

future military construction decisions.

We provided a draft of this report of this report to NGA for review and 

comment. NGA’s comments are reprinted in their entirety in appendix II. 

In comments on our report, NGA stated that it valued our assessment of 

its AOA process, which we judged to have substantially met the 

characteristics of a well-documented, comprehensive, and unbiased 

process, and would use our findings to continue to refine and improve its 

corporate decision making and processes. 

NGA raised a concern about our assessment that its AOA process used 

to select the site for its new NGA West project partially met the best 

practices that demonstrate a credible process. NGA’s specific concern 

was that we concluded that the AOA process did not fully include 

information on risks and sensitivities to cost estimates. In its letter, NGA 

stated that its analysis demonstrated that cost was a factor but not the 

most important factor. Moreover, NGA stated that cost elements and 

details ranged from well-defined costs, such as real estate costs, to 

estimates based on analogy such as an information technology trunk line. 

NGA additionally stated that, due to the conceptual nature of the design 

of the facility at that time, more detailed cost analysis was judged to 

provide no discrimination among alternatives and were thus purposely 

excluded from the initial cost estimates that were used in the AOA 

process. While NGA may have concluded that the project’s cost was not 

the most important factor, the agency estimates that construction of the 

campus will cost about $945 million and NGA used the cost estimate as a 

determining factor to select from the four final alternatives. Moreover, our 

assessment of the credibility characteristic is based only in part on NGA’s 

initial cost estimates and did not penalize NGA for excluding additional 

cost estimates. Rather, we assessed that NGA’s AOA body of work did

not provide evidence of documenting the sensitivity of the cost-benefit or 

Agency Comments 

and Our Evaluation
Page 29 GAO-17-643 Intelligence Community

effectiveness estimates to changes in key assumptions for alternatives, 

nor was a sensitivity or risk analysis applied to the initial cost estimates 

used to make the final site selection.

NGA also stated in its letter that our AOA best practices are not 

applicable in all circumstances, and pointed out that DOD did not concur 

with a recommendation in a prior report38 to develop AOA guidance 

requiring departmental components to use AOA practices, including the 

best practices we identified, for certain future military construction 

projects. Our prior report suggested that such guidance might only apply 

to certain military construction projects as determined by DOD. In 

addition, while DOD’s existing relevant guidance does not require use of 

our AOA best practices, the guidance does not prohibit it either. Further, 

as discussed in our report, NGA officials told us the AOA best practices 

are helpful to such processes, and lacking such DOD guidance NGA had 

to draw on expertise, practices, and procedures from a variety of sources 

to conduct its AOA for the new NGA West site.

Finally, in its letter NGA proposed that two documents—the 

environmental impact statement and record of decision—fulfill the best 

practice to document the AOA process in a single document. Specifically, 

NGA stated that within the context of the environmental impact statement 

process, the record of decision is the authoritative capstone document of 

the process, and that together the two documents include discussions of 

the decision-making and factors considered by the director in selecting 

the agency-preferred alternative. These two documents were prepared to 

fulfill requirements of the National Environmental Policy Act of 1969 in 

order to determine the environmental impacts of the project, as discussed 

earlier in our report. While we recognize that the record of decision and 

the environmental impact statement are significant documents that 

include summaries of aspects of NGA’s AOA process, as NGA indicated 

these are two documents within an expansive AOA body of work. Further, 

many of the elements of NGA’s AOA process are diffused throughout 

these and several other reports and documents—that were specifically 

identified by NGA as the key documentation of its AOA process—rather 

than clearly delineated in a single document as prescribed by the best 

practice (see appendix I). 

 38GAO-16-853
Page 30 GAO-17-643 Intelligence Community

As agreed with your office, unless you publicly announce the contents of 

this report earlier, we plan no further distribution until 30 days from the 

report date. At that time, we will send copies of this report to the 

appropriate congressional committees; the Secretary of Defense; the 

Secretary of the Air Force; the Secretary of the Army; the Under 

Secretary of Defense for Acquisitions, Technology and Logistics; the 

Under Secretary of Defense for Intelligence; and the Director, National 

Geospatial-Intelligence Agency. In addition, the report is available at no 

charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact 

me at (202) 512-4523 or leporeb@gao.gov. Contact points for our Offices 

of Congressional Relations and Public Affairs may be found on the last 

page of this report. GAO staff who made key contributions to this report 

are listed in appendix III.

Sincerely yours,

Brian J. Lepore 

Director, 

Defense Capabilities and Management
Appendix I: Analysis of Alternatives (AOA) 22 

Best Practices with GAO’s Evaluation of 

National Geospatial-Intelligence Agency (NGA) 

AOA Process and Scores

Page 31 GAO-17-643 Intelligence Community

In our earlier discussion of the extent to which NGA’s AOA process met 

best practices for such processes, we presented our analysis for specific 

best practices. These 22 best practices and their definitions were 

originally published and are listed in GAO-16-22.

1 Table 3 summarizes 

our analysis of NGA’s AOA process for selecting the site for the new NGA 

West and our ratings of that process against all 22 best practices.

Table 3: Evaluation of the National Geospatial-Intelligence Agency (NGA) Analysis of Alternatives (AOA) Process against 

GAO’s Best Practices

Best Practices for an AOA Process NGA’s AOA Process

1. Define mission need

Score 5 – Fully Met

Defines mission need in various documents including the two site location studies, the 

Project Management Office (PMO) site evaluation report (PMO Report), and the NGA 

Director’s memo identifying the agency-preferred alternative. For example, both site 

location studies identify the mission need as “to enhance current missions, improve 

resiliency and solve the numerous challenges associated with its current 2nd street 

facilities, NGA is pursuing a new facility in the St. Louis metropolitan area.”

2. Define functional requirements

Score 5 – Fully Met

Provides descriptions of functional (facilities and program) requirements that are realistic, 

organized, clear, and based on mission needs. For example, the first site location study 

identified several physical requirements for the new NGA West campus (e.g. 800,000 

gross square feet, 500-foot security buffer, and 35 percent allowable growth). The first 

site location study also expanded the list to create site minimum requirements which 

established firm criteria to eliminate sites from further consideration. Additionally, the 

PMO Report provides functional requirements to fulfill NGA’s mission and identifies gaps 

between NGA’s current NGA West facilities and its mission-based functional 

requirements.

3. Develop AOA time frame

Score 4 – Substantially Met

Provides evidence of a schedule that was developed prior to starting the site evaluation 

process to select the agency-preferred alternative for NGA West. Several documents 

within the body of work outline that time frame. Specifically, a briefing provided by the 

PMO on the site evaluation process provides significant milestone dates for the 

completion of the site evaluation process, including dates for the evaluation teams’ and 

PMO’s analyses, the National Environmental Policy Act analysis, and the agencypreferred alternative decision. Although the NGA body of work does provide a time line 

for the significant milestones associated with the site evaluation process, the time line 

established during the site evaluation process does not include other key processes such 

as the site location studies, the master planning efforts, or the site selection team 

evaluation. 

 1

GAO, Amphibious Combat Vehicle: Some Acquisition Activities Demonstrate Best 

Practices; Attainment of Amphibious Capability to Be Determined, GAO-16-22 

(Washington, D.C.: Oct. 28, 2015). 

Appendix I: Analysis of Alternatives (AOA) 22 

Best Practices with GAO’s Evaluation of 

National Geospatial-Intelligence Agency 

(NGA) AOA Process and Scores
Appendix I: Analysis of Alternatives (AOA) 22 

Best Practices with GAO’s Evaluation of 

National Geospatial-Intelligence Agency (NGA) 

AOA Process and Scores

Page 32 GAO-17-643 Intelligence Community

Best Practices for an AOA Process NGA’s AOA Process

4. Establish AOA team

Score 5 – Fully Met

Demonstrates evidence of an AOA team with members that are from diverse parts of the 

agency and includes a variety of subject matter experts. For example, the PMO Report 

lays out the overall composition of the personnel involved in the site evaluation process, 

including PMO staff and advisors who are subject matter experts and who provided 

guidance on topics such as legal matters, financial management, human development, 

contracting, and communications. The PMO’s site evaluation process brief also identifies 

the specific makeup of the site evaluation team, which included the mission evaluation 

team, security, infrastructure, and schedule evaluation team, advisors, the National 

Environmental Policy Act documentation team, and the master planning team. 

Additionally, NGA established a site selection team to comprehensively review the 

summaries of all previous site evaluation efforts and reports and develop a path to 

selecting the agency’s preferred alternative. The site selection team consisted of 

representatives from NGA, U.S. Army Corps of Engineers (Corps of Engineers), Air 

Force, and contractor support staff.

5. Define selection criteria

Score 4 – Substantially Met

Identifies and defines selection criteria in numerous documents at various stages of the 

AOA process—such as physical requirements developed by contractors in 2013, 

evaluation factors developed by the PMO in 2015, evaluation sub-factors developed 

during the evaluation teams’ analyses in 2015, strategic effect criteria evaluated by the 

PMO in 2015, and refining criteria developed by the site selection team in consultation 

with the NGA Director in 2016.

In most instances, the criteria were defined before the analysis was done, for example, in 

the case of the physical requirements in the first site location study and the evaluation 

factors in the PMO Report. In one instance, mission criteria used by the site selection 

team to differentiate between the final two alternatives were adjusted so that they were 

defined slightly differently from the criteria that were used in previous analyses, giving the 

appearance that NGA’s criteria had been adjusted after overall analysis began. However, 

NGA officials stated that they used consistent selection criteria in their AOA process and 

that their core criteria did not change during the phases of analyses but were clarified as 

the process matured.

6. Weight selection criteria

Score 3 – Partially Met

Identifies when certain criteria were ranked or weighted at various stages of the process 

and generally demonstrated how this ranking affected the overall scores. For example, 

the mission evaluation team’s workshop results in the PMO Report document how the 

team ranked its mission sub-factors by order of precedence. Specifically, the team 

considered each sub-factor’s relative importance by assessing it according to a high, 

medium, or low order of precedence and then evaluated it from both the near- and farterm perspectives. In one instance, the team ranked its mission sub-factor “Low Crime”

as a high order of precedence and “Proximity to Amenities” as a low order of precedence.

However, the NGA body of work included instances in which criteria were ranked or 

weighted after the analysis began. For example, as documented in the NGA Director’s 

memorandum identifying the agency-preferred alternative, the director adjusted the 

weight of a few criteria toward the end of the AOA process. Also, the evaluation teams, 

the PMO, and the NGA Director used different weights over the course of the evaluation 

process, and there is little justification provided in the body of work behind the ranking of 

the various criteria.

While the iterative nature of NGA’s AOA process allowed for a more thorough review, it 

also resulted in the criteria and weighting being adjusted as the process unfolded, 

possibly allowing for bias to enter the process. 
Appendix I: Analysis of Alternatives (AOA) 22 

Best Practices with GAO’s Evaluation of 

National Geospatial-Intelligence Agency (NGA) 

AOA Process and Scores

Page 33 GAO-17-643 Intelligence Community

Best Practices for an AOA Process NGA’s AOA Process

7. Develop AOA process plan

Score 5 – Fully Met

Includes documents that lay out specific plans and protocols for the site evaluation 

process and address key processes. For example, the PMO Report provides a 

description of the protocol established to complete the site evaluation process. 

Specifically, the PMO Report establishes roles and responsibilities for participants, sets 

key milestones and objectives, establishes evaluation teams and processes, and 

describes the format and content of the required reports. Further, the PMO Report 

includes a site selection methodologies brief that demonstrates that NGA researched and 

relied on methodologies established within the federal government for conducting site 

selections and based its site evaluation process on the U.S. General Service 

Administration’s site selection process.

8. Develop list of alternatives

Score 5 – Fully Met

Documents that NGA considered a diverse range of alternatives and conducted market 

research and surveillance in order to ensure a diverse list of alternatives. For example, 

the first site selection study provides a summary of the thorough analysis taken to identify 

potential site locations for NGA West. This study was conducted by real estate firm using 

“local market databases, mass and selective communications with the local real estate 

community, and interactions with municipal and economic development staff, including 

the U.S. General Services Administration.” This process included narrowing the 

alternative site locations from 186 sites to 6 sites for further analysis. Next, the Corps of 

Engineers conducted a second site location study, identifying additional sites for analysis, 

which ultimately validated the original six sites for further study. During the preliminary 

master planning, four of the six sites identified by the site location studies were 

determined suitable for further analysis to select the agency’s preferred alternative.

9. Describe alternatives

Score 5 – Fully Met

Provides comprehensive descriptions of each of the alternatives at varying levels of 

detail:

1. The first site location study provides descriptions of the top six potential sites, 

including information on the overall size, strengths and weakness, any acquisition or 

development issues, an overall grade, and a conclusion.

2. The Prospective Sites Master Plan lists a detailed analysis of the final four potential 

sites and includes a discussion of the existing buildings’ height, landscape and 

zoning assessments, and aerial, utility, and environmental constraints, among other 

things.

3. The second site location study lists details on additional sites and includes 

information on the advantages and disadvantages.

NGA officials stated the NGA body of work also provided an analysis of cost and noncost data for each potential site alternative, and that the information presented reflected 

the best available assessment of defined benefits in relation to associated costs.
Appendix I: Analysis of Alternatives (AOA) 22 

Best Practices with GAO’s Evaluation of 

National Geospatial-Intelligence Agency (NGA) 

AOA Process and Scores

Page 34 GAO-17-643 Intelligence Community

Best Practices for an AOA Process NGA’s AOA Process

10. Include baseline alternative

Score 3 – Partially Met

Discusses that the status quo, which represents the existing capability’s baseline where 

no action is taken, was evaluated prior to the site selection AOA during NGA’s decisionto-build process in 2010–2012. However, NGA officials stated that once the decision to 

build a new NGA West campus was finalized, NGA did not reevaluate the baseline 

alternative, and the baseline was not documented as an alternative in part of the NGA 

body of work to compare to the final site alternatives in the site evaluation process.

Additionally, as required by the National Environmental Policy Act of 1969 and 

implementing regulations, a “no action alternative” was included in the environmental 

impact statement analysis, which was considered as part of the PMO site evaluation 

process. This analysis compared and evaluated the alternative of taking no action to the 

other alternatives. However, the “no action alternative” serves as the baseline for the 

comparison of environmental and related impacts only.

NGA officials stated that the need for carrying the “status quo” alternative further into their 

AOA process was not necessary given the previous decision to build a new site. 

However, our best practice states that if the baseline alternative is not used to compare 

the alternatives then there is no benchmark for comparison; which can allow for arbitrary 

alternative comparisons. 

11. Assess alternatives’ viability

Score 4 – Substantially Met

Provides evidence of an assessment for each alternative’s viability using predetermined 

factors, including a screening of the alternatives against minimum requirements and 

operational factors as well as assessing viability after the passage of time. Specifically, 

NGA assessed the alternatives’ viability against qualitative technical and operational 

factors at different levels of the site selection process, including the two site location 

studies and preliminary master planning. For example, the first site location study 

narrowed down the full list of sites from 186 to 22 to guide NGA in its analysis of only 

those sites that were available and fully met requirements, including size and cost 

considerations. Additionally, notional master planning was conducted on the final 4 sites 

to ensure that all sites met more detailed specifications.

The NGA body of work also provides documentation of the elimination of nonviable 

alternatives. For example, the second site location study observed that, through 

preliminary master planning, two of the final six sites had become nonviable (NorthPark 

and Weldon Spring) and been eliminated from further analysis. However, according to 

NGA and Corps of Engineers officials, the Mehlville site, one of the final four site 

alternatives, had existing tenants and buildings, but there was no plan for eviction and 

occupation by NGA if the site were to be chosen as the agency-preferred alternative. It is 

therefore unclear whether this site was truly viable. 

12. Identify significant risks and mitigation 

strategies

Score 3 – Partially Met

Provides documentation on the identification and evaluation of some risks and mitigation 

strategies for each alternative at varying stages of the process. For example, the PMO 

Report identifies schedule and negotiation risks that were analyzed for each potential 

site, along with scores for each, and it also includes some estimated mitigation 

requirements. Additionally, the first site location study includes a discussion of risks 

associated with cost, procurement, seismic, and geotechnical/environment factors as part 

of a criterion for the evaluation of viable alternatives,

However, the body of work does not provide evidence that risks were ranked, that risks 

and strategies were documented for each alternative, or that the seven specific types of 

risks identified in this best practice were discussed. Not documenting the risks and the 

associated mitigation strategies for each alternative prevents decision makers from 

performing a meaningful trade-off analysis necessary to choose a recommended 

alternative.
Appendix I: Analysis of Alternatives (AOA) 22 

Best Practices with GAO’s Evaluation of 

National Geospatial-Intelligence Agency (NGA) 

AOA Process and Scores

Page 35 GAO-17-643 Intelligence Community

Best Practices for an AOA Process NGA’s AOA Process

13. Determine and quantify

benefits/effectiveness

Score 3 – Partially Met

Includes discussion of the benefits and effectiveness of each alternative. For example, 

the PMO Report demonstrates that the evaluation teams used a standard process to 

evaluate each alternative and provides a score against various evaluation factors and 

sub-factors. Next, the PMO evaluated the evaluation teams’ scores with different data or 

measures, including applying a strategic effect that placed precedence on sub-factors 

that had higher value and broader impact for NGA’s strategic goals. The PMO then 

scored and weighted the alternatives using some of the scores from the evaluation teams 

with its own level of precedence.

However, NGA did not quantify all benefits/effectiveness of each alternative over the 

alternative’s full life cycle or provide an explanation of why it was not possible to quantify 

benefits. Additionally, the various scalability assessments used by the PMO and the 

evaluation teams led to varying results that are not clearly traceable. Examining 

effectiveness over the life cycle, and determining a standard process to quantify benefits 

are essential to ensuring that biases are not introduced and that decision makers have a 

complete picture.

14. Tie benefits/effectiveness to mission 

need

Score 4 – Substantially Met 

Provides evidence that the benefits of the alternatives were evaluated against NGA’s 

mission needs. For example, the results of the mission evaluation team’s workshop 

provided in the PMO Report demonstrate a filtering process to validate the evaluation 

criteria according to NGA’s mission. The report also determined whether the missionbased criteria were “an enhancer or detractor” to NGA’s strategic goals and identified the 

advantages of the mission-based criteria sub-factors. In another example, the site 

selection team’s memorandum documenting its analysis demonstrates that its criteria are 

linked to NGA’s mission and that the team evaluated the final two sites’ adherence to 

criteria related to NGA’s strategic goal. However, although NGA’s AOA body of work 

provides evidence of evaluating NGA’s current environment against its desired 

environment, this is not done consistently for all criteria throughout the analysis.

15. Develop life-cycle cost estimates

Score 3 – Partially Met

Identifies and compares some land acquisition, site development, transportation 

improvements, and information technology costs for each alternative. Specifically, the 

NGA PMO Report includes Corps of Engineers cost estimates that provide evidence of 

(1) using a common work breakdown structure to compare the four alternatives, (2) 

documenting overarching/general ground rules and assumptions as well as specific 

ground rules and assumptions that were alternative specific, and (3) providing a 

comparison of the costs by alternative that showed the breakout of different cost 

categories for each site.

NGA and the Corps of Engineers referred to these as “initial costs” and confirmed that 

they did not complete full life-cycle cost estimates for their potential site alternatives. NGA 

officials said they chose not to analyze longer term costs for each of the four alternatives, 

because they assumed similar cost rates, since the sites are in the same geographic 

area. Officials added that the lack of final design details constrained their ability to 

develop full life-cycle cost estimates. NGA officials stated that, as a result, they focused 

on the costs they identified as being the most variable, which were the land acquisition, 

site development, transportation improvements, and information technology trunk line 

costs. However, the information technology trunk line costs lacked details and sufficient 

information regarding how the figures had been developed and what data they were 

based on. Without full life-cycle costs, decision makers may not have a complete picture 

of the costs for each alternative and may have difficulty comparing the alternatives, 

because comparisons may not be based on accurate information.
Appendix I: Analysis of Alternatives (AOA) 22 

Best Practices with GAO’s Evaluation of 

National Geospatial-Intelligence Agency (NGA) 

AOA Process and Scores

Page 36 GAO-17-643 Intelligence Community

Best Practices for an AOA Process NGA’s AOA Process

16. Include a confidence interval or range 

for life-cycle cost estimates

Score 2 – Minimally Met

Documents the estimated alternatives’ initial costs, but these estimates do not provide 

evidence that a confidence interval or range for the estimates was developed for each 

alternative. In the initial site development estimates, the Corps of Engineers did include a 

contingency factor for each estimate to identify some risks. Specifically, Corps of 

Engineers officials said that they included the same contingency factor for all four 

alternatives for certain costs based on their own expertise. However, the officials could 

not provide further details regarding how the contingency factors were identified. There 

was no indication of contingency factors or other analyses for the other costs developed 

as part of the NGA AOA process.

For decision makers to make an informed decision, cost estimates must reflect the 

degree of uncertainty or include a range of costs to convey a level of confidence for each 

alternative to achieve a most likely cost.

17. Perform sensitivity analysis

Score 2 – Minimally Met

Does not provide evidence of documenting the sensitivity of the cost-benefit or 

effectiveness estimates for each alternative to risks and changes in key assumptions. 

NGA officials stated that the project had considered how different values and variables 

affect each other during the criteria and evaluation analysis. The NGA body of work also 

documents that some sensitivity analysis or level of risk was analyzed as part of the 

schedule and negotiation analysis, but similar sensitivity analysis was not applied to show 

how schedule sensitivity affected the initial cost estimate or how the cost or benefit 

assumptions that were used in selecting the final site were affected by changes to their 

cost drivers.

NGA officials stated that this best practice has limited application to their AOA process, in 

part because their sensitivity analysis was limited to the considerations that were 

measurable and sensitive to change. While other risk analyses were conducted, such as 

environmental risk analysis, those analyses were not related to the cost or benefit 

estimates, which is the focus of this best practice. Failure to conduct a sensitivity analysis 

to identify the uncertainties associated with different cost and benefit assumptions 

increases the chance the AOA team will recommend an alternative without understanding 

the full effects of costs, which could lead to cost and schedule overruns.

18. Document AOA process in a single 

document

Score 3 – Partially Met

Demonstrates that a series of reports was used to document the entire NGA site 

evaluation process. According to NGA officials, this is due to the analyses being 

conducted over 5 years and documented in various reports as a result of the many 

phases and complexities of the AOA process. NGA officials pointed to four reports that 

illustrate the bulk of the analyses of NGA’s AOA process. These four reports document 

the steps taken to (1) initialize the AOA process, (2) identify, analyze, and select the 

alternatives including details on the criteria used, (3) provide assumption for each 

alternative, (4) identify risks drivers and mitigation techniques, and (5) determine costs 

and schedule risks for each alternative. Although these reports document NGA’s entire 

AOA process, including the final decision by the NGA director, many of the elements of a 

clearly documented AOA process are diffused throughout the various documents rather 

than clearly delineated in a single document. As stated in our best practice, a clear report 

should be compiled in one document to allow for an independent reviewer to assess the 

AOA process.
Appendix I: Analysis of Alternatives (AOA) 22 

Best Practices with GAO’s Evaluation of 

National Geospatial-Intelligence Agency (NGA) 

AOA Process and Scores

Page 37 GAO-17-643 Intelligence Community

Best Practices for an AOA Process NGA’s AOA Process

19. Document assumptions and 

constraints

Score 4 – Substantially Met

Supports that assumptions and constraints were developed and documented. For 

example, the first site location study lists various site location assumptions for all 

alternative locations, such as “only sites that were explicitly available for acquisition were 

included in the evaluation process.” Additionally, the Prospective Sites Master Plan

developed a set of overall assumptions to guide the preliminary planning process and 

identified specific assumptions and constraints for each alternative. Specifically, the plan 

identifies various assumptions and constraints for the four final sites, such as calculations 

of the site boundaries, square footage of buildings, and parking estimates. However, 

although the NGA body of work provides evidence that assumptions and constraints were 

provided for each alternative, justifications for the assumptions and constraints were not 

detailed within the analyses.

20. Ensure AOA process is impartial

Score 3 – Partially Met

Illustrates the intention of impartiality, including (1) hiring an external firm in the first site 

location study to conduct a real estate market analysis based on minimum and initial 

requirements, (2) conducting a second site location study to ensure that no available 

options were overlooked by the omission of public advertisement, (3) including alternative 

designs in the Prospective Sites Master Plan to ensure that there was no predetermined 

design assumption per site, and (4) requesting subject matter experts to conduct an 

independent review of the site evaluation process.

However, the NGA body of work demonstrates that different scoring processes and 

methodologies to rank potential sites existed, such as the different scoring with the 

PMO’s analysis and the site selection team’s evaluation. Such differences can add bias 

to the overall AOA process and its individual reports. The validity of the analysis is 

affected if biases are introduced to the inputs.

21. Perform independent review

Score 3 – Partially Met

Documents external and internal reviews conducted by subject matter experts outside the 

team conducting the AOA. These reviews provide an independent critique of the site 

evaluation criteria and process. Specifically, the PMO Report identifies an external review 

of the site evaluation process by representatives from the Office of the Director of 

National Intelligence, Defense Intelligence Agency, and National Security Agency. 

Additionally, the PMO Report documents an independent internal review by NGA 

personnel independent of the AOA team.

The two groups in the internal review were comprised of experts from selected NGA 

components—one group comprised of senior leaders from Security and Installations and 

the other of senior leaders from Human Development and Diversity Management and 

others. The report stated that the comments received from these groups were key to 

validating the site evaluation process.

However, these reviews did not assess the AOA process as a whole, including the site 

location studies and the site selection team evaluation, and it is unclear whether or how 

NGA addressed the recommendations from the independent reviews. It is recommended 

that experts outside the AOA process review the process to ensure that it is free of 

organizational bias and is sufficiently thorough.
Appendix I: Analysis of Alternatives (AOA) 22 

Best Practices with GAO’s Evaluation of 

National Geospatial-Intelligence Agency (NGA) 

AOA Process and Scores

Page 38 GAO-17-643 Intelligence Community

Best Practices for an AOA Process NGA’s AOA Process

22. Compare Alternatives

Score 2 – Minimally Met

Documents various analyses that compared the alternatives from different perspectives 

with varying criteria over time. Specifically, the alternatives were compared and scored in

a way that allows a decision maker to compare the alternatives at varying levels of detail. 

However, although the NGA body of work demonstrates comparison of the analyses 

throughout the AOA process, it does not provide evidence that the AOA team used net 

present value to compare alternatives—a key element of this best practice. NGA officials 

acknowledged that they did not compare the alternatives using net present value and 

stated that they believed it was not necessary for their review. Our best practice notes 

that an agency must document why it cannot use net present value. NGA’s AOA body of 

work did not document why net present value could not be used, fully describe an 

alternate method that was used to differentiate between alternatives, or explain why that 

method had been applied. Furthermore, comparing alternatives that have not been 

discounted or normalized does not allow for time series comparisons, since alternatives 

may have different life cycles or different costs and benefits.

Source: GAO analysis of NGA Information | GAO-17-643
Appendix II: Comments from the National 

Geospatial-Intelligence Agency

Page 39 GAO-17-643 Intelligence Community

Appendix II: Comments from the National 

Geospatial-Intelligence Agency
Appendix II: Comments from the National 

Geospatial-Intelligence Agency

Page 40 GAO-17-643 Intelligence Community
Appendix III: GAO Contact and Staff 

Acknowledgments

Page 41 GAO-17-643 Intelligence Community

Brian Lepore, (202) 512-4523 or leporeb@gao.gov

In addition to the contact named above, Brian Mazanec, Assistant 

Director; Jim Ashley; Tracy Barnes; Chris Businsky; George Depaoli; 

Richard Johnson; Joanne Landesman; Jennifer Leotta; Jamilah Moon; 

Joseph Thompson; and Sally Williamson made key contributions to this 

report.

Appendix III: GAO Contact and Staff 

Acknowledgments

GAO Contact

Staff 

Acknowledgments
Amphibious Combat Vehicle Acquisition: Cost Estimate Meets Best 

Practices, but Concurrency between Testing and Production Increases 

Risk. GAO-17-402. Washington, D.C.: Apr. 18, 2017.

Joint Intelligence Analysis Complex: DOD Needs to Fully Incorporate 

Best Practices into Future Cost Estimates. GAO-17-29. Washington, 

D.C.: Nov. 3, 2016.

Joint Intelligence Analysis Complex: DOD Partially Used Best Practices 

for Analyzing Alternatives and Should Do So Fully for Future Military 

Construction Decisions. GAO-16-853. Washington, D.C.: Sept. 30, 2016.

Patriot Modernization: Oversight Mechanism Needed to Track Progress 

and Provide Accountability. GAO-16-488. Washington, D.C.: Aug. 25, 

2016.

Amphibious Combat Vehicle: Some Acquisition Activities Demonstrate 

Best Practices; Attainment of Amphibious Capability to Be Determined. 

GAO-16-22. Washington, D.C.: Oct. 28, 2015.

DOE and NNSA Project Management: Analysis of Alternatives Could Be 

Improved by Incorporating Best Practices. GAO-15-37. Washington, D.C.: 

Dec. 11, 2014.

Military Bases: DOD Has Processes to Comply with Statutory 

Requirements for Closing or Realigning Installations. GAO-13-645. June

27, 2013.

Military Bases: Opportunities Exist to Improve Future Base Realignment 

and Closure Rounds. GAO-13-149. Washington, D.C.: Mar. 7, 2013.

Military Base Realignments and Closures: The National GeospatialIntelligence Agency’s Technology Center Construction Project. 

GAO-12-770R. Washington, D.C.: June 29, 2012.

Military Base Realignments and Closures: DOD Is Taking Steps to 

Mitigate Challenges but It Is Not Fully Reporting Some Additional Costs. 

GAO-10-725R. Washington, D.C.: June 24, 2010.

GAO Cost Estimating and Assessment Guide: Best Practices for 

Developing and Managing Capital Program Costs. GAO-09-3SP. 

Washington, D.C., Mar. 2, 2009.

Related GAO Products

(101149)
The Government Accountability Office, the audit, evaluation, and investigative 

arm of Congress, exists to support Congress in meeting its constitutional 

responsibilities and to help improve the performance and accountability of the 

federal government for the American people. GAO examines the use of public 

funds; evaluates federal programs and policies; and provides analyses, 

recommendations, and other assistance to help Congress make informed 

oversight, policy, and funding decisions. GAO’s commitment to good government 

is reflected in its core values of accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no cost is 

through GAO’s website (http://www.gao.gov). Each weekday afternoon, GAO 

posts on its website newly released reports, testimony, and correspondence. To 

have GAO e-mail you a list of newly posted products, go to http://www.gao.gov

and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of production and 

distribution and depends on the number of pages in the publication and whether 

the publication is printed in color or black and white. Pricing and ordering 

information is posted on GAO’s website, http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537.

Orders may be paid for using American Express, Discover Card, MasterCard, 

Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, LinkedIn, Twitter, and YouTube.

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.

Visit GAO on the web at www.gao.gov and read The Watchblog.

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-4400, 

U.S. Government Accountability Office, 441 G Street NW, Room 7125, 

Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

James-Christian Blockwood, Managing Director, spel@gao.gov, (202) 512-4707

U.S. Government Accountability Office, 441 G Street NW, Room 7814, 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Strategic Planning and 

External Liaison

Please Print on Recycled Paper.