Intelligence Community: Analysis of Alternatives Approach for a New Site Reflects Most Characteristics of a High-Quality Process
Released: 2017-08-28T00:00:00

Letter

Background

Decision to Build a New NGA West

The existing NGA West campus consists of 15 facilities on 27 acres. Some of the buildings’ original construction dates back to the early 1800s, and 22 acres of the site are on the National Register of Historic Places, according to NGA documents. In 2009 through 2010, NGA contracted with an independent firm to assess the condition of the existing NGA West. This assessment gave the facilities an overall condition rating of “poor,” generally because of the insufficiency of the anti-terrorism and force protection measures, the average age of the structures, numerous code and accessibility shortfalls, and lack of seismic protection.

Near the end of the completion of the NGA East headquarters consolidation in 2011, NGA focused its attention on the need to improve the operational capacity, security requirements, and modernization of its NGA West facilities. From approximately 2009 through 2012, NGA conducted a series of evaluations to inform its efforts to modernize NGA West. These analyses included a condition assessment of the existing facilities; an economic analysis of alternatives to evaluate the options of building a new facility (“build new”), fully renovating the existing facilities (“modernize”), or remaining in the current facilities with minimum essential repairs (“status quo”); and a qualitative analysis of non-cost considerations for the build new, modernize, and status quo options identified in the economic analysis.

In 2012 NGA determined that a new NGA West would best meet the agency’s mission and resource needs. After examining the options of renovating its current facility, leasing, or building a new government- owned facility, NGA determined that building a new, government-owned facility was the preferred option. NGA officials stated that they are in the process of soliciting design-build proposals and that the final design-build contract is planned for award near the end of fiscal year 2018. Construction is expected to begin approximately in the summer of 2019.

Best Practices for an Analysis of Alternatives Process

We identified 22 best practices for an AOA process in October 2015, based on government and private-sector guidance and input from subject-matter experts. Many federal and industry guides have described approaches to analyses of alternatives; however, there was no single set of practices for the AOA process that was broadly recognized by both government and private-sector entities. We developed these best practices by (1) compiling and reviewing commonly mentioned AOA policies and guidance used by different government and private-sector entities and (2) incorporating experts’ comments on a draft set of practices to develop a final set of practices. The 22 best practices are grouped into four characteristics that describe a high-quality, reliable AOA process and can be used to evaluate whether an AOA process meets the characteristics of well-documented, comprehensive, unbiased, and credible. These practices can be applied to AOA processes for a broad range of capability areas, projects, and programs, including military construction projects and decision-making processes, in which an alternative must be selected from a set of possible options.

In September 2016, we recommended that DOD develop guidance that requires the use of AOA best practices, including those practices we identified, when conducting AOA processes for certain types of military construction decisions. DOD did not concur with this recommendation and disagreed that these best practices apply to military construction decision-making processes. We continue to believe that this recommendation is valid and that the principles demonstrated by the best practices—and the practices themselves—draw from related DOD and other practices.

Our best practices also parallel those found in DOD and Air Force guidance on military construction and analysis for decision making. For example, according to an Air Force instruction governing the planning and programming of military construction projects, one of the required planning actions is to evaluate alternative solutions. According to a DOD directive pertaining to military construction, DOD must monitor the execution of its military construction program to ensure—among other things—that the program is accomplished in the most cost-effective way. This guidance for cost effectiveness aligns with our AOA best practice Develop Life-cycle Cost Estimates, which focuses on providing decision makers with the information they need to assess the cost- effectiveness of alternatives. Further, DOD Instruction 7041.03, on economic analysis for decision making, contains numerous cost estimating principles and procedures that align with those called for in our AOA best practices. As we reported in 2016, these policy documents and instructions align with the general intent of our best practices, and there are many similarities between our best practices and the department’s guidance.

Additionally, in our previous work reviewing AOA process for other national security facilities, agencies generally concurred with our recommendations to consider including our best practices in future guidance. For example, in 2014 we assessed three National Nuclear Security Administration construction projects and found each project’s AOA partially met our best practices for conducting an AOA process. The Department of Energy agreed with our recommendation and has begun implementation.

NGA’s Site Selection Process Began in 2012, and Potential Sites Were Evaluated Based on Mission, Security, Environment, Cost, and Schedule Considerations

Site Selection Process Began in 2012 and Concluded with the June 2016 Record of Decision

NGA launched its search for a new NGA West site in 2012 with a site location study conducted by an outside real estate firm, and it concluded the search with the issuance of a record of decision in June 2016. The site location study included a check for existing federal sites that could accommodate NGA West’s workforce and mission. This search resulted in a total of 186 sites being identified initially as possible options; the list was narrowed to 6 sites in the St. Louis metropolitan area for further study. During preliminary master planning, 4 of the 6 sites identified by the site location studies were determined to be suitable for further analysis to select the agency’s preferred alternative. Three of these sites are in the Missouri cities of Fenton, Mehlville, and St. Louis, and one is in St. Clair County, Illinois, near Scott Air Force Base. See figure 1 for the geographic distribution of the 4 sites.

The subsequent site selection process included an environmental impact statement as required by the National Environmental Policy Act of 1969, analysis of NGA and the Corps of Engineers’ compliance with related DOD policies and other federal laws and requirements, preliminary master planning conducted by the Corps of Engineers, and a site evaluation process initiated by the NGA West Program Management Office (PMO), which is responsible for managing the NGA West project. To select the final site from the four alternatives, NGA initiated a site evaluation process in August 2015 that was led by the NGA West PMO. This process involved various teams of experts analyzing the sites and evaluating them against defined criteria to identify the advantages and disadvantages of each site.

Figure 2 provides an overview of the key elements and milestones of NGA’s site selection process, beginning with its earlier decision to build and concluding with its 2016 selection of the new site and issuance of its record of decision.

According to NGA officials, there was no NGA or DOD policy or set of practices to comprehensively guide NGA’s site selection and AOA process. As a result, NGA relied on various DOD policies and instructions, other federal guidance, and industry standards. It incorporated these practices into the site selection process to ensure that it complied with federal requirements and industry practice to develop its AOA process, according to NGA and Corps of Engineers officials. Additionally, NGA officials stated that our AOA best practices would have been helpful in planning the site selection process for NGA West, but the process began in 2012, and our 22 best practices were not published until October 2015.

Potential Sites Were Assessed Based on Key Factors, and NGA and the Corps of Engineers Conducted Additional Analysis in Order to Reach a Final Determination

Potential Sites Were Assessed Based on Mission, Security, Environment, Cost, and Schedule Factors

At the outset of the site evaluation process in August 2015, the PMO set forth broad sets of criteria to use in analyzing the four alternatives. These broad sets of criteria, referred to as “evaluation factors,” were mission, security, development and sustainability, schedule, cost, and environment. In addition, each site was assessed to ensure that it complied with key laws, regulations, and directives. The PMO divided the analysis of the evaluation factors among NGA and Corps of Engineers teams.

The mission, security, and development and sustainability factors were assigned to two NGA evaluation teams of subject-matter experts—the “mission evaluation team” and “security, infrastructure and schedule evaluation team” (referred to here as security evaluation team). Each of these teams used its expertise to develop “sub-factors” to assess the advantages and disadvantages of each site. For example:

The mission evaluation team developed 10 mission-related sub- factors based on the PMO guidance, NGA’s mission, and the strategic goals outlined in the 2015 NGA Strategy. The mission-related sub- factors focused largely on elements pertaining to NGA’s workforce and partnerships, such as the sites’ proximity to the existing workforce, their distance from NGA’s Arnold facility, and the likelihood that the sites would attract mission partners to create a “GEOINT Valley.”

The security evaluation team developed 13 sub-factors related to security and infrastructure based on PMO guidance, DOD and other federal security and energy requirements, threat analysis, and other subject-matter expertise. Examples of the sub-factors include a 500- foot setback, perimeter security elements, sustainable characteristics, and infrastructure resilience.

Separate evaluations of cost, schedule, and environmental considerations were conducted by the Corps of Engineers in its role as construction agent as part of the environmental impact analysis. In addition, NGA and the Corps of Engineers conducted an assessment of relevant laws and regulations.

The PMO integrated these analyses and provided an additional layer of review to each of the evaluation factors, in some cases adjusting them.

For instance, the PMO reorganized the 10 mission-related sub-factors for its review. Specifically, while the mission evaluation team focused the sub-factors largely on NGA’s strategic goals related to workforce and partnerships, the PMO’s analysis reorganized those same mission-related sub-factors by how they supported all four of the 2015 strategic goals. The PMO listed under three “strategic effects”—“Create GEOINT Valley,” “Enhance Operations,” and “Attract and Sustain the Workforce”—all of the sub-factors related to that strategic effect. The PMO re-analyzed the sites by weighting those strategic effects and sub-factors that were linked to multiple strategic goals higher than those that were linked to fewer such goals. The PMO also adjusted some of the sub-factors used in the evaluation for security and for development and sustainability.

The PMO’s additional analysis did not change the overall outcome of the evaluation of the sites; rather, it validated the mission evaluation team’s conclusion and generally supported all but one of the overall findings of the other analyses. At the conclusion of the PMO’s analysis in December 2015, the PMO’s conclusion was that no one site had emerged as a clear preferred alternative.

NGA Director Requested Additional Analysis in Order to Reach a Decision

Because the master planning and site evaluation process concluded that all four sites—Fenton, Mehlville, St. Louis City, and St. Clair—could meet the overall requirements and that no single site held substantial advantage over another, the NGA Director requested additional analysis with refined criteria to more clearly differentiate among the final four sites. Consequently, in January 2016 NGA initiated a new site selection team— consisting of NGA and Corps of Engineers personnel who had previously been involved in various stages of the process—to reassess the sites against refined criteria and perspectives in order to determine the agency- preferred alternative.

The site selection team carried forward five of the six original evaluation criteria from the start of the site evaluation process, as well as compliance with federal law, policy, and other regulations, to develop its six “refined criteria.” In reviewing these refined criteria, the site selection team determined that cost and schedule accounted for the greatest differences among the sites. The team therefore used the cost and schedule assessments completed as part of the PMO process to narrow the sites, concluding that because the Mehlville and Fenton sites were the most expensive and posed the greatest schedule risk they should be eliminated from final consideration.

The site selection team then focused its analysis on the final two sites— St. Clair and St. Louis City—to inform the Director’s selection. The team used the following six refined criteria to evaluate the sites: (1) cost, (2) schedule, (3) security, (4) mission efficiency and expansion, (5) applicability of and compliance with federal policies, executive orders, and federal initiatives; and (6) environmental considerations. The team proposed narrowing the relevant sub-criteria to those that provided the greatest differentiation among the sites, according to officials on the team. For example, the security criterion was narrowed to include 3 of the original 13 security and infrastructure evaluation sub-factors, and the adjusted “mission efficiency and expansion” criterion included one of the mission evaluation team’s 10 original mission sub-factors.

Subsequently, the NGA Director provided additional direction, including adding a review of potential support from Scott Air Force Base, based on the support NGA East receives from being located at Ft. Belvoir, as well as ensuring that the security-related sub-factors carried over from prior analyses were consistently defined. Additionally, the director added 2 sub-criteria to the mission-related criterion to ensure that the site evaluation continued in terms of NGA’s strategic goals of partnership and people: 1. “Team GEOINT,” which refers to NGA’s current and future partnerships with academic, public, and private sector partners, and which parallels the “GEOINT Valley” element evaluated by the mission evaluation team and PMO. 2. “Team NGA,” which refers to the potential effects of workforce recruitment and retention that were also analyzed in the mission evaluation team and PMO analyses.

According to NGA officials, while certain sub-factors or criteria were adjusted to provide further layers of analysis, the most important factors were always seen as mission and security. Additionally, NGA and Corps of Engineers officials said that adding these two sub-criteria expanded the analysis of the mission-related criteria to resemble the scope of the PMO’s analysis and incorporated the NGA Director’s mission and vision perspective.

Finally, the NGA Director determined the weighting of the final criteria to evaluate the last two sites, the site selection team provided input on which of the sites was more advantageous with respect to each criterion, and in March 2016 this information was used to inform the NGA Director’s selection of the agency-preferred alternative. The weighting and final decisions are shown in table 1.

The NGA Director selected the St. Louis City site as the agency-preferred alternative. It was identified in the publication of the final environmental impact statement and finalized with the issuance of the record of decision in June 2016.

NGA’s AOA Process for Selecting the New NGA West Site Substantially Met Three and Partially Met One Characteristic of a High-Quality, Reliable AOA Process but Lacked Important Cost Information

Site Selection Process Substantially Met Three of the Four Characteristics for a High-Quality, Reliable AOA Process but Could Have Been Strengthened

We compared NGA’s AOA process for selecting a site for the new NGA West campus to our AOA best practices and determined that NGA’s process substantially met three and partially met one characteristic of a high-quality, reliable AOA process. Although NGA’s AOA process substantially met most of the characteristics, we did find areas where the process could have been strengthened if NGA had more fully incorporated the AOA best practices. See table 2 for a summary of our assessment and appendix I for additional details on our scoring of NGA’s alignment with each of the 22 best practices.

NGA’s AOA process for selecting a site for the new NGA West substantially met the well-documented characteristic of a high-quality, reliable AOA process, although we did find areas for improvement.

For example, NGA’s AOA body of work demonstrated that the assumptions and constraints for each alternative for the site selection process were documented. NGA West’s Prospective Sites Master Plan included a set of overall assumptions that guided the preliminary planning process and provided specific assumptions and constraints for each alternative. Specifically, the plan identified various assumptions and constraints for the four final sites, such as calculations of the site boundaries, the estimated number of parking spaces, the square footage of the buildings and estimates of the building’s height, site utilities, and environmental constraints, among other things. In one instance, the plan documented the assumption that if the Mehlville site were to be used, all utilities would need to be removed from within the property line and existing buildings, parking lots, and roads would have to be demolished.

In another example, the Corps of Engineers conducted a schedule and negotiation risk assessment and recorded scores for each site and some mitigation strategies for specific issues. The assessment documented risks to meeting the site acquisition schedule with the St. Louis site because, among other reasons, the site needed environmental cleanup that was expected to take six months. The Fenton site had high negotiation risks, in part because the asking price of the site was significantly higher than the appraised value. However, NGA did not provide information on other risks, such as technical feasibility and resource risks, and did not rank the risks or provide over-arching mitigation strategies for each alternative. According to the best practice, not documenting the risks and related mitigation strategies for each alternative prevents decision makers from performing a meaningful trade- off analysis, which is necessary to select an alternative to be recommended.

Comprehensive Characteristic: Substantially Met

NGA’s AOA process for selecting a site for the new NGA West substantially met the comprehensiveness characteristic of a high-quality AOA process, but although it had strengths, we identified some limitations.

NGA’s AOA process considered a diverse range of alternatives to meet the mission need and conducted market surveillance and market research to develop as many alternative solutions as possible. According to our best practices, an AOA process that encompasses numerous and diverse alternatives ensures that the study provides a broad view of the issue and guards against biases to the AOA process. Specifically, NGA’s AOA process included a site location study that provided a summary of the thorough analysis that NGA conducted to identify potential site locations for the new NGA West campus. The study relied on information from local real estate market databases and input from the local real estate community, multiple municipal officials and organizations, and the public to identify an original set of 186 possible sites and narrow that list to a final 6 for further analysis.

However, although the NGA body of work provides evidence that the Corps of Engineers developed initial cost estimates that compared each alternative using different cost categories, NGA’s AOA process did not include life-cycle cost estimates for the final 4 sites. NGA officials chose not to analyze total construction and other facility sustainment costs, because they assumed that since the sites were in the same geographic area, construction and operating costs would be similar. However, the estimates did not include sufficient details regarding all of the costs examined—specifically, how the cost estimates were developed for information technology trunk line costs. NGA stated that this best practice had limited application to its AOA process because it had determined that variation in the life-cycle cost estimates based on the location of the four sites—all in the St. Louis metropolitan area—was negligible. NGA officials also stated that the lack of final project design details constrained their ability to develop full life-cycle cost estimates. However, without estimates for full life-cycle costs, decision makers may not have a complete picture of the costs for each alternative and may have difficulty comparing the alternatives, because comparisons may not be based on accurate and complete information. NGA and Corps of Engineers officials said that they are in the process of developing full life-cycle cost estimates for the construction and design of the new NGA West campus, for the agency- preferred alternative.

Unbiased Characteristic: Substantially Met

NGA’s AOA process for selecting a site for the new NGA West substantially met the characteristic of an unbiased AOA process, although we did identify some limitations.

NGA’s AOA body of work demonstrated that NGA had developed functional requirements based on the mission need without a predetermined solution and that the requirements were realistic, organized, and clear. For example, NGA’s AOA body of work provided facilities requirements and specifically listed 11 site location and campus requirements that were tied to mission needs, including requiring a facility that will support future changes to mission requirements and allow for continuity of NGA operations. NGA’s AOA body of work also identified physical requirements for the new NGA West campus, for example, that the new facility must have at least 800,000 gross square feet and a 500- foot security buffer, and it must allow for a possible expansion in the future.

However, although the NGA AOA body of work demonstrated a thorough comparison of the alternatives throughout the site evaluation process, it did not provide evidence that net present value was used to compare or differentiate among the alternatives, nor did it provide a rationale for why net present value could not be used. NGA officials acknowledged that they did not compare the alternatives using net present value. They stated that they had normalized some of the costs but that it was not necessary to normalize all costs, because the estimates were all done during the same time period. According to our best practice, if net present value is not used to compare the alternatives, then the AOA team should document the reason why and explain and describe the other methods applied. Additionally, comparing items that have been discounted or normalized with net present value allows for time series comparisons, since alternatives may have different life cycles or different costs and benefits.

Site Selection Process Only Partially Met the Characteristic for a Credible AOA Because It Lacked Important Information on Cost Risks and Sensitivities

NGA’s AOA process for selecting the site for the new NGA West campus partially met the credible characteristic for an agency’s AOA process. Although NGA’s AOA process had strengths, it also had limitations, such as lacking important information related to cost risks and sensitivity analyses for both cost and benefits identified.

NGA’s AOA body of work described the alternatives in sufficient detail to allow for robust analysis. Specifically, it provided descriptions of each of the alternatives at varying levels of detail. For example, the first site location study provided descriptions of the top 6 potential sites, including information on size, the sites’ strengths and weaknesses, and any acquisition or development issues. The NGA AOA body of work also provided evidence that site master planning was conducted to provide additional details on the physical and environmental attributes of each site, as well as constraints and benefits. For example, the NGA West Prospective Sites Master Plan described the Mehlville site as having landscape features such as mature trees, waterways, areas of steep topography, options for public transportation, bike-friendly streets, and existing utility infrastructure.

However, NGA did not fully include key information on either the risk or the uncertainty related to cost estimates or the sensitivity to the costs and benefits identified as part of its AOA process. For example, the NGA body of work did not include a confidence interval or range for the cost estimates for each viable alternative in order to document the level of risk associated with the estimate. NGA’s AOA body of work documented the estimated alternatives’ initial costs and included contingency costs across all four alternatives. Corps of Engineers officials told us that they had developed a 30 percent design and 5 percent construction contingency cost factor across the four alternatives to account for cost risks in those areas. However, the NGA AOA body of work did not provide evidence of a confidence interval or range for the costs provided. NGA acknowledged that while its AOA body of work did not identify the risk associated with specific cost elements for each alternative, it did provide a “level of confidence,” because the methodology behind the cost components in the estimate implied a high level of confidence. Although we agree that NGA did provide a contingency factor for the site development costs and provided cost estimates for all four viable alternatives, NGA did not develop a confidence interval or risk range for those estimates. NGA’s cost estimates were used as a determining factor in the final decision among the four alternatives. However, without understanding the cost risk and the uncertainty of those costs as outlined in the best practice, a decision maker might be unable to make an informed decision.

Additionally, the NGA AOA body of work did not demonstrate that NGA had conducted a sensitivity analysis for the cost and benefit and effectiveness estimates for each alternative in order to examine how changes in key assumptions would affect the cost and benefit estimates. The NGA AOA body of work documented that some sensitivity analysis or level of risk was analyzed as part of the schedule analysis, and NGA officials stated that the project considered how different values and variables affect each other during the criteria and evaluation analysis. However, the NGA AOA body of work did not document the sensitivity of cost and benefit estimates to changes in key assumptions for each alternative, and a sensitivity analysis was not applied to the initial cost estimates or benefit assumptions that were used to make the final site selection.

NGA officials stated that this best practice has limited application to its AOA process, because the lack of variables between sites constrained their ability to develop full life-cycle cost estimates and complete a sensitivity analysis. NGA officials stated that their sensitivity analysis was limited to those considerations that were measurable and sensitive to change—predominantly schedule risk associated with land acquisition activities. Further, NGA officials explained that because all the site alternatives were located within the St. Louis metropolitan area, any variations in conditions would have equal effect. Although we agree that NGA did conduct a sensitivity analysis for schedule risks, NGA neither documented how the schedule sensitivity affected its cost or benefit estimates nor performed a sensitivity analysis for the various assumptions used to develop the cost or benefit for each alternative. According to the DOD instruction on economic analysis, a sensitivity analysis is a “what-if” exercise that should be performed to test the conclusions and assumptions of changes in cost and benefit variables and should always be performed when the results of the economic analysis do not clearly favor any one alternative. According to our best practice, not conducting a sensitivity analysis to identify the uncertainties associated with different assumptions increases the chances that an AOA team will recommend an alternative without understanding the full effects of costs, which could lead to cost and schedule overruns.

Although NGA’s AOA process did not reflect all of the characteristics of a high-quality process, we are not making recommendations in this report, in part because NGA plans to conduct additional cost analysis and in part because we made an applicable recommendation to DOD in 2016. Specifically, although NGA’s AOA process is complete, NGA and Corps of Engineers officials said that they are developing full life-cycle cost estimates for the construction and design of the new NGA West campus and that these estimates will include many elements from our best practices. Further, we continue to believe that our September 2016 recommendation that DOD develop guidance requiring the use of AOA best practices for certain military construction decisions could help ensure that future AOA processes conducted by DOD agencies like NGA are reliable and that agencies identify a preferred alternative that best meets mission needs. While DOD did not concur with our recommendation, as we reported in 2016, our best practices are based on longstanding, fundamental tenets of sound decision making and economic analysis. Additionally, our best practices align with many DOD and military policies, directives, and other guidance pertaining to military construction. Further, during this review NGA officials stated that DOD did not have a set of best practices for conducting an AOA to help NGA make decisions regarding its military construction project, and that our AOA best practices would have been helpful had they been published prior to the start of NGA’s site selection process in 2012. Accordingly, we continue to believe our prior recommendation is relevant and that unless DOD has guidance directing that certain military construction AOA processes be conducted in accordance with identified best practices, it may not be providing Congress with complete information to inform its oversight of DOD’s future military construction decisions.

Agency Comments and Our Evaluation

We provided a draft of this report of this report to NGA for review and comment. NGA’s comments are reprinted in their entirety in appendix II.

In comments on our report, NGA stated that it valued our assessment of its AOA process, which we judged to have substantially met the characteristics of a well-documented, comprehensive, and unbiased process, and would use our findings to continue to refine and improve its corporate decision making and processes.

NGA raised a concern about our assessment that its AOA process used to select the site for its new NGA West project partially met the best practices that demonstrate a credible process. NGA’s specific concern was that we concluded that the AOA process did not fully include information on risks and sensitivities to cost estimates. In its letter, NGA stated that its analysis demonstrated that cost was a factor but not the most important factor. Moreover, NGA stated that cost elements and details ranged from well-defined costs, such as real estate costs, to estimates based on analogy such as an information technology trunk line. NGA additionally stated that, due to the conceptual nature of the design of the facility at that time, more detailed cost analysis was judged to provide no discrimination among alternatives and were thus purposely excluded from the initial cost estimates that were used in the AOA process. While NGA may have concluded that the project’s cost was not the most important factor, the agency estimates that construction of the campus will cost about $945 million and NGA used the cost estimate as a determining factor to select from the four final alternatives. Moreover, our assessment of the credibility characteristic is based only in part on NGA’s initial cost estimates and did not penalize NGA for excluding additional cost estimates. Rather, we assessed that NGA’s AOA body of work did not provide evidence of documenting the sensitivity of the cost-benefit or effectiveness estimates to changes in key assumptions for alternatives, nor was a sensitivity or risk analysis applied to the initial cost estimates used to make the final site selection.

NGA also stated in its letter that our AOA best practices are not applicable in all circumstances, and pointed out that DOD did not concur with a recommendation in a prior report to develop AOA guidance requiring departmental components to use AOA practices, including the best practices we identified, for certain future military construction projects. Our prior report suggested that such guidance might only apply to certain military construction projects as determined by DOD. In addition, while DOD’s existing relevant guidance does not require use of our AOA best practices, the guidance does not prohibit it either. Further, as discussed in our report, NGA officials told us the AOA best practices are helpful to such processes, and lacking such DOD guidance NGA had to draw on expertise, practices, and procedures from a variety of sources to conduct its AOA for the new NGA West site.

Finally, in its letter NGA proposed that two documents—the environmental impact statement and record of decision—fulfill the best practice to document the AOA process in a single document. Specifically, NGA stated that within the context of the environmental impact statement process, the record of decision is the authoritative capstone document of the process, and that together the two documents include discussions of the decision-making and factors considered by the director in selecting the agency-preferred alternative. These two documents were prepared to fulfill requirements of the National Environmental Policy Act of 1969 in order to determine the environmental impacts of the project, as discussed earlier in our report. While we recognize that the record of decision and the environmental impact statement are significant documents that include summaries of aspects of NGA’s AOA process, as NGA indicated these are two documents within an expansive AOA body of work. Further, many of the elements of NGA’s AOA process are diffused throughout these and several other reports and documents—that were specifically identified by NGA as the key documentation of its AOA process—rather than clearly delineated in a single document as prescribed by the best practice (see appendix I).

As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies of this report to the appropriate congressional committees; the Secretary of Defense; the Secretary of the Air Force; the Secretary of the Army; the Under Secretary of Defense for Acquisitions, Technology and Logistics; the Under Secretary of Defense for Intelligence; and the Director, National Geospatial-Intelligence Agency. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-4523 or leporeb@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix III.

Appendix I: Analysis of Alternatives (AOA) 22 Best Practices with GAO’s Evaluation of National Geospatial-Intelligence Agency (NGA) AOA Process and Scores

In our earlier discussion of the extent to which NGA’s AOA process met best practices for such processes, we presented our analysis for specific best practices. These 22 best practices and their definitions were originally published and are listed in GAO-16-22. Table 3 summarizes our analysis of NGA’s AOA process for selecting the site for the new NGA West and our ratings of that process against all 22 best practices.

Appendix II: Comments from the National Geospatial-Intelligence Agency

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Brian Mazanec, Assistant Director; Jim Ashley; Tracy Barnes; Chris Businsky; George Depaoli; Richard Johnson; Joanne Landesman; Jennifer Leotta; Jamilah Moon; Joseph Thompson; and Sally Williamson made key contributions to this report.

Related GAO Products

Amphibious Combat Vehicle Acquisition: Cost Estimate Meets Best Practices, but Concurrency between Testing and Production Increases Risk. GAO-17-402. Washington, D.C.: Apr. 18, 2017.

Joint Intelligence Analysis Complex: DOD Needs to Fully Incorporate Best Practices into Future Cost Estimates. GAO-17-29. Washington, D.C.: Nov. 3, 2016.

Joint Intelligence Analysis Complex: DOD Partially Used Best Practices for Analyzing Alternatives and Should Do So Fully for Future Military Construction Decisions. GAO-16-853. Washington, D.C.: Sept. 30, 2016.

Patriot Modernization: Oversight Mechanism Needed to Track Progress and Provide Accountability. GAO-16-488. Washington, D.C.: Aug. 25, 2016.

Amphibious Combat Vehicle: Some Acquisition Activities Demonstrate Best Practices; Attainment of Amphibious Capability to Be Determined. GAO-16-22. Washington, D.C.: Oct. 28, 2015.

DOE and NNSA Project Management: Analysis of Alternatives Could Be Improved by Incorporating Best Practices. GAO-15-37. Washington, D.C.: Dec. 11, 2014.

Military Bases: DOD Has Processes to Comply with Statutory Requirements for Closing or Realigning Installations. GAO-13-645. June 27, 2013.

Military Bases: Opportunities Exist to Improve Future Base Realignment and Closure Rounds. GAO-13-149. Washington, D.C.: Mar. 7, 2013.

Military Base Realignments and Closures: The National Geospatial- Intelligence Agency’s Technology Center Construction Project. GAO-12-770R. Washington, D.C.: June 29, 2012.

Military Base Realignments and Closures: DOD Is Taking Steps to Mitigate Challenges but It Is Not Fully Reporting Some Additional Costs. GAO-10-725R. Washington, D.C.: June 24, 2010.

GAO Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Capital Program Costs. GAO-09-3SP. Washington, D.C., Mar. 2, 2009.