Joint Intelligence Analysis Complex: DOD Needs to Fully Incorporate Best Practices into Future Cost Estimates
Released: 2016-12-05T00:00:00

Letter

Background

The JIAC Includes Elements of Three Joint Intelligence Operations Centers and Supports Other Intelligence-Related Functions

The JIAC includes elements of three intelligence operations centers—one supporting EUCOM, a second supporting U.S. Africa Command, and a third supporting the North Atlantic Treaty Organization—as well as several other organizations that perform intelligence-related functions. According to DOD guidance, joint intelligence operations centers support the geographical combatant commands and other defense organizations, serving as focal points for intelligence planning, collection management, analysis, and production.

EUCOM Joint Intelligence Operations Center Europe executes intelligence operations that are synchronized and integrated with theater component, national, and partner nation organizations; enables EUCOM planning and execution; and enhances senior leaders’ decision-making across the entire spectrum of military operations.

U.S. Africa Command Directorate for Intelligence at RAF Molesworth manages and executes defense intelligence for U.S. Africa Command, including protecting U.S. personnel and facilities, preventing and mitigating conflict, and building defense capabilities in order to promote regional stability and prosperity.

North Atlantic Treaty Organization Intelligence Fusion Center provides intelligence to warn of potential crises and to support the planning and execution of the North Atlantic Treaty Organization’s operations.

Regional Joint Intelligence Training Facility trains students from EUCOM, U.S. Africa Command, and the North Atlantic Treaty Organization nations, including the United Kingdom.

United States Battlefield Information Collection and Exploitation Systems plans, builds, and operates the Coalition Intelligence and Information Enterprise to provide on-demand coalition information- sharing solutions for both episodic and enduring missions.

DOD Organizations Involved in the JIAC Consolidation Process and Related Efforts

A number of DOD organizations have been involved in the JIAC consolidation process. Overall guidance for DOD’s military construction efforts was provided by the Office of the Assistant Secretary of Defense for Energy, Installations, and Environment. This office is responsible for overseeing various aspects of the department’s military construction efforts. These responsibilities include, among other things, monitoring the execution of the military construction program to ensure the most efficient, expeditious, cost-effective accomplishment of the program, and issuing guidance for the implementation of DOD military construction policy.

Other DOD organizations—including U.S. Air Force Headquarters, the Basing Office of the Office of the Secretary of Defense, and the headquarters of both EUCOM and U.S. Africa Command—made up the team that conducted the JIAC analysis of alternatives. The participating organizations provided subject matter experts who were involved in the team’s day-to-day work and developed the analysis that is the foundation of the decision to consolidate the JIAC at RAF Croughton. DOD’s team conducted work from the initial concept proposal in the fall of 2009 to the Resource Management Decision issued by the Secretary of Defense in April 2013.

In July 2016, we reported on DOD’s analysis of alternatives process and recommended that the Secretary of Defense direct the Assistant Secretary of Defense for Energy, Installations, and Environment to develop guidance requiring the use of best practices for analysis of alternatives—including those practices we identified in the report—and that in this guidance, the Assistant Secretary should define the types of military construction decisions for which use of these best practices should be required. DOD did not agree with our recommendation, stating that the best practices do not wholly apply to decision-making processes for military construction projects.

Table 1 lists the roles and responsibilities of DOD components related to the JIAC consolidation, including their involvement in preparing information in response to congressional requests for information on the analysis of alternatives process and on Lajes Field as a possible location for the JIAC.

Lajes Field, Azores, Portugal

In response a statutory requirement, DOD issued a memorandum that certified that Lajes Field, Azores (Portugal) was not the optimal location for the JIAC, based on an analysis of U.S. operational requirements and an evaluation of key criteria. The Azores is an autonomous region of Portugal situated about 850 miles west of continental Portugal. There are nine major Azorean islands, including Terceira, home of Lajes Field. Lajes Field is a dual military and civilian airfield and is also a Portuguese military base; the 65th Air Base Group, a U.S Air Force unit, is also stationed there. The 65th Air Base Group’s mission supports DOD, allied nations, and other authorized aircraft in transit; its core mission is to service in-transit aircraft en route to eastern and southern destinations.

In 2010, the Air Force recommended a plan to reduce personnel and operations at Lajes Field and divest approximately 500 U.S. military and civilian billets, leaving approximately 165 U.S. personnel at Lajes Field to support the mission requirements. The Secretary of Defense approved the recommendation and announced his decision to streamline Lajes Field in October 2012. According to the Secretary of Defense, the frequency and volume of flights using Lajes Field had decreased, and the base was operating well below its capacity. The Air Force recommended reducing U.S. operations at Lajes from 24 hours, 7 days a week to 8 hours, 7 days a week and downsizing the 65th Air Base Wing to an Air Base Group. According to the Secretary of Defense, the presence at Lajes Field exceeded mission requirements, and the mission requirements at Lajes Field could be supported with a smaller force. DOD has estimated that the streamlined footprint would yield approximately $35 million in annual savings. This Secretary of Defense’s October 2012 decision was subsequently reaffirmed by the European Infrastructure Consolidation assessment, which the Secretary of Defense initiated on January 25, 2013, to perform a comprehensive review of DOD facilities in Europe.

The National Defense Authorization Act for Fiscal Year 2014 required that the Secretary of Defense provide Congress with certification that the actions taken to realign military forces at Lajes Field were supported by the European Infrastructure Consolidation assessment. The act required that DOD’s certification include an assessment of the efficacy of Lajes Field in support of the United States’ overseas force posture. On January 6, 2015, the Secretary of Defense issued a memorandum certifying that the European Infrastructure Consolidation assessment supported DOD’s plan to adjust its presence at Lajes Field. Further, the Secretary of Defense noted that DOD had conducted a comprehensive review and determined that the reduction of U.S. personnel at Lajes Field supported the U.S. military’s European force posture.

DOD officials said that they will continue to provide 24-hour-a-day, 7-day- a-week tower operations at Lajes Field after the base personnel are reduced, along with crash, fire, and rescue services and attendant airfield operations for the joint Portuguese military and civilian airfield. Air Force officials told us that, as of July 2016, U.S. flights to Lajes Field average two per day. Air Force officials told us that, as part of this effort to streamline and reduce the personnel at Lajes Field, they have identified various excess buildings, facilities, and housing units that are no longer needed. Specifically, Air Force officials have identified 350 housing units as excess and are in the process of returning those units back to the Portuguese government; the remaining housing will be used to support unaccompanied personnel at Lajes Field. In July 2016, Air Force officials said that they were in negotiations with the Portuguese government on the return of excess facilities per the personnel streamlining efforts. Specifically, the officials stated that there was disagreement on how the United States and Portugal interpret the technical agreement and policy on the return of excess facilities—the United States’ position was that all property excess to the needs of the United States would be turned over to Portugal in useable condition, while Portugal sought for the United States to demolish a majority of the facilities rather than returning them. According to these officials, Portugal also sought environmental remediation commitments from the United States. According to DOD officials, the demolition and environmental remediation of these facilities was contrary to long-standing DOD policy.

Air Force’s February 2015 Cost Estimate for the JIAC Military Construction Project Did Not Fully Meet Best Practices

Our assessment of the Air Force’s February 2015 cost estimate for JIAC consolidation showed that it did not fully meet cost estimation best practices. According to the GAO Cost Estimating and Assessment Guide, a cost estimate created using best practices exhibits four characteristics—it is comprehensive, well documented, accurate, and credible. Each of the four characteristics is associated with a specific set of best practices. Our assessment found that the JIAC cost estimate partially met three and minimally met one of the four characteristics of a reliable cost estimate. If any of these characteristics is not substantially or fully met, then the cost estimate does not reflect the characteristics of a high-quality estimate and cannot be considered reliable. Table 2 lists each of the four characteristics, along with our summary assessment of the JIAC cost estimate.

The following summarizes our analysis of the JIAC cost estimate for each of the four characteristics. Appendix I provides greater detail on our assessment.

Comprehensiveness (Partially Met). According to best practices, agencies should develop accurate life-cycle cost estimates. Further, a life-cycle cost estimate should encompass all past (or sunk), present, and future costs for every aspect of the program, regardless of funding source—including all government and contractor costs. However, the JIAC cost estimate included only MILCON costs and did not include costs associated with the life cycle of the project. Air Force and Office of the Secretary of Defense officials said they do not consider the JIAC cost estimate a life-cycle cost estimate and that the estimate’s scope is in line with DOD guidance on the development of budget requests for MILCON projects. According to DOD officials, the department does not have a full life-cycle cost estimate for the entire JIAC consolidation effort. DOD and Air Force officials further stated that the estimate covers the costs of infrastructure for the JIAC’s facilities, other supporting infrastructure (e.g., utilities serving the JIAC facilities), and certain other facilities for functions related to JIAC, such as family support (e.g., expanded capacity of the child development center). The associated Operation and Maintenance costs (e.g., family support costs like living allowances) were not included in the estimate, because they were considered out of scope, according to DOD and Air Force officials. However, without fully accounting for life- cycle costs, management may have difficulty successfully planning and programming resource requirements for the JIAC consolidation and making sound decisions.

Documentation (Partially Met). According to best practices, documentation is essential for validating and defending a cost estimate. The JIAC cost estimate is generally consistent with the sizing assumptions included in DOD documentation, laying out requirements for the JIAC’s facilities, inputs from appropriate experts, and relevant DOD guidance, such as the DOD Facilities Pricing Guide. Also, DOD documented both the data sources and the methodology used for the JIAC cost estimate in an Excel spreadsheet model and a parametric cost engineering estimate summary. However, the documentation for the JIAC cost estimate is not complete. Specifically, the cost estimate does not provide sufficient documentation so that a cost analyst unfamiliar with the program could understand what had been done and replicate it. The cost estimate uses the DOD Facilities Pricing Guide (heretofore referred to as the Pricing Guide), which provides planning assumptions and prices for a variety of types of facilities, such as office buildings. In the cost calculation spreadsheet for the JIAC cost estimate, the cost estimators’ judgements regarding which type of facility to use from the Pricing Guide were not always consistent with the categories listed in the Pricing Guide. For example, there is no mention of intelligence facilities in the Pricing Guide, and we were unable to independently trace all of the unit costs from the JIAC cost estimate back to it. Air Force officials were able to show where in the Pricing Guide these numbers were drawn from; however, this is not documented in the estimating model, and there is no rationale provided to show that an intelligence facility would be the same as a communications center. Without a well-documented cost estimate, the Air Force may not be able to present a convincing argument for the validity of the JIAC cost estimate and answer decision makers’ and oversight groups’ questions.

Accuracy (Partially Met). According to best practices, the cost estimate should provide results that are unbiased, and it should not be overly conservative or overly optimistic. An estimate is accurate when it is based on an assessment of most likely costs, adjusted properly for inflation, and contains no more than a few minor mistakes—if any. In addition, a cost estimate should be updated regularly to reflect changes in the program, for example when schedules or other assumptions change or actual costs change, so that it always reflects the current status. The JIAC estimate used historical data, did not contain mathematical errors, showed evidence of being updated, underwent a review process before final approval, and follows DOD construction cost estimation guidance on how to account for inflation. However, while the JIAC cost estimate has been updated, it has not been updated regularly. Specifically, the April 2013 estimate was updated in June 2013 to align with an update to the Pricing Guide but was not updated to align with the two subsequent updates to the Pricing Guide that occurred before the February 2015 JIAC cost estimate was submitted. Air Force officials said that the JIAC cost estimate was updated to reflect foreign currency fluctuations. According to these officials, the MILCON process assumes flexibility in the project timeline to allow adjustments to the estimate and focuses on establishing the project’s scope (e.g., the square feet or square meters associated with a project) in the Air Force’s project development justification forms. These Air Force officials also stated that the costs associated with MILCON projects are updated only with significant changes to the program and are typically permitted to be adjusted by as much as plus or minus 25 percent of the total costs— even after funding has been appropriated—without needing to be reprogrammed. While updating an estimate in this way may be permissible within the established MILCON process, it is not consistent with cost estimating best practices, because the estimate is not updated regularly. Without updating the JIAC cost estimate on a regular basis, DOD and the Air Force may have difficulty analyzing changes in program costs for the consolidation project and may hinder the collection of up-to-date cost and technical data to support future JIAC cost estimates.

Credibility (Minimally Met). According to best practices, a cost estimate should discuss any limitations of the analysis resulting from uncertainty or biases surrounding data or assumptions. Major assumptions should be varied, and other outcomes recomputed, to determine how sensitive the cost estimates are to changes in the assumptions. Also, risk and uncertainty analysis should be performed to determine the level of risk associated with the estimate. Without a sensitivity analysis that reveals how a cost estimate is affected by a change in a single assumption, the cost estimator will not fully understand which variable most affects the cost estimate. The use of a sensitivity analysis is not specified in cost estimation guidance for MILCON projects from either DOD or the Air Force, and the JIAC cost estimate did not include such an analysis. According to Office of the Secretary of Defense and Air Force officials, a sensitivity analysis is part of the underlying unit cost development, because costs are developed through the use of both historical data and industry averages. These officials further stated that the Office of the Secretary of Defense uses actual data underpinned by relevant sensitivity and range analyses to develop its cost estimates. For example, Office of the Secretary of Defense and Air Force officials said that the Office of the Secretary of Defense uses the DOD Selling Price Index—which averages three commonly accepted national indexes for construction price escalation—to calculate actual project award cost data. However, for sensitivity analysis to be useful in informing decisions, careful assessment of the underlying risks and supporting data related to a specific MILCON project is also necessary. In addition, the sources of the variation should be well documented and traceable. Without conducting sensitivity analysis for the JIAC cost estimate to identify the effect of uncertainties associated with different assumptions, DOD and the Air Force increase the risk that decisions will be made without a clear understanding of the effects of these assumptions on costs.

Another key to establishing an estimate’s credibility is its review process. According to best practices, the estimate’s cost drivers should be crosschecked, and an independent cost estimate conducted by a group outside the acquiring organization should be developed to determine whether other estimating methods produce similar results. While the Air Force has a review process, the review it conducted for the JIAC cost estimate did not include the use of a checklist provided as a sample in DOD MILCON cost estimation guidance. The sample checklist, while not required, could have helped the Air Force to confirm the validity of assumptions and the logic used in estimating the cost of the JIAC construction tasks. Air Force officials stated that their review primarily looks at the numbers provided and the ranges from the Pricing Guide to see whether the estimate is within those ranges. These officials added that they would use the checklist only if there was a difference from the Pricing Guide.

However, the first phase of the JIAC cost estimate did not identify the stage of the estimate; did not separate costs for labor, equipment, or material; and did not calculate prime and subcontractor profit by the weighted guidelines method, which are items listed in the sample checklist.

When we shared the results of our analysis with officials from the Office of the Secretary of Defense, they said that they did not agree our best practices for cost estimating were entirely applicable to the JIAC cost estimate, since the estimate focused on MILCON costs. Furthermore, Office of the Secretary of Defense and Air Force officials said that construction is discussed in our Cost Estimating and Assessment Guide as a subsidiary cost to be included in the life-cycle cost estimate. For example, these officials said that construction costs are to be considered as part of the overall ground rules and assumptions for a cost estimate. However, the methodology outlined in our Cost Estimating and Assessment Guide is a compilation of best practices that federal cost estimating organizations and industry use to develop and maintain reliable cost estimates, and this methodology can be used across the federal government for developing, managing, and evaluating capital program cost estimates, including military construction estimates. Furthermore, DOD guidance for estimating construction costs states that in the MILCON program, construction cost estimates are prepared throughout the planning, design, and construction phases of a construction project. These construction cost estimates are categorized as follows: programming estimate, concept estimate, final estimate, and government estimate. The Air Force provided us with the JIAC consolidation programming estimate for analysis, because it was the most complete and updated estimate at the time of our review. Even though our analysis shows that the programming estimate did not meet all of the four characteristics of a high-quality, reliable estimate, the Air Force will have opportunities to incorporate our best practices as it prepares future cost estimates for subsequent phases of the JIAC consolidation program. Without incorporating a methodology that is more closely aligned with our best practices for cost estimation and incorporates all four characteristics of a high-quality, reliable estimate, the Air Force will not be providing comprehensive and high-quality information for decision makers to use.

DOD Conducted Multiple Reviews of Lajes Field as a Possible JIAC Location

After its 2013 decision to consolidate the JIAC at RAF Croughton, DOD conducted multiple reviews to provide information on Lajes Field as a potential alternative location for the JIAC, in response to congressional interest and inquiries. These reviews were developed by different organizations within DOD during 2015 and 2016 and included both one- time and recurring costs. The reviews produced different cost estimates, in particular for communications infrastructure and housing, because the DOD organizations that developed the reviews used different assumptions. However, all of the reviews found that consolidating the JIAC at Lajes Field would be more costly than consolidating it at RAF Croughton. Additionally, in response to statutory requirements, DOD issued a memorandum certifying that the department had determined that RAF Croughton was the optimal location for the JIAC and, conversely, that Lajes Field was not the optimal location, given the JIAC’s operational requirements.

DOD Has Conducted Multiple Reviews of Lajes Field as a Possible Location for the JIAC

From 2015 through 2016, DOD conducted multiple reviews of Lajes Field as a potential location for the JIAC, in response to congressional interest and inquiries. Lajes Field was not originally included in DOD’s analysis of alternatives for the consolidation of the JIAC. DOD officials told us that the reviews of 2015 and 2016 were not conducted with the same level of rigor as a formal cost estimate, because DOD had already completed its analysis of alternatives, and the decision to consolidate JIAC at RAF Croughton had already been made. DOD officials also told us that no credible new evidence had been produced to indicate the department should revisit its initial decision. Figure 1 includes the key events and reviews related to DOD’s analysis of the JIAC, including reviews related to Lajes Field, the European Infrastructure Consolidation study, JIAC consolidation, authorization, and appropriations, and the execution of the JIAC consolidation project.

According to officials from the Office of the Secretary of Defense, DOD did not alter or change its original decision to consolidate the JIAC at RAF Croughton based on the results of these reviews and found that consolidating the JIAC at Lajes Field would be more costly than consolidating it at RAF Croughton. Additionally, according to the Deputy Secretary of Defense, DOD’s reviews determined that Lajes Field was not a suitable location for the JIAC, based both on operational requirements and costs including housing availability and the lack of adequate secure communications infrastructure. These reviews were led by EUCOM, CAPE, and DISA.

EUCOM’s September 2015 review. EUCOM developed an analysis comparing RAF Croughton with Lajes Field as potential locations for the JIAC. EUCOM officials told us that this review was in response to congressional interest and requests, and it included inputs from U.S. Air Forces Europe and officials at Lajes Field. The review compared cost estimates associated with locating JIAC at RAF Croughton with those associated with locating it at Lajes Field. These cost estimates included one-time costs, such as construction costs for the JIAC facilities, communications infrastructure, and housing, as well as recurring costs—including sustainment costs for base and communications infrastructure. The review estimated the one-time costs associated with locating the JIAC at RAF Croughton at $357 million and the one-time costs associated with locating it at Lajes Field at $1.65 billion. For recurring costs, the review estimated that RAF Croughton would cost approximately $68 million annually and Lajes Field approximately $94 million annually. The largest differences between the cost estimates were in the one-time costs for the communications and housing infrastructure necessary to support the JIAC. Officials from the Office of the Secretary of Defense told us that DOD had provided this review, with its appendixes, to the House and Senate Armed Services Committees and the House and Senate Appropriations Committees in September 2015.

This review included two appendixes developed by DISA and DIA on the communications infrastructure needed to support the JIAC. DISA’s July 2015 Azores Telecommunications Feasibility Report provided an analysis of the telecommunications infrastructure on the Azores Islands. This report indicated that the Azores did not have sufficient communications infrastructure to be a feasible location for a DISA telecommunications hub. The DIA Azores Communications Cost Estimate provided a brief summary on the current and proposed communications systems within the Azores Islands, as well as the costs associated with modernizing the systems. The appendix noted that it was developed in response to a request from the DIA Office of Congressional Affairs. For communications infrastructure, the DIA estimated that locating the JIAC at Lajes Field would require approximately $449 million in one-time costs and $32.7 million in recurring annual sustainment costs.

CAPE’s April 2016 cost verification for the JIAC. CAPE conducted an independent review of the cost estimates presented in EUCOM’s September 2015 review and those developed by the House Permanent Select Committee on Intelligence for its July 2015 review. CAPE developed its own cost assumptions, which included housing and communications infrastructure costs, among other things, in its review of the cost calculations in the EUCOM and House Permanent Select Committee on Intelligence reviews, which produced alternative cost totals. CAPE officials told us that this review was in response to direction from the Deputy Secretary of Defense and that they briefed this review to the House Permanent Select Committee on Intelligence in May 2016 and the House and Senate Armed Services Committees in April 2016. CAPE’s review estimated the one-time costs associated with locating the JIAC at RAF Croughton at $356 million and one-time costs associated with locating it at Lajes Field at $1.43 billion (compared with EUCOM’s estimates of $357 million and $1.65 billion respectively). For recurring costs, CAPE’s review estimated that RAF Croughton would cost approximately $53 million annually and Lajes Field approximately $59 million annually (compared with EUCOM’s estimates of $68 million and $94 million, respectively).

DISA’s May 2016 review on the JIAC communications infrastructure requirements. In this update to its July 2015 review, DISA assessed and compared the communications infrastructures at RAF Croughton and Lajes Field with the intelligence mission support requirements, including the communications and technical requirements for the JIAC. DISA officials told us that this review included more refined cost estimates for the communications infrastructure than prior estimates and reflected new technical standards, such as operational bandwidth requirements. The review found that the communications infrastructure at Lajes Field did not meet technical and critical infrastructure requirements. To upgrade the communications infrastructure at Lajes Field, the review estimated a minimum of $267.7 million in one-time costs to procure and install three undersea cables and $6.8 million in annual sustainment costs. For locating the JIAC at RAF Croughton, the review determined that no procurement would be required and estimated sustainment costs for the communications infrastructure at $5.5 million annually. DOD officials told us that they briefed the results of this review to the House Armed Services Committee in September 2016.

DOD’s Multiple Reviews of Lajes Field Produced Different Cost Estimates Because They Were Based on Different Assumptions

DOD’s multiple reviews of Lajes Field as an alternative location for the JIAC produced different cost estimates, because these reviews relied on different assumptions in developing the cost estimates for communications infrastructure and housing.

Assumptions for Communications Infrastructure Costs Differed

DOD’s multiple reviews provide different cost estimates for the communications infrastructure that would be needed to support the JIAC at Lajes Field, because the reviews relied on different assumptions. Specifically, the reviews varied in the costs they included and the number of annual fiber cable breaks they expected would occur, among other details. The three reviews all assumed that three new fiber cables would be needed for Lajes Field. However, the distribution of these fiber cables differs in the reviews. Specifically, the September 2015 EUCOM review and the May 2016 DISA review assume one fiber cable from Lajes Field to mainland Portugal, one fiber cable to the United Kingdom, and one fiber cable to the United States, while the April 2016 CAPE review assumes two fiber cables from Lajes Field to the United States and one fiber cable to the United Kingdom. DOD officials told us that the cable distribution cited in the September 2015 EUCOM and the May 2016 DISA reviews reflect the current JIAC operational requirements, based on a May-June 2015 DIA operational assessment and that the CAPE review reflects a different JIAC operational design. Officials from DIA said that this change in operational requirements was made in various discussion sessions conducted among subject matter experts, and that the decision was not documented. Figure 2 shows the current cable configuration at Lajes Field and the new cables that would be necessary based on the September 2015 EUCOM review, the May 2016 DISA review, and the April 2016 CAPE review.

The May 2016 DISA review had the lowest estimate for communications costs of the three reviews. DISA officials told us that these cost estimates were deliberately built on assumptions that would generate the lowest possible costs. However, DOD officials told us that DISA has not been able to validate all of its assumptions. Table 3 shows the cost estimates and supporting assumptions included in DOD’s multiple reviews for communications infrastructure associated with locating the JIAC at Lajes Field.

Appendix II contains additional information on the requirements for the communications capabilities related to the JIAC.

Assumptions for Housing Costs Differed

Two of DOD’s reviews provided cost estimates for the housing needed to support the JIAC at Lajes, but the estimates were based on different assumptions. Specifically, EUCOM estimated the one-time housing costs for locating the JIAC at Lajes Field at $390.5 million, while CAPE estimated these costs at $188 million. EUCOM’s review assumed that 1,031 new housing units would be needed on the base at Lajes Field, and CAPE’s review assumed that as few as 385 new units would be needed on base. Additionally, EUCOM’s estimate assumes that there would be a 252-person dorm unit shortfall for unaccompanied military personnel, and CAPE assumed a shortfall of 368 dorm units. Table 4 shows the two cost estimates for the housing and the assumptions that each review used.

EUCOM and CAPE used different assumptions when developing their cost estimates for the number of housing units needed to support the JIAC at Lajes Field. Specifically, EUCOM reported that 1,812 housing units were needed to support the JIAC and that those units would include not only housing for the accompanied personnel working at the JIAC (around 1,200 personnel) but also housing units for the additional base operations and support personnel to support the JIAC (around 330 personnel) and personnel associated with the reversal of the personnel streamlining initiative at Lajes Field (around 751 personnel). EUCOM determined its estimate for accompanied housing units required for the JIAC at Lajes Field by using Air Force personnel standards and JIAC planning factors. On the other hand, CAPE’s estimates assumed that 1,260 accompanied housing units would be needed, that dorm units would be used, and unaccompanied civilians would live off base to minimize the effect on military family housing. A factor in the difference between EUCOM’s and CAPE’s housing cost estimates is the addition of 751 personnel (resulting in a need for 451 additional accompanied housing units) that EUCOM included in its cost estimate when the Lajes personnel streamlining initiative was reversed. CAPE officials told us that their estimate did not assume that reversing the personnel streamlining initiative would result in the addition of so many personnel, and that therefore there would be a reduced need for additional housing. However, CAPE assumed that more base operations and support personnel would be needed (CAPE assumed that 500 support personnel would be needed, while EUCOM assumed 330) to support the JIAC at Lajes Field.

Both reviews also provided housing estimates for unaccompanied military personnel. EUCOM’s review assumed that 469 unaccompanied military personnel would reside in the existing 217 dorm spaces at Lajes Field, and there would be a shortfall of 252 dorm spaces (32 of those dorm units would be built using the JIAC military construction funds and the other 220 units represent the shortfall). CAPE’s review assumed that unaccompanied military personnel would reside in the existing 217 dorm units at Lajes Field and that DOD would build two additional dorms (for 168 and 200 personnel) to accommodate the unaccompanied military personnel. Additionally, EUCOM’s review assumed that 204 unaccompanied civilians would live on base and CAPE’s review assumed that 91 unaccompanied civilians would live off base in small family housing.

EUCOM’s review used the military family housing inventory of the U.S. Air Forces in Europe to determine the number of housing units on the base (456 housing units), while CAPE’s estimate assumes that there were 550 housing units at Lajes Field. In July 2016, Air Force personnel at Lajes Field confirmed that there were 456 available housing units at Lajes Field. Also, both CAPE’s and EUCOM’s reviews used the January 2007 Housing Requirements Market Analysis for Lajes Field to determine the number of housing units that were available for rent off base (approximately 229 housing units). However, according to information provided by Terceira’s municipalities, there are currently 1,693 houses in the Island of Terceira available for rent, and almost 400 were recently occupied by U.S. military personnel and their families. According to the 2007 Housing Requirements and Market Analysis DOD conducted at Lajes Field, the Lajes rental market is separated into two areas—housing units that are specifically marketed to U.S. military personnel, have been inspected for suitability, and are listed in the Lajes Field housing rental database, and housing units that are part of the local rental market but not of sufficient quality and without the amenities required by U.S. military and civilian personnel. Further, EUCOM officials told us that the 1,693 housing units available on the Island of Terceira may not all be suitable for U.S. military forces. Air Force officials told us that there were only 225 rental properties in their off-base referral database. Both EUCOM’s and CAPE’s estimates assumed that there were 225 rental properties on the island and that another 100 would be built (for a total of 325) to support the personnel for the JIAC.

Both reviews assumed that no additional housing units would be necessary at RAF Croughton, based on the 2016 Housing Requirements Market Analysis for RAF Croughton. According to Air Force officials, past housing and United Kingdom basing trends indicate that personnel associated with the JIAC would live off base in the private rental market, and the private rental market could sufficiently absorb the housing needs of the JIAC personnel. Further, EUCOM reported that the United Kingdom had the capacity to absorb the number of personnel associated with the JIAC move and that they would not need to build additional military family housing at RAF Croughton. The 2016 Housing Requirements Market Analysis for RAF Croughton reported that the private rental market was very active, that there was a total private rental stock of 69,364 rental units, and that the housing supply was projected to grow to 72,905 units by 2020.

DOD Responded to Statutory Requirements by Certifying that RAF Croughton is the Optimal Location for the JIAC

In addition to its multiple reviews, DOD issued a memorandum in March 2016 stating that the department had determined that RAF Croughton remained the optimal location for the JIAC and that Lajes Air Field is not an optimal location for the JIAC. Specifically, the Deputy Secretary of Defense issued a memorandum in response to several requirements in Section 2310 of the National Defense Authorization Act for Fiscal Year 2016; House Report 114-144 accompanying HR 2596, the Intelligence Authorization Act for Fiscal Year 2016; and Section 8114 of the DOD Appropriations Act for Fiscal Year 2016 (division C). The memorandum states that DOD’s decision was based on an analysis of U.S. operational requirements and an evaluation of multiple locations using five criteria: effect on intelligence operations (critical criterion); impact on bilateral and multinational intelligence collaboration (critical criterion); impact on international agreements and relationships; impact on community quality of life; and business case analysis. According to officials from the Office of the Secretary of Defense, DOD reviewed existing analysis and did not conduct new in-depth analysis to support the certification memorandum. The analysis DOD used to support the memorandum was based on the original analysis of alternatives process that DOD developed for the JIAC consolidation—which did not include Lajes Field as an alternative location—and on subsequent comparisons of Lajes Field and RAF Croughton. The officials stated that no additional in-depth analysis was warranted because no credible new evidence had been produced to indicate the department should revisit its initial decision.

Conclusions

To address costly sustainment challenges and instances of degraded theater intelligence capabilities associated with the current JIAC facilities at RAF Molesworth, DOD plans to spend almost $240 million for the Air Force to consolidate and relocate the JIAC’s facilities at RAF Croughton. However, the Air Force’s cost estimate did not fully meet cost estimating best practices that are intended, when followed, to produce high-quality, reliable estimates. For example, the JIAC cost estimate included only MILCON costs and did not include costs associated with the life cycle of the project. Without fully accounting for life-cycle costs, management may have difficulty successfully planning and programming resource requirements for the JIAC consolidation and making sound decisions. Furthermore, the JIAC cost estimate lacked a sensitivity analysis, which would assess the underlying risks and supporting data. Without identifying the effects of uncertainties associated with different assumptions for the JIAC consolidation project, there is an increased risk that decisions will be made without a clear understanding of these effects on costs. Unless DOD uses best practices as it prepares future cost estimates for the remaining design and construction phases of the JIAC consolidation project, decision makers will not receive complete and reliable information on the total anticipated costs for the JIAC consolidation efforts for which they need to conduct oversight and make informed funding decisions. Furthermore, addressing limitations in future JIAC cost estimates can provide DOD better information to predict costs and make informed decisions about the JIAC consolidation.

Recommendation for Executive Action

To better enable DOD to provide congressional decision makers with complete and reliable information on the total anticipated costs for the JIAC consolidation efforts, we recommend that the Office of the Assistant Secretary of Defense for Energy, Installations, and Environment’s Basing Office—in coordination with the Office of the Assistant Secretary of the Air Force Installations, Environment and Energy—update future construction cost estimates for consolidating the JIAC at RAF Croughton using best practices for cost estimating as identified in the GAO Cost Estimating and Assessment Guide. Specifically, cost estimates for the JIAC consolidation should fully incorporate all four characteristics of a high-quality, reliable estimate.

Agency Comments and Our Evaluation

We provided a draft of this report to DOD for review and comment. DOD provided written comments on our recommendation, which are reprinted in appendix III. The department also provided technical comments that we incorporated as appropriate.

In its written comments, DOD did not concur with our recommendation. DOD agreed that many components in the GAO Cost Estimating and Assessment Guide are broadly applicable in the decision process leading up to a military construction budget request. However, DOD further stated that once military construction funds are authorized and appropriated by Congress, the department transitions to a project management mode, and it would be a waste of resources to continue to generate cost estimates once they have transitioned to managing project execution using actual cost data. However, as we note in the report, DOD guidance for estimating construction costs, DOD’s Unified Facilities Criteria 3-740- 05, states that in the MILCON program, construction cost estimates are prepared throughout the planning, design, and construction phases of a construction project to account for the refinement of the project’s design and requirements. The final estimate should document the department’s assessment of the program's most probable cost and ensure that enough funds are available to execute it. As of October 2016, the military construction funds had not been authorized by Congress for the third phase of the JIAC construction project. According to DOD officials, construction is not scheduled to begin until fall of 2017, and the contract has not yet been awarded.

Further, the GAO Cost Estimating and Assessment Guide states that regardless of whether changes to the program result from a major contract modification or an overtarget budget, the cost estimate should be regularly updated to reflect all changes. This is also a requirement outlined in OMB’s Capital Programming Guide. The purpose of updating the cost estimate is to check its accuracy, defend the estimate over time, and archive cost and technical data for use in future estimates. After the internal agency and congressional budgets are prepared and submitted, it is imperative that cost estimators continue to monitor the program to determine whether the preliminary information and assumptions remain relevant and accurate. Keeping the estimate updated gives decision makers accurate information for assessing alternative decisions. Cost estimates must also be updated whenever requirements change, and the results should be reconciled and recorded against the old estimate baseline.

Therefore, we continue to believe that DOD’s implementation of our recommendation to update future JIAC cost estimates using the best practices identified in the GAO Cost Estimating and Assessment Guide would assist in ensuring that decision makers have complete and reliable information about costs associated with the JIAC consolidation and as the third phase of the JIAC project is authorized. Implementing our recommendation would also ensure that DOD develops a reliable historical record for the cost of the JIAC that can be used to estimate other similar projects in the future.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the date of this report. At that time, we will send copies of this report to the appropriate congressional committees and to the Secretaries of Defense, the Army, the Navy, and the Air Force; the Commandant of the Marine Corps; and the Assistant Secretary of Defense for Energy, Installations, and Environment. In addition, this report will be available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-4523 or leporeb@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix IV.

Appendix I: Assessment of the Air Force’s February 2015 Cost Estimate for the Joint Intelligence Analysis Complex

According to the GAO Cost Estimating and Assessment Guide, a cost estimate is a critical element in any acquisition process to help decision makers evaluate resource requirements at milestones and other important decision points. Cost estimates establish and defend budgets and drive affordability analysis. The guide identifies four characteristics of a high-quality, reliable cost estimate: it is comprehensive, well documented, accurate, and credible. A cost estimate is considered comprehensive when it accounts for all possible costs associated with a project, details all cost-influencing ground rules and assumptions, is technically reasonable, is structured in sufficient detail to ensure that costs are neither omitted nor double-counted, and the estimating teams’ composition is commensurate with the assignment; well documented when supporting documentation for the estimate is accompanied by a narrative explaining the process, sources, and methods used to create the estimate and contains the underlying data used to develop the estimate; accurate when the estimate is neither overly conservative nor too optimistic and is based on an assessment of the costs most likely to be incurred; and credible when the estimate has been cross-checked with independent cost estimates, the level of confidence associated with the point estimate—the best guess at the cost estimate given the underlying data—has been identified, and a sensitivity analysis has been conducted. During the sensitivity analysis, the project will have examined the effect of changing one assumption related to each project activity while holding all other variables constant in order to identify which variable most affects the cost estimate.

Our analysis of the Air Force’s February 2015 cost estimate for the Joint Intelligence Analysis Complex (JIAC) showed that, when compared with best practices, it minimally met one and partially met three of the four characteristics of a reliable cost estimate (see table 1). According to the GAO Cost Estimating and Assessment Guide, a cost estimate is considered reliable if the overall assessment ratings for each of the four characteristics are substantially or fully met. If any of the characteristics is not met, minimally met, or partially met, then the cost estimate does not fully reflect the characteristics of a high-quality estimate and cannot be considered reliable.

Appendix II: Information on Communications Infrastructure Requirements for the Joint Intelligence Analysis Complex

The May 2016 review by the Defense Information Systems Agency (DISA) discussed technical requirements and developed minimum standards for developing its cost estimates for communications infrastructure associated with locating the Joint Intelligence Analysis Complex (JIAC) at Lajes Field. The standards support the Department of Defense’s (DOD) position that locating the JIAC at Lajes Field would require the procurement and installation of three undersea cables. The DISA review stated that capabilities for global intelligence telecommunications at the JIAC must be secure, highly available, reliable, and redundant. The review also listed technical requirements based on these four characteristics, none of which—according to the review—the infrastructure at Lajes Field currently meets. One of these requirements is a critical infrastructure protection practice from the DOD Director of National Intelligence, which prohibits the use of communication paths that could result in denial of service or could compromise the integrity of information. The review characterizes DOD’s meeting this requirement at Lajes Field as a high risk, noting that non-DOD personnel from Huawei, a Chinese telecommunications company, could disconnect one fiber of the two-fiber ring at Lajes Field, which would eliminate the redundancy of the two cables and increase the risk that JIAC personnel would not be able to use the communications infrastructure to meet their operational requirements. The DISA review also listed a technical requirement that the communications infrastructure be able to operate at 56 gigabits per second, which the review noted is the minimum operational requirement for Non-classified Internet Protocol Router Network; Secret Internet Protocol Router Network; Joint Worldwide Intelligence Communications System; and voice, video, and data. According to the review, the current capabilities at Lajes Field do not meet this requirement. In comparison, the capacity at RAF Croughton allows for 800 gigabits per second, the capacity at U.S. Central Command also allows for 800 gigabits per second, and the capacity at U.S. Pacific Command allows for 100 gigabits per second. According to the review, DOD technical requirements also specify that the communications cables must be available at 99.999 percent or higher, which equates to just a few minutes of downtime per year. As indicated by DISA in its review, this level of availability requires sufficient redundancy. The capabilities at Lajes Field do not meet this requirement, according to DISA’s review. DISA officials provided us data on the number of average fiber cable outages per week in 2015—6.8 average outages per week for transatlantic cables and 4.4 average outages per week for Pacific cables. The frequency with which the cables experience outages highlights the need for redundancy in fiber cable routes. Without procuring and installing three undersea cables, Lajes Field would not have the availability, redundancy, capacity, and security necessary to house the JIAC.

The September 2015 review by U.S. European Command (EUCOM) also references standards; however, it did not discuss these standards in detail. Its appendix on communications infrastructure, developed by DIA, says that the minimum threshold for fiber cables is two protected pathways to mainland Europe and one to the continental United States. Additionally—similar to the May 2016 DISA review—the EUCOM review indicated that three new undersea systems would have to be installed at Lajes Field to meet DOD requirements. DIA officials also told us that their assessment was based on DOD guidance and requirements, such as the Joint Intelligence Operations Center Enterprise Functional Requirements document and the Chairman of the Joint Chiefs of Staff Instruction 6211.02D Defense Information Systems Network (DISN) Responsibilities, (Jan. 24, 2012).

The review by the Office of Cost Assessment and Program Evaluation (CAPE) did not discuss requirements or standards for the communications infrastructure, because it relied on DISA’s previous cost estimates. CAPE officials stated that they had deferred to DISA’s estimate, because DISA is the authoritative source for communications infrastructure design.

Appendix III: Comments from the Department of Defense

Appendix IV: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Brian Mazanec (Assistant Director), Jennifer Andreone, Tracy Barnes, Jennifer Echard, Justin Fisher, Joanne Landesman, Jennifer Leotta, Amie Lesser, Jamilah Moon, Carol Petersen, and Sam Wilson made key contributions to this report.