Joint Intelligence Analysis Complex: DOD Needs to Fully Incorporate Best Practices into Future Cost Estimates
Released: 2016-12-05T00:00:00


Highlights: section_title: Why GAO Did This Study
paragraphs: DOD's JIAC, which provides critical intelligence support for the U.S. European and Africa Commands and U.S. allies, is currently located in what DOD has described as inadequate and inefficient facilities at RAF Molesworth in the United Kingdom. To address costly sustainment challenges and instances of degraded theater intelligence capabilities associated with the current JIAC facilities, the Air Force plans to spend almost $240 million to consolidate and relocate the JIAC at RAF Croughton in the United Kingdom.

GAO was asked to review analysis associated with consolidating and relocating the JIAC. This report (1) assesses the extent to which DOD's cost estimate for the JIAC consolidation at RAF Croughton aligns with best practices and (2) describes key reviews DOD has conducted since spring of 2013 related to an alternative location for JIAC consolidation. GAO compared the Air Force's February 2015 JIAC cost estimate with GAO best practices for developing federal cost estimates, reviewed key DOD analysis of Lajes Field as a potential alternative location for the JIAC, and interviewed DOD officials.

section_title: What GAO Found
paragraphs: GAO assessed the cost estimate for the military construction project to consolidate and relocate the Joint Intelligence Analysis Complex (JIAC) at Royal Air Force (RAF) base Croughton and found that it partially met three and minimally met one of the four characteristics of a reliable cost estimate defined by GAO best practices, as shown in the table below. For example, it minimally met the credibility standard because it did not contain a sensitivity analysis; such analyses reveal how the cost estimate is affected by a change in a single assumption, without which the estimator will not fully understand which variable most affects the estimate. Unless the Department of Defense's (DOD) methodology incorporates all four characteristics of a high-quality, reliable estimate in preparing future cost estimates for the JIAC construction project, it will not be providing decision makers with reliable information.

After DOD's 2013 decision to consolidate the JIAC at RAF Croughton, DOD organizations conducted multiple reviews in response to congressional interest in Lajes Field, Azores (Portugal) as a potential alternative location for the JIAC, including

U.S. European Command (EUCOM) September 2015 review , a cost comparison and location analysis of RAF Croughton and Lajes Field;

Office of the Secretary of Defense Cost Assessment and Program Evaluation April 2016 cost verification for JIAC , an independent review of EUCOM's September 2015 cost estimates and those developed by the House Permanent Select Committee on Intelligence in July 2015; and

Defense Information Systems Agency May 2016 review on JIAC communications infrastructure requirements , an assessment and comparison of the communications infrastructures at Lajes Field and RAF Croughton with the intelligence mission support requirements, including the communications and technical requirements, for the JIAC.

These reviews produced different cost estimates, in particular for housing and communications infrastructure, because the DOD organizations that developed them relied on different assumptions. DOD officials said that these reviews were not conducted with the same level of rigor as formal cost estimates, because DOD had concluded its analysis of alternatives and no credible new evidence had been produced to indicate the department should revisit its initial decision to consolidate the JIAC at RAF Croughton.

section_title: What GAO Recommends
paragraphs: GAO recommends that DOD update its future construction cost estimates for consolidating the JIAC at RAF Croughton to comply with best practices for cost estimating identified by GAO. DOD did not agree, stating it would waste resources to continue to generate cost estimates once DOD transitions to managing the project with actual cost data. GAO continues to believe that its recommendation is valid, as discussed in this report.

section_title: Letter
paragraphs: The Joint Intelligence Analysis Complex (JIAC) in the United Kingdom  provides intelligence support for the U.S. European Command (EUCOM),  U.S. Africa Command and U.S allies. The Department of Defense (DOD)  has stated that although the JIAC is a critical intelligence asset, its 21  current facilities—located at Royal Air Force (RAF) base Molesworth—are  inadequate and inefficient. According to DOD, the facilities at RAF  Molesworth were not designed to support the JIAC’s intelligence  missions, and this has led to costly sustainment challenges and instances  of degraded theater intelligence capabilities. For example, the U.S. Air  Force has reported that the JIAC has experienced numerous outages to  facility-related information technology systems that have severely  degraded theater command and control and intelligence capabilities for  several hours at a time. To address these issues, DOD will invest  approximately $240 million in military construction (MILCON) for the Air  Force to consolidate and relocate the JIAC to provide additional space  specifically designed to support and fully enable the current intelligence  mission.

In the fall of 2009, DOD began its analysis of alternatives process for  consolidating the JIAC and, as part of this analysis, considered potential  alternative locations in Europe and the United States. By the spring of  2013, DOD had concluded that its preferred alternative was to move the  JIAC from its current location at RAF Molesworth and consolidate its  facilities at RAF Croughton in the United Kingdom. According to the  Deputy Secretary of Defense, the decision to locate the JIAC at RAF  Croughton was the result of careful consideration of intelligence and  operational requirements and factors and was informed by other criteria,  such as diplomatic relationships and economic factors. The Secretary of  Defense issued a Resource Management Decision in April 2013 to move  ahead with the JIAC consolidation. After the selection of RAF Croughton,  the Air Force developed a MILCON cost estimate and initiated a three- phase MILCON project which, according to Air Force officials, it plans to  complete by early 2021.

The consolidation of the JIAC is proceeding in three phases. In Phase I,  according to the Air Force budget documentation, the Air Force plans to  build EUCOM’s Intelligence Analytic Center, a data processing center, a  warehouse, and various supporting facilities. In Phase II it plans to build  the U.S. Africa Command’s Intelligence Analytic Center, the Intelligence  Fusion Center of the North Atlantic Treaty Organization, a battlefield  information system, and additional supporting facilities. Phase III will  include a regional Joint Intelligence Training Facility and a physical fitness  facility. In December 2014 and November and December 2015, Congress  authorized and appropriated $92.2 million for Phase I and $94.2 million  for Phase II. In September 2016, Congress appropriated $53.1 million for  Phase III. The United Kingdom is serving as executive agent for JIAC  construction, and in August 2015, DOD obligated Phase I funding to the  United Kingdom. DOD officials told us that DOD obligated Phase II  funding to the United Kingdom in July 2016. As of September 2016, DOD  officials said they anticipate awarding a contract for Phases I and II in  mid-2017 with construction to start in the fall of 2017.

Since 2013, Congress has required DOD to provide information on its  JIAC consolidation effort and has taken a number of oversight actions.  For example, House Report 113-102 directed DOD to provide detailed  information on its plans for the JIAC consolidation at RAF Croughton.  House Report 113-02 also directed the Secretary of Defense to submit a  report on RAF Croughton and, among other things, provide a description  of why the current facility, RAF Molesworth, was not sufficient to meet  JIAC’s requirements and how the facility at RAF Croughton would meet  them. In other statutory provisions, Congress has prohibited DOD from  obligating or expending military construction funds in the United Kingdom  or awarding construction contracts in the EUCOM area of responsibility  until it provides certain required information to Congress. In the National  Defense Authorization Act for Fiscal Year 2016, Congress authorized  funding for the JIAC consolidation but prohibited the expenditure of funds  until the Secretary of Defense certifies to the congressional defense  committees that RAF Croughton remains the “optimal location” for the  consolidation of the JIAC and explains why. According to DOD officials,  the department has provided the required certifications in response to  committee direction and statutory provisions.

We were asked to review the analysis associated with consolidating and  relocating the JIAC. This report (1) assesses the extent to which DOD’s  cost estimate for JIAC consolidation at RAF Croughton aligns with best  practices and (2) describes key reviews DOD has conducted since spring  of 2013 related to an alternative location for JIAC consolidation.

We performed our work at various DOD organizations, including U.S. Air  Force Headquarters, U.S. Air Forces in Europe, the Basing Office of the  Office of the Secretary of Defense, the Office of Cost Assessment and  Program Evaluation (CAPE), the Defense Intelligence Agency (DIA), the  Defense Information Systems Agency (DISA), the Office of the DOD  Chief Information Officer, and the headquarters of EUCOM. In addition,  we visited Lajes Field, Azores (Portugal) and met with representatives of  the 65th Air Base Group.

To determine the extent to which DOD’s cost estimate for JIAC  consolidation at RAF Croughton aligns with best practices, we gathered  information on the department’s cost estimation process. We obtained the  Air Force’s most recently completed and approved cost estimate for JIAC  consolidation—dated February 2015—at the time of our review, and we  obtained detailed information on the estimate, including DOD’s cost  model, from officials responsible for the estimate. Using best practices  criteria in GAO’s Cost Estimating and Assessment Guide, we evaluated  the extent to which the JIAC cost estimate was comprehensive, accurate,  well documented, and credible. The methodology outlined in GAO’s  Cost Estimating and Assessment Guide is a compilation of best practices  that federal cost estimating organizations and industry use to develop and  maintain reliable cost estimates throughout the life of a government  program. We met with DOD officials to discuss areas we identified  through our analysis where the cost estimate did not conform to cost  estimation best practices.

To describe the key reviews DOD has conducted since spring of 2013  related to alternative locations for the JIAC, we reviewed key analyses  developed by DOD to provide more information to Congress comparing  the requirements for consolidating the JIAC at RAF Croughton with the  requirements for consolidating at Lajes Field. Those analyses include  work in response to statutory requirements—such as DOD’s responses to  requirements in the National Defense Authorization Act for Fiscal Year  2016—that DOD certify RAF Croughton as the optimal location for the  JIAC and to determine the operational viability of Lajes Field, Azores  (Portugal) for other functions, as required by the DOD Appropriations Act,  2016. Those analyses also include key reviews that DOD conducted to  develop cost estimates for relocating the JIAC to Lajes Field, such as  EUCOM’s September 2015 JIAC Location Analysis, which included  DISA’s Azores Telecommunications Feasibility Report and the DIA’s  Azores Communications Cost Estimate; CAPE’s April 2016 Cost  Verification for the JIAC; and the DISA’s updated May 2016 review of the  JIAC Communications Infrastructure Requirements. We also reviewed  DOD’s functional requirements for joint intelligence operations centers.  Additionally, we met with DOD officials from EUCOM, DISA, the Office of  the DOD Chief Information Officer, DIA, and CAPE to determine the  context for these reviews, understand their analysis and efforts to develop  these cost estimates, and identify underlying assumptions. We also met  with Air Force officials from the 65th Air Base Group to determine the  status of personnel streamlining efforts and current housing inventory and  communications infrastructure on base, and to identify their efforts in  developing an analysis on the JIAC locating to Lajes Field. Additionally,  we received written answers from the Azorean Regional Government  Association related to local operations on the Azores islands.

We conducted this performance audit from April 2016 to November 2016  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
subsections: section_title: The JIAC Includes Elements of Three Joint Intelligence Operations Centers and Supports Other Intelligence-Related Functions
paragraphs: The JIAC includes elements of three intelligence operations centers—one  supporting EUCOM, a second supporting U.S. Africa Command, and a  third supporting the North Atlantic Treaty Organization—as well as  several other organizations that perform intelligence-related functions.  According to DOD guidance, joint intelligence operations centers support  the geographical combatant commands and other defense organizations,  serving as focal points for intelligence planning, collection management,  analysis, and production.

EUCOM Joint Intelligence Operations Center Europe executes  intelligence operations that are synchronized and integrated with  theater component, national, and partner nation organizations;  enables EUCOM planning and execution; and enhances senior  leaders’ decision-making across the entire spectrum of military  operations.

U.S. Africa Command Directorate for Intelligence at RAF Molesworth  manages and executes defense intelligence for U.S. Africa  Command, including protecting U.S. personnel and facilities,  preventing and mitigating conflict, and building defense capabilities in  order to promote regional stability and prosperity.

North Atlantic Treaty Organization Intelligence Fusion Center provides  intelligence to warn of potential crises and to support the planning and  execution of the North Atlantic Treaty Organization’s operations.

Regional Joint Intelligence Training Facility trains students from  EUCOM, U.S. Africa Command, and the North Atlantic Treaty  Organization nations, including the United Kingdom.

United States Battlefield Information Collection and Exploitation  Systems plans, builds, and operates the Coalition Intelligence and  Information Enterprise to provide on-demand coalition information- sharing solutions for both episodic and enduring missions.

section_title: DOD Organizations Involved in the JIAC Consolidation Process and Related Efforts
paragraphs: A number of DOD organizations have been involved in the JIAC  consolidation process. Overall guidance for DOD’s military construction  efforts was provided by the Office of the Assistant Secretary of Defense  for Energy, Installations, and Environment. This office is responsible for  overseeing various aspects of the department’s military construction  efforts. These responsibilities include, among other things, monitoring  the execution of the military construction program to ensure the most  efficient, expeditious, cost-effective accomplishment of the program, and  issuing guidance for the implementation of DOD military construction  policy.

Other DOD organizations—including U.S. Air Force Headquarters, the  Basing Office of the Office of the Secretary of Defense, and the  headquarters of both EUCOM and U.S. Africa Command—made up the  team that conducted the JIAC analysis of alternatives. The participating  organizations provided subject matter experts who were involved in the  team’s day-to-day work and developed the analysis that is the foundation  of the decision to consolidate the JIAC at RAF Croughton. DOD’s team  conducted work from the initial concept proposal in the fall of 2009 to the  Resource Management Decision issued by the Secretary of Defense in  April 2013.

In July 2016, we reported on DOD’s analysis of alternatives process and  recommended that the Secretary of Defense direct the Assistant  Secretary of Defense for Energy, Installations, and Environment to  develop guidance requiring the use of best practices for analysis of  alternatives—including those practices we identified in the report—and  that in this guidance, the Assistant Secretary should define the types of  military construction decisions for which use of these best practices  should be required. DOD did not agree with our recommendation,  stating that the best practices do not wholly apply to decision-making  processes for military construction projects.

Table 1 lists the roles and responsibilities of DOD components related to  the JIAC consolidation, including their involvement in preparing  information in response to congressional requests for information on the  analysis of alternatives process and on Lajes Field as a possible location  for the JIAC.

section_title: Lajes Field, Azores, Portugal
paragraphs: In response a statutory requirement, DOD issued a memorandum that  certified that Lajes Field, Azores (Portugal) was not the optimal location  for the JIAC, based on an analysis of U.S. operational requirements and  an evaluation of key criteria. The Azores is an autonomous region of  Portugal situated about 850 miles west of continental Portugal. There are  nine major Azorean islands, including Terceira, home of Lajes Field.  Lajes Field is a dual military and civilian airfield and is also a Portuguese  military base; the 65th Air Base Group, a U.S Air Force unit, is also  stationed there. The 65th Air Base Group’s mission supports DOD, allied  nations, and other authorized aircraft in transit; its core mission is to  service in-transit aircraft en route to eastern and southern destinations.

In 2010, the Air Force recommended a plan to reduce personnel and  operations at Lajes Field and divest approximately 500 U.S. military and  civilian billets, leaving approximately 165 U.S. personnel at Lajes Field to  support the mission requirements. The Secretary of Defense approved  the recommendation and announced his decision to streamline Lajes  Field in October 2012. According to the Secretary of Defense, the  frequency and volume of flights using Lajes Field had decreased, and the  base was operating well below its capacity. The Air Force recommended  reducing U.S. operations at Lajes from 24 hours, 7 days a week to 8  hours, 7 days a week and downsizing the 65th Air Base Wing to an Air  Base Group. According to the Secretary of Defense, the presence at  Lajes Field exceeded mission requirements, and the mission  requirements at Lajes Field could be supported with a smaller force. DOD  has estimated that the streamlined footprint would yield approximately  $35 million in annual savings. This Secretary of Defense’s October 2012  decision was subsequently reaffirmed by the European Infrastructure  Consolidation assessment, which the Secretary of Defense initiated on  January 25, 2013, to perform a comprehensive review of DOD facilities in  Europe.

The National Defense Authorization Act for Fiscal Year 2014 required that  the Secretary of Defense provide Congress with certification that the  actions taken to realign military forces at Lajes Field were supported by  the European Infrastructure Consolidation assessment. The act required  that DOD’s certification include an assessment of the efficacy of Lajes  Field in support of the United States’ overseas force posture. On January  6, 2015, the Secretary of Defense issued a memorandum certifying that  the European Infrastructure Consolidation assessment supported DOD’s  plan to adjust its presence at Lajes Field. Further, the Secretary of  Defense noted that DOD had conducted a comprehensive review and  determined that the reduction of U.S. personnel at Lajes Field supported  the U.S. military’s European force posture.

DOD officials said that they will continue to provide 24-hour-a-day, 7-day- a-week tower operations at Lajes Field after the base personnel are  reduced, along with crash, fire, and rescue services and attendant airfield  operations for the joint Portuguese military and civilian airfield. Air Force  officials told us that, as of July 2016, U.S. flights to Lajes Field average  two per day. Air Force officials told us that, as part of this effort to  streamline and reduce the personnel at Lajes Field, they have identified  various excess buildings, facilities, and housing units that are no longer  needed. Specifically, Air Force officials have identified 350 housing units  as excess and are in the process of returning those units back to the  Portuguese government; the remaining housing will be used to support  unaccompanied personnel at Lajes Field. In July 2016, Air Force  officials said that they were in negotiations with the Portuguese  government on the return of excess facilities per the personnel  streamlining efforts. Specifically, the officials stated that there was  disagreement on how the United States and Portugal interpret the  technical agreement and policy on the return of excess facilities—the  United States’ position was that all property excess to the needs of the  United States would be turned over to Portugal in useable condition, while  Portugal sought for the United States to demolish a majority of the  facilities rather than returning them. According to these officials, Portugal  also sought environmental remediation commitments from the United  States. According to DOD officials, the demolition and environmental  remediation of these facilities was contrary to long-standing DOD policy.

section_title: Air Force’s February 2015 Cost Estimate for the JIAC Military Construction Project Did Not Fully Meet Best Practices
paragraphs: Our assessment of the Air Force’s February 2015 cost estimate for JIAC  consolidation showed that it did not fully meet cost estimation best  practices. According to the GAO Cost Estimating and Assessment Guide,  a cost estimate created using best practices exhibits four  characteristics—it is comprehensive, well documented, accurate, and  credible. Each of the four characteristics is associated with a specific set  of best practices. Our assessment found that the JIAC cost estimate  partially met three and minimally met one of the four characteristics of a  reliable cost estimate. If any of these characteristics is not substantially or  fully met, then the cost estimate does not reflect the characteristics of a  high-quality estimate and cannot be considered reliable. Table 2 lists  each of the four characteristics, along with our summary assessment of  the JIAC cost estimate.

The following summarizes our analysis of the JIAC cost estimate for each  of the four characteristics. Appendix I provides greater detail on our  assessment.

Comprehensiveness (Partially Met). According to best practices,  agencies should develop accurate life-cycle cost estimates. Further, a  life-cycle cost estimate should encompass all past (or sunk), present,  and future costs for every aspect of the program, regardless of  funding source—including all government and contractor costs.  However, the JIAC cost estimate included only MILCON costs and did  not include costs associated with the life cycle of the project. Air Force  and Office of the Secretary of Defense officials said they do not  consider the JIAC cost estimate a life-cycle cost estimate and that the  estimate’s scope is in line with DOD guidance on the development of  budget requests for MILCON projects. According to DOD officials, the  department does not have a full life-cycle cost estimate for the entire  JIAC consolidation effort. DOD and Air Force officials further stated  that the estimate covers the costs of infrastructure for the JIAC’s  facilities, other supporting infrastructure (e.g., utilities serving the JIAC  facilities), and certain other facilities for functions related to JIAC, such  as family support (e.g., expanded capacity of the child development  center). The associated Operation and Maintenance costs (e.g.,  family support costs like living allowances) were not included in the  estimate, because they were considered out of scope, according to  DOD and Air Force officials. However, without fully accounting for life- cycle costs, management may have difficulty successfully planning  and programming resource requirements for the JIAC consolidation  and making sound decisions.

Documentation (Partially Met). According to best practices,  documentation is essential for validating and defending a cost  estimate. The JIAC cost estimate is generally consistent with the  sizing assumptions included in DOD documentation, laying out  requirements for the JIAC’s facilities, inputs from appropriate experts,  and relevant DOD guidance, such as the DOD Facilities Pricing  Guide. Also, DOD documented both the data sources and the  methodology used for the JIAC cost estimate in an Excel spreadsheet  model and a parametric cost engineering estimate summary.  However, the documentation for the JIAC cost estimate is not  complete. Specifically, the cost estimate does not provide sufficient  documentation so that a cost analyst unfamiliar with the program  could understand what had been done and replicate it. The cost  estimate uses the DOD Facilities Pricing Guide (heretofore referred to  as the Pricing Guide), which provides planning assumptions and  prices for a variety of types of facilities, such as office buildings. In the  cost calculation spreadsheet for the JIAC cost estimate, the cost  estimators’ judgements regarding which type of facility to use from the  Pricing Guide were not always consistent with the categories listed in  the Pricing Guide. For example, there is no mention of intelligence  facilities in the Pricing Guide, and we were unable to independently  trace all of the unit costs from the JIAC cost estimate back to it. Air  Force officials were able to show where in the Pricing Guide these  numbers were drawn from; however, this is not documented in the  estimating model, and there is no rationale provided to show that an  intelligence facility would be the same as a communications center.  Without a well-documented cost estimate, the Air Force may not be  able to present a convincing argument for the validity of the JIAC cost  estimate and answer decision makers’ and oversight groups’  questions.

Accuracy (Partially Met). According to best practices, the cost  estimate should provide results that are unbiased, and it should not be  overly conservative or overly optimistic. An estimate is accurate when  it is based on an assessment of most likely costs, adjusted properly  for inflation, and contains no more than a few minor mistakes—if any.  In addition, a cost estimate should be updated regularly to reflect  changes in the program, for example when schedules or other  assumptions change or actual costs change, so that it always reflects  the current status. The JIAC estimate used historical data, did not  contain mathematical errors, showed evidence of being updated,  underwent a review process before final approval, and follows DOD  construction cost estimation guidance on how to account for inflation.  However, while the JIAC cost estimate has been updated, it has not  been updated regularly. Specifically, the April 2013 estimate was  updated in June 2013 to align with an update to the Pricing Guide but  was not updated to align with the two subsequent updates to the  Pricing Guide that occurred before the February 2015 JIAC cost  estimate was submitted. Air Force officials said that the JIAC cost  estimate was updated to reflect foreign currency fluctuations.  According to these officials, the MILCON process assumes flexibility  in the project timeline to allow adjustments to the estimate and  focuses on establishing the project’s scope (e.g., the square feet or  square meters associated with a project) in the Air Force’s project  development justification forms. These Air Force officials also stated  that the costs associated with MILCON projects are updated only with  significant changes to the program and are typically permitted to be  adjusted by as much as plus or minus 25 percent of the total costs— even after funding has been appropriated—without needing to be  reprogrammed. While updating an estimate in this way may be  permissible within the established MILCON process, it is not  consistent with cost estimating best practices, because the estimate is  not updated regularly. Without updating the JIAC cost estimate on a  regular basis, DOD and the Air Force may have difficulty analyzing  changes in program costs for the consolidation project and may  hinder the collection of up-to-date cost and technical data to support  future JIAC cost estimates.

Credibility (Minimally Met). According to best practices, a cost  estimate should discuss any limitations of the analysis resulting from  uncertainty or biases surrounding data or assumptions. Major  assumptions should be varied, and other outcomes recomputed, to  determine how sensitive the cost estimates are to changes in the  assumptions. Also, risk and uncertainty analysis should be performed  to determine the level of risk associated with the estimate. Without a  sensitivity analysis that reveals how a cost estimate is affected by a  change in a single assumption, the cost estimator will not fully  understand which variable most affects the cost estimate. The use of  a sensitivity analysis is not specified in cost estimation guidance for  MILCON projects from either DOD or the Air Force, and the JIAC cost  estimate did not include such an analysis. According to Office of the  Secretary of Defense and Air Force officials, a sensitivity analysis is  part of the underlying unit cost development, because costs are  developed through the use of both historical data and industry  averages. These officials further stated that the Office of the Secretary  of Defense uses actual data underpinned by relevant sensitivity and  range analyses to develop its cost estimates. For example, Office of  the Secretary of Defense and Air Force officials said that the Office of  the Secretary of Defense uses the DOD Selling Price Index—which  averages three commonly accepted national indexes for construction  price escalation—to calculate actual project award cost data.  However, for sensitivity analysis to be useful in informing decisions,  careful assessment of the underlying risks and supporting data related  to a specific MILCON project is also necessary. In addition, the  sources of the variation should be well documented and traceable.  Without conducting sensitivity analysis for the JIAC cost estimate to  identify the effect of uncertainties associated with different  assumptions, DOD and the Air Force increase the risk that decisions  will be made without a clear understanding of the effects of these  assumptions on costs.

Another key to establishing an estimate’s credibility is its review  process. According to best practices, the estimate’s cost drivers  should be crosschecked, and an independent cost estimate  conducted by a group outside the acquiring organization should be  developed to determine whether other estimating methods produce  similar results. While the Air Force has a review process, the review it  conducted for the JIAC cost estimate did not include the use of a  checklist provided as a sample in DOD MILCON cost estimation  guidance. The sample checklist, while not required, could have  helped the Air Force to confirm the validity of assumptions and the  logic used in estimating the cost of the JIAC construction tasks. Air  Force officials stated that their review primarily looks at the numbers  provided and the ranges from the Pricing Guide to see whether the  estimate is within those ranges. These officials added that they would  use the checklist only if there was a difference from the Pricing Guide.

However, the first phase of the JIAC cost estimate did not identify the  stage of the estimate; did not separate costs for labor, equipment, or  material; and did not calculate prime and subcontractor profit by the  weighted guidelines method, which are items listed in the sample  checklist.

When we shared the results of our analysis with officials from the Office  of the Secretary of Defense, they said that they did not agree our best  practices for cost estimating were entirely applicable to the JIAC cost  estimate, since the estimate focused on MILCON costs. Furthermore,  Office of the Secretary of Defense and Air Force officials said that  construction is discussed in our Cost Estimating and Assessment Guide  as a subsidiary cost to be included in the life-cycle cost estimate. For  example, these officials said that construction costs are to be considered  as part of the overall ground rules and assumptions for a cost estimate.  However, the methodology outlined in our Cost Estimating and  Assessment Guide is a compilation of best practices that federal cost  estimating organizations and industry use to develop and maintain  reliable cost estimates, and this methodology can be used across the  federal government for developing, managing, and evaluating capital  program cost estimates, including military construction estimates.  Furthermore, DOD guidance for estimating construction costs states that  in the MILCON program, construction cost estimates are prepared  throughout the planning, design, and construction phases of a  construction project. These construction cost estimates are categorized  as follows: programming estimate, concept estimate, final estimate, and  government estimate. The Air Force provided us with the JIAC  consolidation programming estimate for analysis, because it was the most  complete and updated estimate at the time of our review. Even though  our analysis shows that the programming estimate did not meet all of the  four characteristics of a high-quality, reliable estimate, the Air Force will  have opportunities to incorporate our best practices as it prepares future  cost estimates for subsequent phases of the JIAC consolidation program.  Without incorporating a methodology that is more closely aligned with our  best practices for cost estimation and incorporates all four characteristics  of a high-quality, reliable estimate, the Air Force will not be providing  comprehensive and high-quality information for decision makers to use.

section_title: DOD Conducted Multiple Reviews of Lajes Field as a Possible JIAC Location
paragraphs: After its 2013 decision to consolidate the JIAC at RAF Croughton, DOD  conducted multiple reviews to provide information on Lajes Field as a  potential alternative location for the JIAC, in response to congressional  interest and inquiries. These reviews were developed by different  organizations within DOD during 2015 and 2016 and included both one- time and recurring costs. The reviews produced different cost estimates,  in particular for communications infrastructure and housing, because the  DOD organizations that developed the reviews used different  assumptions. However, all of the reviews found that consolidating the  JIAC at Lajes Field would be more costly than consolidating it at RAF  Croughton. Additionally, in response to statutory requirements, DOD  issued a memorandum certifying that the department had determined that  RAF Croughton was the optimal location for the JIAC and, conversely,  that Lajes Field was not the optimal location, given the JIAC’s operational  requirements.
subsections: section_title: DOD Has Conducted Multiple Reviews of Lajes Field as a Possible Location for the JIAC
paragraphs: From 2015 through 2016, DOD conducted multiple reviews of Lajes Field  as a potential location for the JIAC, in response to congressional interest  and inquiries. Lajes Field was not originally included in DOD’s analysis of  alternatives for the consolidation of the JIAC. DOD officials told us that  the reviews of 2015 and 2016 were not conducted with the same level of  rigor as a formal cost estimate, because DOD had already completed its  analysis of alternatives, and the decision to consolidate JIAC at RAF  Croughton had already been made. DOD officials also told us that no  credible new evidence had been produced to indicate the department  should revisit its initial decision. Figure 1 includes the key events and  reviews related to DOD’s analysis of the JIAC, including reviews related  to Lajes Field, the European Infrastructure Consolidation study, JIAC  consolidation, authorization, and appropriations, and the execution of the  JIAC consolidation project.

According to officials from the Office of the Secretary of Defense, DOD  did not alter or change its original decision to consolidate the JIAC at RAF  Croughton based on the results of these reviews and found that  consolidating the JIAC at Lajes Field would be more costly than  consolidating it at RAF Croughton. Additionally, according to the Deputy  Secretary of Defense, DOD’s reviews determined that Lajes Field was not  a suitable location for the JIAC, based both on operational requirements  and costs including housing availability and the lack of adequate secure  communications infrastructure. These reviews were led by EUCOM,  CAPE, and DISA.

EUCOM’s September 2015 review. EUCOM developed an analysis  comparing RAF Croughton with Lajes Field as potential locations for  the JIAC. EUCOM officials told us that this review was in response to  congressional interest and requests, and it included inputs from U.S.  Air Forces Europe and officials at Lajes Field. The review compared  cost estimates associated with locating JIAC at RAF Croughton with  those associated with locating it at Lajes Field. These cost estimates  included one-time costs, such as construction costs for the JIAC  facilities, communications infrastructure, and housing, as well as  recurring costs—including sustainment costs for base and  communications infrastructure. The review estimated the one-time  costs associated with locating the JIAC at RAF Croughton at $357  million and the one-time costs associated with locating it at Lajes Field  at $1.65 billion. For recurring costs, the review estimated that RAF  Croughton would cost approximately $68 million annually and Lajes  Field approximately $94 million annually. The largest differences  between the cost estimates were in the one-time costs for the  communications and housing infrastructure necessary to support the  JIAC. Officials from the Office of the Secretary of Defense told us that  DOD had provided this review, with its appendixes, to the House and  Senate Armed Services Committees and the House and Senate  Appropriations Committees in September 2015.

This review included two appendixes developed by DISA and DIA on the  communications infrastructure needed to support the JIAC. DISA’s July  2015 Azores Telecommunications Feasibility Report provided an analysis  of the telecommunications infrastructure on the Azores Islands. This  report indicated that the Azores did not have sufficient communications  infrastructure to be a feasible location for a DISA telecommunications  hub. The DIA Azores Communications Cost Estimate provided a brief  summary on the current and proposed communications systems within  the Azores Islands, as well as the costs associated with modernizing the  systems. The appendix noted that it was developed in response to a  request from the DIA Office of Congressional Affairs. For communications  infrastructure, the DIA estimated that locating the JIAC at Lajes Field  would require approximately $449 million in one-time costs and $32.7  million in recurring annual sustainment costs.

CAPE’s April 2016 cost verification for the JIAC. CAPE conducted an  independent review of the cost estimates presented in EUCOM’s  September 2015 review and those developed by the House  Permanent Select Committee on Intelligence for its July 2015  review. CAPE developed its own cost assumptions, which included  housing and communications infrastructure costs, among other things,  in its review of the cost calculations in the EUCOM and House  Permanent Select Committee on Intelligence reviews, which produced  alternative cost totals. CAPE officials told us that this review was in  response to direction from the Deputy Secretary of Defense and that  they briefed this review to the House Permanent Select Committee on  Intelligence in May 2016 and the House and Senate Armed Services  Committees in April 2016. CAPE’s review estimated the one-time  costs associated with locating the JIAC at RAF Croughton at $356  million and one-time costs associated with locating it at Lajes Field at  $1.43 billion (compared with EUCOM’s estimates of $357 million and  $1.65 billion respectively). For recurring costs, CAPE’s review  estimated that RAF Croughton would cost approximately $53 million  annually and Lajes Field approximately $59 million annually  (compared with EUCOM’s estimates of $68 million and $94 million,  respectively).

DISA’s May 2016 review on the JIAC communications infrastructure  requirements. In this update to its July 2015 review, DISA assessed  and compared the communications infrastructures at RAF Croughton  and Lajes Field with the intelligence mission support requirements,  including the communications and technical requirements for the  JIAC. DISA officials told us that this review included more refined cost  estimates for the communications infrastructure than prior estimates  and reflected new technical standards, such as operational bandwidth  requirements. The review found that the communications  infrastructure at Lajes Field did not meet technical and critical  infrastructure requirements. To upgrade the communications  infrastructure at Lajes Field, the review estimated a minimum of  $267.7 million in one-time costs to procure and install three undersea  cables and $6.8 million in annual sustainment costs. For locating the  JIAC at RAF Croughton, the review determined that no procurement  would be required and estimated sustainment costs for the  communications infrastructure at $5.5 million annually. DOD officials  told us that they briefed the results of this review to the House Armed  Services Committee in September 2016.

section_title: DOD’s Multiple Reviews of Lajes Field Produced Different Cost Estimates Because They Were Based on Different Assumptions
paragraphs: DOD’s multiple reviews of Lajes Field as an alternative location for the  JIAC produced different cost estimates, because these reviews relied on  different assumptions in developing the cost estimates for  communications infrastructure and housing.
subsections: section_title: Assumptions for Communications Infrastructure Costs Differed
paragraphs: DOD’s multiple reviews provide different cost estimates for the  communications infrastructure that would be needed to support the JIAC  at Lajes Field, because the reviews relied on different assumptions.  Specifically, the reviews varied in the costs they included and the number  of annual fiber cable breaks they expected would occur, among other  details. The three reviews all assumed that three new fiber cables would  be needed for Lajes Field. However, the distribution of these fiber cables  differs in the reviews. Specifically, the September 2015 EUCOM review  and the May 2016 DISA review assume one fiber cable from Lajes Field  to mainland Portugal, one fiber cable to the United Kingdom, and one  fiber cable to the United States, while the April 2016 CAPE review  assumes two fiber cables from Lajes Field to the United States and one  fiber cable to the United Kingdom. DOD officials told us that the cable  distribution cited in the September 2015 EUCOM and the May 2016 DISA  reviews reflect the current JIAC operational requirements, based on a  May-June 2015 DIA operational assessment and that the CAPE review  reflects a different JIAC operational design. Officials from DIA said that  this change in operational requirements was made in various discussion  sessions conducted among subject matter experts, and that the decision  was not documented. Figure 2 shows the current cable configuration at  Lajes Field and the new cables that would be necessary based on the  September 2015 EUCOM review, the May 2016 DISA review, and the  April 2016 CAPE review.

The May 2016 DISA review had the lowest estimate for communications  costs of the three reviews. DISA officials told us that these cost estimates  were deliberately built on assumptions that would generate the lowest  possible costs. However, DOD officials told us that DISA has not been  able to validate all of its assumptions. Table 3 shows the cost estimates  and supporting assumptions included in DOD’s multiple reviews for  communications infrastructure associated with locating the JIAC at Lajes  Field.

Appendix II contains additional information on the requirements for the  communications capabilities related to the JIAC.

section_title: Assumptions for Housing Costs Differed
paragraphs: Two of DOD’s reviews provided cost estimates for the housing needed to  support the JIAC at Lajes, but the estimates were based on different  assumptions. Specifically, EUCOM estimated the one-time housing costs  for locating the JIAC at Lajes Field at $390.5 million, while CAPE  estimated these costs at $188 million. EUCOM’s review assumed that  1,031 new housing units would be needed on the base at Lajes Field, and  CAPE’s review assumed that as few as 385 new units would be needed  on base. Additionally, EUCOM’s estimate assumes that there would be a  252-person dorm unit shortfall for unaccompanied military personnel, and  CAPE assumed a shortfall of 368 dorm units. Table 4 shows the two cost  estimates for the housing and the assumptions that each review used.

EUCOM and CAPE used different assumptions when developing their  cost estimates for the number of housing units needed to support the  JIAC at Lajes Field. Specifically, EUCOM reported that 1,812 housing  units were needed to support the JIAC and that those units would include  not only housing for the accompanied personnel working at the JIAC  (around 1,200 personnel) but also housing units for the additional base  operations and support personnel to support the JIAC (around 330  personnel) and personnel associated with the reversal of the personnel  streamlining initiative at Lajes Field (around 751 personnel). EUCOM  determined its estimate for accompanied housing units required for the  JIAC at Lajes Field by using Air Force personnel standards and JIAC  planning factors. On the other hand, CAPE’s estimates assumed that  1,260 accompanied housing units would be needed, that dorm units  would be used, and unaccompanied civilians would live off base to  minimize the effect on military family housing. A factor in the difference  between EUCOM’s and CAPE’s housing cost estimates is the addition of  751 personnel (resulting in a need for 451 additional accompanied  housing units) that EUCOM included in its cost estimate when the Lajes  personnel streamlining initiative was reversed. CAPE officials told us that  their estimate did not assume that reversing the personnel streamlining  initiative would result in the addition of so many personnel, and that  therefore there would be a reduced need for additional housing.  However, CAPE assumed that more base operations and support  personnel would be needed (CAPE assumed that 500 support personnel  would be needed, while EUCOM assumed 330) to support the JIAC at  Lajes Field.

Both reviews also provided housing estimates for unaccompanied military  personnel. EUCOM’s review assumed that 469 unaccompanied military  personnel would reside in the existing 217 dorm spaces at Lajes Field,  and there would be a shortfall of 252 dorm spaces (32 of those dorm units  would be built using the JIAC military construction funds and the other  220 units represent the shortfall). CAPE’s review assumed that  unaccompanied military personnel would reside in the existing 217 dorm  units at Lajes Field and that DOD would build two additional dorms (for  168 and 200 personnel) to accommodate the unaccompanied military  personnel. Additionally, EUCOM’s review assumed that 204  unaccompanied civilians would live on base and CAPE’s review assumed  that 91 unaccompanied civilians would live off base in small family  housing.

EUCOM’s review used the military family housing inventory of the U.S. Air  Forces in Europe to determine the number of housing units on the base  (456 housing units), while CAPE’s estimate assumes that there were 550  housing units at Lajes Field. In July 2016, Air Force personnel at Lajes  Field confirmed that there were 456 available housing units at Lajes  Field. Also, both CAPE’s and EUCOM’s reviews used the January 2007  Housing Requirements Market Analysis for Lajes Field to determine the  number of housing units that were available for rent off base  (approximately 229 housing units). However, according to information  provided by Terceira’s municipalities, there are currently 1,693 houses in  the Island of Terceira available for rent, and almost 400 were recently  occupied by U.S. military personnel and their families. According to the  2007 Housing Requirements and Market Analysis DOD conducted at  Lajes Field, the Lajes rental market is separated into two areas—housing  units that are specifically marketed to U.S. military personnel, have been  inspected for suitability, and are listed in the Lajes Field housing rental  database, and housing units that are part of the local rental market but  not of sufficient quality and without the amenities required by U.S. military  and civilian personnel. Further, EUCOM officials told us that the 1,693  housing units available on the Island of Terceira may not all be suitable  for U.S. military forces. Air Force officials told us that there were only  225 rental properties in their off-base referral database. Both EUCOM’s  and CAPE’s estimates assumed that there were 225 rental properties on  the island and that another 100 would be built (for a total of 325) to  support the personnel for the JIAC.

Both reviews assumed that no additional housing units would be  necessary at RAF Croughton, based on the 2016 Housing Requirements  Market Analysis for RAF Croughton. According to Air Force officials,  past housing and United Kingdom basing trends indicate that personnel  associated with the JIAC would live off base in the private rental market,  and the private rental market could sufficiently absorb the housing needs  of the JIAC personnel. Further, EUCOM reported that the United Kingdom  had the capacity to absorb the number of personnel associated with the  JIAC move and that they would not need to build additional military family  housing at RAF Croughton. The 2016 Housing Requirements Market  Analysis for RAF Croughton reported that the private rental market was  very active, that there was a total private rental stock of 69,364 rental  units, and that the housing supply was projected to grow to 72,905 units  by 2020.

section_title: DOD Responded to Statutory Requirements by Certifying that RAF Croughton is the Optimal Location for the JIAC
paragraphs: In addition to its multiple reviews, DOD issued a memorandum in March  2016 stating that the department had determined that RAF Croughton  remained the optimal location for the JIAC and that Lajes Air Field is not  an optimal location for the JIAC. Specifically, the Deputy Secretary of  Defense issued a memorandum in response to several requirements in  Section 2310 of the National Defense Authorization Act for Fiscal Year  2016; House Report 114-144 accompanying HR 2596, the Intelligence  Authorization Act for Fiscal Year 2016; and Section 8114 of the DOD  Appropriations Act for Fiscal Year 2016 (division C). The memorandum  states that DOD’s decision was based on an analysis of U.S. operational  requirements and an evaluation of multiple locations using five criteria:  effect on intelligence operations (critical criterion); impact on bilateral and  multinational intelligence collaboration (critical criterion); impact on  international agreements and relationships; impact on community quality  of life; and business case analysis. According to officials from the Office  of the Secretary of Defense, DOD reviewed existing analysis and did not  conduct new in-depth analysis to support the certification memorandum.  The analysis DOD used to support the memorandum was based on the  original analysis of alternatives process that DOD developed for the JIAC  consolidation—which did not include Lajes Field as an alternative  location—and on subsequent comparisons of Lajes Field and RAF  Croughton. The officials stated that no additional in-depth analysis was  warranted because no credible new evidence had been produced to  indicate the department should revisit its initial decision.

section_title: Conclusions
paragraphs: To address costly sustainment challenges and instances of degraded  theater intelligence capabilities associated with the current JIAC facilities  at RAF Molesworth, DOD plans to spend almost $240 million for the Air  Force to consolidate and relocate the JIAC’s facilities at RAF Croughton.  However, the Air Force’s cost estimate did not fully meet cost estimating  best practices that are intended, when followed, to produce high-quality,  reliable estimates. For example, the JIAC cost estimate included only  MILCON costs and did not include costs associated with the life cycle of  the project. Without fully accounting for life-cycle costs, management may  have difficulty successfully planning and programming resource  requirements for the JIAC consolidation and making sound decisions.  Furthermore, the JIAC cost estimate lacked a sensitivity analysis, which  would assess the underlying risks and supporting data. Without  identifying the effects of uncertainties associated with different  assumptions for the JIAC consolidation project, there is an increased risk  that decisions will be made without a clear understanding of these effects  on costs. Unless DOD uses best practices as it prepares future cost  estimates for the remaining design and construction phases of the JIAC  consolidation project, decision makers will not receive complete and  reliable information on the total anticipated costs for the JIAC  consolidation efforts for which they need to conduct oversight and make  informed funding decisions. Furthermore, addressing limitations in future  JIAC cost estimates can provide DOD better information to predict costs  and make informed decisions about the JIAC consolidation.

section_title: Recommendation for Executive Action
paragraphs: To better enable DOD to provide congressional decision makers with  complete and reliable information on the total anticipated costs for the  JIAC consolidation efforts, we recommend that the Office of the Assistant  Secretary of Defense for Energy, Installations, and Environment’s Basing  Office—in coordination with the Office of the Assistant Secretary of the Air  Force Installations, Environment and Energy—update future construction  cost estimates for consolidating the JIAC at RAF Croughton using best  practices for cost estimating as identified in the GAO Cost Estimating and  Assessment Guide. Specifically, cost estimates for the JIAC consolidation  should fully incorporate all four characteristics of a high-quality, reliable  estimate.

section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to DOD for review and comment. DOD  provided written comments on our recommendation, which are reprinted  in appendix III. The department also provided technical comments that we  incorporated as appropriate.

In its written comments, DOD did not concur with our recommendation.  DOD agreed that many components in the GAO Cost Estimating and  Assessment Guide are broadly applicable in the decision process leading  up to a military construction budget request. However, DOD further stated  that once military construction funds are authorized and appropriated by  Congress, the department transitions to a project management mode,  and it would be a waste of resources to continue to generate cost  estimates once they have transitioned to managing project execution  using actual cost data. However, as we note in the report, DOD guidance  for estimating construction costs, DOD’s Unified Facilities Criteria 3-740- 05, states that in the MILCON program, construction cost estimates are  prepared throughout the planning, design, and construction phases of a  construction project to account for the refinement of the project’s design  and requirements. The final estimate should document the department’s  assessment of the program's most probable cost and ensure that enough  funds are available to execute it. As of October 2016, the military  construction funds had not been authorized by Congress for the third  phase of the JIAC construction project. According to DOD officials,  construction is not scheduled to begin until fall of 2017, and the contract  has not yet been awarded.

Further, the GAO Cost Estimating and Assessment Guide states that  regardless of whether changes to the program result from a major  contract modification or an overtarget budget, the cost estimate should be  regularly updated to reflect all changes. This is also a requirement  outlined in OMB’s Capital Programming Guide. The purpose of updating  the cost estimate is to check its accuracy, defend the estimate over time,  and archive cost and technical data for use in future estimates. After the  internal agency and congressional budgets are prepared and submitted, it  is imperative that cost estimators continue to monitor the program to  determine whether the preliminary information and assumptions remain  relevant and accurate. Keeping the estimate updated gives decision  makers accurate information for assessing alternative decisions. Cost  estimates must also be updated whenever requirements change, and the  results should be reconciled and recorded against the old estimate  baseline.

Therefore, we continue to believe that DOD’s implementation of our  recommendation to update future JIAC cost estimates using the best  practices identified in the GAO Cost Estimating and Assessment Guide  would assist in ensuring that decision makers have complete and reliable  information about costs associated with the JIAC consolidation and as the  third phase of the JIAC project is authorized.  Implementing our  recommendation would also ensure that DOD develops a reliable  historical record for the cost of the JIAC that can be used to estimate  other similar projects in the future.

As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  date of this report. At that time, we will send copies of this report to the  appropriate congressional committees and to the Secretaries of Defense,  the Army, the Navy, and the Air Force; the Commandant of the Marine  Corps; and the Assistant Secretary of Defense for Energy, Installations,  and Environment. In addition, this report will be available at no charge on  the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-4523 or leporeb@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made key contributions to this report  are listed in appendix IV.

section_title: Appendix I: Assessment of the Air Force’s February 2015 Cost Estimate for the Joint Intelligence Analysis Complex
paragraphs: According to the GAO Cost Estimating and Assessment Guide, a cost  estimate is a critical element in any acquisition process to help decision  makers evaluate resource requirements at milestones and other  important decision points. Cost estimates establish and defend budgets  and drive affordability analysis. The guide identifies four characteristics of  a high-quality, reliable cost estimate: it is comprehensive, well  documented, accurate, and credible. A cost estimate is considered  comprehensive when it accounts for all possible costs associated with  a project, details all cost-influencing ground rules and assumptions, is  technically reasonable, is structured in sufficient detail to ensure that  costs are neither omitted nor double-counted, and the estimating  teams’ composition is commensurate with the assignment; well documented when supporting documentation for the estimate is  accompanied by a narrative explaining the process, sources, and  methods used to create the estimate and contains the underlying data  used to develop the estimate; accurate when the estimate is neither overly conservative nor too  optimistic and is based on an assessment of the costs most likely to  be incurred; and  credible when the estimate has been cross-checked with independent  cost estimates, the level of confidence associated with the point  estimate—the best guess at the cost estimate given the underlying  data—has been identified, and a sensitivity analysis has been  conducted. During the sensitivity analysis, the project will have  examined the effect of changing one assumption related to each  project activity while holding all other variables constant in order to  identify which variable most affects the cost estimate.

Our analysis of the Air Force’s February 2015 cost estimate for the Joint  Intelligence Analysis Complex (JIAC) showed that, when compared with  best practices, it minimally met one and partially met three of the four  characteristics of a reliable cost estimate (see table 1). According to the  GAO Cost Estimating and Assessment Guide, a cost estimate is  considered reliable if the overall assessment ratings for each of the four  characteristics are substantially or fully met.  If any of the characteristics  is not met, minimally met, or partially met, then the cost estimate does not fully reflect the characteristics of a high-quality estimate and cannot be  considered reliable.

section_title: Appendix II: Information on Communications Infrastructure Requirements for the Joint Intelligence Analysis Complex
paragraphs: The May 2016 review by the Defense Information Systems Agency  (DISA) discussed technical requirements and developed minimum  standards for developing its cost estimates for communications  infrastructure associated with locating the Joint Intelligence Analysis  Complex (JIAC) at Lajes Field. The standards support the Department of  Defense’s (DOD) position that locating the JIAC at Lajes Field would  require the procurement and installation of three undersea cables. The  DISA review stated that capabilities for global intelligence  telecommunications at the JIAC must be secure, highly available, reliable,  and redundant. The review also listed technical requirements based on  these four characteristics, none of which—according to the review—the  infrastructure at Lajes Field currently meets. One of these requirements is  a critical infrastructure protection practice from the DOD Director of  National Intelligence, which prohibits the use of communication paths that  could result in denial of service or could compromise the integrity of  information. The review characterizes DOD’s meeting this requirement at  Lajes Field as a high risk, noting that non-DOD personnel from Huawei, a  Chinese telecommunications company, could disconnect one fiber of the  two-fiber ring at Lajes Field, which would eliminate the redundancy of the  two cables and increase the risk that JIAC personnel would not be able to  use the communications infrastructure to meet their operational  requirements. The DISA review also listed a technical requirement that  the communications infrastructure be able to operate at 56 gigabits per  second, which the review noted is the minimum operational requirement  for Non-classified Internet Protocol Router Network; Secret Internet  Protocol Router Network; Joint Worldwide Intelligence Communications  System; and voice, video, and data. According to the review, the current  capabilities at Lajes Field do not meet this requirement. In comparison,  the capacity at RAF Croughton allows for 800 gigabits per second, the  capacity at U.S. Central Command also allows for 800 gigabits per  second, and the capacity at U.S. Pacific Command allows for 100 gigabits  per second. According to the review, DOD technical requirements also  specify that the communications cables must be available at 99.999  percent or higher, which equates to just a few minutes of downtime per  year. As indicated by DISA in its review, this level of availability requires  sufficient redundancy. The capabilities at Lajes Field do not meet this  requirement, according to DISA’s review. DISA officials provided us data  on the number of average fiber cable outages per week in 2015—6.8  average outages per week for transatlantic cables and 4.4 average  outages per week for Pacific cables. The frequency with which the cables  experience outages highlights the need for redundancy in fiber cable  routes. Without procuring and installing three undersea cables, Lajes  Field would not have the availability, redundancy, capacity, and security  necessary to house the JIAC.

The September 2015 review by U.S. European Command (EUCOM) also  references standards; however, it did not discuss these standards in  detail. Its appendix on communications infrastructure, developed by DIA,  says that the minimum threshold for fiber cables is two protected  pathways to mainland Europe and one to the continental United States.  Additionally—similar to the May 2016 DISA review—the EUCOM review  indicated that three new undersea systems would have to be installed at  Lajes Field to meet DOD requirements. DIA officials also told us that their  assessment was based on DOD guidance and requirements, such as the  Joint Intelligence Operations Center Enterprise Functional Requirements  document and the Chairman of the Joint Chiefs of Staff Instruction  6211.02D Defense Information Systems Network (DISN) Responsibilities,  (Jan. 24, 2012).

The review by the Office of Cost Assessment and Program Evaluation  (CAPE) did not discuss requirements or standards for the  communications infrastructure, because it relied on DISA’s previous cost  estimates. CAPE officials stated that they had deferred to DISA’s  estimate, because DISA is the authoritative source for communications  infrastructure design.

section_title: Appendix III: Comments from the Department of Defense

section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact

section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Brian Mazanec (Assistant  Director), Jennifer Andreone, Tracy Barnes, Jennifer Echard, Justin  Fisher, Joanne Landesman, Jennifer Leotta, Amie Lesser, Jamilah Moon,  Carol Petersen, and Sam Wilson made key contributions to this report.


Question-summary annotations: questions: question: What did GAO conclude in their assessment?
answer: GAO assessed the cost estimate for the military construction project to consolidate and relocate the Joint Intelligence Analysis Complex (JIAC) at Royal Air Force (RAF) base Croughton and found that it partially met three and minimally met one of the four characteristics of a reliable cost estimate defined by GAO best practices, as shown in the table below.
child_questions: question: What is one example of this finding?
answer: For example, it minimally met the credibility standard because it did not contain a sensitivity analysis; such analyses reveal how the cost estimate is affected by a change in a single assumption, without which the estimator will not fully understand which variable most affects the estimate.

question: Why is it unlikely for DOD to improve its estimates?
answer: Unless the Department of Defense's (DOD) methodology incorporates all four characteristics of a high-quality, reliable estimate in preparing future cost estimates for the JIAC construction project, it will not be providing decision makers with reliable information.

questions: question: What is the purpose of GAO's analysis?
answer: GAO was asked to review analysis associated with consolidating and relocating the JIAC. This report (1) assesses the extent to which DOD's cost estimate for the JIAC consolidation at RAF Croughton aligns with best practices and (2) describes key reviews DOD has conducted since spring of 2013 related to an alternative location for JIAC consolidation.
child_questions: question: How did GAO conduct their analysis?
answer: GAO compared the Air Force's February 2015 JIAC cost estimate with GAO best practices for developing federal cost estimates, reviewed key DOD analysis of Lajes Field as a potential alternative location for the JIAC, and interviewed DOD officials.

questions: question: What recommendation did GAO make for DOD?
answer: GAO recommends that DOD update its future construction cost estimates for consolidating the JIAC at RAF Croughton to comply with best practices for cost estimating identified by GAO.
child_questions: question: How did DOD react to GAO's recommendation?
answer: DOD did not agree, stating it would waste resources to continue to generate cost estimates once DOD transitions to managing the project with actual cost data.
child_questions: question: How did GAO react to this response?
answer: GAO continues to believe that its recommendation is valid, as discussed in this report.