Defense Infrastructure: Navy Can Improve the Quality of Its Cost Estimate to Homeport an Aircraft Carrier at Naval Station Mayport
Published: 2011-03-03T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The 2001 Quadrennial Defense Review called for the Navy to provide more warfighting assets more quickly to multiple locations. Subsequently, the Navy made a preliminary decision to homeport a nuclear-powered aircraft carrier at Naval Station Mayport, Florida, which was affirmed by the 2010 Quadrennial Defense Review. In House Report 111-491, accompanying a proposed bill for the Fiscal Year 2011 National Defense Authorization Act (H.R. 5136), GAO was directed to develop an independent estimate of the total federal costs for the proposed homeporting. GAO's objectives were to (1) develop an independent estimate of the full life-cycle costs to homeport a nuclear aircraft carrier at Mayport and (2) determine to what extent the Navy's estimate meets the characteristics of a high-quality cost estimate. To do this, GAO worked with a firm experienced in preparing life-cycle cost estimates for major federal acquisitions and compared the Navy's cost estimating practices with the best practices in GAO's "Cost Estimating and Assessment Guide."
section_title: What GAO Found
paragraphs: GAO's independent cost estimate suggests that the total one-time cost of homeporting a nuclear-powered aircraft carrier at Naval Station Mayport is expected to be between $258.7 million and $356.0 million, in base year 2010 dollars. The Navy's estimate of the one-time cost is $537.6 million--also in base year 2010 dollars--which is outside the upper range of GAO's estimate. Unlike GAO's estimate, the Navy did not conduct a risk and uncertainty analysis on its one-time costs; as a result, its estimate does not include a range. The largest difference between GAO's estimate of one-time costs and the Navy's estimate is the cost of constructing new facilities at Mayport. Based on the historical costs of constructing similar facilities, GAO estimates at the 65 percent confidence level that the cost for constructing the controlled industrial facility will be $70.5 million, and the cost for constructing the ship maintenance support facilities will be $45.6 million. The Navy estimates the construction costs to be much higher at $139.1 million and $157.2 million, respectively. Navy officials told GAO the difference is due to the increased cost involved in protecting the buildings from a potential storm surge associated with a Category 4 hurricane. GAO included a hurricane factor in its estimate to account for this increase, but GAO and the Navy used different estimating methods in developing the estimates for the construction costs. GAO used adjusted actual costs from similar construction projects, while the Navy used a detailed engineering estimate. For recurring costs, GAO's independent cost estimate suggests that the total is expected to be between $9.0 million and $17.6 million per year. The Navy's estimate of $15.3 million per year is within GAO's estimated range. The Navy's estimate did not fully meet any of the four characteristics-- comprehensive, accurate, well documented, and credible--for producing a high-quality cost estimate. Specifically, although the estimate included almost all of the life-cycle costs related to homeporting a nuclear aircraft carrier at Mayport, it partially met the criteria for being comprehensive because it does not fully describe the cost-influencing ground rules and assumptions. The estimate was only minimally accurate and well documented in that although many elements of the estimate are based on actual experiences from other comparable programs, it is difficult to say if the cost estimates are the most likely costs since the Navy did not conduct a risk and uncertainty analysis. Further, the estimate contains very little step-by-step description of how the estimate was developed so that a cost analyst unfamiliar with the program could independently replicate it. The Navy had to recreate several portions of the estimate in order to provide GAO with supporting documentation. Further, the Navy's estimate does not meet the GAO best practice for a credible estimate because it does not include a sensitivity analysis and was not compared by the Navy to an independent cost estimate conducted by a group outside the Navy. Without fully meeting the characteristics of a high-quality estimate, the Navy's ability to present a convincing argument of the estimate's affordability and credibly answer decision makers' and oversight groups' questions about the estimate is hampered.
section_title: What GAO Recommends
paragraphs: GAO recommends DOD take several actions to improve the quality of its Mayport cost estimate. DOD partially concurred with two recommendations and disagreed with one, generally stating that additional direction or change is not required. GAO believes the recommendations remain valid as discussed in the report.

section_title: Letter
paragraphs: Until the U.S.S. John F. Kennedy—a conventional aircraft carrier—was  retired in 2007, the Navy had homeported at least one aircraft carrier at  Naval Station Mayport, Florida for many decades. Prior to the U.S.S.  John F. Kennedy’s retirement, the 2001 Quadrennial Defense Review  called for the Navy to provide more warfighting assets more quickly to  multiple locations and to meet this requirement, the Navy made a  preliminary decision to homeport additional surface ships at Mayport. The  Navy subsequently prepared an environmental impact statement to  evaluate a broad range of strategic homeport and dispersal options for  Atlantic Fleet surface ships in Mayport and on January 14, 2009, issued its  decision to pursue an option that would include homeporting a nuclear- powered aircraft carrier at Mayport. The Navy’s decision was reviewed as  part of the Department of Defense’s (DOD) 2010 Quadrennial Defense  Review, which in its report supported the Navy’s decision to homeport a  nuclear-powered aircraft carrier in Mayport, indicating that homeporting  an East Coast carrier in Mayport would contribute to mitigating the risk of  a terrorist attack, accident, or natural disaster. Additionally, officials in  DOD's Office of Cost Assessment and Program Evaluation, which was part  of the review's working group assessing the Navy's Mayport decision,  reviewed the Navy's presentations and supporting documentation, but did  not develop an independent cost estimate for comparison.
In House Report 111-491, accompanying a proposed bill for the Fiscal  Year 2011 National Defense Authorization Act (H.R. 5136), the House  Committee on Armed Services directed GAO to submit a report no later  than February 15, 2011, that contains an independent estimate of the total  direct and indirect costs to be incurred by the federal government for the  proposed homeporting at Naval Station Mayport. In response to this  report, our objectives were to (1) develop an independent estimate of the  full life-cycle costs associated with the Navy’s plan to establish a second  East Coast homeport for a nuclear-powered aircraft carrier at Naval  Station Mayport and determine to what extent it differs from the Navy’s  estimate and (2) determine to what extent the Navy’s estimate meets the  characteristics of a high-quality cost estimate.
To develop the independent cost estimate, we worked with an outside cost  estimating consulting company with expertise in preparing estimates of  the life-cycle costs of major federal acquisitions. We followed the 12 steps  outlined in the GAO Cost Estimating and Assessment Guide, which have  been identified as best practices by GAO and experts in the cost estimating  community, to develop a risk-adjusted range of cost estimates. We  compared our risk-adjusted range of estimates, at an 80 percent  confidence interval, to the Navy’s point estimates to determine whether  the Navy’s estimates were within our risk-adjusted range. For a direct  element-by-element comparison between our estimate and the Navy’s, we  also compared our estimates at the 65 percent confidence level with the  Navy’s point estimates.Additionally, we reviewed the Navy’s cost  estimate and interviewed Navy officials with knowledge of the  methodology and key underlying assumptions used in developing the  Navy’s estimate. We reviewed the Navy’s environmental impact study  used in evaluating options for homeporting surface ships at Mayport. We  also obtained background information about nuclear-powered aircraft  carrier maintenance, visited locations where nuclear-powered aircraft  carrier maintenance is performed, and spoke with Navy officials  knowledgeable about the personnel and facilities requirements for  nuclear-powered aircraft carrier maintenance. From this information we  developed a list of individual cost elements and used standard cost  estimating methodologies to estimate the cost of each element. We also  conducted sensitivity and risk and uncertainty analyses on our estimate.  We compared our estimate with the Navy’s and interviewed Navy officials  to determine the reasons for any differences. Through document reviews  and interviews with agency officials knowledgeable about the Navy’s cost  data and the systems that produced them and the internal controls used to  maintain the integrity of the data, we determined the data were sufficiently  reliable for our purposes.
To determine to what extent the Navy’s estimate has the characteristics of  a high-quality cost estimate, we compared the Navy’s cost estimating  practices against those outlined in the GAO Cost Estimating and  Assessment Guide. We analyzed the extent to which the Navy’s cost  estimate was (1) comprehensive, (2) accurate, (3) well documented, and  (4) credible—the four key characteristics of high-quality cost estimates  and assigned each characteristic a rating of not met, minimally met,  partially met, substantially met, or met. More detailed information about  our scope and methodology is provided in appendix I.
We conducted this performance audit from July 2010 to February 2011 in  accordance with generally accepted government auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings, and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
subsections: section_title: Background
subsections: section_title: Planned Infrastructure and Facility Improvements at Naval Station Mayport to Support a Nuclear- Powered Aircraft Carrier
paragraphs: The U.S. Navy currently maintains 11 nuclear carriers—5 homeported on  the West Coast of the United States, 5 on the East Coast of the United  States, and 1 in Yokosuka, Japan. Of the West Coast carriers, 3 are  homeported in San Diego, California; 1 in Bremerton, Washington; and 1 in  Everett, Washington. All 5 East Coast carriers are homeported in Norfolk,  Virginia. As some of the most technologically advanced ships in the world,  nuclear-powered aircraft carriers require continuous and regularly  scheduled maintenance to keep them ready to meet mission requirements.
Naval Station Mayport has served as a conventional aircraft carrier  homeport at various times since the 1950s, including homeporting two  conventional carriers at one time, but it has not served as homeport for a  nuclear-powered aircraft carrier. Consequently, to homeport a nuclear  powered aircraft carrier, several major infrastructure upgrades are  required at Mayport. These infrastructure improvements include  construction of four new buildings: (1) controlled industrial facility,  (2) ship maintenance facility, (3) maintenance support facility, and  (4) parking garage. The controlled industrial facility would be used for the  inspection, modification, and repair of radiologically controlled equipment  and components associated with naval nuclear propulsion plants. The ship  maintenance facility is necessary to perform nonnuclear maintenance on  the carrier’s nuclear propulsion plants. The maintenance support facility  would provide the needed administrative and technical staff offices to  support propulsion plant maintenance and a central area for receiving,  inspecting, shipping, and storing materials. The parking garage would  replace parking spaces lost through construction of the three new  maintenance buildings. The infrastructure upgrades also include  improvements to Wharf F, which would receive improvement to its  structural components, berthing capability, and utility and security  systems in order to make it capable of homeporting a nuclear carrier.  Additionally, traffic improvements would be made to the Massey Avenue  corridor to rebuild a boulevard-style roadway with two lanes each way  divided by a grass median. Dredging would also be necessary to deepen  the Mayport turning basin to allow access for a fully loaded nuclear  aircraft carrier. Figure 1 shows the location and expected time line for  completion of the planned infrastructure and facility improvements at  Naval Station Mayport.
Massey Ave.
section_title: Navy’s Estimate for Establishing a Nuclear- Powered Aircraft Carrier at Naval Station Mayport
paragraphs: The Navy’s estimate of one-time and recurring costs for establishing a  nuclear-powered aircraft carrier homeport at Naval Station Mayport is  contained in two DOD information papers dated February 25, 2010, and  August 5, 2010. The February 2010 information paper was produced by  the Navy’s Office of the Chief of Naval Operations in response to a request  from the Congressional Research Service and provided only the one-time  costs of the carrier homeport. Those one-time costs were primarily drawn  from a series of DD Form 1391s, which DOD uses as budget justification  material to support funding requests for military construction. However in  addition to military construction costs, there are also one-time costs  associated with planning and design, the initial equipment outfitting for  the new maintenance facilities to be built in Mayport, and the permanent  change of station cost for the relocation of the carrier crew to Mayport.
The August 2010 information paper was also prepared by staff working for  the Chief of Naval Operations, in response to questions from GAO about  the recurring costs for the planned homeporting. This information paper  contained estimates of the recurring costs associated with the carrier  homeport. There were nine recurring costs listed in the information paper  including: (1) permanently assigned labor for the new maintenance  facilities, (2) base operating support, (3) facilities sustainment,  (4) facilities restoration and modernization, (5) transportation of  equipment for the carrier air wing, (6) travel and per diem for public  shipyard workers traveling from Norfolk to Mayport to work on the  carrier’s nuclear propulsion systems, (7) the additional biennial  maintenance dredging needed to allow access for a fully loaded nuclear  aircraft carrier, (8) the differential between Norfolk and Mayport in basic  allowance for housing costs, and (9) the differential between Norfolk and  Mayport in utilities costs. The recurring costs do not represent the total  cost of operating a nuclear-powered aircraft carrier. Rather, they are the  subset of costs associated with nuclear carrier maintenance in which there  is a cost differential between conducting the maintenance in Mayport and  conducting the maintenance in Virginia.
On December 28, 2010, the Chief of Naval Operations transmitted a report  to the congressional defense committees that essentially consolidated the  estimates provided in the two information papers. However, the report  differed from the information papers in its treatment of inflation for one- time costs and in the total for the one-time permanent change of station  cost. The February 2010 information paper showed the one-time military  construction costs as budgeted amounts in then-year dollars, which  include the effects of inflation, while the December 2010 report showed al one-time cost in base year 2010 dollars, which exclude the effects of  inflation and use 2010 prices throughout the life cycle. The one-time  permanent change of station cost was reduced from $26 million to  $10 million in the December 2010 report. The report did not provide any  additional description or documentation that was not contained in t DOD information papers.
section_title: Characteristics of a High- Quality Cost Estimate
paragraphs: A reliable cost estimate is critical to the success of any program because it  can provide the basis for informed investment decision making, realistic  budget formulation and program resourcing, meaningful progress  measurement, proactive course correction when warranted, and  accountability for results. In March 2009, GAO issued a cost estimating  and assessment guide that compiled cost estimating best practices drawn  from across industry and government. Specifically, the guide includes four  characteristics of a high-quality cost estimate by which an organization’s  management can assess cost estimates in making informed decisions. A  cost estimate is:  e when it accounts for all possible costs associated with  a project, is structured in sufficient detail to ensure that costs are  neither omitted nor double-counted, and the estimating teams’  composition is commensurate with the assignment;    well documented when supporting documentation is accompani narrative explaining the process, sources, and metho the estimate and contains the underlying data used to develop the  estimate;    accurate when it is based on an assessment of the costs most likely d and not overly conservative or too optimistic; and     credible when it has been cross-checked with an independent cost  estimate, the level of confidence associated with the point estim been identified through a risk and uncertainty analysis, and a  sensitivity analysis has been conducted—that is, the project has  ate has  examined the effect of changing one assumption related to eac activity while holding all other variables constant in order to iden which variable most affects the cost estimate.
In addition, GAO’s cost estimating and assessment guide l steps that, when followed, should result in high-quality cost estimates that  are comprehensive and accurate and that can be easily and clearly traced ,  replicated, and updated. The guide also contains hundreds of best  practices drawn from across industry and government for carrying out  these steps. Table 1 shows these 12 key steps and how they relate t four characteristics of a high-quality cost estimate.
section_title: GAO’s Independent Cost Estimate Is Lower Than the Navy’s Estimate for Homeporting a Nuclear Aircraft Carrier at Naval Station Mayport
paragraphs: Our independent estimate of the total cost for homeporting a nuclear- powered aircraft carrier at Naval Station Mayport is lower than the Navy’s  estimate. Specifically, the Navy’s estimate of one-time costs is outside the  upper range of our estimate, while its estimate of recurring costs falls  within the upper range of our estimate.
subsections: section_title: GAO Estimates One-Time Costs to Be Between $258.7 Million and $356.0 Million, Lower Than the Navy’s Estimate
paragraphs: Our independent cost estimate suggests that the total one-time cost of  homeporting a nuclear-powered aircraft carrier at Naval Station Mayport  will be between $258.7 million and $356.0 million, in base year 2010  dollars. The Navy’s estimate of the one-time costs is $537.6 million, also in  base year 2010 dollars, which is outside the upper range of our estimate.  Unlike our estimate, the Navy did not conduct a risk and uncertainty  analysis on its one-time costs; as a result, its estimate does not include a  range. Reporting a range of costs around a point estimate is useful to  decision makers because it conveys a level of confidence in achieving the  most likely cost.
Table 2 shows a comparison between our estimated range and the Navy’s  estimate for one-time costs. Specifically, the table shows our estimated  range at an 80 percent confidence interval and whether the Navy’s  estimate falls into that range. The low value of the estimate range  ($258.7 million) represents a 10 percent chance that the cost will be that  amount or less, and the high value of the estimated range ($356.0 million)  represents a 90 percent chance that the cost will be that amount or less.  The last column in the table identifies whether the Navy’s estimate is  within our estimated range.
Our initial point estimate of the one-time dredging cost based on historical  data was $44.3 million, which was $2 million less than the Navy’s estimate.  However, in accordance with the best practices recommended by the GAO  Cost Estimating and Assessment Guide, we have updated our estimate  based on the $26.6 million contract awarded in August 2010. Our estimated  range now reflects the contract award price plus the Navy’s overhead and  contingency amounts. Navy officials attributed the lower than estimated  contract costs to supply and demand within the private dredging sector,  which because of lower demand has caused firms to compete at lower  cost.
We also compared the Navy’s point estimate to our estimate at the  65 percent confidence level. A confidence level is the chance that the  actual costs will be equal to or less than the adjusted amount. In table 2,  we reported the 10 percent confidence level estimate as our low estimate  range and the 90 percent confidence level estimate as our high estimate  range. For an element-by-element comparison of our estimate with the  Navy’s point estimate, we chose the 65 percent confidence level since we  consider this confidence level to be an appropriate budgeting range.
Experts agree that program cost estimates should be budgeted to at least  the 50 percent confidence level, but budgeting to a higher level, such as  70 percent to 80 percent, or the mean, is now a common practice.
Table 3 shows our 65 percent confidence level estimate in comparison to  the Navy’s point estimate. To facilitate comparisons against the Navy’s  estimate, the one-time costs are expressed in base year 2010 dollars, which  represent amounts based on 2010 prices, with the impact of inflation  removed. While useful for comparisons against the Navy’s estimate, base  year 2010 dollars should not be used as the basis for budgetary decisions.   In order to support a budgetary amount, base year 2010 dollars would  need to be converted into then-year dollars.
Table 3 shows the largest difference between the two point estimates is in  the construction of the controlled industrial facility and the ship  maintenance support facilities, where the Navy’s estimates were  $68.3 million and $111.6 million higher than our 65 percent confidence  level estimates, respectively. The Navy’s estimate of these costs is based  on a detailed engineering study developed by an engineering firm. In  contrast, our estimate is based on historical costs observed for analogous  facility construction projects at Naval Air Station North Island in San  Diego in 1995. These analogous costs are adjusted to account for  differences in size, inflation, geographical differences between the two  areas, and any identified complexities or requirements unique to the  Mayport construction. These factors were not present in the San Diego  projects.  One known complexity of the Mayport construction projects is  that the controlled industrial facility must be built to withstand a storm  surge from a category 4 hurricane. Our estimate includes a factor of  10 percent of the total construction cost to account for hurricane-related  construction complexities. Other complexity factors included  environmental considerations and costs associated with antiterrorism and  force protection that were not required for the construction at Naval  Station North Island. We included both of these factors in our estimate  based on a percentage of total construction cost. Neither GAO nor the  Navy has identified any other specific complexity factors that would  account for the significant discrepancy between the two estimates.
Additionally, other elements of the Navy’s estimate were higher than our  own. Specifically, the Navy’s estimate of initial equipment outfitting was  $24.6 million higher than ours and the Navy’s estimate of the permanent  change of station cost for relocating the carrier crew was $4.5 million  higher than ours. Finally, the Navy’s planning and design estimate was  $10.3 million higher than ours. Both we and the Navy estimated planning  and design as a percentage of the overall construction cost. The  $10.3 million difference between the Navy’s estimate and ours results from  the Navy’s higher estimates for construction of the parking garage and  controlled industrial facility and ship maintenance support buildings.  Appendix II includes our complete cost estimate and explains in more  detail our cost estimating results for each element of the one-time costs.
section_title: GAO Estimates Recurring Costs to Be Between $9.0 and $17.6 Million Per Year, and the Navy’s Estimate Falls Within This Range
paragraphs: Our independent cost estimate indicates that the annual recurring cost of  homeporting a nuclear-powered aircraft carrier at Naval Station Mayport  is expected to be between $9.0 million and $17.6 million each year, and the  Navy’s $15.3 million per year estimate of the annual recurring cost is  within the range of our estimate. Because the Navy did not conduct a risk  and uncertainty analysis on its recurring costs, its estimate does not  include a range.
Table 4 shows a comparison between our estimated range and the Navy’s  estimate for recurring costs. Specifically, the table shows our estimate  range at an 80 percent confidence interval and whether the Navy’s  estimate falls into that range. The low value of the estimated range  ($9.0 million) represents a 10 percent chance that the cost will be that  amount or less, and the high value of the estimated range ($17.6 million)  represents a 90 percent chance that the cost will be that amount or less.  The last column in the table identifies whether the Navy’s estimate is  within our estimated range.
As we did with one time costs, we also compared our 65 percent  confidence level estimates with the Navy’s point estimates for a direct  element-by-element comparison between our estimate and the Navy’s, as  shown in table 5.
The largest difference between the two estimates is in the cost of facilities  restoration and modernization, where the Navy’s estimate is $3.9 million  higher. Since the Navy determined this cost as a percentage of total  building construction costs for their estimates, the Navy’s higher cost  results from its higher estimate for construction of the controlled  industrial facility and the ship maintenance support buildings. Some of the  recurring cost elements, such as the permanently assigned labor for  nuclear facilities and base operating support, are slightly higher in our  estimate than the Navy’s estimate. For these cost elements we used the  Navy’s estimated costs because we did not have access to a source of  information from which we could estimate the costs independently from  the Navy. Although we used the same estimate as the Navy, our numbers  at the 65 percent confidence level are slightly higher than the Navy’s  because they are risk-adjusted, whereas the Navy’s point estimates have  not been adjusted for risk. In using the Navy’s calculations, we reviewed  the Navy’s ground rules and assumptions to the extent possible based on  the information that the Navy provided to us.
In addition, we included two additional recurring costs in our estimate  that were not included in the Navy’s estimate—the additional travel costs  that the private sector may incur if they have to bring in workers from  outside of the Mayport area and the permanent change of station for the  aircraft carrier crew during the 10.5-month docking maintenance  availabilities and the 3-year midlife refueling overhaul. The recurring  costs have been annualized over the 50-year carrier life cycle. Consistent  with the Navy’s estimate, we have not identified the recurring costs as  beginning in any particular fiscal year. Appendix II provides our complete  cost estimate and explains in greater detail our cost estimating results for  each element of recurring costs. It also contains additional information on  the differences between GAO’s and the Navy’s estimates.
section_title: Navy’s Estimate Does Not Fully Meet Any of the Four Characteristics for Producing a High- Quality Cost Estimate
paragraphs: The Navy’s estimate did not fully meet any of the four characteristics— comprehensive, accurate, well documented, and credible—for producing a  high-quality cost estimate. Specifically, the Navy’s estimate partially met  the criteria for being comprehensive; however, the estimate was minimally  accurate and well documented and overall, did not meet the criteria for  being credible.
subsections: section_title: The Navy’s Estimate Is Partially Comprehensive
paragraphs: The Navy’s estimate partially meets the criteria for being comprehensive.  Specifically, it includes almost all of the life-cycle costs related to  homeporting a nuclear-powered aircraft carrier at Naval Station Mayport.  The Navy did not include two small recurring costs: (1) the additional  travel costs that the private sector may incur if they have to bring in  workers from outside of the Mayport area and (2) the permanent change  of station for the carrier crew during the 10.5-month docking planned  incremental availability and 39-month midlife refueling overhaul. However,  the estimate does not fully document the cost-influencing ground rules  and assumptions. Additionally, the Navy only minimally provided a cost  element breakdown structure in its estimate, and in conducting our  assessment, we had to develop our own more robust structure to ensure  that cost elements were neither omitted nor double counted. Individual  segments of the Navy’s cost estimate were completed by separate Navy  teams and then pieced together into two documents that represent the  Navy’s cost estimate. According to Navy officials, there was no single  person or team in charge of ensuring that the entire Mayport homeporting  estimate was packaged into a comprehensive and well documented final  estimate.
According to GAO’s guide, a cost estimate is comprehensive when it  includes both government and contractor costs of the program over its full  life cycle, from inception of the program through design, development,  deployment, and operation and maintenance to retirement of the program.  It should also completely define the program, reflect the current schedule,  and be technically reasonable. Comprehensive cost estimates should be  structured in sufficient detail to ensure that cost elements are neither  omitted nor double counted. Specifically, the GAO guide states that the  cost estimate should be based on a product-oriented work breakdown  structure that allows a program to track cost and schedule by defined  deliverables, such as hardware or software components. Finally, where  information is limited and judgments must be made, the cost estimate  should document all cost-influencing ground rules and assumptions.  Appendix III provides our assessment of the extent to which the Navy’s  estimate met the best practices for the four characteristics of a high- quality cost estimate—comprehensive, well documented, accurate, and  credible—described in the GAO Cost Estimating and Assessment Guide.
section_title: The Navy's Estimate Is Minimally Accurate
paragraphs: The Navy’s estimate is minimally accurate. Some elements of the estimate  are based on a historical record of cost estimating and actual experiences  from other comparable programs. Many of the recurring costs—such as  the facilities sustainment, operations, and the biennial dredging costs—are  based on a historical record of past costs. However, no evidence was  provided by the Navy to show that a risk and uncertainty analysis was  conducted. Therefore, the confidence level of the estimate can not be  determined, and it is difficult to say if the cost estimates are the most  likely costs. In addition, the initial cost estimate provided to us by the  Navy contained calculation errors, although the Navy identified the errors  after discussions with GAO and corrected the errors in subsequent  versions of its estimate.
An estimate is considered accurate when it is based on an assessment of  the costs most likely to be incurred, according to GAO’s cost estimating  guide. The results should be unbiased, that is not overly conservative or  optimistic. Additionally, the estimate should be based on a historical  record of cost estimating and actual experiences from other comparable  programs. The Navy based its estimate of construction costs on a detailed  engineering estimate produced by a private architecture and engineering  firm, but prior to GAO’s review, the Navy had not reviewed the firm’s  engineering estimate to verify the accuracy and completeness of the  estimate and did not check whether the engineering estimate relied on  overly conservative assumptions or fell within the range of historical  costs. Without ensuring that its military construction estimates are  accurate, the Navy may be overestimating the costs of new construction at  Mayport, which could result in the Navy requesting more funding than is  needed and limit the amount of money available to fund other projects.
section_title: The Navy’s Estimate Is Minimally Well Documented
paragraphs: Although the Navy’s documentation contains some discussion of the  technical information used in the estimate, the Navy’s estimate is  minimally well documented. The documentation contains very little step- by-step description of how the estimate was developed so that a cost  analyst unfamiliar with the program could understand what was done and  replicate it. Additionally, the documentation does not capture the source  data used, the reliability of the data, and how the data were standardized.  Moreover, Navy officials had to recreate several portions of the estimate to  provide us with supporting documentation.
Cost estimates, according to GAO’s cost estimating and assessment guide,  are well documented when they can be easily repeated or updated and can  be traced to original sources. The documentation should explicitly identify  the primary methods, calculations, assumptions, and sources of the data  used to generate each cost element. The estimating process should be  described and an explanation provided for why particular methods and  data sets were chosen and why these choices are reasonable. All the steps  involved in developing the estimate should be documented so that a cost  analyst unfamiliar with the program could recreate it with the same result.
In addition, documentation for the cost estimate should reflect changes in  technical or program assumptions or new program phases or milestones.  Insufficient documentation makes it difficult for outside organizations to  validate the Navy’s life-cycle cost estimate for homeporting an aircraft  carrier at Naval Station Mayport. Not having adequate documentation also  impedes the Navy’s ability to develop future estimates and facilitate  oversight by Congress.
section_title: The Navy's Estimate Does Not Meet GAO's Best Practices for a Credible Estimate
paragraphs: Overall, the Navy's estimate does not meet the GAO best practices’ criteria  for a credible estimate, because it does not include a sensitivity analysis  and was not compared by the Navy to an independent cost estimate  conducted by a group outside the acquiring organization. Although DOD's  Office of Cost Assessment and Program Evaluation did review the Navy's  presentations and supporting documentation in conjunction with the 2010  Quadrennial Defense Review working group, the office only performed a  general review of the Navy's costs, not a full independent cost estimate  with a sensitivity analysis. Further, while the Navy did identify some risks,  it did not conduct a formal risk and uncertainty analysis that would  identify a level of confidence in the estimate. In addition, although some  cost elements in the Navy’s estimate were derived from models that have  undergone validation, verification, and accreditation, others were not. For  example, the model that was used to develop the estimate of the recurring  travel cost for the maintenance workers who would travel from Norfolk  Naval Shipyard to work on the carrier’s nuclear propulsion systems in  Mayport underwent a documented validation, verification, and  accreditation process. Similarly, the DOD’s facilities sustainment model  that was used to calculate the recurring facilities sustainment costs was  also validated, verified, and accredited. However, we did not find any  evidence that any validated, verified, or accredited model was used for  calculating the costs of transporting equipment for the carrier air wing, the  cost of the differential in the basic allowance for housing between Norfolk  and Mayport, the cost of the differential in utilities charges between  Norfolk and Mayport, or the cost of the permanent change of station for  the aircraft carrier crew. In terms of risk analysis, only the military  construction estimates contained any consideration of risk, in the form of  contingency factors.
Moreover, we found that the Navy has not assigned a single office with the  responsibility for ensuring that the overall estimate for Mayport  homeporting was packaged into a comprehensive and well documented  final estimate and that a sensitivity analysis was conducted on the  estimate to determine which cost elements, and which factors of those  cost elements, most influenced the overall cost. When an organization  does not assign a single office responsibility for assembling the estimate  into a comprehensive package, it is difficult to conduct proper sensitivity  and risk analyses. Additionally, different offices preparing parts of the  estimate may use conflicting assumptions. For example, different offices  may use different assumptions regarding the treatment of inflation.  Some  costs may be expressed in base year dollars, while others are expressed in  then-year dollars.  This was the case with the Navy’s original estimates  which showed one-time costs in then-year dollars and recurring costs in  base year 2010 dollars.  Even if amounts are all expressed using the same  type of dollars, different contributors may use different inflation indexes,  resulting in conflicting underlying assumptions.
According to GAO’s guide, cost estimates are credible when    major assumptions have been varied and other outcomes recomputed  to determine how sensitive outcomes are to changes in the  assumptions,  risk and uncertainty analyses have been performed to determine the  level of risk associated with the estimate, and  the estimate’s results have been cross-checked and an independent  cost estimate has been developed to determine whether other  estimating methods produce similar results.
Sensitivity and risk analyses should be included in all cost estimates as a  best practice because all estimates have some uncertainty. A sensitivity  analysis addresses some of the estimating uncertainty by testing discrete  cases of assumptions and other factors that could change. By examining  each assumption or factor independently, while holding all others  constant, the cost estimator can evaluate the results to discover which  assumptions or factors most influence the estimate. However, because  many parameters could change at the same time, a risk analysis should  also be performed to capture the cumulative effect of additional risks. Risk  analysis adds to the credibility of a cost estimate because it quantifies the  uncertainty and provides a level of confidence associated with the point  estimate. The results of a high-quality, reliable cost estimate should also be  cross-checked, and an independent cost estimate should be developed to  determine whether other estimating methods produce similar results. An  independent cost estimate is considered to be one of the most reliable  validation methods and is typically performed by organizations higher in  the decision-making process, or entirely outside of the office performing  the baseline cost estimate, using different estimating techniques and,  where possible, different data sources from those used to develop the  baseline cost estimate.
Performing a sensitivity analysis for the Mayport aircraft carrier estimate  would help to identify and focus on key elements with the greatest effects  on cost and understand the potential for cost growth and the reasons for  it. Additionally, because the Navy has not conducted a risk and uncertainty  analysis, it is unable to provide Congress with a reasonably high  confidence level for its Mayport aircraft carrier cost estimate. Performing  a risk and uncertainty analysis would enable the Navy to quantify the risk  and uncertainty associated with the cost model; provide a level of  confidence for its cost estimate; and give decision makers perspective on  the potential variability of the cost estimate should facts, circumstances,  and assumptions change.
Navy officials told us that short time frames and budgetary constraints  prevented them from following the criteria for comprehensiveness,  accuracy, documentation, and credibility in developing their estimate.  For elements of the recurring costs, such as the travel and per diem for  public shipyard workers, the estimates were produced in response to  different congressional requests for information often requiring quick  turnaround—in some cases as short as 3 days. Navy officials told us that in  the case of the military construction estimates, their intention was not to  develop budget quality estimates, but rather an estimate more suitable for  initial programming and planning purposes. Navy officials noted that this  is a typical approach for construction projects that are not scheduled to  begin for several years. According to the Navy, it would have been cost  prohibitive to develop budget quality estimates for every proposed military  construction project. Navy officials stated that the Navy further reviews  and refines the construction estimates as they proceed through the DOD  budget process. Because many of the projects planned for Mayport would  be included in future budgets, these projects have not yet undergone this  review. Consistent with prior experience, it may not have been feasible for  the Navy to follow cost estimating best practices to the fullest extent in  developing initial estimates, especially given the short time frames often  available. However, as the Navy continues to respond to congressional  interest in the costs associated with homeporting a nuclear carrier at  Mayport, it has the opportunity to more completely incorporate cost  estimating best practices in its revised estimate, thereby improving the  estimate’s quality and making it easier to defend in future budgets and  decision making.
section_title: Conclusions
paragraphs: The Navy’s ability to produce a comprehensive, accurate, well  documented, and credible cost estimate for homeporting a nuclear- powered aircraft carrier at Naval Station Mayport will continue to be  hampered until it makes certain fundamental changes to the process it  uses to develop, document, and update its overall estimate of Mayport  homeporting costs. Specifically, without full documentation of the data  sources, assumptions, and calculation methods it uses, the Navy cannot  assure that its estimate can be validated or defended or any differences  between estimated and actual costs can be explained—an important step  in improving and updating the estimate. Additionally, without detailed  documentation that describes how the estimate was derived, the Navy can  neither present a convincing argument of the estimate’s affordability, nor  credibly answer decision makers’ and oversight groups’ questions about  specific details in the estimate. Further, without conducting sensitivity and  risk and uncertainty analyses on its cost estimate, the Navy is unable to  identify and focus on major cost drivers, analyze the potential for cost  growth, and quantify the risk and uncertainty associated with the cost  estimate. Moreover, without a comprehensive, accurate, well documented,  and credible cost estimate, Congress cannot have reasonable confidence  that it has a complete understanding and an accurate and realistic  determination of the projected costs to evaluate and make decisions on  the Navy’s planned homeporting of a nuclear-powered aircraft carrier at  Mayport.
section_title: Recommendations for Executive Action
paragraphs: To improve the Navy’s life-cycle cost estimate for the planned  homeporting of a nuclear-powered aircraft carrier at Naval Station  Mayport, Florida, we recommend that the Secretary of Defense direct the  Secretary of the Navy to take the following three actions to incorporate to  a greater extent the best practices identified by GAO for developing a high- quality cost estimate in future revisions of its Mayport nuclear carrier  homeporting cost estimate as part of the annual budgetary process or in  response to future congressional requests:  1.  To improve the comprehensiveness of its cost estimate, the Navy  should    include all potential recurring costs, and     clearly describe the ground rules and assumptions underlying the  estimation of each cost element;  2.  To improve the quality and transparency of the Navy’s estimate, the  Navy should thoroughly document the life-cycle costs associated with  homeporting a nuclear-powered aircraft carrier at Naval Station  Mayport. Specifically, documentation should   identify the source data used, their reliability, and how the data  were normalized,   describe the steps used in developing the overall estimate so that it  can be clearly understood and easily replicated, and    describe in sufficient detail the estimating methodology and  calculations performed to derive each element’s cost; and  3.  To improve the accuracy and credibility of its cost estimate, the Navy  should assign a single office with the responsibility for assembling the  overall estimate into a comprehensive and well documented package  and for performing a sensitivity and risk and uncertainty analyses on  the overall estimate to identify the    major cost drivers,    extent to which estimates could vary due to changes in key cost  assumptions, and  level of confidence in the estimate.
section_title: Agency Comments and Our Evaluation
paragraphs: In written comments on a draft of this report, DOD partially concurred  with two and nonconcurred with one of our three recommended actions.  DOD also provided technical comments that we have incorporated into  this report where applicable. DOD’s written comments are reprinted in  appendix IV.
DOD partially concurred with our two recommendations that the  Secretary of Defense direct the Secretary of the Navy to improve the  (1) comprehensiveness of the Navy’s life-cycle cost estimate for the  planned homeporting of a nuclear-powered aircraft carrier at Naval  Station Mayport by including all potential recurring costs and clearly  describing the ground rules and assumptions underlying the estimation of  each cost element and (2) quality and transparency of the Navy’s estimate  by thoroughly documenting the life-cycle costs associated with the  planned homeporting. DOD stated that the elements, or cost estimating  best practices, cited in our recommendations may contribute to improved  quality in cost estimates, but they may not be ideally suited to the myriad  situations in which the Navy is required, or may be requested, to provide  cost estimates to Congress.
We recognize that it may not have been feasible for the Navy to always  follow cost estimating best practices to the fullest extent in developing  some of its initial estimates, especially when the preparation time for some  estimates may have been limited. However, both recommendations were  directed at improving future Navy cost estimates associated with the  Mayport carrier homeporting. As the Navy continues to respond to  congressional interest, we believe it has the opportunity to more  completely incorporate the cost estimating best practices we identified in  our report in its revised estimates for Mayport in order to improve the  quality of its estimates. Additionally, DOD states that the similarity  between our initial estimate and the Navy’s budget estimate for dredging  costs suggests that the Navy’s programming/budgeting process leads to  high-quality, credible cost estimates. However, the Navy’s overall Mayport  estimate currently includes several cost elements that have not been  refined through a programming/budgeting process. Furthermore, as  described in our report, GAO’s criteria for assessing high-quality, credible  estimates involve factors that go beyond the accuracy of any individual  point estimate. The convergence between our estimate and the Navy’s  estimate on a single cost element is not an indication by itself of the  quality of the Navy’s overall estimate for homeporting a nuclear carrier at  Mayport. Improving the Mayport estimate’s quality will allow the Navy to  more easily defend its plans in future decisionmaking and budgets within  DOD. We believe that in order to best inform the budget decisionmaking  process of the likely costs, affordability, and scheduling of funding needed  to support the Navy’s plans for Mayport, DOD should take every available  opportunity to provide Congress with the highest quality cost estimates  possible. We therefore believe these two recommendations remain valid.
DOD disagreed with our third recommendation that the Secretary of  Defense direct the Secretary of the Navy to improve the accuracy and  credibility of its cost estimate by assigning a single office with the  responsibility for assembling the overall estimate into a comprehensive  and well documented package and for performing sensitivity and risk and  uncertainty analyses on the overall estimate. DOD stated that  responsibilities for centralized supervision and coordination of  programming and budgeting within the Department of the Navy are well- established and that Navy program/budget estimates and exhibits are  prepared and submitted in accordance with the Office of the Secretary of  Defense guidance. DOD further noted that reports to Congress are  typically assigned to a central office for preparation and coordination and  that where necessary, instructions to the offices providing inputs can  include guidance on baseline assumptions.
As we state in our report, the Navy’s level of coordination used in  packaging its final cost estimate did not ensure that the offices providing  inputs adequately and completely documented the cost estimates and did  not provide for the implementation of sensitivity and risk and uncertainty  analyses. Additionally, the Navy’s final cost estimate was not consistent in  its use of various baseline assumptions, such as consistency in using either  then-year or constant dollars in the individual original estimates that made  up the Navy’s final estimate for Mayport. This lack of consistency limits  the Navy’s ability to appropriately combine the estimates from different  offices into a single, coherent cost estimate. High-quality cost estimating  requires good organization and a consistent approach in pulling together  disparate data for each cost element and packaging it as an accurate and  credible whole. Thus, without a single office—within the already  established programming/budgeting process—that is responsible for  assembling future revisions of its Mayport estimate, the Navy will continue  to lack assurance that its overall estimate is comprehensive, well  documented, accurate, and credible. We therefore believe this  recommendation remains valid.
We are sending copies of this report to interested congressional  committees, the Secretary of Defense, the Secretary of the Navy, and the  Director, Office of Management and Budget. The report is also available at  no charge on the GAO Web site at http://www.gao.gov.
If you or your staff have any questions concerning this report, please  contact me at (202) 512-4523 or leporeb@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on the  last page of this report. Key contributors to this report are listed in  appendix V.
section_title: Appendix I: Scope and Methodology
paragraphs: To develop an independent cost estimate of the full life-cycle costs  associated with the Navy’s plan to establish a second East Coast homeport  for a nuclear-powered aircraft carrier we worked with a consultant from a  cost estimating company—Technomics, Inc.—with expertise in preparing  estimates of the full life-cycle costs of major federal programs. We  followed the 12 steps for developing a high-quality cost estimate outlined  in the GAO Cost Estimating and Assessment Guide.
subsections: section_title: Step 1: Define Estimate’s Purpose
paragraphs: The purpose of the estimate is to satisfy the direction provided to GAO in  House Report 111-491, accompanying a proposed bill for the Fiscal Year  2011 National Defense Authorization Act (H.R. 5136), to develop an  independent estimate of the total direct and indirect costs to be incurred  by the federal government for the proposed homeporting of an nuclear- powered aircraft carrier at Naval Station Mayport. The estimate will  provide the defense committees with an independent cost estimate against  which they can evaluate the Navy’s estimate and give Congress greater  confidence that it has an accurate understanding of the total cost  associated with establishing a homeport for a nuclear-powered aircraft  carrier at Naval Station Mayport.
section_title: Step 2: Develop Estimating Plan
paragraphs: Upon identification of the direction in the committee report, we identified  resources and developed a design matrix that included identification of  information required and sources, and the proposed scope and  methodology for addressing the direction. Additionally, in accordance  with Generally Accepted Government Auditing Standards (GAGAS), we  conducted an entrance conference with the Navy, collected cost data, and  conducted interviews with program officials.
section_title: Step 3: Define Program Characteristics
paragraphs: We collected information directly from the Navy that enabled us to define  the program’s technical and programmatic baseline. Specifically, the Navy  provided us with its estimates of one-time and recurring costs, including  any supporting documentation that outlined the underlying assumptions  and ground rules supporting the estimates. We also reviewed the Navy’s  environmental impact study along with background information about  nuclear-powered aircraft carrier maintenance, visited locations where  nuclear-powered aircraft carrier maintenance is performed, and spoke  with Navy officials knowledgeable about the personnel and facilities  requirements for nuclear-powered aircraft carrier maintenance.
section_title: Step 4: Determine Estimating Structure
paragraphs: We developed a cost element structure that was derived from the Navy's  cost estimate and supporting documentation. Where necessary, we added  subelements or additional elements. At the highest level of our estimate,  costs were categorized by one-time costs (1.0 elements) and recurring  costs (2.0 elements). For each cost element, the most appropriate  estimating approach was identified according to data availability and  credibility. Estimating approaches applied included analogy, expert  opinion, and parametric methods.
section_title: Step 5: Identify Ground Rules and Assumptions
paragraphs: Estimating ground rules and assumptions were identified and  documented. Assumptions were minimized, relying instead on real data  when possible. In order to develop an independent estimate, we generally  used methods and data that differed from the Navy’s, relying instead on  two types of estimating methods: scaled analogy and parametric. Scaled  analogy is a cost estimating method that relies on using actual costs from  analogous projects. Adjustments to cost were made for the relative size of  the analogous project as well as appropriate adjustments for complexity  factors. A parametric estimate is a method that uses an established  relationship between cost and one or more input variables. The input  variables become parameters in an equation that is used to predict cost.  It is important to note, however, that the Navy was the only source of  information for certain cost subelements, such as the number of man-days  of maintenance required for aircraft carrier availabilities and the cost of  transporting equipment for the carrier air wing and this limited the degree  to which our cost estimate is completely independent from the Navy’s for  those elements.
section_title: Step 6: Obtain Data
paragraphs: Cost, technical, and programmatic data were solicited and collected  through interviews and supporting documentation. Interviews were  conducted with and supporting documentation was obtained from officials  in the Office of the Chief of Naval Operations, Naval Sea Systems  Command, Regional Maintenance Centers, as well as officials from  Norfolk Naval Shipyard, Naval Station Mayport, and Naval Air Station  North Island. Specific documents and data we collected included Navy  responses to information requests, programming models, and construction  planning documents. Additionally, we collected and reviewed data on  unrelated but analogous Navy and DOD construction projects. In addition,  we analyzed the uncertainty associated with our estimates to help gauge  their accuracy. We assessed the reliability of all data provided by the Navy  by (1) reviewing existing information about the data and the systems that  produced them, (2) interviewing agency officials knowledgeable about the  data to determine the steps taken to ensure the accuracy and  completeness of the data, and (3) reviewing the related internal controls.  Based on this assessment, we determined the data were sufficiently  reliable for our purposes.
section_title: Step 7: Develop Point Estimate
paragraphs: Data collected were thoroughly analyzed, resulting in the development of  several different cost estimating methodologies. These methodologies  were integrated into a comprehensive cost model using Microsoft ® Excel,  resulting in a detailed point estimate.
section_title: Step 8: Conduct Sensitivity Analysis
paragraphs: A comprehensive sensitivity analysis was conducted on the cost model  also using Microsoft ® Excel. Sensitivity analysis addresses the question  of which cost model inputs have the greatest impact on the overall cost  estimate. Inputs that have the greatest impact are often referred to as cost  drivers. Our sensitivity analysis identified all of the inputs used in each  cost element. Each input was changed, in isolation, to determine the  impact on the bottom line cost. Some inputs, such as geographic  adjustment factors, have an impact on multiple elements. Other inputs  only impact a specific cost element.
section_title: Step 9: Conduct Risk and Uncertainty Analysis
paragraphs: Risk and uncertainty analysis was conducted in order to transform the  static point estimate into a probabilistic range based on the risk and  uncertainty inherent in the model input variables. Within the risk analysis  section of our estimate, each cost input and each cost estimating method  was assigned a low and high value based on the relative uncertainty  associated with each value. Inputs that had little or no uncertainty (such  as actual costs, or known square footages) had a small, or possibly a zero  range between the high and low values. In contrast, variables with a high  level of uncertainty had a wider range. We then assigned a triangular  probability distribution to each element, defined as either a low value, a  most likely value (the point estimate), or a high value. Using a Monte Carlo  simulation, the risk analysis was then assigned a random value, according  to the probability distribution of each variable. Total cost was then  calculated and the process was repeated to obtain a range of possible total  cost outcomes. The simulation resulted in establishing probability  distributions about the estimated values, enabling the identification of the  confidence level of the point estimate.
section_title: Step 10: Document the Estimate
paragraphs: Detailed documentation was generated that provides the cost estimate  results, as well as ground rules and assumptions, and all underlying cost  estimating methodology. The cost model was thoroughly referenced to  source documents and then traced and verified in accordance with  generally accepted government auditing standards.
section_title: Step 11: Present Estimate to Management for Approval
paragraphs: The resulting cost estimates were presented to various levels of GAO  management. Additionally, GAO presented the cost estimate and  underlying methodology in detail to the Navy in January 2011.
section_title: Step 12: Update the Estimate to Reflect Actual Costs and Changes
paragraphs: Upon presenting the cost estimate and underlying methodology in detail to  Navy, the Navy provided comments, questions, and some additional data  which we incorporated as appropriate in our estimate.
Within each cost element, we compared our estimate to the Navy's.  Wherever possible, the reasons for significant differences were discussed  with the Navy and explained in the report. Our estimate does not estimate  or evaluate any of the benefits of a potential homeport move, and  therefore does not address or evaluate the military advantages or  disadvantages associated with the homeporting decision. Additionally, our  estimate only includes estimated costs incurred by the Navy and it does  not include any estimate of the economic impact to the local economy in  Virginia or Florida.
To determine to what extent the estimate adheres to the characteristics of  a high-quality cost estimate, we evaluated the Navy’s life-cycle cost  estimate to determine whether it met key characteristics identified in the  GAO Cost Estimating and Assessment Guide. Our guide, which is based  on extensive research of best practices for estimating program schedules  and costs, indicates that a high-quality, valid, and reliable cost estimate  should be well documented, comprehensive, accurate, and credible. We  analyzed the cost estimating practices used by the Navy against these best  practices. Specifically, to determine whether the Navy’s estimate was well  documented, we examined whether the Navy estimate’s documentation  explicitly identified the primary methods, calculations, results, rationales  or assumptions, and sources of data used to generate each cost element.  To determine whether the estimate was comprehensive, we examined  whether the Navy’s cost estimate included all life-cycle costs required to  homeport a nuclear aircraft carrier at Naval Station Mayport. To determine  whether the Navy’s estimate was accurate, we verified the calculations in  the Navy’s estimate and examined whether the estimate accounted for all  costs, including indirect costs. Finally, to determine whether the Navy’s  estimate was credible, we examined the extent to which the Navy tested  key cost elements for sensitivity or used other cost estimating techniques  to cross-check the reasonableness of their ground rules and assumptions.  We also interviewed Navy officials from the Naval Sea Systems Command  and the Office of the Chief of Staff of Naval Operations about the process  used to prepare the life-cycle cost estimates and the assumptions used to  prepare the Navy’s estimate. We shared the GAO Cost Estimating and  Assessment Guide and the criteria against which we would be evaluating  the Navy’s cost estimate with the Navy.
We conducted our performance audit from July 2010 through February  2011 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
section_title: Appendix II: Additional Information on GAO’s Independent Cost Estimate
subsections: section_title: Nuclear-Powered Aircraft Carrier
section_title: Contents
subsections: section_title: Purpose of the Estimate Program Characteristics Estimating Structure Ground Rules and Assumptions Methodology Overview Comparison of GAO’s 65 Percent Confidence Level
section_title: Purpose of the Estimate
subsections: section_title: Satisfy the direction in House Report 111-491, accompanying
paragraphs: a proposed bill for the Fiscal Year 2011 National Defense  Authorization Act (H.R. 5136), to develop an independent  estimate of the total direct and indirect costs to be incurred by  the federal government for the proposed homeporting of a  nuclear-powered aircraft carrier at Naval Station Mayport.
Provide the congressional defense committees with an  independent cost estimate against which they can evaluate  the Navy’s estimate and give Congress greater confidence  that it has an accurate understanding of the total cost  associated with establishing a homeport for a nuclear- powered aircraft carrier at Naval Station Mayport, Florida.
section_title: Program Characteristics
subsections: section_title: The controlled industrial facility and ship maintenance support facilities in
paragraphs: Mayport have been modeled on the facilities built at Naval Air Station North  Island, San Diego, California in the late 1990s to support the first nuclear- powered aircraft carriers to be homeported in San Diego.
Nuclear-powered aircraft carrier maintenance is performed on a 32-month  cycle that includes two 1-month long Carrier Incremental Availabilities and  either one 6-month Planned Incremental Availability or a 10.5-month  Docking Planned Incremental Availability. There is also one refueling  complex overhaul around the carrier’s midlife that lasts about 39 months.
The Carrier Incremental Availabilities and Planned Incremental  Availabilities will be performed in Mayport, while the Docking Planned  Incremental Availabilities will be performed at Norfolk Naval Shipyard.
section_title: Estimating Structure
subsections: section_title: The cost element structure used in this estimate was derived
paragraphs: from the Navy’s cost estimate and associated documentation.   Where necessary, we have added subelements or additional  elements.  At the highest level, costs are categorized by one- time costs (1.0 elements) and recurring costs (2.0 elements).
section_title: Estimating Structure
paragraphs: 2.1 Permanently Assigned Labor for Nuclear Facilities 2.2 Base Operating Support for New Facilities 1.3.1 Parking 1.3.2 Road Improvements 1.3.3 Wharf F Improvements 1.3.4 Controlled Industrial Facility (CIF) 1.3.5 Ship Maintenance Facility (SMF) 1.3.6 Maintenance Support Facility (MSF)
section_title: Ground Rules and Assumptions
subsections: section_title: In order to develop an independent estimate, we generally
paragraphs: use methods and data that differ from the methods and data  used in the Navy’s cost estimate.  However, the Navy is likely  the only source of information for certain cost subelements such as the number of man-days of maintenance required for  aircraft carriers and the cost of transporting equipment for the carrier air wing.
The time frame for one-time costs will be consistent with the  Navy’s estimate, which spans from fiscal year 2010 to fiscal  year 2019.
section_title: Ground Rules and Assumptions
subsections: section_title: The recurring costs have been annualized over the 50-year
paragraphs: carrier life cycle. Consistent with the Navy’s estimate, we  have not identified an exact start date for the recurring costs.
We estimated the cost differentials associated with moving a  nuclear-powered aircraft carrier to Mayport.  This means that  our estimate only includes costs that differ from what the  Navy would spend under the current mode of homeporting all  East Coast carriers in Norfolk, Virginia.  This assumption is  consistent with the Navy’s cost estimate.
section_title: Ground Rules and Assumptions
subsections: section_title: Both one-time and recurring costs will be reported in base-year
paragraphs: 2010 dollars, which have the effects of inflation removed. In  calculations where inflation needs to be added or removed, the  estimate uses the appropriation-specific cost indexes published  annually by the Naval Center for Cost Analysis.
We do not estimate or evaluate any potential economic impacts that  may be experienced by either the Norfolk or Mayport communities  from the planned homeporting.  As directed in House Report 111- 491, accompanying a proposed bill for the Fiscal Year 2011  National Defense  Authorization Act (H.R. 5136), our estimate is limited to the costs to be incurred by the federal government in homeporting a nuclear-powered aircraft carrier at Naval Station  Mayport.
section_title: Methodology
subsections: section_title: To develop the cost estimates, we worked with an outside cost
paragraphs: estimating company with expertise in preparing estimates of the  life-cycle costs of major federal acquisitions.
We then conducted a sensitivity analysis to determine which of the  cost model inputs had the greatest impact on the overall cost  element.  Those inputs are known as cost drivers.
section_title: Methodology
subsections: section_title: Next, we conducted a risk and uncertainty analysis as a way to
paragraphs: assess the variability in the point estimate. Risk analysis addresses  the uncertainty that is inherent within cost inputs and cost estimating  methods. Inputs that have little or no uncertainty (such as sunk costs, or  known square footages) will have a small, or possibly a zero range  between the high and low values.  In contrast, variables with a high level of  uncertainty have a wider range.
The results of the risk analysis can also be viewed as an “S-Curve”, which  plots various risk-adjusted cost totals against a cumulative confidence  level.  The probabilities shown on the Y axis of an S curve represent the  probability that costs will be equal to or less than the amount shown on the  X axis.  For example, to obtain the risk-adjusted cost estimate at which  there is a 90 percent probability that the true costs will be equal to or less  than the risk-adjusted total, determine the dollar value from the X axis  where the S curve crosses 90 percent.
section_title: Overview
subsections: section_title: Our independent cost estimate suggests that the one-time cost for
paragraphs: establishing a nuclear-powered aircraft carrier homeport at Naval Station  Mayport is expected to be between $258.7 million and $356.0 million, with  a 65 percent confidence level estimate of $322.0 million.  The recurring  costs are expected to be between $9.0 million and $17.6 million per year,  with a 65 percent confidence level estimate of $14.3 million.
The Navy’s point estimate of $537.6 million for the one-time cost falls  outside the upper range of our estimate, but the Navy’s point estimate of   $15.3 million/year for recurring costs does fall within our estimated range.   The largest difference between our estimate and the Navy’s is in the one- time construction costs for the new controlled industrial facility and ship  maintenance support facilities in Mayport.
The parameters related to the construction of the controlled  industrial facility and the ship maintenance support facilities are the  cost drivers for the one-time costs, while the parameters related to  the travel cost are the cost drivers for the recurring costs.
section_title: Total Homeporting Cost
section_title: One-Time Cost
paragraphs: It is not statistically appropriate to add each of the individual 65 percent confidence level cost estimates to come up with an overall 65 percent confidence level.  The overall 65 percent confidence level is calculated separately from the individual elements.
section_title: Construction Cost
paragraphs: It is not statistically appropriate to add each of the individual 65 percent confidence level cost estimates to come up with an overall 65 percent confidence level.  The overall 65 percent confidence level is calculated separately from the individual elements.
section_title: Recurring Costs
paragraphs: It is not statistically appropriate to add each of the individual 65 percent confidence level cost estimates to come up with an overall 65 percent confidence level.  The overall 65 percent confidence level is calculated separately from the individual elements.
section_title: Risk-adjusted Range: One-Time Cost
subsections: section_title: The 65 percent confidence level estimate ($322.0 million) is plotted,
paragraphs: along with the 10 percent ($258.7 million) and 90 percent ($356.0  million) confidence levels, which when reported as a range  represent an 80 percent confidence interval.
A confidence interval describes a range of possible costs, based on  a specified probability level that is determined through a risk  analysis.  The low and high values of the confidence interval can be  expressed as confidence levels that represent the probability that  the point estimate will be met.  Our 80 percent confidence interval is  the range of costs from the 10 percent confidence level to the 90  percent confidence level. At the 10 percent confidence level, there  is a 10 percent chance that the cost will be at or below the estimate  and a 90 percent chance that the cost will be above the estimate.  Likewise at the 90 percent confidence level, there is a 90 percent  chance that the cost will be at or below the estimate and a 10  percent chance that the final cost will be above the estimate.
section_title: Risk-adjusted Range: One-Time Cost
section_title: Risk-adjusted Range: Recurring Costs
subsections: section_title: The 65 percent confidence level estimate ($14.3 million) is
paragraphs: plotted, along with the 10 percent ($9.0 million) and 90  percent ($17.6 million) confidence levels, which when  reported as a range represent an 80 percent confidence  interval.
section_title: Risk-adjusted Range: Recurring Costs
section_title: One-Time Cost Element: Planning and Design
subsections: section_title: The Navy’s estimate is outside of GAO’s risk-adjusted range.
paragraphs: Both GAO and the Navy estimated planning and design as a  percentage of the overall construction cost.  The difference  between the two planning and design estimates reflects the  difference in the overall estimates of construction costs.
section_title: One-Time Cost Element: Initial Dredging
subsections: section_title: GAO risk-adjusted range - $31.3 million to $33.1million GAO 65 percent confidence level estimate - $ 32.3 million Navy point estimate - $ 46.3 million
paragraphs: GAO originally used a scaled analogy based on similar dredging at  Norfolk Naval Shipyard and produced an estimate similar to the  Navy’s.  However, we have updated our estimate for dredging at  Mayport to reflect the cost of the contract that was awarded in  August 2010.  The contract amount was $26.6 million, and the  GAO estimate includes overhead and contingency costs. The Navy  stated that the contract price came in lower than the estimate due  to the widely varying nature of costs in the dredging industry.  According to a Navy official, the dredging price is largely based on  equipment availability.  If equipment happens to be available at the  required time, then the dredging cost will be lower.
section_title: One-Time Cost Element: Parking Garage
subsections: section_title: The Navy’s estimate is within the GAO risk-adjusted range.
paragraphs: GAO’s estimate is based on cost per square foot, adjusted  with cost factors for supervision, inspection, and overhead;  contingency; and a geographic factor for northeast Florida.   The Navy’s estimate is based on a detailed engineering  estimate developed by a private engineering firm.
section_title: One-Time Cost Element: Road Improvements
subsections: section_title: The Navy’s estimate is within GAO’s risk-adjusted range.
paragraphs: GAO’s estimate is based on a scaled analogy of road  construction projects for other locations.  The Navy’s  estimate is based on a detailed engineering estimate  developed by a private engineering firm.
section_title: One-Time Cost Element: Wharf F Improvements
subsections: section_title: The Navy’s estimate is within GAO’s risk-adjusted range.
paragraphs: GAO’s estimate is based on an average of analogous wharf  construction projects from other locations.  Adjustments were  made for size, geographic location, and inflation.  The Navy’s  estimate is based on a detailed engineering estimate  developed by a private engineering firm.
section_title: One-Time Cost Element: Controlled Industrial Facility
subsections: section_title: GAO risk-adjusted range - $35.2 million to $94.9 million GAO 65 percent confidence level estimate - $70.5 million Navy estimate - $139.1 million
paragraphs: The Navy’s estimate is not within GAO’s risk-adjusted range.  The Navy’s  estimate is based on a detailed engineering study developed by a private  engineering firm.  The GAO estimate is based on historical costs observed for  analogous facility construction projects at Naval Station North Island (San  Diego).  These analogous costs are adjusted to account for differences in size,  inflation, geographical differences between the two areas, and any identified  complexities or requirements unique to the Mayport construction. Those  complexity factors included construction to withstand Class 4 hurricane storm  surge and costs associated with compliance with environmental standards and  with antiterrorism and force protection. These factors were included in GAO’s  estimate based on a percentage of the total construction cost. No other  specific complexity factors that would account for the discrepancy in the two  estimates have been identified by either GAO or the Navy.
section_title: One-Time Cost Element: Ship Maintenance Support Facilities
subsections: section_title: The Navy’s estimate is substantially outside of GAO’s risk-adjusted range.
paragraphs: The Navy’s estimate is based on a detailed engineering study developed  by a private engineering firm.  The GAO estimate is based on historical  costs observed for analogous facility construction projects at Naval Station  North Island.  These analogous costs are adjusted to account for differences in size, inflation, geographical differences between the two  areas, and any identified complexities or requirements unique to the  Mayport construction. Those complexity factors included compliance with  environmental standards and with antiterrorism and force protection.  These factors were included in GAO’s estimate based on a percentage of  the total construction cost. No other specific complexity factors that would  account for the significant difference between the two estimates have been  identified by either GAO or the Navy.
section_title: One-Time Cost Element: Initial Equipment Outfitting
subsections: section_title: The Navy’s estimate is outside of GAO’s risk-adjusted range.
paragraphs: GAO’s estimate is based on a scaled analogy of equipment  purchased at Naval Air Station North Island, San Diego.      The Navy’s estimate is based on a detailed estimate that  lists all equipment purchase requirements, with the cost of  commercial pricing estimated for each item.
section_title: One-Time Cost Element: Initial Permanent Change of Station
subsections: section_title: The Navy’s estimate is outside of GAO’s risk-adjusted range.
paragraphs: GAO’s estimate is based on published DOD dislocation allowance  rates, by rank, for crew members with and without dependents.  It  assumes a single permanent change of station move when the  homeport location is changed.  The Navy’s estimate appears to be based on additional costs other than the dislocation allowance  rates, but the exact nature of those costs was not specified.
section_title: Recurring Cost Element: Permanently Assigned Labor
subsections: section_title: The Navy’s estimate is within GAO’s risk-adjusted range. GAO’s
paragraphs: estimate uses the Navy’s assumption of 12 full-time employees at  a rate of $100,000 per year.  We confirmed that this assumption is  consistent with the staffing level and compensation for employees  working at the nuclear carrier maintenance facilities at Naval Air  Station North Island, San Diego.  GAO’s 65 percent confidence  level estimate is slightly higher than the Navy’s point estimate because it has been adjusted through a risk and uncertainty  analysis, whereas the Navy’s estimate has not.
section_title: Recurring Cost Element: Base Operating Support
subsections: section_title: The Navy’s estimate is within GAO’s risk-adjusted range. GAO’s
paragraphs: estimate uses the Navy’s assumptions of 10 full-time guards at  $75,000 per year and $225,000 per year for facilities services  based on a fixed unit cost per square foot.  The unit cost is based  on an average cost for Naval Station Mayport in fiscal year 2008.  GAO’s 65 percent confidence level estimate is slightly higher than  the Navy’s point estimate because it has been adjusted through a risk and uncertainty analysis, whereas the Navy’s estimate has not.
section_title: Recurring Cost Element: Facilities Sustainment
subsections: section_title: The Navy’s estimate is within GAO’s risk-adjusted range.
paragraphs: Both estimates are based on facilities sustainment rates  published in the DOD Facilities Pricing Guide, a program  analysis and budget tool that lists replacement, sustainment,  modernization, and operation unit costs for DOD facilities.   Adjustments are made for geographic location.
section_title: Recurring Cost Element: Facilities Restoration and Modernization
subsections: section_title: The Navy’s estimate is outside GAO’s risk-adjusted range. GAO’s
paragraphs: estimate is based on modernization factors published in the DOD  Facilities Pricing Guide.  Each factor is multiplied against the estimated construction cost.  The Navy’s estimate assumes a 67- year recapitalization rate. The modernization factors and 67-year  recapitalization rate result in very similar percentages, but because  the Navy’s construction estimates are higher, the restoration and  modernization rates are correspondingly higher in the Navy’s  estimate.
section_title: Recurring Cost Element: Operations
subsections: section_title: The Navy’s estimate is within GAO’s risk-adjusted range.
paragraphs: GAO’s estimate is based on the assumptions and amounts  used in the Navy’s estimate. GAO’s 65 percent confidence  level estimate is slightly higher than the Navy’s point  estimate because it has been adjusted through a risk and  uncertainty analysis, whereas the Navy’s estimate has not.
section_title: Recurring Cost Element: Travel for Navy Shipyard Workers
subsections: section_title: The Navy’s estimate is within GAO’s risk-adjusted range. GAO’s
paragraphs: estimate is based on a scaled analogy of travel expenses incurred  during a 2009 planned incremental availability on the USS Ronald  Reagan.  Adjustments were made for inflation and differing airfare  and per diem rates in the Jacksonville, Florida area.  The Navy’s  estimate is based on a programming model that breaks down the  travel expense into component costs for airfare, per diem, hotel tax, rental car, and gas.
section_title: Recurring Cost Element: Biennial Maintenance Dredging
subsections: section_title: The Navy’s estimate is within GAO’s risk-adjusted range.
paragraphs: GAO’s estimate is based on the assumptions and amounts  used in the Navy’s estimate.  For this cost element, GAO’s  65 percent confidence level estimate is identical to the  Navy’s point estimate even though the Navy’s estimate was  not adjusted through a risk and uncertainty analysis like  GAO’s.
section_title: Recurring Cost Element: Basic Allowance for Housing
subsections: section_title: The Navy’s estimate is outside GAO’s risk-adjusted range. Both estimates
paragraphs: are based on published rates, by rank, for crew members with and without  dependents.  The difference between the estimates is due to the fact that  GAO’s estimate assumes a crew size of 3,140, which is consistent with the  Navy’s Environmental Impact Study and other sources.  The Navy’s estimate assumes a crew size of 2,728, which lowers the total savings. allowance is less in Jacksonville, Florida than it is in Norfolk, Virginia.  This  means that there is a net savings for the Navy for this cost element.
section_title: Recurring Cost Element: Utilities
subsections: section_title: The Navy’s estimate is within GAO’s risk-adjusted range.
paragraphs: Both estimates are based on the same assumed amount of  power usage multiplied by an average power cost  differential.  GAO’s estimate uses average power costs  published by the Department of Energy for Virginia and  Florida, which is a differential of 2.51 cents.  The Navy’s  estimate assumes a 2.00 cent differential.
section_title: Recurring Cost Element: Recurring Permanent Change of Station
subsections: section_title: Recurring Cost Element: Private Sector Travel
paragraphs: GAO risk-adjusted range - $0.1 million to $1.2 million  GAO 65 percent confidence level estimate - $0.6 million  Navy point estimate - N/A  To account for the risk that private sector shipyards may have to bring in  workers from outside of the Mayport area during surges in workload and  thereby incur travel expenses that would be passed on to the federal  government, the GAO estimate calculates potential private sector travel  costs using the public sector travel and per diem rates.  The GAO estimate  assumes a 60/40 public/private ratio for maintenance work in Mayport and  sets the most likely private sector travel cost at zero, with a high-value risk  input that assumes 30 percent of the private sector workforce has to travel  from outside of the Mayport area.
section_title: Appendix III: GAO’s Assessment of the Navy’s Cost Estimating Practices
paragraphs: Appendix III: GAO’s Assessment of the Navy’s  Cost Estimating Practices The cost estimate includes all life-cycle costs.
The cost estimate completely defines the program, reflects the current  schedule, and is technically reasonable.
The cost estimate work breakdown structure is product-oriented,  traceable to the statement of work/objective, and at an appropriate level  of detail to ensure that cost elements are neither omitted nor double- counted.
The estimate documents all cost-influencing ground rules and  assumptions.
The documentation should capture the source data used, the reliability of  the data, and how the data were normalized.
The documentation describes in sufficient detail the calculations  performed and the estimating methodology used to derive each  element’s cost.
The documentation describes step by step how the estimate was  developed so that a cost analyst unfamiliar with the program could  understand what was done and replicate it.
The documentation discusses the technical baseline description and the  data in the baseline are consistent with the estimate.
The documentation provides evidence that the cost estimate was  reviewed and accepted by management.
The cost estimate results are unbiased, not overly conservative or  optimistic, and based on an assessment of most likely costs.
The estimate has been adjusted properly for inflation.
The estimate contains few, if any, minor mistakes.
The cost estimate is regularly updated to reflect significant changes in  the program so that it is always reflecting current status.
Variances between planned and actual costs are documented,  explained, and reviewed.
The estimate is based on a historical record of cost estimating and actual  experiences from other comparable programs.
The cost estimate includes a sensitivity analysis that identifies a range of  possible costs based on varying major assumptions, parameters, and  data inputs.
A risk and uncertainty analysis was conducted that quantified the  imperfectly understood risks and identified the effects of changing key  cost driver assumptions and factors.
Cost drivers were cross-checked to see whether results were similar.
An independent cost estimate was conducted by a group outside the  acquiring organization to determine whether other estimating methods  produce similar results.
Not met  provided complete evidence that satisfies the entire criterion.
section_title: Appendix IV: Comments from the Department of Defense
section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: Acknowledgments
paragraphs: In addition to the contact listed above, key contributors to this report  include Mark J. Wielgoszynski, Assistant Director; Darnita Akers;  Shawn Arbogast; David Brown (Technomics, Inc.); Russell Bryan;  Jennifer Echard; Mary Jo LaCasse; Erik Wilkins-McKee; and  Michael Shaughnessy.