Defense Infrastructure: Navy Can Improve the Quality of Its Cost Estimate to Homeport an Aircraft Carrier at Naval Station Mayport
Released: 2011-03-03T00:00:00

Letter

Background

Planned Infrastructure and Facility Improvements at Naval Station Mayport to Support a Nuclear- Powered Aircraft Carrier

The U.S. Navy currently maintains 11 nuclear carriers—5 homeported on the West Coast of the United States, 5 on the East Coast of the United States, and 1 in Yokosuka, Japan. Of the West Coast carriers, 3 are homeported in San Diego, California; 1 in Bremerton, Washington; and 1 in Everett, Washington. All 5 East Coast carriers are homeported in Norfolk, Virginia. As some of the most technologically advanced ships in the world, nuclear-powered aircraft carriers require continuous and regularly scheduled maintenance to keep them ready to meet mission requirements.

Naval Station Mayport has served as a conventional aircraft carrier homeport at various times since the 1950s, including homeporting two conventional carriers at one time, but it has not served as homeport for a nuclear-powered aircraft carrier. Consequently, to homeport a nuclear powered aircraft carrier, several major infrastructure upgrades are required at Mayport. These infrastructure improvements include construction of four new buildings: (1) controlled industrial facility, (2) ship maintenance facility, (3) maintenance support facility, and (4) parking garage. The controlled industrial facility would be used for the inspection, modification, and repair of radiologically controlled equipment and components associated with naval nuclear propulsion plants. The ship maintenance facility is necessary to perform nonnuclear maintenance on the carrier’s nuclear propulsion plants. The maintenance support facility would provide the needed administrative and technical staff offices to support propulsion plant maintenance and a central area for receiving, inspecting, shipping, and storing materials. The parking garage would replace parking spaces lost through construction of the three new maintenance buildings. The infrastructure upgrades also include improvements to Wharf F, which would receive improvement to its structural components, berthing capability, and utility and security systems in order to make it capable of homeporting a nuclear carrier. Additionally, traffic improvements would be made to the Massey Avenue corridor to rebuild a boulevard-style roadway with two lanes each way divided by a grass median. Dredging would also be necessary to deepen the Mayport turning basin to allow access for a fully loaded nuclear aircraft carrier. Figure 1 shows the location and expected time line for completion of the planned infrastructure and facility improvements at Naval Station Mayport.

Massey Ave.

Navy’s Estimate for Establishing a Nuclear- Powered Aircraft Carrier at Naval Station Mayport

The Navy’s estimate of one-time and recurring costs for establishing a nuclear-powered aircraft carrier homeport at Naval Station Mayport is contained in two DOD information papers dated February 25, 2010, and August 5, 2010. The February 2010 information paper was produced by the Navy’s Office of the Chief of Naval Operations in response to a request from the Congressional Research Service and provided only the one-time costs of the carrier homeport. Those one-time costs were primarily drawn from a series of DD Form 1391s, which DOD uses as budget justification material to support funding requests for military construction. However in addition to military construction costs, there are also one-time costs associated with planning and design, the initial equipment outfitting for the new maintenance facilities to be built in Mayport, and the permanent change of station cost for the relocation of the carrier crew to Mayport.

The August 2010 information paper was also prepared by staff working for the Chief of Naval Operations, in response to questions from GAO about the recurring costs for the planned homeporting. This information paper contained estimates of the recurring costs associated with the carrier homeport. There were nine recurring costs listed in the information paper including: (1) permanently assigned labor for the new maintenance facilities, (2) base operating support, (3) facilities sustainment, (4) facilities restoration and modernization, (5) transportation of equipment for the carrier air wing, (6) travel and per diem for public shipyard workers traveling from Norfolk to Mayport to work on the carrier’s nuclear propulsion systems, (7) the additional biennial maintenance dredging needed to allow access for a fully loaded nuclear aircraft carrier, (8) the differential between Norfolk and Mayport in basic allowance for housing costs, and (9) the differential between Norfolk and Mayport in utilities costs. The recurring costs do not represent the total cost of operating a nuclear-powered aircraft carrier. Rather, they are the subset of costs associated with nuclear carrier maintenance in which there is a cost differential between conducting the maintenance in Mayport and conducting the maintenance in Virginia.

On December 28, 2010, the Chief of Naval Operations transmitted a report to the congressional defense committees that essentially consolidated the estimates provided in the two information papers. However, the report differed from the information papers in its treatment of inflation for one- time costs and in the total for the one-time permanent change of station cost. The February 2010 information paper showed the one-time military construction costs as budgeted amounts in then-year dollars, which include the effects of inflation, while the December 2010 report showed al one-time cost in base year 2010 dollars, which exclude the effects of inflation and use 2010 prices throughout the life cycle. The one-time permanent change of station cost was reduced from $26 million to $10 million in the December 2010 report. The report did not provide any additional description or documentation that was not contained in t DOD information papers.

Characteristics of a High- Quality Cost Estimate

A reliable cost estimate is critical to the success of any program because it can provide the basis for informed investment decision making, realistic budget formulation and program resourcing, meaningful progress measurement, proactive course correction when warranted, and accountability for results. In March 2009, GAO issued a cost estimating and assessment guide that compiled cost estimating best practices drawn from across industry and government. Specifically, the guide includes four characteristics of a high-quality cost estimate by which an organization’s management can assess cost estimates in making informed decisions. A cost estimate is: e when it accounts for all possible costs associated with a project, is structured in sufficient detail to ensure that costs are neither omitted nor double-counted, and the estimating teams’ composition is commensurate with the assignment; well documented when supporting documentation is accompani narrative explaining the process, sources, and metho the estimate and contains the underlying data used to develop the estimate; accurate when it is based on an assessment of the costs most likely d and not overly conservative or too optimistic; and credible when it has been cross-checked with an independent cost estimate, the level of confidence associated with the point estim been identified through a risk and uncertainty analysis, and a sensitivity analysis has been conducted—that is, the project has ate has examined the effect of changing one assumption related to eac activity while holding all other variables constant in order to iden which variable most affects the cost estimate.

In addition, GAO’s cost estimating and assessment guide l steps that, when followed, should result in high-quality cost estimates that are comprehensive and accurate and that can be easily and clearly traced , replicated, and updated. The guide also contains hundreds of best practices drawn from across industry and government for carrying out these steps. Table 1 shows these 12 key steps and how they relate t four characteristics of a high-quality cost estimate.

GAO’s Independent Cost Estimate Is Lower Than the Navy’s Estimate for Homeporting a Nuclear Aircraft Carrier at Naval Station Mayport

Our independent estimate of the total cost for homeporting a nuclear- powered aircraft carrier at Naval Station Mayport is lower than the Navy’s estimate. Specifically, the Navy’s estimate of one-time costs is outside the upper range of our estimate, while its estimate of recurring costs falls within the upper range of our estimate.

GAO Estimates One-Time Costs to Be Between $258.7 Million and $356.0 Million, Lower Than the Navy’s Estimate

Our independent cost estimate suggests that the total one-time cost of homeporting a nuclear-powered aircraft carrier at Naval Station Mayport will be between $258.7 million and $356.0 million, in base year 2010 dollars. The Navy’s estimate of the one-time costs is $537.6 million, also in base year 2010 dollars, which is outside the upper range of our estimate. Unlike our estimate, the Navy did not conduct a risk and uncertainty analysis on its one-time costs; as a result, its estimate does not include a range. Reporting a range of costs around a point estimate is useful to decision makers because it conveys a level of confidence in achieving the most likely cost.

Table 2 shows a comparison between our estimated range and the Navy’s estimate for one-time costs. Specifically, the table shows our estimated range at an 80 percent confidence interval and whether the Navy’s estimate falls into that range. The low value of the estimate range ($258.7 million) represents a 10 percent chance that the cost will be that amount or less, and the high value of the estimated range ($356.0 million) represents a 90 percent chance that the cost will be that amount or less. The last column in the table identifies whether the Navy’s estimate is within our estimated range.

Our initial point estimate of the one-time dredging cost based on historical data was $44.3 million, which was $2 million less than the Navy’s estimate. However, in accordance with the best practices recommended by the GAO Cost Estimating and Assessment Guide, we have updated our estimate based on the $26.6 million contract awarded in August 2010. Our estimated range now reflects the contract award price plus the Navy’s overhead and contingency amounts. Navy officials attributed the lower than estimated contract costs to supply and demand within the private dredging sector, which because of lower demand has caused firms to compete at lower cost.

We also compared the Navy’s point estimate to our estimate at the 65 percent confidence level. A confidence level is the chance that the actual costs will be equal to or less than the adjusted amount. In table 2, we reported the 10 percent confidence level estimate as our low estimate range and the 90 percent confidence level estimate as our high estimate range. For an element-by-element comparison of our estimate with the Navy’s point estimate, we chose the 65 percent confidence level since we consider this confidence level to be an appropriate budgeting range.

Experts agree that program cost estimates should be budgeted to at least the 50 percent confidence level, but budgeting to a higher level, such as 70 percent to 80 percent, or the mean, is now a common practice.

Table 3 shows our 65 percent confidence level estimate in comparison to the Navy’s point estimate. To facilitate comparisons against the Navy’s estimate, the one-time costs are expressed in base year 2010 dollars, which represent amounts based on 2010 prices, with the impact of inflation removed. While useful for comparisons against the Navy’s estimate, base year 2010 dollars should not be used as the basis for budgetary decisions. In order to support a budgetary amount, base year 2010 dollars would need to be converted into then-year dollars.

Table 3 shows the largest difference between the two point estimates is in the construction of the controlled industrial facility and the ship maintenance support facilities, where the Navy’s estimates were $68.3 million and $111.6 million higher than our 65 percent confidence level estimates, respectively. The Navy’s estimate of these costs is based on a detailed engineering study developed by an engineering firm. In contrast, our estimate is based on historical costs observed for analogous facility construction projects at Naval Air Station North Island in San Diego in 1995. These analogous costs are adjusted to account for differences in size, inflation, geographical differences between the two areas, and any identified complexities or requirements unique to the Mayport construction. These factors were not present in the San Diego projects. One known complexity of the Mayport construction projects is that the controlled industrial facility must be built to withstand a storm surge from a category 4 hurricane. Our estimate includes a factor of 10 percent of the total construction cost to account for hurricane-related construction complexities. Other complexity factors included environmental considerations and costs associated with antiterrorism and force protection that were not required for the construction at Naval Station North Island. We included both of these factors in our estimate based on a percentage of total construction cost. Neither GAO nor the Navy has identified any other specific complexity factors that would account for the significant discrepancy between the two estimates.

Additionally, other elements of the Navy’s estimate were higher than our own. Specifically, the Navy’s estimate of initial equipment outfitting was $24.6 million higher than ours and the Navy’s estimate of the permanent change of station cost for relocating the carrier crew was $4.5 million higher than ours. Finally, the Navy’s planning and design estimate was $10.3 million higher than ours. Both we and the Navy estimated planning and design as a percentage of the overall construction cost. The $10.3 million difference between the Navy’s estimate and ours results from the Navy’s higher estimates for construction of the parking garage and controlled industrial facility and ship maintenance support buildings. Appendix II includes our complete cost estimate and explains in more detail our cost estimating results for each element of the one-time costs.

GAO Estimates Recurring Costs to Be Between $9.0 and $17.6 Million Per Year, and the Navy’s Estimate Falls Within This Range

Our independent cost estimate indicates that the annual recurring cost of homeporting a nuclear-powered aircraft carrier at Naval Station Mayport is expected to be between $9.0 million and $17.6 million each year, and the Navy’s $15.3 million per year estimate of the annual recurring cost is within the range of our estimate. Because the Navy did not conduct a risk and uncertainty analysis on its recurring costs, its estimate does not include a range.

Table 4 shows a comparison between our estimated range and the Navy’s estimate for recurring costs. Specifically, the table shows our estimate range at an 80 percent confidence interval and whether the Navy’s estimate falls into that range. The low value of the estimated range ($9.0 million) represents a 10 percent chance that the cost will be that amount or less, and the high value of the estimated range ($17.6 million) represents a 90 percent chance that the cost will be that amount or less. The last column in the table identifies whether the Navy’s estimate is within our estimated range.

As we did with one time costs, we also compared our 65 percent confidence level estimates with the Navy’s point estimates for a direct element-by-element comparison between our estimate and the Navy’s, as shown in table 5.

The largest difference between the two estimates is in the cost of facilities restoration and modernization, where the Navy’s estimate is $3.9 million higher. Since the Navy determined this cost as a percentage of total building construction costs for their estimates, the Navy’s higher cost results from its higher estimate for construction of the controlled industrial facility and the ship maintenance support buildings. Some of the recurring cost elements, such as the permanently assigned labor for nuclear facilities and base operating support, are slightly higher in our estimate than the Navy’s estimate. For these cost elements we used the Navy’s estimated costs because we did not have access to a source of information from which we could estimate the costs independently from the Navy. Although we used the same estimate as the Navy, our numbers at the 65 percent confidence level are slightly higher than the Navy’s because they are risk-adjusted, whereas the Navy’s point estimates have not been adjusted for risk. In using the Navy’s calculations, we reviewed the Navy’s ground rules and assumptions to the extent possible based on the information that the Navy provided to us.

In addition, we included two additional recurring costs in our estimate that were not included in the Navy’s estimate—the additional travel costs that the private sector may incur if they have to bring in workers from outside of the Mayport area and the permanent change of station for the aircraft carrier crew during the 10.5-month docking maintenance availabilities and the 3-year midlife refueling overhaul. The recurring costs have been annualized over the 50-year carrier life cycle. Consistent with the Navy’s estimate, we have not identified the recurring costs as beginning in any particular fiscal year. Appendix II provides our complete cost estimate and explains in greater detail our cost estimating results for each element of recurring costs. It also contains additional information on the differences between GAO’s and the Navy’s estimates.

Navy’s Estimate Does Not Fully Meet Any of the Four Characteristics for Producing a High- Quality Cost Estimate

The Navy’s estimate did not fully meet any of the four characteristics— comprehensive, accurate, well documented, and credible—for producing a high-quality cost estimate. Specifically, the Navy’s estimate partially met the criteria for being comprehensive; however, the estimate was minimally accurate and well documented and overall, did not meet the criteria for being credible.

The Navy’s Estimate Is Partially Comprehensive

The Navy’s estimate partially meets the criteria for being comprehensive. Specifically, it includes almost all of the life-cycle costs related to homeporting a nuclear-powered aircraft carrier at Naval Station Mayport. The Navy did not include two small recurring costs: (1) the additional travel costs that the private sector may incur if they have to bring in workers from outside of the Mayport area and (2) the permanent change of station for the carrier crew during the 10.5-month docking planned incremental availability and 39-month midlife refueling overhaul. However, the estimate does not fully document the cost-influencing ground rules and assumptions. Additionally, the Navy only minimally provided a cost element breakdown structure in its estimate, and in conducting our assessment, we had to develop our own more robust structure to ensure that cost elements were neither omitted nor double counted. Individual segments of the Navy’s cost estimate were completed by separate Navy teams and then pieced together into two documents that represent the Navy’s cost estimate. According to Navy officials, there was no single person or team in charge of ensuring that the entire Mayport homeporting estimate was packaged into a comprehensive and well documented final estimate.

According to GAO’s guide, a cost estimate is comprehensive when it includes both government and contractor costs of the program over its full life cycle, from inception of the program through design, development, deployment, and operation and maintenance to retirement of the program. It should also completely define the program, reflect the current schedule, and be technically reasonable. Comprehensive cost estimates should be structured in sufficient detail to ensure that cost elements are neither omitted nor double counted. Specifically, the GAO guide states that the cost estimate should be based on a product-oriented work breakdown structure that allows a program to track cost and schedule by defined deliverables, such as hardware or software components. Finally, where information is limited and judgments must be made, the cost estimate should document all cost-influencing ground rules and assumptions. Appendix III provides our assessment of the extent to which the Navy’s estimate met the best practices for the four characteristics of a high- quality cost estimate—comprehensive, well documented, accurate, and credible—described in the GAO Cost Estimating and Assessment Guide.

The Navy's Estimate Is Minimally Accurate

The Navy’s estimate is minimally accurate. Some elements of the estimate are based on a historical record of cost estimating and actual experiences from other comparable programs. Many of the recurring costs—such as the facilities sustainment, operations, and the biennial dredging costs—are based on a historical record of past costs. However, no evidence was provided by the Navy to show that a risk and uncertainty analysis was conducted. Therefore, the confidence level of the estimate can not be determined, and it is difficult to say if the cost estimates are the most likely costs. In addition, the initial cost estimate provided to us by the Navy contained calculation errors, although the Navy identified the errors after discussions with GAO and corrected the errors in subsequent versions of its estimate.

An estimate is considered accurate when it is based on an assessment of the costs most likely to be incurred, according to GAO’s cost estimating guide. The results should be unbiased, that is not overly conservative or optimistic. Additionally, the estimate should be based on a historical record of cost estimating and actual experiences from other comparable programs. The Navy based its estimate of construction costs on a detailed engineering estimate produced by a private architecture and engineering firm, but prior to GAO’s review, the Navy had not reviewed the firm’s engineering estimate to verify the accuracy and completeness of the estimate and did not check whether the engineering estimate relied on overly conservative assumptions or fell within the range of historical costs. Without ensuring that its military construction estimates are accurate, the Navy may be overestimating the costs of new construction at Mayport, which could result in the Navy requesting more funding than is needed and limit the amount of money available to fund other projects.

The Navy’s Estimate Is Minimally Well Documented

Although the Navy’s documentation contains some discussion of the technical information used in the estimate, the Navy’s estimate is minimally well documented. The documentation contains very little step- by-step description of how the estimate was developed so that a cost analyst unfamiliar with the program could understand what was done and replicate it. Additionally, the documentation does not capture the source data used, the reliability of the data, and how the data were standardized. Moreover, Navy officials had to recreate several portions of the estimate to provide us with supporting documentation.

Cost estimates, according to GAO’s cost estimating and assessment guide, are well documented when they can be easily repeated or updated and can be traced to original sources. The documentation should explicitly identify the primary methods, calculations, assumptions, and sources of the data used to generate each cost element. The estimating process should be described and an explanation provided for why particular methods and data sets were chosen and why these choices are reasonable. All the steps involved in developing the estimate should be documented so that a cost analyst unfamiliar with the program could recreate it with the same result.

In addition, documentation for the cost estimate should reflect changes in technical or program assumptions or new program phases or milestones. Insufficient documentation makes it difficult for outside organizations to validate the Navy’s life-cycle cost estimate for homeporting an aircraft carrier at Naval Station Mayport. Not having adequate documentation also impedes the Navy’s ability to develop future estimates and facilitate oversight by Congress.

The Navy's Estimate Does Not Meet GAO's Best Practices for a Credible Estimate

Overall, the Navy's estimate does not meet the GAO best practices’ criteria for a credible estimate, because it does not include a sensitivity analysis and was not compared by the Navy to an independent cost estimate conducted by a group outside the acquiring organization. Although DOD's Office of Cost Assessment and Program Evaluation did review the Navy's presentations and supporting documentation in conjunction with the 2010 Quadrennial Defense Review working group, the office only performed a general review of the Navy's costs, not a full independent cost estimate with a sensitivity analysis. Further, while the Navy did identify some risks, it did not conduct a formal risk and uncertainty analysis that would identify a level of confidence in the estimate. In addition, although some cost elements in the Navy’s estimate were derived from models that have undergone validation, verification, and accreditation, others were not. For example, the model that was used to develop the estimate of the recurring travel cost for the maintenance workers who would travel from Norfolk Naval Shipyard to work on the carrier’s nuclear propulsion systems in Mayport underwent a documented validation, verification, and accreditation process. Similarly, the DOD’s facilities sustainment model that was used to calculate the recurring facilities sustainment costs was also validated, verified, and accredited. However, we did not find any evidence that any validated, verified, or accredited model was used for calculating the costs of transporting equipment for the carrier air wing, the cost of the differential in the basic allowance for housing between Norfolk and Mayport, the cost of the differential in utilities charges between Norfolk and Mayport, or the cost of the permanent change of station for the aircraft carrier crew. In terms of risk analysis, only the military construction estimates contained any consideration of risk, in the form of contingency factors.

Moreover, we found that the Navy has not assigned a single office with the responsibility for ensuring that the overall estimate for Mayport homeporting was packaged into a comprehensive and well documented final estimate and that a sensitivity analysis was conducted on the estimate to determine which cost elements, and which factors of those cost elements, most influenced the overall cost. When an organization does not assign a single office responsibility for assembling the estimate into a comprehensive package, it is difficult to conduct proper sensitivity and risk analyses. Additionally, different offices preparing parts of the estimate may use conflicting assumptions. For example, different offices may use different assumptions regarding the treatment of inflation. Some costs may be expressed in base year dollars, while others are expressed in then-year dollars. This was the case with the Navy’s original estimates which showed one-time costs in then-year dollars and recurring costs in base year 2010 dollars. Even if amounts are all expressed using the same type of dollars, different contributors may use different inflation indexes, resulting in conflicting underlying assumptions.

According to GAO’s guide, cost estimates are credible when major assumptions have been varied and other outcomes recomputed to determine how sensitive outcomes are to changes in the assumptions, risk and uncertainty analyses have been performed to determine the level of risk associated with the estimate, and the estimate’s results have been cross-checked and an independent cost estimate has been developed to determine whether other estimating methods produce similar results.

Sensitivity and risk analyses should be included in all cost estimates as a best practice because all estimates have some uncertainty. A sensitivity analysis addresses some of the estimating uncertainty by testing discrete cases of assumptions and other factors that could change. By examining each assumption or factor independently, while holding all others constant, the cost estimator can evaluate the results to discover which assumptions or factors most influence the estimate. However, because many parameters could change at the same time, a risk analysis should also be performed to capture the cumulative effect of additional risks. Risk analysis adds to the credibility of a cost estimate because it quantifies the uncertainty and provides a level of confidence associated with the point estimate. The results of a high-quality, reliable cost estimate should also be cross-checked, and an independent cost estimate should be developed to determine whether other estimating methods produce similar results. An independent cost estimate is considered to be one of the most reliable validation methods and is typically performed by organizations higher in the decision-making process, or entirely outside of the office performing the baseline cost estimate, using different estimating techniques and, where possible, different data sources from those used to develop the baseline cost estimate.

Performing a sensitivity analysis for the Mayport aircraft carrier estimate would help to identify and focus on key elements with the greatest effects on cost and understand the potential for cost growth and the reasons for it. Additionally, because the Navy has not conducted a risk and uncertainty analysis, it is unable to provide Congress with a reasonably high confidence level for its Mayport aircraft carrier cost estimate. Performing a risk and uncertainty analysis would enable the Navy to quantify the risk and uncertainty associated with the cost model; provide a level of confidence for its cost estimate; and give decision makers perspective on the potential variability of the cost estimate should facts, circumstances, and assumptions change.

Navy officials told us that short time frames and budgetary constraints prevented them from following the criteria for comprehensiveness, accuracy, documentation, and credibility in developing their estimate. For elements of the recurring costs, such as the travel and per diem for public shipyard workers, the estimates were produced in response to different congressional requests for information often requiring quick turnaround—in some cases as short as 3 days. Navy officials told us that in the case of the military construction estimates, their intention was not to develop budget quality estimates, but rather an estimate more suitable for initial programming and planning purposes. Navy officials noted that this is a typical approach for construction projects that are not scheduled to begin for several years. According to the Navy, it would have been cost prohibitive to develop budget quality estimates for every proposed military construction project. Navy officials stated that the Navy further reviews and refines the construction estimates as they proceed through the DOD budget process. Because many of the projects planned for Mayport would be included in future budgets, these projects have not yet undergone this review. Consistent with prior experience, it may not have been feasible for the Navy to follow cost estimating best practices to the fullest extent in developing initial estimates, especially given the short time frames often available. However, as the Navy continues to respond to congressional interest in the costs associated with homeporting a nuclear carrier at Mayport, it has the opportunity to more completely incorporate cost estimating best practices in its revised estimate, thereby improving the estimate’s quality and making it easier to defend in future budgets and decision making.

Conclusions

The Navy’s ability to produce a comprehensive, accurate, well documented, and credible cost estimate for homeporting a nuclear- powered aircraft carrier at Naval Station Mayport will continue to be hampered until it makes certain fundamental changes to the process it uses to develop, document, and update its overall estimate of Mayport homeporting costs. Specifically, without full documentation of the data sources, assumptions, and calculation methods it uses, the Navy cannot assure that its estimate can be validated or defended or any differences between estimated and actual costs can be explained—an important step in improving and updating the estimate. Additionally, without detailed documentation that describes how the estimate was derived, the Navy can neither present a convincing argument of the estimate’s affordability, nor credibly answer decision makers’ and oversight groups’ questions about specific details in the estimate. Further, without conducting sensitivity and risk and uncertainty analyses on its cost estimate, the Navy is unable to identify and focus on major cost drivers, analyze the potential for cost growth, and quantify the risk and uncertainty associated with the cost estimate. Moreover, without a comprehensive, accurate, well documented, and credible cost estimate, Congress cannot have reasonable confidence that it has a complete understanding and an accurate and realistic determination of the projected costs to evaluate and make decisions on the Navy’s planned homeporting of a nuclear-powered aircraft carrier at Mayport.

Recommendations for Executive Action

To improve the Navy’s life-cycle cost estimate for the planned homeporting of a nuclear-powered aircraft carrier at Naval Station Mayport, Florida, we recommend that the Secretary of Defense direct the Secretary of the Navy to take the following three actions to incorporate to a greater extent the best practices identified by GAO for developing a high- quality cost estimate in future revisions of its Mayport nuclear carrier homeporting cost estimate as part of the annual budgetary process or in response to future congressional requests: 1. To improve the comprehensiveness of its cost estimate, the Navy should include all potential recurring costs, and clearly describe the ground rules and assumptions underlying the estimation of each cost element; 2. To improve the quality and transparency of the Navy’s estimate, the Navy should thoroughly document the life-cycle costs associated with homeporting a nuclear-powered aircraft carrier at Naval Station Mayport. Specifically, documentation should identify the source data used, their reliability, and how the data were normalized, describe the steps used in developing the overall estimate so that it can be clearly understood and easily replicated, and describe in sufficient detail the estimating methodology and calculations performed to derive each element’s cost; and 3. To improve the accuracy and credibility of its cost estimate, the Navy should assign a single office with the responsibility for assembling the overall estimate into a comprehensive and well documented package and for performing a sensitivity and risk and uncertainty analyses on the overall estimate to identify the major cost drivers, extent to which estimates could vary due to changes in key cost assumptions, and level of confidence in the estimate.

Agency Comments and Our Evaluation

In written comments on a draft of this report, DOD partially concurred with two and nonconcurred with one of our three recommended actions. DOD also provided technical comments that we have incorporated into this report where applicable. DOD’s written comments are reprinted in appendix IV.

DOD partially concurred with our two recommendations that the Secretary of Defense direct the Secretary of the Navy to improve the (1) comprehensiveness of the Navy’s life-cycle cost estimate for the planned homeporting of a nuclear-powered aircraft carrier at Naval Station Mayport by including all potential recurring costs and clearly describing the ground rules and assumptions underlying the estimation of each cost element and (2) quality and transparency of the Navy’s estimate by thoroughly documenting the life-cycle costs associated with the planned homeporting. DOD stated that the elements, or cost estimating best practices, cited in our recommendations may contribute to improved quality in cost estimates, but they may not be ideally suited to the myriad situations in which the Navy is required, or may be requested, to provide cost estimates to Congress.

We recognize that it may not have been feasible for the Navy to always follow cost estimating best practices to the fullest extent in developing some of its initial estimates, especially when the preparation time for some estimates may have been limited. However, both recommendations were directed at improving future Navy cost estimates associated with the Mayport carrier homeporting. As the Navy continues to respond to congressional interest, we believe it has the opportunity to more completely incorporate the cost estimating best practices we identified in our report in its revised estimates for Mayport in order to improve the quality of its estimates. Additionally, DOD states that the similarity between our initial estimate and the Navy’s budget estimate for dredging costs suggests that the Navy’s programming/budgeting process leads to high-quality, credible cost estimates. However, the Navy’s overall Mayport estimate currently includes several cost elements that have not been refined through a programming/budgeting process. Furthermore, as described in our report, GAO’s criteria for assessing high-quality, credible estimates involve factors that go beyond the accuracy of any individual point estimate. The convergence between our estimate and the Navy’s estimate on a single cost element is not an indication by itself of the quality of the Navy’s overall estimate for homeporting a nuclear carrier at Mayport. Improving the Mayport estimate’s quality will allow the Navy to more easily defend its plans in future decisionmaking and budgets within DOD. We believe that in order to best inform the budget decisionmaking process of the likely costs, affordability, and scheduling of funding needed to support the Navy’s plans for Mayport, DOD should take every available opportunity to provide Congress with the highest quality cost estimates possible. We therefore believe these two recommendations remain valid.

DOD disagreed with our third recommendation that the Secretary of Defense direct the Secretary of the Navy to improve the accuracy and credibility of its cost estimate by assigning a single office with the responsibility for assembling the overall estimate into a comprehensive and well documented package and for performing sensitivity and risk and uncertainty analyses on the overall estimate. DOD stated that responsibilities for centralized supervision and coordination of programming and budgeting within the Department of the Navy are well- established and that Navy program/budget estimates and exhibits are prepared and submitted in accordance with the Office of the Secretary of Defense guidance. DOD further noted that reports to Congress are typically assigned to a central office for preparation and coordination and that where necessary, instructions to the offices providing inputs can include guidance on baseline assumptions.

As we state in our report, the Navy’s level of coordination used in packaging its final cost estimate did not ensure that the offices providing inputs adequately and completely documented the cost estimates and did not provide for the implementation of sensitivity and risk and uncertainty analyses. Additionally, the Navy’s final cost estimate was not consistent in its use of various baseline assumptions, such as consistency in using either then-year or constant dollars in the individual original estimates that made up the Navy’s final estimate for Mayport. This lack of consistency limits the Navy’s ability to appropriately combine the estimates from different offices into a single, coherent cost estimate. High-quality cost estimating requires good organization and a consistent approach in pulling together disparate data for each cost element and packaging it as an accurate and credible whole. Thus, without a single office—within the already established programming/budgeting process—that is responsible for assembling future revisions of its Mayport estimate, the Navy will continue to lack assurance that its overall estimate is comprehensive, well documented, accurate, and credible. We therefore believe this recommendation remains valid.

We are sending copies of this report to interested congressional committees, the Secretary of Defense, the Secretary of the Navy, and the Director, Office of Management and Budget. The report is also available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staff have any questions concerning this report, please contact me at (202) 512-4523 or leporeb@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix V.

Appendix I: Scope and Methodology

To develop an independent cost estimate of the full life-cycle costs associated with the Navy’s plan to establish a second East Coast homeport for a nuclear-powered aircraft carrier we worked with a consultant from a cost estimating company—Technomics, Inc.—with expertise in preparing estimates of the full life-cycle costs of major federal programs. We followed the 12 steps for developing a high-quality cost estimate outlined in the GAO Cost Estimating and Assessment Guide.

Step 1: Define Estimate’s Purpose

The purpose of the estimate is to satisfy the direction provided to GAO in House Report 111-491, accompanying a proposed bill for the Fiscal Year 2011 National Defense Authorization Act (H.R. 5136), to develop an independent estimate of the total direct and indirect costs to be incurred by the federal government for the proposed homeporting of an nuclear- powered aircraft carrier at Naval Station Mayport. The estimate will provide the defense committees with an independent cost estimate against which they can evaluate the Navy’s estimate and give Congress greater confidence that it has an accurate understanding of the total cost associated with establishing a homeport for a nuclear-powered aircraft carrier at Naval Station Mayport.

Step 2: Develop Estimating Plan

Upon identification of the direction in the committee report, we identified resources and developed a design matrix that included identification of information required and sources, and the proposed scope and methodology for addressing the direction. Additionally, in accordance with Generally Accepted Government Auditing Standards (GAGAS), we conducted an entrance conference with the Navy, collected cost data, and conducted interviews with program officials.

Step 3: Define Program Characteristics

We collected information directly from the Navy that enabled us to define the program’s technical and programmatic baseline. Specifically, the Navy provided us with its estimates of one-time and recurring costs, including any supporting documentation that outlined the underlying assumptions and ground rules supporting the estimates. We also reviewed the Navy’s environmental impact study along with background information about nuclear-powered aircraft carrier maintenance, visited locations where nuclear-powered aircraft carrier maintenance is performed, and spoke with Navy officials knowledgeable about the personnel and facilities requirements for nuclear-powered aircraft carrier maintenance.

Step 4: Determine Estimating Structure

We developed a cost element structure that was derived from the Navy's cost estimate and supporting documentation. Where necessary, we added subelements or additional elements. At the highest level of our estimate, costs were categorized by one-time costs (1.0 elements) and recurring costs (2.0 elements). For each cost element, the most appropriate estimating approach was identified according to data availability and credibility. Estimating approaches applied included analogy, expert opinion, and parametric methods.

Step 5: Identify Ground Rules and Assumptions

Estimating ground rules and assumptions were identified and documented. Assumptions were minimized, relying instead on real data when possible. In order to develop an independent estimate, we generally used methods and data that differed from the Navy’s, relying instead on two types of estimating methods: scaled analogy and parametric. Scaled analogy is a cost estimating method that relies on using actual costs from analogous projects. Adjustments to cost were made for the relative size of the analogous project as well as appropriate adjustments for complexity factors. A parametric estimate is a method that uses an established relationship between cost and one or more input variables. The input variables become parameters in an equation that is used to predict cost. It is important to note, however, that the Navy was the only source of information for certain cost subelements, such as the number of man-days of maintenance required for aircraft carrier availabilities and the cost of transporting equipment for the carrier air wing and this limited the degree to which our cost estimate is completely independent from the Navy’s for those elements.

Step 6: Obtain Data

Cost, technical, and programmatic data were solicited and collected through interviews and supporting documentation. Interviews were conducted with and supporting documentation was obtained from officials in the Office of the Chief of Naval Operations, Naval Sea Systems Command, Regional Maintenance Centers, as well as officials from Norfolk Naval Shipyard, Naval Station Mayport, and Naval Air Station North Island. Specific documents and data we collected included Navy responses to information requests, programming models, and construction planning documents. Additionally, we collected and reviewed data on unrelated but analogous Navy and DOD construction projects. In addition, we analyzed the uncertainty associated with our estimates to help gauge their accuracy. We assessed the reliability of all data provided by the Navy by (1) reviewing existing information about the data and the systems that produced them, (2) interviewing agency officials knowledgeable about the data to determine the steps taken to ensure the accuracy and completeness of the data, and (3) reviewing the related internal controls. Based on this assessment, we determined the data were sufficiently reliable for our purposes.

Step 7: Develop Point Estimate

Data collected were thoroughly analyzed, resulting in the development of several different cost estimating methodologies. These methodologies were integrated into a comprehensive cost model using Microsoft ® Excel, resulting in a detailed point estimate.

Step 8: Conduct Sensitivity Analysis

A comprehensive sensitivity analysis was conducted on the cost model also using Microsoft ® Excel. Sensitivity analysis addresses the question of which cost model inputs have the greatest impact on the overall cost estimate. Inputs that have the greatest impact are often referred to as cost drivers. Our sensitivity analysis identified all of the inputs used in each cost element. Each input was changed, in isolation, to determine the impact on the bottom line cost. Some inputs, such as geographic adjustment factors, have an impact on multiple elements. Other inputs only impact a specific cost element.

Step 9: Conduct Risk and Uncertainty Analysis

Risk and uncertainty analysis was conducted in order to transform the static point estimate into a probabilistic range based on the risk and uncertainty inherent in the model input variables. Within the risk analysis section of our estimate, each cost input and each cost estimating method was assigned a low and high value based on the relative uncertainty associated with each value. Inputs that had little or no uncertainty (such as actual costs, or known square footages) had a small, or possibly a zero range between the high and low values. In contrast, variables with a high level of uncertainty had a wider range. We then assigned a triangular probability distribution to each element, defined as either a low value, a most likely value (the point estimate), or a high value. Using a Monte Carlo simulation, the risk analysis was then assigned a random value, according to the probability distribution of each variable. Total cost was then calculated and the process was repeated to obtain a range of possible total cost outcomes. The simulation resulted in establishing probability distributions about the estimated values, enabling the identification of the confidence level of the point estimate.

Step 10: Document the Estimate

Detailed documentation was generated that provides the cost estimate results, as well as ground rules and assumptions, and all underlying cost estimating methodology. The cost model was thoroughly referenced to source documents and then traced and verified in accordance with generally accepted government auditing standards.

Step 11: Present Estimate to Management for Approval

The resulting cost estimates were presented to various levels of GAO management. Additionally, GAO presented the cost estimate and underlying methodology in detail to the Navy in January 2011.

Step 12: Update the Estimate to Reflect Actual Costs and Changes

Upon presenting the cost estimate and underlying methodology in detail to Navy, the Navy provided comments, questions, and some additional data which we incorporated as appropriate in our estimate.

Within each cost element, we compared our estimate to the Navy's. Wherever possible, the reasons for significant differences were discussed with the Navy and explained in the report. Our estimate does not estimate or evaluate any of the benefits of a potential homeport move, and therefore does not address or evaluate the military advantages or disadvantages associated with the homeporting decision. Additionally, our estimate only includes estimated costs incurred by the Navy and it does not include any estimate of the economic impact to the local economy in Virginia or Florida.

To determine to what extent the estimate adheres to the characteristics of a high-quality cost estimate, we evaluated the Navy’s life-cycle cost estimate to determine whether it met key characteristics identified in the GAO Cost Estimating and Assessment Guide. Our guide, which is based on extensive research of best practices for estimating program schedules and costs, indicates that a high-quality, valid, and reliable cost estimate should be well documented, comprehensive, accurate, and credible. We analyzed the cost estimating practices used by the Navy against these best practices. Specifically, to determine whether the Navy’s estimate was well documented, we examined whether the Navy estimate’s documentation explicitly identified the primary methods, calculations, results, rationales or assumptions, and sources of data used to generate each cost element. To determine whether the estimate was comprehensive, we examined whether the Navy’s cost estimate included all life-cycle costs required to homeport a nuclear aircraft carrier at Naval Station Mayport. To determine whether the Navy’s estimate was accurate, we verified the calculations in the Navy’s estimate and examined whether the estimate accounted for all costs, including indirect costs. Finally, to determine whether the Navy’s estimate was credible, we examined the extent to which the Navy tested key cost elements for sensitivity or used other cost estimating techniques to cross-check the reasonableness of their ground rules and assumptions. We also interviewed Navy officials from the Naval Sea Systems Command and the Office of the Chief of Staff of Naval Operations about the process used to prepare the life-cycle cost estimates and the assumptions used to prepare the Navy’s estimate. We shared the GAO Cost Estimating and Assessment Guide and the criteria against which we would be evaluating the Navy’s cost estimate with the Navy.

We conducted our performance audit from July 2010 through February 2011 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Additional Information on GAO’s Independent Cost Estimate

Nuclear-Powered Aircraft Carrier

Contents

Purpose of the Estimate Program Characteristics Estimating Structure Ground Rules and Assumptions Methodology Overview Comparison of GAO’s 65 Percent Confidence Level

Purpose of the Estimate

Satisfy the direction in House Report 111-491, accompanying

a proposed bill for the Fiscal Year 2011 National Defense Authorization Act (H.R. 5136), to develop an independent estimate of the total direct and indirect costs to be incurred by the federal government for the proposed homeporting of a nuclear-powered aircraft carrier at Naval Station Mayport.

Provide the congressional defense committees with an independent cost estimate against which they can evaluate the Navy’s estimate and give Congress greater confidence that it has an accurate understanding of the total cost associated with establishing a homeport for a nuclear- powered aircraft carrier at Naval Station Mayport, Florida.

Program Characteristics

The controlled industrial facility and ship maintenance support facilities in

Mayport have been modeled on the facilities built at Naval Air Station North Island, San Diego, California in the late 1990s to support the first nuclear- powered aircraft carriers to be homeported in San Diego.

Nuclear-powered aircraft carrier maintenance is performed on a 32-month cycle that includes two 1-month long Carrier Incremental Availabilities and either one 6-month Planned Incremental Availability or a 10.5-month Docking Planned Incremental Availability. There is also one refueling complex overhaul around the carrier’s midlife that lasts about 39 months.

The Carrier Incremental Availabilities and Planned Incremental Availabilities will be performed in Mayport, while the Docking Planned Incremental Availabilities will be performed at Norfolk Naval Shipyard.

Estimating Structure

The cost element structure used in this estimate was derived

from the Navy’s cost estimate and associated documentation. Where necessary, we have added subelements or additional elements. At the highest level, costs are categorized by one- time costs (1.0 elements) and recurring costs (2.0 elements).

Estimating Structure

2.1 Permanently Assigned Labor for Nuclear Facilities 2.2 Base Operating Support for New Facilities 1.3.1 Parking 1.3.2 Road Improvements 1.3.3 Wharf F Improvements 1.3.4 Controlled Industrial Facility (CIF) 1.3.5 Ship Maintenance Facility (SMF) 1.3.6 Maintenance Support Facility (MSF)

Ground Rules and Assumptions

In order to develop an independent estimate, we generally

use methods and data that differ from the methods and data used in the Navy’s cost estimate. However, the Navy is likely the only source of information for certain cost subelements such as the number of man-days of maintenance required for aircraft carriers and the cost of transporting equipment for the carrier air wing.

The time frame for one-time costs will be consistent with the Navy’s estimate, which spans from fiscal year 2010 to fiscal year 2019.

Ground Rules and Assumptions

The recurring costs have been annualized over the 50-year

carrier life cycle. Consistent with the Navy’s estimate, we have not identified an exact start date for the recurring costs.

We estimated the cost differentials associated with moving a nuclear-powered aircraft carrier to Mayport. This means that our estimate only includes costs that differ from what the Navy would spend under the current mode of homeporting all East Coast carriers in Norfolk, Virginia. This assumption is consistent with the Navy’s cost estimate.

Ground Rules and Assumptions

Both one-time and recurring costs will be reported in base-year

2010 dollars, which have the effects of inflation removed. In calculations where inflation needs to be added or removed, the estimate uses the appropriation-specific cost indexes published annually by the Naval Center for Cost Analysis.

We do not estimate or evaluate any potential economic impacts that may be experienced by either the Norfolk or Mayport communities from the planned homeporting. As directed in House Report 111- 491, accompanying a proposed bill for the Fiscal Year 2011 National Defense Authorization Act (H.R. 5136), our estimate is limited to the costs to be incurred by the federal government in homeporting a nuclear-powered aircraft carrier at Naval Station Mayport.

Methodology

To develop the cost estimates, we worked with an outside cost

estimating company with expertise in preparing estimates of the life-cycle costs of major federal acquisitions.

We then conducted a sensitivity analysis to determine which of the cost model inputs had the greatest impact on the overall cost element. Those inputs are known as cost drivers.

Methodology

Next, we conducted a risk and uncertainty analysis as a way to

assess the variability in the point estimate. Risk analysis addresses the uncertainty that is inherent within cost inputs and cost estimating methods. Inputs that have little or no uncertainty (such as sunk costs, or known square footages) will have a small, or possibly a zero range between the high and low values. In contrast, variables with a high level of uncertainty have a wider range.

The results of the risk analysis can also be viewed as an “S-Curve”, which plots various risk-adjusted cost totals against a cumulative confidence level. The probabilities shown on the Y axis of an S curve represent the probability that costs will be equal to or less than the amount shown on the X axis. For example, to obtain the risk-adjusted cost estimate at which there is a 90 percent probability that the true costs will be equal to or less than the risk-adjusted total, determine the dollar value from the X axis where the S curve crosses 90 percent.

Overview

Our independent cost estimate suggests that the one-time cost for

establishing a nuclear-powered aircraft carrier homeport at Naval Station Mayport is expected to be between $258.7 million and $356.0 million, with a 65 percent confidence level estimate of $322.0 million. The recurring costs are expected to be between $9.0 million and $17.6 million per year, with a 65 percent confidence level estimate of $14.3 million.

The Navy’s point estimate of $537.6 million for the one-time cost falls outside the upper range of our estimate, but the Navy’s point estimate of $15.3 million/year for recurring costs does fall within our estimated range. The largest difference between our estimate and the Navy’s is in the one- time construction costs for the new controlled industrial facility and ship maintenance support facilities in Mayport.

The parameters related to the construction of the controlled industrial facility and the ship maintenance support facilities are the cost drivers for the one-time costs, while the parameters related to the travel cost are the cost drivers for the recurring costs.

Total Homeporting Cost

One-Time Cost

It is not statistically appropriate to add each of the individual 65 percent confidence level cost estimates to come up with an overall 65 percent confidence level. The overall 65 percent confidence level is calculated separately from the individual elements.

Construction Cost

It is not statistically appropriate to add each of the individual 65 percent confidence level cost estimates to come up with an overall 65 percent confidence level. The overall 65 percent confidence level is calculated separately from the individual elements.

Recurring Costs

It is not statistically appropriate to add each of the individual 65 percent confidence level cost estimates to come up with an overall 65 percent confidence level. The overall 65 percent confidence level is calculated separately from the individual elements.

Risk-adjusted Range: One-Time Cost

The 65 percent confidence level estimate ($322.0 million) is plotted,

along with the 10 percent ($258.7 million) and 90 percent ($356.0 million) confidence levels, which when reported as a range represent an 80 percent confidence interval.

A confidence interval describes a range of possible costs, based on a specified probability level that is determined through a risk analysis. The low and high values of the confidence interval can be expressed as confidence levels that represent the probability that the point estimate will be met. Our 80 percent confidence interval is the range of costs from the 10 percent confidence level to the 90 percent confidence level. At the 10 percent confidence level, there is a 10 percent chance that the cost will be at or below the estimate and a 90 percent chance that the cost will be above the estimate. Likewise at the 90 percent confidence level, there is a 90 percent chance that the cost will be at or below the estimate and a 10 percent chance that the final cost will be above the estimate.

Risk-adjusted Range: One-Time Cost

Risk-adjusted Range: Recurring Costs

The 65 percent confidence level estimate ($14.3 million) is

plotted, along with the 10 percent ($9.0 million) and 90 percent ($17.6 million) confidence levels, which when reported as a range represent an 80 percent confidence interval.

Risk-adjusted Range: Recurring Costs

One-Time Cost Element: Planning and Design

The Navy’s estimate is outside of GAO’s risk-adjusted range.

Both GAO and the Navy estimated planning and design as a percentage of the overall construction cost. The difference between the two planning and design estimates reflects the difference in the overall estimates of construction costs.

One-Time Cost Element: Initial Dredging

GAO risk-adjusted range - $31.3 million to $33.1million GAO 65 percent confidence level estimate - $ 32.3 million Navy point estimate - $ 46.3 million

GAO originally used a scaled analogy based on similar dredging at Norfolk Naval Shipyard and produced an estimate similar to the Navy’s. However, we have updated our estimate for dredging at Mayport to reflect the cost of the contract that was awarded in August 2010. The contract amount was $26.6 million, and the GAO estimate includes overhead and contingency costs. The Navy stated that the contract price came in lower than the estimate due to the widely varying nature of costs in the dredging industry. According to a Navy official, the dredging price is largely based on equipment availability. If equipment happens to be available at the required time, then the dredging cost will be lower.

One-Time Cost Element: Parking Garage

The Navy’s estimate is within the GAO risk-adjusted range.

GAO’s estimate is based on cost per square foot, adjusted with cost factors for supervision, inspection, and overhead; contingency; and a geographic factor for northeast Florida. The Navy’s estimate is based on a detailed engineering estimate developed by a private engineering firm.

One-Time Cost Element: Road Improvements

The Navy’s estimate is within GAO’s risk-adjusted range.

GAO’s estimate is based on a scaled analogy of road construction projects for other locations. The Navy’s estimate is based on a detailed engineering estimate developed by a private engineering firm.

One-Time Cost Element: Wharf F Improvements

The Navy’s estimate is within GAO’s risk-adjusted range.

GAO’s estimate is based on an average of analogous wharf construction projects from other locations. Adjustments were made for size, geographic location, and inflation. The Navy’s estimate is based on a detailed engineering estimate developed by a private engineering firm.

One-Time Cost Element: Controlled Industrial Facility

GAO risk-adjusted range - $35.2 million to $94.9 million GAO 65 percent confidence level estimate - $70.5 million Navy estimate - $139.1 million

The Navy’s estimate is not within GAO’s risk-adjusted range. The Navy’s estimate is based on a detailed engineering study developed by a private engineering firm. The GAO estimate is based on historical costs observed for analogous facility construction projects at Naval Station North Island (San Diego). These analogous costs are adjusted to account for differences in size, inflation, geographical differences between the two areas, and any identified complexities or requirements unique to the Mayport construction. Those complexity factors included construction to withstand Class 4 hurricane storm surge and costs associated with compliance with environmental standards and with antiterrorism and force protection. These factors were included in GAO’s estimate based on a percentage of the total construction cost. No other specific complexity factors that would account for the discrepancy in the two estimates have been identified by either GAO or the Navy.

One-Time Cost Element: Ship Maintenance Support Facilities

The Navy’s estimate is substantially outside of GAO’s risk-adjusted range.

The Navy’s estimate is based on a detailed engineering study developed by a private engineering firm. The GAO estimate is based on historical costs observed for analogous facility construction projects at Naval Station North Island. These analogous costs are adjusted to account for differences in size, inflation, geographical differences between the two areas, and any identified complexities or requirements unique to the Mayport construction. Those complexity factors included compliance with environmental standards and with antiterrorism and force protection. These factors were included in GAO’s estimate based on a percentage of the total construction cost. No other specific complexity factors that would account for the significant difference between the two estimates have been identified by either GAO or the Navy.

One-Time Cost Element: Initial Equipment Outfitting

The Navy’s estimate is outside of GAO’s risk-adjusted range.

GAO’s estimate is based on a scaled analogy of equipment purchased at Naval Air Station North Island, San Diego. The Navy’s estimate is based on a detailed estimate that lists all equipment purchase requirements, with the cost of commercial pricing estimated for each item.

One-Time Cost Element: Initial Permanent Change of Station

The Navy’s estimate is outside of GAO’s risk-adjusted range.

GAO’s estimate is based on published DOD dislocation allowance rates, by rank, for crew members with and without dependents. It assumes a single permanent change of station move when the homeport location is changed. The Navy’s estimate appears to be based on additional costs other than the dislocation allowance rates, but the exact nature of those costs was not specified.

Recurring Cost Element: Permanently Assigned Labor

The Navy’s estimate is within GAO’s risk-adjusted range. GAO’s

estimate uses the Navy’s assumption of 12 full-time employees at a rate of $100,000 per year. We confirmed that this assumption is consistent with the staffing level and compensation for employees working at the nuclear carrier maintenance facilities at Naval Air Station North Island, San Diego. GAO’s 65 percent confidence level estimate is slightly higher than the Navy’s point estimate because it has been adjusted through a risk and uncertainty analysis, whereas the Navy’s estimate has not.

Recurring Cost Element: Base Operating Support

The Navy’s estimate is within GAO’s risk-adjusted range. GAO’s

estimate uses the Navy’s assumptions of 10 full-time guards at $75,000 per year and $225,000 per year for facilities services based on a fixed unit cost per square foot. The unit cost is based on an average cost for Naval Station Mayport in fiscal year 2008. GAO’s 65 percent confidence level estimate is slightly higher than the Navy’s point estimate because it has been adjusted through a risk and uncertainty analysis, whereas the Navy’s estimate has not.

Recurring Cost Element: Facilities Sustainment

The Navy’s estimate is within GAO’s risk-adjusted range.

Both estimates are based on facilities sustainment rates published in the DOD Facilities Pricing Guide, a program analysis and budget tool that lists replacement, sustainment, modernization, and operation unit costs for DOD facilities. Adjustments are made for geographic location.

Recurring Cost Element: Facilities Restoration and Modernization

The Navy’s estimate is outside GAO’s risk-adjusted range. GAO’s

estimate is based on modernization factors published in the DOD Facilities Pricing Guide. Each factor is multiplied against the estimated construction cost. The Navy’s estimate assumes a 67- year recapitalization rate. The modernization factors and 67-year recapitalization rate result in very similar percentages, but because the Navy’s construction estimates are higher, the restoration and modernization rates are correspondingly higher in the Navy’s estimate.

Recurring Cost Element: Operations

The Navy’s estimate is within GAO’s risk-adjusted range.

GAO’s estimate is based on the assumptions and amounts used in the Navy’s estimate. GAO’s 65 percent confidence level estimate is slightly higher than the Navy’s point estimate because it has been adjusted through a risk and uncertainty analysis, whereas the Navy’s estimate has not.

Recurring Cost Element: Travel for Navy Shipyard Workers

The Navy’s estimate is within GAO’s risk-adjusted range. GAO’s

estimate is based on a scaled analogy of travel expenses incurred during a 2009 planned incremental availability on the USS Ronald Reagan. Adjustments were made for inflation and differing airfare and per diem rates in the Jacksonville, Florida area. The Navy’s estimate is based on a programming model that breaks down the travel expense into component costs for airfare, per diem, hotel tax, rental car, and gas.

Recurring Cost Element: Biennial Maintenance Dredging

The Navy’s estimate is within GAO’s risk-adjusted range.

GAO’s estimate is based on the assumptions and amounts used in the Navy’s estimate. For this cost element, GAO’s 65 percent confidence level estimate is identical to the Navy’s point estimate even though the Navy’s estimate was not adjusted through a risk and uncertainty analysis like GAO’s.

Recurring Cost Element: Basic Allowance for Housing

The Navy’s estimate is outside GAO’s risk-adjusted range. Both estimates

are based on published rates, by rank, for crew members with and without dependents. The difference between the estimates is due to the fact that GAO’s estimate assumes a crew size of 3,140, which is consistent with the Navy’s Environmental Impact Study and other sources. The Navy’s estimate assumes a crew size of 2,728, which lowers the total savings. allowance is less in Jacksonville, Florida than it is in Norfolk, Virginia. This means that there is a net savings for the Navy for this cost element.

Recurring Cost Element: Utilities

The Navy’s estimate is within GAO’s risk-adjusted range.

Both estimates are based on the same assumed amount of power usage multiplied by an average power cost differential. GAO’s estimate uses average power costs published by the Department of Energy for Virginia and Florida, which is a differential of 2.51 cents. The Navy’s estimate assumes a 2.00 cent differential.

Recurring Cost Element: Recurring Permanent Change of Station

Recurring Cost Element: Private Sector Travel

GAO risk-adjusted range - $0.1 million to $1.2 million GAO 65 percent confidence level estimate - $0.6 million Navy point estimate - N/A To account for the risk that private sector shipyards may have to bring in workers from outside of the Mayport area during surges in workload and thereby incur travel expenses that would be passed on to the federal government, the GAO estimate calculates potential private sector travel costs using the public sector travel and per diem rates. The GAO estimate assumes a 60/40 public/private ratio for maintenance work in Mayport and sets the most likely private sector travel cost at zero, with a high-value risk input that assumes 30 percent of the private sector workforce has to travel from outside of the Mayport area.

Appendix III: GAO’s Assessment of the Navy’s Cost Estimating Practices

Appendix III: GAO’s Assessment of the Navy’s Cost Estimating Practices The cost estimate includes all life-cycle costs.

The cost estimate completely defines the program, reflects the current schedule, and is technically reasonable.

The cost estimate work breakdown structure is product-oriented, traceable to the statement of work/objective, and at an appropriate level of detail to ensure that cost elements are neither omitted nor double- counted.

The estimate documents all cost-influencing ground rules and assumptions.

The documentation should capture the source data used, the reliability of the data, and how the data were normalized.

The documentation describes in sufficient detail the calculations performed and the estimating methodology used to derive each element’s cost.

The documentation describes step by step how the estimate was developed so that a cost analyst unfamiliar with the program could understand what was done and replicate it.

The documentation discusses the technical baseline description and the data in the baseline are consistent with the estimate.

The documentation provides evidence that the cost estimate was reviewed and accepted by management.

The cost estimate results are unbiased, not overly conservative or optimistic, and based on an assessment of most likely costs.

The estimate has been adjusted properly for inflation.

The estimate contains few, if any, minor mistakes.

The cost estimate is regularly updated to reflect significant changes in the program so that it is always reflecting current status.

Variances between planned and actual costs are documented, explained, and reviewed.

The estimate is based on a historical record of cost estimating and actual experiences from other comparable programs.

The cost estimate includes a sensitivity analysis that identifies a range of possible costs based on varying major assumptions, parameters, and data inputs.

A risk and uncertainty analysis was conducted that quantified the imperfectly understood risks and identified the effects of changing key cost driver assumptions and factors.

Cost drivers were cross-checked to see whether results were similar.

An independent cost estimate was conducted by a group outside the acquiring organization to determine whether other estimating methods produce similar results.

Not met provided complete evidence that satisfies the entire criterion.

Appendix IV: Comments from the Department of Defense

Appendix V: GAO Contact and Staff Acknowledgments

Acknowledgments

In addition to the contact listed above, key contributors to this report include Mark J. Wielgoszynski, Assistant Director; Darnita Akers; Shawn Arbogast; David Brown (Technomics, Inc.); Russell Bryan; Jennifer Echard; Mary Jo LaCasse; Erik Wilkins-McKee; and Michael Shaughnessy.