Presidential Helicopter Acquisition: Program Makes Progress in Balancing Requirements, Costs, and Schedule
Released: 2013-04-09T00:00:00

Letter

Background

The Marine Corps’ HMX-1 squadron uses a fleet of 19 VH-3D and VH- 60N helicopters to transport the President in the national capital region, as well as when the President is traveling in the continental U.S. and overseas locations. These aircraft have been in service for decades. The events following the September 11, 2001, terrorist attacks on the United States highlighted the need for improved transportation, communication, and security capabilities for the presidential helicopter fleet. As a result, a program (subsequently designated the VH-71 program) was initiated in April 2002 to develop aircraft to replace the helicopters currently in service. Initial plans to field the VH-71 by 2011 were accelerated in response to a November 2002 White House memorandum directing that a replacement helicopter be fielded by the end of 2008. By 2009 significant cost growth, plus schedule delays and performance issues resulted in the decision to terminate the VH-71 program. At the time of termination, in June 2009, the estimated VH-71 program cost had doubled from about $6.5 billion at development start in 2005 to $13 billion. Because there remained a need to replace the current in-service presidential helicopters, the Office of the Under Secretary of Defense for Acquisition Technology and Logistics (OUSD(AT&L)) directed the Navy by late June 2009 to present a plan to develop options for a new program to acquire replacement aircraft, now designated VXX.

The Navy’s VXX efforts began immediately with the initiation of an AOA to assess options on how to proceed toward developing and fielding the replacement presidential helicopter. It was focused, at least in part, on one of the primary lessons learned from the VH-71 program experience— the need to establish and maintain a sound business case. A sound business case is one in which a balance is established between requirements, costs, and schedule that reflects an executable program with acceptable risk. According to program officials, the program would be aligned to pursue a best practices knowledge-based acquisition approach with the intent of establishing and maintaining an affordable business case.

Last year, we reported that VXX program’s entry into development had been delayed as the program worked to provide a sound initial business case for development, which is a best practice that was not followed by the terminated VH-71 program. The Navy had produced an initial AOA report under June 2010 study guidance from DOD’s Office of Cost Assessment and Program Evaluation (CAPE). This initial work presumed an acquisition strategy under which the program would start in the technology development phase of DOD’s acquisition process. The Navy released this initial AOA report to DOD. While CAPE found this initial work sufficient, OUSD(AT&L) did not accept its results. Rather, it identified the need for a 2012 update to address using a streamlined acquisition strategy focused on mitigating cost drivers identified in the 2010 AOA study. Additional guidance was provided by the Office of the Secretary of Defense (OSD) in December 2011. That guidance reflected insights on requirements gained in the 2010 study and expectations of using a streamlined acquisition approach proposed by the Navy. The proposed approach would leverage mature technologies being developed outside of the program before including them on aircraft selected for the program with their adoption being facilitated by open systems architectures. This would allow the program to start with Milestone B approval for engineering and manufacturing development (EMD) and then selecting an existing in-production commercial or military platform and working to integrate communications and mission systems provided by the government, which are expected to be mature by that time. Figure 1 depicts the program’s entry into DOD’s acquisition process as currently anticipated.

DOD is investing in the current fleet of presidential helicopters to increase their service life and address capability gaps while working to field VXX aircraft. The current inventory of 19 aircraft are sometimes stressed to meet operational demands—demands that have been growing—making it difficult to take them out of service for lengthy upgrades. A larger VXX inventory of 21 aircraft is expected to help address this.

Program Makes Progress Toward Establishing a Sound Business Case

The Navy has made progress in the past year toward establishing a sound business case for development that reflects a rational balance between requirements, costs and schedule. The Navy completed the AOA, which was deemed sufficient by CAPE to inform future acquisition decisions, and OSD has approved the program to proceed to a Milestone B decision. The CAPE did note, however, some areas of caution, for example, that some air vehicles would require aggressive efforts to manage the weight of the VXX while other air vehicles would be more challenged in other respects. We reviewed that AOA and found that it included elements of a sound AOA.

Navy Completes AOA

The Navy, building on its initial 2010 study, completed its AOA on April 4, 2012, and concluded that the currently proposed acquisition approach of using mature technologies from outside the program on an in-production commercial or military helicopter was acceptable. The initial 2010 study, which considered nine alternative aircraft, revealed that technology development and recertification of aircraft for airworthiness were primary cost drivers of the total projected program cost under the approach it presumed. The 2012 updated study (focused on the most promising aircraft) assessed that mature, certified, and capable in-production commercial and military aircraft exist that can be modified for presidential requirements and be procured under the proposed strategy using a “Customized” rather than a “Min Mod” approach. It determined that there are candidate aircraft with performance characteristics that can meet to varying degrees the February 2012 draft Capability Development Document (CDD) requirements used to conduct the study and found that the Navy’s proposed streamlined acquisition strategy is feasible and would reduce the program’s expected schedule, cost, and risk.

Specifically, the 2012 study estimates suggest that using the proposed approach of having the program enter the acquisition process in the engineering and manufacturing development phase rather than the technology development phase, as was anticipated in the 2010 study, would reduce investment cost by approximately $1.5 billion (19.7 percent) and shorten the development schedule by about 18 percent. The 2012 study also revealed, however, that the alternatives offered varying degrees of individual system performance with no single alternative meeting all of the VXX requirements. For example, while one alternative met most of the requirements, it would require strict weight and requirements management throughout its life to avoid a more costly Min Mod approach. While other alternatives provided differing capacities for weight growth, they would be challenged in meeting other requirements, such as range, transportability, landing zone suitability, or material supportability requirements. Further, for all of the alternatives in the 2012 study, it was found that if they were required to meet the military’s airworthiness standards (as opposed to the certifying authority standard they currently meet), the weight growth associated with meeting some of these standards would likely trigger a more costly Min Mod approach.

The 2012 VXX AOA study team made a number of recommendations including:

To mitigate aircraft certification risk, the airworthiness certifying authority actively participate in all government development activities for the cockpit, communications, and mission systems and be involved in the source selection process for the aircraft;

To reduce the risk of having to resort to a Min Mod approach, an active and aggressive life cycle weight management effort should be put in place if the selected platform does not provide a large enough margin to accommodate future weight growth; and

The release of a request for proposals be contingent upon achieving acceptable technical maturity of critical government developments, such as communications and mission systems.

AOA Accepted and VXX Program Approved to Proceed

The Director of CAPE in a May 30, 2012, memorandum concluded that the 2012 study achieves a logical outcome and was sufficient to inform future acquisition decisions. The CAPE found that the AOA demonstrates that each of the alternatives examined can be provided in a manner consistent with the streamlined acquisition approach, though with assessed limitations as described in the report. It concluded that the study also shows that an approach that avoids recertification was feasible for some of the options considered, and, if adopted, offers potential for reduced cost and schedule.

There are some areas of caution, however, in the CAPE’s assessment.

The alternatives examined in the 2012 study identified similar mission limitations as those seen in the 2010 study. In particular, it was noted that some air vehicles studied exhibited weight sensitivities that would require aggressive weight management for the program’s lifecycle. Other air vehicles exhibited better performance in some aspects including allowing for weight growth, but were more challenged in others, for example, landing zone suitability and transportability.

The AOA did not examine the integration risk of the government- developed communications package and mission systems—key system components that under the Navy’s acquisition strategy are being developed outside of the program but must then be successfully integrated into the selected aircraft.

The CAPE’s assessment also stated that the validity of the study results was contingent on the reduced requirements in the draft CDD—reduced from the requirements for the VH-71 acquisition—becoming finalized as documented and that a return to the previous requirements would require additional analysis. The Joint Requirements Oversight Council (JROC) subsequently approved the CDD on January 3, 2013. While program officials informed us that there were changes to requirements reflected in the approved CDD, they stated that none would affect the validity of the AOA or require the additional analysis mentioned in CAPE’s memo. A CAPE official subsequently informed us that they had reviewed the CDD and do not believe that any of the changes affect the AOA’s validity. In addition, the official stated that they reviewed the requirements changes and were satisfied that they made sense.

In an August 28, 2012 Acquisition Decision Memorandum, the USD (AT&L) approved the VXX to proceed to Milestone B—approval to enter engineering and manufacturing development—as the program’s initial acquisition milestone. The Under Secretary decided that:

Milestone B, scheduled for 3rd quarter of fiscal year 2014, will be the first formal acquisition system milestone for VXX; however, a Pre- EMD Review will occur prior to release of the Request for Proposals for development, integration, and production;

Prior to the pre-EMD review, the Navy and the JROC are to approve the CDD and the Director, Cost Assessment and Program Evaluation is to develop an Independent Cost Estimate based on the approved CDD; and

The Under Secretary will establish affordability targets for the VXX program concurrent with CDD approval by the JROC.

The Under Secretary concluded that based on utilization of mature technologies and a proven, mature, existing aircraft, this approach would not require a technology development phase. In addition, the Under Secretary expects to waive a requirement in the Weapon Systems Acquisition Reform Act of 2009, as amended, for competitive prototyping because its anticipated cost outweighs the expected risk reduction and life cycle benefit it would provide.

Our Assessment of the AOA

In our prior two reports on this acquisition, we stated that when the AOA was issued we would assess it for its robustness—the range of alternatives it considered, its depth of analysis, and its consideration of trade-offs. Based on our review of the AOA report, supporting material, and interviews of program and other defense officials, we found the AOA to be sufficient for this stage of the acquisition. It included elements that GAO has reported should be part of a robust AOA. We also found it used a cost estimating process that was substantially compliant with GAO identified best practices.

An AOA compares the operational effectiveness, suitability, and life-cycle cost estimates of alternatives that appear to satisfy established capability needs. Cost estimating and analysis are significant components of an AOA. We have previously reported on the importance of a robust AOA as a key element in ensuring a program has a sound, executable business case prior to program initiation. Our work has found that programs that conduct a limited AOA (failing to consider a broad range of alternatives or assess technical and other risks associated with each alternative) tended to experience poorer outcomes—including cost growth.

We found that the AOA study team considered a broad range of alternatives. The initial 2010 study effort evaluated 9 platforms and 19 possible alternatives to satisfy the mission and the 2012 update studied the most promising platforms in the 2010 study to document the impact of the Navy’s proposed streamlined acquisition strategy would have on the merits of each of those alternatives. The study team assessed effectiveness, suitability, technical, schedule, and operational risks associated with the alternatives, though, according to CAPE, it did not assess the risks of integrating government-furnished communications and mission control systems into those alternatives. The study director noted that this integration risk could not be assessed by the study team, given the maturity of these subsystems at the time. Rather, the AOA identified the need to consider this issue at a future engineering review when sufficient maturity existed and an accurate assessment could be made.

The AOA process reflected and influenced performance trade-offs. The initial 2010 analysis was based on performance requirements that were lower in a number of areas than for the VH-71 program. The 2012 AOA study reflected additional trade-offs made with regard to cost, schedule, risk and performance. The performance trade-offs enabled the Navy’s revised strategy, and is expected to result in reduced costs and schedule. Following the 2012 study the performance requirements were further refined as reflected in the final CDD. Table 1 illustrates some of the performance trade-offs made by comparing the minimum requirements for VXX aircraft as captured in the final CDD to the minimum VH-71 requirements captured in the equivalent Operational Requirements Document for that program.

We also assessed the cost estimating procedure for the AOA using GAO’s criteria for cost estimating and assessment and found that it was substantially compliant with those criteria. For the purposes of this review, we collapsed the best practices identified in the GAO Cost Estimating and Assessment Guide into four general characteristics: well documented, comprehensive, accurate, and credible. The cost estimating best practices associated with each of those characteristics used in judging the AOA are provided in appendix I.

We found the AOA cost estimate to be comprehensive and well documented. We also found that it was substantially accurate and partially met our criteria for being credible. The AOA cost estimate was properly adjusted for inflation, relied on historical analogous aircraft data, contained no significant calculation errors, and had been recently updated from the estimate contained in the 2010 study. While the documentation stated that the estimate reflected most likely costs, it did not specifically identify potential contingency costs and no cost risk analysis was performed to determine a level of confidence for the cost estimate. As result, we were unable to determine if the costs were indeed most likely. In addition, the AOA cost estimate was deemed as having partially met the best practices criteria for being credible as there was evidence that a robust sensitivity analysis had been performed but not an independent cost estimate or cost risk analysis.

Although there was not a cost risk analysis, a detailed technical risk assessment process was followed for identifying technical risks, their likelihood of occurring, and the consequences if they occurred. The technical risks were mitigated by incorporating costs into the cost estimate through derivation of realistic and reasonable staffing levels and sufficient schedule for design, development and testing of each alternative. In addition, while an independent cost estimate had not been conducted, the USD(AT&L)’s August 28, 2012, acquisition decision memorandum directs CAPE’s completion of one prior to the Pre-EMD Review. This is to occur in the second quarter of fiscal year 2013. As a result, the program could have confirmation of the AOA cost estimating results at that point. The program will then continue to work on its cost estimate to result in a more refined estimate supporting the Milestone B decision in the third quarter of fiscal year 2014.

Efforts Intended to Leverage Technologies into VXX Aircraft Progress

The Navy’s currently proposed acquisition approach relies on the government’s providing, as government furnished equipment, mature technologies for integration into aircraft. Those technologies either already exist or are in development, some as legacy fleet aircraft upgrades. Their provision will be an important factor in the Navy’s achieving the reduced cost and schedule it seeks through its proposed acquisition approach. The program assesses their risks for integration into VXX aircraft as low—supporting the Navy’s goal of providing initial operational capability in fiscal year 2020.

While the program reports the key technologies to be provided by the government for integration are on track, there are risks that they will not work out as planned. For example, the Navy had originally anticipated that the cockpit technologies leveraged into the VXX acquisition would include a “glass cockpit” display system installed as an upgrade on VH- 60N aircraft. It dropped its planned use of this system. Adopting its use would likely necessitate an airworthiness recertification of the platform selected to be the VXX aircraft, a costly and time-consuming endeavor. As a result, the Navy now plans on the prime contractor using the display systems already in use in its certified aircraft—systems that the program manager noted are as capable if not more capable than the VH-60N’s.

Even if individual technologies work out as anticipated, they will still have to be successfully integrated in the aircraft. The program depends on a number of government-defined sub-systems and technologies being hosted in a new airframe. Environmental issues such as size, weight, thermal profile, and stability will have to be ascertained, not separately but in totality as a dynamic system. Only then will it be known whether key performance parameters are met, how closely, and what, if any, refinements need to be applied. In the past, we found integration issues can be significant. For example, in fiscal year 2004 DOD rebaselined the Joint Strike Fighter program extending its development by 18 months and adding resources to address problems discovered during systems integration and the preliminary design review. To mitigate integration risk, though, the VXX program is making use of a systems integration laboratory and also plans to install the communications and mission systems into a test aircraft and do demonstration testing before integration efforts begin on the VXX platform. Table 2 provides more information on the technologies to be provided by the government for integration in VXX aircraft.

Concluding Observations

The program has made progress toward establishing a sound business case for development, one that rationally balances requirements, costs, and schedule. The program still faces challenges that will need to be actively managed to provide greater assurance that a sound business case is maintained throughout development as the program moves forward. These challenges include:

Maintaining the VXX requirements without significant deviation throughout the acquisition process. Subsequent requirement changes will need to be considered carefully in the context of their implications for cost, schedule, risk, and performance and the program will need to effectively manage technology maturation and integration to achieve success.

Managing weight growth of the platform during development so as to not trigger the need for modifications that could then require a flight recertification of the VXX platform. Also, having a weight margin once fielded will place the program in a better position in the future to more readily enhance the platform over its anticipated 40-year service life.

Ensuring the technologies being developed for integration into the selected VXX platform develop as needed and integration mitigation efforts are adequately planned, resourced, and executed.

Failing to address these challenges could impact the program’s ability to stay on track and delay replacement of the in-service helicopter fleet, which is currently stressed at times to meet demand. Additionally, in our prior reports we described both VH-71 lessons learned and acquisition best practices that, if heeded, should help the program remain on track.

Agency Comments

DOD provided written comments on a draft of this report. The comments are reprinted in appendix II.

In commenting on a draft of this report, DOD stated that it would ensure that mitigations are in place to address potential risk areas. It believes its efforts are aligned with GAO’s best practices and the recommendations in GAO’s 2011 report on the program and plans to continue to monitor program progress in view of these standards.

We are sending copies of this report to interested congressional committees; the Secretary of Defense; the Under Secretary of Defense for Acquisition, Technology and Logistics; and the Secretary of the Navy. This report also is available at no charge on GAO’s website at http://www.gao.gov.

Should you or your staff have any questions on the matters covered in this report, please contact me at (202) 512-4841 or sullivanm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix III.

Appendix I: Cost Estimating Best Practices

Characteristic Comprehensive

The cost estimate includes all life cycle costs. The cost estimate completely defines the program, reflects the current schedule, and is technically reasonable. The cost estimate work breakdown structure (WBS) is product-oriented, traceable to the statement of work/objective, and at an appropriate level of detail to ensure that cost elements are neither omitted nor double-counted. The estimate documents all cost-influencing ground rules and assumptions. The documentation should capture the source data used, the reliability of the data, and how the data were normalized. The documentation describes in sufficient detail the calculations performed and the estimating methodology used to derive each element’s cost. The documentation describes step by step how the estimate was developed so that a cost analyst unfamiliar with the program could understand what was done and replicate it. The documentation discusses the technical baseline description and the data in the baseline is consistent with the estimate. The documentation provides evidence that the cost estimate was reviewed and accepted by management. The cost estimate results are unbiased, not overly conservative or optimistic and based on an assessment of most likely costs. The estimate has been adjusted properly for inflation. The estimate contains few, if any, minor mistakes. The cost estimate is regularly updated to reflect significant changes in the program so that it is always reflecting current status. Variances between planned and actual costs are documented, explained, and reviewed. The estimate is based on a historical record of cost estimating and actual experiences from other comparable programs. The cost estimate includes a sensitivity analysis that identifies a range of possible costs based on varying major assumptions, parameters, and data inputs. A risk and uncertainty analysis was conducted that quantified the imperfectly understood risks and identified the effects of changing key cost driver assumptions and factors.

Major cost elements were cross-checked to see whether results were similar.

An independent cost estimate was conducted by a group outside the acquiring organization to determine whether other estimating methods produce similar results.

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

Key contributors to this report were Bruce H. Thomas, Assistant Director; Jerry W. Clark, Analyst-in-Charge; Bonita J.P. Oden; Karen A. Richey; Jennifer K. Echard; Tisha D. Derricotte; Marie P. Ahearn; Hai V. Tran; and Robert S. Swierczek.