Defense Acquisitions: Navy Strategy for Unmanned Carrier-Based Aircraft System Defers Key Oversight Mechanisms
Released: 2013-09-26T00:00:00

Letter

Background

The Navy’s UCLASS system will be the first unmanned aircraft system deployed on an aircraft carrier. Efforts to develop an unmanned combat air system for the Navy can be traced back to 2003 when DOD established a joint Navy and Air Force program called the Joint Unmanned Combat Air System (J-UCAS). This joint effort drew on knowledge that the Air Force had gained through early development of the Unmanned Combat Air Vehicle, an effort that began in the late 1990s. The J-UCAS program was canceled in late 2005. The following year, the Navy initiated the Unmanned Combat Air System Demonstration (UCAS- D) program—the immediate predecessor to UCLASS—with the intent to design, develop, integrate, test, and demonstrate the technical feasibility of operating unmanned air combat systems from an aircraft carrier. In 2013, the Navy successfully launched and landed a UCAS-D on an aircraft carrier. In total the Navy invested more than $1.4 billion in the UCAS-D program. In 2011, as UCAS-D efforts were ongoing, the Navy received approval from DOD to begin planning for the UCLASS acquisition program.

In our past work examining weapon acquisition and best practices for product development, we found that leading commercial firms and successful DOD programs pursue an acquisition approach that is anchored in knowledge, whereby high levels of knowledge are demonstrated at critical junctures. Specifically, there are three critical junctures—knowledge points—in an acquisition program at which decision makers must have adequate knowledge to make large investment decisions. If the knowledge attained at each juncture does not confirm the business case on which the acquisition was originally justified, the program does not go forward. At the first knowledge point, a match must be made between the customers’ needs and the available resources—technical and engineering knowledge, time, and funding— before a system development program is started. At the second knowledge point, about midway through development, the developer must demonstrate that the system’s design is stable and that it can meet performance requirements. At the third knowledge point, the developer must show that the system can be manufactured within cost, schedule, and quality targets and that it is reliable before beginning production. The first knowledge point is the most critical point of the three. At that point programs should present their business case for review and approval, which establishes an acquisition program baseline. This baseline describes the cost, quantity, schedule, and performance goals of a program and provides a framework for effective oversight and accountability. This first knowledge point typically coincides with a substantial financial commitment.

DOD’s acquisition policy and guidance encourage the use of a knowledge-based acquisition approach, in which major decision reviews are aligned with the start of key acquisition phases, including technology development, system development—referred to as engineering and manufacturing development—and production. Figure 1 aligns the knowledge points with key decision points in DOD’s acquisition process.

According to DOD acquisition policy, the purpose of the technology development phase is to reduce technology risk, determine and mature the appropriate set of technologies to be integrated into a full system, and to demonstrate critical technology elements on prototypes. A system level preliminary design review is to be held during the technology development phase to inform requirements trades; improve cost estimation; and identify remaining design, integration, and manufacturing risks. The results of the preliminary design review are to be reported to decision makers at Milestone B—the decision review in DOD’s process that corresponds with knowledge point 1 and initiates system development. The purpose of system development is to develop a system or an increment of capability, complete full system integration, develop an affordable and executable manufacturing process, and demonstrate system integration, interoperability, safety, and utility, among other things.

System development provides a critical opportunity for objective oversight before beginning production.

At Milestone B, major defense acquisition programs are required by DOD policy to have approved requirements, an independent cost estimate, and an acquisition program baseline; begin tracking unit cost changes and report unit cost growth against Nunn-McCurdy statutory thresholds; and periodically report to Congress on the cost, schedule, and performance status of the program in Selected Acquisition Reports. At that time, major defense acquisition programs are also required by statute to present a business case analysis and certify on the basis of that analysis that the program is affordable, has reasonable lifecycle cost and schedule estimates, and that technologies have been demonstrated in a relevant environment, among other things. Taken together, these requirements form the basic oversight framework to ensure that Congress and DOD decision makers are adequately informed about the program’s cost, schedule, and performance progress. In addition, the information is valuable for identifying areas of program risk and its causes, and helps to ensure that decision makers consider the full financial commitment before initiating a new development program. Once initiated at Milestone B, major defense acquisition programs are required to measure program performance against the program’s baseline estimate. Changes to the baseline are only authorized under certain conditions, including a program restructure that is approved by the milestone decision authority, or a breach of the critical Nunn-McCurdy statutory threshold where DOD certifies continuation of the program to Congress.

The Navy’s UCLASS Acquisition Strategy Defers Key Oversight Mechanisms

In fiscal year 2014, the Navy plans to commit to investing an estimated $3.7 billion to develop, produce, and field from 6 to 24 aircraft and modify 1 to 4 aircraft carriers as an initial increment of UCLASS capability— referred to as an early operational capability. The Navy plans to manage UCLASS as if it were a technology development program, although its strategy encompasses activities commensurate with a program in system development and early production. Accordingly, it is not planning to hold a Milestone B review to formally initiate a system development program— which would trigger key oversight mechanisms—until after the initial capability is fielded in fiscal year 2020. This strategy means the program will not be subject to these oversight mechanisms including an acquisition program baseline; Nunn-McCurdy unit cost growth thresholds; and periodic reporting of the program’s cost, schedule, and performance progress. This strategy will likely limit Congress’s ability to oversee this 6- year multibillion dollar program. Navy officials believe that their approach effectively utilizes the flexibility in DOD’s acquisition policy to ensure that UCLASS requirements and concept of operations are well understood and achievable before formally beginning a system development program. Yet they emphasize that by fiscal year 2020 they may have accumulated enough knowledge to allow them to bypass a formal development program and proceed directly to production at Milestone C. Figure 2 illustrates the Navy’s strategy.

As indicated above, the Navy plans to award four firm fixed-price contracts in fiscal year 2013 to competing contractors to develop preliminary designs for the UCLASS air vehicle. The following year, the Navy plans to review those preliminary designs, conduct a full and open competition, and award a contract to develop and deliver the UCLASS air vehicles, effectively ending competition within the air vehicle segment. A review of the full system level preliminary design—including the air vehicle, carrier, and control segments—is scheduled for fiscal year 2015. DOD policy and best practices indicate that around this review point a program would typically be expected to hold a Milestone B review and transition from technology development to system development. Figure 3 illustrates the later point in the process in which the Navy plans to establish the UCLASS acquisition program baseline and formally initiate a development program.

Although the Navy does not plan to hold a Milestone B review until 2020, if at all, it is effectively committing to system development and early production in fiscal year 2015. According to the Navy’s strategy, system development and early production activities, including system integration and air vehicle fabrication, will begin in fiscal year 2015 around the time of the system-level preliminary design review. The Navy also expects to increase annual funding for the UCLASS system from $146.7 million to $522.5 million between fiscal years 2014 and 2015. Testing to demonstrate the system’s capabilities is scheduled to take place from fiscal year 2017—scheduled first flight—through fiscal year 2020, when an early operational capability is expected to be achieved. If the program proceeds according to the Navy’s plan, by 2020, it will have completed many of the activities typically authorized by a Milestone B decision. Moreover, since enough quantities of UCLASS are expected to be delivered for operational use on one or more aircraft carriers, the strategy could also be seen as having begun early production before a Milestone C decision is held.

In a March 2007 report we identified oversight challenges presented by an acquisition strategy that calls for proceeding into system development, demonstration, manufacturing, and fielding without the benefit of a Milestone B decision. A framework of laws make major defense acquisition programs accountable for their planned outcomes and cost, give decision makers a means to conduct oversight, and ensure some level of independent program review. The application of these acquisition laws is typically triggered by a program’s entry into system development. While the activities the UCLASS program plans to undertake exemplify that the program is entering into system development, these laws will not be triggered because the program is not holding a Milestone B review and formally initiating a development program. Therefore, the UCLASS program will not be accountable for establishing a program baseline or for reporting any cost growth to that baseline to DOD and Congress.

UCLASS Faces Cost, Schedule, and Program Management Risks

The UCLASS system faces several risks related to cost, schedule, and program management that, if not addressed, could lead to additional cost and significant schedule delays for the system. The Navy recognizes that many of these risks exist and has mitigation plans in place to address them.

UCLASS cost estimates are uncertain and could exceed available funding: Preliminary cost estimates completed by the Navy indicate that the development and fielding of the initial UCLASS system through fiscal year 2020 could cost between $3.7 and $5.9 billion, all of which is expected to be development funding. However, the Navy has only projected funding of $3.2 billion for the system through fiscal year 2020. The variability in the cost estimates is due largely to cost estimating ground rules and assumptions. For example, Navy officials stated that the $3.7 billion cost estimate reflects an assumed savings of 15 to 20 percent that they believe is achievable since competing contractors’ preliminary designs will be relatively mature. Navy and DOD officials we spoke with emphasized that no true sense of cost will be known until after the air vehicle segment preliminary design reviews have been completed and a single contractor has been selected. If the preliminary designs are less mature than assumed, costs could increase significantly, further exceeding budgeted resources.

Source selection schedule is compressed: After the four competing contractors have completed their preliminary air vehicle designs, the Navy plans to conduct a full and open competition before awarding the air vehicle segment contract. The Navy’s strategy allows for about 8 months between the time that it issues its request for air vehicle proposals and the time it awards the contract. According to OSD officials, this type of contract award process typically takes approximately 12 months.

UCLASS is dependent on development and delivery of other systems: The Navy identifies the delivery of the Common Control System software as a risk and notes that if it is delayed, alternative control system software would be needed to achieve the established deployment timeline. Using alternative software would increase integration costs and extend the testing timeline, resulting in duplicated development, integration, and testing once the common control system software is delivered. The Navy expects this risk to be mitigated over time as individual segments of the control system software are built, delivered, integrated, and tested.

UCLASS is also critically dependent on the development and fielding of the Joint Precision Approach and Landing System (JPALS), which is a global positioning system-based aircraft landing system that guides the aircraft to make a safe landing on the aircraft carrier deck. However, in a March 2013 report, we found that the JPALS program has experienced significant schedule delays. Additional JPALS delays would likely affect the Navy’s UCLASS schedule, in which case the Navy may need to identify an alternative landing system for UCLASS, thus increasing the cost and delaying delivery of the capability. The Navy recognizes this risk. The program office holds weekly integrated master schedule reviews with the JPALS program and plans to mitigate risk through JPALS testing, initial deployments, and continued communication with the JPALS program and other Navy offices.

UCLASS system integration will be challenging: The Navy plans to act as the lead systems integrator for all three segments through the development and fielding of the initial UCLASS system. The Navy will have three separate but interrelated segments to manage, the timing and alignment of which are crucial to success of the overall system. The system is reliant on 22 existing government systems, such as JPALS. The Navy recognizes that there is risk associated with its role as the lead systems integrator, as it does not routinely act in this capacity. Therefore, the Navy plans to manage this risk through interaction with industry and regular system level reviews. According to program officials, this integration effort will require the number of full time equivalent staff in the program office to double from its current level of 150 staff to around 300 staff.

UCLASS Acquisition Strategy Contains Aspects of a Knowledge-Based Approach

While the Navy has not yet established a business case or acquisition program baseline, the UCLASS strategy reflects aspects of a knowledge- based approach. Some of these aspects are discussed in more detail below:

Leveraging significant knowledge gained from prior technology development efforts: The Navy is planning to maximize the use of technologies for carrier-based unmanned aircraft systems operations that have been developed under other efforts like the UCAS-D program, which recently demonstrated the feasibility of launching and landing an unmanned aircraft on an aircraft carrier. Navy officials note that they plan to leverage navigation and control technologies, among other things, from the demonstration program. By effectively leveraging these types of previous investments, along with other existing systems and technologies, the Navy could reduce cost and schedule for the UCLASS system and promote affordability.

Incorporating an open systems design approach: We reported in July 2013 that the Navy is planning to use an open systems approach for the UCLASS system. The Navy has identified key system interfaces and, according to program officials, plans to require contractors to comply with particular open system standards, which it believes will reduce acquisition costs and simplify integration. The Navy also plans to incorporate an open systems architecture developed by OSD for the UCLASS system control segment. This architecture implements a common framework, user interfaces, software applications, and services, and is designed to be common across unmanned aircraft systems. DOD estimates that the open architecture will reduce costs and allow for rapid integration of payloads.

Matching requirements to available resources: In 2012, the Joint Requirements Oversight Council issued a memorandum that required the Navy to reduce its UCLASS requirements because at that time they were deemed unaffordable. The Joint Requirements Oversight Council specifically noted that the Navy’s requirements should focus on achieving an affordable, adaptable platform that supports a wide range of missions within 3 to 6 years. As a result, the Navy scaled down the UCLASS requirements and updated its analysis of alternatives to include requirements that are more affordable and feasible. Our prior work has found that matching requirements with resources before beginning a system development program increases the likelihood that the program will meet cost and schedule objectives.

Holding competition for preliminary designs: In fiscal year 2013, the Navy plans to award four firm fixed-price contracts to competing contractors to develop and deliver preliminary air vehicle designs. The Navy then plans to review those preliminary designs, conduct a full and open competition, and award a single air vehicle segment contract. The Navy believes that this competition will drive efficiencies and ultimately result in cost savings across the system’s life cycle. This strategy reflects recent DOD initiatives that emphasize the importance of competition, which we have noted in the past, can help reduce program costs.

Conclusions

The Navy plans to manage UCLASS as a technology development program, although its strategy encompasses activities commensurate with system development and early production. The Navy believes the strategy provides considerable latitude to manage UCLASS development and to demonstrate significant knowledge before the Milestone B decision. Indeed, we have often reported that programs tend to move forward with Milestone B and system development before they have demonstrated enough knowledge. But the Navy’s plan to develop, manufacture, and field operational UCLASS systems on up to four aircraft carriers before holding a Milestone B decision would defer the decision and mechanisms that would otherwise enable oversight of these very program activities until after they are over. Without a program baseline and regular reporting on progress, it will be difficult for Congress to hold the Navy accountable for achieving UCLASS cost, schedule, and performance goals. As we have noted, these kinds of risks are present in the program and warrant such oversight. Looking ahead to fiscal year 2020, when the UCLASS system is already being delivered, Congress may have few options other than to continue authorizing funding for UCLASS manufacturing and fielding.

If the UCLASS program can be executed according to the Navy’s strategy, it would be consistent with the normal DOD acquisition process that applies to most weapon system programs, with the exception of the deferral of the Milestone B review. In fact, the timing of the Milestone B review notwithstanding, the actual program activities planned are consistent with a knowledge-based acquisition approach. For example, the Navy is leveraging knowledge gained from prior technology development programs, incorporating an open systems design, matching resources with requirements, and utilizing competition. Given the competitive preliminary design process planned and subsequent competitive contract award, it seems reasonable that a Milestone B decision could be held following the competition and before the beginning of system development, providing a solid oversight framework with little or no change to the strategy’s schedule.

Matters for Congressional Consideration

To enhance program oversight and accountability given that the Navy does not plan to modify its acquisition strategy and hold a Milestone B decision review for the UCLASS system following the system level preliminary design review in fiscal year 2015, Congress should consider directing the Navy to hold a Milestone B review for the system after the system level preliminary design review is complete.

If the Navy does not comply, Congress should consider limiting the amount of funding available for the UCLASS system until the Navy provides the basic elements of an acquisition program baseline, such as development and production cost estimates, unit costs, quantities, schedules, annual funding profiles, and key performance parameters needed for such a large investment. The Navy should also be required to periodically report the program’s status against the baseline.

Recommendation for Executive Action

In order to provide for increased congressional oversight and program accountability, we recommend that the Secretary of Defense direct the Secretary of the Navy to hold a Milestone B decision review for the UCLASS system following the system level preliminary design review— which is currently scheduled in fiscal year 2015.

Agency Comments and Our Evaluation

The Navy provided us with written comments on a draft of this report. The Navy’s comments are reprinted in appendix II. The Navy also provided technical comments, which were incorporated as appropriate.

The Navy did not concur with our recommendation to hold a Milestone B decision review for the UCLASS system following its planned system level preliminary design review in 2015. The Navy stated that the Under Secretary of Defense for Acquisition, Technology, and Logistics approved its UCLASS acquisition strategy in 2013 and certified that the strategy was compliant with the Weapon Systems Acquisition Reform Act of 2009, the amendments made to that Act, and DOD policy. The Navy pointed out that DOD’s policy defines the technology development phase as an “iterative process designed to assess the viability of technologies while simultaneously refining user requirements.” The Navy went on to state that the UCLASS user requirements and Concept of Operations will be refined during the early operational capability fleet exercises currently scheduled to begin in fiscal year 2020 and that, at that time, the Navy plans to request approval to hold a Milestone B review to continue development of the UCLASS capability.

While the Navy’s UCLASS acquisition strategy may be compliant with laws and DOD policy, the development, production, and fielding of an operational system before holding a Milestone B review will limit congressional oversight of a significant investment in weapon system development. An estimated development cost of $3.7 billion makes this UCLASS investment larger than the majority of DOD’s current major weapon system development programs. We agree that the technology development phase of an acquisition program is intended to assess the viability of technologies while refining requirements. However, the system development and early production activities included in the Navy’s UCLASS acquisition strategy go well beyond technology development and requirements refinement, and thus warrant oversight commensurate with a major weapon system development program. Thus, we continue to believe that our recommendation is valid and are making two matters for congressional consideration to ensure Congress has information available to oversee the UCLASS system and to hold the Navy accountable for achieving UCLASS cost, schedule, and performance goals.

We are sending copies of this report to the Secretary of Defense, the Secretary of the Navy, and interested congressional committees. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you have any questions about this report or need additional information, please contact me at (202) 512-4841 or sullivanm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

The National Defense Authorization Act for Fiscal Year 2012 mandated that GAO evaluate the Unmanned Carrier-Launched Airborne Surveillance and Strike (UCLASS) system acquisition strategy. This report (1) assesses the Navy’s UCLASS acquisition strategy, (2) identifies key areas of risk facing the UCLASS system, and (3) notes areas where the Navy’s strategy contains good practices.

In order to assess the Navy’s UCLASS acquisition strategy, we collected, reviewed, and compared the UCLASS acquisition strategy with best practice standards for using knowledge to support key program investment decisions. These standards are based on GAO’s extensive body of work in this area. Additionally we compared the Navy’s strategy against DOD acquisition policy.

In order to identify any key areas of risk facing the UCLASS system and note areas where the Navy’s strategy contains good practices, we collected and reviewed additional UCLASS documentation, such as the analysis of alternatives, capabilities development document, and other relevant Navy management documents. We discussed the Navy’s UCLASS acquisition strategy with officials from the UCLASS system program office, the Naval Air Systems Command, the Chief of Naval Operations, and organizations within the Office of the Secretary of Defense (OSD) including the Director of OSD Cost Assessment and Program Evaluation, the Deputy Assistant Secretary of Defense for Systems Engineering, and the Under Secretary of Defense for Acquisition, Technology, and Logistics.

We conducted this performance audit from July 2013 to September 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our finding based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Michael J. Sullivan, (202) 512-4841 or sullivanm@gao.gov.

Staff Acknowledgments

In addition to the contact named above, key contributors to this report were Travis Masters, Assistant Director; Laura Greifner; Julie Hadley; Kristine Hassinger; Laura Jezewski; Matt Lea; John Pendleton; Dr. Timothy M. Persons; and Roxanna Sun.