Defense Acquisitions: Action Is Needed to Provide Clarity and Mitigate Risks of the Air Force's Planned Advanced Battle Management System
Released: 2020-04-16T00:00:00

Letter

Background

The Air Force has identified ABMS as its solution to support broad Department of Defense (DOD) efforts to develop Joint All-Domain Command and Control (JADC2) capabilities. These capabilities will eventually allow U.S. forces from all of the military services, as well as allies, to conduct military operations across all warfighting domains. Command and control is the collection and sharing of information to enable military commanders to make timely, strategic decisions; take tactical actions to meet mission goals; and counter threats to U.S. assets. Figure 1 shows the concept of DOD operations within a joint all-domain environment.

When the Air Force began planning for ABMS in 2017, officials stated the intent was to replace and modernize the capabilities of the Airborne Warning and Control System (AWACS), which provides the warfighter with the capability to detect, identify, and track airborne threats, among other capabilities. According to officials, the Air Force currently plans to operate AWACS aircraft through 2035. In July 2018, the DOD Joint Requirements Oversight Council approved an ABMS Initial Capabilities Document that describes which capabilities would need to be developed and which associated gaps in current capabilities the Air Force would need to address.

According to Air Force officials, after the Initial Capabilities Document was approved, the Air Force determined that its planned approach to ABMS was no longer compatible with the most recent National Defense Strategy, released in January 2018. The 2018 National Defense Strategy outlines DOD’s strategy for maintaining the defense of the United States based on new and reemerging threats from competitors, such as Russia and China. It also defines expectations for how DOD and its military departments should be prepared to engage those threats during future conflicts: forces would be expected to strike a diverse range of targets inside adversarial air and missile defense networks; forces would need capabilities to enhance close combat lethality; and

DOD would prioritize investments that enabled ground, air, sea, and space forces to deploy, operate, and survive in all domains while under attack.

Air Force officials stated that these expectations led the department to reassess requirements for ABMS and assess new options for developing more robust and survivable systems that could operate within contested environments. For example, the Air Force officially canceled a recapitalization program for the Joint Surveillance Target Attack Radar System (JSTARS)—an aircraft that provides surveillance and information on moving ground targets—in December 2018. The cancellation was linked to the 2018 National Defense Strategy, which calls for a more survivable and networked solution, among other things. A June 2018 Air Force report to Congress identified concerns regarding the survivability of the JSTARS aircraft in a contested environment and stated that the Air Force was instead planning for ABMS to eventually provide JSTARS’s capabilities. The Air Force determined that it could continue using some of its JSTARS aircraft into the 2030s.

Officials stated the Air Force subsequently changed the scope and intent of ABMS to align with the 2018 National Defense Strategy and broader requirements for JADC2. According to senior Air Force officials, they concluded that, to align with the new defense strategy, ABMS needed to do far more than replace AWACS and JSTARS. They also concluded that no single platform, such as an aircraft, would be the right solution to providing command and control capabilities across multiple domains. In an April 2019 congressional testimony, the Air Force announced a new vision for ABMS as a multidomain command and control family of systems enabling operations in air, land, sea, space, and cyber domains. In that testimony, Air Force leadership explained the need to move away from a platform-centric approach (such as JSTARS) to a network-centric approach, one that connects every sensor to every shooter. The Air Force, however, did not formally document its decision to change the scope of ABMS. In November 2019, according to Air Force officials, ABMS was determined to be the Air Force solution for JADC2 in response to a July 2019 Joint Requirements Oversight Council memo outlining DOD requirements for command and control systems requirements across all domains.

In May 2019, we reported that Air Force leadership determined that it would not designate ABMS as a major defense acquisition program because it would be a family of systems. The Air Force also determined that ABMS would be directed by a Chief Architect working across PEOs, rather than a traditional acquisition program manager. According to Air Force officials, the Chief Architect role will be instrumental in integrating the various programs and technologies into an overall system and is the first of its kind within the Air Force.

Additionally, Air Force officials stated that they intend to use a flexible acquisition approach to develop ABMS, one that is outside of traditional pathways such as a major defense acquisition program or middle tier acquisition. According to the Chief Architect, this approach will allow ABMS to develop and rapidly field capabilities. Specifically, the Air Force intends to break up technology development into many short-term efforts, generally lasting 4 to 6 months each. The Chief Architect stated that the goal of breaking up development into smaller increments is to increase innovation by requiring multiple contractors—including those that may not usually engage with DOD—to compete for contracts more frequently. These short-term efforts will include prototyping and demonstrations to prove that the capabilities work. Those that are proven will be delivered to the warfighter. By using this approach, the Air Force intends to field capabilities sequentially and more quickly than if all were developed and delivered at one time as is typically done for traditional acquisitions. Additionally, Air Force officials indicated that this approach will not lock the Air Force into long-term development efforts with just one contractor and will allow the Air Force to more easily move on from unsuccessful development efforts.

The Air Force Has Not Established a Business Case for ABMS, Increasing Development Risks

The Air Force has not established a plan or business case for ABMS that identifies its requirements, a plan to attain mature technologies when needed, a cost estimate, and an affordability analysis. As a result of recent ABMS management and scope changes, the Air Force remains early in the planning process and has not yet determined how to meet the capabilities or identify systems that will comprise ABMS. In December 2019, Air Force officials stated an overall plan for ABMS did not exist and would be difficult for the Air Force to develop in the near term due to the unclear scope of ABMS requirements. To date, the Air Force has not identified a development schedule for ABMS, and it has not formally documented requirements.

As previously stated, ABMS will be managed as a family of systems and not as a traditional acquisition program typically governed by DOD Instruction 5000.02, nor as a middle tier acquisition. As a result, Air Force officials initially told us that they did not intend to develop most of the typical acquisition documentation, such as a cost estimate, that is generally required of major defense acquisition programs before entering the development phase. In March 2020, after we sent a copy of this report to DOD for comment, the Air Force provided us a draft tailored acquisition plan for ABMS in lieu of an acquisition strategy. Based on our initial review, this document includes some elements of a traditional acquisition strategy, such as contract and test strategies. However, this tailored acquisition plan does not include key information such as the overall planned capabilities and estimated cost and schedule for ABMS. We will continue to monitor the Air Force’s planning efforts as the program progresses. The Air Force also began preparing an analysis of alternatives in January 2019 to assess options for delivering capabilities such as surveilling moving targets and battle management command and control. The Air Force expects to complete the analysis in 2020, but Air Force officials expect it will inform only some aspects of ABMS planning. The Air Force has not defined what additional planning documentation it will develop to help it establish a business case for ABMS. For example, major defense acquisition programs are generally required to develop acquisition planning documents, such as a cost estimate.

We have previously reported on the importance of establishing a solid, executable business case before committing resources to a new development effort. A business case demonstrates that (1) the warfighter’s needs are valid and that they can best be met with the chosen concept and (2) the chosen concept can be developed and produced within existing resources. In addition to an acquisition strategy, other basic elements of a sound acquisition business case include firm requirements, a plan for attaining mature technologies, and a reliable cost estimate and affordability analysis, further described below. 1. Firm requirements are the requisite technological, software, engineering, and production capabilities needed by the user. Acquisition leading practices state that requirements should be clearly defined, affordable, and informed. Deciding how best to address requirements involves a process of assessing trade-offs before making decisions. Unstable or ill-defined requirements can lead to cost, schedule, and performance shortfalls. 2. A plan to attain mature technologies when needed is critical in establishing that technologies can work as intended before integration into a weapon system. The principle is not to avoid technical risk but rather address risk early and resolve it ahead of the start of product development. Identifying technologies and defining a plan to ensure mature technologies can be attained when needed help guide development activities and enable organizations to track development and inform decisions on next steps. 3. A reliable cost estimate and affordability analysis are critical to the successful acquisition of weapon systems. GAO’s Cost Estimating and Assessment Guide states that a reliable cost estimate is comprehensive, well-documented, accurate, and credible. Leading practices have shown that realistic cost estimates allow program management to obtain the knowledge needed to make investment decisions and match requirements with resources. A cost estimate is the basis of an affordability analysis, which validates whether a program’s budget is adequate for the planned acquisition strategy.

The process of developing and documenting a business case builds knowledge needed to match customer needs with available resources, including technologies, timing, and funding. The fact that the Air Force does not plan to establish such a business case for ABMS increases the risk of cost and schedule overruns and may impact Congress’s ability to exercise its oversight responsibilities. The status of key elements for the ABMS business case follows: Status of requirements. The Air Force has not established well-defined, firm requirements for ABMS, but Congress required that the Air Force start defining requirements for the networked data architecture necessary for ABMS to provide multidomain command and control and battle management capabilities by June 2020. The Air Force has not defined the changes in ABMS’s requirements, such as the need to provide multidomain command and control capabilities in support of joint operations. As a result, the only existing documentation of ABMS’s requirements resides in the ABMS Initial Capabilities Document from 2018, which generally focuses on the capabilities needed to replace AWACS. That document does not address the expanded JADC2 requirements and capabilities ABMS is expected to eventually fulfill. Air Force officials stated that ABMS requirements and the family of systems, or programs, that compose ABMS will be defined over time as they gain more knowledge.

Given the lack of specificity regarding ABMS, Congress has kept a close eye on the effort and has implemented several reporting requirements. Since 2018, the Air Force has been required to provide quarterly updates to the defense committees on the status of ABMS development and associated technologies. In addition, the National Defense Authorization Act for Fiscal Year 2020 required the Air Force to provide ABMS-related documentation that describes certain requirements, a development schedule, and the current programs that will support ABMS, among other things, by June 2020.

While the Air Force has not established firm requirements for ABMS to date, it has informally identified some broad requirements. For example, the Air Force anticipates that ABMS will provide interoperability between systems, present real-time information to military decision makers, and fully utilize the range of sensor data and capabilities across DOD to create a common battlespace operational picture. In addition, Air Force officials stated that ABMS would be developed as a government-owned open architecture family of systems, which would allow any system to be integrated into ABMS.

The Air Force has identified seven different development categories that it plans to simultaneously address to meet its broad ABMS requirements. According to the Air Force, the categories are not intended to be comprehensive and may change as development progresses. These development categories include:

Apps Although the Air Force has not defined these seven development categories, it has identified 28 development areas that fit within the categories. For example, one of these development areas, which falls under the “secure processing” category, is called cloudONE. It is intended to store and process data using a cloud infrastructure for multiple levels of classified and unclassified data. These development areas will eventually compose the architecture and technologies that make up ABMS. In January 2020, the Air Force provided us with a draft version of high-level descriptions of the 28 development areas; however, the document did not fully define the requirements or capabilities for the development areas nor identify which organizations would lead each effort. For example, the cloudONE description does not indicate specific technical requirements that must be met, such as amount of storage, the number of users, or data transmission rate.

Although ABMS requirements are not fully defined, the Air Force awarded several short-term development contracts for ABMS. According to Air Force officials, these efforts are intended to show that its nontraditional development approach is feasible rather than to develop specific capabilities that will be integrated into ABMS. For example, the Air Force awarded several development contracts totaling approximately $8 million for gatewayONE, one of the 28 development areas that is intended to enable communication between platforms. As part of this effort, the Air Force conducted a demonstration in a joint military exercise in December 2019. While the exercise demonstrated some data transfer capability, it did not directly address the intent of gatewayONE to enable communication between multiple platforms using government-owned systems. According to Air Force officials, ongoing and future efforts will allow the Air Force to better define ABMS requirements and determine what existing and emerging technologies can fulfill those capabilities. The Air Force has not determined what development efforts will follow these early demonstration efforts, in part because it has not fully defined its requirements.

Status of plan to attain mature technologies when needed. The Air Force has started development activities without first identifying what technologies are needed for the 28 development areas for ABMS. According to Air Force officials, they do not plan to identify all technologies needed while pursuing development activities. Therefore, the Air Force cannot assess whether technologies required for ABMS are mature or determine the necessary steps to ensure those technologies are mature when needed. Air Force officials stated that as ABMS development progresses, they plan to select commercially available or other mature technologies for integration. However, without first identifying the technologies it needs, the Air Force cannot develop a plan, or technology roadmap, with detailed actions to ensure those technologies will be mature when needed. For example, the Air Force plans for ABMS to assume the capabilities of AWACS and JSTARS aircraft, which are set to retire in the 2030s. However, the Air Force has not defined the technologies ABMS will need or established a roadmap to ensure those technologies are mature before the retirement of legacy aircraft. This increases the risk that the requisite technologies will not be mature before the Air Force must integrate them into ABMS, which increases the likelihood that those capabilities will not be developed when needed.

The Chief Architect and other Air Force senior leaders stated that the ABMS development effort is an ambitious undertaking for the Air Force. Our prior work has found that some DOD programs related to ABMS development have posed challenges in the past, in part because technologies were not sufficiently mature when needed, as shown in table 1.

Additionally, the Office of Cost Assessment and Program Evaluation assessed previous DOD programs that were similar to ABMS development and noted that the scope of ABMS will be larger than any of those individual programs. Officials from that office concluded that ABMS is a high-risk effort and the Air Force has not provided sufficient programmatic detail. As a result, they could not conclude that the Air Force would be able to overcome the cost, schedule, and performance challenges of these past programs. Air Force officials stated that the Air Force’s approach to ABMS development will avoid these past challenges because only mature technologies will be integrated into ABMS and the Air Force is expected to frequently evaluate development progress. However, since the Air Force has not identified what the technology needs for ABMS are, it cannot yet determine if those technologies are mature or will be mature when needed. We have previously found that starting development without first identifying and assessing the maturity of technologies increases the likelihood that those technologies are not mature when needed, which often results in cost overruns and schedule delays.

Status of cost estimate and affordability. The Air Force has not developed a cost estimate for ABMS or an affordability analysis. According to the GAO Cost Estimating and Assessment Guide, even in cases where limited information is available, cost estimates should still be developed to inform budget requests. To date, the Air Force has requested nearly $500 million for ABMS efforts through fiscal year 2021. The Air Force, however, currently has no plans to develop a life-cycle cost estimate, which would provide a comprehensive account of ABMS costs, or an independent cost estimate, which would confirm the credibility of estimated costs.

Officials stated that the Air Force has not developed a cost estimate because the capabilities, technologies, and systems that will compose ABMS are still to be determined and will change over time. Officials stated they intend to develop cost estimates for each of the 28 development areas in the future but did not identify a timeline. The GAO Cost Estimating and Assessment Guide acknowledges that cost estimating is more difficult when requirements—and the technologies and capabilities to meet them—are changing and the final product design is not known while the system is being built. In these cases, leading practices call for developing cost estimates that should be updated more frequently to reflect changes in requirements. Without a realistic and current cost estimate for ABMS efforts, the Air Force will be unable to effectively allocate resources and conduct informed long-range investment planning.

The Air Force has also not determined if it can afford ABMS. Affordability is the degree to which the funding requirements for an acquisition effort fit within the service’s overall portfolio plan. Whether an acquisition effort is affordable depends a great deal on the quality of its cost estimate and other planned outlays. To conduct an affordability analysis, the budget requirements for the entire portfolio are identified for future years. This can help determine whether the funding needs are relatively stable or if the portfolio will require a funding increase in the future. The GAO Cost Estimating and Assessment Guide states that, as part of the cost estimating process, management should review and approve an affordability analysis to identify any funding shortfalls. Air Force officials stated that the Air Force does not plan to conduct a comprehensive affordability analysis for ABMS because it is managing it as a family of systems. They stated that any costs to the Air Force will be determined in the future by the various organizations that manage the systems that will eventually support ABMS. However, without an affordability analysis, the Air Force will be unable to determine whether it can commit sufficient resources for ABMS in future years.

Air Force Has Established an ABMS Management Structure, but Decision-Making Authorities Are Unclear

While the Air Force has taken some steps to establish an ABMS management structure, the authorities of Air Force offices to plan and execute ABMS efforts are unclear. Internal controls, which provide standards on effective management of programs, state that management should establish the organizational structure and authority necessary to enable the entity to plan, execute, control, and assess the organization in achieving its objectives. The Air Force, however, has not fully defined or communicated ABMS decision-making authorities to Air Force offices, and documentation to date regarding ABMS management has been limited. Several Air Force offices are involved in ABMS management, as shown in figure 2.

Air Force Acquisition. This office is headed by the Assistant Secretary of the Air Force for Acquisition, Technology and Logistics, who is generally responsible for all acquisition functions within the Air Force. In an October 2018 memorandum, Air Force Acquisition established the position of the Chief Architect and stated that any unresolved ABMS issues between the Chief Architect and PEOs are to be brought to Air Force Acquisition for resolution.

Chief Architect. The Air Force established this position in October 2018 to execute the overarching vision and strategy for ABMS. According to the Air Force, the Chief Architect will determine the overall design of ABMS, coordinate with the service-level commands and the acquisition programs involved to ensure their efforts are aligned with the overall design and development of ABMS, and identify the enabling technologies that will compose the ABMS family of systems.

An October 2018 memorandum stated that individual PEOs and program managers that oversee programs supporting ABMS will retain all authority and responsibility for executing their respective programs.

In November 2019, Air Force Acquisition issued additional ABMS management guidance that stated that the Chief Architect would select and fund ABMS development projects for PEOs to execute. However, the guidance did not address whether the Chief Architect has authority to direct the execution of efforts initiated and originally funded by the PEOs, which may support ABMS. Specifically, there is no documentation to clarify whether the Chief Architect would have the authority to realign PEO priorities or funding for ABMS projects. For example, the PEO for Space is currently executing a data integration project, which aligns with the cloudONE development area. Although some ABMS funds have been obligated for this project, there is no documentation to support that the Chief Architect will be able to direct the PEO to change the project objectives or timeline to align with ABMS requirements once they are defined.

Air Force Warfighting Integration Capability (AFWIC). In October 2017, the Air Force established AFWIC. According to Air Force officials, AFWIC will ensure forces are operationally ready to perform JADC2 missions using ABMS technologies. According to an AFWIC senior official, in April 2019 AFWIC began leading multidomain command and control efforts for the Air Force. An October 2018 memorandum directed the Chief Architect to coordinate with AFWIC regarding the development of ABMS. Other documentation on ABMS execution indicates that AFWIC will also coordinate with major commands on Air Force doctrine and operations in support of ABMS. However, the documentation did not further define this coordination or indicate whether AFWIC would have any authority in directing ABMS activities.

Chief Architect Integration Office. In December 2019, the Air Force established the Chief Architect Integration Office at Wright-Patterson Air Force Base to coordinate and integrate ABMS development efforts across PEOs and other organizations. Air Force officials stated that this office is in the process of being staffed and the roles and responsibilities still need to be formalized. However, as currently envisioned, this office would lead technology development risk reduction efforts by working with the PEOs and other organizations, such as federally funded research and development centers, to conduct ABMS demonstrations and prototypes. Air Force officials told us the Chief Architect Integration Office is expected to resolve issues across Air Force organizations, such as sharing of resources and personnel. An Air Force Life Cycle Management Center- led task force is currently developing an overall strategy for the office, to include resource and organizational requirements. Air Force officials stated that a proposed strategy will be completed in March 2020. Until the Chief Architect Integration Office has been fully established, it is unclear whether the office will have the required authorities to execute the mission of integrating ABMS development efforts across the Air Force.

Air Force officials stated that the decision-making authorities across these offices will be developed over time. According to officials, details on these authorities have not been developed or communicated to the offices supporting ABMS and the Air Force has not established a timeline for doing so. The Air Force expects that multiple organizations within the Air Force will be responsible for executing ABMS development efforts. Internal controls, which provides standards for effective management of programs, states that organizational structure and authority is necessary to plan, execute, and assess progress. The absence of fully defined and documented decision-making authorities, which are communicated to all those involved, increases the risk to the Air Force’s ability to successfully plan, execute, and assess ABMS development efforts.

Conclusions

The Air Force started ABMS development activities without a business case that defines ABMS requirements, a plan to ensure technologies are mature when needed, a cost estimate, and an affordability analysis. Developing these key elements of a business case helps to build a solid foundation for any successful technology and product development effort, even one using a nontraditional acquisition approach. Congress has already required the Air Force to define and report on certain ABMS requirements, among other aspects of ABMS planning, by June 2020. However, the Air Force does not intend to develop the other elements of a business case, even though it is requesting over $300 million for ABMS development activities in fiscal year 2021. Given the criticality of the battle management command and control mission and the planned retirement of legacy programs, the lack of an ABMS business case introduces uncertainty regarding whether the needed capabilities will be developed within required time frames. For example, without a plan to mature technologies needed to field ABMS capabilities, the Air Force cannot be certain those technologies will be ready when needed.

While it may be difficult for the Air Force to formulate a complete ABMS business case at this time, due to the recent changes in ABMS’s scope, the Air Force is not precluded from beginning the process of defining and formalizing a business case. As ABMS continues to evolve, so too can the Air Force’s business case. For example, the Air Force does not yet know the total life cycle costs of ABMS, but it could provide Congress with a cost estimate based on its knowledge today and update the cost estimate over time. This would allow the Air Force to assess whether ABMS is affordable. Furthermore, the Air Force is already required to provide quarterly briefs to congressional defense committees on the status of ABMS, which affords the Air Force the opportunity to present Congress with information on its ABMS business case and explain any changes over time. Specifically, including updates on the scope of the Air Force’s plans to ensure ABMS will have mature technologies when needed, an overall cost estimate, and an affordability assessment would provide important information to Congress.

Finally, the Air Force has started to execute ABMS development efforts without clearly defined decision-making authorities that have been communicated to the offices supporting those efforts. The absence of these defined authorities may hinder management’s ability to execute and assess ABMS development across multiple organizations within the Air Force.

Recommendations for Executive Action

We are making the following four recommendations to the Secretary of the Air Force to direct the Assistant Secretary of the Air Force for Acquisition, Technology and Logistics: The Assistant Secretary of the Air Force for Acquisition, Technology and Logistics should direct the Chief Architect to develop a plan to attain mature technologies when needed for each ABMS development area, which includes an initial list of technologies and an assessment of their maturity that is updated to reflect changes, and update Congress quarterly. (Recommendation 1)

The Assistant Secretary of the Air Force for Acquisition, Technology and Logistics should direct the Chief Architect to prepare a cost estimate that is developed in accordance with cost estimating leading practices, to include regularly updating the estimate to reflect ABMS changes and actual costs, and update Congress quarterly. (Recommendation 2)

The Assistant Secretary of the Air Force for Acquisition, Technology and Logistics should direct the Chief Architect to prepare an affordability analysis that should be regularly updated, and update Congress quarterly. (Recommendation 3)

The Assistant Secretary of the Air Force for Acquisition, Technology and Logistics should formalize and document acquisition authority and decision-making responsibilities of the Air Force offices involved in the planning and execution of ABMS, to include the Chief Architect. This document should be included as part of the submission to Congress in June 2020 and communicated to the Air Force offices that support ABMS. (Recommendation 4)

Agency Comments

We provided a draft of this product to the Department of Defense for comment. In its comments, reproduced in appendix I, the Department of Defense concurred with our recommendations. We will continue to monitor the Air Force’s actions to respond to these recommendations.

We are sending copies of this report to the appropriate congressional committees. We are also sending a copy to the Secretary of Defense, the Secretary of the Air Force, and other interested parties. In addition, this report is available at no charge on GAO’s website at http://www.gao.gov.

Should you or your staff have questions, please contact me at (202) 512- 4841 or MakM@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix II.

Appendix I: Department of Defense Comments

Appendix II: GAO Contact and Staff Acknowledgments

GAO Contact:

Staff Acknowledgments:

In addition to the contact above, the following staff members made key contributions to this report: Justin Jaynes, Assistant Director; Jessica Karnis, Analyst-in-Charge; and Lauren Wright. Other contributions were made by Brian Bothwell, Rose Brister, Brian Fersch, Miranda Riemer, Megan Setser, Hai Tran, and Robin Wilson.