Defense Acquisitions: DOD Efforts to Adopt Open Systems for Its Unmanned Aircraft Systems Have Progressed Slowly
Released: 2013-07-31T00:00:00

Letter

Background

One way for DOD to minimize the cost and time needed to modify weapon systems is by using an open systems approach for system design and development. An open system allows system components to be added, removed, modified, replaced, or sustained by consumers or different manufacturers in addition to the manufacturer that developed the system. It also allows independent suppliers to build components that can plug into the existing system through the open connections. Fundamental elements of an open systems approach include: designing a system with modular components that isolate functionality. This makes the system easier to develop, maintain, and modify because components can be changed without majorly impacting the remainder of the system. developing and using open, publicly-available standards for the key interfaces, or connections, between the components. Interface standards specify the physical, power, data, and/or other connections between components. All interfaces in a system do not need to use open standards for a system to be considered “open,” and it can be costly and impractical to manage hundreds or thousands of interfaces within a system. Rather, open standards should be identified at key interfaces between the modules that are likely to change, may frequently fail or need to be replaced, or are needed for interoperability. obtaining data rights to interfaces when open standards are not available. DOD describes the acquisition of technical data, such as design drawings, specifications, and standards, as critical to enabling the department opportunities for competition for modification and sustainment of weapon systems throughout their life cycles.

Many consumer products, including U.S. appliances and smartphones, are considered to be open systems because they use widely-available hardware and software standards at key interfaces. For example, U.S. appliances are designed to use a particular wall socket standard, so that they can plug into any power outlet without consumers needing to worry about which brand product is compatible in their homes. Similarly, headphone jacks use a common, open standard, which enables consumers to purchase headphones made by different manufacturers, and plug them into many types of devices, including their MP3 players, cell phones, and stereos, which may be built by other manufacturers. In addition, the AndroidTM operating system, used on smartphones, allows individual software applications introduced by third-party developers to connect to the Android operating system through an open software interface. This allows customers to have a lot of choices and helps keep prices low.

Incorporating an open systems approach prior to the start of development increases the likelihood that open systems considerations are included in program requirements and inform a program’s future competitive strategy, which can significantly reduce upgrade and maintenance costs later. Introducing this approach later in a program’s life cycle, such as a planned modification or upgrade, is more difficult, complex, and costly to do as it may require significant modifications to an already-developed system. Legacy DOD programs that we have reported on in the past did not implement an open systems approach early and are now facing difficult and costly system modifications later in their respective service lives. The B-2 bomber and the C-130 aircraft are two examples:

The Air Force is spending over $2 billion to upgrade the B-2 bomber’s communications, networking, and defensive management capabilities. Because the B-2 program’s prime contractor is the sole system integrator in possession of proprietary technical data and software, there is no opportunity for competition—a critical tool for achieving the best return on investment and driving down costs.

The Air Force planned to replace the aging avionics systems on its C-130 aircraft with open architecture avionics systems. However, the individual cockpits had been modified throughout years of maintenance and differed from one another to such an extent that the upgrades required custom-built hardware and software; less expensive commercial off-the-shelf hardware and software could not be used. Because of the various configurations, C-130 modernization cost estimates increased from $4 billion to over $6 billion and the Air Force significantly reduced the number of planes it planned to modernize from 519 to 221 between fiscal years 2001 and 2011. The Air Force terminated the program in its fiscal year 2013 budget submission because of its escalating costs.

DOD understands the efficiencies of open systems and has taken steps to incorporate an open systems approach into its policy over the past two decades. As far back as 1994, the Under Secretary of Defense for Acquisition and Technology directed the use of an open systems approach in the acquisition of weapon system electronics. That same year, DOD established an Open Systems Joint Task Force to oversee the military services’ transition to open systems and coordinate open systems-related efforts, such as training and standards development. The task force developed several tools to aid program managers with this approach, but was later disbanded. In 1998, the Defense Science Board cited pockets of success in leveraging open systems, but noted that DOD lacked a unifying concept and required a reconfiguration of management processes and aggressive leadership to facilitate open systems implementation. This concern was addressed in DOD’s 2000 acquisition directive and in its most recent update in 2003, which states that “a modular, open systems approach shall be employed, where feasible.”DOD’s 2008 acquisition instruction also provides that “program managers shall employ a modular open systems approach to design for affordable change, enable evolutionary acquisition, and rapidly field affordable systems that are interoperable.”

Recently, the Under Secretary of Defense for Acquisition, Technology and Logistics has placed renewed emphasis on an open systems approach as part of the 2010 and 2012 Better Buying Power initiatives to increase efficiency in defense spending through effective competition. Specifically, the initiatives aim to improve the department’s early planning for open architectures, including making open systems considerations part of the design process.

Open Systems Provide a Variety of Benefits

An open systems approach has great potential to generate efficiencies for product manufacturers, lower total ownership costs for consumers, and transform industries because it spurs industry growth, competition, and innovation. To get the most from open systems, the approach is best implemented at the start of product design because this is where initial modularity, key interface, and data ownership decisions are made and would result in costly redesign if implemented later. Open systems have been successfully used by commercial companies, such as those in the personal computer and software markets, and has helped customers avoid sole source prices and manufacturers address obsolescence and diminishing resource issues. DOD has also used open systems to some extent and reports that open systems have resulted in cost and schedule efficiencies in the development and upgrade of some of its acquisition programs.

Office of the Under Secretary of Defense, Acquisition, Technology and Logistics Memorandum: “Better Buying Power: Guidance for Obtaining Greater Efficiency and Productivity in Defense Spending” (Sept. 14, 2010). Office of the Under Secretary of Defense, Acquisition, Technology and Logistics Memorandum: “Better Buying Power 2.0: Continuing the Pursuit for Greater Efficiency and Productivity in Defense Spending” (Nov. 13, 2012). technologically advanced products. In addition, the capability to accept plug-in components results in reduced development time and costs for manufacturers because parts are easier to integrate. Furthermore, upgrades and repairs take less time and are less costly for consumers. Finally, when different products adhere to the same standards, an open system design enhances interoperability among products.

An open systems approach can also spur industry growth and entrepreneurial creativity, transforming an industry and offering benefits to manufacturers, suppliers, and consumers. A prime example where this has occurred is in the personal computer industry. In 1981, International Business Machines (IBM) Corporation introduced its personal computer, which was designed as an open system. IBM used already existing components, including the monitor from another IBM computer, and commercially-available off-the-shelf parts such as software, floppy drives, and an Intel processor. Upon releasing the personal computer, IBM openly published its hardware and software specifications, allowing other manufacturers to develop compatible software and peripheral hardware, such as the monitor, keyboard, and mouse. Figure 2 illustrates the personal computer as an open system, made up of compatible components that can be produced and integrated by different manufacturers.

Since the time IBM publicly released the specifications of its personal computer, the market has grown exponentially in terms of manufacturers developing computers and related devices such as printers and scanners, third-party suppliers developing software applications that can be used on the computers, and consumers purchasing computers, software, and peripherals. Increased competition and technological innovation brought on by the use of an open systems approach, among other things, has helped make computers affordable to consumers. For example, according to IBM, one of its predecessors to the personal computer sold for $90,000; its first personal computer was sold in retail stores for $1,565.In the present marketplace, multiple computer manufacturers are developing computers that have 500 times the processing power of IBM’s early personal computer and sell for as little as $400.

The personal computer industry has evolved over the past three decades to meet consumer demand and to leverage new technologies developed by the large number of manufacturers and suppliers competing in the marketplace. For example, in the mid-1990s, a group of companies, including Intel, Microsoft, and Hewlett-Packard, developed the Universal Serial Bus (USB) interface standard to reduce the number of different connectors, such as parallel and serial ports, that were required to allow various components to work together. The USB standard is now widely used in various devices beyond computer peripherals, including cell phone chargers and music players. Other standards, such as the High- Definition Multimedia Interface have been developed to address the need for standards for high-definition televisions and computer monitors. Perhaps the biggest transformation in the computer industry has been the Internet, with an estimated 2.7 billion users worldwide. communications technology is supported by open, non-proprietary standards which allow consumers to have easy access to information on a wide variety of topics, music, shopping opportunities, online banking, friends, their workplace, and more.

In 2013, the International Telecommunication Union, a United Nations agency, estimated that there are 2.7 billion worldwide Internet users. number of modernized submarines. In 2009, the Navy reported reductions of:

17 percent in program development and production costs,

13 percent in operating and support costs, and

80 percent in development time—developing and installing the sonar on the first ship 2 years after the program started, as opposed to a decade or more.

The Navy also used an open system architecture to upgrade its Virginia- class submarine program to counteract the effects of obsolescence and ensure the submarines have the capability to respond quickly to changing missions and threats. In 2010 the Navy reported $96 million in cost avoidance since the upgrade program’s inception in 2001.

In addition, in 2008, Congress directed DOD to develop a strategy for commonality and standardization in UAS ground control station architecture. As part of a joint service/industry effort, the UAS Task Force developed a common, open ground control station architecture, called the UAS Control Segment (UCS), which can be integrated into any UAS program’s ground control station. The UCS architecture enables the reuse of individual software applications across different types of ground control stations, such as those for weather or vehicle status, thus eliminating the need to redevelop the applications for each new ground control station. The first version of the UCS architecture was released in 2010, and the ability to share software applications was first made available to UAS programs in January 2013. OSD and service officials point to several benefits from using UCS software. For example, one task force official stated that a demonstration of the UCS architecture in 2010 showed that the average time to integrate 10 applications was less than 2 weeks; significantly shorter than the traditional integration time for a new ground control station capability, which can be about 1 year. DOD also estimates that competition for software applications can eliminate up to 90 percent of their cost. In addition, the architecture will enable rapid integration of new aircraft sensors, weapons, and other payloads.

Navy Leads Other Services in Implementation of an Open Systems Approach for UAS Acquisition Programs

The services vary in the extent to which they have adopted open systems for DOD’s 10 largest UAS, with the Navy leading the other services. Three of the Navy’s four current and planned UAS programs incorporated, or are planning to incorporate, an open systems approach from the start of development in key components of their UAS—the air vehicle, ground control station, and payloads (i.e., cameras and radar sensors). Conversely, none of the Army or Air Force UAS programs incorporated the approach from the start of development because, according to Army and Air Force officials, legacy UAS programs tried to take advantage of commercial off-the-shelf technology or began as technology demonstration programs. Several of these programs (4 of 6) are starting to incorporate the approach, primarily for the ground control station of fielded aircraft during planned upgrades. For example, the Army did not initially include an open systems approach for its three UAS programs, but has since developed a universal ground control station with open interfaces that each of its programs will use. None of the Air Force’s three UAS programs were initially developed as an open system, and only one is being upgraded to include an open systems approach. Each of the programs that have adopted an open systems approach expects to achieve cost and schedule benefits, such as reduced upgrade costs and quicker upgrade times. Figure 3 identifies the UAS programs that introduced an open systems approach at the start of development for the air vehicle, ground control station, and payloads.

Three of the Navy’s four current and planned UAS programs—the Small Tactical UAS (STUAS), Triton, and Unmanned Carrier-Launched Airborne Surveillance and Strike (UCLASS)—which are less than 5 years old, included or are planning to include an open systems approach from the start of development for the key components of their systems. The Fire Scout program, which began in 2000, initially incorporated an open systems approach for its ground control station and later included open system elements in its air vehicle and payloads during an upgrade effort. The Navy expects significant benefits in return, such as reduced development and integration time and costs, as well as increased future competition for new system payloads. Since these systems, for the most part, either have not been fielded or have not begun development, program officials were only able to provide us with estimates of potential savings.

The STUAS program, which started in 2010, made modularity a key requirement for the system, as documented in its acquisition strategy and systems engineering plan, and required the contractor to provide the program office with rights to key interfaces in the development contract. For example, the STUAS program owns the specifications for the data link between the air vehicle and ground control station, as well as the specifications for the payload interfaces. Officials noted that these elements allow the program to compete contracts for both the air vehicle and ground control station in the future for upgrades or to meet changing requirements. In addition, program and contractor officials noted that by having the rights and specifications to the payload interfaces, the program will be able to integrate and test third-party designed payloads within a matter of days or months, as opposed to years typically required to test new system payloads. Program officials also anticipate that they will be able to independently integrate at least 32 different payloads developed by 24 different manufacturers.

The Triton program, which started in 2008, developed an open systems management plan and described in detail the steps it plans to take to achieve an open system in its acquisition strategy and systems engineering plan. The Triton includes both hardware and software modularity, which gives the program the ability to integrate new payloads and introduce software upgrades without affecting the rest of the system. For example, officials noted that the system’s camera is a separate subsystem, which makes it easier to upgrade the capability individually, and the flight control software is separated from the rest of the system’s software, resulting in less required testing when new capabilities are added. Program officials estimated that software testing could be reduced by as much as 66 percent compared to systems that do not have an open system design, in part because the program does not have to test the entire system when introducing new software or upgrades. In addition, the program anticipates that the open system design will facilitate increased competition among payload suppliers, which could result in lower prices and better payload capabilities.

The Navy is planning to use an open systems approach for its future UCLASS program. The program has identified and designated key system interfaces and according to program officials, it plans to require contractors to use particular open standards, such as an open payload interface for electro-optical/infrared sensors, which is expected to reduce acquisition costs and simplify integration, as well as an open avionics architecture. The Navy developed the architectures and provided them to industry beginning in 2012. Officials developing the avionics architecture estimated that development and integration costs for multiple platforms using the avionics could be reduced significantly, as compared to using a closed avionics architecture.

The Fire Scout program included an open systems design for its original ground control station when it began development in 2000, but the rest of the system remained proprietary. However, when the program upgraded from the MQ-8A to the MQ-8B air vehicle in 2004, it secured the data rights to key system interfaces and introduced modularity into the air vehicle and payloads. As a result of a new modular software design, officials noted that payload data links are separated from the flight control software and other mission systems, which is expected to cut the time needed to test software and integrate a payload by about half. For example, program officials said they were able to integrate a new radar in only 18 months; officials estimated this integration previously would have taken 3 years.

The Army and Air Force Are Incorporating Open Systems to a Limited Extent on Fielded UAS

The Army and Air Force did not originally plan to make the air vehicle, ground control station, or payloads of their UAS programs open systems. Instead, program officials stated that they relied on commercial off-the- shelf systems and technology demonstration programs that prime contractors developed and then modified and upgraded over time, without the benefit of competition that could help keep costs low. Over the past several years, the Army has taken steps to make the ground control stations open systems for all three of its fielded UAS and the payload interfaces open in two of those systems, while the Air Force plans to make the ground control station an open system for one of its three UAS.

The Army’s three UAS programs—Hunter, Shadow, and Gray Eagle— were all initially developed as proprietary systems and did not include an open systems approach for all three key components—the air vehicle, ground control station, and payloads. Moreover, the Army’s UAS ground control stations limited interoperability and resulted in the Army paying for ground control stations that provided similar capabilities. The Army eventually developed a common ground control station for the three UAS; however, the new station was still proprietary. All three of the Army’s UAS programs are now upgrading to a universal ground control station that incorporates an open architecture to address obsolescence issues and increase interoperability. According to Army program officials, ground control stations require continuous hardware and software upgrades as the technology becomes obsolete. Even though an open systems approach is being incorporated later in the programs’ life cycles, officials believe the benefits—reduced obsolescence issues, reduced upgrade costs, and increased interoperability—outweigh the costs. For example, the Army’s new universal ground control station will give Army operators in the field the ability to fly Hunter, Shadow, and Gray Eagle from one ground control station. This was not possible with the Army’s legacy ground control stations that did not use open architectures.

The Army’s Gray Eagle and Shadow programs are also incorporating open system elements for the systems’ payloads to save integration time and cost. For example, the Shadow program modified its air vehicle in 2008 to include a universal payload pod that allows any vendor that can meet the standard power and data connections to design a payload to be integrated onto the system. Program officials noted that, to date, four third-party vendors have used the universal pod for their products. Similarly, Gray Eagle program officials stated that the program owns the data rights to all payload interface specifications, which allows third-party vendors to develop a payload using these specifications. However, officials noted that the program has to rely on the prime contractor to integrate new payloads because the Army does not have the expertise to do so.

The Air Force has had limited success in modernizing its UAS to include open systems. For example, the Reaper plans to upgrade to an open ground control station, but the remainder of the system remains proprietary. The other two programs—the Predator and Global Hawk— included language in their planning documents stating their intention to introduce open system elements later in their respective life cycles. However, Predator’s age and Global Hawk’s fiscal constraints prevented them from adopting an open systems approach. As a result, the two systems remain largely proprietary and are now facing challenges sustaining and upgrading their systems.

The Predator program began in 1994 as an advanced concept technology demonstration program and is one of the oldest systems in DOD’s UAS portfolio. Program officials stated that the Predator’s software is not modular and the program has no intention of modifying the software because the Air Force is planning to divest itself of Predator aircraft once more Reapers are fielded. Predator officials also noted that sustainment and obsolescence challenges remain a risk area for the program. Officials from the Global Hawk program, which started development in 2001, also stated that obsolescence is a major problem for that program, particularly for the ground control station. The program recently had planned to develop a new ground control station that utilized an open systems architecture. However, the Air Force cancelled the upgrade effort in 2013 due to what program officials described as fiscal constraints, even though it plans to use the aircraft through at least 2032. The Air Force is now planning to continue to maintain the legacy Global Hawk ground control station and communications system, although contractor obsolescence and other sustainment challenges contributed to a critical Nunn-McCurdy cost breach in 2011.

Policies and Leadership Can Help Drive an Open Systems Approach for Weapon Acquisition Programs, but Challenges Exist

Policies and leadership support are two fundamental ways of getting DOD’s weapon acquisition community to shift from a culture of relying on contractors to provide proprietary systems to one where systems are designed to be open. In general, policies are needed to establish the overall plan and acceptable procedures that guide acquisition strategies, and leadership is needed to champion policies and new initiatives, as well as ensure that technical expertise is available to implement them. DOD has cited a preference for acquiring open systems in its policies since 1994 and most recently in its Better Buying Power initiative, which requires programs to outline an approach for using open systems architectures at milestone B—the start of development. The Navy developed an open systems policy in 2005 and its UAS programs largely followed that policy from the start of development. The Air Force and Army also have open systems policies, but their UAS programs did not implement them from the start of development and, because of this, some programs are now having difficulties upgrading their systems. Further, while DOD leadership is placing a renewed emphasis on an open systems approach through its Better Buying Power initiatives, it is not currently tracking the extent to which weapon acquisition programs are adopting open systems across the department or whether program offices have enough technical expertise to effectively implement an open systems approach. The Naval Air Systems Command has taken steps to ensure that its UAS program offices have the expertise to implement open systems by establishing a small group of experts that can assist program offices with developing their technology development and acquisition strategies with open systems in mind, but other program offices may lack their own expertise or access to similar expertise.

Air Force and Army Are Lagging in Open Systems Implementation

Although DOD acquisition policy directs program managers to employ a modular, open systems approach for acquisition programs to minimize life-cycle costs, the Navy is the only service that did so while developing its UAS programs. Based on our review of UAS programs, the Navy leads the other services in adopting an open systems approach for its acquisition programs, and Navy program officials cited their service’s open systems policy as the driver for adoption. The Navy’s policy required its programs to incorporate open architecture principles into program requirements beginning in 2005, which means that UAS programs would include an open systems approach as part of their acquisition strategies prior to the start of program development. The Navy also established an oversight and reporting structure to provide reasonable assurance that its programs are following this policy. For example, the Navy assigns overall responsibility and authority for directing the effort to one office, the Deputy Assistant Secretary of the Navy for Research, Development, Test and Evaluation. This office is tasked with defining an overarching strategy, providing systems engineering leadership to other Navy acquisition offices, and overseeing and reporting on implementation efforts.

The Air Force and Army also have policies that require open systems to be included in a program’s acquisition strategy, which would occur prior to the start of program development. However, none of their UAS programs incorporated an open systems approach at the start of development, including a program that began after the policies were issued. Instead, these services favored off-the-shelf systems or those that could be quickly fielded. In addition to not including open systems during the initial design of a program, these service officials stated that they are having some difficulties acquiring the necessary funding to make their legacy systems open as they are being upgraded and we found they are not consistently implementing the new UCS architecture developed for all ground control stations. For example, the Global Hawk program planned to use an open systems approach for its ground control station to address obsolescence issues. However, as mentioned earlier, the Air Force terminated the effort because of funding constraints, even though it plans to use the aircraft through at least 2032 and the ground control station will require upgrades and costly support during this time span. In addition, the Air Force’s Global Hawk and Predator and the Navy’s STUAS do not plan to adopt the recently developed UCS architecture at this time. The Global Hawk had planned to incorporate UCS as part of its now cancelled upgrade effort and Predator officials stated that the Air Force is not investing in upgrades, including UCS.

Further complicating the UCS architecture effort is the Air Force, which developed a “complementary” ground control station architecture without the involvement of the other services or OSD. The Air Force plans to use both architectures on the Reaper system. Officials explained that they developed the complementary architecture because the Air Force wanted to be able to attain the capability faster. However, both architectures will be fielded simultaneously on the Reaper and contractors developing UAS ground control station software stated that adopting both architectures will be more costly for DOD.

To improve implementation of an open systems approach, DOD’s 2010 Better Buying Power initiative now requires programs, at milestone B, to outline an approach for using open systems architectures and acquiring technical data rights to ensure sustained consideration of competition in the acquisition of the weapon systems. The approach is to be based on a business case analysis and engineering trade analysis.

DOD Leadership Lacks Visibility into Open Systems Implementation and Military Service Expertise

DOD recognizes that it needs to increase its leadership efforts to implement open systems for its weapon acquisition programs, but does not know the extent to which programs are implementing this approach or if the services have the requisite technical expertise. According to federal internal control standards, agencies should monitor programs to ensure that policies are being implemented, such as by establishing and tracking program metrics, and also ensure that their workforce has the required skills to implement policies. Thus, DOD should monitor open systems implementation and also assess whether program offices have the expertise to carry out the open systems policies. Various independent standards experts, DOD contractors, and DOD systems engineering officials identified DOD leadership as key to effective adoption of an open systems approach. Several of these officials cautioned that prime contractors may be resistant to providing open systems because they are able to achieve greater financial benefits by selling DOD proprietary products, which they alone can integrate, upgrade, and maintain. Specifically, these officials believe that successful implementation of an open systems approach requires that DOD provide clear and consistent direction for its approach and that programs plan for an open systems approach prior to the start of development. This includes defining the open systems approach in a program’s technology development and acquisition strategies prior to initiating the technology development phase at milestone A and the start of engineering and manufacturing development at milestone B in DOD’s acquisition process, respectively. Further, they believe that contracts and requests for proposal should include appropriate language that describes the open systems architecture, defines open standards, and establishes requirements for control documents to ensure the government retains rights for the identified key interfaces.

Acquiring a new weapon system using an open systems approach could be more expensive at the beginning, particularly regarding ownership of data rights. Yet, the potential benefits of obtaining the data rights, in terms of greater competition and reduced upgrade and repair costs, are widely believed to significantly outweigh the costs, especially if done from the start of development. DOD could avoid purchasing data rights if it works with industry to develop standards. However, in these cases, standards organizations’ officials we spoke with said that DOD would have to work with standards organizations several years in advance to ensure that the standards are available at the start of development.

The Better Buying Power initiative for open systems is one of DOD’s efforts to increase competition and drive efficiency in acquisitions. While competition is difficult to do at the system level because there are a limited number of prime contractors that develop DOD weapon systems, an open systems approach can boost opportunities for competition at the subsystem level. As part of the Better Buying Power initiatives, the Under Secretary of Defense for Acquisition, Technology and Logistics formed an Open Systems Architecture Data Rights Team co-led by the Office of the Deputy Assistant Secretary of Defense for Systems Engineering (SE) and the Deputy Assistant Secretary of the Navy for Research, Development, Test and Evaluation.

The team is developing guidance, tools, and training in an effort to gain more momentum for the use of an open systems approach and the acquisition of appropriate data rights across DOD. This includes a contract guidebook to be issued in the summer of 2013 intended to assist program managers in incorporating open systems principles into their acquisition programs. Officials from two of the UAS programs we reviewed, UCLASS and Triton, told us they had used a draft version of this guidebook to help inform their contracting strategies. The team also updated the systems engineering guidance in the Defense Acquisition Guidebook to recommend that program managers update their technology development strategy, acquisition strategy, and systems engineering plan throughout their program’s life cycle to reflect their respective open systems strategies. Finally, the team is leveraging other resources previously developed by the Open Systems Joint Task Force, such as an analytical tool for programs to monitor and evaluate their open systems implementation and a program manager’s guide intended to assist program offices in integrating an open systems approach into their acquisition strategies for new and legacy systems. According to officials from the Open Systems Architecture Data Rights Team, the team is required to report to the Under Secretary of Defense for Acquisition, Technology and Logistics on its progress in meeting initiatives related to Better Buying Power by October 2013, but that reporting has not yet begun.

The Deputy Assistant Secretary of Defense for Systems Engineering indicated that several new programs are incorporating an open systems approach during early planning. For example, the Navy’s Next Generation Jammer included open systems language in its 2012 request for proposal. Additionally, the Army’s Ground Combat Vehicle program and the Air Force’s Space Fence program both considered open systems as part of their future design prior to or during technology development. However, it is unclear the extent to which programs are adopting an open systems approach across the department because this information is not being tracked.

In addition, it is unclear whether the services have sufficient technical expertise to implement an open systems approach and enable DOD to be a savvy buyer of weapon systems. Program offices need open systems expertise to conduct systems engineering activities, even before a contract is awarded, to be able to determine how an open systems strategy fits into program requirements; identify which system interfaces are key to maintaining competitive opportunities; ensure sufficient data rights are obtained to support the competitive strategy; translate open system requirements into contracts; address any risks associated with the open systems approach; and finally, validate that the contractor is providing the open system that the program required. However, officials from 11 offices/companies we spoke with, including a DOD contractor; the UAS Task Force; 6 of 10 Air Force, Army, and Navy UAS programs; the Naval Air Systems Command technical experts; the DOD Open Systems Architecture Data Rights Team; and an open systems expert who consulted with DOD on an open systems approach, stressed that DOD does not have adequate expertise across the department to effectively implement an open systems approach.

The Defense Acquisition University (DAU), the organization responsible for training the acquisition workforce DOD-wide, offers training on open systems as part of its core acquisition curriculum required for both engineers and program managers. However, officials from every program we talked to who had taken this training told us that the training is only sufficient to familiarize participants with the concept of open systems, but not in-depth enough to allow participants to implement the approach. DAU is developing additional training resources in support of open systems to be rolled out beginning fall 2013. This new training, according to a DAU official, will be geared toward providing program managers and data management personnel information on data rights. Open systems will be discussed, but primarily to give the participants awareness of the approach. There are currently no plans for new in-depth training focused on an open systems approach. Some of the material may be added to the core acquisition curriculum, but officials said this transition can take years.

As required by the Weapon Systems Acquisition Reform Act of 2009, the Deputy Assistant Secretary of Defense for Systems Engineering is responsible for reviewing the systems engineering capabilities of the military departments and identifying areas that require changes or improvements. To meet this statutory requirement, SE requests that the Air Force, Army, and Navy submit a self-assessment of their systems engineering capabilities, which SE reviews to identify needed changes or improvements to the services’ capabilities. SE conducts this review annually and publishes its findings in annual reports to Congress, most recently in March 2013. However, the services have not provided an analysis of their open systems capabilities as part of their self- assessments. Therefore, the Deputy Assistant Secretary cannot be sure that program offices have adequate systems engineering resources to effectively implement an open systems approach.

The Naval Air Systems Command has a small group of open systems technical experts that its acquisition programs can use when developing their open systems approach. Officials from the UCLASS, STUAS, and Triton programs said they relied on experts from this team to assist them in planning and executing their open systems approach, including developing a request for proposal, selecting open standards, and responding to contractor bids. UCLASS program officials also said they received assistance from the Deputy Assistant Secretary of the Navy for Research, Development, Test and Evaluation—the Navy’s lead office for open systems—when developing its request for proposal.

Conclusions

The adoption of an open systems approach in DOD acquisition can provide significant cost and schedule savings for DOD. Based on projections from several Navy UAS programs, for example, the Navy could avoid considerable repair and upgrade costs on individual programs, as well as improve performance, by incorporating an open systems approach on programs prior to the start of development. This is because multiple suppliers can compete to quickly replace key components on the UAS with more capable components. Traditionally, DOD has acquired proprietary weapon systems that limit these opportunities and make these systems more costly to develop, procure, upgrade, and support. DOD has cited a preference for acquiring open systems in its policy since 1994 and each of the services have since issued open systems policies. Yet, we found the Army and Air Force have been slow to make their UAS open systems, particularly from the start of development. The Navy, on the other hand, has generally designed its UAS to be open from the start of development where it can reap the most benefits.

To successfully change the services’ inclination toward buying proprietary systems and shift towards the acquisition of open systems, DOD needs to address policy and leadership challenges. Moving to open systems requires strong leadership to overcome preferences for acquiring proprietary systems. While DOD’s Better Buying Power initiative requires programs to outline an approach for using open systems architectures at milestone B, OSD does not have adequate insight of the extent to which an open systems approach is being used by individual weapon acquisition programs. Further, OSD does not know if program offices have the systems engineering expertise required for effective implementation of an open systems approach or if additional expertise is needed. Without adequate knowledge of policy implementation and program office expertise, DOD cannot have reasonable assurance that an open systems approach is being implemented effectively by the services. Until DOD takes action to overcome these challenges, the department will likely continue to invest in costly proprietary systems. These steps should increase DOD’s ability to promote more competition, save taxpayer dollars, and more quickly field new capabilities to the warfighter, particularly if an open systems approach is incorporated into program strategies prior to the start of development at milestone B.

Recommendations for Executive Action

We are making four recommendations to improve the department’s implementation of an open systems approach for UAS and other weapon acquisition programs, as well as its visibility of open systems implementation and program office expertise.

We recommend that the Secretary of Defense direct the Secretaries of the Air Force and Army to implement their open systems policies by including an open systems approach in their acquisition strategies.

We recommend that the Secretary of Defense direct the Under Secretary of Defense for Acquisition, Technology and Logistics to define appropriate metrics to track programs’ implementation of an open systems approach.

We also recommend that the Secretary of Defense direct the Secretaries of the Air Force, Army, and Navy to take the following actions: require their acquisition programs to include open systems metrics developed by the Under Secretary of Defense for Acquisition, Technology and Logistics in their systems engineering plans, track progress in meeting these metrics, and report their progress to the Under Secretary of Defense for Acquisition, Technology and Logistics at key acquisition milestones; and assess their respective service-level and program office capabilities relating to an open systems approach and work with the Deputy Assistant Secretary of Defense for Systems Engineering to develop short-term and long-term strategies to address any capability gaps identified. Strategies could include the Navy’s cross-cutting approach where a team of a few technical experts within the Naval Air Systems Command could be available to work with program offices, as necessary, to help develop open systems plans.

Agency Comments and Our Evaluation

DOD provided us with written comments on a draft of this report. DOD partially concurred with all four recommendations. DOD’s comments are reprinted in appendix II.

DOD partially concurred with our first recommendation, agreeing on the value of implementing an open systems approach, but citing existing department policies and guidance which it believes are sufficient for the military departments to implement open systems architecture in acquisition programs. DOD also noted in its comments that the decision to implement an open systems approach in a particular program’s acquisition strategy is made on a case-by-case basis based on a number of considerations to include mission, threat, vulnerability assessment, operating environment, and business case. We agree that an open systems approach should be informed by these considerations, and we also cited both OSD and service-level policies and guidance governing an open systems approach in our report. However, a number of Air Force and Army unmanned aircraft programs missed opportunities to adopt an open systems approach early in their life cycles, but did try to do so later on when it becomes more costly and complex. Navy programs fared better partly because they augmented policy by addressing the open systems approach in individual acquisition strategies. We maintain that the Air Force and Army would benefit by including an open systems approach in their acquisition strategies before the start of system development.

DOD partially concurred with our second and third recommendations. DOD noted that program implementation of an open systems approach should be assessed through its existing milestone decision process. DOD further noted that acquisition strategies and systems engineering plans, which document a program’s open systems strategy, are assessed at multiple decision reviews and are considered by the milestone decision authority in the overall context of the program. We agree that the milestone decision process is the appropriate venue to review programs’ open systems strategies. However, as discussed in our report, we found that OSD does not have adequate insight of the extent to which an open systems approach is being used by weapon acquisition programs and thus cannot have reasonable assurance of the widespread use of an open systems approach across the department. Based on our review of unmanned aircraft programs, we found that the Navy had generally embraced an open systems approach for its acquisition programs before the start of development, whereas the Air Force and Army had not. Further, it is unclear the extent to which the Air Force and Army will adopt the approach for its programs as DOD’s recent Better Buying Power initiative requires. To provide clearer insight into whether the services are planning for and implementing an open systems approach, we maintain that the Under Secretary of Defense for Acquisition, Technology and Logistics should establish metrics and track weapon acquisition programs’ progress in meeting them at key milestones. For example, this could include tracking whether programs are including open systems in their acquisition strategy documents prior to milestone B and are following through on those plans at subsequent acquisition milestones.

DOD partially concurred with our fourth recommendation but did not explain its position or what, if anything, it would do in response. DOD did not comment on whether the military services and their program offices have sufficient capabilities with respect to open systems. As we discussed in our report, we found that OSD does not know if program offices have the systems engineering expertise required for effective implementation of an open systems approach or if additional expertise is needed. To address this possible gap, we continue to believe that the Air Force, Army, and Navy should assess their respective service-level and program office capabilities relating to an open systems approach and work with the Deputy Assistant Secretary of Defense for Systems Engineering to develop short-term and long-term strategies to address any capability gaps identified. As we suggested, one such strategy could be the Naval Air Systems Command’s cross-cutting approach where a team of a few technical experts could be available to work with program offices, as necessary.

We are sending copies of this report to the Secretary of Defense, the Secretary of the Air Force, the Secretary of the Army, the Secretary of the Navy, and interested congressional committees. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you have any questions about this report or need additional information, please contact me at (202) 512-4841 or sullivanm@gao.gov. Contact points for our Office of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

The House Armed Services Subcommittee on Tactical Air and Land Forces requested that we assess the Department of Defense’s (DOD) progress in implementing open systems in unmanned aircraft system (UAS) acquisitions. Since an open systems approach can facilitate competition and upgrades for any type of weapon system acquisition, and because DOD is directing its current open system initiatives department- wide, we evaluated the benefits of an open systems approach and challenges DOD is facing in adopting the approach broadly. Therefore, this report examines (1) the characteristics and benefits of an open systems approach, (2) DOD’s efforts in applying an open systems approach to its UAS portfolio, and (3) challenges, if any, DOD is encountering in implementing this approach.

To determine the characteristics and benefits of an open systems approach, we reviewed relevant DOD policies, guidance, and handbooks and conducted a literature search to identify open systems-related examples from private companies and standards development organizations. We interviewed officials from four standards organizations that we selected based on our literature search: the Aerospace Industries Association, the Peripheral Component Interconnect Industrial Computer Manufacturers Group, SAE International, and the Universal Serial Bus Implementers Forum. These organizations have responsibility for developing open standards for their respective industries, and their experience is commensurate with the age of their industries. For example, the Universal Serial Bus Implementers Forum has existed for almost 20 years, and SAE International has existed for over 100 years. We also interviewed personnel from DOD contractors, including AAI, Dreamhammer, General Atomics-Aeronautical Systems Group, Insitu, Northrop Grumman, and Raytheon/Solipsys to identify their leading practices in developing open standards and implementing an open systems approach. In addition, we interviewed an expert from Carnegie Mellon’s Software Engineering Institute about experiences working with DOD on an open systems approach. We also interviewed DOD officials knowledgeable about open systems and open standards development from the Army, Navy, and the Office of the Secretary of Defense (OSD). We analyzed interviews with the open systems expert, DOD officials, standards organizations, and all four of the contractors who we asked questions related to open systems best practices (General Atomics, Northrop Grumman, Raytheon/Solipsys, and Dreamhammer) to identify key practices for successfully adopting an open systems approach.

To determine the extent to which DOD is applying an open systems approach to its UAS portfolio, we interviewed and collected documentation, including briefings, acquisition strategies, and systems engineering plans, from the UAS Task Force and UAS program offices on open systems efforts. In addition, we developed a questionnaire to collect information on which elements of an open systems approach UAS programs incorporated, their use of guidance and tools, and their contracting strategies. After we drafted the questionnaire, we asked our Chief Technologist as well as a member of DOD’s Open Systems Architecture Data Rights Team to review and validate our questions. We revised the questionnaire to reflect comments from these reviews, then distributed the electronic Microsoft Word questionnaire by e-mail to 10 current and planned UAS acquisition programs from Groups 2-5—the larger UAS programs—in October 2012; all 10 responded that month. We excluded Group 1 UAS from our analysis because these are the smallest UAS systems and recent DOD open systems efforts focus on Group 2-5 UAS. Using each program’s questionnaire response and follow-up interviews conducted with each program office to corroborate the responses, we evaluated the key areas where UAS programs implemented an open systems approach—the air vehicle, the ground control station, and the system payloads—and determined at what point in the acquisition life cycle that UAS programs introduced their approach.

To determine challenges acquisition programs face in implementing an open systems approach, we synthesized information from DOD policies and guidance, program questionnaires, and interviews with officials from the Office of the Deputy Assistant Secretary of Defense for Systems Engineering; the Office of the Secretary of Defense’s UAS Task Force; the Defense Acquisition University; the Office of the Deputy Assistant Secretary of the Navy for Research, Development, Test and Evaluation; acquisition offices within the Air Force, Army, and Navy; UAS program offices; and UAS contractor personnel.

We conducted this performance audit from July 2012 through July 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Department of Defense

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, key contributors to this report were Cheryl Andrew, Assistant Director; Richard Burkard; Lisa Fisher; Danielle Greene; Laura Jezewski; Deanna Laufer; Brian Lepore; Jean McSween; Paige Muegenburg; Brian Mullins; and Roxanna Sun.