Defense Acquisitions: DOD Efforts to Adopt Open Systems for Its Unmanned Aircraft Systems Have Progressed Slowly
Published: 2013-07-31T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: For fiscal year 2014, DOD requested over $11 billion to modify existing weapon systems--more than 10 percent of its total procurement budget. Traditionally, DOD has acquired proprietary systems, which are costly to upgrade and limit opportunities for competition. Through its Better Buying Power initiatives, DOD has re-emphasized the use of an open systems approach as a way to reduce costs through effective competition.
GAO was asked to examine DOD's progress in implementing an open systems approach for UAS acquisitions. This report addresses (1) the characteristics and benefits of an open systems approach, (2) DOD's efforts in implementing an open systems approach for its UAS portfolio, and (3) challenges, if any, DOD is encountering in implementing this approach. GAO analyzed relevant literature and DOD policies on open systems and interviewed agency and private industry officials to understand how open systems have been implemented and their benefits. In addition, GAO assessed acquisition documents and questionnaire responses from 10 current and planned UAS programs to determine their open system strategies.
section_title: What GAO Found
paragraphs: An open systems approach, which includes a modular design and standard interfaces, allows components of a product (like a computer) to be replaced easily. This allows the product to be refreshed with new, improved components made by a variety of suppliers. Designing weapons as open systems offers significant repair, upgrade, and competition benefits that could translate to millions of dollars in savings as the weapons age.
The services vary in their use of open systems on the Department of Defense's (DOD) 10 largest unmanned aircraft systems (UAS). The Navy used an open systems approach at the start of development for the air vehicle, ground control station, and payloads (i.e., cameras and radar sensors) for three of its four current and planned UAS and anticipates significant efficiencies. For example, Navy and contractor officials expect the Small Tactical UAS to be able to integrate at least 32 payloads developed by 24 manufacturers, some in a matter of days or months rather than years as previous programs experienced. Conversely, none of the Army or Air Force UAS programs initially implemented an open systems approach, relying instead on prime contractors to upgrade and modernize the UAS. The Army is now developing an open ground control station for each of its three legacy UAS programs. Only one of the Air Force's three UAS programs plans to implement an open systems approach on fielded aircraft.
Policies and leadership can help drive DOD's acquisition community to use an open systems approach, but challenges exist. Although DOD and the services have policies that direct programs to use an open systems approach, the Navy is the only service that largely followed the policy when developing its UAS. In addition, while new open systems guidance, tools, and training are being developed, DOD is not tracking the extent to which programs are implementing this approach or if programs have the requisite expertise to implement the approach. Navy UAS program officials told us they relied on technical experts within Naval Air Systems Command to help develop an open systems approach for their programs. Until DOD ensures that the services are incorporating an open systems approach from the start of development and programs have the requisite open systems expertise, it will continue to miss opportunities to increase the affordability of its acquisition programs.
section_title: What GAO Recommends
paragraphs: GAO recommends that the Air Force and Army implement their open systems policies, DOD develop metrics to track open systems implementation, and the services report on these metrics and address any gaps in expertise. DOD partially concurred and stated that its current policies and processes are sufficient. GAO maintains additional action is needed.

section_title: Letter
paragraphs: The Department of Defense (DOD) intends to spend over $11 billion in  fiscal year 2014 to modify and modernize its existing weapon systems— more than 10 percent of its total weapon system procurement budget.  Given the current fiscal environment, budgetary constraints are adding  pressure on DOD to look for ways to reduce life-cycle costs for its  capabilities. Traditionally, DOD has acquired proprietary systems that limit  opportunities for competition and cannot readily be upgraded because the  government is locked into the original suppliers. Acquisition of proprietary  systems is costly and can impede interoperability and the reuse of  components among systems. Conversely, an open system, such as a  personal computer, that incorporates modular design and open standards  for key interfaces, can readily accept upgrades from a variety of suppliers  without redesign of the entire system, providing numerous cost, schedule,  and performance benefits. We previously reported, in 2003, that DOD  was not giving due consideration to acquiring open systems, known as an  open systems approach, for its weapon acquisition programs and that  implementation was limited. In addition, in July 2009, we recommended  that the Secretary of Defense require new unmanned aircraft system  (UAS) programs to take an open systems approach. DOD concurred  with the recommendation and plans to incorporate the approach in a  future UAS program.
Given the potential for open systems benefits, you requested that we  assess DOD’s progress in implementing an open systems approach for  UAS acquisitions. Since an open systems approach can facilitate  competition and ease upgrades for any type of weapon system  acquisition, and because DOD directs its open system initiatives  department-wide, we also evaluated the benefits and challenges of  implementing an open systems approach broadly. Therefore, this report  examines (1) the characteristics and benefits of an open systems  approach, (2) DOD’s efforts in applying an open systems approach to its  UAS portfolio, and (3) challenges, if any, DOD is encountering in  implementing this approach.
To determine the characteristics and benefits of an open systems  approach, we reviewed relevant DOD policies, guidance, and handbooks  and conducted a literature search to identify open systems-related  examples from private companies and standards development  organizations. We interviewed officials from selected standards  organizations, DOD contractors, and academia to identify their leading  practices in developing open standards and implementing an open  systems approach. We also interviewed knowledgeable officials from the  Office of the Secretary of Defense (OSD) and from Army and Navy  acquisition offices. To determine DOD efforts in applying an open  systems approach to its UAS portfolio and the challenges it is  encountering implementing this approach, we interviewed and collected  documentation on open systems efforts from OSD’s UAS Task Force and  UAS program offices. In addition, we collected and analyzed information  from the 10 largest current and planned UAS programs—Group 2-5 UAS  programs—on their open systems strategies using an electronic  questionnaire distributed to the programs in October 2012; all 10  program offices responded that month. We synthesized information  received from the questionnaires; additional documentation collected from  the program offices; and interviews with officials from OSD, the Defense  Acquisition University (DAU), acquisition offices within the Air Force,  Army, and Navy, UAS program offices, and UAS contractor personnel to  assess Air Force, Army, and Navy progress in implementing an open  systems approach and to identify related benefits and challenges. See  appendix I for additional information on our objectives, scope, and  methodology.
We conducted this performance audit from July 2012 through July 2013 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: One way for DOD to minimize the cost and time needed to modify  weapon systems is by using an open systems approach for system  design and development. An open system allows system components to  be added, removed, modified, replaced, or sustained by consumers or  different manufacturers in addition to the manufacturer that developed the  system. It also allows independent suppliers to build components that can  plug into the existing system through the open connections. Fundamental  elements of an open systems approach include: designing a system with modular components that isolate  functionality. This makes the system easier to develop, maintain, and  modify because components can be changed without majorly  impacting the remainder of the system. developing and using open, publicly-available standards for the key  interfaces, or connections, between the components. Interface  standards specify the physical, power, data, and/or other connections  between components. All interfaces in a system do not need to use  open standards for a system to be considered “open,” and it can be  costly and impractical to manage hundreds or thousands of interfaces  within a system. Rather, open standards should be identified at key  interfaces between the modules that are likely to change, may  frequently fail or need to be replaced, or are needed for  interoperability. obtaining data rights to interfaces when open standards are not  available. DOD describes the acquisition of technical data, such as  design drawings, specifications, and standards, as critical to enabling  the department opportunities for competition for modification and  sustainment of weapon systems throughout their life cycles.
Many consumer products, including U.S. appliances and smartphones,  are considered to be open systems because they use widely-available  hardware and software standards at key interfaces. For example, U.S.  appliances are designed to use a particular wall socket standard, so that  they can plug into any power outlet without consumers needing to worry  about which brand product is compatible in their homes. Similarly,  headphone jacks use a common, open standard, which enables  consumers to purchase headphones made by different manufacturers,  and plug them into many types of devices, including their MP3 players,  cell phones, and stereos, which may be built by other manufacturers. In  addition, the AndroidTM operating system, used on smartphones, allows  individual software applications introduced by third-party developers to  connect to the Android operating system through an open software  interface. This allows customers to have a lot of choices and helps keep  prices low.
Incorporating an open systems approach prior to the start of development  increases the likelihood that open systems considerations are included in  program requirements and inform a program’s future competitive strategy,  which can significantly reduce upgrade and maintenance costs later.  Introducing this approach later in a program’s life cycle, such as a  planned modification or upgrade, is more difficult, complex, and costly to  do as it may require significant modifications to an already-developed  system. Legacy DOD programs that we have reported on in the past did  not implement an open systems approach early and are now facing  difficult and costly system modifications later in their respective service  lives. The B-2 bomber and the C-130 aircraft are two examples:
The Air Force is spending over $2 billion to upgrade the B-2 bomber’s  communications, networking, and defensive management capabilities.  Because the B-2 program’s prime contractor is the sole system  integrator in possession of proprietary technical data and software,  there is no opportunity for competition—a critical tool for achieving the  best return on investment and driving down costs.
The Air Force planned to replace the aging avionics systems on its  C-130 aircraft with open architecture avionics systems. However, the  individual cockpits had been modified throughout years of  maintenance and differed from one another to such an extent that the  upgrades required custom-built hardware and software; less  expensive commercial off-the-shelf hardware and software could not  be used. Because of the various configurations, C-130 modernization  cost estimates increased from $4 billion to over $6 billion and the Air  Force significantly reduced the number of planes it planned to  modernize from 519 to 221 between fiscal years 2001 and 2011. The  Air Force terminated the program in its fiscal year 2013 budget  submission because of its escalating costs.
DOD understands the efficiencies of open systems and has taken steps  to incorporate an open systems approach into its policy over the past two  decades. As far back as 1994, the Under Secretary of Defense for  Acquisition and Technology directed the use of an open systems  approach in the acquisition of weapon system electronics. That same  year, DOD established an Open Systems Joint Task Force to oversee the  military services’ transition to open systems and coordinate open  systems-related efforts, such as training and standards development. The  task force developed several tools to aid program managers with this  approach, but was later disbanded. In 1998, the Defense Science Board  cited pockets of success in leveraging open systems, but noted that DOD  lacked a unifying concept and required a reconfiguration of management  processes and aggressive leadership to facilitate open systems  implementation. This concern was addressed in DOD’s 2000 acquisition  directive and in its most recent update in 2003, which states that “a  modular, open systems approach shall be employed, where feasible.”DOD’s 2008 acquisition instruction also provides that “program managers  shall employ a modular open systems approach to design for affordable  change, enable evolutionary acquisition, and rapidly field affordable  systems that are interoperable.”
Recently, the Under Secretary of Defense for Acquisition, Technology  and Logistics has placed renewed emphasis on an open systems  approach as part of the 2010 and 2012 Better Buying Power initiatives to  increase efficiency in defense spending through effective competition. Specifically, the initiatives aim to improve the department’s early planning  for open architectures, including making open systems considerations  part of the design process.
section_title: Open Systems Provide a Variety of Benefits
paragraphs: An open systems approach has great potential to generate efficiencies for  product manufacturers, lower total ownership costs for consumers, and  transform industries because it spurs industry growth, competition, and  innovation. To get the most from open systems, the approach is best  implemented at the start of product design because this is where initial  modularity, key interface, and data ownership decisions are made and  would result in costly redesign if implemented later. Open systems have  been successfully used by commercial companies, such as those in the  personal computer and software markets, and has helped customers  avoid sole source prices and manufacturers address obsolescence and  diminishing resource issues. DOD has also used open systems to some  extent and reports that open systems have resulted in cost and schedule  efficiencies in the development and upgrade of some of its acquisition  programs.
Office of the Under Secretary of Defense, Acquisition, Technology and Logistics  Memorandum: “Better Buying Power: Guidance for Obtaining Greater Efficiency and  Productivity in Defense Spending” (Sept. 14, 2010). Office of the Under Secretary of  Defense, Acquisition, Technology and Logistics Memorandum: “Better Buying Power 2.0:  Continuing the Pursuit for Greater Efficiency and Productivity in Defense Spending” (Nov.  13, 2012). technologically advanced products. In addition, the capability to accept  plug-in components results in reduced development time and costs for  manufacturers because parts are easier to integrate. Furthermore,  upgrades and repairs take less time and are less costly for consumers.  Finally, when different products adhere to the same standards, an open  system design enhances interoperability among products.
An open systems approach can also spur industry growth and  entrepreneurial creativity, transforming an industry and offering benefits to  manufacturers, suppliers, and consumers. A prime example where this  has occurred is in the personal computer industry. In 1981, International  Business Machines (IBM) Corporation introduced its personal computer,  which was designed as an open system. IBM used already existing  components, including the monitor from another IBM computer, and  commercially-available off-the-shelf parts such as software, floppy drives,  and an Intel processor. Upon releasing the personal computer, IBM  openly published its hardware and software specifications, allowing other  manufacturers to develop compatible software and peripheral hardware,  such as the monitor, keyboard, and mouse. Figure 2 illustrates the  personal computer as an open system, made up of compatible  components that can be produced and integrated by different  manufacturers.
Since the time IBM publicly released the specifications of its personal  computer, the market has grown exponentially in terms of manufacturers  developing computers and related devices such as printers and scanners,  third-party suppliers developing software applications that can be used on  the computers, and consumers purchasing computers, software, and  peripherals. Increased competition and technological innovation brought  on by the use of an open systems approach, among other things, has  helped make computers affordable to consumers. For example, according  to IBM, one of its predecessors to the personal computer sold for  $90,000; its first personal computer was sold in retail stores for $1,565.In the present marketplace, multiple computer manufacturers are  developing computers that have 500 times the processing power of IBM’s  early personal computer and sell for as little as $400.
The personal computer industry has evolved over the past three decades  to meet consumer demand and to leverage new technologies developed  by the large number of manufacturers and suppliers competing in the  marketplace. For example, in the mid-1990s, a group of companies,  including Intel, Microsoft, and Hewlett-Packard, developed the Universal  Serial Bus (USB) interface standard to reduce the number of different  connectors, such as parallel and serial ports, that were required to allow  various components to work together. The USB standard is now widely  used in various devices beyond computer peripherals, including cell  phone chargers and music players. Other standards, such as the High- Definition Multimedia Interface have been developed to address the need  for standards for high-definition televisions and computer monitors.  Perhaps the biggest transformation in the computer industry has been the  Internet, with an estimated 2.7 billion users worldwide. communications technology is supported by open, non-proprietary  standards which allow consumers to have easy access to information on  a wide variety of topics, music, shopping opportunities, online banking,  friends, their workplace, and more.
In 2013, the International Telecommunication Union, a United Nations agency,  estimated that there are 2.7 billion worldwide Internet users. number of modernized submarines. In 2009, the Navy reported  reductions of:
17 percent in program development and production costs,
13 percent in operating and support costs, and
80 percent in development time—developing and installing the sonar  on the first ship 2 years after the program started, as opposed to a  decade or more.
The Navy also used an open system architecture to upgrade its Virginia- class submarine program to counteract the effects of obsolescence and  ensure the submarines have the capability to respond quickly to changing  missions and threats. In 2010 the Navy reported $96 million in cost  avoidance since the upgrade program’s inception in 2001.
In addition, in 2008, Congress directed DOD to develop a strategy for  commonality and standardization in UAS ground control station  architecture. As part of a joint service/industry effort, the UAS Task  Force developed a common, open ground control station architecture,  called the UAS Control Segment (UCS), which can be integrated into any  UAS program’s ground control station. The UCS architecture enables the  reuse of individual software applications across different types of ground  control stations, such as those for weather or vehicle status, thus  eliminating the need to redevelop the applications for each new ground  control station. The first version of the UCS architecture was released in  2010, and the ability to share software applications was first made  available to UAS programs in January 2013. OSD and service officials  point to several benefits from using UCS software. For example, one task  force official stated that a demonstration of the UCS architecture in 2010  showed that the average time to integrate 10 applications was less than 2  weeks; significantly shorter than the traditional integration time for a new  ground control station capability, which can be about 1 year. DOD also  estimates that competition for software applications can eliminate up to  90 percent of their cost. In addition, the architecture will enable rapid  integration of new aircraft sensors, weapons, and other payloads.
section_title: Navy Leads Other Services in Implementation of an Open Systems Approach for UAS Acquisition Programs
paragraphs: The services vary in the extent to which they have adopted open systems  for DOD’s 10 largest UAS, with the Navy leading the other services.  Three of the Navy’s four current and planned UAS programs  incorporated, or are planning to incorporate, an open systems approach  from the start of development in key components of their UAS—the air  vehicle, ground control station, and payloads (i.e., cameras and radar  sensors). Conversely, none of the Army or Air Force UAS programs  incorporated the approach from the start of development because,  according to Army and Air Force officials, legacy UAS programs tried to  take advantage of commercial off-the-shelf technology or began as  technology demonstration programs. Several of these programs (4 of 6)  are starting to incorporate the approach, primarily for the ground control  station of fielded aircraft during planned upgrades. For example, the Army  did not initially include an open systems approach for its three UAS  programs, but has since developed a universal ground control station with  open interfaces that each of its programs will use. None of the Air Force’s  three UAS programs were initially developed as an open system, and  only one is being upgraded to include an open systems approach. Each  of the programs that have adopted an open systems approach expects to  achieve cost and schedule benefits, such as reduced upgrade costs and  quicker upgrade times. Figure 3 identifies the UAS programs that  introduced an open systems approach at the start of development for the  air vehicle, ground control station, and payloads.
Three of the Navy’s four current and planned UAS programs—the Small  Tactical UAS (STUAS), Triton, and Unmanned Carrier-Launched Airborne  Surveillance and Strike (UCLASS)—which are less than 5 years old,  included or are planning to include an open systems approach from the  start of development for the key components of their systems. The Fire  Scout program, which began in 2000, initially incorporated an open  systems approach for its ground control station and later included open  system elements in its air vehicle and payloads during an upgrade effort.  The Navy expects significant benefits in return, such as reduced  development and integration time and costs, as well as increased future  competition for new system payloads. Since these systems, for the most  part, either have not been fielded or have not begun development,  program officials were only able to provide us with estimates of potential  savings.
The STUAS program, which started in 2010, made modularity a key  requirement for the system, as documented in its acquisition strategy and  systems engineering plan, and required the contractor to provide the  program office with rights to key interfaces in the development contract.  For example, the STUAS program owns the specifications for the data  link between the air vehicle and ground control station, as well as the  specifications for the payload interfaces. Officials noted that these  elements allow the program to compete contracts for both the air vehicle  and ground control station in the future for upgrades or to meet changing  requirements. In addition, program and contractor officials noted that by  having the rights and specifications to the payload interfaces, the program  will be able to integrate and test third-party designed payloads within a  matter of days or months, as opposed to years typically required to test  new system payloads. Program officials also anticipate that they will be  able to independently integrate at least 32 different payloads developed  by 24 different manufacturers.
The Triton program, which started in 2008, developed an open systems  management plan and described in detail the steps it plans to take to  achieve an open system in its acquisition strategy and systems  engineering plan. The Triton includes both hardware and software  modularity, which gives the program the ability to integrate new payloads  and introduce software upgrades without affecting the rest of the system.  For example, officials noted that the system’s camera is a separate  subsystem, which makes it easier to upgrade the capability individually,  and the flight control software is separated from the rest of the system’s  software, resulting in less required testing when new capabilities are  added. Program officials estimated that software testing could be reduced  by as much as 66 percent compared to systems that do not have an open  system design, in part because the program does not have to test the  entire system when introducing new software or upgrades. In addition, the  program anticipates that the open system design will facilitate increased  competition among payload suppliers, which could result in lower prices  and better payload capabilities.
The Navy is planning to use an open systems approach for its future  UCLASS program. The program has identified and designated key  system interfaces and according to program officials, it plans to require  contractors to use particular open standards, such as an open payload  interface for electro-optical/infrared sensors, which is expected to reduce  acquisition costs and simplify integration, as well as an open avionics  architecture. The Navy developed the architectures and provided them to  industry beginning in 2012. Officials developing the avionics architecture  estimated that development and integration costs for multiple platforms  using the avionics could be reduced significantly, as compared to using a  closed avionics architecture.
The Fire Scout program included an open systems design for its original  ground control station when it began development in 2000, but the rest of  the system remained proprietary. However, when the program upgraded  from the MQ-8A to the MQ-8B air vehicle in 2004, it secured the data  rights to key system interfaces and introduced modularity into the air  vehicle and payloads. As a result of a new modular software design,  officials noted that payload data links are separated from the flight control  software and other mission systems, which is expected to cut the time  needed to test software and integrate a payload by about half. For  example, program officials said they were able to integrate a new radar in  only 18 months; officials estimated this integration previously would have  taken 3 years.
subsections: section_title: The Army and Air Force Are Incorporating Open Systems to a Limited Extent on Fielded UAS
paragraphs: The Army and Air Force did not originally plan to make the air vehicle,  ground control station, or payloads of their UAS programs open systems.  Instead, program officials stated that they relied on commercial off-the- shelf systems and technology demonstration programs that prime  contractors developed and then modified and upgraded over time, without  the benefit of competition that could help keep costs low. Over the past  several years, the Army has taken steps to make the ground control  stations open systems for all three of its fielded UAS and the payload  interfaces open in two of those systems, while the Air Force plans to  make the ground control station an open system for one of its three UAS.
The Army’s three UAS programs—Hunter, Shadow, and Gray Eagle— were all initially developed as proprietary systems and did not include an  open systems approach for all three key components—the air vehicle,  ground control station, and payloads. Moreover, the Army’s UAS ground  control stations limited interoperability and resulted in the Army paying for  ground control stations that provided similar capabilities. The Army  eventually developed a common ground control station for the three UAS;  however, the new station was still proprietary. All three of the Army’s UAS  programs are now upgrading to a universal ground control station that  incorporates an open architecture to address obsolescence issues and  increase interoperability. According to Army program officials, ground  control stations require continuous hardware and software upgrades as  the technology becomes obsolete. Even though an open systems  approach is being incorporated later in the programs’ life cycles, officials  believe the benefits—reduced obsolescence issues, reduced upgrade  costs, and increased interoperability—outweigh the costs. For example,  the Army’s new universal ground control station will give Army operators  in the field the ability to fly Hunter, Shadow, and Gray Eagle from one  ground control station. This was not possible with the Army’s legacy  ground control stations that did not use open architectures.
The Army’s Gray Eagle and Shadow programs are also incorporating  open system elements for the systems’ payloads to save integration time  and cost. For example, the Shadow program modified its air vehicle in  2008 to include a universal payload pod that allows any vendor that can  meet the standard power and data connections to design a payload to be  integrated onto the system. Program officials noted that, to date, four  third-party vendors have used the universal pod for their products.  Similarly, Gray Eagle program officials stated that the program owns the  data rights to all payload interface specifications, which allows third-party  vendors to develop a payload using these specifications. However,  officials noted that the program has to rely on the prime contractor to  integrate new payloads because the Army does not have the expertise to  do so.
The Air Force has had limited success in modernizing its UAS to include  open systems. For example, the Reaper plans to upgrade to an open  ground control station, but the remainder of the system remains  proprietary. The other two programs—the Predator and Global Hawk— included language in their planning documents stating their intention to  introduce open system elements later in their respective life cycles.  However, Predator’s age and Global Hawk’s fiscal constraints prevented  them from adopting an open systems approach. As a result, the two  systems remain largely proprietary and are now facing challenges  sustaining and upgrading their systems.
The Predator program began in 1994 as an advanced concept technology  demonstration program and is one of the oldest systems in DOD’s UAS  portfolio. Program officials stated that the Predator’s software is not  modular and the program has no intention of modifying the software  because the Air Force is planning to divest itself of Predator aircraft once  more Reapers are fielded. Predator officials also noted that sustainment  and obsolescence challenges remain a risk area for the program. Officials  from the Global Hawk program, which started development in 2001,  also stated that obsolescence is a major problem for that program,  particularly for the ground control station. The program recently had  planned to develop a new ground control station that utilized an open  systems architecture. However, the Air Force cancelled the upgrade effort  in 2013 due to what program officials described as fiscal constraints, even  though it plans to use the aircraft through at least 2032. The Air Force is  now planning to continue to maintain the legacy Global Hawk ground  control station and communications system, although contractor  obsolescence and other sustainment challenges contributed to a critical  Nunn-McCurdy cost breach in 2011.
section_title: Policies and Leadership Can Help Drive an Open Systems Approach for Weapon Acquisition Programs, but Challenges Exist
paragraphs: Policies and leadership support are two fundamental ways of getting  DOD’s weapon acquisition community to shift from a culture of relying on  contractors to provide proprietary systems to one where systems are  designed to be open. In general, policies are needed to establish the  overall plan and acceptable procedures that guide acquisition strategies,  and leadership is needed to champion policies and new initiatives, as well  as ensure that technical expertise is available to implement them. DOD  has cited a preference for acquiring open systems in its policies since  1994 and most recently in its Better Buying Power initiative, which  requires programs to outline an approach for using open systems  architectures at milestone B—the start of development. The Navy  developed an open systems policy in 2005 and its UAS programs largely  followed that policy from the start of development. The Air Force and  Army also have open systems policies, but their UAS programs did not  implement them from the start of development and, because of this, some  programs are now having difficulties upgrading their systems. Further,  while DOD leadership is placing a renewed emphasis on an open  systems approach through its Better Buying Power initiatives, it is not  currently tracking the extent to which weapon acquisition programs are  adopting open systems across the department or whether program offices  have enough technical expertise to effectively implement an open  systems approach. The Naval Air Systems Command has taken steps to  ensure that its UAS program offices have the expertise to implement  open systems by establishing a small group of experts that can assist  program offices with developing their technology development and  acquisition strategies with open systems in mind, but other program  offices may lack their own expertise or access to similar expertise.
subsections: section_title: Air Force and Army Are Lagging in Open Systems Implementation
paragraphs: Although DOD acquisition policy directs program managers to employ a  modular, open systems approach for acquisition programs to minimize  life-cycle costs, the Navy is the only service that did so while developing  its UAS programs. Based on our review of UAS programs, the Navy leads  the other services in adopting an open systems approach for its  acquisition programs, and Navy program officials cited their service’s  open systems policy as the driver for adoption. The Navy’s policy required  its programs to incorporate open architecture principles into program  requirements beginning in 2005, which means that UAS programs would  include an open systems approach as part of their acquisition strategies  prior to the start of program development. The Navy also established an  oversight and reporting structure to provide reasonable assurance that its  programs are following this policy. For example, the Navy assigns overall  responsibility and authority for directing the effort to one office, the Deputy  Assistant Secretary of the Navy for Research, Development, Test and  Evaluation. This office is tasked with defining an overarching strategy,  providing systems engineering leadership to other Navy acquisition  offices, and overseeing and reporting on implementation efforts.
The Air Force and Army also have policies that require open systems to  be included in a program’s acquisition strategy, which would occur prior to  the start of program development. However, none of their UAS programs  incorporated an open systems approach at the start of development,  including a program that began after the policies were issued. Instead,  these services favored off-the-shelf systems or those that could be  quickly fielded. In addition to not including open systems during the initial  design of a program, these service officials stated that they are having  some difficulties acquiring the necessary funding to make their legacy  systems open as they are being upgraded and we found they are not  consistently implementing the new UCS architecture developed for all  ground control stations. For example, the Global Hawk program planned  to use an open systems approach for its ground control station to address  obsolescence issues. However, as mentioned earlier, the Air Force  terminated the effort because of funding constraints, even though it plans  to use the aircraft through at least 2032 and the ground control station will  require upgrades and costly support during this time span. In addition, the  Air Force’s Global Hawk and Predator and the Navy’s STUAS do not plan  to adopt the recently developed UCS architecture at this time. The Global  Hawk had planned to incorporate UCS as part of its now cancelled  upgrade effort and Predator officials stated that the Air Force is not  investing in upgrades, including UCS.
Further complicating the UCS architecture effort is the Air Force, which  developed a “complementary” ground control station architecture without  the involvement of the other services or OSD. The Air Force plans to use  both architectures on the Reaper system. Officials explained that they  developed the complementary architecture because the Air Force wanted  to be able to attain the capability faster. However, both architectures will  be fielded simultaneously on the Reaper and contractors developing UAS  ground control station software stated that adopting both architectures will  be more costly for DOD.
To improve implementation of an open systems approach, DOD’s 2010  Better Buying Power initiative now requires programs, at milestone B, to  outline an approach for using open systems architectures and acquiring  technical data rights to ensure sustained consideration of competition in  the acquisition of the weapon systems. The approach is to be based on a  business case analysis and engineering trade analysis.
section_title: DOD Leadership Lacks Visibility into Open Systems Implementation and Military Service Expertise
paragraphs: DOD recognizes that it needs to increase its leadership efforts to  implement open systems for its weapon acquisition programs, but does  not know the extent to which programs are implementing this approach or  if the services have the requisite technical expertise. According to federal  internal control standards, agencies should monitor programs to ensure  that policies are being implemented, such as by establishing and tracking  program metrics, and also ensure that their workforce has the required  skills to implement policies. Thus, DOD should monitor open systems  implementation and also assess whether program offices have the  expertise to carry out the open systems policies. Various independent  standards experts, DOD contractors, and DOD systems engineering  officials identified DOD leadership as key to effective adoption of an open  systems approach. Several of these officials cautioned that prime  contractors may be resistant to providing open systems because they are  able to achieve greater financial benefits by selling DOD proprietary  products, which they alone can integrate, upgrade, and maintain.  Specifically, these officials believe that successful implementation of an  open systems approach requires that DOD provide clear and consistent  direction for its approach and that programs plan for an open systems  approach prior to the start of development. This includes defining the  open systems approach in a program’s technology development and  acquisition strategies prior to initiating the technology development phase  at milestone A and the start of engineering and manufacturing  development at milestone B in DOD’s acquisition process, respectively.  Further, they believe that contracts and requests for proposal should  include appropriate language that describes the open systems  architecture, defines open standards, and establishes requirements for  control documents to ensure the government retains rights for the  identified key interfaces.
Acquiring a new weapon system using an open systems approach could  be more expensive at the beginning, particularly regarding ownership of  data rights. Yet, the potential benefits of obtaining the data rights, in terms  of greater competition and reduced upgrade and repair costs, are widely  believed to significantly outweigh the costs, especially if done from the  start of development. DOD could avoid purchasing data rights if it works  with industry to develop standards. However, in these cases, standards  organizations’ officials we spoke with said that DOD would have to work  with standards organizations several years in advance to ensure that the  standards are available at the start of development.
The Better Buying Power initiative for open systems is one of DOD’s  efforts to increase competition and drive efficiency in acquisitions. While  competition is difficult to do at the system level because there are a  limited number of prime contractors that develop DOD weapon systems,  an open systems approach can boost opportunities for competition at the  subsystem level. As part of the Better Buying Power initiatives, the Under  Secretary of Defense for Acquisition, Technology and Logistics formed an  Open Systems Architecture Data Rights Team co-led by the Office of the  Deputy Assistant Secretary of Defense for Systems Engineering (SE) and  the Deputy Assistant Secretary of the Navy for Research, Development,  Test and Evaluation.
The team is developing guidance, tools, and training in an effort to gain  more momentum for the use of an open systems approach and the  acquisition of appropriate data rights across DOD. This includes a  contract guidebook to be issued in the summer of 2013 intended to assist  program managers in incorporating open systems principles into their  acquisition programs. Officials from two of the UAS programs we  reviewed, UCLASS and Triton, told us they had used a draft version of  this guidebook to help inform their contracting strategies. The team also  updated the systems engineering guidance in the Defense Acquisition  Guidebook to recommend that program managers update their  technology development strategy, acquisition strategy, and systems  engineering plan throughout their program’s life cycle to reflect their  respective open systems strategies. Finally, the team is leveraging other  resources previously developed by the Open Systems Joint Task Force,  such as an analytical tool for programs to monitor and evaluate their open  systems implementation and a program manager’s guide intended to  assist program offices in integrating an open systems approach into their  acquisition strategies for new and legacy systems. According to officials  from the Open Systems Architecture Data Rights Team, the team is  required to report to the Under Secretary of Defense for Acquisition,  Technology and Logistics on its progress in meeting initiatives related to  Better Buying Power by October 2013, but that reporting has not yet  begun.
The Deputy Assistant Secretary of Defense for Systems Engineering  indicated that several new programs are incorporating an open systems  approach during early planning. For example, the Navy’s Next Generation  Jammer included open systems language in its 2012 request for  proposal. Additionally, the Army’s Ground Combat Vehicle program and  the Air Force’s Space Fence program both considered open systems as  part of their future design prior to or during technology development.  However, it is unclear the extent to which programs are adopting an open  systems approach across the department because this information is not  being tracked.
In addition, it is unclear whether the services have sufficient technical  expertise to implement an open systems approach and enable DOD to be  a savvy buyer of weapon systems. Program offices need open systems  expertise to conduct systems engineering activities, even before a  contract is awarded, to be able to determine how an open systems  strategy fits into program requirements; identify which system interfaces  are key to maintaining competitive opportunities; ensure sufficient data  rights are obtained to support the competitive strategy; translate open  system requirements into contracts; address any risks associated with the  open systems approach; and finally, validate that the contractor is  providing the open system that the program required. However, officials  from 11 offices/companies we spoke with, including a DOD contractor;  the UAS Task Force; 6 of 10 Air Force, Army, and Navy UAS programs;  the Naval Air Systems Command technical experts; the DOD Open  Systems Architecture Data Rights Team; and an open systems expert  who consulted with DOD on an open systems approach, stressed that  DOD does not have adequate expertise across the department to  effectively implement an open systems approach.
The Defense Acquisition University (DAU), the organization responsible  for training the acquisition workforce DOD-wide, offers training on open  systems as part of its core acquisition curriculum required for both  engineers and program managers. However, officials from every program  we talked to who had taken this training told us that the training is only  sufficient to familiarize participants with the concept of open systems, but  not in-depth enough to allow participants to implement the approach.  DAU is developing additional training resources in support of open  systems to be rolled out beginning fall 2013. This new training, according  to a DAU official, will be geared toward providing program managers and  data management personnel information on data rights. Open systems  will be discussed, but primarily to give the participants awareness of the  approach. There are currently no plans for new in-depth training focused  on an open systems approach. Some of the material may be added to the  core acquisition curriculum, but officials said this transition can take  years.
As required by the Weapon Systems Acquisition Reform Act of 2009, the  Deputy Assistant Secretary of Defense for Systems Engineering is  responsible for reviewing the systems engineering capabilities of the  military departments and identifying areas that require changes or  improvements. To meet this statutory requirement, SE requests that the  Air Force, Army, and Navy submit a self-assessment of their systems  engineering capabilities, which SE reviews to identify needed changes or  improvements to the services’ capabilities. SE conducts this review  annually and publishes its findings in annual reports to Congress, most  recently in March 2013. However, the services have not provided an  analysis of their open systems capabilities as part of their self- assessments. Therefore, the Deputy Assistant Secretary cannot be sure  that program offices have adequate systems engineering resources to  effectively implement an open systems approach.
The Naval Air Systems Command has a small group of open systems  technical experts that its acquisition programs can use when developing  their open systems approach. Officials from the UCLASS, STUAS, and  Triton programs said they relied on experts from this team to assist them  in planning and executing their open systems approach, including  developing a request for proposal, selecting open standards, and  responding to contractor bids. UCLASS program officials also said they  received assistance from the Deputy Assistant Secretary of the Navy for  Research, Development, Test and Evaluation—the Navy’s lead office for  open systems—when developing its request for proposal.
section_title: Conclusions
paragraphs: The adoption of an open systems approach in DOD acquisition can  provide significant cost and schedule savings for DOD. Based on  projections from several Navy UAS programs, for example, the Navy  could avoid considerable repair and upgrade costs on individual  programs, as well as improve performance, by incorporating an open  systems approach on programs prior to the start of development. This is  because multiple suppliers can compete to quickly replace key  components on the UAS with more capable components. Traditionally,  DOD has acquired proprietary weapon systems that limit these  opportunities and make these systems more costly to develop, procure,  upgrade, and support. DOD has cited a preference for acquiring open  systems in its policy since 1994 and each of the services have since  issued open systems policies. Yet, we found the Army and Air Force have  been slow to make their UAS open systems, particularly from the start of  development. The Navy, on the other hand, has generally designed its  UAS to be open from the start of development where it can reap the most  benefits.
To successfully change the services’ inclination toward buying proprietary  systems and shift towards the acquisition of open systems, DOD needs to  address policy and leadership challenges. Moving to open systems  requires strong leadership to overcome preferences for acquiring  proprietary systems. While DOD’s Better Buying Power initiative requires  programs to outline an approach for using open systems architectures at  milestone B, OSD does not have adequate insight of the extent to which  an open systems approach is being used by individual weapon  acquisition programs. Further, OSD does not know if program offices  have the systems engineering expertise required for effective  implementation of an open systems approach or if additional expertise is  needed. Without adequate knowledge of policy implementation and  program office expertise, DOD cannot have reasonable assurance that an  open systems approach is being implemented effectively by the services.  Until DOD takes action to overcome these challenges, the department will  likely continue to invest in costly proprietary systems. These steps should  increase DOD’s ability to promote more competition, save taxpayer  dollars, and more quickly field new capabilities to the warfighter,  particularly if an open systems approach is incorporated into program  strategies prior to the start of development at milestone B.
section_title: Recommendations for Executive Action
paragraphs: We are making four recommendations to improve the department’s  implementation of an open systems approach for UAS and other weapon  acquisition programs, as well as its visibility of open systems  implementation and program office expertise.
We recommend that the Secretary of Defense direct the Secretaries  of the Air Force and Army to implement their open systems policies by  including an open systems approach in their acquisition strategies.
We recommend that the Secretary of Defense direct the Under  Secretary of Defense for Acquisition, Technology and Logistics to  define appropriate metrics to track programs’ implementation of an  open systems approach.
We also recommend that the Secretary of Defense direct the Secretaries  of the Air Force, Army, and Navy to take the following actions:  require their acquisition programs to include open systems metrics  developed by the Under Secretary of Defense for Acquisition,  Technology and Logistics in their systems engineering plans, track  progress in meeting these metrics, and report their progress to the  Under Secretary of Defense for Acquisition, Technology and Logistics  at key acquisition milestones; and assess their respective service-level and program office capabilities  relating to an open systems approach and work with the Deputy  Assistant Secretary of Defense for Systems Engineering to develop  short-term and long-term strategies to address any capability gaps  identified. Strategies could include the Navy’s cross-cutting approach  where a team of a few technical experts within the Naval Air Systems  Command could be available to work with program offices, as  necessary, to help develop open systems plans.
section_title: Agency Comments and Our Evaluation
paragraphs: DOD provided us with written comments on a draft of this report. DOD  partially concurred with all four recommendations. DOD’s comments are  reprinted in appendix II.
DOD partially concurred with our first recommendation, agreeing on the  value of implementing an open systems approach, but citing existing  department policies and guidance which it believes are sufficient for the  military departments to implement open systems architecture in  acquisition programs. DOD also noted in its comments that the decision  to implement an open systems approach in a particular program’s  acquisition strategy is made on a case-by-case basis based on a number  of considerations to include mission, threat, vulnerability assessment,  operating environment, and business case. We agree that an open  systems approach should be informed by these considerations, and we  also cited both OSD and service-level policies and guidance governing an  open systems approach in our report. However, a number of Air Force  and Army unmanned aircraft programs missed opportunities to adopt an  open systems approach early in their life cycles, but did try to do so later  on when it becomes more costly and complex. Navy programs fared  better partly because they augmented policy by addressing the open  systems approach in individual acquisition strategies. We maintain that  the Air Force and Army would benefit by including an open systems  approach in their acquisition strategies before the start of system  development.
DOD partially concurred with our second and third recommendations.  DOD noted that program implementation of an open systems approach  should be assessed through its existing milestone decision process. DOD  further noted that acquisition strategies and systems engineering plans,  which document a program’s open systems strategy, are assessed at  multiple decision reviews and are considered by the milestone decision  authority in the overall context of the program. We agree that the  milestone decision process is the appropriate venue to review programs’  open systems strategies. However, as discussed in our report, we found  that OSD does not have adequate insight of the extent to which an open  systems approach is being used by weapon acquisition programs and  thus cannot have reasonable assurance of the widespread use of an  open systems approach across the department. Based on our review of  unmanned aircraft programs, we found that the Navy had generally  embraced an open systems approach for its acquisition programs before  the start of development, whereas the Air Force and Army had not.  Further, it is unclear the extent to which the Air Force and Army will adopt  the approach for its programs as DOD’s recent Better Buying Power  initiative requires. To provide clearer insight into whether the services are  planning for and implementing an open systems approach, we maintain  that the Under Secretary of Defense for Acquisition, Technology and  Logistics should establish metrics and track weapon acquisition  programs’ progress in meeting them at key milestones. For example, this  could include tracking whether programs are including open systems in  their acquisition strategy documents prior to milestone B and are following  through on those plans at subsequent acquisition milestones.
DOD partially concurred with our fourth recommendation but did not  explain its position or what, if anything, it would do in response. DOD did  not comment on whether the military services and their program offices  have sufficient capabilities with respect to open systems. As we  discussed in our report, we found that OSD does not know if program  offices have the systems engineering expertise required for effective  implementation of an open systems approach or if additional expertise is  needed. To address this possible gap, we continue to believe that the Air  Force, Army, and Navy should assess their respective service-level and  program office capabilities relating to an open systems approach and  work with the Deputy Assistant Secretary of Defense for Systems  Engineering to develop short-term and long-term strategies to address  any capability gaps identified. As we suggested, one such strategy could  be the Naval Air Systems Command’s cross-cutting approach where a  team of a few technical experts could be available to work with program  offices, as necessary.
We are sending copies of this report to the Secretary of Defense, the  Secretary of the Air Force, the Secretary of the Army, the Secretary of the  Navy, and interested congressional committees. In addition, the report is  available at no charge on the GAO website at http://www.gao.gov.
If you have any questions about this report or need additional information,  please contact me at (202) 512-4841 or sullivanm@gao.gov. Contact  points for our Office of Congressional Relations and Public Affairs may be  found on the last page of this report. Key contributors to this report are  listed in appendix III.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: The House Armed Services Subcommittee on Tactical Air and Land  Forces requested that we assess the Department of Defense’s (DOD)  progress in implementing open systems in unmanned aircraft system  (UAS) acquisitions. Since an open systems approach can facilitate  competition and upgrades for any type of weapon system acquisition, and  because DOD is directing its current open system initiatives department- wide, we evaluated the benefits of an open systems approach and  challenges DOD is facing in adopting the approach broadly. Therefore,  this report examines (1) the characteristics and benefits of an open  systems approach, (2) DOD’s efforts in applying an open systems  approach to its UAS portfolio, and (3) challenges, if any, DOD is  encountering in implementing this approach.
To determine the characteristics and benefits of an open systems  approach, we reviewed relevant DOD policies, guidance, and handbooks  and conducted a literature search to identify open systems-related  examples from private companies and standards development  organizations. We interviewed officials from four standards organizations  that we selected based on our literature search: the Aerospace Industries  Association, the Peripheral Component Interconnect Industrial Computer  Manufacturers Group, SAE International, and the Universal Serial Bus  Implementers Forum. These organizations have responsibility for  developing open standards for their respective industries, and their  experience is commensurate with the age of their industries. For  example, the Universal Serial Bus Implementers Forum has existed for  almost 20 years, and SAE International has existed for over 100 years.  We also interviewed personnel from DOD contractors, including AAI,  Dreamhammer, General Atomics-Aeronautical Systems Group, Insitu,  Northrop Grumman, and Raytheon/Solipsys to identify their leading  practices in developing open standards and implementing an open  systems approach. In addition, we interviewed an expert from Carnegie  Mellon’s Software Engineering Institute about experiences working with  DOD on an open systems approach. We also interviewed DOD officials  knowledgeable about open systems and open standards development  from the Army, Navy, and the Office of the Secretary of Defense (OSD).  We analyzed interviews with the open systems expert, DOD officials,  standards organizations, and all four of the contractors who we asked  questions related to open systems best practices (General Atomics,  Northrop Grumman, Raytheon/Solipsys, and Dreamhammer) to identify  key practices for successfully adopting an open systems approach.
To determine the extent to which DOD is applying an open systems  approach to its UAS portfolio, we interviewed and collected  documentation, including briefings, acquisition strategies, and systems  engineering plans, from the UAS Task Force and UAS program offices on  open systems efforts. In addition, we developed a questionnaire to collect  information on which elements of an open systems approach UAS  programs incorporated, their use of guidance and tools, and their  contracting strategies. After we drafted the questionnaire, we asked our  Chief Technologist as well as a member of DOD’s Open Systems  Architecture Data Rights Team to review and validate our questions. We  revised the questionnaire to reflect comments from these reviews, then  distributed the electronic Microsoft Word questionnaire by e-mail to 10  current and planned UAS acquisition programs from Groups 2-5—the  larger UAS programs—in October 2012; all 10 responded that month. We  excluded Group 1 UAS from our analysis because these are the smallest  UAS systems and recent DOD open systems efforts focus on Group 2-5  UAS. Using each program’s questionnaire response and follow-up  interviews conducted with each program office to corroborate the  responses, we evaluated the key areas where UAS programs  implemented an open systems approach—the air vehicle, the ground  control station, and the system payloads—and determined at what point  in the acquisition life cycle that UAS programs introduced their approach.
To determine challenges acquisition programs face in implementing an  open systems approach, we synthesized information from DOD policies  and guidance, program questionnaires, and interviews with officials from  the Office of the Deputy Assistant Secretary of Defense for Systems  Engineering; the Office of the Secretary of Defense’s UAS Task Force;  the Defense Acquisition University; the Office of the Deputy Assistant  Secretary of the Navy for Research, Development, Test and Evaluation;  acquisition offices within the Air Force, Army, and Navy; UAS program  offices; and UAS contractor personnel.
We conducted this performance audit from July 2012 through July 2013 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Comments from the Department of Defense
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, key contributors to this report  were Cheryl Andrew, Assistant Director; Richard Burkard; Lisa Fisher;  Danielle Greene; Laura Jezewski; Deanna Laufer; Brian Lepore; Jean  McSween; Paige Muegenburg; Brian Mullins; and Roxanna Sun.