Unmanned Aircraft Systems: DOD Needs to More Effectively Promote Interoperability and Improve Performance Assessments
Published: 2005-12-13T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Unmanned aircraft systems (UAS) consist of an unmanned aircraft; sensor, communications, or weapons, carried on board the aircraft, collectively referred to as payloads; and ground controls. UAS have been used successfully in recent operations, and are in increasingly high demand by U.S. forces. To meet the demand, the Department of Defense (DOD) is increasing its investment in and reliance on UAS, and often deploying them while still in development. GAO has previously found that DOD's approach to developing and fielding UAS risked interoperability problems which could undermine joint operations. GAO was asked to review (1) UAS performance in recent joint operations and (2) the soundness of DOD's approach to evaluating joint UAS operational performance.
section_title: What GAO Found
paragraphs: DOD has achieved certain operational successes using UAS, including identifying time-critical targets in Iraq and Afghanistan, and striking enemy positions to defeat opposing forces. Some missions effectively supported joint operations, and in other cases, the missions were service-specific. DOD has encountered challenges which have hampered joint operations at times. First, some UAS cannot easily transmit and receive data with other communication systems because they are not interoperable. Although DOD guidance requires interoperability, detailed standards for interoperability have not been developed; DOD has relied on existing, more general standards; and the services developed differing systems. For now, U.S. forces have developed technical patches permitting transmission but slowing data flow, potentially hampering time-critical targeting. Second, some sensor payloads cannot be interchangeably used on different UAS because DOD has not adopted a payload commonality standard. Some UAS missions may have to be delayed if compatible unmanned aircraft and payloads are not available. Based on its experience with UAS in Persian Gulf operations, U.S. Central Command believes communications interoperability and payload commonality problems occur because the services' UAS development programs have been service-specific and insufficiently attentive to joint needs. Lastly, the electromagnetic spectrum needed to control the flight of certain unmanned aircraft and to transmit data is constrained and no standard requiring the capability to change frequencies had been adopted because the problem was not foreseen. Thus, some systems cannot change to avoid congestion and consequently some missions have been delayed, potentially undermining time-critical targeting. In addition to the joint operational challenges, inclement weather can also hamper UAS operations. Unmanned aircraft are more likely to be grounded in inclement weather than manned aircraft and DOD had not decided whether to require all-weather capability. While DOD has acknowledged the need to improve UAS interoperability and address bandwidth and weather constraints, little progress has been made. Until DOD adopts and enforces interoperability and other standards, these challenges will likely remain and become more widespread as new UAS are developed and fielded. DOD's approach to evaluating UAS joint operational performance has been unsound because it was not systematic or routine. DOD has deployed UAS before developing a joint operations performance measurement system, even though results-oriented performance measures can be used to monitor progress toward agency goals. DOD has generally relied on after-action and maintenance reports which have useful but not necessarily joint performance information. DOD has also relied on short-duration study teams for some performance information but had not established ongoing or routine reporting systems. Thus, while continuing to invest in UAS, DOD has incomplete performance information on joint operations on which to base acquisition or modification decisions. In May 2005, U.S. Strategic Command began developing joint performance measures.

section_title: Letter
paragraphs: Unmanned aircraft systems (UAS) consist of unmanned aircraft; sensors,  weapons, and communications equipment carried on board the aircraft,  known as “payloads”; and ground control stations that control the flight of  the aircraft and receive information collected and transmitted by the  payloads.  UAS have been used successfully in recent military operations  on intelligence, surveillance, and reconnaissance; and offensive strike  missions.  Due to the successes, U.S. forces are increasingly demanding  that more UAS be supplied to them, prompting the Department of Defense  (DOD) to try and rapidly develop and field these emerging technologies.   Moreover, to meet the demand, DOD has substantially increased its  investment in these systems from about $363 million in fiscal year 2001 to  about $2.2 billion in fiscal year 2005, not including supplemental  appropriations.
In March 2004, we reported that DOD’s approach to planning for  developing and fielding UAS did not provide reasonable assurance that its  investment strategy will facilitate their efficient integration into the force  structure.  We also reported that DOD’s approach increased the risk of  future interoperability problems, which could undermine joint operations,  and would likely be insufficient to prevent duplication of effort from one  service-specific program to another.  As a result, we recommended that  DOD develop a strategic plan for these systems’ development and fielding  and assign the UAS Planning Task Force or other appropriate entity within  DOD with sufficient authority to enforce program direction specified in the  plan.  DOD partially concurred with our recommendation to develop a  strategic plan and nonconcurred with the recommendation to place an  entity in charge.  DOD’s rationale for nonconcurring was that the  Undersecretary of Defense (Acquisition, Technology, and Logistics) and by  extension its UAS Planning Task Force had sufficient authority to develop  and enforce interoperability and other standards, and that the Joint  Capabilities Integration and Development System process promoted joint  war fighting and thus would avoid interoperability problems. In addition,  DOD has issued guidance requiring interoperable communications  capabilities in DOD weapons and other systems.
As requested, we reviewed (1) the operational performance of UAS in  recent joint operations and (2) the soundness of DOD’s approach to  evaluating joint UAS operational performance.  To evaluate the operational  performance of UAS in recent operations, we examined DOD or service  regulations, directives, instructions, after-action reports, performance  evaluations, and other documents.  We also met with key DOD and service  officials to discuss current UAS operational status, future plans, initiatives  to address emerging challenges, and related issues.   To review DOD’s  approach to evaluating joint UAS performance, we obtained relevant DOD  directives, instructions, and other documents, and met with DOD and  service officials to identify the performance measurement systems in place  and operating.  We determined that the data on the numbers and types of  missions performed by UAS were sufficiently reliable for this review.  We  performed our work from July 2004 to October 2005 in accordance with  generally accepted government auditing standards.
subsections: section_title: Background
paragraphs: DOD defines UAS as a powered aircraft that does not carry a human  operator; can be land-, air-, or ship-launched; uses aerodynamic forces to  provide lift; can be autonomously or remotely piloted; can be expendable  or recoverable; and can carry a lethal or nonlethal payload.  Generally, UAS  consist of the aircraft; a flight control station; information and retrieval or  processing stations; and, sometimes, wheeled land vehicles that carry  launch and recovery platforms.  UAS carry a payload including sensors for  intelligence, surveillance, or reconnaissance to provide real-time  intelligence to battlefield commanders.  When used on an intelligence,  surveillance, or reconnaissance mission, generally, the aircraft carries a  sensor payload capable of detecting heat, movement, or taking  photographs or video of ground-based targets.  This information is then  transmitted to ground stations or satellites via a communications payload  for retransmission to forces needing the information to support operations.   Unmanned aircraft can also be armed for offensive strike missions and be  used to attack ground-based targets.  UAS require adequate intra- or  intertheater communications capabilities using the electromagnetic  spectrum to permit operators to control certain aircraft, and also permit  communications equipment to transmit the information obtained by the  sensor payload to ground commanders or other users.
Effective joint operations are critical because combatant commanders  operate in a joint environment by applying military force appropriate for  their operational circumstance using the unique capabilities of each of the  services.  In a changing security environment, joint operations are  becoming more important given the complex nature of military operations.   This importance is being driven by the combatant commands’ need to  combine the capabilities of multiple services to address the global threat,  as well as the growing interdependence of capabilities among the services.   Moreover, effective joint operations permit combatant commanders to  leverage the capabilities associated with each service to accomplish  operational missions.  As with manned aircraft, UAS provide another  capability that can be applied by combatant commanders in joint  operations.
subsections: section_title: Evolution of UAS Development and Use
paragraphs: Initially, UAS were seen as complementary systems that augmented  existing war fighting capabilities.  However, UAS are also evolving into  more significant roles, for which they can provide primary capability.  For  example, the Global Hawk UAS may eventually replace the U-2  reconnaissance aircraft, and the Unmanned Combat Aerial System may  eventually perform electronic warfare missions currently performed by the  EA-6 Prowler aircraft as well as offensive deep strike missions.  Moreover,  UAS are figuring prominently in plans to transform the military into a more  strategically responsive force and are expected to be an integral part of this  information-based force.  For example, the Army is developing the Future  Combat System and a new generation of unmanned aircraft and other  systems to enable information to flow freely across the battlefield.
Since 2001, DOD has significantly increased its planned expenditure for  UAS and associated systems, and, more recently, the systems have  continued to be heavily used in Afghanistan and Iraq.  In fact, over 10  different types of UAS have been used in Afghanistan and Iraq.  According  to the UAS Planning Task Force, as of August 2005, DOD had  approximately 1,500 unmanned aircraft in Iraq and Afghanistan.  In  addition, the budget request for UAS grew significantly between fiscal  years 2001 and 2005, from about $363 million to about $2.2 billion, and  further growth is likely.  These figures do not include any supplemental  appropriations.
Fewer than half of the UAS in Iraq and Afghanistan at the time of our report  had reached full-rate production or initial operating capability.  They were  still considered developmental, and consequently were covered by DOD  Directive 5000.1, The Defense Acquisition System and DOD Instruction  5000.2, Operation of the Defense Acquisition System, both issued in May  2003.  The directive mandates that systems, units, and forces shall be able  to provide and accept data, information, materiel, and services to and from  other systems, units, and forces, and shall effectively interoperate with  other U.S. forces, among other things. The instruction implements the  directive and is intended to provide DOD officials with a framework for  identifying mission needs and technology to meet the needs, as the basis  for weapons system acquisitions.  Finally, the 2002 Roadmap emphasizes  the need for interoperable unmanned aircraft and payloads by identifying a  number of existing standards that are to be complied with in systems’  development in such areas as common data links, interoperable data links  for video systems, and electromagnetic spectrum frequencies that should  be used for data transmission under a variety of circumstances.
section_title: Prior GAO Reviews of DOD’s Planning for Developing and Fielding UAS
paragraphs: In March 2004, we reported that DOD’s approach to planning for  developing and fielding UAS does not provide reasonable assurance that its  investment will facilitate efficient integration into the force structure and  avoid interoperability problems, although DOD had taken some steps to  improve UAS program management.  For example, in 2001, DOD  established the Joint Unmanned Aerial Vehicles Planning Task Force (now  known as the UAS Planning Task Force) in the Office of the Undersecretary  of Defense (Acquisition, Technology, and Logistics). To communicate its  vision and promote commonality of UAS, the Task Force published the  2002 Unmanned Aerial Vehicle Roadmap, which described current  programs, identified potential missions, and provided guidance on  emerging technologies. While the Roadmap demonstrated some elements  of a strategic plan, neither it nor other key documents represented a  comprehensive strategic plan to ensure that the services and DOD agencies  develop systems that complement each other, perform all required  missions, and avoid duplication. Moreover, the Task Force served in an  advisory capacity to the Undersecretary, but had little authority to enforce  program direction. For their part, service officials told us that they  developed service-specific planning documents to meet their own needs  and operational concepts without considering those of other services or  the Roadmap. In consequence, we concluded that without a strategic plan  and an oversight body with sufficient authority to enforce program  direction, DOD risked interoperability problems, which could undermine  joint operations.  Thus, in our 2004 report, we recommended that DOD  establish a strategic plan and assign an office authority and responsibility  to enforce program direction communicated in the plan to promote joint  operations.
DOD partially concurred with our recommendation to establish a strategic  plan and nonconcurred with our recommendation to assign an office with  authority and responsibility to enforce program direction.  DOD asserted  that the Undersecretary had sufficient authority to integrate UAS into joint  operations and that the Task Force had been established to promote  payload commonality, develop and enforce interface standards, and ensure  multiservice coordination.  Moreover, DOD indicated that the Joint  Capabilities Integration and Development System process focuses on  developing integrated joint warfighting capabilities and thus would avoid  interoperability problems that we believed were likely.
section_title: UAS Have Achieved Certain Mission Successes but DOD Faces Emerging Interoperability and Other Challenges on Joint Operations
paragraphs: DOD has achieved certain operational successes with UAS including  collecting intelligence with unmanned aircraft sensor payloads and  conducting offensive strike missions with weapons payloads in  Afghanistan and Iraq.  Nonetheless, U.S. forces employing UAS have  encountered certain communications and payload interoperability  problems (called payload commonality problems), electromagnetic  spectrum constraints, and inclement weather groundings of unmanned  aircraft during recent operations.  While DOD has acknowledged the need  to improve UAS interoperability and address bandwidth and weather  constraints that undermine unmanned aircraft operations, little progress  has been made.
subsections: section_title: UAS Have Played an Integral Role in Mission Accomplishments
paragraphs: DOD has achieved certain operational successes from its use of a variety of  unmanned aircraft and their sensor, communications, and armaments  payloads.   In operations in Iraq or Afghanistan since 2002, U.S. forces have  used UAS in integral roles on intelligence, surveillance, reconnaissance,  and offensive strike joint or service-specific missions.  For example:  The Air Force used its Predator unmanned aircraft with sensor or  armaments payloads on over 5,800 sorties or totaling more than 80,000  hours of flight on a variety of intelligence, surveillance, and  reconnaissance; close air support; armed strike; and other missions in  Iraq and Afghanistan from 2002 through 2005.  For example, the  Predator’s sensor and communications payloads have provided video  images to ground forces to support their operations or to strike enemy  targets with Hellfire missiles.
Certain Air Force units used the Global Hawk unmanned aircraft’s  sensor payloads to identify 55 percent of the time-critical targets to  defeat enemy air defenses in Iraq in March and April 2003.  To enhance  joint operations, the Air Force developed procedures and tactics to  allow the Global Hawk’s sensor payloads to provide more direct support  to ground force missions.
In 2004, an Army force used its Hunter unmanned aircraft and sensor  payload to locate an enemy antiaircraft artillery weapon that had been  firing at coalition force aircraft.  Then the Air Force sent a Predator  armed with a Hellfire missile to attack the enemy weapon.  Within  minutes of the Predator strike, the Army unit sent its Hunter back to  transmit information needed for battle damage assessment.
In 2004, an Army force operating an I-Gnat unmanned aircraft in Iraq  detected a potential ambush of Marine Corps forces and the Army unit  used information from the I-Gnat’s sensor payload to successfully adjust  mortar fire onto the enemy position.
Recently, the Air Force, Army and Marine Corps forces have used their   unmanned aircraft and their sensor and communications payloads to  locate numerous targets in Iraq and Afghanistan to permit U.S. forces to  destroy the targets.
section_title: UAS Interoperability Remains a Challenge
paragraphs: While achieving certain successes with the use of unmanned aircraft and  their payloads, certain interoperability challenges have also emerged  during recent operations despite certain DOD directives requiring  interoperability and the emphasis on interoperability in the 2002 Roadmap.   First, DOD Directive 5000.1 specifies that systems, units, and forces shall  be able to provide and accept data and information to and from other  systems and shall effectively interoperate with other U.S. forces.  Second,  the Roadmap specifies five data standards for formatting data, a  communication standard to ensure adoption of a common data link, and a  variety of file transfer, physical media, and other standards applicable to  unmanned aircraft or their sensor and communications payloads.   However, the 2005 edition of the Roadmap indicates that the detailed  standards for interoperability have not been developed.  In effect, the  absence of such standards has led to the development of UAS that are not  interoperable.  In operations in Afghanistan and Iraq, interoperability  problems have emerged.  Specifically, during operations, DOD has learned  that unmanned aircraft sensor and communications payloads and ground  stations were not designed to common data standards and thus are not  interoperable, even within a single service in certain circumstances.  For  example:  Army forces operate both the Shadow and Hunter unmanned aircraft  and associated ground stations but discovered that these systems are  not interoperable.  Specifically, while the Shadow’s sensor and  communications payload is able to transmit information to its own  ground station, it is unable to transmit to a Hunter ground station.   Similarly, the Hunter’s sensor and communication payloads are able to  transmit to a Hunter ground station but not Shadow’s.  Onward  transmission to forces needing the information is equally constrained if  they do not have compatible equipment for receiving the information.   As a result, the Army has missed an opportunity to effectively leverage  the technology inherent in either system for the benefit of operational  forces that need the information.  At the time of our review, the Army  had begun an initiative to make the Shadow and Hunter unmanned  aircraft ground stations compatible with either aircraft.
When communication systems are incompatible, operating forces may be  prompted to operate their own UAS, thus increasing the numbers of  systems operating in the same area.   To permit the sharing of tactical  intelligence obtained by unmanned aircraft sensor payloads, the services or  combatant commands have developed certain technical patches permitting  compatibility but slowing data transmission.  As we pointed out in 2003, in  some cases, DOD needs hours or days to transmit information to multiple  services. However, slow intelligence data transmission can undermine U.S.  forces’ ability to attack time-critical targets or allow the targets to escape.   U.S. Central Command acknowledges that timely data dissemination is  critical to combat operations.
Communications interoperability problems are a long-standing problem.  In  2001, we reported that each of the military services plans, acquires, and  operates systems to meet its own operational concepts but not necessarily  the requirements of joint operations in spite of the DOD directive requiring  interoperability.  In our 2004 unmanned aerial vehicle report, we reported  that the services engaged in little coordination in developing their  unmanned aerial vehicle roadmaps and that they did not view the UAS  Planning Task Force’s 2002-2027 Roadmap as a strategic plan or an  overarching architecture for integrating UAS into the force structure.  In  the absence of adequately developed and implemented standards and in  contravention of the DOD guidance, the services have continued to develop  their unmanned systems to their own standards, but without regard to the  others’ standards.  At the same time, DOD continues to develop and field  UAS without adjusting the standards, likely causing the problem to become  even more widespread.  Moreover, the UAS used in current operations  were built before the Joint Capabilities Integration and Development  System became fully operational and thus has had little impact on the  problem.  Consequently, the information collected cannot always be  quickly transmitted to users needing it, undermining joint operations and  potentially leading to future costly initiatives to modify existing unmanned  aircraft, sensors and communications payloads, and ground stations to  overcome interoperability problems.
In addition to communications interoperability problems, payload  interoperability (commonly referred to as “payload commonality”)  problems also exist.  DOD has developed at least six different sensor  payloads each able to collect different types of information.  These sensor  payloads are attached to an unmanned aircraft and flown over operational  areas to observe activity of interest on the ground in a target area and to  transmit observations to ground or air forces or other users as tactical  intelligence.  As an example, figure 1 displays a Predator unmanned aircraft  with a sensor payload attached underneath.
However, many sensor payloads can be attached to only one type of  unmanned aircraft because DOD has not adopted a payload commonality  standard even though this problem was identified nearly 20 years ago.  As a  result, commanders may have to delay missions if the appropriate sensor is  available but no unmanned aircraft is able to carry it.  We discussed this  problem in 1988 when we reported that DOD had not adequately  emphasized payload commonality for unmanned aircraft and that Congress  had stressed the need for DOD to consider payload commonality in 1985.   The 2002 Roadmap acknowledged the need for sensor payload  commonality where practical, but limited progress has been made.
In addition to the flexibility inherent in the communications standards,  according to U.S. Central Command based on its experience in Persian Gulf  operations, unmanned aircraft development has been service-centric and  lacks an overarching employment doctrine to shape development to  achieve aircraft and sensor interoperable communications and payload  commonality.  Furthermore, a Joint Forces Command official told us that  combatant commanders can not take full advantage of the dissimilar  unmanned aircraft or the sensor payload data produced due to the  interoperability problems.
section_title: Congested Electromagnetic Spectrum Has Hampered UAS Operational Effectiveness
paragraphs: Unmanned aircraft and their sensor, armaments, and communications  payloads depend on reliable access to the electromagnetic spectrum.   However, the spectrum is increasingly constrained, potentially  undermining joint operations by requiring delays in an unmanned aircraft  flight or, if the problem worsens, cancellation.  Unmanned aircraft  operators use the electromagnetic spectrum to maintain contact with the  aircraft to control its flight, fire its weapons if armed, and receive  information collected by the sensor payloads.  Certain spectrum  frequencies are sometimes referred to as bands and the amount of the  spectrum needed to permit transmission of information is referred to as  bandwidth.  DOD officials told us that more bandwidth is needed to  transmit video and other information obtained by sensor payloads than to  maintain flight control of the aircraft.  Numerous weapons also use  electromagnetic spectrum and share it with UAS but they can interfere with  each other during operations if they operate on the same frequency at the  same time.
The military services have experienced bandwidth capacity constraints,  limiting both the number of UAS and other systems that can be effectively  operated simultaneously and the amount of available data that can be  transmitted from the unmanned aircraft communications payload.  For  example, insufficient bandwidth limits U.S. forces’ ability to download  video and radar images via satellite from more than one aircraft at a time.   As a result, data transmission and relay are delayed, undermining U.S.  forces’ ability to engage time-critical targets and possibly permitting the  target to escape, unless alternative information sources are available on a  timely basis.  Army officials informed us that data link limitations are due  primarily to frequency congestion.  Table 1 displays the bands used by 12  different unmanned aircraft or models of unmanned aircraft for flight  control and sensor payload data transmission.  As shown in the table,  several UAS rely on the C-band for their data transmission capability, and  only 2 of the 12 UAS can be reprogrammed to another band.
The 2002 Roadmap established a goal of modifying the Army’s Shadow  UAV to permit it to operate a common tactical data link in Ku-band and not  the more congested C-band.  This goal had not been met at the time of our  review and the Shadow unmanned aircraft still operated in C-band.   Similarly, the 2002 Roadmap established a goal of moving the Air Force’s  Predator unmanned aircraft video sensor payload from C-band to Ku-band  for line of sight operations.  However, the goal had not been met at the time  of our report.
Moreover, the problem cannot be easily overcome without potentially  costly modifications to existing systems because DOD has not required  unmanned aircraft or sensor payloads to be reprogrammable from one  band to another and therefore has not established such standards.  As a  result, most have been designed and built without the flexibility to operate  in differing frequencies or bands to avoid congested frequencies, thus  sometimes preventing timely information transmission or delaying their  flight without interfering with or experiencing interference from other UAS  or other weapons systems.
section_title: Inclement Weather Limits Some Unmanned Aircraft Operations
paragraphs: Unmanned aircraft are more likely to be grounded by inclement weather  than manned aircraft due in part to their lighter weight.  Dust storms,  strong winds, rain, or icing prevent some unmanned aircraft from flying,  thus denying U.S. forces critically needed information unless alternative  data collection or offensive strike capabilities are available.  Specifically,  winds up to 80 miles per hour in Iraq and Afghanistan have reduced the  availability of most unmanned aircraft and dust storms have undermined  the use of some sensor payloads.  Moreover, the 2002 Roadmap indicates  that icing has been a primary factor in two accidents involving the Hunter  unmanned aircraft and three crashes of the Predator unmanned aircraft.   The Roadmap established a goal to incorporate all-weather capabilities  into future UAS.  However, little progress has been made because DOD has  not adopted standards for all-weather capability to be considered in  development, despite the Roadmap’s stating the goal.  As a result, systems  have been developed without it.  At the same time, according to a UAS  Planning Task Force official, developing unmanned aircraft with all- weather capabilities may result in some degradation in performance, such  as a reduced flying range.  At the time of our review, DOD had not  determined whether all-weather capability was worth the trade-off of  potentially degraded performance.
section_title: DOD Has Made Little Progress in Addressing the Challenges
paragraphs: While DOD has acknowledged the need to improve UAS interoperability  and address bandwidth and weather constraints that undermine unmanned  aircraft operations, little progress has been made. On the one hand, to  begin to address the problems, DOD has taken a number of steps as listed  below:  In August 2005, DOD issued an updated version of its roadmap, entitled  2005 Unmanned Aircraft Systems Roadmap, to guide acquisition and  interoperability.  Among other things, the 2005 Roadmap establishes the  goal of enhancing joint service collaboration as a means to improve joint  operations.
At the time of our review, the Office of the Secretary of Defense was  preparing an action plan to address a number of shortfalls including  interoperability and other problems within U.S. Central Command’s area  of responsibility, although the plan was limited to just this command and  would not necessarily solve the problems that UAS might encounter  elsewhere.
DOD plans to reemphasize the role that the Joint Capabilities  Integration and Development System could play in all new UAS  developments by trying to ensure that DOD develops systems to support  joint operations, achieve commonality to the extent practical, and  identify gaps in DOD’s ability to carry out its warfighting missions.
U.S. Joint Forces Command has developed certain initiatives to improve  UAS interoperability by conducting experiments to demonstrate aircraft  modifications and new concepts of operations, although such  modifications can be costly.
In addition, on June 1, 2005, DOD’s Joint Requirements Oversight Council  established a new Joint Unmanned Aerial Vehicle Center of Excellence and  a Joint Unmanned Aerial Vehicle Overarching Integrated Process Team.   The Joint Unmanned Aerial Vehicle Overarching Integrated Process Team  has subsequently been renamed the Joint Unmanned Aircraft Systems  Material Review Board.  These joint forums will help the services manage  development of new UAS or modifications to existing UAS, and they will  help the services to develop new or revised concepts of operations for  more effective use.  At the same time, the UAS Planning Task Force will try  to ensure that the services’ UAS acquisition programs are coordinated, and  a Task Force representative is to be a member of the Joint Overarching  Integrated Process Team.  DOD views these changes as means to more  effectively manage service UAS programs.  While these changes appear to  be steps in the right direction, it is too early for us to tell if they will solve  the interoperability and other problems that we identified.
Furthermore, payload commonality, interoperability of communications  and data transmission systems, and inclement weather flying capabilities  that we identified as impacting recent operations, had been identified  previously as problems already occurring or likely to occur.  First, our 1988  unmanned aerial vehicle report indicated that DOD had not adequately  emphasized payload commonality for these aircraft.  Second, our 2001  report found interoperability problems due to the services’ continued  practice of acquiring systems to support their own operations but not  necessarily that of the other services. Third, DOD’s guidance requires  interoperability but the detailed standards have not been developed.   Lastly, the 2002 edition of the Roadmap identified the need to improve  interoperability of communications systems for UAS and also identified  inclement weather capability as a problem undermining UAS operations  and established goals to address it.  Despite all the emphasis, problems  related to communications and payload interoperability, and all-weather  capability problems remain.  DOD acknowledges that it (1) did not foresee  the rapid technological development experienced with unmanned aircraft,  sensor or communications payloads, and ground stations; (2) has provided  unmanned aircraft and payloads rapidly to deployed forces to meet forces’  demands for them; and (3) has not always adopted new or enforced  existing standards that might have prevented or mitigated some of these  problems.  As a result, while DOD has issued a directive, instructions,  guidance, and roadmaps, and established at least five different  organizations to promote UAS interoperability and address other  unmanned aircraft and payload developmental needs, no organization has  or has exercised sufficient authority to enforce program direction, or  ensure that the standards and guidance are in concurrence.  As a result, the  services continued to develop and field these systems without fully  complying with the interoperability requirements stated in key guidance or  addressing known payload commonality problems.
section_title: DOD’s Approach to Evaluating Joint UAS Performance on Operational Deployments Has Been Unsound
paragraphs: DOD’s approach to evaluating joint UAS performance on operational  deployments is unsound because it has not implemented a systematic  approach to evaluating such performance.  Instead, DOD has relied on  systems for evaluating performance that are not focused on joint  operations and are nonroutine, and as a result the department has little  assurance that the information that has been collected represents the key  performance indicators needed to assess performance on joint operations.
subsections: section_title: DOD Lacks Performance Indicators to Assess Certain Joint UAS Performance
paragraphs: DOD has not implemented a systematic approach to evaluating joint UAS  performance on operational deployments.  As we previously noted in our  2004 report, the Government Performance and Results Act’s strategic  planning framework specifies that results-oriented performance measures  can be used to monitor progress toward agency goals and that such  performance measurements should be developed and used to monitor  progress. At the time of our report, DOD was only beginning to decide on  key indicators of performance that would be used to assess unmanned  aircraft, payload, and ground station performance on joint operations.  To  date, DOD has relied on service-specific information that addressed certain  UAS performance.  For example, some forces filed after-action reports and  maintenance reports addressing UAS performance.  While producing some  useful information, these reports have not necessarily been specifically  targeted to joint UAS operations, nor do they systematically identify key  indicators for collection which could be used to develop joint operational  performance baselines and permit performance measurement against the  baseline.  Thus, DOD has little assurance that the information that has been  collected represents the key performance indicators needed to assess joint  operations performance.
DOD officials told us that they have tried to keep pace with operating  forces’ demands for more unmanned aircraft and their payloads, and  therefore the services have deployed them while still under development  within the DOD acquisition system.  These deployments have often  occurred before identification of key performance indicators that would  need to be collected to be used to evaluate performance.  In effect, the  services have bought and deployed unmanned aircraft, sensor and  communications payloads, and ground stations and tried to evaluate their  effectiveness all at the same time.  On the one hand, this has permitted  DOD to provide operating forces with new capabilities represented by the  aircraft and their payloads. On the other hand, it has also resulted in DOD  and the services sometimes learning of joint performance problems based  on reporting from actual operations only if after-action reports or other  reporting mentioned the problem. Nonetheless, without appropriate  performance measures and baselines against which to assess performance  on joint operations, even anecdotal information can have limited utility  because officials are less likely to be able to assess the magnitude of the  problem, or even become aware of it if no reports identify it.
DOD has acknowledged the need to develop specific performance  indicators for unmanned aircraft and their payloads on joint operations and  had begun to develop them at the time of our report.  First, the Army  recently began an initiative to develop performance indicators and a  baseline against which to assess performance.  However, while this  approach may produce useful information on which to assess the  performance of Army-operated unmanned aircraft, payloads, and ground  stations, it was not designed to address joint performance.  The other three  services had not started to develop specific performance indicators and  baselines for unmanned aircraft at the time of our review.  Second, in May  2005, DOD assigned U.S. Strategic Command responsibility for the  development of joint performance indicators but the effort was just getting  started at the time of our review.
section_title: DOD’s UAS Joint Performance Reporting Has Not Been Routine
paragraphs: In addition to anecdotal performance reporting, DOD has not established  routine performance reporting mechanisms for UAS operations but instead  has relied on sometimes short-duration study teams to gather relevant joint  operational performance information.  For example, in November 2004,  DOD established a group known as a “Tiger Team” to identify opportunities  for improving the joint operational effectiveness of UAS.  However, this  team was established on a temporary basis and had a limited mission to  identify improvements only in the U.S. Central Command area of  responsibility.  The Tiger Team did identify a number of areas needing  improvement.  For example, it determined that forces in the region need  Full Motion Video capability to provide images of actual events as they  occur.  The team also determined that a need exists to address  electromagnetic spectrum limitations hampering UAS operations.   However, the team identified the electromagnetic spectrum problem only  after the UAS had been deployed and U.S. forces had tried to use them on  operational missions.  In addition, also in 2004, another DOD short- duration study team evaluated the operational performance of the Shadow  unmanned aircraft.  Lastly, the Army conducted a one-time comprehensive  review of the effectiveness of its UAS in theaters of operations.  While  these teams developed useful performance information, the approach does  not represent a systematic or long-term means to obtain joint UAS  performance information since the teams are not permanently established  and they did not use consistent study parameters.
Finally, even in the instances where some ongoing processes were used,  the information obtained was relevant only on a service-specific but not a  joint basis.  For example, the Marine Corps uses its Operational Advisory  Group process to determine needed improvements in its UAS operations.   While this group has developed useful information that may assist the  Marine Corps in enhancing its ability to effectively use UAS in operations,  the information developed is likely to have limited utility for joint  operations.
DOD acknowledges that the speed with which unmanned aircraft,  payloads, communications, and associated technology are being  developed, along with the imperative to provide emerging technologies  quickly to operating forces, have resulted in the deployment of  developmental systems before adequate performance reporting systems  have been established.  Consequently, while the systems are being  successfully used in overseas operations, DOD does not have reasonable  assurance that it is well informed on opportunities to further enhance the  ability of operational forces to take advantage of UAS capabilities.
section_title: Conclusions
paragraphs: DOD has achieved certain operational successes with UAS but certain  challenges have also emerged that have hampered joint operations or  prevented effective employment of UAS.  These challenges are caused by  the limited attention paid to interoperability standards for UAS and the lack  of detailed interoperability standards.  Development and implementation of  appropriate interoperability, payload commonality, and other standards  help to ensure that such problems are addressed during development and  any problems are fixed prior to deployment.  Moreover, until DOD assesses  the extent to which a lack of detailed standards undermines the purpose of  the broad standards by allowing development of noninteroperable systems  and enforces common standards among the services, problems are likely to  continue and possibly be repeated and made more widespread as new  unmanned aircraft, sensor and communication payloads, ground stations,  and related equipment are developed and fielded.  In addition, costly  modifications might be needed later.
The unsoundness in the approach DOD has taken to assessing joint UAS  performance in operational deployments was due to a lack of accepted  performance indicators and a routine system for collecting performance  information.  Until DOD develops specific indicators of UAS joint  operational performance, establishes appropriate baselines against which  to measure performance, and communicates which indicators operating  forces should systematically collect and report to appropriate users, DOD  will lack reasonable assurance that it is adequately informed on UAS  performance on joint operations.  Moreover, DOD may also be poorly  informed as to its progress in addressing interoperability and other  problems and may therefore be less likely to avoid the same problems in  future UAS development and fielding.
Lastly, in our 2004 report, we recommended that DOD establish a strategic  plan and an office with sufficient authority to enforce program direction to  avoid interoperability problems and for other purposes.  In nonconcurring  with our recommendation to assign an office with sufficient authority to  enforce program direction, DOD indicated that the UAS Planning Task  Force and Joint Capabilities Integration and Development System had  sufficient authority and would address interoperability, payload  commonality, and other problems.  However, these problems persist.   Consequently, we continue to believe that sustained management attention  is warranted.  Without such attention, DOD continues to risk undercutting  the benefit of its continued investment in UAS.  Consequently, we continue  to believe that our prior recommendation has merit, but we are not  reiterating it because DOD indicated that it will not implement it.
section_title: Recommendations for Executive Action
paragraphs: To address the challenges emerging in joint operations, we recommend that  the Secretary of Defense direct the Undersecretary of Defense  (Acquisition, Technology, and Logistics), the Chairman of the Joint Chiefs  of Staff, the service secretaries, and other appropriate organizations to  work together to take the following four actions  develop or adjust communications interoperability standards and  electromagnetic frequency reprogramming capabilities standards and  ensure that they are applied to new or modified unmanned aircraft,  sensor and communications payloads, ground stations, and related  equipment;   develop sensor and other payload commonality standards where  practical and enforce such standards when modifying existing  unmanned aircraft or payloads and developing new ones;  develop appropriately detailed UAS interoperability standards; and   determine whether unmanned aircraft need all-weather flying  capabilities, identify any performance degradation associated with all- weather flying capabilities, and obtain all-weather capabilities where  appropriate.
To improve joint operational performance reporting, we recommend that  the Secretary of Defense direct the Commander of the U.S. Strategic  Command to ensure that the performance measurement system being  developed by the command at a minimum  measures how effectively UAS perform their missions by identifying  quantifiable goals and comparing results with desired outcomes;  identifies the specific performance indicator information that needs to  be collected to adequately assess joint performance;  develops indicators that assess communications and payload  interoperability, and the extent to which electromagnetic spectrum  congestion is undermining joint operations;  establishes baselines and applies the identified indicators against the  baselines to gauge success in joint UAS performance; and  develops a way to systematically collect identified performance  information and routinely reports it to organizations that develop and  field UAS.
section_title: Agency Comments and Our Evaluation
paragraphs: DOD provided written comments on a draft of this report.  These  comments are reprinted in their entirety in appendix II.  We made five  recommendations and DOD fully or partially concurred with them.  It also  provided technical comments, which we incorporated into our report as  appropriate.
First, DOD concurred with our recommendation for the appropriate DOD  organizations to work together to develop or adjust communications  interoperability standards and electromagnetic frequency reprogramming  capabilities standards and ensure that they are applied to new or modified  unmanned aircraft, sensor and communications payloads, ground stations,  and related equipment.  In concurring, DOD indicated that it recognized the  utility of communications interoperability and the need to improve this  capability and will direct the services to use common frequencies and data  links to enhance communications interoperability.
Second, in partially concurring with our recommendation to develop and  enforce sensor and other payload commonality standards where practical,  DOD commented that it does not typically focus on payload  interchangeability.  Instead, DOD pointed out that unmanned aircraft  payload procurement is a service responsibility and is dependent on  service mission requirements, unmanned aircraft physical design  limitations, and rapid  technological evolution.  Our report recognizes that  it is not practical for all unmanned aircraft sensors and payloads to be  common due to the various sizes of some aircraft and we worded our  recommendation accordingly.
Third, DOD fully concurred with our recommendation that the appropriate  DOD organizations work together to develop appropriately detailed UAS  interoperability standards.  DOD indicated that the UAS Roadmap 2005- 2030 released in August 2005 discusses the preferred framework,  methodology, and standards for achieving UAS interoperability.  DOD  outlined a number of actions that it has taken to address UAS  interoperability standards, including ratifying a North Atlantic Treaty  Organization Standards Agreement aimed at achieving joint and combined  interoperability.  The Joint Chiefs of Staff has tasked the newly formed  Joint UAS Material Review Board and Joint UAV Center of Excellence to  provide recommendations for continuing to improve UAS interoperability.
Fourth, DOD fully concurred with our recommendation to determine  whether unmanned aircraft need all-weather flying capabilities, identify  any performance degradation associated with all-weather capabilities, and  obtain all-weather capabilities where appropriate.  DOD commented that  combatant commanders should expect UAS to support operations in  diverse weather conditions.  Further, DOD indicated that as UAS  capabilities improve, the weather conditions these systems will need to  operate in will also increase.  However, DOD also points out that it is not  cost effective to expect all classes of unmanned aircraft to have an all- weather capability.  We agree.  The intention of our recommendation is for  DOD to determine those UAS for which all-weather capabilities are cost  effective and to add such capabilities when appropriate.
Finally, DOD partially concurred with our recommendation that U.S.  Strategic Command  ensure that the performance measurement system  being developed at a minimum includes quantifiable goals, performance  baselines, systematic collection procedures, measures of communications  and payload interoperability, and performance indicators against which to  measure performance.  DOD indicated that the U.S. Strategic Command  has drafted a Joint Functional Component Concept of Operations that  includes metrics to gauge the force’s ability to meet intelligence,  surveillance, and reconnaissance requirements.  Moreover, DOD stated that  in conjunction with the services, intelligence community, combatant  commanders, and other DOD organizations, this action would facilitate not  only the evaluation of UAS performance but would enable DOD to have the  necessary information available to assess such factors as UAS  requirements, mission accomplishment, UAS capabilities, and customer  satisfaction.  DOD also pointed out that the performance measures are in  development and will require service participation to define the specific  data and methodology which will result in useful information.  While we  acknowledge that these actions should address many of the data elements  that we believe are necessary to evaluate UAS, we continue to believe that  effective communications, interoperability, and avoidance of frequency  congestion are important contributors to the success of joint operations.   Therefore, we continue to believe that DOD should ensure that, at a  minimum, the U.S. Strategic Command includes the data elements we  recommended in its performance measurement system.  In addition, we  agree that other organizations including the services, should participate in  the development of this measurement system if appropriate.
We are sending copies of this report to other appropriate congressional  committees, the Secretary of Defense, the secretaries of the Army, the  Navy, and the Air Force; the Commandant of the Marine Corps; the  Chairman of the Joint Chiefs of Staff; and the Director, Office of  Management and Budget.  We will also make copies available to other  interested parties upon request.  In addition, the report will be available at  no charge on the GAO Web site at http://www.gao.gov.
If you or your staff have any questions about this report, please call me at  (202) 512-9619 or email at pickups@gao.gov.  Contact points for our Office  of Congressional Relations and Public Affairs may be found on the last  page of this report.  The GAO contact and key contributors are listed in  appendix III.
section_title: Scope and Methodology
paragraphs: To evaluate the operational performance of unmanned aircraft systems  (UAS) in recent operations, we examined the Department of Defense  (DOD) regulations, directives, and instructions as well as service guidance  and documentation on UAS.  We met with key DOD and service officials,  including those from the UAS Planning Task Force and UAS program  managers, to discuss the current status and future plans for these systems.    We reviewed the Unmanned Aerial Vehicles Roadmap 2002-2027 because  this document establishes an overall DOD management framework for  developing and employing UAS DOD-wide and the update, 2005  Unmanned Aircraft Systems Roadmap. During discussion and visits with  DOD and service officials, we obtained and reviewed DOD and service  analyses, briefings, and summary reports describing each of the UAS used  in supporting recent combat and combat support operations.  This included  obtaining detailed information on current and future UAS operational  capabilities.   Additionally, we obtained information on the numbers and  types of missions performed by UAS, as well as the methods used by the  services to evaluate UAS performance in accomplishing those missions.  To  assess the reliability and types of missions provided to us by DOD, we (1)  interviewed knowledgeable officials about the processes for collecting and  maintaining the data and (2) reviewed the data for completeness and  reasonableness by comparing it to other sources of information.  We  determined that the data were sufficiently reliable for the purposes of this  review.  DOD and service officials also provided specific examples of  operational successes and emerging challenges.  We discussed actions  taken and processes used by DOD and service officials and the Joint  Capabilities Integration and Development System to address identified  challenges.  We also held discussions with Joint Staff officials to discuss  their efforts to address joint UAS issues via the Tiger Team.
The specific military activities that we visited and/or obtained written  responses to questions from include the following:  Office of the Undersecretary of Defense (Acquisition, Technology, and  Logistics) and its Joint UAS Planning Task Force; Washington, D.C.;   Headquarters, Department of the Army; Washington, D.C.;  U.S. Army Redstone Arsenal, Huntsville, Alabama;  U.S. Marine Corps, Systems Command, Quantico, Virginia;  U.S. Navy Naval Sea Systems Command, Naval Air Station Patuxent   U.S. Air Force Air Combat Command Directorate of Requirements,  Langley Air Force Base, Virginia;  U.S Air Force, Air Force Material Command, Wright Patterson Air Force  Base Dayton, Ohio;  U.S. Joint Forces Command, Norfolk, Virginia;  U.S. Central Command, MacDill Air Force Base, Tampa, Florida;  U.S. Special Operations Command, MacDill Air Force Base, Tampa,   U.S. Joint Staff, Washington, D.C., and  U.S Strategic Command, Omaha, Nebraska.
We also obtained documents describing the mission and planned  operations of the new Joint Unmanned Aerial Vehicle Center of Excellence  and Joint Unmanned Aerial Vehicle Overarching Integrated Process Team.
To assess the soundness of DOD’s approach to evaluating UAS operational  performance, we interviewed DOD and service officials to discuss the  criteria and processes used to assess performance.  We also obtained and  reviewed DOD and Army UAS Operations Assessments to identify issues  and concerns regarding performance. Additionally, we held discussions  with U.S. Strategic Command officials to obtain information on the status  of their efforts to establish measures for assessing joint UAS performance.   We also held discussions with service officials to determine the extent to  which they are required to capture information on the use and performance  of UAS in their existing lessons-learned systems.  Finally, we obtained and  reviewed DOD and service specific UAS or unmanned aerial vehicle  roadmaps.
We performed our work from July 2004 to October 2005 in accordance with  generally accepted government auditing standards.
section_title: Comments from the Department of Defense
section_title: GAO Contacts and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Acknowledgments
paragraphs: In addition to the person named above, Brian J. Lepore, Assistant Director;  Harry E. Taylor, Jr.; Patricia F. Albritton; Jeanett H. Reid; Elisha T. Matvay;  Robert B. Brown; Cheryl A. Weissman; Ron La Due Lake; and Kenneth E.  Patton also made major contributions to this report.
section_title: Related GAO Products
paragraphs: Unmanned Aerial Vehicles: Improved Strategic and Acquisition  Planning Can Help Address Emerging Challenges.  GAO-05-395T.  Washington, D.C.: March 9, 2005.
Unmanned Aerial Vehicles: Changes in Global Hawk’s Acquisition  Strategy Are Needed to Reduce Program Risks.  GAO-05-6.  Washington,  D.C.: November 5, 2004.
Unmanned Aerial Vehicles: Major Management Issues Facing DOD’s  Development and Fielding Efforts.  GAO-04-530T.  Washington, D.C.:  March 17, 2004.
Force Structure: Improved Strategic Planning Can Enhance DOD’s  Unmanned Aerial Vehicles Efforts. GAO-04-342. Washington, D.C.: March  17, 2004.
Nonproliferation: Improvements Needed for Controls on Exports of  Cruise Missile and Unmanned Aerial Vehicles. GAO-04-493T. Washington,  D.C.: March 9, 2004.
Nonproliferation: Improvements Needed to Better Control Technology  Exports for Cruise Missiles and Unmanned Aerial Vehicles. GAO-04-175.  Washington, D.C.: January 23, 2004.
Defense Acquisitions: Matching Resources with Requirements Is Key to  the Unmanned Combat Air Vehicle Program’s Success. GAO-03-598.  Washington, D.C.: June 30, 2003.
Unmanned Aerial Vehicles: Questionable Basis for Revisions to Shadow  200 Acquisition Strategy. GAO/NSIAD-00-204. Washington, D.C.:  September 26, 2000.
Unmanned Aerial Vehicles: Progress of the Global Hawk Advanced  Concept Technology Demonstration. GAO/NSIAD-00-78. Washington, D.C.:  April 25, 2000.
Unmanned Aerial Vehicles: DOD’s Demonstration Approach Has  Improved Project Outcomes. GAO/NSIAD-99-33. Washington, D.C.: August  30, 1999.
Unmanned Aerial Vehicles: Progress toward Meeting High Altitude  Endurance Aircraft Price Goals. GAO/NSIAD-99-29. Washington, D.C.:  December 15, 1998.
Unmanned Aerial Vehicles: Outrider Demonstrations Will Be Inadequate  to Justify Further Production. GAO/NSIAD-97-153. Washington, D.C.:  September 23, 1997.
Unmanned Aerial Vehicles: DOD’s Acquisition Efforts. GAO/T-NSIAD-97- 138. Washington, D.C.: April 9, 1997.
Unmanned Aerial Vehicles: Hunter System Is Not Appropriate for Navy  Fleet Use. GAO/NSIAD-96-2. Washington, D.C.: December 1, 1995.
Unmanned Aerial Vehicles: Performance of Short-Range System Still in  Question. GAO/NSIAD-94-65. Washington, D.C.: December 15, 1993.
Unmanned Aerial Vehicles: More Testing Needed Before Production of  Short-Range System. GAO/NSIAD-92-311. Washington, D.C.: September 4,  1992.
Unmanned Aerial Vehicles: Medium Range System Components Do Not  Fit. GAO/NSIAD-91-2. Washington, D.C.: March 25, 1991.
Unmanned Aerial Vehicles: Realistic Testing Needed Before Production of  Short-Range System. GAO/NSIAD-90-234. Washington, D.C.: September 28,  1990.
Unmanned Vehicles: Assessment of DOD’s Unmanned Aerial Vehicle  Master Plan. GAO/NSIAD-89-41BR. Washington, D.C.: December 9, 1988.