Unmanned Aircraft Systems: DOD Needs to More Effectively Promote Interoperability and Improve Performance Assessments
Released: 2005-12-13T00:00:00

Letter

Background

DOD defines UAS as a powered aircraft that does not carry a human operator; can be land-, air-, or ship-launched; uses aerodynamic forces to provide lift; can be autonomously or remotely piloted; can be expendable or recoverable; and can carry a lethal or nonlethal payload. Generally, UAS consist of the aircraft; a flight control station; information and retrieval or processing stations; and, sometimes, wheeled land vehicles that carry launch and recovery platforms. UAS carry a payload including sensors for intelligence, surveillance, or reconnaissance to provide real-time intelligence to battlefield commanders. When used on an intelligence, surveillance, or reconnaissance mission, generally, the aircraft carries a sensor payload capable of detecting heat, movement, or taking photographs or video of ground-based targets. This information is then transmitted to ground stations or satellites via a communications payload for retransmission to forces needing the information to support operations. Unmanned aircraft can also be armed for offensive strike missions and be used to attack ground-based targets. UAS require adequate intra- or intertheater communications capabilities using the electromagnetic spectrum to permit operators to control certain aircraft, and also permit communications equipment to transmit the information obtained by the sensor payload to ground commanders or other users.

Effective joint operations are critical because combatant commanders operate in a joint environment by applying military force appropriate for their operational circumstance using the unique capabilities of each of the services. In a changing security environment, joint operations are becoming more important given the complex nature of military operations. This importance is being driven by the combatant commands’ need to combine the capabilities of multiple services to address the global threat, as well as the growing interdependence of capabilities among the services. Moreover, effective joint operations permit combatant commanders to leverage the capabilities associated with each service to accomplish operational missions. As with manned aircraft, UAS provide another capability that can be applied by combatant commanders in joint operations.

Evolution of UAS Development and Use

Initially, UAS were seen as complementary systems that augmented existing war fighting capabilities. However, UAS are also evolving into more significant roles, for which they can provide primary capability. For example, the Global Hawk UAS may eventually replace the U-2 reconnaissance aircraft, and the Unmanned Combat Aerial System may eventually perform electronic warfare missions currently performed by the EA-6 Prowler aircraft as well as offensive deep strike missions. Moreover, UAS are figuring prominently in plans to transform the military into a more strategically responsive force and are expected to be an integral part of this information-based force. For example, the Army is developing the Future Combat System and a new generation of unmanned aircraft and other systems to enable information to flow freely across the battlefield.

Since 2001, DOD has significantly increased its planned expenditure for UAS and associated systems, and, more recently, the systems have continued to be heavily used in Afghanistan and Iraq. In fact, over 10 different types of UAS have been used in Afghanistan and Iraq. According to the UAS Planning Task Force, as of August 2005, DOD had approximately 1,500 unmanned aircraft in Iraq and Afghanistan. In addition, the budget request for UAS grew significantly between fiscal years 2001 and 2005, from about $363 million to about $2.2 billion, and further growth is likely. These figures do not include any supplemental appropriations.

Fewer than half of the UAS in Iraq and Afghanistan at the time of our report had reached full-rate production or initial operating capability. They were still considered developmental, and consequently were covered by DOD Directive 5000.1, The Defense Acquisition System and DOD Instruction 5000.2, Operation of the Defense Acquisition System, both issued in May 2003. The directive mandates that systems, units, and forces shall be able to provide and accept data, information, materiel, and services to and from other systems, units, and forces, and shall effectively interoperate with other U.S. forces, among other things. The instruction implements the directive and is intended to provide DOD officials with a framework for identifying mission needs and technology to meet the needs, as the basis for weapons system acquisitions. Finally, the 2002 Roadmap emphasizes the need for interoperable unmanned aircraft and payloads by identifying a number of existing standards that are to be complied with in systems’ development in such areas as common data links, interoperable data links for video systems, and electromagnetic spectrum frequencies that should be used for data transmission under a variety of circumstances.

Prior GAO Reviews of DOD’s Planning for Developing and Fielding UAS

In March 2004, we reported that DOD’s approach to planning for developing and fielding UAS does not provide reasonable assurance that its investment will facilitate efficient integration into the force structure and avoid interoperability problems, although DOD had taken some steps to improve UAS program management. For example, in 2001, DOD established the Joint Unmanned Aerial Vehicles Planning Task Force (now known as the UAS Planning Task Force) in the Office of the Undersecretary of Defense (Acquisition, Technology, and Logistics). To communicate its vision and promote commonality of UAS, the Task Force published the 2002 Unmanned Aerial Vehicle Roadmap, which described current programs, identified potential missions, and provided guidance on emerging technologies. While the Roadmap demonstrated some elements of a strategic plan, neither it nor other key documents represented a comprehensive strategic plan to ensure that the services and DOD agencies develop systems that complement each other, perform all required missions, and avoid duplication. Moreover, the Task Force served in an advisory capacity to the Undersecretary, but had little authority to enforce program direction. For their part, service officials told us that they developed service-specific planning documents to meet their own needs and operational concepts without considering those of other services or the Roadmap. In consequence, we concluded that without a strategic plan and an oversight body with sufficient authority to enforce program direction, DOD risked interoperability problems, which could undermine joint operations. Thus, in our 2004 report, we recommended that DOD establish a strategic plan and assign an office authority and responsibility to enforce program direction communicated in the plan to promote joint operations.

DOD partially concurred with our recommendation to establish a strategic plan and nonconcurred with our recommendation to assign an office with authority and responsibility to enforce program direction. DOD asserted that the Undersecretary had sufficient authority to integrate UAS into joint operations and that the Task Force had been established to promote payload commonality, develop and enforce interface standards, and ensure multiservice coordination. Moreover, DOD indicated that the Joint Capabilities Integration and Development System process focuses on developing integrated joint warfighting capabilities and thus would avoid interoperability problems that we believed were likely.

UAS Have Achieved Certain Mission Successes but DOD Faces Emerging Interoperability and Other Challenges on Joint Operations

DOD has achieved certain operational successes with UAS including collecting intelligence with unmanned aircraft sensor payloads and conducting offensive strike missions with weapons payloads in Afghanistan and Iraq. Nonetheless, U.S. forces employing UAS have encountered certain communications and payload interoperability problems (called payload commonality problems), electromagnetic spectrum constraints, and inclement weather groundings of unmanned aircraft during recent operations. While DOD has acknowledged the need to improve UAS interoperability and address bandwidth and weather constraints that undermine unmanned aircraft operations, little progress has been made.

UAS Have Played an Integral Role in Mission Accomplishments

DOD has achieved certain operational successes from its use of a variety of unmanned aircraft and their sensor, communications, and armaments payloads. In operations in Iraq or Afghanistan since 2002, U.S. forces have used UAS in integral roles on intelligence, surveillance, reconnaissance, and offensive strike joint or service-specific missions. For example: The Air Force used its Predator unmanned aircraft with sensor or armaments payloads on over 5,800 sorties or totaling more than 80,000 hours of flight on a variety of intelligence, surveillance, and reconnaissance; close air support; armed strike; and other missions in Iraq and Afghanistan from 2002 through 2005. For example, the Predator’s sensor and communications payloads have provided video images to ground forces to support their operations or to strike enemy targets with Hellfire missiles.

Certain Air Force units used the Global Hawk unmanned aircraft’s sensor payloads to identify 55 percent of the time-critical targets to defeat enemy air defenses in Iraq in March and April 2003. To enhance joint operations, the Air Force developed procedures and tactics to allow the Global Hawk’s sensor payloads to provide more direct support to ground force missions.

In 2004, an Army force used its Hunter unmanned aircraft and sensor payload to locate an enemy antiaircraft artillery weapon that had been firing at coalition force aircraft. Then the Air Force sent a Predator armed with a Hellfire missile to attack the enemy weapon. Within minutes of the Predator strike, the Army unit sent its Hunter back to transmit information needed for battle damage assessment.

In 2004, an Army force operating an I-Gnat unmanned aircraft in Iraq detected a potential ambush of Marine Corps forces and the Army unit used information from the I-Gnat’s sensor payload to successfully adjust mortar fire onto the enemy position.

Recently, the Air Force, Army and Marine Corps forces have used their unmanned aircraft and their sensor and communications payloads to locate numerous targets in Iraq and Afghanistan to permit U.S. forces to destroy the targets.

UAS Interoperability Remains a Challenge

While achieving certain successes with the use of unmanned aircraft and their payloads, certain interoperability challenges have also emerged during recent operations despite certain DOD directives requiring interoperability and the emphasis on interoperability in the 2002 Roadmap. First, DOD Directive 5000.1 specifies that systems, units, and forces shall be able to provide and accept data and information to and from other systems and shall effectively interoperate with other U.S. forces. Second, the Roadmap specifies five data standards for formatting data, a communication standard to ensure adoption of a common data link, and a variety of file transfer, physical media, and other standards applicable to unmanned aircraft or their sensor and communications payloads. However, the 2005 edition of the Roadmap indicates that the detailed standards for interoperability have not been developed. In effect, the absence of such standards has led to the development of UAS that are not interoperable. In operations in Afghanistan and Iraq, interoperability problems have emerged. Specifically, during operations, DOD has learned that unmanned aircraft sensor and communications payloads and ground stations were not designed to common data standards and thus are not interoperable, even within a single service in certain circumstances. For example: Army forces operate both the Shadow and Hunter unmanned aircraft and associated ground stations but discovered that these systems are not interoperable. Specifically, while the Shadow’s sensor and communications payload is able to transmit information to its own ground station, it is unable to transmit to a Hunter ground station. Similarly, the Hunter’s sensor and communication payloads are able to transmit to a Hunter ground station but not Shadow’s. Onward transmission to forces needing the information is equally constrained if they do not have compatible equipment for receiving the information. As a result, the Army has missed an opportunity to effectively leverage the technology inherent in either system for the benefit of operational forces that need the information. At the time of our review, the Army had begun an initiative to make the Shadow and Hunter unmanned aircraft ground stations compatible with either aircraft.

When communication systems are incompatible, operating forces may be prompted to operate their own UAS, thus increasing the numbers of systems operating in the same area. To permit the sharing of tactical intelligence obtained by unmanned aircraft sensor payloads, the services or combatant commands have developed certain technical patches permitting compatibility but slowing data transmission. As we pointed out in 2003, in some cases, DOD needs hours or days to transmit information to multiple services. However, slow intelligence data transmission can undermine U.S. forces’ ability to attack time-critical targets or allow the targets to escape. U.S. Central Command acknowledges that timely data dissemination is critical to combat operations.

Communications interoperability problems are a long-standing problem. In 2001, we reported that each of the military services plans, acquires, and operates systems to meet its own operational concepts but not necessarily the requirements of joint operations in spite of the DOD directive requiring interoperability. In our 2004 unmanned aerial vehicle report, we reported that the services engaged in little coordination in developing their unmanned aerial vehicle roadmaps and that they did not view the UAS Planning Task Force’s 2002-2027 Roadmap as a strategic plan or an overarching architecture for integrating UAS into the force structure. In the absence of adequately developed and implemented standards and in contravention of the DOD guidance, the services have continued to develop their unmanned systems to their own standards, but without regard to the others’ standards. At the same time, DOD continues to develop and field UAS without adjusting the standards, likely causing the problem to become even more widespread. Moreover, the UAS used in current operations were built before the Joint Capabilities Integration and Development System became fully operational and thus has had little impact on the problem. Consequently, the information collected cannot always be quickly transmitted to users needing it, undermining joint operations and potentially leading to future costly initiatives to modify existing unmanned aircraft, sensors and communications payloads, and ground stations to overcome interoperability problems.

In addition to communications interoperability problems, payload interoperability (commonly referred to as “payload commonality”) problems also exist. DOD has developed at least six different sensor payloads each able to collect different types of information. These sensor payloads are attached to an unmanned aircraft and flown over operational areas to observe activity of interest on the ground in a target area and to transmit observations to ground or air forces or other users as tactical intelligence. As an example, figure 1 displays a Predator unmanned aircraft with a sensor payload attached underneath.

However, many sensor payloads can be attached to only one type of unmanned aircraft because DOD has not adopted a payload commonality standard even though this problem was identified nearly 20 years ago. As a result, commanders may have to delay missions if the appropriate sensor is available but no unmanned aircraft is able to carry it. We discussed this problem in 1988 when we reported that DOD had not adequately emphasized payload commonality for unmanned aircraft and that Congress had stressed the need for DOD to consider payload commonality in 1985. The 2002 Roadmap acknowledged the need for sensor payload commonality where practical, but limited progress has been made.

In addition to the flexibility inherent in the communications standards, according to U.S. Central Command based on its experience in Persian Gulf operations, unmanned aircraft development has been service-centric and lacks an overarching employment doctrine to shape development to achieve aircraft and sensor interoperable communications and payload commonality. Furthermore, a Joint Forces Command official told us that combatant commanders can not take full advantage of the dissimilar unmanned aircraft or the sensor payload data produced due to the interoperability problems.

Congested Electromagnetic Spectrum Has Hampered UAS Operational Effectiveness

Unmanned aircraft and their sensor, armaments, and communications payloads depend on reliable access to the electromagnetic spectrum. However, the spectrum is increasingly constrained, potentially undermining joint operations by requiring delays in an unmanned aircraft flight or, if the problem worsens, cancellation. Unmanned aircraft operators use the electromagnetic spectrum to maintain contact with the aircraft to control its flight, fire its weapons if armed, and receive information collected by the sensor payloads. Certain spectrum frequencies are sometimes referred to as bands and the amount of the spectrum needed to permit transmission of information is referred to as bandwidth. DOD officials told us that more bandwidth is needed to transmit video and other information obtained by sensor payloads than to maintain flight control of the aircraft. Numerous weapons also use electromagnetic spectrum and share it with UAS but they can interfere with each other during operations if they operate on the same frequency at the same time.

The military services have experienced bandwidth capacity constraints, limiting both the number of UAS and other systems that can be effectively operated simultaneously and the amount of available data that can be transmitted from the unmanned aircraft communications payload. For example, insufficient bandwidth limits U.S. forces’ ability to download video and radar images via satellite from more than one aircraft at a time. As a result, data transmission and relay are delayed, undermining U.S. forces’ ability to engage time-critical targets and possibly permitting the target to escape, unless alternative information sources are available on a timely basis. Army officials informed us that data link limitations are due primarily to frequency congestion. Table 1 displays the bands used by 12 different unmanned aircraft or models of unmanned aircraft for flight control and sensor payload data transmission. As shown in the table, several UAS rely on the C-band for their data transmission capability, and only 2 of the 12 UAS can be reprogrammed to another band.

The 2002 Roadmap established a goal of modifying the Army’s Shadow UAV to permit it to operate a common tactical data link in Ku-band and not the more congested C-band. This goal had not been met at the time of our review and the Shadow unmanned aircraft still operated in C-band. Similarly, the 2002 Roadmap established a goal of moving the Air Force’s Predator unmanned aircraft video sensor payload from C-band to Ku-band for line of sight operations. However, the goal had not been met at the time of our report.

Moreover, the problem cannot be easily overcome without potentially costly modifications to existing systems because DOD has not required unmanned aircraft or sensor payloads to be reprogrammable from one band to another and therefore has not established such standards. As a result, most have been designed and built without the flexibility to operate in differing frequencies or bands to avoid congested frequencies, thus sometimes preventing timely information transmission or delaying their flight without interfering with or experiencing interference from other UAS or other weapons systems.

Inclement Weather Limits Some Unmanned Aircraft Operations

Unmanned aircraft are more likely to be grounded by inclement weather than manned aircraft due in part to their lighter weight. Dust storms, strong winds, rain, or icing prevent some unmanned aircraft from flying, thus denying U.S. forces critically needed information unless alternative data collection or offensive strike capabilities are available. Specifically, winds up to 80 miles per hour in Iraq and Afghanistan have reduced the availability of most unmanned aircraft and dust storms have undermined the use of some sensor payloads. Moreover, the 2002 Roadmap indicates that icing has been a primary factor in two accidents involving the Hunter unmanned aircraft and three crashes of the Predator unmanned aircraft. The Roadmap established a goal to incorporate all-weather capabilities into future UAS. However, little progress has been made because DOD has not adopted standards for all-weather capability to be considered in development, despite the Roadmap’s stating the goal. As a result, systems have been developed without it. At the same time, according to a UAS Planning Task Force official, developing unmanned aircraft with all- weather capabilities may result in some degradation in performance, such as a reduced flying range. At the time of our review, DOD had not determined whether all-weather capability was worth the trade-off of potentially degraded performance.

DOD Has Made Little Progress in Addressing the Challenges

While DOD has acknowledged the need to improve UAS interoperability and address bandwidth and weather constraints that undermine unmanned aircraft operations, little progress has been made. On the one hand, to begin to address the problems, DOD has taken a number of steps as listed below: In August 2005, DOD issued an updated version of its roadmap, entitled 2005 Unmanned Aircraft Systems Roadmap, to guide acquisition and interoperability. Among other things, the 2005 Roadmap establishes the goal of enhancing joint service collaboration as a means to improve joint operations.

At the time of our review, the Office of the Secretary of Defense was preparing an action plan to address a number of shortfalls including interoperability and other problems within U.S. Central Command’s area of responsibility, although the plan was limited to just this command and would not necessarily solve the problems that UAS might encounter elsewhere.

DOD plans to reemphasize the role that the Joint Capabilities Integration and Development System could play in all new UAS developments by trying to ensure that DOD develops systems to support joint operations, achieve commonality to the extent practical, and identify gaps in DOD’s ability to carry out its warfighting missions.

U.S. Joint Forces Command has developed certain initiatives to improve UAS interoperability by conducting experiments to demonstrate aircraft modifications and new concepts of operations, although such modifications can be costly.

In addition, on June 1, 2005, DOD’s Joint Requirements Oversight Council established a new Joint Unmanned Aerial Vehicle Center of Excellence and a Joint Unmanned Aerial Vehicle Overarching Integrated Process Team. The Joint Unmanned Aerial Vehicle Overarching Integrated Process Team has subsequently been renamed the Joint Unmanned Aircraft Systems Material Review Board. These joint forums will help the services manage development of new UAS or modifications to existing UAS, and they will help the services to develop new or revised concepts of operations for more effective use. At the same time, the UAS Planning Task Force will try to ensure that the services’ UAS acquisition programs are coordinated, and a Task Force representative is to be a member of the Joint Overarching Integrated Process Team. DOD views these changes as means to more effectively manage service UAS programs. While these changes appear to be steps in the right direction, it is too early for us to tell if they will solve the interoperability and other problems that we identified.

Furthermore, payload commonality, interoperability of communications and data transmission systems, and inclement weather flying capabilities that we identified as impacting recent operations, had been identified previously as problems already occurring or likely to occur. First, our 1988 unmanned aerial vehicle report indicated that DOD had not adequately emphasized payload commonality for these aircraft. Second, our 2001 report found interoperability problems due to the services’ continued practice of acquiring systems to support their own operations but not necessarily that of the other services. Third, DOD’s guidance requires interoperability but the detailed standards have not been developed. Lastly, the 2002 edition of the Roadmap identified the need to improve interoperability of communications systems for UAS and also identified inclement weather capability as a problem undermining UAS operations and established goals to address it. Despite all the emphasis, problems related to communications and payload interoperability, and all-weather capability problems remain. DOD acknowledges that it (1) did not foresee the rapid technological development experienced with unmanned aircraft, sensor or communications payloads, and ground stations; (2) has provided unmanned aircraft and payloads rapidly to deployed forces to meet forces’ demands for them; and (3) has not always adopted new or enforced existing standards that might have prevented or mitigated some of these problems. As a result, while DOD has issued a directive, instructions, guidance, and roadmaps, and established at least five different organizations to promote UAS interoperability and address other unmanned aircraft and payload developmental needs, no organization has or has exercised sufficient authority to enforce program direction, or ensure that the standards and guidance are in concurrence. As a result, the services continued to develop and field these systems without fully complying with the interoperability requirements stated in key guidance or addressing known payload commonality problems.

DOD’s Approach to Evaluating Joint UAS Performance on Operational Deployments Has Been Unsound

DOD’s approach to evaluating joint UAS performance on operational deployments is unsound because it has not implemented a systematic approach to evaluating such performance. Instead, DOD has relied on systems for evaluating performance that are not focused on joint operations and are nonroutine, and as a result the department has little assurance that the information that has been collected represents the key performance indicators needed to assess performance on joint operations.

DOD Lacks Performance Indicators to Assess Certain Joint UAS Performance

DOD has not implemented a systematic approach to evaluating joint UAS performance on operational deployments. As we previously noted in our 2004 report, the Government Performance and Results Act’s strategic planning framework specifies that results-oriented performance measures can be used to monitor progress toward agency goals and that such performance measurements should be developed and used to monitor progress. At the time of our report, DOD was only beginning to decide on key indicators of performance that would be used to assess unmanned aircraft, payload, and ground station performance on joint operations. To date, DOD has relied on service-specific information that addressed certain UAS performance. For example, some forces filed after-action reports and maintenance reports addressing UAS performance. While producing some useful information, these reports have not necessarily been specifically targeted to joint UAS operations, nor do they systematically identify key indicators for collection which could be used to develop joint operational performance baselines and permit performance measurement against the baseline. Thus, DOD has little assurance that the information that has been collected represents the key performance indicators needed to assess joint operations performance.

DOD officials told us that they have tried to keep pace with operating forces’ demands for more unmanned aircraft and their payloads, and therefore the services have deployed them while still under development within the DOD acquisition system. These deployments have often occurred before identification of key performance indicators that would need to be collected to be used to evaluate performance. In effect, the services have bought and deployed unmanned aircraft, sensor and communications payloads, and ground stations and tried to evaluate their effectiveness all at the same time. On the one hand, this has permitted DOD to provide operating forces with new capabilities represented by the aircraft and their payloads. On the other hand, it has also resulted in DOD and the services sometimes learning of joint performance problems based on reporting from actual operations only if after-action reports or other reporting mentioned the problem. Nonetheless, without appropriate performance measures and baselines against which to assess performance on joint operations, even anecdotal information can have limited utility because officials are less likely to be able to assess the magnitude of the problem, or even become aware of it if no reports identify it.

DOD has acknowledged the need to develop specific performance indicators for unmanned aircraft and their payloads on joint operations and had begun to develop them at the time of our report. First, the Army recently began an initiative to develop performance indicators and a baseline against which to assess performance. However, while this approach may produce useful information on which to assess the performance of Army-operated unmanned aircraft, payloads, and ground stations, it was not designed to address joint performance. The other three services had not started to develop specific performance indicators and baselines for unmanned aircraft at the time of our review. Second, in May 2005, DOD assigned U.S. Strategic Command responsibility for the development of joint performance indicators but the effort was just getting started at the time of our review.

DOD’s UAS Joint Performance Reporting Has Not Been Routine

In addition to anecdotal performance reporting, DOD has not established routine performance reporting mechanisms for UAS operations but instead has relied on sometimes short-duration study teams to gather relevant joint operational performance information. For example, in November 2004, DOD established a group known as a “Tiger Team” to identify opportunities for improving the joint operational effectiveness of UAS. However, this team was established on a temporary basis and had a limited mission to identify improvements only in the U.S. Central Command area of responsibility. The Tiger Team did identify a number of areas needing improvement. For example, it determined that forces in the region need Full Motion Video capability to provide images of actual events as they occur. The team also determined that a need exists to address electromagnetic spectrum limitations hampering UAS operations. However, the team identified the electromagnetic spectrum problem only after the UAS had been deployed and U.S. forces had tried to use them on operational missions. In addition, also in 2004, another DOD short- duration study team evaluated the operational performance of the Shadow unmanned aircraft. Lastly, the Army conducted a one-time comprehensive review of the effectiveness of its UAS in theaters of operations. While these teams developed useful performance information, the approach does not represent a systematic or long-term means to obtain joint UAS performance information since the teams are not permanently established and they did not use consistent study parameters.

Finally, even in the instances where some ongoing processes were used, the information obtained was relevant only on a service-specific but not a joint basis. For example, the Marine Corps uses its Operational Advisory Group process to determine needed improvements in its UAS operations. While this group has developed useful information that may assist the Marine Corps in enhancing its ability to effectively use UAS in operations, the information developed is likely to have limited utility for joint operations.

DOD acknowledges that the speed with which unmanned aircraft, payloads, communications, and associated technology are being developed, along with the imperative to provide emerging technologies quickly to operating forces, have resulted in the deployment of developmental systems before adequate performance reporting systems have been established. Consequently, while the systems are being successfully used in overseas operations, DOD does not have reasonable assurance that it is well informed on opportunities to further enhance the ability of operational forces to take advantage of UAS capabilities.

Conclusions

DOD has achieved certain operational successes with UAS but certain challenges have also emerged that have hampered joint operations or prevented effective employment of UAS. These challenges are caused by the limited attention paid to interoperability standards for UAS and the lack of detailed interoperability standards. Development and implementation of appropriate interoperability, payload commonality, and other standards help to ensure that such problems are addressed during development and any problems are fixed prior to deployment. Moreover, until DOD assesses the extent to which a lack of detailed standards undermines the purpose of the broad standards by allowing development of noninteroperable systems and enforces common standards among the services, problems are likely to continue and possibly be repeated and made more widespread as new unmanned aircraft, sensor and communication payloads, ground stations, and related equipment are developed and fielded. In addition, costly modifications might be needed later.

The unsoundness in the approach DOD has taken to assessing joint UAS performance in operational deployments was due to a lack of accepted performance indicators and a routine system for collecting performance information. Until DOD develops specific indicators of UAS joint operational performance, establishes appropriate baselines against which to measure performance, and communicates which indicators operating forces should systematically collect and report to appropriate users, DOD will lack reasonable assurance that it is adequately informed on UAS performance on joint operations. Moreover, DOD may also be poorly informed as to its progress in addressing interoperability and other problems and may therefore be less likely to avoid the same problems in future UAS development and fielding.

Lastly, in our 2004 report, we recommended that DOD establish a strategic plan and an office with sufficient authority to enforce program direction to avoid interoperability problems and for other purposes. In nonconcurring with our recommendation to assign an office with sufficient authority to enforce program direction, DOD indicated that the UAS Planning Task Force and Joint Capabilities Integration and Development System had sufficient authority and would address interoperability, payload commonality, and other problems. However, these problems persist. Consequently, we continue to believe that sustained management attention is warranted. Without such attention, DOD continues to risk undercutting the benefit of its continued investment in UAS. Consequently, we continue to believe that our prior recommendation has merit, but we are not reiterating it because DOD indicated that it will not implement it.

Recommendations for Executive Action

To address the challenges emerging in joint operations, we recommend that the Secretary of Defense direct the Undersecretary of Defense (Acquisition, Technology, and Logistics), the Chairman of the Joint Chiefs of Staff, the service secretaries, and other appropriate organizations to work together to take the following four actions develop or adjust communications interoperability standards and electromagnetic frequency reprogramming capabilities standards and ensure that they are applied to new or modified unmanned aircraft, sensor and communications payloads, ground stations, and related equipment; develop sensor and other payload commonality standards where practical and enforce such standards when modifying existing unmanned aircraft or payloads and developing new ones; develop appropriately detailed UAS interoperability standards; and determine whether unmanned aircraft need all-weather flying capabilities, identify any performance degradation associated with all- weather flying capabilities, and obtain all-weather capabilities where appropriate.

To improve joint operational performance reporting, we recommend that the Secretary of Defense direct the Commander of the U.S. Strategic Command to ensure that the performance measurement system being developed by the command at a minimum measures how effectively UAS perform their missions by identifying quantifiable goals and comparing results with desired outcomes; identifies the specific performance indicator information that needs to be collected to adequately assess joint performance; develops indicators that assess communications and payload interoperability, and the extent to which electromagnetic spectrum congestion is undermining joint operations; establishes baselines and applies the identified indicators against the baselines to gauge success in joint UAS performance; and develops a way to systematically collect identified performance information and routinely reports it to organizations that develop and field UAS.

Agency Comments and Our Evaluation

DOD provided written comments on a draft of this report. These comments are reprinted in their entirety in appendix II. We made five recommendations and DOD fully or partially concurred with them. It also provided technical comments, which we incorporated into our report as appropriate.

First, DOD concurred with our recommendation for the appropriate DOD organizations to work together to develop or adjust communications interoperability standards and electromagnetic frequency reprogramming capabilities standards and ensure that they are applied to new or modified unmanned aircraft, sensor and communications payloads, ground stations, and related equipment. In concurring, DOD indicated that it recognized the utility of communications interoperability and the need to improve this capability and will direct the services to use common frequencies and data links to enhance communications interoperability.

Second, in partially concurring with our recommendation to develop and enforce sensor and other payload commonality standards where practical, DOD commented that it does not typically focus on payload interchangeability. Instead, DOD pointed out that unmanned aircraft payload procurement is a service responsibility and is dependent on service mission requirements, unmanned aircraft physical design limitations, and rapid technological evolution. Our report recognizes that it is not practical for all unmanned aircraft sensors and payloads to be common due to the various sizes of some aircraft and we worded our recommendation accordingly.

Third, DOD fully concurred with our recommendation that the appropriate DOD organizations work together to develop appropriately detailed UAS interoperability standards. DOD indicated that the UAS Roadmap 2005- 2030 released in August 2005 discusses the preferred framework, methodology, and standards for achieving UAS interoperability. DOD outlined a number of actions that it has taken to address UAS interoperability standards, including ratifying a North Atlantic Treaty Organization Standards Agreement aimed at achieving joint and combined interoperability. The Joint Chiefs of Staff has tasked the newly formed Joint UAS Material Review Board and Joint UAV Center of Excellence to provide recommendations for continuing to improve UAS interoperability.

Fourth, DOD fully concurred with our recommendation to determine whether unmanned aircraft need all-weather flying capabilities, identify any performance degradation associated with all-weather capabilities, and obtain all-weather capabilities where appropriate. DOD commented that combatant commanders should expect UAS to support operations in diverse weather conditions. Further, DOD indicated that as UAS capabilities improve, the weather conditions these systems will need to operate in will also increase. However, DOD also points out that it is not cost effective to expect all classes of unmanned aircraft to have an all- weather capability. We agree. The intention of our recommendation is for DOD to determine those UAS for which all-weather capabilities are cost effective and to add such capabilities when appropriate.

Finally, DOD partially concurred with our recommendation that U.S. Strategic Command ensure that the performance measurement system being developed at a minimum includes quantifiable goals, performance baselines, systematic collection procedures, measures of communications and payload interoperability, and performance indicators against which to measure performance. DOD indicated that the U.S. Strategic Command has drafted a Joint Functional Component Concept of Operations that includes metrics to gauge the force’s ability to meet intelligence, surveillance, and reconnaissance requirements. Moreover, DOD stated that in conjunction with the services, intelligence community, combatant commanders, and other DOD organizations, this action would facilitate not only the evaluation of UAS performance but would enable DOD to have the necessary information available to assess such factors as UAS requirements, mission accomplishment, UAS capabilities, and customer satisfaction. DOD also pointed out that the performance measures are in development and will require service participation to define the specific data and methodology which will result in useful information. While we acknowledge that these actions should address many of the data elements that we believe are necessary to evaluate UAS, we continue to believe that effective communications, interoperability, and avoidance of frequency congestion are important contributors to the success of joint operations. Therefore, we continue to believe that DOD should ensure that, at a minimum, the U.S. Strategic Command includes the data elements we recommended in its performance measurement system. In addition, we agree that other organizations including the services, should participate in the development of this measurement system if appropriate.

We are sending copies of this report to other appropriate congressional committees, the Secretary of Defense, the secretaries of the Army, the Navy, and the Air Force; the Commandant of the Marine Corps; the Chairman of the Joint Chiefs of Staff; and the Director, Office of Management and Budget. We will also make copies available to other interested parties upon request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staff have any questions about this report, please call me at (202) 512-9619 or email at pickups@gao.gov. Contact points for our Office of Congressional Relations and Public Affairs may be found on the last page of this report. The GAO contact and key contributors are listed in appendix III.

Scope and Methodology

To evaluate the operational performance of unmanned aircraft systems (UAS) in recent operations, we examined the Department of Defense (DOD) regulations, directives, and instructions as well as service guidance and documentation on UAS. We met with key DOD and service officials, including those from the UAS Planning Task Force and UAS program managers, to discuss the current status and future plans for these systems. We reviewed the Unmanned Aerial Vehicles Roadmap 2002-2027 because this document establishes an overall DOD management framework for developing and employing UAS DOD-wide and the update, 2005 Unmanned Aircraft Systems Roadmap. During discussion and visits with DOD and service officials, we obtained and reviewed DOD and service analyses, briefings, and summary reports describing each of the UAS used in supporting recent combat and combat support operations. This included obtaining detailed information on current and future UAS operational capabilities. Additionally, we obtained information on the numbers and types of missions performed by UAS, as well as the methods used by the services to evaluate UAS performance in accomplishing those missions. To assess the reliability and types of missions provided to us by DOD, we (1) interviewed knowledgeable officials about the processes for collecting and maintaining the data and (2) reviewed the data for completeness and reasonableness by comparing it to other sources of information. We determined that the data were sufficiently reliable for the purposes of this review. DOD and service officials also provided specific examples of operational successes and emerging challenges. We discussed actions taken and processes used by DOD and service officials and the Joint Capabilities Integration and Development System to address identified challenges. We also held discussions with Joint Staff officials to discuss their efforts to address joint UAS issues via the Tiger Team.

The specific military activities that we visited and/or obtained written responses to questions from include the following: Office of the Undersecretary of Defense (Acquisition, Technology, and Logistics) and its Joint UAS Planning Task Force; Washington, D.C.; Headquarters, Department of the Army; Washington, D.C.; U.S. Army Redstone Arsenal, Huntsville, Alabama; U.S. Marine Corps, Systems Command, Quantico, Virginia; U.S. Navy Naval Sea Systems Command, Naval Air Station Patuxent U.S. Air Force Air Combat Command Directorate of Requirements, Langley Air Force Base, Virginia; U.S Air Force, Air Force Material Command, Wright Patterson Air Force Base Dayton, Ohio; U.S. Joint Forces Command, Norfolk, Virginia; U.S. Central Command, MacDill Air Force Base, Tampa, Florida; U.S. Special Operations Command, MacDill Air Force Base, Tampa, U.S. Joint Staff, Washington, D.C., and U.S Strategic Command, Omaha, Nebraska.

We also obtained documents describing the mission and planned operations of the new Joint Unmanned Aerial Vehicle Center of Excellence and Joint Unmanned Aerial Vehicle Overarching Integrated Process Team.

To assess the soundness of DOD’s approach to evaluating UAS operational performance, we interviewed DOD and service officials to discuss the criteria and processes used to assess performance. We also obtained and reviewed DOD and Army UAS Operations Assessments to identify issues and concerns regarding performance. Additionally, we held discussions with U.S. Strategic Command officials to obtain information on the status of their efforts to establish measures for assessing joint UAS performance. We also held discussions with service officials to determine the extent to which they are required to capture information on the use and performance of UAS in their existing lessons-learned systems. Finally, we obtained and reviewed DOD and service specific UAS or unmanned aerial vehicle roadmaps.

We performed our work from July 2004 to October 2005 in accordance with generally accepted government auditing standards.

Comments from the Department of Defense

GAO Contacts and Staff Acknowledgments

GAO Contact

Acknowledgments

In addition to the person named above, Brian J. Lepore, Assistant Director; Harry E. Taylor, Jr.; Patricia F. Albritton; Jeanett H. Reid; Elisha T. Matvay; Robert B. Brown; Cheryl A. Weissman; Ron La Due Lake; and Kenneth E. Patton also made major contributions to this report.

Related GAO Products

Unmanned Aerial Vehicles: Improved Strategic and Acquisition Planning Can Help Address Emerging Challenges. GAO-05-395T. Washington, D.C.: March 9, 2005.

Unmanned Aerial Vehicles: Changes in Global Hawk’s Acquisition Strategy Are Needed to Reduce Program Risks. GAO-05-6. Washington, D.C.: November 5, 2004.

Unmanned Aerial Vehicles: Major Management Issues Facing DOD’s Development and Fielding Efforts. GAO-04-530T. Washington, D.C.: March 17, 2004.

Force Structure: Improved Strategic Planning Can Enhance DOD’s Unmanned Aerial Vehicles Efforts. GAO-04-342. Washington, D.C.: March 17, 2004.

Nonproliferation: Improvements Needed for Controls on Exports of Cruise Missile and Unmanned Aerial Vehicles. GAO-04-493T. Washington, D.C.: March 9, 2004.

Nonproliferation: Improvements Needed to Better Control Technology Exports for Cruise Missiles and Unmanned Aerial Vehicles. GAO-04-175. Washington, D.C.: January 23, 2004.

Defense Acquisitions: Matching Resources with Requirements Is Key to the Unmanned Combat Air Vehicle Program’s Success. GAO-03-598. Washington, D.C.: June 30, 2003.

Unmanned Aerial Vehicles: Questionable Basis for Revisions to Shadow 200 Acquisition Strategy. GAO/NSIAD-00-204. Washington, D.C.: September 26, 2000.

Unmanned Aerial Vehicles: Progress of the Global Hawk Advanced Concept Technology Demonstration. GAO/NSIAD-00-78. Washington, D.C.: April 25, 2000.

Unmanned Aerial Vehicles: DOD’s Demonstration Approach Has Improved Project Outcomes. GAO/NSIAD-99-33. Washington, D.C.: August 30, 1999.

Unmanned Aerial Vehicles: Progress toward Meeting High Altitude Endurance Aircraft Price Goals. GAO/NSIAD-99-29. Washington, D.C.: December 15, 1998.

Unmanned Aerial Vehicles: Outrider Demonstrations Will Be Inadequate to Justify Further Production. GAO/NSIAD-97-153. Washington, D.C.: September 23, 1997.

Unmanned Aerial Vehicles: DOD’s Acquisition Efforts. GAO/T-NSIAD-97- 138. Washington, D.C.: April 9, 1997.

Unmanned Aerial Vehicles: Hunter System Is Not Appropriate for Navy Fleet Use. GAO/NSIAD-96-2. Washington, D.C.: December 1, 1995.

Unmanned Aerial Vehicles: Performance of Short-Range System Still in Question. GAO/NSIAD-94-65. Washington, D.C.: December 15, 1993.

Unmanned Aerial Vehicles: More Testing Needed Before Production of Short-Range System. GAO/NSIAD-92-311. Washington, D.C.: September 4, 1992.

Unmanned Aerial Vehicles: Medium Range System Components Do Not Fit. GAO/NSIAD-91-2. Washington, D.C.: March 25, 1991.

Unmanned Aerial Vehicles: Realistic Testing Needed Before Production of Short-Range System. GAO/NSIAD-90-234. Washington, D.C.: September 28, 1990.

Unmanned Vehicles: Assessment of DOD’s Unmanned Aerial Vehicle Master Plan. GAO/NSIAD-89-41BR. Washington, D.C.: December 9, 1988.