Report to Congressional Committees

United States General Accounting Office

GAO

August 2002 MILITARY

TRANSFORMATION

Actions Needed to

Better Manage DOD’s

Joint Experimentation

Program

GAO-02-856
Page i GAO-02-856 Military Transformation

Letter 1

Results in Brief 2

Background 4

Participation in Joint Experimentation Continues to Evolve 7

No Recommendations for Change Have Been Approved or

Implemented 16

Some Key Management Elements Are Missing 19

Conclusions 24

Recommendations for Executive Action 25

Agency Comments and Our Evaluation 26

Scope and Methodology 26

Appendix I Chronology of Key Events Important to Joint

Experimentation 29

Appendix II Additional Information on Joint Concept

Development and Experimentation Activities 31

Appendix III Comments from the Department of Defense 33

Appendix IV Organizations and Offices Contacted 35

Appendix V GAO Contacts and Staff Acknowledgments 38

Tables

Table 1: Organizations Involved in Joint Experimentation 8

Table 2: First Three Recommendations Submitted by Joint Forces

Command 17

Table 3: Types of Activities and Experiments Associated with Joint

Experimentation 32

Contents
Page ii GAO-02-856 Military Transformation

Figures

Figure 1: Joint Concept Development and Experimentation Process 6

Figure 2: Involvement of Organizations in Preparing Joint

Experimentation Guidance and Campaign Plan 12

Figure 3: Joint Concept Development and Experimentation

Activities 31
Page 1 GAO-02-856 Military Transformation

August 29, 2002

The Honorable Carl Levin

Chairman

The Honorable John W. Warner

Ranking Minority Member

Committee on Armed Services

United States Senate

The Honorable Bob Stump

Chairman

The Honorable Ike Skelton

Ranking Minority Member

Committee on Armed Services

House of Representatives

The Department of Defense (DOD) considers the transformation1

 of the

U. S. military a strategic imperative to meet the security challenges of the

new century. In October 1998, DOD established a joint concept

development and experimentation program to provide the engine of

change for this transformation. The U.S. Joint Forces Command, the

executive agent of the program, carries out experiments—with support

from the military services, the combatant commands, and other DOD

organizations—to explore and analyze new joint operational and

organizational concepts. The experiments can include studies and

analyses, conferences, workshops, seminars, war games, and simulated or

live field experiments.

On the basis of these experiments, the Joint Forces Command

recommends changes aimed at achieving significant improvements in joint

 1

 DOD states that transformation of the military “results from the exploitation of new

approaches to operational concepts and capabilities, the use of old and new technologies,

and new forms of organization that more effectively anticipate new or still emerging

strategic and operational challenges and opportunities and that render previous methods of

conducting war obsolete or subordinate.”

United States General Accounting Office

Washington, DC 20548
Page 2 GAO-02-856 Military Transformation

capabilities. The Joint Requirements Oversight Council2 reviews the

Command’s recommendations, and the Chairman of the Joint Chiefs of

Staff is responsible for approving them. The Council also oversees the

implementation of approved recommendations. The cumulative changes

are expected to contribute to the way U.S. military forces are transformed.

In November 2001, DOD also created an Office of Force Transformation to

foster innovation and experimentation and assist the Secretary of Defense

to develop guidance and identify strategies and policies for

transformation.

Because of the central role DOD’s joint experimentation program plays in

military transformation, we reviewed related activities to determine

whether changes are needed to improve the program’s ability to focus on

future joint operations, military transformation, and the rapid integration

of results. Specifically, we assessed the extent to which (1) the Joint

Forces Command obtains participation from all relevant sources in

developing and conducting its experimentation activities;

(2) the recommendations flowing from the joint experimentation process

have been approved and implemented; and (3) the necessary management

elements are in place to foster program success. We are providing this

report to you because of your oversight responsibilities and interest in

military transformation issues.

In the nearly 4 years since becoming the executive agent for joint concept

development and experimentation, the Joint Forces Command has

increased the participation of key DOD stakeholders—the military

services, the combatant commands, and other organizations and

agencies—in its experimentation activities. The Command has also

expanded the participation of federal agencies and departments,

academia, the private sector, and some foreign allies. The Command uses

many different ways to obtain input—including meetings, seminars,

workshops, video conferencing, E-mail, and the Internet—and integrates

the results of military operations, technology efforts, and other DOD

organizations’ experiments into its activities. Although stakeholders

 2

 The Joint Requirements Oversight Council is an advisory council that assists the

Chairman of the Joint Chiefs of Staff in identifying and assessing the priorities for joint

military requirements to achieve current and future military capabilities. Chaired by the

Vice Chairman of the Joint Chiefs of Staff, the Council is comprised of a senior officer from

each of the military services. Representatives from other DOD entities, such as the

combatant commands and the Joint Staff, serve in an advisory role to the Council.

Results in Brief
Page 3 GAO-02-856 Military Transformation

determine the extent of their involvement in joint experimentation

activities, many more organizations are in fact involved in the planning

and execution of this year’s major field experiment than there were in a

similar experiment 2 years ago. Most DOD officials whom we spoke with

were generally satisfied with the way the Command reached out to obtain

their input, but many also believe more frequent contacts with the Joint

Forces Command would improve communications. Some believe that the

Command could do more to expand the participation of non-DOD

agencies and departments and allies and coalition partners, and the

Command is planning initiatives to do so, though this outreach would

involve broader agency-resource and national-security considerations.

No recommendations flowing from joint experimentation have been

approved or implemented. Although the Joint Forces Command issued

three recommendations nearly a year ago, they were not approved by the

Joint Requirements Oversight Council because of confusion among the

Joint Staff and the Joint Forces Command about a proposed change in

guidance that required additional data (on costs and timelines) be

included when submitting these recommendations. As a result, it is not

clear when these recommendations will contribute to military

transformation. The Command plans to re-submit the recommendations

with the required data later this year. However, at the time we concluded

our review, the official guidance on what information should accompany

joint experimentation recommendations had not been approved and

issued. In addition, although it is too early to assess how DOD integrates

the results of its experimentation into its plans, programs, and budgets,

several DOD officials expressed concern that the resource allocation

process may be too slow to provide rapid and timely funding for the

implementation of new concepts emerging from joint experimentation and

that other, more rapid resource mechanisms may be needed.

While DOD has been providing more specific and clearer guidance for

joint experimentation, DOD and the Joint Forces Command are missing

some key management elements that are generally considered necessary

for successful program management. DOD and the Joint Forces Command

thus lack the following very important management tools needed to carry

out their joint experimentation responsibilities:

• It is not yet clear what role the Office of Force Transformation will have in

joint experimentation and how it will coordinate its activities with other

DOD stakeholders. As a result, the Secretary of Defense may not be

effectively using this new office to interact with stakeholders to foster and
Page 4 GAO-02-856 Military Transformation

monitor joint experimentation as a means to advance DOD’s

transformation agenda.

• DOD has not developed meaningful performance measures to assess joint

experimentation’s contribution to the advancement of military

transformation. Therefore, DOD cannot gauge the progress of joint

experimentation efforts and of the program’s overall contribution to

transformation.

• The Joint Forces Command does not have the strategic-planning tools

(a strategic plan with associated performance plans and reports) that

would allow it to clearly define what it is seeking to accomplish, identify

what strategy it will use to achieve desired results, and finally measure

how well it is succeeding in meeting results-oriented goals and objectives.

We are making recommendations designed to improve the processes and

the management oversight and accountability for the joint concept

development and experimentation program and to clarify the role of the

Office of Force Transformation. In its comments on a draft of this report,

DOD agreed with our recommendations and stated that our findings

provide a factual summary of the history and progress of joint

experimentation and identify the critical weaknesses currently being

addressed by DOD. It expects that a forthcoming Transformation

Planning Guidance or subsequent guidance documents will address the

issues and problems with the joint experimentation program identified in

our report.

The Joint Forces Command, in coordination with the Joint Staff,3

 the

services, and other combatant commands and DOD agencies, is

responsible for creating and exploring new joint war-fighting concepts, as

well as for planning, designing, conducting, and assessing a program of

joint experimentation. The Command executed its second large-scale field

experiment, Millennium Challenge 2002,4

 this year, and it plans another

one in 2004 and others every third year thereafter. These experiments are

intended to examine how well the concepts previously explored by the

 3

 The staff that assists the Chairman of the Joint Chiefs of Staff in executing the Chairman’s

responsibilities.

4

 Millennium Challenge 2002 focuses on how the U.S. military can use its current and

near-term capabilities differently in this decade. It will involve the U.S. Special Operations

Command and forces from all the services. Another major field experiment—Olympic

Challenge—is planned for 2004.

Background
Page 5 GAO-02-856 Military Transformation

Command in smaller venues will work when applied with the emerging

concepts being developed by the services and other combatant

commands. For example, Millennium Challenge 2002 tested how well

U.S. forces fared against a regional power with a sizable conventional

military force and so called “anti-access” capabilities—which can include

advanced surface-to-air missiles, antiship missiles and mines, and

chemical and biological weapons—and validated the results of earlier

experiments to develop the Command’s “rapid decisive” operations

concept. The aim of the experiment was to come up with changes that can

be made during the current decade. (App. I provides a chronology of major

events important to joint experimentation.)

Over the next several years, the Command’s experimentation will focus

primarily on two concepts: one to develop a standing joint force

headquarters to improve joint command and control; another to conduct

more effective joint operations through “rapid decisive” operations.

In November 2001, the Chairman of the Joint Chiefs of Staff5

 directed that

the Command make development of the prototype headquarters its highest

near-term priority.6

 Additionally, the Command will develop a number of

other concepts aimed at specialized issues or operational problems to

support the two primary concepts.

Joint experimentation is a continuous process that begins with the

development of new operational and organizational concepts that have the

potential to improve significantly joint operations (see fig. 1). The Joint

Forces Command identifies new joint concepts including those developed

by other DOD organizations (such as the Joint Staff, services, and

combatant commands) and the private sector and tests them in

experiments that range from simple (workshops, seminars, war games,

and simulations) to complex (large-scale virtual simulations and “live”

field experiments). Appendix II provides additional information on joint

experimentation program activities.

 5

 The Unified Command Plan, April 30, 2002, assigns the Chairman of the Joint Chiefs of

Staff responsibility for providing overarching guidance on joint experimentation.

6

 The Command is directed to develop this prototype headquarters by 2004 and make it

available for implementation by the regional combatant commands during fiscal year 2005.

The headquarters will provide uniform, standard operating procedures, tactics, techniques,

and technical system requirements, with the ability to move expertise among commands.
Page 6 GAO-02-856 Military Transformation

Figure 1: Joint Concept Development and Experimentation Process

Source: Joint Forces Command.

After analyzing experimentation data, the Command prepares and submits

recommendations to the Joint Requirements Oversight Council for review

and, ultimately, to the Chairman of the Joint Chiefs of Staff for approval.

7

Before submitting them to the Council, however, the Command submits its

recommendations to the Joint Staff for preliminary review and

coordination. The recommendations are distributed for review and

comment to the Joint Staff directorates, the military services, the

combatant commands, and other DOD and federal government

organizations. The Council then reviews the recommendations and advises

the Chairman of the Joint Chiefs of Staff on whether they should be

approved. The changes, if approved, provide the basis for pursuing the

capabilities needed to implement a specific operational concept.

The Council is also responsible for overseeing the implementation of the

recommendations, but it can designate an executive agent, such as the

Joint Forces Command, to do so. The Council (or its designated executive

 7

 We recently reported on the status of actions taken by the Chairman of the Joint Chiefs of

Staff to improve the Joint Requirements Oversight Council’s oversight and the joint

requirements process, including establishing procedures for submitting recommendations

stemming from joint experimentation to the Council. See U.S. General Accounting Office,

Defense Plans: Status of DOD’s Efforts to Improve Its Joint Warfighting Requirements

Process, GAO-02-100R (Washington, D.C.: Nov. 9, 2001).
Page 7 GAO-02-856 Military Transformation

agent) is responsible for obtaining the resources needed to implement the

recommendations through DOD’s Planning, Programming, and Budgeting

System.8

 The Council also assists the Chairman, in coordination with the

combatant commands, the services, and other DOD organizations, to

identify and assess joint requirements and priorities for current and future

military capabilities. The Council considers requirements (and any

proposed changes) for joint capabilities across doctrine, organizations,

training, materiel, leadership and education, personnel, and facilities.

The Department of the Navy’s budget9

 provides funding to the Joint Forces

Command for joint experimentation and other Command missions.

In fiscal year 2002, the Command received from the Navy about

$103 million for its joint concept development and experimentation

program, and it planned to spend about half of this amount for Millennium

Challenge 2002. The Command has requested that the Navy provide about

$98 million for the program in fiscal year 2003. The Command also

provides some funds to the services, the combatant commands, and other

DOD organizations for efforts that support its program activities.

However, the services fund the operations and support costs of forces

participating in joint experimentation. Also, the individual

experimentation efforts of the services and the combatant commands are

funded from within their own budgets.

Since it first began joint experimentation, the Joint Forces Command has

broadened and deepened the inclusion of other DOD organizations,

federal agencies and departments, the private sector, and allies and

coalition partners in its process for capturing and identifying new joint

ideas and innovations. Organizations participating in joint experimentation

are generally satisfied with current opportunities for their ideas to be

considered, and many have increased their participation in the program.

However, the participation of different stakeholders—the extent of which

 8

 The Planning, Programming, and Budgeting System is designed to assist the Secretary of

Defense in making choices about the allocation of resources among a number of competing

or possible programs and alternatives to accomplish specific national security objectives.

The objective of the system is to provide the optimal mix of forces, equipment, and support

that can be achieved within fiscal constraints.

9

 DOD Directive 5100.3, Support of the Headquarters of Combatant and Subordinate Joint

Commands, Nov. 15, 1999, designates the Navy as the executive agent responsible for

arranging for the program-funding of the Joint Forces Command’s activities, including the

joint experimentation program.

Participation in Joint

Experimentation

Continues to Evolve
Page 8 GAO-02-856 Military Transformation

is determined by the stakeholder—varies considerably and some would

like more visits and contacts with the Command. The Command is

planning initiatives to increase stakeholder participation in the future,

particularly for federal agencies and departments and key allies, but this

increase will depend on agency-resource and national-security

considerations.

As the program gradually evolved, the Joint Forces Command solidified a

process to involve the military services, the combatant commands, and

other DOD organizations in the planning and execution of its joint

experimentation activities. Because future joint operations will involve

diplomatic, information, and economic actions, as well as military

operations, many DOD, federal, and private organizations and

governments participate and provide input into the joint experimentation

program (see table 1). The Joint Forces Command functions as a

facilitator to solicit and coordinate the involvement of these organizations

and incorporate their input, as appropriate, into concept development and

experimentation activities.

Table 1: Organizations Involved in Joint Experimentation

Organizational area

Examples of organizations and government

entities

DOD headquarters and support

organizations

• Office of the Secretary of Defense

• Chairman of the Joint Chiefs of Staff and Joint Staff

• Office of Force Transformation

• Joint Advanced Warfighting Program

• Defense Intelligence Agency

• National Defense University

Military services • Army

• Navy

• Air Force

• Marine Corps

Combatant commands • U.S. Central Command

• U.S. European Command

• U.S. Pacific Command

• U.S. Southern Command

• U.S. Special Operations Command

• U.S. Transportation Command

• U.S. Space Command

• U.S. Strategic Command

Federal agencies and

departments

• Department of State

• Department of Justice

• Central Intelligence Agency

Private sector • Academia

• Think tank groups

• Industry

Participation Has Grown

and Continues to Expand
Page 9 GAO-02-856 Military Transformation

Organizational area

Examples of organizations and government

entities

Allies and potential coalition

partners

• Some North Atlantic Treaty Organization countries

• Australia

Source: GAO interviews and DOD data.

Because the stakeholders determine the extent of their participation in the

program, it can vary considerably. However, Joint Forces Command

officials stated that participation by the services, the combatant

commands, and other DOD organizations has grown steadily since the

program was created and continues to grow, as participants become

increasingly aware of the strong emphasis that DOD leaders are placing on

experimentation. For example, in contrast to the first field experiment

in 2000, which had limited involvement by the services, this year’s

Millennium Challenge has seen the services more actively involved in early

planning, and their individual experiments better coordinated and

integrated into the field experiment. Our comparison of participation in

the Command’s major field experiment in 2000 with plans for this year’s

experiment found a significant increase in the diversity and number of

participating organizations and in the number of concepts and initiatives

proposed by these organizations. For example, the total number of

organizations participating in Millennium Challenge 2002 more than

doubled from the prior experiment in 2000 (from 12 to 29 organizations),

and the total number of service initiatives increased from 4 to 29.

The Command provides several ways for organizations to participate and

provide inputs: they can review program plans and strategies; attend

meetings, seminars, and workshops; take part in experimentation

activities; and use various communication tools such as E-mail, Internet,

and video conferencing. Additionally, the Command obtains input from the

various experimentation and research and development organizations of

the military services and of some combatant commands and DOD

organizations.10 The Command also considers the results of Advanced

 10 Participating organizations would include each of the services’ experimentation

programs, service battle labs and supporting research and development organizations,

DOD organizations such as Defense Advanced Research Projects Agency and Joint

Advanced Warfighting Program, and federally funded research and development

organizations such as the Institute for Defense Analyses, the RAND Corporation, and the

Center for Naval Analysis.
Page 10 GAO-02-856 Military Transformation

Concept Technology Demonstrations efforts,11 innovations, and recent

military operations in developing its program. For example, as a result of

its operational experiences in Kosovo, the U.S. European Command

identified various joint capability shortfalls in its recent list of Command

priorities as a means of guiding the Joint Forces Command in selecting

focal areas and activities for experimentation. Further, the Command is

taking steps to (1) align its experimentation activities with the schedules

of major service and combatant command exercises and (2) adjust its

program to allow for earlier consideration of new concepts proposed by

the services and the combatant commands in the input process. These

adjustments would improve synchronization of experiments with the

availability of forces and the training schedules of the services and the

combatant commands, allow for greater involvement of these entities in

the process, and increase the likelihood that joint requirements are

sufficiently considered early in the development of concepts.

Participating organizations also provide input during the annual

preparation of two key joint experimentation-program documents:

the Chairman of the Joint Chiefs of Staff’s guidance on joint

experimentation and the Joint Forces Command’s Joint Concept

Development and Experimentation Campaign Plan12 (see fig. 2). Each

year the Chairman provides guidance to the Joint Forces Command to use

in developing its Campaign Plan for joint concept development and

experimentation. The basis for the Chairman’s guidance is derived from

several sources, including strategy and planning documents, studies, and

other assessments.13 Additionally, key DOD stakeholders, including the

Chairman’s Joint Warfighting Capability Assessment teams14 and the Joint

 11 DOD initiated the Advanced Concept Technology Demonstration program to determine

the extent to which a given mature technology will improve military capabilities before

entering the normal acquisition process and, by using mature technology, reduce the length

of time to develop and acquire weapon systems.

12 The Campaign Plan provides the structure for executing experimentation and outlines

the Joint Forces Command’s objectives, methods, and events to achieve those objectives.

The most recent Campaign Plan is: U.S. Joint Forces Command, The Joint Concept

Development and Experimentation Campaign Plan Fiscal Years 2002-2007 (Norfolk,

Va.: 2002).

13 Examples of these sources include the President’s National Security Strategy, the

Secretary of Defense’s annual defense planning guidance, and the Chairman of the Joint

Chief of Staff’s Joint Vision and National Military Strategy.

14 Teams of DOD war-fighting and functional-area experts appraise various aspects of joint

war fighting within their respective areas. The Chairman of the Joint Chiefs of Staff uses

the teams’ assessments to develop guidance and to support decision making.
Page 11 GAO-02-856 Military Transformation

Requirements Oversight Council, provide input to the Joint Staff to use in

developing the Chairman’s guidance. The Joint Forces Command uses this

guidance, with additional input from DOD stakeholders, in preparing its

Campaign Plan, which is the primary vehicle for synchronizing its joint

experimentation activities and coordinating resources. The Command also

solicits and considers input for the Campaign Plan from some other

federal agencies and departments, academia, private sector, and allies.

After review and endorsement by the combatant commands, the services,

and the Joint Requirements Oversight Council, the Chairman approves the

Campaign Plan.
Page 12 GAO-02-856 Military Transformation

Figure 2: Involvement of Organizations in Preparing Joint Experimentation

Guidance and Campaign Plan

Source: Joint Staff and Joint Forces Command.

Officials at the military services, the combatant commands, and other

DOD organizations we talked with said they were generally satisfied with

the opportunities for input provided by the Joint Forces Command. At the

same time, DOD stakeholders have taken various actions to increase their

participation. Some, however, would like more contacts and

communication with the Command. The Command is responding with

some initiatives.

Stakeholders Generally

Satisfied but Point to Some

Possible Improvements
Page 13 GAO-02-856 Military Transformation

Each service, the Joint Staff, the U.S. Special Operations Command, the

U.S. Space Command, as well as some DOD and federal agencies (such as

the National Imagery and Mapping Agency and the National Security

Agency) have assigned liaison officers at the Joint Forces Command.15

However, officials at the Central, Pacific, and Southern Commands stated

that their staffing levels currently do not allow them to devote personnel

in this role. Combatant command officials indicated that the frequency and

number of meetings, conferences, and other events held at the Joint

Forces Command often make it difficult for their organizations to attend.

The officials believe that as a result, the views and positions of their

organizations are not always fully captured in some discussions and

deliberations. Some of the combatant commands have or are planning to

establish their own joint experimentation offices. Officials from the Pacific

and Special Operations Commands stated that although their respective

joint experimentation offices are largely focused on supporting their own

experimentation efforts, the offices provide a cadre of staff who can better

coordinate and participate more consistently in the Joint Forces

Command’s joint experimentation program. For example, Pacific

Command officials said that their own experimentation efforts to improve

the command of joint operations over the past few years have contributed

to joint experimentation by providing significant insights for the Joint

Forces Command’s development of the standing joint-force headquarters

concept. Central Command and Southern Command officials said their

Commands have plans to establish similar offices soon.

While satisfied with their participation and their ability to provide input

into the program, officials at some combatant commands believe that a

number of things could be done to improve the program, assuming

resources are available. They believe that the Joint Forces Command

could increase its visits to and participation in combatant-command

activities. Some of the officials also believe that if the Joint Forces

Command assigned liaison officers to their commands, the Command

would obtain first-hand knowledge and a better appreciation of the

various commands’ individual requirements. These officials believe that

such a presence at their commands would demonstrate the Joint Forces

Command’s commitment to joint experimentation and would allow for

interaction with staff throughout their commands. The Joint Forces

Command does not favor doing this because of the cost and the difficulty

 15 Additionally, Germany, the United Kingdom, Canada, and Australia have each assigned a

liaison officer at the Joint Forces Command.
Page 14 GAO-02-856 Military Transformation

in providing the staff necessary to fulfill this role. Officials at the Pacific,

Central, and Southern Commands also believe that some level of funding

should be provided to the combatant commands for their use in

supporting individual command and the Joint Forces Command

experimentation efforts. Combatant command officials stated that

currently, funds from other command activities must be diverted to

support these efforts.

Out of concern about the need to improve communications and

participation in joint experimentation planning, the Joint Forces

Command is planning some initiatives such as the following:

• It plans to create a virtual planning-center site for joint experimentation on

its Intranet to provide DOD stakeholders with easily accessible weekly

updates to information on planned experiments; participants; goals and

objectives; and ongoing experimentation by the Joint Forces Command,

the services, the combatant commands, and DOD agencies. It plans to

develop the requirements for the site during fall 2002 and to initiate the

project soon after.

• It established Project Alpha—a “think-tank” group—in early 2002 to

provide another source of input and outputs. The project will interface

with researchers throughout DOD, Department of Energy national

laboratories, private industry, and academia to find cutting-edge

technologies for inclusion in service and joint experimentation.

This relationship will provide an opportunity for the Joint Forces

Command to leverage the work of these organizations and similarly, for

these organizations to gain a better understanding of and include their

work in the joint experimentation program.

As the joint experimentation program matured, participation by non-DOD

federal agencies and departments gradually increased. Participation,

however, depends upon the agencies’ desire to be involved and their

available resources. Lack of involvement could lead to missed

opportunities. And participation by allies and coalition partners has been

limited by security concerns.

The Joint Forces Command’s input process allows individual federal

agencies and departments, such as the Departments of State and Justice,

to participate in joint experimentation events as they choose. Interagency

participation is improving, according to Command officials. For example,

federal agencies and departments are participating in Millennium

Expanding Interagency

and Foreign Participation

in Experimentation Events

Is Being Considered
Page 15 GAO-02-856 Military Transformation

Challenge 2002 to assist the Command in developing its standing jointforce headquarters concept.

However, resource and staffing constraints prevent some agencies and

departments from taking part in experiments. For example, according to a

Joint Forces Command official, the Department of Transportation and the

Central Intelligence Agency decided not to send representatives to

Millennium Challenge 2002 because of staffing constraints. Not only could

non-DOD agencies provide important insights and contributions to joint

operations, but also some important opportunities could be missed if these

agencies do not consistently participate in joint experimentation events.

While federal agencies and departments are beginning to increase their

role in joint experimentation, several service and combatant command

officials we spoke with believe that greater involvement is needed because

of the role these organizations are likely to have in future joint operations.

For example, these non-DOD federal agencies and departments would

provide support (economic, diplomatic, and information actions) to

U.S. military forces in their conduct of operations aimed at defeating an

adversary’s war-making capabilities—support that is critical to

implementation of the Joint Forces Command’s rapid decisive operations

concept.

Several DOD (service, combatant command, Office of the Secretary of

Defense, and other DOD organizations) officials we spoke with believe

that the Joint Forces Command should explore ways to boost the

participation and involvement of allies and coalition partners in joint

experimentation. Joint Forces Command officials agree and believe that

such cooperation would foster a better understanding of allied

perspectives, allow the Command to leverage concept development work,

expand available capabilities, and facilitate the development of

multinational capabilities. The Command recently created a multinational

concept-development and experimentation site on its Intranet to facilitate

the involvement of allies and coalition partners in joint experimentation.

However, some DOD officials believe that the Joint Forces Command

should do more because future U.S. military operations will likely be

conducted with other countries. The officials stress that other nations’

military personnel should be included in experiments to develop new

operational concepts, if these concepts are to be successful. Joint Forces

Command officials pointed out, however, that the participation and

involvement of other countries are often constrained by restrictions on

access to sensitive security information. For example, North Atlantic

Treaty Organization countries only participated as observers in
Page 16 GAO-02-856 Military Transformation

Millennium Challenge 2002 because of security information restrictions.

The Command, however, plans to develop ways to better handle these

restrictions to allow greater participation by other nations in its next

major field experiment in 2004.

Nearly 4 years after the program was established, only three

recommendations have flowed from the joint experimentation program,

and none of them have been approved. Confusion about proposed changes

in guidance regarding the information required for submitting these

recommendations has partly delayed their approval. At the time we

concluded our review, official guidance on what information should

accompany joint experimentation recommendations had not been

approved. In addition, several DOD officials expressed concern that the

process used to review and approve recommendations, the same as that

used for major acquisition programs, may not be the most appropriate for

a program whose aim is to integrate changes quickly. However, the

officials could not pinpoint any specific impasses in the approval process.

The DOD officials are also concerned about potential delays in the

integration of new concepts because of the lengthy DOD resource

allocation process.

The Joint Forces Command submitted one recommendation to the

Chairman of the Joint Chiefs of Staff in August 2001 and two more in

November 2001 (see table 2). At the time we ended our review, none of the

recommendations had been approved. The recommendations to improve

the planning and decision-making capabilities of joint forces and provide

better training for personnel conducting theater missile defense

operations were based on analyses of results of experiments carried out in

the first 3 years of joint experimentation. Inputs included two major

experiments: Millennium Challenge 2000 (live field experiment in

August-September 2000) and the Unified Vision 2001 (virtual simulation

experiment in May 2001). The first recommendation was submitted for

review just 3 months after the end of the last experiment.

No Recommendations

for Change Have Been

Approved or

Implemented

Concerns Raised about the

Delays in Approving

Recommendations
Page 17 GAO-02-856 Military Transformation

Table 2: First Three Recommendations Submitted by Joint Forces Command

Recommendation Description Changes requireda

Date submitted

for review

Collaborative

environment and

collaborative tools

Develop deployable

Internet-like tools to

improve the

collaborative planning

and decision-making

capabilities of joint

forces.

• Joint doctrine

• Training

• Leadership and

education

• Personnel

August 18, 2001

Joint intelligence

preparation of the

battle space

Develop objectives to

train and exercise joint

forces on intelligence

requirements for

preparing the battle

space, with a focus on

theater missile defense.

• Joint doctrine

• Training

• Leadership and

education

November 1,

2001

Theater missile

defense and timecritical targeting

team training

Develop a training

course to improve the

effectiveness of teams

involved in theater

missile defense and

time-critical targeting

activities.

• Training

• Leadership and

education

• Personnel

• Facilities

November 1,

2001

a

Recommendations can propose changes be made to doctrine, organizations, training, materiel,

leadership and education, personnel, and/or facilities.

Source: Joint Forces Command.

According to a Joint Staff official, however, approval of the

recommendations has been delayed because Joint Forces Command and

Joint Staff officials were confused about proposed changes in guidance.

In May 2001, the Joint Requirements Oversight Council proposed new

guidance, which would require that information on costs and timelines be

included in joint experimentation recommendations. Prior guidance did

not require such information. Although the recommendations went

through preliminary review by the Joint Staff, the omission was not caught

until the recommendations were to be scheduled for review by the Joint

Requirements Oversight Council. Joint Forces Command officials told us

that they were not aware of the change in guidance until that time. When

we ended our review, Joint Forces Command officials were working with

the Joint Staff to assess how much data could be prepared and when.

Command officials said that the recommendations will be resubmitted in

fall 2002 together with other recommendations emerging from Millennium

Challenge 2002. As a result, no recommendations have yet been reviewed

or approved. Also, at the time we ended our review, the draft guidance on

joint experimentation recommendations had not been approved and
Page 18 GAO-02-856 Military Transformation

issued. This guidance will become especially important because joint

experimentation is expected to produce new recommendations more

rapidly as the program matures.

The requirement for costs and timeline data is consistent with that of

recommendations for major weapon-system-acquisition programs.

However, joint experimentation officials at the Joint Forces Command

believe that requiring this type of information on joint-experimentation

recommendations may not be appropriate because (1) these

recommendations are generally intended to convince decision makers to

develop particular joint capabilities, not specific weapon systems;

(2) the new requirement may slow the preparation of future

recommendations; and (3) it will be difficult to provide accurate estimates

of costs and timelines for recommendations that span further into the

future. It is too early to determine whether these concerns are valid.

Some DOD officials were also concerned that the system currently used to

allocate resources to implement joint-experimentation

recommendations—DOD’s Planning, Programming, and Budgeting

System—may not be the most efficient because it usually takes a long time

to review, approve, and provide funding in future budgets.

A recommendation approved in 2002, for example, would not be

incorporated into DOD’s budget until 2004 or even later. This delay could

result in missed opportunities for more rapid implementation.

A Joint Staff official told us that the Joint Staff and the Joint Forces

Command recently adjusted the timing of events to better align the joint

experimentation process with the Planning, Programming, and Budgeting

System. Additionally, DOD established a special fund for the Joint Forces

Command to use as a temporary funding source to speed up the

implementation of certain critical or time-sensitive recommendations.

This source will provide early funding for implementation until funding is

provided through DOD’s Planning, Programming, and Budgeting System.

However, Joint Forces Command and other DOD officials believe other

ways to implement new joint capabilities within the framework of existing

budget and oversight practices may need to be considered.

Concerns Raised about the

Process for Funding

Implementation of

Recommendations
Page 19 GAO-02-856 Military Transformation

DOD has been providing more specific and clearer guidance on its goals,

expectations, and priorities for the joint experimentation program.

Nevertheless, the management of joint experimentation is missing a

number of key elements that are necessary for program success: some

roles and responsibilities have not yet been defined; current performance

measures are not adequate to assess progress; and the Joint Forces

Command lacks strategic planning tools for the program.

DOD officials stated that the joint experimentation program had difficulty

in its first years because guidance was evolving and was not specific:

DOD’s transformation goals were not adequately linked to transformation

efforts, and roles and responsibilities were not clearly defined.

16 Over time,

the Secretary of Defense and the Chairman of the Joint Chiefs of Staff

have provided more specific guidance on the goals and expectations for

joint experimentation and its contribution to DOD’s transformation

efforts. Guidance for joint experimentation has evolved gradually over the

program’s nearly 4-year life span, partly because of shifting defense

priorities and lack of clarity about the roles of various DOD stakeholders.

Roles and responsibilities have also matured with the program.

17

The Secretary of Defense’s 2001 Quadrennial Defense Review Report

18

established six transformation goals, which include improving

U.S. capabilities to defend the homeland and other bases of operations,

denying enemies sanctuary, and conducting effective information

operations. According to DOD officials, the Secretary of Defense’s most

recent planning guidance tasked the Joint Forces Command to focus its

experimentation on developing new joint operational concepts for these

goals. To begin meeting these goals, the Chairman has also provided the

Joint Forces Command with clarifying guidance that identified specific

areas for the Command to include in its experimentation, such as the

 16 We have pointed this difficulty out in prior transformation-related reports. See U.S.

General Accounting Office, Military Transformation: Navy Efforts Should Be More

Integrated and Focused, GAO-01-853 (Washington, D.C.: Aug. 2, 2001) and U.S. General

Accounting Office, Military Transformation: Army Has A Comprehensive Plan for

Managing Its Transformation but Faces Major Challenges, GAO-02-96 (Washington, D.C.:

Nov. 16, 2001).

17 The roles and responsibilities for most DOD organizations are broadly defined in the

Chairman of the Joint Chiefs of Staff Instruction 3010.02A, Joint Vision Implementation

Master Plan, Apr. 15, 2001.

18 Department of Defense, Quadrennial Defense Review Report (Washington, D.C.: 2001).

Some Key

Management

Elements Are Missing

Guidance More Clearly

Defined
Page 20 GAO-02-856 Military Transformation

development of a standing joint-force headquarters concept and of a

prototype to strengthen the conduct of joint operations. The Command

has reflected this new guidance in its latest Joint Concept Development

and Experimentation Campaign Plan. Additionally, the Secretary of

Defense reassigned the Command’s geographic responsibilities to focus it

more clearly on its remaining missions, particularly transformation and

joint experimentation.

DOD officials at both headquarters and the field believe that the recent

guidance begins to provide a better framework for the Joint Forces

Command to establish and focus its joint experimentation efforts.

Some officials, however, believe that future guidance should further clarify

the link between joint experimentation and DOD priorities and the

required resources necessary to support joint experimentation. DOD, in its

comments to a draft of this report, stated that it expects the

Transformation Planning Guidance—currently being prepared by the

Office of the Secretary of Defense—will establish the requirements

necessary to link experimentation to changes in the force.

While roles and responsibilities for DOD organizations are now broadly

defined, the new DOD Office of Force Transformation’s role in joint

experimentation and its relationship to other stakeholders have not yet

been clearly established. The Office’s charter or terms of reference have

not been released. DOD plans to issue a directive later this year that will

include a charter and description of the Office’s authorities and

responsibilities. However, there is still uncertainty about the extent of

authority and involvement the Office will have in the joint experimentation

program and the Office’s ability to link the program with DOD’s overall

transformation efforts. Joint Forces Command and other DOD officials

consider having a transformation advocate in the Office of the Secretary of

Defense as a beneficial link between the Joint Forces Command’s, the

services’, and the combatant commands’ joint experimentation programs

and DOD’s overall transformation agenda.

According to DOD’s 2001 Quadrennial Defense Review Report, the Office

of Force Transformation, created in November 2001, is to play a role in

fostering innovation and experimentation and should have an important

responsibility for monitoring joint experimentation and for providing the

Secretary of Defense with policy recommendations.

An Office of Force Transformation official told us that the Office will be an

advocate for transformation and will help develop guidance and make

Role of Office of Force

Transformation Is Not

Clear
Page 21 GAO-02-856 Military Transformation

recommendations on transformation issues to the Secretary of Defense

(the Office provided comments on the Secretary’s annual planning

guidance and developed instructions for the services on preparing their

first transformation road maps).

The Office has also decided to take a cautious approach in carrying out its

mission because of possible resistance from other DOD organizations, the

same official said. The Office plans to offer its assistance to DOD

organizations in their transformation efforts and attempt to influence their

thinking on key issues, rather than asserting itself directly into their

efforts, for example by funding military use of existing private-sector

technology to act as a surrogate for evaluating possible concepts, uses,

and designs.

Joint Forces Command officials stated that as of May 2002, they had had

only limited discussions with the Office and had not established any

working agreements on how the Office would participate in the joint

experimentation program.

The Office of Force Transformation has only recently assembled its staff

and is beginning to plan its work and establish contacts within DOD and

with other organizations. The Office’s budget for fiscal years 2002 and 2003

is about $18 million and $35 million, respectively.

DOD’s performance measures (or metrics) for assessing joint

experimentation—by measuring only the number of experiments carried

out—do not provide a meaningful assessment of the program’s

contribution toward meeting its performance goal for military

transformation because they are only quantitative. Consistent with good

management practices and in order to effectuate the purposes of the

Government Performance and Results Act of 1993,19 federal agencies

devise results-oriented metrics that provide an assessment of outcomes or

the results of programs as measured by the difference they make. In its

fiscal year 2000 performance report,20 the most recent it has issued, DOD

described the performance indicators for the joint experimentation

 19 Pub.L. 103-62, sec. 2(b), Aug. 3, 1993, sometimes referred to as the Results Act.

20 Department of Defense, Government Performance and Results Act: Department of

Defense Fiscal Year 2000 Performance Report (Washington, D.C.: 2001). As of August

2002, DOD did not submit a performance report to the Congress for fiscal year 2001.

Current Performance

Measures Are Not a Useful

Assessment Tool
Page 22 GAO-02-856 Military Transformation

program in terms of the number of experiments conducted against a target

goal for the prior, current, and following fiscal years. In fiscal year 2000,

DOD exceeded its target number of experiments and did not project any

shortfalls in meeting its target in the next fiscal year. Although this

measure does provide a quantitative assessment of experimental activity,

it does not provide a meaningful method for assessing how joint

experimentation is helping to advance military transformation.

An Office of the Secretary of Defense official stated that DOD recognizes

that better performance measures are needed for assessing how joint

experimentation advances transformation and for two other metrics

currently used to assess its military transformation goal.21 The official

stated that developing such measures is a challenge because joint

experimentation does not easily lend itself to traditional measurement

methods. For example, most programs consider a failure as a negative

event, but in joint experimentation, a failure can be considered as a

success if it provides insights or information that is helpful in evaluating

new concepts or the use of new technologies.

An Office of the Secretary of Defense official told us that the RAND

Corporation and the Institute for Defense Analyses recently completed

studies to identify possible performance measures for assessing the

progress of transformation. DOD is evaluating them and is preparing the

Transformation Planning Guidance to provide more specific information

on the priorities, roles, and responsibilities for executing its

transformation strategy. The same official stated that the new guidance

will include a discussion of the types of performance measures needed for

assessing transformation progress or will assign an organization to

determine them. In either case, measures will still need to be developed

and implemented. DOD plans to issue the new guidance later in 2002 but

has not determined how new performance measures would be

incorporated into its annual performance report.

The Joint Forces Command has not developed the strategic planning

tools—a strategic plan, an associated performance plan, and performancereporting tools—for assessing the performance of the joint

 21 DOD’s fiscal year 2000 performance report included three metrics for assessing its

transformation of U.S. military forces performance goal: joint experimentation,

procurement spending, and defense technology objectives.

Joint Forces Command’s

Program Lacks Strategic

Planning Tools
Page 23 GAO-02-856 Military Transformation

experimentation program. Strategic planning is essential for this type of

program, especially considering its magnitude and complexity and its

potential implications for military transformation. Such planning provides

an essential foundation for defining what an organization seeks to

accomplish, identifies the strategy it will use to achieve desired results,

and then determines—through measurement—how well it is succeeding in

reaching results-oriented goals and achieving objectives. Developing

strategic-planning tools for the joint experimentation program would also

be consistent with the principles set forth in the Government Performance

and Results Act of 1993, which is the primary legislative framework for

strategic planning in the federal government.

The Joint Forces Command prepares an annual Joint Concept

Development and Experimentation Campaign Plan that broadly

describes the key goals of its program, the strategy for achieving these

goals, and the planned activities. However, a February 2002 progress

report, prepared by the Joint Forces Command’s Joint Experimentation

Directorate, on the development of the Directorate’s performance

management system indicated that one-fourth of those organizations

providing feedback22 on the Campaign Plan believed that the Plan lacks

specificity in terms of the program’s goals and objectives and an

associated action plan that outlines the activities to be carried out in order

to achieve those goals. Officials we spoke with at the military services, the

combatant commands, and the Joint Forces Command all cited the need

for more specific and clearer goals, objectives, and performance measures

for the program.

In the progress report, the Command acknowledged the benefits of

strategic planning and the use of this management tool to align its

organizational structure, processes, and budget to support the

achievement of missions and goals. The report proposed that the

Command develop a strategic plan, possibly by modifying its annual

Campaign Plan, and subsequently prepare a performance plan and a

performance report. Command officials indicated that the basic

requirements of a strategic plan could be incorporated into the

Campaign Plan, but they were unsure, if such an approach were taken,

whether the changes could be made before the annual Campaign Plan is

 22 DOD organizations providing comments to the draft Campaign Plan included various

offices and activities of the Army, the Navy, the Air Force, the Marine Corps, the

U.S. Special Operations Command, the U.S. Strategic Command, the U.S. Southern

Command, the U.S. Pacific Command, and the U.S. European Command.
Page 24 GAO-02-856 Military Transformation

issued later this year. Similarly, the Joint Forces Command has had

difficulty in developing specific performance measures for joint

experimentation. A Command official stated that the Command has tried

to leverage the performance measures developed by other organizations

like itself, but found that there is widespread awareness throughout the

research and development community, both within and outside DOD, that

such measures are needed but do not exist. Additionally, a Joint Forces

Command official stated that whatever metrics the Command develops

must be linked to its mission-essential tasks for joint experimentation and

that the Command is currently developing these tasks. At the time we

ended our review, the Command had identified six broad areas for which

specific metrics need to be developed. These included quality of life,

customer relationships, and experimentation process management.

After nearly 4 years, the Joint Forces Command’s process for obtaining

inputs for the development and execution of DOD’s joint experimentation

program has become more inclusive. However, questions continue about

whether the program is the successful engine for change envisioned when

it was established. Since the program’s inception, only three

recommendations have flowed from experimentation activities and their

review, approval, and implementation have been delayed from confusion

over a change in guidance that required additional information be included

in the recommendations. As a result, no recommendations for change have

been approved or implemented to date. To the extent that the draft

guidance on what should be submitted with joint experimentation

recommendations can be officially approved and issued, future

recommendations could be submitted for approval and implementation

more quickly. Underscoring the need to finalize the guidance is the

anticipated recommendations to be made after this year’s major field

experiment, Millennium Challenge 2002.

The lack of strategic planning for joint experimentation deprives the

Joint Forces Command of necessary tools to effectively manage its

program. Implementation of strategic planning at the Joint Forces

Command would create a recurring and continuous cycle of planning,

program execution, and reporting and establish a process by which the

Command could measure the effectiveness of its activities as well as a

means to assess the contributions of those activities to the operational

goals and mission of the program. Such planning could also provide a

tool—one that is currently missing—to identify strengths and weaknesses

in the development and execution of the program and a reference

document for the effective oversight and management of the program.

Conclusions
Page 25 GAO-02-856 Military Transformation

Performance measures developed under the Command’s strategic

planning could provide the standard for assessing other experimentation

efforts throughout DOD, which are also lacking such metrics.

The lack of a meaningful performance measure for assessing the

contribution of the joint experimentation program to advance

DOD’s transformation agenda limits the usefulness and benefit of this

management tool to assist congressional and DOD leaders in their

decision-making responsibilities. Establishing a “meaningful” joint

experimentation performance measure for its annual performance report

would provide congressional and DOD leadership a better assessment of

the program’s contribution and progress toward advancing

transformation. Such a metric would also be consistent with the intent of

the Results Act to improve the accountability of federal programs for

achieving program results. Because the role and relationships of the

Secretary of Defense’s new Office of Force Transformation have not yet

been clarified, the Secretary may not be effectively using this office in

DOD’s transformation efforts. This office, if given sufficient authority,

could provide the Secretary with a civilian oversight function to foster and

monitor the joint experimentation program to ensure that it is properly

supported and provided resources to advance the DOD’s overall

transformation agenda. Rectifying these shortcomings is critical in view of

the importance that DOD has placed on joint experimentation to identify

the future concepts and capabilities for maintaining U.S. military

superiority.

To improve the management of DOD’s joint experimentation program, we

recommend that the Secretary of Defense direct the Chairman of the Joint

Chiefs of Staff to

• approve and issue guidance that clearly defines the information required

to accompany joint experimentation recommendations for the Joint

Requirements Oversight Council’s review and approval and

• require the Commander in Chief of the U.S. Joint Forces Command to

develop strategic planning tools to use in managing and periodically

assessing the progress of its joint experimentation program.

We further recommend that the Secretary of Defense

• develop both quantitative and qualitative performance measures for joint

experimentation in DOD’s annual performance report to provide a better

Recommendations for

Executive Action
Page 26 GAO-02-856 Military Transformation

assessment of the program’s contribution to advancing military

transformation and

• clarify the role of the Office of Force Transformation and its relationship

to the Chairman of the Joint Chiefs of Staff, the Joint Forces Command,

and other key DOD stakeholders in DOD’s joint experimentation program.

We received written comments from DOD on a draft of this report, which

are included in their entirety as appendix III. DOD agreed with our

recommendations and indicated that it expects that a forthcoming

Transformation Planning Guidance and subsequent guidance will be

responsive to them by clarifying roles and missions across DOD,

implementing recommendations for changes, and establishing clear

objectives. We believe such strategic guidance from the Secretary of

Defense could provide a significant mechanism for better linking and

clarifying the importance of the joint experimentation program with

DOD’s transformation agenda. DOD also provided technical comments to

the draft that were incorporated in the report where appropriate.

To determine the extent to which the Joint Forces Command obtains input

from stakeholders and other relevant sources in developing and

conducting its joint experimentation activities, we reviewed an array of

documents providing information about participants in joint

experimentation, including guidance and other policy documents, position

papers, fact sheets, reports, and studies of the military services, the

combatant commands, the Joint Staff, and other DOD organizations.

We also reviewed Joint Forces Command plans and reports. Additionally,

we made extensive use of information available on public and DOD

Internet web sites. To assess the change in participation by various

stakeholders over time, we compared the differences in the numbers of

participating organizations and initiatives provided by these organizations

between the Joint Forces Command’s first two major field experiments in

2000 and 2002 (Millennium Challenge 2000 and Millennium Challenge

2002). We conducted discussions with officials at five combatant

commands, the Joint Staff, the military services, and other DOD

organizations, such as the Joint Advanced Warfighting Program and the

Defense Advanced Research Projects Agency. Appendix IV lists the

principal organizations and offices where we performed work. At the

Joint Forces Command, we discussed with joint experimentation officials

the process for soliciting and incorporating inputs for joint

experimentation from the military services and the combatant commands.

We also attended conferences and other sessions hosted by the

Agency Comments

and Our Evaluation

Scope and

Methodology
Page 27 GAO-02-856 Military Transformation

Joint Forces Command to observe and learn about joint experimentation

participants and their contributions and coordination. For example, we

attended sessions for the Command’s preparation of its annual Joint

Concept Development and Experimentation Campaign Plan and

planning for this year’s Millennium Challenge experiment. With officials

from each of the services and the combatant commands, we discussed

perceptions of the effectiveness of coordination and participation in joint

experimentation. We also obtained observations about participants’

involvement from several defense experts who track joint experimentation

and military transformation. Although we did not include a specific

assessment of the individual experimentation efforts of the services and

combatant commands, we did discuss with service and command officials

how their efforts were coordinated and integrated into joint

experimentation. We also did not determine the extent that individual

inputs obtained from various participating organizations were considered

and incorporated into the joint experimentation program.

To determine the extent to which recommendations flowing from the joint

experimentation process have been approved and implemented, we

reviewed and analyzed data that tracked the progress of the first three

joint experimentation recommendations submitted by the Joint Forces

Command. We also obtained and analyzed relevant guidance and held

discussions with Joint Staff, Joint Forces Command, and Office of the

Secretary of Defense officials on the Joint Requirements Oversight Council

process for reviewing and approving joint experimentation

recommendations. We also discussed issues relating to implementation of

joint experimentation recommendations through DOD’s Planning,

Programming, and Budgeting System.

To assess whether key management elements, such as policy,

organization, and resources, were in place for the program, we conducted

a comprehensive review of current legislative, policy, planning, and

guidance documents and reports and studies. We used the principles laid

out in the Government Performance and Results Act of 1993 as an

additional benchmark for assessing the adequacy of performance

measures established for the program and of tools used to manage the

program. We also discussed the status and evolution of joint

experimentation oversight and management, including office roles and

responsibilities and joint experimentation metrics, with officials at the

Joint Forces Command, the Joint Staff, the services, the combatant

commands, the Office of the Secretary of Defense, the Office of Force

Transformation, and other DOD organizations. Several defense experts

who follow joint experimentation and military transformation discussed
Page 28 GAO-02-856 Military Transformation

with us joint experimentation oversight and management and gave us their

impressions regarding current joint experimentation management

practices.

Our review was conducted from October 2001 through May 2002 in

accordance with generally accepted government auditing standards.

We are sending copies of this report to interested congressional

committees, the Secretary of Defense, the Chairman of the Joint Chiefs of

Staff, and the Commander in Chief, U.S. Joint Forces Command. We will

also make copies available to others upon request. In addition, this report

will be available at no charge on the GAO Web site at http://www.gao.gov.

Please contact Richard G. Payne at (757) 552-8119 if you or your staff have

any questions concerning this report. Key contacts and contributors to this

report are listed in appendix V.

Henry L. Hinton, Jr.

Managing Director

Defense Capabilities and Management
Appendix I: Chronology of Key Events

Important to Joint Experimentation

Page 29 GAO-02-856 Military Transformation

Date Event Relevance to joint experimentation

May 1996 Chairman of the Joint Chiefs of Staff

issued Joint Vision 2010.

This vision of future war fighting provides a conceptual template for

the Department of Defense’s (DOD) transformation efforts across all

elements of the armed forces.

May 1997 DOD’s Report of the Quadrennial

Defense Review issued.

Report discussed the importance of preparing for future national

security challenges. It concluded that DOD needed to institutionalize

innovative investigations, such as war-fighting experiments, to

ensure future concepts and capabilities are successfully integrated

into the forces in a timely manner.

May 1998 Secretary of Defense designated

Commander in Chief, U.S. Joint

Forces Command,a

 as executive agent

for joint experimentation.

The Secretary of Defense tasked the Joint Forces Command to

design and conduct joint war-fighting experimentation to explore,

demonstrate, and evaluate joint war-fighting concepts and

capabilities.

April 1998 Joint Advanced Warfighting Program

established.

DOD established the program at the Institute for Defense Analyses to

serve as a catalyst for achieving the objectives of Joint Vision 2010

(and later Joint Vision 2020). To that end, the program is to develop

and explore breakthrough operational concepts and capabilities that

support DOD’s transformation goals.

October 1998 Joint Forces Command assumed

responsibility as the executive agent

for joint experimentation.

Joint concept development and experimentation program initiated.

June-August 1999 Joint Advanced Warfighting Program

conducted the first joint experiment for

Joint Forces Command.

An experiment—J9901—that investigated approaches for attacking

critical mobile targets. Experiment allowed the Joint Forces

Command to begin its learning process on how to conduct joint

experimentation.

September 1999 Report of the Defense Science Board

Task Force on DOD Warfighting

Transformation issued.

Report proposed several recommendations to promote military

transformation.

June 2000 Chairman of the Joint Chiefs of Staff

issued Joint Vision 2020.

Updated vision statement described the joint war-fighting capabilities

required through 2020.

August-September

2000

Millennium Challenge 2000 major field

experiment conducted.

The first major field experiment coordinated by the Joint Forces

Command among the services and other stakeholders.

April 2001 Chairman of the Joint Chiefs of Staff

issued updated Joint Vision

Implementation Master Plan.

Guidance described the process for generation, coordination,

approval, and implementation of recommendations emerging from

joint experimentation and defined the roles and responsibilities of

DOD stakeholders.

April 2001 Transformation Study Report:

Transforming Military Operational

Capabilities issued.

Study conducted for the Secretary of Defense to identify capabilities

needed by U.S. forces to meet the twenty-first century security

environment. Made several recommendations directed at improving

joint experimentation.

May 2001 Joint Forces Command conducted

Unified Vision 2001 experiment.

A major joint experiment—largely modeling and simulation—

conducted to refine and explore several war-fighting concepts, such

as “rapid decisive” operations.

August 2001 Secretary of Defense’s planning

guidance issued.

Required studies by defense agencies and the Joint Staff to develop

transformation road maps and a standing-joint-force headquarters

prototype.

September 2001 DOD’s Quadrennial Defense Review

Report issued.

The report established priorities and identified major goals for

transforming the Armed Forces to meet future challenges. It called

for new operational concepts, advanced technological capabilities,

and an increased emphasis on joint organizations, experimentation,

and training.

Appendix I: Chronology of Key Events

Important to Joint Experimentation
Appendix I: Chronology of Key Events

Important to Joint Experimentation

Page 30 GAO-02-856 Military Transformation

Date Event Relevance to joint experimentation

November 2001 Chairman of the Joint Chiefs of Staff

issued joint experimentation guidance.

The guidance directed the Joint Forces Command to focus its nearterm experimentation on developing a standing joint force

headquarters prototype.

November 2001 Office of Force Transformation

established.

Office assists the Secretary of Defense in identifying strategy and

policy, and developing guidance for transformation.

April 2002 Unified Command Plan 2002 issued. Plan reduced the number of missions assigned to the Joint Forces

Command to allow the Command to devote more attention to its

remaining missions such as joint experimentation.

May 2002 Secretary of Defense’s planning

guidance issued.

The guidance directed the Joint Forces Command to develop new

joint concepts that focus on the six transformation goals set forth in

the 2001 Quadrennial Defense Review Report.

July-August 2002 Joint Forces Command conducted

Millennium Challenge 2002.

Second major field experiment conducted to culminate a series of

experiments to assess “how” to do rapid decisive operations in this

decade.

a

 In October 1999, the U.S. Atlantic Command was redesignated as the U.S. Joint Forces Command.
Appendix II: Additional Information on Joint

Concept Development and Experimentation

Activities

Page 31 GAO-02-856 Military Transformation

The Joint Forces Command uses various types of assessment activities to

develop, refine, and validate joint concepts and associated capabilities.

As shown in figure 3, the Command begins to move through the five joint

concept development phases by conducting workshops, seminars, and war

games to develop information and identify possible areas to explore in

developing new concepts and associated capabilities and then uses

simulated or live experiment events to confirm, refute, or modify them.

These activities vary in scale and frequency, but each activity becomes

larger and more complex. They can involve a small group of retired flag

officers and academics, up to 100 planners, operators, and technology

experts, or several thousand in the field. Near the end of the process, the

Command will conduct a large-scale simulation experiment (such as

Unified Vision 2001), followed by a major field experiment (such as

Millennium Challenge 2002). The process continuously repeats itself to

identify additional new concepts and capabilities.

Figure 3: Joint Concept Development and Experimentation Activities

Source: Joint Forces Command.

Appendix II: Additional Information on Joint

Concept Development and Experimentation

Activities
Appendix II: Additional Information on Joint

Concept Development and Experimentation

Activities

Page 32 GAO-02-856 Military Transformation

Table 3 provides additional information about the characteristics, scale,

and frequency of these and other associated activities and experiments.

Table 3: Types of Activities and Experiments Associated with Joint Experimentation

Type of activities and

experiments Characteristics Scale and frequency

Workshop Brief, intense experimentation program used to discover

and generate concepts. Participants identify focus areas

and research topics. Workshops are conducted live and

may have virtual components. Subject matter experts

participate.

Small scale

Many each year

Seminar Discussion forums used to discover and define problem

boundaries, pose solutions, and exchange information.

Seminars are conducted live, and may have virtual

components. Subject matter experts participate.

Small scale

Many each year

War game An event, using simulated and emulated systems, centering

on the interaction of two or more opposing forces depicting

a possible or actual scenario. Identifies key variables in an

experiment; refines concepts and assesses alternatives.

Scale varies

One or more each year

Limited objective experiment An event, limited in scope, that permits participants to see

how specific, focused ideas or concepts will work in a given

context and works with supporting elements of the

integrating concept. It confirms or refines experiments and

is primarily virtual or simulated. May include workshops,

seminars, and war games.

Small scale

Varies (6 in 2001)

Leveraged event Service or other organization’s event in which the Joint

Forces Command and its partners share lessons learned.

The event may involve the use of live forces in a theater

scenario.

Scale varies

Frequency varies

Spiral A group of smaller events and meetings requiring live

participants; concepts are refined and plans are made for

larger live Joint Forces Command experiments.

Small to medium scale

A few each year

Major simulation experiment A constructive, virtual, or live experiment that makes

extensive use of models and simulation to provide detailed

information on the expected outcomes of an event or

sequence of events. It may use live and/or simulated

participants.

Large scale

Not more than one every 2 years

Major field experiment Integrates several smaller, previous events. Determines

how effectively the concepts previously explored in smaller

experiments will operate when integrated with the concepts

of the other services and combatant commands. It validates

concepts and requires live participants as well as simulated

opponents.

Large scale

Not more than one each year
Appendix III: Comments from the Department

of Defense

Page 33 GAO-02-856 Military Transformation

Appendix III: Comments from the

Department of Defense

Report number is now:

GAO-02-856.
Appendix III: Comments from the Department

of Defense

Page 34 GAO-02-856 Military Transformation

Now on p. 26.

Now on p. 25.

Now on p. 25.
Appendix IV: Organizations and

Offices Contacted

Page 35 GAO-02-856 Military Transformation

Department of Defense

• Office of the Secretary of Defense, Program Analysis and Evaluation

• Office of the Under Secretary of Defense for Policy

• Office of the Under Secretary of Defense for Acquisition, Technology,

and Logistics

• Joint Advanced Warfighting Program

• Defense Advanced Research Project Agency

• Office of Force Transformation

The Joint Staff

• Operational Plans and Interoperability Directorate

• Joint Vision and Transformation Division

• Command, Control, Communications, and Computers Directorate

• Force Structure, Resources, and Assessment Directorate

Department of the Army

• Directorate of Training

• Directorate of Integration

• Directorate for Strategy, Concepts, and Doctrine

Department of the Navy

• Office of the Deputy Chief of Naval Operations for Warfare

Requirements and Programs

• Marine Corps Combat Development Command

Department of the Air Force

• Wargaming and Experimentation Division

Other Organizations

• Booz Allen Hamilton

• The Carlyle Group

• Center for Strategic and Budgetary Assessments

• Hicks & Associates, Inc.

Appendix IV: Organizations and Offices

Contacted

Washington, D.C.,

Area
Appendix IV: Organizations and

Offices Contacted

Page 36 GAO-02-856 Military Transformation

U.S. Pacific Command

• Joint Experimentation Division

• Intelligence Architecture and Systems Division

• Exercise Division

• Regional Strategy and Policy Division

U.S. Pacific Fleet

• Joint Experimentation Office

U.S. Army Pacific

• Internal Review Office

U.S. Pacific Air Forces

• Operational Requirements

Marine Forces Pacific

U.S. Southern Command

• Future Operations

• Programs and Resources Directorate

• Office of Analysis and Simulations

Navy Warfare Development Command

U.S. Joint Forces Command

• Joint Experimentation Directorate

• Joint Integration and Interoperability Directorate

Honolulu, Hawaii,

Area

Miami, Fla., Area

Newport, R.I., Area

Norfolk, Va., Area
Appendix IV: Organizations and

Offices Contacted

Page 37 GAO-02-856 Military Transformation

Department of the Army

• Training and Doctrine Command

• Army Joint Venture Directorate

• Army Audit Agency

Department of the Air Force

• Air Combat Command

• Air Force Experimentation Office

Joint Forces Staff College

U.S. Central Command

• Futures Division

• Exercise Division

U.S. Special Operations Command

• Special Operations Requirements and Resources

• Joint Experimentation Division

• Special Operations Acquisition and Logistics

• Intelligence and Information Operations

Tampa, Fla., Area
Appendix V: GAO Contacts and

Staff Acknowledgments

Page 38 GAO-02-856 Military Transformation

Sharon L. Pickup (202) 512-9619

Richard G. Payne (757) 552-8119

In addition to the individuals named above, Carol R. Schuster,

Mark J. Wielgoszynski, John R. Beauchamp, Kimberley A. Ebner,

Lauren S. Johnson, and Stefano Petrucci made key contributions to this

report.

Appendix V: GAO Contacts and Staff

Acknowledgments

GAO Contacts

Acknowledgments

(350120)
The General Accounting Office, the investigative arm of Congress, exists to

support Congress in meeting its constitutional responsibilities and to help

improve the performance and accountability of the federal government for the

American people. GAO examines the use of public funds; evaluates federal

programs and policies; and provides analyses, recommendations, and other

assistance to help Congress make informed oversight, policy, and funding

decisions. GAO’s commitment to good government is reflected in its core values

of accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no cost is

through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and fulltext files of current reports and testimony and an expanding archive of older

products. The Web site features a search engine to help you locate documents

using key words and phrases. You can print these documents in their entirety,

including charts and other graphics.

Each day, GAO issues a list of newly released reports, testimony, and

correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site

daily. The list contains links to the full-text document files. To have GAO e-mail

this list to you every afternoon, go to www.gao.gov and select “Subscribe to daily

E-mail alert for newly released products” under the GAO Reports heading.

The first copy of each printed report is free. Additional copies are $2 each. A

check or money order should be made out to the Superintendent of Documents.

GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a

single address are discounted 25 percent. Orders should be sent to:

U.S. General Accounting Office

441 G Street NW, Room LM

Washington, D.C. 20548

To order by Phone: Voice: (202) 512-6000

TDD: (202) 512-2537

Fax: (202) 512-6061

Contact:

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800

U.S. General Accounting Office, 441 G Street NW, Room 7149

Washington, D.C. 20548

GAO’s Mission

Obtaining Copies of

GAO Reports and

Testimony

Order by Mail or Phone

To Report Fraud,

Waste, and Abuse in

Federal Programs

Public Affairs