Report of the  

Defense Science Board Advisory Group on Defense

Intelligence

Operations Research Applications for Intelligence,

Surveillance and Reconnaissance (ISR)

January 2009

Office of the Under Secretary of Defense

for Acquisition, Technology, and Logistics

Washington, D.C. 20301‐3140
This report is a product of the Defense Science Board (DSB).    The DSB is a Federal Advisory

Committee established to provide independent advice to the Secretary of Defense.  Statements,

opinions, conclusions, and recommendations in this report do not necessarily represent the

official position of the Department of Defense.  

The DSB Advisory Board Group on Defense Intelligence Task Force on Operations Research for ISR

completed its information gathering in September 2008.   

This report is UNCLASSIFIED and releasable to the public.

This page is intentionally blank.
OFFICE OF THE SECRETARY OF DEFENSE 

3140 Defense Pentagon 

Washington, DC 20301-3140 

DEFENSE SCIENCE 

BOARD 

December 9, 2008 

MEMORANDUM FOR THE CHAIRMAN, DEFENSE SCIENCE BOARD 

SUBJECT: Final Report of the Defense Science Board (DSB) Advisory Group on Defense 

Intelligence Task Force on Operations Research Applications for Intelligence, Surveillance and 

Reconnaissance (ISR) 

Attached is the final report of the DSB Advisory Group on Intelligence Task Force on 

Operations Research Applications for ISR. The Task Force was asked to examine the use of 

operations research (OR) methods to support ISR decision making within the Department of 

Defense (DoD). 

The Terms of Reference asked the Task Force to survey the departments and agencies to 

determine how much OR is being performed; assess how the results of OR are being used in 

decision making; recommend a test case(s) for using OR methodologies; and recommend steps 

the Department can take to institutionalize the use of OR in future DoD decision making. 

The Task Force reviewed the application of OR – which it views as decision support analysis 

based on information collected and analyzed using the scientific method – throughout the DoD 

and the Services focusing primarily on the resources available to conduct OR, as opposed to 

reviewing specific programs. The Task Force examined the commitment of decision makers to 

the use of OR and the extent to which these resources reflected the application of standard OR 

processes and practices and the manner in which the results appeared to be employed. The Task 

Force also looked at several private-sector OR application models and at recent and historical 

uses of OR in support of national security requirements. 

Based on its inquiry the Task Force arrived at the following observations: 

• Operations Research represents a powerful tool to help improve the quality of 

investment decision making by illuminating key issues, assumptions, and sources of 

information. 

• Operations Research is applied inconsistently throughout the Defense and ISR 

communities and each lacks standard OR processes and practices, and consistent 

organizational models or commitments. 
• OR and the use of it can be strengthened in the Defense and ISR communities 

through effective institutionalization. 

• The utility of OR can be more firmly established through appropriate test cases. 

Based on its findings, the Task Force recommends establishing an organic OR oversight function 

and investment decision capability within the USD(I). In response to the TOR, the Task Force 

recommends two test cases, one related to the balance in the intelligence cycle and the other 

related to investment decision making in biometrics technologies. 

The attached report provides the rationale for the Task Force’s findings and recommendations, 

reflects on the historical use of OR, details current OR practices, specifies attributes required for 

the test cases, and offers potential institutionalization models. 

We appreciate the contributions made by the Department of Defense, Intelligence Community, 

and private sector professionals who took the time to provide us with their knowledge and 

expertise; the members of this study; the Executive Secretary; and the DSB Secretariat and its 

military assistant. 
_______________________________________________________________________TABLE OF CONTENTS

TABLE OF CONTENTS

Executive Summary......................................................................................................................... 3 

Chapter 1:  Introduction and Tasking ............................................................................................. 7 

Chapter 2:  The Task Force’s Understanding of Operations Research ........................................... 9 

2.1 Desired Attributes of Operations Research.................................................................... 9 

2.2 Advantages of Operations Research............................................................................. 10 

2.3 ISR Decisions to which Operations Research is Applicable .......................................... 10 

2.4 Conditions under which Operations Research can be Institutionalized ...................... 10 

Chapter 3:  Operations Research – Historical Context ................................................................. 13 

3.1 General Lessons Learned from Historical Experience .................................................. 18 

Chapter 4:  Relevance of Operations Research to Intelligence, Surveillance, and Reconnaissance

....................................................................................................................................................... 21 

4.1 Current ISR Issues that can Benefit from the Application of OR .................................. 21 

4.2 Past ISR Investment Issues that have Benefited from the Application of OR .............. 22 

4.3 Complications in Applying OR to ISR Investment Issues .............................................. 22 

4.4 Challenges in Applying OR to ISR Issues ....................................................................... 23 

Chapter 5:  Current Operations Research Practices (TOR Tasks 1 and 2) .................................... 25 

Chapter 6:  ISR Test Case(s) Attributes (TOR Task 3).................................................................... 29 

6.1 Importance.................................................................................................................... 29 

6.2 Leadership Commitment .............................................................................................. 29 

6.3 OR Process Credibility ................................................................................................... 30 

6.4 Test Case Scope............................................................................................................. 30 

6.5 Test Case Timing and Duration ..................................................................................... 32 

6.6 A Note on ISR Test Case Challenges.............................................................................. 32 

Chapter 7:  Institutionalizing Operations Research (TOR Task 4)................................................. 33 

Chapter 8:  Findings and Recommendations................................................................................ 37 

Appendix A:  Terms of Reference ................................................................................................. 39 

Appendix B:  Task Force Membership .......................................................................................... 43 

Appendix C:  Briefings Received ................................................................................................... 45 

Appendix D:  Test Cases................................................................................................................ 47 

D.1 Biometrics..................................................................................................................... 47 

D.2 Investing in a Balanced Intelligence Cycle and Imagery UASs ‐ a Value Proposition ... 49 

Appendix E:  Glossary.................................................................................................................... 53 

OPERATIONS RESEARCH APPLICATIONS FOR ISR ___________________________________________________i
TABLE OF CONTENTS_______________________________________________________________________

This page is intentionally blank.

ii__________________________________DEFENSE SCIENCE BOARD ADVISORY GROUP ON DEFENSE INTELLIGENCE
_____________________________________________________________________ EXECUTIVE SUMMARY

EXECUTIVE SUMMARY

The Advisory Group on Defense Intelligence (the Task Force) of the Defense Science

Board (DSB) was tasked by the Under Secretary of Defense for Intelligence (USD(I)) to

examine the manner and extent to which Operations Research (OR) is employed by the

Department of Defense; how OR can be used to support Intelligence, Surveillance, and

Reconnaissance (ISR) decision making; and the manner in which OR can be

institutionalized in the Department of Defense (DoD).

The Task Force reviewed the application of OR1 – which it views as decision support

analysis based on information collected and analyzed using the scientific method –

throughout the DoD and the Services.    The review was done on a selective basis,

focusing principally on the resources available to conduct OR as opposed to specific

programs.  The Task Force looked at decision makers’ commitment to the use of OR, the

extent to which resources reflected the application of standard processes and practices,

and the manner in which the results of resources appeared to be employed by the

organizations of which they are a part.  The Task Force also looked at several private‐

sector OR application models (notably FedEx), and at recent and historical use of OR in

support of national security requirements.   

Four aspects of our inquiry stand out:

1. Operations Research represents a powerful tool to help improve the quality

of investment decision making by illuminating key issues, assumptions, and

sources of information.

2. Operations Research is applied inconsistently throughout the Defense and

ISR communities.  These communities do not possess standard OR processes

and practices, a consistent organizational model, or a consistent

commitment to the use of OR.

3. OR – and its use – can be strengthened in the Defense and ISR communities

through effective institutionalization.    The Task Force commends to the

USD(I) for further consideration models employed by the private sector (of

which FedEx appears to be a strong examples) and the Army (TRAC and CAA).  

The Military Operations Research Society (MORS) is a domain expert

resource that should be used in building a plan for institutionalization.

4. The utility of OR can be more firmly established through appropriate test

cases.    The Task Force points to two cases for USD(I) consideration:

Biometrics and Investment in a Balanced Intelligence Cycle, with specific

emphasis on Unmanned Aerial Systems (UASs).

 

1 The analytical study of military problems undertaken to provide responsible commanders and staff agencies with a scientific basis for

decision on action to improve military operations. Also called operational research; operations analysis.” (JCS Pub 1‐02)

OPERATIONS RESEARCH APPLICATIONS FOR ISR ___________________________________________________3
EXECUTIVE SUMMARY _____________________________________________________________________  

The Task Force believes Operations Research can strengthen the quality of ISR decision

making in terms of the ability of investment decisions to more clearly reflect

requirements, alternatives, resources, and technologies.    However, OR is currently

employed throughout DoD and the Services on an inconsistent basis, reflecting varying

levels of leadership commitment and support, a lack of standards regarding OR training

and methodologies, and uncertain career recognition and prospects for OR

practitioners.  Although the Federal Acquisition Regulations (FAR) require the use of OR

in support of Major Acquisition Programs, OR is used inconsistently in ISR investment

decision making, even on major systems, and the resources employed do not reflect

standard practice.  In general, OR is used to configure existing resources or to support

(or refute the need for) additional “buys” of existing systems.  OR is also used to support

complex mission management decisions regarding operational systems.    Overall,

however, the use of Operations Research is diminished by a seeming lack of leadership

support for its application in support of program investment, career investment, and

sufficient resource investment to create sustained excellence in its practice throughout

the ISR and larger DoD communities.    In contrast, OR (and science method‐based

analysis) were disciplines prized for their ability to help decision makers meet significant

challenges, particularly in support of U.S. military operations during World War II, in

determining force mix and sufficiency after WWII, and in rationally mobilizing the U.S.

industrial base in support of national security requirements.  While there exist islands of

OR practice today – some of them quite competent – in DoD and the Services,

institutional capacity to apply science‐based analysis to complex decisions is far less

than it has been in the past, and less than it should be.

Institutionalization of OR for ISR investment decision making requires a sound

management model and the sustained support of DoD leaders, both in its application

and sustainment of the resources necessary to keep OR a robust discipline throughout

the Department and the Services.  Such commitment will be required to reach a level of

competent OR practice throughout the ISR community.  At present, competent OR cells

exist, but are poorly connected as a professional community and inconsistently utilized

by decision makers.  The Task Force believes OR should be institutionalized, practiced

consistently (from a standards and methodology perspective), and used routinely to

support high value decisions involving complex requirements and significant resources.  

The Task Force understands clearly that OR does not “make” decisions; rather OR can

and should make visible the sources of data and information used to make decisions,

help drive the development of consistent assumptions and requirements, and give an

objective basis against which decisions can be weighed.  The Task Force recognizes that

complex decisions exist within a wider context of policy and changing requirements; OR

provides an objective “anchor” that decision makers should consider.

The Task Force recommends that the Federal Express (FedEx) management model be

given strong USD(I) consideration as a sound example of the use of OR in the private

sector.    In addition, the work being done by the Army’s Training and Doctrine

Command’s (TRADOC) Analyses Center (TRAC) and the Center for Army Analysis (CAA)

4__________________________________ DEFENSE SCIENCE BOARD ADVISORY GROUP ON DEFENSE INTELLIGENCE
_____________________________________________________________________ EXECUTIVE SUMMARY

provide good examples of utilizing OR within the government.  The private sector model

used by FedEx employs OR at a corporate level to decide which lines of business are

competitive and at what level; employs OR at the line‐of‐business level for investment

decision making and configuration of operational resources; and provides corporate

oversight to ensure the presence of consistent OR standards methodology throughout

the company’s family of businesses.  FedEx does not “require” the use of OR instead the

FedEx leadership commitment to OR‐based decision making creates a culture which

gives its practitioners substantial influence during high value decision making.   At the

same time, the Task Force believes that TRAC may represent a strong example of an

embedded, “line‐of‐business” capability, analogous to the OR capabilities in FedEx’s

various lines‐of‐business.  Overall, the Task Force recommends the USD(I) establish an

organic2 ISR OR oversight function that requires, sustains, and reviews OR standard

processes and practices in DoD ISR investment decisions.    

In response to the TOR, the Task Force recommends two ISR test cases.  The first related

to investment decision making in biometrics technologies in support of ISR

requirements; and the second to investment decision making for balance in the overall

intelligence cycle, and specifically in regard to UASs.    These test cases should be

structured to illuminate challenges and opportunities to strengthen OR decision making

in the ISR investment domain.  The biometrics test case provides an opportunity to help

biometrics meet its ISR potential through stronger investment strategy decision making,

creation of a real biometrics information architecture, and development of a biometrics

information sharing environment useful to both intelligence analysts and warfighters.

Building a more effective OR capability to support ISR is a difficult but meaningful

challenge.    The Task Force believes that the Military Operations Research Society

(MORS) is a resource the USD(I) can use to help implement this study’s

recommendations.

In summary, the decision maker must commit to using the OR results in support of

decisions to be made.    While it is not realistic to believe that an important ISR

investment decision will be driven solely by OR analytical results, it should be clear to

stake‐holders that the results were carefully considered in the decision‐making process.

The Task Force looks forward to discussing with the USD(I) and the ISR community its

recommendations and is grateful for this opportunity to serve the larger national

security community.

 

2 An in‐house OR function.  

OPERATIONS RESEARCH APPLICATIONS FOR ISR ___________________________________________________5
EXECUTIVE SUMMARY _____________________________________________________________________  

This page is intentionally blank.

6__________________________________ DEFENSE SCIENCE BOARD ADVISORY GROUP ON DEFENSE INTELLIGENCE
_______________________________________________________CHAPTER 1:  INTRODUCTION AND TASKING

CHAPTER 1:  INTRODUCTION AND TASKING

As a result of the recent High Value Targets (HVT) Study performed by Edge Consulting,

Inc. to examine the application of ISR assets in support of operations in the U.S. Central

Command’s Area of Responsibility (USCENTCOM AOR), senior leaders in the Office of

the Under Secretary of Defense for Intelligence (OUSD(I)), Program Analysis and

Evaluation (PA&E), and the Joint Staff engaged in a discussion about the merits of

operations research in support of planning, programming, and operations.3    This

discussion led to a request by the USD(I) to the Defense Science Board (DSB) to perform

a study on the use of Operations Research methods to support decision making within

the Department, with a focus on Intelligence, Surveillance, and Reconnaissance (ISR)

investment choices.    The DSB assigned this task to its Advisory Group on Defense

Intelligence (the Task Force).  The Terms of Reference (TOR)  directed the Task Force to

examine the manner and extent to which OR is employed by the Department of Defense

(DoD), how OR can be used to support ISR decision making, and the manner in which OR

can be institutionalized in the DoD.  The TOR included the following four tasks:  

1. Survey the departments and agencies to determine how much OR is being

performed;

2. Assess how the results of that OR are being used in decision making;

3. Based on these findings, recommend a test case for using OR methodologies

to improve DoD ISR investment decisions; and

4. Recommend steps the Department can take to “institutionalize” the use of

OR in DoD decision making in the future.  

The Task Force discussed the scope of the OR inquiry that would best serve the USD(I)’s

needs and considered limiting the inquiry to an examination of OR as a component of

technology and acquisition strategy.    The Task Force also considered extending the

inquiry to ways in which OR can be used to strengthen operational planning.  In the end,

the Task Force concluded that a broad inquiry was appropriate to the TOR tasks

provided by the USD(I).  Such an inquiry included examination and discussion of OR as a

component of operational planning, as an element in resource allocation in support of

operations, and as an element of acquisition and technology strategy pertinent to

supporting resource allocation and operational planning.  The Task Force believes this

broad‐based approach is particularly relevant to the ISR challenges facing the

Department now and in the foreseeable future.    Secretary of Defense Robert Gates

 

3 The Office of the Undersecretary of Defense (Intelligence), Program Analysis and Evaluation, and the Joint Staff co‐sponsored Edge

Consulting, Inc. to deliver a series of quantitative ISR performance analyses.    The Department's objective was to enrich the

standard methods ‐ interviews, surveys, war games ‐ with quantitative analysis of empirical operations and intelligence performance data.  

As one example, the Department tasked Edge with quantifying the contribution of a dozen Intelligence Surveillance and Reconnaissance

(ISR) systems to the special operations campaign against High Value Targets (HVT).  Working in close cooperation with the Department,

U.S. Central Command, and Special Operations Forces, Edge gathered over one million empirical records associated with thousands of

HVT operations.   Operations Research techniques were adapted and applied to quantify the contribution of ISR systems to successful

operations.  Causal factor analysis was applied to understand root causes of poor performance.  The Department and special operations

forces applied these results to adjust both acquisition and operational decisions.

OPERATIONS RESEARCH APPLICATIONS FOR ISR ___________________________________________________7
CHAPTER 1:  INTRODUCTION AND TASKING_______________________________________________________  

described some of these challenges – related to requirements of ever increasing

complexity and constrained resources – in a speech on September 29, 2008 at the

National Defense University.4

    His speech highlighted the challenges associated with

employing complex platforms and capabilities against an ever‐changing set of missions

and requirements.    These challenges apply clearly to current and prospective ISR

investments.

Given this complex environment, the Task Force believes that OR can strengthen end‐

to‐end analysis associated with coupling long‐term investment strategies, requirements,

operational concepts, and operational plans.  The Task Force received 22 briefings and

questioned representatives of the Services, various operational agencies, planning and

evaluation specialists from PA&E, and program management cadres within the DoD and

Office of the Director of National Intelligence (ODNI).  The Task Force also spoke with

industry representatives who provided useful insights regarding ways in which OR can

be used for strategic investment planning and in support of missions associated with

specific lines of business.  Appendix C lists the organizations that briefed Task Force.   

The Task Force believes its broad approach to the TOR tasking provides the USD(I) with

insights pertinent to a full range of ISR investment needs, ranging from long‐range

technology strategy to the efficient use of existing ISR resources.    In regard to

methodology, the Task Force focused principally on the resources available to conduct

OR, instead of reviewing specific programs, a task beyond the means and time available

for this effort.  The Task Force looked at the extent to which these resources reflected

the application of standards and the manner in which the results appeared to be

employed by the respective organizations.    This approach allowed the Task Force to

infer a lack of OR use and standard processes and practices in ISR investment decision

making.  Inquiries into specific programs and investments could more precisely establish

the manner and competence with which OR has been or is being employed.

 

4 As delivered by Secretary of Defense Robert M. Gates, Washington, D.C., Monday, September 29, 2008.    See:

http://www.defenselink.mil/speeches/speech.aspx?speechid=1279

8__________________________________DEFENSE SCIENCE BOARD ADVISORY GROUP ON DEFENSE INTELLIGENCE
_________________________________ Chapter 2:  Task Force’s Understanding of Operations Research

CHAPTER 2:  THE TASK FORCE’S UNDERSTANDING OF OPERATIONS RESEARCH

Joint Chiefs of Staff Publication 1‐02 (JCS Pub 1‐02), Department of Defense Dictionary of

Military and Associated Terms, defines Operations Research as “The analytical study of

military problems undertaken to provide responsible commanders and staff agencies

with a scientific basis for decision on action to improve military operations.”  Operations

Research may also be referred to as Operational Research or Operations Analysis.   

OR CAN SIGNIFICANTLY

INCREASE THE DECISION

MAKER’S INSIGHT AND

IMPROVE THE QUALITY OF

DEBATE AROUND  

KEY DECISION ISSUES.

The Task Force received numerous briefings from Operations Research (OR)

ractitioners from a variety of organizations.    Information

leaned from these briefings convinced the Task Force that

he application of structured decision making tools and

rocesses can provide significant assistance to the decision

aker in the analysis of complex operational and investment

ecisions.  OR can increase significantly the decision maker’s

sight and also improve the quality of the debate around

he key decision issues.

p

g

t

p

m    

d

in

t

A

 

s noted in Chapter 1, the Task Force’s understanding of OR

– decision support analysis based on information collected and analyzed using the

scientific method – broadly encompasses analysis of operations as well as strategic

technology, investment, and acquisition planning in support of operations.    OR

practitioners have a variety of tools at their disposal.    These tools consist of applied

mathematical techniques such as statistical and data analysis, linear and non‐linear

regression, queuing theory, stochastics, as well as linear and non‐linear programming.  

Additionally, practitioners have access to a variety of off‐the‐shelf models and

simulations, as well as commercial software packages.  For those problems where off‐

the‐shelf tools are not practicable, OR practitioners can develop models and simulations

using the techniques listed above.

It is the Task Force’s opinion that OR should be used as an important component of an

analytical approach to decision making.    Information provided to the Task Force by

briefers also assisted in formulating a picture of what good OR practices should entail.

2.1 Desired Attributes of Operations Research

The Task Force viewed OR as a scientific method; a method based on gathering

empirical and measurable evidence or data and subjecting it to repeatable principles of

reasoning.    These principles of reasoning often involve the formulation and testing of

hypotheses.    This process requires a thorough identification of the problem to be

solved, which includes advance agreement by decision makers on the objective

function,5 metrics, data provenance, and the study assumptions and boundary

5 The Task Force considered at length the manner in which valid objective functions could be employed in the OR analysis of ISR

investments.  We considered established “attrition models” by which the effectiveness of a system is assessed as a function of the extent

to which it contributes to the attrition of adversary forces, as well as the extent to which an ISR system can reduce the level of uncertainty

OPERATIONS RESEARCH APPLICATIONS FOR ISR __________________________________________________9
CHAPTER 2:  TASK FORCE’S UNDERSTANDING OF OPERATIONS RESEARCH__________________________________  

conditions – all of which are keys to solid analysis.    The best use of OR occurs when

employed by a team composed of operators and analysts capable of investigating the

entire problem and conducting analyses without bias.  In so doing, a good OR team will

seek agreement on assumptions and objective functions, and advocacy from all key

stakeholders up‐front, and continue to cultivate that advocacy throughout the analytic

process.

2.2   Advantages of Operations Research

Good analysis involves all decision makers and stakeholders from the very beginning.  

This involvement includes, but is not limited to, agreement on the actual questions to be

answered by the analysis as well as all assumptions and constraints pertinent to the

decision to be made.  Such involvement by senior leadership facilitates periodic reviews

and assessments of progress, and provides the opportunity to redirect or refine

guidance to the study team if needed.    A repeatable process based on the scientific

method enhances the likelihood that the analysis will be objective and the results will

provide the organizational leadership with the analytic underpinnings to make sound,

informed decisions.    Structured analyses will provide decision makers with clear

traceability from study requirements to final results.    In addition, analyses can often

illuminate problems previously unidentified or unknown to senior leadership.  

2.3 ISR Decisions to which Operations Research is Applicable

OR readily lends itself to assisting organizational leadership in making high‐value

decisions in which significant consequences and resources are at stake.  Such decisions

involve risk and uncertainty, and are often driven by complex requirements involving

multiple stakeholders with competing objectives.  These analytic processes can be used

across all ISR disciplines focused on a prioritized list of current and future threats and

missions.  Tailored approaches can be applied to the entire intelligence cycle including

system requirements analysis, asset tasking, data collection, processing and

exploitation, as well as intelligence analysis.

2.4 Conditions under which Operations Research can be Institutionalized

OR cannot be institutionalized without strong commitment, support, and advocacy from

senior leadership, which is assumed to have the authority to implement decisions

influenced by the OR process.  The senior leadership to which the Task Force is referring

includes those individuals responsible for the outcome of the decision process, that is,

those that will make the decision based upon various inputs.

This approach requires senior leadership to assemble and sustain a critical mass of OR

resources to include a well‐trained and adequately‐funded staff.  OR professionals need

career development and prospects.    The Task Force found noteworthy the approach

 regarding a question of intelligence interest.  The Task Force recognizes that establishing objective functions will continue to challenge

decision makers, although we believe good work has been done to establish Key Performance Parameters (KPPs) for many ISR systems.

10_________________________________ DEFENSE SCIENCE BOARD ADVISORY GROUP ON DEFENSE INTELLIGENCE
_________________________________ Chapter 2:  Task Force’s Understanding of Operations Research

employed by the Army (TRAC) in building and sustaining a cadre of competent OR

professionals.  

The essential requirement for effective OR in support of decision making is the support,

confidence, and early participation of the decision maker.    The decision maker must

ensure that the question asked, the assumptions made, the objective function defined,

and the metrics chosen all meet the needs of the decision process.   

A decision maker who demands good analytical support for decisions will get it.

OPERATIONS RESEARCH APPLICATIONS FOR ISR __________________________________________________11
CHAPTER 2:  TASK FORCE’S UNDERSTANDING OF OPERATIONS RESEARCH__________________________________  

This page is intentionally blank.

12_________________________________ DEFENSE SCIENCE BOARD ADVISORY GROUP ON DEFENSE INTELLIGENCE
___________________________________________ CHAPTER 3:  OPERATIONS RESEARCH –HISTORICAL CONTEXT

CHAPTER 3:  OPERATIONS RESEARCH – HISTORICAL CONTEXT

The Task Force examined, at a high level, OR from a historical perspective.    The Task

Force looked at OR in the context of its development and use principally in the U.S., with

an understanding that the OR discipline is not confined to the U.S.  This review of OR’s

historical context (from World War I to present day requirements and acquisition

processes) provided the Task Force with useful insights regarding the need for

standards, involvement of operators alongside analysts, and senior leadership

commitment.  

World War I

During World War I civilian scientists on both sides of the Atlantic joined together to

apply scientific methods to defeating the forces of the Triple Alliance, specifically the

submarine threat.  In 1915, after reading one of Thomas Edison's speeches on military

preparedness published in The New York Times, Secretary of the Navy Josephus Daniels

came upon the idea of the Naval Consulting Board.  After discussions with Edison, it was

determined that a board consisting of the brightest scientific and industrial minds in the

country would be convened under Edison's chairmanship.  Its primary purpose was to

advise the Navy on systems acquisition.    Edison, fully aware of the threat the U‐boat

posed to merchant shipping and Britain’s survival, focused his talents on developing

systems and tactics to be used in the fledgling anti‐submarine warfare (ASW) effort.  He

employed many applied mathematics techniques, such as time‐motion analysis to

understand the effects of zigzagging in disrupting fire control solutions; graphical and

statistical data analysis to understand U‐boat operations; threat system assessments;

and military wargaming.6

  That same year the British Admiralty instituted the Board of

Invention and Research to encourage the naval service to solicit expert technical

assistance in solving tactical problems.    Chaired by Sir Jackie Fisher, former First Sea

Lord, it was comprised of numerous scientists and divided into six science and

technology sections.  Of these, Section II: Submarines and Wireless Telegraphy, was the

most important.  In the two years of the Board’s existence, its members evaluated over

41,000 submitted inventions.7

British use of OR during World War II

Operations Research (Operational Research in the United Kingdom and Canada) was

formally established in the late 1930’s at a time when Adolf Hitler announced his

intentions to create a Luftwaffe equal to the combined air forces of Britain and France.  

To defend Britain against an attack from Germany, the British government established

radar installations along the coast to provide the Royal Air Force (RAF) with early

warning of incoming German strikes.  However, in 1938, tactical exercises demonstrated

that the system fell short of its operational requirements.  A.P. Rowe, Superintendent of

 

6 Scott, Lloyd N., Naval Consulting Board of the United States, Government Printing Office, Washington, D.C., 1920.

7 Hackmann, Willem, Seek & Strike: Sonar, Anti‐Submarine Warfare and the Royal Navy 1914‐54, Her Majesty’s Stationary Office (HMSO),

London, 1984.

OPERATIONS RESEARCH APPLICATIONS FOR ISR __________________________________________________13
CHAPTER 3:  OPERATIONS RESEARCH –HISTORICAL CONTEXT___________________________________________

the Bawdsey Research Station, proposed that research into operational aspects rather

than technical aspects of the system should be conducted – thus coining the phrase

“operational research.”8, 9

  These studies led to the establishment of formal techniques

of command and control of fighter versus bomber operations in the Battle of Britain.   

Operational Research spread to other organizations throughout the British military

establishment.    Operational Research sections in the Coastal Command and the

Admiralty were responsible for developing convoy procedures to keep Britain’s lifeline

to North America open, perfecting the use of airborne radar in detecting U‐boats, and

shutting down the U‐boat transit lanes from bases along the Bay of Biscay to open

ocean operating areas.10, 11, 12   Other sections were established to support the Army

(primarily the Antiaircraft Command), as well as RAF Bomber Command where studies

in navigation accuracy, radar performance, and the employment of chaff contributed to

improved bombing accuracy and aircraft survivability during the area bombing of

German cities.13  Recent studies in the United Kingdom indicate that the area bombing

campaign undertaken by Bomber Command in 1942, and espoused by the Commander‐

in‐Chief, Sir Arthur Harris, may have been validated by the misapplication of operations

research and was of less impact on the German war economy and civilian morale then

was believed at the time.14

U.S. use of OR during World War II

American scientists became involved in the war effort in June 1940 with the formation

of the National Defense Research Committee (NDRC) and the subsequent formation of

the Office of Scientific Research and Development (OSRD) under Dr. Vannevar Bush.15,

16    Like the Naval Consulting Board of World War I, both the NDRC and OSRD were

comprised of some of the brightest minds from academia and industry.  Upon the entry

of the United States into the war, the Navy took the lead in establishing operations

research organizations.   In March 1942 the Mine Warfare Operations Research Group

(MWORG) was established at the Naval Ordnance Laboratory.  MWORG went through

numerous reorganizations throughout the war, and ultimately supported the Army Air

Forces XXI Bomber Command in planning Operation Starvation in the Far East – the

 

8 Cunningham, W. Peyton, et al, “Of Radar and Operations Research: An Appreciation of A.P. Rowe (1898‐1976),” Operations Research,

Volume 32, Number 4 (July‐August 1984), pp. 958‐967.

9 Larnder, Harold, “The Origin of Operational Research,” Operations Research, Volume 32, Number 2 (March – April 1984), pp. 465‐475.

10 Falconer, N., “On the Size of Convoys: An Example of the Methodology of Leading Wartime OR Scientists,” Operational Research

Quarterly (1970‐1977), Volume 27, Number 2, Part 1 (1976), pp. 315‐327.

11 McCue, Brian, U‐boats in the Bay of Biscay: An Essay in Operations Analysis, National Defense University Press, Washington, DC, 1990.

12 McCloskey, Joseph F., “British Operational Research in World War II,” Operations Research, Volume 35, Number 3 (May – June 1987),

pp. 453‐470.

13 Ibid.

14 Kirby, M. and R. Capey, “The Area Bombing of Germany in World War II: An Operational Research Perspective,” The Journal of the

Operational Research Society, Volume 48, Number 7 (July 1997), pp. 661‐677.

15 Meigs, Montgomery C., Slide Rules and Submarines: American Scientists and Subsurface Warfare in World War II, National Defense

University Press, Washington, DC, 1990.

16 Fortun, M and S.S. Schweber, “Scientists and the Legacy of World War II: The Case of Operations Research (OR),” Social Studies of

Science, Volume 23, Number 4 (November 1993), pp. 595‐642.

14_________________________________ DEFENSE SCIENCE BOARD ADVISORY GROUP ON DEFENSE INTELLIGENCE
___________________________________________ CHAPTER 3:  OPERATIONS RESEARCH –HISTORICAL CONTEXT

mining of Japan’s Inland Sea.    During the campaign, more than 1,000 ships were

damaged or sunk, including over 100 naval vessels, greatly reducing Japanese shipping

in this area.17, 18

When the U.S. entered WWII, the loss of shipping to U‐boats in the Atlantic rapidly

reached catastrophic proportions.  In March 1942, Captain Wilder D. Baker, Commander

of Destroyer Squadron 31 formed the Anti‐Submarine Warfare Unit in Boston.  It did not

take the Unit long to conclude that the major cause of America’s poor ASW campaign

was inadequate tactical doctrine.    By the end of March, Baker approached Professor

Philip M. Morse of MIT and asked him to undertake the role of lead analyst in what

would be called the ASW Operations Research Group (ASWORG).  One of the first issues

ASWORG tackled was the problem of search, by ship or plane, for submarines and

warships.  Its first memorandum, published 1 May 1942, was titled “Preliminary Report

on the Submarine Search Problem.”    The recommendations in this report were

immediately put into effect and improved the tactics of convoy protection and the

search for U‐boats.    As successes grew, ASWORG gradually undertook studies in the

Pacific, first on submarine operations, then naval air activities, and finally on all aspects

of operations.19, 20  During this time Operations Research was successfully employed in

both the Army and Army Air Forces as well.21   

OR and the Cold War

After World War II, acceptance of Operations Research techniques became widespread

throughout industry leading to the establishment of professional societies in the United

States, United Kingdom, and Canada.  OR’s utility to military operations continued with

the introduction of Federally Funded Research and Development Centers (FFRDCs), first

with the Center for Naval Analyses, then Project Air Force, and then followed quickly by

Lincoln Laboratories.22  At the outset of the Cold War, Operations Research found new

missions, most notably in the development of the Single Integrated Operational Plan

(SIOP), a plan and planning process to determine the use of U.S. nuclear weapons in the

event of nuclear war.  The first such plan was submitted in 1960.  Operations Research

tools are still employed to study blast, thermal, and radiation effects, as well as mission‐

asset allocation.23  However, since 1983 this capability has diminished significantly.24

 

17 Shortley, George, “Operations Research in Wartime Naval Mining,” Operations Research, Volume 15, Number 1 (January‐February

1967), pp. 1‐10.

18 McCloskey, Joseph F., “U.S. Operations Research in World War II,” Operations Research, Volume 35, Number 6 (November – December

1987), pp. 910‐925.

19 Tidman, Keith R., The Operations Evaluation Group: A History of Naval Operations Analysis, Naval Institute Press, Annapolis, MD, 1984.

20 Morse, Philip M., “The Beginnings of Operations Research in the United States,” Operations Research, Volume 34, Number 1 (January –

February 1986), pp. 10‐17.

21 Shrader, Charles R., History of Operations Research in the United States Army, Volume I: 1942‐1962, United States Army, Washington,

DC, 2006.

22 U.S. Congress, Office of Technology Assessment, A History of the Department of Defense Federally Funded Research and Development

Centers, OTA‐BP‐ISS‐157, U.S. Government Printing Office, Washington, DC, 1995.

23 Pringle, Peter and William Arkin, SIOP, the Secret U.S. Plan for Nuclear War, Norton Publishing, New York, New York, 1983.

OPERATIONS RESEARCH APPLICATIONS FOR ISR __________________________________________________15
CHAPTER 3:  OPERATIONS RESEARCH –HISTORICAL CONTEXT___________________________________________

Also in the early 1960s, Secretary of Defense Robert McNamara, convinced that the

Secretary of Defense should control the evaluation of military needs, established the

Systems Analysis discipline as an administrative tool to estimate the required quantity

and performance of forces and weapons in relation to their mission and costs known as

the Planning, Programming and Budgeting System (PPBS).25  The effort was not entirely

successful – “McNamara’s systems analysis instilled structure and discipline, too, but in

the Pentagon, it was applied in an atmosphere of competition and distrust.    He

introduced systems analysis explicitly to obtain control over Service programs.    The

mutual trust and regard in the fleet were impossible in Washington.  Decision making by

analysis took on a hostile, jaded, and onerous cast.”26

Goldwater‐Nichols Act

In 1986, the Goldwater‐Nichols Department of Defense Reorganization Act called on the

Joint Chiefs of Staff to advocate the effectiveness of joint operations.    In so doing, it

tasked the Vice Chairman to chair a special council on military requirements known as

the Joint Requirements Oversight Council (JROC) to provide programmatic

recommendations to the Chairman of the Joint Chiefs for consideration and submission

to the Secretary of Defense.27  This Act positioned the Joint Staff to be the only military

body to address force integration and to resolve inter‐service conflict with a total

organizational perspective.  It assigned the Chairman of the Joint Chiefs of Staff (CJCS)

with determining strategy, military requirements, programmatic prioritization, and

budget recommendations.    The fact that the Joint Staff is comprised of military

personnel avoided the problems encountered with the McNamara PPBS.28    In 1994,

Admiral William Owens, Vice Chairman of the JCS established the Joint Warfighting

Capability Assessment (JWCA) process to provide better advice to the CJCS by increasing

the involvement of the Combatant Commanders and the Services in the JROC process.  

The JWCA process “examines relationships and interactions among warfighting

capabilities and identifies opportunities for improving joint effectiveness.”29  Each team

is comprised of representatives from the Unified Commands, OSD, and the Services and

is chartered with assessing areas where capabilities gaps or duplication may exist, as

well as determining where technologies may be brought to bear.  The JWCA findings are

then presented to the JROC for consideration.    The process has been instrumental in

providing a deeper analytic underpinning to the recommendations from the CJCS to the

Secretary.    

 

24 Technology Strategies & Alliances, Prepared for the Office of Net Assessment, Maintaining Advantage in a Multi‐polar Nuclear World:

Declining Resources Final Report, December 2007.

25 Enthoven, Alain and K. Wayne Smith, How Much is Enough?: Shaping the Defense Program 1961‐1969, RAND Corporation, Santa

Monica, CA.

26 Hughes, Wayne P., “Navy Operations Research,” Operations Research, Volume 50, Number 1 (January – February 2002), pp. 103‐111.

27 Owens, William A., “JROC: Harnessing the Revolution in Military Affairs,” Joint Forces Quarterly, Summer 1994, pp. 55‐57.

28 Turcotte, Dr. William E., “Service Rivalry Overshadowed,” Airpower Journal, Fall 1996, pp. 28‐33.

29 Statement of General Joseph W. Ralston, The Vice Chairman of the Joint Chiefs of Staff, to the Senate Armed Services Committee, 10

March 1999.

16_________________________________ DEFENSE SCIENCE BOARD ADVISORY GROUP ON DEFENSE INTELLIGENCE
___________________________________________ CHAPTER 3:  OPERATIONS RESEARCH –HISTORICAL CONTEXT

To bring the historical context up to date, there are two Major Acquisition Program

examples in which OR tools were used effectively.  These examples may be instructive

for ISR investment decision making.

Joint Direct Attack Munition Program

Operations Research techniques were the key element in the process that led to the

definition of the requirements and optimization of the trade space for the Joint Direct

Attack Munition (JDAM).

Following Desert Storm, a review of lessons learned identified the fact that adverse

weather or limited visibility of the ground caused by blowing sand, smoke from ground

fires, fog and cloud cover limited the employment of precision guided weapons.  These

same conditions also limited the delivery accuracy of unguided weapons when delivered

from medium and high altitudes.

Data from Desert Storm was analyzed by a team of operators and requirements officers

from the Tactical Air Command and the Strategic Air Command (later combined into Air

Combat Command).    This process was facilitated by Glenn Kent of the RAND

Corporation.    The team used optimization techniques to identify the needed key

characteristics of the new weapon and sensitivity analysis to identify the available trade

space during acquisition.  

The key characteristics included reduced sensitivity to weather, reduced crew workload

(laser weapons required human‐in‐the‐loop through most of the flight time), launch and

leave, and reduced cost over existing precision weapons.  Air Force Research Laboratory

personnel then joined the process and various guidance schemes were traded‐off

against the key characteristics before arriving at a guidance scheme which featured GPS‐

aided inertial guidance.

The end‐game set of requirements and technology trades defined an Adverse Weather

Precision‐Guided Weapon which became the JDAM following industry competition.

Joint Strike Fighter

Operations Research methodologies were critical to the Joint Strike Fighter (JSF)

Requirements Process, Technology Development Activity and affordability trades.  The

JSF program was initiated when OSD was forced to cancel three separate fighter

development activities with very diverse characteristics.   During the first phase of the

program, called the Joint Advanced Strike Technology Program (JASTP), a joint process

involving service war‐fighters and technologists was employed which identified the key

characteristics of a next‐generation strike system for the USAF, USN, USMC and Royal

Navy.

Structured analysis of the needs against scenarios defined by the Joint Staff led to a set

of key characteristics, associated sensitivity analysis, and a Concept of Operations.  

OPERATIONS RESEARCH APPLICATIONS FOR ISR __________________________________________________17
CHAPTER 3:  OPERATIONS RESEARCH –HISTORICAL CONTEXT___________________________________________

Industry primes then entered the process and began identifying design space around

the key characteristics.    System design attributes were then analyzed against the key

characteristics and cost and displayed in carpet plots which allowed senior Service and

OSD decision makers to understand cost as an independent variable.   

This process resulted in the Services coalescing around a set of requirements that

allowed satisfaction through a “Family of Airplanes” concept that allowed for a high

degree of commonality and program affordability.  

As the industry prime contractors refined their design concepts, they were evaluated by

service warfighters and maintainers in simulation activities conducted by the Program

Office.    This simulation used several core tools provided by and maintained by the

Program Office.    This provided configuration control and increased confidence in the

simulation results.  

As the program entered the System Design and Development (SDD) phase, this

simulation activity and the carpet plots provided decision makers the tools to continue

to make system trades for program affordability.  

3.1   General Lessons Learned from Historical Experience

There are two important lessons to be learned from the historical examples cited above.  

The first lesson is that, for Operations Research to have a positive impact, the analyst

should be imbedded in or closely connected to the decision maker’s organization.   In

1941, in a note prepared by Professor P.M.S. Blackett entitled “Scientists at the

Operational Level,” he stated, “The object of having scientists in close touch with

operations is to enable operational staffs to obtain scientific advice on those matters

which are not handled by the Service technical establishments.”    In addition,

“Operational staffs provide the scientists with the operational outlook and data.    The

scientists apply scientific methods of analysis to these data, and are thus able to give

useful advice.”30   

While outstanding work was done by organizations on both sides of the Atlantic during

World War I, they were never accepted by the Services they were established to assist.  

In the case of the Naval Consulting Board, they failed to have any great effect on the

conduct of naval operations because they were a civilian organization that reported

directly to the Secretary of the Navy.  Not once throughout the course of the War did

the Board establish contact with the operational commanders in the field.31

The widespread success of operations research in World War II can be attributed to the

close relationship the analyst had with the warfighter.  In most cases, the analyst was

deployed to support a specific command in a specific theater of operations.  Data were

readily available at the source, and interactions with staff personnel enabled the analyst

 

30 Air Ministry, The Origins and Development of Operational Research in the Royal Air Force, Her Majesty’s Stationary Office, London, 1963.

31 Whitmore, W.F., “Edison and Operations Research,” Operations Research, Volume 1, Issue 2 (February 1953), pp. 83‐85.

18_________________________________ DEFENSE SCIENCE BOARD ADVISORY GROUP ON DEFENSE INTELLIGENCE
___________________________________________ CHAPTER 3:  OPERATIONS RESEARCH –HISTORICAL CONTEXT

to personally demonstrate the utility of operations analysis through near‐immediate

results.

During the 1960s, McNamara’s “Whiz Kids” failed to take notice of the successes

enjoyed by Operations Research groups during World War II as a result of their close

alignment with the decision maker – an alignment not present with operators – thereby

alienating the Services.  The result was Service “stiff‐arming” of the PPBS process and its

ultimate failure.   With the JROC and JWCA processes, control of the analytic process,

regardless of its rigor, has been returned to the DoD.   

The second lesson to be learned is that Operations Research can be misused if the

analysis team is not free from bias.  The case in point is the deliberate use of analysis to

support “Bomber” Harris’ position to continue area bombing of German cities during

World War II rather than redirecting the forces at his disposal to conduct precision

bombing of military targets (airfields, shipyards, U‐boat pens, etc.).    As early as the

autumn of 1944, Strategic Bombing Surveys were initiated by both the United States

and Britain.    These surveys revealed that neither civilian morale nor the German

economy were as severely impacted as the Bomber Command Operational Research

Section (ORS) claimed.  In fact, it is interesting to note that British analysts stated that

the effects of the London Blitz were not long‐lasting.  Analyses conducted by the ORS

were biased because the analysts were influenced by Harris’ strong personality and the

refusal to depart from pre‐war concepts derived from the Japanese attack on Shanghai

in 1931 and the obliteration of Guernica during the Spanish Civil War by the German

Condor Legion.

OPERATIONS RESEARCH APPLICATIONS FOR ISR __________________________________________________19
CHAPTER 3:  OPERATIONS RESEARCH –HISTORICAL CONTEXT___________________________________________

This page is intentionally blank.

20_________________________________ DEFENSE SCIENCE BOARD ADVISORY GROUP ON DEFENSE INTELLIGENCE
_______________________________________Chapter 4:  Relevance of Operations Research to ISR

CHAPTER 4:  RELEVANCE OF OPERATIONS RESEARCH TO INTELLIGENCE,

SURVEILLANCE, AND RECONNAISSANCE

The Task Force was tasked by the USD(I) to examine how OR can be used to support ISR

decision making.    Although OR may have wider applicability in broader investment

decision making, the focus on ISR is driven by several factors that make it both relevant

and challenging.

A more consistent scientific and quantitative approach to evaluation could assist the

USD(I) and the Director of National Intelligence (DNI) with many key investment issues.  

Below are a few examples of past and current issues that illustrate the trade‐offs that

can be illuminated by the proper application of OR.

4.1 Current ISR Issues that can Benefit from the Application of OR

In the current conflicts, full motion video (FMV) collection has proven to be a very

valuable intelligence input.    However, studies suggest that a large number of UASs,

bandwidth and exploitation resources would be required to meet all the anticipated

requirements for FMV.    A program known as the Wide‐Area Airborne Sensors is

designed to collect a wider field persistent view of the battle space as a way of capturing

information.  Understanding the cost/benefit of these programs and their optimum mix

would be very valuable at this juncture.

With the focus on current operations, resources allocated to science and technology

(S&T) collection and exploitation have been reduced.    S&T data is essential for

identifying new adversary capabilities and determining approaches to collect against

them.    The lack of S&T data creates vulnerabilities in the United States’ ability to

counter new threats and collect information against adversaries when they are using

new technologies.    The key to making acquisition decisions is understanding the

appropriate balance between S&T and current collection capabilities to ensure a

continued ability to adapt and understand new threats.  Acquisition delays and failing

national systems programs require key investment decisions.    These decisions weigh

trading lower risk and lower capability short term solutions against higher risk and

higher capability longer term solutions.  Prudent investment decisions should be based

on the optimization of factors such as how UASs and manned aircraft can work with

national systems to provide support to national and military needs over the next 10

years.

In the Global War on Terror, information about individuals can be more important than

information on military equipment.  This has given rise to the desire to collect and use

biometric data on the battlefield.  However, assessing the value of this information and

how it will be used versus the value of other kinds of collection will be important.  In

addition, as investments in biometrics increase, OR should prove helpful in ensuring that

investments are made in a balanced fashion across the entire intelligence cycle (i.e.,

requirements, tasking, collection, processing, exploitation, dissemination).  

OPERATIONS RESEARCH APPLICATIONS FOR ISR __________________________________________________21
CHAPTER 4:  RELEVANCE OF OPERATIONS RESEARCH TO ISR ___________________________________________  

22_________________________________ DEFENSE SCIENCE BOARD ADVISORY GROUP ON DEFENSE INTELLIGENCE

4.2 Past ISR Investment Issues that have Benefited from the Application of OR

THE SUCCESSFUL USE OF  

OPERATIONS RESEARCH

REQUIRES SOUND

ASSUMPTIONS

The requirements process for the Future Imagery Architecture (FIA), the Integrated

verhead SIGINT Architecture (IOSA), the recent U‐2 /

lobal Hawk trade‐off decision, and the aforementioned

VT Study employed OR in a significant manner.   In all of

hese cases, there was a clear mandate from decision

akers for the use of OR, general consensus in advance on

he objective function, a broad base of community

nvolvement, and sound methodology.    The Task Force

otes, however, that the use of OR cannot guarantee

successful program implementation, particularly if technical capabilities for

implementation assumed to exist during analysis are not present during design and

development.  As always, the successful use of OR requires sound assumptions.   

O

G

H

t

m

t

i

n

4.3 Complications in Applying OR to ISR Investment Issues

ISR programs are complicated by several factors that make investment decision making

difficult.  

To add value for a decision maker, information has to be collected, processed, exploited

and disseminated.    Typically the programs that operate in each of these stages are

controlled by different organizations.  Thus, in evaluating an investment decision and its

impacts, the OR analyst must be careful to look across program elements and across

organizations.   In many cases, the funding streams to support a program may include

both Military Intelligence Program funding and National Intelligence Program funding.  

The first is controlled by the Secretary of Defense under Title 10 authority, the latter by

the DNI under Title 50 authority.

Most ISR systems are used to support a broad variety of missions.    For example, the

same UAS may be tasked to support force protection, battlespace awareness, and

targeting.  National systems are often used to support decisions ranging from national

strategy and policy to striking individual tactical targets.    Evaluating the value of a

system requires looking at its impact on all of the missions it might support.

Most major ISR systems are complex and expensive, which generally results in both a

long development time and a long design life.    These factors often result in an ISR

system operating for more than 20 years after the requirements for the system were

initially developed.  During that time period, threats may change substantially and the

types of uses for which the system was initially designed may become less relevant.  To

make effective investment decisions throughout the life cycle of ISR assets requires a

broad view of what the future may hold so that the system can be evaluated against all

potential future uses.
_______________________________________Chapter 4:  Relevance of Operations Research to ISR

4.4 Challenges in Applying OR to ISR Issues

The proper use of OR to examine ISR investment decisions objectively can address many

of the complications identified above.  However, applying OR to ISR issues has several

key challenges.  The primary challenge is to answer the question: “What is a pound of

ISR worth?”32    That is, what is an objective measure of the value of ISR information?  

Unlike lethal weapon systems, ISR does not directly destroy enemy targets.  Instead, it

produces knowledge that informs decision makers as they make a broad range of

decisions.  This issue makes it difficult to agree on the objective function against which

OR is performed.    To evaluate what this knowledge is worth requires some

understanding of how decisions are made and what contribution ISR provides to those

decisions.    Although there are no universally‐accepted measures that have been

developed to address this issue, there are some approaches that have been used to

measure the impact of a specific ISR contribution in specific circumstances.  

The most common ISR metric used for national systems evaluation is “User

Satisfaction.”  User Satisfaction allows analysts to quantify the impact of an ISR system

by measuring the fraction of collection requirements it is able to meet.    Using this

metric, trade‐off decisions can be made between architectures or systems by assessing

the amount of “User Satisfaction” they produce versus the cost of the system.  

Unfortunately, this approach is heavily dependent on the validity of the requirements

provided, and it assesses only the relative value – not the absolute value – of the

systems being traded.   That is, it does not identify how much is enough or the direct

impact of this information on outcomes.    It is also hard to evaluate systems from

different collection disciplines, since user requirements are generally specified by

discipline.

Other OR approaches attempt to characterize the effect of ISR on outcomes by trying to

quantify the effect of ISR on attrition of enemy forces, a traditional measure of

effectiveness.  There have been many different ways of addressing this challenge, with

most suffering from strong limitations.  Typically the tactical effect of finding, classifying,

and locating a target can be related to the rate and probability of killing the target.  

These “sensor‐to‐shooter” models capture some of the tactical impacts of ISR, but do

not capture the value of situational awareness or the value of ISR in non‐combat

situations.  

Attempts to capture the wider value of ISR are often stymied by the view held by many

practitioners that ISR is a craft that is too complex to be characterized.  This view makes

it difficult to address quantitative measures for system performance.   As a result, the

community will have to continue to study the impact of ISR on decision making and

outcomes to better characterize its value.    The development of community accepted

metrics and the agreement by decision makers on objective functions in advance of an

OR effort is essential to the successful application of OR.

 

32 As quoted by the Honorable James R. Clapper, Jr., Under Secretary of Defense for Intelligence.  

OPERATIONS RESEARCH APPLICATIONS FOR ISR __________________________________________________23
CHAPTER 4:  RELEVANCE OF OPERATIONS RESEARCH TO ISR ___________________________________________  

This page is intentionally blank.

24_________________________________ DEFENSE SCIENCE BOARD ADVISORY GROUP ON DEFENSE INTELLIGENCE
_____________________________ CHAPTER 5:  CURRENT OPERATIONS RESEARCH PRACTICES (TOR TASKS 1 AND 2)

OPERATIONS RESEARCH APPLICATIONS FOR ISR __________________________________________________25

CHAPTER 5:  CURRENT OPERATIONS RESEARCH PRACTICES (TOR TASKS 1 AND 2)

Given the Task Force’s understanding of OR, its attributes, the conditions under which it

can be employed successfully, and in response to TOR Tasks 1 and 2, the Task Force

looked at several specific examples of OR from a broad array of organizations across the

Department of Defense, the Intelligence Community (IC), and industry.  Given the time

available to the Task Force, it was not possible to explore the entirety of OR and OR‐

related activities in the Department and the IC.  However, the representative sampling,

while not exhaustive, provided the Task Force with a good opportunity to assess the

extent to which OR is institutionalized, how it is being employed, and how its

employment might be improved.    This approach should allow the USD(I) to infer the

extent to which OR is being used to support ISR investment decisions.    A list of

organizations that made presentations is located in Appendix C.   

...THE APPLICATION OF OR IS

GENERALLY UNCOORDINATED,

SOMETIMES ORGANIZATIONALLY

BIASED, INCONSISTENTLY

RESOURCED, AND NOT WELL

INTEGRATED...

The Task Force received presentations from entities from each of the Services, from the

National Geospatial‐Intelligence Agency (NGA), the National Security Agency (NSA), the

National Reconnaissance Office (NRO), ODNI, OSD PA&E, and others, including industry.  

Edge Consulting, Inc. provided a briefing on their ongoing work regarding ISR in support

of the search for High Value Targets.  The Navy has pockets of excellence where OR is

conducted, and most importantly has a strong curriculum for OR at the Naval

Postgraduate School (NPS).    The Army, which has made the most consistent

ommitment to institutionalizing OR, uses the NPS

o great advantage to educate their OR workforce.  

he Air Force also maintains an OR degree program

t the Air Force Institute of Technology (AFIT).  The

ost robust OR at the OSD level is performed by

A&E.    Of note, it is now almost exclusively

utsourced ith little internal pabili .    The

ntelligence Community brought multiple examples

o the Task Force of the ongoing decline in OR

in support of planning, programming, and

materiel acquisition.    FedEx presented a compelling example of a model used in the

private sector for OR institutionalization and application.    Finally, the expertise

embodied in the MORS, a professional organization dedicated to defense‐related OR,

represents a unique resource for the Department.

c

t

T

a

m

P

o w ca ty

I

t

capability

From an overarching perspective, there are some findings that became clear to the Task

Force.  While OR is used widely across the Services, especially in support of operations,

the application of OR is generally uncoordinated, sometimes organizationally biased,

inconsistently resourced, and not well integrated into processes and programs.  There is

no strong central champion for OR at the Department level, and therefore there is no

clear ability to identify and support best practice transfer, or to capture, document, or

share OR lessons learned.   Consequently there is not a jointly accepted OR review or

accrediting organization.   While Federal Acquisition Regulations require OR for Major
CHAPTER 5:  CURRENT OPERATIONS RESEARCH PRACTICES (TOR TASKS 1 AND 2)_____________________________  

Acquisition Programs, the application of OR in response to this requirement is

inconsistent in level of effort and quality, and often requires rework when submitted for

OSD review.

Efforts to support and resource OR have generally been in decline in the Department,

but this trend is even more stark in the Intelligence Community.    In the cases of the

NGA, NSA, and the NRO, it was clear that OR capabilities for programming and

acquisition decision making are atrophying rapidly.    The Task Force did note that in

contrast, these same agencies use significant and effective OR in support of current

operations.   Notably, there is a nascent analytic capability in the Defense Intelligence

Operations Coordination Center (DIOCC), and while its parent organization, the National

Intelligence Coordination Center (NIC‐C) was originally intended to have a strong OR

capability, that effort has not been resourced.    Finally, there is no evidence of OR

leadership or direction from the staff of the Director of National Intelligence, nor any

significant OR capability noted on that staff.    The recent decision to terminate the

Integrated Concepts Development Office (ICDO) is further evidence of the apparent

decline of interest in OR at the ODNI.

Of the many presentations given to the Task Force, two stood out in a positive manner.  

First, the Army has continued to support a historically strong OR program.   The Army

has developed and sustained an officer career field for OR which includes opportunities

for advanced degrees and promotion.  The Army has continued to support institutions

that promulgate OR best practices, including the Army Training and Doctrine Command

(TRADOC) Analyses Center (TRAC).    The Army has traditionally maintained a strong

champion on the Army Staff with direct access to the most senior leadership.  The Army

uses OR both in support to current operations, (with deployed OR specialists embedded

with units in theater) as well as to support strategic planning, programming, materiel

acquisition, and the generation of requirements.

Second, a model which may have direct applicability to the Department is the

application of OR in the private sector, as evidenced by FedEx.  FedEx has a strong and

consistent commitment to OR, and uses OR extensively in its decision making, both for

daily operations and long‐term strategic planning.  The FedEx corporate model is a very

small (approximately 200 people) headquarters overseeing multiple robust independent

operating units.    FedEx has a hybrid model for OR which is both centralized and

distributed.  A small OR group of approximately eight to ten people provides corporate‐

level OR support, and each of the line operations maintains their own OR entities.  OR is

valued as part of the culture at all levels of the organization, but also seen through a

pragmatic lens.   While OR is often used to great advantage, OR entities within FedEx

have no line authority.  The leader of the corporate‐level OR organization made it clear

that in no case would OR be utilized if there was not clear leadership support, and

articulated that even the most well‐executed OR would be of no value if the decision

maker did not support it.

26_________________________________ DEFENSE SCIENCE BOARD ADVISORY GROUP ON DEFENSE INTELLIGENCE
_____________________________ CHAPTER 5:  CURRENT OPERATIONS RESEARCH PRACTICES (TOR TASKS 1 AND 2)

OPERATIONS RESEARCH APPLICATIONS FOR ISR __________________________________________________27

THERE IS NO CHAMPION ADVOCATE

FedEx uses OR at the corporate level to determine which lines of business are

appropriate, how much capability is required to support each line of business, and what

FedEx should be prepared to invest in support of those lines of business.    The FedEx

corporate OR organization does not provide decision support at the operational level

within the separate lines of business, but does directly support decision making for the

C‐level (CEO, CFO, CIO, etc.) corporate leadership, and can assist the C‐level leaders with

OR‐related expertise as they review the analytics submitted by the operating units.  The

corporate OR entity also provides some measure of oversight to the practice of OR in

the operating units to ensure quality and consistency.  Within the line operating units,

OR is used to optimize efficiency, to test alternative operating concepts, to guide

nvestment riorities, and to support other

ecisions as management deems necessary.   

i p

d

It

s

i

c

d

m

r t i

a

IN THE DEPARTMENT, NO COHERNT

STRATEGY, AND NO ORGANIZATION

OR STAFF ELEMENT IDENTIFIED TO

PROVIDE LEADERSHIP IN THIS

DOMAIN.

was clear to the Task Force that OR is used, in

ome form or another, in every Service and the

ntelligence Combat Support Agencies.    It was also

lear that save for a few “points of light,” OR is in

ecline and not universally valued in decision

aking, especially at the strategic level, and with

espect o nvestment decisions and systems

cquisition.   There is no champion advocate in the

Department, no coherent strategy, and no organization or staff element identified to

provide leadership in this domain.  Moreover, while the IC conducts truly world class OR

in determining the daily allocation of scarce collection resources against myriad

increasing requirements, there has been an astounding decay in the ability of these

same agencies to conduct OR in support of planning, programming, and acquisition.  In

the case of both the Army and FedEx, however, the Task Force identified some

significant best practices that may have immediate use as models for a way ahead in OR

to support ISR investment decisions.
CHAPTER 5:  CURRENT OPERATIONS RESEARCH PRACTICES (TOR TASKS 1 AND 2)_____________________________  

This page is intentionally blank.

28_________________________________ DEFENSE SCIENCE BOARD ADVISORY GROUP ON DEFENSE INTELLIGENCE
____________________________________________ CHAPTER 6:  ISR TEST CASE(S) ATTRIBUTES (TOR TASK 3)

CHAPTER 6:  ISR TEST CASE(S) ATTRIBUTES (TOR TASK 3)

In approaching TOR Task 3, which is to recommend an ISR test case in which OR might

be used to reach a better decision, the Task Force found it useful to examine the

attributes that possible test cases should possess.  It is unlikely that every possible test

case will be characterized by every attribute discussed herein, but the Task Force is

persuaded that successful cases should be characterized by all or most elements

described in the sections below.  

The Task Force considered several test cases and believes two cases in particular would

serve the USD(I) in establishing ways to employ OR effectively in support of ISR.  These

cases are:

• Biometrics technology investment and architecture

• Balance intelligence cycle investment  

These test cases are discussed in more detail in Appendix D.

6.1 Importance   

The Task Force considers it critical that potential test cases be viewed by ISR decision

makers as involving issues of great importance – certainly to the ISR community and

especially to USD(I) and DNI leadership.  If the issue(s) chosen to examine with OR do

not meet the common sense test of importance, they are unlikely to receive the kind of

leadership “buy‐in” and resource support that will be required for success.  While it is

true that the Task Force believes that OR could significantly contribute to ISR decisions

for a multitude of operational, architectural, and investment or acquisition issues, we

believe that it is the latter area (investment/acquisition of significant resources) that will

provide the most unique benefit to the USD(I) and DNI missions.   

6.2 Leadership Commitment  

Closely related to, and even driven by, the importance attribute described above, is the

issue of leadership commitment.  If any selected test case is to successfully demonstrate

the value of OR in ISR decision making, USD(I) (and probably DNI) leadership must be

fully committed to the case and the process.    The following elements of leadership

commitment to the test case must be present and evident:

1. Leadership must ensure that stakeholders agree to the terms of reference

and assumptions for the test case.    It is critical that the questions to be

answered and assumptions to be used are clear and properly framed.   

2. The test case sponsor must commit to providing the resources necessary to

seeing the test case through to completion, and should provide a “time‐

certain” date when the project is to be completed if it is to have value in

decision making.   

OPERATIONS RESEARCH APPLICATIONS FOR ISR __________________________________________________29
CHAPTER 6:  ISR TEST CASE(S) ATTRIBUTES (TOR TASK 3)_____________________________________________  

3. The test case sponsor must commit to using the OR results in support of the

decision.  While it is not realistic to believe that an important ISR investment

decision will be driven solely by the OR analytical results, it should be clear to

stake‐holders that the results were carefully considered in the decision‐

making process.    This is particularly true if, for some reason, the ultimate

decision diverges significantly from the direction suggested by the OR

analysis, and it would be helpful if the decision maker provided feedback as

to how the OR analysis was used and the degree to which it was helpful

and/or could have been more useful.

6.3 OR Process Credibility   

All decision makers and stake‐holders involved with the chosen test case must be

convinced of the credibility and integrity of the OR process, even if they ultimately might

disagree with the results.  This means that everything used in the test case OR process

must be credible and transparent – terms of reference, questions to be answered, data

to be used, OR tools to be applied, OR practitioners to be utilized, and the decision(s)

intended to be supported by the process.    In addition, decision makers and stake‐

holders must agree in advance on the objective functions against which the analysis is

conducted.  If the initial test case(s) conducted do not “ooze” credibility, the prospects

for the institutionalization of OR for making ISR investment decisions are slim.  

6.4 Test Case Scope   

The issue of test case scope is a challenge.  The Task Force has already highlighted the

attribute of importance, but to be important, the test case does not have to be “huge” –

especially in terms of scope.    The broader the scope the more difficult it will be to

achieve stakeholder “buy‐in” on the credibility of the process and its contribution to

good decision making.  In this regard, an optimal test case may be one that employs OR

to examine the potential impacts of decisions that are already being contemplated or

nearing “finality,” which by definition are likely to meet the “importance” test already

described.    Generic examples of these kinds of “impact” examinations are provided

below:

1. Examining the impact of a contemplated ISR investment decision on its

applicability against a range of threats, current and future.    For example,

assume that a costly investment in additional high altitude/long endurance

UASs equipped for imagery and full motion video is contemplated primarily

for GWOT, and specifically for OIF and OEF, and that the coverage factor and

output of those UASs is credibly quantified and modeled.  An OR examination

of the contemplated investment could be conducted in other scenarios – for

example a “defense of Taiwan” or “conflict with Iran.”  In this example, the

need for the UASs for GWOT may be the overriding requirement, but one

could definitively analyze and understand in a credible OR process whether

the same UASs equipped with the same sensors will be applicable to the

30_________________________________ DEFENSE SCIENCE BOARD ADVISORY GROUP ON DEFENSE INTELLIGENCE
____________________________________________ CHAPTER 6:  ISR TEST CASE(S) ATTRIBUTES (TOR TASK 3)

other scenarios and consider future alternate utility (either positive or

negative) into the investment decision.   

2. Examining a contemplated ISR investment decision in a specific new

discipline for its potential impact (specifically cost and capability) on other

parts of the relevant intelligence cycle.  For example, consider the impact of

a costly investment in biometric data collection capability.   OR techniques

could be used to analyze the capability of the entire ISR system to effectively

use the contemplated new collection capability and/or understand what

additional costs would have to be incurred.  An obvious case in point would

be whether or not appropriate investment has been made in the analytical

resources (specific skills, recruitment and training) and dissemination

capability needed to handle the volume of new product that would be

produced by a UAS investment.  Additionally, in an area like biometrics, are

there legal or privacy impediments that make the collection capability more

difficult or more expensive to direct, task, and use, and have the costs of

these factors been credibly analyzed and considered?

3. Examining the impact of new war‐fighting capabilities on the ability of the

ISR system to support them.    A notable historic instance of this was the

impact on the ISR system of fielding large numbers of precision munitions

since Desert Storm and the continuing impact of fielding “extreme precision”

systems today.  For example, GPS‐guided weapons are only as precise as the

quality of the Digital Terrain Elevation Data (DTED) that is collected and

analyzed to provide exact target coordinates.    The demand for

extraordinarily detailed and high‐quality imagery to support precision

application of military capability (strike or SOF) continues to grow

significantly.   Can OR be used to understand and plan for new weapons or

strike capabilities on the horizon that will produce new quality or volume

demands on the ISR system?    One possible example of this opportunity

would be the extension of precision strike into ground and naval force

artillery.    The dramatic increase in precision strike potential brought on by

such an exponential increase in precision weapons would potentially bring

about a similar exponential increase in the need for highly accurate target

geographic coordinates and/or high quality target imagery.   OR techniques

seem ideally suited to examine this critical question.

4. Examining different intelligence disciplines/portions of the intelligence

cycle to determine where  ‐  for a given problem  ‐  investment of limited

resources can provide the most effective and efficient prospects for mission

satisfaction. Since this type of examination is often hindered by the fact that

resource sponsors or political proponents are aligned parochially by

intelligence discipline, organizational interests, budget authorities, or

constituent concerns versus mission optimization; unbiased OR using good

quality data and credible analytical techniques could make a significant

contribution.    Even if the challenges were such that OR did not “drive the

OPERATIONS RESEARCH APPLICATIONS FOR ISR __________________________________________________31
CHAPTER 6:  ISR TEST CASE(S) ATTRIBUTES (TOR TASK 3)_____________________________________________  

decision,” it is still possible that applying good OR techniques could improve

the quality of the debate and create the environment for a more favorable

outcome.    Examples of the kind of problems are numerous: HUMINT vs.

SIGINT as the route toward discovering enemy intentions, faster/broader

dissemination vs. more imagery collection, and/or the value of more

investment in analysis vs. just adding collection capability.

6.5 Test Case Timing and Duration   

While not as critical as the other attributes, the Task Force considers it optimal if the

first test case(s) could begin soon, address a near term issue, and be completed

relatively quickly.  It will obviously create more IC interest if the OR process dealt with

issue(s) that are currently important and relevant to ongoing resource or operational

deliberations.  Similarly, if the first test case(s) were suitable for a credible OR analysis

that could be completed in three to six months it would be helpful in getting an early

determination of the value of the OR process for ISR decision support.      

6.6 A Note on ISR Test Case Challenges   

While the Task Force believes that the ISR test case attributes cited above are valid, it is

under no illusion about the difficulties of applying OR in the ISR community.  OR is hard

for ISR and challenges abound:

1. Typically, models exist primarily for collection systems, making the “SR”

(surveillance and reconnaissance) part of the ISR acronym much easier to

model and measure than the “I” (Intelligence).   In fact, probably the single

biggest challenge for using OR for ISR investment decision support is the fact

that we have never been able to successfully measure or model the value of

the product of the intelligence process – knowledge about an enemy’s

capabilities and intent.    In short, the impact of intelligence on decision

making is not easily measured or understood.

2. ISR platforms are often multi‐purpose, making their total output and relative

value difficult to measure, model, and compare.

3. The ISR community lacks analytic models that reflect the utility of ISR product

assessment.

4. Funding and control streams are in different organizations, and often in

different departments of government.

5. The same ISR platforms and agencies often serve many different customers

whose requirements range from tactical through strategic, and whose

missions can range from combat to operational planning or national level

policy.

32_________________________________ DEFENSE SCIENCE BOARD ADVISORY GROUP ON DEFENSE INTELLIGENCE
___________________________________CHAPTER 7:  INSTITUTIONALIZING OPERATIONS RESEARCH (TOR TASK 4)

CHAPTER 7:  INSTITUTIONALIZING OPERATIONS RESEARCH (TOR TASK 4)

The Task Force looked at a number of current OR and decision‐support institutions in

use by the Services, DoD, the IC, and industry.  The Task Force found that OR, systems

analysis, and other decision‐support disciplines (including modeling and simulation) are

used widely in the Services and DoD, as well as the IC.  However, the application of OR

(and related disciplines) is uneven in these organizations, as is the extent to which OR is

recognized as a distinct discipline and career field.  Some organizations (e.g., the NRO)

use OR to optimize the use of current ISR resources to support specific mission

requirements, while others (e.g., the NSA) employ modeling and simulation to support

architecture analysis.  The former example represents robust and sustained use of OR,

albeit without strong coupling of the resources of OR analysis to strategic investment

decision making.    The latter example typifies a situation in which structured decision

making has fallen into disuse and in which its practitioners struggle to find opportunities

to apply their discipline.    Of note, the Army has done the most to sustain an

institutionalized OR capability, one that is well instantiated at the TRAC.  TRAC’s Website

states:

“TRAC conducts operations research (OR) on a wide range of military

topics, some contemporary but most often set 5 to 15 years in the future.  

How should Army units be organized?    What new systems should be

procured?  How should soldiers and commanders be trained?  What are

the costs and benefits of competing options?  What are the potential risks

and rewards of a planned military course of action?    TRAC directly

supports the mission of the Army’s major command, the Training and

Doctrine Command (TRADOC), to develop future concepts and

requirements while also serving the decision needs of many military

clients.  TRAC analysis enables change.”33

While the Task Force did not assess in detail the extent to which TRAC is performing

successfully in all of the roles described above, our discussions allowed us to conclude

that TRAC is comprised of dedicated professionals who enjoy leadership commitment

and support and whose work is visible and highly regarded in the Army planning and

operational communities.

As described in Chapter 5, the FedEx example also represents a seemingly robust

approach to institutionalizing OR and structured decision making.  This model employs

OR at a corporate level to help decide on the lines of business in which the company

should compete, and at what level.  FedEx also employs OR at the line‐of‐business level

to support investment decision making and to configure operational resources; it

provides corporate oversight to ensure the presence of consistent OR standard

processes and methodology throughout the FedEx family of businesses.  The company

 

33 http://www.trac.army.mil/  

OPERATIONS RESEARCH APPLICATIONS FOR ISR __________________________________________________33
CHAPTER 7:  INSTITUTIONALIZING OPERATIONS RESEARCH (TOR TASK 4)___________________________________  

does not “require” the use of OR so much as the leadership commitment of the

company to OR‐based decision making gives its practitioners substantial influence as

high value decisions are made.  That leadership extends to empowering the company’s

corporate OR organizational to ensure that appropriate professional standards,

processes, and practices are employed in the conduct of OR throughout the company’s

various lines‐of‐business.

Four common elements unite the Army and FedEx cases.

• The OR professionals in both organizations enjoy leadership commitment

and support.  In both organizations, senior leaders employ the results of OR

in a visible fashion to support key strategic decisions.    In the FedEx case,

discussions regarding important business decisions are expected to be

supported by corporate or line‐of‐business specific OR analyses.  

Recommendations not supported risk being dismissed.

• In both the Army and FedEx, OR is regarded as a discreet career field.   OR

professionals are recruited, and in the case of the Army, developed.    OR

professionals in both organizations enjoy respect; career prospects; and the

opportunity to develop, refine, and demonstrate their capabilities.    In the

other Services, the Task Force found that a career in Operations Research can

lead to careers removed from the mainstream resulting in diminished career

prospects.

• The Army and FedEx use OR broadly.   The TRAC mission unites investment

decision making with operational concepts.  While many companies employ

OR in decision making, the Task Force found the FedEx model very relevant

to the functioning of DoD.  FedEx uses OR to help decide in which businesses

to compete and how to allocate resources within specific lines of business in

support of current operations.    In the FedEx process model, OR proves its

value constantly.

• The Army and FedEx appear to have aligned closely their respective OR

cadres, decision makers, and operators.   Analyses employing OR appear to

reflect current operational experience.  The Task Force notes that this factor

is also demonstrated in the historical context in which OR has been used.

In the past DoD tried to institutionalize structured ISR decision support.    In the mid‐

1990’s it established a Joint C4ISR Decision Support Center (the Center).  In particular,

the FY97 Defense Planning Guidance (DPG) directed that the Office of the Assistant

Secretary of Defense for Command, Control, Communications and Intelligence (OASD

(C3I)) lead an effort in coordination with the USD(AT&L), Joint Staff, CINCS, Services and

Agencies to develop a DoD Joint C4ISR Conceptual Architecture which would identify

opportunities to invest in Joint and integrated C4ISR Systems.    The purpose of this

directive was to increase efficiency in the performance of C4ISR systems in Joint

operations and to reduce the apparent duplicative expenditures in C4ISR systems across

34_________________________________ DEFENSE SCIENCE BOARD ADVISORY GROUP ON DEFENSE INTELLIGENCE
___________________________________CHAPTER 7:  INSTITUTIONALIZING OPERATIONS RESEARCH (TOR TASK 4)

the Department of Defense.    $10M per year was made available for FY97‐2001 in

support of this effort.  The Center’s challenge was specific; it was expected to provide

analysis results to the Joint C4ISR DSC Senior Steering Group, Joint Requirements

Oversight Council (JROC) and the Defense Acquisition Board (DAB) to assist senior

decision makers in making Joint C4ISR investment decisions.

The Center was used actively in support of a number of studies including Sensor‐to‐

Shooter, Precision Force C4I Operational Architecture, and other efforts.   The Center’s

efforts, however, were not generally coupled to acquisition strategies.  Over time, the

Center’s resources diminished and, while it still exists, the Center conducts fewer

studies and enjoys limited visibility.    Overall, the Center lacked sustained leadership

support, resources, and the mandate to conduct meaningful analyses coupled to

acquisition and investment decisions.

The Task Force recognizes that several organizational models are available to the USD(I)

to institutionalize Operations Research (and scientific method‐based decision‐support)

that can address these problems.  The Task Force urges the USD(I) and the Department

to look closely at the private sector and the FedEx model, one in which there exists

“corporate” governance regarding the necessity to employ OR for key decisions,

enforcement of OR standards processes and practices, oversight of line‐of‐business‐

specific OR capabilities, and evaluation of the quality of OR products.   The company’s

approach encompasses a compact, but influential corporate organization (for

governance, oversight, and OR analysis in support of corporate strategy decisions), and

more fully elaborated OR organizations embedded in the lines of business.  The Army’s

TRAC organization may represent a sound example of such an embedded organization,

performing a wide range of decision‐support analyses.   However the USD(I) proceeds,

the MORS represents a powerful domain expert resource that can help build a plan to

institutionalize OR.

OPERATIONS RESEARCH APPLICATIONS FOR ISR __________________________________________________35
CHAPTER 7:  INSTITUTIONALIZING OPERATIONS RESEARCH (TOR TASK 4)___________________________________  

This page is intentionally blank.

36_________________________________ DEFENSE SCIENCE BOARD ADVISORY GROUP ON DEFENSE INTELLIGENCE
_____________________________________________________ CHAPTER 8:  SUMMARY RECOMMENDATIONS

CHAPTER 8:  FINDINGS AND RECOMMENDATIONS

The USD(I) should recognize explicitly that the value, quality, and use of OR in support of

a given decision is dependent on the decision maker’s perception of its value and

relevance; that the quality of ISR decisions can be improved by increased use of high

quality OR in decision making; and that the state of OR is advancing.  

OR often gets attention on an ad hoc basis, but when immediate demand subsides,

interest in sustaining its use tends to wane.  Institutionalization of OR is missing in the

overall decision‐making process.  It is important to develop a set of required standard

processes and practices, milestones, methodologies, and thresholds for objective

application of OR to high‐value decisions.    The Task Force recommends the USD(I)

closely examine the FedEx and TRAC models.    Buttressing these views are four

important observations:

1. Operations Research represents a powerful tool to help improve the quality

of investment decision making by illuminating key issues, assumptions, and

sources of information.

2. Operations Research is applied inconsistently throughout the Defense and

ISR communities.  These communities do not possess standard processes and

practices relating to OR practice, a consistent organizational model, or a

consistent commitment to the use of OR.

3. OR – and its use – can be strengthened in the Defense and ISR communities

through effective institutionalization.   

4. The utility of OR can be more firmly established through appropriate test

cases.  

The USD(I) should employ OR to support ISR investment decisions relating to the

dynamic set of capabilities found throughout the intelligence cycle that need to be

maintained, upgraded, developed, or acquired to satisfy the changing requirements that

the intelligence cycle must support.  This OR capability should be organic to the Office of

the USD(I), enjoy the full confidence and support of the USD(I), and be comprised of OR

professionals who can provide objective OR‐based recommendations to guide ISR

investment decisions.

The Task Force offers the following recommendations:

1. As Battlespace Awareness portfolio manager, USD(I) should:

1.1. Establish an organic ISR OR oversight function that requires, sustains, and

reviews OR standard processes and practices in DoD ISR investment decisions.

• Consider the approach employed by FedEx as an institutional OR model,

i.e., decentralized OR‐based decision support performed in lines of

OPERATIONS RESEARCH APPLICATIONS FOR ISR __________________________________________________37
CHAPTER 8:  SUMMARY RECOMMENDATIONS______________________________________________________  

business; a compact corporate OR organization provides standard

processes and practices, and oversight of OR use, and conducts OR on

strategic corporate issues.

• Consider MORS as a domain expert resource to assist in ISR OR

institutionalization.

1.2. Establish a USD(I) capability to conduct OR for strategic investment decisions

(includes delegation of specific OR analyses to subordinate ISR organizations).

• Create a professional OR community of interest and practice across DoD

and the IC.

• Coordinate development and application of OR models for specific ISR

investment decisions.

1.3. Define requirements for broad based studies for decision making milestones

and a joint capability for accrediting the studies, models, and data used and

develop best practices/models for ISR effects.

1.4. Ensure OR requirements for MAPs and programs below Milestone Decision

Authority are understood and met.

2. This OUSD(I) OR function should:

2.1. Serve as the implementation mechanism for the institutionalization of ISR OR

within the DoD.

2.2. Advocate OR as a required input to ISR decision‐making processes.

2.3. Oversee the application of OR to run test cases and scenarios.

2.4. Apply OR to validate assumptions.

2.5. Use OR to test investment decisions for proper balance.

2.6. Track the state of OR and improve OR tools supporting ISR decisions and

employ them in the decision‐making process.

3. Test Cases:  

3.1. In response to the TOR, the Task Force recommends two test cases: one

related to investment decision making in biometrics technologies, and one related

to the balance in the intelligence cycle.   These test cases should be structured to

illuminate challenges and opportunities to strengthen OR decision making in the ISR

investment domain.   

38_________________________________ DEFENSE SCIENCE BOARD ADVISORY GROUP ON DEFENSE INTELLIGENCE
____________________________________________________________ APPENDIX A: TERMS OF REFERENCE

APPENDIX A:  TERMS OF REFERENCE

OPERATIONS RESEARCH APPLICATIONS FOR ISR __________________________________________________39
APPENDIX A: TERMS OF REFERENCE ____________________________________________________________  

40_________________________________ DEFENSE SCIENCE BOARD ADVISORY GROUP ON DEFENSE INTELLIGENCE
____________________________________________________________ APPENDIX A: TERMS OF REFERENCE

OPERATIONS RESEARCH APPLICATIONS FOR ISR __________________________________________________41
APPENDIX A: TERMS OF REFERENCE ____________________________________________________________  

This page is intentionally blank.

42_________________________________ DEFENSE SCIENCE BOARD ADVISORY GROUP ON DEFENSE INTELLIGENCE
_________________________________________________________ APPENDIX B: TASK FORCE MEMBERSHIP

APPENDIX B:  TASK FORCE MEMBERSHIP

CO‐CHAIRMEN

Dr. Robert Lucky Telcordia Technologies

Maj Gen Richard O’Lear, USAF (Ret.) Lockheed Martin

MEMBERS

Mr. Dennis Fitzgerald Private Consultant

Mr. Sid Fuchs OAO Technology Solutions

Mr. Keith Masback U.S. Geospatial Intelligence Foundation

Mr. Ken McGruther Northrop Grumman

Lt Gen George Muellner, USAF (Ret.) Private Consultant

Lt Gen Gene Santarelli, USAF (Ret.) S'relli Consulting, LLC

Mr. Howard Schue Technology Strategies & Alliances

Mr. Bill Thoet Booz Allen  

Mr. Samuel S Visner CSC

VADM Tom Wilson, USN (Ret.) Alliant Techsystems, Inc. (ATK)

OBSERVERS

Mr. Jack Keane MORS

EXECUTIVE SECRETARY

Capt Keith May, USN OUSDI(PPR/ISRP)

DSB SECRETARIAT

Mr. Brian Hughes Defense Science Board

LTC Karen Walter, USA Defense Science Board

SUPPORT

Ms. Michelle Ashley SAIC

Mr. Jimmy Hyatt DUSD(PPR/ISRP) (Scitor)

Ms. Amely Moore SAIC

Ms. Lauren York SAIC

OPERATIONS RESEARCH APPLICATIONS FOR ISR __________________________________________________43
APPENDIX B: TASK FORCE MEMBERSHIP _________________________________________________________  

This page is intentionally blank.

44_________________________________DEFENSE SCIENCE BOARD ADVISORY GROUP ON DEFENSE INTELLIGENCE
_____________________________________________________________ APPENDIX C: BRIEFINGS RECEIVED

APPENDIX C:  BRIEFINGS RECEIVED

April 2‐3, 2008

Col Del Delapena, OUSD(I) HVT

May 14‐15, 2008

Mr. Kevin Meiners, OUSD(I) BA Portfolio Management

COL Michael Wilmer, Army Capabilities

Integration Center

Dr. Chris Morey, TRADOC Analysis Center

Army OR

CDR Eric Law, Navy N81 Navy OR

Mr. Martin Steele, NASA Kennedy Space

Center

NASA/KSC Overview

Mr. Jack Keane, Military Operations

Research Society

MORS Overview

Dr. Jacqueline Henningsen, Air Force A9 Air Force OR

Dr. George Akst, Marine Corps Combat

Development Command

USMC OR

June 11‐12, 2008

Lt Col Michael Fredley, NRO OR at NRO

Mr. Jack Stewart, NSA OR at NSA

Mr. Tim Grayson, ODNI OR at ODNI

Mr. Will Payson, FedEx FedEx Use of OR for Decision Support

Dr. George Stone, Alion Science &

Technology

The Nexus of Operations Research and

Modeling & Simulation in DoD Decision

Making

Mr. Winston Beauchamp, NGA OR at NGA

Mr. Brian Peters, NSA RT‐RG

July 9‐10, 2008

Mr. Dave Terry, NIC‐C DIOCC Brief

Mr. Eric Benn and Mr. Keith Barber, NGA Hi‐Res Utilization & Hi‐Res Characterization

OPERATIONS RESEARCH APPLICATIONS FOR ISR__________________________________________________45
APPENDIX C: BRIEFINGS RECEIVED _____________________________________________________________  

July 30‐31, 2008

Dr. Scott Comes, PA&E OR at PA&E

August 27‐28, 2008

Mr. Josh Kerbel, ODNI Lessons Learned

Center

ODNI Perspectives on Lessons Learned

COL Thomas Cioppa, TRADOC OIF Strategic Communications Analysis and

Assessment

September 17‐18, 2008

Mr. Scott Thayer, NSA Cyber Discussion  

46_________________________________DEFENSE SCIENCE BOARD ADVISORY GROUP ON DEFENSE INTELLIGENCE
___________________________________________________________________APPENDIX D: TEST CASES

APPENDIX D:  TEST CASES

D.1 Biometrics

The Secretary of Defense (SecDef) has recently taken steps to organize DoD biometrics

activities.    Department of Defense Directive (DoDD) 8521.01E of February 21, 2008

“...designates the Director, Defense Research & Engineering (DDR&E) as the Principal

Staff Assistant (PSA) responsible for oversight of DoD biometrics programs and policy..."  

8521.01E also “...designates the Secretary of the Army as the DoD Executive Agent (EA)

for DoD biometrics...”    Section 5.3 of the Directive assigns to the Under Secretary of

Defense for Intelligence (USD(I)) a broad range of responsibilities to ensure biometrics

are used effectively to meet intelligence and counter‐intelligence challenges.34

Although the Directive clarifies DoD Biometrics responsibilities, questions remain

regarding the application of biometrics to military operations in general, and to the ISR

domain in particular.  Although the Administration has requested funding from Congress

for biometrics applications and the Army has established a Biometrics Fusion Center

(BFC), no clear path exists yet for investment in biometrics.    Concepts of operations

(CONOPS) for application of biometrics attributes are lacking, as are the requirements

important to making biometrics investment decisions.    As a result, the DSB Advisory

Group on Defense Intelligence regards biometrics as an appropriate test case for the use

of Operations Research in ISR investment decision making.

In the Task Force’s view, OR can be used in three ways pertinent to biometrics

investment decision making, each of which would help the Department understand OR’s

potential as a decision‐support discipline and lead to concrete results in the short term

relevant to biometrics.

First, the Department can employ OR to establish and refine concepts of operations for

the application of biometrics, end‐to‐end, in support of military operations in general

and to meet ISR requirements in particular.    OR has been used traditionally by the

Services to model operational requirements, desired effects, and the application of

resources to meet those requirements and achieve those effects.   OR can be used to

help refine the understanding of the ways in which the insertion of specific biometrics

data can support and improve the effectiveness of specific military operations.  In this

regard, OR can also help improve the architecture of military operations that employ

biometrics.

Second, the USD(I) can work with the Office of the Director of National Intelligence

(ODNI) to strengthen the information architecture associated with biometrics

databases, information sharing, as well as biometrics collection, analysis, and

dissemination.    While laudable work has already been done at the National

Counterterrorism Center (NCTC), Federal Bureau of Investigation (FBI), and elsewhere in

 

34 Department of Defense Directive (DoDD) 8521.01E, dated February 21, 2008.

OPERATIONS RESEARCH APPLICATIONS FOR ISR__________________________________________________47
APPENDIX D: TEST CASES___________________________________________________________________

the Intelligence Community to employ biometrics information, a unified information

architecture for the application of biometric attributes (i.e., biometrics) by the

Intelligence Community does not exist.  Work remains to be done to build capabilities to

query a wide variety of biometrics databases on a federal basis.   Indeed, information

sharing schemes remain rudimentary, dependent on security restrictions, and hampered

by a lack of data standards.   Nonetheless, investment continues in a wide variety of

programs associated with counter‐terrorism and war‐fighter support, many of them

dependent on the availability and use of biometrics data.  In addition, the IC needs, but

does not possess, a sound analytical concept for the use of biometric attributes, a fact

that can also impair the ability of the IC to define biometrics ISR requirements and an

effective information architecture that encompasses the use of biometrics.  Indeed, the

Task Force is aware of broad concerns that biometrics collection capabilities have

outpaced the ability of the Intelligence and Defense Communities to share and employ

biometrics data, both for analysis and for tactical military operations.  As a result, the

“promise” of biometrics has yet to be fulfilled, even through biometrics collection

capabilities (e.g., retinal, facial, and voice recognition) and tools to “enroll” and compare

an individual with biometrics data (e.g., the Biometrics Automated Toolset) have

continued to develop.

OR should be assessed as a mechanism for defining intelligence operations including

tasking, collection, and analysis pertinent to biometrics data, and dissemination of

intelligence products encompassing a biometrics component.  Such an application of OR

could have several distinct benefits:

• It can lead to better, Community‐wide biometrics information architecture.

• It can help the Community achieve true biometrics information sharing.

• It can help the USD(I) and others understand how OR might be employed to

build information‐centric inter‐agency and Community‐wide ISR capabilities.

Third, the Secretary of the Army and the BFC can use OR to conduct trade studies on a

variety of specific biometrics technologies and acquisition strategies, consistent with

DoD acquisition program management requirements.    OR can be used to compare

Technology Readiness Levels, expected investment levels, and likely operational

capabilities associated with competing technologies, acquisition and investment

strategies, and funding arrangements.  In this instance, OR would employ biometrics as

a technology development and acquisition strategy test‐bed.    Such an approach is

particularly appropriate, given recent moves to strengthen DoD biometrics oversight,

enhance the financial resources associated with biometrics investment, and upcoming

acquisition activities on the part of the Biometrics Fusion Center.  In this instance, the

Department should choose an upcoming procurement activity related to biometrics

collection or enrollment and employ OR to help define and frame a set of investment

decisions related to two or more specific technologies and two or more specific

acquisition strategies.

48_________________________________DEFENSE SCIENCE BOARD ADVISORY GROUP ON DEFENSE INTELLIGENCE
___________________________________________________________________APPENDIX D: TEST CASES

Overall, the three elements described above could help the USD(I) understand how OR

could improve warfighter operations through the use of biometrics as a component of

ISR, lead to a stronger analytic concept for the use of biometrics, contribute to the

development of a biometric information architecture, and strengthen biometrics

technology and program investment decision making.

D.2 Investing in a Balanced Intelligence Cycle and Imagery UASs ‐ a Value Proposition

The Intelligence Community has a history of investing in new, discipline‐specific

capabilities without proper consideration of the impact of decisions on the entire

intelligence cycle.  In the past this resulted in large outlays of resources that, because of

this imbalance, were either not very useful or were marginalized because of an inability

to realize full value from the investment.  For purposes of this discussion, the Task Force

defines the “Intelligence Cycle” as follows: Planning and Direction, Collection,

Exploitation, Analysis, Dissemination, and Feedback (Note: The Task Force recognizes

that, depending upon the collection system being considered, dissemination may occur

at multiple places in this cycle, for example between collection and exploitation, again

between exploitation and all‐source analysis, and finally to the end‐user).  Some historic

examples of where cycle imbalance caused significant problems or challenges include

the following:

1. Inability to adequately disseminate national imagery to tactical units during

Desert Shield/Desert Storm.  This is perhaps the most notable example of where

a lack of balance in the intelligence cycle made a magnificent collection product

not available in a timely fashion to warfighters in the volume or time‐frame

needed.    In this case an enormous investment in imagery collection capability

was not matched by the investment needed in dissemination capability or

technology.  The result was air strike target imagery or amphibious landing zone

imagery was available at national and/or combatant command levels, but not to

those with the mission of striking the targets or landing on the beaches.    In

addition, even if dissemination had been much more capable, imagery analysts

were not available in the numbers needed for exploitation of the imagery

product.    The reasons for this imbalance are multiple (failure to consider

dissemination or analysis requirements by a collection system acquirer, failure to

properly invest in dissemination capability or technology by military Service

users) so the purpose here is not to try to establish blame, but to point out that

an OR approach to examining the dissemination and exploitation issues at the

time of collection system investment decision would have been helpful.

2. Inability to adequately provide timely feedback to collection system planning

and direction, especially when the feedback to planning and direction reaches

across collection disciplines.    This challenge is often referred to as “cross‐

cueing,” and it is one that is rarely provided for in the design and procurement of

a collection system.  U.S. war‐fighting history is replete with examples of where

adequate and timely feedback from one collection discipline to another (notably

OPERATIONS RESEARCH APPLICATIONS FOR ISR__________________________________________________49
APPENDIX D: TEST CASES___________________________________________________________________

among and between SIGINT, ELINT, and IMINT) might have resulted in high‐value

target location or important intelligence discovery.   

3. Inability to adequately provide specialized analysis to support new collection

capabilities.  Measurement and Signature Intelligence (MASINT) is an area where

new technical collection capability and capacity may be significantly outpacing

the ability to realize the full value of new kinds of technical collection.   Many

MASINT products require specialized exploitation and specifically trained and

experienced analysts to make good use of products.    It is a problem for

intelligence cycle effectiveness and utility if investment in exploitation and

analysis for MASINT does not match the developing collection capacity and it is

probably better to re‐balance the investment than continue to pour more

MASINT data into the system.    An historical example of where an effective

MASINT intelligence cycle was achieved through balanced investment in

planning and direction, collection, exploitation, analysis, dissemination and

feedback was the U.S. Navy’s use of Acoustic Intelligence (ACINT) during the Cold

War.    Over several decades, very large investment by the Navy – in both

operations and intelligence – resulted in strong technical and tactical ACINT

collection matched by exploitation of the data and use in all‐source intelligence

analysis to provide cueing information to operational forces and technical

information to weapons systems developers.    Coincidentally, this same highly

developed (and expensive) system (SOSUS, ASWOCS, FOSICS, and FOSIFS) made

very good use of cross‐cueing techniques between ACINT and

IMINT/SIGINT/ELINT.   U.S. Navy anti‐submarine warfare is strongly rooted in a

history of Operations Research, and the degree to which OR played a part in

developing the ASW/ACINT system might be a useful case study in its own right.   

Significant discussion is underway in the Intelligence Community regarding the

appropriate balance of resource investment in the Tasking, Processing, Exploitation, and

Dissemination (TPED) segments of ISR systems.    The Task Force suggests that to look

only at TPED may be “shooting short of the mark.”    Instead, a test case using OR

techniques, tools, and practitioners analyze an ISR resource decision with the goal of

optimizing allocation of resources in a balanced way across the entire intelligence cycle

as described in paragraph 1 above.  A potential test case would be to look at a planned

investment in imagery UASs, whose collection potential can surely be examined in

detail using valid coverage factors and collection capability.    However, all the

additional parts of the intelligence cycle should be analyzed similarly with a view

toward answering the following questions about other parts of the intelligence cycle

relevant to the potential UAS collection system investment.

1. Planning and direction:   What is the command and control apparatus and are

there the right number of sufficiently trained personnel in that system to do

mission planning relevant to the type of missions envisioned?  If not, how many

are needed and what is the cost associated with recruiting, training, and

retaining them?    Are the planned UASs fully compatible with existing ground

50_________________________________DEFENSE SCIENCE BOARD ADVISORY GROUP ON DEFENSE INTELLIGENCE
___________________________________________________________________APPENDIX D: TEST CASES

station infrastructure and is there sufficient capacity in them to handle the

envisioned new system?    If ground station additions/modifications and

additional ground station personnel are required, have the costs been fully

analyzed and included in the planned investment?   

2. Collection:    Even assuming that the collection capability of the planned UAS

system has been correctly measured and modeled, have the costs of future

potential upgrades and/or alternative collection capabilities been estimated?   

3. Dissemination:    Is the contemplated new UAS collection capability fully

compatible with existing dissemination networks and is there sufficient

communications capacity to handle the increased volume of collection?  

Additionally, once exploitation has been conducted, is there sufficient secondary

dissemination capacity to get exploited data into the hands of users?    The

bottom line is to ensure that there is sufficient, credible analysis to answer the

questions relative to primary and secondary dissemination requirements, and

that associated cost data are included in the investment planning.

4. Exploitation:  An historic problem associated with imagery collection systems is a

failure to plan and pay for sufficient imagery exploitation personnel to handle

the increased data quantity.    An OR analysis approach could help answer this

question, as well as provide cost estimates of what the additional personnel and

their training would cost so that these costs could be included in the cost of the

overall procurement plan.

5. Analysis:    There is a difference between exploitation and analysis, and if the

planned new collection capability is to fully contribute to actual knowledge

development, the capacity must exist to disseminate and incorporate the

exploited information into the all‐source analysis system.    While all‐source

analysis is not a part of the collection system per se, pre‐decisional OR analysis

should be conducted to ensure that the envisioned system is fully compatible

with all source analysis requirements in all parts of the ISR system that could

benefit from the information.

6. Feedback:   Often the most overlooked part of the intelligence cycle, any new

system acquisition should include plans for providing feedback – to the entire ISR

system if required.  Perhaps the most sophisticated form of feedback is “cross‐

cueing,” where exploited data from one collection system can immediately cue

(or be cued by) other collection systems.  This type of feedback is critical to an

effective and efficient ISR system and has rarely been incorporated in system

acquisition decisions, but should be analyzed and accounted for during

investment decision deliberations.    

OPERATIONS RESEARCH APPLICATIONS FOR ISR__________________________________________________51
APPENDIX D: TEST CASES___________________________________________________________________

This page is intentionally blank.

52_________________________________DEFENSE SCIENCE BOARD ADVISORY GROUP ON DEFENSE INTELLIGENCE
___________________________________________________APPENDIX E: ACRONYMS & GLOSSARY OF TERMS

APPENDIX E:  GLOSSARY

ACINT Acoustic Intelligence  

AFIT Air Force Institute of Technology

AOR Area of Responsibility

ARCIC Army Capabilities Integration Center

A/S Assistant Secretary

ASD Assistant Secretary of Defense

ASW Anti‐Submarine Warfare

ASWORG Anti‐Submarine Warfare Operations Research Group

BA Battlespace Awareness

BAT Biometrics Automated Toolkit

BDC Biometrics Fusion Center

BFC Biometrics Fusion Center

C3I Command, Control, Communications, and Intelligence  

C4ISR Command, Control, Communications, Computers, Intelligence,

Surveillance and Reconnaissance  

CAA Center for Army Analysis

CIA Central Intelligence Agency

CINCs Commanders in Chief

CJCS Chairman of the Joint Chiefs of Staff

COCOM Combatant Command

CONOPS Concept of Operations

DAB Defense Acquisition Board

DARPA Defense Advanced Research Projects Agency

DDRE Director of Defense Research and Engineering

DEPSECDEF Deputy Secretary of Defense

DFAR Defense Federal Acquisition Regulation

DIA Defense Intelligence Agency

DIOCC Defense Intelligence Operations Coordination Center

DNI Director of National Intelligence

OPERATIONS RESEARCH APPLICATIONS FOR ISR__________________________________________________53
APPENDIX E: ACRONYMS & GLOSSARY OF TERMS___________________________________________________

DoD Department of Defense

DoDD DoD Directive

DPG Defense Planning Guidance

DSB Defense Science Board

DTED Digital Terrain Elevation Data

DTRA Defense Threat Reduction Agency

EA Executive Agent

ELINT Electromagnetic Intelligence  

FAR Federal Acquisition Regulations

FBI Federal Bureau of Investigation  

FedEx Federal Express

FFRDC Federally Funded Research and Development Center

FIA Future Imagery Architecture

FMV Full Motion Video

FYDP Future Year Defense Plan

GWOT Global War on Terror

HUMINT Human Intelligence  

HVT High Value Target

IC Intelligence Community

ICDO Integrated Concepts Development Office

IOSA Integrated Overhead SIGINT Architecture

ISR Intelligence, Surveillance, and Reconnaissance

JASTP Joint Advanced Strike Technology Program

JCS Joint Chiefs of Staff

JDAM Joint Direct Attack Munition  

JROC Joint Requirements Oversight Council

JSF Joint Strike Fighter

JWCA Joint Warfighting Capability Assessment

MAP Major Acquisition Program

MASINT Measurement and Signature Intelligence  

MIT Massachusetts Institute of Technology

54_________________________________DEFENSE SCIENCE BOARD ADVISORY GROUP ON DEFENSE INTELLIGENCE
___________________________________________________APPENDIX E: ACRONYMS & GLOSSARY OF TERMS

MOA Memorandum of Agreement  

MORS Military Operations Research Society  

MOU Memorandum of Understanding

MWORG Mine Warfare Operations Research Group  

NCTC National Counterterrorism Center

NDRC National Defense Research Committee  

NGA National Geospatial‐Intelligence Agency  

NIC‐C National Intelligence Coordination Center  

NPS Naval Postgraduate School

NSA National Security Agency

NRO National Reconnaissance Office

OASD Office of the Assistant Secretary of Defense  

ODNI Office of the Director of National Intelligence  

OIF Operation Iraqi Freedom

OR Operations Research

ORS Operational Research Section

OSD Office of the Secretary of Defense

OSRD Office of Scientific Research and Development  

OUSD(AT&L) Office of the Under Secretary of Defense for Acquisition, Technology

and Logistics

OUSD(I) Office of the Under Secretary of Defense for Intelligence  

PA&E Program Analysis and Evaluation  

PPBS Planning, Programming and Budgeting System

PSA Principal Staff Assistant

R&D Research and Development

RAF Royal Air Force

RDT&E Research, Development, Test and Evaluation

S&T Science and Technology  

SA Systems Analysis

SDD System Design and Development

SecDef Secretary of Defense  

OPERATIONS RESEARCH APPLICATIONS FOR ISR__________________________________________________55
APPENDIX E: ACRONYMS & GLOSSARY OF TERMS___________________________________________________

SIGINT Signals Intelligence  

SIOP Single Integrated Operational Plan

SOF Special Operations Forces  

SOSUS Sound Surveillance System

TOR Terms of Reference  

TPED Tasking, Processing, Exploitation, Dissemination  

TRAC TRADOC Analyses Center

TRADOC U.S. Army Training and Doctrine Command

TSWG Technical Support Working Group

UAS Unmanned Aerial System

USA United States Army

USAF United States Air Force

USD(AT&L) Under Secretary of Defense for Acquisition, Technology, and Logistics

USD(I) Under Secretary of Defense for Intelligence

USMC United States Marine Corps

USN United States Navy

U.S. United States

VCJCS Vice Chairman of the Joint Chiefs of Staff

56_________________________________DEFENSE SCIENCE BOARD ADVISORY GROUP ON DEFENSE INTELLIGENCE