DEPLOYED ANALYST HANDBOOK

(DAHB)

JUNE 2016

DISTRIBUTION RESTRICTION: Approved for public release; distribution is unlimited.

This publication is prepared by Center for Army Analysis (CAA). It reflects the current thought of this 

Agency and conforms to Department of the Army (DA) doctrine.
DISCLAIMER

The findings of this report are not to be construed as an official 

Department of the Army position, policy, or decision unless so 

designated by other official documentation. Comments or 

suggestions should be addressed to:

Director

Center for Army Analysis 

ATTN: CSCA-OA

6001 Goethals Road

Fort Belvoir, VA 22060-5230
REPORT DOCUMENTATION PAGE Form Approved

OMB No. 0704-0188

The public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data 

sources, gathering and maintaining the data needed, and completing and reviewing this collection of information. Send comments regarding this burden estimate or any other 

aspect of this collection of information, including suggestions for reducing this burden, to Department of Defense, Washington Headquarters Services, Directorate for Information 

Operations and Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. Respondents should be aware that notwithstanding any other 

provision of law, no person shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number.

PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS.

1. REPORT DATE (DD-MM-YYYY) 2. REPORT TYPE 3. DATES COVERED (From –To)

June 2016 Handbook Apr 2015 – Feb 2016

4. TITLE AND SUBTITLE 5a. CONTRACT NUMBER

Deployed Analyst Handbook

DAHB 5b. GRANT NUMBER

5c. PROGRAM ELEMENT NUMBER

6. AUTHOR(S) 5d. PROJECT NUMBER

V. Bahm, G. Collins, R. Kollhoff, J. Lindquist, M. Stollenwerk, M. Gaul, E. Jesse, F. 

Dunaway, S. Thambidurai, S. Huddleston, H. Hall, N. Delara, M. Haskell, A. Shilling, and J. 

Anderson.

5e. TASK NUMBER

5f. WORK UNIT NUMBER

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION 

REPORT NUMBER

Center for Army Analysis

6001 Goethals Road

Fort Belvoir, VA 22060-5230 CAA-2015094

9. SPONSORING / MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR'S 

ACRONYM(S)

FA49 Proponent Office, HQDA DCS G-8

Pentagon, Room 3C353

Pentagon, Washington, DC 20310 11. SPONSOR/MONITOR'S REPORT 

NUMBER(S)

12. DISTRIBUTION / AVAILABILITY STATEMENT

DISTRIBUTION A. Approved for public release: distribution unlimited.

13. SUPPLEMENTARY NOTES

14. ABSTRACT

The Center for Army Analysis (CAA) has provided Operations Research /Systems Analysis (ORSA) support to forward deploying 

headquarters with both personnel and reachback project execution for over 14 years. Changes in the operational environment 

prompted this revision of the Deployed Analyst Handbook to prepare analysts for deployments in support of the warfighter. This 

handbook highlights ORSA methods and techniques that are expected to be used by ORSA analysts into the foreseeable future.

15. SUBJECT TERMS

Center for Army Analysis (CAA); Deployed Analyst Handbook (DAHB); assessments; ORSA capabilities; data management and 

analytical techniques; survey development and analysis; geospatial analysis; statistical analysis; reachback support; operation 

assessment

16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF 

ABSTRACT

18. NUMBER 

OF PAGES

19a. NAME OF RESPONSIBLE 

PERSON

a. REPORT b. ABSTRACT c. THIS PAGE

UU 122

Mr. Russell Pritchard

U U U

19b. TELEPHONE NUMBER

(include area code)

(703) 806-5421

Standard Form 298 (Rev. 8/98)

Prescribed by ANSI Std. Z39.18
(THIS PAGE INTENTIONALLY LEFT BLANK)
CAA-2015094

i

DEPLOYED ANALYST HANDBOOK

(DAHB)

JUNE 2016

SUMMARY

THE PROJECT PURPOSE: This handbook is intended to be a quick reference guide for all 

deploying Operations Research/Systems Analysis (ORSA) analysts, whether military, civilian, or 

contractor. 

THE PROJECT SPONSOR: Functional Area (FA) 49 Proponency Office. 

THE PROJECT OBJECTIVES: 

Present ORSA analysts with a wide range of topics from specific analytical products to ways to 

prepare and communicate analyses.

Present examples of analyses developed in theater and through reachback support.

Present examples of analyses that ORSA analysts can expect to execute in a combat theater.

Provide references for ORSA analysts who are interested in additional resources and examples.

THE SCOPE OF THE PROJECT: The information contained in this handbook was

developed from the experiences of ORSA analysts who deployed to operational theaters in Iraq, 

Afghanistan, the Philippines, the Horn of Africa, the Middle East, and elsewhere over the past 

14-plus years. 

COMMENTS AND QUESTIONS may be sent to the Director, Center for Army Analysis, 

ATTN: CSCA-OA, 6001 Goethals Road, Suite 102, Fort Belvoir, VA 22060-5230
CAA-2015094

ii

(THIS PAGE INTENTIONALLY LEFT BLANK)
CAA-2015094

iii

PREFACE

Dedication. This Deployed Analyst Handbook (DAHB) is dedicated to the 142 ORSA analysts 

who deployed under Center for Army Analysis (CAA) auspices to Iraq, Afghanistan, the Horn of 

Africa, and the Philippines from 2002 through June of 2016.

The Center would also like to thank all Operations Research/Systems Analysis (ORSA) analysts

who contributed to the development of the DAHB, as well as reviewing agencies, to include the 

U.S. Army Materiel Systems Analysis Activity (AMSAA), the Marine Corps Combat 

Development Command (MCCDC), the U.S. Army Training and Doctrine Command Analysis 

Center (TRAC), and the Functional Area (FA) 49 Proponency Office.

Intended Audience. Military FA 49s, civilian career program (CP) 36/1515s, and ORSA 

contractors who directly support the warfighter at brigade and higher levels of command.

Reference Guide. The DAHB is a quick reference guide for all deploying ORSA analysts and 

contains a wide range of techniques and analytical products that ORSA analysts have developed

in theater and through reachback support, and have used in operational theaters. The lessons 

learned and best practices contained in the DAHB were consolidated from numerous ORSA

analysts who have deployed in support of Operations Iraqi Freedom (OIF) and New Dawn 

(OND) in Iraq; Operations Enduring Freedom (OEF) and Freedom's Sentinel (OFS) in 

Afghanistan; Operation Inherent Resolve (OIR) in the Levant, and other Overseas Contingency 

Operations (OCOs) in the Philippines, the Horn of Africa, and elsewhere around the world over 

the past 14-plus years. The DAHB is informative, relevant, and full of information—not easily 

obtained elsewhere—that will prepare deploying ORSA analysts for their unique and highly 

critical assignments at the tactical, operational, and strategic levels of war.

Topic Selections. The information and examples presented in this handbook highlight the types 

of analyses that ORSA analysts can expect to conduct in the future. CAA thoughtfully chose 

these specific examples through working groups and advice received from experts throughout the 

analytical community.

Motivation for Using Handbook. This handbook complements the CAA Deploying Analyst 

Course (DAC) by providing a companion text that assists deploying ORSA analysts who are 

junior military officers, civilians, and contractors. This handbook helps prepare ORSA analysts

to provide analytical support to their respective commands and staffs by exposing them to best 

practices and lessons learned, and informing them of the reachback processes.
CAA-2015094

iv

Additional Deployed Analyst Information and Insights: 

 CAA Deployed Analyst History Report Vol. I (Analytic Support to Combat Operations in 

Iraq (2002 – 2011)), March 2012, available at 

http://www.caa.army.mil/DAHR%20Volume%201%20-%20Iraq%20FinalMarch%202012.pdf.

 CAA Deployed Analyst History Report (DAHR) Vol. II (Analytic Support to Combat 

Operations in The Philippines (2011 - 2014)), December 2015, available at 

http://www.caa.army.mil/DAHR-Philippines(Final%20Report)20151203.pdf.
CAA-2015094

v

CONTENTS Page

CHAPTER 1 COMMUNICATING ORSA CAPABILITIES ...................................................1

1.1 What is Operations Research? .................................................................................................................................1

1.2 Concisely Communicating the ORSA Profession to Others....................................................................................1

1.3 How to Recognize When Others Would Benefit from ORSA.................................................................................2

1.4 Valuable Applications of the ORSA Skill Set in a Deployed Environment ............................................................3

1.5 Ensuring One's Relevancy .......................................................................................................................................3

1.6 Summary..................................................................................................................................................................5

CHAPTER 2 DATA MANAGEMENT METHODS..................................................................7

2.1 Why Data Management? .........................................................................................................................................7

2.2 Data Management Goals and Objectives.................................................................................................................7

2.3 Data Management Process Overview......................................................................................................................7

2.4 Three Common Characteristics of Data Collection .................................................................................................8

2.5 High Quality Data Attributes...................................................................................................................................8

2.6 Data Entry................................................................................................................................................................9

2.7 Data Verification .....................................................................................................................................................9

2.8 Data Validation......................................................................................................................................................10

2.9 Data Analysis.........................................................................................................................................................10

2.10 Data Dissemination..............................................................................................................................................11

2.11 Data Organization................................................................................................................................................11

2.12 Data Maintenance ................................................................................................................................................11

2.13 Version Control ...................................................................................................................................................12

2.14 Operational Competence .....................................................................................................................................12

2.15 Way Ahead ..........................................................................................................................................................12

CHAPTER 3 DATA ANALYSIS TECHNIQUES....................................................................15

3.1 Introduction ...........................................................................................................................................................15

3.2 Data Cleaning ........................................................................................................................................................15

3.3 Statistical Analysis ................................................................................................................................................15

3.4 Other Analytical Methods......................................................................................................................................16

3.5 Software Utilization...............................................................................................................................................17

3.6 Examples of Available Software Packages............................................................................................................18

3.7 Case Study: Time Series Analysis in a Theatre Strategic Headquarter .................................................................19

3.8 Summary................................................................................................................................................................47

CHAPTER 4 COMMUNICATING CLEARLY.......................................................................49

4.1 Purpose ..................................................................................................................................................................49

4.2 Constraints, Limitations, and Assumptions (CLA)................................................................................................49

4.3 Roles and Tenets of CLA ......................................................................................................................................50

4.4 Summary................................................................................................................................................................61

CHAPTER 5 PUBLIC PERCEPTION SURVEY DEVELOPMENT AND ANALYSIS.....63

5.1 Introduction ...........................................................................................................................................................63

5.2 Practical Applications and Examples.....................................................................................................................63

5.3 Data Analysis.........................................................................................................................................................69

5.4 Report Presentation and Delivery..........................................................................................................................70

5.5 Limitations of Surveys...........................................................................................................................................72

5.6 Existing Data Sources/Surveys..............................................................................................................................73

5.7 Summary................................................................................................................................................................73

5.8 Further Reading .....................................................................................................................................................74

CHAPTER 6 OPERATIONS ASSESSMENTS........................................................................75

6.1 Introduction ...........................................................................................................................................................75

6.2 Assessment Framework.........................................................................................................................................75

6.3 More on Causality .................................................................................................................................................80
CAA-2015094

vi

6.4 Alternatives to MOE-MOP Constructs..................................................................................................................80

6.5 More on Assessment Products...............................................................................................................................81

6.6 To read more..........................................................................................................................................................84

CHAPTER 7 GEOSPATIAL ANALYSIS.................................................................................85

7.1 Introduction ...........................................................................................................................................................85

7.2 Geospatial Software and Its Uses ..........................................................................................................................85

7.3 Examples of Geospatial Analysis..........................................................................................................................85

7.4 Summary................................................................................................................................................................89

CHAPTER 8 REACHBACK CAPABILITIES ........................................................................91

8.1 Why Reachback? ...................................................................................................................................................91

8.2 Motivational Factors for Reachback......................................................................................................................91

8.3 Reachback Considerations.....................................................................................................................................93

8.4 Reachback Support Topics....................................................................................................................................94

8.5 The Reachback Process .........................................................................................................................................95

8.6 Knowledge Management .......................................................................................................................................97

8.7 Formulating Reachback Problems.........................................................................................................................98

8.8 Coordinating Intra-/Inter-Theater Reachback Efforts............................................................................................99

8.9 Pre-Deployment Training Opportunities for Deploying Analysts.......................................................................100

APPENDIX A LIST OF ACRONYMS....................................................................................103

APPENDIX B REFERENCES .................................................................................................107

FIGURES

Figure 1. Steps in data flow model. ................................................................................................7

Figure 2. Weekly enemy-initiated attacks in Afghanistan (1 January 2007- 31 December 2012).24

Figure 3. Seasonal decomposition of weekly enemy-initiated attacks in Afghanistan (1 January 

2007- 31 December 2012). ....................................................................................26

Figure 4. Observed versus predicted enemy-initiated attacks (EIAs)...........................................29

Figure 5. Selected residual analyses. ............................................................................................32

Figure 6. Seasonal ARIMA prediction model for insurgent activity............................................35

Figure 7. Results of a two-stage least squares model to identify unexpected trends....................39

Figure 8a. Static, geospatial scatter plot visualizing individual enemy-initiated attacks. .............41

Figure 8b. Static geospatial density plot visualizing changes in enemy-initiated attacks ............42

Figure 9. Average number of Afghans living within 1 km of each enemy-initiated attack (EIA).43

Figure 10. ANDP survey model....................................................................................................54

Figure 11. Annual ANDP spending per fiscal year ......................................................................54

Figure 12. Numbers of attacks and casualties...............................................................................55

Figure 13. Casualties per attack ....................................................................................................56

Figure 14. Numbers of participating ISF/US/CF security forces .................................................57

Figure 15. The answer "Road improvements would most improve our lives?" ...........................58

Figure 16. The age-percentages of ANDP 3.0 survey respondents..............................................59

Figure 17. MEDEVAC helicopter response times and number of events....................................60

Figure 18. Security situation by province .....................................................................................61

Figure 19. Survey process map.....................................................................................................63

Figure 20. Line chart – depict changes in perception over time...................................................71

Figure 21. Geospatial Displays – Show the distribution across an area. ......................................71
CAA-2015094

vii

Figure 22. Clustered Bars – Illustrate a rank order among several choices..................................71

Figure 23. Pie Charts – Show demographics or categorical compostions....................................72

Figure 24. Staced Charts – Display Likert-scale data across categories.......................................72

Figure 25. Mental model for linking outcomes to indicators........................................................78

Figure 26. Assessment product example 1. ..................................................................................82

Figure 27. Assessment product example 2. ..................................................................................83

Figure 28. Napoleon's march to Moscow in 1812 ........................................................................86

Figure 29. MEDEVAC coverage..................................................................................................87

Figure 30. District assessments in Afghanistan ............................................................................88

Figure 31. Map of areas polled in the Philippines........................................................................89

Figure 32. Example code (script) written to support reachback request.......................................92

Figure 33. Example web application developed to support reachback request ............................93

Figure 34. CAA Operations Analysis Division SharePoint Site...................................................98

Figure 35. Reachback Request Form............................................................................................98
CAA-2015094

viii  COMMUNICATING ORSA CAPABILITIES DAHB

(THIS PAGE INTENTIONALLY LEFT BLANK)
CAA-2015094

DAHB COMMUNICATING ORSA CAPABILITIES  1

CHAPTER 1 COMMUNICATING ORSA CAPABILITIES

1.1 What is Operations Research?

General. What is Operations Research/Systems Analysis? It is difficult to answer this question 

concisely without using words like stochastic, combinatorial, and heteroscedasticity. However, 

effectively communicating one's unique ORSA capabilities to the command is as critical to one's 

success as is the quality of one's work.

In a deployed environment, ORSA analysts often do not have the luxury of external resources to 

assist in communicating their skill set to the command and others who would benefit from their 

assistance. Many or most ORSA analysts are introverted by nature. More often than not, 

socializing one's skill set does not come naturally. However, it is a talent that can be developed 

with practice.

This chapter offers techniques for ORSA analysts to use (1) in explaining their profession 

concisely to others, (2) in applying their skills in a deployed environment, and (3) in effectively 

presenting their work. Much of the content of this chapter parallels information from the 

Institute for Operations Research and the Management Sciences (INFORMS) on communicating 

the ORSA profession.

1.2 Concisely Communicating the ORSA Profession to Others

ORSA Defined. ORSA is the acronym for the scientific field of "Operations Research/Systems 

Analysis," but ORSA is often used colloquially to mean an "ORSA analyst." Introducing oneself 

as an ORSA peaks the attention of others and leads to further inquiry. As senior leaders become 

familiar with the highly skilled, academic nature of an ORSA analyst's work, they gain a sober 

appreciation for the tremendous value ORSA analysts bring to the fight. In short order, senior 

leaders associate ORSA analysts with insight and solutions for solving complex problems at the 

tactical, operational, and strategic levels of war.

Responsibilities of a deployed ORSA:

1

ORSAs use analytical methods and mathematically based procedures to enable leadership decisions 

in a constantly changing global environment. [ORSAs] introduce quantitative and qualitative 

analysis to the military's decision-making processes by developing and applying probability models, 

statistical inference, simulations, optimization, and economic models. [ORSAs work in] diverse 

disciplines that include personnel management, doctrine and force development, training 

management, system testing, system acquisition, decision analysis, and resource management, as 

well as tactical, operational and strategic planning from division through combatant command, and 

from Army Command (ACOM) through the highest levels of the [Department of Defense] DOD 

(Department of the Army Pamphlet [DA PAM] 600-3, 2014, p. 310).

ORSAs integrate military knowledge and experience with the scientific and managerial fields. 

They serve as subject matter experts in designing forces, allocating resources, analyzing effects, 

 

1 This discussion about the responsibilities of a deployed Army Officer ORSA taken from DA 

PAM 600-3, 2014 is substantially applicable to deployed Army, sister service, and partner nations’ 

military, civilian, and contractor ORSAs.
CAA-2015094

2  COMMUNICATING ORSA CAPABILITIES DAHB

performing course of action and trade-off analysis, and they effectively communicate potential 

solutions to complex problems to decision makers. The ORSA officer will typically serve in one 

of several general assignments:

(1) As a combat analyst on a division, corps, Army Service Component Command (ASCC), (or 

equivalent joint headquarters staff), or Combatant Command headquarters staff.

(2) As an analyst on an Army, joint, or defense agency staff, a field-operating agency (FOA) or a 

direct reporting unit, (e.g., CAA, Human Resources Command (HRC), or the Army Test and

Evaluation Command).

(3) As an analyst in an ACOM, i.e., U.S. Army Training and Doctrine Command (TRADOC), 

U.S. Army Forces Command (FORSCOM), or U.S. Army Materiel Command (AMC), either in 

the command headquarters or in an organization whose principal mission is to provide analysis 

that supports the organizing, equipping, manning, training and operations of military forces. 

Such organizations include the TRADOC Army Capabilities Integration Center (ARCIC), 

TRADOC Analysis Center (TRAC), and Army Centers of Excellence (CoE).

(4) As an instructor teaching Operations Research/Systems Analysis and/or mathematics courses 

at the United States Military Academy (USMA), Naval Postgraduate School (NPS), Air Force 

Institute of Technology or a TRADOC School. (DA PAM) 600-3, 2014, p. 311).

The ORSA Skill Set. ORSA analysts (1) investigate essential factors that illuminate 

understanding; (2) communicate complex ideas to decision makers at brigade and higher levels 

of command; and (3) recommend solutions to complex problems.

ORSA analysts apply their skill set through (1) problem analysis, using various analytical 

techniques; (2) probability and statistics analysis to gain valuable insight; and (3) optimization 

analysis to quantify options and select the best option for a complex problem that has multiple 

solution sets. 

Application examples include

 analyzing significant event data in order to identify spatial and temporal trends;

 developing strategies to effectively allocate or employ limited resources;

 improving supply chain and logistics operations;

 conducting qualitative and quantitative assessments of current operations;

 measuring risk and uncovering factors critical to managing and reducing risk; and

 evaluating the potential benefit of changes to tactics, techniques, and procedures (TTPs).

1.3 How to Recognize When Others Would Benefit from ORSA

An ORSA analyst can determine when others would benefit from their skill set through a variety 

of indicators including identifying staff

 with a large amount of data but do not know what is important or how to use it;

 who are responsible for inputting and storing their data in Excel but do not have the skills 

to do so;
CAA-2015094

DAHB COMMUNICATING ORSA CAPABILITIES  3

 who do not know how to display their data to senior leaders;

 who are dealing with a complex problem with many decision points and need a method to 

analytically evaluate their options and choose the best one;

 who are risk-averse and do not have effective strategies for limiting, reducing, or 

managing risk; or

 who have variables they want to measure but do not know how to measure them.

1.4 Valuable Applications of the ORSA Skill Set in a Deployed Environment

Most ORSA analysts do not have the time required to use advanced ORSA techniques in a 

deployed environment; however, there are less complex applications that are highly valuable to 

theater commands. These include

 managing, analyzing, and displaying data;

 organizing large amounts of data;

 communicating technical information to commanders and senior leaders;

 thinking critically, i.e., "outside the box", and developing strategies to solve complex 

problems;

 advocating change and providing clear answers to problems others are reluctant to tackle;

 developing metrics to measure campaign progress;

 providing recommendations that are tactically, operationally, and strategically relevant; 

and

 teaching basic and advanced Excel and PowerPoint skills.

In the ORSA community, these tools are often taken for granted, e.g., pivot tables and charts; 

linking PowerPoint data to external statistical databases; or using Visual Basic for Applications 

(VBA) to automate tasks. The ORSA skill set is in high demand and pays huge dividends for 

theater commanders. For example, teaching staff members Excel technical shortcuts saves them 

a great deal of time (the most highly valued asset in theater). There is a saying, "If you give a 

man a fish, you feed him for a day. If you teach a man to fish, you feed him for a lifetime" 

(author unknown).

1.5 Ensuring One's Relevancy

ORSA analysts often must establish their role with other staff officers in the unit who may be 

confused as to the nature of their duties. This is essential in ensuring that ORSA analysts are 

used appropriately for the services they offer. Listed below are techniques incoming ORSA 

analysts should consider when establishing their role.

Keep others informed. ORSA analysts should strive to become extroverted collaborators, not 

"isolationists." They should be excited about their work and share it with others. Products 

improve when one collaborates with others and involves them in the process. No matter the 

level of expertise, others have opinions, unique histories and backgrounds, and "an extra set of 
CAA-2015094

4  COMMUNICATING ORSA CAPABILITIES DAHB

eyes" to enhance the process. Others cannot help solve a problem or assist with a project unless 

they are aware of it.

Make time for face-to-face interaction with others. In today's digital world, staff work can 

keep professionals at their desks for hours on end. ORSA analysts should get out and mingle 

with their counterparts. This is the best way for everyone to get to know each other and share 

problems, issues, concerns, and ideas. Before one can influence others, one must gain their trust

and respect. ORSA is a quiet profession. ORSA analysts have to get out and make a little noise 

if they want to be utilized. As a rule-of-thumb, one should spend more time interacting with 

others face-to-face than interacting with others over the computer.

Do not emphasize problems. Commanders and staff appreciate solutions. Problems are just 

challenges waiting to be met by determined ORSA analysts and their in and out of theater 

networks of associates. This does not mean the ORSA analyst is an expert in every staff issue. 

However, the ORSA analyst should talk to staff members to uncover solutions that can serve to 

solve problems.

Create the opportunity for follow-on interaction. When an ORSA analyst meets people faceto-face to discuss issues, he or she should set a time to meet again. This helps keep the 

momentum going so others do not lose interest or become sidetracked. One should say 

something like, "when we meet again at 1000 on Thursday, we will discuss progress on x, y, and 

z."

Actively seek out resources, e.g., colleagues, mentors, and "believers." There are people in the 

command who have ORSA backgrounds and/or are aware of the value ORSA analysts bring to 

the table. One should develop a strong rapport with these people, keep them informed of current 

work, and ask these people if they are willing to collaborate on projects. This is especially 

important if one is a "lone" deployed ORSA analyst. ORSA analysts should find others who 

appreciate groundbreaking ideas and support "out of the box" projects.

Familiarize others with ORSA work. The following technique can increase one's exposure: 

When briefing or describing one's work to others, say, "Do you think Colonel Smith would like 

to see this analysis?" When one receives an affirmative answer, one has a perfectly good reason 

to walk into Colonel Smith's office and present one's great work. When possible, an ORSA

analyst should bring along his or her supervisor or counterpart in or near Colonel Smith's office 

or network. This builds relationships within an ORSA analyst's circle of influence.

Start with projects most applicable and important to the command. ORSA analysts could 

spend months simply mining through data sets. As a deployed ORSA analyst, one does not have 

the luxury of time. Deployed ORSA analysts need to produce results in days (or sometimes 

hours). Good work completed quickly is better than great work completed late. People need to 

receive answers while they still remember their questions. Conducting timely analysis that is 

most applicable and important to the command creates a healthy respect for the ORSA 

profession. Once senior leaders understand and appreciate the ORSA skill set, they will seek out 

ORSA analyst assistance on complex problems. By solving problems quickly and well, one 

becomes the go-to person in the command for advice and for solutions.

Answer the question. ORSA analysts should resist the temptation to over promise. When 

presented with a specific question, one should always answer that question first, especially prior 

to explaining to the senior leader that the question he asked is not the "real" question at hand. If 
CAA-2015094

DAHB COMMUNICATING ORSA CAPABILITIES  5

time permits, one can conduct further analysis and provide alternative solutions and 

recommendations. Be humble when explaining new concepts to senior leaders; for example, 

introducing the concept of normalizing casualty statistics against the population of a security 

service, e.g., 52 killed in action/10k troops. Be prepared to explain what you mean in clear 

simple language and how normalizing casualties provides context not found in the raw numbers.

Prepare concise reports and clean presentations. Two or three pages or slides are usually 

sufficient. When possible, an ORSA analyst should present his or her results in one page or 

slide. One should write in Standard English so everyone can understand. Presentations should 

be brief and direct. One should highlight or bold the insights/results of the analysis.

Do not sell the algorithm; sell the results. Hardly anyone cares that one performed a 

Chi-Squared test with 41 degrees of freedom or a logistic regression with 12 independent 

variables with interactions. An ORSA analyst should present findings/results of the analysis and 

their significance to the mission. Too many ORSA analysts spend too much time writing and 

talking about how they did the analysis. Senior leaders do not need the process; they need the 

results. One should share his or her conclusions and make recommendations.

1.6 Summary

This chapter explained the role of the ORSA analyst. It was designed to teach a deploying 

ORSA analyst how to (1) communicate his or her profession to others; (2) apply his or her skill

set in a deployed environment; and (3) become the go-to person in his or her command. Good 

communication skills are vital to one's success as a deployed ORSA analyst. Effective 

communication does not replace sound analytical work; both are required in a deployed 

environment.
CAA-2015094

6  COMMUNICATING ORSA CAPABILITIES DAHB

(THIS PAGE INTENTIONALLY LEFT BLANK)
CAA-2015094

DAHB DATA MANAGEMENT METHODS  7

CHAPTER 2 DATA MANAGEMENT METHODS

2.1 Why Data Management?

Data Management is the process of planning, coordinating, and controlling an organization's data 

resources. Data management is generally a Chief Information Office (CIO)/ G-6 function; 

however, in a deployed environment, ORSA analysts play a significant role in data management. 

Key responsibilities associated with data management include data acquisition/collection, data 

indexing/cataloging/storage, and data verification/validation.

This chapter provides an overview of data management methods. It is not a cookbook of 

detailed procedures or an exploration of abstract, conceptual techniques. Rather, it provides an 

analytical approach that is responsive to the needs of decision makers at the highest levels of 

military command. It includes data management topics ranging from general data considerations 

to specific data requirements: data collection, data entry, data reporting, and data analysis.

2.2 Data Management Goals and Objectives

ORSA analysts seldom render advice based on perfect information. An ORSA analyst's task is 

to discern which data are pertinent and meaningful and then use these data in the problem 

solving process. Inappropriate, inaccurate, and invalid data are sources of fruitless analytical 

effort (sound study efforts based on sound analytical designs are of little value without valid 

data). It is a rookie mistake to base judgments on the unchallenged use of official figures. One 

must challenge his or her sources of information. One must examine the data that researchers 

used to reach their conclusions. One must determine how researchers derived their findings, and 

one must check data for accuracy. A key function of analysis is to reject bad data and inaccurate 

information. Ensuring and maintaining data integrity is fundamental to the military mission and 

requires a considerable investment of time and energy.

2.3 Data Management Process Overview

Data Flow Model. A simple data flow model provides a good starting point for understanding 

how to carry out data handling procedures. Figure 1 identifies five key steps in data flow: 

acquisition, verification, validation, analysis, and dissemination. These steps—plus additional 

procedural details—are described in the following pages. Additionally, storage, maintenance, 

and security issues apply at all stages of data flow.

Figure 1. Steps in data flow model.

Data Acquisition. Acquiring and collecting data from many internal and external sources is 

critical and often complicated. Sometimes there is a need to collect raw data in the field, such as 

post blast details from an improvised explosive device (IED) event. Other times, there is a need 

Storage, Maintenance, Security

Acquisition Verification Validation Analysis Dissemination
CAA-2015094

8  DATA MANAGEMENT METHODS DAHB

to elicit data from other people, e.g., polling/survey data. Data may be acquired through data 

mining or field studies conducted personally or by other analysts. Regardless of one's method of 

collection, one must filter and validate the data to avoid garbage in, garbage out. Data quality is 

an extremely important issue.

2.4 Three Common Characteristics of Data Collection

Data Population for Measurement (Who?) In order to choose the most appropriate statistical 

methods of analysis, one must precisely define the population he or she wishes to measure. The 

specific segments of the general population one samples are called sampling units. Sampling 

units must be unique, easily identifiable, and selectable. For instance, if a poll involves Soldiers 

from multiple units, the researcher must capture certain criteria associated with each Soldier,

e.g., rank, branch, duties, and responsibilities, so he or she can easily identify all measurements 

from this population. If the measured population includes Private Jones, an Infantry Soldier with 

the 101st Airborne Division who served in Afghanistan from May 2014 to May 2015, the 

researcher must identify measurements associated with this Soldier, i.e., personal characteristics 

and professional activities. Sampling units must be both discrete (countable) and measurable 

using standardized methodologies.

Items for Measurement (What?) After one defines the population and sampling units, one 

must define and select the specific data he or she wishes to measure. Measured data must have 

clear definitions, recognizable characteristics, a clearly defined set of observable measurement 

parameters (qualification standards), and a discrete name. For example, when a soldier submits 

an initial spot report, he or she describes enemy activity, gives a clear description of what the 

friendly forces were doing at the time of the incident (patrol, recon, etc.), pinpoints where the 

incident took place (grid location), how many soldiers were involved in the incident, what kind 

of incident occurred, e.g., IED explosion, and what the outcome of the incident was, i.e., battle 

damage assessment. One should treat each of these data points as discrete pieces of the event to 

facilitate further analysis and comparison with future or past events that are similar.

Methods of Performing the Measurement (How?) Data are generally classified as objective 

or subjective, depending on the amount of human judgment involved with measuring them. 

Objective data have physical/tangible properties, e.g., grid location, speed of a vehicle, number 

of vehicles in a convoy and require little or no human judgment. On the other hand, subjective 

data, e.g., lessons learned captured from surveys submitted by redeploying soldiers, require 

human judgment. For some aspects of data collection, objective data may ultimately give 

sensitive, reliable, and valid results. Other data, such as soldier feedback and assessments, 

require subjective measurements and interpretation. A researcher must evaluate, score, and 

assign a rating to each subjective measurement. He or she must assess and measure the validity 

of each. In every case—to the greatest extent possible—a researcher must minimize his or her 

personal biases in order to ensure valid and reliable data collection.

2.5 High Quality Data Attributes

Sensitivity. Sensitivity means the extent to which a small change in a parameter results in a 

relatively large change in the outcome. If a small change in a parameter results in a relatively 

large change in the outcome, the outcome is said to be sensitive to that parameter.
CAA-2015094

DAHB DATA MANAGEMENT METHODS  9

Reliability. Reliability means free from random error in the final measurement, usually 

indicated by consistency among items or stability in the measurements over time. Both sampling 

and non-sampling errors affect reliability. Sampling errors are the inevitable result of basing an 

inference on a random sample rather than on the entire population. Non-sampling errors occur 

from non-response, coding errors, computer processing errors, errors in the sampling frame, 

reporting errors, etc.

Validity. Validity means that the data accurately represent what they are intended to represent. 

Data validity depends partly on having adequate sensitivity and reliability of the measures; 

however, even with adequate sensitivity and reliability, the data may represent something other 

than what they appear to represent. For example, a poorly worded survey question may yield 

results that do not accurately answer the desired question (invalid results). Vague data 

definitions, insufficient interviewer training, casual data collection methods, and operator 

discretion weaken or reduce data validity.

Standardization. Standardization means ensuring uniformity in data collection procedures. 

Standardization ensures that data differences are not procedural differences. The standardization 

attribute is very sensitive to non-sampling errors associated with quality of data, collection 

techniques, and the stability of established data collection procedures.

Completeness. Completeness means that all the expected data elements, records, observations, 

etc. are present. Non-sampling errors affect completeness. Common causes of incomplete data 

include (a) failure to enforce data collection standards outlined in the standardized data entry 

report, (b) failure to report all of the required information, (c) failure to secure records from 

which the data are to be collected, (e) inability to read/scan the data collection form, and (f) 

corrupted files.

2.6 Data Entry

Data entry is the initial set of operations used to transcribe the raw source data (survey results, 

initial spot reports, etc.) into a computerized format, i.e., a database. Data entry begins when 

data collection is complete. Data entry forms and quality control features are extremely 

important in minimizing error. One should standardize data entry forms across all levels of 

command to ensure consistency in data collection. Standardized forms should contain logical 

formats that include discrete data entry choices e.g., drop down menus, to reduce transcription 

errors. Inevitably, the process of transcribing data from field forms to a digital format introduces 

error. To minimize errors, one must standardize data entry forms and quality control measures 

throughout the data management process. For example, data entry forms reduce transcription 

errors through pick lists, value limits, and controlled access to the database. Forms set for "data 

entry only" prevent accidental deletion or alteration of existing data. One can also set key fields 

to prevent duplicate entries such as variations of a unit's name. With standardized forms, one 

can also control the sequence of data entry. A well-developed data entry format enables users to 

search for and retrieve consistent results that are free from duplicate entries.

2.7 Data Verification

Irrespective of how one acquires the data, one must import the data into the database using a 

process that ensures verification. Data verification immediately follows data entry and involves 

checking the accuracy of source data. While one rarely achieves 100% error-free data entry, one 
CAA-2015094

10  DATA MANAGEMENT METHODS DAHB

can minimize transcription errors by using a layered approach to data verification. One should 

use quality control measures throughout the data management process, from the entry of an 

initial report, to the consolidation of massed reports into an official database, to retrieval at 

higher headquarters where one uses the data for analysis and decision making. A good rule of 

thumb is for one to verify 10% of all records for a specified period, focusing on key aspects of 

the report e.g., event location, event category, etc. One should compare these audited records 

with the existing record and include this information in the finalized report. If one finds 

reporting errors, he or she should review and re-verify the entire data set. Once one verifies that 

the computerized data accurately reflect the original field data, he or she can index and archive 

the reports for later retrieval.

2.8 Data Validation

Data validation ensures the accuracy of the original source field reports. Data should be accurate 

and logical. For example, a report stating the convoy speed was 150 mph is illogical and almost 

certainly incorrect, whether or not the original recorder properly transcribed the data from field 

reports. As another example of validated data, spatial data collected within a certain boundary 

should appear within that boundary when viewed in a geographic information system (GIS) 

environment. The process of reviewing computerized data for range and logic errors is the 

validation stage.

Database developers build certain components of data validation into their programs, e.g., range 

limits. ORSA analysts can perform additional data validation during verification if they are 

sufficiently knowledgeable about the data. During data validation, an ORSA analyst should 

identify generic errors, e.g., missing, mismatched, or duplicate records, as well as errors specific 

to particular items and types of activities. For example, validation of enemy threat data includes 

database queries and comparison of data over different years. One query may detect records of 

specific location identification (ID) and a certain type of enemy activity, e.g., IED attack. 

Another query may count the number of IED attack plots per sample site to ensure that personnel 

entered all plots correctly. Additionally, an ORSA analyst can compare current data with 

previous years to identify gross differences. One should ensure consistency between field forms 

and the database by noting how and why personnel made changes to the data on the original field 

forms. Once data validation is complete, the data can be archived for later retrieval.

2.9 Data Analysis

As described by Albright, Winston, and Zappe (2011), there are seven basic steps to data 

analysis: (1) define the problem, (2) collect and summarize the data, (3) formulate a model, (4) 

verify the model, (5) select one or more suitable decisions, (6) present results, and (7) update the 

model as new data become available. These steps aid in planning the proper course of data 

analysis. Simply put, there are a myriad of statistical tests, optimization methods, and simulation 

methods, and one must determine which technique is appropriate for a given situation. 

Analytical tools used to assist in the process include, but are not limited to, Excel, Access, ESRI 

Arc Geographical Information System (ArcGIS), SAS Institute Statistical Analysis System 

(SAS), and IBM's Statistical Package for the Social Sciences (SPSS).2 All data analysis 

 

2 Note: No endorsement is stated or implied by any references to commercial software applications in this 

handbook.
CAA-2015094

DAHB DATA MANAGEMENT METHODS  11

techniques should comport to scientific and professional standards. One should analyze results 

for statistical significant.

2.10 Data Dissemination

One should follow the appropriate review process and share results with all identified users. One 

should display data in a logical and consistent format and follow ownership protocol guidelines. 

To accomplish this goal, one should use standardized procedures to quality-check, analyze, 

document, catalogue, archive, and make data available for further analysis and decision making. 

It is especially important to provide well-documented data in a timely manner to those who 

depend on readily accessible, accurate, and complete data. One should consider the following 

advice:

 Data should be easily discoverable and obtainable.

 Data results should not be released until appropriate authorities have granted permissions.

 Complete metadata (characteristics of information-bearing entities) that clearly establish 

the source and content of the data should be included.

 Data should be properly identified and protected from unauthorized access and 

inappropriate use, e.g., sensitive data in reports, metadata, raw and manipulated spatial 

and non-spatial data, maps. One should clearly identify classified data and follow all 

rules of use and distribution.

At times, one may need to export data from one database to another, e.g., from Access to SPSS, 

ArcGIS, or FalconView for further analysis. Data integrity must be maintained when 

transferring data between software tools. One can most easily control the field order, i.e., order 

of variables in the resulting 'flat' file, while the data is still in the relational database. One should 

determine field types early on in the database development process. One should keep data type 

changes to a minimum in order to minimize risk to data loss. This handbook recommends using 

the American Standard Code for Information Interchange (ASCII) text for its nearly universal 

readability across applications. ASCII text typically delimits fields by commas, tabs, or spaces, 

and encloses text strings by single or double quotes. ASCII text requires extra steps to transfer 

data between applications and extra care regarding data formatting. The preferred alternatives 

are data exports directly from an Access .mdb file (such as ArcGIS) or link to an .mdb file by 

way of a database connection such as an Object Linking and Embedding Database (OLE DB) or 

an Open Database Connectivity (ODBC) data link.

2.11 Data Organization

The various databases, reports, GIS coverages, etc. create a large number of files and folders to 

manage. For example, databases are occasionally stored in two versions of Access in order to 

accommodate data users with different software versions. Additionally, GIS data are sometimes 

stored in two projections, one for navigation and another for use with existing base GIS data. 

Poor file management leads to confusion and data corruption.

2.12 Data Maintenance

Data sets are rarely static. They often change through additions, corrections, and improvements 

following the archival of a data set. There are three main caveats to this process: 
CAA-2015094

12  DATA MANAGEMENT METHODS DAHB

 One should only make changes that improve or update the data and maintain data 

integrity.

 When archiving the data set, one should document change versions made to the data set.

 One should prepare backup files to recover from mistakes made during editing.

2.13 Version Control

Secure data archiving is essential for protecting data files from corruption. Version control of 

records is a critical piece of the data management process. Prior to making any major changes to 

the dataset, one should save the data set using a unique version number. This facilitates tracking 

changes over time. With proper controls and communication, versioning ensures that one uses 

only the most current version in analysis. One should use a logical numbering system and assign 

each additional version a sequentially higher number. When one makes new versions to the data 

set, one should provide all frequent users with the most recently archived version.

2.14 Operational Competence

ORSA analysts must remain operationally competent across the full spectrum of resident skills. 

One becomes operationally competent by integrating oneself into the staff process early in the 

planning process and by remaining engaged all the way through to the execution of operations. 

One's ability to access and utilize multiple data sources ensures that the commander has the most 

accurate, relevant, and timely information available. For example, ORSA analysts deployed with 

the International Security Assistance Force (ISAF), the Combined Joint Special Operations Task 

Force – Afghanistan (CJSOTF-A), the Joint Special Operations Task Force – Philippines 

(JSOTF-P), and the Combined Joint Task Force – Horn of Africa (CJTF-HOA) have utilized 

multiple data sources to

 analyze the projected size of the Afghan National Security Forces (ANSF);

 develop a process to locate, validate, store, and distribute polling data for analytical use 

to support strategic and operational issues in Operation Enduring Freedom (OEF);

 analyze poll results of counterinsurgency operations to gauge the success of efforts to win 

the hearts and minds of the local population;

 develop assessment frameworks to inform commanders and future campaign planners;

 consolidate and format unusable, individual unit reports into a single database using text 

extraction techniques;

 assess the effectiveness of combat and security operations on enemy activities; and

 establish robust database systems to better facilitate analysis of assessments.

2.15 Way Ahead

Data for decision making comes from a variety of sources, both internal and external. Because 

the database management system is one of the major components of most decision management 

support systems, it is important for ORSA analysts to be familiar with the latest developments in 

the field. Organizations are recognizing that their data contain a gold mine of information. 

Consequently, they are warehousing and mining data for users to obtain information on their 
CAA-2015094

DAHB DATA MANAGEMENT METHODS  13

own (through a variety of multidimensional analytical tools and new enterprise-wide system 

architectures) and establish relationships (through data extraction techniques) that were 

previously unknown. New emerging tools, such as Online Analytical Processing (OLAP), 

provide on-the-fly data analysis that is invaluable to analysts. Moreover, a wide variety of data 

formats are becoming available through innovative networked database management systems. 

One can use any or all of these methods to enhance data analysis and assist commanders at all 

levels of command in making highly informed decisions.
CAA-2015094

14  DATA MANAGEMENT METHODS DAHB

(THIS PAGE INTENTIONALLY LEFT BLANK)
CAA-2015094

DAHB DATA ANALYSIS TECHNIQUES  15

CHAPTER 3 DATA ANALYSIS TECHNIQUES

3.1 Introduction

Definition. Data analysis is the act of transforming and modeling data to extract useful 

information and facilitate conclusions that support decision making.

Purpose. This chapter highlights several good data analysis techniques used by previously 

deployed ORSA analysts. As a deployed ORSA analyst, one makes inferences from data to 

recommend courses of action (COAs), assess specified activities, identify trends, etc.

Data analysis is an objective method for determining whether alternative COAs are significantly 

different from each other and for providing the commander with resource allocation 

recommendations. Data analysis underpins operational assessments, e.g., analysis of enemy and 

friendly activities that identify changes and trends in the operational environment. It is an ORSA

analyst's responsibility to determine which data analysis technique will provide commanders 

with the most accurate findings to assist them in their decision making process. Commonly used 

analytical techniques include statistical analysis, simulation, optimization, network flow models, 

and geospatial analysis.

3.2 Data Cleaning

Prior to beginning analysis, one may need to mine and clean the dataset. Typically, datasets 

contain missing data, multiple entries of the same event, and duplicate events entered with 

different terminology, all of which one must identify and correct prior to analysis. Furthermore, 

the necessary data may be located in "text" fields that require one to mine the data and extract the 

information. In a deployed environment, "dirty data" are quite common. ORSA analysts must 

not use "dirty data" in their analyses. These analyses are not trivial; they often become part of 

larger studies. Data cleaning is a labor-intensive task.

3.3 Statistical Analysis

Accurate analyses necessitate selection of the most appropriate technique for a given complex 

problem. One should consider events that may have an impact on the graphed data, e.g., major 

operations, elections, battlespace transition, and/or significant changes in the political 

environment, such as ratification of a constitution/seating of a parliament. These events can 

explain spikes or dips in the data and place the data in context.

Descriptive Statistics. This handbook recommends visually inspecting the data set using a 

graph, e.g., scatter-plot, histogram, or time series chart prior to conducting the analysis. 

Statistics textbooks provide examples of descriptive statistics, e.g., Devore, 2012, and many online resources outline techniques for calculating numerical measures and developing graphs.

Statistical Techniques. One can find the most relevant trend-analysis examples by referencing 

the work of recently deployed ORSA analysts. Several statistical techniques used in recent 

projects conducted in theater and as part of the analytical reachback process include t-test;

regression, e.g., linear, multiple, and logistic; and time series forecasting, e.g., moving average, 

weighted moving average, and Holt-Winters method. This handbook also recommends 

"Forecasting: Methods and Applications" by Makridakis, Wheelright, and Hyndman (1998).
CAA-2015094

16  DATA ANALYSIS TECHNIQUES DAHB

3.4 Other Analytical Methods

Deployed ORSA analysts may find that the following analytical methods exceed combat-theater 

resources, i.e., time, skill, software; however, one can still use these methods through the 

reachback program. 

Forecasting. Forecasting is the process of developing estimates for future outcomes that are 

likely to occur. Modern forecasting falls into one of two broad categories: causal forecasting or 

time series analysis. Causal forecasting predicts how an uncertain quantity relates to other 

quantities. Time series forecasting predicts future values of an uncertain quantity based on past 

values of a known quantity. Forecasting, i.e., predicting future events, is important as the United 

States continues the fight against terrorism. Senior leaders may ask ORSA analysts to develop 

tools and models to forecast enemy activity, both temporally and spatially.

One comprehensive forecasting resource is "Forecasting Principles: Evidence-based Forecasting" 

(2012), available from http://www.forecastingprinciples.com. Makridakis, Wheelwright, and 

Hyndman's "Forecasting: Methods and Applications" (1998) is also a good resource. Makridakis 

et al. provide descriptions and examples of basic forecasting tools, time series decomposition, 

simple regression, etc. Additionally, Sam L. Savage's "Decision Making with Insight" (2003) 

provides good explanations of causal and time series forecasting.

Simulation. Simulation is a mathematical model for recreating a situation repeatedly to more 

accurately estimate the likelihood of various outcomes. One can use simulation to show the 

eventual real effects of alternative conditions and COAs. Analysts widely use Monte Carlo 

computational algorithms for simulating the behavior of various physical and mathematical 

systems, and for other computations. One can use simulation in forecasting as well (for example, 

developing recommendations for resource allocation). Albright et al. (2011) include descriptions 

and examples of the simulation process. Additionally, Excel add-ins, such as @Risk, are 

designed to conduct simulations.

Optimization. One can use optimization, i.e., math programming, to minimize or maximize a 

real function by systematically choosing the values of real or integer variables from within an 

allowed set. One can use several different methods to conduct optimization, to include linear 

programming, network flow models, and critical path models. Over the past several years, 

commanders and senior leaders have asked ORSA analysts for optimal methods of employing 

technology, transporting technology and personnel to theater, optimizing coverage of specific 

assets, e.g., medical evacuation [MEDEVAC] and intelligence, surveillance, and reconnaissance

[ISR] technology. One can conduct simple optimization problems using the built-in "Solver" of 

Excel. Additional Excel software tools are available for sensitivity analysis to determine how 

the optimal solution changes as one or more input variables change. Albright et al. (2011) 

include detailed descriptions of optimization models using linear programming.

Sensitivity Analysis. Sensitivity analysis is the procedure for determining the sensitivity of 

alternative outcomes to changes in one or more parameters. If a small change in a parameter 

results in a relatively large change in the outcome, the outcome is sensitive to that parameter. 

This may mean that one has to determine the parameter very accurately or redesign the outcome 

to be less sensitive. One should incorporate sensitivity analysis into the decision making process 

to determine how sensitive the recommended solution is to various parameters of the model. 

Albright et al. (2011) provide a good example using Excel to conduct sensitivity analysis. 
CAA-2015094

DAHB DATA ANALYSIS TECHNIQUES  17

Additionally, the Precision Tree add-in discussed below is a good tool for conducting this type of 

analysis.

Gap Analysis. ORSA analysts—more often than not—have only incomplete data to answer 

specific questions. Where there are incomplete data, there are potential gaps in the analysis. 

One should identify these gaps in the "limitations" section of one's final written report. There are 

several ways to approach missing data, to include (1) deleting incomplete entries, (2) filling in 

incomplete entries based upon the most similar complete entry ("hot deck imputation"), (3) 

simulation, (4) filling in complete entries with the sample mean ("mean substitution"), or (5) 

using a learning algorithm, e.g., maximum likelihood to infer a missing entry. In one's written 

reports, one should annotate the methods used to overcome the gaps in data. As additional data 

become available, one should conduct gap analysis to "fill in the blanks" or run another iteration 

of the analysis to determine how the additional data change the results.

3.5 Software Utilization

This section summarizes software packages available to assist in the execution of data analysis. 

Senior leaders do not expect ORSA analysts to be experts in all of them; however, they do expect 

ORSA analysts to know the differences between them.

Excel built-in Functions. Several built-in functions in Excel improve the efficiency of analysis. 

In particular, the following three features are very powerful: pivot tables, sort options, and 

filtering. These features allow one to manipulate and visualize data quickly. Moreover, Excel 

has "add-ins" such as Analysis ToolPak for statistical analysis.

Excel add-in Software.3 The following Excel add-in software capabilities are available for 

purchase:

 @Risk for risk analysis. With this feature, one can define probability distributions and 

run simulations for input and output variables within Excel. 

 Precision Tree is a decision analysis add-in. One may find it useful in optimization 

when one is dealing with a set of alternatives such as decisions on using new weapons or 

factoring in decisions at each stage of contracting and development.

 Crystal Ball performs Monte Carlo simulations in spreadsheets that automatically 

calculate thousands of different "what if" scenarios and save the inputs and results of 

each calculation as an individual scenario. Analysis of these scenarios reveals the range 

of possible outcomes, the probability of each occurring, which inputs have the greatest 

effect on the model, and where one should focus one's efforts.

Macros. One can create a Macro to automate repetitive actions, i.e., a series of recorded actions. 

One can write Macros using the built-in toolbar or Visual Basic for Applications (VBA) code in 

the VBA editor.

 

3 Note: No endorsement is stated or implied by any references to commercial software 

applications in this handbook.
CAA-2015094

18  DATA ANALYSIS TECHNIQUES DAHB

3.6 Examples of Available Software Packages4

 SPSS is used for statistical analysis, data management, and data documentation.

 Stata is used for data management, statistical analysis, graphics, simulations, and custom 

programming.

 TRAC creates and maintains visualization tools for DoD analysts.

 DECMAT is a well-known multi-criteria decision matrix, i.e., preference matrix used in 

the military decision making process (MDMP).

 JMP is visual software from SAS that sets itself apart by linking robust statistics with 

graphics on the desktop, producing visual representations of data that reveal context and 

insight impossible to see in a table of numbers. JMP allows one to be more efficient, 

tackle difficult statistical problems, and bring one's data analysis to a whole new level. 

Data and information visualization, design of experiments, and statistical modeling 

techniques from simple to advanced are all within one's grasp with this powerful 

platform. And when one makes JMP one's analytic hub, one can use other favorite tools: 

SAS, Excel, and R.5

 MATLAB is used for algorithm development, data analysis, visualization, and numerical 

computation.

 Minitab is used to analyze one's data and improve one's products and services with the 

leading statistical software used to implement Six Sigma worldwide.

6

 GAMS is used for mathematical programming and optimization.

 R (free software) is used for statistical computing and graphics.

 TABLEAU helps people see and understand data. Tableau helps anyone quickly 

analyze, visualize, and share information.7

 

4 Note: No endorsement is stated or implied by any references to commercial software 

applications in this handbook.

5 JMP software is available from http://www.jmp.com/en_us/software.html

6 Minitab software is available from http://www.minitab.com/en-US/Products

7 Excerpt from the TABLEAU website: http://www.tableau.com/
CAA-2015094

DAHB DATA ANALYSIS TECHNIQUES  19

3.7 Case Study: Time Series Analysis in a Theatre Strategic Headquarter8

Time Series Analysis in a Theatre Strategic Headquarter

Marcus Gaul and Eric Jesse

In: Williams, A., Bexfield, J., Fitzgerald Farina, F., and de Nijs, J., (Eds.). Innovation in 

Operations Assessment: Recent Developments in Measuring Results in Conflict Environments. 

Innovation in Capability Development Series, Vol. 1. Norfolk, VA: NATO Headquarters 

Supreme Allied Commander Transformation (2013).

Abstract

Time series analysis in its many facets and shapes, from data exploration to spatial 

analysis, and from qualitative to quantitative considerations offers a set of highly valuable tools 

for analysts in support of senior leader's decision making. Time series analysis can not only 

generate crucial insights by combining time and spatial domains, but it can help to identify 

unexpected relationships and opportunities that can be exploited by a command, and enable the 

assessment of operational effectiveness and developments in insurgent tactics. In this article we 

present the fundamentals of time series analysis and discuss some of its applications and uses 

stemming from our experiences of working in the Afghan Assessment Group (AAG) at the 

International Security Assistance Force's (ISAF) Headquarters in Kabul, Afghanistan.

Introduction

The AAG is the International Security Assistance Force's (ISAF) Headquarters' principal 

analysis and assessment organization. The AAG provides regular analysis and assessments of 

the campaign to the ISAF Commander, the Government of the Islamic Republic of Afghanistan 

(GIRoA), and the governments of troop contributing nations, the press, and the public. In 

assessing the campaign, AAG uses a variety of analytic methodologies, one of which is time 

series analysis, the focus of this discussion. Time series analysis, in particular, leverages the 

information advantage ISAF has achieved through its advanced theatre-wide data collection 

processes.

Our chapter begins with a general review of time series analysis and associated 

terminology. Then we will cover selected common analytical issues and challenges, to include 

 

8 Used by permission from Marcus Gaul and Eric Jesse In: Williams, A., Bexfield, J., Fitzgerald 

Farina, F., and de Nijs, J., (Eds.). Innovation in Operations Assessment: Recent Developments in 

Measuring Results in Conflict Environments. Innovation in Capability Development Series, Vol. 

1. Norfolk, VA: NATO Headquarters Supreme Allied Commander Transformation (2013).
CAA-2015094

20  DATA ANALYSIS TECHNIQUES DAHB

data collection and consistency, data aggregation, variable selection, seasonality, and nonconstant variance. Aware of these problems that frame analytic approaches, we next discuss 

methods AAG employs to analyze insurgent activity in Afghanistan, covering year-over-year 

differencing, seasonal decomposition, and regression analysis. This carries into a discussion on 

the identification of statistically significant changes in insurgent behavior. Finally, we discuss 

the incorporation of geospatial considerations with temporal data. By adding spatial data, we can 

visualize, assess, and explain trends, which might otherwise go unobserved by analysts and 

decision makers. We conclude with a discussion of the challenges quantitative analysis 

encounters during the ISAF campaign, and the facilitating measures taken to remove them. 

Time Series Analysis

A time series is a collection of observations obtained through repeated measurements 

over time. Time series are most often considered subsets of stochastic models or stochastic 

processes. Stochastic processes concern sequences of events governed by probabilistic laws. 

Stochastic models infer characteristics of a population using temporal or spatial data. Time 

series analysis utilizes the natural temporal order of the data to extract information about the 

underlying stochastic process. 

The extent of assumptions about the underlying data in a statistical analysis defines two 

classes of statistics, parametric and non-parametric. The most common models utilize 

parametric statistics. Parametric statistics assume that the data have some underlying structure, 

most typically about the underlying distribution (often the normal distribution) and variance 

(often constant over time). Some widely known parametric models are ordinary least squares 

regression, generalized least squares, autoregressive and moving average models. Nonparametric (or distribution free) models do not assume an underlying distribution for the data; 

instead, they try to determine the underlying system structure within the data. As a result, larger 

datasets are required for non-parametric models to make similar conclusions as analogous 

parametric test. Non-parametric methods often rely on ranking or ordering of data for analysis. 

The Kolmogorov-Smirnov test is one such well known non-parametric method used to test 

whether a data set follows an assumed probability distribution, or whether two data sets come 

from the same generating process. 

Our discussion of time series analysis focuses on parametric statistics, but we need to be 

aware that non-parametric analysis techniques exist when parametric assumptions are shown to 

be invalid. 
CAA-2015094

DAHB DATA ANALYSIS TECHNIQUES  21

Statistical analysis is further subdivided into linear and non-linear analysis. Linear

analysis assumes the dependent variable (response variables) can be explained by a linear 

combination of independent variables (explanatory variables). Linear analysis assumes an 

additive relationship between variables that enables a closed form solution to estimation. While 

linear models are intuitively easier to interpret and understand, most real world systems do not 

follow strictly additive processes. In non-linear analysis, the dependent variable is explained by 

a non-linear combination of independent variables. Examples of non-linear functions include 

Lorenz curves, exponential functions, logarithmic functions, and Gaussian functions. 

Exponential and logarithmic functions can be transformed into linear forms enabling a closed 

form solution, but in general, non-linear functions use numerical techniques to estimate 

parameters. 

Time series analysis can be used to answer two types of inquiries: descriptive and 

predictive. Descriptive inquiries seek to ascertain the existence of hypothesized relationships 

between the dependent and independent variables. For example, how do enemy-initiated attacks, 

the opportunities for leave, seasonality, or major Islamic holidays affect attrition, i.e., the 

planned and unplanned loss of personnel in the Afghan National Army? Descriptive analyses 

look for significance and direction of relationships between the dependent and independent 

variables. Predictive analysis projects potential future system states based on the historical 

relationship between a dependent variable and a set of independent variables. An example is 

predicting the level of enemy activity in 2013 using the historical relationship between the 

enemy activity level and independent variables such as the number of coalition forces9.

Data Collection and Aggregation

ISAF collects vast amounts of data with most of the data being provided by military units 

using standard reporting procedures. These units are not focused primarily on data collection, so 

data quality, completeness, and accuracy can vary. Reported and collected data also tends to 

evolve with mission objectives, potentially adding further complexity to analysis. The collected 

data are refined with subsequent reports categorized and transmitted to an analytical database 

that also serves as a historical archive. A centralized data collation function provides an 

additional layer of quality assurance by enhancing data consistency, eliminating duplicate 

reports, and identifying and reconciling database anomalies.

 

9 See (Gons, 2012) for an example of such a scenario.
CAA-2015094

22  DATA ANALYSIS TECHNIQUES DAHB

ISAF tracks additional data sources beyond military operational reporting. These reports include 

open source, United Nations, and non-governmental organization (NGO) reporting. Military 

operational reporting is usually entered into the database within 24 hours of occurrence and data 

entry. This lag can mislead or bias near-term analysis. Understanding the data is the foremost 

task of the analysts, since quantity, quality, and reliability of the data is a primary factor shaping 

the analysis.

Data aggregation is the next major consideration for the analyst. Data should not be 

aggregated beyond the relevant level of inquiry. For example, looking at theater-wide 

improvised explosive device (IED) events would be an inappropriate means of assessing a new 

mine rolling system in Kandahar Province.

The aggregation of security-related data in Afghanistan is at least a three-dimensional 

issue, with data having spatial, temporal, and categorical aspects. Spatial disaggregation presents 

a particular challenge due to the number of ways Afghanistan can be divided. ISAF seeks to 

map data with an accuracy of a few meters to a specific military grid reference system (MGRS) 

coordinate, but this level of accuracy is not always achieved, especially for reports from the 

Afghan National Security Force (ANSF). Almost all data are identified at the district level. The 

approximately 400 districts are in 34 provinces, which in turn are part of 6 geographically 

distinct regions in Afghanistan: East, South, Southwest, West, North, and the Capital – the ISAF 

Regional Commands (RCs). Numerous further groupings are possible, for example defined by 

climate, terrain, and population demographics. 

Aggregation can influence analytic results. Simpson's paradox is the phenomenon where 

a trend that appears in data subsets disappears or reverses when these subsets are combined. As 

a hypothetical example, we might find an improving security situation in 28 of Afghanistan's 34 

provinces when analyzed separately, but when looking across the entire theater, the trend could 

be reversed and security would appear to be worsening. This trend could be driven by 

dominating negative conditions in just two provinces, for example Helmand and Kandahar.

Disaggregation can potentially obscure important trends due to the effect of specific 

events, operations, or simple random noise. Disaggregation can also limit the power of statistical 

analysis by reducing the number of observations. There is no "correct" level of aggregation to 

employ across all analysis. Aggregate models are very important to understanding general 

theater trends, while disaggregated models enable the understandings that are important at 

operational and tactical levels. Disaggregate models help with the analysis of local effects of 

actions such as clear/hold operations, cache finds, or weapon seizures. The appropriate level of 
CAA-2015094

DAHB DATA ANALYSIS TECHNIQUES  23

aggregation is a judgment call by the operational analysts based on their understanding of the 

question and the tractability of the data. 

Adjustments for Seasonality

Insurgent activity in Afghanistan follows a strong cyclical process. This type of recurring 

pattern in data is known as seasonality.10 Nationwide insurgent activity in Afghanistan is 

closely tied to the rhythm of daily life, with the progression of the seasons strongly influencing 

engagement levels. Planting and harvesting crops demand the attention of individuals who might 

otherwise be fighting coalition or Government of the Islamic Republic of Afghanistan (GIRoA) 

forces, while the winter cold and snows close mountain passes limits movement that results in 

diminishing contact between insurgents and coalition forces.

As measures for insurgent activity, we consider, for example, security incidents11 or 

enemy-initiated attacks12 (EIAs). EIAs provide a perspective on the active, executed 

components of insurgent engagements against ISAF and the ANSF, while security incidents also 

include elements that failed to execute such as explosive devices that were found and cleared. 

For the purpose of this article, we consider enemy-initiated attacks as an approximation for 

insurgent activity. 

Figure 2 shows the periodic pattern of insurgent activity over the period from 1 January 

2007 through 31 December 2012. In July of 2011, EIAs averaged over 700 per week. Five 

months later, in December 2011, EIAs averaged just over 350 per week, a drop of approximately 

50%. Interpreting this decrease as a sign of an improving security situation could be misleading 

for several reasons. First of all, this comparison fails to account for seasonality, i.e., fluctuations 

in the insurgent activity due to the seasonal temperature changes over the course of a year. We 

can see this by considering EIAs several months later. In July of 2012, EIAs again averaged 

over 700 per week.

 

10 Many time series show cyclic variations that are known as seasonal fluctuations or seasonality. 

Insurgent activity in large parts of Afghanistan is correlated to the seasonal temperature 

fluctuations over the course of a year.

11 Security incidents comprise enemy-initiated attacks (enemy-initiated direct fire, indirect fire, 

surface-to-air fire and executed IED attacks), as well as potential IED attacks such as IEDs and

mines that were found and cleared.

12 Enemy-initiated attacks is a term that is used for the total of enemy-initiated direct fire, 

indirect fire, and surface-to-air fire attacks, as well as executed IED attacks, namely IED 

explosions and mine strikes.
CAA-2015094

24  DATA ANALYSIS TECHNIQUES DAHB

Figure 2. Weekly enemy-initiated attacks in Afghanistan (1 January 2007- 31 December 

2012).

An assessment of the security situation should include not only insurgent activity, but 

also quantitative and qualitative metrics of population perceptions of development and 

governance, as well as interdependencies among the variables. A comprehensive analysis of this 

type extends beyond the scope of this chapter. 

When analyzing the insurgency, it is useful to differentiate between systemic changes

and those changes that are attributable to the cyclical pattern of the conflict. ISAF assessments 

regularly utilize three approaches to adjust for seasonality year-over-year differences, classical 

seasonal decomposition, and regression analysis. 

Year-over-year (YoY) differences allow comparisons between seasonally analogous 

periods, e.g., comparing EIAs in January 2013 with January 2012. The comparison window is 

important, as periods that are too small risk confusing natural variance in the data with 

significant trends. HQ ISAF AAG found that a 1-week period was the shortest possible period to 

identify meaningful changes in insurgent activity from data noise. On the other hand, if YoY 

analysis spans too much time, operationally relevant factors may be obscured, giving the 

commander misleading information. 

When seasonal trends evolve slowly, it is sometimes possible to compare the last period 

with that immediately preceding to search for changes, e.g., for data with an annual seasonal 

trend we can compare the last week with the preceding week. This is an analogous approach to 

more traditional YoY differencing in that the two periods should have very similar seasonal 

influences. When they differ considerably, it could be indications of an emerging trend. 

0

200

400

600

800

1000

1200

1400

1600 Jan 07 Jul 07 Jan 08 Jul 08 Jan 09 Jul 09 Jan 10 Jul 10 Jan 11 Jul 11 Jan 12 Jul 12 Number of Attacks

Weekly Enemy-Initiated Attacks

20 August 2009 

Presidential Election

18 September 2010 

Parliamentary Elections
CAA-2015094

DAHB DATA ANALYSIS TECHNIQUES  25

A second approach commonly employed to seasonally adjust the data is classical 

seasonal decomposition. This method removes seasonal fluctuations from the data allowing the 

identification of long-term trends. The technique factors time series data (Xt) into two parts: a 

seasonal component (St) and a trend component (Tt). While both additive and multiplicative 

decompositions are available, multiplicative decompositions perform best when modeling 

insurgent activity in Afghanistan (Xt = Tt * St). Since we know a priori that the cycle affecting 

the time series of enemy-initiated attack data (Xt) is annual, we can calculate the two 

components of the seasonal decomposition using the following steps:

1) Compute a moving average for the series Xt, with the moving average window being 

of equal length to one season (i.e., 12 months, 52 weeks, 365 days). This step 

removes intra-seasonal variability from the data resulting in a smoothed time series 

(SMt).

2) Compute the ratio of the observed (Xt) and smoothed (SMt) series. This ratio is the 

raw seasonal component. 

3) Next the average seasonal component (St) is computed by averaging the raw seasonal 

components for each point in the time series (i.e., each month, each week, and each 

day). Various forms of averaging the raw seasonal components to determine the 

average seasonal component are possible. We recommend normalized medial 

averaging. This approach excludes outliers from the average, preventing particularly 

abnormal observations from influencing the seasonal adjustment in the same period 

for other cycles. 

4) The original time series (Xt) can now be seasonally adjusted by dividing it by the 

average seasonal component (St), producing the trend component (Tt). 

A seasonal decomposition of weekly EIAs in Afghanistan is depicted in Figure 3 showing the 

usefulness of this approach.
CAA-2015094

26  DATA ANALYSIS TECHNIQUES DAHB

Figure 3. Seasonal decomposition of weekly enemy-initiated attacks in Afghanistan (1 

January 2007- 31 December 2012).

Regression analysis is the third approach to seasonally adjust data and identify trends. 

Regression analysis is used to estimate relationships between a dependent variable and one or 

more independent variables. A major strength of regression analysis is that it assesses the 

relationships between multiple independent variables to the dependent variable. Variable 

selection is the process of identifying control variables in regression analysis. In Afghanistan, 

factors known to influence insurgent activity include coalition and ANSF force levels and 

significant events such as elections and Islamic holy days. The number of insurgent attacks on 

ISAF and ANSF patrols is related to the location and frequency of these patrols. In the case of 

elections and Islamic holy days, insurgent activity has two opposing characteristics. Prior to and 

0

200

400

600

800

1000

1200

1400

1600 Jan 07 Jul 07 Jan 08 Jul 08 Jan 09 Jul 09 Jan 10 Jul 10 Jan 11 Jul 11 Jan 12 Jul 12 Number of Attacks

Weekly Enemy-Initiated Attacks

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

1.8

2.0 Jan 07 Jul 07 Jan 08 Jul 08 Jan 09 Jul 09 Jan 10 Jul 10 Jan 11 Jul 11 Jan 12 Jul 12 Seasonal Component

Seasonal Component

0

200

400

600

800

1000

1200

1400 Jan 07 Jul 07 Jan 08 Jul 08 Jan 09 Jul 09 Jan 10 Jul 10 Jan 11 Jul 11 Jan 12 Jul 12 Trend Component

Trend Component
CAA-2015094

DAHB DATA ANALYSIS TECHNIQUES  27

during elections, insurgents attempting to intimidate the population accelerate attacks. 

Conversely, some Islamic holy days see lower levels of activity due to religious customs. 

When selecting control variables for factors that influence insurgent activity, it is 

important to distinguish between regular periodicity and irregular periodicity, i.e., periods or 

events that recur at the same time each year and those periods or events that occur on different 

calendar dates. Events that occur at the same time annually are subsumed in the seasonal 

adjustment. For example, insurgent activity decreases during the poppy harvest season each 

year, but since the harvest occurs at approximately the same time each year it is captured by the 

seasonal adjustment. Islamic holy days, however, follow a lunar calendar in which a year is 

always 12 lunar months, making the months not linked to the seasons. Key dates shift each year 

10 to 12 days. As a result, the effects of the annual Islamic observances of Eid al-Fitr (the 

"Festival of Fast-Breaking" marking the end of Ramadan) and Eid al-Adha (the "Festival of 

Sacrifice") will not be accounted for in a seasonal adjustment. Regression models, being the 

most powerful of the seasonal adjustment techniques will be explored in greater detail below.

Linear Regression Models

There are several types of regression models. In this section we discuss how simple

linear regression models can be applied to estimating trends in enemy-initiated attacks. Other 

approaches will be addressed in later sections. Determining the best model structure, as 

measured by statistically accurate parameter estimates, can be challenging. Ordinary least 

squares (OLS) regression assumes the independent variables are additive (linear).

In Afghanistan, as discussed earlier, some of the major factors related to insurgent 

activity, in particular seasonality, have a multiplicative relationship. We use a log transformation 

converting an approximately multiplicative process into a linear model.13 Techniques are 

available for treating seasonal adjustments simultaneously with other explanatory variables. If 

 

13 Coalition force levels strongly influence insurgent activity levels, but coalition force levels 

have varied significantly from less than 40,000 troops in 2006 to over 150,000 at the peak of 

the surge in 2011. As a result, the variance in insurgent activity has changed significantly over 

time. Uneven variance over time, heteroskedasticity, causes ordinary least squares estimation 

to be inefficient and overstates statistical confidence in measured estimates. A log 

transformation of the dependent variable makes a multiplicative model additive and in many 

cases significantly reduces or entirely removes the problem of heteroskedasticity. There are 

numerous methods to stabilize variance. A square root transformation is often also appropriate 

for count processes. Other variance stabilizations include the Anscombe transform 

(Anscombe, 1948), Freeman-Tukey transform (Freeman and Tukey, 1950), and Box-Cox 

transform (Box and Cox, 1964) amongst others.
CAA-2015094

28  DATA ANALYSIS TECHNIQUES DAHB

treated separately, seasonal adjustments will include some of the effects of the explanatory 

variables. Treating the adjustment simultaneously with other explanatory factors ensures that the 

seasonal adjustment itself does not include effects that are more correctly attributed to the other 

explanatory factors in the model. In a regression model, the addition of "indicator" variables for 

each observation period in the seasonal cycle (which is 1 year for the Afghan insurgency) is 

equivalent to a classical seasonal decomposition. An indicator variable is assigned a value of 

one when a condition is present and zero otherwise. For example, the seasonal decomposition in 

a regression model using weekly data includes 51 additional indicator variables, one for each 

week. The 52nd week is captured in the intercept, as it is perfectly identified by the 51 other 

weekly indicator variables and therefore cannot be directly estimated.

A simple (multiple) linear regression model of weekly insurgent activity expressed by 

seasonality, coalition and Afghan Nation Army (ANA) force levels, and the holy days of Eid alFitr, and Eid al-Adha, can be written as follows (with each variable considered a vector of the 

data indexed by week):

 

where

 ,

 ,

 ,

 - , where is a scalar function assigning the value of one to a week in 

which the days of Eid al-Fitr fall, and the value of zero otherwise,

 - , where is a scalar function assigning the value of one to a week 

in which the days of Eid al-Adha fall, and the value of zero otherwise,

 , where is a scalar function assigning the value of

one to a week in which an election falls, and the value zero otherwise,

 is the indicator (or dummy) variables for each observation period, i.e., 

week, in the year, and
CAA-2015094

DAHB DATA ANALYSIS TECHNIQUES  29

 is the residual error.

The parameters are computed using OLS, i.e., by minimizing the sum of squared 

deviations between the dependent variable (observations) and the estimate using the independent 

variables. Each parameter measures the observed historical relationship the independent variable 

has with the dependent variable.

The observed and predicted results of this model for the Afghan theater of operation are 

shown in Figure 4. The results of the OLS for nationwide and four Regional Command (RC) 

models are presented in Table 1.

Figure 4. Observed versus predicted enemy-initiated attacks (EIAs).

Coefficient Theater RC South

RC 

Southwest RC East RC North

Intercept

-9.76 (0.26) 

***

-9.31 (0.46) 

***

-15.59 (0.54) 

***

-8.75 (0.29) 

***

-15.59 (0.54) 

***

log(Coalition

)

1.11 (0.07) 

***

0.76 (0.13) 

***

1.98 (0.15) 

***

0.48 (0.08) 

***

1.98 (0.15) 

***

log(ANA)

0.21 (0.06) 

***

0.38 (0.11) 

*** -0.22 (0.12) *

0.63 (0.07) 

*** -0.22 (0.12) *

Eid al-Fitr

-0.26 (0.07) 

***

-0.30 (0.12) 

**

-0.33 (0.14) 

**

-0.21 (0.08) 

***

-0.33 (0.14) 

**

Eid al-Adha -0.10 (0.07) -0.18 (0.12) -0.04 (0.14) -0.04 (0.08) -0.04 (0.14) 

Election 

Week

0.54 (0.11) 

***

0.50 (0.19) 

*** 0.36 (0.22) 

0.63 (0.12) 

*** 0.36 (0.22) 

R

2

0.9525 0.8596 0.8736 0.9423 0.8736

Table Format: Coefficient (Standard Error) Significance 

Significance: *** at 0.01, ** at 0.05, * at 0.1

Table 1. Results of the OLS model by region in Afghanistan.

0

200

400

600

800

1000

1200

1400

1600 Jan 07 Jul 07 Jan 08 Jul 08 Jan 09 Jul 09 Jan 10 Jul 10 Jan 11 Jul 11 Jan 12 Jul 12 Number of Attacks

Weekly Enemy-Initiated Attacks

20 August 2009 

Presidential Election

18 September 2010 

Parliamentary Elections
CAA-2015094

30  DATA ANALYSIS TECHNIQUES DAHB

We see in Table 1 that the parameters point in the hypothesized directions. Insurgent activity 

rises with increases in coalition and ANA force levels, Islamic holidays lead to decreases in 

insurgent activity, and elections result in increased insurgent activity. The only variable that 

does not point in its hypothesized direction is ANA troop level in RC North. This may have 

occurred because the ANA troop levels are modeled at the theater level, and this may be an 

inadequate measure of ANA forces in RC North. It could also have occurred by chance with the 

measured effect being only weakly significant at a confidence level of 0.1, i.e., with a 90% 

chance of being a significant relationship. 

Looking at the nationwide model, assuming all else to be equal, the interpretation of the 

model results is as follows:

 A 10% decrease in the number of coalition forces is correlated with a 11% decrease in 

insurgent activity

 A 10% increase in ANA forces is correlated with a 2% increase in insurgent activity

 A week encompassing Eid al-Fitr has a 26% decrease in insurgent activity

 An election weeks results in a 54% increase in insurgent activity

Finally, it should be noted that each of the five regression results in Table 1 have an 

associated R2. This is sometimes called the coefficient of determinations and it is a measure of 

the proportionate reduction of total variation associated with the use of the independent 

variables. It can take on values between zero and one, with higher numbers like those seen in the 

table implying that the regression model is a good fit to the data.

The levels of significance displayed in Table 1 are calculated assuming that the residuals 

are normally distributed, with constant variance, and are not correlated with each other. In 

general, it is necessary to test these implicit assumptions of parametric models and verify model 

validity. Violation of these assumptions can bias results and mislead conclusions. 

Autocorrelation is one common issue in time series data. It occurs when the current 

observation is correlated with an observation at another time period, e.g., the immediately 

preceding one, violating the independence assumption of OLS.

Non-constant variance is a second potential issue. As coalition force levels have 

varied significantly, from less than 40,000 troops in 2006 to over 150,000 at the peak of the 

surge in 2011, the variance in reported insurgent activity has changed significantly over time. 

Uneven variance over time, heteroskedasticity, may result in overstated statistical confidence in 

parameter estimates. 
CAA-2015094

DAHB DATA ANALYSIS TECHNIQUES  31

We can test the OLS assumptions visually and with more formal tests. Figure 5 shows 

selected residual analysis for the nationwide model in Table 1. The top plot shows studentized 

residuals. Studentized residuals are normalized by their variance and thus should align with a 

standard normal curve with most observations falling between -2 and +2. The bottom plot 

compares a histogram of the residuals to a normal distribution. We can see that the residuals do 

not appear to violate the normality assumption.

The residuals show signs of autocorrelation, with insurgent activity in one period 

appearing strongly related to insurgent activity in adjacent periods, as visible in Figure 5. 

Having homoskedastic residuals (residuals with constant variance over time) that are reasonably 

balanced (centered about zero over time), our estimates are unbiased, but our confidence in these 

estimates is overstated. In order to correct the effects of autocorrelation (or heteroskedasticity if 

balanced) in the error terms from OLS regression, the autocorrelation-consistent Newey-West 

variance-covariance matrix estimator of the residuals is often used. Alternatively, we can utilize 

a generalized linear model and explicitly address the autocorrelation. 
CAA-2015094

32  DATA ANALYSIS TECHNIQUES DAHB

Figure 5. Selected residual analyses.

Autoregressive, Integrated Moving Average (ARIMA) Models

Ordinary least squares is one of many forms of regression analysis (Hamilton, 1994). As 

the event data are actually count data, Poisson or negative binomial models under certain 

conditions can yield much more accurate predictions. If data fidelity improved and limitations in 

computing power were removed, point process models like a Cox process result in even higher 

fidelity characterizations of the data over space and time (Cox 1955; Cox and Isham, 1980; Cook 

and Lawless, 2007; Allen et al., 2008). 

A class of models that perform well in forecasting non-stationary time series data, like 

insurgent activity in Afghanistan, are autoregressive, integrated moving average (ARIMA) 

-4

-3

-2

-1

0

1

2

3

4 Jan 07 Jul 07 Jan 08 Jul 08 Jan 09 Jul 09 Jan 10 Jul 10 Jan 11 Jul 11 Jan 12 Jul 12 Studentized Residuals

Studentized Residuals

0

0.05

0.1

0.15

0.2

0.25

Frequency

Studentized Residuals

Observed Frequency

Normal Distribution

-4 -3.5 -3 -2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5 4
CAA-2015094

DAHB DATA ANALYSIS TECHNIQUES  33

models (Box et al. 2008)14. ARIMA models consist of three parts, an autoregressive (AR) part, 

an integrated (I), and a moving average part (MA). The model is usually referred to as an 

ARIMA(p, d, q) model where the parameters p, d, and q are non-negative integers. The 

parameter p refers to the order of the autoregressive part, i.e., the number of autoregressive 

terms, d refers to the order of the involved differencing (note that a time series that needs to be 

differenced to be made stationary is said to be an "integrated" version of a stationary series), and 

q is the order of the moving average part of the model. An ARIMA model specification can be a 

subset of the autoregressive, integrated, and moving average components. None of the parts of 

the ARIMA is required be included in its final specification. 

An autoregressive model seeks to define the current observation as a function of 

preceding observations. For example, an autoregressive model of order one is given by

 , while the general autoregressive model of order p is given by 

 

 

 

.

The integrated part of the model utilizes successive first differences. For example, for a 

difference of order 2, the relationship analyzed is with the variable defined by the difference 

between the current and previous periods minus the difference between the previous period and 

the period preceding, mathematically expressed as 

 . The most 

common use of such an integrated term is to account for a quadratic trend in the data.

A moving average model defines the current observation as the mean μ of the series, and 

current and previous error terms . For example, a moving average model of order one is given 

by , and a moving average model of order q is given by 

 

 

 

. Because the lagged error terms are not directly observable, iterative non-linear 

fitting procedures need to be used in place of linear least squares to estimate model parameters. 

 A seasonal ARIMA is an adaptation of the standard ARIMA that incorporates a priori

information about the recurring seasonal pattern. In a seasonal ARIMA model with a recurrence 

period of time S, observation is predicted by data values, differences, and errors with time 

lags that are multiples of S (the span of the complete seasonal cycle). For example, a seasonal 

autoregressive model of order two is given by . 

 

14 A stationary process is a stochastic process whose joint probability distribution does not 

change over time. This means parameters like the mean and variances are constant over time, 

i.e., time invariant. The data here exhibits cyclostationary properties - it varies annually. In 

order to perform time series analysis we transform the data to become stationary (Priestley 

1988). 
CAA-2015094

34  DATA ANALYSIS TECHNIQUES DAHB

An ARIMA model as briefly discussed above has several benefits. It allows us to model 

seasonality using fewer terms than in the (multiple) linear regression model described earlier, 

increasing estimation power. It also allows simultaneous estimation of non-seasonal factors like 

force levels, or Islamic holy days Eid al-Adha and Eid al-Fitr. An ARIMA model will also 

identify, capture, and reflect unspecified underlying processes that might be eliminated as noise 

in other types of estimation models, thus making it a very powerful tool for forecasting and 

prediction. 

At the theater strategic level we have used seasonal ARIMA models to forecast insurgent 

activity taking several factors into account, primarily the projected troop level changes. Figure 6

shows the model predictions of insurgent activity in 2012 based on historical data from 2007 to 

2011. The troop level changes in 2012 that are factored in the prediction are shown, as well as 

the confidence bounds on the predicted EIA levels. The seasonal ARIMA has a non-seasonal 

part of (1, 0, 0) and a seasonal specification of (0, 2, 1); it also includes control variables for 

troop levels, Eid al-Fitr, and Eid al-Adha; however, no examination of the structural, systemic 

causes for insurgent activity, or the changes and dynamics are considered.
CAA-2015094

DAHB DATA ANALYSIS TECHNIQUES  35

Figure 6. Seasonal ARIMA prediction model for insurgent activity.

Note that the model's overestimation of insurgent activity for 2011, as seen in Figure 6, is caused 

by not controlling for voter registration periods or the 18 September 2010 Parliamentary 

Election.

Identifying Operationally Relevant Changes

The identification of operationally relevant changes is one goal of analysis at a strategic 

headquarters. The ability to rapidly evaluate and assess changes, identify opportunities first, and 

provide the option to react faster, can provide a significant advantage over the opposition. ISAF 

developed a methodology that utilizes regression models to quickly identify measurable changes 

in insurgent activity and determine the statistical significance of this change (Jesse, 2013).

The procedure begins by defining a model to explain insurgent activity, and estimating it 

against a benchmark period of data. We next compute the residuals for this benchmark period, 

which are defined as the difference between model predictions and observed insurgent activity. 

The residuals, or fitting errors, to regression models often follow a distribution that is, in many 

0

20,000

40,000

60,000

80,000

100,000

120,000

140,000

160,000

180,000

0

200

400

600

800

1000

1200

1400

1600 Feb-11 Mar-11 Apr-11 May-11 Jun-11 Jul-11 Aug-11 Sep-11 Oct-11 Nov-11 Dec-11 Jan-12 Feb-12 Mar-12 Apr-12 May-12 Jun-12 Jul-12 Aug-12 Sep-12 Oct-12 Nov-12 Dec-12 Enemy

Coalition Troops Levels

-Initiated Attacks

Prediction of Enemy-Initiated Attacks for 2012 

Observed EIAs Prediction Upper/Lower 85% Conf. Bound

PROJECTION

Troop Levels
CAA-2015094

36  DATA ANALYSIS TECHNIQUES DAHB

cases, a normal distribution with mean zero. We can estimate the parameters of this residual 

distribution and design a statistical test to search for un-modeled changes in insurgent activity. 

With the model predictions over the benchmark period, we can forecast insurgent activity levels 

for new time periods and compare these with observed insurgent activity. From the sequence of 

errors obtained by this procedure, and the known information about the residuals' distribution, 

we can find the statistical likelihood for a significant change (Chandler and Scott, 2011).

One use of the above approach at ISAF is to monitor daily insurgent activity. We use a 

two-stage least-squares model15 of daily enemy-initiated attacks baselined on the period from 1 

January 2010 to 31 October 2012 to monitor changes in insurgent activity. 

The advantage of a two-stage model is its separate accounting for additive and 

multiplicative relations. The first stage models enemy-initiated attacks using ISAF and ANSF 

troop levels, which explains much of the growth in insurgent activity. For the second stage, the 

observed enemy-initiated attack values are normalized by the predictions from the first stage. 

This procedure results in the dependent variable being normalized around one (seasonal 

fluctuations persist), and it should remove any heteroskedasticity. Fitting the second stage on the 

normalized dependent variable, independent variable coefficients estimate the relative change 

from the mean caused by the variables. In the example, fitting the second stage model, we get 

estimates of the effects of the Islamic holidays of Ramadan, Eid al-Adha, and Eid al-Fitr 

seasonality approximated by month. 

In the next step, we compute the residuals of the two-stage model. The residuals of this 

model should closely approximate a normal distribution with a mean of zero and constant 

standard error. We then employ a t-test to calculate the likelihood that the observed insurgent 

activity deviates significantly from activity explained by modeled factors. Or stated differently, 

it tests the null hypothesis that the mean of a sample population, the residuals, is equal to a 

specified value .

The t-statistic is found by taking the mean of the sample population , minus the test 

value , and dividing this by the sample standard deviation , where is the sample size. 

 

15 A two-stage model can allow implicit correction of heteroskedastic variance in a dependent 

variable. In the first modeling stage we control for additive independent variables and in the 

second stage multiplicative independent variables. This approach can outperform dependent 

variable transformations, as it incorporates a priori knowledge about modeled relationships. 

The approach yields unbiased results as long as the independent variables in the first stage are 

uncorrelated with the independent variables in the second stage.
CAA-2015094

DAHB DATA ANALYSIS TECHNIQUES  37

For most regression models, to include those presented here, the test value is zero, with the 

mean of the errors averaging to zero by construction. The t-statistic is:

 

 

 

The t-statistic comes from a Student's t-distribution, which maps directly to probability 

values allowing the determination of statistical significance. Rearranging the t-test equation, 

thresholds can be defined at the standard levels for statistical significance 0.10, 0.05, and 0.01,

which corresponds to the identification of a change from the baseline with 90%, 95%, and 99% 

certainty16. The determination of the thresholds is accomplished by first finding the tdistribution value for the desired confidence level p, and degrees of freedom in the sample size 

 . The t value, scaled by the sample standard deviation provides upper and lower 

bounds for the sample mean minus the test value:

 

 

 

 

 

 

 

The exact likelihood of having observed a change can be found by solving explicitly for 

t; all the required sample characteristics are known (sample mean , number of events in the 

sample n, test value , and sample variance ):

 

 

 

 

 

16Analyst selection of an appropriate significance level will be dictated by context. It must 

balance type 1 error, an incorrect rejection of the null hypothesis and type II error, falsely 

rejecting the null for the alternative hypothesis when in fact the null hypothesis is true (Pratt et 

al., 2001). 
CAA-2015094

38  DATA ANALYSIS TECHNIQUES DAHB

 

 

 

 

 

 

Utilizing the detailed approach, Figure 7 shows one possible visualization of significant 

unexpected deviations in insurgent activity that occurred in 2012. Significant increases are 

denoted by shades of red, with darker reds indicating increased statistical confidence. 

Significant decreases are denoted by shades of green, with darker shades indicating increased 

statistical confidence. UB/LB means upper and lower confidence bounds, gray circles represent 

non-significant incidents. While the underlying analysis leverages daily residuals, Figure 7 plots 

moving 7-day averages of enemy-initiated attacks, smoothing the trend line for visual analysis.
CAA-2015094

DAHB DATA ANALYSIS TECHNIQUES  39

Figure 7. Results of a two-stage least squares model to identify unexpected trends.

17

The strength of this process is that in recurring analysis it allows us to readily identify 

deviations from expected insurgent activity as they occur. With the identification of significant 

changes, possible explanations for these shifts are as follows:

1) An atypically severe winter might have caused a delay in the start of the fighting 

season. 

2) A poorer harvest likely returned insurgent fighters to the fight earlier. At the same 

time, the burning of Korans at Bagram airbase reaches international and Afghan 

media outlets inciting protests.

3) EIAs decrease significantly surrounding Eid al-Fitr as insurgents travel to and from 

home for the observance. 

 

17 Results of a two-stage least squares model to identify significant unexpected trends in 

nationwide enemy-initiated attacks in Afghanistan in 2012. Significant increases/decreases are 

denoted by shades of red/green, with darker red/green indicating increased statistical 

confidence. UB/LB means upper and lower confidence bounds, gray circles represent nonsignificant incidents

0

100

200

300

400

500

600

700

800

900

1000 Jan-12 Feb-12 Mar-12 Apr-12 May-12 Jun-12 Jul-12 Aug-12 Sep-12 Oct-12 Nov-12 Dec-12Daily Enemy-Initiated Attcks (7-Day Aggregate)

Identification of Significant Trends in Enemy-Initiated Attacks

LB 0.01 LB 0.05 LB 0.1 Not Sig UB 0.1 UB 0.05 UB 0.01 Model Expectations

(3) Fighters travel to 

and from home 

surrounding Eid al-Fitr

(1) Longer winter 

delays start of 

fighting season

(2) Poor harvest leads 

to earlier return to 

fighting and Koran 

burnings incite protests

(4) Unseasonable 

cold followed by an 

irregularly warm 

period leads first to 

a decrease in 

insurgent activity 

then an increase
CAA-2015094

40  DATA ANALYSIS TECHNIQUES DAHB

4) Unseasonable cold temperatures likely led to lower than usual expected insurgent 

activities, but these were almost immediately followed by a period of unseasonable 

warm temperatures that probably led to an increase in activity levels.

Methods for Visualizing Time Series Data

Visualization is a very useful tool for showing changes in patterns or trends that might 

otherwise go unnoticed. Standard ways to display changes over time comprise among others line 

charts, bar charts, or scatter plots such as shown in Figure 7 with a supplementary color 

dimension to convey additional information, in this case statistical significance. Time series data 

can be categorized by considering their context or frame of reference – usually distinguished into 

abstract and spatial. An abstract frame of reference simply refers to data that has been collected 

in a non-spatial context, i.e., without connection to a spatial layout. A spatial frame of reference, 

on the other hand, implies the existence of a spatial layout for the data set - it allows viewing 

space as some kind of indexing system for events (Anselin, 1992; Cressie, 1993; Cressie and 

Wilke, 2011).

In the Afghanistan Theatre of Operations spatial information is collected - its geographic 

location for each event. Although it is important to know the insurgent activity changes over 

time, it is equally important to understand where insurgent actions occur, their spatial structures, 

spatial interactions, and the population that is impacted. The power for spatial data analysis lies 

in its geo-relational database structure, i.e., in the combination of value information and 

locational information. A geographical information system (GIS) is the natural tool to aid in the 

display and analysis of spatial data. The advanced display capabilities contained in a GIS can be 

useful for the visualization of processed, geo-referenced data, or the results of statistical 

analyses. These spatial visualizations can have a temporal dimension. In Figure 8a and Figure 

8b we show examples for two static geospatial representations.18 Figure 8a displays insurgent 

activity in Afghanistan's central Helmand Province relative to the population that is visualized as 

a background layer. The location of each individual insurgent attack is represented by a colored 

point, where the color represents one of the two time periods under consideration – June to 

September 2012 and, for comparability, the same period in 2011. This representation shows that 

insurgent activity moved from higher population densities in the central part (during the 2011 

time period), to areas with lower population densities in 2012. Combined team operations and a 

high operational tempo appear to have had a considerable impact on the insurgent's campaign by 

 

18 Static representations visualize the data in still images (i.e., the representations do no changes 

automatically over time), while dynamic representations convey the time dependency of the 

data (i.e., representations over sequential time periods).
CAA-2015094

DAHB DATA ANALYSIS TECHNIQUES  41

reducing attacks in key population centers in Helmand Province, namely in Marjeh District, as 

well as Lashkar Gah, Nad'Ali, and western Nahr-e Saraj Districts.

Figure 8a shows another static representation of the enemy activity for central Helmand 

Province. Figure 8b shows the difference between insurgent attach densities for 2011 and 2012 

time periods for this same region. This representation reveals a relative change in insurgent 

activity from one year to another. Representations like those in Figure 8a and Figure 8b can 

greatly improve the situational and contextual understanding of both the analyst and more 

importantly the leadership and decision makers.

Figure 8a. Static, geospatial scatter plot visualizing individual enemy-initiated attacks.

Lashkar Gah

Nawah-ye Barakzai

Marjeh

Nad’ Ali

Nahr-e Saraj

Gereshk

Lashkar Gah City

RC-SOUTH

RC-SOUTHWEST

EIAs Jun – Sep 2012

EIAs Jun – Sep 2011

Population 

Density per km2
CAA-2015094

42  DATA ANALYSIS TECHNIQUES DAHB

Figure 8b. Static geospatial density plot visualizing changes in enemy-initiated attacks 

More robust spatial time series analysis can help to quantify these observed trends (Gaul, 

2013). We achieve this by measuring the proximity of the population to insurgent activity, or 

more precisely by considering the average number of Afghans that live within 1 km of each 

enemy-initiated attack. A geographical information system is utilized to conduct the laborious 

spatial analysis.

Figure 9 shows the result, i.e., the average number of Afghans in proximity of an attack 

by month for all of Afghanistan since January 2010. We evidently see a continuing downward 

trend in this metric since 2010 when the average number of Afghans living within 1 km of an 

enemy-initiated attack was approximately 2,000, while in November 2012, around 1,300 

Afghans were objectively affected on average. Each EIA affects (on average) fewer Afghans, or 

in other words, the insurgent attacks migrate to less densely populated areas where they are close 

to fewer and fewer Afghans. The visualized downwards trend is statistically significant. 

Lashkar Gah

Nawah-ye Barakzai

Marjeh

Nad’ Ali

Nahr-e Saraj

Gereshk

Lashkar Gah City

RC-SOUTH

RC-SOUTHWEST

Population 

Density per km2

Year-over-Year 

Insurgent Activity

Difference

Decrease

Increase
CAA-2015094

DAHB DATA ANALYSIS TECHNIQUES  43

Figure 9. Average number of Afghans living within 1 km of each enemy-initiated attack 

(EIA).

Modern warfare is creating vast amounts of data. Leveraging this information across 

time and space can crucially advance operational and strategic goals. It is critical for analysts to 

be equipped with the right tools, skills, and opportunities to process and manipulate 

spatiotemporal data generated by a (military) campaign, thus facilitating expedient analysis of 

enemy actions and blue force operations.

Challenges

During our time at ISAF Headquarters we have encountered several challenges that have 

impeded our ability to perform effective analysis. They include the cost and time needed to 

establish an operational database, i.e., data collection, access to the data and data sharing; data 

quality control and quality assessment; the merging of data sets) and access to analytic software. 

Beyond these structural challenges, there often exists a gap between decision makers and 

analysts. The analysts need to understand the issues facing the top-level leadership in order to 

best focus their analysis.

In addition to the large upfront costs of establishing a database structure useable across a 

command, there is the time and manpower needed to maintain it, e.g., collate operational reports 

into the database, reconciling differences between inputs, manage lost or incomplete data, etc. 

This requires planning and a dedicated funding pool. 

0

500

1000

1500

2000

2500

3000

3500

4000

4500

5000 Jan10 Feb10 Mar10 Apr10 May10 Jun10 Jul10 Aug10 Sep10 Oct10 Nov10 Dec10 Jan11 Feb11 Mar11 Apr11 May11 Jun11 Jul11 Aug11 Sep11 Oct11 Nov11 Dec11 Jan12 Feb12 Mar12 Apr12 May12 Jun12 Jul12 Aug12 Sep12 Oct12 Nov12

0

500

1000

1500

2000

2500

3000 Number of People Within 1km of an EIA Number of Enemy-Initiated Attacks

Avg Number of Afghans Living within 1km of each Enemy-Initiated Attack

Number of EIAs

Avg Number of Afghans w/in 1 km of EIA
CAA-2015094

44  DATA ANALYSIS TECHNIQUES DAHB

Across a theatre of operations different organizations often maintain databases or data 

repositories for various reasons, such as the collection of a specific type of information that is not 

collected elsewhere, or because of a lack of knowledge of existing databases. This can cause the 

analyst additional challenges, the primary ones being 1) lower standards concerning quality 

control or data consistency and data integrity compared to the authoritative "command 

databases," and 2) difficulties in assessing and accessing the data . The ability to access data is 

crucial to analysis and assessment – additional non-authoritative databases/repositories, although 

potentially well intended, can have counterproductive effects.

Understanding data quality and the constraints it places on subsequent analysis is an 

important prerequisite to formulating an assessment and making recommendations. The ability 

to control data quality by cleaning, processing, and organizing data is a necessary skill for 

analysts. In some cases analysis that could be done should be avoided because underlying data 

inconsistencies would lead to conclusions too unreliable to guide decision making.

The data necessary for an analysis may not exist in one repository. The ability to join 

large datasets is what enables many analyses. Combining datasets and looking across previously 

different lines of effort allows for the identification of formerly undetected interdependencies 

and interrelationships that can bring important campaign insights.

Access to analytic software is the final major constraint for the analyst. The tools to 

manage and manipulate large quantities of spatiotemporal data can be expensive. Funding such

tools requires improved awareness of their operational benefits. Similarly, system protocols and 

security related constraints often hinder access to required software such as Microsoft Excel's 

VBA (Visual Basic for Applications) or tools publically available for statistical analysis such as 

R data analysis software. 

Operational analysis can provide commanders critical information about the campaign. 

In this line, analysts need to link directly into the pulse of the campaign and understand current 

commander priorities, while leadership should seek to remove hurdles hindering analysis. A 

good analyst can often help transform commander's priorities and guide valuable research that 

informs future decisions. 

Summary and Conclusion

Leveraging the vast amounts of data collected across time and space in modern military 

campaigns can crucially advance operational and strategic goals. It is critical for analysts to be 

equipped with the right tools, skills, and opportunities to process and manipulate these data. 

Time series analysis is one of these highly valuable tools and can generate crucial insights for a 

command, enabling the assessment of operational effectiveness and developments in insurgent 

tactics. By combining time and spatial domains it is possible to identify further relationships and 
CAA-2015094

DAHB DATA ANALYSIS TECHNIQUES  45

opportunities that can be exploited. In this chapter we have presented the fundamentals of time 

series analysis and discussed some of its uses stemming from our experiences of working in the 

AAG at ISAF Headquarters in Kabul, Afghanistan. We concluded by presenting some of the 

assessment challenges encountered during the ISAF campaign; lessons that may help future 

analysts and commanders better structure their campaigns for success.

References

Allen, O.O., O. Borgan, and H.K. Gjessing. Survival and Event History Analysis: A Process 

Point of View. Statistics for Biology and Health, Springer Science and Business Media, 

LLC, New York. (2008).

Anscombe, F.J. The Transformation of Poisson, Binomial, and Negative Binomial Data. 

Biometrika, 35 (1948), 246-254.

Anselin, Luc. Spatial data analysis with GIS: an introduction to application in the social 

sciences. (1992).

Box, G.E.P. and D.R. Cox. An Analysis of Transformations. Journal of Royal Statistical 

Society, Series B, 26 (1964). 211-252.

Box G.E.P, G.M. Jenkins, and G. C. Reinsel. Time Series Analysis: Forecasting and Control. 

Wiley Series in Probability and Statistics, John Wiley & Sons, Inc. Hoboken, NJ, 4th ed., 

2008.

Chandler, R.E. and E.M. Scott. Statistical Methods for Trend Detection and Analysis in the 

Environmental Sciences. Statistics in Practice, John Wiley & Sons Ltd., Chichester, UK. 

(2011).

Cook, R.J. and J.F. Lawless. The Statistical Analysis of Recurrent Events. Statistics for Biology 

and Health, Springer Science and Business Media, LLC, New York, 2nd ed., 2002. 

Corrected Eighth Printing. (2010). 

Cox, D.R. Some statistical methods connected with series of events (with discussion). Journal of 

the Royal Statistical Society, Series B, 17 (1955), 289-300.

Cox, D.R. and V. Isham. Point Processes. Monographs on Statistics and Applied Probability. 

Chapman and Hall/CRC, Boca Raton, Florida. (1980).

Cressie, N.A.C. Statistics for Spatial Data. Wiley Series in Probability and Mathematical 

Statistics, John Wiley and Sons Inc., New York, Revised Ed. (1993).

Cressie, N.A.C. and C.K. Wilke. Statistics for Spatio-Temporal Data. Wiley Series in 

Probability and Mathematical Statistics, John Wiley and Sons Inc., Hoboken, New 

Jersey. (2011).

Freeman, M.F., and J.W. Tukey. Transformations Related to the Angular and the Square Root. 

The Annals of Mathematical Statistics, 21 (1950), 607-611.
CAA-2015094

46  DATA ANALYSIS TECHNIQUES DAHB

Gaul, Marcus, Ryan McAlinden and Pablo Alvarez. Population-Centric Counterinsurgency: The 

Effects of Violence on Population Clusters. To be published. (2013).

Gons, Eric, Jonathan Schroden, Ryan McAlinden, Marcus Gaul and Bret Van Poppel. 

Challenges of measuring progress in Afghanistan using violence trends: the effects of 

aggregation, military operations, seasonality, weather, and other causal factors. Defense 

& Security Analysis, 28:2 (2012), 100-113.

Hamilton, J.D. Time Series Analysis. Princeton University Press, Princeton, New Jersey. 

(1995).

Jesse, Eric. Identifying Significant Trends in Operational Data. To be published. (2013).

Makridakis, Spyros, Steven C. Wheelwright, Rob J, Hyndman. Forecasting - Methods and 

Applications. John Wiley & Sons. (1998).

Pratt, J.W., H. Raiffa, and R. Schlaifer. Introduction to Statistical Decision Theory. The MIT 

Press, Cambridge, Massachusetts, 1995. Third Printing. (2001).

Priestley, M.B. Non-linear and Non-stationary Time Series Analysis. Academic Press, London. 

(1988).
CAA-2015094

DAHB DATA ANALYSIS TECHNIQUES  47

3.8 Summary

There are numerous analytical techniques one can use to execute analysis. One does not have to 

be an expert in every technique, but one should know which technique is appropriate for a given 

problem. If a deployed ORSA analyst finds that a problem requires a specific technique where 

time, skill, or software availability is at issue, he or she can request analytical support from a 

reachback agency such as CAA or tap into his or her personal ORSA network.
CAA-2015094

48  DATA ANALYSIS TECHNIQUES DAHB

(THIS PAGE INTENTIONALLY LEFT BLANK)
CAA-2015094

DAHB COMMUNICATING CLEARLY  49

CHAPTER 4 COMMUNICATING CLEARLY

4.1 Purpose

The purpose of this chapter is to provide ORSA analysts with best practices for representing data 

and associated analyses in a clear, accurate, and understandable way.

4.2 Constraints, Limitations, and Assumptions (CLA)

CLA bound (scope) a study effort by identifying what must (or must not) and can (or cannot) be 

accomplished; they frame the study space and set the stage for the study team's methodology 

development; they serve as a "contract" between the study sponsor and the study team; and they 

provide a basis for the sponsor to reconcile the study results. CLA provide the framework for 

both the study team and the study sponsor to understand the conditions under which a study's 

results are applicable. 

4.2.1 Constraints

Constraints are restrictions imposed by the study sponsor that limit the study team's options in 

conducting the analysis and/or study. The study tasker and sponsor's guidance may place bounds 

on the study by specifying the

 date by which the study must be completed;

 organizations to participate in the study;

 force structure(s) and force year(s) to consider;

 type(s) of combat operations to consider; and

 scenario(s), threats, and environments to consider.

These bounds may limit one's ability to conduct a full investigation and analysis of the issues at 

hand. One needs to discuss changes to any constraint with the study sponsor prior to removing 

or adjusting the constraint.

4.2.2 Limitations

Limitations are restrictions that limit one's ability to meet the study objectives or fully 

investigate/analyze the study issues within the bounds of the study's sponsor. Limitations may 

consist of the following

 concept immaturity;

 access to study information, e.g., information that requires special clearances;

 availability of data;

 number and type of available scenarios; and,

 available models and their capabilities.

One should not label something as a limitation that one imposes on oneself. If one can overcome 

a limitation by having the study sponsor change a constraint, one should recommend that change 

to the sponsor and identify how it benefits the analysis/study.
CAA-2015094

50  COMMUNICATING CLEARLY DAHB

4.2.3 Assumptions

Assumptions are statements related to the analysis/study that are taken as true in the absence of 

facts, often to accommodate limitations.

Clearly define assumptions used in one's analysis. Assumptions are an integral part of any 

analysis. One needs to clearly define and state all assumptions associated with the analysis. 

Higher headquarters or written plans provide the assumptions associated with the analysis. One 

usually identifies these assumptions up-front to help shape the method of analysis. One may also 

discover other needed assumptions as the analysis progresses. When one discovers additional 

assumptions during the analysis, one needs to discuss these assumptions with the sponsor to 

determine their relevance or to obtain additional data that would eliminate them. Whether stated 

up-front or determined over the course of analysis, one needs to make all recipients of the final 

analytical product aware of all assumptions by incorporating them into the final presentation or 

report.

State how the assumptions will affect analysis results. It is important to state how each 

assumption will affect one's analysis and study results. One should only include those 

assumptions that have an impact on the analysis and results. If one cannot explain the effect—or 

there is no effect on the results—one should not include the assumption. One should explain all 

assumptions during one's presentation or in the notes section of the final product or report. The 

following are examples of relevant assumptions made in past analyses conducted in theater:

 Specific sensor system had line of sight to view significant activities.

 Attrition rate of 6.4%.

 International funds would continue at the same level for the next x number of years.

Based on the available data at the time of the analysis, one makes assumptions to allow analysis 

to continue. If additional data become available during the analysis, one should include them to 

provide greater fidelity and insight. Assumptions, such as those listed above, assist in the 

development of a data collection plan and enhance future iterations of the analysis.

4.3 Roles and Tenets of CLA

CLA are critical to a successful study. They

 bound (scope) a study effort by identifying what must (or must not) or can (or cannot) be 

accomplished;

 frame the study space and set the stage for the study team's method-development;

 serve as a contract between the study sponsor and the study team; and

 provide a basis for the sponsor to reconcile the study results.

CLA must be

 acceptable, i.e., generally agreed upon by all study participants;

 evolving, i.e., continually reviewed and adjusted as the study effort matures;

 necessary, i.e., enable the study effort; and
CAA-2015094

DAHB COMMUNICATING CLEARLY  51

 valid, i.e., sound and supportable.

4.3.1 State All Caveats

One must clearly state all caveats associated with the data and analysis. Caveats are warnings or 

cautions that may include one's interpretation of data, interpretation of terminology, and one's 

graphical representation of results. One may address caveats in the form of limitations. One 

should address caveats up-front to give one's audience a greater perspective while they are 

reviewing the analysis and results. For example, one should include a caveat associated with a 

specific graphical representation of results as a text box or bullet on the same slide as the graphic 

to avoid misrepresentation and/or misunderstanding of the results.

The following are examples of caveats from OIF and OEF:

 Population data were extrapolated from the census conducted in 1975.

 All levels of armoring (1, 2, & 3) were included in the up-armored high mobility 

multipurpose wheeled vehicle (HMMWV) category.

 Only coalition forces (CF) casualty data were included.

 Responses for Afghan National Development Program (ANDP) opinion survey 4.0 were 

worded differently from previous surveys. Responses were binned to compare with 

ANDP surveys 1.0 - 3.0.

Caveats such as these inform the sponsor to begin collection on specific data requirements to 

enhance future iterations of ANDP analysis.

4.3.2 Presenting Results

Know the audience. Before one presents a brief on analytical results, he or she needs to know 

his or her audience. Knowing one's audience assists one in knowing what level of discussion to 

have on the analytical techniques used in the analysis.

Avoid becoming too technical. If one wants senior leaders to accept one's technical study 

results at the highest levels, one must communicate insights as simple statements, without 

becoming too technical. One should not assume that the decision maker will sift through 

briefing data to find the answers. The purpose of a briefing is not to show the decision maker all 

the data and expect him or her to find the answer. One must realize that most decision makers 

are not technical experts. If one displays too much technical data, one further complicates an 

already complex issue. One must package one's results to provide insight and recommendations 

to decision makers. Decision makers focus on decision making, not technical fascination.

Present bottom line up front. One should clearly state the purpose of one's brief/presentation at 

the beginning in the form of a purpose statement or problem statement. One should make these 

statements concise and develop them jointly with the study sponsor/study director. 

Use a logical presentation methodology. One's presentation should have a logical flow to 

allow one's audience to understand the purpose, problem, analysis, and results. One should 

begin one's presentations with the purpose and/or problem statement, i.e., "bottom line up front." 

This technique enables one's audience to understand the end state and results while the 

discussion is unfolding. Any graphical representation of data or analysis should include a "take 

away" bullet. One should include a bullet comment on the most important reason for displaying 
CAA-2015094

52  COMMUNICATING CLEARLY DAHB

the data/results in a given manner. This bullet should stand out using a color scheme, larger bold 

text, or some type of symbol to draw the audience's attention to this important fact.

If there is specific information that should remain displayed throughout the brief, e.g., a map of 

the operational area or a legend of a color scheme), one should create separate visual displays to 

make this possible.

The delivery of one's brief is just as important as the brief itself. One should rehearse one's brief 

to increase one's confidence, correct one's mistakes, and make changes to the flow of one's 

presentation. One should take the time to ensure all technology is operating properly (including 

replacing the batteries in the remote, the bulbs in the projector, the length of extension cords for 

the computer equipment, the volume on the sound, etc). If a remote is not available, one should 

arrange for someone else to work the slide show presentation (if one has time to rehearse, this is 

always a good idea because it limits one's distractions and keeps one's audience focused on the 

presentation). One can have the best analytical product in the world, but if one's delivery is poor, 

the decision maker may discount one's efforts.

After giving the presentation, one should re-assess the overall briefing and determine if one 

achieved one's stated goals. One should determine if follow-on presentations or meetings are 

necessary and appropriate.

Prepare slides to stand alone. One's slides should contain backup notes and potentially stand 

alone. Commanders and staffs at all levels may extract specific slides from one's presentation 

for other presentations/meetings/working groups, etc. Providing backup notes with one's slides 

limits misuse and misinterpretation.

Include the appropriate classification with each bullet. When incorporating classified 

information into one's presentation, one should use a Security Classification Guide or properly 

marked source document(s) to determine the classification of information incorporated into their 

presentation. Every portion in a classified presentation should be marked for classification, and 

every slide should contain a classification banner indicating the highest level of classification 

found on that slide. Reference the governing regulations or memorandums on marking of 

classified information that pertain to your working location.

Use the best method of presentation. Consider one's audience in determining the best method 

to display one's results. One should remember one is conducting the analysis for the customer. 

In some cases, this method is command-driven. If it is not, the following are helpful hints on 

best practices used in theater.

Present information in the clearest manner. One should not make slides so complex that the 

audience has difficulty understanding the main point of one's slides.

Use a combination of graphs or tables with accompanying bullets. One should highlight the 

"take away" of the slide. One should use colored text boxes and larger fonts to make important 

points stand out from other information on the slide. When one uses a common color scheme

throughout the presentation, one's audience quickly recognizes the main point of each slide.

Properly label graphical representation of results. One should label all aspects of each graph, 

i.e., x-axis, y-axis, title, and legend. One should clearly define data table columns and rows. 

One should include the data source with all graphs and tables to enable one's audience to find the 

original data for follow-on analysis and/or validation.
CAA-2015094

DAHB COMMUNICATING CLEARLY  53

Simplify mathematical concepts. One should know one's audience when explaining the 

mathematical concepts behind the analysis. As a rule, one should present concepts simply and 

have the mathematical computations available in one's back-up slides to present to senior leaders 

who request additional information.

Graphically display results and conclusions. Graphical displays are often more powerful than 

the technical analysis behind them. One should take extra time to display one's information in 

such a way that one's audience easily grasps the main points. When one incorporates tables, 

charts, or graphs into a PowerPoint presentation, one should paste them as pictures. One does 

this by using the Edit, Paste Special command in PowerPoint. This prevents anyone else from 

manipulating one's data and charts. It also reduces the size of one's presentation and makes it 

easier to send electronically. The following are some helpful hints for displaying results:

 Visuals are more powerful in color. One should use a specific color for the critical points 

or "take-a-ways" that require audience attention. One should limit the number of colors 

per graphic and keep colors consistent across one's entire presentation. Each graphic 

must contain a detailed legend that defines one's color-coding methodology. Ensure your 

primary audience member, e.g., the commander, is not colorblind, which could render 

your message ineffective.

 One should avoid confusing one's audience; one should keep scales constant across 

multiple charts on the same topic.

 After completing the analysis, one should determine the type of graphic that best 

communicates the results and main "take-a-way" points, e.g., text charts, tables, line 

charts, dual-axis charts, surface charts, vertical bar or column charts, and pie charts).

 Text charts are the easiest type of graphic to prepare. However, one should remember 

that words or bullets are not visuals. Pictures, clouds, and arrows used in conjunction 

with words create visual effects that communicate the information or results better. Text 

charts are effective for introducing topics, such as the methods one used in the analysis

and important ideas or results. One should keep one's text charts as concise as possible; 

one should use bullet statements rather than entire sentences and paragraphs.
CAA-2015094

54  COMMUNICATING CLEARLY DAHB

Figure 10. ANDP survey model

Figure 10 is a text chart displaying the survey model of the ANDP. One can use tables to display 

raw data or study results; however, there are better methods for displaying comparisons or 

trends. One can use a table in conjunction with other graphics if the exact numbers are relevant 

to the "take-a-way" one wants to achieve on a particular slide. When one uses tables, one should 

clearly label all columns and rows (see Figure 11).

Figure 11. Annual ANDP spending per fiscal year

Elections 

Expectations

& 

State of Affairs

Security

Elections 

Delivery

& 

ANSF

Reconstruction Governance 

& Justice

Survey 1.0

August -

October

Survey 2.0

November -

March

Survey 3.0

April - July

Survey 4.0

August - October

Elections 

Expectations

& 

State of Affairs

Security

Elections 

Delivery

& 

ANSF

Reconstruction Governance 

& Justice

Survey 1.0

August -

October

Survey 2.0

November -

March

Survey 3.0

April - July

Survey 4.0

August - October
CAA-2015094

DAHB COMMUNICATING CLEARLY  55

Figure 12. Numbers of attacks and casualties

Line or curve charts effectively show trends and time series data. Line charts display multiple 

lines on a single graph. One should clearly distinguish each line with differing colors or patterns 

(solid, dashed, etc). One should include a legend that defines colors and patterns.

Figure 12 is a sample line chart of notional attack and casualty data. The chart defines the x- and 

y-axes, and includes a legend to specify the representation of each line. The title specifies the 

exact category of the data, e.g., attacks and casualties.

Attacks and Casualties

0

10

20

30

40

50

60

70

80

90

100

Jan-04

Feb-04Mar-04Apr-04

May-04

Jun-04

Jul-04

Aug-04

Sep-04

Oct-04

Nov-04Dec-04

Jan-05

Feb-05Mar-05Apr-05

May-05

Jun-05

Jul-05

Aug-05

Sep-05

Oct-05

Nov-05Dec-05

Date

Number of Attacks and 

Casualties

Attacks Casualties
CAA-2015094

56  COMMUNICATING CLEARLY DAHB

Figure 13. Casualties per attack

Dual axis charts show different data with a varying degree of magnitude between the data sets. 

In Figure 13, the ORSA analyst uses a red line to display notional casualties per attack. The 

dual-axis chart presents all pertinent information in a clear and understandable way.

0

50

100

150

200

250

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Month

Number of Attacks and 

Casualties

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

0.45

0.50

Casualties per Attack

Attacks Casualties Casualties per Attack
CAA-2015094

DAHB COMMUNICATING CLEARLY  57

Figure 14. Numbers of participating ISF/US/CF security forces

In Figure 14, the ORSA analyst uses a surface chart to display data for the Iraqi Security Forces 

(ISF) Update conducted by Multi-National Forces-Iraq (MNF-I). A surface chart is a single line 

chart with the area shaded under the line. The shading emphasizes the size of the total amount 

rather than the differences or changes in the amounts.

Forces in Iraq

0

50000

100000

150000

200000

250000

300000

350000

400000

450000

Sep-04

Oct-04

Nov-04Dec-04

Jan-05

Feb-05Mar-05Apr-05

May-05

Jun-05

Jul-05

Aug-05

Sep-05

Oct-05

Nov-05Dec-05

Iraqi Security Forces U.S. Military Coalition Forces
CAA-2015094

58  COMMUNICATING CLEARLY DAHB

Figure 15. The answer "Road improvements would most improve our lives?"

Figure 15 is a column chart used to display the results of the answer "road improvements" to the 

question in the ANDP opinion surveys 2.0 and 3.0—"What is the one thing that would most 

improve your life?" 

A vertical bar or column chart displays values and series groups as sets of vertical columns 

grouped by category. The height of each column represents each respective quantity and its 

variation across different categories, e.g., time intervals or geographic regions. Excel offers 

different variations of this type of chart, to include the stack column chart, the percentage 

column chart, and the option to graph multiple categories across the same intervals of time or 

geographic regions by displaying each as a different-colored column. Each graph should include 

a legend that defines the visual coding.

0

20

40

60

80

100

120

North

West

North 

North East

East

Central

East

Central 

South

Regions

Number Respondents

ANDP 2.0 ANDP 3.0
CAA-2015094

DAHB COMMUNICATING CLEARLY  59

Figure 16. The age-percentages of ANDP 3.0 survey respondents

Figure 16 displays age category percentages for ANDP 3.0. A pie chart is used to show a 

composite whole and its proportions. Pie slices are compared with each other and with the 

whole. Each slice is represented with color and a legend is used to explain the color-coding 

methodology.

Respondents Ages for ANDP 3.0

10%

26%

36%

28%

18-24

25-34

35-49

50+
CAA-2015094

60  COMMUNICATING CLEARLY DAHB

Figure 17. MEDEVAC helicopter response times and number of events

Figure 17 shows notional Afghanistan MEDEVAC data and helicopter response times created 

with ArcGIS. Shading represents the various response times and red dots represent MEDEVAC 

events.
CAA-2015094

DAHB COMMUNICATING CLEARLY  61

Figure 18. Security situation by province

Figure 18 is an example of a quad chart that summarizes and highlights trends on a specific 

question from the ANDP. Across all four iterations of the ANDP survey, pollsters asked, "How 

is the security situation in your village/city?" This information fed indicators and measures of 

effectiveness (MOEs) to determine if U.S./host nation (HN)/CF were meeting campaign plan 

objectives.

As a deployed ORSA analyst, one most commonly uses the "Quad" chart to summarize project 

status. The Quad chart divides a slide into four sections that usually consist of (1) the 

purpose/problem statement, (2) methods, (3) timeline, and (4) results/insights. One can adjust 

these sections to fit one's specific projects or quickly summarize one's current work effort. One 

can share these charts with agencies and other analysts who are interested in this work or 

collaborating on a reachback project. One can use Quad charts to show relationships. For 

example, in the assessment process, one may find this type of chart helpful in highlighting 

relationships between different indicators and MOEs, or increases/decreases in a specific MOE. 

One can amend the "Quad" chart, e.g., "Tri" or "Bi" chart) as the situation requires.

4.4 Summary

This chapter presented best practices for representing data and associated analyses in a clear, 

accurate, and understandable way. It presented the differences between constraints, limitations, 

and assumptions, and the importance of each. And finally, it highlighted the importance of using 

Good

82% Good

64%

Fair

15% Fair

27%

Good

92% Good

82%

Fair

17%

Fair 6%

Bad 3%

Bad 9%

Bad 2% Bad 1%

Aug '05 Dec '05 May '06 Sep '06

NATIONAL “How is the security situation in your village/city?”

0

5

10

Aug-2005 Dec-2005 May-2006 Sep-2006

North Kabul East West South

RC West

RC South Average of Good, Fair, Bad

Relatively 

stable

Bad

Fair

Good

REGIONAL

Aug-2005 Dec-2005 May-2006 Sep-2006

Nimroz Hilmand Kandahar

Zabul Uruzgan Daikundi

Good=10

Fair=5

Bad=0

Uruzgan

HelmandNimroz

Average Rating 

ANDP1.0 ANDP2.0 ANDP3.0 ANDP4.0

NIM

KDR

ZBL

HLM

UZG

DKD

RC SOUTH - PROVINCIAL RC WEST - PROVINCIAL

Aug-2005 Dec-2005 May-2006 Sep-2006

Ghor Badghis Herat

Farah

Good=10

Fair=5

Bad=0

Average Rating 

ANDP1.0 ANDP2.0 ANDP3.0 ANDP4.0

Farah

Badghis Ghor

Herat Badghis

Ghor Herat

Good

82% Good

64%

Fair

15% Fair

27%

Good

92% Good

82%

Fair

17%

Fair 6%

Bad 3%

Bad 9%

Bad 2% Bad 1%

Aug '05 Dec '05 May '06 Sep '06

NATIONAL “How is the security situation in your village/city?”

0

5

10

Aug-2005 Dec-2005 May-2006 Sep-2006

North Kabul East West South

RC West

RC South Average of Good, Fair, Bad

Relatively 

stable

Bad

Fair

Good

REGIONAL

Aug-2005 Dec-2005 May-2006 Sep-2006

Nimroz Hilmand Kandahar

Zabul Uruzgan Daikundi

Good=10

Fair=5

Bad=0

Uruzgan

HelmandNimroz

Average Rating 

ANDP1.0 ANDP2.0 ANDP3.0 ANDP4.0

NIM

KDR

ZBL

HLM

UZG

DKD

RC SOUTH - PROVINCIAL RC WEST - PROVINCIAL

Aug-2005 Dec-2005 May-2006 Sep-2006

Ghor Badghis Herat

Farah

Good=10

Fair=5

Bad=0

Average Rating 

ANDP1.0 ANDP2.0 ANDP3.0 ANDP4.0

Farah

Badghis Ghor

Herat Badghis

Ghor Herat
CAA-2015094

62  COMMUNICATING CLEARLY DAHB

geospatial analysis as a powerful tool to complement other visual tools for displaying one's 

analysis.
CAA-2015094

DAHB PUBLIC PERCEPTION SURVEY DEVELOPMENT AND ANALYSIS  63

CHAPTER 5 PUBLIC PERCEPTION SURVEY DEVELOPMENT 

AND ANALYSIS

5.1 Introduction

This chapter will discuss the value of public perception surveys, walk through the process to 

design and execute a survey, review analysis and presentation methods, and conclude with a 

discussion on the limitation of survey data and additional resources.

In operational environments where the indigenous population is the center of gravity, decision 

makers consider the population's perspective in their planning calculus. Commanders 

continually make decisions to address, persuade, and change public perceptions of key aspects 

and focus areas of the nation-building process, such as reconstruction, governance, security, etc. 

Surveys play an important role in understanding the population's perception, a unique data point 

to understanding the overall operational environment.

Figure 19. Survey process map

A broad overview of the entire process for designing, executing, and presenting public 

perception data is shown here.

5.2 Practical Applications and Examples

Public perception data can inform a broad range of decisions, spanning from the strategic level to 

more tactical decisions. This section lays out examples of the different ways in which survey 

data can be used. 

Assessment Indicators

Commanders use assessments to determine whether progress has been made towards achieving 

stated objectives. Survey data are often used as a data source for indicators as part of the 

assessment framework. Data collected over time can show how a population's sentiment has 

changed over time, whether towards extremist groups, the U.S., a U.S-Coalition military 

presence, or the host nation government. 

ISAF’s Afghan National Quarterly Assessment Report (ANQAR), a 

nationally representative public perception survey, was used as one 

indicator to inform assessments of the ISAF mission by showing the 

trends of Afghans’ perception of security over time.

Example 1. International Security Assistance Force.
CAA-2015094

64  PUBLIC PERCEPTION SURVEY DEVELOPMENT AND ANALYSIS DAHB

Inform Planning Requirements 

Survey data are also used to inform the development and adjustment of operations, particularly 

those related to civil affairs and military information support operations (MISO). Operators can 

identify target populations for potential operations, identifying areas of opportunity or need 

among the population. This includes identifying pockets of the population that are sympathetic 

towards violent extremists and understand why they are sympathetic. Then, using demographic 

and media usage information gleaned from the survey data, analysts can design a specific, 

targeted MISO series. 

Before Starting

Know the ultimate objective of the survey – what decision(s) is it informing? What are the data 

points stakeholders want from the survey? What information will it provide that other data 

sources cannot? Know who the primary stakeholders are, and what information they want from 

the survey. Ensure those expectations are realistic and fall within the scope of a public 

perception survey.

Process of Designing and Executing a Survey 

The involvement of the ORSA analyst in the survey process will depend on the role of the 

supported command: it may range from designing, implementing, managing the entire survey 

process to simply analyzing data collected through an already-existing survey channel. This 

sections walks through that entire process. 

If operating under a tight timeframe or with limited resources, there may not time to complete all 

these steps (pre-tests, back translations of the questionnaire, etc). Local contractors will almost 

always do the actual fieldwork; locals conducting surveys to local people in their own local 

dialect are much better received than if U.S. personnel were to conduct the surveys.

A note on working with contractors: Be certified as a Contracting Officer's 

Representative (COR). This provides the ability to communicate directly with the 

CAA’s Southern Philippines Public Perception Survey identified pockets 

of the Mindanao population where the people were unfamiliar with the 

Bangsamoro Peace Process. This information was used to develop a 

MISO series to education the population where it was needed, conducted 

in coordination with Philippine counterparts. 

Example 3. Joint Special Operations Task Force - Philippines

CAA used Department of State’s nation-wide surveys in Niger to inform 

the planning decisions of Special Operations Command Africa, who was 

looking to build an airbase in the Agadez Region. The survey data 

provided insights on the public’s sentiment towards the United States, as 

well as identified areas of opportunity, such as infrastructure 

improvement for the United States to build up good will. 

Example 2. Special Operations Command Africa
CAA-2015094

DAHB PUBLIC PERCEPTION SURVEY DEVELOPMENT AND ANALYSIS  65

contractors, instead of going through the Contracting Officer (KO). The KO will almost 

certainly not be a subject matter expert (SME) in public perception surveys, so part of the 

COR role will be to work with the contractor in the development of the sampling plan 

and questionnaire, check in with them to ensure they are following the accepted processes 

and protocols, and follow up directly with the contractor if problems with the data are 

identified, e.g., incorrect coding, responses not translated to English, etc.,. Additionally, 

the COR will review the Statement of Work before it is finalized to be sure the details 

comport with the requirements. This includes sample size, questionnaire length, 

sampling plan, training of interviewers, format of data delivery, timeline for execution, 

data coding requirements, and any required final products from the contractor.

Sampling Plan and Methodology Development 

Professional polling companies have the expertise to recommend the best polling methodology 

based on the survey's purpose. If one uses a contracting agency, inspect the sampling plan prior 

to fieldwork to ensure it comports to acceptable standards. Considerations include:

 Demographics. Typically, the survey sample should be representative of the population, 

as dictated by an authoritative source, such as a census. Depending on the purpose of the 

survey, the sample may need to focus on a particular portion of the population. For 

example, a survey looking at the influence of violent extremist organizations' messaging 

may focus on specific vulnerable populations as determined by the survey stakeholders. 

 Random Sampling. In order to limit the bias associated with surveys, random sampling 

needs to occur at every level of the survey process, from the selection of towns and cities 

to the actual people one will interview. Standard selection methods usually suffice; 

however, there may be times when one needs to consider "non-standard" selection 

methods.

 Sample Size. Even when a survey controls for demographics and random sampling, there 

is a margin of error associated with any opinion poll. There are many methods for 

calculating this margin of error (each with underlying assumptions), the most common 

involves sample size and confidence level. The equation for the margin of error 

calculated at the 95% confidence level is:

 

 

 

, where n represents the sample size.

Much analysis is done with subsets of the sample – male/female, geographic, and religious 

breakdowns, etc. Keep in mind that each of these sub-groups will have larger margins of error as 

the sample size being analyzed shrinks. Make sure to account for statistical significance in the 

final analysis and presentation - do not highlight differences that are not meaningful. 

Questionnaire Design

Designing a questionnaire should be a team effort, with involvement from all stakeholders to 

ensure their subject areas are covered, be it security, governance, development, etc. Typically, 

sections on demographics as well as media usage are standard. Whenever possible, one should 

take advantage of the contracted agency's expertise, both in polling as well as regional expertise, 

to assist with question wording and understand relevant cultural considerations.

When designing a questionnaire, there are several things to keep in mind:
CAA-2015094

66  PUBLIC PERCEPTION SURVEY DEVELOPMENT AND ANALYSIS DAHB

 Length. Cost will limit the number of questions; consider interviewer and respondent 

fatigue when adding questions. Do not be unnecessarily redundant.

 Order. Ask about awareness of an issue before asking an opinion of that issue. Group 

questions thematically.

 Continuity. Keep as many questions from previous waves as possible to enable trend 

analysis.

 Language. Keep it simple. Plain language, clear, and direct. Consider what nuances 

may be lost in translation.

There are several types of questions. These are some general categories:

 Closed-Ended Questions: Respondents are limited to the answer choices provided by the 

interviewer. The answer choices "Don't Know" is typically included for all closed-ended 

questions, however this selection is not provided to the respondent, only marked if that is 

what is said. Reading them aloud will skew results significantly. 

 Attitudinal Questions – typically asked on a five-point scale to represent the range: very 

negative, somewhat negative, neutral, somewhat positive, very positive. Sometimes the 

neutral response is removed.

Bad Question: Do you think the government is doing a good job at fighting corruption and 

providing services for its people?

This question combines multiple sentiments (fighting corruption and providing 

services) into one, which limits the respondent. The respondent may think the 

government is good at one thing and not the other – the data resulting from this 

question would not show this. Additionally, words should be specific enough that 

every respondent interprets the question the same way. What is meant by services? 

It could mean a broad range of things, and this question would not give meaningful 

data to act upon.

Better Question: Please tell me whether the government is doing a very good, somewhat 

good, neither good nor poor, somewhat poor, or very poor job for each of the following 

items: 

i. Fighting corruption.

ii. Providing security services.

iii. Providing education services.

iv. Providing healthcare services.

v. Providing transportation services.

vi. Providing electricity services.

This question asks about different tasks of the government, separately and 

specifically. The four answer choices offer more granularity for respondents. 

Example 4. Question Design
CAA-2015094

DAHB PUBLIC PERCEPTION SURVEY DEVELOPMENT AND ANALYSIS  67

 Multiple Responses – Allow respondent to choose several (usually up to three) responses 

from a given set to answer a question. 

 Agree/Disagree – Provide a statement, and ask respondents if they agree with it. 

 Open-Ended Questions – Typically asked after an attitudinal question, to provide the 

"why." Respondents are free to answer however they want, and typically, contractors 

will bin common answers together to simplify analysis.

Focus Group Discussions

Analysts will sometimes have the contracting agency conduct focus group discussions with the 

population, typically 6 to 8 participants with a trained moderator. This can either be a standalone survey mechanism, or used to augment the typical quantitative survey. Focus groups offer 

deeper insights into why people feel a certain way on a topic. They are usually organized around 

one or two specific areas, such as upcoming elections or the local economy, and the moderator 

facilitates a fairly unstructured discussion with the participants. The composition of the focus 

groups will be particular to the region/subject matter; sometimes the groups are split along 

religious lines, sometimes divided into male and female groups. Focus groups are also a way to 

see whether certain topics make people more uncomfortable and less willing to be forthright 

about their opinion – something that can be used to inform the development of the quantitative 

questionnaire. 

Q1. Have you heard of the Abu Sayyaf Group?

(Yes, No)

Q2. How much of a threat does the Abu Sayyaf Group pose to your 

community?

(Great Deal, Fair Amount, Not Very Much, None)

Q3. Why do you say that?

(Open-ended)

The first question is used as a skip question, so that only people who have heard of ASG are 

asked questions about them.

The second question provides the Likert-scale spectrum to measure the public’s sentiment 

towards the group.

The third question provides insights as to why the public feels a certain way. A respondent 

may say that ASG poses no threat to their community – that could be because ASG is simply 

not present in their community, or it could be because ASG is present and viewed favorably 

by the population. These two reasons have very different implications for the analysis, and 

are impossible to extrapolate from the survey data otherwise.

Example 5. Sample Question Series
CAA-2015094

68  PUBLIC PERCEPTION SURVEY DEVELOPMENT AND ANALYSIS DAHB

Pre-Test 

Particularly for a new survey, or just new questions, one should initiate a field test with a small 

sample size (25) to see how the questions play before the actual fieldwork begins. Things to 

look for: Is the respondent comfortable talking about the subject being asked? Is the question 

understood or is it confusing? Is there a cultural issue that makes this question unsuitable? The 

field test will isolate these questions, and others that may provide non-significant insights, i.e., 

100% of respondents answered "Don't Know" to a question, and allow the survey team to adjust, 

delete, or add questions based on field test results. 

Oversee Fieldwork

When and where possible, one should observe and monitor the training of interviewers, and 

monitor the actual fieldwork to ensure field workers adhere to prescribed standards and 

methodologies. However, consider the sensitivity associated with training observation and 

fieldwork monitoring – will a survey agency's association with U.S. personnel place their 

employees at risk?

Quality Check the Data

When first receiving the raw data, run a basic quality control to check for inconsistencies and/or 

anomalies in the data that may point to incorrect coding or other instances of the contractor 

committing errors during the survey collection and processing period. The quality assurance 

(QA)/quality control (QC) will provide greater assurance that the data collected is valid, offering 

more legitimacy to the analysis that is based off of it. Listed below are some areas to focus on 

for a QC checklist. 

Do you own a gun?

vs.

Is it appropriate for someone in this neighborhood to own a gun to protect his family?

When asked in the Southern Philippine Public Perception Survey, the first question 

above yielded a very low positive response, which was universally agreed upon to be 

highly inaccurate. Filipinos did not want to admit to personally owning a gun. 

However, when asked in the third person, as in the second question above, the response 

rates were much more in line with what operational data had analysts expecting. 

Example 6. Social Sensitivities in Questions
CAA-2015094

DAHB PUBLIC PERCEPTION SURVEY DEVELOPMENT AND ANALYSIS  69

Future waves

If the command is planning on conducting subsequent waves to track trends, try to maintain as 

much consistency as possible. Keep the exact wording of the question and coding the same. 

Maintain a merge file to simplify analysis. Ensure the contractor complies with the exact 

specifications on how to code the data. Be mindful of the timing of the survey, as it relates to 

current events. Is there a desire to capture sentiment before and after a presidential election? 

After a major operation or other significant event? Keep these in mind while planning out the 

timeline of conducting a survey.

5.3 Data Analysis

There are several different types of analysis that can be used on survey data, ranging from basic 

to complex. This section walks through the basics of data analysis, and briefly discusses some of 

the more commonly used tools for survey analysis. 

Common Tools19

 SPSS. The most common survey software is SPSS. SPSS is simple to learn and fairly 

intuitive, and has lots of capability for survey analysis. It allows the development and 

application of weights, conduct basic descriptive statistics and crosstabs, as well as more 

advanced analysis to include clustering, regression, the creation of new variables based 

on calculations of existing ones, etc. The data visualization for SPSS is not as flexible 

and intuitive as Microsoft’s Excel and PowerPoint applications, and some analysts prefer 

use of Tableau software to visualize survey and other data. 

 

19 Note: No endorsement is stated or implied by any references to commercial software 

applications in this handbook.

- Do all SPSS variable labels and value labels in the data set match the final 

questionnaire?

- Are the marginal consistent with regional/cultural knowledge/previous trends?

- Are crosstabs logically consistent?

- Were filter questions properly executed and coded?

- Do the sampling variables match the pre-fieldwork sampling design?

- Does the weighted/unweighted sample match a known population distribution?

- Are lengths and timings of interviews consistent and within the realm of normal? (do 

interviews last 5 minutes, or happen in the middle of the night?)

- Are there overlapping interviews?

- Does one interviewer complete in excess of 10 interviews per day? (The standard is 

10, but should be discussed with the contracting agency beforehand)

- Are there any instances of teleportation? (interviewer concludes one interview, 

begins another immediately after in a different province)

- Are there any duplicate cases in the dataset?

Example 7. QA/QC Questions
CAA-2015094

70  PUBLIC PERCEPTION SURVEY DEVELOPMENT AND ANALYSIS DAHB

 Excel. With basic Excel skills, one can do most survey analysis. There is no easy way to 

apply weights to data though, if necessary. Pivot tables will enable easy crosstabs, 

filtering, and frequencies of the data. Additionally, Microsoft Office is universal, so 

there is no need to request special software. Creating charts in Excel is simple, and they 

are easily manipulated to adjust, re-format, and combine into one PowerPoint 

presentation for stakeholders. 

Basic Analysis

 Frequencies. Basic marginal information offers broad insights into the perceptions on a 

given topic for the population as a whole.

 Crosstabs. Understand relationships between variables, particularly demographics – sex, 

ethnicity, religion, and location are typical subsets that tend to have different attitudes 

towards subjects. Beyond demographics, crosstabs are important to understand 

relationships between different attitudinal questions. 

 Trends. Longitudinal analysis provides stakeholders with the context of the survey 

findings – what was it before? How has it changed? What has been the general 

directionality over time? 

 Correlations. Quantify the degree to which two or more factors show a tendency to vary 

together. It is useful to see how closely responses to one question follow responses to 

another. For example, respondents who have strong distrust for the government may also 

have strong distrust for security sector institutions. 

 Cluster Analysis. Draw out groups of respondents whose response profiles are similar to 

one another, and identify what common characteristics the group has, and what makes it 

distinct from the rest. 

In addition to various quantitative techniques for analyzing the data, the analyst should have 

regional expertise, or at least basic knowledge, to understand, recognize, and expect certain 

cleavages in the data. This supplementary contextual knowledge helps stakeholders better 

understand why the data are a certain way, and isolate the atypical data points from those that 

would be expected, given the population's composition and culture. 

When compiling survey analysis to present to stakeholders, the analyst should try and 

incorporate other atmospheric information, analysis, and findings wherever possible to validate 

particular findings and/or determine areas of emphasis that need further analysis due to 

conflicting results. 

5.4 Report Presentation and Delivery

Tailor the analysis to the stakeholders' needs and preferences. If they prefer less PowerPoint, 

draft a short narrative with minimal charts to capture the key findings from the data. Highlight 

the takeaways that are most relevant to their problem set. Minimize the use of statistical 

language, such as covariance, correlation, statistically significant. 

Do not attribute causation for the way things are. Show correlation, overlay a timeline of current 

events over a trend of a particular sentiment, and offer potential explanations, but do not 

definitively state "why" something is the way it is. 
CAA-2015094

DAHB PUBLIC PERCEPTION SURVEY DEVELOPMENT AND ANALYSIS  71

Data Visualizations 

Use color when it has a meaning, not just for the sake of using color. Do not use it too much to 

make it overwhelming to the audience. Do not choose a color for a category that would be 

counterintuitive; e.g., red should not mean support.

The figures below provide examples of different visualizations for survey data. There are no 

absolute rules for how to present survey analysis, but the figures below capture some examples 

that have worked for stakeholders. Know the stakeholder's preferences in order to create a 

presentation that fits their style best. 

Figure 20. Line chart – depict changes in perception over time.

 

Figure 21. Geospatial Displays – Show Figure 22. Clustered Bars – Illustrate a rank

the distribution across an area order among several choices

3%

4%

7%

9%

12%

13%

14%

19%

0% 5% 10% 15% 20%

Agriculture

Expensive living

Health

Crime/insecurity

Poverty

Unemployment

Roads

Access to drinkable water

What is the most serious problem facing your 

community? Support for the Peace Process in Basilan

0

20

40

60

80

100

Dec-11

Feb-12

Apr-12

Jun-12

Aug-12

Oct-12

Dec-12

Feb-13

Apr-13

Jun-13

Aug-13

Oct-13

Dec-13

Feb-14

Apr-14

Jun-14

Aug-14

Oct-14

Dec-14

Feb-15

Apr-15

Jun-15

Aug-15

Percentage of Respondents Who Trust the MNLF

Zamboanga

Cotabato

Marawi

Isabela

Southern Balisan

Sulu
CAA-2015094

72  PUBLIC PERCEPTION SURVEY DEVELOPMENT AND ANALYSIS DAHB

 

Figure 23. Pie Charts– Figure 24. Stacked Charts - Display LikertShow demographic or categorical scale data across categories 

compositions

5.5 Limitations of Surveys

When using survey data to inform analysis, there are several things to keep in mind. As with any 

data set, there are potential errors in the collection and cleaning processes to watch out for and 

MILF GRP

-100%

-80%

-60%

-40%

-20%

0%

20%

40%

60%

80%

100%

Trust for Organizations

Strong 

Neither 

Not Strong 

FAB Opponents

All Respondents

Demographics of Survey Respondents

50% 50%

Male

Female

78%

22% Muslim

Non-Muslim

58%

42% Urban

Rural
CAA-2015094

DAHB PUBLIC PERCEPTION SURVEY DEVELOPMENT AND ANALYSIS  73

mitigate as possible. Below are some of the more common errors, and how to limit the effects of 

them.

 Sampling Error. Were enough people asked the survey? Was the sample representative 

of the entire population?

Use responsible sampling methodologies. Provide the margin of error when displaying 

analysis to stakeholders. 

 Measurement Error. Did respondents respond honestly? Do the data accurately reflect 

the sentiment of the population?

Vet the questions with focus groups. Have similar questions throughout the survey to 

check for consistency. Provide interviewer with questions on whether they felt the 

respondent was answering faithfully. 

 Coverage Error. Did interviewers go where they were supposed to go?

Particularly problematic with conflict areas. Conduct regular site visits during fieldwork, 

collect status reports from contractors, check data for instances of teleportation or other

inconsistencies. 

 Other Errors. Was there malfeasance? Incorrect coding? Failure to follow interview 

protocol? Compare data to other sources for consistency. Conduct thorough QA/QC 

checks on the data. 

5.6 Existing Data Sources/Surveys

Several organizations, both governmental and non-governmental are already running surveys, 

most commonly at the national level. Before undertaking this process, do research. Department 

of State (DOS)'s Office of Opinion Research conducts surveys in almost every country, Center 

for Army Analysis has access to the Gallup World Poll dataset. Would either of these sources 

fulfill the requirements? What is the U.S. Embassy collecting? Are there other open source data 

available? The Asia Foundation and Social Weather Station conduct regular national and subnational surveys in the Philippines. Surveys are expensive and time-consuming. Make sure the 

information is not already available through another avenue. If there are existing survey 

vehicles, can the analyst add questions to that survey questionnaire instead of beginning a new 

effort? If there are none, look around to see what other organizations would be interested in 

survey data of the specific areas. Can there be a combined effort? Look particularly to United 

States Agency for International Development (USAID) or DOS counterparts. 

5.7 Summary

Surveys are just one data point in understanding the operational environment. Public perceptions 

represent what people think, not necessarily the reality on the ground. And given the complex 

nature of today's current operating environment, there is no way to attribute specific actions to 

changes in perception. In order to get a more complete picture of the situation, include all-source 

information, including operational data; atmospherics; Political, Military, Economic, Social, 

Information, and Infrastructure (PMESII) analysis; intelligence; etc. 
CAA-2015094

74  PUBLIC PERCEPTION SURVEY DEVELOPMENT AND ANALYSIS DAHB

5.8 Further Reading

This is a very basic primer on public perception surveys. See sources below for more detailed 

guidance. The internet is also a valuable resource. 

Survey Research Methods. Fowler, Floyd J. 

The American Association for Public Opinion Research Website (https://www.aapor.org) 

for educational resources, standard definitions, publications.
CAA-2015094

DAHB OPERATIONS ASSESSMENTS  75

CHAPTER 6 OPERATIONS ASSESSMENTS

6.1 Introduction

A paradigm shift is underway in the operation assessment of complex environments. In the past, 

an assessment product was an attempt to determine progress over the preceding time period—in 

essence a history report—that organizations could share with higher authority. Today, an 

assessment process is becoming an increasingly robust attempt to improve our understanding of 

the operational environment (OE), which is informed by rigorously collected data given meaning 

by appropriate analyses, and the process is done explicitly for the purpose of making operations 

more effective. Therefore, assessment is now forward-looking to enable decision makers to take 

actions most appropriate to achieve their desired outcomes given the current state of the OE.

The definition of operation assessment proposed by the assessment community for the next 

revision of Joint Publication (JP) 5-0 makes this clear. Operation assessment is: "An 

operational activity that supports decision making by determining progress toward 

accomplishing a task, creating an effect, or achieving an objective or end state for the 

purpose of making operations and campaigns more effective" (JP 5-0 proposal).

In addition to the primary purpose of making operations more effective, assessment also has a 

secondary purpose of providing accountability to higher authority. Indeed, on many occasions, a 

formal assessment, summarized in a report, is required by law or regulation.

Assessors must also be aware that commanders may also use assessment data or products as 

messaging tools. This activity has implications for the veracity of assessment products used in 

this way (see Mushen and Schroden, (2014) "Are We Winning? A Brief History of Military 

Operations Assessment" available through Google). It is possible that a commander will want to 

"cherry-pick" data or results of analyses that support a particular message.

6.2 Assessment Framework

Assessment Framework

Joint Doctrine Note (JDN) 1-15, Operation Assessment, states that any assessment framework 

consists of three activities: organize and collect data, analyze data, and communicate results and 

recommendations.

Organizing and collecting data includes specifying what data require collection (information 

requirements), and details about who will collect it, how it will be collected, and how the data 

will be stored to support analysis.

Analyzing data consists of, in general, answering these questions:

 Where are we?

 What happened?

 Why do we think it happened? ("we think" is a hat tip to the idea that we are never sure 

"why")

 So what?

 What are the likely future opportunities and risks?
CAA-2015094

76  OPERATIONS ASSESSMENTS DAHB

 What do we need to do?

Analyzing the data is largely an exercise in critical thinking. Mathematical techniques may or 

may not be useful depending upon the nature and quality of the data. Given the nature of the 

operational environment (OE) in complex operations and the quality of data collected, 

quantitative techniques should not be considered superior to critical reasoning. The 

appropriateness, usefulness, and likely validity of quantitative techniques require judgment from 

experienced assessors and others with appropriate expertise.

Analyzing the data is largely about trying to understand what has occurred and why, and 

therefore, what the joint force should do next in order to be most effective. Assessors cannot 

avoid discussions of causality. In practice, assessors will qualitatively examine sets of indicators 

that are related to each other. They will propose likely, possibly multiple, causal explanations 

(candidate hypotheses) for the movement of these indicators. Then they will pose additional 

analytic questions designed to eliminate some of these hypotheses and isolate the most likely 

cause. They do this by going back into the data in order to re-examine and exclude candidate 

hypotheses. 

Much of this analysis is simple math and statistics or simple data visualization, enhanced by any 

more sophisticated quantitative techniques that are appropriate. The overall process should be 

viewed as qualitative in nature, simply because the complexity of the OE defies precise and 

complete measurement—that is, some important feature of the OE will not be precisely 

measureable. 

Communicating results and recommendations includes packaging the pertinent information 

that was identified during the assessment process for sharing with decision makers and, if 

required, for reporting to higher authority. This information includes items, data, or results that 

the commander or other decision makers have asked to see; answers to specific questions that a 

decision maker has posed; and other things the staff has discovered during analysis that they 

think decision makers need to see. 

It is not necessary to prepare slides that show the movement of every indicator. All indicators 

should be examined by functional area experts on the staff, but only a few are sufficiently 

important to occupy senior leaders' time. Moreover, it is usually counter-productive to attempt 

to aggregate several related indicators into some sort of index to reduce the amount of data 

assessors attempt to show senior leaders. Again, the report or brief should include only the 

things senior leaders have asked for and the things that the staff wants to bring to their attention.

It is important to remember that the packaging of information is designed to look forward, to 

inform decisions on future operations in order to make these operations most effective. The 

assessment product, whatever form it takes, is not merely a backward-looking history report.

It is also important to understand that the assessment product, whatever form it takes, is not the 

assessment itself, it is merely a communication of the pertinent portions of the assessment. 

The assessment itself is the improved understanding of the operational environment (OE), 

and our place in it, that we get from a systematic examination of the OE during the 

assessment process. "The assessment" is a cognitive, not a physical or written product, that is 

analogous to a running staff estimate. The assessment product—a brief or a report—is 

essentially the executive summary of this improved understanding. (This is commonly 

misunderstood. Frequently, the words "the assessment" when heard about the headquarters are 
CAA-2015094

DAHB OPERATIONS ASSESSMENTS  77

referring to the report. This is a conceptual problem. For more information, email 

adam.p.shilling.civ@mail.mil.)

JDN 1-15 contains an appendix (Appendix C) that discusses the advantages and disadvantages of 

various means of presentation of assessment results. It is relatively short, and the document is 

available through Google.

Outcomes

"Outcomes" is a single word that includes all of the following: end state, success criteria, 

termination criteria, objectives, sub-objectives, effects, conditions, tasks and purposes, or any 

other words that describe what we want some portion of the operating environment to look like 

in order for us to say that our operations are successful.

All operation assessment is contextual to the desired outcomes of the command performing the 

assessment. Its purpose is to provide the analysis to make operations more effective. Therefore, 

the questions (information requirements) and indicators that provide data for analysis are linked 

explicitly to the set of outcomes the command desires.

In addition to the data a command collects for its own use, higher authority may also require a 

command to report on various items that are of interest to the higher command.

The first duty of an assessor, when possible, is to help the planners write outcome statements that 

are specific. Failure to do this will make these outcomes very difficult to assess, but even more 

importantly, unclear outcomes make it very difficult for subordinate commands to take 

effective action to achieve them. Unclear outcome statements often do not focus or constrain 

subordinates' behavior, making the desired outcomes more difficult to achieve.

Linking Outcomes to Indicators

Once outcomes are as specific as possible, assessors should ask themselves, about each outcome 

statement, "What questions do we need to answer to know we are accomplishing this?" The 

resulting questions are information requirements (IRs). This list should be scrubbed and then 

prioritized to focus collection and reporting on the information that is most valuable. When any 

IR is sufficiently specific that it can be answered by empirical observation, the information that 

answers that question is an indicator.

Adam Shilling, PhD, developed a mental model for linking outcomes to indicators that can help 

assessors identify information requirements and indicators, understand the relationships between

indicators, and organize indicators for collection and storage.
CAA-2015094

78  OPERATIONS ASSESSMENTS DAHB

Figure 25. Mental model for linking outcomes to indicators.

The model is used by beginning at the top with any outcome statement and posing Q1. If the 

statement is judged to be specific enough, the assessor proceeds to Q2. If not, the assessor helps 

the planner break the original statement into more specific outcome statements that "add up" to 

the original outcome. Each of these is evaluated with Q1, and additional iterations through the 

"design loop," until all outcome statements are specific enough to proceed. At this point, 

assessors are helping the planners with "design."

For each specific outcome, assessors pose Q2. This generates a list of questions, also called 

information requirements (IRs), that the command needs to answer to determine whether they are 

progressing toward their desired outcomes. These questions need to be refined until they are 

Outcome Statement

Q1: Is this outcome statement specific or 

does it need to be broken up?

More Specific 

Outcome 

Statements

Q2: What questions do we need to 

answer to know that we doing this?

Specific Broken up

Q3: Can we answer this question 

with empirical evidence?

List of questions 

(information requirements)

-

-

-

-

-

Indicators

-

-

-

Yes Ask more specific 

questions No

More Specific 

Outcome 

Statements Outcome 

Statements Outcome 

Statements

Q2: "What questions do we need to 

answer to know that we are doing this?”
CAA-2015094

DAHB OPERATIONS ASSESSMENTS  79

specific enough to answer with empirical observation (or the judgment of a trusted observer), 

and prioritized in the face of limited collection resources. All of these answers are the set of 

indicators that are optimal. Again, in the real world, all of them may not be collectable, so 

assessors must prioritize the IRs and collect the most important indicators.

A word of caution: some assessors try to skip the list of questions (IRs) portion of the model. 

When they do, the typically develop a set of all possible indicators that have bearing on the 

desired outcomes. This creates a couple of problems: 1. the list of indicators is too long, lacks 

focus, and is hard to prioritize; 2. assessors do not understand the relationship of indicators to 

each other. The formal step of listing IRs (questions) and then identifying indicators that answer 

them helps assessors understand the relationship between several indicators that may answer a 

single question. Also, most people find the list of questions easier to prioritize than an 

exhaustive list of potential indicators. In essence, the formality of posing the questions focuses 

the identification of indicators.

Some observations from experience: Sometimes several questions (IRs) may be answered or 

informed by the same indicator or set of indicators. Some questions may be answered 

individually by one or more indicators; or several related questions may be answered collectively 

by a set of indicators. Some indicators are straight forward and observable, e.g., the number of 

recruits that qualified with their individual weapons in the current class of boot camp. Others 

will require a judgment, which will require an observer(s) trusted to make the judgment, e.g., a 

platoon is proficient in the task "react to ambush." Many more require observers to try to 

classify an empirical event according to set of criteria, e.g., was this attack simple or complex?

Some issues will require a set of IRs and indicators to give the command a thorough 

understanding of each issue. Others may require only enough information to determine that an 

issue is or is not a problem for the command. If a problem exists, the command can focus 

additional effort and resources on understanding and resolving the issue. If not, the command 

may be content with merely monitoring it.

Fully specifying indicators

Selected indicators must be sufficiently well-specified such that any one indicator can be 

collected consistently by multiple observers—at different places or over time. Each will need a 

definition, a plan for collecting the data (who, what, when, why, and how), and be sensitive to 

change within a relevant time frame. If it is calculated, it must have a formula; and it may have a 

target or threshold or a desired rate of change.

Among other pending changes in U.S. doctrine in 2015, the definitions of common terms 

associated with a regime of metrics or indicators are pending change. Joint Doctrine Note (JDN) 

1-15, Operation Assessment, proposes the following new definitions to replace those currently in 

JP 3-0:

Measure of effectiveness (MOE): An indicator used to measure a current system state, with 

change indicated by comparing multiple observations over time.

Measure of performance (MOP): An indicator used to measure a friendly action that is tied to 

measuring task accomplishment. 

Indicator: A specific piece of information that shows the condition, state, or existence of 

something, and provides a reliable means to measure performance or effectiveness. 
CAA-2015094

80  OPERATIONS ASSESSMENTS DAHB

Under previous definitions, a MOE was a criterion used to assess changes in systems, and a 

MOP was a criterion tied to measuring task accomplishment. An indicator provided insight into 

a MOE or MOP. These previous definitions created a hierarchy in which MOEs and MOPs were 

informed by indicators. This old metric regime, in practice, created a number of problems with 

the organization of information for collection and analysis, including that MOEs, as written, 

were rarely specific enough to measure anything. The new definitions destroy this hierarchy and 

provide that both MOEs and MOPs are types of indicator that, by definition, must be specific.

Sources for collecting indicators include routine reporting, intelligence and operations reporting, 

surveys, interagency or international partners, host nation reporting, or sometimes special data 

collection efforts for data not available elsewhere. Frequently, external organizations may 

collect data that are of higher quality on some topic than joint forces can collect themselves, e.g., 

civilian casualty data in Afghanistan are better collected by the United Nations than by ISAF. 

Also, assessors will want to check military data against data owned by external organizations, 

both to double check military collection and to be able to explain why military data may vary 

from other data published in international media.

6.3 More on Causality

Data scientists frequently shy away from talking about causality because the data, if they are of 

the right type, can only demonstrate a statistical correlation of a variable (indicator) to others. In 

assessment however, assessors cannot avoid the issue of causality because ultimately, assessors 

and subject matter experts on the staff must explain why indicators are moving as they are. Any 

operational plan that orders a set of activities in pursuit of a set of outcomes is a set of 

causal hypotheses—that these activities will cause the changes in the OE that are specified 

as outcomes. Assessment can be viewed as a testing of these hypotheses. Therefore, assessors 

must address issues of causality.

Assessors cannot assume, however, that since an outcome occurred, therefore friendly action 

caused it. The OE is full of other actors, some allied or sympathetic to US goals, some opposed 

or adversarial toward American goals, and many third party actors that are pursuing agendas of 

their own. 

Assessors need to consider sets of related indicators together, determine possible causal 

relationships that would account for the movements in indicators, and then go back into the data 

in an attempt eliminate some possible causes, isolating the most likely. 

6.4 Alternatives to MOE-MOP Constructs

Some assessment problems can be addressed through methods other than the formal monitoring 

of specific MOEs and MOPs. Some alternatives include:

The use of published standards, such as Training and Evaluation Outlines (T&EOs) and Soldiers'

Manuals, provide "performance steps" that describe how to correctly perform collective and 

individual military tasks. These can be used as training references, as references for the 

evaluation of training, and as references for the assessment of operations. The correct 

performance of tasks will lead to more effective individuals and units. Moreover, if an outcome 

is not achieved, but we know from a task assessment that the task was performed correctly, then 

we know that performance of that task does not lead directly to that outcome. If the task were 

performed incorrectly, we might conclude the task should be performed again.
CAA-2015094

DAHB OPERATIONS ASSESSMENTS  81

Inspection programs usually feature a qualified observer armed with a checklist. The observer 

answers those checklist questions about the performance or effectiveness of a unit, which is an 

assessment of the unit's capability. He may also mentor the unit on how to do better against the 

items on the checklist, and finally, the inspection itself is an incentive for the evaluated unit to 

periodically examine itself and ensure it is in compliance with standards.

Reports the headquarters receives are assessment documents. Ultimately, assessment data do not 

exist in a sterile database for assessors' analytic use. Observers in the field must witness and 

report the items that are of interest to assessors, and therefore, assessors must tell them what to 

report. Many items are reported routinely already, and gathering that data is a matter of gleaning 

them from existing reporting. For items not currently reported, assessors and staff members will 

need to construct a report format that enables disparate observers to consistently report the 

required information. For example, if a headquarters is tracking naval incursions of an 

adversarial navy into a friendly navy's waters, routine spot reports will provide size, activity, 

location, unit, time, and equipment information that, when analyzed, will answer many of the 

headquarters' questions about the adversary's activity trends and likely intentions. 

Finally, when something out of the ordinary occurs, assessors might need to determine what 

happened and why to either ensure it never happens again (for a negative event) or to institute it 

as a best practice (for a positive event). For example, mission personnel returning from multiday missions in the Horn of Africa reported during their after action reviews that they had 

difficulty contacting the Joint Operations Center (JOC) by phone for submitting evening 

accountability reports. The JOC's representative to the after action review panel investigated and 

determined the failure of the JOC watch to answer the phones was due to the fact that phone 

ringers were turned off during the daily afternoon brief and not consistently turned back on. He 

instituted corrective measures, and reports of the JOC's failure to answer the phones dropped to 

zero.

6.5 More on Assessment Products

Usually, the words, "the assessment," heard about the headquarters refer to the assessment 

product—usually a report or slideshow. In many staff actions, a focus on the "deliverable" is 

helpful to get work done. In operation assessment, this attitude is not as useful. Or more 

accurately, the "deliverable" for assessment is an effective operation, the "product," whatever it 

is, is merely a communication of the pertinent portions of the assessment—the unwritten 

cognitive product—for informing senior leaders or providing accountability.

Assessment products in the past (and often currently) have not reflected this sensibility. Often 

assessment briefs have consisted of many slides that each track an indicator. This is not a 

problem in itself and tracking indicators is critical, but only a few indicators are sufficiently 

important that a senior leader will need to see a slide of its trend. One important example was 

the end strength of the Afghan National Army (see below). Four-star leadership and higher 

authority wanted to see this trend because it was critical to U.S. perception of its success in 

building Afghan institutions. A less important example was the trend of the percentages of 

Afghan army recruits qualifying with their individual weapons. The latter metric is critical to the 

master sergeant who runs the rifle range and to the lieutenant colonel who commands basic 

training, but it is not something general officers need to see. 
CAA-2015094

82  OPERATIONS ASSESSMENTS DAHB

Figure 26. Assessment product example 1.

Many past products also consisted of a simple, often bulleted, summary of a "line of effort." 

Many of these have a stoplight depiction that communicates that the command is on track to 

achieve the outcomes of this line of effort and a depiction of the trend—are we moving in the 

right direction (see below)? These depictions are simplistic. They may be sufficient as 

communication tools provided they stimulate the right discussion among senior leaders, generate 

appropriate recommendations from the staff, and action is taken on them. However, this 

depiction cannot be mistaken for "the assessment" because the staff must still churn through the 

indicators, determine what they mean, posit causal relationships, and make recommendations for 

the command. The staff's mere production of this slide does not constitute assessment.
CAA-2015094

DAHB OPERATIONS ASSESSMENTS  83

Figure 27. Assessment product example 2.

Assessment reports frequently feature narrative descriptions of happenings and their meaning 

that are illustrated by appropriate graphics. A major example is the "Report on Progress Toward 

Security and Stability in Afghanistan," also known as the "1230 report"

20 (available through 

Google). This report provides accountability to the U.S. Congress on military activities in 

Afghanistan. It is a large effort to describe the results of assessment analysis, but it is also a 

model of a narrative-based assessment communication tool that can be scaled down for use 

within a headquarters. Frequently, an effective depiction may be a chart or several charts that are 

captioned with a narrative explaining what they mean, what causal relationships exist, and what 

the command should do.

The bottom line of this discussion is that assessment products may take a variety of forms, some 

of which may or may not be useful, and so the product format must be evaluated according to 

this standard: Does the preparation of this depiction provoke the conversation among the staff 

that promotes nuanced understanding of our OE and help us select appropriate actions for our 

forces? Does this depiction allow the effective communication of our findings and 

 

20 In 2015 the 1230 Report was re-codified by Congress to section 1225 of the US National 

Defense Appropriation Act (NDAA) and now known as the 1225 Report, along with a 

classified annex.
CAA-2015094

84  OPERATIONS ASSESSMENTS DAHB

recommendations, and their implications, to senior leaders? If so, the assessment product is 

probably successful.

One more thing: In general, CAA recommends showing senior leaders only what they ask to see 

and what the staff thinks they need to see. The former is highly correlated with the information 

that is really important, and the latter are the things the staff discovers during the assessment 

process that need to be brought to the attention of senior leaders.

6.6 To read more

Start with Joint Doctrine Note (JDN) 1-15, Operation Assessment (2015) (also available through 

Google). At the time of this writing, it is the most advanced published work on assessment 

doctrine. Older sources are useful for perspective, and there are number of people performing 

assessment in a manner consistent with older publications—you may work for one of these 

people soon—but JDN 1-15 is paradigm shift in why the joint force does operation assessment, 

and older publications reflect obsolete ways of thinking about assessment processes.

CAA also recommends several instructive critiques of the old ways of assessment. These are: 

Jonathon Schroden, Why Operations Assessments Fail (Naval War College Review, Autumn 

2011). 

Stephen Downes-Martin, Operations Assessment in Afghanistan is Broken (Naval War College 

Review, Autumn 2011). 

Ben Connable, Embracing the Fog of War: Assessment and Metrics in Counterinsurgency 

(RAND, 2012).
CAA-2015094

DAHB GEOSPATIAL ANALYSIS  85

CHAPTER 7 GEOSPATIAL ANALYSIS 

7.1 Introduction

Definition. Geospatial analysis seeks to find correlation between physical, cultural, or 

operational features, to answer a specific question. Typically, a Geographic Information System 

(GIS) or geospatial software is used to apply statistical and proximal techniques to 

geographically referenced data to determine if relationships exist within and/or between datasets 

(map layers) to provide insight to the study problem. Simply put, geospatial analysis is the use 

of geographic data to increase the understanding of a given phenomenon. The results can be 

presented in a map, graphic, or in tabular form depending on the nature of the analysis. 

The Purpose of Geospatial Analysis. Geographic analysis is the "where" in the analyst query.

'Where" an event occurs is just as important as "when" and "how" in the context of discerning 

"why." Used appropriately, geospatial analysis can summarize large quantities of data clearly 

and concisely. The addition of a map to operational analysis will provide a reference framework 

to the analysis to enhance and accelerate its understanding. This chapter includes several 

examples to show the advantages of this technique.

7.2 Geospatial Software and Its Uses

Geospatial analysis is a powerful tool that complements other types of analyses. There are 

several geospatial software packages currently used by ORSA analysts, to include (1) ArcGIS, 

(2) FalconView, and (3) Web-Enabled Temporal Analysis System (WebTAS; commonly used 

with the Combined Information Data Network Exchange [CIDNE] database). These tools plot 

geo-referenced data to reveal geospatial relationships within and/or between the datasets, as well 

as other features such as populated places, administrative boundaries, roads, or operating areas. 

Each of the aforementioned software programs has different capabilities and strengths. One can 

use any combination of them to conduct geospatial analysis effectively. ArcGIS and WebTAS 

are able to plot densities, conduct clustering of specific incidents, determine distance/ direction, 

and display results. WebTAS is able to create a "video" of incidents; showing the incident 

pattern both spatially and over time. 

7.3 Examples of Geospatial Analysis

Geospatial analysis has played an important role in analyses and assessments in Iraq, 

Afghanistan, Philippines, the Horn of Africa, and elsewhere. Common themes used in-theater 

include density of attacks, effectiveness of attacks, operating areas, area assessments (security, 

governance, development, and overall), MEDEVAC response times, and polling data 

representation. Geospatial analysis often compresses complex issues to enrich the commander's 

situational awareness. The following are examples of real-world geospatial analysis:
CAA-2015094

86  GEOSPATIAL ANALYSIS DAHB

Figure 28. Napoleon's march to Moscow in 1812

Minard's use of geospatial analysis in 1869 to portray Napoleon's 1812 Russian campaign is 

generally referred to as the best statistical graphic ever drawn (see Figure 28 above). Minard 

uses six separate variables: (1) latitude and (2) longitude of the army as it moved; (3) line width 

continuously marks the size of the army; (4) lines showing the directions the army travels (5) 

temperatures on the return trip; and (6) dates (time). The variables were each instrumental in the 

force's failure. Together, Minard uses these variables to tell the story of Napoleon's forceattrition over the 6 months of the campaign.
CAA-2015094

DAHB GEOSPATIAL ANALYSIS  87

Figure 29. MEDEVAC coverage

Figure 29 depicts medical treatment facilities, aircraft bases, MEDEVAC events, and response 

bands. This illustration is especially useful to a planner who is considering a large operation 

requiring significant MEDEVAC capability. By identifying potential areas of inadequate 

coverage, the planner can work with MEDEVAC units to reposition assets for the operation in a 

way that ensures timely MEDEVAC.
CAA-2015094

88  GEOSPATIAL ANALYSIS DAHB

Figure 30. District assessments in Afghanistan

Figure 30 shows the overall district ratings on a country map. This type of map gives the 

commander a quick snapshot of the general situation to assist in identifying successes and areas 

of concern. While a map does not explain the entire situation, it does provide the commander 

with a clear picture of the current assessment in his or her area of responsibility (AOR). It is 

highly recommended that more detailed, narrative assessments are completed and briefed to the 

commander in addition to the map.
CAA-2015094

DAHB GEOSPATIAL ANALYSIS  89

Figure 31. Map of areas polled in the Philippines

Figure 31 highlights the Zamboanga Peninsula that contractors polled for the BPS and relative 

participation in the SPPPS. This map highlights the Barangay suitable for further analysis, given 

the number of survey results available. 

7.4 Summary

Geospatial analysis is a supplementary tool for the deployed ORSA analyst's toolkit. Geospatial 

software can summarize large quantities of geographically referenced data quickly, clearly, and 

concisely. Geospatial analysis opens a new area to explore and explain the relationships and 

correlations within and/or between multiple geographic datasets. The examples used here are 

only a subset of the range of applications. 
CAA-2015094

90  GEOSPATIAL ANALYSIS DAHB

For training reservations contact the National Geospatial Agency (NGA) College Registrar (571-

588-2968) for the free 1-week courses "Fundamentals of Geographic Information Systems and 

Intermediate Geographic Information Systems (GIS) for Analysis." These courses are available 

in the St Louis, MO and Northern Virginia areas. For additional examples or further assistance, 

contact NGA's Geospatial Intelligence section or the CAA Operations Analysis (OA) reachback

team POCs listed at http://www.caa.army.mil/Commanders_Trifold_Jan2016.pdf. 
CAA-2015094

DAHB APPENDIX A  91

CHAPTER 8 REACHBACK CAPABILITIES

8.1 Why Reachback?

The Generating Force, i.e., institutional army analytical organizations, provide analyses through 

an integrated combination of embedded ORSA analysts—either assigned or attached to Army, 

Joint, and Multi-National headquarters—and through analytical reachback support (also known 

simply as "reachback"). Operations Research/Systems Analysis is vital in reducing commanders' 

uncertainty through sound reasoning and well-constructed analytical models. Reachback allows 

ORSA analysts serving at Division, Corps, ASCCs, and Multi-National Command headquarters 

in deployed environments to draw upon resources and capabilities of Generating Force 

organizations and institutions whose primary mission is to generate and sustain the Operational 

Army's capabilities for employment by Joint Force commanders. 

As a deployed ORSA analyst, one applies logical reasoning and sound processes to solve highly 

complex problems at the operational and strategic levels when no readily apparent solutions 

exist. One applies analytical methods to perform trade-off analysis, compare courses of action, 

allocate critical resources, and assess operational effectiveness. These methods are an integral 

part of Army and Joint leadership decision-making processes to organize, staff, train, equip, 

sustain, and resource current and future forces. One conducts analyses using one's personal 

talents and skill set unless the complexity and scope of a given problem require additional 

resources.

8.2 Motivational Factors for Reachback 

Integration of Analytical Effort. Reachback connects Operational Army analytical efforts to 

Generating Force organizations. This connectivity ensures analytical relevance along with 

greater capability within the operational headquarters. Specifically, reachback provides visibility 

of critical analytical requirements to Generating Force organizations. Additionally, this 

connectivity captures institutional knowledge for future deploying ORSA analysts.

Subject Matter Expertise. Generating Force organizations possess a wide array of expertise. 

Given the limited number of deployed ORSA analysts and the complexity of emerging issues, 

reachback provides a valuable mechanism for expanding one's expertise and analytical 

capabilities. Furthermore, organizations that provide reachback are frequently staffed with 

ORSA analysts who have or will deploy. This provides important continuity for analytical 

support to theater commanders.

Additional Data. Reachback expands available data and offers contextual reference for theaterspecific analytical problems.

Analytical Review Process. Reachback provides a natural review process for analytical efforts. 

Through collaborative reachback and Generating Force organizational capabilities, multiple 

personnel and organizations review reachback analytical products. While the review process 

requires more time, it ensures high quality products.

Analytical Tools. Reachback provides one with access to sophisticated tools and additional 

expertise. Two examples of support provided to operational headquarters are provided below. 

Figure 32 is an example of an R script written to replace a manual process used downrange to 
CAA-2015094

92  APPENDIX A DAHB

download and index information from web pages on the internet, which saved hundreds of hours 

of manual processing time for a deployed headquarters. 

Figure 32. Example code (script) written to support reachback request

Figure 33 is an example of an interactive web-based analytic tool developed for an Army 

operational headquarters to provide time series analysis and visualization of enemy activity. 
CAA-2015094

DAHB APPENDIX A  93

Figure 33. Example web application developed to support reachback request

Required Surge Capability. The connection between Operational Army commands and 

Generating Force organizations, enabled by reachback, allows for analytical surge capability as 

required. For example, from the onset of OIF and OEF, CAA, TRAC, Army Materiel Systems 

Analysis Activity (AMSAA), and the United States Military Academy (USMA) provided 

Deployed Analyst Support Teams (DASTs) to ASCCs in Iraq and Afghanistan. A recent change 

in U.S. military operational focus has resulted in these organizations expanding their analytical 

support to commands located in additional geographic locations, e.g., the Philippines and Africa.

8.3 Reachback Considerations

Timeliness. Deployed operational Army as well as Joint and Combined headquarters (HQs) 

generally operate 7 days a week/12-14 hours a day. An analytical request to/from these 

headquarters often has a quick-turn suspense that requires an answer within a few hours or days. 

generating force organizations performing reachback support must adapt to these "quick-turn"

requirements. Deployed ORSA analysts must adapt to the environment within which generating 

force organizations operate. This environment requires a detailed, time-intensive, review process 

to ensure a high quality product.
CAA-2015094

94  APPENDIX A DAHB

Confidentiality of Derived Analysis. Generating Force organizations are customer-focused. 

They must obtain Operational Army Command release authority before disseminating derived 

analytical products. This release protocol ensures customer confidentiality in what often 

concerns contentious issues.

Organizational Capabilities. One should become familiar with the capabilities of Generating 

Force organizations. This familiarity enables one to incorporate current organizational efforts 

and understand where a problem solution may reside.

Contextual Reference. One must clearly articulate one's analytical requirements when 

requesting reachback support. A well-articulated request ensures that reachback personnel 

address the problem within the context of other competing issues such as time constraints, level 

of model sophistication, and the essential elements of analysis.

8.4 Reachback Support Topics

Generating Force analytical support to the current fight has thus far included campaign plan 

assessments, casualty analyses, theater campaign modeling and analyses, force-on-force 

analyses, stability and support operations analyses, and weapon systems analyses. ORSA topics 

span doctrine, organizations, training, materiel, leadership and education, personnel, and 

facilities (DOTMLPF). Examples of reachback support topics include:

 Campaign plan assessments. Campaign plan assessments include the development of 

appropriate MOEs and MOPs.

 Casualty analyses. Casualty analyses include efficacy of personal equipment and 

implications for operating force TTPs.

 Theater campaign modeling and analysis. Theater campaign modeling focuses on the 

Joint and Combined operational/strategic environment. These models incorporate 

weapons effectiveness data, unit formations, and current war plans in support of 

campaign analyses. Additionally, air and missile defenses, along with weapons of mass 

destruction (chemical, biological, radiological, and nuclear), are analyzed at the tactical, 

operational, and strategic levels of war. The Total Army Analysis (TAA) process uses 

analytical output to determine future force requirements for the Army.

 Force-on-force modeling and simulation. Generating Force ORSA analysts develop 

and maintain a class of warfighting force-on-force models and simulations (M&S) 

ranging from individual objects, e.g., Soldiers, weapons, terrain features to theater-level 

models that often aggregate objects, e.g., battalions at Corps level. AMSAA leads Army 

efforts in modeling platform performance parameters. TRAC focuses on Corps and 

below force-on-force combat models while CAA conducts theater-level campaign 

analysis. Collectively, these models are widely used by DOD, industry, and allied 

nations. 

 Stability and support operations. Generating Force ORSA organizations contribute to 

stability operations with a variety of analytical products, e.g., campaign plan assessment 

methods development, convoy protection analysis, sensor placement recommendations, 

basing analysis, medical asset allocation recommendations, new materiel fielding and 

utilization analysis, Intelligence Preparation of the Battlespace (IPB) assistance, attack 

pattern analysis, economic forecasting, and force structure/size recommendations. The 
CAA-2015094

DAHB APPENDIX A  95

foregoing ORSA capabilities assist in promoting a secure environment and aid in 

diplomatic and economic programs designed to eliminate root causes of instability. 

These contributions complement and reinforce overall Stability and Support Operations 

(SASO).

 Weapon Systems analysis. Weapons Systems analysis enables an understanding of the 

system, its functions, performance and effectiveness measures, and operational criteria. 

The ORSA community can provide a comprehensive assessment of the impact of a given 

friendly or enemy weapons system on the operational environment. The Army Test and 

Evaluation (ATEC) Command is one Generating Force organization with keen weapon 

systems insights.

 Programmatic analysis. The cost of campaigns and major operations strongly affects 

domestic support and helps determine success or failure of a given mission. ORSA 

analysts aid in decision making with analysis of logistics, force structure, and 

mobilization/deployment simulation modeling. These projects include force closure 

estimates, resources for mobilization/deployment data, lift asset requirements, prepositioning recommendations, and high-level quick-turn COA analysis. ORSA analysts 

conduct analysis of support forces, i.e., combat support and combat service support 

requirements and casualty analysis, develop wartime Class V and Class VII requirements, 

and compare theater logistics requirements with capabilities.

8.5 The Reachback Process

General. 

Each Generating Force organization manages its respective reachback mechanisms 

independently. CAA conducts an informal biweekly interagency meeting with deployed ORSA 

analysts in an effort to coordinate reachback efforts. While there is an effort to consolidate and 

formalize reachback contributions from Generating Force organizations, this chapter focuses on 

the overall reachback process.

Army Materiel System Analysis Agency (AMSAA). AMSAA conducts analyses across the 

Materiel Lifecycle to inform critical decisions for current and future Warfighter needs while 

valuing the unique knowledge, experiences, and backgrounds of its people.

Army Test and Evaluation Command (ATEC). ATEC plans, integrates, and conducts 

experiments, developmental testing, independent operational testing, and independent 

evaluations and assessments to provide essential information to acquisition decision makers and 

commanders.

Center for Army Analysis (CAA). CAA conducts analyses across the spectrum of conflict in a 

Joint, Interagency, Intergovernmental, and Multinational (JIIM) context to inform critical seniorlevel decisions for current and future national security issues. CAA supports DA and its major 

subordinate commands with analyses of Army forces and systems within the context of joint and 

combined warfighting. CAA analyzes strategic concepts and military options and estimates 

requirements to support Army input to the Planning, Programming, Budgeting, and Execution 

System (PPBES). CAA evaluates (1) the Army's ability to mobilize and deploy forces, (2) Army 

force capabilities, (3) force alternatives, and (4) theater force-level scenarios and resource 

analyses.
CAA-2015094

96  APPENDIX A DAHB

Marine Corps Combat Development Command (MCCDC). MCCDC fully integrates Marine 

Corps concepts and requirements based warfighting capabilities; including doctrine, 

organization, training, materiel, leadership and education, personnel and facilities in order to 

ensure the Marine Corps is properly organized, trained and equipped now and in the future.

U.S. Army Training and Doctrine Command Analysis Center (TRAC). TRAC serves as the 

principal analytical organization of TRADOC and provides centralized leadership and 

management of analysis for combat, training, and doctrinal developments, while unaligned with 

TRADOC proponents. TRAC conducts studies and analyses for TRADOC and Headquarters, 

Department of the Army (HQDA); conducts studies of the integrated battlefield related to 

doctrine, organization, training, materiel, personnel, and leadership; designs and develops 

models and simulations (M&S) for capabilities development; participates in technical exchange 

programs at the national and international levels; provides analytical support to Army 

Capabilities Integration Center (ARCIC), Centers of Excellence (CoEs), and schools; directs 

research related to methods, models, tools, and analysis; establishes, maintains, and manages the 

databases, scenarios, models, and wargaming tools required to support analyses and studies; and 

reviews and ensures, as directed, the quality of TRADOC studies before their approval.

Non-ORSA Generating Force organizations. The following Generating Force organizations 

also provide relevant information to many ORSA efforts:

 National Ground Intelligence Center (NGIC). NGIC provides All Source and 

Geospatial Intelligence on foreign ground force capabilities and related military 

technologies and integrates with Mission Partners to ensure the U.S. Army, DoD, Joint, 

and National level decision makers maintain decision advantage to protect the U.S. and 

interests abroad.

 Joint Improvised-Threat Defeat Agency (JIDA). JIDA enables Department of 

Defense actions to counter improvised threats with tactical responsiveness and through 

anticipatory, rapid acquisition in support of Combatant Commands' efforts to prepare for, 

and adapt to, battlefield surprise in support of counter-terrorism, counter-insurgency, and 

other related mission areas including counter-improvised explosive device.

Communication Methods. Deployed ORSA analysts and Generating Force organizations 

communicate using the Defense Switched Network (DSN), commercial telephone, Non-secure 

Internet Protocol Router Network (NIPRNET), Secret Internet Protocol Router Network 

(SIPRNET), Secure Video Telephone Conference (SVTC), Information Work Space (IWS), and 

other means beyond the scope of this chapter. Periodic communications ensure established 

relationships between deployed ORSA analysts and their respective organizations. Unclassified 

contact information for the most commonly requested reachback Generating Force organizations 

follows:

 Army Logistics University (ALU) College of Professional and Continuing 

Education. Available from http://www.alu.army.mil/; Commercial: (804)765-

4710/4991/4553 or DSN (539) 4710.

 Army Materiel Systems Analysis Activity (AMSAA). Available from 

http://www.amsaa.army.mil/home.html; Commercial (410) 278-6614 or DSN (298) 6614.
CAA-2015094

DAHB APPENDIX A  97

 Army Test and Evaluation Command (ATEC). Available from 

http://www.army.mil/ATEC; Commercial (443) 861-9999 or DSN (848) 9999.

 Center for Army Analysis (CAA). Available from http://www.caa.army.mil; 

Commercial (703) 806-5580/5680 or DSN (656) 5580.

 Marine Corps Combat Development Command (MCCDC). Available from 

http://www.mccdc.marines.mil; Operations Analysis Directorate; Commercial (703) 784-

6293 or DSN (278) 6293. 

 National Ground Intelligence Center (NGIC). Available from

https://www.inscom.army.mil/msc/NGIC.aspx; Commercial (434) 980-7000. 

 U.S. Army Training and Doctrine Command Analysis Center (TRAC). Available 

from http://www.trac.army.mil; Commercial (913) 684-9150 or DSN (552) 9150.

8.6 Knowledge Management

Concept. The U.S. Army ORSA community provides a centralized reachback team, trained and 

equipped to coordinate effective and timely analytical contributions to U.S. Army ORSA 

analysts who are serving within deployed elements of an operational headquarters, e.g., 

Division/Corps/ASCC/Echelons above Corps [EAC]/Joint/Combined). CAA OA Division—

with participating Generating Force organization assistance—maintains a SharePoint site 

(focused on current operations) on Non-secure Internet Protocol Router Network (NIPRNET) to 

facilitate deployed ORSA operations and analytical reachback support. 

Vision. The CAA OA Division Current Operations Team SharePoint site identifies, creates, 

represents, and distributes knowledge for reuse, thereby enabling awareness and learning—with 

specific emphasis on current operations support—across the ORSA community. Key objectives 

of this site include shared intelligence, improved performance, competitive advantage over 

adversaries, and higher levels of innovation.

Structure. Figure 34 shows the general structure of the CAA Operations Analysis (OA) 

Division NIPRNET SharePoint site available from 

https://army/deps.mil/army/cmds/CAA/OA/SitePages/Home.aspx. CAA designed this 

SharePoint site to facilitate coordination between analytical agencies and deployed ORSA 

analysts regarding reachback and Irregular Warfare (IW) projects and data issues. Featured 

information includes organization points of contact (POCs), a request form for reachback 

support, and contact information for all known deployed ORSA analysts.
CAA-2015094

98  APPENDIX A DAHB

Figure 34. CAA Operations Analysis Division SharePoint Site 

Reachback Request Form. Figure 35 shows the structure of the Request for Reachback Form 

that can be accessed through SharePoint site at http://www.caa.army.mil with U.S. CAC login.

Figure 35. Reachback Request Form 

8.7 Formulating Reachback Problems

In general, Generating Force organizations should adapt to the requirements of deployed ORSA 

analysts, especially in terms of the timeliness of derived products. To aid Generating Force 

organization endeavors, inquiries for reachback support should include the following information 

in as much detail as possible:
CAA-2015094

DAHB APPENDIX A  99

 Problem Statement. One should provide a clear definition of the problem along with 

essential elements of analysis. In particular, what question should reachback answer? 

One should provide a description of all assumptions related to the problem. If possible, 

one should describe potential modeling methods.

 Problem Background. One should explain factors/issues bearing on the problem. This 

guides one's literature review and provides insight to reachback personnel. One should 

describe the ultimate customers and how this reachback effort supports their decision 

making process and how it relates to other theater efforts, i.e., issues surrounding the 

analysis.

 Data. One should provide data or recommendations on how to generate/obtain the data 

necessary for the analysis.

 Timeline and suspense. One should specify desired completion dates and the impact of 

delayed analysis (details on intermediate milestones aid in project management and 

ensure customer satisfaction).

 Communications. One should establish recurrent communication with the supporting 

agency via the CAA Current Operations Team SharePoint site on NIPRNET and other 

communication tools, e.g., SIPRNET.

 Classification. The deployed analyst must be the expert as to the classification of 

(typically coalition derived) data and products, and be able to explain to others how the 

classification relates to the classifications of a home nation and other relevant 

international organizations, e.g., the North Atlantic Treaty Organization (NATO).

 Releasability. One should specify whether the customer has approved the release of 

derived analytical products developed for them. Analytical organizations must maintain 

customer confidentiality; however, release of analytical products of current issues and 

trends—where possible—furthers understanding within the ORSA community.

8.8 Coordinating Intra-/Inter-Theater Reachback Efforts

Documenting Analyses. Proper documentation of ongoing and completed analyses provides an 

initial start point for new analysis and reduces duplication of efforts. The CAA Current 

Operations Team asks deployed ORSA analysts to provide a concise briefing (normally one or 

two slides) on a biweekly basis to the CAA Current Operations meeting. This meeting focuses 

on current ORSA projects, ongoing reachback projects, and other miscellaneous activities that 

support the warfighter. 

 Sharing Data. A mechanism for sharing very large unclassified data files is the U.S. 

Army Aviation and Missile Research Development and Engineering Center (AMRDEC) 

site that can be accessed at https://safe.amrdec.army.mil/safe. AMRDEC allows 

Common Access Card (CAC) holders to send files over AMRDEC SAFE (Safe Access 

File Exchange), and also allows non-CAC holders to send files to CAC holders. On the 

SIPRNET, large data files can be shared through the U.S. Air Forces Central Command 

(USAFCENT) Large File Distribution System (LFDS) at 

https://lfds.shaw.afcent.af.smil.mil/default.aspx. Headquarters Resolute Support (RS) 

command also uses a system called Centaur to allow specially designated and trained 
CAA-2015094

100  APPENDIX A DAHB

U.S. staff officers and analysts to quickly move UP TO RESOLUTE SUPPORT21 (RS) 

SECRET files from the RS Mission SECRET network to SIPRNET network, and from 

the SIPRNET network to the RS Mission SECRET network. Additionally, 

USCENTCOM provides a two-person review air gapping service to transfer files from 

SIPRNET and ISAF/SECRET or CENTRIXS to SIPRNET, NIPRNET, or 

ISAF/SECRET or CENTRIXS – the files must be marked with proper classifications on 

every page or sheet. The files must be emailable and it takes a couple days for 

USCENTCOM to send the files back to you – or – embarrassingly, tell you that your files 

are not marked properly or simply of the wrong classification to move to destination 

network you requested. Here are the links you need:

o Moving files from SIPRNET to NIPRNET, or RS SECRET or CENTRIXS: 

http://www.centrixs.centcom.smil.mil/CFTS.cfm

o Moving files from RS SECRET or CENTRIXS: http://22.11.80.14/cfts/cfm; use 

http://22.11.80.14/CFTS.cfm if the former link does not work.

Generating Force organizations alleviates many of the issues with data storage and analysis. 

Data management and analysis continue to be a source of reachback contributions. 

Recurrent Coordination Meetings. Recurrent meetings provide coordination and collaboration 

between Generating Force organizations and deployed ORSA analysts. Continued support of 

formal and informal coordination meetings enables effective collaboration.

Knowledge Management. The CAA OA Division Current Operations SharePoint site provides 

a mechanism to manage creation, identification, accumulation, and application of knowledge for 

the Operations Research (OR) community. This effort brings together inherent intellectual 

capital.

8.9 Pre-Deployment Training Opportunities for Deploying Analysts

Deploying analysts as well as organizations deploying ORSA analysts to conflict areas should 

work in advance to identify and resource applicable tailored training opportunities for each 

deployer. The following known training opportunities provide ORSA analysts an initial menu of 

courses to consider:

 The CAA hosted no-fee Deploying Analyst Course (DAC), offered twice yearly 

(currently March and September) at Fort Belvoir, VA; POCs at 

http://www.caa.army.mil/Commanders_Trifold_Jan2016.pdf. 

 The Individual Terrorism Awareness Course (INTAC), provided by the Army Corps of 

Engineers in Winchester, VA, POC Bill Bicking at Bill.Bicking@usace.army.mil. 

 Combined Information Data Network Exchange (CIDNE) & Web-Enabled Temporal 

Analysis System (WebTas); POC Jason Thornton, ISS Corp., Alexandria, VA at 

Jason.Thornton@issinc.com. 

 

21 HQ Resolute Support (RS) is the successor coalition organization that replaced HQ 

International Security Force (ISAF) on 1 JAN 2015.
CAA-2015094

DAHB APPENDIX A  101

 Training offered by the deployers future deployed command hosted in the Continental 

United States.

 Geospatial analysis training hosted by the National Geospatial Agency (NGA); see 

course opportunity information at section 7.4, above.

 Contracting Officer's Representative (COR) training; see Defense Acquisition University 

(DAU) website at http://icatalog.dau.mil/onlinecatalog/courses.aspx?crs_id=1731. 

 NATO related training opportunities at the NATO Joint Force Training Centre (JFTC) 

Bydgoszcz.

 The Operations Assessment Course at the NATO School in Oberammergau, Germany. 

See https://www.natoschool.nato.int/Academics/Resident-Courses/Course-Catelog; 

Enrollment POC is Boyd Forbes (Forbes.Boyd@natoschool.nato.int).

 For assistance identifying viable pre-deployment for deploying analysts, contact the CAA 

Operations Analysis (OA) reachback team POCs listed at 

http://www.caa.army.mil/Commanders_Trifold_Jan2016.pdf. 
CAA-2015094

102  APPENDIX A DAHB

(THIS PAGE INTENTIONALLY LEFT BLANK)
CAA-2015094

DAHB APPENDIX A  103

APPENDIX A LIST OF ACRONYMS

AAG Afghan Assessment Group

ACOM Army Command

AMSAA Army Materiel Systems Analysis Activity 

ANA Afghan National Army

ANDP Afghan National Development Program

ANSF Afghan National Security Forces

AOR area of responsibility

ArcGIS Arc Geographic Information System

ASCC Army Service Component Commands

ASCII American Standard Code for Information Interchange

ATEC Army Testing and Evaluation Command

CAA Center for Army Analysis

BPS Bangsamoro Perception Survey

CENTRIX Combined Enterprise Regional Information Exchange System

CF coalition forces

CIDNE Combined Information Data Network Exchange

CIO Chief Information Office

CJSOTF-A Combined Joint Special Operations Task Force - Afghanistan

CJTF-HOA Combined Joint Task Force – Horn of Africa

CLA Constraints, Limitations, and Assumptions

COA course of action

CoEs Centers of Excellence 

COR Contracting Officer's Representative

DA Department of the Army

DA PAM Department of the Army Pamphlet

DAC Deploying Analyst Course

DAHB Deployed Analyst Handbook
CAA-2015094

104  APPENDIX A DAHB

DAHR Deployed Analyst History Report

DAST Deployed Analyst Support Team

DAU Defense Acquisition University

DOD Department of Defense

DOTMLPF Doctrine, Organizations, Training, Materiel, Leadership, Personnel, and Facilities

DSN Defense Switched Network

EAC Echelons Above Corps

FA Functional Area

GIS Geospatial Information System

HN Host Nation

HOA Horn of Africa

HQ Headquarters

HQDA Headquarters, Department of Army

ID Identification

IED Improvised Explosive Device

INTAC Individual Terrorism Awareness Course

INFORMS Institute for Operations Research and the Management Sciences

IPB Intelligence Preparation of the Battlespace

IR Information Requirement

ISAF International Security Assistance Force

ISF Iraqi Security Forces

ISR Intelligence, Surveillance, and Reconnaissance

IW Irregular Warfare

IWS Information Work Space

JP Joint Publication

JSOTF-P Joint Special Operations Task Force – Philippines 

M&S Models and Simulations

MCCDC Marine Corps Combat Development Command 

MDMP Military Decision Making Process
CAA-2015094

DAHB APPENDIX A  105

MEDEVAC Medical Evacuation

MNF-I Multi-National Forces – Iraq 

MOE Measure of Effectiveness

MOP Measure of Performance

NATO North Atlantic Treaty Organization

NGIC National Ground Intelligence Center

NGA National Geospatial Agency

NGO Non-Governmental Organization

NIPRNET Non-secure Internet Protocol Router Network

NPS Naval Post Graduate School

OA Operations Analysis

OCO Overseas Contingency Operation

ODBC Open Database Connectivity

OE Operational Environment

OEF Operation Enduring Freedom

OFS Operation Freedom Sentinel

OIF Operation Iraqi Freedom

OIR Operation Inherent Resolve

OLAP Online Analytical Processing

OLE DB Object Linking and Embedded Database

OMB Office of Management and Budget

OND Operation New Dawn

OR Operations Research

ORSA Operations Research/Systems Analysis

PMESII Political, Military, Economic, Social, Information, and Infrastructure

POC Point of Contact

PPBES Planning, Programming, Budgeting, and Execution System

P-Value Probability Value

RS Resolute Support
CAA-2015094

106  APPENDIX A DAHB

SAS Statistical Analysis System

SASO Stability and Support Operations

SIPRNET Secret Internet Protocol Router Network 

SPPPS Southern Philippines Public Perception Survey

SPSS Statistical Package for the Social Sciences

TAA Total Army Analysis

TRAC TRADOC Analysis Center

TRADOC United States Army Training and Doctrine Command

TTPs Tactics, Techniques, and Procedures

U.S. United States

USAID United States Agency for International Development

USMA United States Military Academy

VBA Visual Basic for Applications

VTC Video Telephone Conference

WebTAS Web-Enabled Temporal Analysis System
CAA-2015094

DAHB APPENDIX B  107

APPENDIX B REFERENCES

Albright, S. C., Winston, W., & Zappe, C. (2011). Data analysis and decision-making. Mason, 

OH: South-Western Cengage Learning.

Connable, B. (2012). Embracing the fog of war: Assessment and metrics in counterinsurgency. 

U.S. Department of Defense. Santa Monica, CA: Rand Corporation. 

http://www.rand.org/content/dam/rand/pubs/monographs/2012/RAND_MG1086.pdf

Devore, J. L. (2012). Probability and statistics for engineering and the sciences. Boston, MA: 

Cengage Learning.

Downes-Martin, S. (2011). Operations assessment in Afghanistan is broken: What is to be done. 

Naval War College Review 64(4), p. 105. Available from 

http://www.usnwc.edu/getattachment/e8262311-4922-4b8c-b14c8714992130ba/Operations-Assessment-in-Afghanistan-Is-Broken--Wh

Forecasting Principles: Evidence-based Forecasting (2012). Available from 

http://www.forecastingprinciples.com

Makridakis, S., Wheelright, S. C., & Hyndman, R. J. (1998). Forecasting: Methods and 

applications. New York, NY: John Wiley & Sons.

NATO Operations Assessment Handbook (2011, January 29). Available from 

https://www.mccdc.usmc.mil/OperationsAnalysis/files/20101101%20NATO%20Operati

ons%20Assessment%20HB.pdf

Savage, S. L. (2003). Decision making with insight. South Western Educational Publishing.

Schroden, J. (2011). Why operations assessments fail: It's not just the metrics. Naval War 

College Review 64(4), pp. 95-99. Available from 

https://www.usnwc.edu/getattachment/6ed0e5cd-621f-44c3-b40d-913cad7e8c48/WhyOperations-Assessments-Fail--It-s-Not-Just-the

U.S. Department of the Army ATTP 5-0.1 (2011, p. 2-23). Army tactics, techniques, and 

procedures: Commander and staff officer guide. Washington, DC: Headquarters, 

Department of the Army.

U.S. Department of the Army Field Manual 3-24 (2006, December 15). Counterinsurgency. 

Washington, DC: Headquarters, Department of Defense. Available from 

http://www.fas.org/irp/doddir/army/fm3-24.pdf

U.S. Department of the Army Joint Publication 3-0 (2011). Joint Operations. Washington, DC: 

Headquarters, Department of the Army. Available from http://www.dtic.mil/doctrine/
CAA-2015094

108  APPENDIX B DAHB

U.S. Department of the Army Pamphlet 600-3 (2014, December 3). Commissioned officer 

professional development and career management. Washington, DC: Headquarters, 

Department of the Army.

U.S. Department of the Army, Training and Doctrine Command, TRADOC Regulation 71-20 

(2013, June 28). Concept development, capabilities determination, and capabilities 

integration. Washington, DC: Headquarters, Department of the Army. Available from 

www-tradoc.army.mil/tpubs/regs/tr71-20.doc

U.S. Department of Defense Directive Number 2000.19E (2006, February 14). Joint Improvised 

Explosive Device Defeat Organization (JIEDDO). Washington, DC: Headquarters, 

Department of Defense. Available from 

http://www.dtic.mil/whs/directives/corres/pdf/200019p.pdf

U.S. Department of Defense Joint Publication 1-02 (2010, November 8 as amended thru 2012, 

March 15). Dictionary of military and associated terms. Washington, DC: Headquarters, 

Department of Defense. Available from 

http://www.dtic.mil/doctrine/new_pubs/jp1_02.pdf

U.S. Department of Defense Joint Publication 2-01.3 (2009, June 16). Joint intelligence 

preparation of the operational environment. Washington, DC: Headquarters, Department 

of Defense. Available from http://www.fas.org/irp/doddir/dod/jp2-01-3.pdf
(THIS PAGE INTENTIONALLY LEFT BLANK)
110  APPENDIX B DAHB