# Operation Assessment Multi-Service Tactics, Techniques, And Procedures For Operation Assessment

## Atp 5-0.3 Mcrp 5-10.1 Nttp 5-01.3 Afttp 3-2.87 February 2020 Foreword

This multi-Service tactics, techniques, and procedures (MTTP) publication is a project of the Air Land Sea Application (ALSA) Center in accordance with the memorandum of agreement between the Headquarters of the Army, Marine Corps, Navy, and Air Force doctrine commanders directing ALSA to develop MTTP publications to meet the immediate needs of the warfighter.

This MTTP publication has been prepared by ALSA under our direction for implementation by our respective commands and for use by other commands as appropriate.

Major General, US Army Major General, US Marine Corps Director Commanding General Mission Command Center of Excellence Training and Education Command

## J. F. Meier Brad M. Sullivan

Rear Admiral, US Navy Major General, US Air Force Commander Commander
Navy Warfare Development Command Curtis E. Lemay Center for Doctrine Development and Education This publication is available through the following websites:
ALSA (http://www.alsa.mil/);
US Army (https://armypubs.army.mil);
US Marine Corps
(https://homeport.usmc.mil/sites/mcdoctrine/SitePages/Home.aspx);
US Navy at Navy Warfare Library (https://doctrine.navy.mil);
US Air Force Center for Doctrine Development and Education
(http://www.doctrine.af.mil); and Joint Electronic Library Plus
(https://jdeis.js.mil/jdeis/index.jsp?pindex=0).

## Preface

1. Purpose This multi-Service tactics, techniques, and procedures (MTTP) publication serves as a commander and staff guide for integrating assessments into the planning and operations processes for operations conducted at any point along the range of military operations. It provides operation assessment how-to techniques and procedures which complement current joint and Service doctrine. The MTTP is a means for ensuring appropriate assessment information gets to the right decision maker at the right time.

## 2. Scope This Mttp Publication:

Explains assessment techniques and procedures to make operations more effective.

Provides an assessment framework that:

(1) Aligns with Joint Publication 5-0, Joint Planning.
(2) Describes staff and commander actions during each phase of an operation.
(3) Allows for a common reference to enable effective communication between echelons, and between commanders and their staffs.
Describes assessment planning and integration into the planning and operations processes.

Offers operation assessment techniques and procedures adaptable to each component's general circumstance while recognizing Services performing similar assessment activities generally focused on differing domains.

## 3. Applicability This Mttp Publication Applies To Commanders And Their Staffs That Conduct Operations.

4. Implementation Plan Participating Service command offices of primary responsibility will review this publication; validate the information; and, where appropriate, reference and incorporate it in Service manuals, regulations, and curricula as follows:
Army. Upon approval and authentication, this publication incorporates the tactics, techniques, and procedures contained herein into the United States (US) Army Doctrine and Training Literature Program as directed by the Commander, US Army Training and Doctrine Command. Distribution is in accordance with applicable directives listed on the authentication page.

Marine Corps.1 The Marine Corps will incorporate the procedures in this publication in United States Marine Corps (USMC) doctrine publications as directed by Commanding General, Training and Education Command (TECOM). Distribution is in accordance with the Marine Corps Publication Distribution System.


Navy. The Navy will incorporate these procedures in US Navy training and doctrine publications as directed by the Commander, Navy Warfare Development Command (NWDC) (N5). Distribution is in accordance with MILSTRIP/MILSTRAP Desk Guide, Naval Supply Systems Command Publication 409.


Air Force. The Air Force will incorporate the procedures in this publication in accordance with applicable governing directives. Distribution is in accordance with Air Force Instruction 33-360, *Publications and Forms Management*.

## 5. User Information

US Army Combined Arms Center; Headquarters, Marine Corps, DC, CD&I;
NWDC; Curtis E. LeMay Center for Doctrine Development and Education; and Air Land Sea Application (ALSA) Center developed this publication with joint participation of the approving Service commands. ALSA will review and update this publication as necessary.

This publication reflects current joint and Service doctrine, command and control organizations, facilities, personnel, responsibilities, and procedures. Changes in Service protocol, appropriately reflected in joint and Service publications, will be incorporated in revisions to this document.

We encourage recommended changes for improving this publication. Key your comments to the specific page and paragraph and provide a rationale for each recommendation. Send comments and recommendations directly to:

## Army

Commander, US Army Combined Arms Center ATTN: ATZL-MCD Fort Leavenworth KS 66027-6900 DSN 552-4885 COMM (913) 684-4885 E-mail: usarmy.leavenworth.mccoe.mbx.cadd-org-mailbox@mail.mil

## Marine Corps

Commanding General, Training and Education Command MAGTF Training and Education Standards Division, Doctrine Branch ATTN: C466 Quantico VA 22134 DSN 278-8393 COMM (703) 432-8493 E-mail: doctrine@usmc.mil

## Navy

Commander, Navy Warfare Development Command ATTN: N5 1528 Piersey St, Building O-27 Norfolk VA 23511-2723 DSN 341-4185 COMM (757) 341-4185 E-mail: nwdc_nrfk_fleet_pubs@navy.mil

## Air Force

Commander, Curtis E. LeMay Center for Doctrine Development and Education ATTN: DDJ 401 Chennault Circle Maxwell AFB AL 36112-6428 DSN 493-7864/1681 COMM (334) 953-7864/1681 E-mail: afddec.ddj@us.af.mil

## Alsa

Director, ALSA Center 114 Andrews Street Joint Base Langley-Eustis VA 23665-2785 DSN 575-0902 COMM (757) 225-0902 E-mail: alsadirector@us.af.mil This page intentionally left blank.

## Summary Of Changes

ATP 5-0.3/MCRP 5-10.1/NTTP 5-01.3/AFTTP 3-2.87, Multi-Service Tactics, Techniques, and Procedures for Operation Assessment.

This revision: Updates:


The operation assessment process to align with the six steps outlined in Joint Publication 5-0, Joint Planning.

Bridges the gap between strategic assessments to tactical operation assessments.

Increased ways to communicate assessments.
Removes:


Outdated examples from Operation ENDURING FREEDOM and Operation IRAQI FREEDOM.

Assessment terminology from chapter 1, moved the applicable terms to the
glossary.

The detailed discussion on measures of effectiveness and measures of performance as they are both types of indicators.
Adds:


Discussion on each of the six steps of the assessment model.

An appendix on a model to drive a planners thinking, to allow the user to define
each objective and link one or more indicators to each.

An appendix with examples of assessment, collection, and communication plans.

An appendix on the orders format for assessments from Service doctrine.
This page intentionally left blank.

NTTP 5-01.3

AFTTP 3-2.87
ATP 5-0.3
US Army Training and Doctrine Command
Joint Base Langley-Eustis, Virginia
US Army Combined Arms Center
Fort Leavenworth, Kansas
MCRP 5-10.1
Headquarters, USMC, Deputy Commandant, CD&I
Quantico, Virginia
NTTP 5-01.3
Navy Warfare Development Command
Norfolk, Virginia
AFTTP 3-2.87
Curtis E. LeMay Center for Doctrine
Development and Education
Maxwell Air Force Base, Alabama
7 February 2020

## Operation Assessment Multi-Service Tactics, Techniques, And Procedures For Operation Assessment

EXECUTIVE SUMMARY ........ xi CHAPTER I ASSESSMENT OVERVIEW ........ 1
1. Assessment **........ 1** 2. Operation Assessment Process **........ 3**
CHAPTER II FRAME THE ASSESSMENT ........ 9
1. Introduction ........ 9
2. Organizing an Assessment Cell **........ 9** 3. Operation Assessment within the Planning Process **........ 10** 4. Assessment Products Developed during Planning **........ 11** 5. Assessment Planning during Execution **........ 12** 6. Considerations for Planning the Assessment Process **........ 14** 7. Develop the Assessment Approach **........ 14** 8. Develop the Assessment Plan **........ 16** 9. Developing Indicators **........ 16** 10. Designing Effective Indicators **........ 17** 11. Fully Specifying Indicators **........ 17**
3. United States Navy **........ 93** 4. United States Air Force (USAF) **........ 96** 5. North Atlantic Treaty Organization (NATO) **........ 96**
REFERENCES ........ 99 GLOSSARY ........ 103

## List Of Figures

Figure 1.
Assessment Integration into JIPOE ........ 31
Figure 2.
ISAF Campaign Assessment Results Collection Template ........ 34
Figure 3.
Notional Campaign Assessment Summary Slide ........ 35
Figure 4.
Assessment Collection and Analysis ........ 37
Figure 5.
Assessment Working Group Quad Chart Example ........ 38
Figure 6.
Example Grouped Bar Chart ........ 41
Figure 7.
Example Line Chart ........ 42
Figure 8.
Example Pie Chart ........ 43
Figure 9.
Example Pie Chart vs Bar Chart ........ 43
Figure 10. Example Geospatial Chart ........ 44 Figure 11. Sample Histograms ........ 45 Figure 12. COFMS Calculator ........ 46 Figure 13. Sample Assessment Rating Definitions ........ 50 Figure 14. Spider Chart Example........ 59 Figure 15. Spider Chart Depicting an Ordinal Assessment ........ 59 Figure 16. Partner Capability in Building Assessment Communication ........ 60 Figure 17. Assessment Communication Tool ........ 61 Figure 18. Example Stoplight Chart Combined with Staff Assessment........ 62 Figure 19. Commander Decision Cycle Integrated with the Operations and Assessment
Processes ........ 64
Figure 20. Linking Outcomes to Indicators Model ........ 66 Figure 21. An Example of a Starting Point ........ 68 Figure 22. Refining the Outcome Statement ........ 69 Figure 23. Example Recording of Objectives ........ 70
Figure 24. Example Further Refinement ........ 71 Figure 25. Example Second Iteration Through the Design Loop ........ 72 Figure 26. Example Narrowing Objectives Further ........ 73 Figure 27. Example Three Iterations Through the Design Loop ........ 74 Figure 28. Completing the Design Loop and Moving to Question Two ........ 75 Figure 29. Example Question Two to Determine Information Requirements ........ 76 Figure 30. Example First Iteration Through the Assessment Loop ........ 77 Figure 31. Example IRs and Indicators ........ 78 Figure 32. Example Recording of Indicators ........ 79 Figure 33. Example Operational Approach ........ 82 Figure 34. Example Objectives and Effects for LOE 1 ........ 83 Figure 35. Example Indicators in Support of LOE 1 ........ 84 Figure 36. Example Decision Point Template ........ 86 Figure 37. Example Indicators to a Specific Objective or Effect ........ 87
Figure 38. Integrating Assessment Planning Concurrent to Operational Planning ...... 88 Figure 39. Developing Questions to Measure Effectiveness ........ 90 Figure 40. Developing Indicators from Environment and Threat Systems ........ 91 Figure 41. Example Initial Collection Plan ........ 91 Figure 42. Example Tab 1, Collection Plan ........ 96

## List Of Tables

Table 1. Operation Assessment Steps ........ 4 Table 2. Operation Assessment Steps One and Two........ 9 Table 3. Operation Assessment Steps Three and Four ........ 25 Table 4. Example Collection Roles........ 26 Table 5. Example Assessment Data Collection Plan ........ 27 Table 6. Example Data Collection Methodology ........ 30
Table 7. Generic ISAF Campaign Data Organization Method ........ 32 Table 8. Notional Assessment Standards for an Essential Task ........ 33 Table 9. Six Assessment General Questions ........ 39 Table 10. Example Table ........ 40 Table 11. Operation Assessment Steps Five and Six ........ 53 Table 12. Assessment Task Integration during Execution ........ 54 Table 13. Stoplight Chart Example (1230 Report to Congress, July 2013) ........ 57 Table 14. Data Collection Plan Template ........ 85 Table 15. Navy War College Assessment Appendix Example ........ 94 Table 16. North Atlantic Treaty Organization Annex OO Example ........ 97

## Executive Summary Operation Assessment

Multi-Service Tactics, Techniques, and Procedures (MTTP) for Operation Assessment establishes techniques and procedures for staffs to better inform commanders on progress of operations in order to identify risks and resource requirements and make operations more effective. There are three main concepts to introduce for the understanding and context of the manual: the definition of assessment to include the role of the staff and the assessment teams, what the assessment is intended to do for the command, and the many steps on how to conduct the assessment.

Assessment is a continuous process where the staff observes and evaluates the operational environment and the impact of friendly unit actions against their mission to better inform the commander. Successful staffs conduct assessments using existing staff elements, supported and coached by the assessment team. The staff member(s) assigned to conduct operational assessments leverage existing staff leads for particular lines of effort or lines of operation, as outlined in Joint Publication (JP) 5-0, Joint Planning, providing supporting evidence in coordination with and supporting the operations or plans officers in recommending actions to the commander. Although the assessment staff may run a separate working group, their recommendations support either an operational or planning decision. Successful assessment integration from the onset of planning throughout execution gives commanders a means to proactively identify and adjust to emerging opportunities and risks to mission accomplishment. Timely recognition of opportunities and risks affords commanders a distinct advantage by possibly catching the enemy off balance and rapidly ending a battle; refocusing joint force capabilities to minimize disruption; or hastening accomplishment of objectives, conditions, and end states. Conversely, missed opportunities and risks can result in protracted engagements, higher casualties, and increased potential for setbacks.
Successful planning includes integrating the concept of continuous assessment from the receipt of the mission throughout execution. Concepts in the staff integration include information required for the commander to make key decisions, the identification of resource shortfalls, and risk to the accomplishment of the mission.

Best practices in an assessment include starting with the desired outcome, forming line of effort working groups to lead the assessment, nesting higher and lower assessments, conducting a standards-based assessment to measure progress, using strategic questions to communicate data requirements, using theories of change to frame and provide credibility for friendly actions, and publishing a written assessment to clearly communicate well-thought-out results of the assessment.

This MTTP nests within the six operation assessment steps introduced in JP 5-0. This publication will further explain why each of those steps are important and some methods to accomplish them. A summary of each chapter and appendix of this publication follows:

## Chapter I Assessment Overview

Chapter I provides commanders and their staffs an overview of the assessment process and provides the link between JP 5-0 and MTTP discussed in this publication.

## Chapter Ii Frame The Assessment

Chapter II discusses the first and second steps of the assessment process. These two steps develop an operation assessment approach and the assessment plan, and occur during the planning phase of operations.

## Chapter Iii Collect And Analyze

Chapter III discusses the third and fourth steps of the assessment process. These steps collect and analyze information and intelligence, and occur during operations.

## Chapter Iv Communicate The Assessment And Adapt The Plan

Chapter IV describes processes to accomplish the fifth step of the assessment process:
communicate the assessment. This step is most commonly referred to as the assessment, as it entails communicating, through verbal and visual means, the staff's assessment to the commander for decision. Chapter IV briefly discusses step six of the assessment process, which is adapting plans or operations.

## Appendix A Connecting Outcomes To Indicators Model

Appendix A provides a detailed discussion of how to create the correct questions or statements to efficiently collect indicators that will drive assessments. This model can help a new assessor or assessment cell with an assessment plan to support a commander's decision points.

## Appendix B Assessment Plan Examples

Appendix B shows an example assessment plan used by II Marine Expeditionary Force in a past operation. This example will assist by showing a working example of an assessment plan and how it was created. The Naval War College provided several examples for assessors to organize their thoughts, develop indicators, and display a data collection plan.

## Appendix C Example Annexes And Appendices

Appendix C provides the doctrinal operation order assessment formats from Service doctrine and provides the source for each.

## Program Participants

The following commanders and agencies participated in creating this publication:

## Joint

United States (US) Joint Forces Command, J7, Suffolk, Virginia

## Army

US Army Combined Arms Center, Fort Leavenworth, Kansas US Army Training and Doctrine Command, Joint Base Langley-Eustis, Virginia US Army Training and Doctrine Command Analysis Center, Monterey, California US Army Center for Army Analysis, Fort Belvoir, Virginia

## Marine Corps

Training and Education Command, Quantico, Virginia Marine Corps Capabilities Development Directorate, Quantico, Virginia II Marine Expeditionary Force, G3, Camp Lejeune, North Carolina Marine Air Ground Task Force Staff Training Program, Quantico, Virginia Marine Corps Tactics and Operations Group, Twentynine Palms, California

## Navy

Navy Warfare Development Command, Norfolk, Virginia Naval War College, Newport, Rhode Island

## Air Force

Curtis E. LeMay Center for Doctrine Development and Education, Maxwell Air Force
Base, Alabama
This page intentionally left blank.

## Chapter I Assessment Overview 1. Assessment

An assessment is the determination of progress toward accomplishing a task, creating a condition, or achieving an objective. Source: Joint Publication (JP) 3-0, Joint Operations.

The purpose of the operation and desired end state define the commander's intent. An assessment must link and reflect the status of progress of accomplishing tasks, creating effects, and achieving objectives toward accomplishing the commander's end state as defined in the intent.

An assessment is fundamental to a military organization's ability to adapt.

Accordingly, observed and reported actions are of little value unless they can serve as a basis for future decisions and actions. Assessments enable adaptations, providing guidance and direction to make our forces more effective. An assessment is not complete, and all effort spent in it is wasted, if it is not used to change course during operations.

An assessment, along with planning, preparing, and execution, comprise the operations process. As with any cycle, it has no beginning or ending once the process has commenced.

An assessment seeks to answer six general questions:
(1) How has the operational environment (OE) changed?

(2) How much discernable progress exists in accomplishing our operational objectives? (3) What do we think caused progress and/or lack of progress in achieving our objectives? (4) Do the changes in the OE cause a change to operations and/or plans?
(5) What are the resource gaps to accomplishing our objectives and what are the
risks associated with the current resourcing? (6) How does this assessment nest with higher headquarters (HHQ)
assessments and incorporate lower-level assessments?
Effective operation assessment:
(1) Focuses on the commander's objectives, end state, and related information requirements (IRs). (2) Considers specific indicators in context with other indicators and professional military judgment.

(3) Incorporates both quantitative and qualitative indicators. (4) Considers subordinate units' capabilities before assigning assessmentrelated requirements.
(5) Provides analysis that identifies particular trends and changes in the OE, and their impact on operations. (6) Incorporates the insights and expertise of various staff sections and stakeholders.

(7) Leverages objectives, desired effects, and tasks that have effectiveness and performance indicators that forces can observe, measure, refine, and adapt throughout planning and execution.

(8) Conveys the assessment to the commander in a clear and concise manner. (9) Provides analysis and synthesis supported by professional military judgment
achieved in part through scrutiny of relevant evidence and logic. (10) Provides context; i.e., explaining why evidence, arguments, and
recommendations matter to the end state. (11) Measures progress against objectives.
(12) Incorporates best practices in assessments, including standards-based assessments, theory of change, and written assessments.

## Assessment Outcomes Will:

(1) Depict progress toward accomplishing the commander's intent. (2) Deepen understanding of the OE. (3) Inform the commander's decision making.
(4) Produce actionable recommendations. (5) Make operations more effective.
Tenets of an Operation Assessment. The following tenets should guide the commander and the staff throughout the assessment process:

(1) Subordinate Commander Involvement. Assessment teams must include a method for including subordinate assessment results into their assessment. The
best practice is to include gaps and risks from the subordinate commander.
(2) Integration. Operation assessments are the responsibility of commanders, planners, and operators at every level and not the sole work of an individual advisor, committee, or assessment entity.

(3) Integration into the Planning Process and Battle Rhythm. Operation assessments are an integral element of the commander's decision cycle. Assessment planning should coincide with the operation planning efforts. The resulting assessment plan should become an integral element of the command's battle rhythm. (4) Integration of External Sources of Information. To get a more complete understanding of the OE, it is important to receive and share relevant information with the host nation, interagency, multinational, private sector, and nongovernmental partners.

(5) Credibility and Transparency. The assessment report should list methods, collection limitations, and any assumptions that link evidence to conclusions.

(6) Continuous Operation Assessment. Like any process, once underway, an assessment is continuous, both informing and being informed by planning and execution.

## Staff Role In An Assessment.

(1) Effectively supporting the commander requires staff assessment activities to conform to the commander's decision-making style. When it comes to thinking through how best to support commander decision making, several aspects are worth considering.

(a) How does the commander prefer to receive and process information?
(b) How does the commander prefer to make decisions? (c) What role does the commander want to play in the assessment?
(2) Commanders can form assessment cells at all levels of command. However, since an assessment is an inherent staff responsibility within respective functional areas, the commander must determine how the assessment cell's focus will augment the staff's focus.

(a) An assessment benefits greatly from the professional military judgment of staff officers within their area of expertise. A broad range of skills adds balance to the assessment activity and products.

(b) Primary staff officers conduct assessments as part of their normal responsibilities. They can also form and chair standalone assessment groups, joint planning teams, and operational planning teams, as necessary. Staff principals must provide resources to subject matter experts (SMEs) for required subworking groups to ensure continuity and unity of effort.

(3) Effective staffs leverage existing battle rhythm venues to help manage information in support of an assessment in order to reduce the burden to personnel and subordinate units.

## 2. Operation Assessment Process

There is no single way to conduct an assessment. Every mission and OE has its own set of challenges, and every commander assimilates information differently, making every assessment plan unique. The following steps in table 1 (see page 4)
can help guide the development of an assessment plan.

Operations
Step
Input
Personnel Involved
Staff Activity
Output
Process
Activity
Planning
- JIPOE
- Staff estimates - Operational
approach
development
Develop Assessment
Approach
- Commander - Planners
- Primary staff - Special staff - AWG personnel
- JPP
- Joint targeting - AWG
- Develop a
framework
- Select measures
(MOE and MOP)
Develop
Assessment Plan
Planning
- Operations planners
- Intelligence planners
- AWG personnel
- Identify indicators - Develop a
feedback
mechanism
Collect Information
and Intelligence
Execution
- Intelligence analysts
- Current operations - AWG personnel - Assessment cell (if
established)
- Joint targeting - JIPOE - Staff estimates - IR management - ISR planning and
optimization
Execution
- Assessment work
group
Analyze and Synthesize the
Feedback
- Staff estimates
- Primary staff - Special staff - AWG personnel - Assessment cell (if
established)
- Commander - Subordinate
commanders
(periodically)
Execution
Communicate the Assessment and
Recommendations
- Provide a timely
recommendation
to the appropriate decision maker
- Primary staff
- Special staff - AWG personnel - Assessment cell (if
established)
Adapt Plans
Execution Planning
- Joint targeting - JPP
- Commander - Planners
- Primary staff - Special staff - AWG personnel - Assessment cell (if
established)
Legend: AWGassessment working group

IRinformation requirement ISRintelligence, surveillance, and reconnaissance JIPOEjoint intelligence preparation of the operational environment

- Information,
intelligence, and
collection plans
- Clearly defined
end states,
objectives, and
tasks
- Operational
approach
- Assessment plan
- JIPOE - Desired end state - Feedback
mechanism parameters
- Multisource
- Estimates of OE
conditions, enemy disposition, and friendly disposition
intelligence reporting, and joint force resource and disposition information
- Operational
reports
- Intelligence
assessments
- Estimate of joint
force effects on OE
(draft assessment report)
- Staff assessments
- Analysis methods
- Assessment report,
decisions, and
recommendations
to higher headquarters
- Estimate of joint
force effects on
OE (draft
assessment report)
- Changes to the
operation and assessment plan
- Commander's
guidance and
feedback
OEoperational environment JPPjoint planning process MOEmeasure of effectiveness MOPmeasure of performance

Step 1Develop the Operation Assessment Approach.

(1) This step focuses on the linkages with the planners to ensure the assessment approach develops as the plan and operational design also develops. The staff begins to develop the operation assessment approach by
identifying and integrating the appropriate assessment plan framework and structure needed to assess execution effectiveness. If an HHQ assessment plan exists, assessment planners should align applicable elements of that assessment
plan to the plan they are developing. The assessment approach becomes the
framework for the assessment plan and will continue to mature through plan development. The assessment approach should identify the information and intelligence needed to assess progress and inform decision making. (2) Identifying objectives and the desired end state and associated conditions is
critical to determining progress in any operation. Poorly defined objectives or end states typically result in ineffective planning, as well as increase the risks of wasting resources and opportunities to successfully accomplish the mission. The staff should identify clear objectives and tasks having effectiveness and performance criteria that forces can observe, measure, and refine throughout planning and execution. In turn, analysis and synthesis of anticipated and completed tasks should generate assessment recommendations to communicate to the commander.
Step 2Develop the Operation Assessment Plan. This step overlaps step 1
during the identification of the objectives and effects. The assessment plan focuses appropriate monitoring and collection of necessary information and intelligence to inform decision making throughout execution. The assessment plan should link objectives, desired effects, and tasks to observable key indicators. The assessment plan can be developed using the operational approach as a baseline to identify lines of effort (LOEs) or lines of operation (LOOs) that link directly to objectives and the desired end state. The assessment plan should include required information oversight responsibilities to gather, process and exploit, analyze and integrate, disseminate, classify, and archive the required information. Developing the assessment plan is a whole-of-staff effort and should include other key stakeholders to better shape the effort. The assessment plan should identify staff or subordinate organizations to monitor, collect, analyze information, and develop recommendations and assessment products as required. Appendix A provides a model for planners to use to develop indicators based on the desired end state.

Step 3Collect Information and Intelligence. Organizations collect relevant information throughout planning and execution. They refine and adapt information collection requirements about the OE and anticipated and completed actions. Staffs and subordinate commands provide information during execution through applicable battle rhythm events. Intelligence staffs continually provide updates about the OE and the impact in support of the collective staff assessment effort.

Step 4Analyze Information and Intelligence.
(1) Analysis seeks to identify positive or negative movement toward creating desired effects, achieving objectives, or attaining end states. Analysis seeks to identify trends and changes that can significantly impact the OE and the operation. Based on this analysis, the staff estimates the effects of force employment and resource allocation, determines whether forces have achieved their objectives, or have realized that a decision point has emerged. Using these determinations, the staff may identify additional risks or opportunities.
(2) Recommendations generated by staff analyses regarding achievement of the objective or attainment of the desired end state, force employment, resource allocation, validity of planning assumptions, and decision points should enable the staff to develop recommendations for consideration. Recommendations can include:

(a) Update, change, add, or remove critical assumptions.
(b) Transition between phases, stages, parts, steps (as appropriate). (c) Execute branches or sequels. (d) Change resource allocation. (e) Adjust operations. (f) Adjust orders, objectives, and end states. (g) Adjust priorities. (h) Change priorities of effort. (i) Change command relationships. (j) Change task organizations. (k) Adjust decision points. (l) Refine or adapt the assessment plan.
Step 5Communicate Feedback and Recommendations. Assessment products contain recommendations for the commander based upon the commander's guidance. Assessment products inform the commander about current and possible conditions within the OE, evaluate the ability of the force to impact the OE, evaluate progress toward objectives and end states, provide accountability to higher authority, and communicate progress to external stakeholders. Regardless of quality and effort, the assessment process is useless if the communication of its results is deficient or inconsistent with the commander's personal style of assimilating information and making decisions. Additionally, there may be a requirement to provide input to HHQ assessments in which the requirements and feedback could be within a different construct.

Step 6Adapt Plans or Operations. Commanders direct changes or provide additional guidance that dictate operations updates or modifications to drive progress to objectives and end states. Staffs capture the commander's decisions and guidance in order to ensure forces take necessary actions. As the operation and OE evolves, the assessment plan needs to evolve as well.

## Assessment Best Practices

Effective assessment practices clearly articulate progress, gaps, and the risk associated in accomplishing the unit's mission. Gap assessment, strategic questions, standards-based assessments, written assessments, and risk assessments are best practices that provide the tools required to assist operational assessments.

Gap Assessment. One outcome of an assessment process is to determine progress against a mission. When an objective will not be accomplished by the target date, it raises the question of what to do next. A structured method to align assessments to answer this question is gap assessment, which defines the gaps in the critical path to obtain a given objective along a timeline. Gaps fall into the categories of capacity (insufficient forces allocated or assigned to the command, lack of authorities and/or permissions), capability, or shortcomings in the willingness or capability of partner nations. Identifying these gaps and closing them provide the staff with a method to take action leading to the accomplishment of their objectives.

Strategic Questions. In determining progress and gaps for a given line of effort
(LOE) or objective, several common questions arise. Recording these questions and reviewing them on a periodic basis is a best practice in many assessment programs. It allows those responsible for the assessment a method to record the assumptions and the logical lines followed by working groups, in detail, and to determine why they believe they are progressing or retrogressing.

Standards-based Assessment. This method provides the most accurate and successful summation of progress through operational and strategic commands. It is covered in detail in chapter III.

Written Assessment. Possessing a written document is important for many reasons. One such reason is the level of thought, staff coordination, and detail required to articulate an assessment in words and sentences is far greater than what is required to fill out a chart template. Some leaders and analysts recommend exclusive use of written assessments.

Risk Assessment. Chairman of the Joint Chiefs of Staff manual 3105.01, Joint Risk Analysis, provides definitions of military and strategic levels of risk. An example of a risk assessment begins with a statement of the objective or end state, describes the level of progress determined from the standards-based assessment, evaluates the risk of meeting objectives, and identifies the capability and capacity gaps.

Other Best Practices. Other best practices exist in related literature, such as theories of change, which ensure objectives and measures result from a logical process derived from causal assumptions. Additional best practices include using objective development criteria, such as the acronym SMART (specific, measurable, achievable, relevant, and time bound) or the similar initialism RMRR (relevant, measurable, responsive, and resourced). Best practices related to staff organization and functions include assigning senior leaders as LOE leads and gaining championship by the commander.

This page intentionally left blank.

## Chapter Ii Frame The Assessment 1. Introduction

This chapter provides details of the relationship of the assessment cell or function to the planning process and the rest of the staff. JP 5-0 frames the task of planning the assessment as two steps: develop the assessment approach and develop the assessment plan. Table 2 shows these first two planning steps as part of the six-step process.

Operations
Step
Input
Personnel Involved
Staff Activity
Output
Process
Activity
Planning
- JIPOE - Staff estimates - Operational
approach
development
- Information,
intelligence, and
collection plans
Develop Assessment
Approach
- Clearly defined
end-states
objectives and
tasks
- Commander - Planners
- Primary staff - Special staff - AWG personnel
- JPP
- Joint targeting - AWG
- Develop a
framework
- Operational
approach
- Select measures
(MOE and MOP)
- Assessment plan
Develop
Assessment Plan
Planning
- Operations planners
- Intelligence planners
- AWG personnel
- JIPOE - Desired end state - Feedback
mechanism parameters
- Identify indicators - Develop a
feedback
mechanism
MOEmeasure of effectiveness MOPmeasure of performance

Legend: AWGassessment working group JIPOEjoint intelligence preparation of the operational environment JPPjoint planning process Throughout planning, the assessment team must engage with the commander and staff to ensure the assessment plan supports the commander's understanding of the OE. The most successful staffs are those that routinely integrate and implement assessment activity at the onset of the planning process.

LOE leads, as outlined in JP 5-0, should guide the development and assessment of LOE intermediate objectives, critical conditions, indicators, tasks, and associated metrics and recommendations through the LOE working groups. The LOE assessment produces updated findings, insights, and recommendations.

## 2. Organizing An Assessment Cell

Each Service organizes differently to conduct an operation assessment. Some Service organizations have dedicated assessment personnel assigned to a staff, while others establish assessments upon initiation of a planning effort. Common elements include:

(1) Assessment Cell. The assessment cell is the organization that is responsible
for developing the assessment plan and managing the assessment process in execution. The assessment cell may be incorporated into a staff directorate, be a
directorate unto itself, or a special staff section working directly for the commander, deputy commander, or chief of staff. There is no set size to an assessment cell as long as it can support operational planning teams (OPTs), working groups, and key battle rhythm events.

(2) Assessment Working Group (AWG). The AWG is a cross-functional team that vets assessment products and recommendations. The AWG is composed from representatives from across the general and special staff sections, as well as other SMEs depending on mission set. During planning the AWG meets periodically to review, revise, and provide input to that assessment plan as it is being developed and finalized.

## 3. Operation Assessment Within The Planning Process

The focus of this section is to use the joint planning process (JPP) to identify assessment actions executed during each of the planning steps of the JPP. These actions are included in other Service planning processes.
During planning initiation, the assessment cell:

(1) Reviews HHQ order to consider:
(a) HHQ operational approach.
(b) HHQ assessment annex and requirements. (c) HHQ joint intelligence preparation of the operational environment (JIPOE).
(2) Determines the identification of potential data sources, including academic institutions and civilian SMEs. (3) Reviews any current or historically relevant assessment products, either classified or open-source, produced by civilian and military organizations.

(4) If required, conducts operation assessment training with the assessment cell,
AWG, and staff. (5) Actively participates in operational design and approach discussions.
(6) Develops initial template assessment plan and data collection plan based on operational design discussion with the AWG.

(7) Gains commander's preferences for communicating the assessment.
During mission or task analysis, the assessment actions will include:

(1) Review and update JIPOE.
(2) Support the development of risk assessment, success criteria, and initial
commander's critical information requirement development.
(3) Conduct AWGs to continue developing an assessment plan and data collection plan.
During course of action (COA) development, the assessment actions will include:

(1) The assessment cell providing support to each COA development team.
(2) Reviewing JIPOE and if necessary providing updates to the rest of the staff. (3) Conducting AWGs to continue developing an assessment plan and data
collection plan. (4) Providing staff estimate for each COA, if required.
During COA analysis and war gaming, the assessment actions will include:

(1) The assessment cell providing inputs to COA evaluations.
(2) Reviewing JIPOE and if necessary providing updates to the rest of the staff.
(3) Conducting AWGs to continue developing an assessment plan and data
collection plan.
During COA comparison, the assessment actions will include:

(1) The assessment cell providing inputs to COA evaluations.
(2) Reviewing JIPOE and if necessary providing updates to the rest of the staff. (3) Conducting AWGs to continue developing an assessment plan and data
collection plan.
During COA approval, the assessment actions will include:

(1) The assessment cell providing inputs to COA evaluations.
(2) Reviewing JIPOE and if necessary providing updates to the rest of the staff. (3) Providing inputs and analysis to finalize the operational approach.
(4) Conducting AWGs to continue to finalize an assessment plan and data
collection plan. Assessment representatives are actively involved with the collection management working group to ascertain what is going to be collected. What is not collected is recognized as additional risk that is briefed to the
commander.
During plan and order development, the assessment actions will include:

(1) Developing an appropriate assessment annex or appendix, if different from
the assessment plan or data collection plan.
(2) Finalizing the data collection plan. (3) Engaging with the information management section to establish assessment. (4) Prior to execution, continuing to maintain situational awareness and adjusting the assessment.

## 4. Assessment Products Developed During Planning

Assessment Plan. Navy Warfare Publication (NWP) 5-01, *Navy Planning*, and Army Field Manual 6-0, *Commander and Staff Organization and Operations*, provide procedures for developing an assessment plan. This document adapts those procedures for multi-Service use. Once the assessment plan is complete, it guides application of the assessment activity to monitor, evaluate, recommend, and direct continuously throughout the operations process. It is important to recognize, as operational plans are iteratively adjusted and improved, the assessment plan must undergo revisions to ensure alignment with the end state.

Data Collection Plan. To support the assessment plan, the staff may leverage a data collection plan that incorporates assessment requirements identifying where the information is found for each indicator and whom the lead is to collate the information for the assessment. Some data collection plans just identify where the information is found, as the assessment has the capacity to conduct an initial assessment, and then provide it to the assessment or LOE working group to validate or revise. If large amounts of data are collected from polling subordinate units on a consistent basis, it is imperative to engage with the information management section to set up an appropriate data collection site. Expect the data collection plan to evolve during execution as the OE evolves and the worth of indicators changes with their ability to provide the relevant data. Data collection plans are more critical when large amounts of quantitative data are required from other agencies or subordinate units, and less critical when more qualitative or standards-based assessments require less quantitative data, relying more on narrative reports.

Operation Order (OPORD) Annex or Appendix. North Atlantic Treaty Organization (NATO), the United States (US) Army, and the US Navy have identified an assessments annex or appendix for an OPORD. The annex or appendix is based on the assessment plan and data collection plan, so it is developed throughout the planning process. Example OPORD annexes are found in appendix C. Not all assessment processes require an annex or appendix, as assessment cycles often run at a different schedule than the OPORD release timeline.
Commander's Assessment Brief. Some organizations have developed a standard commanders' brief template, which is revalidated prior to a planning effort. As each commander is different, the previous version may no longer be valid, requiring prior template validation. Additionally, as the plan is developed and finalized, the assessment cell or commander may see the need to make changes. Once the plan is approved, the brief template is revalidated.

## 5. Assessment Planning During Execution

As previously noted, as the operation is executed, the original assessment plan will go through revisions based on the OE and the assessment associated with it.

What the assessment cell must be prepared to support during execution of an operation is the requirement to evolve the assessment plan and data collection plan. This effort is associated with requirements listed below:

(1) Development of a branch or sequel. (2) Major revisions to the operational approach.
(3) Additional assessment requirements from HHQ.
The tactical assessment process is not a separate process disjointed from the operation planning process. Assessments must be nested within, and developed concurrently with, the tactical plan. The OPT should form the core of the assessment working group during the operations execution.

(1) Nest Operation Assessment Approach with the Plan. The tactical assessment team develops the assessment framework during the military decision-making process with the support of the staff. Developing situational awareness of the OE and understanding the operation plan is paramount to building the assessment plan. Mission analysis sets the condition for the operation assessment. The operational framework of the plan, derived during mission analysis, serves as the structure of the assessment framework and how the commander views the area of operations. With staff officers representing all war-fighting functions participating in the OPT, mission analysis provides an opportunity for the assessment team to introduce the assessment methodology to the staff and begin generating measures of merit and indicators for the assessment framework. The assessment team will assign offices of primary responsibility (OPRs) for every measure of merit and indicator. This step also includes analyzing higher headquarters assessment plans and taking strides to nest the assessment plan with the next higher headquarters processes. (2)
Develop Operation Assessment Plan. The complete assessment plan synchronizes the assessment framework with measures of merit and indicators corresponding to mission end states, tactical objectives, desired effects, the data collection plan for the assessment framework, AWG framework and agenda, and the overall campaign assessment products tailored to the senior commander. The data collection plan links the tactical end states to the measures of merit and indicators, and assigns OPRs for each measure of merit and indicator. The assessment working group framework allows the staff to provide evidence-based quantitative and qualitative analysis with respect to risk. The comprehensive campaign assessment product provides the senior commander a concise snapshot of how the operation is progressing based on a projection of the plan for a predetermined time in the future. (3) Assessment Execution. The staff and subordinate units use the data collection plan to provide the necessary inputs for the assessment of each end state, effect, objective, LOO, or LOE. Additionally, the staff provides evidencebased qualitative analysis through the AWG framework.

(4) Analyze Information and Intelligence. The AWG provides clear and candid analysis of identified measures of merit and indicators, and their effect on the operations execution and outcome. The AWG will develop possible solutions and refinements to the plan based upon the operation assessment. The assessed time horizon is dependent on the tempo of the executed operation. It is important to incorporate both plans and future operations planning horizons.

(5) Communicate Feedback and Recommendation. It is imperative the AWG
meets before any meeting involving the targeting cycle. The targeting decision board is the preeminent meeting for the senior commander. This meeting allocates echelons above brigade resources, shaping the deep area enabling future maneuver for the subordinate units. (6) Adapt Plans. Changes to the plan require support from the planners and the future operations cell. If the assessment necessitates adjustments to the plan, the plans update board is the forum where the senior commander will approve branches or sequels to the current plan. The chief of assessments will brief the campaign assessments at every plans update board along with changes to the plan as required. If the assessment identifies significant changes to the current operation, the planners will brief a proposed update, either a branch or sequel plan, to the current operation addressing the issues from the assessment and providing a solution to bring the campaign back on track. The chief of assessments requires a forum to fully brief the analysis behind the assessment, and the planners to present a solution to put the campaign on track. This assessment process allows the staff to identify future potential issues, providing time for the planners to produce and publish valid changes to the plan.

## 6. Considerations For Planning The Assessment Process

This section explains conceptually how assessors structure the process to gather, store, and analyze information to better understand the OE, and how to design products that communicate these findings and associated recommendations for more effective operations to senior decision makers. These activities take place separate from, but in parallel with, the planning process, while assessors participate in staff-wide activities.

The task of planning the assessment is defined in two steps: develop the assessment approach and develop the assessment plan. The former makes inputs to initial decisions about the organization of the assessment effort, the relationship of assessors to the rest of the staff, the integration of AWGs into the battle rhythm, and the sort of information that is gathered and analyzed to improve the staff's understanding of the OE. The latter refines, formalizes, and communicates these decisions throughout the organization through written standard operating procedures
(SOPs), an assessment annex, collection matrices, or other intermediate documents.

## 7. Develop The Assessment Approach

The initial framing of the assessment problem will establish an initial organization of the assessment cell, its place in the staff, its contribution to the planning process, and its participation in, or leadership of, battle rhythm events. There are many potential ways to organize the effort, and initial decisions can be refined as the staff learns, the operation evolves, and the OE changes.

Note: A theory of change is a simple statement that defines the expected outcome of friendly actions as an if-then statement. Common examples include: If we clear, hold, and build, then the local population will not support the insurgency and will instead support the government. The theory links the actions of the friendly unit to progress toward the desired outcome or end state. Analysts may use a theory of change to assist in identifying measures of performance and effectiveness. A graphical logic or concept map can also aid in describing the theory of change to the commander and staff. There are numerous sources that provide lists of theories of change for use by analysts. Some of the sources most applicable for military use include the RAND Corporation, United States Agency for International Development, and universities. Analysts have found the following sources particularly helpful: Christopher Paul et al., Assessing and Evaluating Department of Defense Efforts to Inform, Influence, and Persuade: Desk Reference (Santa Monica, CA: RAND Corporation, 2015), 3233, 88102. Susan Allen Nan and Mary Mulvihill, Theories of Change and Indicator Development in Conflict Management and Mitigation (Washington, DC: United States Agency for International Development, June 2010), Appendix A. An example of how to make a graphical theory of change model is detailed in Kilcullen's books: David Kilcullen, Counterinsurgency (New York: Oxford University Press, 2010), 5253;
David Kilcullen, The Accidental Guerrilla: Fighting Small Wars in the Midst of a Big One (New York: Oxford University Press, 2011), 35.

Additional theories of change are listed at: http://start.foxtrotdev.com/
An operation assessment begins when planning begins. This is critical because assessors provide a quality check on the planning or the design process. Assessors help planners be specific and bounded when writing end states, objectives, effects, or other ways of specifying the desired outcome of operations. While assessment considerations should not drive operations, the inability to assess an unclear outcome statement is an excellent indicator that subordinate headquarters will have difficulty planning and executing operations to pursue it. In addition, an assessment is an activity; like any other activity it requires a plan, and the earlier assessors begin planning the assessment process, the more effective it is likely to be.

The assessment plan should reflect the logic of the operational plan, as the two develop simultaneously. For example, if the operational plan specifies LOOs or LOEs, then an assessment plan based on these LOOs or LOEs will likely provide the best understanding of the OE, provided those were a reasonable way to begin understanding the environment based upon the JIPOE. The assessment process may improve the understanding of the OE and this may recommend a change to the plan that organizes operations. In every case, the assessment plan should evolve as the operational plan changes.

Assessment methods and techniques also add value to other staff processes by approaching information gathering and analysis in a structured way. All staff sections gather and analyze information relevant to their functional areas, but trained assessors can help structure collection efforts and information storage to improve analysis and speed up the publication of key products. Added structure may also reduce bias and provide a better empirical basis to staff estimates.

There are multiple implementation methods for the staff organization for assessments. Forming a multifunctional assessments working group led by the assessment team is a more direct approach for smaller staffs. Larger staffs that require more consensus may use LOE working groups, led by operations or plans staffs, so that recommendations are made by the staff lead rather than the assessments team.

Decisions on these issues can be found in an SOP that establishes steady-state functioning of the assessment cell, a blueprint of the assessment process, and a schedule for the publication of assessment products. An assessment annex will communicate information required for people outside of the staff to facilitate assessment functions, particularly in collecting and providing information, or an annex will specify how to adapt from the SOP for particular short-term operations.

## 8. Develop The Assessment Plan

The development of the assessment plan adds detail to the assessment approach that is required to coordinate the efforts of all participants in the assessment process, including senior decision makers, assessors, staff, and even the junior Service members who will report the raw empirical information required for the assessment process as this information appears in the OE.

## 9. Developing Indicators

Developing indicators that reflect the changes in the OE over time, that are pertinent to the operation, is essential for assessments to be effective. In essence, assessors are asking an increasingly specific series of questions about the OE and about the changes in the OE that joint forces are attempting to effect. The questions begin with the general form of: What questions do we need to answer to know we are accomplishing a specific effect or task? The subsequent set of increasingly specific questions forms the logical links between the desired end state and the empirical evidence of change over time that denotes success. The most specific of these questions is answerable with empirical observation. The answer could be as simple as a qualitative yes or no from a trusted observer, answering, for example: Have we seized Hill 802? Or as simple as a quantitative three, answering, for example: How many mortar attacks have there been on the support area this week?

Possible answers may be more complex, but the idea is to frame questions that identify the facts assessors need to gather through direct observation or the judgment of a qualified and trusted observer. Analysis which considers multiple pieces of information simultaneously allows the staff to come to more reasonable conclusions about the change occurring in the battlespace and the causes of it.

An indicator is defined as: a specific piece of information that infers the condition, state, or existence of something, and provides a reliable means to ascertain performance or effectiveness.

Indicators are only important if they answer the correct questions. Therefore, assessors should not fixate on collecting a number of indicators bearing on the operational problem. Instead, they should focus on asking the correct questions to determine if the organization is achieving its desired effects and objectives. These questions provide the logical links between stated objectives and effects and the indicators used to measure their attainment. Finally, collection assets and staff capacity are often limited. Therefore, the staff should not attempt to know all that can be known, but should focus information collection that answers the most important questions. The staff formalizes these questions as IRs.

## 10. Designing Effective Indicators

Assessment indicators generally come in two varieties. Measures of performance
(MOPs) are indicators used to assess friendly actions tied to measuring task accomplishment. MOPs commonly reside in task-execution matrices, and answer general questions such as: Are we doing things correctly? Was the task completed to standard? Measures of effectiveness (MOEs) are indicators used to help measure a current system state, with change indicated by comparing multiple observations over time to gauge the achievement of objectives and attainment of end states. MOEs help answer the question: Are we doing the correct things to create the effects or change in the OE that we desire?

Note: Before the publication of the July 2017 edition of JP 5-0, the terms measure of effectiveness (MOE), measure of performance (MOP), and indicator were defined differently. The assessment community thought the redefinition of these terms simplified the regime of measures and indicators. The terms, MOE and MOP, are useful in that they highlight the difference between performance (performing a task properly) and effectiveness (having the effect on the operational environment that the commander desires). In short, performing a task is insufficient to conclude operations or have been effective because forces may have performed an inappropriate task, or have performed an appropriate task poorly, or the adversaries may have countered friendly operations.

Once this distinction is clearly understood, the terms are less useful, and the label MOE or MOPput on an indicator is less important than the fact that an indicator answers an important question.

Indicators must be relevant, observable or collectable, responsive, and resourced. That is, they should answer the important questions; be collectable at reasonable cost in time, money, or manpower; change perceptively in a time frame relevant to the operation; and have resources made available to collect them.

## 11. Fully Specifying Indicators

Selected indicators must be sufficiently well-specified such that they answer the IRs they are designed to answer, and such that any one indicator can be collected consistently by multiple observers, at different places, or over time. Each will need a definition, a plan for collecting the data (Who, What, When, Why, and How), and be sensitive to change within a relevant time frame. If it is calculated, it must have a formula; and it may have a target or threshold of success or a desired rate of change. This information is formalized in the data collection plan.

Questions (which are also IRs) and answers (which are also indicators) need not have a one-to-one correspondence. It may be that a single indicator answers a single IR, but it is also possible that a single indicator answers several questions, that several indicators answer a single question, or that several related indicators answer a set of related questions. The nature of the OE and the logic of the operational plan will dictate these specifics.

Some indicators are quantitative, that is, a number can express meaningful information about a quantity or amount. Some are qualitative, which means they reflect information about quality or kind. These require a description, which could be a single word, sentence, or paragraph, and which could be simply yes or no. This description could be strictly empirical, such as: The bridge is still standing. Or require some judgment, such as: I rate Alpha Company as trained on this collective task.

Sometimes a qualitative judgment may be summarized as a number on an ordinal scale, such as: The Romanian judge gave the Dutch gymnast a 9.8.; or: The local guide rates this restaurant as 3 stars.
Note: In these cases, assessors must be very careful with applying mathematical techniques to this sort of data. An average of equally weighted ordinal scores, like Olympic gymnastics scoring, is probably acceptable and provides leaders useful information; frequency distributions are also acceptable; but other mathematical techniques, such as adding, subtracting, applying ratios, and especially applying varying weights, to ordinal data, distorts the data and are not valid.

Take caution that some assessment schemes rely on the weighting and aggregation of a number of ordinal scores that represent information about various elements of the organization's performance or effectiveness. These schemes produce some sort of numeric index which people have treated as if this number had meaning; it does not. In general, if an assessor cannot tell the commander what unit of measure a number represents (e.g., attacks, casualties, sorties, etc.), then the number is questionable.

This criticism does not apply to standards-based assessment products which employ a simple ordinal scale, often one to five, as a shorthand to communicate the status of a line of effort. Because the product merely communicates a status, no mathematical operations are performed on the scores, and the appropriate analyses are performed behind the product. This format is commonly used at HHQ.

"Assessors do not understand that the E-4s through E-6s who will collect and report many assessment indicators do not have the assessors' organization-wide perspective. Therefore, assessors seeking data must ask specific questions and be very clear what information they want in reporting or patrol debrief formats."
Dr. Adam Shilling, Center for Army Analysis March 2018

## 12. Considerations For Planning For Collection

There are a number of ways of gathering assessment information. Many potential indicators exist within the headquarters, and are contained in operational, intelligence, sustainment, or civil-military reporting. Other data sources include HHQ, interagency partners, allies, nongovernmental organizations, international organizations, and media reports. Additional important information may require special efforts to collect. Surveys and focus groups can provide information on public opinion, or assessors may draft report formats for operational forces to complete routinely or as needed. Each survey question or line from a report format is stored as a field in a database.
If Service members are reporting on events or situations encountered in the field, the questions embedded in the report format can anticipate assessors answering questions that the staff has not yet identified as important. In addition to the currently tracked indicator, the report may contain the answers to many more assessment questions. For example, a significant activities (SIGACTs) report from Iraq or Afghanistan consisted of a number of fields asking questions to get a unit to describe an event that fit the definition of a SIGACT. When stored in an operational database, this report, and others like it, allowed assessors and other staff analysts to count the number of SIGACTS, to separate attacks from other types of SIGACTs, to separate indirect fire from direct fire from improvised explosive device (IED) attacks, to compare numbers and types of attacks from different time periods, to study geographic and temporal patterns, to analyze which sort of attacks produced the most casualties, and to conduct a wide range of analyses, which could not possibly have been completely anticipated at the outset of operations.

Note: The assessment team ensures standard reports such as: situation reports, intelligence summaries, personnel stats, logistical stats, etc., support the assessment plan, and insert specific reporting requirements if required.

The commander may task collection assets to answer IRs identified through the assessment process the same as for IRs identified by the intelligence or targeting processes; the staff uses the existing intelligence, surveillance, and reconnaissance
(ISR)-collection matrix or Service-specific matrix. The frequency of observation required to satisfy IRs (e.g., hourly, daily, weekly, monthly) or the requirement for repeat versus one-time observation may govern whether one or more collectiontasking matrices are best, and assessment IRs may compete for collection assets with other IRs.

Assessors and other staff members cannot ignore important information about events in the OE because these things are not in the collection plan. For example, an event such as the assassination of a key local national ally can change an operation in unexpected ways. When unexpected events occur, assessors should evaluate these for their importance to the mission, and may recommend changes to the operational plan to mitigate risk or exploit opportunity. They may also recommend changes to the assessment plan to monitor events which are newly recognized as important.

## 13. Considerations For Organizing Information For Analysis

As the plan matures, the assessment cell in conjunction with the AWG develops an assessment plan and a supporting data collection plan. The plan compartmentalizes the OE by end state, phase, and geography (i.e., purpose, time, and space), or by other means appropriate to the situation, as determined by commander's guidance. Assessors review what is recorded, consider the confidence of the data received, and then discern evidence-based conclusions of the current situation. The effective organization of these data lends to a clear understanding of their relevance and limitations, and the underlying logic behind their use; thus, supporting an effective assessment.

As information enters the headquarters it must be quality checked by the assessment cell; obvious errors must be corrected. For example, a map reference that is outside the operational area, a date from the distant past, or a casualty report that is far out of the norm is likely an error. After the quality check, information or data must be organized and stored to facilitate analysis. The method must be sufficient for the intended analysis and might be as simple as a tally kept on a white board, but most assessment data would benefit from a more robust storage technique. Typically, this means storing the information on a database. Frequently, staffs get by with tables stored in a common commercial spreadsheet program to which most Service members have access and can use for simple information storage tasks. Because this database can sort data, create charts and graphs, and do a number of mathematical manipulations on quantitative data, it is usually superior to storing data as storyboards, slides, or printed reports (unless the data require a lot of narrative text). As the assessment problem grows with the complexity of the operation, a common commercial database will prove better, and ultimately, a special-purpose database may be required. Assessors store reports as a series of fields in the database, which facilitates planned analysis, answers key questions, and may answer other questions posed by analysts or key leaders as the staff learns or as the OE changes.

## 14. Considerations For Planning For Analysis

As noted in chapter 1, an assessment is trying to answer six general questions.

These are:

(1) How has the OE changed?
(2) How much discernable progress exists in accomplishing our operational objectives?
(3) What do we think caused progress and/or lack of progress in achieving our
objectives? (4) Do the changes in the operational environment cause a change to operations and/or plans? (5) What are the resource gaps to accomplishing our objectives and what are the risks associated with the current resourcing?
(6) How does this assessment nest with HHQ assessments and incorporate lower-level assessments?
Analysis pursuant to assessment answers specific questions the commander and staff have determined are important to the success of operations, or that analysts or decision makers pose as operations progress. These are often termed assessment questions or strategic questions, and are more difficult to answer than the IRs that empirical indicators answer, in that the former require analysis, critical thought, reasoning, and military judgment.
Indicators answer IRs; therefore, provide the empirical basis for the analysis which answers higher-order questions, and permits the staff to draw relevant conclusions about the OE. This allows the staff to recommend changes to the plan which will make operations more effective. Quantitative indicators are desirable because they are frequently less subject to bias than are qualitative indicators. However, in a complex OE, there are important aspects of it that defy quantitative measurement, and require qualitative description. Therefore, assessors must be able to analyze and synthesize both quantitative and qualitative data into an integrated understanding of the OE and communicate this understanding through an integrated assessment product.

Some quantitative indicators will require statistical description or other mathematical manipulation. Descriptive statistics: means, medians, modes, variances, frequency distributions, percentages of totals, etc., and simple data visualization in a graphic form can aid assessors in making sense of raw data or making relevant comparisons. More sophisticated mathematical techniques require appropriately trained operations research personnel to make the best sense of the data, and to minimize the possibility of misleading analytic errors. For example, a lot of quantitative assessment information is stored as time-series data. The assessor will display the data in appropriate ways, because the trend of the indicator, or the trends of several related indicators viewed together, tell the story that the assessor is trying to extract from the data. The assessor may also perform analytic techniques to smooth the data or to remove seasonality from the data and isolate the trend.

View the related indicators together when viewing the trends of indicators. There may be several reasons why one indicator is moving in a certain way, but some of these explanations can be discarded based on the simultaneous movement of other related indicators. Moreover, an assessment is frequently not as simple as observing an indicator increase or decrease over time. There may be setbacks or adversary activity reflected in the data, or the staff may find that an increase in the value of an indicator in a given time period is good, and a decrease in the same indicator in another time period is also good. For example, if a unit moved into a new area surrounded by a local population, they might begin tracking the number of tips per week; where the local people inform on enemy personnel. An initial increase in tips might indicate that the people are growing in their trust of the friendly unit, but over time, the number of tips per week reach a maximum and then begin to decline. This could indicate a loss of trust, but if contacts with the enemy have also topped out and begun to decline, it is likely the fewer number of tips reflects the fewer number of enemy personnel remaining in the area. In this case, both the increasing trend and the decreasing trend are good.

Viewing several indicators together also tells a more complete story. For example, if enemy-initiated attacks are down, but friendly casualties are up, it would appear that attacks are becoming more effective. The assessor would attempt to determine if this is true, and why it is occurring, so that the command can take action to mitigate this increased level of risk. Indeed, the important assessment question is not what the indicators are doing; but why they are doing it. Assessors must attribute change in the OE, reflected in the indicators, to the correct causes in order to know what joint forces should do next to be most effective.

Those assigned to work with quantitative data frequently shy away from talking about causality because the data, if they are the correct type, can only demonstrate a statistical correlation of an indicator to others. In an assessment, however, assessors cannot avoid the issue of causality because ultimately assessors and functional area experts on the staff must explain why indicators are moving as they are. Any operational plan that orders a set of tasks in pursuit of a set of objectives or effects is a set of causal hypothesesthat these tasks are developed to cause desired changes in the OE. An assessment can be viewed as a testing of these hypotheses; therefore, assessors must address issues of causality.
Assessors cannot assume that a friendly action alone caused an outcome. A
complex OE is full of other actors, some allied or sympathetic to US goals, some opposed or adversarial toward US goals, and many third-party actors that are pursuing agendas of their own. Assessors need to consider sets of related indicators together, determine possible causal relationships that would account for the simultaneous movements in indicators, and then go back into the data in an attempt to eliminate some possible causes, isolating the most likely cause.

Math, statistics, data visualization, and mapping techniques are important means of analysis. It should also be apparent that the most critical skill for assessment is critical thinking, particularly as the staff sorts through qualitative information and combines qualitative information with quantitative data. The assessment process and products are viewed as qualitative in nature for three reasons: in a complex environment, some important features cannot be adequately described by quantitative information; once the staff combines quantitative and qualitative information, the synthesis is qualitative; and human military judgment is required to make sense of the collection of indicators and analytic output.

## 15. Considerations For Planning To Communicate The Assessment

In determining the form of products that communicate the assessment, the most important thing is to remember that the product should not be confused with the assessment itself. The latter is the improved understanding of the OE gained from working through the assessment process; the former is merely a device to communicate the most pertinent portions of this understanding at a point in time to senior leaders.

The purpose of the product drives its form or format. The primary purpose of an assessment is to make operations more effective. This works by helping the commander understand their OE and select appropriate actions for the forces. Therefore, the product will reflect the commander's preferences for the display of information, and it needs only to address the things asked to see, answer specific questions posed, and present things that the staff has discovered that require the commander's attention. It does not need to share the status of every indicator, but most frequently consists of one or more slides with appropriate graphs or graphics which make it clear to the commander what the staff has discovered. This depiction will often form the basis of a conversation between the staff and the commander, or among several commanders, that improves their understanding of the OE, aids their decision making, and leads to more effective operations.

Operations assessment also provides accountability of activities and resources to HHQ, and which aids higher ranking commanders in allocating resources in the future, which in turn, makes the enterprise more effective. In this case, the higher ranking commander may specify what information is required by posing questions, imposing IRs, or specifying a report format. Usually, the product takes the form of a report that contains empirical information explained by a narrative. This report should contain sufficient information about collection methods, sources, and analysis performed to provide transparency and permit a reader to form a judgment as to the quality of the report's conclusions.

An assessment may also assist a number of leaders other than the unit's commander and their higher ranking commander. Staff directors, subordinate commanders and staffs, adjacent commanders, allied leaders, interagency partners, and potentially a large number of audiences may benefit from a thorough assessment product communicated clearly.

Finally, assessment products may be released to the public, and these may become messaging tools to garner support for a greater effort or may be used by other actors to build a case for denying additional resources. Assessors must recognize that products may be used as messaging devices, and they must also understand that the use of products in this way creates pressure to tell a story in a particular way. This has implications for the veracity of assessment products, and requires an ethical commitment from assessors to perform analysis in valid ways so that they are telling the truth as they understand it.

## 16. Evaluating The Effectiveness Of The Assessment Product

There are two criteria for evaluating an assessment product:

(1) Does the analysis conducted by the staff in support of an assessment product promote a nuanced understanding of the OE and help the staff
recommend appropriate actions for forces?
(2) Does this depiction allow the effective communication of the staff's findings
and recommendations, and their implications, to the commander?
If the above criteria are met, the assessment product is useful. If not, it requires revision. Under no circumstances is the mere production of a slide or a report sufficient to constitute an assessment. An assessment is the process behind the depiction.

If the staff is not gaining a nuanced understanding of the environment, and conveying it to the commander for a decision, then a redesign of the assessment process and tools needs to be completed.
This page intentionally left blank.

## Chapter Iii Collect And Analyze 1. Introduction

This chapter focuses on steps three and four of the six steps to an operation assessment. Conducting collection and analysis for an operation assessment is an integral part of planning and execution. Throughout planning and execution, the staff refines and adapts collection as well as analysis methods. This chapter provides tools, concepts, and techniques for enhancing the collection and analysis that leads to actionable recommendations to the commander and staff. Table 3 depicts steps three and four of the assessment process.

Operations
Step
Input
Personnel Involved
Staff Activity
Output
Process
Activity
- Multi-source
Collect Information
and Intelligence
Execution
intelligence reporting and joint
force resource and disposition
information
- Estimates of OE
conditions, enemy
disposition, and
friendly disposition
- Intelligence analysts
- Current operations - AWG personnel - Assessment cell (if
established)
- Joint targeting - JIPOE - Staff estimates - IR management
- ISR planning and
optimization
- Operational
reports
- Intelligence
assessments
Execution
- Assessment work
group
Analyze and Synthesize the
Feedback
- Staff estimates
- Primary staff - Special staff - AWG personnel - Assessment cell (if
- Estimate of joint
force effects on OE
(draft assessment report)
- Staff assessments
- Analysis methods
established)
JIPOEjoint intelligence of the operational environment OEoperational environment

Legend: AWGassessment working group IRinformation requirement ISRintelligence, surveillance, and reconnaissance

## 2. Collect Information And Intelligence

Personnel involved. Information and intelligence collection is a continuous whole-of-staff effort. Effective staffs leverage existing reporting mechanisms, whenever possible, to enable collection of information and intelligence. An example of who is involved in the collection plan and their role is listed in table 4.

| Position                                            |
|-----------------------------------------------------|
| Commander                                           |
| Approve allocation of resources to an               |
| assessment.                                         |
| Deputy Commander or Chief                           |
| of Staff                                            |
| Managing battle rhythm, staff resources, and        |
| personnel dedicated to an assessment.               |
| Assessment Cell                                     |
| Identifying and refining assessment information     |
| requirements and organizing assessment data         |
| collected and stored to inform analysis.            |
| Intelligence Section                                |
| Staff section responsible for identifying and       |
| refining threat and operational environment         |
| information requirements and organizing data        |
| collected and stored to inform intelligence         |
| analysis, for the collection of intelligence and    |
| preparing the joint intelligence preparation of the |
| operational environment.                            |
| Current Operations                                  |
| Staff section responsible for monitoring the        |
| activity of the force. Inputs from subordinate      |
| task forces will likely be received in current      |
| operations.                                         |
| Provide updated staff estimates.                    |
| Special Staff: Public Affairs,                      |
| Civil Affairs, Information                          |
| Warfare, etc.                                       |
| Assessment Working Group                            |
| Provide subject matter expert support to the        |
| assessment team in identifying and refining         |
| relevant information to support the assessment.     |

## Inputs.

(1) The collection step formally starts with an approved assessment plan and its associated data collection plan. (2) The assessment cell should leverage the intelligence collection process and any other staff system appropriate to the assessment plan. It should set requirements for data input to organize the storage of indicators identified for analysis during the development of the assessment plan.

(3) The assessment cell will likely influence collection efforts, and may generate independent assessment tools and forms or formats. (4) Note how the source staff section is included in table 5. This technique enables the assessment cell to assign responsibility for collection, and confirm that the data is available during or before execution.

| Objectives                               | Effects      |
|------------------------------------------|--------------|
| Indicators                               | Source       |
| Number of attacks against integrated air |              |
| defense and battle damage assessment     |              |
| (BDA).                                   |              |
| Number of radar acquisitions on friendly |              |
| aircraft.                                |              |
| Intelligence                             |              |
| Destroy                                  |              |
| enemy air                                |              |
| defenses                                 |              |
| Amount of signals intelligence (SIGINT)  |              |
| activity between enemy radar systems.    |              |
| Intelligence                             |              |
| Number of friendly aircraft lost.        |              |
| Air Liaison                              |              |
| Officer                                  |              |
| Number of long-range artillery units     |              |
| identified and engaged.                  |              |
| Fires                                    |              |
| Number of message campaigns/leaflet      |              |
| drops on long-range artillery.           |              |
| Fires                                    |              |
| BDA of enemy long-range artillery.       | Intelligence |
| Level of SIGINT between enemy artillery  |              |
| headquarters (HQ).                       |              |
| Intelligence                             |              |
| Percentage of friendly missions with     |              |
| effective enemy counterfire.             |              |
| Intelligence/Fires                       |              |
| Destroy                                  |              |
| enemy                                    |              |
| integrated                               |              |
| fire control:                            |              |
| disrupt                                  |              |
| enemy fire                               |              |
| support and                              |              |
| target                                   |              |
| acquisition                              |              |
| systems                                  |              |
| Combat                                   |              |
| power:                                   |              |
| reduce enemy                             |              |
| combat power                             |              |
| to enable                                |              |
| defeat                                   |              |
| Number of effective enemy artillery and  |              |
| missile strikes.                         |              |
| Intelligence/Fires                       |              |
| BDA of enemy attack aviation units.      | Intelligence |
| Number of enemy aviation attacks on      |              |
| friendly units.                          |              |
| Degrade                                  |              |
| enemy                                    |              |
| attack                                   |              |
| aviation                                 |              |
| BDA of enemy maneuver units (tanks,      |              |
| armored personnel carriers).             |              |
| Intelligence                             |              |
| Degrade                                  |              |
| enemy                                    |              |
| maneuver                                 |              |
| Friendly maneuver forces reporting       |              |
| favorable combat power ratios.           |              |
| Subordinate HQ                           |              |
| BDA of countermobility and engineering   |              |
| assets.                                  |              |
| Intelligence                             |              |
| Number of known enemy obstacles.         | Intelligence |
| Degrade                                  |              |
| enemy                                    |              |
| engineer                                 |              |
| assets                                   |              |
| Number of known enemy dug-in             |              |
| positions.                               |              |
| Number of electronic warfare missions    |              |
| conducted.                               |              |
| Fires                                    |              |
| Number of strikes on enemy division      |              |
| and brigade command and control (C2)     |              |
| nodes.                                   |              |
| Command                                  |              |
| and control:                             |              |
| delay enemy                              |              |
| decision                                 |              |
| making                                   |              |
| Strikes and battle damage of high-value  |              |
| items.                                   |              |
| Intelligence/Fires                       |              |
| Disrupt                                  |              |
| enemy                                    |              |
| maneuver                                 |              |
| forces:                                  |              |
| degrade                                  |              |
| enemy                                    |              |
| command                                  |              |
| and control                              |              |
| Number of missions conducted effecting   |              |
| enemy C2.                                |              |
Fires
Intelligence/ Subordinate HQ
Intelligence/ Subordinate HQ Intelligence/Fires Special Technical Ops
| Objectives                                | Effects      |
|-------------------------------------------|--------------|
| Indicators                                | Source       |
| Reduction of SIGINT activity between      |              |
| appropriate HQ.                           |              |
| Intelligence                              |              |
| Reported ability to C2 at brigade level.  | Intelligence |
| Reported ability to C2, and division      |              |
| level and above.                          |              |
| Intelligence                              |              |
| Disrupt enemy                             |              |
| maneuver                                  |              |
| forces: degrade                           |              |
| enemy                                     |              |
| command and                               |              |
| control (cont'd)                          |              |
| Evidence of enemy maneuver ability to     |              |
| conduct effective defense.                |              |
| Subordinate HQ                            |              |
| BDA on reserve force, HQ, and             |              |
| sustainment.                              |              |
| Intelligence                              |              |
| Disrupt                                   |              |
| mechanized                                |              |
| brigade's ability                         |              |
| to reinforce                              |              |
| Reserve units not in position to          |              |
| effectively reinforce enemy main effort   |              |
| or shaping effort.                        |              |
| Number of strikes with BDA on enemy       |              |
| logistics and logistics C2 nodes.         |              |
| Intelligence                              |              |
| Command                                   |              |
| and control:                              |              |
| delay                                     |              |
| enemy                                     |              |
| decision                                  |              |
| making                                    |              |
| (cont'd)                                  |              |
| Number of nonlethal strikes on enemy      |              |
| logistics C2 nodes.                       |              |
| Fires                                     |              |
| Deny, deceive,                            |              |
| and degrade                               |              |
| logistics                                 |              |
| command and                               |              |
| control                                   |              |
| Level of effectiveness of enemy           |              |
| logistics.                                |              |
| Intelligence                              |              |
| Number of nonlethal attacks on enemy      |              |
| leadership credibility.                   |              |
| Fires                                     |              |
| Number of reports indicating mistrust in  |              |
| enemy political leaders.                  |              |
| Intelligence                              |              |
| Undermine                                 |              |
| credibility of                            |              |
| enemy country                             |              |
| leadership                                |              |
| Number of defections observed.            |              |
| Intelligence/                             |              |
| Subordinate HQ                            |              |
| Number of nonlethal strikes targeting     |              |
| enemy command will.                       |              |
| Fires                                     |              |
| Number of enemy forces surrendering       |              |
| or withdrawing.                           |              |
| Reduce enemy                              |              |
| forces will to                            |              |
| resist                                    |              |
| Intelligence reports of reduced enemy     |              |
| will to fight.                            |              |
| Intelligence                              |              |
| Number of nonlethal strikes linked to     |              |
| lethal strikes and friendly maneuver.     |              |
| Fires                                     |              |
| Number of nonlethal strikes linked to     |              |
| high enemy casualty actions.              |              |
| Fires                                     |              |
| Will:                                     |              |
| reduce                                    |              |
| enemy will                                |              |
| in order to                               |              |
| generate an                               |              |
| operational                               |              |
| advantage                                 |              |
| Reports of reactions to lethal strikes or |              |
| maneuver.                                 |              |
| Intelligence                              |              |
| Amplify                                   |              |
| psychological                             |              |
| effects of                                |              |
| enemy                                     |              |
| casualties and                            |              |
| the physical                              |              |
| destruction of                            |              |
| offensive                                 |              |
| operations                                |              |
Intelligence Intelligence/ Subordinate HQ
| Objectives                                |
|-------------------------------------------|
|                                           |
| Source                                    |
|                                           |
| Number of strategic level public affairs  |
| officer (PAO) messages and nonlethal      |
| actions affecting enemy will.             |
| Support                                   |
| international                             |
| strategic                                 |
| communications                            |
| Host nation, enemy, and ally reactions,   |
| international media sentiment.            |
| Number of countermessages to              |
| adversary media.                          |
| PAO                                       |
| Number of nonlethal strikes to reduce     |
| adversary influence.                      |
| Fires                                     |
| Number and significance of outlets and    |
| stories supporting enemy.                 |
| Will:                                     |
| reduce                                    |
| enemy will                                |
| in order to                               |
| generate an                               |
| operational                               |
| advantage                                 |
| (cont'd)                                  |
| Number of enemy messages reaching         |
| and resonating with host-nation           |
| population.                               |
| Reduce                                    |
| adversary                                 |
| medias                                    |
| influence in                              |
| reaching the                              |
| host nation                               |
| population.                               |
| Prevent media                             |
| support to                                |
| insurgency.                               |
| Number of enemy forces remaining in       |
| host-nation territory.                    |
| Intelligence                              |
| Clear host                                |
| nation of enemy                           |
| forces                                    |
| Subordinate HQ established contact        |
| with host-nation authorities.             |
| Enhance                                   |
| federal and                               |
| local law                                 |
| enforcement                               |
| Status of critical law enforcement        |
| entities and infrastructure.              |
| Intelligence, CA                          |
| Number of strikes in vicinity of critical |
| infrastructure or cultural sites.         |
| Fires                                     |
| Protect critical                          |
| infrastructure or                         |
| cultural sites                            |
| Assessed damage to critical               |
| infrastructure or cultural sites.         |
| Intelligence, CA                          |
| Security conditions set to enable other   |
| organizations to assist population.       |
| Subordinate HQ                            |
| Support                                   |
| humanitarian                              |
| assistance                                |
| operations                                |
| Effective distribution of aid where       |
| needed in host nation.                    |
| Number of messages supporting             |
| legitimacy of host nation.                |
| PAO, Fires                                |
| Transition to                             |
| civil                                     |
| governance:                               |
| established                               |
| civil                                     |
| conditions                                |
| for host                                  |
| nation                                    |
| government                                |
| to assume                                 |
| control                                   |
| Number of reports of host-nation          |
| government-personnel assassinations.      |
| Intelligence                              |
| Number and intensity of protests and      |
| violent events.                           |
| Intelligence, CA                          |
| Reinforce                                 |
| legitimacy of                             |
| host nation                               |
| government                                |
| Number of reports of insurgent            |
| sabotage and attacks.                     |
| Intelligence                              |
| Status of displaced persons and camps     |
| (effective support to displaced           |
| population).                              |
| CA                                        |
Fires, PAO
Intelligence, PAO Intelligence,
PAO
Intelligence, PAO
Subordinate HQ/Civil Affairs (CA)
Intelligence, CA, Subordinate HQ

(5) Assessment collection information flow can use normal battle rhythm events and routing reporting channels and documents such as the intelligence, operation, and logistics status summary, and daily updates such as the commander's update brief as well as staff running estimates.

(6) An example of a simplified assessment plan that establishes the rhythm for reporting information requirements established above is shown below in the data collection methodology in table 6.

Source
Information and Intelligence
Frequency and delivery
Requirements
Daily at commander's update brief.
Intelligence
- Assigned measures of effectiveness
(MOEs) and measures of performance
(MOPs).
Daily posted to SharePoint.
- Threat and operational environment
update.
Daily at commander's update brief.
Current Operations and Future Operations
- Shaping objective assessment (based
on defined levels) and one-paragraph description per shaping objective with observed indicators' bullet points.
- Shaping objective assessment (based
on defined levels).
Fires
Daily during targeting working group (attended by assessment cell).
- Assigned MOPs.
- Generate recommendations (adjust
fire support coordination line,
boundaries, priorities).
- Consolidate input and generate
description and highlight key indicators.
Subordinate Headquarters 1
Daily update to
assessments cell.
- Shaping objective assessment (based
on defined levels) and one-paragraph description per shaping objective with observed indicators' bullet points.
- Assigned MOEs and MOPs.
- Shaping objective assessment (based
on defined levels).
Subordinate Headquarters 2
Daily update to
assessments cell.
- Assigned MOEs and MOPs.

(7) Clearly establishing reporting requirements, delivery methods, and the frequency of delivery greatly increases the efficiency of the assessment process and leaves the assessors more time to conduct analysis.

Assessment Cell Data and Information Sources.

(1) JIPOE: The staff continually updates and refines the JIPOE or their respective Service intelligence picture in order to maintain a holistic view of the OE. The process is depicted in figure 1. Assessors seek to leverage this continuous effort to inform the command on the effectiveness of operations and generate recommendations to improve operations.

(2) Staff external sources. Agencies or entities outside of the staff can provide information crucial to assessing operations. Examples include the Department of State, and other governmental and nongovernmental organizations. Assessment cells may also be able to find funding for analysis from private agencies. (3) Other assessment data collection methods:
(a) The assessment cell may determine that reporting through existing staff estimates is not sufficient. The cell may ask subordinates or staff sections to provide input through custom manual forms or even automated collection systems. (b) Assessment cells should carefully consider the necessity of such products before adding reporting requirements to subordinates and other staff sections. (c) The following vignette portrays an assessment that leverages multiple best practices in assessment; standards-based and written assessments.

## International Security Assistance Force (Isaf) Campaign Assessment

In 2012, the ISAF headquarters' assessment effort measured the state of the war and the progress towards achieving strategic and campaign goals. This vignette illustrates the campaign assessment portion of the ISAF assessment effort.
The ISAF commander (COMISAF) guidance for planning the assessment:


The process must assess all aspects of the war in Afghanistan, rather than just the military aspects.

The assessment must stimulate discussion among senior leaders, as opposed to just presenting information.

The results of the assessment must be actionable. COMISAF wanted the process to identify items that could address challenges and opportunities
within COMISAF's span of control, and on which the commander could take, direct, or request action as appropriate to make operations more effective.

Subordinate and supporting commanders must be involved in the assessment's inputs, outputs, and outcomes.

The ISAF assessment cell will leverage the ISAF staff and ISAF's subordinate and supporting commands for necessary expertise. The ISAF assessment cell will not act as an independent entity.

The process will adhere to the quarterly cycle of reporting and the battle rhythm requirements levied by North American Treaty Organization and United States chains of command.
The ISAF assessment cell chose to organize data by purpose. ISAF listed eight essential tasks along with the assertion that accomplishment of the eight tasks would equate to mission accomplishment. The assessment cell identified four fundamental domains across which they would measure progress towards or setbacks from achieving ISAF campaign goals for each essential task. Table 7 depicts the adopted organizational method.

Campaign Goal 1 Campaign Goal 2 Campaign Goal 3
Command Assessments
Security
Governance
Socioeconomic
Regional Relations
Essential Task 1: XXXX Essential Task 2: YYYY Essential Task 3: ZZZZ Essential Task 4: AAA Essential Task 5: BBB Essential Task 6: CCC Essential Task 7: DDD Essential Task 8: EEE

The ISAF assessment cell developed standards for each fundamental domain for each essential task to provide a common framework for thinking about the campaign and provide necessary space for including nuance and context. COMISAF required subordinate and supporting commands to report progress and setbacks for each essential task against the domain standards depicted in the five-point rating definition scale in table 8.

Campaign Essential Task 1: Secure Areas XXXX and YYYY
Category
Level 1
Level 2
Level 3
Level 4
Level 5
Stated areas are not
secured.
Security
Stated areas are partially secured, but
with significant risk of reversion.
Governance
Key government actors are not present in the stated areas.
Some key government actors are present in the stated areas and/or their actions are significantly undermining security.
Socioeconomic
Security conditions in or around the stated areas are significantly
hindering legitimate
socioeconomic activity.
Security conditions in or around the stated areas are moderately
hindering legitimate
socioeconomic activity.
Regional Relations
Other
countries are playing
an overall moderately negative role with respect
to security in the stated areas.
Other
countries are playing
a significantly negative role with
respect to security in
the stated areas.
Stated areas are partially secured, but
with minimal risk of reversion.
Stated areas are fully secured
with minimal risk of reversion.
Stated areas are partially
secured, but with moderate risk of reversion.
All key government actors are present in the stated areas and/or their actions are minimally undermining security.
All key government actors are present in the stated areas and they are actively working to enhance security.
A majority of key government actors are present in the stated areas and/or their actions are moderately undermining security.
Security conditions in/around the stated areas are having no impact on
legitimate socioeconomic activity.
Security conditions in/around the stated areas are enhancing
legitimate socioeconomic activity.
Security conditions in or around the stated areas are having
minimal impact on
legitimate socioeconomic activity. Other
countries are playing
an overall minimally positive role with respect
to security in the stated areas.
Other
countries are playing an
overall moderately positive role with respect
to security in the stated areas.
Other
countries are playing
an overall significantly positive role with respect
to security in the stated areas.

Line of effort (LOE) managers reported their assessment results using the campaign assessment template depicted in figure 2. COMISAF afforded subordinate and supporting commands the ability to select and rate only those tasks and domains that pertained to their specific mission. Subordinate and supporting commands chose the standard that is most representative of their situation for each selected task in each selected domain, and provided narrative justification for their particular standard choices. Subordinate and supporting commands also provided narratives on the most significant obstacles to future progress for each selected task, the most significant opportunities for ISAF to act on, and any other items of interest. Additionally, subordinate and supporting commanders submitted a less structured, personal assessment directly to COMISAF summarizing the heart and mind of the commander regarding their efforts to execute the ISAF operational plan.

Analysis primarily consisted of studying all the commands' responses against the developed standards for each domain of each task. Analysis revealed differences in views among subordinate and supporting commanders as to what was and was not working in the campaign. These differences often served as discussion points among the ISAF staff and for the commanders' quarterly assessment conference. Another key component of analysis was the identification of opportunities and challenges to future effectiveness in each task, and an appraisal of the risk to the overall mission if ISAF failed to overcome the identified challenges. Appropriate actionable recommendations were developed to make operations more effective.

The ISAF assessment cell used two distinct products to validate the analysis results during a series of working group meetings and, subsequently, to communicate the assessment to the COMISAF. The first assessment product was a set of PowerPoint slides summarizing the commands' inputs for each of the eight essential tasks. Having all of these inputs presented on a single slide for each of the eight essential tasks stimulated significant discussion. The inclusion of the actual standards corresponding to the consolidated response in the chart kept the discussion focused on achieving the stated campaign goals. Presenting subordinate and supporting commands' comments verbatim on the slides preserved and effectively communicated the raw information supporting the assessment. The second output of the campaign assessment was a narrative set of issues identified via the overall assessment portion of the campaign assessment template (shown in figure 3).

(4) Staff Estimates: Running staff estimates provide information, including indicators, and provide context to assessments.

(a) Army Doctrine Publication 5-0, *The Operations Process,* provides a template for staff estimates that include: facts, assumptions, friendly force status, enemy activities and capabilities, civil considerations, conclusions, and recommendations. (b) Assessors should work with the chief of staff to ensure all staff sections focus on the "so what" in their estimates; this is usually evident in the conclusions and recommendations section.

(c) A technique includes an assessment section within running estimates presented to the commander and available to the assessment cell.

Output: The collection process is continuous, but sampled as necessary by the assessment cell to conduct analysis. Assessors organize data to enable effective analysis, ensure its relevancy to understanding operational effectiveness, and vet all information collected to ensure accuracy and valid analysis.
Effective Collection Methods:

(1) Linkage. Link indicators to effects and objectives.
(2) Accountability. Firmly establish reporting requirements as tasks for
subordinate units or staff sections.
(3) Availability. Ensure that data and information are availablenote the
absence of critical information required. (4) Purpose. Clearly articulate the purpose of collection tasks.
(5) Relevancy and Focus. Do not over collect, more information is not always better. Only collect data that are necessary for validating analysis. Focusing on fewer measures can often lead to more accurate measurements and greater analytical rigor. (6) Qualitative versus Quantitative. Not all important aspects of the OE can be counted; do not solely focus on collecting numbers.

## 3. Analysis

Assessors analyze purposefully collected information and intelligence products and reports to inform assessment. The purpose of analysis is to identify trends and changes in the OE over time that signal either operational effectiveness or a need to consider adjusting the plan to attain progress towards objectives and end states.

Analysis is vetted and validated through the staff prior to Step 5, Communicate Feedback and Recommendations. All recommendations, and any major issues unable to be resolved by the assessment team, are presented to the commander for approval and implementation guidance.

Analysis must be a whole-of-staff effort that leverages functional expertise. The collection and analysis steps blend; functional experts and subordinates provide analysis in their areas of expertise as depicted in figure 4.

An assessment cell fuses all input into higher-level recommendations for the commander. The cell should staff their analysis to ensure accuracy, and conduct an assessments working group to further refine when time is available.

## Staff Activities. (1) Awgs.

(a) The AWG should improve and validate the staff's analysis before it goes before the commander. Time consuming data-mining and analysis should not be conducted during the actual working group. (b) A best practice is for the assessment cell to send a draft of the assessment product to be communicated to the commander along with supporting analysis to all members prior to the working group. Participants can then prepare appropriately and provide meaningful and efficient input at the meeting. (c) An AWG should gather members of all major staff sections, key special staff if required, and guests from outside agencies or subordinate units when available. (d) The assessment cell can guide the working group's discussion, inputs, outputs, and required attendance with a seven-minute drill. The seven-minute drill gives each staff section no more than seven minutes to discuss the updated information to their running estimate and what they believe the "so what" is from that information.

(e) Time constraints may prevent the staff from conducting an independent AWG, but validation of the assessment cells analysis can occur in or during other venues. Examples include a combined plans and assessments working group, the targeting board, or potentially an intelligence working group. There are many other options, including not having a formal working group at all, but the assessment lead must find a way to involve all key personnel in the creation and validation of the assessment product. Figure 5 shows an example assessment working group quad chart.

## Assessments Working Group Proposed Time: 11001200; Daily

Physical LocationExecutive Conference Room; Virtual LocationVoice (XXX) XXX-XXXX

Purpose and Function:
Inputs:
Product
Time and method of delivery
From whom or
what battle
rhythm event?
Operational
Received at plans update for
(1) Joint planning
next phase. Assessments
before start of
downloaded from SharePoint.
exercise.
approach slides (with end states and objectives).
(2) Plans update.
Enemy
functional-effects
0600 and 1700 daily, prepared by intelligence, briefed by MAJ
assessments.
Schwartz.
Previous targeting
decision brief and
intelligence from
battle update brief.
Brigade
0900 daily. Email to assessment
All brigades.
assessment.
cell.
1500 the previous day. Posted
Last assessments
on SharePoint.
working group.
Consolidated assessments
slides by line of
operation.
Executive
1000 daily. Posted on

(2) The assessment cell and staff answers the six general questions based on the assessment plan. They must synthesize answers with information and intelligence collected, which may be qualitative or quantitative. Conclusions must be evidence based to maintain the assessment's credibility. The six questions and some thoughts on why these are important are discussed in table 9.

Chair: Division Chief of Staff.
Lead: Chief of Assessments.
Attendees by location:
In person: Intel, Sustainment, Commutations, Civil Affairs,
Protection, Air and Missile Defense, Engineer, Provost
Marshall, Public Affairs, Electronic Warfare, Aviation, and
Liaisons.
Virtual: Subordinate Brigade Representatives.
Agenda (simplified as appropriate)
(1) Review operational approach with end states and objectives
by line of operation.
(2) Review or brief changes to indicators.
(3) Assess current status of each line of effort, using each staff
and brigade assessments input.
Outputs/Decisions/Next Actions:
Product
Time and method of
To what battle rhythm
delivery
event?
| Question                                                              |
|-----------------------------------------------------------------------|
| How has the                                                           |
| operational                                                           |
| environment                                                           |
| (OE) changed?                                                         |
| The staff must document key changes in the OE. Their focus is on      |
| understanding the impact of friendly and enemy operations and         |
| the impact of activities conducted during the previous reporting      |
| period. Answering this question determines if the mission, tasks,     |
| and activities executed impact decisive conditions in a positive or   |
| negative way.                                                         |
| Answers to this question help determine progress or lack of           |
| progress along measurable objectives. When progress is difficult      |
| to measure, using standards-based bins allows the staff to            |
| qualitatively relate if there is or is not discernable progress.      |
| How much                                                              |
| discernable                                                           |
| progress exists                                                       |
| in accomplishing                                                      |
| our operational                                                       |
| objectives?                                                           |
| What do we think                                                      |
| caused progress                                                       |
| or lack of                                                            |
| progress in                                                           |
| achieving our                                                         |
| objectives?                                                           |
| Analysis will enable the staff to posit why they think changes in the |
| OE occurred. Professional military judgment enables critical          |
| thinking on attributing causality, but the staff should maintain      |
| caution during this effort to avoid common biases. Leveraging a       |
| theory of change or a causal diagram can assist the staff in          |
| determining complex changes in the environment.                       |
| Answering this question queues the staff to implement branches        |
| or sequels to the plan, ensuring the current plan possesses a         |
| clear path to achieve the end state or objective.                     |
| Do the changes                                                        |
| in the OE cause                                                       |
| a change to                                                           |
| operations and                                                        |
| plans?                                                                |
| What are the                                                          |
| resource gaps to                                                      |
| accomplishing                                                         |
| our objectives                                                        |
| and what are the                                                      |
| risks associated                                                      |
| with the current                                                      |
| resourcing?                                                           |
| Gaps are an important product of the analysis step because they       |
| lead to solid recommendations that the commander can take             |
| action on by either reallocating resources or requesting additional   |
| resources from a higher headquarters (HHQ). Clearly articulating      |
| the risk to the operation relays the criticality of the resource      |
| allocation decision.                                                  |
| See Chairman of the Joint Chiefs of Staff                             |
| manual 3105.01, Joint Risk Analysis                                   |
| , for standardized risk                                               |
| definitions.                                                          |
| How does this                                                         |
| assessment nest                                                       |
| with HHQ                                                              |
| assessments                                                           |
| and incorporate                                                       |
| lower level                                                           |
| assessments?                                                          |
| The assessment informs the commander by articulating progress         |
| and if that progress causes a change to the mission, but it also is   |
| an important communication tool for the commander and staff           |
| because it provides a detailed list of capacity, authority, or        |
| capability gaps and associated risk in a common language to relay     |
| to their HHQ. The details from subordinate headquarters must          |
| provide relevant information that informs the evaluation of           |
| progress, incorporating their gaps and risk if relevant to the higher |
| mission.                                                              |

## Analytical Tools And Techniques.

(1) This section outlines potential analytical tools and techniques that enable assessors to identify trends, patterns, and responses in the OE. The staff can use these to generate answers to the six general questions.

(2) Data Visualization.
(a) Visualizing data provides opportunities to quickly identify trends and patterns within the OE. The following examples guide the use of some of the most common techniques. When time permits, the staff or assessment cell should conduct statistical tests that confirm whether patterns exist or are significant.

(b) Tables.
- Tables are useful analytic tools, but quickly lose their utility as multiple categories and or large amounts of data are accumulated.

- Sorting and filtering of tabular data enables more complete analysis and understanding of trends or identification of outliers.
- See the example table in table 10, and note how it is fairly easy to compare data in the table within one category, but it quickly becomes challenging to make comparisons across categories or to look for potential patterns and trends. - When using tables, limit the volume of data displayed at once to enable effective analysis and eventual communication of analytical results.

Province
A
B
C
D
E
F
G
H
I
J
K
L
Protests
10
12
13
14
15
8
12
12
8
7
3
2
3
5
7
10
4
2
9
2
1
2
3
5
Improvised
explosive
device
events
Humanitarian
3
2
3
2
1
4
7
2
3
4
2
2
aid
distributions

(c) Pivot Tables.
- Pivot tables are extremely useful when conducting analysis. They are tables that summarize data, and are generated by applying operations such as sorting, averaging, or summing data, typically including groupings appropriate to answer an analysis question. - A simple pivot table generated from the data in table 10 might simply contain averages across provinces.

- Microsoft Excel (or similar program) has a dedicated pivot function, and many other applications provide pivot functionality. With the high volume of data often collected, assessors should seek to leverage such tools to rapidly generate analyzable results.

(d) Bar Charts.
- Bar charts are used to compare and display categorical data. Categories can include locations, conditions, groups of persons, or any other relevant factor identified to group measurements and indicators.

- Human visual perception performs length comparisons quickly and with a high level of accuracy, especially when compared to area comparisons. For this reason, a bar chart may be a better option than an area-based graphic such as a pie chart if the intent is to more fully compare similar
values and understand differences.
- The grouped bar chart in figure 6 displays the same data from table 10. This provides the analyst a method for identifying patterns. One can readily discern potential correlations between the categories of information shown by province. The information is much more readily usable for cross comparison than it was in the table format.

- Take care to note the scale of the charts to ensure you understand the relative magnitude of the measurements. It is a best practice to at least begin with a y-axis starting at zero. The analyst can then zoom in to identify potential differences as required.

- Often categories of data have different relative magnitudes, so it may be necessary to display data with multiple y-scales together on the same
chart to effectively conduct the analysis. - Stacked bar charts can be useful to display information with
subcategories with a meaningful sum. One example comparing total
violent events by location or year, might be a stacked bar with subcomponents being types of violent events.
(e) Line Charts.
- Line charts are used to display time-series data. Assessors can identify
trends over time with these charts, especially when contextual information
is overlaid or included in the analysis.
- Note the example in figure 7, an assessor could assume that there was a reduction in attacks in response to increased patrols, but also a subsequent seasonal variation introduced by the start of the traditional summer fighting season. - Start the y-axis of line charts at zero when possible to show the relative magnitude of changes. - Ensure the x-axis or time changes are evenly spaced.
- As shown in figure 7, leveraging smoothing techniques, such as a simple moving average helps to avoid trends being masked by noise.
(f) Pie Charts.
- Pie charts can be useful for comparing data or proportional data with a few categories, but are generally best reserved as a communication technique. See figure 8 for an example of an effective use of a pie chart for comparison.

- Figure 9 demonstrates the limitations of pie charts. An analyst could much more effectively discern the difference in magnitude of measurements 15 when the information is displayed as a bar chart.

(g) Geospatial Chart. A geospatial chart, as shown in figure 10, is a way to analyze geographical or spatial data to search for trends. Geospatial analysis and communication methods can provide nominal information (such as demographics) or it can use ordinal information on a color scale (such as the status of security at the district level). The use of geospatial analysis techniques can cue an analyst or decision maker to areas on a map that requires additional focus. Geospatial charts also can depict the density of events (such as the locations and number of IED or small arms attacks along a specific route). The main limitation of geospatial methods is that the scale of the map can hide important details. For example, a national-level map may depict an entire province as transition ready, while a provincial-level map may expose important areas within the province where major problems still exist.

(3) Statistical Analysis.
(a) Descriptive statistics can be useful to summarize a data sample.

Examples include averages and variance estimators, or potentially higher moments such as skewness and kurtosis.

- Use measures of central tendency (mean, median, and mode) as a
starting point, but take care to examine the shape, or distribution of the data. - Use histograms to assess the shape of data and understand potential
implications on the relevancy of measures of central tendency. Note how in figure 11, all data share the same mean but have very different underlying distributions. The dashed line shows the mean, while the dotted line illustrates the median.
- Due to data distributions, the median and mode are considered during analysis. As in the skewed data plot in figure 11, the median better
represents the center of most of the data. In this way it may be more practically useful. - Always consider variance of the data. A system may perform well on average, but not do so consistently. The option or measure with the
highest mean or median then may not be the best where consistency matters more. See the high and low variance examples in figure 11. - When possible, determine whether there are statistically significant differences between groups. Analysis of variance can offer a method to conduct comparisons of three or more groups.
(b) Regression Analysis. Regression enables an analyst to better understand or estimate the relationships between variables. An example may be assessing how well the density of forces corresponds with violence within a given area. There are computer programs that offer tools for regression analysis. Ensure to use an abundance of caution when implying causation as
the result of any statistical analysis. (c) Advanced Statistical Techniques. When time permits, the analyst may consider using pattern recognition, clustering, multiple regression, or other means to model and/or identify operational effects on the environment.
(4) Correlation of Force and Means (COFMS). COFMS is a calculator used to compare the relative combat power of two forces and estimate the outcome of engagements between them. An example spreadsheet based calculator is shown in figure 12.

(a) An analyst can leverage COFMS calculators to inform operational decisions and improve operational effectiveness. It is often used during planning, but can also be leveraged continuously throughout the execution of a tactical or operational plan.

(b) A system to continually evaluate friendly and enemy combat power and rapidly conduct COFMS analysis informs command decisions on allocation of resources and the timing of operations during execution and can greatly improve operational effectiveness. (c) A COFMS spreadsheet or tool is a useful model, but must be augmented with professional military judgment and contextualized to account for asymmetries and effects of the OE.

(5) Modeling and Simulation. When time and resources permit, combat and stability models can be leveraged by an assessment team to inform future decisions. Models include advanced combat simulations and war-gaming tools or even social interaction and stability models such as Athena. (6) Professional Military Judgment. The assessment cell fuses information from disparate staff sections and data sources into cogent recommendations and conclusions for the commander. This distillation of information requires analysts to make assumptions and often requires experience and military judgment.

(a) The assessor or analyst is often not the most experienced and or knowledgeable source of professional military judgment. They can, however, provide an appropriate framework to leverage subject matter experts in and out of the staff. (b) Analysts must identify where professional military judgment or logic was used to inform assessments and use it carefully, but should also understand that judgment may prove more valuable than any single quantitative or qualitative measurement.

## (7) Sme Elicitation.

(a) Assessors seek input from subject matter experts to gain insights in the form of: expert opinion, subjective judgment, expert forecasts, best estimates, educated guesses, and expert knowledge. (b) Experts can provide estimates on new, rare, complex, or otherwise poorly understood phenomenon. (c) Analysts must identify and account for any potential biases in expert judgment. (d) Experts commonly available to an operational assessment team include subordinate commanders or senior enlisted, primary staff members, political or military advisors, interagency staff, and partnered force leadership.

## (8) Standards-Based Assessment.

(a) In order to gain consistent input from the staff or subordinate elements and to provide an objective basis for analysis in complex situations the staff can define grades of progress, including success using standards-based assessments.
(b) An example is provided in figure 13 (see page 50) for a targeting assessment. The framework provides both a basis for assessment that can be constructed prior to mission execution, and a method for incorporating information that may not have a baseline or prior expectations. (c) Definition-level constructs can also serve to inform SME elicitation and provide a basis for discussion during an AWG if the team is trying to communicate progress or success.

(d) The levels should be constructed with sufficient detail so that they are mutually exclusive and collectively exhaustive, i.e., for a given state of nature, the situation can clearly fit into only one category. If there is a discrepancy in that a condition resides in more than one category, then the definitions are updated to possess sufficient detail. The top level objective is normally reserved for the objective or end state of the LOE or LOO being measured.

The current state is not necessarily the lowest level; the assessor must look historically to see where the lowest possible state could be and use that as the lowest level, as the situation could unexpectedly deteriorate.

## Standards-Based Assessments

A method to provide a summation of progress is standards-based assessments. There are four reasons for the use of standards-based assessments; it is important to display data at the resolution we can effectively measure, assessments must relate to the objective's progress, standards-based binning facilitates gap analysis, and binning forces evaluators to provide compelling evidence. The process results in a method of clearly rating the progress toward an objective. In implementing a standard-based bin, a working group may employ the following steps: 1. Determine the goal. The military objective, normally an intermediate military objective (IMO) end state, is defined as the goal condition. If the end state is not clear at any point in the process, it is revised by adding more detail. This becomes the top bin, or goal state of the objective. 2. Determine the worst case. We define worst case as the worst possible state of progress, including states the IMO could retrogress to in the future. 3. Determine the additional bins. Determine what you want to discern between additional levels, and define the terms you wish to use to make this determination. Break the possible states into natural breaks, normally three to seven bins for a single objective. 4. Refine the bins. Each bin is described in at least a paragraph, in sufficient detail so there is no question as to which bin a scenario belongs. Bins are collectively exhaustive (every observation fits somewhere in the bins) and may possess mutual exclusivity (each observation fits in one bin) or build upon each other (each observation fits into a bin and all the bins below or above it). 5. Additional means. If the division of natural states proves problematic, additional observations are used by taking a similar historic situation and placing the observations on a continuum between the best and worst cases, compiling these into similar bins. Using historical examples is helpful because people relate better to conflicts they have experienced. 6. Plan to achieve the end state. Using the developed bins, plot a course from the present state until the stated date of the objective. Then, using planned activities and operations, determine remaining gaps. Two important disruptions frequently occur; working groups must design bins to prevent constructive credit for task accomplishment rather than effect accomplishment, and accountability for rating the IMO must remain with the working group. Otherwise, narratives diverge into listing activities accomplished rather than effects.

SOURCE: Are We There Yet? Implementing Best Practices in Assessments, Military Review, May-June 2018
COL Lynette Arnhart and LTC Marvin King

(9) Analysis Best Practices.
(a) Credibility. Thorough analysis ensures credibility. The commander and staff should know they can leverage analysis produced to make decisions. This requires citing and validation of all facts, assumptions, and opinions used. (b) Causality. Causality cannot be avoided because the staff is trying to attribute change in the OE to friendly operations. However, causality is very difficult, and other actors are also pursuing their agendas. Therefore, it is very possible to make a mistake when attributing causality, and assessors must have the moral courage to admit a mistake when more information makes a mistake obvious.

(c) Frequency. Take time to consider the OE and an appropriate timeframe for conducting analysis. Understand that variables and measures will respond to operational actions with varying timelines. Gain concurrence with the command on an appropriate timeline for assessment cycles.

(d) Noise. Take care to leverage statistical techniques, judgment, and expert opinion to avoid making conclusions on operational effectiveness based on natural variations in the OE or measurement inaccuracies.

(e) Vetting and Validation. Build an appropriate governance structure for the assessment and validate all assertions prior to publishing to the commander. (f) Leverage Expertise. Do not perform an assessment and analysis in a vacuum. Incorporate as much valid information and opinion as possible.

This page intentionally left blank.

## Chapter Iv Communicate The Assessment And Adapt The Plan

This chapter addresses steps five and six of the operation assessment process: How the staff best communicates its understanding of the OE, subsequent recommendations, and then translates the commander's decision into actionable directives. Table 11 shows steps five and six according to JP 5-0.

Operations
Step
Input
Personnel Involved
Staff Activity
Output
Process
Activity
- Commander. - Subordinate
- Provide the
- Estimate of joint
- Assessment report,
commanders (periodically).
Execution
Communicate the Assessment and Recommendations
assessment and recommendation to the appropriate decision maker.
force effects on OE (draft assessment report).
decisions, and recommendations to higher headquarters.
- Primary staff. - Special staff. - AWG personnel. - Assessment cell (if
established).
- Commander's
- Changes to the
Adapt Plans
Execution Planning
- Joint targeting. - JPP.
guidance and feedback.
operation plan and assessment plan.
- Commander. - Planners. - Primary staff. - Special staff. - AWG personnel. - Assessment cell (if
established).

Legend: AWGassessment working group
  OEoperational environment JPPjoint planning process

## 1. Communicate The Assessment And Recommendations

Regardless of quality and effort, the assessment process is futile if the communication of its results is deficient or inconsistent with the methods by which the commander assimilates information and their personal style for making decisions. While the staff will develop assessment products to communicate the assessment, assessment products are not the assessment itself.

The assessment is the staff's understanding of the OE and its impact on the plan.

The assessment derives from the staff's analysis and synthesis of indicators of changes in the OE. The degree to which the staff can explain why the OE changed the way it did also provides insights into how well did the command understand the problem set, as well as the relevancy of the subsequent COA. Most importantly, the staff's ability to explain why the current OE is changing is the singular basis for recommending changes to the plan to make operations more effective. The most critical aspect is the understanding of the current OE. That understanding makes recommendations self-evident at a broad operational level.
Any assessment products contained in the organization's assessment plan are the result of the staff's interactions with the commander over time to learn how the commander processes information as well as the demands of the mission and the unique nature of the OE. Examples of assessment products below can help the staff best inform the commander's decision making.

Assessment products include recommendations to make operations more effective. They also can inform the commander about current and anticipated conditions within the OE, evaluate the ability of the force to impact the OE, evaluate progress toward objectives and end states, provide accountability to higher authority, communicate gaps to HHQ, relay risk to the mission from those gaps, and communicate progress to external stakeholders. Table 12 shows a refined look at table 11, specifically during steps five and six during execution.

Operations
Associated Staff
Assessment Task
Activity
Personnel
Input
Output
Process
Activity
- Estimate of effects
- Commander. - Subordinate
commanders
(periodically).
- Develop an
assessment.
Execution
Communicate Feedback and
Recommendations
- Assessment report,
and
recommendations
to the commander.
on OE (draft assessment report).
- Provide relevant
recommendations to the commander.
- Primary staff. - Special staff.
- AWG personnel.
- Assessment cell (if
established).
Execution Planning
- Commander's
guidance .
Adapt Plans for Operation and
Assessment
- Changes to the
operation plan and
assessment plan,
or a new plan.
- Commander. - Planners.
- Primary staff. - Special staff. - AWG personnel.
- Assessment cell (if
- Current
Operations,
Future
Operations, Plans
produce planning
directive(s).
established).
  OEoperational environment

The commander has numerous avenues for receiving information to support decision making, among them is the communication of the assessment.

(1) Commanders and staff officers must understand that the depiction of the assessment is NOT the assessment itself. Neither is it data for analysis. Welldesigned assessment processes evaluate changes in indicators describing the
OE and the performance of organizations. They contain a rigor that is not part of the depiction because the commander does not need to see the detail of every indicator. It is the staff's responsibility to organize the data; analyze them (answer
the six questions); and concisely communicate the results of their analysis and
synthesis, i.e., the assessment results, including recommendations for improving
effectiveness to the commander for a decision. (2) The depiction of the assessment is simply a communication method designed to convey information clearly and concisely to decision makers.
Developing the timing and quality of assessment products:
(1) Analyze the operations process and staff battle rhythms to determine the appropriate interval and venue for the staff to communicate the assessment to best support planning, operations, and commander decision making. Determine
the likely method of communicating the assessment based upon the communicated information and the commander's personal preference. (2) Receiving guidance from the commander is a critical step in designing the product that communicates the assessment. Scheduling feedback mechanisms for a time when the commander is normally available is key. (3) Staffs should strive to synchronize outcome products while communicating assessments. Inclusion of various staff products gains efficiencies by possibly eliminating duplicative briefings and decision boards. It also serves to convey proper context and ensure a staff-wide dialogue with the commander. Potential attendees that are available during the communication of both formal and informal assessments include.

(a) Intelligence Representation to Communicate JIPOE and Priority Intelligence Requirements (PIRs) Linkages. Since PIRs link directly to decision points, briefing a PIR assessment can add necessary context to the assessment report. A PIR assessment should relate the ability to collect on the PIR and convey possible decision-point options that the PIR point to. (b) Fires Representative Armed with Targeting Products. Joint targeting cycle and joint integrated prioritized target list (JIPTL) results provide contextual snapshots of operations conducted for attendees not normally in the headquarters for daily battle rhythm events. Inclusion of a holistic JIPTL
review enables clear establishment and shifting of priorities beyond lethal targets. (c) Operations Representatives Armed with Commander's Planning Guidance and Operational Approach. The commander's planning guidance is an accessible reference. An operational approach review provides the opportunity for an azimuth check to reconcile previous guidance with the current assessment. (d) Other Outside Stakeholders and Key Enablers to Answer Questions.

These personnel often are not present in the headquarters on a daily basis. Attendance at an assessment brief provides the opportunity to gain a shared understanding, engage in dialogue, and eliminate ambiguity.

(e) Subordinate Commanders with their Assessments. Attendance can enrich the dialogue and eliminate ambiguity by ensuring key information and messages are not lost while staffs construct the formal assessment report. Consider monthly attendance at the lower tactical level to quarterly attendance at the higher tactical level. Attendance frequency usually depends upon the frequency of assessment cycles and how often the commander desires subordinate commanders' attendance. (f) Military Information Support Operations (MISO) and Media Representatives. MISO and media operations shape operations. Therefore, winning the battle for the narrative is essential to achieving objectives at all levels of warfare. Winning the narrative requires effective monitoring of the information environment. Inclusion of MISO and media in assessment reporting mechanisms and products enables commanders to proactively consider and direct information action to be the first with the truth, to counter enemy messaging, and focus upcoming media engagements on stories the commander wants to tell.

The following considerations apply when communicating the assessment.

(1) Assessment outcomes should generate iterative dialogue wherein the commander and staff challenge assumptions, assessments, or recommendations that are not supported by their current understanding of the OE. As a result of this discussion process, understanding of the OE improves and the quality of the assessment is enhanced. (2) The communication methods the staff selects depend upon the information presented and the preferences of the commander. Regardless of the methods, assessment products must be clear and concise. It is imperative that the communication method answers the general questions.

(3) Assessors fully document any product that leaves the headquarters so it is transparent to readers outside of the organization. When depicting assessment information on a slide, the slide should stand alone with notes, if necessary, to ensure its context. (4) Assessment products guard against known biases, including those of the commander, the staff, and the assessment cell. Avoid common biases such as silver bullets (panaceas); assumed answers (group think); news that the boss does not want to hear; over optimism; confirmation bias (making data conform to foregone conclusions); and expectation bias (what does green really mean?).

The *University of Foreign Military and Cultural Studies Red Team Handbook* and the Air Force Handbook (AFH) 33-337, *Air Force Tongue and Quill* discuss these common biases. (5) Graphic products frequently display a status and a trend of an indicator that represents a fact or judgment. Accurately differentiating between facts and judgments within the assessment enables their accurate communication. An example of a factual indicator would be counting the number of sorties flown in a week in a specified OE against enemy command and control. An example of a judgment-based indicator would be the leader's assessment of the effectiveness of those sorties, enemy command and control is degraded. Metrically, a unit can be green on all individual indicators and judged amber on the assigned task.

Assessors can use various ways to communicate assessment information. While not exclusive, the following is a list of common practices for communicating information, the appropriate use of each, and some advantages and disadvantages of each. Assessors must take care not to allow any displayed indicator to supplant the objective. In other words, the force's objective is to change the OE in support of the end state.

(1) Written Narrative.
(a) The narrative adds context and meaning to empirical information that forms the basis of the assessment result. Alone, a well-written narrative answers the general questions. However, when coupled with some form of graphic depiction of empirical information, the narrative still answers the
questions, but does so in a manner that is usually more convincing than the
narrative alone. A narrative is also the only way to express recommendations and explain risks and opportunities.
(b) A well-written narrative is difficult and time consuming to produce, because it requires logical thinking and clear, concise writing skills. It also requires time and effort on the part of the reader to understand and evaluate the ideas contained in it. Like a table, a poorly written narrative can obscure essential points by providing too much information.

(2) Oral Narrative Supported by Visual Products.
(a) Stoplight Chart (Bubble Chart).
- A stoplight chart, shown in table 13, uses several levels of assessment to depict the status of an indicator. The most common colors used are red, amber, and green, which give the chart its name. Stoplight charts are useful because, universally, commanders understand them, and stoplight charts effectively draw the commander's attention to items that require it.

| 1B Date                                                  | Line of Operation (LOO)    | Current Capability                                 |
|----------------------------------------------------------|----------------------------|----------------------------------------------------|
| Milestone Rating                                         |                            |                                                    |
| LOO #1: Support to Operations                            |                            |                                                    |
| Afghan Ministry of Defense Intelligence Policy           | 4                          | Post 2014                                          |
| Afghan Ministry of Defense Reserve Affairs               | 2B                         | 3Q,14                                              |
| 2B                                                       | 1Q, 14 Army                | Ministry of Defense Chief Constructor and Property |
| Management Division                                      |                            |                                                    |
| General Staff G2 Intelligence                            | 2B                         | 2Q, 14                                             |
| General Staff G3 Operations                              | 2A                         | 3Q, 13                                             |
| General Staff G5 Policy and Planning                     | 1B                         | Achieved                                           |
| General Staff G6 Communications                          | 2A                         | 4Q, 13                                             |
| General Staff G7 Force Structure, Training, and Doctrine | 2A                         | 3Q, 13                                             |
| Ground Forces Command                                    | 2B                         | 4Q, 13                                             |
| Afghan National Army Special Operations Command          | 3                          | 1Q, 14                                             |
| Afghan Air Force Command                                 | 2B                         | Post 2014                                          |
| Medical Command                                          | 2A                         | 4Q, 2013                                           |

## Capability Milestone Rating Legend

1A
Capable of autonomous operations.
1B
Capable of executing functions with coalition oversight only.
2A
Capable of executing functions with minimal coalition assistance; only critical ministerial or institutional functions are covered.
2B
Can accomplish its mission but requires some coalition assistance.
3
Cannot accomplish its mission without significant coalition assistance.
4
Department or institution exists but cannot accomplish its mission.
- Often, stoplight charts are an abbreviated method of providing judgments about the implications of information that may be quantifiable, such as the amount of ammunition on hand or the graduation rate of a partner nation's basic officer course. In this case, the levels need to be clearly defined and generally uniform across subordinate elements. For
example, fewer than five rifle magazines per Service member is amber or a graduation rate greater than 90 percent is green. Assessors should define required thresholds of each color during assessment framework development to increase objectivity and provide a clear understanding of requirements, rather than develop the color standards during data analysis.

- Sometimes, stoplight charts present simple information that is not easily quantifiable, but has a clear order. For example, a unit leader's judgment of the unit's ability to accomplish a tactical task as untrained, needs practice, or trained or the status of a civil affairs project as stalled, on track, or complete. - Stoplights have important limitations. For example, the simplicity of the
communication method may be mistaken for simplicity in the described system or may hide a lack of rigor in the assessment. Additionally, stoplights poorly depict a series of items where most have an indeterminate status. In other words, if all items are amber, the commander is not well informed.

## (B) Spider Or Radar Chart.

- A spider chart allows the depiction of several indicators in the same graphic. A spider chart is useful for comparing alternatives based on several criteria when measuring the criteria in the same unit (i.e., dollars or days). If a best alternative exists, it is best in all or most criteria and the
best alternative becomes obvious. If one alternative is best in one criterion and another alternative is best in some other criterion, the chart is not as useful. - Spider charts also can compare planned conditions to what actually occurred. Figure 14 compares budgeted expenditures in several categories to actual expenditures in the same period.
- The military use of spider charts to depict several ordinal indicators
simultaneously can depict change, as illustrated in figure 15. However,
one cannot directly compare across dimensions because depicted indicators are often not of the same units of measure. These ordinal depictions are the equivalent of several stoplights leaned together, like poles in a teepee, and the chart can be replaced by several stoplight charts in the same space on a product. In these situations, the spider chart appears more scientific than it is. However, the inherent uncertainty of the assessment may be better communicated and the commander's attention better directed with stoplights. - Assessors must avoid the temptation to calculate and compare the geometric areas within the lines that join the ordinal values, such as the
polygons depicted in figure 16 (see page 60). Such calculations are
meaningless.
(1) While the above tools can be used to communicate assessments, commanders may want to see a composite of the different formats on one communication tool. It is best to discuss with the commander early to determine how they best receive information, specifically assessments, and agree to a format. (2) Figure 17 depicts an example of what a division headquarters referred to as the current campaign assessment. It shows the staff's judgment of the overall assessment in the top left, provides a symbol depicting the overall assessment on top, and defines what is meant by each stoplight color and actions required for each. The bottom half of the chart depicts the staff's composite assessment for each end state, or LOO, with current objectives for each. This slide drives discussion between the staff and the commander as well as between the division commander and subordinate commanders. The details, including indicators and data, to each LOO were in subsequent hidden slides for examination should the commander ask a question.

(3) Another example of a composite assessment communication tool is displayed as figure 18 (see page 62). In this example, the objective to be completed next is on the left, moving to the right is a depiction of the last 24
hours and current assessment via a stoplight chart. Moving further to the right shows what changed in the past 24 hours under the description column followed by the data that supports the staff's judgment. (4) No matter what form the staff uses to communicate assessments, the communication methods are what the commander needs to see to make their own personal assessments.

There are a number of substandard visualization methods that are welldocumented to contribute to poor decisions or discredit the assessment processes when used improperly. These methods include:
Thermographs.

Thermographs contain a continuum of colors, normally red to green with yellow between, and the current status marked by a triangle indicating the rating. This technique fails to provide a standard to show progress, leading a staff to move progress indicators subjectively as measures of performance are achieved, not as objective verifiable effects are achieved.
Stoplights Without Standards.

The standard-less stoplight, consists of a red-amber-green scale with the absence of definitions, absolving the briefer of accountability for evaluating progress against a standard. Stoplights should provide the color definitions on the chart and a written narrative detailing the definitions in reserve.

Color Math.

Color math identifies a color for a single indicator, assigning a number value, using it as part of an index with other indicators, and then translating it back into a color. This process treats ordinal variables as continuous; the average of ordinal responses is meaningless and misleading.
Arrows without Amplifying Information.

Arrowsup, down, and sidewaysonly report the change from the last report.

Arrows show short-term advances to demonstrate progress but ignore more important trends based on mission accomplishment.
Indices.

Indices comprise a weighted average of normalized data. The purpose of an index is to have a single indicator summarizing an aspect of a problem. Indices are useful when experts agree on the weights applied to the input data and for comparing like items. Most indices are not transparent enough to provide value, such as when multiple indicators contribute to the increase or decrease, hiding key changes.

One-Hundred-Point Scales.

One-hundred-point scales source data through a survey, using a scale of 1 to 100 with the overall score being the average of the votes. This assumes that the voters have the ability to measure the variable with precision, which is not always the case.
Implementation of an Effects-based Assessment.

There are two possible problems with published efforts to implement an effectsbased assessment; it assumes a deconstructionist mentality, that is, effects roll up into intermediate military objectives (IMOs), and perceived data requirements bloat staff data collection without corresponding benefit. This sometimes results in the expectation to collect vast amounts of quantitative data; efficient assessment sections use an assessment framework to collect only data required to measure the progress of their IMOs.

SOURCE: Are We There Yet? Implementing Best Practices in Assessments,

## Military Review, MayJune 2018 Col Lynette Arnhart And Ltc Marvin King 2. Adapting Plans Or Operations

Commanders continuously visualize, describe, decide, and direct action based upon their personal assessments. Operation assessment can be solely comprised of the commander's personal assessment in a critically time-constrained environment, but the addition of the staff's assessment allows better understanding when time and circumstances permit. Figure 19 (see page 64) depicts the commander's decision cycle integrated with the operations process.

Staffs deliver assessment results by drawing on extant staff estimates and key operations process activities, notably the joint targeting process, JIPOE, and Service specific planning processes. Co-opting existing operations process activities for assessment becomes increasingly important as one moves downward through each echelon of command, due to increasingly constrained staff manning authorizations. The rate at which operation assessment encompasses JIPOE and joint targeting cycles is determined by the pace of operations and the decision type and rapidity with which the commander's decision making is required. Generally, the lower the echelon of command, the tighter the joint targeting and decision cycles become, and the more frequently commanders convene AWGs. Since assessment execution is continual, it also must be supportable and sustainable, and it must allow time for action and thoughtful analysis rather than simply requiring rote attendance at meeting after meeting.
The commander's understanding of the OE, and guidance, will drive staff actions and the actions of the entire force. The staff will disseminate the commander's updated appreciation of the OE to ensure a shared understanding throughout the force. The staff may convey, normally through fragmentary orders, any changes to the current plan, or an anticipated decision to execute a branch or sequel. Another possible decision by the commander, is to reconvene the OPT and develop a new plan, as the current plan may no longer serve as a basis for effective action.

Another requirement satisfied by the assessment process is the need to support the higher headquarters' assessment process. Because each command's mission is unique, respective assessment processes will differ. Invariably, OE perspectives, i.e., how the commander and staff understand the new problem set, will differ too.

Since the understanding of the problem set makes subsequent solutions selfevident, operations assessment creates a natural tension within the force.

Reconciling efforts should avoid compromise solutions opting instead for an enriched understanding of the evolving OE through a sharing of perspectives.

## Appendix A Connecting Outcomes To Indicators Model 1. Introduction

This model is not a rigid step-by-step approach to identifying indicators. It is a breakdown of a holistic thought processa sort of mental handrailto help people get started. As assessors develop mastery, they will perceive this process as nonlinear, very elastic, and malleable to meet the demands of the problem, operational environment, and the command. The key to using the model is to use it loosely.

The model's purpose is to posit and record the links between an outcome and the indicators used to gauge its achievement. Assessors do this for the purpose of making operations more effective. It might also be useful in operational design as it can suggest the more specific outcomes that planners will break a large problem into, which make its solution easier to execute and more likely to be successful.

To use the model, assessors always start with a statement of the desired outcome. Outcomes include: an end state, an objective, an effect, a task and purpose, a condition, a success or termination criterion, or anything else that specifies the change in the operating environment to be achieved. For an outcome to be executable, achievable, and assessable, it must be specific and bounded. Common ways to bound outcomes are by unit size, geography, or time. Another way is to make them specific, measureable, attainable, relevant, and time-bound (SMART). However, assessors or planners should not attach meaningless measures or deadlines simply to comply with SMART criteria. (For example, the objectives on many theater campaign plans will specify that all objectives will be achieved at the same time. This time is the expiration of the plan, and has no relationship to the achievement of each objective in the real world).

## 2. How To Use The Model

Figure 20 (see page 65) shows the complete model. To use it, assessors evaluate an outcome statement with question one (Q1). If the statement is specific, then they begin the process of discovering and recording the links by continuing through the model. If the statement is too vague to guide the effective operations of subordinate organizations, they break it up into a number of more specific outcomes. These more-specific outcomes statements can be phrased as questions or statements that break the original desired outcome into manageable portions for resolution. Starting with an end state, these statements may take the form of, or may suggest, lines of effort (LOEs) or lines of operation (LOOs), success or termination criteria, decision criteria, strategic questions, assessment questions, or any of the outcome statements listed above.

As each specific outcome is articulated, assessors evaluate each of these with Q1, and repeat the process just described. When each is of sufficient specificity, then they can proceed through the model.

To illustrate, if the beginning outcome statement is an end state, one iteration through the design loop in the upper right corner of the model will yield either a set of objectives or several success criteria. A second iteration through the loop may yield effects. When these statements are specific enough, assessors proceed through the remainder of the model.

For each specific outcome statement, assessors ask question two (Q2), and make a list of questions that the unit needs to answer to know it is accomplishing that specific outcome. These questions provide context and focus to the subsequent list of indicators.

Then assessors evaluate each question on this list with question three (Q3) to determine if it can be answered with empirical information. If the answer is yes, then they record the piece of information needed to answer that question as a potential indicator. If the answer to Q3 is no, then they ask more specific questions until each more-specific question can be answered with empirical evidence. Formally, these most-specific questions are called information requirements (IRs) and the evidence which answers them are indicators.

## 3. Tips On The Model's Use

Assessors should use a loose outline format to record the linkages between an end state and indicators with a less-specific question or statement close to the margin and layers of specificity indented underneath each. Successive layers can represent LOEs, objectives, effects, IRs, and indicators, respectively (or whatever other names the planners use for outcome statements).

Assessors should not try to skip the list of IRs and go straight to the list of indicators. When they do, they develop a set of all possible indicators that have bearing on the desired outcomes. This creates a couple of problems:

(1) The list of indicators is too long, lacks focus, and is hard to prioritize. (2) Assessors do not understand the indicators' relationship to each other.
The formal step of listing IRs (questions) and then identifying indicators which answer them helps assessors understand the relationship between several related indicators that may answer related questions. Also, most people find the list of questions easier to prioritize than an exhaustive list of potential indicators. In essence, the formality of posing the questions focuses the identification of indicators.

Questions (IRs) and answers (indicators) need not have a one-to-one correspondence. One indicator may answer several questions, one question may require several indicators, or several related questions may be answered by several related indicators.

Assessment is largely about answering questions about the operational environment (OE); friendly activities; and friendly, adversary, or third-party interaction with it. Once the staff answers the questions, they understand the OE better, and can make intelligent recommendations to increase the unit's effectiveness. Increasing effectiveness is the point of this model.

## 4. An Example Of Using The Model

Figures 21-32 (see pages 68-79) show an example of using the model from a blank sheet of paper to a complete example. As the model is worked through, the details are added to the original blank sheet of paper. As per figure 21, assessors start with a blank sheet of paper, and insert the end state at the top of the page to begin.

Assessors, especially at lower levels of command, may also start from an outcome statement, other than an end state, that is relevant to their unit. An outcome statement can be any of these: an end state, an objective, a sub-objective, an effect, a task and purpose, a condition, a success or termination criterion, or anything else that specifies the change(s) in the operating environment that they are seeking.

## Example

(Start with a blank sheet and write the end state at the top.)
End state: Create professional and self-sustaining security institutions for the host nation.

(1) Figure 22 shows the beginning portion of the model, the design loop, starting with the end state (or other outcome statement). In this case, a security cooperation headquarters abroad wanted to aid the partner nation in creating professional and self-sustaining security institutions, which is the end state at the top of the page.

(a) The first step is to evaluate the first outcome statement, in this case, the end state, with Q1: Is this outcome statement specific enough to guide the activities of subordinate organizations or does it need to be broken up? If the statement is specific, assessors can proceed through the model. If not, they need to break the statement into more specific statements.

(b) In this case, it is useful to break up the end state into more specific statements. A more specific outcome is any statement that increases the specificity of the outcome desired. At this point, an assessor is helping with design and planning.

(2) In this scenario, assessors conclude that achieving the five things listed in figure 22 are sufficient to accomplish the end state. They update the blank sheet to reflect these objectives as shown in figure 23.

## Example

(Now the sheet looks like this.)
End state: Create professional and self-sustaining security institutions for the host nation.

Objectives (in support of the end state):

1. Security forces are properly manned.
2. Security forces are properly trained.
3. Security forces are properly equipped.
4. Security forces are properly sustained. 5. Security forces have institutional infrastructure for sustaining 1-4.
(3)
The next step is to evaluate each new, more-specific outcome statement with Q1. Assessors do this as often as is necessary, and proceed through the model once the most-recent outcome statements are sufficiently specific. Assessors then iterate through the design loop as many times as is necessary, and proceed through the model once outcome statements are judged to be sufficiently specific. In real life, assessors would follow this procedure for all outcome statements, but for this example, the one in bold typeface is sufficient to illustrate the procedure. (4) Going back to the model, in figure 24, assessors evaluate the statement with Q1: Security forces are properly trained. They conclude it requires more breakdown, because they are not sure what properly trained means. After some critical thought, they write several more outcomes that, if achieved, are sufficient (in this example) to conclude partner forces are properly trained. Now the sheet of paper looks like figure 25.

## Example

End state: Create professional and self-sustaining security institutions for the host nation.

Objectives (in support of the end state):

1. Security forces are properly manned. 2. Security forces are properly trained.
- Training requirements for individuals and units are published. - Training courses have relevant graduation standards.
- Training evolutions are evaluated against standards.
- Requirements specify skills soldiers need to have.
3. Security forces are properly equipped.
4. Security forces are properly sustained.
5. Security forces have institutional infrastructure for sustaining 1-4.
(5) The assessors then evaluate each of the new outcome statements by Q1. For this example, they will focus on the one in bold typeface. As in figure 26, they conclude one more iteration through the design loop is required. After some critical thought, they conclude that three specific requirements are necessary, and the paper now looks like figure 27.

## Example

End state: Create professional and self-sustaining security institutions for the host nation.

Objectives (in support of the end state):

1. Security forces are properly manned. 2. Security forces are properly trained.
- Training requirements for individuals and units are published. - Training courses have relevant graduation standards.
- Training evolutions are evaluated against standards. - Requirements specify skills soldiers need to have.
- Soldiers can use weapons effectively. - Soldiers can complete common tasks. - Soldiers can identify likely improvised explosive device emplacement (IED).
3. Security forces are properly equipped. 4. Security forces are properly sustained.
5. Security forces have institutional infrastructure for sustaining 1-4.
(6) Now assessors evaluate these three new outcomes with Q1, and they conclude that the outcomes are sufficiently specific, so that they can proceed through the model as in figure 28.

(7) Assessors begin the assessment loop by posing Q2 for each specific outcome statement. They want to determine the specific questions that need to be answered for them to determine how well the joint force is accomplishing each specific outcome. (8) In this example, as seen in figure 29, they identify one or more questions that need to be answered for each outcome. The paper now looks like figure 30.

## Example

End state: Create professional and self-sustaining security institutions for the host nation.

Objectives (in support of the end state):

1. Security forces are properly manned. 2. Security forces are properly trained.
- Training requirements for individuals and units are published.
- Training courses have relevant graduation standards. - Training evolutions are evaluated against standards. - Requirements specify skills soldiers need to have.
- Soldiers can use weapons effectively.
- Can soldiers qualify with individual weapons? - Can crews qualify with crew-served weapons?
- Can junior leaders employ key weapons?
- Soldiers can complete common tasks.
- Can soldiers pass common task training?
- Soldiers can identify likely improvised explosive device (IED) emplacement.
- Can soldiers identify IEDs?
3. Security forces are properly equipped.
4. Security forces are properly sustained.
5. Security forces have institutional infrastructure for sustaining 1-4.
(9) Assessors evaluate each of these questions with the model's Q3 as in figure
31. If each question can be answered with empirical observation, assessors designate it an IR, and begin looking for a way to answer it. If it cannot be answered with empirical observation, then they ask more specific questions with the goal of asking questions that can be answered empirically. (10) Once they have that list of questions, they have their IRs, and the information that answers them empirically are indicators. Now assessors prioritize their IRs and assign collection assets. The paper now looks like figure 32.

## Example

End state: Create professional and self-sustaining security institutions for the host nation.

## Objectives (In Support Of The End State):

1. Security forces are properly manned. 2. Security forces are properly trained.
- Training requirements for individuals and units are published.
- Training courses have relevant graduation standards. - Training evolutions are evaluated against standards. - Requirements specify skills soldiers need to have.
- Soldiers can use weapons effectively.
- Can soldiers qualify with individual weapons?
- % soldiers qualified with individual weapons.
- Can crews qualify with crew-served weapons?
- % crews qualified with assigned weapons.
- Can junior leaders employ key weapons?
- Battalion Commander assessment of platoon leaders.
- Soldiers can complete common tasks.
- Can soldiers pass common task training?
- % soldiers passing common task training.
- Soldiers can identify likely improvised explosive device (IED) emplacement.
- Can soldiers identify IEDs?
- % first time go's on basic IED course.
3. Security forces are properly equipped. 4. Security forces are properly sustained.
5. Security forces have institutional infrastructure for sustaining 1-4.

## 5. Conclusion

Once assessors flesh out the outline seen in figure 32, they have recorded the logical connections between the desired outcomes and the indicators they collect and use to gauge joint force effectiveness.

Using the above model will assist assessors and planners to determine specific objectives, effects, etc., for an end state that will focus units on the things that they need to do to be most effective. It also helps assessors develop appropriate IRs and indicators for collection. By working through this model, the assessor can determine what information is needed for analysis and assessment.

## Appendix B Assessment Plan Examples 1. Introduction

This appendix provides two examples of how an assessment cell can take an operational approach and develop an assessment plan and data collection plan. These documents should then be incorporated in the operation order (OPORD) annex or appendix. The first example has been adapted from II Marine expeditionary force's
(MEF's) assessment cell while the second example is from the Naval War College's (NWC's) College of Maritime Operational Warfare.

## 2. Ii Mef Example

This example was used by the II MEF during Large Scale Exercise 2017. II MEF
established an assessment cell within the staff comprised of three full-time assessors while each staff and functional section provided representation. The deputy commander chaired the working groups to provide command oversight. This method was effective during the exercise, and chronologically in paragraphs 3-7.

Figure 33 (see page 82) is an example of an operational approach. The operational approach is developed throughout planning and is finalized during order production. The assessment cell is a key contributor, as it is developed throughout planning, providing input on the ability to properly assess the lines of effort (LOEs), lines of operation (LOOs), decision points, and ensure that the framework supports the commander's end state. If a higher headquarters (HHQ) does not provide an operational approach, one can be developed from the OPORD to assist planners and assessors. For this example LOE 1: Legitimacy of Operations is used.

## 3. Developing The Assessment Plan From The Operational Approach

As the operational approach is being developed, the assessors are providing input as how potential LOEs, LOOs, and decision points can be assessed. Assessors will also determine if the LOEs, LOOs, and decision points are feasible in addressing the center of gravity and getting to the commander's end state. As the LOEs, LOOs, and decision points are adapted, the assessment cell, with the assistance of the assessment working group, will develop the assessment plan. Figure 34 is an example of objectives and effects developed in support of LOE 1; Legitimacy of Operations.

# Objectives And Effects, Line Of Effort 1: Legitimacy Of Operations.


1.1. Support from Country A.

1.2. Control the narrative.

1.3. Support from international community.

1.4. Coalition maintained.
During the development of objectives and effects for a line of effort or line of operation, and their associated decision points, the assessment cell can provide an initial list, and then have the assessment working group review and revise, or it can all be done in an assessment working group. Time will drive if all the lines of effort, lines of operation, and decision points are developed together or individually.

## 4. Indicators

Once the objectives or effects are finalized and approved, the next step is to identify the indicators that need to be collected for the assessment of the objectives or effects and decision points that are part of the operational approach. After reviewing and revising the indicators, the indicators are finalized and incorporated into the assessment plan. Figure 35 (see page 84) is an example of indicators developed in support of a single objective or effect in support of LOE 1; Legitimacy of Operations.

Note: The more specific the indicators are written, the more detailed information is returned from the collectors. Another way to phrase the statements in figure 35 is to ask specific questions for each indicator (e.g., Is there evidence that messaging efforts against enemy forces X and Y are viewed as legitimate by country A's government, or the coalition?) so that they can be answered with either a binary, ordinal, or short description answer. This is particularly true if assessors are not clear yet what evidence of success or failure looks like. If an ordinal scale is used, assessors should develop definitions for the scale that are used by all collectors. Each indicator may require a different definition to measure success on an ordinal scale. See figure 13 for an example of rating definitions.

## 1.1. IndicatorsSupport From Country A Government. Indicators

1.1.1.  Messaging coalition efforts against enemy forces X and Y viewed as legitimate by Country A government, coalition, and United Nations. 1.1.2.  Country A face on successful operations against enemy forces X and Y. 1.1.3.  Country A forces capable of defeating enemy X elements with minimal support. 1.1.4.  Key Country A clans support Country A government and coalition actions. 1.1.5.  Local Country A population will not significantly affect coalition operations. 1.1.6.  Country A government maintains support of military and police.

1.1.7.  Country A able to support nongovernmental organization access with minimal coalition support to address humanitarian assistance needs. Numerous indicators will be developed in support of the objectives and effects. The assessment team will need to review them, as some may not be measurable, collectable, or feasible, while some may be merged with others. There is no set number of indicators. Be aware of the effort and capability of the assessment team to track them all when viewed in context of the whole assessment plan.

## Figure 35. Example Indicators In Support Of Loe 1

The assessment plan can be developed within a series of PowerPoint slides as provided above, or within a Word document. A technique is to develop within a PowerPoint construct so that it can be incorporated as back up information to support the commander's assessment brief. If the assessment cell is required to write an assessment annex or appendix, a PowerPoint product can be incorporated in the document, along with the data collection plan.

## 5. Data Collection Plan

As the assessment plan is developed, the data collection plan may also be developed to incorporate requirements to support assessment. The data collection plan identifies the sources, and staff support, to collect the indicators that are identified to support the objectives or effects. Table 14 is an example of a data collection plan template. This example includes a column to track the assessments as conducted over time. Bullets of major assessment points can be incorporated and as the next assessment is done, the assessment cell can refer back to the data collection plan to see the history. The data collection plan should be tailored for the particular assessment, leveraging quantitative data, or narrative assessments where required.

| Assessment                                                                    | Line of                                         |
|-------------------------------------------------------------------------------|-------------------------------------------------|
| Effort                                                                        |                                                 |
| Objective                                                                     |                                                 |
| or Effect                                                                     |                                                 |
| Indicators                                                                    | Sources                                         |
| Staff                                                                         |                                                 |
| Support                                                                       | Baseline                                        |
| - Embassy                                                                     | - PAO                                           |
| - POLAD                                                                       | - Local                                         |
| newspaper, radio,                                                             |                                                 |
| or TV                                                                         |                                                 |
| - Coalition                                                                   |                                                 |
| LNOs                                                                          |                                                 |
| - Coalition                                                                   |                                                 |
| 1.1.1. Messaging                                                              |                                                 |
| efforts against                                                               |                                                 |
| enemy forces X                                                                |                                                 |
| and Y viewed as                                                               |                                                 |
| legitimate by                                                                 |                                                 |
| Country A,                                                                    |                                                 |
| coalition, and the                                                            |                                                 |
| United Nations.                                                               |                                                 |
| - International                                                               |                                                 |
| news                                                                          |                                                 |
| - Embassy                                                                     | - PAO                                           |
| - Local news                                                                  | - POLAD                                         |
| 1.1. Support                                                                  |                                                 |
| from Country A.                                                               |                                                 |
| Line of Effort                                                                |                                                 |
| 1: Legitimacy                                                                 |                                                 |
| of Operations.                                                                |                                                 |
| - SOFLE                                                                       |                                                 |
| 1.1.2. Country A                                                              |                                                 |
| face on successful                                                            |                                                 |
| operations against                                                            |                                                 |
| enemy forces X                                                                |                                                 |
| and Y.                                                                        |                                                 |
| - Survey of                                                                   |                                                 |
| population                                                                    |                                                 |
| - SOF                                                                         |                                                 |
| - Coalition                                                                   |                                                 |
| LNOs                                                                          |                                                 |
| - HN                                                                          | - PAO                                           |
| - Embassy                                                                     | - POLAD                                         |
| - SOF                                                                         | - SOFLE                                         |
| 1.1.3. Country A                                                              |                                                 |
| forces capable of                                                             |                                                 |
| defeating enemy X                                                             |                                                 |
| elements with                                                                 |                                                 |
| minimal coalition                                                             |                                                 |
| support.                                                                      |                                                 |
| Note: The data collection plan would have all applicable indicators included. |                                                 |
| Legend:                                                                       |                                                 |
| HNhost nation                                                                | POLADpolitical advisor                         |
| LNOliaison officer                                                           | SOFspecial operations forces                   |
| PAOpublic affairs officer                                                    | SOFLEspecial operations forces liaison element |

Note: Additional information may be added to the data collection plan in table 14. Some organizations add columns titled Metrics or Details in between Indicators and Sources to fully understand and specify the information required. Other organizations omit the Assessment Baseline column and add a column for Tasked Unit or Asset to indicate the collection plan has been formally tasked in a mission type order.

## 6. Decision Points

For decision points, the focus would be on the indicators to support that decision point. Figure 36 (see page 86) is a snapshot of decision-point indicators that would be tracked by the assessment cell to support the commander's decision to conduct an amphibious landing.

## Decision Point 1Conduct Amphibious Landing. Indicators

Note: Each of the indicators in figure 36 would have specific details required to define each indicator. An example is defining significant enemy combat power from indicator 1.2. Another example is defining minimum required aircraft required to support the landing, greater specifying the details to indicator 1.5.

## 7. Commander's Decision Brief

There is no right or wrong technique, as long as it provides the commander the information needed. Each commander receives information and analyzes information differently. If the assessment cell gets good guidance and requirements from the commander on what they want in the assessment brief, it will ensure that these items are included in the assessment plan and data collection plan.

For this example, the commander has decided on a single slide for the assessment brief. It can be incorporated into the normal operation and intelligence brief on an as-needed basis, or provided as part of the planners' update, or as a separate assessment board battle rhythm event. In this example, the assessment cell has decided to build the brief to take the commander to specific indicators using hyperlinks. This allows the commander the ability to drill down into the assessment if there are questions.

Figure 37 takes the brief down to the indicators for each of the objectives or effects. In this template, the focus is on a trending construct with more narrative. This template can also be used for the commander's baseline slide. The slide provides each LOE or LOO, and then an assessment.

## 1.1. IndicatorsSupport From Country A Government.

| Indicator                                        |                                      |
|--------------------------------------------------|--------------------------------------|
| Current Situation:                               | 1.1.1.  Messaging coalition efforts  |
| against enemy forces X and Y                     |                                      |
| viewed as legitimate by Country A                |                                      |
| government, coalition, and United                |                                      |
| Nations.                                         |                                      |
|                                                  |                                      |
| 96 hours out:                                    |                                      |
| Current Situation:                               | 1.1.2.  Country A face on successful |
| operations against enemy forces X                |                                      |
| and Y.                                           |                                      |
|                                                  |                                      |
| 96 hours out:                                    |                                      |
|                                                  |                                      |
| 1.1.3.  Country A forces capable of              |                                      |
| defeating enemy X elements with                  |                                      |
| minimal support.                                 |                                      |
|                                                  |                                      |
| Current Situation:                               |                                      |
| (Insert narrative assessment of                  |                                      |
| the current situation and the assessment working |                                      |
| group prediction of 96 hours out)                |                                      |
|                                                  |                                      |
| 96 hours out:                                    |                                      |
|                                                  |                                      |
| Current Situation:                               | 1.1.4.  Key Country A clans support  |
| country A government and coalition               |                                      |
| actions.                                         |                                      |
|                                                  |                                      |
| 96 hours out:                                    |                                      |
|                                                  |                                      |
| Current Situation:                               | 1.1.5.  Local Country A population   |
| will not significantly affect coalition          |                                      |
| operations.                                      |                                      |
|                                                  |                                      |
| 96 hours out:                                    |                                      |
|                                                  |                                      |
| Current Situation:                               | 1.1.6.  Country A government         |
| maintains support of military and                |                                      |
| police.                                          |                                      |
|                                                  |                                      |
| 96 hours out:                                    |                                      |
|                                                  |                                      |
| Current Situation:                               | 1.1.7.  Country A able to support    |
| nongovernmental organization                     |                                      |
| access with minimal coalition support            |                                      |
| to address humanitarian assistance               |                                      |
| needs.                                           |                                      |
|                                                  |                                      |
| 96 hours out:                                    |                                      |
|                                                  |                                      |

## 8. Nwc, College Of Maritime Operational Warfare Example

This section provides an example of how the maritime operational planner's course exposes planners to operations assessment planning to stimulate the development of an assessment plan concurrent to operational planning. There is both an art and a science to assessment planning. This process aligns with the first two steps of the overall six-step operation assessment process which are: develop the assessment approach and develop the assessment plan. The assessment plan is developed using four major steps.

(1) Step One. Understand and leverage design and intelligence planning efforts.
(2) Step Two. Analyze objectives and effects. (3) Step Three. Develop indicators to support recognition of effectiveness and efficiency (to assist in drawing evidenced-based conclusions for
recommendations during execution).
(4) Step Four. Develop an assessment appendix as well as a concept of support
for assessments in the base directive to guide execution.
Steps one and two form the art piece. Steps two and three transition from the art to the science, to form the basis for the development of the collection plan. Step four is the outcome of this planning with an assessment appendix, a supporting collection plan, and a concept of support to insert into the base directive. Figure 38 depicts these four steps aligned with the naval planning process.

Note: For explanation purposes, a fictitious maritime stability operation in Somalia is used to partially demonstrate the steps.

## Navy Planning Process

Course of
Action
Course of Action
Comparison and
Design
Mission
Analysis
Development
Course
of Action
Analysis
Decision
Directive Development

## Assessment Planning Process

4. Develop the assessment
appendix as well as the
concept of support for the
base directive:
2. Analyze objectives and
effects:
What does success look like? What does the commander want to focus on?
3. Develop indicators to
support recognition of effectiveness and efficiency:
Synchronize with collection management board.
1. Understand and
leverage: Design and Intelligence
efforts: Environment
(PMESII) and operational approach.
IPOE: Systems thinking
(Threat).
Planning Process: Plan
logic and decision support.

[---------Develop Collection Plan (Tab A to Assessment Appendix)---------]
[------------------Art Piece (Understand; Conceptual)------------------]
[--------------------Science Piece (Detailed)---------------------]

Legend:
IPOEintelligence preparation of the operational environment PMESIIpolitical, military, economic, social, information, and infrastructure

## Figure 38. Integrating Assessment Planning Concurrent To Operational Planning

Assessment planners leverage several processes during planning efforts to add rigor to the process. From design efforts, assessment planners gain an increased understanding of the environment as well as the operational approach, particularly with respect to objectives and effects. Planners leverage intelligence work performed to produce systems analysis both in the environment and from a threat perspective. Planners also leverage staff estimates and the outcomes of operational planning.

At the beginning of the process, assessment planners seek guidance from the commander to guide the development of the assessment plan. Data can be organized several ways to ensure the commander understands the assessment by:
end state and objectives, phase intermediate objectives and effects, geography and time, and for decision support.

## 9. Step One. Leverage Design And Intelligence Planning Efforts

The design process provides assessment planners increased understanding of the environment and of the operational approach. Planners should participate in design to understand the current state and the future desired state. Design teams may use various approaches to understand the current environment. An approach is to understand the current conditions and analyze relationships, actors, functions, and tensions. Another approach is to determine the current state in terms of political, military, economic, social, information, and infrastructure.

Assessment planners leverage intelligence products to be aware of who the threats are, where the threats operate, and what equipment and capabiltities the threats have. This type of systems thinking leads to better discernment of indicators later in the process.

## 10. Step Two. Analyze Objectives And Effects

Assessment planners analyze identified friendly objectives and effects by asking a series of questions. These questions are:

(1) Do the effects describe success as it relates to the objectives?
(2) Are the effects relevant and measurable? (3) What refinements would you make (additions, deletions, or restatements)
with respect to the effects? (4) Are the tasks (as a result of task analysis during mission analysis) likely to create the desired effects and accomplish the objectives?
As the assessment team analyzes objectives and effects, the assessment team may determine that some of the desired effects cannot effectively be measured. The team will then recommend changes to the desired effects to ensure the same overall effect is achieved, but in a measurable way. An example is if the original effect was: human trafficking and smuggling of arms, ammunition, and goods are reduced. This effect can be broken into two measurable effects as: human trafficking is reduced and smuggling of arms, ammunition, and goods into country A is reduced. This example could be broken down further into actual measurable effects to ensure reporting is accurate, which creates better assessments.

## 11. Step Three. Developing Indicators

Indicators that measure effectiveness are measures of outcome as the result of action. Indicators that measure efficiency are measures of input that measure whether assigned actions are being executed, to what degree, and by what amount of resources. The indicators must be relevant, able to be resourced, collectible, and reported by some established means. Indicators that are critical to the discernment of progress or efficiency that cannot be collected for any reason may result in added operational risk.

One technique is to ask a series of questions to recognize effectiveness of the force. By asking questions, a degree of specificity emerges. In this example, a series of questions is proposed to answer: How will we know we are achieving this objective? By asking questions to ascertain the recognition of a desired effect, intelligence and information requirements can be derived that may translate into potential indicators. In figure 39 (see page 90), questions are formed to ascertain whether pirates and terrorists are targeting humanitarian aid shipments into country, whether their actions are increasing or decreasing because of the force's presence, and whether the planned amount of aid is actually reaching ports designated to accept this shipping. In execution, the same data points over time may lead to form evidenced-based conclusions to determine whether the force is effective and successful or whether adjustment is required by changing ways or means.

After narrowing down the measures of effectiveness (MOEs) and measures of performance (MOPs), the collection plan needs to ensure that the intelligence and information requirements are collected on. This may be accomplished at a collection management board to determine what can be sourced. The potential result are indicators that are relevant, collectible, and resourced. Unsourced or unsupported indicators may be considered as added operational risk.

Another technique is to leverage systems thinking from intelligence preparation of the operational environment products to discern potential indicators that may demonstrate evidence of change over time. This technique reviews the objective and related effect, reviews enemy systems, then makes inferences based on system analysis to ascertain desired change over time. This critical thinking requires the active participation of members and the exchange of different perspectives for indicator development. Assessment groups must be proactive to form their own perspectives; figure 40 depicts this technique. A review of the objective, effect, the environment, and specifically, the threat system that can prohibit accomplishing the desired effect and prevent achieving the objective leads to brainstorming and discernment of potential derived indicators. This information is brought forward to the collection management board battle rhythm event for consideration and sourcing.

## Figure 40. Developing Indicators From Environment And Threat Systems

The product produced as a result of these efforts forms an initial collection plan that can be used to begin data collection during execution. This plan will evolve as the environment or objectives change in execution. Other refinements may include indicators that are not effective in reaching evidenced-based conclusions as a basis for recommendations. Figure 41 depicts an example initial data collection plan.

TIME
OBJECTIVE
EFFECT
INDICATOR
ACCEPTABLE
CONDITION IF
APPLICABLE
INFORMATION
IS OF VALUE
100,000 per
week
Phase IIV
# of tons of
aid arriving
as expected
in PORT
XXX.
Aid delivered via humanitarian shipping is defined as 100,000 tons per week as an expectation to fill the need. A
typical medium carrier carries
from 15,000 to 18,000 tons on average.
7 per month
Phase IIV
The acceptable rate per week to meet the need is 7 based on a 15,000-ton capacity per ship.
# of aid
ships
arriving as
expected in
PORT XXX.
Terrorists
and pirates
do not
threaten
humanitarian
aid
shipments
into XXX.
0
Phase IIV
# of
reported
attacks in
operational
area A.
Approaches to XXX are defined as the line of communication out to the 12-mile line. Incidents are defined as any event that delays shipping freedom of movement to port.
0
Phase IIII
# of
reported
incidents in
operational
area B.
These incidents are defined as specific piracy events targeting shipments along the entire length of the coast out to 220 nautical miles.
NOTES AND EXPLANATIONS
SOURCE
FREQUENCY
DATA
(DEFINE TERMS FOR
OF
FORMAT
COMMON EXPECTATIONS)
COLLECTION
COLLECTED BY
Embassy
Number
Weekly
Current
Operations
Embassy
Number
Weekly
Current
Operations
Higher
Headquarters
Number
Weekly
Intelligence
Higher
Headquarters
Number
Weekly
Intelligence

Developing indicators to measure efficiency of the force is derived from a determination of critical tasks that are important to measure based on commander's guidance.

## 12. Step Four. Develop The Appendix And Concept Of Support

Upon conclusion of the planning process, the output is an order or directive. Included in the order or directive is the assessment appendix, the collection plan, and the concept of support. These three items work together to create inputs to the assessment cell that are required to complete the act of assessing. Examples of assessment appendices are found in appendix C.

## Appendix C Example Annexes And Appendices

This appendix will provide examples from each Service planning and orders formats for assessments.

## 1. United States Army

Field Manual (FM) 6-0, *Commander and Staff Organization and Operations*, provides a doctrinal operation order (OPORD) format and provides fundamental considerations, formats, and instructions for developing Annex M, (Assessment) Format and Instructions. Commanders and staffs use Annex M as a means to quantify and qualify mission success or task accomplishment. This annex describes the assessment concept of support objectives. For more information refer to FM 6-0.

## 2. United States Marine Corps

The Marine Corps Planning Process (MCPP), does not specify a format to articulate assessments, but does recognize the need to accomplish assessments. The MCPP defines design as the fundamental responsibility of the commander during planning, but also throughout the planning-execution-assessment continuum. It stresses the importance of understanding the problem, the environment, the enemy, and the purpose of the operation. The stress on the continuum states the importance of assessments from the planning process through execution, managing information on the environment, the enemy, and friendly forces in terms of the purpose of the operations. For more information see Marine Corps Warfighting Publication 5-10, Marine Corps Planning Process.

## 3. United States Navy

The method of communicating the assessment framework to the staff, higher headquarters, other components, and subordinates may vary. One proposal includes an annex to appendix C of the base operation order. It may also include the assessment organization, offices of primary responsibility, and concept for assessment. This example includes objectives, effects, measures of effectiveness, and collection responsibilities. See Navy Warfare Publication 5-01, Navy Planning for more information.

Another example of a Navy assessment appendix example, in table 15, is taught at the Naval War College at the maritime operational planner's course.

OFFICIAL DESIGNATION OF COMMAND (Record the name of the command.) PLACE OF ISSUE (Record where the directive was issued from.)
DATE TIME GROUP (Record the date issued.) MESSAGE REFERENCE NUMBER (Record if applicable.)
APPENDIX XX, OPERATION ASSESSMENT TO ANNEX C, OPERATIONS, TO OPERATION ORDER "OPERATION NAME" (U) REFERENCES: (Record references used when developing the appendix.) (U) Warning Order XXXX
(U) Decision Directive XXXX (U) Memorandum XXXX (U) Message XXXX

## Situation

1. (U) General. (Insert a short statement paragraph describing the situation or record this statement.) "REFER TO BASE OPERATION ORDER, SITUATION
PARAGRAPH."
2. (U) Area of Operations. (Insert a short statement paragraph describing the area of operations or record this statement or record this statement.) "REFER TO ANNEX B, INTELLIGENCE."
3. (U) Enemy Forces. (Insert a general statement paragraph describing enemy forces or record this statement.) "REFER TO ANNEX B, INTELLIGENCE."
4. (U) Friendly Forces. (Insert a general statement paragraph describing friendly forces or record this statement.) "REFER TO BASE ORDER, SITUATION PARAGRAPH." 5. (U) Civil Considerations. (Insert a general statement paragraph describing civil consideration applicable to the force or record this statement.) "REFER TO ANNEX G
CIVIL AFFAIRS."
6. (U) Attachments and Detachments. (Insert a statement paragraph describing forces attached or detached for the operation or record this statement.) "REFER TO ANNEX
A TASK ORGANIZATION." MISSION. (Record this statement.) "REFER TO BASE ORDER, MISSION PARAGRAPH."

## Execution

7. (U) Concept of Operations Assessment. (Provide a short lead in paragraph that describes what the assessment will achieve and who may contribute to that effort)

## Table 15. Navy War College Assessment Appendix Example (Cont'D) 8. A. (U) Phase I:

8. A. 1. (U) Purpose. (Record applicable objectives and associated effects in this section that the assessment will focus on to gauge effectiveness of the force.) Record what critical tasks the assessment will focus on to gauge efficiency of the force. (Record how long and if applicable, under what circumstances this particular effort will last). The purpose of the assessment in this phase is to measure the effectiveness and efficiency of the force in that (objective) XXXX. The desired conditions (effects) are YYYY. 8. A. 2. (U) Focus. (Record some core measures the assessment will concentrate on to gauge effectiveness of the force.) The commander will evaluate the effectiveness of the force in this phase by monitoring and evaluating XXXX. The commander will evaluate the efficiency of the force in this phase by monitoring and evaluating ZZZZ. 8. B. (U) Phase II: 8. B. 1. (U) Purpose. (See 8.A.1 above and use this framework) 8. B. 2. (U) Focus. (See 8.A.2 above and use this framework) 8. C. (U) Phase III: 8. C. 1. (U) Purpose. (See 8.A.1 above and use this framework) 8. C. 2. (U) Focus. (See 8.A.2 above and use this framework) 8. D. (U) Phase IV: 8. D. 1. (U) Purpose. (See 8.A.1 above and use this framework) 8. D. 2. (U) Focus. (See 8.A.2 above and use this framework) 8. E. (U) Phase V: 8. E. 1. (U) Purpose. (See 8.A.1 above and use this framework)
8. E. 2. (U) Focus. (See 8.A.2 above and use this framework)
9. (U) Data Collection. (Record a statement similar to this describing relationships required for data collection and, if feasible, how that is accomplished). Requirements for data collection are synchronized through the agreed collection, data sharing process, and the data collection manager. The operations assessment cell, in coordination with the intelligence and current operations, synchronize requirements and agrees to data collection management sharing for threat assessment, friendly force monitoring and task completion information required, and assessment evaluation to facilitate evidenced based conclusions. 10. (U) Coordinating Instructions. (Record any information that applies to two or more organizations of the force. This can include battle rhythm events and times, assessment working group organization, or any other information to be shared that does not fit in other parts of the appendix).

## Admin And Log

11. A (U) Administrative. (Record this statement.) "REFER TO ANNEX D
(LOGISTICS)."
11. B. (U) Logistics. (Record this statement.) "REFER TO ANNEX D (LOGISTICS)."
COMMAND AND CONTROL 12. A. (U) Command. (Record this statement.) "REFER TO THE BASE ORDER, COMMAND PARAGRAPH." 12. B. (U) Liaison Requirements. (Record any liaison requirements internally to the assessment cell or externally to other maritime operations center cross-functional teams.) 12. C. (U) Control. (Record this statement.) "REFER TO ANNEX R (REPORTS)." Tab 1Collection Plan. (See figure 42 for an example of the collection plan.)

## 4. United States Air Force (Usaf)

The USAF does not specify a doctrinal assessment product. The USAF allows commanders and their staffs to develop a product that works for the commander.

## 5. North Atlantic Treaty Organization (Nato)

NATO established annex OO in table 16 as part of their standard six-paragraph operation order format. For more information see the NATO Operations Assessment Handbook, version 3.0.

## Table 16.  North Atlantic Treaty Organization Annex Oo Example 1. Situation

a. General. Introduction to operations assessment, its purpose within the headquarters (HQ), relationship to the plan, and key references used in the design of the assessment plan.

b. Purpose. The purpose of the annex.

## 2. Mission

A clear, concise statement which states the operations assessment mission, with a clear purpose in support of the commander's decision making.

## 3. Concept Of Operations

a. General Concept for Operations Assessment. The general overview of the assessment to be conducted including the measures of effectiveness (MOEs) and measures of performance (MOPs), data collection, how the data is analyzed to develop outputs, where the assessments are used, and what decisions the assessments are likely to support. Include reference to how lessons learned are captured and the assessment refined.

b. Operation Assessment Model or Process. A schematic drawing representing an overview of the process of operations assessment within the command.

c. Operations Assessment Results. How will the assessment products be presented? Where and who will use the output from the assessments?

d. Data Collection Plan. Reference to how data is collected using the data collection plan detailed in appendix I.

## 4. Execution

a. Operations Assessment Battle Rhythm. How the operations assessment is executed with a battle rhythm and its relationship with the wider HQ battle rhythm.

b. Coordinating Instructions.

i. Subordinate Command Actions. Actions or responsibilities for subordinate commands.

ii. Supporting Command Actions. Actions or responsibilities for supporting commands.

iii. Host-nation Requests. Requests to host nation for support. Identify overlaps with host-nation assessment capabilities.

iv. Civilian-organization Requests. Requests to civilian organizations for support. Identify overlaps with civilian-organization assessment capabilities.

## Table 16. North Atlantic Treaty Organization Annex Oo Example (Cont'D)

c. Use of Tools for Operations Planning Functional Area Services (TOPFAS) or other operations assessment related software. How the assessment is executed using software applications, including databases and tools such as the campaign assessment tool within TOPFAS.

## 5. Service Support

Financial Management Support. All Service contracts are to be established conducting an operations assessment, cost-based analysis, and using the fiscal triad, composed of resource management, contracting services, and finance operations assets through the lens of the legal office.

## 6. Command And Signal

a. Command and Control. Describe the relationship with other assessment cells. b. Liaison and Coordination. Describe how to deal with issues and who the key
points of contacts are within the command.
c. Reporting. Detail key reports and timings for submission.
SIGNATURE BLOCK APPENDIX LIST

## Appendix IData Collection Plan

Annex OO write very specific MOEs and MOPs. As the plan reviews, the annex may become obsolete in some essential aspects, requiring adjustment through mechanisms other than the plan review. Bearing this in mind, a plan which includes the following information for the purposes of data collection:
MOE or MOP with associated planning elements such as operational objective, decisive condition, supporting effect, task, etc. Include all reference numbers.

Detailed description of MOE or MOP including definitions.

Goals of the MOE or MOP. Type of data being collected (including units of measurement).

Data source.

Office of primary responsibility for data collection.

Data format to be reported in. Frequency data to be reported.

## References Joint Publications

CJCSM 3105.01, *Joint Risk Analysis,* 14 October 2016
DOD Dictionary of Military and Associated Terms, November 2019

JP 1, *Doctrine for the Armed Forces of the United States,* 25 March 2013 Incorporating
Change 1, 12 July 2017
JP 2-0, *Joint Intelligence*, 22 October 2013 JP 3-0, *Joint Operations,* 17 January 2017 JP 3-13.3, *Operations Security*, 6 January 2016
JP 5-0, *Joint Planning*, 16 June 2017

## Army

ADP 5-0*, The Operations Process,* 31 July 2019

FM 6-0, *Commander and Staff Organization and Operations,* May 2014 Incorporating
Change 2, 22 April 2016

## Marine Corps

MCWP 5-10*, Marine Corps Planning Process,* 2 May 2016

## Navy

MILSTRIP/MILSTRAP Desk Guide, Naval Supply Systems Command Publication 409,
May 2003
NTRP 1-02, *Navy Supplement to the DOD Dictionary of Military and Associated Terms*,
October 2019
NWP 5-01, *Navy Planning,* December 2013

## Air Force

AFH 33-337, *Air Force Tongue and Quill,* 27 MAY 2015, Certified Current 27 July 2016
AFI 33-360, *Publications and Forms Management,* 1 December 2015

## Other Publications

Kilcullen, David. *Counterinsurgency*, Oxford University Press, 2010
Kilcullen, David. The Accidental Guerrilla: Fighting Small Wars in the Midst of a Big
One, New York: Oxford University Press, 2011
Military Review, Are We There Yet? Implementing Best Practices in Assessments,
MayJune 2018
NATO Operations Assessment Handbook, version 3.0, 1 July 2015

The Red Team Handbook*, The Army's Guide to Making Better Decisions,* version 9.0,
May 2019

## Suggested Reading

Agoglia, John, Michael Dziedzic, and Barbara Sotirin. Measuring Progress in Conflict
Environments (MPICE): A Metrics Framework. Defense Technical Information Center, 2010
CJCSI 3100.01D*, The Joint Strategic Planning System*, 20 July 2018

Claflin, Bobby, Dave Sanders, and Greg Boylan, Improving Analytical Support to the
Warfighter: Campaign Assessments, Operational Analysis, and Data Management,
Military Operations Research Society, 2010
Commander's Handbook for Assessment Planning and Execution, Joint Staff, 2011
Connable, Ben, Embracing the Fog of War: Assessment and Metrics in
Counterinsurgency, RAND Corporation, 2012
Diehl, Paul F., and Daniel Druckman, *Evaluating Peace Operations.* Lynne Rienner
Publishers, 2010
DODD 7045.14, *The Planning, Programming, Budgeting, and Execution Process,* 25
January 2013
Downes-Martin, Stephen. *Operations Assessment in Afghanistan is Broken,* Naval War
College Review 64.4, 2011, pp. 103-125
Fitzpatrick, Jody L, James R Sanders, and Blaine R Worthen, Program Evaluation:
Alternative Approaches and Practical Guidelines. 4th ed. Boston: Pearson, 2011
Flynn, Michael T., Matt Pottinger, and Paul D. Batchelor. Fixing Intel: A Blueprint for
Making Intelligence Relevant in Afghanistan, Center for New American Security, 2010
Insights and Best Practices Focus Paper: Assessment, Second Edition, Joint Staff J7,
2013
LaRivee, Dave, Best Practices Guide for Conducting Assessments in
Counterinsurgencies, Small Wars Foundation, 2011
Marquis, Jefferson P., Michael J. McNerney, S. Rebecca Zimmerman, Merrie Archer,
Jeremy Boback, and David Stebbins, Developing an Assessment, Monitoring, and Evaluation Framework for U.S. Department of Defense Security Cooperation, RAND Corporation, 2016
Mattis, James N., USJFCOM Commander's Guidance for Effects-Based Operations,
Joint Forces Quarterly, 2008
MORS Special meeting: Assessments of Multinational Operations: Report on
Proceedings, MacDill AFB, Tampa, FL, 5-8 November 2012
Mushen, Emily, and Jonathan Schroden, Are We Winning? A Brief History of Military
Operations Assessment. CNA Analysis and Solutions, 2014
Operations Assessment (OA) Process at the Operational Level (NOAH V3.0), NATO,
Assessment Staff at MN JHQ Ulm, September 2016
Participation in the PPBE process, CJCSI 8501.01B. DoD, 2012
Roginski, J.W., *Operational Assessments in the Garrison Environment, Infantry,* v104
n2, Apr-Jun 2015
Rowlett, Rick; Young Carl A.; Mangan, Alan F.; and Townsend, Steven M., The Way
Ahead for Joint Operations and Planning Doctrine. Joint Forces Quarterly, 77.2,
2015
Schroden, Jonathan., *Why Operations Assessments Fail.* Naval War College Review
64.4, 2011
Schroden, Jonathan; William Rosenau; and Emily Warner. Asking the Right Questions:
A Framework for Assessing Counterterrorism Actions. CNA Analysis and Solutions, 2016
Schroden, Jonathan, et al. A New Paradigm for Assessment in Counter-insurgency,
Military Operations Research 18.3, 2013
Schroden, Jonathan*, A Best Practice for Assessment in Counterinsurgency,* Small Wars
& Insurgencies 25.2, 2014
Shilling, Adam, A Quick Reference Guide to the New Paradigm of Operation
Assessment. 2016
Shilling, Adam, *Assessment Training Position Paper,* 2016
Tufte, Edward R, *The Visual Display of Quantitative Information*, Graphics Press, 1983
Upshur, W., J.W. Roginski, D. Kilcullen, *Recognizing Systems in Afghanistan,* Prism, v3
n3, June 2012
Wholey, J. S.; H. P. Hatry; and K. E. Newcomer, eds, Handbook of Practical Program
Evaluation, 3rd ed. San Francisco, CA: John Wiley & Sons, 2010
Williams, Andrew; James Bexfield; Fabrizio Fitzgerald Farina; Johannes de Nijs, eds.
Innovation in Operations Assessment: Recent Developments in Measuring Results
in Conflict Environments, NATO Communications and Information Agency, 2013
This page intentionally left blank.

## Glossary Part IAbbreviations And Acronyms

A
ABCT
armored brigade combat team
ADA
air defense artillery
ANSF
Afghan National Security Forces
AOA
amphibious objective area
APOD
aerial port of debarkation
AWG
assessment working group
B
BDA
battle damage assessment
BDE
brigade
BN
battalion
C
C2
command and control
CA
civil affairs
CAB
combat aviation brigade
CATF
commander, amphibious task force
CFMCC
combined forces maritime component commander
CLF
commander, landing force
CMD
command
COA
course of action
COFMS
correlation of force and means
COMISAF
commander, International Security Assistance Force
CR
cavalry regiment
D
DOD
Department of Defense
DR
disaster relief
E
ECA
enable civil authorities
F
FA
field artillery
FLOT
forward line of own troops
FM
field manual
FRAGORD
fragmentary order
G
GIRoA
Government of the Islamic Republic of Afghanistan
H
HA
humanitarian assistance
HHQ
higher headquarters
HN
host nation
HQ
headquarters
HUMINT
human intelligence
HVI
high value individual
I
IBCT
infantry brigade combat team
IDP
internally displaced persons
IED
improvised explosive device
IGO
inter-governmental organization
IMO
intermediate military objective
INT
intelligence
IPOE
intelligence preparation of the operational environment
IR
information requirement
ISAF
International Security Assistance Force
ISR
intelligence, surveillance, and reconnaissance
J,K
JIPOE
joint intelligence preparation of the operational environment
JIPTL
joint integrated prioritized target list
JOC
joint operations center
JP
joint publication
JPP
joint planning process
L
LNO
liaison officer
LOE
line of effort
LOO
line of operation
M
MARSEC
maritime security
MCPP
Marine Corps Planning Process
MEF
Marine expeditionary force
MIB
mechanized infantry brigade
MISO
military information support operations
MOE
measure of effectiveness
MOP
measure of performance
MTOE
modified table of organization and equipment
N
NATO
North Atlantic Treaty Organization
NGO
nongovernmental organization
NWC
Naval War College
NWP
Navy warfare publication
O
OE
operational environment
OPORD
operation order
OPR
office of primary responsibility
OPS
operations
OPT
operational planning team
OPTEMPO
operating tempo
P,Q
PAO
public affairs officer
PIR
priority intelligence requirement
PMESII
political, military, economic, social, information, and infrastructure
PN
partner nation
POLAD
political advisor
PUB
Plans Update Board
R
RAND
Research and Development
RMRR
relevant, measurable, responsive, and resourced
RPG
rocket propelled grenade
S
SIGACT
significant activity
SIGINT
signals intelligence
SMA
simple moving average
SMART
specific, measurable, achievable, relevant, and time bound
SME
subject matter expert
SOF
special operations forces
SOFLE
special operations forces liaison element
SOP
standard operating procedure
SP
self-propelled
SPOD
seaport of debarkation
T
TOPFAS
Tools for Operations Planning Functional Area Services
TW
towed
U,V,W,X,Y,Z
US
United States
USAF
United States Air Force
USD
United States dollar

## Part IiTerms And Definitions

assessment1. A continuous process that measures the overall effectiveness of
employing capabilities during military operations. (DOD Dictionary. Source: JP 3- 0) 2. Determination of the progress toward accomplishing a task, creating a
condition, or achieving an objective. (DOD Dictionary. Source: JP 3-0)
assumptionA specific supposition of the operational environment that is assumed to
be true, in the absence of positive proof, essential for the continuation of planning. (DOD Dictionary. Source: JP 5-0)
commander's intentA clear and concise expression of the purpose of the operation
and the desired military end state that supports mission command, provides focus to the staff, and helps subordinate and supporting commanders act to achieve the commander's desired results without further orders, even when the operation does not unfold as planned. (DOD Dictionary. Source: JP 3-0)
condition1. Those variables of an operational environment or situation in which a
unit, system, or individual is expected to operate and may affect performance. (DOD Dictionary. Source: JP 3-0) 2. A physical or behavioral state of a system
that is required for the achievement of an objective. (DOD Dictionary. Source: JP 3-0)
decision pointA point in space and time when the commander or staff anticipates
making a key decision concerning a specific course of action. (DOD Dictionary.
Source: JP 5-0)
effect1. The physical or behavioral state of a system that results from an action, a set
of actions, or another effect. (DOD Dictionary. Source: JP 3-0) 2. The result,
outcome, or consequence of an action. (DOD Dictionary. Source: JP 3-0) 3. A
change to a condition, behavior, or degree of freedom. (DOD Dictionary. Source: JP 3-0)
end stateThe set of required conditions that defines achievement of the
commander's objectives. (DOD Dictionary. Source: JP 3-0)
evaluateUsing indicators to judge progress toward desired conditions and
determining why the current degree of progress exists. (Source: ADP 5-0)
indicator1. In intelligence usage, an item of information which reflects the intention or
capability of an adversary to adopt or reject a course of action. (DOD Dictionary.
Source: JP 2-0) 2. In operations security usage, data derived from friendly
detectable actions and open-source information that an adversary can interpret
and piece together to reach conclusions or estimates of friendly intentions, capabilities, or activities. (DOD Dictionary. Source: JP 3-13.3) 3. In the context of
assessment, a specific piece of information that infers the condition, state, or
existence of something, and provides a reliable means to ascertain performance or effectiveness. (DOD Dictionary. Source: JP 5-0)
measure of effectivenessAn indicator used to measure a current system state, with
change indicated by comparing multiple observations over time. (DOD Dictionary. Source: JP 5-0)
measure of performanceAn indicator used to measure a friendly action that is tied to
measuring task accomplishment. (DOD Dictionary. Source: JP 5-0)
mission1. The task, together with the purpose, that clearly indicates the action to be
taken and the reason therefore. (DOD Dictionary. Source: JP 3-0)
objective1. The clearly defined, decisive, and attainable goal toward which an
operation is directed. (DOD Dictionary. Source: JP 5-0) 2. The specific goal of
the action taken which is essential to the commander's plan (DOD Dictionary. Source: JP 5-0)
operation1. A sequence of tactical actions with a common purpose or unifying
theme. (DOD Dictionary. Source: JP 1) 2. A military action or the carrying out of a strategic, operational, tactical, service, training, or administrative military mission.
(DOD Dictionary. Source: JP 3-0).
operation assessment1. A continuous process that measures the overall
effectiveness of employing capabilities during military operations in achieving
stated objectives. (DOD Dictionary. Source: JP 5-0) 2. Determination of the
progress toward accomplishing a task, creating a condition, or achieving an
objective. (DOD Dictionary. Source: JP 5-0)
operational approachA broad description of the mission, operational concepts,
tasks, and actions required to accomplish the mission. (DOD Dictionary. Source: JP 5-0)
operational designThe conception and construction of the framework that underpins
a campaign or operation plan or order. (DOD Dictionary. Source: JP 5-0)
operational environmentA composite of the conditions, circumstances, and
influences that affect the employment of capabilities and bear on the decisions of
the commander. (DOD Dictionary. Source: JP 3-0)
staff estimateA continual evaluation of how factors in a staff section's functional area
support and impact the planning and execution of the mission. (DOD Dictionary. Source: JP 5-0)
systemA functionally, physically, and/or behaviorally related group of regularly
interacting or interdependent elements; that group of elements forming a unified whole. (DOD Dictionary. Source: JP 3-0)
taskA clearly defined action or activity specifically assigned to an individual or
organization that must be done as it is imposed by an appropriate authority.
(DOD Dictionary. Source: JP 1)
threshold of successA level, point, or target desired for an indicator. Attainment of
the target indicates success for the associated task, objective, or end state and signals the opportunity to reallocate resources. (Source: NTRP 1-02)
varianceThe difference between the desired situation and actual situation at a
specified time. Based on the impact of the variance on the mission, the staff makes recommendations to the commander on how to adjust operations to accomplish the mission more effectively. (Source: NATO Operations Assessment Handbook)
This page intentionally left blank. This page intentionally left blank. This page intentionally left blank. This page intentionally left blank. This page intentionally left blank.