¢ “The enemy anticipates that our force will be at location X.”

¢ “The enemy will detect the movement of our force to location X.”

e “There are good crossing sites that we missed.”

e “The enemy doesn’t know how good a location X is.”

e “The enemy doesn’t have any river crossing assets. He can’t cross the river at all.”

e “The enemy’s river crossing assets are so good that he can cross elsewhere.”

¢ “The enemy has a large enough force that he can accept casualties crossing

elsewhere.”

e “The enemy's objectives are different. He doesn’t need to cross at all.”

e “The enemy will use air assault to get across the river, rather than cross it.”

These nine ideas are exceptions to the officer's recognitional assessment of the situation.
Some may be implausible (such as the suggestion that the enemy can afford high casualties in a
river crossing). Others may require defensive planning. For example. the enemy could anticipate
the current location of own forces, but moving forces surreptitiously would protect against this.

Table 1: Example of exceptions to the interpretation of evidence, and methods of handling the

exceptions.

Assessment: The enemy will cross at X.

Exceptions

Responses

‘The enemy anticipates that our force will be at
location X.”

Place our forces elsewhere then move at the last
minute.

“The enemy will detect the movement of our force to
location X.”

Same as above.

“There are good crossing sites that we missed.”

Consult with a specialist in river crossings or scout
enemy movements tcwards other crossing sites.

“The enemy doesn’t know how good a location X
is.”

Dismiss this as implausible given recent intel on
enemy surveying and scouting activity.

“The enemy deesn’t have any river crossing assets.
He can't cross the river at all.”

Dismiss this. It presents no problem if true.

“The enemy's river crossing assets are so good that
he can cross elsewhere.”

Collect intel to verify or disprove.

“The enemy has a large enough force that he can
accept casualties crossing elsewhere.”

Dismiss this as implausible given current, reliable
intel.

“The enemy's objectives are different. He doesn’t Adopt a contingency pian.
need to cross at all.”
“The enemy will use air assault to get across the Accept this risk.

river, rather than cross it.”

11



Ol

suoydwinssy ueppiy Bulpul-y Hulules| JUaSsassy UOHeNS

R| ‘UolNdaoxa

FONSI BOTY PTR LATA RTE ETAT

“and} aq O} Ajay!) eo UoNdwnsse ey] eyeW 0} UNO aye) AeW NOA

ony Ss} uondwinsse

guns ATA ETE RRS SRT ANAT ATEN

OU) J! 89S O} UOPeUOJUI JaYYNY JoJo Aew NO,

“YSU 84} Jdedoe 0} aploap Aew No,

FESCUE sw Ef oigisnejdu on se uondons a a

| “8Iqisnejduu! 00} se uoNdeoxe ayy ssiusip Aew no,

ae SESE SAPIENS SEES EPPS REY REITERATE NY TET GTN SN ETN TOPPER

‘INO Wa] pajns 40 WeY}

Pe}a1109 pue -- sassouyeem jeUajod Japisuod oO} JJasuNoA Paosoy BABY NOA ‘adUapyUoD
4] INOA NYVWA NOA sayew ying ‘Buco S| | BAOJd },USBOp UOISNJ9UOD e 0} SUOI|deoxa ajqissod
Bl 242 eJau} esnedaq jsnf “‘uoISN|OUCD JNOA Jo ains Sse] NOA ayeW O} BABY j,USEOP poyjOW SIU
4

suoldaaxq ajqissod ay} ayenjeag ‘mon

32


9 suonduinssy uappi} Suipul4 Burures jUusWSSsesSsy uoHeNnysS

REGRETS MCR

aa} > dais 0} yoeq oF
iBuowm si uoeuRjdxa SIY} Yeu. NOA sjjaq
MOU 8ONOS soUAabyJe}U! ,JOQJed, ay] “p

a as.

UAE SUT RCL TT
‘Buolm aq
Pinos y Moy uleldxy *¢

‘(asje Buiyjou s,asay
MUIYL NOA ft UaAV aTIYM |-——
® 10} Bulo6 daayy)

T

PRION TERN SEIS SATAN COSTTE PTS IC AE ET
“‘Buoum

S! JUlUssasse INGA jo Led siqy Uy NOA S118} (jpeq [eySAUO
© ayill) 8o1NOS aoUaHI}O]UI dosed, e yey} aulBew ‘z

ALTAR ANG EA Te TER,

YW 0 ains ale nod > ru
NOA }! Uane -- JUaWSsesse INOK Jo ped JBOHUS B yVaJaSg “1

PREP CTI TS EET PR TTS SS NNT TOTES ETE rrerremmwamneerolern

“suonduinssy USsppIH Bulpuls

28


GL suondunssy ueppiy Buipul4

Hululed| jusWssessy UOHeNIS |

“syoeye
42 40 Asayyjpse Jo AWQissod
OY} 404 URI Jou Og ‘Ss
4! BJ9YM aseg Bo] ay) doay

“suoljdunsse
feulBu0 INOA YUM yous
0} pue aiqisnejdui
suolyeurjdxe
M8U BU} PUI} 0} UOSBaL
poof eney Kew no,

y ‘aseq bof
R] 24) J2a}0J1d oO} paxojdwa are
li] Suodeem esuajap sie ainsuz

‘ayqisnejd
suoneurldxe
MAU a4} JO BUO pul}
no }! suondwnsse
4Jnof aBueyo
Ayyenjoe ues no,

yoeye sre so Arayie
40 Ayiqissod ay) jsurebe

sainseaw erlsuajap enissed
asn JC seas ay} 0} aseq

5o} ay} aaoww ‘ajdwexa 40-4

“suolduinsse
BU} O} BIGesaUINA sso}
ae JO -- Suondwnsse
84} UO puadap Jafuo}
ou suejd snoA yeuy
Os uoHoe Jo asinoo
4nof aBueyo ues no,

37



Metacognitive knowledge: Overview

Metacognition consists of functions that monitor and regulate thought. It has been defined
as “individuals' knowledge of the states and processes of their own mind and/or their ability to
control or modify these states and processes” (Gavelek and Raphael, 1985).

Metacognition is the focus of considerable attention by developmental psychologists
(e.g., Flavell, 1979; Forrest-Pressley, MacKinnon, and Waller, 1985), interested in how children
learn to manage the cognitive activities involved in reading, comprehending, memorizing, and
paying attention. Evidence for similar processes is found in the literature on expert-novice
differences in problem solving.

While experts may be said to “recognize” familiar problems, recognition is sometimes
achieved through the evaluation of intermediate results. For example, according to Larkin,
McDermott, Simon, & Simon (1980), physics experts often construct and examine a sketch of
the superficial objects and relations in a physics problem in order to determine the next step: If
the depicted system is familiar, the expert may proceed directly to the equations required for
solution. If the system is unfamiliar, the expert constructs an idealized representation (i.e., a free-
body diagram), which is then used in the generation of solution equations. According to Chi,
Glaser, and Rees (1982), this qualitative analysis of a problem is not a discrete phase that is
concluded prior to the generation of quantitative equations. They found that experts returned to.
and refined, the initial gross representation when necessary throughout the course of the problem.
In short, experts manipulate the situation until they understand it. That is, they change their
representation of the problem until it makes contact with their knowledge. Metacognitive skill is
Tequired in judgments of familiarity and of how best to transform the problem to make it familiar.

Metacognition also plays a role after the problem has been recognized and (apparently)
“solved.” Physics experts utilize the abstract physical representation of a problem to verify the
correctness of their method and result, e.g., by checking whether all forces are balanced, whether
all entities in the diagram are related to givens in the problem, etc. Similarly, in chess, Simon
(1972) observed that some masters search the space of future moves and countermoves to verify
that the moves they recognized as best are in fact in the subset of good moves. More recent
research has found that differences in search skill (i.e., depth, breadth, and speed) are correlated
with chess expertise (Charness, 1981; Holding and Reynolds, 1982). Key aspects of searching to
verify recognized answers are metacognitive: the processes of initiating search, monitoring and
evaluating its results, and deciding when it should be terminated.

In sem, metacognitive processes are crucial in two phases of intuitive decision making:
¢ Constructing a situation model or plan when recognition is uncertain.
¢ Verifying the results of recognition

We have devised a model that incorporates both recognition and facilitative processes,
and we call it the Recognition/Metacognition model (Cohen, 1993a; Cohen, 1993b). It has three
key components (see figure 2): Quick Test, Critiquing, and Correcting. Each represents a
different category of skill in situation understanding and decision making. We will address each
in turn.


Lv peyedxeun oy} Buljpuey

Bulures | juewssessy uoHens

«2 9Q SIY} UBD MOH “AuesjUOD BU} 0} SUOSeaL JeUIBLO
JnoA ay!dsap oauJ09 si JUaWISSasse Meu INOA, SKes adINOS goUa

jul eyed ay) yeu) auIBew]

“ayenbape s} you ul ulewa |

“YINOS U! esUajep UIE Bye |IIM aM SeAaI|oq
SJOJasOU} PUB O|qeJOAR} S| Ule18} WeyINos ay) yeUy
azjuBoo9s em smouy Awaug :(uoNdesep) Awauy,

\ “JUEWEAOW JO JoOLedNs S| UleIIE) WeY]NOS

“painyded jou si

Hod j! UeAs suo}esedo pod dojs |IIM Jo}9es WeYyLOU
J eiNZIas Jey Sersljeg AWaug :(sojoe}) AWoeUZ
‘einseelw uoNdedep se Lod 0} ejno) snoiaqo

sso} ‘JaBuo] ydacoe jm Awoug :(uoydesap) Aweug,

Mnoalqo Ajax!] S| YINOS UI Lod

“W edxe jsea] om e1oym Poys

urew eye 0} spuaju! pue YINos ey) U! AjlAReY eioW
PUBJEp IM eM Sanel|aq ey esnedaq eULDOp Woy
Buneinep Aleuonueyuy si Aweuz :(uondedap) Awouy,
“eBpuq

JO UOHONYSEp JO esnedsq eUl}OOp YIM BulAjdwoo

JO ajqedeo jou sj Aweug :(seniqedeo) Aweu3

“ssaoons Wey) uojules AWaua seyeyoip aulj00q

suoieuer|dxy

s]uaAq

“You aly Ul YORE Ure siy YouNe] oF suRjd AWoUs Oy jJUsLUSSESsSY

‘juoussesse
Mou INoA
UM jU9}SISUCD
Wau} eyew
0} sJuaAe pue
UO!PELUOJU!
poyoedxeun
3u}
NIV1dX4 “2

65



peyedxeun oy} Burpuey Burures | juolussessy UOlenus

SVAUSEURTRIATNN

‘pajoadxeun ay) Buljpuey 10} poujew e sjuaseid uoldes siy]

47

ELC asatis STEIN AT EINE SPINS ON SANSA RENTS SI
‘JUBA peyoadxeun ayy jo Bulpueysiapun
4JNOA YM Wa|qoid & aq pjnoo }| Buoim si juawssesse Inok ueow

A\essaoau j,useop }! Ing “‘Alsnoiias uaye} aq Jsnu! JUaAs peyoedxeun
uy ‘snosebuep aq ues sasuodse Yjog "Jueussesse Inok ehueyo
pure BduUap!jUod eso] Ajaye|peww (Z) “Wey JUNOOSIp JO e10ub| (1)

‘S]JU8A9 payoedxaun 0} sesuodsa! UoWWOO Om} ase a1ey “NDD0 ABW

SJUSA9 pajoadxeun pue ‘joeped Wopjas ale sjuewssesse uoyentis
ing “Uol}Oe Jo BsiNnOd punos k asiAap puke ‘sjuaAe eunyny joIpeid
‘s]u9Ae Jsed urejdxe 0} No Smo||e JUatussesse UoHENys poob y



oo

payedxeur eu) Buljpuey

Hulules| Juaussessy uolens

45



Ze peyedxeun eu} Buljpuey Burures | JUBLUSSESSVV uonenys |

SRURETT o

: f 2 ATRIA GOUT
éS!4} se yons sjuane payoedxeun 9a]puey djey ueo enbiuyse} JEUM

ESTAR I RA VTA TI TELA ABATE TR ATER E

Son ST ORIS N I PR Ng EI

‘asay) yore JOfew B Jonpucs oO} Aye UMO sey) Buniuy sny) ‘juod

4184) 0} eBpiig sofew e peosjsap aary saos0j AWaue UsJayINOS ay) Jey) UOdas a2uabi/aju! UB aAlaIas NOA :ajepdry

REATIRTNSUSNNEIS ANN SET ET ETT TT

OREM CTT EY Teta bene eite ra

YHOU ayy U yore AeuolsieAlp ‘Auepuogas 2 Yyoune] jm AaY, “JuaWeoIOJUIOL
4NOA JUaAadd puk YOd ay} EzIas 0} YINOS AY) UI yoRNe UIE Si! a1NJaXa 0} SURI AWauUA ay ‘jUQWSsasse ay,

51

RAT SRATIS TRIER 2 SRI LORELEI RTE TETRIS SEERA AY ASST TALEY STEN ERS TT
“Ssaaons yojdx

8 0} SI ‘sayja1 Awaua ay) YOIYM UO ‘aULJDOP JaIAOg ‘(J0]98S UJEYYOU ey} JOU jnq)
40}98S UJEYINOS INOA Buyjoeye sseoons payseuw pey sey Aweaue ayy ‘asowsayuny ‘Awaue 2Y} 0} jobse) aBANIBIYe
ue sasod }) pue ‘10}9as NOK jo ped UBYINOS BY} U} SI UOd BY, “Od ay} O} YYed JOaJIP a1owW eB sey adic) Awaua |
UJ9YINOS AY] puke ‘JUEWEAOW JOWe pides yoddns sAempeos puke UlesIa) UJ@YINOS ay | JWNIIYIP 84ay) JUBUIAAOU
JOUUE BYeW YOU ay} U/ UlesIa} YONOJ puke speos sood YeraMo}] “yedJajUNOI WaYINosS siy UBY}) PanuaLadXO
BJO S} JEPUBUILIOD S}! PUB '8DJOJ UJ@YINOS AY} uey} sBulssola Janu asay) 40J paddinba 49J2q S! eds0) Awaua
UsSBYOU BY “AANDEIGO 1104] YOCAJ O} SIBALI [BIAS OSIBA bi} Jsnw Awaue ay} ‘UoHedo] JEYIIa WOlJ ‘yseayjnos
JNO 0} puk JseaypoU JNOA oO} Aeme WY OOL Inoge paAeue ale saosoj AWeuzZ ‘BulAe ale sjuaWadJOjUlas ;
yoy YBnosy) Yod ay} Buipuayap si OYM JEpUBUIWOD UOISIAIp 8040) AQUBBUNUOD B ale NOA -uonentis ayy

: T
OlMeU3S9DS



Le pajsedxeun eu} Buljpuey

Bulures| Juswussessy uolens |

‘jualussesse
4noA eBueyo

‘MoRsHes si
uojeurdxe ou 3}

“UO OS pure ase} SI 1 aseO
ul sued AoueBuguos ydope ‘anuj yi aye
0} pleyemeg ay) adeys ‘yt Ajuen 0} An ues
NOA ‘uoHduinsse ue awiodeq MOU sey }|

| “alqisne|d Aj\qeuoseas suolyeue|dxe Soy} Jo

ANO LSV31 LV pul Isnw nod ‘jugwssesse
feulBO 84} 0} UO PJOY 0} YsIM NOA J]

RARELY ONES ROA

“suoljeue|dxe
aseu}
ALVNIVAA '€

‘Awoue oy) Aq ones

® Sem a6p1iq 04) Jo uoHIONASep oy) sdeysog :(oyeIsIW) AWOUS
sojes

WayiNOS BY) U} eq JOU Im YORNE ULEW BY YUIY) NOK ayeW

01 uoudaoep bs} uononusep oy) sdeyeg :(Uondasep) Awauy
“aBpuq ay pakonsep

Tey) ‘Aweua ey} 10u ‘s8010) UMO INO sem 11 Sdeyog :sdoo1) ,

quody sty OF
eBpiiq Jofew e peXosjsap sey soso) Awaua uweynos

suolyeue|dxy

s]ueAq

YInos OY} U| YORHe UleEW eB puke ‘YOU au) UI yoRNe Areuolssanip @ suejd Awaua aul :JueluSSesSSY

55



Le

suolduinssy ueppiy Bulpuly

Hulures| jJuawssessy UolenS

[ “g jeyoed asn

PER RAFTS NUE TPS AT 2 PPO BIRCH STRAT TROT

‘Jaluea peal nod yey) oueUads pues} aU;
WoJ} senss| jeseaes uO UeBe anbiuyoe} ayy ALL

43



In this example, the decision maker has used the crystal ball and METT-T categories to
generate the interpretations of each event (in the right column). The METT-T category of each
explanation is presented in brackets. Explanations marked with an asterisk are those the officer
considers the most credible.

The most plausible three explanations for the events in the example concern troops,
enemy deception, and enemy tactics. However, it may be improbable to the officer that all three
of these explanations are independently true. Thus, for example, he may choose a somewhat
simpler account, in which a single cause — our own aggressive interdiction campaign —
accounts for the destruction of bridges and radio silence, and a second assumption concerning
enemy tactics explains reinforcements in the north. This account balances credibility and
simplicity.

Thus far, this illustration employs the first three steps of the method presented above.
However, an officer may find that he cannot construct a plausible account of all of the
unexpected events. There may be too many events to explain. Suppose that the example were
extended to include another unexpected event: the movement of two enemy motorized rifle
regiments from the south to the north. Suppose further that every explanation for this event (such
as enemy deception or errors) significantly diminishes the plausibility of the account. That is,
there is no credible and simple story that supports the assessment that the enemy plans a main
attack in the south and a diversionary attack in the north. In this case, a new assessment is in
order, per the fourth step of the procedure outlined above.

The officer examines the unexpected events to date and generates a new assessment, in
this case that the enemy plans its main attack in the north. He then fills the events column of a
table (identical in structure to Table 2) with data that appear to contradict the new assessment.
For example, he may recall that the southern terrain is superior for movement and that a southern
port is a likely enemy objective. To each unexpected datum, he applies the method of handling
unexpected events. That is. he generates multiple interpretations of each datum, and selects from
among all explanations the few that tell a plausible story.

Training materials

The text used in training this method is presented in Appendix A on pages A-18 to A-51.
It is divided into three sections. The first introduces the method, the second applies it to
explaining unexpected events (using the first three steps of the method), and the third concerns
revising the situation assessment (beginning with the third and fourth steps, then cycling back to
the first).

The introduction begins with a motivating statement, an outline of the four-step method
in flow-chart form, and a scenario consisting of a situation description, an assessment, and an
update describing an unexpected development, upon which illustrations of the method are based.

The next section, on explaining unexpected events, draws on the scenario to fill out a
table containing slots for the situation assessment, unexpected events, and explanations of those
events. Instructions and tips are presented in floating captions beside the table, as it is filled in
over several pages. This section ends with a statement of the conditions of plausibility under
which the assessment should be revised.

15



LL

suoyduinssy uappiy Bulpuly

Hulures| Juawussessy uoHenS

ueid Aouebuayuoo ydopy
ueid Aouebunuoo ydopy

AjayJUN 00} Se Ssisiq

9INS BYEW O} JDIJOD

wajqoid & jon

JOAU pulyaq syyed UO Saniasal aoe
JOA pulysq syyed uO sanlasal aoRld

8AOW UdU}
‘eleymasje seoi0} aoejd - uondesap asp

SSOJ9 },UOM--]/NeSSe JI
SSOJ9 },UOM--aADa[go JUsIEWIG

salyenseo \decoe--a910j sabe]
aJeymas|a SSOJo--sjasse Jayog
Ssolo },UueD

X passiw AoyL

SOUS PASSILU BAA

9910} 1NO yOe]Eq
8010 JNO eyedionuy

0g ued aM HUM

suondaoxea ajqissog

*X ye SSOJO |IIM Awaue ayy
:Juawissassy

suondsasxy ay} Bulyenjeag :ajdwiexy

33



Ov pejoedxeun au) Buyjpuey Bujures | jJusuuSsessy UOleENYNS

‘Juawussesse
mou
JNOA JO MAIA JO
#| Julod oy} Woy
pajoedxeun
MOU Je 8Se8yL
‘Juawssesse
jUaWerow 40 Jouedns s1 yfe1I0} woyr6S | dl jeulBuo
[ E| anof peyoddns
 yeEU} SJUBAS pue
} UOITEWOJU! BY}
earjoalqo A\dyI| SI YNOS Ul WOg YAEWAWSE

“ssaoons Weyjnos BOIOJUIOL Awaue sej]ejoIp BUuL}I0G i “uolyenys au}
suoneue|dxy S]UeAy jo jusussasse
J : EST Mau JNO,

~ penis

“‘YvoU 984i Ut yee WEL siy Youne| 0} suBid AWoUD ey, 7]UBLUSSOSSYY

64



FOREWORD

The U.S. Army strives to have competent officers trained in the art of battle command.
One discipline of that art is the assessment of battlefield situations. An experimental program of
research was conducted to determine ways to improve the cognitive skills used in situation
assessment. The training concepts presented in this report were a key component of the U.S.
Army Research Institute for the Behavioral and Social Sciences (ARI) Fort Leavenworth
Research Unit’s research program. This research was initiated in 1994 at the special request of the
Commander of the U.S. Army Training and Doctrine Command.

This report describes the rationale and intent of instruction to safeguard against uncertain
or unreliable information and how to handle information conflicts. A companion report, ART
Technical Report 1050, provides experimental evidence collected on Army leaders that shows the
merits of the instruction.

ZITA M. SIMUTIS EDGAR M. JOHNSON
Technical Director Director



ov pajoedxeun eu} Buljpuey Burures | jusluSSsessy uonenis |

‘VOD JNOA asiAal 0} JUBUSSESse MAU 9] BS)

DET PEY LOTTI PET TEN Ee TAR ETN AT RESIN .
“SuOI}OIPOId

asoy} jo Aoeinooe ay} }S9} pue sJUsAe

@inyny JoIpeid 0} yUatUSSasse ay} esp

(‘sseooid 3

Aeso} pue

67

f| SNONUI}UOO & JUBLUSSESse UOI}ENYS fi = “juaUIssasse maul suspen.
SOF YUM SJEYL “UOHPULOJU! pue B) 40 suoddns yeu} uoeuoyul a}}OD

s]UeAe peyoedxaun aiow ayeiaueb
Aew sdejs esau} yeu} 9}ON)

“leq yeysAuo, uy
asp ‘suonduinsse ueppiy 10} yoreas

EE VENERATED ERE ‘ cr ERERE SORTASE

z vi DEAT AMR ATTEN
r “juowssesse Aue op nok Aem awes OU} JUBLUSSESSe UOHeENIIS MEU BU} B|PUBH



Zp pajoadxeun au} Buljpuey

Bulures | JuaWSsessy UOTeNS

yunoooe ajduiis ‘ajGuls e sapiaoid pue ‘uolewojul
JO s0aId yore jo uoNeuRydxa aqisnejd & s} UoONdes0eq

\

“ayenbepe s} you U} urea |

“yNos U! esUajep UTE exeW |IIM eM SeAatjag
@10jaJ04) PUR e/GeJOARY Si UNeII9} WeYyNos By} Jey)
ezjuBooe1 em smouy Awauz :(uondeoep) Aweu3,

JUsWeAOW JO} JoVWedNs S| UNeLIE} WeYINOS

“painjdeo jou st

vod | UeAe suoleiedo pod dojs |}!M Jo}oas Wey}oU
JO eiNZlas yey] sers|jeq Awaug :(soyoe}) AWouy
“einseaw uoydesap se yod 0} a}no! snolAqo

ss9| ‘4a6Buo] }daooe jm AWauy :(uodeoep) Aweuz,

eanoalqo Ajex] St yNOS Ul Og

“Wh yoedxe jsea} EM a1aYyM LOyo
ujew eye 0} Spueju! pue YyNos EY} Ul AjIABaY aIOW
q BY EsNedeq euUL}OOp WoL)
6unelep Ayjeuonuayu! s} Awouz :(uondesep) Aweuy,

“eBpuq
JO UOHONIsep JO esnedeq euLDOp YM BulAjdwoo
Jo e\qudvo jou s! Awouz :(s

yedeo) Awoux | *

IS WO\NOS OdJOjUJo) AWeUe Se}HVIP OULDOG

suojeue|dxy

s]uaAq

“YHOU eu} UL YoRNe ure siy YouNe| Oo} suBId AWoUE EYL [}JUBLUSSOSSY

“yuN0INe |}218A0
Butoulauoo
JSOW ayy
SUILUIa}ap 0} ALL
*suoneue|dxe ay}
ALVNIVAd '€

66



Gv peyadxeun ou} Buyjpuey Bulures | JUBWUSSESSY UOIeENUS

‘juauissesse
SIy} UO peseq YOO B dojanep No{ ‘sunojUOD apisy|ily pe}seso} ‘dee}s
BHuoye janes} |IIM ay Jeu SI ‘elojor0y} ‘WUeWssasse Ino, , BuIUI|AYS, Aq
einsodxe Juaneid 0} aul] eBpu ay) Buoje Bulnow plone jim af] “ysnque
PIOAR O} S]!e} $40 jane} [IM AWauUa ayy yeU) adxe NOA “salANOe
eyjWenB yonpuos 0} YOU jane} [JIM AwWoUs OU} yeu] apNjoUcS No,

SSPE A SNS IT EG MPR TER ENEG. ON TRAE ITO TIO SH TAPE
“ sauljeBpul pesodxe pue sadinalo Aaljen
ay} Buoye Ajuo uns sires] os ‘daays ose sedojg ‘vale aajoelqo ay} pue
f| $2010) Aweue ey} Usemjaq sal SAa|jen deep pue siily daays jo uoibes
Vv ‘“yjnos ay} ul AqUaiiNo s| AWaue ay, “YOU ay) UI pays!|dwiogs9e
@q AjU0 ued sanoalgo asa! ‘sonIAoe JayjO pue ‘uoeLodsue),
‘uoleolunuiwos Bundnisip 40 solo} Ajpuaiy Bursseuequia
SAjOAu! sjeoh jussedde esoym 2910} e||LUaNnB e si Aweua ay

69

asioJexy





Table 1 includes several responses to the explanations generated above. These responses
are of several types: testing the truth of an assumption through recall of evidence or data
collection, adopting proactive strategies, adopting contingency plans, and accepting the
assumption as a known risk.

We have found this technique of critical thinking to be fast and highly productive. It is
particularly effective for officers when they are confident of their assessments. It raises issues
that force officers to earn their own confidence by strengthening their assessments and plans to
compensate for potential weaknesses.

Training Materials

The text used in training this method is presented in Appendix A on pages A-5 to A-17.
The text begins with a motivating statement concerning the method: “Hidden assumptions are a
source of weakness in situation assessments, and a source of risk in the courses of action based
on them.” We then present the method in the form of a flow chart, and apply it to a military
scenario, Next, some ways to use the output of the method are briefly described. We then present
a final motivating statement, and a practice exercise with sample solutions.

This unit is designed to be used in conjunction with brief lectures and interactive
exercises based on pretest materials or the officers’ personal experiences. A particularly effective
classroom exercise, when trainees have had some operational experience, is to ask for a
description of a situation in which someone was very sure of something (e.g., that the enemy
would cross the river in a certain location). When the question is put this way, the example
usually invoives a situation in which the assessment turned out correct. The instructor then
“brings out” the crystal ball, which says that this assessment is wrong, and the class is asked to
generate ways in which that could happen. The exception conditions are listed and the class is
asked to evaluate their seriousness and plausibility, and asked what they would do to handle each
possibility.

The next unit of training deals with situations in which assessments turn out to be wrong.

Handling the Unexpected

The Method

Despite careful assessments and planning, officers sometimes find themselves
confronting unexpected events. The second training method we have developed is designed to
help officers handle events that conflict with a current assessment, rather than simply disregard
or discount them. At the same time, an unexpected event does not necessarily imply that the
current assessment is wrong. In this training, officers begin by explaining events in terms of the
current assessment, but if these explanations are implausible, they alter the assessments
themselves. The procedure consists of these steps:

1. Notice unexpected events.

2. Explain how an unexpected event could occur even if your current assessment is
correct. (If there have been previous unexpected events, try to find the simplest
plausible explanation of all of them.)

3. Evaluate the plausibility of your explanations.

12



Hinton, G. E., & Sejnowski, T. J. (1986). Learning and relearning in Boltzmann machines. In J. L.
McClelland, D. E. Rumelhart, and the PDP Research Group (Eds.), Parallel distributed
processing: Explorations in the microstructure of cognition: Volume 1: Foundations.
Cambridge, MA: The MIT Press.

Holding, D. H., & Reynolds, R. I. (1982). Recall or evaluation of chess positions as determinants
of chess skill. Memory and Cognition, 10(3), 237-242.

Klein, g. A. (1993). a recognition-primed decision (RPD) model of rapid decision making. In G.
A. Klein, J. Orasanu, R. Calderwood, & C. E. Zsambok (Eds.), Decision making in action:
Models and methods. Norwood, NJ: Ablex Publishing.

Larkin, J. H., McDermott, J., Simon, D. P., & Simon, H. A. (1980). Models of competence in
solving physics problems. Cognitive Science, 4, 317-348.

McClelland, J. L., & Rumelhart, D. E. (1986). A distributed model of human learning and
memory. In J. L. McClelland, D. E. Rumelhart, & the PDP Research Group (Eds.),
Parallel distributed processing: Explorations in the microstructure of cognition: Volume 2:
Psychological and biological models. Cambridge, MA: The MIT Press.

Neisser, U. (1976). Cognition and reality. Principles and implications of cognitive psychology.
San Francisco: Freeman.

Palincsar, A. S., & Brown, A. L. (1984). Reciprocal teaching of comprehension fostering and
comprehension monitoring activities. Cognition and Instruction, 1(2), 117-175.

Rasmussen, J. (1993). Deciding and doing: Decision making in natural contexts. In G. A. Klein, J.
Orasanu, R. Calderwood, & C. E. Zsambok (Eds.), Decision making in action: Models
and methods. Norwood, NJ: Ablex Publishing.

Simon, H. A. (1972). Theories of bounded rationality. In C. B. Radner & R. Radner (Eds.),
Decision and organization. Amsterdam: North Holland.

18



new assessment may be compared by asking how convincing a “story” can told in support of
each; in other words, how plausible and simple are the assumptions that each entails.

In some cases, a new assessment that is stronger than either of the previous ones can be
generated by combining their strengths. In this strategy, the decision maker focuses on the
assumptions that the assessments require, dropping the unreliable ones and adopting relatively

plausible elements from both “stories.”

Illustration

An example is presented here in a table format used during training (see Table 2). The
table consists of an assessment concerning the enemy’s plan of attack (on the top row), and
several events that appear to contradict the assessment in the left-hand column. Across from each
conflicting event, in the right-hand column, are explanations that have been generated to explain
the conflicting events to make them consistent with the assessment.

Table 2: Table format used to train both finding hidden assumptions and handling the

unexpected.

Assessment: The enemy plans a diversionary
attack in the north, and a main attack in the south.

Events

Explanations

Southern enemy force has destroyed a major bridge
to his front

“(Troops] Perhaps it was our own forces, not the
enemy, that destroyed the bridge.

{Enemy (deception)] Perhaps the destruction is a
deception to make you think the main attack will not
be in the southem sector.

{Enemy (mistake)} Perhaps the destruction of the
bridge was a mistake by the enemy.

Enemy has initiated radio silence in the north and
the south.

[Troops] The enemy is not responsible for radio
silence. Our interdiction campaign may have
destroyed critical enemy radio facilities.

*{Enemy (deception)] The enemy is concealing the
location of the attack by instituting radio silence
everywhere.

[Enemy (equipment)] The enemy’s C2 equipment
has failed.

Reinforcements have arrived in the north.

{Enemy (deception)] The enemy is “showing” arrival
in the north before shifting forces stealthily to the
south.

*[Enemy (tactics)] The enemy may be putting green
forces on line in the north to execute a secondary
attack.

{Enemy (mistake)] The northern movement of
reinforcements may be a mistake.

14



suondwinssy uappiy Buipul-| Bujures | jusLUSSesSsy UOHeNsS

OSES STR RIT TRAE EARN OT RR PNR Ppa
““suonduunsse uappiy
| Suipuy 10) spouyjaw ore aseuy ‘Ajayeunyo4

TESTED nTer

TERNS ARTE

“WAL] UO paseq UOHOL JO Sasinod at] UL YSU

fs S04uNOs e puke ‘sjuatussesse UOHeNIIS UI ssauyeom
JO 801NOS eB s1e suoNduuNnsse UappIy YyonsS “pajeys

BJOU oe suolduinsse 1ayO “Way} WIYUOS 0} Ay Aew

Au} pue ele eset yeym mouy yeys ayy “poudxe

oe suonduinsse awos “6-QOLLS jenuew Bururey
YYOMUSABE) 1-4 U! Paye}s se ,‘s]oe} UMOUyUN

Jo Bulssiwi yng Aresseoeu aoeldel, yeu) suoljduinsse
SEW O} SI AjUreLSOUN YIM jeap Aay) Kem Ue)

Aureysaoun 9ovy Ajjue}suoS SuaYeW UOISIOSp AVE]

27



was false. Conflict may thus trigger a metacognitive process of exposing hidden assumptions and
questioning their reliability. The process of revising beliefs to explain conflict requires a variety
of metacognitive skills: awareness that conflict exists, an ability to uncover implicit assumptions
that have created the conflict, sufficient awareness of the structure of one's beliefs to identify the
assumptions that are central to a variety of models and plans, and sensitivity to factors that
indicate unreliability of assumptions, for example, by recalling past episodes in which the same
beliefs led to a conflict. Finally, the process of assumption revision calls for balancing the
plausibility and the power of the resulting models and plans.

TRAINING METACOGNITIVE SKILLS

Training Principles

We derive from the R/M model several principles for training (to be described at greater
length in another report). These principles contrast with those that reflect different models of
decision making. For example, if we view decision making as an analytical process, a perfectly
appropriate stance in some circumstances, then it follows that instruction should focus on the
transformations required to represent problems within the analytical model (for example,
quantitative representations of probabilities and utilities). Practice may be partitioned such that
trainees study parts of the method before attempting to execute the whole. Examples should
come from varied domains to demonstrate the generality of the method, and they should be
presented in a graduated sequence progressing from problems that clearly afford the analytical
technique (that is. they require little in the way of transformation) to those that superficially seem
inappropriate to the technique (that is, they are quite difficult to completely and accurately
represent).

If decision making is taken to be a recognitional process, again a reasonable assumption
in some situations, then the following inferences about training hold. Instruction should focus on
goals, environmental conditions, and actions. In procedural terms, instruction should convey the
following type of knowledge: if your goal is X amidst events Y, then perform action Z. Practice
conditions should be realistic to promote accurate recognition and repetitive to facilitate
automatic performance. Examples should be representative of the domain, though feedback may
be made more explicit and immediate to speed training.

Training that assumes an adaptive model of decision making, such as the R/M model,
should first of all convey the iterative nature of problem solving — from recognition to
metacognitrve checks, critiques, and corrections, back to recognition. It must also aim to sensitize
trainees to domain-specific cues concerning the time constraints, stakes, and familiarity of
problems, as well as the nature of conflict, completeness and reliability in specific domains. It
must convey methods of critiquing situation assessments and plans, and of making corrections
that will enhance understanding. Practice should be designed to make metacognitive processes
explicit. Such practice might involve reciprocal teaching (Palincsar and Brown, 1984), team
exercises, explicit labeling of one’s own activities, listening to experts label their thought
processes, or critiquing the performance of peers. All of these methods help trainees make public
otherwise hidden cognitive activities. Making principles explicit may helps trainees transfer what
they have learned to varied settings (Collins, Brown, & Newman, 1989). In addition, it may be
appropriate to give trainees control over practice parameters that cue metacognitive activities;
these include time, immediacy of feedback, quality of feedback, and difficulty. Exercises should
employ non-routine cases, as these discourage simple recognitional solution and encourage the

9



Vv pejoedxeun ayy Buljpuey

Bulures| jJuawWssessy uoITeNUS

“playaeq
84} UO SUOHIPUOD MoU PU UO!EUOJU!
MOU 0} }depe oO} JUaLUSSESsse INOA
HulBueyo 10) sauljapin6 syuasaid 1 puy

“juawussesse
Jno aBueyo 0} usyM ‘s! yeyy
-- SJU8A@ peyoedxoun ,Aeme 6

jueuussesse juaino
4noA eAiesaid yey sXem ul sjuand
Bujure|dxe 40} anbiuyos} e sepinoid }

{COS PRE RAPE TEMT REL AT RUE

, UOH}eWOJU! pue s}UaAe Pe}oadxeun lejpuey No djey
0} pauBlsep s! paipnys isn aa,noA poyjew ey} ‘wns uy

“Ajuatoyoid

68



6c

payoedxeun ay) Buljpuey

Bulures{ juatussassy uolens

‘JoyUN, Way} jnoqe
UY] 0} BUH BY} a42} Pinoys

no ‘sindv0 sjusne payoadxeun
JO Jaquinu jueoyuBbis ® \I
Jo ‘yBiy s! aye}siW e Jo JS09 OU) 4]

RIG MEAS ROU ReAn PEs cat Rsk

iSLNAAA GaLOSdXANN SOLON

ENTE TANS nae RTE

LSva1 LV SAVM1V LN ‘esey dojs ueyodui
S1OW O18 SE]OU JeyjO Jo ‘auUN OU S| a10U) J]

suolyeue|dxy

quod) S1y 0}
aBpug iofew e paxonsep sey @040y Auiaua wayjnos

s}UuaAq

YINOs 84} UI YORE UeW B pue ‘YOU ay) UI yoeye Areuolsieaip & sued Aweue euL jusuussessy

ARSE AATR ETAL

‘Juaussasse
Aay 2 0}
pajejas Juana
peyoadxeun
yore

AOILON ‘fF

53



METHODS FOR TRAINING COGNITIVE SKILLS IN BATTLEFIELD
SITUATION ASSESSMENT

INTRODUCTION

This report describes a training method for enhancing two metacognitive skills that are
critical to officers performing battlefield situation assessment. The skills concern finding hidden
assumptions in existing assessments and plans, and handling data that conflict with expectations.

The training methods convey techniques that have several attributes to recommend them.
First, the techniques are simple. They can be executed entirely in the head, if necessary, and
officers report that the techniques are comprehensible and memorable. Second, these are rapid
techniques. They can be performed in minutes, not hours. Third, they can be executed by an
individual or a group (an aspect of the methods that we will discuss in a future report). Third they
can be used to refine plans as well as assessments. Fourth, they produce a narrative account or
reasoning that may benefit individual or group decision making. Whereas recognitional decision
making processes are automatic (and thus unavailable for reflection or reporting) and analytical
processes are too abstract to communicate easily in groups. the decision making processes trained
here are both explicit and concrete. Officers who use the reasoning processes trained here should
be able to explain their decisions and participate ’in critiques of their decision making.

The training techniques are based on a cognitive framework for battlefield situation
assessment, which was itself based on the results of extensive interviews with active-duty
officers as well as findings and research in cognitive psychology. This report begins witha
review of the battlefield situation assessment framework upon which the training is based
(Cohen, Adelman, Tolcott, Bresnick, & Marvin. 1993). The second section. concerning train:
opens with a description of training principles that derive from the situation assessment
framework, and continues to describe the training methods themselves. For each training method,
we present a description of the technique meant io convey the target skill, an illustration of the
technique. and a summary have been pilot tested and evaluated in experimental sessions with
officers from three different operational divisions. A subsequent report will describe the results

of these tests.

THE BATTLEFIELD SITUATION ASSESSMENT FRAMEWORK

Overview

The framework that we have used to analyze battlefield situation assessment framewors
consists, at the most general level, of four components (see figure |):

1. The real-world environment.

2. Memory and knowledge structures;
3. Actions, goals, and values; and
4. Processes for regulating and monitoring cognition

The figure represents these components as arcs that comprise a circle. The diagram
partitions these components with concentric circles, which represent different aspects of memory,
from the immediate focus of working memory in the innermost ring, to the contents of long-term
semantic and episodic memory in the outer ring (as discussed in the following section).



uononpoyu|

Hulures | JUawssessy UOHeNIS

S]U9A9 pa}oedxeun

Y3514V Juaussesse
uolyenyis nod

Buluyay -- pejoedxeun
ayy Buypuey

RELATE AE ETE

“suuajqoid
nod asneo Au}
FYO4Fg Juewssesse
uolyenyis uno ul
sessouyeem Bulpul4
-- suoyduinsse

UBUISSESSE UO!ENYS O} /eIONID s||4S OM} UO Sasnoo} Buure SIU.

24



9g payedxeun ey} Buypuey

Hulures| jJusussessy uolVenS

EET EATEN a ead RIAD RAG RT TO ATE

Tatas an

“suonduunsse paijie@aun

mau Aue 00} sasinbas sesuiduns sno} |e Bulure|dxy

‘ayeisiw B oq Aew sjuawiie:
PeZUO}OW sy) Jo WeWerow WayyoU ayy :(ayeIsIW) AWeUR
I$ 810)9q
UuOU ey UI feAWe ,Sulmoys, si AWaUe ey, :(uondedep) AweU3

“YHOU ay} O} YINOS ay) Wo) BuiAoWw
ale (SYYW) S}uawiBas ayy pazuojow Aweua OM

FEAL

‘oyeisiw @ aq Aew

siuawaosojulo, jo uaweaow WayyoU ey, (ayeisiW)) AWOUR
oeue Arepuogas v 81NVax~ 0} YOU By) U! AUT}

Uo seos0 uaal6 Bunind aq Aew Awaue ay :(sonoei) Awou3,
“UINOS By} 0} Ayiea}s $9010} BuRjIYs a10}9q

YBou 24) UI JeAWJe ,BuImoys, si AWaua ay :(UoNdedap) Awau3

“UBQU 94} U! pane eABY sIbaWeoIOjUIOY

‘paltey Sey jvawidinbe Zo s,Awaua ay) :(juawdinds) Awauy
“s18yuane uals oIpes Bunmsu! Aq HOeNe ayy

JO uoNeo9} a4 Guyeeouod si AuIOUD OY (Ui p) Awioug,
“sonmoe)

o1pes Awaue eon paosjsop eney ABW UBlEdwes UONOIpsaIUI
JNO “eduaIIS o1peA 40) BIq!Suodsas jou s! Aweua ay) :sdoos 1

|

unos ay

pue YnOU ay} U! eouells oIpes payeniu! sey AWeUy

BETTE!

“Ausoue oy) Aq oyeistw

B sem a6piiq ay) Jo UOHOMuISep ay! sdeysog :(ayeIsILU) AWeUg
ojes

WeuINOs ey) U! Eq JOU [WM YORNE IEW ayy yUIy) NOA oyeWL

01 uondeoep & S| uononysap ay) sdeweg :(uondasep) Aweu3
“aBpug ayn pekonsap
Jeu) ‘Auaue ays tou 's3010) umo sno sem 41 sdeyag :sdool),

uo} SIY 0}
aBpug Jofew e peAosjsep sey e010) Awoua wayinog

suolyeue|dxy

s|uaAq

4INOS OU} UL YORNE UleW e PUR ‘YOU ay) UI JORNe AleuOIssanIp B sUBId AWOUD ou Jueuussessy

AAS TANT
“uoHenyis oy}
jO jJualussasse
JnoA BulBueyo
Japlsuoo
0} yeuBis e si SIU,
‘jJusuussesse
JUBLIND INOA Jo
adoos 984} UUM
SJUBAB OY} [Ie JO
junoooe ajqisne|d
pue ajduwis
Ajqeuoseeal ou
SI aay “ure|dxe
0} Auew 00} au0
ysnf si yey) JUeAe
ue Jajunooua
Aienquane
Aew no, “p

60


8

pajoedxeun ay) Buljpuel

Bulures, Juawssessy UOTeNUS

juauissassy ino, BHulbueyD

:payoedxeuf ou} Huljpuey

62



6E

pejedxeun au} Buljpuey Buyures | juslussessy UOHeENUS

‘Jualussasse
MAN JNOA jo suui9} Ul Wau} Ure|dxe 0} ALL

‘juowssesse
MAN JNOA Jo main jo yulod ay} Woy
yS]JU8AS pajoedxeun, MoU ase ABU} ‘OS 3]

; “juausseasse jeulbuo
4Jno& 10} suosead pey Ajqewnsaid no,

LES MAROTTA RETNA
“UONELWJOJU! BU} [}e JO JuNODIe
MAN ® e}eeJ9 ysnwW NOA
yeu} Sues LOHENYIS Bu} JO
juaussasse JnoA BulbueyD

jsuoseel jeulBuo JNO 19610} 1,U0q'

63



H

ov

|
| | Teep-pes (€0L

LOTT VA uO) STIpLY|
our] moaor 00ZP

|
'
'
j
i

"PMO
yu YPawasay YLOMUDAVI‘] FAO pT
|
i

an “VEU toanuat

|
|
Yp.vasay AULLY

jueussassy UuOHenUsS pjayayeg a

Buluresay e)



6r payoedxeun ay) BuypueH

Bulures Juauussessy UuOHeNyS

“MOS 00} 9q |IIM S|feu]
ay} JO UOeBIAeN ‘JUaWISsesse

—

JeulBu0 INOK jo suu9}
ul peeds sejtuen6 ayy urejdxe 0}
HNOIJIP OO} MOU SI }] ‘ValLee eAeB
nof suoyeue|dxe ayy jo awos

cm

“‘peyoadxe uey}
JeyolU) pue J9\/e} yonw s| uolHes
jesjuao ‘AyjIy ayy UL UOe}aHan ay,

UMOP S}JOOYS UOHEWOJU! SIU

“‘pajoedxe uey} Appnw

JO asusp SsEj SI ULE19} BpIs}|y.eyL
‘pajoedxe uey} Buy00} Jano}

aJOW SsaplAoid ureda} apis}|iy ay L

*‘payoedxe
uey) sie}4y6y aj6un{ peousuedxe
a1ouw ase seljuenb ayy

‘pajoedxe ueuy Ayub]
a1ou Huljanes} ose seqjen6B out

"Z-D ay} Woy siskyeue
Ule118} MOU & BAIGIE1 MOU NOA

‘payoadxe no uey} sayyny
%0G YUOU pajene4} sAeY seIWEND

suoneuer|dxy

sju9aq

7}JUsUSSEssy

73



METHODS FOR TRAINING COGNITIVE SKILLS IN BATTLEFIELD SITUATION
ASSESSMENT

EXECUTIVE SUMMARY

Research Requirements:

Situation assessment provides the basis for decisions by battlefield commanders and their
staff during both planning and operations. There is a need for training that enhances the
component skills in situation assessment. Such training should be based on a model of the
knowledge representations and cognitive processes that are utilized in situation assessment.

Procedure:

A framework for battlefield commanders’ situation assessment has been developed, based
on interviews with active-duty command staff and on published work in cognitive psychology.
Principles for improving situation assessment skills were developed based on the framework. The
situation assessment framework and principles for improving situation assessment skills were
developed based on the framework and the improvement principles; these training techniques are
described in the present report. The training techniques have been experimentally tested with
active-duty officers, and the results of that testing are described in a companion report.

Findings:

Two training methods have been developed. Both methods focus on metacognitive skills
involved in critiquing and correcting assessments and plans. The skills concern finding hidden
assumptions in existing assessments and plans, and handling data that conflict with assessments.

The first method is tinted to help officers find and assess the reliability of hidden
assumptions before those assumptions cause problems. In essence, the method helps counteract
overconfidence. It includes a devil’s advocate technique that forces officers to imagine that their
exploration of how potential problems with an assessment can be handled, e.g., by collecting
additional data to verify assumptions, planning against the possibility that assumptions will fail, or
accepting an assumption as a known risk.

The second method is designed to help officers find and resolve conflicting evidence. This
second method deals with assumptions after they cause problems, through inadequate assessments
or plans. This method improves the chances of noticing conflicting data by sensitizing officers to it
and providing tools for handling it when it occurs. These tools include techniques for trying to
explain the conflicting data in terms of the current assessment, evaluating the plausibility of the
explanations, and generating alternative assessments. The training discusses methods for
evaluating competing assessments in terms of the plausibility of the assumptions they require.

vii



ACKNOWLEDGMENTS

Weare grateful to other members of the project team who contributed significantly to the
research underlying these training concepts: Terry Bresnick, F. Freeman Marvin, Leonard
Adelman, Martin Tolcott, and GEN Charles P. Otstott (U.S. Army, retired).

We also thank Jon Fallesen of the U.S. Army Research Institute for the Behavioral and
Social Sciences (ARI) for this advice and support in the preparation of this report and throughout
the project. We also appreciate the comments and feedback of many others at the ARI Fort
Leavenworth Research Unit.

vi



use of metacognitive skills. We have attempted to bear these guidelines in mind while
developing training to teach metacognitive skills.

The training methods we have developed focus on two metacognitive skills. Both
methods are designed to hone officers’ skills in critiquing and correcting assessments and plans.
The first method is intended to help officers find and assess the reliability of hidden assumptions
before those assumptions cause problems. In essence, the method helps counteract
overconfidence. The second method is designed to help officers find and resolve conflicting
evidence. This second method deals with assumptions after they cause problems, through
inadequate assessments or plans. The second method may be a cure for underconfidence, or
confusion. We describe each of the training methods, below.

Finding Hidden Assumptions

The Method
The method of finding hidden assumptions consists of four steps.
1. Select a critical part of your assessment — even if you are confident of it.

2. Imagine that an infallible crystal ball (or some other “perfect” intelligence source)
tells you that this part of your assessment is wrong.

Explain how this part of your assessment could be wrong.

4, Imagine that the crystal ball now tells you that your explanation is wrong and directs
you back to step 3. (Continue until your explanations cover a variety of different
mechanisms, which seem representative.)

The product of this exercise is a list of ways an assessment could be wrong. To maintain
confidence in the assessment, one must be confident that none of these is the case. It may be
possible to dismiss some possibilities as implausible. Others it may be advisable to verify by
collecting additional data. Still others can be planned against. The end result may be that one is
as confident in the assessment as when you started. But such confidence is now earned by taking
seriously the possibility that the assessment is wrong, asking how that could happen, and
handling each possibility. Unless each possible exception to the original assessment can be dealt
with in some way, the original confidence may not be justified. This simple method reveals
hidden assumptions that underlie assessments upon which plans are based.

An Illustration
The following illustration — drawn from an interactive classroom exercise — illustrates
the method.

An officer’s assessment includes the claim that the enemy will cross the river at location
X. He bases this claim on arguments concerning the distance the enemy must travel to his
supposed objective via point X, the shallow depth of the river at X, and better concealment
opportunities along the bank at X. The officer is confident of this assessment; however stakes are
high and there is time to critique the assessment, so he does so. Following the method, above, he
imagines that a crystal ball tells him that the enemy will not cross at location X, and demands
that he explain this failure in his interpretation of the evidence. He cycles through steps three and
four to generate the following list of explanations (or negations of assumptions):

10



METHODS FOR TRAINING COGNITIVE SKILLS IN BATTLEFIELD SITUATION
ASSESSMENT

CONTENTS

Page
INTRODUCTION . 1
THE BATTLEFIELD SITUATION ASSESSMENT FRAMEWORK .....0...0::0: ce ceeeteeseeeees 1

Situation Knowledge: Enemy Intent Schemas & Action Strategies.
Value/action Structures ..
Metacognitive knowledge: Overview....... .
Metacognitive knowledge: The Quick Test
Metacognitive knowledge: Critiquing

Metacognitive knowledge: Correcting .......0..0.00cccccceeneeees nnn

TRAINING METACOGNITIVE SKILLS 00.00.00. cscceeceseecs ees eeseeeseeseseneenescesnessesseessseceseesessneeeees 9

Training Principles

Finding Hidden Assumption: .

Handling the Unexpected ......0.0..0.ccccccccccccceseeseseesesesessessssesissssssssssssseeessessssncesseseeseseness
CONCLUSION..
REFERENCES ..0.0.....scscccscssescsesvseceeseseeesesesscscsssnscsesesisacssscsesessescessesecussessessnsnsnsusisesenseeneneseseeeens 17
APPENDIX : Training Materials ..0..........ccccccccscseesesesesessesecseseseseseesessseseseessessesenenssiciseneeenseeie® 19

LIST OF TABLES

Table 1. Example of exceptions to the interpretation of evidence, and methods of handling
the ExCeptions 0.2... cecccccceeeseessrsceesesesesesessesesssesescesssssseassesuesessseecsesessesenssereneasinsenees 11

2. Table format used to train both finding hidden assumptions and handling the

UMEXPECted ooo. ee cece ccs eseeeseeseseseessesnescsesssseesnsessesseessseeeisisaeacseenenenensieeeeeteseeneeeee 14

LIST OF FIGURES
Figure 1. Framework for the adaptive decision-making model...............0...0cccscscssessesseeeeeeeees 2
2. Metacognitive knowledge and process flOW .0.........c:cccececsesecsestesestestseeseeseeteseeseneesentenees 6



Lg papedxeuy ay) Buypuey

Bulures| Juauussessy UOHeNUS

‘soysnque

dn jes pue puy 0} sn 40} Jepuey
ase Aa} os ‘umosB8A0 PeEeyMEWOS
Aiqeqoud ase suyed Agen ou

“ysnqwie JO YS 84} BONpad O}
suyed Age plone |jim Awaue ayy

. BuluyAys, INOYWM asa} |eAeI}
ues AWaUsd au} OS ‘UO}E}eHan
yBiy ym UMoIBeA0 are seBpu oul

» BululjAys,, JUBAeId 0} JapsO
ul SoBpu plone jjim Awaue aul

suoneue|dxy

sjueAq

*sunoJUod apisiiy UeYyy

Jayyes seul eBpu pue syresy Aayjea asn {jim Awoue ey sjuawissessy

ee
UONEWOJU!
Joud Buiurejdxe
jo Shem
awos pue
juauwssesse
MOU

75



vs

pejoedxeun ou} Buljpuey

Bulures| JUewssessy uoTeENS

suolyeue|dxy

suaAq

:}USWUSSESsSy

78



assumptions that support it, and to actively test it by predicting future events. The section ends
with a motivating summary of the method, a new, practice scenario with sample solutions, and a
pointer to an interactive class exercise.

CONCLUSION
The present report is part of a series of reports. Their purposes are to:

¢ describe a framework for battlefield situation assessment skill in terms of cognitive
theory

e derive implications of the framework for improving battlefield situation assessment
performance,

© generate techniques for training battlefield situation assessment, and
© experimentally test the effectiveness of the training techniques.

The training methods described above only illustrate the kinds of training techniques that
might be developed for enhancing critical thinking skills. Nevertheless, they appear to have wide
potential applicability. The kinds of skills they address seem appropriate in very diverse domains
where initial judgments and responses can be verified and improved if time, stakes, and novelty
warrant. They may apply in fields ranging from combat to fire fighting to medical decision
making. Certainly, they appear to apply across a wide spectrum of Army battlefield environments
and force configurations.

Training in effective thinking skills, as illustrated in this report, can be introduced quite
early in an officer's training and can be continued and expanded as training progresses. At the
basic level, the training could be applied to simpler tactical situations and decisions, while joint
tactics as well as operational and strategic decisions would be addressed in more advanced
training and exercises. Across all these stages of training, and across a variety of Army battlefield
specializations, the thinking processes that are taught would share a consistent set of concepts
and procedures. That same framework could serve as an aid in the design and evaluation of
exercises and wargames. The groundwork for shared situation understanding and effective
coordination would thus be laid as early as possible and would evolve into an explicit common
framework as training progressed.

16



Research Product 97-07

Methods for Training Cognitive Skills in
Battlefield Situation Assessment

Jared T. Freeman and Marvin S. Cohen
Cognitive Technologies, Inc.

19970401 046

January 1997

Fort Leavenworth Research Unit

U.S. Army Research Institute for the Behavioral and Social Sciences

Approved for public release; distribution is unlimited.



SS

peyoedxeun ay} Buljpuey

Buluyes, juawssessy uolentis

suolyeue|dxy

s|uaAq

‘JUeluSsSESsy

79



Zl suojduinssy ueppiy Bulpul4 Buyures JUBLUSSOSSY UOHeENYS

USERS DON AeA ENT TS RATAN NBT ES IO
‘sdooy NOK enow jj! NOK YoIUM ye juiod uolsioap seaj9 e sI sIu] “ssed
au} SiajUs By se Ajsnonoidsuos puke Aj1e9 }}WWOS JsnW ay ‘YoU SeAoW ey
J! JOABMOH “ssed UJeYyLOU au) ye JO YINOs aU Jaye ul HBuryoeye jo ajqedeo
s! Aweue ayy “ssed oy) wos seBiewos ey Se Yue} WaUNos siy uo Aweue
SUT Y 0} pajoe}epun pue A)y49Inb YoU eAow 0} peuoNisod aye savi0}
ANOA Jo sourjeq ay “‘SsaUuyeeMm YORE 0} aULOOP JBIAOS YUM eoUepsODOR
Ul “} O4U}S 0} Jdwsye jim Awaue oj yeu} Hulnsijeq ‘ssed ayy jo see au}
| 0} (-)apeBuq e UoMsod 0} URId NO, “UleNa} ajqisuayap ‘poob ym siiy Burjo.
jo sease uado om) Usamjaq Jsom 0} sea BuluuNs ssed urejunow e ye ‘you
4NOA 0} upes49} Yons puy NA *AWauUe oy} UO S@1l} SSEW O} YOIYM U! UTE}
Bulziweued Jo} yoo] NoA ‘playayeq ayy adeys 0} }dwaye Nod ‘sppo snoA
JonEq OL “INjssecons aq j|IM asuajep ay) YEU] AjoyIJUN SI}! Sa10} UoLIND
YUM puk ‘Awaue au} JO JOA} Ul BUC 0} XIS aq O} ONVeL 9010} a4} Ssasse
NOX "jSe9 OU) Ul SALO@lqo AwWaus Ue oO} YoRoidde jo sonuane jsulebe
ISOM OU} PAEMO} PUSJOP O} UOISSILU B YIM JOBPUBWLOD UOISIAIP B Ble NOA

39

9s1019xq


example of the potential cycles of assessment, an initial knowledge structure may be checked by
metacognitive processes, modified, and checked again, before leading to an action plan, which is
also checked by metacognition before being implemented in the environment, resulting in new
knowledge. When time is limited, metacognitive critiquing and correcting may be omitted.

To understand the potential of the model, it is important to examine each of its three
cognitive components (excluding the environment) more closely.

Situation Knowledge: Enemy Intent Schemas & Action Strategies

We partition knowledge structure in this model, as do Adams, Tenney, and Pew (1991),
into four segments:

1. The explicit focus of working memory (representing the currently attended part of the
situation),

2. The implicit focus of memory (containing the full situation model),

3. Current episodic memory (containing the history of the current problem), and

‘

+. Long-term memory (with both semantic and episodic contents).

Long-term knowledge takes the form of memory schemas. These are used to organize
situation assessment information. More proficient situation assessors appear to use a variety of
schemas. Among the most important are the enemy intent schema and associated action
Strategies.

Commanders use enemy intent structures to organize information about enemy interests,
strengths, and location, and describe how they lead to intentions, actions, and consequences.
Take, for example, a situation in which a U.S. officer must infer the intent and actions of an
enemy who may attack along northern or southern fronts. The officer knows that tanks are a
prime source of enemy strength, and that U.S. forces have situated tank killing systems in a
southern region. Enemy interests (a term we broadly define to include values, doctrine, and
goals) include doctrine to avoid opposing strength, such as northern U.S. tank killing systems.
Finally. enemy location allows attack in the north or south because terrain is hospitable in both
areas. These factors enable the U.S. officer to infer that the enemy intent is to attack to the north,
that this will involve such actions as moving artillery. massing troops, and command facilities to
the north. The enemy’s intended consequence will be a northern breakthrough.

Action strategies build on the enemy intent structure. They include three modes of
thinking about enemy intent: proactive, predictive, reactive.

In the proactive mode, the officer’s conception of the situation is predicated on molding
enemy intent by shaping the battlefield, and specifically by altering the enemy’s perception of his
own interests, strength or location. For example, an officer’s assessment of a situation may
assume that his planned deceptions will influence enemy estimations of relative strength, and that
the deceptions will persuade the enemy to adopt an intent to attack a force that is, in fact,
superior to his own.

The predictive orientation is one in which the commander uses his knowledge of enemy
interests, strength and location to predict enemy intent. For example, the officer who understands
the enemy interest, strength and location described above might predict the northern attack.

In the reactive mode, a commander infers enemy intent from the actions the enemy carries
out, or by observing their consequences. The commander hit from the north by enemy tanks

3



(potentially available

Actual present environment
(data, tasks, stressors)

|

Sat ples
infldences

Monitoring
& Regulation

Metacognitive skills

Figure 1: Framework for the adaptive decision-making model.

The basic form of the framework is inspired by Neisser (1976). In his concept of the
perceptual cycle, knowledge structures called schemas actively direct actions such as attending to
and exploring the environment. The real-world information generated by that exploration then
causes changes in the schemas. These interactions cycle continuously as the observer gains
understanding of the actual world. Connolly and Wagner (1988) extended Neisser's concept to
include decision cycles, in which exploration of the environment causes decision makers to
refine their goals. We have incorporated this extension, and have added the iterative role of
metacognition, i.e., monitoring and regulating one's own cognitive processes, in learning both
about the world and about one’s own goals.

Neisser's perceptual cycle consisted only of a path from knowledge to action to real-world
and back to knowledge. Our notion of cognitive cycle, however, includes many other pos-
sibilities. The metacognitive component may critique and correct situation knowledge and plan
knowledge; plan knowledge directs sampling of the environment; these perceptual samples
modify situation knowledge, which can direct the composition of plan knowledge. As just one

2



8¢

peyedxeun au} Buyjpuey Burures | Jusussessy UOl}eENUS

S}JU8Aq poyoedxeun Buiuresdx3
:payedxeun eu} Buljpuey

52



€L suoldwnssy uappiy Bulpul4 Burured | juslussessy UOlyeENsS

a]
Mou JOY YBnoue §,JeYL,

ERA ET TYE ORE ONT

[ (esse a a

 Arayywe yuouibne
0} woddns se asojo asn 0} suRjd Awaua ay, °G4 UOHeUeIaXy

AUT

BPAY AEE CPR NIT

RE PREETI NPI TE SE
. WY OZ 1SBe] Je Yeas UBD says ay, “BUOIM
{ sem abues (aye Awaua ay jnoge eouabyjejuj,, :p# UONPUE|AX

10} ‘Buomm | SI uoneuejdxe s
SIYL, ‘Seda a2JNOS ay | fqn

Perce!

SDN A TENEMENT ITC

, ase Boj ay} Jo
By UF OG UIYIIM 0} Aipides perow aq jm Asayive au.L,, ‘64 voneueldxy

« BUOIM os]e SI 1 voneue)dxa chet
SIVL, ‘Shes eoinos

aregeerestaet

SCPONRACA TNS AAO TU NITY AE soe

20u06I/84U! «J08HE0, UL waseq Boj ay} JO Wy OG UIYIM Ayjead Sty “Ajjoa109U) pazijeD0]

uaaq sey Aayipe pajoejep ay, :Z# UoNeuR|dxXy asiaap no,

. BUOIM SI UOHeURIAXE
SI], ‘Shes aoinos

ER Fa TTA TE TEPER RAT TEREEOT REBUILT TITIES

BIUaBI/EJU! JOGA, BY

, seq Bo} ay} jo wy
aT 0S IY Aaye pejejepun s} asay |, °1 # UoNeUL|AXZ asiaap NO,

35



oS

peyedxeun au; Buljpuey

Bulures | JUsussessy UoTenS

‘GG 0} eg sebed
se payequnu seed
94} UO Sejqe}] yue|q

84} pue - Jex0ed asp

om 3

‘Jala peal NOA yey} OUeUaDS

pue|s| ayy

Wwo4j sanss| je1enes uo ureBe anbiuyos} ayy ALL

76



uononpouyu| Hulures| JUoWSssessy UOTENS

“yes pue siapuewWoo ‘Auuy peouariodxe
jO esiedxe ay} uo paseg s| eeyew Bululed) sul

‘sue|d pue uolnoe jo sesinoo 4a}yeq sonpoid « un} ul ‘ pinoys suolyenys
Bulssesse ye paypjs SJOW B12 OYM SIBIIJJO “S||D{S JUBwISsasse UOHeNS
JOY} BUYO1 SIBOYJO 1ayIO djay o} sanbiuyoe} padojanep BACY OM ‘SMAIAOIUI

yareg ssesse Aoy}
MOY jNO PUI} O} SIBOIJJO Jye]s AW paduaHadxe OE uey} ai0W pemealAejul
‘oyNsU] YJeesey Au ay) YIM Buryiom “ou] ‘saiBojouyoe| aayub09

are

GST

‘uoljen}Is ureaoun pue xejdwoo e ssesse
Ajayesndoe O} SI JBOYJO JE]S 10 JAaPULLULUOD
Aue 10} S¥Se} }NONJIP JSOW By} Jo BUD '

23



8h

suoyduwinssy ueppiy Bulpul4 Burures | Juslussessy UOl}eNS

Wi opebuq

BY | SRR TERCERA RTE LOT
ssed au} Buissoo

\o sywuwo0s AWweuy

uMO

40



Pd



Ge payoedxeun au} Bulyjpuey

Buiureiy juaussessy UOHeENyS

BL LEE ARTA SEEMS

AE

“yoeye Auepuosas ay} jo oddns ul ase you
OU} U! SJUBWAIIOJUIAJ JEU) JNg ‘aouUa}IS OIPes ay} pasneod
pue abpuq ay} paAosjsap saoJ0} JNO yey} S| JUNODIE |]/218A0
Buloulauos jsow ayy yey} puy Aew nod ‘ajdwrexe 104

SONV IVa Isnw
YW ysadwis ay) eq

ayeisiw @ aq Kew
sluawaosojule) jo juawerofu WAYUdWOYL :(axeISIW) AWEUR
“yorne Asepyooas b a1nod)

uo se0i0} uael6 Bund eq/Kew Aipeua ayy:
“unos ayy 0} Anhuyeas saas0j Bunyys a10)aq

pou oy) Ul jeaWe ,GuIMoYs, si Awa ey :(UoNdad9p) AW2UR

jou paau juNoDIe
|Je19A0 JSEq OUL

“YHOU a4) U! PAALWE eABY s}UeWEDIOJUIBY

“pajiey sey juawdinbe Zo s,Awaua
“a1oymuend eoualls o1pes
J0 Uo}

olpes Awaue jeonuo pako.}sep envy ABW UBIO
JNO “B0UalIs OIPe! Jo) alqysuodser Jou si AWAI

aul sdoory

puByyou oi

“yynos ayy
| aoualIs OIpes payenul sey AWaUy

‘hwaue oy Aq oyeisiw

e sem aBpug ay) jo uononhsep ay) sdeweg :(ayeisiw) AWoUz
“401998

wwayinos ey) ul aq jou ym yorne UIeW oy) xUIYL NOA oyLUU

0] uonndaoap es} UoonsaA au sdeyeg (UoNdedep) AWeu3

“aBpug ey) pafosisep SIU O}
yeu ‘Awaus a2 jou ‘seos0j Umo sno sem y! sdeweg :sdoosy, | @BpHi lew e paAoysap sey eos0) Awaud

Bera MA OE

‘(uondasap

‘ajduexa
40}) Aem ewes ey}

suojyeur|dxg

S]U9A@ BAI) [1B
suiejdxe junov0e

sjuang

FREES

unos ey) Ul YORHe UleEW e pue ‘yyoU ay) Ul yoRWe AreuolssaAIp

ysojduis ay

e sueid Aweue ey, JUBLUSSOSSY

59



Utilization of Findings:

The training methods appear to have wide potential applicability. The kinds of skills
they address seem appropriate in very diverse domains where initial judgments and responses
can be verified and improved if time, stakes, and novelty warrant. They may apply in fields
ranging from combat to fire fighting to medical decision making. Certainly, they appear to apply
across a wide spectrum of Army battlefield environments and force configurations.

Training in effective thinking skills, as illustrated in this report, can be introduced quite
early in an officer’s training and can be continued and expanded as training progresses. At the
basic level, the training could be applied to simpler tactical situations and decisions, while joint
tactics as well as operational and strategic decisions would be addressed in more advanced
training and exercises. Across all these stages of training, and across a variety of Army
battlefield specializations, the thinking processes that are taught would share a consistent set of
concepts and procedures. That same framework could serve as an aid in the design and
evaluation of exercises and wargames. The groundwork for shared situation understanding and
effective coordination would thus be laid as early as possible and would evolve into an explicitly
common framework as training progressed.

viii



4. If the explanations are implausible (and if time is available and the stakes are high),
take up another assessment and repeat the process from step 1.

The first step is crucial. Noticing unexpected events is more likely if officers have
thought about the events implied by their assessments. (Event templates and war-gaming
facilitate this.) This training may also increase the likelihood of noticing unexpected events, in
two ways: by sensitizing officers to conflicting evidence through presentation of examples, and
by teaching them methods for handling it when it occurs.

In some cases, the first step of the method can be the last. That is. it is sometimes enough
simply to notice an unexpected event. This is the case if there is no time to analyze the event, if
the stakes of the situation are below some threshold, or if one easily recognizes the implications
of the event. However, unexpected events should be remembered: Officers may want to revisit
them if additional unexpected events occur whose implications are not as clear, or if the stakes of
the situation should change.

To execute the second step, explaining the event, the officer can use two tools. The
crystal ball (or “perfect” information source) is productive. It instructs the officer to imagine that
the assessment is true, and that he must explain the surprising event. Secondly, trainees often find
it useful to focus the technique by looking for explanations within each of the categories of
METT-T (mission, enemy, troops, terrain and weather, and time available). For example, the
trainee may consider how an event might be explained in terms of actions of his own troops, the
terrain or weather, enemy tactics, enemy equipment, and so forth.

The product of these methods is a set of ways in which the current assessment could be
true, despite the conflicting evidence. In order to hold onto the assessment. then, it is necessary to
accept at least one of these conditions as plausible. If each possibility is implausible, or is
disconfirmed by actual data collection, then alternative assessments should be considered.

The third step of the method, thus, is an evaluation of the plausibility of the explanations.
An important feature of this step is to look at explanations for all the conflicting data that have
been observed, not just the most recent item. The total set of explanations of all this conflicting
data must be judged plausible if the assessment is to be kept unchanged. The most plausible
overall account must consist of individually plausible explanations, but it should also be simple
— ie., not require a large number of independent assumptions. For example, an account of
events that relies heavily on enemy deception to explain all of the unexpected events is
maximally simple, but not necessarily plausible because deception may be a dubious explanation
for several specific events. A plausible account must balance simplicity against the credibility of
individual explanations.

The fourth step in the method involves taking up another assessment for consideration.
To generate a new assessment, officers can focus on the unexpected evidence by itself as a
catalyst for a new assessment. Officers examine the unexpected events they have encountered
and answer such questions as, “What do you make of these events?” or “\Vhat do these events
usually mean?” Thus, we spur the creative process by restricting the evidence from which
inferences must be drawn. Once a new assessment is formulated, however, the officer must
consider the rest of the evidence, i.e., the evidence that supported the original assessment. These
events will conflict with the new assessment. Thus, these events must be explained, using the
methods described above, in a manner that is consistent with the new assessment. These
explanations must then be evaluated for plausibility and simplicity. The old assessment and the

13



€S

payedxeun ey} Buljpuey

BHuluresy Juaussessy uOHeNyS

77

suojyeue|dxy

sjuaaq

Juawssessy



96

pejedxeun eu} Buljpuey

Bulures | juawssessy uoHeNnS

8M Jeu} OLBUSDS BU} UO pase SI UONeUSN]II

‘aBed yxau ay} uo Juasaid

“poyjew au}
gsn Jo/pue Jaquawes nod djay Aew yeu} a1qe} Y “Z

“juauussasse juaino InoA abueyo
0} Way} Buisn Aq Jo Juaussesse juan
JNOA Jo suusa} Ul Way} Bujure|dxe Aq s}juans
peyoedxeun Buyjpuey 40} seunpedoid ‘|

No MOUS [IM OMA

50



os

peyedxeun ay) Buypuey-y

Bulures) jJuawussessy UuoHenS

au} asf) {JUBWSsesse
mou JnoA

JO SWUJ9} Ul! UOPEUOJU!
Joud urejdxe noA
ued MOH gj]UaWSsesse

MAU JNOA SI yEUAA

i |

suoneue|dxy

s}uaAq]

sJuaWSsessy

74


es peyedxeun eu} Bul|puey

Bulures| Juewssessy uoyens

“ueld uondasap Awaua ue jo SW9} UI
8q Yjog osje Aew Aay, ‘Suojoe UMO JNO jo SadUaNbesuCD
se poulejdxe aq yjog jyBiw sjuane om) esau) ‘aidwexe 104

paurejdxa

“payrey sey juawdinba zo sAwaue ey
“e1eymfuane eouals oIpes Bujimnsul Aq ¥OLNe out

JO uo}e09} 84) Bulfeeou0D s} AWaue ey{ :(UORdedep) AWIOUR,
“sanjory

o1pes Awaua jeonvo pakousap eaey Aew ubledweo UoNoIpialul
JN “e0uaI!s o1pes 40) B/q!Suodse) JOU si AwWoUS ayy :sdoos,

pue you ay) ul sougs oIpes payel

“yynos ayy
sey Aweug

‘Auraue oy Aq ayeisi

® Sem aBpuq oy) Jo UoHOMsep ay sdeweg :(axeISIW) AWeUy
“10)99S

way}NOS a4) Ul eq JOU III YOR UIEW OY) 4UIYI NOA ayeul

0) uondaoap & S| UoHonsep ay) sdeyleg :(uondedsp) Aweuy
“a6puq au pekouisap

Jeu) "AWeUD 04) JOU 'SO910} UMO JNO SUM } SdLYog :sU00I],

quod} SIY O}

iq Joluw v pofo.dsop sey e910) Aweua WeUnos

suolyeur|dxy

sjuenq

UNOS 94) UI YORNe UleW e pue ‘YoU ayy U! yOeWe AVeUOISJeAIp eB suRjd AWaua oul UelUSSEessy

“JUBAB BUO UBU}
9J0W JO} JuNODOe
yey} SuolyeuR|dxe
@WWOS YM
dn awos 0} Aly
‘sJUaAd JaILeE
JO} poyesoueb
no suoieue|dxe
ay} ye YOO]

57



8 suonduinssy ueppiy Buipul-| Hulures| juaussessy uonenys

« Sjasse Bujssoso
[JOA Aue aney },,usa0p Awiaua ay |, :G# uojjeuejdxz

sue opman

FS ROOTS RRR PD RICCI MRRP RN ECR: NP ISTE

«SIX UOleI0]
pjpoob MOY MOUY j,USaOp AWaua AY |, :p# UoeUuRldxy

«/00} ‘Buowm st voneueici gp

SIYL,, ‘Saljdas ainos ayy senses

ON EERE RH

y OTA 8

} Jey) Salis Bulssos9 pooB ale aiay |, -E# uojeueldxy

 BUuOJM ose si uoeuRldxa L
| SIYL,, ‘Shes aoinos

OR = Sabor

aouabiijajul ,JELEd, ay]

« X OHIO] O} DOJ 1NO JO JUBWIBAOW AY}
joejep jim Awaue ay), :2# uoyeuesdxy asinap no,

r=

« BUOIM SI UoHeUe|dxa
SIYL, ‘Shes aoinos

EASE NOME STAY

feet error RTS IPLS ARTE OT NST HLT ATEN
. « X FE 8 |I/M 89404 1NO JeYy

iy SaJedioyUe AWaUa OY |, :L# uojjeuejdxy asinap no,
| ! YL,

souabyjajur JIapad, ayy

— « JUBW]BIDU09
- a pue ‘yidap ‘aoueysip uj

~ ayidsep ‘x ye ssouo Jou |jIm Awaua
EY} @q PINOD YW Moy Ure|dx3,

30



ov

pejedxeun au} Buljpuey Burures |. JUSWSSESSY UOIENYS

‘Gp , abed se 2 poiequunu 9qey
Ue/q ey] es Z}JUeWSSesse jUaLIND
4NoA jo adoos ey} UIyYIM UOIeWUOJU!
sly) ure|dxe no pinoo mo}

“‘payedxe nod vey} Jaypny
%OS YOu pajenei} sey aH “jYyBiujeAo
pajenes) sey Aweus ay} yey) seyeoipul

yoda, s,burusow yxeu ay) :erepdp !

70



6 suonduinssy uappiy Burpul-j Bujures Jueussessy uolTens

“MOU JO} yBnouo s SIEYL,

= - ARAPAHOE IOS TSR ENSSAT
.« Bulssos9 UBY] JEYJeI JOA AY Ssouze Jab
Oo} ynesse 412 asn jim Awauea ay, ‘64 uoneuejdxz

‘00} ‘Buoum s} uoyeurjdxa
SIYL, ‘Saljdas aoinos ayy F

EE STENT REST {

Sx Pre ma

« [l@ 18 SSO 0] paeu uop Asay quaiayip
ale sanioalqo s,Awaua ay |,, ‘gi uoeueldxy

ESTEE LT RD A CAM ENS ERB TE PERE LRT ON SONG 2 TR PR OPA TET SO

.« BUOIM osye SI uojeuejdxa “|
SIWL, ‘Shes aainos & ‘
aouabyjaju! joayed, yf

abvey e sey Aweauea oy, :Z# uoneuejdxy asinap no,

« a1ayMas|a Bujsso1o
Ul salyensea jdeooe ues Aay} yey) e010) yYBnou~a

‘Buosm SI uoyeueldxa
SIUL, ‘Shes aoinos

TEPER NSE STIS TSN BIH UTE NFU OE NS TED NN

aouabiyjaju! ,JOapad, auf

 1BYMasa
sso ued fay) poob os ase sjasse Bulssol9
AAU S,AWUBUA OY], :9# UO|eUuRIdXy asinap no,

'

31





APPENDIX

The following pages include the training materials described in the text. The material is
divided into the following sections:

Introduction 23
Finding Hidden Assumptions 25
Handling the Unexpected 45

19



Go

peyoedxeur ay} Buypuey

Hulures| Juauussessy UOHeNyS

-suoneurjdxa
OU) ALVA TVA ©

‘aunqoid uolyenys
4nok FONVHO
‘aiqisnejd jou
ese suoipeuedxe 4 “py

]

EPSTEIN SOE TN GeO ME TE SPILT
“aunjold
uolyenyis JUaIuND INOA Jo
SW] Ul S}JUaAd pajyoedxeun

NIV1dxXa 9} ALL °Z

ae

EER IVR TNR FER WIT STS I NEM TREO ALINE ENG AT DURES H STRUTT ETN BT,

iSJU9AB payoedxeun AOLLON ‘|

LON oa PRL

=

49



pe pejedxeun ayy Buyjpuey

Bulures| yuaussessy uolyenys

faa Sass ACES LA

“ajigsnieyd st yjosy Aq yova yGnoyy uong ‘suondwnsse juosayip 9014) Sobor y asnvseq
BHuloulauod sso SI yuNosoR siyy “yoRye Alepuodes oy} 0} ENP aJe YLOU OU} Ul SJUsWADIOJUIOI
pure ‘uoydedep 0} anp si! aauaiis o1pes ‘eBpiiq ey) peAoysep $800} UMO INC :ajdwex~a 104

‘ayes e oq ew

sjueweosojUjas Jo JUawenow Wayyou ay) :(eeISILU) AAU
“yoeye Alepuooas e ejnoaxa 0} YOU ay} U! aul}

uo $2010) ueai6 Bunind eq Aew Auiaue ey (sonoe}) AWoUS,
“yinos 64) 0} Auyyears sao10) Bunyiys e10}0q

uou ay} ul eaWe ,uIMoYs, s! AWeUa ey, :(UoNdasep) AUeUy

“yuoyeu

Poause erey s}UsWadIOJUIOY

“paliey sey uawidinba zo sAweue ayy
“eraymuane aouails o1pes 6

JO vonea9} ay Buljeaou09 s} AWaUa ay),

atudinba) Awauz
ul Aq oene OM
(uondesap) Awau3,

Jeu) ‘AWaue ey) Jou ‘seo10j UMO INO sem | Sdeysad :sdooly,

olpes Aweua jeonuo pekossep aney Aew ubledweo u “yinos au)
4NQ “2dUA}!S OIpes 40) B/qISUOdSO, Jou Ss! AWaUE By) pue You Ay) Ul eoUa|Is O|pes payenjul sey AwaeUy
“Auiaue oyy Aq oyeysiws
8 sem e6p1q @y) Jo UoHNsep ay) sdewag :(oyeIsIW) AWeUZ
‘sopes
WEYINOS BY) U! q JOU |IIM YORYE urew ayy yUIY) NOK oyeW
0} uondeoap es} uoyonsep ey) sdeyeg (uondadep) AwoUy
“epg ey) peAoyjsep quo4sy siy 0}

Bpuq sofew e pedosjsep sey 9010) Aweue Weyynog

suoneue|dxy

sjusay

UNOS EU UI HORE YEW B PUe “YOU oy) UI YoBNE AiBUOISJeAIp e sURId AWeUD EYL "JUSLUSSOSS\YY

"JUaAe

auo ue} a0
0} Ajdde yeuy
suoneuerjdxa

0} WHIM B1yxe
@WOS SAID

58


Research Product 97-07

Methods for Training Cognitive Skills in Battlefield
Situation Assessment

Jared T. Freeman and Marvin S. Cohen
Cognitive Technologies, Inc.

Fort Leavenworth Research Unit
Stanley M. Halpin, Chief

U.S. Army Research Institute for the Behavioral and Social Sciences
5001 Eisenhower Avenue, Alexandria, Virginia 22333-5600

Office, Deputy Chief of Staff for Personnel
Department of the Army

January 1997

Army Project Number Personnel Systems and
20262785A790 Performance Technology

Approved for public release; distribution is unlimited...



OL suojduinssy ueppiy Burpul4

Hulures| Jualussessy uolyenys

-uonoe 0: inok juowssesse Inok oul

Way} UO ezifeyded Jo ‘Wey ysureBe ued ‘way AyWaA 0} Au) ‘Ayjiqisne|d
4ouy) ayenjeag :suojdwnsse Aue eipuey pinom nod se eseuy e]puey
“suolduinsse uspply puy ued nof ‘ase oq yyBIw yuawssesse unoA
jo ped Aey e moy Bulurejdxe Ag ‘sesseuyeam jenuejod JEPISUOD 0} NOA
$8910} | ‘JUBWSsesse INOA Jo ped Aay eB ynoqe jee} AewW NOA jUapyUoD
E| Moy Jaye ON “sjuaussesse enbyiJo 0} noA sdjay poyjew siyy ‘wins uj

38



VE

suojdwinssy ueppiy Bulpuly

Hujures| Juawssessy UOIeNS

SEPTATE LT

8 URES RIAN NL ERTS TRLIS

‘sasy Alayeqsajunoo
pue Jepes Buyjezo}
Buisn Aq Buyjays Avaye
isuyeBe puajap 0} uel

‘sjasse
uoddns se Awaua jabse 1
Auayie Ajddnse 0} pasn
aq 146i Jey) S907 AoNseq

SENS

“asjej ino sun}

uondunsse ue seo

ul suejd AouaBuquoo
dojensp ues no,

‘and oe Suondwnsse
ey} yey) ons
Oye O} playaleq ayy
adeys 0} Aly ued no,

RIES ON REMASTER EE

‘saonjaeid yoddns se asojo
Jo ‘spoyjou abeynoweo
‘ebues Avaye Awauae
Bujuseouod UONeWOJU)
asow s8yjeb ‘ajdwiexa 10-4

CeROPRR TRE

‘suolduinsse ayy
| Aju@A o} Aly ue NO,

suoljdaoxy ay} Buryenjeag

36



LE

pepedxeun ou} Burypuey

Hulules| justussessy uoeNUsS

“anuy Ajsnoaueynuis
ese suoieueldxe JnoA
B) $0 [JE yey} Ajoysun SI}

*s]UaAe peyoedxeun
SU] JO} UoeUR]dxa
alqisnejd e ayeaio
},uUe0 NOA UsYyM JO

Sy ae Cae aha ind Sata
“SJUBAO
peyoedxeun jo Jaquinu
i] e618] & palajunoous
98A,NOA UBM

| 912 10.98 Ue JO S}SOO
9y} UAYM UOHeNIS ouUy
JO Juolussesse JNOA 1
eBueyo pinoys no,

61



QUICK TEST

Situation
Model

(1) Do | have some time before | must
commit to a decision?
(2) Are stakes of an error high?

Verity

Real World

(3) Is the situation unfamiliar, atypical?
Or is there some specific problem with
the model or plan?

Verify

No to 1,2, or 3: Yes to 1,2, &3

CRITIQUING

(a) Discover conflicts (Mentally simulate
expectations, compare with data or goals;
get others’ views; adopt different points of

view).

Problem well
understood?

(b) Uncover unreliable assumptions
(Imagine how each step of reasoning could

be wrong). Yes
(c) Test for incompleteness in model or |
plan (Checklist, template, SOP; mental

simulation). CORRECTING

(1) Collect more data,

ised situati: 2) Activa’ ditional s of
ie etgae—]_ ) Activate acettional part
-- and/or --
(3) Adjust assumptions, select an
explanation

Figure 2: Metacognitive knowledge and process flow

Metacognitive knowledge: The Quick Test

The Quick Test is a gate-keeping function that determines whether (1) to engage
critiquing and correcting processes that might improve problem recognition or (2) whether the
current level of understanding can (or must) suffice.

This process answers the question: Is there some reason to think more about my current
model or plan, or should I act immediateiy? The answer is based on three more specific
questions: (1) Do I have time before it is necessary to commit to a decision? (2) Are the stakes of
an error high? and (3) Is there significant reason to doubt my initial situation assessment or plan?
Quick Test skills thus involve sensitivity to the availability of time and potential costs of delay:
sensitivity to the costs of errors that might occur if one acts now on the best solution to date: and

6



ve

peypedxeun uy Buljpue}

Bulures| Jugussessy uolens

0} Aresseoau si 1] UayM pue}siapuNn
noA sdjey jueseid am poujow ey

‘pulw JnoA aBueyo

 RAATALONTIT

“UOIPELOJUL MOU OU}

4O} S}UNODOe Ja}yeq
yey} uoHenys ayy
j$O JUSLUSSESsSe MoU
Be aye9J90 ue noA

‘NOILWALIS SHL
dO LNAWSSaSSV
YNOA SONVHO

“SJU9AO
payoedxeun Jo} suojeurjdxe
Jayjoqg ayees0 O} J -) LAW pue ,ifeq
JeysAuo,, ay) asn Oo} MOY NOA smous
alay Juasaid am poujaw oy

C2 | ROTTER
‘juolussesse
yeu} BurlBueyo
| INOUWM ‘JUsLUSSEsse
JUALIND ANOA YM Sy
UOHEWIOJU] MOU BU}
Moy JO uoHeuR;dxe

ue asiAap ued

NOA *LI NIV1dXA3

SARWAN PRE TTT TRIE REM EN ITI

‘uonIsodsip Awaus uo squab
jsed jo soda se yons ‘uoeuoju! pue sjuane peypedxeyn
0} ayelu ued NOA sesuodsad jo sadA} Om) Ose O190U

JU! JUBLIND JO ‘AYAOe Awwaua

48



O€ payedxeun ey) Buyjpuey

Hulules | JuauUssessy UOTeNIS

“Aobeye9 J - | |W yors

uly suoleuR|dxe yo yULY) 0} Au) UBD NOA,

PTI RTE

w SIY} Ule|dx>] “UaAe payoedxeun
ay} ay!dsep ‘anu S] JusWssesse juan INo,,,
‘swule|9 891NOS UONeWHOJUI ,JOSLed, B yeu] eulbew|

“Awaue ou Aq axe}

B sem oBpLiq oy) jo UoHoNASap oY) sdeYog :(oywIsIW) KtuOUy
4010s

WEYINOS BU) UI 8q JOU jIIm HORNE ULEW ay) YUILY NOA ayeW

01 uondaoap k s| uojionuisap ay sdeyeg :(uojdeoep) Aweug
‘aBpug 8y) pekousep

Jey) ‘Aweue ey) jou ‘se0s0j UMO sno sem 1 Sdeyseg :sdoos 1,

yuo) st
eBpug iofew e peXoysep sey 2010; kuoue woatiea|

suolyeue|dx9

NY

P—___sjueny

4INOS OU) UL YoRHE Ue e pUe ‘YHOU ey) UL yoRHe AJeUOISIEAIp B SUID AWoUS OU :JuauuSsessy

AER TERRY

‘juaussesse
juouno ayy
lM Jua}sisuoo
y ayew
0} UoeWOJUI
peyoedxeun
au NIVIdXS
OV AIL *Z

54



problem is found, the third major function of metacognition is enlisted: facilitating the
construction of an improved model or plan. Whatever problem is discovered, three methods are

available to solve it:

1. Collecting more data to fill gaps in the model or plan, confirm or disconfirm an
assumption, or to resolve conflict

2. Activating existing knowledge in long-term memory, for the same purposes

3. Adding assumptions to fill gaps or resolve conflict, and dropping or replacing
assumptions when they appear unreliable or conflict with other information or
assumptions

Metacognitive processes play a role in choosing among these processes, and in regulating
the process that is chosen: (1) in selecting the amount and type of data collection, (2) in directing
the search for knowledge in long-term memory, and (3) in adjudicating among competing
possible assumptions.

Data collection. Sometimes there is time and opportunity to collect additional data to
flesh out or resolve ambiguity in a model or plan, or confirm or disconfirm doubtful assumptions.
The decision to collect more data rather than simply think about the problem involves
metacognitive judgments regarding the amount of available time, the cost and potential risks of
data collection, and the trustworthiness of information sources.

Knowledge activation. Metacognitive processes are crucial in guiding the serial
activation of knowledge in long-term memory. Such searches are conceived of as controlled
spreading activation. Executive processes determine which components of the current model will
be attended, thus influencing the portions of long-term memory likely to be activated next
(McClelland & Rumelhart, 1986). The values of the attended nodes are fixed, or clamped, at a
high level of activation (in effect, accepting them provisionally or by assumption) in order to
explore their implications. In the next cycle, new nodes may be clamped, and so on, until
knowledge is activated that satisfies the goals of the search (or quick verification determines that
time has run out). Different officers will attend to aspects of the situation in different orders.
Some may focus attention on knowledge of terrain, others on knowledge of enemy strength.
others on knowledge about enemy goals, and others on knowledge of enemy actions.

Metacognitive control may influence search in another way, by adjusting the degree of
similarity required for a match between patterns in active memory and stored structures (cf..
Hinton & Sejnowski, 1986). When the threshold is set low, the activation net is cast wide, and
far-fetched ideas have a significant chance of being considered. When the threshold is set high.
an idea must have a very high degree of association with currently active beliefs to have a chance
of being activated. Low thresholds may be used. for example, when schema is readily available
that fits all the data, or no plan is readily available that can achieve important goals.

Adjusting assumptions. If data collection is infeasible because of limitations in
resources, time, or sources of information, and if definitive knowledge is not available or cannot
be accessed from long-term memory, the situation assessor may adopt assumptions to remedy
incompleteness in his model of the situation. Metacognitive processes are crucial in the
interpretative process of evaluating and revising assumptions. Decision makers think and act as if
assumptions were true until there is some reason to doubt them. Conflict between data and a
situation model, or between two competing models, provides such a reason for doubt. Conflict
indicates that at least one of the beliefs involved in building the models or interpreting the data

8



REPORT DOCUMENTATION PAGE

1. REPORT DATE 2. REPORT TYPE 3. DATES COVERED (from. . . to)
1997, January Final January 1994-September 1996
4, TITLE AND SUBTITLE 5a. CONTRACT OR GRANT NUMBER
MDA903-92-C-0053
Methods for Training Cognitive Skills in Battlefield Situation 5b. PROGRAM ELEMENT NUMBER
Assessment 0602785A
6. AUTHOR(S) 5c. PROJECT NUMBER
A790
Jared T. Freeman and Marvin S. Cohen (CTI) 5d. TASK NUMBER
111
Se. WORK UNIT NUMBER
C04
7, PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION REPORT NUMBER
Cognitive Technologies, Inc.
4200 Lorcom Lane
Arlington, VA 22207 95-2
9, SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. MONITOR ACRONYM
US. Army Research Institute for the Behavioral and Social Sciences ARI
ATTN: PERI-RK
5001 Eisenhower Avenue 41. MONITOR REPORT NUMBER
Alexandria, VA 22333-5600
: Research Product 97-07

12. DISTRIBUTION/AVAILABILITY STATEMENT

Approved for public release; distribution is unlimited.

13. SUPPLEMENTARY NOTES
For related research see ARI Technical Report 1050, Training Critical Thinking Skills for Battlefield Situation Assessment: An
Experimental Test by Marvin S. Cohen, Jared T. Freeman, Jon J. Fallesen, F. Freeman Marvin, and Terry A. Bresnick.

14. ABSTRACT (Maximum 200 words):

Situation assessment provides the basis for decisions by battlefield commanders and their staff during both planning and
operations. In previous work, we developed a framework for battlefield commanders’ situation assessment from interviews
with active-duty command staff and from published work in cognitive psychology. The present report describes methods for
training cognitive skills in situation assessment based on that framework.

Two training methods have been developed, both of which focus on metacognitive skills involved in verifying and
improving assessments and plans. The first method helps officers find and assess the reliability of hidden assumptions. It
includes a devils’ advocate technique that forces officers to imagine that their assessment is wrong and to explain why, as well
as techniques for handling potential problems that are found. The second method helps officers find and resolve conflicting
evidence. It includes techniques for trying to explain the conflicting data in terms of the current assessment, evaluating the
plausibility of the explanations, and generating alternative assessments. The training techniques have been experimentally
tested with active-duty officers, and the results of that testing are described in a companion report.

The training methods appear to have wide potential applicability in military as well as nonmilitary contexts.

15, SUBJECT TERMS.

Situation assessment Decision making Battle Command Training
Critical thinking Planning Cognition Metacognition
: SECURITY CLASSIFICATION OF

19. LIMITATION OF | 20. NUMBER 21. RESPONSIBLE PERSON

ABSTRACT OF PAGES (Name and Telephone Number)
16. REPORT 17. ABSTRACT | 18. THIS PAGE

Unclassified | Unclassified | Unclassified | Unlimited 83



Z suonjduunssy uappiq Buipul{ Bu uled| JUsSsSessy UOHeNS

« ule|dxe aseatq

*X UOT}ED0]
| 6 JE OAL OU} SSOIO
\ you jim Aweue oul,

“suoljdwinsse uepply 10}
Yyoreas 0} aploap no os ‘yBiy
Alaa ase Saye]S BY) “EAEMOH

*X ye Ssodo {JIM Awaua
Ou} Jey) JUBPYUOD AIBA ja0} NOA

:shes aounos
aQquebljayu!
wWoeped, ay

mares een

ich cd ocr x 2 Pema Peers aie
*X UO!Te00] Je seoJ0} i *X UOHTeDO] Je JAA 4} SSOJD j||IM AWauUa 9U) ‘salUNUoddo
UMO ayeIUBDUOD g 1UOW}eadU09 puke “aA ayy jo Ydep ‘jane.] 0} BoUR]SIP UO paseg
ueid lapow UoHENys

AIdNVX4

29



Lv

payedxeun eu) Buljpuey

Bulures| Juawssessy UOHeENS

suojeue|dxy

sjuaaq]

‘JuUsluSSEsSsSY

71



REFERENCES

Adams, M. J., Tenney, Y. J., & Pew, R. W. (1991). Strategic workload and the cognitive
management of advanced multi-task systems. BBN Systems and Technologies.

Beach, L. R. (1990). Image theory: Decision making in personal and organizational contexts.
New York: Wiley.

Beach, L. R. (1993). Image theory: Personal and organizational decisions. In G. A. Klein, J.
Orasanu, R. Calderwood, & C. E. Zsambok (Eds.), Decision making in action: Models
and methods. Norwood, NJ: Ablex Publishing.

Charness, N. (1981). Search in chess: Age and skill differences. Journal of Experimental
Psychology Human Perception and Performance, 7(2), 467-476.

Chase, W. G., & Simon, H. A. (1973). Perception in chess. Cognitive Psychology, 4, 55-81.

Chi, M., Glaser, R., & Rees, E. (1982). Expertise in problem solving, In R. S. Steinberg (Ed.),
Advances in the psychology of human intelligence (vol. 1). Hillsdale, NJ: Erlbaum.

Cohen. M. S. (1993a). Metacognitive strategies in support of recognition. Proceedings of the
1993 Human Factors and Ergonomics Society 37th Annual Meeting, Seattle, WA.

Cohen, M. S. (1993b). The naturalistic basis of decision biases. In G. A. Klein, J. Orasanu, R.
Calderwood, & C. E. Zsambok (Eds.), Decision making in action: Models and methods.
Norwood, NJ: Ablex Publishing.

Cohen, M. S., Adelman, L., Tolcott, M. A., Bresnick, T. A., & Marvin, F. F. (1993). A cognitive
framework for battlefield commanders’ situation assessment. Arlington, VA: Cognitive
Technologies.

Collins, A., Brown, J. S., & Newman, S. E. (1989). Cognitive apprenticeship: Teaching the crafts
of reading, writing, and mathematics. In L. B. Resnick (Ed.), Knowing, learning, and
instruction: Essays in honor of Robert Glaser. Hillsdale, NJ: Erlbaum.

Connolly, T., & Wagner, W. G. (1988). Decision cycles. In R. L. Cardy, S. M. Puffer, & M. M
Newman (Eds.), Advances in information processing in organizations (Vol. 3).
Greenwich, CT” JAI Press.

Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-
developmental inquiry. American Psychologist, 34, 906-911.

Forrest-Pressley, D. L., MacKinnon, G. E., & Waller, T. G. (eds.). (1985). Metacognition,
cognition, and human performance (2 vols.). New York: Academic Press.

7



U.S. ARMY RESEARCH INSTITUTE
FOR THE BEHAVIORAL AND SOCIAL SCIENCES

A Field Operating Agency Under the Jurisdiction
of the Deputy Chief of Staff for Personnel

EDGAR M. JOHNSON
Director

el
Research accomplished under contract
for the Department of the Army

Cognitive Technologies, Inc.

Technical review by

Rex Michel, ARI
Julia Pounds, Pounds Associates

NOTICES

FINAL DISPOSITION: This Research Product may be destroyed when it is no longer needed.
Please do not return it to the U.S. Army Research Institute for the Behavioral and Social Sciences.

NOTE: This Research Product is not to be construed as an official Department of the Army
position, unless so designated by other authorized documents.



6h

suonduinssy ueppipq Bulpul{ Bulures | juslussessy uohentis

id c5 RFR
“suolyeue|dxa euos
94M 0} ped ay) Woy

Jaded Jo jaays eB ascy)

REET

41

Zueddey siy} pinoo mo ‘ssed auy yBnosu)
ORHE JOU |IIM AWaua ay} yeu} spears ,|1eq feyshio, ay, “ssed
Syl ye yoeYe 0} JUajul sAWeua ay} UO sebuly juaWssesse INO,

anbiuyos} ayy Addy



suondwinssy ueppiy Bulpuly Burures | jusLUSSEsSsy UOeNHS

ETRE KEN SI SHOU FU TAT DERI THY OMRON PRESENTA TRIPE MEYER NON NERS TENA

suoljduinssy usppiy Bulpul4

25



ch

suondwnssy uappiy Buipul4

Hulures| JUaUssessy UOTeENS

“ea Awaua Woy ayes
JOU Ss! aseq sonsiIBo} oyL,,

:sAes aounos
9a0ueb||}a}U!
wOopad, oy]

‘suondwnsse
UsppPlY JO} Youeas 0} aploap
no& os ‘yBiy Aiaa ase seyeys
OU} ‘JEAEMOH “ody Aua}|Ie
Wwo4 ayes s| aseq Boj nok
yey] JUBPYUOD AYaA 9a} NOA

‘ebues Suayjipe Awaua episjno

Ajajes sj aseq Boy ino (aouas
Wy Og B sey Auayiyie Jo adf} pajoadsns OU) Je} Sajejs UojeuUOjU) BdUabyjaj}U] ‘UOHISOd /valNE
Auiaua payiodes jse| 84} WOL Wy 09 pa}eo0] Mou Ss} aSeq PsJEMJOJ INOA ‘YORE ay) Loddns oO}
PIEMIOJ [[9M aseq Sd1}sIBO] [eID & panow aaey Nok ‘uoyoe aaisuayo Buswos e 404 uoesedasd Uy

RET
asus Ue) eIojasayy ‘abues

ASIOYUAXA

34





would, clearly, infer the enemy’s intent, and possibly reconstruct the causal chain to that intent
from enemy interests, strength and location.

These strategies are not mutually exclusive. A predictive strategy may employ reactive
methods (i.e., observations of enemy actions) to confirm the predictions. A proactive strategy
may use predictive methods to decide what actions would produce the desired enemy intent, and
may use reactive methods to confirm that the attempt to influence enemy intent was successful.

There are a range of other schemas that officers bring to bear on situation assessment. We
will not discuss them here, but the following list conveys their character:

¢ Enemy goal structures describe the hierarchical and compensatory relationships
among values, goals, and actions.

¢ Temporal plan execution structures provide a more detailed description of the
temporal durations, precedence relations, and causal contingencies among actions and
events.

¢ Enemy planning structures describe the enemy roles and activities involved in produc-
ing, communicating, and implementing plans.

e Terrain structures relate terrain features to expected enemy actions and prescribed
friendly actions.

Value/action structures

Actions, goals, and values reflect a qualitatively different way of viewing knowledge.
They represent how possible states of affairs are valued, whereas situation knowledge represents
how strongly they are believed. Values or preferences are importantly different from strengths of
belief. They influence beliefs, but are separate from them. We draw on Beach’s (1990) concepts
of the value image, trajectory image, and strategic image to define these segments.

High-level values are relatively permanent knowledge about what the decision maker
regards as desirable, important, and worth pursuing.

Current goals reflect the desired sequence of states from the present into the future: they
are concrete realizations of high-level values in the current situation. Goals thus involve episodic
memory. Goals are used to generate plans and provide the larger meaning for specific actions.

The current plan is the detailed set of actions and action contingencies that the decision
maker has adopted in the current situation. It includes the specific actions (e.g., "move up follow-
up forces,” “emplace artillery," look for a kill zone, etc.) undertaken to realize goals.

The part of the plan active in working memory is the immediate focus of evaluation. Such
evaluation may occur prior to implementation as part of the decision making process, or during
implementation by monitoring an on-going action for its success in achieving goals.

Processing of actions, goals and values can be either top down or bottom up (Beach,
1990). Actions and plans may be generated and evaluated based on goals, and modified or
rejected if they fail to achieve them. Similarly, goals may be generated and evaluated based on
values. On the other hand, from a bottom-up perspective, goals may be revised if no actions can
be found to achieve them. Even high-level values (such as maintaining the initiative, or attacking
the enemy’s center of gravity) might be revised (perhaps rationalized away) if they are not
achievable by realistic goals or actions.



cf

payedxeun ey} Buypuey

Hulures| juswssessy uolenS

“suoljeue|dxe

ayy Jo Ayjiqisnejd
OW. ALVNIVAA €

SRR TO LY NECTAR TARAS SR

(eq jeIsAu9,
9@u] esp) JUaWssesse

jualin INOA Jo adoos
8Ud UUM WENIW 1d XA 2%

PSA ENTENMARET AE

“paltey sey juewdinbs zo sAwoue ayy, :(juoU Wwauy
“esoynvuane aoualts oipei BuIMyNSUt Aq yOeN_OuR

J0 YoNedO} ey) BueadUOd s} AWaUE ey) (UONda0ap) AWaUZ,
“sanyson)

o1pes Awaue jeonuo pafonsap erey Aew ubiedweo uonoIpseyul
INQ ‘2DU9}!S OIpes Jo} ajqisuodsay Jou si AWaua oy, :sdoory

a ets
BIUEyiV-)

payadxeun ey} JOLLON ‘1

“yinos ayy
1 sey Awaug

Pue you ay) u! eouayis olpes payer

“Auoue ayy Aq oyeisiws
J (0 ) AWOL
sojas

WayINos ay) U! Bq JOU III YORWE ULEW ey) YUIy) NOA aye

01 uondaoap ® si uoNoNsEp ey) sdeWeg :(uondedep) AWeUR
‘a6piq ay) pakonsep

Jey) ‘Awaue 94) Jou 'se010) UMO JNO Sem II Sdeed :sdoosy,

8 sem OBpiiq 94) Jo UoHTONsISo

Ju04} S1y 0}
abpuig iofew e pakoisep sey oso} Aweua weyjnos

suolyeuejdxy

s]uangq

Yynos ey) Ul yORNe UleW e pUe ‘YOU ey) Ul yoRNe

Aseuoissanip e suejd Aweue ay ‘Juslussessy

y

A CR RN R I

40} einpesoid

“MO||O}
yey} s}uaAa
poejoedxeun

Jaujo Aue

owes
Siu] Mo||O4

56


8V

payedxeun eu) Suljpuey

Bulules| juswussessy uoHens

éjno yooyo Aay}

aseu] jo Aue }|

fl OP Ing “en eq |INspjnoo
| |Uaussesse jeulBLO uNOA
jno syoeyo suo|eueRjdxe

“‘pajoedxe uey} Appnw

JO asuap Ss9] SI ULE9} ApIs||ly BU,
‘pajoedxe uey} Buljoo) Jano}

SOW SEPIAOId UNE} ApIsiiy OU_L
“pejedxa

uey} si9]46y a/6un!{ peouaedxe
alow ae seljuen6 ey.

“‘pejoedxe ueuy Ayy6

a0 Buljenesy ave sejjien6 eu

{PASTA ARETRRANE

a

UO!PEWOJU! SIY}
JO suolyeuerjdxe
aiqissod awos

72

“ele

‘peyoadxe nod uey} Jeyyny
%0S YOu pajenes] avey seyWeNndy

suolyeue}dxy

sjusaq

‘seul| eBpi uo pue sAayjen ul

JJ} PIOAR |IIM SeIILWEND

‘JUsUSSESsy



02d suonduunssy uappiy Bulpul4 Burures | JUSWSSESSY UOIeENUS

PETIA TRE SEER zi as
“sJUSWOAOWW “OAIAINS
doo} INOA apiy oy 0} 80981 AWaua
suolneoeid jeuolippe ewWOs Moje }sNW
eye} UBD NOA sdeylag 8010) INOA ‘asojasay
EES OE NARI TT
“yInos ay] Ul SsouUyeaM Mau INOA Rea P| TIN Q
yoeye pue ‘Juowenow Bupjuey *8010} Buanoo nod Aq peAossep
® Joadsns ‘you ayy 0} s8d10} S|] 89081 siY je asNedaq YOU eu}
ule INOA jo JuUBWaAOW JOe}ep ul eam ase NOA yeu} ULE] JaASU
qyBiw Aweue ay, :z# UoIpeuR|dxy Aew Awaus ayy :1# uoleur|dxy

ERASER PONT LET OE RTP TAT PITT TIA “3 n
‘JaA00uN UB | |
anbluyos} siyj yey] suoeuejdxe om) ale ale}





sensitivity to the typicality of a situation and, conversely, to the presence of unusual or troubling
features. If all three of the Quick Test conditions are satisfied, Quick Test inhibits the
recognition-based response and triggers a process of knowledge-based reasoning. If at least one
of the Quick Test conditions is not satisfied, the initial model or plan is accepted, and no
critiquing or correcting takes place.

The R/ M model integrates concepts from other models. According to Klein (1993),
rapid recognition-primed decision making is expected under conditions of high time pressure.
According to Connolly and Wagner (1988), rapid processing may occur when there is low cost of
an error. According to both Klein (1993) and Rasmussen (1993), it is expected in highly familiar
situations, that is from decision makers with situation-specific expertise. If any one of these
conditions is true, then the answer to the corresponding Quick Test question is “no,” and
correcting and critiquing do not take place.

The Quick Test can be a relatively explicit and conscious process, or a form of
recognitional processing at a higher level that can be extremely rapid and virtually automatic.

Metacognitive knowledge: Critiquing

An initial situation model or plan may fail the Quick Test because specific problems are
apparent or simply because the situation is complex or unfamiliar. In the latter case, the next goal
in metacognitive processing is to answer the question: Are there specific potential problems with
the current model or plan? Critiquing is the stage of ferreting out such problems.

Critiquing can result in the discovery of three kinds of problems in the current model or
plan: incompleteness, unreliability, or conflict. Situation understanding or planning is incomplete
if conclusions and options are not specific and detailed enough. Understanding and planning may
be complete but unreliable if the link between data and evidence, or the link between actions and
goals, is uncertain or conditional. Finally, even if understanding and planning are complete and
free of obvious unreliable assumptions, there may be an alternative, conflicting conclusion that
better accounts for some of the data, or an alternative incompatible action that better achieves
some of the goals.

Some critiquing methods are general-purpose. They are capable of uncovering problems
of all three kinds. Mental simulation and retrieval of similar cases from experience are general-
purpose in this sense. For example, by mentally simulating a course of action, an officer might be
able to see if the current plan has any gaps, if it reliably achieves goals, and whether it conflicts
with other goals. Similarly, comparison of a plan with an analogous previous experience might
reveal gaps in the present plan, suggest places where the plan might not work reliably, or suggest
alternative actions that have been adopted in the past.

Other critiquing methods are more specialized in the kinds of problems they can uncover.
For example, decision makers might use a checklist or standard operating procedure to ensure
that all required components of a model or plan have been specified (completeness). They might
adopt a devil’s advocate technique in order to ferret out unreliable assumptions in an assessment
or prediction. Data collection can determine if observations are consistent or conflicting with the
current situation model or plan.

Metacognitive knowledge: Correcting

If no specific problem with the model or plan is identified by either the quick test or
critiquing, then metacognitive processing in the current cycle is complete. But if a specific

7