Toward the Development of a Predictive Computer Model of 

Decision Making During Uncertainty for Use in Simulations 

of U.S. Army Command and Control Systems 

by Sam E. Middlebrooks and Brian J. Stankiewicz 

ARL-TR-3719 January 2006 

Approved for public release; distribution is unlimited.
NOTICES 

Disclaimers 

The findings in this report are not to be construed as an official Department of the Army position 

unless so designated by other authorized documents. 

Citation of manufacturer’s or trade names does not constitute an official endorsement or 

approval of the use thereof. 

DESTRUCTION NOTICE⎯Destroy this report when it is no longer needed. Do not return it to 

the originator. 
Army Research Laboratory 

Aberdeen Proving Ground, MD 21005-5425 

ARL-TR-3719 January 2006 

Toward the Development of a Predictive Computer Model of 

Decision Making During Uncertainty for Use in Simulations 

of U.S. Army Command and Control Systems 

Sam E. Middlebrooks 

Human Research and Engineering Directorate, ARL 

Brian J. Stankiewicz 

The University of Texas at Austin 

Approved for public release; distribution is unlimited. 
ii 

REPORT DOCUMENTATION PAGE Form Approved 

OMB No. 0704-0188

Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining 

the data needed, and completing and reviewing the collection information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for 

reducing the burden, to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, 

VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to any penalty for failing to comply with a collection of information if it does not 

display a currently valid OMB control number. 

PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS. 

1. REPORT DATE (DD-MM-YYYY)

 January 2006 

2. REPORT TYPE 

Final 

3. DATES COVERED (From - To) 

 

5a. CONTRACT NUMBER 

5b. GRANT NUMBER 

4. TITLE AND SUBTITLE 

 Toward the Development of a Predictive Computer Model of Decision 

 Making During Uncertainty for Use in Simulations of U.S. Army 

 Command and Control Systems 5c. PROGRAM ELEMENT NUMBER 

5d. PROJECT NUMBER 

 62716AH70 

5e. TASK NUMBER 

6. AUTHOR(S) 

 Sam E. Middlebrooks (ARL); Brian J. Stankiewicz (UTA) 

5f. WORK UNIT NUMBER 

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 

 U.S. Army Research Laboratory 

 Human Research and Engineering Directorate 

 Aberdeen Proving Ground, MD 21005-5425

8. PERFORMING ORGANIZATION 

 REPORT NUMBER 

ARL-TR-3719

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR'S ACRONYM(S) 

11. SPONSOR/MONITOR'S REPORT 

 NUMBER(S) 

12. DISTRIBUTION/AVAILABILITY STATEMENT 

 Approved for public release; distribution is unlimited. 

13. SUPPLEMENTARY NOTES 

14. ABSTRACT 

 In today’s increasingly complex world of digital command and control, it is seldom obvious or intuitive how the introduction of new 

automation systems will affect the overall performance of battlefield command and control (C2) systems. Field observations can account 

for performance factors that are directly observable, such as rates of communication flow, rates of flow, and quality of incoming 

intelligence. However, what the human mind does under the influence of all these factors is not directly observable and is the subject of 

considerable experimentation. 

This research addresses this limitation through the development of predictive quantitative models of decision making during conditions of 

uncertainty such as exist in many aspects of human performance and certainly in battlespace management. Using Bayesian statistical 

approaches implemented through Partially Observable Markov Decision Processes (POMDP) that describe experiential decision processes 

moderated by Monte Carlo effects to account for performance variability, we are developing a series of computer simulations with the 

goal of predicting the quality of decisions possible from a given set of input conditions. 

These simulations are based on cognitive models being developed in a collaborative effort through a series of empirical studies that 

investigate human performance in a sequential decision making with uncertainty task using human subjects. Through this collaboration, 

the results of these studies are being applied at each stage of the research to predictive computer simulations of Army battlefield 

performance where battlefield automated C2 systems are involved. These simulations, when operational, will allow cognitive effects, 

such as predictive levels of effective decisions possible from a given set of circumstances, to be assessed as a battlefield metric. The 

usefulness of these simulations will be realized in their ability to predict cognitive performance improvements that can potentially be 

realized through modifications of the work system such as organizational changes, new system components, and changes in training levels 

of the team members. 

15. SUBJECT TERMS

command and control systems; computer simulation; decision making during uncertainty; partially observable Markov decision processes

16. SECURITY CLASSIFICATION OF:

19a. NAME OF RESPONSIBLE PERSON

 Sam E. Middlebrooks 

a. REPORT 

Unclassified 

b. ABSTRACT 

Unclassified 

c. THIS PAGE 

Unclassified 

17. LIMITATION 

OF ABSTRACT 

SAR 

18. NUMBER 

OF PAGES 

30 19b. TELEPHONE NUMBER (Include area code) 

 254-288-9379 

Standard Form 298 (Rev. 8/98) 

Prescribed by ANSI Std. Z39.18
iii 

Contents 

List of Figures iv

List of Tables iv

Acknowledgments v

1. Introduction 1

1.1 Modeling the Human Ability to Make Efficient Decisions During Conditions of 

Uncertainty ......................................................................................................................2 

1.1.1 The Optimal Observer .........................................................................................3 

1.1.2 Defining the State Space .....................................................................................4 

1.1.3 Defining the Observation Vector ........................................................................4 

1.1.4 Defining the Transition Matrix............................................................................4 

1.1.5 Belief Revision ....................................................................................................5 

1.2 Applying the Decision Making During Uncertainty Model............................................8 

1.2.1 Simulation Design ...............................................................................................8 

1.2.2 The Decision Making During Uncertainty Decision Loop ...............................10 

1.2.3 Evaluating the Current Information ..................................................................11 

1.2.4 Revising the Belief Vector ................................................................................12 

2. Decision and Results 13

2.1 Action Sequence Assessment: Recon1, Strike1, Recon2-N.........................................14 

2.2 Action Sequence Assessment: Recon1, Strike1-N.......................................................15 

2.3 Action Sequence Assessment: Recon1-N ....................................................................17 

2.4 Action Sequence Assessment: Strike1-N.....................................................................18 

3. Conclusions and Continuing Work 20

4. References 21

Distribution List 24 
iv 

List of Figures 

Figure 1. C3TRACE C2 simulation vignette................................................................................. 9 

Figure 2. Decision making during uncertainty model. ................................................................ 10 

Figure 3. Belief vector revision. .................................................................................................. 12 

Figure 4. State space. ................................................................................................................... 13 

Figure 5. Belief graph for action sequence recon1, strike1, recon2-N. ....................................... 15 

Figure 6. Belief graph for action sequence recon1, strike1, strike2-N. ....................................... 16 

Figure 7. Belief graph for action sequence recon1-N.................................................................. 18 

Figure 8. Belief graph for action sequence strike1-N.................................................................. 19 

List of Tables 

Table 1. The set of actions and their observations for the current seek and destroy task.............. 5 

Table 2. Probabilities for observation from artillery strike............................................................ 5 

Table 3. Probabilities for killing enemy from artillery strike. ....................................................... 5 

Table 4. Probabilities for sequence recon1, strike1, recon2-N. ................................................... 14 

Table 5. Probabilities for sequence recon1, strike1-N................................................................. 16 

Table 6. Probabilities for sequence recon1-N.............................................................................. 17 

Table 7. Probabilities for sequence strike1-N.............................................................................. 19 
v 

Acknowledgments 

This work has been supported by a grant from the Congressionally funded University XXI 

program in a partnership between the faculty and staff of The University of Texas at Austin 

through the Institute for Advanced Technology and the Human Research and Engineering 

Directorate (HRED) of the U.S. Army Research Laboratory (ARL). 

Thanks go to colleagues who provided technical reviews of this manuscript. From ARL’s 

HRED, these are Dr. Bruce Sterling at Fort Knox, Kentucky, Dr. Jim Ainsworth at Fort Hood, 

Texas, and Dr. Paul Rose and Dr. Dallas Johnson at Aberdeen Proving Ground, Maryland. From 

The University of Texas at Austin’s Department of Psychology is Associate Professor Dr. 

Bradley Love. 

The authors would also like to thank the following members of the faculty at the U.S. Military 

Academy at West Point, New York, for their support during this project: 

LTC Garry Lambert, Ph.D., ARL-U.S. Military Academy Research and Development 

Coordinator, Department of Mathematics, and 

Dr. Michael D. Matthews, Professor of Engineering Psychology, Department of Behavioral 

Sciences and Leadership. 

Special thanks also go to West Point Cadet Edward Klein for his contributions to the project 

during his Advanced Individual Academic Development tour during the summer of 2005 while 

at Fort Hood, Texas, and The University of Texas at Austin campus. 
vi 

INTENTIONALLY LEFT BLANK
1 

1. Introduction 

The time is now. The place is a U.S. Army command center in a faraway land, which is 

conducting combat operations. To the uninitiated observer, this would seem like a place of 

uncontrolled chaos. Computers, computer-generated tactical projection displays, and 

communications devices fill every available space. People are running back and forth yelling at 

each other with each one having a seemingly singular role to do whatever s/he is focused upon 

with no regard to other activities around him or her. Multiple radios are blaring from all sides of 

the work space. Individual groups of two or three people are engrossed in huddles in various 

corners of the area and are oblivious of everything else going on around them. The commander 

is yelling at a computer operator to get a computer-generated tactical map projection display 

revised and current with the admonition at a high fever pitch “you are killing me!!!”. The 

computer operator is feverishly and frantically calling for help from adjacent operators and 

noncommissioned officers. Although it is not obvious to the casual observer, there is structure to 

this chaos and there is order to the disorder, resulting in a carefully balanced mix of people, 

machines, and weapons conducting an orchestrated performance on the battlefield. 

Into the midst of all this apparent pandemonium, a new communications device is brought on 

line that has been developed by the best and brightest minds that the Army acquisition 

community can bring to bear on the problem. Its design promises to increase the rate of 

communications flow into the tactical operations center by 150% with an improvement in data 

quality by 90% (sample numbers for an imaginary system). However, there has been no chance 

to validate its design promises because of a rapid fielding initiative that put it into the field in an 

accelerated timeline. Even if an opportunity had been afforded to field test the system, the 

results of the test would typically have validated that the system is or is not working as designed 

and whether it provides greater communications flow at a higher fidelity. Even if these field 

tests had been conducted, there would still be the unanswered question of the overall effect on 

the battlefield that the introduction of this new system would have. This effect would be the 

result of the introduction of a new system component into the command and control (C2) system 

with the potential to change the overall performance of the total system. Furthermore, although 

the physical parameters of the new system can be measured and quantified, the effect of these 

changes on the cognitive performance of the work group and their ability to adapt to changes in 

operational paradigms is much more difficult to estimate. In fact, even in optimal laboratory 

conditions, it is a complex undertaking to assess cognitive performance factors such as 

situational awareness, individual and team performance, and the effects that these factors have 

on decision-making performance. Furthermore, many laboratory-based evaluations are 

conducted as a part of a basic research effort and are left to future research to apply to specific 

application areas. 
2 

Thus, the development of predictive computer-based models of optimal performance has a 

significant potential to aid in the evaluation of overall work systems in which the human is an 

integral component. This project has an objective to develop computer models of optimal 

performance in the area of decision making during conditions of uncertainty, which is the result 

of a coordinated effort that allows basic research using empirical investigations to be directly 

applied into the structure of a predictive computer simulation of decision making. 

1.1 Modeling the Human Ability to Make Efficient Decisions During Conditions of 

Uncertainty 

One of the most fundamental aspects of human cognition is the ability to make decisions. 

Humans can make decisions in a broad range of domains. For example, everyday decisions are 

made as to what time an individual will depart for work, where to eat or what will be eaten for 

lunch, what will be worn to work, and thousands of other decisions. These decisions often seem 

mundane because the ramifications of a “poor” decision are not significant. However, other 

decisions appear to be more critical and the ramifications of a “poor” decision can appear (and 

often are) more detrimental. For example, military decisions, medical decisions, and fault 

detection can all have significant ramifications.

It is important to recognize that most decisions that are made are not “one off” decisions in 

which the decision is made and then the rewards reaped or the punishment endured. Instead, 

most decisions that are made have future ramifications and affect the options and decisions that 

are available later. One challenge faced by any decision maker is the uncertainty that the 

decision maker has about the true state of the system. In most circumstances, the true state of 

the system is unknown or hidden. That is, it cannot be directly observed. For example, in 

military decisions, often there is uncertainty about an enemy’s position, strength, and morale. 

Given that the true state is hidden, there are things that can be done to reduce the decision 

maker’s uncertainty about these states. For example, the decision maker may attempt to 

determine the enemy’s position by sending reconnaissance to a location where the enemy is 

believed to be located. When the reconnaissance returns with either an “enemy sighted” or 

“enemy not sighted” report, the decision maker must revise his or her belief about the location of 

the enemy. 

If the observations and actions were all deterministic, revising a belief would be relatively 

simple. However, in almost all conditions, the observations and actions are probabilistic. That 

is, the probability of getting an observation, given the true state of the environment, is not 

necessarily 0.0 or 1.0. In the example, there is a certain non-zero probability that the 

reconnaissance mission was sent to the right location and will miss the enemy and send a report 

of “enemy not sighted”. Furthermore, there may be a non-zero probability that the 

reconnaissance mission falsely sent a report of “enemy sighted” (or false alarmed) when the 

enemy was not actually at the location. 
3 

Given that the observation and actions are probabilistic, revising a belief, given an observation 

and an action, can become cognitively difficult. Furthermore, evaluating the added benefit of a 

specific piece of equipment that changes these probabilities can also become difficult. The 

current research focuses on a task that is commonly faced by decision makers in the military, 

namely, a seek-and-destroy task. In this task, the decision maker is trying to localize and destroy 

an enemy within a specific region. At the decision maker’s disposal are actions that allow him or 

her to gain information about the true state of the system (i.e., the location of the enemy) in 

addition to changing the state of the system (e.g., moving the enemy from being at a specific 

location to the state of destroyed). The former actions are reconnaissance actions and the latter 

are artillery actions. The outcomes of these actions are probabilistic. That is, reconnaissance 

actions will not always detect the enemy when a sensor is sent to the enemy’s location. 

Furthermore, the reconnaissance may also falsely report that the enemy is seen at a location 

where he is not. Furthermore, the artillery will not always move the enemy to the “destroyed” 

state, even when it is sent to the right location. 

1.1.1 The Optimal Observer 

To best evaluate performance in a task that leads to uncertainty and probabilistic actions, it is 

useful to define the optimal performance within the task. The optimal performance can be 

calculated with Bayesian statistics. However, because of the nature of the current type of task, 

simple Bayesian statistics are insufficient. That is, with simple Bayesian statistics, the likelihood 

of the true state of the system can be optimally estimated, but this likelihood does not indicate 

what action should be selected. In order to do action selection, not only must the current state be 

calculated, given the previous actions and observations, but the optimal action to be performed in 

a given belief state must also be calculated wherein a belief state is a particular probability 

distribution across all the possible states in the environment. 

A variation on classical Bayesian statistics that may well add some additional predictive power 

for sequential decision making during uncertainty is the Partially Observable Markov Decision 

Processes (POMDP) (Cassandra, 1998; Cassandra, Kaelbling, & Kurien, 1996; Cassandra, 

Kaelbling, & Littman, 1994; Kaelbling, Littman, & Cassandra, 1998; Sondik, 1971). By 

defining the State Space, Observation Vector, Transition Matrix, and the Reward Structure, we 

can compute the expected reward for a particular action. In the following sections, a description 

of these actions is provided. In addition, a description of how to optimally revise an individual’s 

belief, given these definitions, is provided. 

An Ideal Observer Model provides optimal performance, given the information, available in the 

task. Typically, ideal observers are not proposed as models of human cognition. Instead, the 

ideal observer provides a benchmark by which to compare human performance. More 

specifically, these models illustrate what optimal performance should look like. When human 

performance matches that of the ideal observer model, one can conclude that the human is 

employing all the information in the task. When the human under-performs the ideal observer, 
4 

specific discrepancies between the human data and the ideal data may illuminate the constraints 

imposed by the human information processing system. 

Ideal observer analysis has been used to understand perceptual functions from the quantum limits 

of light detection (Hecht & Shlaer, 1942) to many forms of visual pattern detection and 

discrimination (Geisler, 1989), to reading (Legge & Hooven, 2002; Legge & Klitz, 1997) object 

recognition (Liu & Knill, 1995; Tjan & Braje, 1995; Tjan & Legge, 1998) eye movements 

(Najemnik & Geisler, 2005) and in reaching tasks (Trommershäuser & Gepshtein, 2004). 

1.1.2 Defining the State Space 

In all problems that are solved with a POMDP architecture, there is a set of states that the 

problem can be in. In a POMDP problem, the true state (StateTrue) is not directly observable (i.e., 

it is hidden). For the problems used in this project, the hidden state was defined as the enemy’s 

current position within the 5x5 state space area plus an additional “destroyed” state that the 

enemy could move into after an artillery strike at its current position for a total state space matrix 

of 26 states. 

1.1.3 Defining the Observation Vector 

Although the true state is hidden, the observer typically has actions and observations that provide 

information about the true state of the problem. In the current problem, the observer can fire 

artillery at a specific position or reconnaissance can be sent to a particular location within the 

environment (i.e., one of the 25 locations). The current problem has three possible observations: 

Enemy Sighted, No Enemy Sighted, or No Information. When the observer decides to send 

reconnaissance to a particular location, one of two observations will be received: “enemy 

sighted” or “enemy not sighted”. In the current problem, the artillery returns only one possible 

observation: “no information”. This replicates the fact that the artillery firing unit does not see 

the effects of its fires because it is an indirect firing unit and is not able to see where the artillery 

rounds fall. It must rely on forward observer assets to report what is termed “battle damage 

assessment” (BDA) in military jargon. In these models, it is the unmanned aerial vehicle (UAV) 

that provides the BDA. 

1.1.4 Defining the Transition Matrix 

Thus, in this “seek and destroy” problem, the commander has 51 possible different actions. 

There are 25 reconnaissance actions (one to each of the 25 locations in the environment), 25 

artillery actions (again, one to each of the 25 locations within the environment) and the “declare 

destroyed” option. The transition matrix defines the probability of the resulting state if the 

observer generates a particular action in a specified state (i.e., p(s'|s,a)). In the static form of the 

seek and destroy problem, there is only one state transition that could occur. When the 

commander fires artillery to where the enemy is located, the enemy will transition into the 
5 

“destroyed” state with a probability of 0.75. Probabilities that are assumed for the purpose of 

this analysis, which are merely estimations for the purposes of this discussion, are shown in 

tables 1 through 3. These values are estimates only and are not to be construed as factual. The 

determination of valid probabilities is left for future field and empirical work. 

Table 1. The set of actions and their observations for the current seek and destroy task. 

Action Observation State Probability 

Recon Enemy Sighted Enemy Present 0.75 

Recon Enemy Not Sighted Enemy Present 0.25 

Recon Enemy Sighted Enemy Not Present 0.2 

Recon Enemy Not Sighted Enemy Not Present 0.8 

(The observations for the reconnaissance action depend on whether the enemy is actually within the viewing region of the 

reconnaissance. Thus, the two possible states are “enemy present” and “enemy not present”.) 

Table 2. Probabilities for observation from artillery strike. 

Action Observation State Probability 

Strike NoInfo Enemy Present 1.0 

Strike NoInfo Enemy Not Present 1.0 

Table 3. Probabilities for killing enemy from artillery strike. 

Action Result State Probability of Dead 

Strike Probability of Enemy being killed. Enemy Present 0.75 

Strike Probability of Enemy not being killed. Enemy Present 0.25 

Strike Probability of Enemy being killed. Enemy Not Present 0.0 

1.1.5 Belief Revision 

Given an initial probability distribution over the state space, the Observation Matrix, and the 

Transition Matrix, hypotheses can be generated about the current state of the problem following 

an action and the returned observation. Equation 1 provides the Bayesian revision rule. 

( | , , ) ( | , )) ( |,,) (|,)p o s ba ps ba ps boa po ba

′ ′ ′ = (1) 

in which 

s’ ≡ true state (of the condition being present within the total of all states, S), 

represented as: s’ ∈ S 

b ≡ prior belief 

o ≡ observation 

a ≡ action that was generated 

Equation 1 specifies how the ideal observer would revise his or her belief that s' is the true state, 

given the prior belief (b), the observation (o), and the action that was generated (a). 

To illuminate the process of belief revision, a simple example of a smaller seek and destroy 

problem is provided. The Transition and Observation Matrices used in the empirical studies will 

be used here. To simplify the process, however, a three-state problem will be used instead of a 
6 

26-state problem. More specifically, the enemy will be in one of three states: State1, State2, or 

StateDestroyed. The initial (prior) probability of the state will be State1=0.5, State2=0.5, and 

StateDestroyed =0.0 (to simplify, this is represented as [0.5, 0.5, 0.0]) meaning that there is a 50% 

probability of the enemy of being in State1, a 50% probability of the enemy being in State2, and a 

0% probability of the enemy being in State Dead, i.e., the enemy is alive. 

Assume that the enemy is in State1 and that the observer decides to do reconnaissance to State1

and receives a “enemy sighted” observation. What is the likelihood of the belief that the enemy 

is in State1, State2 or StateDead? 

With equation 1, the likelihood that the enemy is in State1 can be revised. That is, the desire is to 

compute p(State1| [0.5, 0.5, 0.0], “EnemySighted”,Recon1). 

First, compute p(o|s',b,a) or p(“EnemySighted”|State1, [0.5, 0.5, 0.0],Recon1). To do this, the 

likelihood of obtaining an observation of “enemy sighted” if State1 were the true state is needed. 

In the Observation Matrix section, the likelihood of correctly identifying the enemy as 0.75 is 

defined. The likelihood of the true state being State1, given the previous belief and the action of 

Recon1 is needed. Because no transition is possible, these remain at the prior probabilities of 0.5. 

Finally, the likelihood of receiving the observation “enemy sighted” when reconnaissance is 

made at State1 or p(‘EnemySighted’|[0.5,0.5, 0],Recon1) needs to be computed. 

po ba ( | , ) ((0.5 0.75) (0.5 0.2) (0.0 0.2)) = × +×+× (2) 

= 0.475 (3) 

1 1

0.75 0.5 ( | [0.5,0.5,0.0], " ", 0.475 p State Enemy Sighted Recon × = (4) 

= 0.7895 (5) 

Furthermore, 

2 1

0.20 0.5 ( | [0.5,0.5,0.0], " ", 0.475 p State Enemy Sighted Recon × = (6) 

= 0.2105 (7) 

and finally, 

1

0.2 0.0 ( | [0.5,0.5,0.0], " ", 0.475 p Destroyed Enemy Sighted Recon × = (8) 

= 0.0 (9) 

Thus, if the first action is to observe at State1, the new belief vector would be 

[0.7895,0.2105,0.0]. 
7 

Now, imagine that the observer selected the action Strike1 following the action Recon1. To 

revise the belief that the enemy is in State1, the conditional probability p(State1|[0.7985, 0.2105, 

0.0], “NoInfo,” Strike1) is computed. 

First, compute p(o|s',b,a), which is the probability of receiving the “NoInfo” observation, given 

that the true state is State1, the current belief ([0.7895,0.2105,0.0]), and the action Strike1. The 

probability of receiving this observation is 1.0. Regardless of the state of the problem, a strike 

always returns the observation “NoInfo” (see table 2). This same probability and logic hold for 

computing p(o|b,a). 

The conditional probability p(s'|b,a) also needs to be computed. That is, what is the probability of 

the true state being State1, given the current belief and the action Strike1? As described in 

section 1.1.4, the probability of transitioning the problem into the destroyed state is 0.75 if the 

enemy is at the location where the artillery strike occurred. This means that there is a probability 

of 0.25 that the enemy’s state will not change or that the enemy will remain in State1 if it was 

initially in State1. 

1 1

( | , , ) 1.0

( | , ) 0.7895 0.25

( | , ) 1.0

( | [0.7895,0.2105,0.0]," ", ) 1.0 0.7895 0.25

1.0

0.1996

po s ba

ps ba

po ba

p State NoInfo Strike

′ =

′ = ×

=

=× ×

=

 (10) 

1

2 1

1

2 1

(" " | ,[0.7895,0.2105,0.0], ) 1.0

( | [0.7895,0.2105,0.0], ) 0.2105

(" " | [0.7895,0.2105,0.0], ) 1.0

( | [0.7895,0.2105,0.0]," ", ) 1.0 0.2105

1.0

p NoInfo Destroyed Strike

p State Strike

p NoInfo Strike

p State NoInfo Strike

=

=

=

= ×

= 0.2105

 (11) 

1 p NoInfo Destroyed Strike (" " | ,[0.7895,0.2105,0.0], ) 1.0 = (12) 

1 p Destroyed Strike ( | [0.7895,0.2105,0.0], ) 0.7895 0.75 = × (13) 

1 p NoInfo Strike (" " | [0.7895,0.2105,0.0], ) 1.0 = (14) 

1

1.0 0.7895 0.75 ( | [0.7895,0.2105,0.0]," ", ) 1.0 p Destroyed NoInfo Strike × × = (15) 

= 0.5989 

Thus, the belief vector, following Recon1 with the observation of “enemy sighted” followed by 

Strike1 with an observation of “No Information,” is [0.1996,0.2105,0.5989], or 
8 

p(State1) = 0.1996 

p(State2) = 0.2105 

p(Dead) = 0.5989 

1.2 Applying the Decision Making During Uncertainty Model 

Using cognitive models such as those described now allows computer simulations of C2 systems 

configured around task performance analysis based on previous work (Middlebrooks, 2001, 

2003, 2004; Middlebrooks et al., 1999a, 1999b; Middlebrooks & Williges, 2002; 

Wojciechowski, Plott, & Kilduff, 2005) to now be structured to incorporate cognitive decision 

making as a performance metric with this belief revision model. The steps in this process 

resemble the well-known observe-orient-decide-act (OODA) model (Belknap, 1996; Salas, 

Morgan, Glickman, Woodard, & Blaiwes, 1986). The decision actions here described consist of 

gathering information, revising the belief about the environment or state space, taking an action 

to accomplish an objective in the state space, and then making a decision of whether to continue 

the mission or terminate it with an assessment of mission success or failure. The example in this 

military C2 scenario employs a UAV to gather the intelligence, artillery to take an action to 

destroy an enemy somewhere within the state space, and belief revision to evaluate the situation 

after each action and then continue or declare “mission complete”. 

To structure this scenario in a computer simulation, the programming environment of Kilduff, 

Swoboda, and Barnette (2005); Plott (2002); Plott, Quesada, Kilduff, Swoboda, and Allender 

(2004) is employed. Command, Control, and Communications: Techniques for the Reliable 

Assessment of Concept Execution (C3TRACE) , developed through funding by the Human 

Research and Engineering Directorate of the U.S. Army Research Laboratory, is an adaptation of 

the commercial discrete event programming language MicroSaint1 (Schunk & Plott, 2004). 

Although the basic MicroSaint programming language allows for the development of task-based 

computer simulations of real-world systems and processes to be represented, C3TRACE has 

embedded data structures that augment MicroSaint to allow for representation of Army C2 

systems. 

1.2.1 Simulation Design 

C3TRACE programs are implemented with the use of discrete event language constructs 

common to any MicroSaint simulation program. The top level of a C2 sub-workgroup within a 

sample organization is shown in the example depicted in figure 1. Here, messages received by 

the radio operator are distributed according to their subject content. Situation reports (SITREPs) 

are passed to the S3 operations officer, logistics reports are passed to the S4 logistics officer for 

action, and so on. If, for example, a mission directive such as seek and destroy an enemy is 

 1MicroSaint is a trademark of MicroAnalysis and Design, Inc.
9 

received, it is passed to the commander for action. There are different reactions that might be 

experienced to such a directive. The commander might communicate with the originating 

authority to clarify information: an initial estimate of the situation before taking action might be 

performed; a revision of the situational awareness before taking action might be performed; or 

the mission might be undertaken as directed. In this case, as depicted in the green box in 

figure 1, what is referred to as the decision making during uncertainty process would be initiated 

to execute the mission. 

Figure 1. C3TRACE C2 simulation vignette. 

Figure 2 illustrates a modeled decision process that is very similar to the OODA model. This 

diagram represents an iterative process where the decision maker makes an initial estimate of the 

situation and then begins an iterative process of gathering additional information (flying a UAV 

mission) or taking an action to destroy the enemy (firing artillery). When the commander 

believes that the enemy has been destroyed, a mission complete decision is made and the results 

of the decision are realized. If the enemy was destroyed and the decision maker made that 

correct assessment, then a positive reward resulting from a good decision is applied to the 

performance of the overall system. If the enemy was not destroyed and the decision maker 

believed that he was destroyed, then a negative battlefield outcome is applied to the simulation. 

Likewise, if the enemy was destroyed but the decision maker believed he was not, then the 

results of poor decision making are applied. This process of iterative action can be generalized 

to similar scenarios where information is gathered (Observe), belief revision occurs (Orient), 
10 

decisions are made for mission success (Decide), and actions are taken to accomplish the mission 

(Act). The examples of employing a UAV and firing artillery are used here to simply provide a 

tangible example of how this type of activity might occur. 

Figure 2. Decision making during uncertainty model. 

1.2.2 The Decision Making During Uncertainty Decision Loop 

Referring to figure 2, the top-level logic for this model can be examined. After initiating the 

decision sequence and performing an initial estimate of the situation, the commander revises his 

belief vector, defined as the belief about the current situation regarding the enemy, and then 

begins an iterative process of looking for information or taking an action to accomplish the 

mission. When this process has reached some level of belief that the mission is accomplished, 

the commander terminates the action and completes the decision process by declaring a mission 

“success” or “failure”. 

If the initial desire is to obtain additional information, a UAV is sent to a specified location to 

attempt to locate the enemy. The UAV is the information gathering or BDA tool available to the 
11 

commander to revise his belief vector about the enemy. If the target is already dead from 

previous artillery action, then there is no correct location for the enemy because he does not exist 

or is dead. If the enemy is alive and the UAV is sent to the correct location, then it has a 

probability, according to table 1, of detecting or not detecting the enemy representing the 

accuracy of the UAV. From this, it will correctly or incorrectly report that the enemy was found. 

Likewise, if it is sent to a location where the enemy is not located or if the enemy is already 

dead, the UAV may correctly or incorrectly report the enemy sighted again, according to table 1. 

The values in table 1 are only sample estimates for use in the development of this model and do 

not represent any actual system currently in existence. After the UAV mission is flown, the 

commander evaluates the report from the UAV through the process of revising his belief vector 

(described next) and using this new information, decides what process to invoke next. 

If the commander decides to fire artillery (which is representative of taking a positive action to 

do something to accomplish the mission), then the probability exists that the right or wrong 

location will be fired upon. If the artillery fires on the wrong location, then the only outcome 

will be to miss the target. If the correct location is fired upon, then the artillery will kill or not 

kill the enemy according to the circular area of probability for the type of artillery fired. 

Independent of where the artillery is fired, the only report that is sent to the commander is that 

the artillery fired upon the location directed. This represents the fact that artillery is an indirect 

fire weapon and the firing unit never actually sees the target. The forward observer (in this case 

the UAV) must report the actual target situation, i.e., to provide the BDA. The commander must 

then evaluate the firing data and information from previous UAV reconnaissance missions to 

decide if to continue the mission or declare the enemy is dead and end the mission. 

When the commander believes that the enemy has been destroyed, then mission complete is 

declared and the commander is faced with the rewards of a successful or good decision sequence 

where the enemy was killed, meaning that the mission was accomplished, or the effects of a bad 

decision where the mission was not accomplished. 

1.2.3 Evaluating the Current Information 

Figure 3 illustrates the input feeding the sequence of evaluating the current situation and revising 

the belief vector and the resulting choice for the next action. 

The actual logic for each of these activities occurs within the C3TRACE program with C sharp 

(C#) code statements embedded within the beginning and ending effects sections of each of the 

task blocks in the diagram. 
12 

Figure 3. Belief vector revision. 

A verbal description of the computer logic for the “evaluate report” task takes the form 

For UAV Mission Cases: 

If (UAV mission ordered) & (Enemy Located at specified location) 

 Then, There is a positive probability that the enemy will be reported sighted. 

If (UAV mission ordered) & (Enemy Not Located at specified location, or is dead) 

 Then, There is a small probability that the enemy will still be reported sighted. 

For Arty Mission Cases: 

If (Arty mission ordered) & (Enemy Located at specified location) 

 Then, There is a positive probability that the enemy will be destroyed. 

If (Arty mission ordered) & (Enemy Not Located at specified location) 

 Then, There is a zero probability that the enemy will be destroyed. 

For all Arty mission cases, 

 Report = “No Info”. 

1.2.4 Revising the Belief Vector 

Figure 3 illustrates the revise belief vector task that follows the evaluate report task. The pseudo 

logic that is implemented in C# for this task is 

For UAV Mission Cases: 

If (UAV mission ordered) & (Enemy Located at specified location) 

 Then, Revise Belief Vector with positive information for State Where Reconnaissance was 

Performed. 

 Then, Revise Belief Vector with negative information for State Where Reconnaissance was Not 

Performed. 

 Then, Revise Belief Vector with negative information for State Dead. 

If (UAV mission ordered) & (Enemy Not Located at specified location, or is dead) 
13 

 Then, Revise Belief Vector with positive information for State Where Reconnaissance was Not 

Performed. 

 Then, Revise Belief Vector with negative information for State Where Reconnaissance was 

Performed. 

 Then, Revise Belief Vector with negative information for State Dead. 

For Arty Mission Cases: 

If (Arty mission ordered) & (Enemy Located at specified location) 

 Then, Revise Belief Vector with positive information for State Fired Upon. 

 Then, Revise Belief Vector with negative information for State Not Fired Upon. 

 Then, Revise Belief Vector with positive information for State Dead. 

If (Arty mission ordered) & (Enemy Not Located at specified location) 

 Then, Revise Belief Vector with negative information for State Fired Upon. 

 Then, Revise Belief Vector with negative information for State Not Fired Upon. 

 Then, Revise Belief Vector with negative information for State Dead. 

2. Decision and Results 

Current work has focused on the simplest possible state space which consists of two location 

states and a state representing the status of the target enemy, dead or alive. While a location state 

of only two conditions appears to be trivial and unrelated to any actual human performance 

condition, even this simple arrangement can relate to actual performance. Although this simple 

three-state model is not sufficient for actual analysis, it forms the basis for future, more complex 

models to become predictive of selected decision processes. When faced with a decision choice, 

a human decision maker many times has one of two selections to make. Referring to figure 4, 

whether it is right versus left, good versus bad, right versus wrong, high payoff versus low 

payoff, cheap versus expensive, or whatever the criterion, a two-state response choice augmented 

by one result state (in this case dead), can be descriptive of actual conditions. Also, even in this 

most simple of state environments, the conditional probability logic can become exhaustive. 

Computer runs with these conditions illustrate the conditional probabilities resulting from 

relatively simple actions in this state space. 

Figure 4. State space. 
14 

As shown in figure 4 in this simple state space case, the enemy is located in State1 or State2 for a 

binary location condition, with a condition of dead or alive, as indicated by State3. 

2.1 Action Sequence Assessment: Recon1, Strike1, Recon2-N 

In order to examine the conditional probability logic associated with actions in this state space, 

some examples of actions and resulting belief vectors will be examined. The assumptions are 

that the enemy is located in State1 and that he is static, i.e., not moving. Also, there is an equal 

probability in the belief of the commander that the enemy could be in either of the location states 

and that the enemy is alive. The initial belief vector is thus [0.5, 0.5, 0.0], meaning a 50% 

chance of being in location State1, a 50% chance of being in location State2, and a 0.0% chance 

of being in State Dead, i.e., the enemy is alive. Assume the following sequence of actions: 

Conduct a UAV mission into State1, called Recon1 

Fire artillery strike into State1, called Strike1 

Conduct continuous UAV missions into State1, called Recon2 to ReconN. 

Using the POMDP methodology in the C3TRACE DMDC model, examine the resulting beliefs 

that the decision maker has following the decision to perform this series of actions. The output 

from this simulation run is illustrated in table 4 and figure 5. 

Table 4. Probabilities for sequence recon1, strike1, recon2-N. 

NumDMDC_

Iterations Action State_Space[1,1] -

Cell1

State_Space[1,2]-

Cell2

State_Space_

Dead

1 Recon1 0.5000 0.5000 0.0000

2 Strike1 0.7895 0.2105 0.0000

3 Recon2 0.1974 0.2105 0.5921

4 Recon3 0.4797 0.1365 0.3838

5 Recon4 0.7757 0.0588 0.1655

6 Recon5 0.9284 0.0188 0.0528

7 Recon6 0.9799 0.0053 0.0149

8 Recon7 0.9945 0.0014 0.0040

9 Recon8 0.9985 0.0004 0.0011

10 Recon9 0.9996 0.0001 0.0003

11 Recon10 0.9999 0.0000 0.0001

12 Recon11 1.0000 0.0000 0.0000

13 Recon12 1.0000 0.0000 0.0000

14 Recon13 1.0000 0.0000 0.0000
15 

Figure 5. Belief graph for action sequence recon1, strike1, recon2-N. 

These actions (first a reconnaissance, then a strike, and then only reconnaissance) produce a 

predicted belief pattern of almost 80% after the first reconnaissance that the enemy is in fact at 

State1. After the strike, the belief of being in State1 drops to 21% while the belief that the enemy 

is dead goes from 0% to almost 60%. However, if the only succeeding actions at this point are to 

perform reconnaissance to State1 (which have no ability to harm the enemy because only an 

artillery strike can cause damage), then, because of successive incremental probabilities that the 

enemy might still be at State1, and therefore not dead, the belief value for State1 asymptotes at 

100% while the values for State2 and State3 Dead go to 0%. 

2.2 Action Sequence Assessment: Recon1, Strike1-N 

Now consider the action sequence of an initial reconnaissance, Recon1, followed by multiple 

artillery strikes, Strike1-N. The previous assumptions for enemy location and initial belief 

vector are the same. 

Sequence (All to State1 w/Enemy @ State1): Initial, Recon1, Strike1, Recon2-

ReconN

0.0000

0.2000

0.4000

0.6000

0.8000

1.0000

1.2000

12345 6 7 8 9 10 11 12 13

DMDC Iterations

Belief Probability

State_Space[1,1] - Cell1

State_Space[1,2]- Cell2

State_Space_Dead
16 

Table 5. Probabilities for sequence recon1, strike1-N. 

Figure 6. Belief graph for action sequence recon1, strike1, strike2-N. 

After the first two belief revisions, which are the same as in the previous runs, the belief values 

for the enemy being dead continue to rise with successive artillery strikes but asymptote at 78% 

NumDMDC_

Iterations Action State_Space[1,1] -

Cell1

State_Space[1,2]-

Cell2

State_Space_

Dead

1 Recon1 0.5000 0.5000 0.0000

2 Strike1 0.7895 0.2105 0.0000

3 Strike2 0.1974 0.2105 0.5921

4 Strike3 0.0493 0.2105 0.7401

5 Strike4 0.0123 0.2105 0.7771

6 Strike5 0.0031 0.2105 0.7864

7 Strike6 0.0008 0.2105 0.7887

8 Strike7 0.0002 0.2105 0.7893

9 Strike8 0.0000 0.2105 0.7894

10 Strike9 0.0000 0.2105 0.7895

11 Strike10 0.0000 0.2105 0.7895

12 Strike11 0.0000 0.2105 0.7895

Sequence (All to State1 w/Enemy @ State1): Initial, Recon1, Strike1, Strike2-

StrikeN

0.0000

0.1000

0.2000

0.3000

0.4000

0.5000

0.6000

0.7000

0.8000

0.9000

123 456 7 8 9 10 11 12

DMDC Iterations

Probability

State_Space[1,1] - Cell1

State_Space[1,2]- Cell2

State_Space[1,3]- Dead
17 

while the belief value for the enemy still being in State1 goes to zero. This is offset by a 

suspected belief condition for the enemy being at State2 which is 21%. Since the belief 

probabilities for all three states must sum to 100%, representing the complete belief condition, 

this causes the predicted belief for the enemy actually being dead to only achieve maximum at 

78% even with successive artillery strikes each of which has the ability to completely kill the 

enemy. Actually, the C3TRACE model had the enemy killed after the first artillery strike, but 

the predicted belief by the commander has the values shown. 

2.3 Action Sequence Assessment: Recon1-N 

To check the response of the model, additional runs were made of only reconnaissance missions 

and only artillery missions. Table 6 and figure 7 show the results for the case of only flying the 

UAV for multiple reconnaissance into State1. Initial conditions remain the same as before. 

Table 6. Probabilities for sequence recon1-N. 

Action State_Space[1,1] - 

Cell1

State_Space[1,2]- 

Cell2

State_Space_

Dead

Recon1 0.5000 0.5000 0.0000

Recon2 0.7895 0.2105 0.0000

Recon3 0.9336 0.0664 0.0000

Recon4 0.9814 0.0186 0.0000

Recon5 0.9950 0.0050 0.0000

Recon6 0.9987 0.0013 0.0000

Recon7 0.9996 0.0004 0.0000

Recon8 0.9999 0.0001 0.0000

Recon9 1.0000 0.0000 0.0000

Recon10 1.0000 0.0000 0.0000

Recon11 1.0000 0.0000 0.0000

Recon12 1.0000 0.0000 0.0000
18 

Figure 7. Belief graph for action sequence recon1-N. 

Here, as would be predicted by common sense logic, the dead state remains at zero as 

reconnaissance missions have no ability to damage the enemy. The belief probabilities for the 

enemy at State1 and State2 complement each other with the State1 belief that asymptotes at 100% 

while the State2 belief asymptotes at 0%. 

2.4 Action Sequence Assessment: Strike1-N 

Finally, table 7 and figure 8 show the results for the case of only firing artillery for multiple 

strikes at State1. Initial conditions remain the same as before. In this case, while the artillery has 

the ability to kill the enemy, it does not have the ability to report its effects. Thus, the belief 

probability of the commander for where the enemy is located and what his dead state is, is 

nonexistent as artillery strikes only report that the mission was fired. As a result, only firing 

artillery will provide no information to the commander about the status of the enemy even 

though the artillery might have actually destroyed the enemy. Thus, successive artillery strikes 

to State1 result in no change in the belief that the enemy is at State1 and cause only a maximum 

belief of 50% that the enemy might have been destroyed by the artillery missions. These models 

represent the total belief possible in the commander with an indication of 100% summed from all 

the individual beliefs. As a result of this 100% possible belief, this causes the belief that the 

enemy is at State2 to asymptote at 0%. 

Sequence (All to State1 w/Enemy @ State1): Initial, Recon1, -ReconN

0.0000

0.2000

0.4000

0.6000

0.8000

1.0000

1.2000

123 456 7 8 9 10 11 12

DMDC Iterations

Probability

State_Space[1,1] - Cell1

State_Space[1,2]- Cell2

State_Space[1,3]- Dead
19 

Table 7. Probabilities for sequence strike1-N. 

Figure 8. Belief graph for action sequence strike1-N. 

NumDMDC_

Iterations Action State_Space[1,1] -

Cell1

State_Space[1,2]-

Cell2

State_Space_

Dead

1 Strike1 0.5000 0.5000 0.0000

2 Strike2 0.1250 0.5000 0.3750

3 Strike3 0.0313 0.5000 0.4688

4 Strike4 0.0078 0.5000 0.4922

5 Strike5 0.0020 0.5000 0.4980

6 Strike6 0.0005 0.5000 0.4995

7 Strike7 0.0001 0.5000 0.4999

8 Strike8 0.0000 0.5000 0.5000

9 Strike9 0.0000 0.5000 0.5000

10 Strike10 0.0000 0.5000 0.5000

11 Strike11 0.0000 0.5000 0.5000

12 Strike12 0.0000 0.5000 0.5000

13 Strike13 1.0000 0.0000 0.0000

Sequence (All to State1 w/Enemy @ State1): Initial, Strike1, Strike2-StrikeN

0.0000

0.1000

0.2000

0.3000

0.4000

0.5000

0.6000

123 456 7 8 9 10 11 12

DMDC Iterations

Probability

State_Space[1,1] - Cell1

State_Space[1,2]- Cell2

State_Space[1,3]- Dead
20 

3. Conclusions and Continuing Work 

This project is at the initial stage of developing a quantitative predictive model of optimal 

decision making performance in a form usable in simulations of C2 activities. The Bayesian 

statistical approach to the modeling is not intended to be construed as an attempt to present a 

human cognitive model of decision making but is an attempt to understand what optimal 

performance could be in a given set of circumstances. Once this optimal performance is 

understood, then the Bayesian models can be used to compare predicted optimal performance 

against actual observed human performance data to gain an understanding of how human 

cognitive limitations can be affected by changes in different components of the human-computer 

interfaces in the work system. Some of the human performance areas where this technique might 

be applied include memory, decision strategy, and perception, to name a few. This technique has 

the promise to allow investigations of how modifying technology can affect decision strategies 

as they are represented by battlefield success and the cost of achieving that success. 

The work to date has been and continues to be a partnership between (a) basic empirical research 

that is investigating optimal performance through the modification of belief presentations and the 

accuracy of belief vectors and (b) applied research working to develop simulations of highly 

dynamic battlefield performance, which are moderated by predicted optimal performance 

conditions within the work group by employing the C3TRACE modeling environment. 

These initial efforts have developed the framework for a more complex series of models by 

developing performance algorithms based on a simple three-element state space of two location 

states and one status state (dead). The results presented here are not intended to provide a basis 

for actual investigative work. These data are included only for the purpose of illustrating the 

potential for this research. Future work will expand the effort to a five-state space environment 

of a 2 x 2 location grid + Dead and to a 26-state space environment of a 5 x 5 location grid + 

Dead. The current empirical work is based upon the 5 x 5 location grid + Dead, or 26-state space 

environment condition. Also, the current work is based upon an enemy statically located in the 

grid. Future work will incorporate a dynamic enemy moving within the location grid according 

to some predetermined algorithm. 
21 

4. References 

Belknap, M. Military Decision Making in the Information Age. Unpublished Monograph, Naval 

War College, Newport, RI. Joint Military Operations Department. Ad-a307-648-6-xab, 

1996. 

Cassandra, A. R. Exact and Approximate Algorithms for Partially Observable Markov Decision 

Processes. Unpublished Dissertation, Brown University, 1998. 

Cassandra, A. R.; Kaelbling, L. P.; Kurien, J. A. Acting under Uncertainty: Discrete Bayesian 

Models for Mobile-Robot Navigation. In Proceedings of IEEE/RSJ International 

Conference on Intelligent Robots and Systems, 1996. 

Cassandra, A. R.; Kaelbling, L. P.; Littman, M. L. Acting Optimally in Partially Observable 

Stochastic Domains. In Proceedings of the Twelfth National Conference on Artificial 

Intelligence. (AAAI), 1023--1028. http://citeseer.ist.psu.edu/cassandra94acting.html 

Seattle, Washington. MIT Press, 1994. 

Geisler, W. S. Ideal-Observer Theory in Psychophysics and Physiology. Physica Scripta 1989, 

39, 153-160. 

Hecht, S.; Shlaer, S. Energy, Quanta, and Vision. Journal of General Physiology 1942, 25, 819-

840. 

Kaelbling, L. P.; Littman, M. L.; Cassandra, A. R. Planning and Acting in Partially Observable 

Stochastic Domains. Artificial Intelligence 1998, 101, 99-134. 

Kilduff, P. W.; Swoboda, J. C.; Barnette, B. D. Command, Control, and Communications: 

Techniques for the Reliable Assessment of Concept Execution (C3TRACE) Modeling 

Environment: The Tool; ARL-MR-0617; U.S. Army Research Laboratory, Human Research 

and Engineering Directorate: Aberdeen Proving Ground, MD, 2005. 

Legge, G. E.; Hooven, T. A. Mr. Chips 2002: New Insights from an Ideal-Observer Model of 

Reading. Vision Research 2002, 42, 2219-2234. 

Legge, G. E.; Klitz, T. S. Mr. Chips: An Ideal-Observer Model of Reading. Psychological 

Review 1997, 104, 524-553. 

Liu, Z.; Knill, D. C. Object Classification for Human and Ideal Observers. Vision Research

1995, 35 (4), 549-568. 

Middlebrooks, S. E. Experimental Interrogation of Network Simulation Models of Human Task 

and Workload Performance in a U.S. Army Tactical Operations Center. Unpublished 

Thesis, Virginia Polytechnic Institute and State University, Blacksburg, VA, 2001. 
22 

Middlebrooks, S. E. The Compass Paradigm for the Systematic Evaluation of U.S. Army 

Command and Control Systems Using Neural Network and Discrete Event Computer 

Simulation. Unpublished Dissertation, Department of Industrial and Systems Engineering. 

Virginia Polytechnic Institute and State University, Blacksburg, VA, 2003. 

Middlebrooks, S. E. Neural Net and Discrete Event Simulations of C2 Performance in Army 

Tocs. In Proceedings of the Human Factors and Ergonomics Society's 48th Annual Meeting, 

September 20-24, 2004 (pp. 513-517). New Orleans, LA: HFES, 2004. 

Middlebrooks, S. E.; Knapp, B. G.; Barnette, B. D.; Bird, C. A.; Johnson, J. M.; Kilduff, P. W.; 

Schipani, S. P.; Swoboda, J. C.; Wojciechowski, J. Q.; Tillman, B. W.; Ensing, A. R.; 

Archer, S. G.; Archer, R. D.; Plott, B. M. Cohost - Computer Simulation Models to 

Investigate Human Performance Task and Workload Conditions in a U.S. Army Heavy 

Maneuver Battalion Tactical Operations Center. In Proceedings of the HFES 43rd Annual 

Meeting (pp. 242-246). Houston, TX: Human Factors and Ergonomics Society, 1999a. 

Middlebrooks, S. E.; Knapp, B. G.; Barnette, B. D.; Bird, C. A.; Johnson, J. M.; Kilduff, P. W.; 

Schipani, S. P.; Swoboda, J. C.; Wojciechowski, J. Q.; Tillman, B. W.; Ensing, A. R.; 

Archer, S. G.; Archer, R. D.; Plott, B. M. Cohost (Computer Modeling of Human Operator 

System Tasks) Computer Simulation Models to Investigate Human Performance Task and 

Workload Conditions in a U.S. Army Heavy Maneuver Battalion Tactical Operations Center; 

ARL-TR-1994; U.S. Army Research Laboratory Human Research and Engineering 

Directorate: Aberdeen Proving Ground, MD 21005. DTIC ADA368587, 1999b. 

Middlebrooks, S. E.; Williges, R. C. Experimental Design Interrogation of Network Simulation 

Models of U.S. Army Command and Control Centers. In Proceedings of the 46th Annual 

Meeting of the Human Factors and Ergonomics Society (pp. 458-462). Baltimore, MD: 

HFES, 2002. 

Najemnik, J.; Geisler, W. S. Optimal Eye Movement Strategies in Visual Search. Nature 2005, 

434, 387-391. 

Plott, B. Software User's Manual For: C3TRACE (Command, Control and Communication - 

Techniques for Reliable Assessment of Concept Execution); U.S. Army Research Laboratory, 

Human Research and Engineering Directorate: Aberdeen Proving Ground, MD, 2002. 

Plott, B.; Quesada, S.; Kilduff, P.; Swoboda, J.; Allender, L. Using an Information-Driven 

Decision Making Human Performance Tool to Assess U.S. Army Command, Control, and 

Communication Issues. In Proceedings of the Human Factors and Ergonomics Society's 

48th Annual Meeting, September 20-24, 2004 (pp. 2396-2400). New Orleans, LA: HFES, 

2004. 
23 

Salas, E.; Morgan, B. B., Jr.; Glickman, A. S.; Woodard, E. A.; Blaiwes, A. S. Measurement of 

Team Behaviors in a Navy Training Environment; NTSC TR-86-014; Human Factors 

Division, Department of the Navy. DTIC ADA185237, 1986. 

Schunk, D.; Plott, B. Use of Discrete Event Simulation to Model Human Performance. In 

Proceedings of the Human Factors and Ergonomics Society's 48th Annual Meeting, 

September 20-24, 2004 (pp. 2126-2130). New Orleans, LA: HFES, 2004. 

Sondik, E. J. The Optimal Control of Partially Observable Markov Processes. Unpublished 

Dissertation, Stanford University, Stanford, California, 1971. 

Tjan, B. S.; Braje, W. L. Human Efficiency for Recognizing 3-D Objects in Luminance Noise. 

Vision Research 1995, 35 (21), 3053-3069. 

Tjan, B. S.; Legge, G. E. The Viewpoint Complexity of an Object Recognition Task. Vision 

Research 1998, 15 (16), 2335-2350. 

Trommershäuser, J.; Gepshtein, S. Optimal Compensation for Changes in Effective Movement 

Variability. Nature Neuroscience Under Review, 2004. 

Wojciechowski, J. Q.; Plott, B.; Kilduff, P. Human Performance Model Development of a 

Battalion Tactical Operations Center; ARL-TR-2635; U.S. Army Research Laboratory, 

Human Research and Engineering Directorate: Aberdeen Proving Ground, MD, 2005. 
24 

NO. OF 

COPIES ORGANIZATION

 1 DEFENSE TECHNICAL 

 (PDF INFORMATION CTR 

 ONLY) DTIC OCA 

 8725 JOHN J KINGMAN RD 

 STE 0944 

 FORT BELVOIR VA 22060-6218 

 1 US ARMY RSRCH DEV & ENGRG CMD 

 SYSTEMS OF SYSTEMS 

 INTEGRATION 

 AMSRD SS T 

 6000 6TH ST STE 100 

 FORT BELVOIR VA 22060-5608 

 1 INST FOR ADVNCD TCHNLGY 

 THE UNIV OF TEXAS AT AUSTIN 

 3925 W BRAKER LN STE 400 

 AUSTIN TX 78759-5316 

 1 THE UNIV OF TEXAS AT AUSTIN 

 DEPT OF PSYCHOLOGY 

 ATTN B STANKIEWICZ 

 1 UNIVERSITY STN A8000 

 AUSTIN TX 78712-0187 

 1 DIRECTOR 

 US ARMY RESEARCH LAB 

 IMNE ALC IMS 

 2800 POWDER MILL RD 

 ADELPHI MD 20783-1197 

 1 DIRECTOR 

 US ARMY RESEARCH LAB 

 AMSRD ARL CI OK TL 

 2800 POWDER MILL RD 

 ADELPHI MD 20783-1197 

 2 DIRECTOR 

 US ARMY RESEARCH LAB 

 AMSRD ARL CS OK T 

 2800 POWDER MILL RD 

 ADELPHI MD 20783-1197 

 3 US ARMY NATICK SOLDIER CTR 

 ERGONOMICS TEAM/SSTD 

 ATTN J B SAMPSON 

 100 KANSAS ST 

 NATICK MA 01760-5020 

 1 RDECOM-NATICK SOLDIER CTR 

 MANAGEMENT TEAM/PEO SOLDIER/PM 

 FUTURE FORCE WARRIOR TECH PRO OFC 

 ATTN C FITZGERALD 

 100 KANSAS ST 

 NATICK MA 01760-5020 

NO. OF 

COPIES ORGANIZATION

 1 RDECOM-NATICK SOLDIER CTR 

 HUMAN PERF/MANPRINT/SSI LEAD 

 FUTURE FORCE WARRIOR TECH PRO OFC 

 ATTN C L BLACKWELL 

 100 KANSAS ST 

 NATICK MA 01760-5020 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR M DR M STRUB 

 6359 WALKER LANE SUITE 100 

 ALEXANDRIA VA 22310 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MA J MARTIN 

 MYER CENTER RM 2D311 

 FT MONMOUTH NJ 07703-5630 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MC A DAVISON 

 320 MANSCEN LOOP STE 166 

 FT LEONARD WOOD MO 65473-8929 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MD T COOK 

 BLDG 5400 RM C242 

 REDSTONE ARSENAL AL 35898-7290 

 1 COMMANDANT USAADASCH 

 ATTN ATSA CD 

 ATTN AMSRD ARL HR ME MS A MARES 

 5800 CARTER RD 

 FT BLISS TX 79916-3802 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MI J MINNINGER 

 BLDG 5400 RM C242 

 REDSTONE ARSENAL AL 35898-7290 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MM DR V RICE 

 BLDG 4011 RM 217 

 1750 GREELEY RD 

 FT SAM HOUSTON TX 78234-5094 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MG R SPINE 

 BUILDING 333 

 PICATINNY ARSENAL NJ 07806-5000 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MH C BURNS 

 BLDG 1002 ROOM 117 

 1ST CAVALRY REGIMENT RD 

 FT KNOX KY 40121 
25 

NO. OF 

COPIES ORGANIZATION

 1 ARMY RSCH LABORATORY - HRED 

 AVNC FIELD ELEMENT 

 ATTN AMSRD ARL HR MJ D DURBIN 

 BLDG 4506 (DCD) RM 107 

 FT RUCKER AL 36362-5000 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MK MR J REINHART 

 10125 KINGMAN RD 

 FT BELVOIR VA 22060-5828 

 20 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MV HQ USAOTC 

 S MIDDLEBROOKS 

 91012 STATION AVE ROOM 111 

 FT HOOD TX 76544-5073 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MY M BARNES 

 2520 HEALY AVE STE 1172 BLDG 51005 

 FT HUACHUCA AZ 85613-7069 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MP D UNGVARSKY 

 BATTLE CMD BATTLE LAB 

 415 SHERMAN AVE UNIT 3 

 FT LEAVENWORTH KS 66027-2326 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR M DR B KNAPP 

 ARMY G1 MANPRINT DAPE MR 

 300 ARMY PENTAGON ROOM 2C489 

 WASHINGTON DC 20310-0300 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MJK MS D BARNETTE 

 JFCOM JOINT EXPERIMENTATION J9 

 JOINT FUTURES LAB 

 115 LAKEVIEW PARKWAY SUITE B 

 SUFFOLK VA 23435 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MQ M R FLETCHER 

 US ARMY SBCCOM NATICK SOLDIER CTR 

 AMSRD NSC SS E BLDG 3 RM 341 

 NATICK MA 01760-5020 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MT DR J CHEN 

 12350 RESEARCH PARKWAY 

 ORLANDO FL 32826-3276 

NO. OF 

COPIES ORGANIZATION

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MS MR C MANASCO 

 SIGNAL TOWERS RM 303A 

 FORT GORDON GA 30905-5233 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MU M SINGAPORE 

 6501 E 11 MILE RD MAIL STOP 284 

 BLDG 200A 2ND FL RM 2104 

 WARREN MI 48397-5000 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MF MR C HERNANDEZ 

 BLDG 3040 RM 220 

 FORT SILL OK 73503-5600 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MW E REDDEN 

 BLDG 4 ROOM 332 

 FT BENNING GA 31905-5400 

 1 ARMY RSCH LABORATORY - HRED 

 ATTN AMSRD ARL HR MN R SPENCER 

 DCSFDI HF 

 HQ USASOC BLDG E2929 

 FORT BRAGG NC 28310-5000 

 1 US ARMY CECOM-NVESD 

 ATTN AMSEL RD NV LWD D MOODY 

 10221 BURBECK RD 

 FORT BELVOIR VA 22060-5806 

 ABERDEEN PROVING GROUND

 1 DIRECTOR 

 US ARMY RSCH LABORATORY 

 ATTN AMSRD ARL CI OK TECH LIB 

 BLDG 4600 

 1 DIRECTOR 

 US ARMY RSCH LABORATORY 

 ATTN AMSRD ARL CI OK S FOPPIANO 

 BLDG 459 

 1 DIRECTOR 

 US ARMY RSCH LABORATORY 

 ATTN AMSRD ARL HR MR F PARAGALLO 

 BLDG 459