U.S. Army Research Institute 

for the Behavioral and Social Sciences 

Research Report 1847 

Preliminary Evaluation of a Novel Simulation-Based 

Tool for Training Rapid Decision-Making Skills 

Richard E. Christ 

U.S. Army Research Institute 

February 2006 

Approved for public release; distribution is unlimited. 
U.S. Army Research Institute 

for the Behavioral and Social Sciences

A Directorate of the Department of the Army 

Deputy Chief of Staff, G1 

Authorized and approved for distribution: 

 

MICHELLE SAMS ZITA M. SIMUTIS 

Technical Director Director 

Technical review by 

Jennifer S. Tucker, U.S. Army Research Institute 

Paula J. Durlach, U.S. Army Research Institute 

NOTICES 

DISTRIBUTION: Primary distribution of this Research Report has been made by ARI. 

Please address correspondence concerning distribution of reports to: U.S. Army 

Research Institute for the Behavioral and Social Sciences, Attn: DAPE-ARI-MS, 

2511 Jefferson Davis Highway, Arlington, Virginia 22202-3926. 

FINAL DISPOSITION: This Research Report may be destroyed when it is no longer 

needed. Please do not return it to the U.S. Army Research Institute for the Behavioral 

and Social Sciences. 

NOTE: The findings in this Research Report are not to be construed as an official 

Department of the Army position, unless so designated by other authorized documents.
i

REPORT DOCUMENTATION PAGE 

1. REPORT DATE (dd-mm-yy)

February 2006 

2. REPORT TYPE

 Interim 

3. DATES COVERED (from. . . to) 

 November –December 2005 

4. TITLE AND SUBTITLE 5a. CONTRACT OR GRANT NUMBER 

Preliminary Evaluation of a Novel Simulation-Based Tool for Training Rapid 

Decision-Making Skills 5b. PROGRAM ELEMENT NUMBER 

 622785 

5c. PROJECT NUMBER 

 A790 

5d. TASK NUMBER 

 215 

6. AUTHOR(S) 

Richard E. Christ (U.S. Army Research Institute)

5e. WORK UNIT NUMBER 

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 

Infantry Forces Research Unit 

ATTN: DAPE-ARI-IJ 

P.O. Box 2086 

Fort Benning, Georgia 31995-2086 

8. PERFORMING ORGANIZATION 

REPORT NUMBER 

10. MONITOR ACRONYM 

 ARI 

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 

U. S. Army Research Institute for the Behavioral and Social Sciences 

2511 Jefferson Davis Highway 

Arlington, Virginia 22202-3926 

11. MONITOR REPORT NUMBER 

 Research Report 1847 

12. DISTRIBUTION/AVAILABILITY STATEMENT 

Approved for public release; distribution is unlimited 

13. SUPPLEMENTARY NOTES 

Subject Matter POC: Richard E. Christ 

14. ABSTRACT (Maximum 200 words): 

This report describes an initial evaluation of a desktop training tool named the Simulated Field Exercise (SimFX). 

SimFX is different from other desktop trainers in that it uses a discrete, outcome-driven simulation for training leader 

decision making rather than a simulation driven by inputs from the virtual operating environment. The training 

scenarios used in SimFX exploit the cognitive realism that occurs when the leader is engaged in an interactive, 

branching storyline. The SimFX simulation advances the leader from one decision point to the next, and therefore 

focuses the leader on making decisions using available information rather than on experiencing the subtleties of a 

virtual environment. The SimFX was evaluated using the results from a questionnaire completed by 19 participants 

in a hands-on workshop that was conducted to introduce SimFX to a broad cross-section of trainers and training 

developers at Fort Benning, Georgia. The findings of this preliminary evaluation of SimFX suggest that it may be 

both an effective and an efficient means to train information-processing and decision-making skills. Equally 

important, the SimFX tool has the capability to be an aid in the development of the cognitive skills required for both 

the current force and for the envisioned future force environment. 

15. SUBJECT TERMS 

Training Decision Making Training Simulation Discrete Outcome-Driven Simulation 

Future Force Desktop Simulation Authorable Simulation Infantry Operations Phase II SBIR

 SECURITY CLASSIFICATION OF 19. LIMITATION 

OF ABSTRACT

20. NUMBER OF 

PAGES

21. RESPONSIBLE PERSON

16. REPORT 

Unclassified 

17. ABSTRACT 

Unclassified 

18. THIS PAGE 

Unclassified 

 

Unlimited 

 Ellen Kinzer 

Technical Publication Specialist 

703-602-8047 

 Standard Form 298
ii
Research Report 1847 

Preliminary Evaluation of a Novel Simulation-Based Tool for 

Training Rapid Decision-Making Skills

Richard E. Christ 

U.S. Army Research Institute 

Infantry Forces Research Unit 

Scott E. Graham, Chief

U.S. Army Research Institute for the Behavioral and Social Sciences 

2511 Jefferson Davis Highway, Arlington, VA 22202-3926 

February 2006 

Army Project Number Personnel Performance 

622785A790 and Training Technology

Approved for public distribution; distribution is unlimited

iii
ACKNOWLEDGEMENTS 

The hands-on workshop for the SimFX tool could not have been conducted without 

the full participation of its developers from Micro Analysis and Design, Inc. Special 

acknowledgement is given to this team of developers that included: Alan Brockett, who 

led the workshop, Rick Archer, Walter Warwick, and Patty McDermott. The participants

of the workshop gave their full attention to the procedures used to introduce them to the 

SimFX tool and their completion of the SimFX Questionnaire. While the participants 

were insured their names would not be used in the reporting of the results, the author 

wishes to acknowledge the leaders of the Fort Benning activities and units that did 

authorize and encourage their participation. These Fort Benning activities and units 

include each of the following: Infantry Futures Group, TRADOC Systems Manager – 

Soldier, Directorate of Operations and Training, Henry Caro Noncommissioned Officer 

Academy, Combined Arms and Training Directorate, 3rd Brigade of the 3rd Infantry 

Division, and the 11th Infantry Regiment. 

iv
PRELIMINARY EVALUATION OF A NOVEL SIMULATION-BASED TOOL FOR 

TRAINING RAPID DECISION-MAKING SKILLS 

EXECUTIVE SUMMARY 

Research Requirement 

The Army’s Future Force concept exploits the opportunities made possible by advances 

in our capacity to quickly gather, organize, and distribute battlespace information 

available from multiple sensor and database systems. However, as systems are being 

planned to train the decision-making skills required by these advanced communication 

and information systems, there is a continuing need to develop the decision-making 

competence of small-unit leaders who operate in current conventional environments. A

novel computer-based training tool, called the Simulated Field Exercise (SimFX) tool, 

was developed that takes into consideration various combinations of conventional and 

more technologically advanced operational environments. This report documents the 

approach and the results of a preliminary evaluation of the SimFX training tool. 

Procedure 

The SimFX was evaluated using the results from a questionnaire completed by 19 

participants in a hands-on workshop conducted for a broad cross-section of trainers and 

training developers at Fort Benning, Georgia. One of the developers of SimFX 

demonstrated a full-mission scenario and two deliberative practice exercises. Then, 

each participant was encouraged to use and explore the properties of each training 

component of SimFX. The developer also demonstrated how an authoring tool could 

make changes to the full-mission scenario and create a new scenario from scratch. 

Findings 

This preliminary evaluation of the SimFX tool showed that the workshop participants 

believed it could be an effective tool for training information-processing and decisionmaking skills and that the users would be personally involved in the training they 

received. The participants also indicated that SimFX was relatively easy to use for 

training and for authoring existing and new training scenarios and practice exercises. 

Equally important, the SimFX tool was perceived as being an aid for the development of 

the cognitive skills required for both the current force and for the envisioned Future 

Force environment. 

Utilization and Dissemination of Findings 

The results of this preliminary evaluation of SimFX have been presented to senior 

leaders of Infantry training and training development at Fort Benning. These key 

Infantry training personnel have been invited to participate in a follow-on workshop 

designed to further the transition of this research product into the pool of training 

systems available to institutional activities and operational units. 

v
vi
PRELIMINARY EVALUATION OF A NOVEL SIMULATION-BASED TOOL FOR 

TRAINING RAPID DECISION-MAKING SKILLS 

CONTENTS

Page 

INTRODUCTION ...................................................................................................... 1 

Key Feature of SimFX ......................................................................................... 2 

Purpose of This Report ....................................................................................... 2 

METHOD .................................................................................................................. 3 

 Participants ......................................................................................................... 3 

 Procedure ........................................................................................................... 3 

The SimFX Questionnaire ................................................................................... 4 

RESULTS ................................................................................................................. 5 

Rated Opinions About SimFX ............................................................................. 5 

Written Comments About SimFX ........................................................................ 7 

DISCUSSION ........................................................................................................... 8 

CONCLUSION ......................................................................................................... 8 

REFERENCES ......................................................................................................... 9 

Appendix A. A description of the SimFX training tool. ............................................. A-1 

Appendix B. SimFX Questionnaire .......................................................................... B-1 

Appendix C. Detailed Results from the SimFX Questionnaire.................................. C-1 

List of Tables 

Table 1. Number of Respondents by Status and Agency Represented .................. 3 

vii
viii
Preliminary Evaluation of a Novel Simulation-Based Tool 

for Training Rapid Decision-Making Skills 

INTRODUCTION 

The Infantry Forces Research Unit of the U.S. Army Research Institute for the 

Behavioral and Social Sciences is conducting research to evaluate the training potential 

of desktop simulations of dismounted Infantry operations (Beal, 2005; Beal & Christ, 

2004, 2005; Centric, Beal, & Christ, 2005). The desktop simulations evaluated were 

developed to provide Infantry leaders with low-cost, repetitive opportunities to 

experience realistically the consequences of executing an operations order and the 

challenges inherent in making hasty changes to those orders in response to emerging 

tactical conditions. Generally, the declared purpose of these simulations was to train 

the types of cognitive skills needed by Infantry leaders to make optimal decisions in the 

current or contemporary operating environments. 

However, in keeping with the Army’s modernization plan, there is a simultaneous 

need to develop and evaluate desktop training tools that have a high potential to 

enhance the types of cognitive skills that would enable rapid decision making in the 

proposed future operating environment (Department of the Army, 2005). The Army’s 

Future Force concept exploits the enormous opportunities made possible by advances 

in our capacity to quickly gather, organize, and distribute battlespace information. It can 

be argued that the transition from the current to the future operating environments may 

produce changes in the nature of the information-processing and decision-making skills 

currently required to achieve decision-making proficiencies (Mosier, 2001). 

Several years ago the author developed a topic statement for the Small Business 

Innovation Research (SBIR) program that asked for the development of a computerbased system that could be used to train rapid decision-making skills of small unit 

leaders regardless of the level of technology used in their operating environment. The 

training tool was to be initially developed for use by dismounted Infantry platoon leaders 

but also have the capability to be used by leaders at both higher and lower echelons. 

Further, the emphasis of the tool was to train the leader’s ability to access, integrate, 

and effectively use information from multiple sources to improve his decision-making 

proficiency. Finally, special importance was placed on the role of the information 

provided or available to the leader, but not on the source of the information, the format 

and structure of the information (i.e., its analog or digital format), or the means through 

which the information was presented (i.e., the user-system interface or knobology). 

Based on the quality of the background work and plans accomplished during a phase I 

effort, a phase II SBIR contract for this topic was awarded to Micro Analysis and Design 

(MAAD), Inc. During the phase II effort, MAAD developed a tool, called the Simulated 

Field Exercise or SimFX, that met the objectives of the SBIR topic. 

1
Key Feature of SimFX 

The one feature of SimFX that distinguishes it from other desktop simulations that 

have been evaluated at Fort Benning is that it uses a discrete, outcome-driven 

simulation for training leader decision making rather than a simulation driven by inputs 

from the virtual operating environment. (A more complete description of the concepts 

and methods that underlie the development of the SimFX training tool is provided in 

Appendix A.) Unlike most other desktop simulations, the approach used to create 

SimFX training scenarios does not depend on the development of an immersive virtual 

reality in which the user performs his small unit leader tasks. Consequently, the SimFX 

software does not need to model autonomous intelligent adversaries, and it does not 

need to adjust for differences in the simulated versus real world states to meet the 

training objectives. These features characteristic of most other desktop simulations 

make them hard to maintain and almost impossible to adjust for changes in the training 

tasks and conditions. 

The outcome-driven simulation incorporated in SimFX exploits the cognitive realism 

that is created when the learner engages in an interactive, branching storyline. The 

SimFX simulation advances the leader from one decision point to the next, and, 

therefore, focuses the leaders on making decisions using information at their disposal

rather than on their experiencing the subtleties of a virtual environment. The developer 

of SimFX utilized techniques to manage the size of the decision tree that drives the 

decision-making points in the story line. These techniques permit ground truth to be 

adjusted as necessary to meet the training objectives and permit easy modifications of a 

scenario and the creation of new scenarios. 

It became apparent during an in-progress review meeting for this SBIR contract in 

mid-October 2005 that the SimFX tool had been developed to the point that it should be 

reviewed and evaluated by Infantry trainers and training developers at Fort Benning. 

The software development was sufficiently complete to support both authoring and 

execution of training scenarios. An in-house evaluation of the initial beta-version of the 

software established a proof of concept with a simple test bed scenario. The test of the 

beta software demonstrated successfully how decisions points of different types can be 

implemented to insure the face validity of the story line, and to produce an engaging 

training scenario. The developers created a doctrinally correct set of four full-mission 

scenarios and two deliberative practice exercises. The latter presented repeatedly a 

single type of decision task, e.g., deciding whether two images of an environmental 

feature were the same or different. It was decided that a revised beta-version of the 

SimFX software could be demonstrated during a hands-on workshop. This 

demonstration would provide a basis for conducting a preliminary evaluation of its 

potential for training in the Infantry community. 

Purpose of This Report 

This report documents the preliminary evaluation the SimFX training tool. The 

report provides a brief description of the procedures used for conducting a hands-on 

workshop in which SimFX was demonstrated. It also describes the outcome of the 

2
workshop based on an assessment of the perceptions and opinions of the Infantry 

trainers and training developers who participated in the workshop. 

METHOD 

Participants 

To encourage participation in the workshop, the author conducted a series of 

meetings to briefly describe the SimFX training tool and the hands-on workshop to the 

leaders of ten key training and training development activities and units across Fort 

Benning. In addition, the author provided information about the SimFX tool and the 

workshop to his colleagues in the Infantry Forces Research Unit and in two contractor 

firms. The contractor firms regularly provide support for training and training 

development to Fort Benning and to the Infantry Forces Research Unit. Eight of the 

invited Fort Benning activities and agencies, as well as the three civilian organizations 

sent representatives to the workshop. About 30 individuals participated in all or most of 

the workshop. Of these participants, 19 completed the questionnaire that provided the 

data used in this report. 

Table 1 provides a general breakdown of the respondents who completed the 

questionnaire. It was possible to identify the status and the activity or unit represented 

by all but two of these respondents. The respondents were told that their responses to 

the questionnaire would be anonymous. There was no attempt to gather any 

information about the respondents’ experience in Infantry training or training 

development. However, it is reasonable to conclude that most of the senior 

commissioned officers, noncommissioned officers (NCOs), and civilian respondents 

possessed considerable experience in these areas. 

Table 1. Number of Respondents by Status and 

Agencies Represented 

Status Number of 

Respondents 

Number of Different 

Agencies Represented 

Military

Officer 5 3

NCO 6 4

Civilian 6 3

Unknown 2 ~

Note: Two of the 19 respondents elected not to provide their names 

and their status and affiliation could not be determined. 

Procedure 

The SimFX workshop was conducted in a classroom at Fort Benning in which each 

participant was able to use the SimFX software with his or her own desktop computer. 

The workshop began with an introductory presentation that lasted about 15 minutes by 

the lead member of the MAAD team. During this presentation the operating sets of 

assumptions and principles that drove the development of the SimFX training tool were 

described. 

3
Following the introductory presentation, the participants were told to turn on their 

computers to access to the revised beta-version of SimFX. For the next fifteen minutes 

the lead contractor demonstrated the full-mission scenario and the two deliberative 

practice exercises that were contained in the software. The participants were 

encouraged to ask questions during this developer-led demonstration of the revised 

beta-version of the SimFX software. 

During the next 60 minutes, the participants were encouraged to use and explore 

the properties of each component of the training software. While the participants used 

the SimFX tool, the lead developer and three of his colleagues (as well as the author) 

answered questions raised by the participants and provided information. At the 

conclusion of this 60-minute hands-on demonstration, the participants were told they 

could take a short 15-minute break. (Up to one-half the participants elected not to take 

a break or to take only an abbreviated break. These participants used some or all of the 

break time to continue investigating the properties of the SimFX training tool.) 

Following the break, the lead developer demonstrated how the authoring tool could 

be used to make changes to the full mission scenario that was contained in the betaversion of SimFX software. He then demonstrated how the authoring tool could be 

used to create a new, three-node scenario from scratch. A hand-out was provided to 

the participants to guide them through this application of the authoring tool. During and 

following this demonstration, the participants were encouraged to raise questions or to 

offer comments to the developers. The demonstration of the authoring capability of the 

SimFX tool lasted about 30 minutes. 

At the end of this final demonstration period, each participant was given a copy of 

the SimFX Questionnaire. The participants were encouraged to complete the SimFX 

Questionnaire before they left the classroom. The author collected all the 

questionnaires that were completed. 

The SimFX Questionnaire 

The SimFX Questionnaire was developed to capture the opinions of the participants 

about their experiences using the SimFX tool during the workshop. A copy of the 

questionnaire is given in Appendix D. Successive parts of the questionnaire asked the 

respondents to rate the following:

• Training value of the SimFX 

• Extent to which users of SimFX would be personally involved with the training 

• Ease of use or the usability of SimFX for training and for editing or authoring the 

training material presented with SimFX. 

Participants also were asked to provide written comments about SimFX in terms of its 

advantages and disadvantages as a training tool and as an editing or authoring tool. 

In the first two parts of the questionnaires, respondents used a seven-point rating 

scale to indicate their responses to questions about SimFX. In these two parts of the 

4
questionnaire, a scale value of 1 reflected the most negative rating for SimFX, a scale 

value of 4 a neutral or indifferent rating, and a scale value of 7 the most positive rating 

for SimFX. In part three of the questionnaire, respondents used a five-point rating scale 

to indicate their reactions to positive statements about the usability of SimFX. The five 

scale values were verbally labeled Strongly disagree, Disagree, Neither agree nor 

disagree, Agree, and Strongly agree, and assigned numeric values of 1 through 5, 

respectively. Many of the items to be rated in the questionnaire were modified from 

those used to assess the training effectiveness of other Army simulation-based training 

tools (Beal & Christ, 2004, 2005). 

In the last part of the questionnaire the respondents were asked to describe their 

overall opinions of the training and authoring capabilities of SimFX. They were asked to 

describe up to three positive and three negative features of SimFX for training and up to 

two positive and two negative features of SimFX for authoring training scenarios and 

exercises. Observations of and oral comments by the participants during the workshop 

were not formally recorded and are not presented in this report. 

RESULTS 

Rated Opinions About SimFX

Training value of the SimFX tool. The participants indicated that they believed 

the SimFX would have training value. Across the eleven items in this part of the 

questionnaire, between 68 and 95 percent of the respondents used one of the three 

highest rating categories to indicate a positive opinion of the training value of the 

SimFX. (Details of the ratings for training value are presented in Appendix Table C-1). 

The five items on which over half of the respondents used one of two most positive 

rating categories to indicate their opinion of the training value of the SimFX tool are 

listed below (in descending order of the percentage of respondents using one of these 

two rating categories). 

• Would training with SimFX help small unit leaders focus on critical factors that 

influence tactical decisions? 

• Would using SimFX provide a Soldier opportunity to practice making sound 

tactical decisions? 

• Would training with SimFX be a valuable learning experience? 

• Would training with SimFX improve a Soldier's ability to make more rapid tactical 

decisions? 

• To what extent will training with SimFX help a Soldier to make sound tactical 

decisions? 

User involvement during SimFX training. The participants indicated that users 

would become personally involved while training with the SimFX tool. Across the five 

questionnaire items that assessed this factor, between 52 and 95 percent of the 

participants used one of the three highest rating categories to indicate that a user would 

become involved with the SimFX training. (Details of the involvement ratings are 

presented in Appendix Table C-2). The two items on which over half the respondents 

5
used one of two highest rating categories to indicate their belief that users would be 

personally involved during SimFX training are shown below (in descending order of the 

percentage of respondents using one of these two rating categories). 

• How involved were you in the decision-making experiences provided by SimFX? 

• How often might a Soldier training with SimFX be completely focused on the 

decision-making task? 

Usability of SimFX as a training tool. The participants indicated that SimFX 

would be relatively easy to use and that they would use it as a training tool. Across the 

19 items used to assess this aspect of the SimFX tool, between 61 and 100 percent of 

the respondents either agreed or strongly agreed with positive statements about the 

usability of SimFX for training. (Details of the ratings of SimFX usability for training are 

presented in Appendix Table C-3). The four items on which over a third of the 

respondents used the strongly agree rating category for the usability of SimFX for 

training are listed below (in descending order of the percentage of respondents using 

this highest rating category). 

• If it were available, I would use SimFX as a training tool. 

• If the SimFX tool were available I would use it to train my Soldiers. 

• The "clock" shown in the SimFX display to indicate how much time remains to 

complete a mission was easy to understand. 

• The steps required to use SimFX for training purposes are easy to perform. 

Usability of SimFX for authoring training scenarios and exercises. The 

participants indicated they believed the SimFX software would make it relatively easy to 

modify existing training exercises and to create new ones more to their liking. They 

further indicated that if the software were available to them, they would exercise this 

capability. Across the 6 items in this part of the questionnaire, between 61 and 94 

percent of the respondents either agreed or strongly agreed with positive statements 

about the usability of SimFX for training. (Details of the ratings for the usability of SimFX 

authoring capabilities are presented in Appendix Table C-4). The three items on which 

over a third of the respondents used the most positive rating category for the authoring 

capability of SimFX are listed below (in descending order of the percentage of 

respondents using the strongly agree rating category). 

• If it were available, I would personally create some new SimFX scenarios or 

exercises. 

• If it were available, I would personally modify existing SimFX scenarios or 

exercises. 

• With just a little practice I could become very good at modifying and creating 

SimFX scenarios or exercises. 

6
Written Comments About SimFX 

Seventeen of the 19 respondents provided 101 written comments about positive

and negative features of the SimFX tool. Sixteen respondents gave 36 positive 

comments and 15 gave 28 negative comments about SimFX training capabilities. 

Sixteen respondents gave 24 positive comments and 9 gave 13 negative comments 

about the authoring capability of SimFX. These written comments are provided in 

Appendix Tables C5-C8. The author’s attempt to organize these written comments into 

meaningful clusters is reflected in these four appendix tables and is summarized in this 

section of the report. 

Written comments about using SimFX for training. The respondents’ positive 

comments about SimFX training followed a pattern similar to their rated opinions. The 

majority of the positive comments referred to the ease of using SimFX for training and 

its use specifically to train decision-making skills. Another cluster of the positive 

comments spoke specifically about the potential use of SimFX scenarios to 

accommodate other current and emerging training requirements. 

Some of the negative comments about training with SimFX were concerned with 

details specific to the scenario that was demonstrated in the workshop. For example, 

remote and robotic sensor systems played a key role in the demonstrated scenario but 

most of the workshop participants were not familiar with the capabilities of these 

emerging technologies. However, most of the negative comments about using SimFX 

for training highlighted issues that are common to the use of many training tools, such 

as the difficulty inherent in developing good scenarios and conditions for training, the 

difficulty of getting computers to use for training, and the preference for live training. 

Other negative comments showed that some features of SimFX were not clearly 

demonstrated during the workshop. 

Written comments about the SimFX authoring capability. Positive comments 

about the authoring capability of SimFX reinforced the rated opinions given for its 

usability. Most of the positive comments spoke to the relative ease of use and the 

flexibility provided by the authoring software. Many respondents indicated they would 

be able to tailor SimFX scenarios to fit their specific training requirements. Several 

respondents pointed specifically to the use of this capability to convert currently used 

paper-and-pencil-based decision exercises into SimFX scenarios and to use them in 

SimFX to enhance the training of tactical decision-making skills. 

Most of the negative comments about SimFX authoring capability addressed 

perceived difficulties in learning how to author scenarios. Another cluster of negative 

comments were concerned with whether the authoring capability could be used to 

create scenarios for the current force that does not have access to the remote sensor 

technology that was highlighted in the demonstrated scenario. 

7
DISCUSSION 

The results showed clearly that the participants in the hands-on SimFX workshop 

thought the SimFX tool has the potential to effectively train decision-making skills. Most 

respondents gave very high ratings and wrote positive comments about the value and 

ease of using SimFX for training. Over half the respondents indicated that Soldiers 

would become involved in the training provided by the SimFX tool. Further, most 

respondents indicated that, if SimFX were available, they would use it to train decisionmaking skills. Some respondents specifically recommended that currently used paperand-pencil decision-making exercises be converted into SimFX scenarios. 

While the rated opinions of workshop participants were quite positive toward the 

training value of the SimFX tool, their written comments indicated also some perceived 

negative aspects of training with the SimFX tool. Many of these negative comments 

about training with SimFX were keyed to only the specific scenario that was 

demonstrated in the workshop. These perceived negative aspects of the SimFX 

scenario could be neutralized or reversed if the participants had been exposed to other 

training scenarios that did not employ remote and robotic sensors. The other problems 

noted by the respondents were generally applicable to all training tools and simulations 

(e.g., difficulty in preparing good training scenarios) or are a reflection of more 

widespread problems in acquiring training resources (e.g., scarcity of computers for 

training purposes). 

The rated and written opinions of the workshop participants about the authoring 

capability built into the SimFX tool were more equivocal than those about its training 

capability. On the one hand, the respondents indicated that with a little practice they 

thought they would be able to use the authoring capability to modify existing scenarios 

as well as to create new scenarios to meet their training responsibilities. However, 

some respondents also indicated that it might be difficult to use the authoring capability 

of SimFX. One possible reason for the more ambivalent opinion about the authoring 

capability than the training capability of SimFX is that participants had hands-on 

experience using the training aspects of the SimFX tool but received only a description 

of its authoring capability. It is anticipated that planned future hands-on demonstration 

of both the training and the authoring capabilities of the final version of the SimFX tool 

will reinforce the positive user opinions of the tool and reduce if not eliminate any 

perceived problems with its use. 

CONCLUSION 

The findings of this preliminary evaluation of the SimFX tool suggest that the tool 

may be both an effective and an efficient means to train information-processing and 

decision-making skills. Equally important, the SimFX tool has the capability to aid to the 

development of the cognitive skills required for both the current force and for the 

envisioned future force environment. Given the relatively low level of development of 

the SimFX tool used in this evaluation, the short duration of the workshop used for this 

evaluation, and, most importantly, the absence of a hands-on demonstration of its 

authoring capability, it is important to conduct appropriate follow-on evaluations of this 

desktop training simulation as it is further refined and developed. 

8
REFERENCES 

Beal, S. A. (2005). Using Games for Training Dismounted Light Infantry Leaders: 

Emergent Questions and Lessons Learned (Research Report 1841). Arlington VA: 

U.S. Army Research Institute for the Behavioral and Social Sciences. (ADA 438042) 

Beal, S.A. and Christ, R. E. (2004). Training Effectiveness Evaluation of the Full 

Spectrum Command Game (Technical Report 1140) Alexandria, VA: U.S. Army 

Research Institute for the Behavioral and Social Sciences. (ADA 419670) 

Beal, S. A. and Christ, R. E. (2005). Soldier Perceptions of the Rapid Decision Trainer

(Research Report 1835). Arlington VA: U.S. Army Research Institute for the 

Behavioral and Social Sciences. (ADA 436993) 

Centric, J. H., Beal, S. A. and Christ, R. E. (2005). Train-the-Trainer Package for the 

Full Spectrum Warrior Game (Research Product 2005-02). Arlington, VA: U.S. Army 

Research Institute for the Behavioral and Social Sciences. (ADA 428443) 

Department of Army (2005). 2005 Army Mod Plan. Washington, DC: Author. 

Mosier, K. (2001). Cognition in the automated cockpit: A coherence perspective. 

Proceedings of the 45th Annual Meeting of the Human Factors and Ergonomics

Society. Santa Monica, CA, 68-72. 

9
10
Appendix A. A description of the SimFX training Tool 

SIMULATION BASED TRAINING FOR THE FUTURE FORCE WARRIOR ∗

Walter Warwick, Rick Archer, Alan Brockett, and Patty McDermott 

Micro Analysis and Design 

Boulder, CO 

Abstract: In this paper we describe techniques we have adopted to develop a 

computer-based, outcome-driven simulator to train digital information skills for 

small unit leaders of the Army’s Future Force Warrior program. We begin by 

contrasting attempts to engender “virtual realism” in simulation based training

against attempts to engender cognitive realism by way of the branching storylines 

at the heart of an outcome-driven simulation. We next present an example of how 

such an approach might be applied to train digital information skills before 

turning to a more general discussion of the problems that such an approach 

entails—namely, crafting an engaging story while minimizing the combinatorial 

explosion in a branching storyline. We describe how we have dealt with these 

problems both by streamlining storylines and by decoupling student input from 

the branching process. Finally, we allude to a software tool we have created that 

allows the training developer to author and execute such outcome-driven 

simulations.

INTRODUCTION 

Realism is an essential component of simulation-based training. For many computer-based 

simulations, this realism is accomplished with the construction of a highly detailed, carefully 

rendered, synthetic environment coupled with some sort of input device that allows the student to 

interact with the simulated environment. So-called virtual reality permits the student to explore 

the simulated training environment in real-time, in a perceptual-motor situation similar to the real 

world. In principle, anything the student might do in the actual environment could be done via 

simulation in a virtual environment. As long as the simulated environment reflects the salient 

interactions of the actual environment, the student can gain valuable experience performing tasks 

that are either too dangerous, or too expensive, or simply impossible to perform in the actual 

environment. While effective for some types of training, immersion in a virtual reality comes 

with its own issues and significant overhead that do not justify its application in every training 

domain. (In fact, it is not clear that highly realistic synthetic environments provide useful 

training for dismounted Soldiers. Cf. Pleban (2003), Christ (2004)) Outcome-driven simulation 

has recently emerged as one possible alternative (e.g., Gordon, 2004). In outcome-driven 

simulation the goal is no longer to immerse the student in a virtual reality but, rather, to exploit 

the cognitive realism that follows from engaging the student in a story or vignette where the 

student must make a series of decisions that ultimately affects how the story plays out. 

Outcome-driven simulation trades the continuous environment of virtual reality for a set of 

discrete choice points built into a narrative structure. By scripting together a series of choice 

points in a branching storyline, the training developer maintains control over the interactions

 

∗ Published in the Proceedings of the 49th Annual Meeting of the Human Factors and Ergonomics Society

(Orlando, FL, 2005). 

A- 1
Appendix A. A description of the SimFX training Tool 

between student and simulation—ensuring that the student will encounter specific decisions at 

specific times rather than wandering aimlessly or unpredictably through a virtual environment. 

However, crafting the branching story line that constitutes an outcome-driven simulation places a 

significant burden on the developer to come up with an engaging yet tractable scenario. If the 

training developer constructs a scenario with too few choice points he runs the risk of 

constructing a simulation that is no more engaging than a short multiple choice exam. At the 

other extreme, if the developer tries to string together too many choice points he will quickly 

find himself lost in a combinatorial explosion. The training developer must strike a balance 

between engaging the student and managing the complexity of a scenario maintaining all the 

while some semblance of continuous flow and believability throughout the scenario, no matter 

which choices the trainee makes. 

In this paper we describe how we have struck such a balance in the development of a 

computer-based, outcome-driven training simulator. The simulator is intended to train the small 

unit leaders of the U.S. Army’s Future Force Warrior (FFW) program to make sense of the ever 

wider array of information technologies available as the Army transforms itself to fight on the 

“digital battlefield.” The emphasis on digital technology and the ability of the Soldier to fuse 

information from various sources makes this domain especially well suited to an outcome-driven 

simulation. A series of discrete choice points can be used to present the Soldier with particular 

pieces of information that must be fused to make the “right” decision (i.e., the decision that 

allows the scenario to unfold in the manner intended by the training developer). Still, crafting 

the training scenarios has been far from trivial. We describe below how we have exploited some

of the “chapter-based” techniques from Gordon (2004) to manage combinatorial explosion, along 

with insights from the gaming community that inspired us to decouple the overt choices made by 

the student from those that actually dictate the flow of control through the scenario. In this way 

we present the illusion that the student is interacting with a simulated world that exists

independently without incurring the overhead that would otherwise be necessary to maintain a 

consistent world state. We also allude to a general software tool we’ve built that allows us to 

implement these techniques in the development of computer-based, outcome-driven training 

scenarios. 

SCENARIOS TO TRAIN DIGITAL SKILLS FOR THE FFW 

Although a good deal has been written recently about the impacts of digital technologies 

and their implications for training, our work was motivated by the straightforward observation 

that expertise is generally built on a foundation of hands-on experience. So, rather than focus 

training on the specifications and capabilities of new digital technologies—the “knobology” of 

new technology—we set out to provide students with computer-based scenarios that would force 

them to resolve ambiguous or contradictory sensor readings, fuse disparate sources of 

information, filter information, manage resources, (e.g., time, network bandwidth) and learn how 

to employ sensors to the greatest effect in a tactical situation.

For example, at one decision point we might ask the student to pick among three routes to 

a waypoint. The paths are presented on an electronic display of a map. The student has the 

ability to query various information sources. In addition to traditional information sources (e.g., 

an operations order, radio communications, map overlays) the student can query unattended 

acoustic sensors, reconnaissance from unmanned air and ground vehicles and spot reports from a

A- 2
Appendix A. A description of the SimFX training Tool 

densely connected communications network. Choosing the correct path means querying the 

appropriate sensor and making good use of the information it provides. In this case, the decision 

was crafted so that the student must recognize that the indication of foot traffic reported by an 

unattended acoustic sensor in the vicinity of one route is inherently ambiguous and that the 

determination of whether it is due to enemy or friendly activity along the route depends on 

querying another sensor—perhaps inspecting recent aerial reconnaissance. The choice can be 

further complicated by layering tactical considerations and time management demands (e.g., the 

shortest route offers less cover). 

Although seemingly straightforward, implementing this decision point depended on the 

solutions to several interrelated questions. First, we had to decide how information would be 

presented. While we wanted to preserve the “look and feel” of the information sources, we 

didn’t want the student to become mired in the painstaking analysis of a grainy reconnaissance 

photograph or the interpretation of a particular acoustic signature in a noisy signal. Instead, we 

opted to present information from these sources abstractly (usually as text-based reports from a 

notional intelligence analyst who reviews sensor data), to emphasize how the student should 

integrate such facts once presented rather than train interpretation of raw data. More generally, 

the abstract representation reflects the desire to steer away from a detailed underlying model 

where a consistent “world state” can be maintained and presented to the student (via additional 

and comparably complex sensor models). Instead, the training developer simply specifies the 

information provided to the student at each decision point, tweaks the simulated world as 

necessary (e.g., adding enemies at a location, removing assets), much in the same way that a socalled observer-controller will change the course of a live training exercise to suit the training 

objectives. But while the training developer gains greater control of the simulation in this way, it 

comes at a cost. Without detailed, underlying models to maintain a consistent world state, it falls 

on the training developer to manage the complexity and consistency of the unfolding scenario. 

As indicated above, managing this complexity is difficult. 

Even with only a few choices at each decision point, keeping track of all the possible paths 

through a scenario becomes unmanageable after a handful of decisions. While some degree of 

combinatorial explosion is inevitable, it can be minimized in a number of ways. First, as Gordon 

(2004) describes, a branching scenario can be pruned by introducing “chapters” whereby a series 

of decisions ultimately funnel back to a single decision. For example, we ask the student a short 

series of questions, each of which asks where he’d move to next given the available information 

sources (which change from decision to decision). But rather than ramify the student’s decisions 

throughout the entire scenario, we introduce a new series of questions by discontinuously 

moving the student to a new location that could plausibly be reached no matter which route the 

student chose previously. 

A second technique for minimizing combinatorial explosion is simply to avoid it in the 

first place by posing non-branching decisions. Such decisions either ask the student to provide 

factual responses about digital technologies (e.g., “Can your unmanned acoustic sensor field at 

the objective detect truck traffic on the road just east of the objective?”), or we can ask the 

student to estimate the resources required to execute particular phases of the mission, or, finally, 

use rhetorical strategies to force the student to deepen his thought about the tactical situation 

(e.g., “Have you considered how your operational tempo would be affected if your were flanked 

A- 3
Appendix A. A description of the SimFX training Tool 

en route to the objective?”). While the student will be prompted for a response, the response 

does not change how the scenario advances. 

Finally, borrowing techniques from the gaming community, we have found it is possible to 

present the student with a seemingly genuine decision (i.e., a choice that affects outcomes) 

without having to represent those outcomes in the scenario. The trick here is not to predicate the 

outcome of the decision on what the student actually chooses, but rather, on what the student 

knew or should have known before he made his choice (which we can infer by keeping track of 

which information assets were queried). Much in the same way that a video game designer will 

program a monster to appear in whatever room the player enters, we can ensure that bad things 

will happen whenever a student fails to make the best use of the information assets at his 

disposal. Returning to our earlier example, independent of route the student actually chooses, we 

can place an enemy ambush whenever the student fails to disambiguate the reports from his 

acoustic sensors. Conversely, the enemy will be absent whenever the appropriate combination of 

sensors is queried and we will reward the student for recognizing the original ambiguity, thus 

reinforcing the training objective. While this style of question requires the training developer to 

specify training feedback (i.e., outcomes) for a potentially large number of sensor combinations, 

it allows scenarios, or large parts of them, to be developed without any branching and so the 

level of effort tends to grow linearly rather than exponentially in the depth of the scenario. 

DISCUSSION 

We built a software tool that allows us to implement all three styles of decision points. 

The training developer is able to specify the scenario structure using a Decision Tree Editor 

(depicted in Figure 1 below). The Decision Point Editor (depicted in Figure 2 below) allows the 

training developer to specify each decision point, including the information that will be

available, the choices the student can make and the feedback the student will receive. 

Figure 1 Decision Tree Editor

A- 4
Appendix A. A description of the SimFX training Tool 

Figure 2 Decision Point Editor

Our effort to date has been on software development, as opposed to content development. 

We have created simplistic scenarios to anchor intuitions and focus our efforts, but our real 

interest is in tool development (although we have just begun working with an infantry subject 

matter expert to develop more robust training scenarios). Accordingly, our focus on evaluation 

has been limited to the usability of the software and its ability to represent the kinds of

information-rich decision points we think might be important training points for the FFW. The 

evaluation of training effectiveness will have to wait until richer content can be developed. 

While our approach places a non-trivial demand on the developer to produce a wellcrafted outcome driven scenario, the required effort pales in comparison to that required to 

develop, maintain and use more immersive simulation environments. At the other extreme, we 

are inspired by a generation of paper and pencil exercises, called Tactical Decision Games, that 

the U.S. Marines use to suspend disbelief and engage the story behind the training scenario. We

see our approach as a middle road between high-fidelity simulations and pencil-and-paper 

exercises. 

References 

Beal, S.A. & Christ, R.E. (2004). Training Effectiveness Evaluation of the Full Spectrum 

Command Game (Technical Report 1140). Arlington, VA: U.S. Army Research Institute 

for the Behavioral and Social Sciences. 

Gordon, A. (2004). Authoring Branching Storylines for Training Applications. Proceedings of 

the Sixth International Conference of the Learning Sciences (ICLS-04), Santa Monica, CA. 

Pleban, R. J. & Salvetti, J. (2003). Using Virtual Environments for Conducting Small Unit 

Dismounted Mission Rehearsals (Research Report 1806). Arlington, VA: U.S. Army

Research Institute for the Behavioral and Social Sciences. 

A- 5

Appendix B. SimFX Questionnaire 

Simulated Field Exercise (SimFX) Tool Questionnaire

Print your full name________________________________________________ Please note: Your name and your responses to this questionnaire will never be made public. 

ARI might use your name if it wishes to contact you to follow-up on your experiences with SimFX. 

Based on your admittedly short experience with SimFX and on your experience as a trainer in the 

Infantry, respond to the questions in Parts 1 and 2 of this questionnaire by marking an "X" in the 

appropriate box of the 7-point scale. Please consider the entire 7-point scale before making your 

responses. 

Part 1. Overall Potential Training Value of SimFX 

 1. Would using SimFX provide a Soldier opportunity to practice making sound tactical decisions? 

|________|________|________|________|________|________|________| 

 NO SOME GREAT 

 OPPORTUNITY OPPORTUNITY OPPORTUNITY 

 2. Would training with SimFX improve a Soldier’s ability to make more rapid tactical decisions? 

|________|________|________|________|________|________|________| 

 WILL NOT MAY WILL DEFINITELY 

 IMPROVE IMPROVE IMPROVE 

 3. Would training with SimFX make a Soldier more confident in his ability to make tactical decisions? 

|________|________|________|________|________|________|________| 

 NOT MORE SOMEWHAT MORE MUCH MORE 

 CONFIDENT CONFIDENT CONFIDENT 

 4. How challenging is the overall experience provided by training with SimFX?

|________|________|________|________|________|________|________| 

 NOT MODERATELY VERY 

 CHALLENGING CHALLENGING CHALLENGING 

 5. Would training with SimFX have a valuable impact on small unit leaders?

|________|________|________|________|________|________|________| 

 NO SOME GREAT 

 VALUE VALUE VALUE 

 6. Would training with SimFX help small unit leaders focus on critical factors that influence tactical 

decisions? 

|________|________|________|________|________|________|________| 

 NO SOME EXCELLENT 

 FOCUS FOCUS FOCUS 

 7. To what extent could SimFX teach the Soldier something new about decision making that 

is not now or not easily covered in normal classroom or field training? 

|________|________|________|________|________|________|________| 

 NOT AT ALL SOMEWHAT VERY MUCH 

Continue on the next page

B - 1
Appendix B. SimFX Questionnaire 

 8. To what extent will training with SimFX help a Soldier to make sound tactical decisions? 

|________|________|________|________|________|________|________| 

 NOT AT ALL SOMEWHAT VERY MUCH 

 9. To what extent will training with SimFX permit a Soldier to practice the types of decisions he must 

make as a small unit leader? 

|________|________|________|________|________|________|________| 

 NOT AT ALL SOMEWHAT VERY MUCH 

10. Would training with SimFX be a valuable learning experience?

|________|________|________|________|________|________|________| 

 NO SOME GREAT 

 VALUE VALUE VALUE

11. As a trainer, how satisfied were you with the training opportunities provided by SimFX? 

|________|________|________|________|________|________|________| 

 NOT SOMEWHAT COMPLETELY

 SATISFIED SATISFIED SATISFIED

Part 2. User Involvement or Immersion in SimFX

 1. How captivated or drawn in would a Soldier be by the tactical events and actions presented in 

SimFX? 

|________|________|________|________|________|________|________| 

 NOT AT ALL SOMEWHAT COMPLETELY 

 2. How often might a Soldier training with SimFX be completely focused on the decision-making task? 

|________|________|________|________|________|________|________| 

 NONE OCCASIONALLY FREQUENTLY 

 3. How involved were you with tactical events and actions portrayed in SimFX?

|________|________|________|________|________|________|________| 

 NOT AT ALL SOMEWHAT COMPLETELY 

 4. How involved were you in the decision-making experiences provided by SimFX? 

|________|________|________|________|________|________|________| 

 NOT MILDLY COMPLETELY 

 INVOLVED INVOLVED ENGROSSED 

 5. Were you so involved while using SimFX that you lost track of time? 

|________|________|________|________|________|________|________| 

 NOT AT ALL SOMEWHAT COMPLETELY 

Continue on the next page

B - 2
Appendix B. SimFX Questionnaire 

Part 3. The Degree to Which SimFX is Easy or Difficult to Use

In Part 3 of the questionnaire draw a circle around the letter that best indicates the extent to which you 

agree or disagree with each statement. Write the letters NA to the left of the statement number to 

indicate that you have no basis for having an opinion about the statement. 

Strongly disagree 

Disagree | 

Neither agree nor disagree | | 

Agree | | | 

Strongly agree | | | | 

| | | | |

Section A: Using SimFX to Train Decision-Making Skills | | | | |

 1. If it were available, I would use SimFX as a training tool. ....................................... A B C D E 

 2. The steps required to use SimFX for training purposes are easy to perform. ........ A B C D E 

 3. The abbreviated operations order provided to the user at the beginning 

 of a scenario were adequate to understand the training mission. .......................... A B C D E 

 4. The description of the factors that defined each decision point were 

 adequate for the purpose of the training exercise. ................................................. A B C D E 

 5. The types of decision-making tasks possible with SimFX are similar 

 to those encountered in an operational environment. .............................................. A B C D E 

 6. It was easy to request and obtain information while executing the mission. .......... A B C D E 

 7. The reporting process was adequate to keep my CO informed. ............................ A B C D E 

 8. The signals used on the SimFX screen for alerting the user that a source 

 of information is available were easy to understand. .............................................. A B C D E 

..9. The printed text-based information available to the user of SimFX 

 were easy to understand. ........................................................................................ A B C D E 

10. The printed text-based information available to the user of SimFX were 

 useful for selecting a course of action at the decision point. .................................. A B C D E 

11. The graphic- or photographic image-based information available to the user

 of SimFX were easy to understand. ........................................................................ A B C D E 

12. The graphic- or photographic image-based information available to the user 

 of SimFX were useful for selecting a course of action at the decision point. .......... A B C D E 

13. The time pressure SimFX imposes on the decision maker is appropriate. ............ A B C D E 

14. The planning map provided an appropriate amount of detail. ................................ A B C D E 

15. The “clock” shown in the SimFX display to indicate how much time remains 

to complete a mission time was easy to understand. .................................................... A B C D E 

Continue on the next page

B - 3
Appendix B. SimFX Questionnaire 

Strongly disagree 

Disagree | 

Neither agree nor disagree | | 

Agree | | | 

Strongly agree | | | | 

| | | | | 

Section A: Using SimFX to train decision-making skills (continued) | | | | | 

16. The alternative courses of action provided with each decision point are 

 similar to those likely to arise in an operational environment. ................................. A B C D E 

17. SimFX can be used to train Soldiers about the proper use of robotic sensors. ...... A B C D E 

18. The feedback provided following a decision was appropriate for the purpose. 

 of training decision-making skills. ........................................................................... A B C D E

19. If the SimFX tool were available I would use it to train my Soldiers. ...................... A B C D E 

Section B. Using SimFX to modify/create mission scenarios and practice exercises

 1. If it were available, I would personally modify existing SimFX scenarios 

 or exercises. ............................................................................................................ A B C D E 

 2. If it were available, I would personally create some new SimFX scenarios 

 or exercises. ............................................................................................................ A B C D E 

 3. The procedures for modifying or creating a SimFX scenario or exercise 

 are quite logical. ...................................................................................................... A B C D E 

 4. The steps required to modify an existing SimFX scenario or exercise 

 are easy to perform. ................................................................................................ A B C D E 

 5. The steps required to create a new SimFX scenario or exercise are easy 

 to perform. ............................................................................................................... A B C D E 

 6. With just a little practice I could become very good at modifying and 

 creating SimFX scenarios or exercises. .................................................................. A B C D E 

B - 4
Appendix B. SimFX Questionnaire 

Part 4. Overall Opinion of the SimFX for training the decision-making skills 

Section A. The training capabilities of SimFX 

1. What would be the chief advantages for using SimFX as a tool for training the decision-making skills of 

small unit leaders? Describe up to three positive features of SimFX for training that would encourage you 

to use it as a training tool. 

2. What would be the chief disadvantages for using SimFX as a training tool for improving the decisionmaking skills of small unit leaders? Describe up to three negative features of SimFX for training that 

would prevent you from using it as a training tool. 

Section B. The authoring capabilities of SimFX to modify/create scenarios or exercises 

1. Describe up to two positive features of the authoring capabilities of SimFX. 

2. Describe up to two negative features of the authoring capabilities of SimFX. 

If you wish to be kept informed about the final status of this research product please print your 

phone number and e-mail address below. 

Phone number: _______________________ 

Email address: _____________________________ 

Thank you for participating in this demonstration of SimFX and for completing this questionnaire.

ARI POC: Dr. Richard E. Christ 

706-545-2207, ChristR@benning.army.mil

B - 5

Appendix C. Detailed Results from the SimFX Questionnaire 

Table C-1. Results for Part 1: Overall Potential Training Value of SimFX 

Percentage o

Statistic 

Ra

Item Number and Item

N M SD 1 2 3 

01. Would using SimFX provide a Soldier opportunity to practice making sound 

 tactical decisions? 19 5.8 0.9 ~ ~ ~ 5

02. Would training with SimFX improve a Soldier's ability to make more rapid 

tactical decisions? 19 5.6 0.9 ~ ~ ~ 1

03. Would training with SimFX make a Soldier more confident in his ability to make 

tactical decisions? 19 5.5 1.0 ~ ~ ~ 1

04. How challenging is the overall experience provided by training with SimFX? 19 4.8 1.4 ~ 10.5 5.3 105. Would training with SimFX have a valuable impact on small unit leaders? 19 5.5 0.9 ~ ~ ~ 106. Would training with SimFX help small unit leaders focus on critical factors 

that influence tactical decisions? 

19 5.8 0.9 ~ ~ ~ 1

07. To what extent could SimFX teach the Soldier something new about decision 

 making that not now or not easily covered in normal classroom or field training? 

19 5.1 1.1 ~ ~ 5.3 2

08. To what extent will training with SimFX help a Soldier to make sound tactical 

 decisions? 19 5.5 0.8 ~ ~ ~ 1

09. To what extent will training with SimFX permit a Soldier to practice the types of 

 decisions he must make as a small unit leader? 

19 5.4 0.9 ~ ~ ~ 1

10. Would training with SimFX be a valuable learning experience? 19 5.7 0.8 ~ ~ ~ 511. As a trainer, how satisfied were you with the training opportunities provided by

 SimFX? 19 5.5 0.8 ~ ~ ~ 5

C - 1 
Appendix C. Detailed Results from the SimFX Questionnaire 

Table C-2. Results for Part 2: User Involvement or Immersion in SimFX 

Percentage o

Statistic Item Number and Item Ra

N M SD 1 2 3 

01. How captivated or drawn in would a Soldier be by the tactical events and actions 

 presented in SimFX? 

19 5.1 0.9 ~ ~ 5.3 1

02. How often might a Soldier training with SimFX be completely focused on the 

 decision-making task? 

18 5.7 0.9 ~ ~ ~ 

03. How involved were you with tactical events and actions portrayed in SimFX? 19 5.6 1.2 ~ ~ 5.3 104. How involved were you in the decision-making experiences provided by SimFX? 19 5.7 0.8 ~ ~ ~ 05. Were you so involved while using SimFX that you lost track of time? 19 4.4 1.6 5.3 5.3 21.1 1

C - 2 
Appendix C. Detailed Results from the SimFX Questionnaire 

Table C-3. Results of Part 3: The Degree to Which SimFX is Easy or Difficult to Use. 

Section A: Using SimFX to Train Decision-Making Skills 

Percentag

Statistic Item Number and Item 

N M SD 1 2 

01. If it were available, I would use SimFX as a training tool. 19 4.4 0.7 ~ ~ 02. The steps required to use SimFX for training purposes are easy to perform. 18 4.2 1.0 5.6 ~ 03. The abbreviated operations order provided to the user at the beginning of a scenario were

 adequate to understand the training mission. 18 4.0 0.8 ~ 5.6 

04. The description of the factors that defined each decision point were adequate for the purpose 

 of the training exercise. 19 3.9 0.9 ~ 10.5 

05. The types of decision-making tasks possible with SimFX are similar to those encountered in

 an operational environment. 18 3.6 0.8 ~ 11.1 

06. It was easy to request and obtain information while executing the mission. 19 3.7 0.9 ~ 10.5 07. The reporting process was adequate to keep my CO informed. 19 3.8 0.6 ~ ~ 08. The signals used on the SimFX screen for alerting the user that a source of information is 

 available were easy to understand. 19 3.9 0.8 ~ 5.3 

09. The printed text-based information available to the user of SimFX was easy to understand. 19 4.1 0.5 ~ ~ 10. The printed text-based information available to the user of SimFX were useful for selecting a 

 course of action at the decision point. 19 3.9 0.7 ~ 5.3 

C - 3 
Appendix C. Detailed Results from the SimFX Questionnaire 

Table C-3. (Continued) 

Percentag

Statistic Item Number and Item 

N M SD 1 2 

11. The graphic- or photographic image-based information available to the user of SimFX were 

 easy to understand. 19 4.2 0.4 ~ ~ 

12. The graphic- or photographic image-based information available to the user of SimFX were 

 useful for selecting a course of action at the decision point. 19 4.0 0.6 ~ ~ 

13. The time pressure SimFX imposes on the decision maker is appropriate. 18 3.8 1.0 5.6 5.6 14. The planning map provided an appropriate amount of detail. 18 3.7 0.8 ~ 5.6 15. The "clock" shown in the SimFX display to indicate how much time remains to complete a 

 mission time was easy to understand. 19 4.4 0.6 ~ ~ 

16. The alternative courses of action provided with each decision point are similar to those likely 

to arise in an operational environment. 18 3.7 0.8 ~ 5.6 

17. SimFX can be used to train Soldiers about the proper use of robotic sensors. 19 3.7 0.9 ~ 10.5 18. The feedback provided following a decision was appropriate for the purpose of training

 decision-making skills. 19 3.7 0.7 ~ 5.3 

19. If the SimFX tool were available I would use it to train my Soldiers. 18 4.3 0.7 ~ ~ 

C - 4 
d Results from the SimFX Questionnaire 

C - 5 

Table C-4. Results for Part 3. The Degree to Which SimFX is Easy or Difficult to Use. 

Section B: Using SimFX to Modify/Create Mission Scenarios and Practice Exercises 

Percenta

Statistic Item Number and Item 

N M SD 1 2 

01. If it were available, I would personally modify existing SimFX scenarios or exercises. 19 4.4 0.8 ~ 5.3 02. If it were available, I would personally create some new SimFX scenarios or exercises. 18 4.6 0.6 ~ ~ 03. The procedures for modifying or creating a SimFX scenario or exercise are quite logical. 18 4.1 0.8 ~ 5.6 04. The steps required to modify an existing SimFX scenario or exercise are easy to perform. 18 3.8 0.9 ~ 5.6 05. The steps required to create a new SimFX scenario or exercise are easy to perform. 18 3.9 0.9 ~ 5.6 06. With just a little practice I could become very good at modifying and creating SimFX scenarios 

 or exercises. 18 4.2 0.7 ~ ~ 

Appendix C. Detaile
Appendix C. Detailed Results from the SimFX Questionnaire 

Table C-5. Positive Written Comments About Using SimFX for Training 

(Respondent ID) 

Simple to use (5) 

Flexible (11) 

Easy to understand and use (15) 

Easy to use (19) 

Able to make up your own scenarios (19) 

Easy to adjust scenarios (5) 

This system appears to be unlimited in its capabilities as a training tool. The only limitation 

 would be the trainer and his scenarios. (12) 

Can be developed by the trainer to suit his situation (14) 

The scenarios can easily be modified to teach a wide variety of decision making skills. (17) 

Visual aid to tactical exercise (11) 

Graphics (19) 

Expand data base of exercises in unit (11) 

Making good decisions (19) 

Allows the exercise to focus on decision making (14) 

Requires quick sound decisions (1) 

Requires quick decision with limited info (3) 

The time limits make them form decisions quickly, as well as analyze info. quickly (10) 

Allows PLs to practice making decisions in a time constrained environment (9) 

Allows Jr. leaders to practice skills involved in decision making such as map reading and 

 graphic control measures (9) 

It makes Soldiers form decisions with limited information in a safe environment (10) 

Great way for leader to learn how not to make hasty decision (think first) (18) 

Post H-hour decision making skills are hard to replicate for Co commanders. 

 This is an option. (8) 

Enables one to actually execute as opposed to simply planning an operation (7)

Teaches you to utilize assets (1) 

Build confidence not only in the Soldier but confidence on using technology (5) 

Gets reader to think about resources available (18) 

Forcing Soldiers to use available assets and inform higher will teach them to do so (10) 

Reminds you to communicate with higher (1) 

Gives real world feedback without spending the money for troops or supplies (6) 

It gives feedback to student on the spot (4) 

Soldier can use SimFX as an AAR tool to iron out SOPs (6) 

It takes away all distractions normally involved with field training exercises (6) 

The user is drawn into the scenario and is actively engaged which promotes learning. (17) 

Use to reinforce doctrine, tactics, or what ever you choose (8) 

Allows me to set up more realistic responses (3)

Doesn’t require a lot of computer space (14) 

Do not need to “train” the Soldier on the actual software before training them 

 on the content (17) 

C - 6 
Appendix C. Detailed Results from the SimFX Questionnaire 

Table C-6. Negative Written Comments About Using SimFX for Training 

(Respondent ID) 

More to fighting than robotic sensors (7) 

It may put the platoon in a failed situation because the PL did not check an asset 

 which is not realistic. (10) 

Many of the assets are not available at the platoon level and might confuse things (10) 

Automation usage (11) 

Time required to develop scenario (14) 

It takes some time for instructors to develop the scenarios (17) 

The scenarios need to be doctrinally correct in order to not confuse junior leaders or 

 teach them the wrong information (9) 

Scenarios must be carefully designed and reviewed to insure that no negative actions 

 are reinforced – especially actions that are peripheral to the main objective (15) 

It takes engaged instructors to provide additional feedback and teaching points to the 

 students (17) 

Need to develop new icons for conventional units (14) 

To design and input a Co-level activity with appropriate options, graphics, etc. would be 

 a major task (8) 

The need for computers to get everyone training on the same day (1) 

Lack of computers (3) 

Soldiers are sometimes skeptical about using simulations to train. (They would rather 

 train in the field.) (17)

Nothing fully replaces “boots on the ground” (1) 

Sometimes there is no correct answer, not everything you do on the battlefield is black 

 and white (3) 

A student is relying on the SimFX answer (4) 

Mission change/ Programmed response (11) 

From what I experienced, it doesn’t allow for different types of platoons, such as 

 mechanized, lights, or mortar, etc. (10) 

Limited storyline (11) 

As you make decisions it should show visual progress on the map (19) 

Overlays should stay up at all times (6) 

You have to start over with every fatal decision –should restart from there (6) 

Viewing in the 1st person as opposed to icons to on a map is more effective (7) 

No stress involved (2) 

No overall statement of strengths or weaknesses of decision making skills (6) 

Some trainers (SMEs) do not fully understand the nature of their own expertise (14)

C - 7 
Appendix C. Detailed Results from the SimFX Questionnaire 

Table C-7. Positive Written Comments About Using SimFX for Authoring 

(Respondent ID) 

Easy system to understand. User friendly (8) 

Simple (2) 

Looks simple (11) 

Looks flexible (11) 

Seems to be simple (12) 

Very easy to use! (17) 

Able to change the icons (19) 

Ability to use desired terrain (7) 

The ability to add images, photos, etc. (1) 

It allows the instructor to change the scenario to include assets or delete assets. This feature 

 can make SimFX more applicable to the current force capabilities (10)

Point and click (15) 

Word worksheet to develop initial scenario (14)

Branching chart to visualize learning path (14) 

A unit can create scenarios tailored to its unit mission (1) 

Very flexible – can create a wide array of scenarios (17) 

You can tailor make it to your mission (6) 

Set up how you want (3) 

Create training specific to your unit (3) 

Able to change with little effort (4) 

Work well with every changing COE (4) 

It evolves as the mission continues (6) 

This would be great for QDXs for the NCOA schools for tactics (5) 

I think the system would allow you to take current TDEs and convert them to SimFX (9) 

C - 8 
Appendix C. Detailed Results from the SimFX Questionnaire 

Table C-8. Negative Written Comments About Using SimFX for Authoring 

(Respondent ID) 

It will take a little time to become familiar with the software (17) 

To design and input a Co-level activity with appropriate options, graphics, etc. would be 

 a major task (8) [same comment made with respect to SimFX training value] 

Would tale a long time to truly depict reality (7) 

It puts more pressure on the instructor to develop new scenarios or modification 

 to the old ones (10) 

Different instructors may modify it differently, so there is not one standard in a company or 

 battalion (10) 

On the current battlefield we set battle updates as mission changes. I think some 

 scenarios that should happen because it will adjust the thinking process of the Soldiers (6) 

Depending on author’s perception can force students on what to think, not how to think (7) 

Enough features to create scenarios for the current force to use? (It was hard to know 

 this from the demo.) 

A good addition might be short audio input for voice messages 

Programmed enemy actions 

% of occurrence 

Software should keep the overlay displayed 

C - 9