Study 

Report 

2007- 01 

 

Immersive Simulation Training 

 for the Dismounted Soldier 

 Bruce W. Knerr 

 U.S. Army Research institute

 

 

 United States Army Research Institute 

 for the Behavioral and Social Sciences 

February 2007 

 Approved for public release; distribution is unlimited. 
U.S. Army Research Institute 

for the Behavioral and Social Sciences 

A Directorate of the Department of the Army 

Deputy Chief of Staff, G1 

Authorized and approved for distribution: 

 

 

SCOTT E GRAHAM MICHELLE SAMS

Acting Technical Director Acting Director 

Technical review by 

 

Timothy Sayers, Army’s Combat Developer Project Lead 

 for Dismounted Soldier Simulations 

Diana Tierney, Chief, S&T Research, Studies and Experimentation 

 Division, TPCRD, DCSOPS&T, TRADOC 

NOTICES 

DISTRIBUTION: Primary distribution of this Study Report has been made by ARI. 

Please address correspondence concerning distribution of reports to: U.S. Army 

Research Institute for the Behavioral and Social Sciences, Attn: DAPE-ARI-MS, 

2511 Jefferson Davis Highway, Arlington, Virginia 22202-3926. 

FINAL DISPOSITION: This Study Report may be destroyed when it is no longer 

needed. Please do not return it to the U.S. Army Research Institute for the Behavioral 

and Social Sciences. 

NOTE: The findings in this Study Report are not to be construed as an official 

Department of the Army position, unless so designated by other authorized documents. 
REPORT DOCUMENTATION PAGE 

1. REPORT DATE (dd-mm-yy)

 February 2007 

2. REPORT TYPE

Final 

3. DATES COVERED (from. . . to) 

October 2005 – October 2006 

5a. CONTRACT OR GRANT NUMBER 

 

4. TITLE AND SUBTITLE

Immersive Simulation Training for the Dismounted Soldier 

5b. PROGRAM ELEMENT NUMBER 

665803 

5c. PROJECT NUMBER 

D730

6. AUTHOR(S) 

Bruce W. Knerr 

 5d. TASK NUMBER 

338 

5e. WORK UNIT NUMBER 

H01 

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)

U. S. Army Research Institute for the Behavioral & Social 

Sciences 

ATTN: DAPE-ARI-IF 

12350 Research Parkway 

Orlando, FL 32826-3276 

8. PERFORMING ORGANIZATION REPORT NUMBER 

 

10. MONITOR ACRONYM

 

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 

U. S. Army Research Institute for the Behavioral & Social 

Sciences 

2511 Jefferson Davis Highway

Arlington, VA 22202-3926 

11. MONITOR REPORT NUMBER 

 Study Report 2007-01 

12. DISTRIBUTION/AVAILABILITY STATEMENT 

Approved for public release; distribution is unlimited. 

13. SUPPLEMENTARY NOTES 

 Subject Matter POC: Bruce W. Knerr

14. ABSTRACT (Maximum 200 words): 

A study was conducted to document the need for immersive dismounted virtual Soldier and leader training 

and the available research evidence regarding the effectiveness of virtual training for training Soldiers and 

leaders in complex skills. A literature search of research reports, journal articles, and conference 

proceedings to identify evaluations and experiments related to the study topic of the training effectiveness 

of immersive virtual simulations was conducted. Particular attention was paid to an expended series of 

evaluations conducted by the Army R&D organizations during the period 1997 – 2005. The major findings 

are organized around the topics of training effectiveness, Soldier task performance, and advantages and 

disadvantages of immersive virtual simulations. Soldiers and small unit leaders report that their skills 

improve as a result of training in virtual simulations, and these self-reports by have generally, if informally, 

been confirmed by observers. While the simulators impose constraints on the performance of some 

Soldier activities, this should limit training effectiveness only if those activities that cannot be performed in 

the simulator are not trained by other means. Advantages and disadvantages of immersive simulations 

are also described. 

 

15. SUBJECT TERMS 

Immersive simulation. virtual simulation, virtual environment, mission rehearsal, collective training, Infantry 

training

 SECURITY CLASSIFICATION OF 19. LIMITATION OF

ABSTRACT

20. NUMBER 

OF PAGES

21. RESPONSIBLE PERSON

16. REPORT

Unclassified 

17. ABSTRACT 

Unclassified 

18. THIS PAGE

Unclassified 

 

 Unlimited 

 

63 

Ellen Kinzer 

Technical Publications Specialist 

703-602-8047 

 Standard Form 298 

i
ii
Study Report 2007-01 

Immersive Simulation Training for the Dismounted Soldier 

Bruce W. Knerr 

U.S. Army Research Institute 

Simulator Systems Research Unit 

Stephen L. Goldberg, Chief 

U.S. Army Research Institute for the Behavioral and Social Sciences 

2511 Jefferson Davis Highway, Arlington, Virginia 22202-3926 

February 2007 

Army Project Number Personnel and Training 

665803D730 Analysis Activities 

Approved for public release; distribution is unlimited. 

iii 
iv 
IMMERSIVE SIMULATION TRAINING FOR THE DISMOUNTED SOLDIER 

EXECUTIVE SUMMARY 

Research Requirement: 

ARI was tasked to conduct a study to develop a training/benefit analysis documenting the 

need for and expected benefits of immersive dismounted virtual Soldier and leader training, and 

to prepare a report documenting the available research evidence regarding the effectiveness of 

virtual training for training Soldiers and leaders in complex skills, the cognitive and decision 

making skills that will foster adaptability and the capability to respond to rapidly changing 

situations. 

Procedure: 

ARI conducted a literature search of research reports in the Defense Technical 

Information Center (DTIC), and journal articles and conference proceedings in the open 

literature to identify evaluations and experiments related to the study topic of the training 

effectiveness of immersive virtual simulations. Particular attention was paid to an expended 

series of evaluations conducted by the U.S. Army Research Institute (ARI), Program Executive 

Office, Simulation, Training and Instrumentation (PEO STRI), the Army Research Laboratory 

(ARL), and the Research, Development, and Engineering Command Simulation and Training 

Technology Center (RDECOM STTC) during the period 1997 – 2005. This series of related 

development and evaluation activities constitutes the most extensive examination of questions of 

dismounted Soldier performance and training effectiveness in immersive simulations. 

Findings: 

It is difficult to draw definitive conclusions about the effectiveness of immersive virtual 

simulation for training dismounted Soldiers and leaders because the technology is relatively new, 

and organizations have not had the resources to conduct evaluations of sufficient scope to 

provide definitive results about training effectiveness. The evidence that is available comes from

assessments involving small numbers of Soldiers and units who received limited amounts of 

training. In these assessments, Soldiers and small unit leaders have consistently reported that 

their skills improved as a result of training in virtual simulations, with the most improvement in 

controlling, coordinating, communicating, and planning (conducted prior to use of the 

simulators), and less improvement in the mechanics of tasks and battle drills. These self-reports 

have generally been confirmed by observers. They are also consistent with evaluations of 

collective marksmanship trainers, which use different technology, but are functionally similar to 

immersive virtual simulators. There is objective performance data indicating that the 

marksmanship training provide effective training in tactical skills as well as marksmanship. 

Virtual immersive simulators necessarily do limit how some tasks can be performed, 

particularly those involving locomotion and touch. This limits training effectiveness only if 

those actions that cannot be performed in the simulator are not trained by other means. Even 

v 
though the simulator may not allow Soldiers to perform physical actions as they do in the real 

world, it does allow them to practice and learn the cognitive skills they need. These include the 

planning that needs to be done prior to missions, the skills required to maintain situational 

awareness, and learning to think on their feet and make appropriate decisions in complex 

situations. 

Virtual simulations possess a number of advantages relative to the use of live simulation 

for dismounted Soldier training. They can represent any terrain or environment that has been 

modeled, weather, and other environmental conditions. Use of immersive virtual simulation is 

likely to reduce the amount of time required by a unit to conduct a training exercise relative to 

live training. Fewer human role players should be required to fill the positions of enemy units, 

other friendly units, and civilians in virtual simulations than in live simulations. The reduced 

risk of accident or injury makes it possible to more realistically depict the effects of demolitions 

or indirect fire than is possible in live simulations. Virtual simulations require less physical 

space than a live simulation facility, and reduce the need for use of actual equipment. 

Virtual After Action Reviews (AARs) provide unique capabilities that are not currently 

possible with live simulations. They can provide an overall view of the action that is not 

possible with video cameras, which provide multiple separate views of portions of the action. 

They can provide a view of the action from any perspective, not limited by camera locations. 

They capture and provide an accurate representation of ground truth. Finally, they can provide a 

variety of aids to analyze and depict events. 

Virtual simulations possess a number of disadvantages relative to the use of live 

simulation for dismounted Soldier training. Some physical activities cannot be practiced. 

Weapons effects may not be modeled correctly, or position and location tracking may not be 

accurate enough to depict realistic weapons effects. Soldiers may, therefore, develop 

inappropriate expectations. Virtual simulations require improved facilities (enclosed space, air 

conditioning, and power) and technical support to implement training scenarios and operate and 

maintain computers and other equipment. Trainees may not take virtual simulations seriously 

because they are similar to video games, and might, therefore, practice actions that are 

inappropriate in the real world, or attempt to “game” the system. Some trainees may experience 

“simulator sickness.” 

Virtual immersive simulations can provide the opportunity for small unit leaders and unit 

members to practice, in the context of realistic simulated operations, the cognitive and decision 

making skills that will foster adaptability and the capability to respond to rapidly changing 

situations. Focused, repetitive, deliberate practice, with feedback based on performance, is an 

effective method for training the recognition of situations and developing expertise. Cognitive 

and decision making skills can be trained even if some physical tasks cannot be performed in the 

situation. 

Utilization and Dissemination of Findings 

It is recommended that the following actions be taken as the next steps toward acquisition 

and fielding of an immersive virtual training system for dismounted Soldiers. 

vi 
Conduct a large-scale evaluation using current technology. This evaluation should consider 

objective measurement of both skill improvement on the simulator and transfer of those skills to 

a live simulation environment. A sufficient number of units should participate so that any 

meaningful real differences in the training effectiveness of different can be detected. The 

evaluation should be embedded in the unit’s normal training progression, and unit personnel 

should be involved in its development and delivery.

Consider cost-effectiveness of fully immersive and desktop or laptop systems. There is a large 

cost differential between fully-immersive simulators and simulators using desktop or laptop 

computers to provide the same functionality. In contrast, the research evidence indicates that any 

difference in training effectiveness between immersive and desktop systems is likely to be small, 

although immersive systems have a small advantage when training spatially-oriented tasks. 

Whether they are better when training squads to conduct urban or counter-insurgency operations, 

and if so, whether the difference is large enough to justify the increased cost, are unknown, but 

should be investigated. 

Conduct a survey of potential officer and enlisted trainees to determine game-playing skill. An 

assumption underlying the growing popularity of the use of video and PC-based games for 

training is that the potential trainees are interested in and experienced with such games. They, 

therefore, need little training in their use, and are motivated to play. However, data from small 

samples of Soldiers who participated in evaluations suggests that this may not be the case. Since 

game playing experience has implications for the types of games that could be used for training 

and the amount and type of prerequisite game playing training required, we would like to have a 

much better picture of the both current and projected Soldier and leader familiarity and 

experience with different types of interactive games. 

Continue to improve virtual simulation capabilities and assess their training effectiveness. Some

aspects of immersive virtual simulation technology, such as large-screen displays, will continue 

to improve, or become less expensive, as a result of commercial market forces. However, it will 

require a continuing science and technology investment by the Army to produce a tailored 

system that provides cost-effective training. 

vii 
viii 
IMMERSIVE SIMULATION TRAINING FOR THE DISMOUNTED SOLDIER 

CONTENTS 

 

Page 

INTRODUCTION ...........................................................................................................................1 

 Study Objectives..................................................................................................................1 

IMMERSIVE SIMULATIONS FOR DISMOUNTED SOLDIERS: THE ARMY TRAINING 

NEED...............................................................................................................................................1 

Capabilities to be provided by Immersive Simulation.........................................................2 

What is Virtual Simulation?.................................................................................................4 

Background on Simulators...................................................................................................7 

Results of research on the use of simulators for military training .......................................8 

RESULTS OF RESEARCH ON THE USE OF IMMERSIVE SIMULATION FOR TRAINING 

DISMOUNTED LEADERS, SOLDIERS, AND UNITS................................................................9 

 Introduction ......................................................................................................................9 

 Methodological Considerations...........................................................................................9 

 Analyses ....................................................................................................................10 

 Experimental Efforts..........................................................................................................13 

 Marksmanship Trainers .........................................................................................13 

The Dismounted Warrior Network Experiments...................................................14 

 The Virtual Environment Science and Technology Objective ..............................18 

 The Virtual-Integrated MOUT Training System (V-IMTS) Assessment..............22 

Situational Awareness in Virtual Environments (SAVE)......................................23 

 Results ....................................................................................................................23 

Other Experiments and Evaluations.......................................................................28 

Discussion of Immersive Simulation Results....................................................................29 

 Limits of Experimentation .....................................................................................29 

Research on Simulator Fidelity..............................................................................30 

THE USE OF COMPUTER GAMES FOR TRAINING ..............................................................33 

Issues in Game-based Training..........................................................................................35 

Data on game playing ............................................................................................36 

SIMULATOR SICKNESS ............................................................................................................38 

DISCUSSION................................................................................................................................41 

SUMMARY OF FINDINGS.........................................................................................................42 

RECOMMENDATIONS...............................................................................................................47

ix 
CONTENTS (continued)

REFERENCES ..............................................................................................................................49 

APPENDIX A: LIST OF ACRONYMS .................................................................................... A-1 

LIST OF TABLES 

TABLE 1. TRAINING NEEDS STATEMENTS ...........................................................................3 

TABLE 2. 1994 SME RATINGS OF TTES FUNCTIONS..........................................................12 

TABLE 3. 1995 SME RATINGS OF TTES FUNCTIONS..........................................................13 

TABLE 4. COMBINED RATINGS OF SIMULATOR CAPABILITY (2002 – 2005)...............24 

TABLE 5. SUMMARY OF SIMULATOR CAPABILITY RATINGS BY ASSESSMENT ......25 

TABLE 6. TRAINING EFFECTIVENESS QUESTIONNAIRE RESULTS...............................26 

TABLE 7. POTENTIAL FACTORS ASSOCIATED WITH SIMULATOR SICKNESS IN 

 VIRTUAL ENVIRONMENTS ..................................................................................39 

LIST OF FIGURES

FIGURE 1. A GRAPHIC DEPICTION OF THE TTES ...............................................................12 

FIGURE 2. A SOLDIER IN VIC APLHA....................................................................................15 

FIGURE 3. A SOLDIER IN VIC BRAVO ...................................................................................16 

FIGURE 4. 2001 CE SYSTEM CONFIGURATION, SOLDIER BATTLE LAB, FORT .............. 

 BENNING..................................................................................................................21 

FIGURE 5. V-IMTS PHYSICAL CONFIGURATION................................................................23 

FIGURE 6. V-IMTS TRAINING EFFECTIVENESS RATINGS FOR LEADERS AND 

 SOLDIERS ................................................................................................................27 

x 
IMMERSIVE SIMULATION TRAINING FOR THE DISMOUNTED SOLDIER 

Introduction 

TRADOC DCSOPS&T and TPIO Virtual requested that ARI conduct a study to develop 

a training/benefit analysis documenting the need for and expected benefits of immersive 

dismounted virtual Soldier and leader training. This report documents the available research 

evidence regarding the effectiveness of virtual training for training Soldiers and leaders in 

complex skills. It is based on an extensive review of research findings from all Services and 

industry that impact upon the use of immersive simulation for the dismounted Soldier. 

Study Objectives 

The objectives of this study are: (a) to review and synthesize the accumulated evidence, 

from the research literature and reports of field experiments, that is related to the effectiveness of

virtual simulation for dismounted Soldier training; (b) to use this information as a basis for 

conclusions about the expected effectiveness of virtual training approaches; and (c) to present 

this information in a straightforward, non-technical format. 

To accomplish those objectives, this report is organized around fiver major questions. 

• What is the Army’s training need which is to be met through the application of virtual 

simulation and gaming technology?

• What virtual simulation and gaming technology exists today to meet this need?

• What evidence is there that virtual simulation and gaming technology can meet those 

training needs?

• What recommendations for future actions can be made based on this evidence?

• What are the unresolved issues and further research/study needs?

Immersive Simulations for Dismounted Soldiers: The Army Training Need 

This report will not provide a detailed or exhaustive task-level analysis of the specific 

skills that immersive virtual simulation is expected to train. However, the Army Strategic 

Planning Guidance 2005 (Department of the Army, 2005) and the Soldier CATT Operational 

Requirements Document (ORD; Department of the Army, 2004) provide a broad picture of 

requirements. Three major needs emerge. First, Soldiers and small unit leaders need to be 

adaptable and capable of responding to and taking advantage of rapidly changing situations. 

Second, they must be attuned to cultural conditions and knowledgeable in the culture, history, 

and language of the area of operations. Third, they must be able to rapidly and accurately assess 

the evolving situation using all of the tools at their disposal. Officer and junior noncommissioned officers must also be able to communicate effectively with a variety of different 

governmental and non-governmental organizations, to include media representatives. Table 1 

provides more detail about these needs. 

1 
Capabilities to be provided by Immersive Simulation 

The Soldier CATT ORD also describes the capabilities to be provided by immersive 

simulation. The following excerpts provide a high level summary of the capabilities. 

[The] centerpiece is the “Virtual Warrior,” a high fidelity dismounted leader 

trainer providing a natural virtual environment in which dismounted leaders see 

the battlefield in three dimensions, control subordinate virtual Soldiers through 

voice recognition/voice synthesis, and communicate via FM voice 

communications. (p. 6) 

Soldier CATT is a Soldier and leader collective combined arms virtual training 

simulation focused on dismounted infantry centric training. It will provide a 

sufficient quantity of Virtual Warrior simulators to support from individual 

Soldier training at the squad level up to platoon and infantry company leader 

training. Soldier CATT is designed to train ... the full-spectrum of operations in 

a combined arms and joint environment. Leaders will train such tasks as the 

command and control of units, fire coordination, employment of joint enablers, 

engineer support (obstacle integration) and logistical support. Soldier CATT will 

support full spectrum training (e.g., infantry tasks of raid, dismounted assault and 

area presence; Reconnaissance, Surveillance, and Target Acquisition (RSTA) 

tasks of multidimensional area reconnaissance, multidimensional route 

reconnaissance and screen). Additionally, Soldier CATT will support training for 

urban operations (UO) and operations on complex terrain. (p. 7) 

Soldier CATT will provide ... a home station and deployed unit training

capability ideally suited for walk-level and sustainment training as well as

mission rehearsal training. (p. 7) 

2 
Table 1 

Training Need Statements 

Recent experience in Iraq and Afghanistan has shown the need for Soldiers who are not only well trained and 

equipped, but who are also adaptable and capable of responding to rapidly changing situations and are attuned to

cultural conditions. In addition to rigorous, repetitive training in weapons and fieldcraft, Soldiers also require 

training in how to adapt to quickly evolving situations, not just how to react to changes, but also how to shape the 

environment to create the best possible outcomes. (Army Strategic Planning Guidance 2005, p. 4)

Iraq in particular has proven to be a non-linear battlefield. Given the security environment, we can expect this to

become the norm. This requires all Soldiers understand they are warriors first and that they must be ready to fight. 

The distinctions between combatant and noncombatant have blurred, as have the distinctions between combat 

operations and stability operations. Simultaneous operations across the range of military operations, rather than

sequential operations will likely be the rule. To succeed in such an environment, Soldiers and leaders must be 

capable of rapidly and accurately assessing the evolving situation. In order to make accurate assessments, they must 

also be capable of using all the tools at their disposal. This requires Soldiers and leaders who understand and are

capable of leveraging the latest technology and Joint Capabilities. It also requires Soldiers and leaders who are 

knowledgeable of culture, history and the language of the area of operations. (Army Strategic Planning Guidance

2005, p. 5)

Today’s environment requires lieutenants and junior noncommissioned officers to communicate regularly with a 

variety of actors; from interagency partners, to members of non-governmental organizations, to local leaders, to

members of the U.S. and foreign media. To succeed, leaders at all levels must have situational understanding that 

extends beyond the tactical level. This requires a robust leader development system that grows leaders who are 

prepared, versatile and adaptive and who have a broad understanding of the political and military objectives of the 

campaign, as well as the potential implications of their actions upon those objectives. (Army Strategic Planning 

Guidance 2005, pp. 5-6)

Recent operations, such as operations in Afghanistan and Iraq, identified an immediate and critical need for a virtual 

training simulation to build and sustain the training readiness of dismounted infantry-centric units ... required to

conduct close combat operations in complex/restrictive terrain against asymmetric forces, integrating multiple 

enablers and platforms and the associated fires and effects of fires in the joint and/or multinational operational 

battlespace. (Soldier CATT ORD (revised 8 June 2004), p. 1). 

Virtual training is required to prepare and build confident and adaptive dismounted infantry leaders and Soldiers, 

that can rapidly dominate battlefield situations, and dilemmas in varying conditions (combined arms, joint 

interagency multi-national, contemporary operational environment, urban and complex terrain), through small unit 

actions, while dispersed, deployed, or distributed, and remain self aware while digitally connected or networked 

between shooters and sensors, and dismounted and mounted forces. Since every Soldier must be prepared to be a

leader, virtual training will improve the dismounted Soldier’s ability to remotely control weapon and sensor systems 

to include robotic systems and unmanned platforms to enhance his ability to see, understand, and act first. (Soldier 

CATT ORD (revised 8 June 2004), p. 1)

The dismounted Soldier and leader require training in a multiplicity of environments and terrain (e.g. urban, desert, 

mountainous, jungle) that generally cannot be replicated at homestation training areas or the Combat Training

Centers. (Soldier CATT ORD (revised 8 June 2004), p. 2)

It is a critical requirement to prepare junior leaders to successfully lead multifunctional units, effectively use 

digitized systems, integrate into combined arms and joint organization and conduct decentralized and dispersed 

operations across the full-spectrum of possible missions. To develop and sustain the tactical and technical

proficiency required of current and future Soldiers and leaders, units need home station and deployable training tools 

that enable frequent and repetitious complex task training. (Soldier CATT ORD (revised 8 June 2004), p. 2)

3 
The Soldier CATT ORD provides a more detailed description of the training 

requirements that a virtual simulation system should satisfy: 

• General Requirements: 

o Support training for dismounted infantry from squad to company level. 

o Allow the expansion of the training audience from individual Soldier up to battalion 

level leader. 

o Support the mounted training requirements of the Stryker Brigade Combat Team. 

o Support initial leader training at institutions (USAIS, USJFKSWCS, and the 

USAARMC). 

o Support unit sustainment training. 

o Provide structured, multi-echelon, collective task training to active and reserve 

component dismounted infantry units. 

o Support Soldier and leader training during peacetime, mobilization and deployment. 

o Support conduct of command and control and tactical training. 

o Support external evaluations, mission training and combined arms exercises. 

o Complement live training. 

o Allow dismounted leaders, mounted leaders and crews, and units to operate in 

combined L-V-C exercises. 

• Institutional Training Requirements:

o Enhance leader skill training in subjects including decision making, planning for 

operations, command and control of small units, direct fire coordination, fire support 

coordination, integration of obstacles and logistical support. 

o Incorporated into programs of instruction for infantry, SOF and scout officer and 

enlisted leader courses. 

• Unit Training Requirements: 

o Support TOE unit training requirements for active and reserve component training at 

home station and in deployed locations worldwide. 

o Teach and sustain proficiency in individual and collective leader tasks. 

o Live force-on-force exercises required to validate proficiency. 

What is Virtual Simulation? 

Virtual simulation is defined in TRADOC Pamphlet 350-37 as “a simulation involving 

real people operating simulated systems. Virtual simulations inject humans-in-the-loop in a 

central role, by exercising motor skills, decision skills, or communication skills.” (TRADOC 

Pamphlet 350-37, 9 June 2003) 

The Columbia Electronic Encyclopedia provides a more detailed and more descriptive 

definition of largely equivalent and interchangeable terms virtual reality (VR) or virtual 

environment (VE). “Computer-generated environment with and within which people can 

interact. The advantage of VR is that it can immerse people in an environment that would 

normally be unavailable due to cost, safety, or perception restrictions. A successful VR 

4 
environment offers users immersion, navigation, and manipulation. VR encompasses a range of 

interactive computer environments, from text-oriented on-line forums and multiplayer games to 

complex simulations that combine audio; video, animation, or three-dimensional graphics; and 

scent. Some of the more realistic effects are achieved using a helmet like apparatus with tiny 

computer screens, one in front of each eye and each giving a slightly different view so as to 

mimic stereoscopic vision. Sensors attached to the participant (e.g., gloves, bodysuit, footwear) 

pass on his or her movements to the computer, which changes the graphics accordingly to give 

the participant the feeling of movement through the scene. Computer-generated physical 

feedback adds a “feel” to the visual illusion, and computer-controlled sounds and odors reinforce 

the virtual environment. Other VR systems, such as flight simulators, use larger displays and 

enclosed environments to create an illusion. Less-complicated systems for personal computers 

manipulate an image of three-dimensional space on a computer screen. In a virtual network 

many users can be immersed in the same simulation, each perceiving it from a personal point of 

view. VR is used in some electronic games, in amusement-park attractions, in military exercises, 

and to simulate construction designs. Experimental and envisioned uses include education, 

industrial design, surgical training, and art.” (virtual reality, 2003) 

For purposes of this report, immersive simulations are defined to include a broad range of 

virtual simulations. At one extreme, it will include the use of videogame platforms (e.g., Full 

Spectrum Warrior). At the other extreme, it will include networked fully immersive simulators 

with full-screen or head-mounted displays (e.g., the Virtual Integrated MOUT Training System

(V-IMTS). Between these two extremes, a variety of PC-based simulations are possible and will 

be considered. 

Campbell, Knerr, and Lampton (2004) have provided a list of necessary functions that 

must be performed in order to use virtual simulations for dismounted Soldier training. 

• Realistic representation of terrain in different conditions (e.g., night, fog) – How will the 

Soldier see his environment? 

• Locomotion devices that provide realistic perception of movement and some degree of 

accurate energy expenditure – How can he move or change position? How can he cross a 

field or climb stairs? 

• Ability to select weapons and other systems and use them with realistic effects – How will 

the Soldier be able to use his M4 rifle, throw smoke grenades or concussion grenades, and 

use binoculars or night vision devices?

• “Intelligent” computer-controlled forces to represent enemy, friendly, and neutral forces – 

How can the simulated people react to the Soldier’s actions? Will they fall when shot, and 

walk around rather than walk into Soldiers and other obstacles? 

• Dynamic terrain showing damage to structures, rubble and other micro-terrain obstacles – 

How can scenery features (such as trees, cars, and street lights) also “behave” like their real 

world counterparts? Will buildings and other structures show effects of the Soldier’s 

actions?

5 
• Accurate portrayal of Soldier actions and movements in the virtual environment –How do the 

Soldiers see each other or interact? 

• Voice and gesture recognition by simulated “Soldiers” – How can leader trainees control 

computer-generated Soldiers with voice and hand-arm signals?

• Performance recording, analysis, and tailorable feedback at an AAR station – How can a 

trainer know what’s going on in the simulation, so there can be appropriate feedback and 

performance improvement?

While there are multiple ways the essential components and capabilities required for 

dismounted Soldier simulation could be configured or described, the following categorization 

covers the key elements. 

• Computer environment generator/processor. This component displays the simulated 

environment and processes calculations to determine environments and weapons effects. It is 

usually a PC, although it may also be a game console. 

• Network. This transmits information among the participating simulators and between the 

simulators and the management system. Network characteristics depend on the simulation. 

High-end military systems typically use a Distributed Interactive Simulation (DIS) or High 

Level Architecture (HLA) network. Systems based on commercial game engines use 

protocols developed for use over the internet. 

• Trainee Control Suite. This consists of the components of the individual simulator by which 

the trainee affects the simulation. The major components are: 

o Weapon Controller – A device for controlling the trainee’s weapon (aim point, 

ammunition selection, firing rate, etc.). It could be as sophisticated as an 

instrumented, demilitarized weapon, or as simple as a game controller. 

o Locomotion Controller – A device for controlling the trainee’s posture, 

orientation, and movement. The range of sophistication is quite varied. It could be 

combined with the Weapon Controller. 

o Voice Input device – A microphone is used to capture the voice signal, but there 

are many ways to process and transmit it. 

o Other Devices – To select or control additional equipment such as digital systems, 

night vision goggles, etc. 

• Trainee Display Suite. This consists of the components of the individual simulator which 

present information to the trainee. The major components are: 

o Visual Display – There are three approaches: 

 Large-screen displays. These have typically been rear-screen projection 

displays, although some systems have used front-projection displays. They 

may be either single-screen or multi-screen. Large Light Emitting Diode 

(LED) and plasma displays may become a reasonable alternative. 

 Head-mounted displays (HMD). 

 Standard-size PC monitor. 

6 
o Auditory Display – Speakers or headphones. They deliver environmental sounds 

and voice communications. 

o Haptic Display – Uses pressure or vibration to provide cues such as physical 

contact with environmental objects or people. 

o Other Devices – To receive input from additional equipment such as digital 

systems, night vision goggles, etc. 

• Management System. This includes a variety of capabilities used to set up training scenarios, 

track unit and individual progress, control computer-generated entities (people and objects), 

and provide feedback (AAR) to the trainees. 

Background on Simulators 

It is beyond the scope of this report to provide a detailed review of the use and 

effectiveness of simulators for military training. However, the long history of use of simulators 

has established both “conventional wisdom” and empirical data about the effectiveness of 

simulators. Orlansky, Dahlman, Hammon, Metzko, Taylor, & Youngblut (1994) provided a list 

of advantages and disadvantages of the use of simulators. These are cited as “conventional 

wisdom,” which may not always be correct when applied to a particular situation. 

Advantages 

• Trains many tasks as well as the use of the actual equipment. 

• Costs less than actual equipment to procure and use. 

• Provides training that otherwise requires use of actual equipment. 

• Permits training for dangerous and potentially catastrophic conditions not otherwise 

possible in actual equipment. 

• Reduces risks, safety hazards, and wear and tear from use and maintenance of actual 

equipment, and extends useful life of equipment needed for combat. 

• Reduces impact that actual equipment has on environment, and the noise and use of 

ranges have on communities. 

• Protects operational security of tactics and mission rehearsal, of electronic warfare, and 

sensitive performance characteristics of new weapons. 

• Provides features needed for instruction and feedback, including performance measures. 

• Saves on use of fuel, munitions, and support facilities. 

• Provides initial familiarization, trains procedures (less so for advanced students), and

avoids risk to actual equipment by novices. 

• Permits training not readily affordable with actual equipment. 

• Permits use of actual or anticipated threat conditions to examine doctrine, effect of the 

combat environment, and potential responses. 

• Permits testing of effectiveness of plans and tactics prior to actual operations. 

• Permits joint training of DoD forces and of coalition forces. 

• Can recreate significant battles to identify key events and to train leaders.

7 
Disadvantages 

• Could adversely affect training readiness if use of simulators reduces Operational Tempo 

(OPTEMPO), as well as confidence in use of actual equipment. 

• If uninformed by data from actual performances of weapons and service crews, virtual 

and constructive simulations can be dubiously valid. 

• Require funds to procure and use. 

• May instill habits that are incorrect for use in actual equipment. 

• Limited fidelity of visual resolution, sensors, aerodynamics, and motion platforms can 

lead to inadequate or misleading training. 

• Differing response characteristics of visual displays and of motion platforms often leads 

to motion sickness. 

• There is a potential compromise of security if information about sensitive tactics and 

equipment can be gained by access to networks that support interactive simulators. 

• Accurate performance of weapons and platforms in simulations requires accurate 

engineering models and data that may not be available or are expensive to develop. 

Results of research on the use of simulators for military training 

Orlansky et al (1994), in a review of research on the use of simulators for individual 

flight and maintenance training in the military, report that a combination of real equipment and 

simulators were always as effective as use of the actual equipment alone, and cost less. For 

example, an hour of aircraft simulator training was equivalent to .48 hours of actual aircraft time, 

while operating costs of the simulator averaged about 10% of those of the actual aircraft, and 

total costs about 20% of actual aircraft. 

A review by Hayes et al (1992) (based on 26 of 247 documents) concluded that the use of 

both a simulator and the actual aircraft for jet aircraft training consistently resulted in better 

performance in the aircraft than did the use of the aircraft for training alone. Caretta and Dunlap 

(1998) reviewed 67 articles, papers and reports regarding the use of simulators for flying training 

written after the Hayes et al review. Of these, 13 actually reported measures of transfer from the 

simulator to the aircraft. All showed that at least some of the training received in the simulator 

transferred to the actual aircraft. 

This research summarized above is largely concerned with the use of simulators for 

individual training in institutions. However, as Orlansky et al point out, this differs from one of

the situations of interest to us, collective training in units. First, the objective of individual 

training in institutions is to achieve standards of proficiency at the lowest possible cost, while the 

objective of collective training in units should be to get the best possible training at little or no 

increase in cost. Second, there is very little data to indicate how effective simulators and 

simulations are for unit and collective training. Several reviewers (Boldovici, Bessemer, and 

Bolton, 2002; Simpson, 1999) discuss the fact that it is difficult and highly resource intensive to 

do such evaluations. 

8 
Results of Research on the Use of Immersive Simulation for Training Dismounted Leaders, 

Soldiers, and Units 

Introduction 

This section will provide a review of research on the use of immersive simulations to 

train Army personnel for tactical training. It will include individual Soldier, leader, and 

collective tasks. The criterion for including studies and experiments has been fairly liberal with 

regard to the definition of “immersive” simulation. Two marksmanship trainers, one of which 

used videodisc-based technology, are included. On the other hand, technology descriptions or 

demonstrations which obtained no structured feedback from trainees or subject matter experts 

(SMEs) were not included. 

The Army began to be interested in the use of immersive simulation for dismounted 

Soldier training in the early 1990’s. This was largely a result of the influence of Gorman (1990), 

who was an early proponent of the use of VE for dismounted infantry (DI) training. Partly as a 

result of his efforts, a conference was held in Snowbird, Utah in 1990 to discuss individual 

Soldier systems and the role that an individual immersive simulator, which he termed an 

individual portal (or I-Port), would play in their development (Goldberg and Knerr, 1997). The 

conference did create an awareness of the difference in simulation capabilities between those 

available to support training of Soldiers fighting from vehicles, and those fighting dismounted. 

Although consensus was achieved on the need for an I-Port, Operation Desert Storm preempted 

the initiation of a large-scale cooperative research and development program to develop such a 

simulation capability. The meeting did, however, provide the impetus for organizations to initiate 

Research and Development (R&D) programs in the area. 

Methodological Considerations 

The evaluation activities which followed can roughly be divided into three categories: 

analyses, laboratory experiments, and field experiments. 

 In analyses, SMEs use information about the capabilities and characteristics of the 

simulation or simulator and the tasks to be trained to assign ratings to each task or task 

component (activity) which indicate how well they believe it can be performed or trained in the 

simulation. Rules can be used to aggregate these ratings from task components to higher level 

tasks. The quality of the results from an analytic approach is highly dependent on the familiarity 

of the rater with both the simulation and the tasks to be trained. It also may require making some 

sophisticated distinctions between being able to perform a task in a simulation and learning in a 

simulation to perform a task in the real world. For example, in one currently available simulator, 

a Soldier can throw a simulated grenade by selecting the appropriate type of grenade from a 

menu, choosing direction and launch angle by pointing his mock weapon, choosing the amount 

of force desired by adjusting a slider with a pair of buttons on the weapon, and pulling the 

trigger. While this bears little resemblance to the physical act of throwing a real grenade, it does 

provide practice in deciding which type of grenade to use, and where and when to employ it. 

9 
In laboratory experiments, trainees (who may or may not be Soldiers) perform or learn to 

perform tasks under highly controlled conditions. The tasks may be actual Soldier tasks or 

synthetic Soldier tasks (designed to have the key elements of Soldier tasks but modified so that 

they can be performed or learned by non-Soldiers). Laboratory experiments compare 

performance or learning under different conditions. Trainee performance is measured 

objectively. 

In field experiments, Soldiers perform or learn to perform Soldier tasks under controlled 

conditions (although the conditions may be less tightly controlled than under laboratory 

experiments because of operational constraints). There may or may not be multiple groups for 

comparison. Trainee performance may or may not be measured objectively. 

Some experiments provide more convincing evidence of effectiveness than others. There 

are two important aspects to the design of an experiment: the training methods being compared, 

and the measures of training effectiveness or performance that are used. In terms of the 

comparison of training methods being made, the best evidence of training effectiveness is 

provided by those experiments which compare simulation-based training with an alternative 

training approach and/or a control group which receives no training. Experiments that compare 

only pre-training and post-training performance of Soldiers trained in a simulation provide less 

convincing evidence. In addition, some criteria used to evaluate training effectiveness are better 

than others. The best criterion is performance on similar tasks in a live simulation as measured 

by objective performance data or SME ratings. The next best would be objective measures of 

performance or SME ratings on the tasks as performed in the simulation, again as measured by 

objective performance data or SME ratings. Self-reports by the trainees of what and how much 

they learned are less informative than objective measures. Trainees may not be able to assess 

how much they learned accurately, and their ratings may also be affected by other factors, such 

as the novelty of the training situation or how much they enjoyed the training. These self reports 

can vary from very general and global to highly detailed and specific. The more specific they are 

the better. 

Analyses 

The first attempts to evaluate the training potential of immersive simulations for Army 

training had to be analytical, since they preceded the development of prototypes designed for 

Army training. They were instead based on knowledge of limited off-the-shelf commercial 

systems, systems under development in university laboratories, and projections of the 

capabilities of future technology. Levison and Pew (1993) were the first to conduct an analysis of 

simulation requirements for dismounted Soldier training. Their report is of interest now only 

largely for historical reasons, since it was conducted at a fairly gross level of detail, and VE 

technology was primitive at the time, however, it is interesting to note that they saw benefit in 

the use of then current simulation technology: 

(Current) technology is considered adequate for most CPT (Combat Proficiency 

Training) and MPR (Mission Planning and Rehearsal) training requirements. 

More sophisticated visual display technology will be required to train in situations 

that provide rapidly-unfolding activities in close proximity to the Soldier 

10 
(operations in urban and other close-in environments), where the resulting large 

and rapid head movements place a higher demand on the VE technology needed 

to supply the visual information adequate for training. An advanced haptic display 

interface is required for tasks that rely on the sense of touch (such as assessing the 

surface condition for supporting heavy equipment), or for situations where it is 

deemed important to have realistic tactile feedback from control and manipulation 

of weapons, equipment, and other physical objects. (p.79) 

Levison and Pew (1993) also made a key distinction between “performance fidelity” and 

“training fidelity” that is still relevant. Full performance fidelity is achieved when a Soldier 

performs a task with the same result in the simulator as in the real world, according to any 

objective measure of performance. Full training fidelity is achieved when training in the 

simulator allows a Soldier to perform a task in the real world as well or better than if the training 

had been conducted in the real world. Performance fidelity, they said, would be required for 

mission planning and rehearsal, and training fidelity would be required for training. 

Jacobs et al (1994) followed up on this work by conducting an analysis of four Infantry 

and Special Forces Army Training and Evaluation Plan Mission Training Plans (ARTEP MTPs) 

and drills (MTP for the Infantry Rifle Platoon and Squad, Battle Drills for the Infantry Rifle 

Platoon and Squad, MTP for the Special Forces Company: Special Reconnaissance, and MTP 

for the Special Forces Company: Direct Action). From these, they selected 67 unique tasks 

which they further broke down into 252 individual Soldier activities. They then determined the 

simulation capabilities required in order for each of those activities to be performed in a VE, and 

compared this with the then current and projected state of VE technology. They concluded that 

none of the missions or tasks analyzed could be fully supported by VE technology at that time, 

but that the great majority could be partially supported. The list of 252 individual Soldier 

activities has provided a basis for subsequent evaluations of what Soldiers could do in virtual 

simulations. 

In retrospect, it is clear that VE technology has not progressed as rapidly as projected in 

these early analytical efforts. Capabilities in the areas of speech recognition, position and 

orientation tracking, computer recognition of human gestures, and low-cost high-resolution wide

field-of-view HMDs has not become available as rapidly as forecast. However, the emergence 

of PCs with the capacity to drive the simulations and generate the graphics required was not 

projected. 

Perhaps surprisingly, the Navy was the first service to produce a prototype of a virtual 

individual combatant simulator. The Team Tactical Engagement Simulator (TTES), a graphic of 

which is shown in Figure 1, consisted of a 8’ X 10’ rear projection display, demilitarized rifle 

and tracker, head tracker, computer graphics generator, and system software. Trainees moved 

through the virtual world through the use of a foot pedal; pressure on the front of the pedal 

moved the trainee forward (in the direction of gaze), while pressure on the back of the pedal 

moved the trainee in the opposite direction. TTES was expected to serve multiple applications: 

individual marksmanship trainer, individual and unit tactical trainer, and decision making trainer. 

TTES was first evaluated in 1994 (Lind and Adams, 1994) by 21 military and civilian SMEs 

11 
who rated the importance, current capability, and potential capability of TTES for each of those 

three applications. Each application was defined by multiple events or activities which were 

Figure 1. A graphic depiction of the TTES (From Lind and Adams, 1994). 

rated separately. Prior to providing their ratings, SMEs received a briefing describing TTES and 

projected future enhancements, used the simulator for approximately 25 minutes, and had the 

opportunity to observe others using it. Major results are summarized in Table 2. Ratings were 

aggregated and converted so that the numbers shown in the table are expressed as a percentage of 

the highest possible rating. Lind and Adams recommended as a result that emphasis be placed on 

developing the functions that had the perceived highest importance and potential capability 

ratings, Marksmanship and Discretionary Decisions. However, it is also clear that the SMEs 

considered the current version of TTES to be well short of its potential. 

Table 2. 

1994 SME Ratings of TTES Functions 

Function Importance Potential

Capability 

Current 

Capability 

(Tactical) Training Situations 76.4 76.4 45.0 

Marksmanship 85.0 80.0 46.3

Discretionary Decisions 83.5 85.5 41.3

Note: The numbers in the cells represent the percentage of the maximum possible rating assigned 

to the events in that category. 

12 
Lind (1995) conducted a similar evaluation one year later in which 28 SMEs participated. 

Procedures were generally the same, although not all of the SMEs participated in the TTES 

exercises (some merely observed the participation of others), and those who participated did so 

as two-man teams using two networked TTES, rather than as individuals. The questionnaire was 

changed slightly. The list of functions was expanded to include Mission Preview and Mission 

Rehearsal, and “Importance” and “Potential Capability” were replaced by “Potential 

Usefulness.” The results are shown in Table 3. Based on the high Potential Usefulness ratings 

(93.9% and higher), Lind concluded that simulators such as TTES could produce “phenomenal 

savings in time, ammunition, and other costs related to training, while providing the capability to 

train to higher standards and to challenge trainees in ways not currently possible” (p.3). It is 

perhaps more significant to note that the Current Capability ratings for the (Tactical) Training 

Situations, Marksmanship, and Discretionary Decisions functions increased substantially from

1994 to 1995, presumably a result of improvements in the TTES. 

Table 3. 

1995 SME Ratings of TTES Functions 

Function Potential Usefulness Current Capability 

(Tactical) Training Situations 96.4 69.5 

Marksmanship 95.0 71.1

Discretionary Decisions 95.2 70.0 

Mission Preview (walkthrough) 94.8 72.6 

Mission Rehearsal 93.9 73.0 

Note: The numbers in the cells represent the percentage of the maximum possible rating 

assigned to the events in that category. 

These two TTES evaluations probably best serve to document the fact that SMEs 

perceived TTES to have the potential to provide effective training in a variety of areas. The 

TTES implementation, however, was perceived to be short of that potential. Most importantly, 

no actual training was conducted or evaluated. 

 

Experimental Efforts 

Marksmanship trainers 

Eisley, Hagman, Ashworth, and Viner (1990) conducted an evaluation of the Squad 

Engagement Training System (SETS) as both an individual marksmanship trainer and a squad 

tactical trainer. While SETS does not use computer-generated graphics, it could be considered to 

be immersive. It uses computer-controlled, videodisc-based visual imagery projected on a wide 

screen, and laser-instrumented individual weapons to train both individual marksmanship and to 

a limited extent, squad tactics. Conceptually, it seems very similar to the current Engagement 

Skills Trainer (EST 2000). This evaluation provides sound evidence that simulators can provide 

effective tactical training for small dismounted units. 

Thirty-six squads of National Guard Soldiers (18 Infantry and 18 support) were randomly 

assigned to either the experimental or the control group. Experimental group squads received 

13 
two hours of SETS training that included both simulated firing for record and two tactical 

defensive scenarios. They then zeroed their weapons on a live-fire range, conducted two tactical 

live-fire exercises, and then fired for record on a live-fire range. Control group squads had no 

SETS training. They only zeroed their weapons on a live-fire range, conducted two tactical livefire exercises, and then fired for record on a live-fire range. Squad Leader performance, as rated 

on 27 tasks extracted from ARTEP standards, improved from the first to the second SETS 

exercise, and was higher on the live-fire exercises for the experimental group than the control 

group. This difference was particularly apparent for the leaders of the support squads. The fire 

distribution of experimental group squads on the live fire test was also superior to that of the 

control group squads. While the qualification scores of the Soldiers in the experimental group 

were slightly (but significantly) higher than those of Soldiers in the control group, the 

relationship between live fire and SETS qualification scores was not sufficiently strong that 

SETS could be recommended as a substitute for live fire qualification. The authors concluded 

that SETS had the potential to support home-station training of basic defensive tactical skills. 

This experiment showed conclusively that SETS provided a training benefit for leaders, 

Soldiers, and units that transferred to the real world. However, this benefit was achieved through 

two hours of additional training. There is no indication of how this would compare with two 

hours of live training. The authors also stressed the instructor dependence of SETS-based 

training. It is up to the instructors to establish the training context, determine how to evaluate 

leader performance, and evaluate it. 

In a related evaluation with more current technology, Miller and Zeisset (2002) 

conducted a Post-Fielding Training Effectiveness Evaluation of the EST 2000. The EST 2000 is 

a small arms simulator design to augment and substitute for individual marksmanship training 

and squad collective training. It provides simulated weaponry that simulates the physical and 

operational characteristics of several weapons. Unlike the Eisley et al (1990) experiment, which 

used the simulator as a means of providing additional training, Miller and Zeisset compared a 

control group which trained using live fire procedures with an experimental group that 

substituted EST 2000 training for a portion of the live fire. The extent of the training, either in 

terms of the number of rounds fired or time required, was not described. The largest number of 

Soldiers trained on the M16A2 384 using live fire and 76 using the EST. There were no 

significant differences in the qualification scores between the EST and the live fire groups, either 

in terms of mean scores or in the proportion qualifying as Expert, Sharpshooter, or Marksman. 

The results for the M240B machine gun were similar, in that no significant differences were 

found, although the sample sizes were so small (22 Soldiers in each group) that little confidence 

can be placed in this conclusion. 

The Dismounted Warrior Network Experiments 

Involvement of Army Soldiers in what we would today think of as immersive virtual 

simulation began with the Dismounted Warrior Network (DWN) program in 1997. DWN was a 

U.S. Army Simulation, Training and Instrumentation Command (STRICOM)1

 program to 

develop a reliable, low-cost, easy-to-use capability to insert dismounted Soldiers into VE. The 

 

1 What was then the R&D component of STRICOM is now the U.S. Army Research, Development, and Engineering 

Command Simulation and Training Technology Center. 

14 
DWN program (and its successor, DWN Enhancements for Restricted Terrain (ERT)) did not 

evaluate training effectiveness, but did obtain data, through automated data capture, Soldier 

questionnaires, and observation, about task performance in the virtual simulators used in the 

program. 

A series of experiments was conducted during 1997 to investigate different interfaces for 

inserting dismounted Soldiers into virtual simulations. Four Virtual Individual Combatant 

Simulators (VICS) were selected for evaluation based on three criteria: a desire to have a diverse 

mixture of characteristics to examine; a cost/benefit assessment of system characteristics; and 

system availability. The VICS were designated Alpha, Bravo, Charlie, and Foxtrot. 

Figure 2. A Soldier in VIC Alpha. 

VIC Alpha used full body tracking with multiple fixed-position video cameras and 

reflective markers on the head, body, and weapon to track posture, orientation and weapon 

aiming. Locomotion used a “human joystick” approach, with movement direction determined by 

body orientation, and speed of movement determined by distance from a marked center point on 

the floor; the further the user was from the center point, the faster the movement. Return toward 

the center point was required to slow down or stop. A low resolution (420 x 230 pixels) wireless 

15 
HMD was used to provide a view of the virtual world. A photograph of a Soldier using VIC 

Alpha is shown in Figure 2. The white spheres are the reflective markers for the tracking system. 

VIC Bravo projected visual images on four rear-projection screens that surrounded the 

user. Locomotion was provided by the Omni-Directional Treadmill (ODT), which used 

interlocked belts oriented at right angles in conjunction with a centering harness to permit the 

user to walk in any direction while remaining in the same position. A Soldier in VIC Bravo is 

shown in Figure 3. 

Figure 3. A Soldier in VIC Bravo. 

There were two versions of VIC Charlie, one used during the Engineering Experiments 

and the other used during the User Experiments. While they had different simulation software, 

the interface for both was a computer monitor and joystick. 

VIC Foxtrot was the then-current version of the Navy TTES, described earlier and shown 

in Figure 1. It used a foot pedal for control of movement, a single rear projection screen, and a 

tracking system to determine head and weapon position and orientation. 

16 
The DWN evaluations were conducted in two phases. Eight Soldiers participated in each. 

The first was a set of Engineering Experiments, which focused on how well individual Soldier 

tasks (detect targets, engage targets, move through a building, etc) could be performed in the 

simulators. The Engineering Experiments made extensive use of objective measures of Soldier 

performance. After the Engineering Experiments were completed, the VICS, a DI SemiAutomated Forces (DISAF) station, an Exercise Support Station, and an AAR station were tied 

into a DIS network and installed at the Soldier Battle Lab (SBL)2

, Fort Benning. A set of User 

Experiments was then conducted which focused on the performance of team and squad tactical 

tasks. The User Experiments relied heavily on the use of questionnaires to obtain feedback from

Soldiers regarding how well they could perform tasks in the simulators. Soldiers rated each VIC 

on the extent to which they could perform selected tasks, and the difficulty of performing those 

tasks. Tasks covered movement, orientation, visual recognition, and weapon engagement. The 

detailed results are presented in Lockheed Martin (1997) and Pleban, Dyer, Salter, & Brown 

(1998). 

The DWN experiments were successful in showing that Soldiers could perform basic 

Infantry tasks in the simulators. All VICS were somewhat effective for engaging targets, 

simulating locomotion, and identifying people and objects. However, the experiments also 

revealed many limitations of the VICS. Soldier performance on visual tasks was closely related 

to the resolution of the visual display: the higher the resolution, the better the performance. 

Weapon use was difficult to implement with an HMD. It required presenting a graphic image of 

the weapon in the HMD in a position that corresponded with the actual position of the weapon 

with little margin for error. In addition, the bulk of the HMD prevented the Soldier from

bringing the weapon into its proper aiming position. Some of the interfaces were difficult to 

master. DISAF did not work in buildings. System reliability was a major factor in determining 

Soldier preferences. Soldiers preferred VIC Bravo over VIC Alpha, but otherwise there were no 

differences. 

A follow-on project to DWN, entitled DWN ERT focused on Military Operations in 

Urban Terrain (MOUT). Some VICS were modified and new ones developed based on lessons 

learned in the DWN experiments. New locomotion methods were introduced, improved visual 

systems were incorporated, and new aiming techniques were implemented. In addition, DISAF 

was modified to support operations inside buildings. Engineering experiments, live tests at the 

Fort Benning McKenna MOUT site, and user experiments were conducted in July 1998 with 

these modified systems. The goal of this round of experiments was to investigate how well a fire 

team of VICS and DISAF could support MOUT tasks at the individual Soldier, fire team, and 

squad, levels. 

The four VICS included an improved version of VIC Alpha from the 1997 experiments, 

and three VICs based on the Soldier Visualization Station (SVSTM). Improvements to VIC 

Alpha included changing locomotion control from the “human joystick” to a weapon-mounted 

joystick, and upgrading the HMD. All SVSTM-based VICs used two PCs for calculations and 

image generation, a demilitarized weapon, and a tracking system to determine head position and 

weapon position and orientation. VIC Delta was the basic SVSTM used with a 10’ x 8’ rear 

projection screen and a rifle-mounted joystick for movement. VIC Echo was the basic SVSTM

 

2

 This is the current name for the facility. There have been several name changes since its founding.

17 
used with a front projection dome display which provided a field of view approximately 150 

degrees wide by 40 degrees high. As in VIC Delta, movement was controlled by a rifle-mounted 

joystick. VIC Golf used the SVSTM

 software with an HMD and the ODT from the 1997 

experiments. 

Eight Soldiers participated in tests of the systems over a two week period. The detailed 

results are documented in the DWN ERT Final Report (Lockheed Martin Corporation, 1998) and 

Salter, Eakin, & Knerr (1999). The results are to some extent tied to the specific (and now 

outdated) equipment employed, but they also serve to illustrate some trends that continue to the 

present. First, locomotion in the virtual world was difficult. It took the Soldiers approximately 

four times as long to complete a route through a virtual building than it did to complete the same

route in the actual building at the McKenna MOUT site. Inability to track weapon positions

accurately led to the weapon aiming portion of the test being curtailed. Even at the shortest 

range (25 meters), the best VICs hit only 67% of the targets. In addition, the accuracy of the 

trackers decreased with the distance of the weapon from the tracker (which was mounted 

overhead). This resulted in the prone position being the least accurate firing position. 

In both the engineering and user experiments, the Soldiers rated the VICS on the 

dimensions of similarity to the real world, and speed, quality, and difficulty of performance. In 

the engineering experiments, the activities rated were Detect Targets, Engage Targets, Move 

Inside Buildings and Move Outside Buildings. In the user experiments the activities rated were 

Tactical Movement, Room Clearing, React to Contact/Engage, and Communicate. When there 

were significant differences among the VICs, it generally indicated that VIC Golf, with the 

combination of an HMD and the ODT, performed less well than the others. VIC Golf also 

produced more symptoms of simulator sickness. (See separate section on simulator sickness). 

Salter, Eakin, and Knerr (1999) identified four major areas for improvement. The first 

was position and orientation tracking. Improved tracking was necessary to improve weapons 

accuracy. The second was the means of simulating locomotion. While walking in the real world 

is natural and is usually accomplished without much conscious effort, “walking” in the virtual 

world was not. Within the amount of time available, Soldiers did not acquire full proficiency in 

the simulators and consequently moved more slowly in the virtual world than in the real world. 

They also frequently collided with walls. Locomotion required conscious effort that may have 

impaired the performance of other tasks or interfered with training. The third was providing a 

means of non-verbal communication, such as gestures and facial expressions. The fourth was 

wide-angle visual displays. 

The Virtual Environment Science and Technology Objective (STO) 

During the period 1999 – 2005 there were a series of related technology development and 

transition activities which shared a common assessment methodology and used nearly identical 

measures of task performance and training effectiveness in VE. In the following sections, the 

assessment activities will be described, and then the collective results twill be described and 

discussed. 

18 
Following the completion of the DWN experiments, a four-year (FY 1999-2002) STO 

effort was initiated to develop a demonstration virtual dismounted leader trainer at the fire team, 

squad, and platoon level. Five Army organizations3

 collaborated on the research and 

development. Each had a particular area of interest, but all worked together to explore concepts 

and systems and to recommend directions for further work on training, concept development, 

and mission rehearsal. 

One of the emphases during the first year of the STO (FY 1999) was on identifying highpayoff small unit and leader training applications. Pleban, Eakin, and Salter (2000) took the task 

lists from analytic efforts (including Jacobs et al, 1994), and eliminated those tasks that could not 

be supported by the technology as represented in the then currently available SVSTM; (VIC Delta

in the DWN ERT experiments). Tasks were: move through built-up areas, reconnoiter area, 

react to contact, assault, and clear a building. They then refined their findings by having six 

Soldiers carry out 15 tactical scenarios in the simulators as three-man teams, and then additional 

scenarios as a six-man team. After completing a set of three similar scenarios, they rated their 

ability to perform each of the component activities of that task on a four-point scale (Very Poor, 

Poor, Good, or Very Good). Ratings were then aggregated to produce an overall rating for each 

task. The activities and the rating scheme developed by Pleban Eakin, and Salter (2000) formed 

the basis for Soldier ratings of their ability to perform tasks in virtual simulations in a number of 

subsequent investigations performed during the conduct of the STO and even subsequent efforts. 

Each year of the STO a culminating event (CE) was conducted. The CEs were 

comprehensive demonstrations and assessments conducted with Soldiers, using as much of the 

developed technology as was feasible in a realistic training exercise. Three of the four CEs 

(1999, 2001, and 2002) were conducted at the SBL and will be discussed here. 

The 1999 CE consisted of squad-level exercises which were conducted on three 

consecutive days, with different squads involved on each of those days. All Soldiers participated 

using what was then called the Squad Synthetic Environment (SSE), a network of individual 

simulators, each of which consisted of a SVSTM and a virtual radio. The SVSTM is a PC based, 

DIS compatible DI simulator developed by Reality By Design (RBD), and was essentially VIC 

Delta. 

Squads generally consisted of a Squad Leader and two fire teams. Each fire team was 

composed of three Soldiers and a fire team leader. The focus of the evaluation was as the squad 

leader as the primary trainee. Five exercises were conducted for each squad, although not all 

squad members were involved in each. The first and fifth exercises were assessment exercises 

and the entire squad participated in both. The second, third, and fourth exercises were conducted 

with different mixes of virtual Soldiers and DISAF. Each squad leader conducted exercises with 

an all-virtual squad, an all-DISAF squad, and a Half-SAF squad (one virtual fire team and one 

DISAF fire team). AARs and, to a limited extent, coaching were used improve learning. 

 

3

 The organizations were the U.S. Army Research Institute Simulator Systems and Infantry Forces Research Units, 

the U.S. Army Simulation Training and Instrumentation Command, and the U.S. Army Research Laboratory Human 

Research and Engineering and Computational and Information Sciences Directorates. 

19 
At the completion of their exercises, all Soldiers who were not squad or fire team leaders 

rated their ability to perform 48 tasks (from Pleban Eakin, and Salter, 2000) in the simulators as 

very poor, poor, good, or very good. Squad and fire team leaders instead completed a 

questionnaire which asked them to rate their improvement in 11 areas as a result of their training. 

The areas were: Plan a Tactical Operation, Coordinate Activities with your Chain of Command, 

Assess the Tactical Situation, Control Squad or Fire Team Movement during the Assault, 

Communicate with Members of your Team or Squad, Clear a Building, Clear a Room, Control 

Squad or Fire Team while not in Contact with the Enemy, Control your Squad or Fire Team, 

React to Contact Battle Drill, and Locate Known or Suspected Enemy Positions. These same

questionnaires, with minor variations in the Soldiers tasks, would be used repeatedly in 

subsequent assessments, and provide a basis for year-to-year comparisons. 

By the third year of the STO (2000-2001), the focus was broadened to include a heavy 

emphasis on the human interface aspects of the VE systems for training, including the use of 

automatic speech recognition to control DISAF, development of an AAR system, and 

incorporation of a broad range of DISAF behaviors. At the same time, the technologies were 

becoming more mature and stable, and interoperability issues were being addressed with 

increasing success. Training continued to be a principal area of investigation. 

The FY 01 CE was held in September 2001 at the SBL (Knerr et al 2002). The key 

technologies included: a Dismounted Infantry Virtual AAR System; new behaviors and 

improved operator control for DISAF; fire team leader control of DISAF through Voice 

Recognition and Synthesis; enhancements to the SVSTM; an improved ODT; and a dynamic 

terrain server. 

Infantry Soldiers participated in scenarios in the simulators, with one group of six 

Soldiers participating for one day, and two groups of six soldiers participating for two days each. 

The network configuration for the CE is shown in Figure 4. The following items were connected 

to the network: 

• Five SVSTM individual Soldier simulators. These were used by the squad leader, the 

two fire team leaders, and two of the three Fire Team A members. The simulators were 

identical, except for additional equipment in the Fire Team B leader’s area for the voice 

recognition system. All SVSs were equipped with ASTiTM radio headsets, which 

permitted verbal communication on up to two channels, depending on the duty position. 

The squad leader could talk to the platoon leader (a role player) and his fire team leaders. 

Each fire team leader could talk to the squad leader and his subordinates. Fire team

members could talk among themselves and with their fire team leader. 

• One ODT station. The third A fire team member used this simulator. Like the SVSs, 

this station was equipped with an ASTiTM radio headset. 

• One Voice Recognition PC. 

• One AAR System PC. 

• One Dynamic Terrain Server. 

• One BattleMaster/DISAF Operator Station. The DISAF Operator and the Exercise 

Controller used this station. 

• One Desktop SVSTM used by a role player. 

20 
21 

Figure 4. 2001 CE System Configuration, Soldier Battle Lab, Fort Benning. The 2002 System

. 

general situation and eight different scenarios were developed. Each of the scenarios 

was de

a

t

, 

nario

n the final year of the STO (FY 2002), there was a concerted effort to document the 

training 02

for

DISAF Operator

After

Action 

Review

System

Role Player

SVS Desktop

Simulator

DIS

Rifle Rifle 

SVS SVS 

Voice

Recognition

ASTi Radio Network

Squad Ldr

SVS

Dynamic

Terrain

Server

Rifle

DISim/ODT

Voice

Recog

Team Ldr

SVS

Alpha Fire Team Bravo Fire Team

DISAF

Platoon

Ldr/

Exercise

Controller

Team Ldr

SVS DISAF

(during and after

exercise)

Configuration was similar, except the DISim with ODT was replaced with an additional SVSTM

A

signed to be about 20 minutes in duration. They focused specifically on forcing the 

Soldiers to use the various new devices and capabilities in the VE. Each scenario provided

different mission so the capabilities would be used under differing circumstances. Scenarios 

were also designed to actively involve as many squad members as possible. Seven of the eigh

scenarios took place in a virtual representation of the Shugart-Gordon MOUT facility at Ft. Polk

LA. The eighth scenario took place in a high-rise office building described as being in an 

adjacent town. The scenarios covered a variety of wartime and SASO. The descriptive sce

titles were: Support Operations Checkpoint, Hostage Rescue, Support by Fire, Assault and Clear 

a Building, Roving Patrol, Air Assault and Clear a Building, Crowd Control, and Downed 

Helicopter. 

I

potential of the systems, and to prepare recommendations for the road ahead. The FY

CE was held in August 2002 at the SBL (Knerr et al, 2003). Both the hardware and software 

configuration and the scenarios were similar to those of 2001. Also, three squads participated
two days each. Two different versions of the Hostage Rescue and Deliberate Attack scenarios 

were designed to be comparable in terms of mission type and difficulty, although the starting 

points and objectives were different. By using these parallel versions, researchers were able to

identify improvements in performance or effects of simulation fatigue across time. In addition t

the Leader ratings of changes in their performance that had been obtained previously, SMEs 

rated squad leader performance on 14 different factors during each scenario. 

o

The Virtual –Integrated MOUT Training System (V-IMTS) Assessment 

V-IMTS was a short-term project to speed the transition of the virtual simulation 

technol d

ance 

The Cassidy Combined Arms Collective Training Center at Ft. Campbell, KY was the 

initial t t

s

n

he assessment took place in September 2004. Twenty-seven Soldiers from three squads 

particip

he

the

ed

ogy developed under the STO that specifically considered the integration of live an

virtual training. The objective of the assessment was to obtain information about the perform

of Soldier tasks and training effectiveness in the V-IMTS configuration. 

ransition site for V-IMTS and the site for the assessment (Knerr and Lampton, 2005). I

consists of a 28-building complex of one- to four-story buildings representing a small town. As 

part of the V-IMTS program, a deployable shelter approximately 40 feet square was placed next 

to the Cassidy control center. Three Immersive SVSTM simulators (SVSI) and six Desktop 

SVSTM simulators (SVSD) were installed inside. The squad leader and two fire team leader

used the SVSI. The remaining Soldiers used the SVSDs located immediately behind their fire

team leader. DIVAARS, DISAF, and SVSD stations for human opposing forces were located i

the control room. 

T

ated. The squads were actual units, not groups formed specifically for this event. While 

68% of the squad members had served a tour in Iraq or Afghanistan, all three squad leaders, and

three of the six team leaders, had served in their current duty position for one month or less. 

Their unit personnel, assisted by technical personnel, controlled the exercises and conducted t

AARs. The scenarios consisted of multiple variants of two missions: Search and Cordon a 

Building and Attack/Assault a Building. The variants differed in the buildings involved and

positions and actions of the enemy. Each exercise consisted of the delivery of the operations 

order to the squad leader, a planning period, mission execution, and an AAR (usually conduct

by the Platoon Leader). 

22 
Alpha

Fire Team

Leader

HVAC

40 ft

Fire

Team A

Fire

Team B

SAF AAR

Battle

Master

Bravo

Fire Team

Leader

Squad 

Leader

40 ft

10 ft 10 ft 10 ft 10 ft

AAR

Facility

Internet 

Connection

Figure 5. V-IMTS physical configuration. 

Each squad completed two live exercises separated by two, three, or six virtual exercises. 

Squad 1 participated in six virtual exercises lasting a total of 96 minutes (mean = 16.0 minutes), 

squad 2 participated in four virtual exercises lasting a total of 77 minutes (mean = 19.2 minutes), 

and squad 3 participated in three virtual exercises lasting a total of 44 minutes (mean = 14.7 

minutes). Overall mean duration of the virtual exercises was 16.7 minutes. 

Situational Awareness in Virtual Environments (SAVE) 

The SAVE assessment was conducted at the SBL in 2005. The purpose of the 

assessment was to try out tools for collection data on Squad Leader Situation Awareness. 

However, since it involved full squads conducting tactical exercises in immersive simulators (the 

SVSI), Soldier rating of simulator capabilities and Soldier and leader ratings of training 

effectiveness were obtained. Each of three squads participated in three building clearing 

exercises and six urban operations exercises (three security patrol and three search for arms

cache) over a two-day period. Average scenario duration was about 30 minutes. 

Results 

Table 4 summarizes the results of the immersive simulator capability ratings for the 

assessments/exercises described above conducted during the period 2002 – 2005. The complete 

breakout of the ratings for each assessment is shown in Appendix X. Fifty-three Soldiers and 

leaders participated in those exercises using immersive simulators. While simulation technology 

in general and the SVSTM in particular were evolving, the capabilities were fairly static during 

the 2002-2005 period, and for this reason those ratings have been combined to produce a 

summary rating. Activities are ordered from best (Very Good, 3.0) to worst (Very Poor, 0.0). 

Thirty of the fifty-four activities were rated Good or better (2.00 and above); eighteen were rated 

between the Good/Poor midpoint (1.50) and Good (2.00); and six were rated Poor (1.00-1.49). 

Activities which were rated highly included outdoor movement, identification of types of people 

(civilians, non-combatants within a room, enemy soldiers), identification of tactically significant

areas (sectors of observation and responsibility), and individual weapons use (but not grenades). 

23 
Table 4. 

Combined Ratings of Simulator Capability (2002 – 2005) 

Task Mean Rating

Move through open areas as a widely separated group. 2.49 

Identify civilians. 2.47 

Execute planned route. 2.42 

Move in single file. 2.38 

Fire weapon in short bursts. 2.35 

Understand verbal commands. 2.31 

Locate assigned areas of observation, e.g. across the street. 2.30 

Identify assigned sectors of observation. 2.29 

Move according to directions. 2.29 

Identify non-combatants within a room. 2.27 

Identify sector of responsibility. 2.25 

Execute the assault as planned. 2.25 

Communicate enemy location to team member. 2.22 

Move quickly to the point of attack. 2.21 

Aim weapon. 2.21 

Communicate spot reports to squad leader. 2.21 

Fire weapon accurately. 2.19 

Coordinate with other squad members. 2.17 

Identify covered and concealed routes. 2.16 

Identify safe and danger areas. 2.13 

Assume defensive positions. 2.11 

Locate support team positions. 2.11 

Use hand-held illumination (flares). 2.10 

Identify enemy soldiers. 2.08 

Locate buddy team firing positions. 2.06 

Employ tactical hand-held smoke grenades. 2.03 

Maintain position relative to other team members. 2.00 

Identify areas that mask supporting fires. 2.00 

Maneuver below windows. 2.00 

Use flash-bang grenades to help clear rooms. 2.00 

Take hasty defensive positions. 1.98 

Engage targets within a room. 1.98 

Scan from side to side. 1.90 

Look around corners. 1.89 

Determine other team/squad members’ positions. 1.84 

Take position to one side of a doorway. 1.81 

Locate enemy soldiers inside buildings firing at your unit. 1.80 

Move close to walls. 1.77 

Scan the room quickly for hostile combatants. 1.75 

Maneuver/move around obstacles. 1.75 

Use fragmentation grenades. 1.70

Estimate distances from self to a distant object/point. 1.67 

Maneuver close to others. 1.66 

Take a tactical position within a room. 1.66 

Move past furniture in a room. 1.63 

Climb up or down stairs. 1.59 

Maneuver around corners. 1.53 

Visually locate the source of enemy fire. 1.52 

Move quickly through doorways. 1.40 

Distinguish between friendly and enemy fire. 1.39 

Maneuver past other personnel within a room. 1.36 

Determine the direction enemy rounds are coming from. 1.33 

Scan vertically. 1.19 

Determine the source of enemy fire by sound. 1.15 

24 
25 

Poorly rated items included maneuver indoors (close to others, past furniture, close to walls, 

around objects, past other personnel, around corners, through doorways, up and down stairs), and 

identifying the source and type of fire (enemy or friendly), either by auditory or visual cues. 

Table 5. 

Summary of Simulator Capability Ratings by Assessment 

Assessment Pleban 

1999 

VE STO 

1999 

VE STO 

2001 

VE STO 

2002 

V-IMTS 

2004 

SAVE 

2005 

N 6 18 18 18 9 26 

Mean 

Rating 2.27 1.90 1.72 2.13 1.67 1.90 

What is most striking about the ratings, however, as shown in Table 5, is that there has 

been no systematic improvement in the ratings over a six-year period. In fact, the highest ratings 

were obtained in 1999 (mean = 2.27) and the lowest in 2004 (mean = 1.67). Possible reasons for 

this will be discussed below. 

Table 6 summarizes the results of the administration of the Training Effectiveness 

Questionnaire. This was administered to Squad and Fire Team Leaders in every assessment 

beginning with the VE STO CE in 1999, and to Soldiers (fire team members) in the V-IMTS and 

SAVE assessments. The pattern here is much more consistent, with the mean ratings increasing 

consistently every year, from 0.82 (less than slight improvement) in 1999 to 2.06 (moderate 

improvement) in 2005. Leaders and Soldiers report learning different things. Leaders report the 

most improvement in controlling their units, assessing the tactical situation, and communication. 

Soldiers report the most improvement in communication and planning a tactical operation.
26 

Table 6. Training Effectiveness Questionnaire Results 

VE STO 1999 

VE STO 

2001 

VE STO 2002 V-IMTS 2004 SAVE 2005 

Question Leaders Leaders Leaders Leaders Soldiers Combined Leaders Soldiers Combined LeaN 9 9 9(18)a 9 18 27 9 17 26 2Assess the tactical situation. 0.99 1.44 1.72 2.33 1.76 1.96 2.22 1.59 1.80 Control of squad/fire team movement during the assault. 1.10 1.22 1.50 2.44 1.71 1.96 2.56 1.50 1.87 Communicate with members of your team or squad. 1.10 1.56 1.39 1.89 2.00 1.96 2.11 1.65 1.80 Coordinate activities with your chain of command. 0.77 1.78 1.39 2.00 2.00 2.00 2.00 1.41 1.61 Plan a tactical operation. 0.55 1.22 1.50 2.11 2.18 2.15 1.56 1.31 1.40 Control squad or fire team movement while not in contact with the enemy. 

1.10 1.22 1.44 1.67 1.69 1.68 2.22 1.06 1.45 

React to Contact Battle Drill. 0.55 1.22 1.44 1.11 1.65 1.46 2.11 1.41 1.64 Control your squad or fire team. 1.10 1.22 1.50 1.67 1.59 1.62 2.33 1.00 1.44 Clear a building. 0.66 0.78 1.29 1.33 1.94 1.73 1.89 1.12 1.37 Locate known or suspected enemy positions. 0.55 1.11 1.39 1.22 1.59 1.46 2.00 1.35 1.57 Clear a room. 0.55 0.88 1.44 1.33 1.88 1.69 1.67 0.94 1.18 Mean 0.82 1.24 1.45 1.74 1.82 1.79 2.06 1.30 1.55 

a Each of the nine leaders completed the questionnaire at the end of each day of training, resulting in a total of 18 
0123

React to Contact

Assess tactical situation

Control movement assault

Locate enemy positions

Clear room

Clear building

Control movement not in contact

Plan tactical operation 

Control squad or team

Coordinate with chain of command

Communicate with team or squad.

Overall

Mean Rating

Soldier

Leader

Figure 6. V-IMTS training effectiveness ratings for leaders and Soldiers.

In the V-IMTS assessment, the Squad and Fire Team Leaders used immersive simulators 

(SVSI) while the Soldiers used desktop simulators (SVSD). The training effectiveness ratings 

from the V-IMTS assessment are presented in Figure 6. The mean rating for training 

effectiveness was 1.8 on a scale where 1.0 equaled slight improvement and 2.0 equaled moderate 

improvement. Interestingly, both the leaders and the Soldiers reported about the same amount of 

overall improvement (mean rating of 1.76 vs. 1.66), although the skills on which they reported 

improving the most differed. The SVSI and SVSD ratings were positively correlated (r = .62), 

indicating that in general the same activities tended to be rated similarly on both simulators. 

Leaders reported the greatest improvement on Control of squad/fire team movement during the 

assault, Assess the tactical situation, Plan a tactical operation, and Coordinate activities with 

your chain of command. Soldiers reported the most improvement on Plan a tactical operation, 

Coordinate activities with your chain of command, and Communicate with members of your team 

or squad. Perhaps because of the small numbers involved, the differences between leader and 

Soldier ratings approached statistical significance for only one task, Control of squad/fire team 

movement during the assault. 

The 2002 VE STO CE included an objective measure of unit performance, a fourteenitem checklist of unit behaviors that was scored independently by three raters for each scenario. 

Possible scores ranged from a low of 1.0 to a high of 5.0. Squads received a mean score of 3.47 

on the first occurrence of a scenario, and a mean score of 3.72 on the subsequent occurrence of a 

similar scenario. This provides rare evidence, beyond trainee opinion, that the training was 

effective. 

27
Other Experiments and Evaluations 

Lampton, McDonald, Rodriguez, Cotton, Morris, Parsons, and Martin (2001) found that 

practice of a building search task in VE by two-person teams results in improved performance 

under a variety of training conditions (practice only, demonstration, and AAR).Experimental 

participants were college students. 

Pleban, Eakin, Salter, and Matthews (2001) conducted an evaluation of the use of the 

SVSTM for training and assessment of decision making skills. The trainees were Army

Lieutenants and Captains playing the role of Infantry platoon leaders. Role players and DISAF 

were used to fill the other positions required to conduct the training scenarios. Thus this was an 

individual training situation. Four different scenarios were conducted in a simulated virtual 

village: Stability and Security Operations, Company Assault, Defend, and Secure Village. Role 

players filled the Company Commander, Platoon Sergeant, and Squad Leader positions, and 

DISAF was used to fill out the remainder of the platoon. Objective measures of the accuracy of 

the decisions the officers made improved significantly over the four training sessions (scenarios). 

The officers also reported that they believed that their decision making skills had improved as a 

result of the training. 

Pleban and Salvetti (2003) evaluated the transfer of training received in immersive 

simulators (again, the SVSs in the SBL) to live simulation. Four Infantry squads participated. In 

the Real-World condition, two squads conducted rehearsals of an assault in a real building 

similar to the one on which they would subsequently be tested. In the VE condition, two squads 

conducted rehearsals in a VE representation of the actual building on which they would be 

tested. All squads were then tested on two missions in the actual building (the transfer task). 

Two observer/controllers scored performance on each mission. Performance differences between 

the two groups on the final two missions were considered to be “small to negligible.” However, 

Soldier opinions about the effectiveness of the VE training differed depending on when they 

received it. One-half of the Soldiers in the VE condition felt that it was just as effective as the 

real world for conducting mission rehearsals. However, all of the Soldiers in the Real-World 

condition, who received training in the VE after completing the transfer task, felt that VE was 

inferior to real world training. 

Pleban and Salvetti (2003) note that performance in the VE was hampered by simulator 

constraints that made precision movement in confined areas and use of grenades difficult. They 

concluded however, that while immersive simulators lacked realistic qualities in certain areas, 

they could be used effectively for certain types of training, specifically during the walk phase of 

training, for improving decision making, situation awareness, and communication and 

coordination skills. 

Loftin, Scerbo, McKenzie, Catanzaro, Bailey, Phillips, and Perry (2004) investigated the 

use of an immersive CAVE (CAVE Automatic Virtual Environment) simulator to train 

checkpoint operations. The CAVE visual display consisted of two 10 X10 foot screens each 

with 1280 X 1040 resolution. Other capabilities included a speech recognition system, position 

tracking, and sophisticated avatars for the computer-controlled entities in the scenarios. Two 

groups of 16 college students participated. The experimental group participated in two 45-

28
minute shifts, receiving feedback on their performance between the shifts. The control group 

participated in only a single shift. The experimental group performed better on their second shift 

than on their first, and in addition, performed better on shift 2 than the control group did on shift 

1. Both comparisons indicate that the training was effective. In a second experiment, Loftin et al 

replicated the conditions of the first experiment using a monitor and touch screen in place of the 

CAVE. Although the number of participants was smaller, they were basically able to replicate 

the findings of the first experiment: the simulation (in this case delivered by a monitor) 

significantly reduced the number of errors committed in the scenarios. They also reported that 

the trainees in the fully immersive system (CAVE) consistently performed better than those in 

the monitor and touch screen condition. However, they concluded that: 

 Although this difference was found across groups and conditions, the magnitude 

of the difference was not dramatic. Thus, the ability to port a similar training 

experience to a less expensive PC platform without major performance 

differences underscores the potential for providing greater access to this type of 

VE training in a much more cost effective medium(p. 12). 

Singer, Kring, and Hamilton (2006) investigated the use of immersive simulation to train 

positioning and maneuver skills in an urban environment. They found significant improvements 

in both performance and the time required to perform the tasks. The basic patterns for the time 

to perform the tasks is relatively clear, with steadily decreasing amounts of time required for 

steadily improving (overall) performance. 

Discussion of Immersive Simulation Results 

Limits of Experimentation 

Despite a decade of interest and research and development activity in the use of 

immersive simulation, evaluations of the technology using Soldiers performing Soldier tasks 

have been limited. There are a number of reasons for this. First, experimentation is limited by 

the available simulators, and the only facility available with sufficient simulators to outfit a full 

squad has been the SBL. The lack of availability of Soldiers to participate in evaluations has 

been a second constraint. Operational and training demands, have greatly limited the number of 

squads and the amount of time that they can participate in an evaluation. Finally, the costs of an 

evaluation, in terms of facilities rental, technical and support personnel, and travel costs, limit 

exercise duration. 

SVSs have been the predominant simulators for experimentation, not necessarily because 

they are the best (they might or might not be), but because they exist in sufficient quantities at an 

Army site. This means that to some extent the results obtained are limited to that simulator. 

It is somewhat surprising that Soldier ratings of SVSTM capabilities have not 

systematically improved between 1999 and 2005. There are several possible explanations for 

this. One is that as capabilities have been added to the SVSTM, system performance has 

deteriorated. While this might explain one dip in the ratings, the SVSs have been upgraded 

29
regularly so long-term effects are unlikely. A second possibility is that the training scenarios 

have become more challenging and complicated in an attempt to keep pace with current 

operational conditions. They have pushed the Soldiers to perform more complicated tasks in the 

simulators, and therefore they were more likely to encounter the limits of the simulators. 

Moving more of the action inside buildings, for example, placed an emphasis on precise 

movement, which in turn emphasized simulator weaknesses in that area. The unavailability of 

hand grenades made it extremely difficult to clear buildings, but adding hand grenades increased 

the complexity of using the simulator. It is also likely that Soldier ratings of simulator capability 

are strongly influenced by the amount of time they spend in the simulator acquiring proficiency. 

This has not been controlled over the years. Finally, the impact of the increasing sophistication 

of computer and video games may have caused a gradual escalation of Soldier standards for 

simulator performance. The simulator capabilities are being compared with increasingly realistic 

and sophisticated commercial products. This may have, in effect, raised the standards by which 

automated entities and environments are judged. 

What factors are responsible for the increase in training effectiveness ratings? First, the 

complexity and realism of the scenarios has improved. Second, the AAR capabilities have 

improved. Recall that the VE STO program did not develop an AAR capability until 2001, and 

that it has been incrementally improved since. Third, capabilities such as grenades, smoke, and 

improved DISAF behaviors have been added. However, remember that the trainer, whether 

called an observer/controller or an AAR leader, is key in setting the effectiveness of the training. 

The results of the V-IMTS assessment, combined with the results of the Loftin et al 

(2004) experiment, raise the question of how realistic simulators need to be in order to be 

effective. An interesting and unexpected aspect of the V-IMTS results is the lack of a significant 

difference in rated training effectiveness between the Soldiers trained in the desktop simulators 

and the leaders trained in the immersive simulators. Admittedly, they were performing and 

learning different tasks, but nevertheless it seems to suggest that the features of a fully immersive 

simulator might not produce a training benefit that justifies the cost premium over a simple 

desktop computer. Loftin et al (2004) found small differences in effectiveness between 

immersive and desktop simulators, and questioned the effectiveness of the immersive simulator. 

Research on Simulator Fidelity 

Other research addressing the question of interface fidelity on training effectiveness is 

limited. The basic concept is that immersive VE systems, as compared to desktop VE systems, 

provide the trainee with more information about their orientation in and movement through 

physical space. Systems with head-tracked HMDs provide vestibular and kinesthetic cues about 

changes in direction of movement and viewing direction. Systems with a walking mechanism,

such as a treadmill, or which use sensors to implement walking in place, provide kinesthetic cues

about distance traveled. Systems with wide FOV displays provide visual cues in the periphery of 

the visual field about distance and speed of movement. If these factors make a difference in 

effectiveness, they would most likely affect learning routes through and the configuration of 

physical spaces, an area of research known as wayfinding. Researchers in this area make a 

distinction between route knowledge (which allows one to successfully move from one known 

point to another known point along a specific route, using landmarks and waypoints, but does not 

30
allow for deviations from the route), and configuration or survey knowledge (which enables uses 

to create a map of an area or create new routes). 

As background, it should be noted that the question of whether VR systems can be used 

to train configuration and route knowledge is not in dispute. A large number of experiments 

have shown that they can (e.g., Witmer, Bailey and Knerr, 1995; Bliss, Tidwell, and Guest, 

1997; Tate, Siebert, and King, 1997). The question is whether more realistic and sophisticated 

interfaces enhance that learning. 

Grant and Magee (1997) compared a walking in place interface with a joystick in an 

experiment investigating the acquisition and transfer of spatial knowledge. All participants 

viewed the VE through a head-tracked HMD. The participants' task was to learn the location of a 

number of landmark objects in a science museum. During a transfer test in the actual building, 

the walking simulator group walked significantly less distance to find all of the landmarks than 

did the joystick trained group, but they did not find them significantly faster. Grant and Magee 

claim this as evidence that there is something learned from the walking interface in addition to

the visual information that can be used for spatial navigation. 

Singer, Allen, McDonald, and Gildea (1997) found that training a Hi-VE interface (headlinked stereoscopic HMD and treadmill) produced better configuration knowledge and more 

rapid response time than a Low-VE interface (non-head-linked stereoscopic HMD and joystick) 

in a terrain learning task. Students trained using the Hi-VE interface also did better than those 

trained using topological maps. 

Waller, Hunt, and Knapp (1998) compared blindfolded maze-walking performance of 

groups trained with immersive VE (HMD with no head-tracking plus joystick), desktop VE 

(monitor plus joystick) and map, among other conditions. While the performance of all groups 

improved, the map condition was superior to the immersive VE and desktop VE conditions, 

which did not differ. However, their immersive VE condition provided no additional kinesthetic 

or vestibular cues relative to the desktop VE condition. 

Jacquet (2002) found no significant differences between groups trained to perform tank 

turret troubleshooting tasks using immersive (HMD and 3D mouse) and desktop (two monitors 

and a mouse) versions of an M1 tank turret simulator. 

Lathrop and Kaiser (2005) compared immersive and desktop interfaces in terms of search 

performance and maintenance of orientation. Immersive users wore a head-tracked HMD, 

changed direction of movement by rotating while seated in a chair, and clicked a button on a 

hand-held mouse to move forward or backward. Desktop users viewed a Monitor and used the 

hand-held mouse to both turn and move forward or backward. Immersive users were able to 

locate target objects in a complex environment more quickly than desktop users, but did not 

develop better configuration knowledge of the space they searched. 

Singer, Ehrlich, Cinq-Mars, and Pappin (1995), investigating the effects of head-coupling 

and stereoscopic displays on the performance of simple tasks in VE, found that those two 

variables had a significant effect only on distance estimation at short distances (10 feet or less). 

31
In summary, it appears that head-tracked visual displays, body controlled movement, or a 

combination of the two can improve the performance of spatially-oriented tasks and acquisition 

of spatial knowledge, but this difference, based on a limited number of experiments, does not 

appear to be large. 

32
The Use of Computer Games for Training 

Over the past ten years, there has been increasing interest in the use of computer games, 

played on either a PC or a game console, such as the X-Box, Playstation, or Gameboy, for Army

training. Once these games were believed to be visually attractive, but too limited in terms of the 

actions they permitted the players and those of the simulated entities, and too unrealistic in terms 

of how players made those actions (e.g., using a game controller, joystick, or mouse), to be 

considered as a means of training. Today, games are being seriously considered for a variety of 

reasons. 

First, it is assumed that most Soldiers play computer games. Two additional assumptions 

follow from this one. The first is that Soldiers are motivated to play games designed to train, 

even to the extent that they will do so in their off-duty hours. The second is that Soldiers are 

skilled in playing games, and therefore they need little introductory train-up or preparation, such 

as game concepts or learning how to use game controllers, prior to using these games for 

training. 

Second, game consoles and PCs, particularly laptop PCs, are readily transportable and 

deployable. They are also low cost relative to special purpose simulators. 

Third, many games can be networked to provided collective training. Some games 

(massively multiplayer games) can include hundreds of simultaneous participants. 

Finally, the use of computer games provides a way to capitalize on the investments 

already made by game developers. This can translate into low per copy acquisition costs, 

particularly if the game can be used as is. 

The issue of training effectiveness, however, is the most important consideration. If 

games do not provide effective training, then the reasons given for their use do not matter. What 

evidence is there that computer games can provide effective training? 

Hayes (2005) conducted an extensive review of the use of games for education and 

training. Of 270 documents he identified in a literature search, only 48 reported evaluations in 

which empirical data on effectiveness was obtained. Of those, six used military personnel as 

trainees. Skills trained in those six experiments included periscope skills; chemical, biological, 

and radiological defense principles; and technical, electrical, or electronics skills and 

terminology. Three experiments showed better learning or retention with the game, one worse, 

one no difference, and one showed transfer of the skills learned in the game but had no control 

group for comparison. Hayes’ review showed that games can provide effective training for a 

variety of training audiences and a variety of skills. However, he concluded that there is no 

evidence that instructional games are more effective than other well-designed instructional 

activities. 

While a few of the evaluations reviewed by Hayes represent military tasks, fewer address 

the issue we are most concerned with, Soldier training of combat tasks. 

33
DeBrine and Morrow (2000) demonstrated that a computer off-the-shelf (COTS) 

computer game (Quake III ArenaTM) could be modified to support conduct of a walkthrough of a 

real building. It took about 300 hours of labor for unskilled programmers to modify the game

code and implement the building model. No try-outs or evaluations of the resulting game were 

conducted. 

 

Beal and Christ (2004) conducted an evaluation of Full Spectrum Command (FSC), a PC 

game designed specifically for use in the Infantry Captains Career Course (ICCC). Fifty-four 

ICCC students participated. One-half of the students participated in their normal course work. 

The rest did that and in addition played FSC for an average of 4.8 hours (in addition to 4.8 hours 

of demonstration and train-up). The decision making of officers in both groups was then 

assessed as they conducted individual exercises using the JANUS Simulation. No differences in 

decision making performance during the JANUS exercises were found. Overall, the trainees 

reported highly positive opinions about the overall training value of FSC. They reported that the 

scenarios were challenging, the realism of the battlefield events and actions was quite good, and 

playing FSC provided practice in many of the tasks that were relevant to course objectives. On 

the other hand, the majority indicated that they had to focus their attention on the game’s display 

and control devices, rather than on the experiences created by the game. 

Beal and Christ (2004) believe that they failed to find a difference between the two 

groups in performance on the JANUS exercise because of limited time available to play FSC, the 

lack of controlled procedures for use of FSC (e.g., trainee selection of scenarios), and the lack of 

any feedback or evaluation of decision making performance during or after the FSC scenarios. 

While an AAR capability was built into FSC, it was not used due to operational constraints. 

Belanich, Sibley, and Orvis (2004) conducted an experiment using the “America’s 

Army” game, a first person shooter designed as a recruiting tool. Twenty delayed entry recruits 

and one Reserve Officer Training Corps cadet played the basic training section of the game, 

consisting of Army background information, marksmanship training, obstacle course, weapons 

familiarization, and a MOUT mission. After playing the game, the participants answered a series 

of multiple choice questions about information presented in the game. The participants 

remembered procedures better than facts, and information presented in graphics and spoken text 

better than information presented in written text (perhaps because the written text could be 

skipped over without reading). They also recalled information relevant to progressing through 

the game better than irrelevant information. 

Diller, Roberts, Blankenship, and Nielsen, (2004) evaluated the DARWARS AMBUSH! 

at Fort Lewis, WA, in May 2004. AMBUSH! is a computer-based collective training game that 

lets units experience and respond to ambush situations. A total of 18 participants (16 enlisted 

personnel and two officers) used the system in three groups. Each group participated for about 

three hours, to include a training session and the conduct of three missions. At the conclusion of 

the training, participants rated their overall satisfaction with the system as a convoy operations 

training tool as 5.88, on a scale of 1 (low) to 7 (high). All participants gave ratings at or above 

the midpoint of the scale (4). Participants were even more positive about using the system as a 

tactics, techniques, and procedures training tool (mean = 6.36). Trainee performance was not 

assessed. 

34
Beal and Christ (2005) conducted an assessment of the Rapid Decision Trainer (RDT). 

The RDT is a PC-based simulation built on game engine software. It was designed to prepare 

Infantry lieutenants to serve as platoon leaders during the Infantry Officer Basic Course (IOBC) 

platoon Live Fire Exercise (LFX). It requires the trainee to conduct mission analysis and 

planning and to make decisions in response to conditions that emerge during the execution of the 

simulated mission. This assessment used a single RDT scenario, which closely resembled the 

terrain and battle conditions that occur during the IOBC LFX. Twenty lieutenants participated in 

the RDT scenario as two person teams, a platoon leader and an observer, who rotated positions. 

Nineteen lieutenants participated as a large group, some of whom rotated through the platoon 

leader position while the others observed the exercise. Questionnaires were used to obtain 

feedback from the trainees after the RDT training but both before and after the LFX. No 

performance data was collected. Trainees indicated that the RDT: 

• Permitted training and rehearsal of the types of decisions a platoon leader must make 

during a LFX (85% of ratings in highest three [of seven] categories) 

• Provided meaningful practice for exercising command and control of platoon 

operations (82%) 

• Accurately simulated tasks and conditions specified in current platoon-level battle 

drills (80%)

• Had a valuable impact on preparing them for leading a platoon in a unit (77%) 

• Provided meaningful practice for planning a platoon mission (72%) 

Despite concluding that the RDT had training value, the authors also provided a number of 

cautions: the sample size was small, no performance measures were available, training time on 

the RDT was limited, only one scenario was available, and the simulation was still under 

development. 

Nolan and Jones (2005) assessed the use of Delta Force: Black Hawk Down – Team 

SabreTM to train IOBC students to perform the React to Contact Battle Drill. In this case, full 

squads at individual computers were networked together to conduct the training scenarios. Two 

squads each completed two eight-hour training sessions (completing six or seven missions) using 

the game in addition to their regular training. Two control squads received only their regular 

training. All squads then conducted a live exercise, and completed questionnaires. All 

participants in the game condition rated the training effectiveness of the game as “good” or 

better. All game participants also reported that the game had a “positive” or “very positive” 

effect on their performance on the subsequent live exercise. No performance data on the live 

exercise were obtained. 

Issues in Game-Based Training 

ARI hosted a symposium on the use of PC-based simulations and gaming for military

training in November 2003 (Belanich, Mullin, and Dressel, 2004). Approximately 50 researchers 

and trainers participated. Beal (2005) documented lessons learned and issues based on his 

conduct of evaluations of the RDT, Full Spectrum Command, and FSW. There is considerable 

similarity between the symposium conclusions and Beal’s lessons learned. 

35
First and most important, the consensus is that games can be effective for training, but the 

empirical evidence to support this is lacking. In particular, there is a belief that games can be 

effective in training cognitive skills prior to their application in field exercises. In other words, 

games could bridge the gap between classroom training and field exercises. 

Instructors are believed to be highly important to the successful use of games for training. 

Beal (2005) gives two general areas of instructor support. The first is in explaining the purpose 

of the game, demonstrating game functions, and explaining game limitations and shortfalls. The 

second is in providing coaching and feedback, to include the conduct of AARs. Feedback to the 

trainee by the instructor was the most frequently mentioned instructional techniques for use with 

games. This feedback can range from immediate feedback during the game or scenario to an 

AAR at its conclusion. Feedback should both highlight positive lessons learned and diminish the 

possibility that the trainees will learn suspect behaviors that may enhance success in the game, 

but not in the real world. 

Also important is the need for game design to be based on training objectives. Clear 

training objectives and follow-through by instructors are important for successful 

implementation. 

Data on game playing 

Both the ARI symposium and Beal (2005) called into question the assumption that all 

Army trainees are skilled game players. The attractiveness of these games, in addition to 

presumed training effectiveness, is that Soldiers are assumed to already be familiar with them, 

and motivated to train with them, even in off-duty hours. Are these assumptions justified? 

Symposium participants considered the time required to learn to play a game is a barrier to 

successful implementation. 

Roberts, Foehr, and Rideout (2005) conducted an extensive survey of American youth 

and media in 2004. They found that 8-18 year olds spent an average of 19 minutes per day 

playing PC games and 49 minutes playing console and handheld games. Fifty-nine percent (68% 

of males and 51% of females) play some type of interactive games on any given day. These 

results seem to be fairly consistent across race and parent’s education and income. However, it 

cannot be determined from the report what percentage of youth rarely or never play interactive 

games. 

No recent large-scale surveys of Soldier game playing have been conducted. However, 

Soldiers participating in evaluations of simulations and PC games have been asked about their 

game-playing experience. Soldiers in the V-IMTS evaluation described earlier reported a mean 

of 9.5 hours (median 6.5 hours) per week playing computer or video games. A sample of 27 

Infantry Soldiers from a separate assessment conducted in FY 2004 (Knerr, Garrity, and 

Lampton, 2004) reported a mean of 8.5 hours (median 5.0 hours) per week playing computer or 

video games. However, 7% and 22% of the Soldiers in these samples, respectively, did not play 

video games. Beal and Christ (2005) found that 30% of their sample of 39 Infantry lieutenants 

involved in the evaluation of the RDT did not play video games, and 51% described themselves 

as Novice PC game players. Only 13% reported playing more than two hours per week. In their 

36
evaluation of the FSC game, Beal and Christ (2004) found that 57% of their sample of 54 

Infantry lieutenants and captains did not play computer games. Diller, Roberts, Blankenship, 

and Nielsen (2004), when evaluating the game DARWARS Ambush!, found that 14 of their 18 

participants reported playing computer games for at least one hour per week (mean = 12.9 hours 

per week). Their sample included 2 officers and 16 enlisted personnel, three of whom were 

women. Nolan and Jones (2005) reported that 10 of the 41 IOBC students in their experiment 

had played no computer games in the previous two years, and only 15 had played weekly or 

more frequently. 

The results involving enlisted personnel seem fairly consistent with those of the Roberts, 

Foehr, and Rideout (2005) survey data reported above. The results involving junior officers 

seem very different. Whether this is a result of differences in age, education, social factors, or the 

particular questions asked is not clear. It is safe to assume that a large portion of our potential 

trainees are familiar with computer or video games. However, we cannot assume that all of them

are skilled or enthusiastic game players. This does not mean that games could not provide 

effective training for them, but it does indicate a need to provide adequate training in playing the 

game before training begins in earnest. Beal (2005) found that leaders differed in their 

proficiency with games and personal computers. More than 70% of the participants in the FSC 

evaluation reported that they had to focus on computer control devices and functions rather than 

the experiences created by the simulation. He reported finding similar results in other 

evaluations. Experimenters tended to overestimate trainee proficiency and underestimate 

training time. 

A related question is whether Soldiers have access to computers to use for training in 

their off-duty hours. The ARI Army Personnel Survey Office conducted a survey in the fall of 

2002 and found that 99.1% of all Army officers and 87.4% of all enlisted personnel reported 

having access to the internet, while 98.7% and 84.1%, respectively, reported having access to a 

PC. The group with the lowest access was junior enlisted personnel, with only 80.4% having 

access to the internet and 74.8% having access to a PC. These data may have changed, but no 

recent survey data are available. Moreover, having a computer and internet access in the 

barracks or residence does not guarantee that those resources can be used by the Soldier for 

training in off-duty hours. 

37
Simulator Sickness 

Simulator sickness refers to unwanted side effects and aftereffects that may result from 

using simulators, but does not result from similar use of the actual equipment. Major symptoms

include nausea, dizziness, and headache or eyestrain. Simulator sickness is closely related to, but 

nevertheless different from, forms of motion sickness such as sea sickness, car sickness, and air 

sickness. Simulator sickness is a concern because it can affect the well being of the trainee and 

reduce training effectiveness, even if it does not produce severe symptoms. Discomfort in the 

simulator may distract the trainee. It may also cause trainees to adopt behaviors that reduce 

discomfort in the simulator, but will be detrimental if transferred to the actual task (such as 

reducing side-to-side scanning). Aftereffects involving the sense of balance or visual flashbacks 

could impair the trainees' ability to drive safely or perform skilled motor tasks after leaving the 

simulator. The training value of a simulator is also reduced if simulator sickness causes a 

reduction in the frequency or duration of its use. 

Experts do not agree about the causes of simulator sickness (Johnson, 2005; Kolasinski, 

1995). However, according to the most popular theory, simulator sickness is thought to result, at 

least in part, because simulated movement results in a conflict between the body's mechanical 

systems and visual systems for sensing movement. Normally, our sense of balance is maintained 

by a complex interaction of several senses including: the eyes, which monitor where the body is 

in space and direction of motion; the inner ears, which monitor the directions of motion; the skin 

pressure receptors such as in the feet and seat, which indicate what part of the body is down and 

touching the ground; and the muscle and joint sensory receptors, which indicate what parts of the 

body are moving. When the relationships among these senses differ from our normal experience 

(as when our eyes perceive us moving down a corridor while our other senses indicate that we 

are standing still), discomfort results. 

Kolasinski (1995) has documented a variety of factors that affect the incidence and 

severity of simulator sickness. These are shown in Table 7. It is not necessary to review these in 

detail. However, it is important to recognize that the extent to which a trainee will experience 

simulator sickness symptoms will depend on their individual characteristics, the characteristics 

of the simulator, and the characteristics of the task they are trying to perform. 

Simulator sickness is usually measured by using questionnaires. A commonly used 

questionnaire is the Simulator Sickness Questionnaire (SSQ), (Kennedy, Lane, Berbaum, & 

Lilienthal, 1993). The SSQ consists of 16 symptoms, each of which is rated by the simulator user 

on a 4-point scale (0=none, 1=slight, 2=moderate, 3=severe). These ratings form the basis for 

three subscale scores (Nausea, Oculomotor Discomfort, and Disorientation) as well as a Total 

Severity score. The symptoms making up the three factor scores are as follows: Nausea - general 

discomfort, increased salivation, sweating, nausea, difficulty concentrating, stomach awareness, 

and burping; Oculomotor - general discomfort, fatigue, headache, eyestrain, difficulty focusing, 

difficulty concentrating, and blurred vision; and Disorientation - difficulty focusing, nausea, 

fullness of head, blurred vision, dizzy (eyes open), dizzy (eyes closed), and vertigo. One 

complication is that these symptoms as not unique to simulator sickness. They can be caused by 

a variety of factors, such as insufficient sleep, cold, flu, or alcohol consumption. 

38
Table 7 

Potential Factors Associated with Simulator Sickness in Virtual Environments 

Individual Simulator Task

• age 

• concentration level 

• ethnicity 

• experience with real-world 

task 

• experience with simulator 

(adaptation) 

• flicker fusion frequency 

threshold 

• gender 

• illness and personal 

characteristics 

• mental rotation ability 

• perceptual style 

• postural stability 

• binocular viewing 

• calibration 

• color 

• contrast 

• field of view 

• flicker 

• inter-pupillary distance 

• motion platform

• phosphor lag 

• position-tracking error 

• refresh rate 

• resolution 

• scene content 

• time lag (transport delay) 

• update rate (frame rate) 

• viewing region 

• altitude above terrain 

• degree of control 

• duration 

• global visual flow 

• head movements 

• luminance level 

• method of movement 

• rate of linear or rotational 

acceleration

• self-movement speed 

• sitting versus standing 

• type of application 

• unusual maneuvers 

• vection 

Is simulator sickness really a problem for users of immersive simulators? Knerr et al 

(1998) reported that simulator sickness symptoms among college students who participated in a 

series of VE laboratory experiments were sufficiently severe that 5.6% (29/517) withdrew from

the experiments. Fourteen (4.1% percent) of the 338 participants who were specifically asked 

indicated vomiting. 

Soldiers have typically reported fewer and less severe symptoms. Lockheed Martin 

(1997) reported low levels of symptoms among the Soldiers during the DWN engineering 

experiments (about one “slight” symptom for every four Soldiers). Scores for the DWN ERT 

experiments were higher. For three of the VICS, Soldiers reported symptoms equivalent to two 

slight symptoms at the end of each day. However, Soldiers who used VIC Golf reported the 

equivalent of five slight symptoms. Recall that VIC Golf used a combination of HMD and ODT. 

 In the 2001 VE STO CE, total symptom scores were slightly higher at the end of the day 

than at the start (0.67 vs. 0.31) but the difference was slight, equating to an increase of about one 

slight symptom for every three soldiers. The most frequently reported symptoms at the end of 

the day were fatigue and eyestrain, both with mean symptom scores of 0.17, a level of 

occurrence equal to about one slight symptom reported for every six soldiers. 

In the 2002 VE STO CE, the occurrence of simulator sickness symptoms was very low, 

averaging the equivalent of 1.36 slight symptoms per soldier at the end of the day, compared 

with 1.16 at the beginning. The largest increases were in the symptoms headache and eyestrain 

(+.17 each). 

In V-IMTS evaluation, there was an increase in the overall frequency and severity of 

symptoms following simulator use, but the increase was slight, averaging about the equivalent of 

39
one-half (0.55) a unit increase in one symptom. The increase was larger for Solders in the 

immersive simulators (1.00) than for Soldiers using the desktop simulators. (0.32). 

The low level of occurrence of symptoms likely results from the use of short scenarios 

(approximately 20 minutes each) separated by non-immersive activities, and the use of rearprojection displays in the SVSs. Simulator sickness does not seem to be an obstacle to longerduration training scenarios. 

There was one additional assessment that obtained information about Soldier simulator 

sickness using immersive simulators. It has not been discussed previously because it was 

concerned strictly with simulator capability, not training effectiveness. It involved three squads 

conducting exercises with immersive and desktop SVSs, and three different versions of wearable 

simulators which used HMDs. All of the HMDs permitted peripheral views of the real world 

around the sides and bottom. The SVSs and two of the three wearable simulators produced 

symptom scores in the 1.19 – 1.56 range, but the third wearable simulator produced a means 

symptom score of 4.33. The highest scoring symptoms for that system were eyestrain (.58), 

general discomfort (.44), headache (.47), difficulty focusing (.35) and blurred vision (.26). This 

suggests that the visual display system was a causal factor. The same pattern remained when we 

looked at the scores for three different dimensions of simulator sickness, Nausea, Oculomotor 

Discomfort, and Disorientation: one simulator produced higher scores than the others, which did 

not differ among themselves. This was probably a result of the particular HMD it used. It had a 

very small exit pupil, which meant that it had to be precisely positioned on the head. Head 

movements could cause a temporary loss of the image. It also had an adjustment for interpupillary distance, which each Soldier had to make. If this was not done properly, the images 

seen by the two eyes would not fuse, and eyestrain would result. There was no way to check to 

see it the adjustment had been made properly. 

What can we conclude? First, consider that in all situations involving Soldiers, exposures 

were short, usually under 30 minutes per exercise (although Soldiers participated in multiple 

exercises per day). Longer exercises, particularly with Soldiers who have little experience with 

the simulators, may cause more problems. Second, with a few exceptions, the Soldiers used the 

SVSTM immersive simulator, which provides view of the real world (the floor and walls of the 

simulator enclosure) while viewing the simulation on a rear-projection display. Other display 

systems, such as wide FOV projection systems or HMDs which completely obscure the view of 

the real world are more likely to cause problems. 

Simulator sickness should be viewed as a real but manageable problem. 

40
Discussion 

Let us review some of the results presented earlier. A number of reviews have concluded 

that simulation is a cost-effective means of training. It is not necessarily more effective or 

efficient than the real world or use of the actual equipment, but it typically costs less. Simulators 

employing older technology, such as the Squad Engagement Training System and the 

Engagement Skills Trainer 2000, have proven to be effective for leader and weapons training. 

SME ratings of virtual simulators have tended to indicate potential, if not the actual effectiveness 

of the simulators. 

With regard to individual training, there seems to be a consistent finding that training in 

virtual simulations can be effective. Whether the objective is wayfinding and navigation, 

checkpoint operations, tactical positioning and maneuver, or squad leader decision making, there 

is ample objective evidence that trainee skills improve with practice in the simulation.

The results with regard to collective training are more ambiguous. Trainee ratings 

indicate that they can perform a wide variety of individual tasks in the simulators at a level of

“good” or better, which is a necessary but not sufficient condition for effective training. Trainee 

ratings have indicated that they believe that their performance improved as a result of training in 

the virtual simulators. Observer/Controller ratings, while limited, have indicated that leader/unit 

performance improved as a result of training in virtual simulators. 

Whether or not one concludes that immersive simulations are effective for collective 

training of dismounted Soldiers depends largely on the standard of proof employed. The “gold 

standard” would be an improvement in objectively measured performance in actual combat or 

live exercises, in controlled experiments so that the improvement can be shown to result from the 

use of the immersive simulation to the exclusion of other factors, and a sufficient number of 

units trained to insure confidence that the results do not result from chance. Unfortunately, there 

is no evidence that meets this standard. Conversely, there is no evidence that immersive 

simulation could not meet this standard. Such an evaluation has not been conducted. 

Without the evidence that a “gold standard” evaluation would provide, we must rely on 

research results that provide a lower standard of proof. Soldiers and small unit leaders who have 

participated in virtual simulation exercises have consistently reported that their skills have 

improved as a result of the training. There is ample evidence that Soldiers report that their skills 

have improved as a result of the use of immersive simulation, but there is little objective 

evidence (such as trainer ratings or mission success) to confirm this, or to show how the 

effectiveness of training in virtual simulations compares with other approaches. Those 

evaluations that have been conducted have also generally involved fewer Soldiers and units than 

would be required to establish statistical significance. 

The problem is that there has not been an organizational commitment to conduct such an 

evaluation. Largely because of resource constraints, particularly on the availability of Soldier 

trainees to participate in evaluations, relatively few virtual simulation programs for dismounted 

Soldiers have evaluated collective training effectiveness using objective measures of Soldier 

performance. For the same reason, virtual simulation has not generally been compared to other 

41
forms of training. Ideally, we would like to have more rigorous data. Realistically, this data does 

not exist for other training methods, either. 

In addition to the unresolved questions about the acquisition of skills in the simulators, 

there are some questions about the transfer of what is learned in the virtual simulators to the real 

world. Pleban and Salvetti (2003) found that practicing building clearing in a VE did not 

improve performance on the task in the real building. Statkus and Spring (2003) found 

differences in Soldier task performance between MOUT tasks performed in virtual simulation 

(SVSTM) and performance of the same tasks in a live simulation with simunitions. Soldiers in 

the SVSTM engaged targets more rapidly and fired fewer rounds than did Soldiers conducting the 

same exercises at the McKenna MOUT site. They attributed the differences in performance to a 

combination of environmental effects (lighting and temperature), load carried (BDU only vs. full 

fighting load), increased fatigue in the live simulation (as a result of the previous two factors), 

and the adverse consequence of being shot with a simunition. 

PC-based simulations, or PC games modified for training purposes, are receiving 

increasing attention. A number have been developed or modified for Army training. The 

general evaluation finding is that trainees respond positively to the game, but objective measures 

of training effectiveness are not obtained. An ARI workshop on the topic concluded that “while 

most researchers seem to agree that computer games can be effective training tools, there is very 

little empirical evidence to support that conclusion” (Belanich, Mullen, and Dressel, 2004). 

In addition to the question of effectiveness, there is also the question of whether potential 

Solider trainees, both enlisted and junior officer, possess the computer and game-playing skills 

that would enable them to master PC games used for training readily. There are no wide-scale 

survey data on this question, but data from experiments with relatively small numbers of 

participants suggests that 20-57% of junior officers and 7-22% of enlisted personnel do not play 

interactive games. This indicates a need for introductory practice prior to use of interactive 

games for training, but the existing data are very limited. 

Immersive simulations (or PC-based simulations) should not be used by themselves. 

They can provide a transition between initial or classroom training and live simulations. They 

appear to be best suited to training cognitive skills (not motor skills), decision making, 

communication, and teamwork. They require an instructor to provide a context and feedback. 

Desktop or laptop simulators should be considered as an alternative to fully-immersive 

simulators which use large rear-projection or HMD. Fully-immersive head-tracked displays are 

more effective, particularly when performing spatially-oriented tasks and acquiring spatial 

knowledge, but based on a limited number of experiments, this difference does not appear to be 

large, and may not justify the increased cost. 

Summary of Findings 

The major findings are summarized in the following sections, organized around the topics 

of training effectiveness, Soldier task performance, and advantages and disadvantages of 

immersive virtual simulations. 

42
What have we learned about training effectiveness? 

The following are based on data and observation from five assessments of immersive 

simulation systems involving 45 squad and fire team leaders and 94 squad members during the 

period 1999 – 2005. 

First, Soldiers and small unit leaders report that their skills improve as a result of training 

in virtual simulations. They report the most improvement in controlling, coordinating, 

communicating, and planning (conducted prior to use of the simulators), and less improvement 

in the mechanics of tasks and battle drills. 

Second, these self-reports by Soldiers have generally, if informally, been confirmed by 

observers. In one of the assessments objective ratings of unit performance by two independent 

raters were obtained. These ratings showed improvement in performance over the course of the 

training. 

These results are consistent with findings involving collective marksmanship trainers. 

Using different technology, but functionally similar to immersive virtual simulators, these 

trainers have produced objective data indicating that they provide effective training in tactical 

skills as well as marksmanship. 

It should be noted that these results were obtained despite a number of limitations. The 

total amount of time that trainees spent conducting training scenarios was actually relatively 

short (no more than three hours). In most cases, the software was still under development, and 

not fully problem free. The scenarios were usually not tailored to the skill level of the leader or 

unit. The trainer/AAR leader was frequently unfamiliar with the technology and did not 

necessarily use it to best advantage. 

What we have learned about what Soldier task performance? 

Fifty-one Soldiers and leaders participated in tactical exercises using immersive 

simulators (all variants of the immersive Soldier Visualization Station, or SVSTM) to conduct 

tactical exercises in assessments conducted during the period 2002 – 2005. While simulation 

technology in general and the SVSTM in particular were evolving, the capabilities were fairly 

static during that period. After Soldiers participated in the scenarios, they rated how well they 

could perform each of 54 Soldier activities in the simulators as Very Poor, Poor, Good, or Very 

Good. Overall, 30 of the 54 activities were rated Good or better; 18 were rated between the 

Good/Poor midpoint and Good; and 6 were rated Poor. Activities which were rated highly 

included outdoor movement, identification of types of people (civilians, non-combatants within a 

room, enemy soldiers), identification of tactically significant areas (sectors of observation and 

responsibility), and individual weapons use (but not grenades). Poorly rated items included 

maneuver indoors (close to others, past furniture, close to walls, around objects, past other 

personnel, around corners, through doorways, up and down stairs), and identifying the source 

and type of fire (enemy or friendly), either by auditory or visual cues. 

43
The issue of fine or precision movement in confined areas, such as movement indoors, 

has consistently emerged as the most important improvements required. It appears to be a more 

complicated problem than can be solved by building a new movement control device. It involves 

a number of related issues: the size and complexity of the “bounding box” which detects 

collisions between Soldiers and other objects, and between Soldiers; the limited visual texture 

and shading cues on the walls inside buildings; the relatively narrow FOV of the SVSTM display; 

the problem representing objects which are located between the Soldier and the rear-projection 

screen; and the linkage between direction of gaze and direction of movement. 

Precise movement in buildings is the primary example of the limitation on the physical 

actions that trainees could perform in the SVSTM simulator. Other simulators may differ in the 

specifics of the physical actions that can be performed. But no matter what the design, there are 

likely to be limitations, particularly with regard to locomotion and touch. For example, trainees 

in the SVSTM could drop to the ground, but they could not drop to the ground and roll. If they 

tried, they were likely to get tangled up in wires. They could not physically “throw” grenades. 

They selected a type of grenade from a menu, chose direction and launch angle by pointing their 

mock weapon, choose the amount of force desired by adjusting a slider with a pair of buttons on 

the weapon, and pulled the trigger to launch the grenade. While this bears little resemblance to 

the physical act of throwing a real grenade, it does provide practice in deciding which type of 

grenade to use, and where and when to employ it. They could not physically touch other people 

in the simulation, or objects other than their individual weapon and equipment. They 

consequently could not practice some aspects of room entry. 

These simulator constraints limit training effectiveness only if those actions that cannot 

be performed in the simulator are not trained by other means. For example, the SVSTM approach 

to grenade throwing is a problem only if Soldiers are not taught to throw grenades by other 

means. It is important that immersive virtual simulation be conducted as part of a planned 

sequence of instruction that provides them with the prerequisite skills prior to the simulation, and 

provides subsequent live training to improve the physical skills that cannot be trained in the 

virtual simulation. 

Even though the simulator may not allow Soldiers to perform many of the physical 

actions they need to perform in the way they need to perform them, it does allow them to 

practice and learn the cognitive skills they need. These include the planning that needs to be 

done prior to missions, the skills required to maintain situational awareness, and learning to think 

on their feet and make appropriate decisions in complex situations. These are all positive things 

that are one level of analysis above the physical actions Soldiers can or cannot do in the virtual 

environments with current technology. 

What are the advantages of immersive virtual simulations? 

Virtual simulations possess a number of advantages relative to the use of live simulation 

for dismounted Soldier training. These are described in the following paragraphs. 

Virtual simulation can represent any terrain or environment that has been modeled, 

whether a real location or a generic one. It is not limited to the physical area where it is located. 

44
Therefore, it can provide an urban training area with high rise buildings at a remote location, or a 

Middle Eastern village in a downtown armory. Environments can be selected based on training 

needs, rather than availability. 

Weather and other environmental conditions (rain, fog) can be represented. Nighttime 

missions can be trained during the day and daytime missions can be trained during the night. 

There is no need to wait for the right conditions. 

 

Use of immersive virtual simulation is likely to reduce the amount of time required by a 

unit to conduct a training exercise relative to live training. The following requirements should be 

reduced: transportation to and from the training site; drawing, cleaning, and returning weapons, 

ammunition, and gear such as MILES/OneTESS; positioning equipment and support personnel 

prior to the exercise; movement to the exercise start position; movement to the AAR area; and 

cleaning up the training site. 

Both instrumented live and virtual simulation require support personnel, and both require 

support personnel with technical skills to maintain and operate the equipment. However, the 

requirements for human role players fill the positions of enemy units, other friendly units, and 

civilians should be much smaller for virtual simulations. OneSAF can fill many of these roles 

(enemy units, crowds, etc.). Human role players can fill multiple roles (although not 

simultaneously) without regard to physical appearance or location in the environment. 

The risk of accident or injury is lower in virtual than in live simulation. Because of this, 

virtual simulations make it possible to more realistically depict the effects of demolitions (blow a 

hole in the building anywhere) or indirect fire than is possible in live simulations. 

 

While virtual simulations require improved physical facilities, they require less physical 

space than a live simulation facility. They also reduce the need for use of actual equipment. 

Virtual AARs provide unique capabilities for viewing events that are not currently 

possible with live simulations. First, they can provide an overall view of the action that is not 

possible with video cameras, which provide multiple separate views of portions of the action. 

Second, they can provide a view of the action from any perspective, not limited by camera 

locations. Third, they capture and provide an accurate representation of ground truth on which to 

base AARs so they can learn from their mistakes. Finally, they can provide a variety of aids to 

analyze and depict events (e.g., shot lines, movement paths, FOV, etc.). 

These capabilities make it possible for virtual immersive simulations to be used to 

provide the opportunity for small unit leaders and unit members to practice, in the context of 

realistic simulated operations, the cognitive and decision making skills that will foster 

adaptability and the capability to respond to rapidly changing situations. Focused, repetitive, 

deliberate practice, with feedback based on performance, is an effective method for training the 

recognition of situations and developing expertise. Cognitive and decision making skills can be 

trained even if some physical tasks cannot be performed in the situation. 

45
What are the disadvantages of immersive virtual simulations?

Virtual simulations possess a number of disadvantages relative to the use of live 

simulation for dismounted Soldier training. These are described in the following paragraphs. 

Some physical activities cannot be performed in the simulators, or must be performed 

differently in the simulators than in the real world. These currently include actions such as 

throwing grenades, or touching or grasping other participants or objects. There is a risk that 

Soldiers may learn inappropriate behaviors. This risk can be reduced by having trainers 

emphasize the differences between the simulation and the real world, and by providing additional 

live training on the affected tasks. 

Weapons effects may not be modeled correctly, or position and location tracking may not 

be accurate enough to depict realistic weapons effects. Soldiers may, therefore, develop 

inappropriate expectations. This disadvantage applies to live simulation as well. Any specially 

developed simulation system should model weapons effects, particularly individual Soldier 

weapons, accurately. Tracking accuracy may be a problem for fully immersive systems, and if 

so, would suggest that training scenarios not involve long range use of individual weapons by 

trainees. Inaccurate modeling may be a problem for use of commercial products. 

Virtual simulations require improved facilities (enclosed space, air conditioning, and 

power). They also require technical support to implement training scenarios and operate and 

maintain computers and other equipment. 

Trainees may not take virtual simulations seriously because they are similar to video 

games, and might, therefore, practice actions that are inappropriate in the real world, or attempt 

to “game” the system. This risk can be reduced by trainer emphasis. 

Some trainees may experience “simulator sickness.” This is unlikely to be a major 

problem. Risks can be reduced by making initial exposures to the simulators relatively short, and 

gradually increasing the duration. 

46
Recommendations 

Conduct a large-scale evaluation using current technology. 

As discussed above, the evidence that immersive simulation can provide effective 

training is not sufficient to justify the immediate acquisition of such systems on a large scale. 

However, there is enough evidence that they can be effective to justify acquisition of a prototype 

system using current technology and use it to conduct a rigorous training effectiveness 

evaluation which would permit quantification of the effectiveness of the training and comparison 

with alternatives. While this would be costly, in terms of both dollars and Soldier time, it would 

lead to a more informed acquisition decision. 

This evaluation should consider objective measurement of both skill improvement on the 

simulator and transfer of those skills to a live simulation environment. It should involve a 

sufficient number of units (probably squads) that any meaningful real differences in the training 

effectiveness of different can be detected. The evaluation should be embedded in the unit’s 

normal training progression, and unit personnel should be involved in its development and 

delivery. 

Consider cost-effectiveness of fully immersive and desktop or laptop systems. 

The cost differential between fully-immersive simulators using large projection displays 

or HMDs and simulators using desktop or laptop computers to provide the same functionality 

can be enormous. Knerr (2006), analyzed cost data provided by Loftin, Dryer, Belfore, Petty, 

Phillips, Garcia, Seevi, Lusso, Mastaglio, and Park (2004), and estimated the difference at over 

$75,000 per simulator. While the cost difference is likely to decrease as the cost of large visual 

displays decreases, the fact remains that the major cost drivers for fully immersive simulators are 

the interface devices, particularly the position and orientation trackers and visual display 

systems. The computers are relatively cheap. The interface devices also increase space and 

support requirements. 

In contrast, the research evidence indicates that any difference in training effectiveness between 

immersive and desktop systems is likely to be small. Immersive systems are more effective, 

particularly when training spatially-oriented tasks, such as learning a complex route or floor 

plan. Whether immersive simulators are also better when training squads to conduct urban or 

counter-insurgency operations, and if so, whether the difference is large enough to justify the 

increased cost, are unknown, but should be investigated, perhaps in conjunction with the largescale evaluation recommended above. 

Conduct a survey of potential officer and enlisted trainees to determine game-playing skill. 

One of the assumptions underlying the growing popularity of the use of video and PCbased games for training is that the potential trainees, if not avid game players, are at least 

interested in and experienced with such games. They, therefore, need little training in their use, 

and are motivated to play. However, data from small samples of Soldiers who participated in 

evaluations suggests that this may not be the case. While a fairly large percentage of male 

47
enlisted personnel, possibly 75% or higher, play computer or video games weekly or more 

frequently, officers appear to play less frequently. In addition, based on civilian data, we would 

expect that female Soldiers would play less frequently than male Soldiers. We know little about 

the types of games that Soldiers play (e.g., first person shooter vs. role player). Since game

playing experience has implications for the types of games that could be used for training and the 

amount and type of prerequisite game playing training required, we would like to have a much 

better picture of the both current and projected Soldier and leader familiarity and experience with 

different types of interactive games. 

Continue to improve virtual simulation capabilities and assess their training effectiveness. 

Some aspects of immersive virtual simulation technology, such as large-screen displays, 

will continue to improve, or become less expensive, as a result of commercial market forces. 

However, it will require a continuing science and technology investment by the Army to produce 

a tailored system that provides cost-effective training. 

48
References 

Beal, S.A. (2005). Using Games for Training Dismounted Light Infantry Leaders: Emergent 

Questions and Lessons Learned (RR 1841). Arlington, VA: U.S. Army Research Institute 

for the Behavioral and Social Sciences. 

Beal, S.A. and Christ, R.R. (2004). Training Effectiveness of the Full Spectrum Command Game 

(TR 1140). Arlington, VA: U.S. Army Research Institute for the Behavioral and Social 

Sciences. (ADA419670) 

Beal, S.A. and Christ, R.R. (2005). Soldier Perceptions of the Rapid Decision Trainer (RR1835). 

Arlington, VA: U.S. Army Research Institute for the Behavioral and Social Sciences. 

(ADA436993) 

Belanich, J., Mullen, L.N., and Dressel, J.D. (2004). Symposium on PC-based Simulation and 

Gaming for Military Training (RP 2005-1). Arlington, VA: U.S. Army Research Institute 

for the Behavioral and Social Sciences. (ADA415918) 

Belanich, J., Sibley, D.E., and Orvis, K.L. (2004). Instructional Characteristics and Motivational 

Features of a PC-Based Game (ARI RR 1822). Alexandria, VA: U.S. Army Research 

Institute for the Behavioral and Social Sciences.

Bliss, J.P., Tidwell, P.D., and Guest, M.A. (1997). The effectiveness of virtual reality for 

administering spatial navigation training to firefighters. Presence: Teleoperators and 

Virtual Environments, 6(1), 73-86. 

Boldovici, J.A., Bessemer, D.W., and Bolton, A.E. (2002). The Elements of Training Evaluation.

Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social Sciences. 

Campbell, C.H., Knerr, B.W., and Lampton, D.R. (2004). Virtual Environments for Infantry 

Soldiers (ARI Special Report 59). Alexandria, VA: U.S. Army Research Institute for the 

Behavioral and Social Sciences. 

Caretta, T.R., and Dunlap, R.D. (1998). Transfer of Training Effectiveness in Flight Simulation:

1986 – 1997 (AFRL-HE-AZ-TR-1998-0078). Mesa, AZ: U.S. Air Force Research 

Laboratory. 

DeBrine, J.D., & Morrow, D.E. (2000). Re-Purposing Commercial Entertainment Software for 

Military Use. (Master’s Thesis). Monterey CA: Naval Postgraduate School. 

Department of the Army. (8 June 2004). Operational Requirements Document for the Soldier 

Combined Arms Tactical Trainer. 

Department of the Army. (2005). Army Strategic Planning Guidance 2005.

49
Diller, D.E., Roberts, B., Blankenship, S. & Nielsen, D. (2004, December). DARWARS 

Ambush! – Authoring Lessons Learned in a Training Game. Proceedings of the 2004 

Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC). 

Orlando, FL. 

Eisley, M.E., Hagman, J.D., Ashworth, R.L., and Viner, M.P. (1990). Training Effectiveness 

Evaluation of the Squad Engagement Training System (SETS) (RR 1562). Alexandria, 

VA: U.S. Army Research Institute for the Behavioral and Social Sciences. 

Goldberg, S. L., & Knerr, B. W. (1997). Collective training in virtual environments: Exploring 

performance requirements for dismounted soldier simulation. In R.J. Seidel & P.R. 

Chatelier (Eds.), Virtual reality, training’s future? New York: Plenum Press. 

Gorman, P. F. (1990). Supertroop via I-Port: Distributed simulation technology for combat 

development and training development (IDA Paper P-2374). Alexandria, VA: Institute 

for Defense Analysis. 

Grant, S.C., & Magee, L.E. (1997, December). Navigation in a virtual environment using a 

walking interface. Proceedings of the NATO RSG-18 Workshop on Capability of Virtual 

Reality to Meet Military Requirements. Orlando, FL. 

Hayes, R.T. (2005). The effectiveness of instructional games: A literature review and discussion

(TR 2005-004). Orlando, FL: Naval Air Warfare Center Training Systems Division.

Hayes. R.T., Jacobs, J.W., Prince, C., and Salas, E. (1992). Requirements for future research in 

flight simulation training: Guidance based on a meta-analytic review. The International 

Journal of Aviation Psychology. 2(2), 143-158. 

Jacobs, R. S., Crooks, W. H., Crooks, J. R., Colburn, E., Fraser, II, R. E., Gorman, P. F., 

Madden, J. L., Furness, III, T. A., & Tice, S. E. (October, 1994). Behavioral 

Requirements for Training and Rehearsal in Virtual Environments. (TR-1011), U.S. 

Army Research Institute for the Behavioral and Social Sciences, Alexandria, VA. 

(ADA286311) 

Jacquet, C.R. (2002). A Study of the Effects of Virtual Reality on the Retention of Training.

Unpublished Master’s Thesis, University of Central Florida. Orlando, FL. (ADA401640) 

Kennedy, R. S., Lane, N. E., Berbaum, K. S., & Lilienthal, M.G., (1993). A simulator sickness 

questionnaire (SSQ): A new method for quantifying simulator sickness. International

Journal of Aviation Psychology. 3(3), 203-220. 

Knerr, B.W. (2006, June). Current Issues in the Use of Virtual Simulations for Dismounted 

Soldier Training. Paper presented at the NATO Human Factors & Medicine Panel 

Workshop on “Virtual Media for Military Applications”. West Point, NY. 

50
Knerr, B.W., Garrity, P.J., and Lampton, D.R. (2004, December). Embedded training for future 

force warriors: An assessment of wearable virtual simulators. Proceedings of the 24th

Army Science Conference. Orlando, FL. (ADA433376) 

Knerr, B.W., and Lampton, D.R. (2005). An Assessment of the Virtual-Integrated MOUT 

Training System (V-IMTS) (TR 1163). Alexandria, VA: U.S. Army Research Institute for 

the Behavioral and Social Sciences. (ADA438351) 

Knerr, B.W., Lampton, D.R., Crowell, H.P., Thomas, M.A., Comer, B.D., Grosse, J.R., Centric, 

J.H., Garfield, K.A., Martin, G.A., & Washburn, D.A. (2002) Virtual environments for 

dismounted soldier simulation: Results of the FY 2001 Culminating event (ARI Technical 

Report 1129). Alexandria, VA: U.S. Army Research Institute for the Behavioral and 

Social Sciences. 

Knerr, B.W., Lampton, D.R., Thomas, M., Comer, B.D., Grosse, J.R.., Centric, J., 

Blankenbeckler, P., Dlubac, M., Wampler, R.L., Siddon, D., Garfield, K., Martin, G.A., 

& Washburn, D.A. (2003). Virtual environments for dismounted soldier simulation, 

training, and mission rehearsal: Results of the FY 2002 culminating event (ARI 

Technical Report 1138). Alexandria, VA: U.S. Army Research Institute for the 

Behavioral and Social Sciences. (ADA417360) 

Kolasinski, E.M. (1995). Simulator sickness in virtual environments (Technical Report 1027). 

Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social Sciences. 

(ADA295861) 

Lathrop, W.B., and Kaiser, M.K. (2005). Acquiring Spatial Knowledge while Traveling Simple 

and Complex Paths with Immersive and Non-immersive Interfaces. Presence, 14 (3).

249-263. 

Levison, W. H., & Pew, R. W. (1993). Use of virtual environment training technology for 

individual combat simulation. (Technical Report 971). Alexandria, VA: U.S. Army

Research Institute for the Behavioral and Social Sciences. (ADA263546) 

Lind, J.H. (1995). Perceived Usefulness of the Team Tactical Engagement Simulator (TTES): A 

Second Look (NPS-OR-95-005). Monterey, CA: Naval Postgraduate School. 

Lind, J.H. and Adams, S.R. (1994). Team Tactical Engagement Simulator (TTES): Perceived 

Training Value (NAWCWPNS TM 7724). China Lake, CA: Naval Air Warfare Center 

Weapons Division. 

Lockheed Martin Corporation. (1997). Dismounted warrior network front end analysis 

experiments (Advanced Distributed Simulation Technology II, Dismounted Warrior 

Network DO #0020, CDRL AB06, ADST-II-CDRL-DWN-9700392A). Orlando, FL: US 

Army Simulation, Training and Instrumentation Command. (ADA344365) 

51
Lockheed Martin Corporation. (1998). Dismounted warrior network enhancements for restricted 

terrain (Advanced Distributed Simulation Technology II, Dismounted Warrior Network 

DO #0055, CDRL AB01, ADST-II-CDRL-DWN-9800258A). Orlando, FL: US Army 

Simulation, Training and Instrumentation Command. (ADA370504) 

Loftin, R.B., Dryer, D., Belfore, L., Petty, M., Phillips, M., Garcia, H., Seevi, J., Lusso, A., 

Mastaglio, T., and Park, N. (2004). Soldier CATT front end analysis: Results of 

Technology Assessment. Portsmouth, VA: MYMIC LLC. 

Loftin, R.B., Scerbo, M.W., McKenzie, R., Catanzaro, J.M., Bailey, N.R., Phillips, M.A., and 

Perry, G. (2004). Training in peacekeeping operations using virtual environments. Paper 

presented at the RTO HFM Symposium on “Advanced Technologies for Military 

Training.” (ADA428142) 

Miller, C. and Zeisset, M. (2002). Engagement Skills Trainer Post-Fielding Training 

Effectiveness Analysis (PFTEA) (TRAC-WSMR-PFTEA-02-025). White Sands Missile 

Range, NM: TRADOC Analysis Center – White Sands Missile Range. 

Nolan., J.M., and Jones, J.M. (2005). Games for Training: Leveraging Commercial off the Shelf 

Multiplayer Gaming Software for Infantry Squad Collective Training (Masters Thesis).

Monterey, CA: Naval Postgraduate School. 

Orlansky, J., Dahlman, C.J., Hammond, C.P., Metzko, J., Taylor, H.L., and Youngblut C. (1994). 

The Value of Simulation for Training (IDA Paper P-2982). Alexandria, VA: Institute for 

Defense Analysis. 

Pleban, R. J., Dyer, J. L. Salter, M. S., & Brown, J. B. (1998). Functional capabilities of four 

virtual individual combatant (VIC) simulator technologies: An independent assessment

(Technical Report No. 1078). Alexandria, VA: U.S. Army Research Institute for the 

Behavioral and Social Sciences. (ADA343575)

Pleban, R.J., Eakin, D.E., and Salter, M.S. (2000) Analysis of Mission-Based Scenarios for 

Training Soldiers and Small Unit Leaders in Virtual Environments (RR 1754). 

Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social Sciences). 

Pleban, R.J., Eakin, D.E., Salter, M.S., and Matthews, M.D. (2001). Training and Assessment of 

Decision-Making Skills in Virtual Environments (RR 1767). Alexandria, VA: U.S. Army 

Research Institute for the Behavioral and Social Sciences. 

Pleban, R.J., and Salvetti, J. (2003). Using Virtual Environments for Conducting Small Unit 

Dismounted Mission Rehearsals (RR 1806). Alexandria, VA: U.S. Army Research 

Institute for the Behavioral and Social Sciences.

Roberts, D.F., Foehr, U.G., and Rideout, V. (2005). Generation M: Media in the lives of 8 – 18 

year-olds (Report No. 7251). Menlo Park, CA: The Henry J. Kaiser Family Foundation. 

52
Salter, M. S., Eakin, D. E., & Knerr, B. W. (1999). Dismounted warrior network enhancements 

for restricted terrain (DWN ERT): An independent assessment (Research Report No. 

1742). Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social 

Sciences. (ADA364607 

Singer, M.J., Allen, R.C., McDonald, D.P., & Gildea, J.P. (1997). Terrain appreciation in virtual 

environments: Spatial knowledge acquisition (Technical Report 1056). Alexandria, VA: 

U.S. Army Research Institute for the Behavioral and Social Sciences. (ADA325520) 

Simpson, H. (1999). Evaluating Large-Scale Training Simulations: Volume 1: Reference Manual

(DMDC Technical Report 99-05). Seaside, CA: Defense Manpower Data Center. 

Statkus, M.J., and Spring, R. (2003). Human Science/Modeling and Analysis Data: Comparision 

of Results from the Squad Synthetic Environment and McKenna Military Operations in 

Urban Terrain (MOUT) Site Studies (Natick/TR03/019L). Natick, MA: U.S. Army

Soldier and Biological Chemical Command Solider Systems Center. 

Tate, D.L., Siebert, L., and King, T. (1997). Virtual environments for shipboard firefighting 

training. Proceedings of IEEE Virtual Reality Annual International Symposium (VRAIS 

’97), 61-68. 

U.S. Army Training and Doctrine Command (2003). Objective Force Embedded Training 

(OFET) Users’ Functional Description (TRADOC Pamphlet 350-37). Fort Monroe, VA: 

U.S. Army Training and Doctrine Command, 

"virtual reality." The Columbia Electronic Encyclopedia, Sixth Edition. Columbia University 

Press., 2003. Answers.com 13 Sep. 2006. http://www.answers.com/topic/virtual-reality

Waller, D., Hunt, E., and Knapp, D. (1998). The transfer of spatial knowledge in virtual 

environment training. Presence: Teleoperators and Virtual Environments, 7(2), 129-143. 

Witmer, B. G., Bailey, J. H., & Knerr, B. W. (1994). Training dismounted soldiers in virtual 

environments: Route learning and transfer (Technical Report 1022). Alexandria, VA: 

US Army Research Institute for the Behavioral and Social Sciences.

53
Appendix A 

List of Acronyms

AAR After Action Review 

ARI U.S. Army Research Institute for the Behavioral and Social 

Sciences 

ARL U.S. Army Research Laboratory 

ARTEP Army Training and Evaluation Program

CATT Combined Army Tactical Trainer 

CAVE CAVE Automatic Virtual Environment 

CE Culminating Event

CPT Combat Proficiency Training 

COTS Commercial off-the-Shelf

DARPA Defense Advanced Projects Agency

DARWARS DARPA Training Superiority Program 

DCSOPS&T Deputy chief of Staff for Operations and Training 

DI Dismounted Infantry

DIS Distributed Interactive Simulation

DISAF Dismounted Infantry Semi-Automated Forces 

DIVAARS Dismounted Infantry Virtual After Action Review System

DoD Department of Defense 

DTIC Defense Technical Information Center 

DWN Dismounted Warrior Network

DWN ERT Dismounted Warrior Network Enhancements for Restricted 

Terrain 

EST Engagement Skills Trainer 

FOV Field of View 

FSC Full Spectrum Command

HLA High Level Architecture

HMD Head Mounted Display or Helmet Mounted Display 

ICCC Infantry Captains Career Course 

IOBC Infantry Officer Basic Course 

I-Port Individual Portal

LED Light Emitting Diode 

LFX Live Fire Exercise 

L-V-C Live – Virtual - Constructive 

MILES Multiple Integrated Laser Engagement System

MOUT Military Operations in Urban Terrain 

MPR Mission Planning and Rehearsal 

MTP Mission Training Plan 

ODT Omni-Directional Treadmill

OEF Operation Enduring Freedom

OIF Operation Iraqi Freedom

One SAF One Semi-Automated Forces 

OneTESS One Tactical Engagement Simulation System

OPTEMPO Operational Tempo

A - 1
ORD Operational Requirements Document

PEO STRI Program Executive Office for Simulation, Training, and 

Instrumentation 

PC Personal Computer

R&D Research and Development 

RDECOM STTC Research, Development, and Engineering Command Simulation, 

and Training Technology Center 

RBD Reality by Design 

RDT Rapid Decision Trainer 

RSTA Reconnaissance, Surveillance, and Target Acquisition 

SASO Support and Sustainment Operations 

SAVE Situational Awareness in Virtual Environments 

SBL Soldier Battle Lab 

SETS Squad Engagement Training System

SME Subject Matter Expert 

SOF Special Operations Forces 

SSE Squad Synthetic Environment 

SSQ Simulator Sickness Questionnaire 

STO Science and Technology Objective 

STRICOM US Army Simulation, Training and Instrumentation Command 

SVSTM Soldier Visualization Station 

SVSD Desktop SVSTM simulator 

SVSI Immersive SVSTM simulator 

TOE Table of Organization and Equipment 

TPIO TRADOC Program Integration Office 

TRADOC U.S. Army Training and Doctrine Command 

TTES Team Tactical Engagement Simulator 

UO Urban Operations

USAARMC U.S. Army Armor Center 

USAIC U.S. Army Infantry Center 

USAIS U.S. Army Infantry School 

USJFKSWCS U.S. John F. Kennedy Special Warfare Center and School 

VE Virtual Environment

V-IMTS Virtual Integrated MOUT Training System

VICS Virtual Individual Combatant Simulator 

VR Virtual Reality 

A - 2