U.S. Army Research Institute 

for the Behavioral and Social Sciences 

Research Report 1788 

 COLLECTIVE STAFF TRAINING IN A VIRTUAL 

LEARNING ENVIRONMENT 

William R. Sanders 

U.S. Army Research Institute 

March 2002 

i
i

REPORT DOCUMENTATION PAGE

1. REPORT DATE (dd-mm-yy) 2. REPORT TYPE

Final 

3. DATES COVERED (from. . . to) 

May 2001 to August 2001 

4. TITLE AND SUBTITLE 5a. CONTRACT OR GRANT NUMBER

Collective Staff Training In A Virtual Learning Environment 

5b. PROGRAM ELEMENT NUMBER 

0602785A

5c. PROJECT NUMBER

A790

5d. TASK NUMBER

211

6. AUTHOR(S) 

William R. Sanders (U.S. Army Research Institute for the Behavioral 

and Social Sciences) 

5e. WORK UNIT NUMBER

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 

U.S. Army Research Institute for the Behavioral and Social Sciences 

ATTN: TAPC-ARI-IK 

2423 Morande Street 

Fort Knox, KY 40121-5620

8. PERFORMING ORGANIZATION REPORT NUMBER

10. MONITOR ACRONYM 

ARI

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 

U.S. Army Research Institute for the Behavioral and Social Sciences 

ATTN: TAPC-ARI-IK 

5001 Eisenhower Avenue 

Alexandria, VA 22333-5600

11. MONITOR REPORT NUMBER 

12. DISTRIBUTION/AVAILABILITY STATEMENT 

Approved for public release; distribution is unlimited.

13. SUPPLEMENTARY NOTES 

14. ABSTRACT (Maximum 200 words): 

As the Army transitions to modern digital command and control technology it faces a major challenge in designing webdelivered training to support the acquisition, retention, and transfer of collective staff skills required to apply these advanced 

technology capabilities to real-world tasks. This report identifies training program design features based on principles of 

cognitive psychology that can be incorporated into Virtual Learning Environment (VLE) collective training. A prototype 

Cognitive Training Techniques Checklist was developed and applied in a review of U.S. Army Armor School Virtual Tactical 

Operational Center (VTOC) collective training. From this review candidate training program design features were identified 

that can support VLE training requirements. The examination of VTOC training also provided insights for Train-the-Trainer 

products, and a description of potential future C2 system operational capabilities and challenges. The research represents a 

logical extension and continuation of previous U. S. Army Research Institute for the Behavioral and Social Sciences cognitive 

skills training, and distance learning research conducted to assist the U.S. Army Armor School in assessing the individual selfpaced phase of training for a new Armor Captains Career Course – Distance Learning (AC3 DL) offered primarily as web-based 

instruction. 

15. SUBJECT TERMS 

Cognitive Training Techniques Collective Training C4I Training 

Virtual Learning Environments Distance Learning Future Combat Systems

SECURITY CLASSIFICATION OF 

16. REPORT 

Unclassified

17. ABSTRACT 

Unclassified

18. THIS PAGE 

Unclassified

19. LIMITATION OF 

ABSTRACT 

Unlimited 

20. NUMBER 

OF PAGES 

21. RESPONSIBLE PERSON

(Name and Telephone Number) 

Dr. James W. Lussier 

DSN 464-3450

NSN 7540-01-280-5500 Standard Form 298 
ii
Research Report 1788 

 COLLECTIVE STAFF TRAINING IN A VIRTUAL 

LEARNING ENVIRONMENT 

William R. Sanders 

U.S. Army Research Institute 

for the Behavioral and Social Sciences 

Armored Forces Research Unit

Barbara A. Black, Chief 

U.S. Army Research Institute for the Behavioral and Social Sciences 

5001 Eisenhower Avenue, Alexandria, Virginia 22333-5600 

March 2002 

Army Project Number Personnel Performance and 

20262785A790 Training Technology 

Approved for public release; distribution is unlimited 

iii
iv
FOREWORD 

The Future Battlefield Conditions (FBC) Team of the Armored Forces Research Unit 

(AFRU), U.S. Army Research Institute for the Behavioral and Social Sciences (ARI) has a work 

package (211) FUTURETRAIN: Techniques and Tools for Command, Control, 

Communications, Computers, Intelligence, Surveillance, and Reconnaissance (C4ISR) Training 

of Future Brigade Combat Team Commanders and Staffs. Recent work under this work package 

has involved research and development concerning collective training for digital staffs. 

Rapid advances in computing power have resulted in an unprecedented capability to 

bring geographically dispersed students together in Virtual Learning Environments (VLEs). As 

the Army transitions to the new web-based training technologies, research is needed to ensure 

that VLE training program design features exploit the full capabilities of these training systems. 

The present research represents a logical extension and continuation of previous ARI distance 

learning research conducted in partnership with the U.S. Army Armor School (USAARMS). 

This research extends the application of cognitive principles to collective skills training design to

fully realize the training potential of VLE training technologies. 

The information provided in this report is valuable to the USAARMS and other 

organizations involved in developing VLE collective training for leaders and staffs. The 

cognitive psychology principles for collective VLE training in this report are applicable to a wide 

range of current and future systems. The results of this research were briefed to representatives 

of the USAARMS on October 31, 2001. 

 

 MICHAEL G. RUMSEY 

 Acting Technical Director 

v
ACKNOWLEDGEMENTS 

The author wishes to acknowledge the essential contributions made by personnel in the 

U.S. Army Armor School (USAARMS) who supported this research. Much of the value of the 

present research comes from documenting the detailed comments and recommendations 

provided by the Armor Captains’ Career Course Distance Learning trainers and students. The 

present research required observations and data collection from Virtual Tactical Operation 

Center (VTOC) class exercises. Mr. Ron Offutt, Mr. Pete Borosky, and Mr. Jim Harrison of AB 

Technologies Incorporated provided essential information regarding the effectiveness of the 

VTOC training techniques, and the challenges facing the Army in developing the next generation 

of VTOC instructors. Dr. Connie Wardell served as the senior USAARMS representative for the 

present research, and provided valuable information regarding the issues underlying the 

development of distance learning programs. 

vi
COLLECTIVE STAFF TRAINING IN A VIRTUAL LEARNING ENVIRONMENT 

EXECUTIVE SUMMARY 

Research Requirement: 

As the Army transitions to modern digital command and control (C2) technology it faces 

a major challenge in designing training for the collective staff skills required to apply these 

advanced technology capabilities to real-world tasks. Web-based Virtual Learning Environment 

(VLE) training technologies offer great promise as collective training tools. However, to fully 

exploit the potential of VLE collective training, research is needed to identify the principles of 

cognitive psychology that support collective skills, to identify how these cognitive principles can 

be incorporated as specific VLE collective training design features, and to identify potential 

Train-the-Trainer products. The investigation of VLE collective staff training should also 

provide early insights into future C2 system operational capabilities and challenges. 

Procedure: 

As a first step, cognitive psychology principles and training techniques identified in 

previous research were organized into a checklist tool to provide a framework for identifying 

specific techniques currently implemented in VLE training. Next, a representative VLE training 

system was selected where insights from instructors, students, and previous research could be 

gathered. Interviews with VLE instructors were conducted, and training sessions were observed. 

Particular attention was paid to identify training program requirements driven by the transition 

from classroom to VLE training. The review of current VLE training issues also identified 

potential operational capabilities and challenges for future distributed digital C2 systems. 

Findings: 

Several products were generated from the present research. A prototype Cognitive 

Training Techniques Checklist was developed and used to review a current VLE collective 

training program. From this review candidate Training Program Design Features were identified 

that can support VLE training requirements. The VLE training program review also provided 

insights for future Train-the-Trainer products, and a description of potential future C2 system

operational capabilities and challenges. 

 

Utilization of Findings: 

 

The results of this research should support Army efforts to design VLE collective training 

for a broad range of current and future systems. The prototype Cognitive Training Techniques 

Checklist, and Candidate Training Program Design Features product demonstrate how the 

principles and techniques of cognitive psychology can be utilized in real world VLE training 

design. Insights gained into future C2 system operational capabilities and challenges should be 

of value to managers involved in future systems acquisition and development. 

vii
viii
COLLECTIVE STAFF TRAINING IN A VIRTUAL LEARNING ENVIRONMENT 

CONTENTS

 

 Page 

INTRODUCTION ........................................................................................................................ 1 

 Requirement...................................................................................................................... 1 

 Background....................................................................................................................... 2 

 Research Approach ........................................................................................................... 4 

VIRTUAL LEARNING ENVIRONMENT TRAINING ISSUES IDENTIFICATION.............. 4 

An Exemplar Virtual Learning Environment ................................................................... 4 

Training Issues for Collective Staff Training ................................................................... 6 

VTOC Trainer Interview................................................................................................... 6 

VTOC Training Observations .......................................................................................... 7 

Assessment of Initial Delivery of the Armor Captains’ Career Course (DL)................... 8 

Commander’s Survey: Armor Captains’ Career Course (DL) ........................................10 

Suggested VLE Research Directions................................................................................11 

Summary of VTOC Related Training Issues....................................................................12 

COGNITIVE PRINCIPLES AND TRAINING TECHNIQUES .................................................13 

Cognitive Theory Support for Training ............................................................................13 

Cognitive Training Techniques Checklist ........................................................................13 

 Candidate Training Program Design Features..................................................................15 

TRAINING THE TRAINER........................................................................................................16 

Train-the-Trainer Issues....................................................................................................16 

Skill Requirements for VTOC Trainers............................................................................17 

Train-the-Trainer Products................................................................................................18 

FUTURE DISTRIBUTED DIGITAL C2 OPERATIONAL CAPABILITIES............................19 

Future Operational Capabilities........................................................................................19 

Leadership Expertise Reach Back ....................................................................................19 

Sanctuary Off-Battlefield Manpower Sites.......................................................................20 

Synchronization of Assets Across Digitally Networked Units.........................................20 

Fostering Innovation ........................................................................................................21

Distributed Training..........................................................................................................21 

Multifunctional Staff.........................................................................................................22 

Training for Degraded Communications..........................................................................22 

ix
CONTENTS (Continued) 

 Page 

 Digital Media Induced Performance Problems.................................................................23 

SUMMARY AND DISCUSSION ..............................................................................................23 

Training Program Design Features for VLE Training......................................................23 

Train-the-Trainer Issues....................................................................................................23 

Future Digital C2 Systems Operational Capabilities and Challenges ..............................24 

REFERENCES .............................................................................................................................25 

APPENDIX A. List of Acronyms.......................................................................................... A-1 

B. Issues From VTOC Small Group Instructor Interview................................ B-1 

C. Issues From Observation of VTOC Training............................................... C-1 

D. Issues From Initial Delivery of AC3 DL Course Student 

 Interview and Survey ................................................................................ D-1 

E. Issues From Initial Delivery of AC3 DL Course SGI Interview.................. E-1 

 F. Cognitive Training Techniques Checklist .................................................... F-1 

LIST OF TABLES 

Table 1. VTOC Trainer Issues..................................................................................................... 6 

 2. VTOC Training Observation Issues............................................................................... 7 

 3. AC3 DL Student Issues.................................................................................................. 9 

 4. AC3 DL Small Group Instructor Issues.........................................................................10 

 5. VTOC Issues: Commanders’ AC3 DL Survey.............................................................11 

 6. Suggested Research Directions......................................................................................12 

 7. Cognitive Training Techniques Checklist .....................................................................14 

 8. Candidate Training Program Design Features...............................................................15 

x
CONTENTS (Continued) 

 Page 

LIST OF FIGURES

Figure 1. VTOC computer screen home page ............................................................................. 5 

xi
COLLECTIVE STAFF TRAINING IN A VIRTUAL LEARNING ENVIRONMENT 

Introduction 

Requirement 

It has been estimated that in the 21st century, half of all learning will be conducted on the 

Internet, and when online learning is combined with more interactive and facilitative in-person 

learning, it could easily outperform today’s one-size-fits-all traditional lecture delivery system

(Draves, 2000). With the rapid advance of computer hardware and software capabilities, a great 

deal of effort is being devoted to the development of web-based training to replace or 

supplement the traditional exposition-application classroom lecture education. Researchers 

caution us, however, that the training potential of emerging technologies stems not simply from

technologically advanced capabilities, but from the ability to vary instructional methods and 

media systematically according to the cognitive demands of learning tasks (Hooper & Hannafin, 

1991). Research is needed to ensure that the development of sound teaching techniques for 

employing web-based Distance Learning training technology does not lag far behind the pace of

the development of the technology. For the present research, Distance Learning (DL) will be 

defined as “planned learning that normally occurs in a different place from teaching and as a 

result requires special techniques for course design, special instructional techniques, special 

methods of communication by electronic and other technology, as well as special organizational 

and administrative arrangements.” (Moore & Kearsley, 1996, p. 2) 

The Army has made the decision to employ the Internet as a central means of delivering 

training. Army training developers have stated that for learners who cannot attend full length 

resident training, a collective DL environment is needed for battle staff training to capture the 

peer interaction and the creative energy present in real small group interactions. These training 

developers view DL technology as offering unexpected capabilities, and actively support efforts 

to identify learning principles and training techniques that can be incorporated into current and 

future DL applications, stating that “At this point in DL development we need to set future 

targets, not current, for technology, to be a risk taker or you will never get beyond current 

capabilities” (C. Wardell, personal communication, March 28, 2001). 

While advances in computer technology have led to the large scale introduction of webbased training applications, many opportunities exist to enhance this training. The Army has 

developed learning environments where individual web-based learning is combined with 

collective learning experiences that harness the power of performance modeling, peer tutoring, 

and peer motivation to achieve success in training collective battle staff skills. The present 

research seeks to identify principles of cognitive psychology that support DL collective skills 

training, to identify how these cognitive principles can be incorporated as specific DL collective 

training design features, and to identify potential Train-the-Trainer products. The investigation 

of DL collective staff training should also provide early insights into future digital command and 

control (C2) system operational capabilities and challenges. 

1
Background

 

Overview. The present research represents an extension of previous individual staff skills 

work to address requirements for collective staff skills training in a Virtual Learning 

Environment (VLE). This research must first consider collective staff skill requirements, as 

effective training should derive from applying established cognitive principles to learning 

requirements, rather than from a focus on VLE technological solutions. In particular, the central 

role of the trainer in employing these cognitive approaches to staff skills training needs to be 

identified. Research on staff skills training must also be future oriented to ensure that training 

solutions will be available to meet the anticipated demands of a future digitally networked force. 

Related staff skills research efforts. The U.S. Army Research Institute for the Behavioral 

and Social Sciences (ARI), Armored Forces Research Unit (AFRU) at Fort Knox has recently 

conducted research to assist the U.S. Army Armor School in assessing the individual self-paced 

phase of training for a new Armor Captains’ Career Course – Distance Learning (AC3 DL) 

offered primarily as web-based instruction (Sanders & Burnside, 2001; Sanders & Guyer, 2001). 

This research gathered information from AC3 DL instructors and students to identify strengths of 

the training program, and also areas of training that could be improved. In a related effort, 

research was conducted to identify opportunities to apply cognitive psychology principles to the 

development of training techniques for computer delivered individual training to enhance digital 

system skill acquisition, retention, and transfer (Sanders, 2001). The cognitive training 

techniques derived in this research were incorporated into the development of prototype 

individual procedural skills training modules (Deatz & Campbell, 2001). 

Virtual learning environments. Early training applications on the Internet tended to 

divide the traditional method of teaching into two parts – cognitive learning, which could be 

accomplished with online learning; and affective learning, which could be accomplished better in 

a small group discussion setting (Draves, 2000). Here, cognitive learning includes facts, data, 

knowledge, and mental skills, while affective learning involves emotions and social skills 

essential in team building. Advances in technology have now permitted the development of 

VLEs where both cognitive and affective learning can take place. The VLEs have been 

described as learning management software systems that synthesize the functionality of 

computer-mediated communications software (e.g., shared text and graphics documents, e-mail, 

voice, bulletin boards, and information reference resources such as maps and manuals) and online methods of delivering course materials (e.g., the World Wide Web). The VLEs are intended 

not simply to reproduce the classroom environment “online,” but to use the technology to 

provide learners with new tools to facilitate their learning. They aim to accommodate a wider 

range of learning styles and goals, and to encourage collaborative learning (Britain & Liber, 

2000). 

Cognitive principles for collective training. Cognitive psychology learning paradigms 

include learning theories (e.g., behaviorism, cognitivism, and constructivism), which consider 

the nature of individual learning as well as how knowledge is constructed in social situations. 

For behaviorism, learning reflects a new behavioral pattern being repeated until it becomes 

automatic, while cognitivism views learning in terms of the thought process behind the behavior. 

Constructivism is distinguished by its premise that learners construct their own perspective of the 

2
world, based on individual experiences and schema (Wilhelmsen, Asmul, & Meistad, 1998). 

The constructivist theories of learning are particularly appropriate for collective skills training in 

VLEs as this learning approach leverages the power of performance modeling, peer tutoring, and 

peer motivation as tools to enhance learning. The review of theories of learning can provide the 

basis for identifying theory-driven technology-based approaches to meeting training 

requirements (Lajoie, 2000). The need exists to identify how cognitive principles and 

pedagogical design can be applied to meet battle staff collective training goals. Cognitive 

principles and techniques that would apply to VLE collective training have recently been 

documented by Sanders (2001) and Lussier, Ross, and Mayes (2000). 

Train-the-Trainer. The “Train-the-Trainer” requirement associated with the introduction 

of the DL training approach needs to be identified and documented as an essential contributor to

the success of the training system. An essential goal of any training system is that it be robust, 

and scalable as the demand for classes increases. A key concern that drives the “Train-theTrainer” requirement is the potential for a shortage of instructors as DL courses proliferate and 

the instructor demand increases in turn. The development of instructor training, establishment 

of student performance standards, and development of reliable student performance assessment 

tools are essential to ensure consistent training across instructors for a robust training program. 

The introduction of advanced DL training technology, and the requirements for instructors to 

perform as role model and mentor, leveraging a growing set of VLE technology capabilities, can 

create a significant Train-the-Trainer requirement. 

Future distributed digital C2. One unanticipated benefit of collective battlestaff VLE 

training is that it can provide an early look at the operational capabilities and challenges facing 

the Army as it moves to distributed battalion C2 digitized systems. The future of Army 

commander and staff mission planning and execution tasks bears a strong resemblance to the 

VLE training of today, where the geographically dispersed instructor (commander) and students 

(staff) work collectively to develop planning products. The Army is undergoing a transformation 

to a rapidly deployable yet fully capable force built around Future Combat Systems (FCS) 

internetted combined arms operations. It is anticipated that the FCS will offset platform

limitations in the close fight with internetted combined arms with “point and shoot” fire support. 

While commanders and staff have traditionally been co-located for mission planning and 

execution, the transformation to FCS will likely require that geographically dispersed leaders and 

staff communicate and coordinate their collective actions rapidly across digital C2 links. This 

ability to plan and execute actions across digital C2 links will be essential to the successful 

employment of Networked Fires, Robotics, Organic Three-Dimensional Targeting and Mobile 

C2 enabling technologies (St. Onge, 2000). 

 Distributed training. It is likely that in the future training capabilities will be required to 

support Distributed Training (DT) requirements. For the present research, DT will be defined as 

training that can be delivered electronically across geographically dispersed leaders and 

subordinates to support unit skill sustainment training requirements. While DL and VLEs 

typically are an extension of the schoolhouse that supports initial skill acquisition, DT will 

leverage the capabilities of modern training technology to meet the requirement for leaders to 

conduct individual and collective sustainment training, whether in garrison or deployed. The 

investigation of collective battlestaff training in VLEs can provide insights into the training 

3
requirements associated with future C2 systems, particularly with respect to the commander’s 

role and support requirements as primary trainer for his battlestaff. One challenge will be to 

identify training system designs, to include trainer aids and automated performance assessment 

and feedback tools, that can support sustainment and proficiency training within and between 

digitally equipped forces in the future. 

Research Approach 

The research approach basically follows the sequence of topics identified in the 

Background Section. Cognitive psychology principles and training techniques identified in 

previous research were used as a framework to identify specific techniques that are currently 

implemented in VLE training, and opportunities to incorporate additional techniques into 

training. As a first step, training issues documented in previous research that specifically address 

VLE training issues were reviewed and summarized. Next, a representative DL VLE training 

system was selected where insights from instructors and students could be gathered. Interviews 

with VLE instructors were conducted, and training sessions were observed. Particular attention 

was paid to identify Train-the-Trainer requirements driven by the transition from classroom to 

VLE training. The review of current VLE training issues provided a valuable opportunity to 

identify potential operational capabilities and challenges for future distributed digital C2 

systems. Definitions for the acronyms used in this report are provided in Appendix A. 

 Virtual Learning Environment Training Issues Identification 

 

An Exemplar Virtual Learning Environment 

The Virtual Tactical Operations Center (VTOC) located at the U. S. Army Armor School 

(USAARMS), Fort Knox, was selected as an exemplar VLE training system for the purpose of 

identifying and documenting cognitive training techniques applicable to a representative Army

VLE. The VTOC is a virtual reality environment representing a battalion–level tactical 

operations (TOC) center, where geographically dispersed National Guard students and 

instructors participating in AC3 DL training come together over the Internet for real-time 

collective staff training. The VTOC allows for online collective task performance training 

through audio, text, and graphic overlay tools to accomplish battalion-level commander and staff 

tasks. Tasks include using the Military Decision Making Process, developing the Operations 

Order, preparing a Warning Order, and preparing a Mission Analysis Brief. 

The VTOC training is presented in Phase 1 of the AC3 DL program. In Phase 1 students 

complete a year-long course of study based on individual (Asynchronous) Internet-based selfstudy modules addressing brigade (BDE) and battalion (BN) battle staff skills, and Company 

Team operations. Seven weekend VTOC collective (Synchronous) training sessions are 

distributed across the Phase 1 training program to provide students with an opportunity to 

practice applying this new knowledge. While the individual delivering VTOC instruction is 

formally referred to as the Small Group Instructor (SGI) he will be referred to here as a “trainer” 

to highlight the fact that he does not lecture or provide instruction in VTOC, instead acting as a 

mentor for the students. Students who successfully complete Phase 1 training complete the 

course with two weeks of Resident Training (Phase 2) at Fort Knox. The AC3 DL replaces the 

4
previous Armor Officer Advanced Course – Reserve Component (AOAC RC) self-study bookbased correspondence course. The AC3 DL program is the National Guard’s alternative to the 

traditional 18 week Armor Captains’ Career Course – (AC3) taught for Active Component (AC) 

officers as a resident course at Fort Knox. 

Principles of cognitive psychology were incorporated into the pedagogical design of 

VTOC training delivery from the outset. The new VTOC course design incorporates elements of 

the constructivist approach to instruction, which places the learner in an active role in which 

knowledge is discovered in interaction with the environment (AB Technologies Inc., 1998, July). 

Using the constructivist model, the VTOC classroom environment should closely represent the 

conventional battalion TOC task environment. The students are actively involved in the learning 

process, constructing their own understanding of how knowledge can be used. The trainer 

intervenes only when the students are no longer able to make progress. The after action review 

(AAR) is a key component of this constructivist learning approach (Ross & Yoder, 1999). 

Figure 1 presents the home page of the VTOC computer screen interface. The large 

window at the upper left side of the screen provides a virtual reality environment where students 

and instructors move about as avatars in a conventional battalion TOC, communicating via voice 

and text chat, accessing maps, manuals, orders, and preparing staff products by working 

individually or collectively on templated documents and graphic overlays. The large window at 

the upper right can display various data, to include templated documents for collective editing. 

Specific interface features available in VTOC have been identified by Offutt and Borosky 

(2000). A typical VTOC training session would involve a trainer and 8-12 students taking on the 

roles of a battalion task force commander and staff. Training is conducted one weekend each 

month for students, with each class participating in approximately seven training sessions. 

Additional information describing the AC3 DL VTOC is available at their web site (U.S. Army

Armor School, 2001). 

Figure 1. VTOC computer screen home page (source: U.S. Army Armor School, 2001). 

5
Training Issues for Collective Staff Training 

Issues for collective staff training were identified from several sources. These issues 

s-on

ed,

TOC Trainer Interview

The ARI staff interviewed the senior VTOC trainer, also referred to as the senior SGI, 

rimari

e was

ified

able 1

TOC Trainer Issues

___________________________________________________________

raining Program Design Features: 

were used to identify features of the VTOC VLE training program design that might be 

incorporated to enhance VLE training in general. Interviews were conducted with VTOC

trainers, VTOC training sessions were observed, and software was obtained to allow a hand

exploration of the VTOC environment. Issues were also identified from a review of previous 

AC3 DL research which included data from structured interviews with AC3 DL students and 

trainers, and student surveys. A brief description of each data collection effort will be present

annotated with a summary of training issues derived from each source. Based on this review of 

training issues, and the stated research goals, three categories were chosen for use in presenting 

and reviewing the training issues: “Training Program Design Features,” “Train-the-Trainer

Requirements,” and “Implications for Future C2 Operations.” 

V

p ly to investigate VTOC Train-the-Trainer requirements, using a structured interview 

guide. Previous research (Sanders, 2001) had noted that there might be a shortage of trainers

available for current VTOC classes, while the demand for DL throughout the Army was 

expected to increase dramatically. Facing this increasing demand for DL trainers the issu

raised of how best to meet this need through a Train-the-Trainer program. Topics of discussion

included the trainer role and activities in VTOC class sessions, availability of written materials 

such as a trainer’s handbook, and recommendations for additional materials that could be 

developed that would help new trainers deliver VTOC training. A selection of issues ident

from this interview are presented in Table 1. The full listing of issues identified in the interview

is presented in Appendix B. 

T

V

___________________

T

re is no formal recorded assessment of student leadership 

• of each weekend

• written VTOC 

in

Table Continues 

• Performance Assessment: The

skills, or individual student performance during VTOC training. 

Structured Training: Group AAR is informal, conducted at the end

training session with teaching points taken from the SGI’s notes. 

Performance Feedback: Students are not individually rated on their

products. Formal evaluation of important VTOC-trained competencies occurs later,

the two-week resident phase of instruction formal evaluation. Scores from this later 

evaluation do go into the student’s record. 

6
Table 1 (Continued)

Train-the-Trainer Requirements: 

e senior VTOC trainer stated that trainers will need very little 

r

• r knowledge is presented in the

•

Implications for Future C2 Operations:

• Training Requirement: Th

VTOC-specific training. The senior VTOC trainer stated that instructors selected as 

VTOC trainers will already know how to train battle staff competencies based on thei

previous Army training and leadership experience. 

Training Process: Instruction to convey subject matte

Asynchronous portion of the course. Using the VTOC, the trainer guides collective 

experiential learning by mentoring students and monitoring their progress. 

Instructor Support: There is no VTOC trainer handbook. 

 

s: VTOC performance assessment and feedback tools 

______________________________________________________________________________ 

TOC Training Observations

ARI staff observed VTOC Class #3 weekend training sessions, primarily to investigate 

able 2

TOC Training Observation Issues 

________________________________________________

raining Program Design Features: 

• Future Commander Training Skill

are not available to serve as a basis for building training aids to assist a future commander

who serves as primary trainer for his staff. The current VTOC approach is to hire retired 

officers as VTOC instructors, and not develop Train-the-Trainer programs. 

V

VTOC Train-the-Trainer requirements, using a structured data collection guide. A selection of 

issues identified during these observations are presented in Table 2. A full listing of issues 

identified during the training observation is provided in Appendix C. 

T

V

______________________________

T

e introduced during a VTOC session to require students to 

•

u

• remembering where text and graphic 

• that trainees start sending email to each other a week 

Table Continues 

• Skill Flexibility: New tasks ar

adapt to change quickly. Students were given additional tasks that were not expected 

(e.g., one-page additional commander’s guidance posted on VTOC message board). 

Advance Organizer: The trainer described the link between past, present, and future 

work at the beginning of VTOC training: “Expand on what we did last time today, yo

will see this again in Resident phase training.” 

Digital Media Skills: Some students had trouble

digital files from previous VTOC session were stored, and which files represented the 

most recently updated version. 

Skill Rehearsal: Student stated

before VTOC training to review what they did in the previous session. 

7
Table 2 (Continued)

Train-the-Trainer Requirements: 

ormally crosswalking training design: course goals, training 

• discussion, and absent students

e

l

• Feedback: The VTOC trainer role plays the BDE and BN Commander 

at

ctor’s

plications for Future C2 Operations:

• Training Goals: Consider f

methods, performance assessment, and feedback. 

Performance Modeling: There were long pauses in

during the Mission Brief to the commander. It was stated that students can hide in th

VTOC training, compared to a real life briefing where the battlestaff is present in the 

briefing room. Consider presenting a video clip demonstration early in lesson to mode

the desired briefing format and protocol, establishing the collective performance 

standards. 

Performance

(CDR), BN Executive Officer (XO) and Operations Officer (S3). The trainer stated th

his instructor role combines art and science. The “Science” is the multi-page task 

checklist used to outline task requirements for the students. The “Art” is the instru

ability to assess whether the student’s plan is feasible. There are no set solutions, so the 

instructors must be prepared to assess a broad range of plan options. 

Im

class started staff tasks while some students were still 

cs

• Product Preferences and Expectations: Students used a variety of available 

•

• appeared that sometimes student individual staff products were 

______________________________________________________________________________ 

ssessment of Initial Delivery of the Armor Captains’ Career Course (DL) 

Data originally collected to assess the self-paced portion of a prototype training course 

were re

uested

C3

• Degraded Communications: The

not logged on, and some logged on but with degraded communications. Students were 

told to inform others when they came online. Unplanned loss and return of students 

occurred due to communications problems, but students quickly adjusted using 

alternative communication lines (email, cell phones, commercial text and graphi

software). 

Commercial

communication resources not provided in VTOC to accomplish tasks: Commercial text 

and graphics software, public email, cell phones. Students may expect high functionality

from Army digital systems, at the same level as commercial hardware/software systems. 

Digital Media Design: Students cannot file all the information that they would like to 

file. “I wish I could store information like the previously developed Operations Order 

and refer back to it.” 

Digital Media Risks: It

not collectively reviewed by all staff members before being briefed, raising concerns 

about internal coordination of collectively developed digital documents. 

A

viewed for the present research to identify issues related to the VTOC-delivered 

collective staff training portion of the course. In June 1999, the Director USAARMS req

the ARI AFRU provide assessment support to the USAARMS review of the AC3 DL. The ARI

researchers worked with USAARMS to develop an interview protocol for both students and 

instructors, and a multiple-choice survey for students to identify training issues for the first A

DL class to complete training. A 12-item AC3 DL Resident Course Survey was developed to 

8
identify how well elements of the program meet training needs. A 16-item AC3 DL Student 

Interview Guide was also prepared to identify Army policies and incentives that could impact

RC officer participation in the AC3 DL training program. Both the survey and interview were

administered to students during the 2-week resident portion of the AC3 DL program (Sanders &

Burnside, 2001). 

selection of VTOC-related issues identified by students in the survey and questionnaire 

is prese

ed by

able 3

C3 DL Student Issues

__________________________________________________________

aining Program Design Features: 

A

nted in Table 3. One active duty SGI experienced as a VTOC trainer participated in the 

detailed interview addressing specific strengths and weaknesses of the AC3 DL course, to 

include issues associated with VTOC training. A selection of VTOC-related issues identifi

the SGI is presented in Table 4. The full list of VTOC-related issues identified by students in the

AC3 DL Resident Course Survey and Student Interview appears in Appendix D. The complete 

list of VTOC-related SGI Interview issues appears in Appendix E. 

T

A

____________________

 

Tr

Students stated that AC3 DL is superior to AOAC RC as 

• the Army can't have RC company commanders 

Train-the-Trainer Requirements:

• Peer Motivation and Tutoring:

it includes peer motivation and peer tutoring as part of the learning process, and better 

prepares students for resident training. 

Part-task Training: Students stated that

doing (weekend-long) Synchronous training instead of drill, that so many students 

dropped out that the class lost key positions. 

 

re one-on-one feedback from trainers on student progress. 

plications for Future C2 Operations:

• Trainer Feedback: Need mo

• Performance Evaluation: Students stated that they need a detailed AAR at end of the 

VTOC training sessions. 

Im 

udents stated that it would be better if they could work 

______________________________________________________________________________ 

• Centralized vs. Distributed C2: St

together in co-located small groups for VTOC training. This suggests that face-to-face 

staff work may have performance advantages over distributed C2. 

9
Table 4

C3 DL Small Group Instructor Issues 

_____________________________________________

raining Program Design Features: 

A

_________________________________

T

experience students are more likely to make a decision. 

• he role of XOs or S3 which involves supervising, 

• des peer motivation, peer tutoring as part of 

• tudents teamed in VTOC portion of training arrive at the 2-week 

rain-the-Trainer Requirements: 

• Decision Making: With VTOC

• Communication Skills: Students have more confidence, experience speaking in front of 

peers, students know SGI better. 

Supervising Skills: Students play t

controlling, and directing other students in subordinate staff roles. Students say they 

need to learn to lead, and to be a follower. 

Teaching/Counseling Skills: AC3 DL inclu

the learning process. 

Team Development: S

resident training phase ready to perform as a team. 

T

VTOC Instructor must identify students with stronger 

S3) 

• s students

• t evaluated. 

plications for Future C2 Operations:

• Student Skills Assessment:

leadership skills early in training, and put them in more demanding positions (e.g.,

first, then rotate with other students. Assessment of student strength comes through 

Asynchronous Phase performance, and dialogue between SGI and student. 

Behavior Modeling: The SGI plays the role of the CDR and/or XO, and walk

through wargaming the Course Of Action analysis. First the SGI shows the students 

what to do, then appoints students to play the role of XO or S3 which does involve 

supervising, controlling, and directing students in subordinate staff roles. 

Performance Assessment: Individual student VTOC written products are no

Im

egraded communications training is essential for future 

• m various staff roles during planned

.

______________________________________________________________________________ 

ommander’s Survey: Armor Captains’ Career Course (DL) 

Data originally collected to address AC3 DL course policy issues were reviewed for the 

present

o

with

• Use of Available C2 Systems: D

digital C2 operations, particularly developing the staff mindset that workarounds can be 

found. The VTOC students demonstrated the ability to adapt “on the fly” to 

communications problems after having had lots of experience with degraded 

communications during their VTOC training. 

Flexible Staff Roles: Students learned to perfor

rotations through staff assignments. Students also got experience at having to quickly

assume staff functions while filling in for staff members who were temporarily off-line

C

research to identify issues specifically related to the VTOC collective staff training. In 

June 2000, the Director USAARMS requested that the ARI AFRU provide assessment support t

the USAARMS review of the AC3 DL course. The ARI researchers worked with training 

developers and instructors to construct a survey for both students and unit leaders involved

10
AC3 DL. The survey requested demographic information, and presented 14 open-response 

questions regarding policy issues that might impact student participation in the training (San

& Guyer, 2001). In responding to the survey 45% of students and 32% of unit leaders indicated 

that AC3 DL is a good course, and described it as an improvement over the previous paper-based

correspondence course. It was noted that the DL format does not lead to a reduction in the 

requirement for instructors, as instructors are essential to on-line training success for DL 

students. The full set of VTOC-related issues identified by students and unit leaders are 

presented in Table 5. 

ders

able 5

TOC Issues: Commanders’ AC3 DL Survey 

_______________________________________

raining Program Design Features: 

T

V

_______________________________________

T

time interactions with others.

o use unfamiliar VTOCrain-the-Trainer Requirements: 

• Peer Tutoring: Need more real-

• Digital Skills Mental Model: Students reported that learning t

specific C2 interface software wasted a lot of training time. 

T

at does not lead to a reduction in the requirement for 

• e

plications for Future C2 Operations:

• Trainer Role: The DL form

instructors, as instructors are essential to on-line training success for DL students.

Trainer Target Audience Skills: Consider integrating Active Component and Activ

Guard Reserve instructors to counter shortage in authorized instructor personnel. 

Im 

rs will expect that the C2 interface will operate with the 

______________________________________________________________________________ 

uggested VLE Research Directions 

The issues identified from ARI’s AC3 DL training program research were presented to 

the AR

oping

• C2 Interface: Future staff membe

same functionality and reliability as commercial products. 

S

I Director (Chief Psychologist of the Army), on August 8, 2001 at ARI Fort Knox to 

gather guidance and suggestions for VLE research. The briefings noted that as the Army 

transitions to new digital communications technologies, it faces a major challenge in devel

integrated training programs that support the collective leader and staff planning and decision 

making skills required to exploit the capabilities of these systems. It was noted that previous 

research by Abell (2000) has identified the importance of including both whole-task, and parttask presentations in training. The ARI Director suggested that research could be conducted to

investigate several cognitive training techniques to address the identified VLE training issues. 

The suggested research directions are summarized in Table 6. 

11
Table 6

uggested Research Directions

___________________________________________________

• Training Design Checklist: Translate the cognitive principles and methods for training 

• l cognitive technique is the use of a part-task 

hat

• formance should be presented 

• VLE AAR needs to include teaching points to make it 

te

______________________________________________________________________________ 

ummary of VTOC Related Training Issues

Issues for VTOC collective staff training were identified from interviews with VTOC 

instruct

of

e.

an

ith regard to Train-the-Trainer requirements, VTOC training delivery currently relies 

heavily

nical

ng staff

ed,

S

___________________________

into a checklist, using clear behavioral terms, and use this to systematically review a 

representative VLE training program. 

Part-task Training: An additional usefu

approach to focus training emphasis on difficult and problematic tasks. It was noted t

VTOC training takes an analogue training approach, where each element of a 

conventional task is played out sequentially in real time. 

Performance Standards: Clear behavioral standards for per

early in collective DL exercises. 

Performance Demonstrations: The

a learning event. It was noted that Reserve Component students will not have the 

experience and background that their AC instructors do. Trainers might demonstra

performance standards with tools such as realistic video demonstrations. 

S

ors, and the observation of VTOC training sessions. Previous AC3 DL research which

included VTOC-related data from structured interviews with AC3 DL students and SGIs, and 

student surveys was also reviewed for training issues identification. The review suggests that a

number of constructivist training techniques are currently employed in VTOC training. Staff 

officer subject matter knowledge training is delivered in the self-paced Asynchronous portion

the AC3 DL course. In VTOC training the trainer does not lecture, but instead monitors, 

mentors, and provides guidance to students, and thus needs the skills to perform in this rol

VTOC training might benefit from incorporating additional cognitive techniques supporting 

performance assessment and feedback, and the introduction of part-task training exercises as

alternative to the current single vignette per weekend approach. 

W

on recruiting retired military officers with the requisite skills, rather than implementing a

training program to develop these required skills in future instructors. The concern exists that as 

VTOC-type VLE training increases there will be a greater need to develop instructor skills 

through training. The review of VTOC training revealed the fortuitous situation where tech

problems in VTOC communications led the students to become proficient at finding 

workarounds to continue their staff tasks. It appears that there is great benefit in havi

members develop a flexible mindset for dealing with degraded communications. This 

experience with degraded modes of communications might be an essential, but overlook

element of digital staff training for future distributed digital C2 systems. 

12
Cognitive Principles and Training Techniques 

ognitive Theory Support for Training 

 Constructivist psychology principles can address many of the training design needs for 

VLEs.

ing

n

onstructivist theory proposes that knowledge is individually constructed and socially 

co-cons g

cial

attle

or the present research it was important to consider the role of the trainer in employing 

constru

The

ing

ognitive Training Techniques Checklist 

The interviews and surveys conducted with students and VTOC trainers served to 

identify tential

nd the

into

ues.

prototype checklist, annotated with VTOC training observations, appears in Appendix F. 

C

 

For the present research a checklist was developed to demonstrate how training 

programs can be reviewed to identify principles of cognitive psychology and related train

techniques that have been incorporated into the design of training. The checklist was based on

previous ARI research which identified a number of constructivist psychology principles that ca

be applied to VLE staff training to enhance skill retention (Sanders, 2001). 

C

tructed by learners based on their interpretations of experiences in the world. Followin

this principle, training should consist of experiences in realistic task environments that facilitate 

knowledge construction (Jonassen, 1999). Constructivist theory lends itself to ill-defined tasks, 

and seeks to address higher level cognitive tasks, particularly those involving collective task 

performance. Constructivist theory suggests that VLEs should provide an environment for so

interaction, social negotiation, and collaboration, important skills for collective commander and 

battle staff planning and decision making tasks. Thus it might have great utility in providing a 

training approach for battalion staff collective C2 tasks in an environment that contains a 

fluctuating mixture of both digital and conventional message traffic. For one example of b

staff training, Lussier, et al. (2000) have developed training to facilitate "adaptive thinking" in 

response to unanticipated circumstances; this training combines the constructivist principles of

coaching and scaffolding with the behaviorist technique of deliberate practice. 

F

ctivist training techniques. Employing the "cognitive apprenticeship" instructional 

approach, students will model skills demonstrated by an expert trainer in a realistic setting.

sequence of feedback in coaching would start with a high degree of external support or 

"scaffolding" for beginning learners and move to a gradual reduction of support as learn

progresses, an instructional technique referred to as "fading" (Samarapungavan, Beishuizen,

Brazier & Sanders, 1993). 

C

the strengths of VTOC training particularly in leveraging the powerful training po

of peer motivation and peer tutoring. These data gathering efforts also identified issues 

regarding the absence of systematic individual and collective performance assessment, a

reliance on a single training approach involving a 2-day real-time staff planning task. The 

framework for constructivist principles previously presented in Sanders (2001), was revised

checklist format using descriptive behavioral terms, and used to review VTOC training features. 

This example serves to illustrate how constructivist principles are currently implemented in 

VTOC training, and identifies further opportunities to leverage constructivist training techniq

A sample of a prototype constructivist principles checklist is provided as Table 7. The full 

13
Table 7 

Cognitive Training Techniques Checklist 

SYSTEM 

 

DATE RATER 

 

Integrated in

Training? 

Cognitive Principle 

Yes No NA

Rater Comments/Suggestions 

1. Use context-based training to 

build situated cognition of tasks. 

 

 1a. Knowledge to be trained 

should be presented in a realistic 

context. 

 

 1b. The training problem should 

be ill-defined, ill-structured, to 

stimulate original thinking. 

 

2. Training should emphasize 

learner control and the ability to 

manipulate information in the task 

environment. 

 

 2a. Learner should have multiple 

ways (interface options) to access 

information. 

 

 2b. Learner should be able to 

access manipulate multiple types of 

information (e.g., text, voice, 

graphics). 

 

3. Training should provide multiple 

modes and perspectives for 

representing instructional content. 

 

 3a. Training should provide 

learner selectable interface media, 

individualizable to meet learner 

preferences.

 

 3b. Team training should 

provide for cross-training of team

member roles.

 

14
Candidate Training Program Design Features

niques identified in the Cognitive Training 

echniques Checklist require that training program design features be in place to ensure their 

success. For example, scaffolding techniques require that trainers are able to make reliable 

assessments of relevant aspects of individual students performance over a period of time, using 

these performance assessments to either sustain, or fade the amounts of trainer assistance to each 

i e incorporation of hn aining 

programs have established clear training desired performance identified in behavioral 

terms that will allow program developers to as whether program goals are being met, or not. 

T ng de gn f ture that uld be introduced into VTOC 

t of cognitive training techniques, and enhance training 

effectiveness. 

T

Candidate Training Program Design Features 

Training Design Features 

Many of the cognitive training tech

T

ndividual student. Th cognitive tec

goals with

iques is also facilitated where tr

sess

able 8 presents a number of traini

raining to support the incorporation

si ea s co

able 8

Training Needs and Issue Source 

1. The VTOC training develope

rosswalk stated training goals to training 

rs need to Build a training goals and techniques matrix. 

Establish performance criteria for training 

pr gram ffectiveness assessment. Collect 

em ance data to support 

training techniques effectiveness assessment. 

c

methods, and student performance 

assessment. 

Issue Source: Table 4 

o

pirical student perform

e

2. The VTOC should provide a bro

f experience f

ader range

or learners, greater staff role

s .

0

Develop a variety of part-task vignettes as an 

alternative to the current weekend-long 

training scenario. Use video teamwork 

examples to make up for lack of visual 

examples of teamwork in VTOC. 

o

rotation. Focus practice on difficult ta

Issue Source: Table 7, and Abell (200

ks

)

3. Provide “Train-the-Trainer” experience in 

nd fe bac

Develop digital video VTOC screen capture 

examples of student performance that new 

trainers can practice evaluating. Establish 

observable student performance criteria that 

can be reliably scored across trainers. 

student performance assessment a

to enhance performance assessment 

consistency across trainers. 

Issue Source: Table 2 

ed k

4. Formal documentation of student s

is needed in the event that substitute t

must take over the class. The stude

s needed to make student staff role

t gth

ra er 

nt record

Make a record of student progress that can be 

accesse y a substitute teacher. Employ a 

structured rating sheet similar to the “cutsheet” used by some instructors in Resident 

phase training, and update student assessments 

to support student staff role assignments. 

ren

in

s

i

assignments. 

 Issue Source: Table 2 

d b

Table Continues

15
Table 8 (Continued) 

5. Th ing goals.

ion

e VTOC training developers need to

clearly identify the leadership behaviors to be 

trained. Develop methods to demonstrate the 

desired behaviors. 

Issue Source: Table 2. 

Identify key leadership behavior train

Develop methods to demonstrate desired 

behaviors (e.g., decision making, informat

organization, task allocation) such as digital 

video files, graphic or text descriptions. 

6. The VTOC training needs to include a 

means of demonstrating the execution of the 

staff plan. 

Issue Source: Future VTOC capability. 

Incorporate a tactical simulation game in 

VTOC for plan execution and assessment. 

Identify evaluation criteria and opportunities 

for automated performance assessment. 

7. The VTOC training needs a structured 

AAR, with a re

Structured AAR product should leverage 

cord of student performance 

TOC training weekends to support

program effectiveness assessment. 

available VTOC multimedia to identify what to

sustain, and what to improve. Record AAR 

issues for review at next VTOC lesson. 

across V

training

Issue Source: Table 7. 

Traini

rain-the-Trainer Issues

ng th

T

t of Train-the-Trainer req

s s

w p

delivery, and which will allow for training progra L

P d wi ersus

A , 2001 ess of

V g

o s

 

ent associa

approach needs to be identified and documented a

t T

f a

b m e

i sis

previously, a key concern that drives the Train-th

shortage of instructors as VLE courses proliferate

VTOC training program e

ecision to employ retired officers as instructors, and to consider using Reserve C

fficers as instructors.

An essential goal of any training system is that it be robust, and scalable as the demand 

r classes increases. The training system must be able to adapt on occasions when an individual 

structor is not available to teach. It is hazardous to follow a practice where essential student 

e Trainer

The establishmen uirements is an essential component of large 

cale training system design. Train-the-Trainer i

ill be very valuable in identifying a training ap

cte

sues identified from AC3 DL VTOC training 

roach to ensure consistency in training 

m assessment. The assessment of AC3 D

th an empirical comparison of AC3 DL v

). A similar evaluation of the effectiven

hase 1 Asynchronous training was condu

OAC RC test scores (Sanders & Burnside

TOC training, quite useful in justifying fundin

f standardized and validated performance asses

The Train-the-Trainer requirem

requests, would first require the development

ment tools. 

ted with the introduction of the VLE training

s an essential contributor to the success of the

raining system. The introduction of advanced V OC training technology, and the requi

or instructors to perform as role model, mentor,

urden. Train-the-Trainer issues identified fro

n identifying a training approach to ensure con

rements

nd trainer, can create a significant training 

AC3 DL VTOC training will be very valuabl

tency in training delivery. As noted

e-Trainer requirement is the potential for a 

and the instructor demand increases in turn. 

Even with the small scale , a shortage of instructors has resulted in th

d

o

omponent 

fo

in

16
performance information for an entire class resides only in one instructor’s memory. Formal 

udent performance assessment, establishment of student performance standards, and 

d ial to

a g

b s t

not be known if instructors, or instructors-in-train s

i il st

r such ve

noted frustration and difficulty when encounterin

p

instructors. While the perceived change in perfor

R als

i s ha

i

v er in

l

kill Requirements for VTOC Trainers 

The role of the VTOC trainer is evolving. At

g how products were produced. Now instructors monitor, mentor, and 

he students produce the staff planning products. It is assumed that the 

tudent

s

d

phase formal evaluation which 

es go

rviews VTOC trainers were asked to describe the training that new 

nstruct

st

evelopment of instructor training are essent

ssessment across instructors for a robust trainin

ackgrounds hold students to different standard

ensure consistent performance standards and 

program. If instructors from different 

hen training will also be inconsistent. It will 

ing, use the same reliable standards unless thi

s checked through estimates of inter-rater reliab ity against a common class session, or again

epeatable examples of student performance

rogress from AC3 DL VTOC civilian (retired m

as video or audio recordings. Students ha

g different performance standards as they 

ilitary) to AC3 DL Resident Phase AC 

mance standards could reflect a Crawl-Walkun progression, the different standards might 

ndividual instructors. Experienced instructor

nsights from the limited (audio, text) information available in VTOC. These insights should be 

alidated, so that they can be shared with oth

o reflect the personal decisions of the 

ve learned how to gather student performance 

structors, and integrated into Train-the-Trainer 

essons.

S

 first the VTOC trainer provided instruction, 

lecturing and demonstratin

provide guidance, letting t

s s get the subject matter knowledge instruction they need from the Asynchronous selfstudy portion of the course. The AC3 DL Course Design Document identifies a number of 

student competencies as training goals for the total course which suggest areas where instructor

will need training skills: 1) Decision Making, 2) Planning, 3) Communicating, 4) Technical an

tactical proficiency, 5) Use of available systems, 6) Supervising, 7) Teaching/counseling, 

modeling behaviors, and 8) Soldier team development (AB Technologies Inc., 1988, July). 

While these student competencies are developed in VTOC training, no student performance 

record is kept. Formal evaluation of VTOC trained competencies occurs in follow-on 

Asynchronous training individual projects, and in the Resident

do into the student’s record.

The role of VTOC trainer is tied in with that of the Resident phase SGI. The current 

VTOC trainers are retired military, and they benefit from the Active Duty Captain resident phase 

instructors’ current knowledge of doctrine and detailed knowledge of company team operations.

VTOC trainers will typically manage 60-80 students progressing through the Asynchronous selfstudy material, and two VTOC classes with 8-12 students in each. The VTOC classes are 

conducted on weekends (all day Saturday, half-day Sunday) so that a VTOC trainer will devote

two weekends each month to VTOC training, and devote weekdays to managing Asynchronous

training. 

In structured inte

i ors should receive to prepare for VTOC. The VTOC trainers stated that they have been 

doing collective leader training for years as active duty military to develop these competencies in 

students and subordinates, and that they do not need additional training to perform this role for 

17
the VTOC. They stated that the AC3 DL Asynchronous and VTOC phase trainer needs to be

Army Lieutenant Colonel (LTC) retiree, with experience as a BN XO or S3, have comp

Command and General Staff College (CGSC) training, and be Armor branch qualified. It was 

stated that the candidate trainer must have the personal disposition needed for VTOC inte

with students, and must present the proper image of a leader. Most of the individuals in this 

selection group have been Army instructors. Potential VTOC trainers must first complete the 

AC3 DL Asynchronous course before serving as a trainer. The current VTOC trainers are retired

LTCs, have experience as BN XOs, have completed CGSC, are Armor branch qualified, have 

passed the interview for proper image, and are experienced Fort Knox Armor Captain Cours

instructors. 

Currently there is not a VTOC Trainer Handbook, but in the VTOC there is an 

instructional guide for students that new trainers can use for orientation. The VTOC trainers firs

attend a 1-week Instructor Training Course, then the standard Army 2-week SGI course. 

Normally the new VTOC trainer will attend two VTOC training sessions getting guidance from

the trainer

an

leted

ractions

e

t

before teaching a class on their own. One VTOC SGI interviewed had assisted an 

structor in three VTOC sessions before serving as the primary trainer, and stated that new 

t just watch or “right seat” the VTOC course. The new 

TOC trainers also go through the Asynchronous phase of the AC3 DL course as additional 

training

While 

ccur

e

e introduction of VTOC-specific Train-the-Trainer products. 

al

e on

entioned previously in Table 8, some of the goals of having the new trainer attend a 

TOC class might be achieved efficiently by presenting multimedia examples of both desirable 

and und the

in

trainers really need this, to assist and no

V

. No formal practice is conducted for new trainers to observe and evaluate individual 

student performance in the VTOC environment, or rate student performance against a standard to 

ensure reliability across trainers. 

Train-the-Trainer Products 

New trainer preparation is essential, but time consuming, typically requiring that the 

individual attend a VTOC course to learn the trainers role through first hand observation.

in the course the new trainer has an opportunity to gain experience, witnessing examples of 

effective and ineffective performance on the part of both the experienced trainer, and the 

students. However, this experience would typically be limited to those situations that o

within one iteration of the VTOC course. Given this limitation the opportunity exists to enhanc

new trainer preparation through th

The review of survey, interview, and previous AC3 DL research issues suggests sever

candidate Train-the-Trainer products that could be developed through follow-on research. As an 

example, Train-the-Trainer products might support the skills necessary for a VTOC trainer to 

play the role of a more senior officer for a limited set of structured staff planning vignettes. 

Alternatively, Train-the-Trainer products could provide up-to-date subject matter knowledg

lower echelon doctrine for retired senior officers serving as VTOC trainers. 

As m

V

esirable trainer and student performance. This is particularly true for VTOC, where

trainer and students normally interact via electronic media, rather than face-to-face. Vignette 

examples of trainer performance could also provide the new trainer with real world examples of 

how cognitive principles such as scaffolding can be employed in the VTOC environment. 

18
Vignette examples could provide the less experienced trainer with detailed examples of the BDE 

commander role, or coaching techniques, that are appropriate for the training goals of specific

VTOC vignettes. A combination of written guidelines and digital video clip VTOC screen 

captures examples of performance might also be used to provide VTOC trainers with a common 

set of standards for student performance. By having new trainers rate the digital video clip 

student performance examples training managers could ensure that consistent standards are bei

applied to students by trainers who might come from different backgrounds, whether active duty, 

reserve component, or retired. 

ng

Future Distributed Digital C2 Operational Capabilities 

e FCS

2

Future FCS C2 operational capabilities could include the ability for deployed units to 

xperience available online, to access staff manpower located in 

Sanctuary” off-battlefield sites, and to achieve greater lethality through digitally networked 

coordin ent 

l

ill allow “virtual leaders” and operational planners to add professional qualities to forces with 

mited

rs and staff are

geographically dispersed, but linked digitally, may actually serve to increase the tactical leader 

experie

l

Future Operational Capabilities 

The examination of VTOC training can provide valuable insights to powerful futur

C2 operational capabilities. Currently the VTOC training system seeks to replicate a 

conventional TOC environment where staff meet face-to-face for collective planning and 

decision making. However, in the VTOC all trainer and student (commander and staff) 

interactions are computer mediated, presenting a task environment that possesses key 

characteristics of proposed future fully digital distributed C2. Information gained from VTOC

trainer and student interviews and surveys were reviewed to gain insights into possible future C

system operational capabilities, training support requirements, and training design issues. 

“Reach Back” to leadership e

“

ation across on-battlefield units. Projecting the future operational and threat environm

of 2015, the U.S. Army Training and Doctrine Command (TRADOC) has predicted that the new 

C2 communication systems and information technology have the potential to provide tactica

experience, accelerate planning, and foster innovation (TRADOC, 2000). The TRADOC

predicts that leaders and units within an area of conflict will be able to link electronically to 

allies, external organizations, or individuals outside the area of responsibility to obtain ideas, 

plans, or guidance for dealing with tactical problems. Advanced communications capabilities 

w

li experience, training, and skill. “The experience, advice and assistance received could 

make all the difference in future close combat operations” (TRADOC, 2000, p. 16). 

Leadership Expertise Reach Back 

The development of a future FCS C2 “Reach Back” where commande

nce available to assist in decision making. Rather than being limited to the personnel on 

hand, future C2 cells linked in a virtual TOC could share information and request guidance from

experienced leaders around the world. Face-to-face video discussions could be held with 

experienced allied personnel who have fought the same opponent, or fought in the same tactica

environment. Access to tactical experience could also be achieved via links to reach back to 

19
web-based information sources, such as bulletin boards of tactical lessons learned from sources 

such as the Center for Army Lessons Learned (CALL). 

Potential disadvantages of having an on-line mentor available can also be identified. 

Observations of VTOC training suggested that the ease of access to the subject matter exper

trainer might lead students to rely on this leader excessively for guidance, or to defer to 

commander/instructor solutions, rather than making a decision on their own. In the VTOC 

Mission Planning briefings it wa

t

s observed that student staff members continued to request 

formation and guidance from the trainer role-playing the battalion commander even during 

their Mission Plannin n the staff and

commander, and the ability to constantly update digital documents, may have contributed to a 

off making their own decisions in hopes of gaining more 

formation for last minute integration into plans. This would be a dangerous trend if continued 

nits to

ing

the planning process. Much as the remotely located VTOC instructor reviews student staff 

plannin e

g.

s

nits to

chieve greater lethality through digitally networked synchronization across on-battlefield units. 

ble to bring geographically dispersed leaders into the planning 

rocess via digital C2 is that this can support improved allocation of assets across traditional unit 

bounda

r

in

g briefing. Here, the lack of face-to-face contact betwee

tendency for staff members to put

in

into future digital C2 operations. 

Sanctuary Off-Battlefield Manpower Sites 

Future FCS C2 operational capabilities could include the ability for deployed u

access additional staff manpower located in “Sanctuary” off-battlefield sites. Using a VTOCtype distributed digital C2 interface virtual staff members located far from the battlefield in

Sanctuary sites could provide professional manpower to assist on-battlefield units, accelerat

g products, experienced virtual staff members at Sanctuary sites could participate in th

preparation of mission planning products. In the past the staff would have to monitor unit 

activities, sending information requests and consolidating information for purposes of plannin

With digital communications links across personnel, vehicles, and commands, unit status reports 

could be automatically generated, posted to a common data base, updated, and accessed as 

needed by staff at a Sanctuary location to meet planning information requirements. The VTOC

observations suggest that new methods for allocating tasks across digitally-linked staff may be 

needed, as it was observed that some misunderstandings persisted as to the specific staff role

and responsibilities in collectively developed planning products. 

Synchronization of Assets Across Digitally Networked Units 

Future FCS C2 operational capabilities could include the ability of deployed u

a

One potential advantage of being a

p

ries, in contrast to traditional stove-piped unit support. This will be an increasingly 

important capability as the Army seeks to direct resources to specific units that need them, rathe

than dispersing limited resources into individual unit stockpiles. The enhanced coordination 

achieved through digital C2 might allow greater support between units, where (for example) one 

unit might contribute fires across unit boundaries to support another unit’s mission. 

20
Fostering Innovation 

The new C2 communication systems and information technology have the potential to 

foster i

ts had

have

sed

nd

an

uld be the development of standardized digital templates for 

lanning documents which speeds up student preparation of products, as it would for future 

ed

igital

abilities of future distributed 

igital C2 systems interface. Students mentioned that problems occurred when trying to keep 

The students were able to identify 

hanges to VTOC digital file structures that might help to keep track of document changes, and 

the des ons

re

he commander will likely serve as the primary trainer in future C2 cells, and will need 

automa d training support systems to facilitate this training. The operational capabilities of 

forces commanded by FCS C2 cells will likely not have 2-day planning tasks like current 

nnovation. Observations of VTOC training sessions revealed that students were using 

available media in novel ways, and that the students could identify desired changes in design 

features to facilitate task accomplishment. One VTOC instructor stated that most studen

purchased an additional home phone line, or kept a cell phone available, so that they would

an additional voice channel to use to coordinate their efforts. It was observed that students u

their personal commercial text and graphics software and email in innovative ways to create a

share information, as the commercial tools were considered more user-friendly for some tasks

than those developed specifically for VTOC. The flexible commercial text-editing software, 

graphics, and email tools were familiar to students, demanded few attention resources, and c

be leveraged in novel ways to accomplish C2 planning goals. One example of innovation 

emerging from VTOC training wo

p

fielded digital C2 systems. 

The VTOC lessons learned provide innovative ideas and prototype products that could

support the introduction of future C2 systems. As an example, one VTOC instructor encourag

students to develop a Battle Book of commonly needed products that can be modified to meet 

new requirements. The instructor stated that the Battle Book would be useful for students 

throughout AC3 training, and also when assigned to a unit. Research could be conducted to

document VTOC student and instructor innovative digital command post Battle Book features 

that would apply to future distributed C2 systems. The VTOC Battle Book could contain d

files of worked document examples, document templates, and commonly used graphics. By 

systematically identifying staff information needs for a digital TOC a VTOC Battle Book could 

be developed with input from training developers, instructors, and students to identify 

information and tools that could be incorporated into future C2 systems as performance aids to 

enhance their operational capabilities.

VTOC research could be valuable in identifying desired cap

d

track of changes and updates made to collective documents.

c

ire for digital work areas to develop products, which might prove to be innovative less

learned for future C2 system design efforts. In the face of system problems students learned 

innovative ways to circumvent temporary communications problems. In this case the 

unintentional introduction of problems into tasks resulted in students learning to be flexible and 

find workarounds, a valuable skill that should be incorporated in training for staffs using futu

digital C2 systems. 

Distributed Training 

T

te

21
conventional battalion forces, and will not do VTOC-style two-day planning (or 2-day long 

aining sessions for this). It is more likely that future units will face short planning timelines, 

and wil

in

e a

y

e

C2 staffs to be quickly assembled and task organized to meet the 

emands of a specific mission. In AC3 VTOC and Resident phase training, students rotate staff 

roles, g

ment 

and

ff

e field.

Trainin

d

essary to

s 

which temporarily dropped the instructor and students off the VTOC 

et, degrading or eliminating modes of communication such as voice communications or text 

nanticipated benefit resulting from this random occurrence of problems 

as that students learned to adapt to the changing environment very well, becoming adept at 

recover

ilities

tr

l need to be proficient at dynamic replanning in response to changing events during 

mission execution. These new short suspense planning requirements will need to be reflected

both initial and sustainment staff training. Particularly for sustainment training there should b

requirement for short training vignettes and part-task training that deployed units can more easil

accommodate in the regular duty schedule, rather than a continued focus on weekend-long 

planning sessions. Training developers could support the transition to future digital C2 leader 

conducted training in the unit by initiating the development of structured vignettes, automated 

performance assessment and feedback, and integrated wargaming simulation capabilities to 

playout mission plans, and courses of action. 

Multifunctional Staff 

Future C2 systems can support multifunctional leaders capable of performing multipl

staff roles. This would allow

d

aining experience in the full range of staff tasks. The SGIs stated that they identify 

stronger students and put them in more demanding positions (e.g., S3) first, then rotate the staff 

roles with other students. This technique of rotating staff assignments during training supports

the development of multifunctional leaders, and would need to be continued during sustain

training to meet the demands of future C2 systems. Current VTOC training sessions last one

a half days, with students maintaining the same role throughout the session. Future C2 staff 

sustainment training will require shorter training sessions. The use of a part-task training 

approach could be implemented to repeatedly practice the more difficult to train aspects of sta

tasks, rather than simply performing all the detailed aspects of staff planning in analogue realtime fashion, and would allow for greater rotation of staff assignments. The development of 

automated evaluation and feedback tools to support staff training should greatly facilitate 

sustainment training in garrison or in th

g for Degraded Communications

While future C2 systems will rely heavily on an established network of digital 

communications links, leaders will need to be prepared to operate in an environment of degraded

and changing communications capabilities. This requirement calls for the development of 

collective training vignettes that require leaders to utilize available communications media, an

conventional backup capabilities. The VTOC training hardware and software tools nec

link the 8-10 students and instructor and perform collective tasks across the Internet are still 

evolving. From observations of VTOC training it was noted that a number of technical problem

occurred during training

n

chat messages. One u

w

ing from system crashes, continuing to perform collaborative tasks with partial or 

changing communications links, and optimizing available capabilities in realtime. Students 

learned to begin staff tasks without all participants online, and learned to “cover” responsib

22
and otherwise take care of those staff members temporarily off-line or operating with limited

capabilities. It was observed that students would use email as an alternative to voice 

communication, commercial graphics software as an alternative to the VTOC map editing 

program, and personal cell phones to bypass computer modem telephone line constraints. 

Digital Media Induced Performance Problems

The move from the conventional face-to-face TOC environment to the computermediated distributed VTOC may require changes in the way that C2 is conducted, and a need fo

new leader support tools. Observations of VTOC training sessions raised the issue of whethe

new structure for briefings presented by distribu

r

r a

ted leaders and staff is needed, and whether the 

onventional TOC metaphor should be the basis for designing future C2 communications. In 

here the commander could not see the staff face-to-face it appeared that 

the formality of conventional staff briefings was lost. As one example, the Executive Officer 

once to

ent, 

es

s

aining

Checklist was developed to provide a systematic 

and comprehensive approach for reviewing VLE training program features based on trainer and 

observations of training. The checklist review of the 

TOC training program suggested candidate training program design features that could be 

incorpo uld be

and

lified

t

c

VTOC virtual reality w

ld the Battalion Commander to “Wait-One” in response to a question from the Battalion

Commander, and left the virtual TOC for several minutes to review digital files in search of his 

response. Research should be conducted to explore whether the conventional TOC briefing 

format of verbal discussion and graphics will remain valid for the future digital C2 environm

or whether a new digital TOC briefing format optimized for digital capabilities and constraints 

needs to be developed.

Summary and Discussion 

 

Training Program Design Features for VLE Training 

The present research identified VLE training program design features based on principl

of cognitive psychology that can support collective staff training. The program design feature

documented in this research should be generalizable to a broad range of web-based VLE tr

applications. A Cognitive Training Techniques

student surveys, interviews, and first-hand

V

rated in the training. For the VTOC VLE training system additional features co

developed to support individual and group performance scoring, development of short vignettes 

for part-task training, incorporation of access-on-demand video demonstrations of individual

collective performance, and development of structured AARs that are archived for later review.

Future ARI supported training development should investigate the planned incorporation of 

Battle Games into VTOC training, first identifying the training goal, and then considering 

performance assessment and feedback techniques that will be required of instructors to gain the 

full benefit from this training. 

Train-the-Trainer Issues 

The rapid move to adopt VTOC-type training will create a parallel demand for qua

VLE trainers possessing collective training, performance assessment, and feedback skills tha

match the capabilities and constraints of the VLE training environment. The present research 

23
suggests that the role played by the trainer is essential to the success of VLE training, however

in the case of VTOC the required trainer skills are selected for during hiring rather tha

developed through a formal training program. Future research should identify methods of 

instructor training required to develop the skills associated with the cognitive training 

,

n being

techniques, such as practice in role playing, coaching, and student performance assessment. 

Future Digital C2 Systems Operational Capabilities and Challenges 

structures did not exactly fit their needs as they would tend to lose track 

f shared digital text files containing individual and collective product updates, which are key 

features of future C2 staff work. Th ext media problem area, and may 

flect a broad real-world problem encountered when staffs perform collaborative writing and 

ion

VTOC training can provide insights to powerful future FCS C2 operational capabilities. 

Currently the VTOC training system seeks to replicate a conventional TOC environment where 

staff would meet face-to-face for collective planning and decision making. However, in the 

VTOC all instructor and student (commander and staff) interactions are computer mediated, 

presenting a task environment that possesses key characteristics of proposed future fully digital 

distributed C2. The present research suggests that distributed digital C2 systems can serve to 

expand the tactical experience base available to decision makers, allowing for accelerated 

planning, and innovation. Observations from VTOC training suggest that training can be 

designed to prepare multifunctional officers capable of assuming a variety of staff roles. One

unexpected benefit of VTOC training observations has been the identification of training 

conditions that foster skills in handling degraded communications. Some students reported that

the available VTOC file

o

is may represent a digital t

re

distributed-decision making tasks. 

Future research should investigate a number of potential operational capabilities that 

distributed digital C2 may afford. Research should investigate whether there are any 

geographical limits to the distances that might exist between distributed staff members. 

Currently the VTOC battalion level commander and staff are dispersed across several states 

while they collaborate on plans, and make decisions. If geographical dispersion does not 

introduce performance problems, then experienced subject matter experts might be able to assist

in staff planning for forward deployed units, increasing the speed and quality of planning and

decision making. While VTOC training appears to be effective in establishing some sense of 

teamwork between students and instructor it may not provide essential interpersonal aspects 

provided in traditional face to face small group training situations. The one student co-located 

with the VTOC instructor enthusiastically stated that the co-location arrangement provided him

with much greater access to the instructor, and with a greater opportunity to pass on informat

gained from the instructor to the other students. Future research needs to identify the 

psychological teamwork issues as well as the communications technology aspects of distributed 

command and control. 

24
References 

Abell, M. (2000, December). Soldiers as distance learners: What Army trainers need to kn

Paper presented at the meeting of the Interservice/Industry Training Simulation a

Education Conference, Orlando, FL. 

ow.

nd

Reserve Component

Distributed Learning Course Design. Alexandria, VA. 

structivist Learning Environments. In C. M. Reigeluth 

(Ed.), Instructional-design theories and models: A new paradigm of instructional theory 

V

ptive

,

, P. H. (2001). Small group instruction in a virtual environment. 

Unpublished manuscript. AB Technologies Inc. 

Ross, K. & Yoder, M. (1999). Producing computer literacy for the digitized battlespace of the 

future. Paper presented at the meeting of the Interservice/Industry Training Simulation and 

Education Conference, Orlando, FL. 

AB Technologies Inc. (1998, July). Armor Officer Advanced Course:

Britain, S. & Liber, O. (2000). A framework for pedagogical evaluation of virtual learning 

environments. University of Wales – Bangor. [On-line]. Available: 

http://www.jtap.ac.uk/ reports/htm/jtap-041.html

Deatz, R. C., & Campbell, C. (2001). Application of cognitive principles in distributed 

 computer-based training (Research Product 2001-03) Alexandria, VA: U.S. Army

 Research Institute for the Behavioral and Social Sciences. 

Draves, W. A. (2000). Teaching online. River Falls, Wisconsin. LERN Books. 

Hooper, M. J., & Hannafin, S. (1991). Psychological perspectives on emerging instructional 

technologies: A critical analysis. Educational Technology Research and Development, 

26(1), 69-95. 

Jonassen, D. (1999). Designing Con

ol. 2. (pp. 215-240). Mahwah, NJ: Erlbaum.

Lajoie, S. (2000). Computers as cognitive tools Vol. 2. Mahwah, NJ: Erlbaum.

Lussier, J. W., Ross, K. G., & Mayes, B. (2000, December). Coaching techniques for ada

thinking. Paper presented at the meeting of the Interservice/Industry Training Simulation 

and Education Conference, Orlando, FL. 

Moore, M., & Kearsley, G. (1996). Distance Education: A systems view. Belmont, CA: 

Wadsworth Publishing Company. Cited in Wisher, R.A., Champagne, M.V., Pawluk, J.L.

Eaton A., Thornton, D. M., & Curnow, C. K. (1999). Training through distance learning:

An assessment of research findings (Technical Report 1095). Alexandria, VA: U.S. Army

Research Institute for the Behavioral and Social Sciences. 

Offutt, R. D., & Borosky

25
Samarapungavan, A., Beishuizen, J., Brazi anders A. F. (1993). Implications of 

advances in cognitive science for intelligent tutoring systems. In Seidel, R. J., and 

anders, W. (2001). Cognitive psychology principles for digital systems training (Research 

ocial

eer

U.S. Army Research 

ted at

Association of the United States Army Symposium, May 16, 2000, Williamsburg, VA. 

TRAD n

Chief of Staff for Intelligence, U.S. Army Training and Doctrine 

Command, Fort Monroe, VA, 2000.

U.S.

ief survey of 

the changing views of learning. Department of Information Science, University of 

 

 

er, F. T., & S

Chatelier, P. R., (Eds.). Advanced technologies applied to training design (pp. 219-259). 

NY: Plenum Press. 

S

Report 1773). Alexandria, VA: U.S. Army Research Institute for the Behavioral and S

Sciences. 

Sanders, W. & Burnside, B. (2001). Assessment of initial delivery of the armor captains’ car

course (distance learning) (Research Report 1775). Alexandria, VA:

Institute for the Behavioral and Social Sciences.

Sanders, W. & Guyer, C. (2001). Commanders’ survey: Armor captains’ career course

 (distance learning) (Research Report 1771). Alexandria, VA: U.S. Army Research

 Institute for the Behavioral and Social Sciences. 

St. Onge, R. (2000, May). Army transformation overview. Briefing slides presen

OC (2000, October). Future operational and threat environment: A view of the world i

2015. Deputy

Army Armor School. (2001). AC3-DL Course. [On-line]. Available: 

http://147.238.160.41 

Wilhelmsen, S., Asmul, S. I., & Meistad, O. (1998). Psychological theories; a br

 Bergen, Norway. [On-line]. Available: http://www.uib.no/People/sinia/CSCL/ 

web_struktur-836.htm

26
Appendix A 

List of Acronyms

AR After Action Review 

AC3

AC3 Armor Captains’ Career Course – Distance Learning 

FRU Armored Forces Research Unit 

tions, Computers, and Intelligence 

BC attlefield Conditions 

ions,

aining of 

re Brigade Combat Team Commanders and Staffs 

TC Lieutenant Colonel 

 Operations Officer 

GI Small Group Instructor 

TOC Tactical Operations Center 

TRADOC U.S. Army Training and Doctrine Command 

USAARMS U.S. Army Armor School 

VLE Virtual Learning Environment 

VTOC Virtual Tactical Operations Center 

XO Executive Officer 

A

AC Active Component 

 Armor Captains’ Career Course 

-DL

A

ARI U.S. Army Research Institute for the Behavioral and Social Sciences 

AOAC-RC Armor Officer Advanced Course – Reserve Component 

BDE Brigade 

BN Battalion 

C2 Command and Control 

C4I Command, Control, Communica

C4ISR Command, Control, Communications, Computers, Intelligence, 

 Surveillance, and Reconnaissance 

CALL Center for Army Lessons Learned 

CDR Commander

CGSC Command and General Staff College 

DL Distance Learning

DT Distributed Training 

F Future B

FBCB2 Force XXI Battle Command Brigade and Below 

FCS Future Combat Systems 

FUTURETRAIN Techniques and Tools for C4ISR (Command, Control, Communicat

 Computers, Intelligence, Surveillance, and Reconnaissance) Tr

Futu

L

S3

S

A-1
Appendix B 

Issues From VTO tructor Interview

Group Instructor (SGI) on August 2, 2001, 

” requirements. A structured interview guide 

C training. A 

ints 

conduct training

at instructors selected as VTOC trainers 

staff competencies in VTOC based on their previous 

s. 

OC to learn the interface. 

the

ork).

students.

tudents become proficient in 

matter competency when student prepares the 

n follow-on Asynchronous lesson. 

mpetency goals to 1) competency 

) required Train the Trainer support. 

here a no d cific competencies for 

strengths and

weaknesses. 

competency ratings and feedback. Students are 

not individually rated on their written VTOC products. Cut sheets are used in the twoe, typically not in VTOC. 

elated) leadership skills for years. 

ormally recorded in the course data 

al evaluation of important 

 Resident phase formal evaluation, this goes 

ormalized. No formal written 

XO are working the hardest.

ated that Asynchronous teaches bits

astery of 

C Small Group Ins

The ARI staff interviewed one VTOC Small

primarily to investigate VTOC “Train the Trainer

was used. Issues identified from this interview follow: 

• Individual student performance is not formally evaluated during VTO

group AAR is conducted at the end of each weekend training session with teaching po

taken from the SGI’s informal/unstructured notes. 

• VTOC SGI stated that instructors do not need VTOC-specific training to

in the VTOC environment. The SGI stated th

already know how to train battle

experience as trainers and leader

• Instructors need some familiarization with VT

• VTOC Program logic: 

1. Identified leadership (not just doctrinal knowledge) competencies to train in

 DL course design plan (ex. Decision Making, Communicating, ... Teamw

2. Assign standard BN Command Group task to

3. SGI believes (through observation) that all s

 leadership competencies/skills. 

4. No assessment of leadership skills. 

5. Assess individual doctrinal/subject

 whole Operations Order by himself i

• No direct crosswalk from VTOC leadership co

evaluation and feedback, 2) required trainer skills, 3

T re ata to indicate that VTOC works for general or spe

group or individuals in the group.

• VTOC training places a heavy demand on instructors to role play commander and battle 

staff competencies, mentor students, and evaluate and shape individual learning exercises 

(staff positions) for each student based on an assessment of the students

• Performance assessment: Formal student

week Resident Training phase of the cours

• Instructors have been training all the (VTOC r

• What student performance scores/comments are f

base? 

1. No student performance record kept for VTOC. Form

 VTOC trained competencies occurs in

 into the record. 

2. VTOC assessment is to group from notes, not f

 evaluation in VTOC. In VTOC the S3, S2, and

 VTOC final product is OPORD. The SGI st

 and pieces, that students practice in VTOC, evaluate individual m

B-1
 knowledge in follow-on Asy ere final product is an individually

 prepared OPORD.

• Does VTOC p

- Role-Playing (Avatars): The avatar has an important useful purpose; it shows that there 

The absence of face-to-face is not a 

 

ld. 

• t

Army LTC (Retired). No “Train-the-Trainer” 

s are selected

trained. Current LTC (Retired) SGIs have been doing the 

trai

• The

new chronous. 

• Res aptain.

• There is no VTOC instructor handbook. 

nch class wh

resent unique demands on the instructor? 

 is another person working – instructor and other students can see him checking a file or

 map. 

 - Coaching: This occurs through text and audio.

problem. 

- Social interaction, collaboration between students: Students feel that they are part of a 

group, they learn more than they thought they wou

• The SGI monitors, mentors, provides guidance to students. Instructional content comes 

from Asynchronous portion of the course. 

The VTOC role plays BN CDR, other staff positions, provides performance assessmen

and feedback. Instructor needs to be

program can prepare a person to serve as VTOC trainer. The VTOC SGI

for necessary skills, not

ning tasks required in VTOC throughout their Army career. 

SGIs all take 1 week Instructor Training Course, then 2-week SGI course. Normally 

instructor will right-seat a course, and will complete Asyn

ident Course instructor needs to be “Fast Track” top 10% Active Component C

B-2
Appendix C 

in ng

First O

Issues From Observation of VTOC Tra i

bservation of VTOC Training

“Train-the-Trainer” requirements. A structured data collection guide was 

could – video role playing both 

•

given 

escribing in detail the 

sk is to....Time 

e on the stated task, did not allocate time

correctly to this task, and then began work on follow-on tasks. Students “Got Lost In The 

Woods” the details of follow-on activities, spent too much time on immediate task. 

• Class starts tasks while some students are still not logged on, some logged on but 

degraded communications. Students told to inform others when they come online. 

• Student staff roles are rotated across the seven sessions. Unplanned loss and return of 

students occurs due to common problems, but students quickly adjust using alternative 

communication lines (email, cell phones, MS Word, MS PowerPoint). 

• New tasks introduced during VTOC session to require students to adapt to change 

quickly. The trainer dropped off the VTOC net frequently, had to reboot computer. 

• The trainer claims there should be a one-to-one correspondence between Asynchronous 

lessons, and the seven VTOC practice exercises. Currently not an exact match. 

• The trainer said it would be smart to crosswalk course goals, training methods, 

assessment, and feedback. 

• Student briefing was late, appears that students did not allocate time to review the 

product (Warning Order) prior to briefing it. 

• Lots of dead air, missing students during brief. Students can hide, compared to real life 

with all battlestaff present in briefing room. 

• Instructors carry a lot of student performance assessment information in their head, not 

systematic performance evaluation records. What if one gets sick? 

The ARI staff observed VTOC Class #3 training on August 3, 2001, primarily to 

investigate VTOC

developed and used. Issues identified from observations follow: 

• The VTOC doesn’t incorporate as much multi-media as it

individual and collective, examples of good and bad performance, briefings, products, 

and time allocation graphics. 

• Code Red virus forced Army to restrict network access, students could not access 

Asynchronous training, could access VTOC training. 

On-site student claimed value in being able to get insights directly from the trainer, pass

on to other students. Implies that face to face has advantages over digital. Students

highly structured five page Mission Analysis Briefing Checklist d

products they will need to produce (ex. Situation Template Overlay). The trainer clearly 

states product and time requirements: The learning point here is....Your ta

due .... Who provides product. (Task, conditions, standard). However students 

misunderstood, used all the time availabl

C-1
Second Observation of VTOC Training

The ARI staff obse , 2001 to gain additional

insights on DL VLE training. Issues identified from observations follow: 

main tent. Can’t they see each other as avatars? Do the 

students use avatars? 

• re 

•

rd).

t we

e training.” When you find 

•

•

• e so

re, 

lay

l

n

• p graphics 

iss an

riety

erbal)

paper the student could keep as a reference. 

need

ade last time – it was a month ago.” In 

dates.

• Students start sending email to each other a week before VTOC training to review what 

they did in the previous session. 

• Students can’t file some of the information that they would like to file. “I wish I could 

store information like previously developed Operations Order and refer back to it. 

rved VTOC Class #3 training on September 1

• Student asks if anyone is in the

Instructor calls students by phone to see if they are going to come online. Is a mo

structured approach needed?

• The VTOC Instructor and student log-on takes a long time, start logging on at 0830 for 

0900 training. 

Students were given additional tasks not expected (one page additional CDR guidance 

posted on VTOC message boa

• The trainer describes link between past, present, and future work. “Expand on wha

did last time today, you will see again in resident phas

information save it as a Battle Book/Smart Book for resident phase and when you go to a

command. 

The trainer has student by staff position matrix for the seven VTOC training sessions. 

The trainer claims that not seeing each other is no problem. “They almost do see each 

other – working the documents and verbally.” 

The trainer stated “Almost all S’s have gotten an additional line for email/cell phon

they can use these additional common links to perform VTOC tasks.” Maps: MapEdit

topographical digital map has battlefield symbology that does not scale down. Vehicle 

icons that are too large to place on map (ex. tank icon takes up almost a full grid squa

icon is 700 meters long, 400 meters wide. Students use generic graphics tool to disp

simple battlefield geometry for planning. The detailed digital topographical map too

provided with VTOC is too detailed, icons not scalable, takes too long to put graphics o

map. Students get a 1:50K paper map to use in lesson projects. 

Students use MS PowerPoint as an alternative for MapEdit in sharing ma

• Students get threat information from web sites instead of available VTOC FMs. M

opportunity to become familiar with validated doctrinal materials. Students use a va

of available communication resources not provided in VTOC to accomplish tasks: 

Microsoft Word and PowerPoint, public email, cell phones. 

• The trainer stated that it might be good to develop a more formalized (not just v

AAR – provide digital text/

• One student stated that students lose files from previous month’s training that they

for today’s tasks. 

• Students can’t remember where files are from previous VTOC session, which files were 

updated. “I don’t remember what change we m

this collective document editing task the students don’t know which files are most up to

date, what material or changes other students might have added as up

C-2
• The trainer role plays the BDE and BN CDR, BN XO and S3. The trainer role combines 

art and science. Science is the task checklist. Art is the ability to assess whether the 

repared ARR might have less student involvement, lose a training 

• get confusing – month later lose track of changes made to 

• d that “In VTOC the S3 position is the most important, others just pitch 

get done?

ng on

stration, digital file example, previous student work. 

.

• n, concerns about internal consistency. 

student’s plan is feasible, no set solutions, requires coaching a broad range of subject 

matter. 

• Current verbal AAR has trainer asking questions and getting student answers, a learning 

event. High structure p

opportunity. 

One student stated that “Files

pieces, what is most up-to-date.” 

One student state

in.” Do we get diffusion of staff responsibility/ lose responsibility for specific staff 

duties, to the point where they don’t

• The trainer stated request for an “update” was misinterpreted by students. Is there a 

human bias toward presenting detailed digital products, briefings, instead of focusi

information content? Do students need clear example of expected product, could be a 

video demon

• Students prefer to use alternative phone and email common lines not provided in VTOC

to share files. Student roles evolve, one appeared to have the role of “PowerPoint” guy

Digital products not reviewed before presentatio

 Each student confirms he has the latest update of text and graphics prior to brief, an 

important concern with digital C2. 

C-3
 Appendix D 

Issues From Initial Delivery of AC3 DL Course Student Interview and Survey 

AC3 DL Student Interview Issues: 

People in command positions can't miss a drill. 

Problem: whe

•

• n writing an OPORD you would normally have a battle staff to support 

"

g in

nts

•

•

•

time requirements associated with the course. The 

chain of command does not realize the time demands. 

• Problem if you don't have weekends off, or rotate days off. 

• The VTOC facilitates later resident training. 

• The VTOC group training is a problem when one member can’t participate. 

AC3 DL Student Written Survey Issues:

you. Provide a Battle Book - Battle Book helped tremendously - Map Edit only allows 6

view of map, and it crashed a lot. 

• Progression - should we start w/ TF, CO, BN? Start big and work down, or start at the 

bottom and work up? Trying to write CO/TM orders - student says they are "swingin

the dark." 

• The VTOC is good - everybody worked from home. It would be much better if stude

worked in teams because it is too easy to "hide" at home. 

So many students dropped that you lost key positions. 

Command and control through a port hole. 

• Can't have commanders’ company/team doing Synchronous training instead of drill. 

If you could get small groups together for VTOC - real world together - then do it.

• If you have the chance to work face-to-face do it. Like S3 and S2 tasks, will need to get 

chain of command to support it. 

• There is not an understanding of the

• Seventy three percent of students rate the quality of AC3 DL VTOC training as Very 

High, or High. 

• Students and the SGI stated that AC3 DL is superior to AOAC RC as it includes peer 

motivation and peer tutoring as part of the learning process, and better prepares students 

for resident training. 

• Do not do VTOC every month for several months in a row (miss drill and have more 

conflicts with civilian job). 

• Need more one-on-one feedback from SGIs on student progress. 

• Enforce standards (e.g., suspense dates for work completion). 

• Compress the timeline. Run Phase I in tandem with Phase II (VTOC). 

• A lot of straight OPORDs and tactics. However, as an armor CO CDR I spend more time 

with training issues and plans, soldier issues, recruiting, armory activities, and paperwork 

than I do with OPORDs. Need to focus on other aspects of command also. 

• Need detailed AAR to follow course. 

• Work the software conflicts with VTOC, Map Edit etc., so they are not training 

distracters. 

• Big blocks of instruction are a problem. Start late and see a nine-hour block of VTOC 

and sign off – can’t do big blocks. 

D-1
Appendix E 

Issues From Initial Delivery of AC3 DL Course SGI Interview 

The SGI was asked to give his overall assessment of the AC3 DL course and offered the 

foll i

d

• ivision of labor between students, identifying the 

nous training, and from discussions with each student. 

actice tactical operations prior to 

The G

com t

follo i

•

front of peers, students know 

roducts not evaluated. 

b and

s get more experience with computer usage (future C2 

•

ritten exam. 

as part of the

ow ng comments related to VTOC training:

• The VTOC training is valuable because it provides repetition in Orders Preparation, an

provides additional experience for students in briefing planning products. 

The VTOC trainer must decide the d

stronger students who can take on more demanding positions (ex. S3) first. The trainer

makes his preliminary assessment of student skills through a review of their performance 

in aysnchro

• Student load: Instructor can handle two VTOC classes at the same time, equal to two 

weekends each month. 

• Shortcoming: VTOC does not provide opportunity to pr

Resident Training. 

S I was asked to describe how VTOC provides training for nine specific student 

pe encies, and to describe the training needed for SGIs to support this training. The 

w ng comments were provided:

• Decision Making: Students more likely to make a decision. 

Planning: More planning experience through VTOC. 

• Communicating: More confidence, experience speaking in

SGI better. Individual student VTOC written p

• Technical and tactical proficiency: Students have better knowledge of the jo

tactical doctrine (compared to old AOAC RC Correspondence Course). 

• Use of available systems: Student

flexibility, degraded modes of operation). 

Supervising: Synchronous VTOC training first addresses Military Decision Making 

Process, the SGI plays the role of the XO and/or CDR, and walks students through 

wargaming the Course Of Action analysis. First the SGI shows the students what to do, 

then appoints students to play the role of XO or S3 which does involve supervising, 

controlling, and directing students in subordinate staff roles. The previous AOAC RC

course did not train staff processes, course content was all Company Offense (for 

commanders), then Company Team Defense, Operations Order, w

• Professional ethics: Not specifically addressed in VTOC. 

• Teaching/counseling: AC3 DL includes peer motivation, peer tutoring

learning process. Students say they need to learn to lead, and to be a follower. 

• Soldier team development: In Asynchronous identify Team Leaders. Students teamed in 

VTOC portion of training. The bond between students was there when they showed up 

(for 2-week Resident phase of training).

E-1
Appendix F 

2001 SANDERS 

Cognitive Training Techniques Checklist 

SYSTEM 

VTOC 

DATE 

X OCT

RATER 

Integrated in

Training? 

Cognitive Principle Rater Comments/Suggestions 

Yes No NA

1. Use

build si

context-based training to

tuated cognition of tasks.

 The VTOC tasks are performed 

collectively by groups of students in 

virtual battalion TOC with avatars to 

represent battlestaff (students). 

 1a. Knowledge to be trained 

sho

context

 The VTOC tasks are performed with 

conventional. Better reflects the 

future digital C2 interface context. 

uld be presented in a realistic

.

√ digital communication links vs. 

 1b. The training problem should 

be ill-defined, ill-structured, to 

stimulate original thinking. 

 

√

Problem is highly defined an

structured, using a 5-page Miss

d

ion

Analysis Briefing Checklist. 

2. T i

con l

informa

ra ning should emphasis learner 

tro and the ability to manipulate 

tion in the task environment. 

The VTOC students role-play staff

e information. 

positions and use generic map and 

text tools to exchang

2a.

way i

informa

Learner should have multiple The VTOC interface provides 

s ( nterface options) to access

tion.

√ multiple text and graphics tools:

bookshelf metaphor, maps, chat line, 

and email. 

2b.

access

of infor

graphic

Learner should be able to

and manipulate different types 

mation (e.g., text, voice, 

s).

√

Learner can access and share 

information using personal computer 

files, telephone, non-VTOC text and

graphics software applications. 

3. Trai

modes

rep e

ning should provide multiple 

and perspectives for

res nting instructional content. 

 The VTOC does provide the stu

with multiple methods for exam

dent

ining 

material.

3

lear r

indi d

prefere

a. Training should provide The VTOC provides voice traffic,

ne selectable interface media, 

vi ualizable to meet learner 

nces.

√ digital text messages, simulated faxes, 

information from maps, doctrinal 

documents, and plans. 

for cross-training of team m

3b.

ember 

roles. 

√

The VTOC students rotate assigned 

staff roles, cover for off-line staff, 

learn how to lead and how to follow. 

Team training should provide

F-1
SYSTEM 

VTOC

D

X OCT 2001 

RATER 

 SANDERS 

ATE

Integrated in

Training? 

Cognitive Training Techniq

NA

ues Rater Comments/Suggestions 

Yes No

 4. Promote case based reasoning 

ted cases or

worked examples to enable CBR and 

cognitive flexibility. 

(CBR). Provide rela

 The VTOC tr orates

nique

solutions. Vignettes build on Phase 1 

self-study tutorial themes. 

aining incorp

realistic vignettes that require u

 4a. Provide multiple short 

vignettes. A broad range of training

examples can promote greater 

flexibility and reinforce the learning 

√

per

ss,

points. 

The VTOC presents one vignette

weekend. Recommend incorporating

Part-task training of short vignettes, 

or selected portions of staff proce

to increase the breadth of examples.

4b. Present brief case studies a

worked e

nd

xamples of staff process and

products. 

 

√

Recommend adding multimedia

examples (video, illustrations) of 

effective and ineffective staff 

collaboration. 

5. Assessment focused on trans

knowledge and skills to similar but 

fer of

different tasks.

tutorial

course knowledge to practical 

The VTOC training does involve 

transfer of Phase 1 self-study

exercises. 

 5a. Include novel vignettes to 

assess transfer of skills to similar but 

not identical situations.

√

planning process. However, VTOC 

ed by

at. 

Trainers present learners with 

unanticipated changes during the 

tasks and learner roles are limit

weekend-long single vignette form

5b. Performance assessment 

ould measure information √ ducts require information 

not just

sh

synthesis and integration skills, and

not just recall of facts. 

The VTOC trainers stress that 

planning pro

synthesis and integration, and

digital file cut-and-paste. 

6. Coaching develops learner 

expertise through performance 

ents

assessment, feedback and goals 

tailored to the individual stud

needs (Scaffolding). 

 The VTOC training does not include 

formal individual performance 

standards, or student performance 

record keeping. 

6a. Coaching requires assessment The VTOC relies on informal learner 

of individual learner skills. √ performance assessment for coaching,

without standardized performance 

criteria. 

6b. Standardization of assessment 

tors. 

 eck of

and performance documentation 

allows for coaching by multiple 

instruc

√

There is currently no ch

performance assessment consistency

across trainers. 

F-2
SYSTEM 

VTOC 

DA

X O

TE RA

CT 2001

TER 

SANDERS 

Inte

Tra

grated in

ining?

Cognitive Principle 

Yes o A

Rater Comm

N N

ents/Suggestions 

 7. Use conflicting perspectives, 

and cognitive dissonance, to force 

students to rethink their

assumptions. 

The

req

ass

VTOC training presents changing

uirements to force students to rethink 

umptions. 

 7a. Put student in an 

immersive problem-solving 

context from the beginning of

 

√

training. 

The

rep

and

VTOC immerses student in a virtual

resentation of the task environment 

problem context. 

7b. Learning environment 

should promote rethinking of 

assumptions, and restructuring of 

decision making knowledge. 

√

Trainers introduce changes (e.g., add or 

delete battalion assets) to force students 

to rethink and adjust their plans. 

8. Learning requires social

interaction and collaboration 

between learners, testing their 

own understanding against that of 

others. 

The

scenario-based tasks which leverage peer 

ass mote

lear

VTOC training includes collective 

essment and peer tutoring to pro

ning.

 8a. Facilitate social interaction 

through face to face, voice, and √

The

ava

text chat communications tex

channels. 

VTOC provides virtual battlestaff

tars on-screen to enhance voice and 

t social interaction.

8b. Collaborative tasks

provide a venue for students to

share ideas and understanding. 

√

The VTOC provides planning docum

pre

col

ent 

paration tasks that require 

laborative team work. 

9. Have students model skills 

demonstrated by an expert tutor 

or coach (Cognitive 

Apprenticeship). 

The e

commander, and serve as subject m

experts. 

VTOC trainers role play the brigad

atter 

 9a. Key skills should be 

odeled by the trainer.

 

m √

VT

Com

dem

per

OC trainer play the role of

mander. Trainer does not 

onstrate examples of staff officer task 

formance. 

9b. Incorporate an expert tutor

to demonstrate skills. √

Mig

dem

ineffective s e performance. 

ht add multi-media video clips to 

onstrate examples of effective, and

taff collectiv

F-3
SYSTEM 

VTOC 

DATE 

X OCT 2001

RATER 

SANDERS 

Integrated in

Training? 

Cognitive Principle 

Yes No NA

Rater Comments/Suggestions 

10. Training should provide a 

high degree of external support 

and coaching that links n

to previous tra

ew skills

ining and

information (Scaffolding) which 

gradually fades. 

The VTOC virtual learning environmen

provides problem representation, 

knowledge-mo

t

deling, performancesupport, and information-gathering tools. 

 10a. Trainer must assess

learner’s strengths and 

weakness

each

es to scaffold new skills 

n previously mastered 

√ gths of each 

learner. Trainers provide guidance and 

o assigns battle staff roles to learners when 

knowledge. 

 The VTOC trainers make a subjective 

assessment of the stren

they feel the learner is sufficiently 

prepared. 

 10b. Trainer support of 

learners should gradually fade as √ volunteer less guidance to learners as they 

learning progresses. 

The VTOC trainers are careful to

progress through the VTOC training 

sessions. 

F-4