DOCUMENT RESUME

ED 215 221 CE 032 157

AUTHOR Jorgensen, C. C.; Hoffer, P. L.

TITLE Early Formation of Training Programs for Cost

Effectiveness Analysis.

INSTITUTION Army Research Inst. for the Behavioral and Social

Sciences, Alexandria, Va.

REPORT NO ARITR-472

PUB DATE Jul 78

NOTE 137p.; Research conducted by the Army Research

Institute Field Unit at Fort Bliss, TX.

EDRS PRICE M701/PC06 Plus Postage.

DESCRIPTORS *Cost Effectiveness; *Cost Estimates; Data Analysis;

Equipment Utilization; *Evaluation Methods; Military

Air Facilities; *Military Training; Models;

Predictive Measurement; Predictor Variables; *Program

Costs; Program Design; *Program Evaluation; Research

Design; Research Methodology; Training

IDENTIFIERS Air Force; *Cost and Training Effectiveness Analysis;

Weapons

ABSTRACT

Developed for application in Air Force weapons

systems training programs, a research project investigated the

methods, procedures, and data available for conducting cost and

training effectiveness analyses (CTEA). The primary objective of the

project was to develop a user-oriented procedure for early

formulation of training programs that can be input into cost and

training effectiveness analyses. While the starting point for the

CTEA procedure developed is always a task description, the procedure

has not been made dependent upon any one participation task format,

but instead recJrds task information into a set of descriptive

variables. Functions of training programs addressed by the procedure

include (1) the stimulus media presenting the training information;

(2) the response media accepting the responses made by the student to

the stimulus media; (3) the feedback media performing evaluations of

student response; and (4) the method within which the media are used.

Steps of the CTEA procedure include preparing task descriptions,

choosing the task media, describing the functional context of the

group task, recording efficiency for each task, selecting the group

method, reducing the media set, costing the program, forming

bfficiency ratios, and normalizing costs for cost/efficiency ratios.

(MN)

***********************************************************************

* Reproductions supplied by EDRS a e the best that can be made * * from the original document. * ***********************************************************************
Technical Report 472

EARLY FORMATION OF TRAINING PROGRAMS

FOR COST EFFECTIVENESS ANALYSIS

C.C. Jorgensen and P.L. Hoffer

Submitted by: Michael H. Strub, Chief

ARI FIELD UNIT AT FORT BLISS, TEXAS

U S DEPARTMENT OF EDUCATION

NATIONAL INSTITUTE OF EDUCATION

EDUCATIONAL RESOURCES INFORMA PION /

CENTER (ERIC,

Thrs document has Peen reproduced as

re,inied Irons the person or organization

,riginating it

Minor changes have beet, made ' t improve

reprt dor non giratilv

Pr ants of view or opinions stated m this doer,

dc not pertessarilv represent cf4ictel NIE

pt, in of [10'.G/

Approved by:

E. Ralph Dusek, Director

PERSONNEL AND TRAINING

RESEARCH INSTITUTE

U.S. ARMY RESEARCH INSTITUTE FOR THE BEHAVIORAL AND SOCIAL SCIENCES

5001 Eisenhower Avenue. Alexandria. Virginia 22333

Office, Deputy Chief of Staff for Personnel

Department of the Army

July 1978

Army Project Number

20763743A771

Approved for public release; distribution unlimited.

111

System Embedded

Training Development
U. S. ARMY RESEARCH INSTITUTE

FOR THE BEHAVIORAL AND SOCIAL SCIENCES

A Field Operating Agency under the Jurisdiction of the

Deputy Chief of Staff for Personnel

JOSEPH ZEIDNER

Technical Director

L. NEALE COSBY

Colonel, IN

Commander

NOTICES

DISTRIBUTION: Primary distribution of this report has been made by ARI.

Please address correspondence concerning distribution of reports to: U.S.

Army Research Institute for the Behavioral and Social Sciences, ATTN:

PERI-TST, 5001 Eisenhower Avenue, Alexandria, Virginia 22333.

FINAL DISPOSITION: This report may be destroyed when it is no longer

needed. Please do not return it to the U.S. Army Research Institute for

the Behavioral and Social Sciences.

NOTE: The findings In this report are not to be construed as an official

Department of the Army position, unless so designated by other authorized

documents.
ARI Research Reports and Technical Reports are intended for sponsors of

R&D tasks and for other research and military agencies. Any findings ready

for implementation at the time of publication are presented in the last part

of the Brief. Upon completion of a major phase of the task, formal recommendations for official action normally are conveyed to appropriate military

agencies by briefing or Disposition Form.

I iv
FOREWORD

The research reported here is part of a broader program on air defense

weapon systems being conducted by the Army Research Institute Field Unit at

Fort Bliss, Since 1975, the Fort Bliss Field Unit has been engaged in a

vigorous program of research to discover the methods, procedures and data available for conducting cost and training effectiveness analyses (CTEA). An especially challenging issue is the development of a methodology applicable to weapon systems still in the formative or conceptual stage of development.

The methodology presented in this report begins with preliminary duty or

task description lists that include probable weapon characteristics. The methodology then utilizes a series of selection procedures to generate a profile of

equipment and methods that could be used to train the tasks. The selection

procedures also yield an efficiency ratio that indicates the relative opportunity costs of making alternate training decisions and assumptions. The final

projected training requirements are formulated to be costed in dollars sia a

computer program.

While the methodology was developed with a view toward application on the

many new air defense systems projected for fielding in the mid-80's, it is by

no means restricted to air defense application. The methodology is intended

for use in any man-machine system for which early estimations of cost and training effectiveness are required.

The entire program is responsive to the requirements of RDTE Project

2Q763743A771 and the Director of Training Developments, U.S. Army Air Defense

School, Fort Bliss, Tex.

v

O

EPH DNER

hnical Director
EARLY FORMULATION OF TRAINING PROGRAMS

FOR COST EFFECTIVENESS ANALYSIS

BRIEF

Increased accuracy in early budgetary prediction has become more important

in recent years and has led to a corresponding need within all branches of the

military services to better estimate future training programs and their impacts.

Recent examples of this need have included the intelservice procedures for instructional system develoi,mentl (ISD), TRADOC Pamphlet 71-10,2 and the Army

Training Study.3

The need to assess the value of training programs and weapon systems has

also led to the development of cost and operational effectiveness analysis

(COEA) and associated analyses, such as cost and training effectiveness analysis (CTEA) and reliability, availability, and maintainability analysis (RAM).

Each decision making tool, however, assumes a methodology by which the required

information can be generated. In CTEA the methodologies are still in the developmental stages as a result of both instructional decision r ing and the

current state of the art in training analysis. This paper addr,sses the latter

area. Its concern is with developing a user-oriented procedure for early for- , mulation of training programs that can be input into cost and training effectiveness analyses.

The research reported in this paper begins with a theoretical discussion

of the procedure and proceeds to the specific. The first section (Section A)

deals mainly with the underlying logic used in constructing the procedure. The

discussion becomes more applied in Section B. Finally, in Section C a simplified example is presented designed to show how the procedure would function to

choose a sample training method and its support equipment.

An overview of the procedure is shown in the flowchart in Figure 1. Each

box in the flowchart corresponds to a key process that must be performed to

formulate a future training program. Four of these processes: Choose media,

record efficiency, describe functional context, and select a group method are

considered in detail. The remaining boxes are discussed only in enough detail

to illustrate how the processes work together for input into CTEA.

The starting point for this procedure is a task description. Much of the

initial tasx analysis information for developing weapon systems is not available in the suggested Instructional Systems Developmen (ISD) Handbook format.

Thus, the method in this paper has not been made dependent upon any particular

task format, but instead recodes task information into a set of descriptive

variables. CTEA processes are, therefore, defined in terms of interactions

among these variables. Four separate variable matrices have been generated to

select training media and methods. Each matrix corresponds to some function of

a training progrun. The four matrices include: (1) The stimulus media (equipment/materials) presenting the training information, (2) the response media

(equipment/materials) accepting the responses made by the student to the

vii

t)
stimulus information, (3) the feedback media (equipment/materials) performing

evaluations of the student response, and (4) the method, i.e., the organizational structure of student-instructor interrelationships, within which the

media are used.

I---

CHOOSE MEDIA FOR

EACH TASK

RECORD EFFICIENCY

FOR EACH TASK

REDUCE MEDIA SET

PREPARE TASK

DESCRIPTIONS

GROUP TASKS

DESCRIBE FUNCTIONAL

CONTEXT OF TASK GROUP

SELECT GROUP METHOD

FORM EFFICIENCY

RATIOS

COST PROGRAM I

Figure 1. System formulation.

NORMALIZE COSTS FOR

COST/EFFICIENCY RATIO

In the three media matrices, physical information derived from each task

description is used to select an "ideal" piece of training equipment. This

physical information is derived not only from the task description but also

from the referenced characteristics of the system hardware upon which the task

is performed.

Unlike the media matrices where physical information and system hardware

derive the selection of stimulus, response, and feedback equipment, the training

method matrix is based on variables which describe operational context. Because

most of the commonly used methods are loosely defined, a separate procedure is

used to more Precisely define alternative training approaches.

viii
C

Although the method and media variab]es aid training program selection decisions, a technique is also required to monitor effects produced by managerial

changes. This paper presents a process called "efficiency ratio calculation"

to track the psychological training gains or losses caused by program modifications. A costing process to track dollar values is also considered. Both measures comprise a cost/efficiency ratio which can be used by the CTEA analyst to

compare the relative merits of various training programs.

ix

4
EARLY FORMULATION OF TRAINING PROGRAMS

FOR COST EFFECTIVENESS ANALYSIS

CONTENTS

I. TECHNICAL SUPPLEMENT

Page

A. The Logic of Going from Task Description to

Training Program 1 1. A String Analogy 3

2. Linking the Task Data with a Training Program 4

3. Developing the Process Matrices 5 B. Application of the Method 7

1. Generating the Training Variables 9

2. Implementing the Model 12

a. Method Selection 13

b. Media Selection 16

c. Content and Structure Selection 16

d. Effectiveness Prediction 17

e. The Costing Procedure 18

C. An Example 21

D. References 33

II. APPENDIXES

A. Media 35

B. Method 53

C. Future Research Resource Variables 81

D. Sample Cost Sheets 99

xi
CONTENTS (Continued)

LIST OF FIGURES

Page

Figure 1. System formulation viii

2. Mapping from task situations to training programs 2

3. Linking training variables to string elements 6

4. An example of a device matrix being premultiplied

by a task variable vector 8

5. The generation of a single efficiency ratio

and plot of possible ratios 19

6. The flow of training program formulation 22

7. Media selection matrix 26

8. Media selection matrix 28

9. Media selection matrix 29

10. Method selection matrix 30

xii

i I )
EARLY FORMULATION OF TRAINING PROGRAMS

FOR COST EFFECTIVENESS ANALYSIS

I. TECHNICAL SUPPLEMENT

A. THE LOGIC OF GOING FROM TASK DESCRIPTION TO TRAINING PROGRAM

To go from a task description to a costed training program, a procedure

needs to be set up for defining the output (the training program elements) in

terms of a series of manipulations on the input (task analysis data). This

procedure, or mapping, involves finding a common set of variables by which both

the training program and the task analysis can be described. (See Fig. 2.)

Once a common set of variables is found, it is then possible to determine hcw

well a particular training concept represents critical task information by com- paring similarities or differences for each variable.

Implicit in the use of such variables are some distinct methodological as- sumptions which should be examined. Most training experts agree that two important factors determining the success of a training program are functional

and physical fidelity. Physical fidelity means that the training program must

capture the essential operations and physical conditions present in the real

life situation for which an individual is being trained. In the case of functional fidelity, the critical actions or functions performed by the system user are captured by either exact replication of user actions or by using a psychologically useful analog. Specific rules for generating such an analog are not

yet (in the opinion of the authors) sufficiently developed to be useful in CTEA

application. The importance of both types of fidelity depends upon a wide variety of factors in the learning situation.

It is obvious that one way to maximize physical and functional fidelity is

to do all training on the actual system equipment and in the real environment.

Allowing for the fact that personnel must still be taught certain prerequisite

skills and knowledges before they can begin to use the actual equipment, this

approach is both highly expensive and unrealistic. Consequently, the problem

becomes one of choosing some subset of the full range of physical and functional

requirements, combining them with groups of devices and methods, and managing

the interactions that are generated as a result.

There are at least two approaches for determining subsets. One approach

is to select important fidelity related variables by analyzing the current literature on training interactions. One difficulty in assimilating this literature, however, is that the contexts within which studies take place are so varied that it is often difficult to identify causal relations. Good summaries

have been produced, e.g., Meister, 1976,4 however, and are a areat aid in pulling together this diversified literature.

A second approach is to use survey information drawn from panels of experts. This type of information is generally more relevant for specific applications, but it often lacks experimental rigor and is difficult to validate.

If forced to choose, summary literature seems to present the more immediate

1
TASK

DESCRIPTION

Physical Requirements ---4 Physical

1. Variables --)

2. --)

3.

''-',

Functional Requirements Functional

1. Variables _4

2. --4

3.

Social Requirements

n Context

1. Variables

2.

3.n Real World Constraints ---.4 Variable

Weightings

--4

--)

111111.

Figure 2. Mapping from task situations to training programs.

TRAINING

PROGRAM
data base for preliminary models (Haverland, 19745). The model developed in

this paper has utilized summary literature to a large extent.

A.1. A STRING ANALOGY

The model assumes an underlying logic about how the interactions and processes in training programs can be represented. The logic can be visually illustrated by a string of letters, for example: ABCCBFCCBA. Within any string,

several things may vary. First, the string is composed of elements, represented

here by capital letters. Second, the elements may occupy different positions

and occur at different frequencies within the string. Third, the string may

vary in length. Finally it may not contain all possible elements that could

have been chosen. A training program can be similarly represented.

In a training program there are two basic levels of interest, the program

level and the content level. The program level is concerned with the physical

equipment and facilities used ill the training environment and the social si.ructure or method within which the physical equipment is applied. The content

level is concerned with the units of training information which are flowing

through the program to a student and are important to a course writer. The interrelationships between information units represent structure, which when specified, create a course text. Both levels of interest are presented below.

Pror'ram level Content level

Equipment & facilities Information units

(Media) (Content)

Social interrelationships Information unit interrelationships

(Method) (Structure)

Both the program level and the content level can be thought of as a string

of elements, as described above. For the program level, the elements are the

physical media. The interrelationships represented in the ordering of the

string are the methods. For the content level, the elements are the units of

information to be presented and the interrelationships are the orderings and

sequences of information units.

As with the string analogy, elements at the program level may occur at

different frequencies, may be selected frcm a larger pool, may be grouped into

strings of different lengths, and may occupy different positions. Elements at

the content level may themselves be composed of smaller elements, occur at different frequencies of presentation, and so on.

3

1. A
Other characteristics of a string concern relational properties which the

elements have to one another. For example, at the program level, the concern

is how the media interrelate. This interrelationship is called the training

method. Using the string as an analogy, a convenient breakdown for training

progams is made possible. The program breakdown in turn leads to analytical

representations which can be paired to training variables. For example, five

questions were derived, using the string analogy, to capture the basic differences between existing training methods. First, if a string of training elements does not have a constant sequence over time, who or what controls the

sequence? Second, how many times in an instructional string does an element

appear? Third, where does each appearance occur? Fourth, if an element influences the sequence of other elements, what degree of influence does it have?

For example, if the elements were an instructor, a student teacher, and students, what would be the degree of control of the student teacher relative to

the instructor in a given method? Finally, what percentage of the total instructional time is taken up by each of the elements?

These five question:: were systematically applied to a number of commonly

accepted training "methods" to give them a more precise definition for CTEA

usage. Twenty-three such methods are defined in Appendix B-54-76, along with

their common narrative descriptions. This procedure not only allows one to

describe an existing metnod but also allows one to produce a number of new

training methods based on combinations of answers to the five basic questions.

Until now the string analogy has been used only to consider the program

level, but it is equally applicable for the content level. For this level, the

elements are units of specific training material, i.e., blocks of course content at any convenient level of description, and their interrelationships, the

variables affecting the structure, i.e., the sequencing or patterning of material. At this level of analysis, the problem is one of specific course material and structure, reeler than the determination of physical equipment and

method. In this way the string can be used as a convenient framework for the

division and clarification of processes going on in a training program.

It should be noted that a training program does not operate in a vacuum

and that its success or failure is influenced markedly by its environment.

Consequently, training program formulation must allow for external impacts,

such as managerial limitations, costs, personnel, and so on. In the context of

a string, these may be considered modifying factors which change an idealized

string of instructional events to a conforming string concerned with practical

realities. In this paper, such modifying factors are seen as having a direct

impact on the five questions used to define the training methods.

A.2. LINKING THE TASK DATA WITH A TRAINING PROGRAM

In the earlier discussion of the training variables, it was stated that

their purpose was to form a common linkage between the physical and functional

properties of the task :nyironment and the potential training program. The way

in which this linkage is made will now be considered in greater detail.

4
Matrix algebra is a technique that can be used in the application rf both

the string analogy and the variables. Because of its flexibility, matrix algebra is a powerful tool for characterizing any model. In this particular effort,

matrices are at the heart of the method and media selection processes. A sample process will be examined to show how matrix techniques are used.

Recall that the variables are chosen to link the task environment to some aspect of the training environment. Figure 3 shows a sample matrix. Along the

X-axis are elements representing either program or content elements. Along the

Y-axis are variables representing some subset of training variables corresponding to a particular training program function su "h as media selection. Each

variable along the Y-axis is given a value based on its applicability or nonapplicability to each program or content element along the X-axis. For example, if the element is a "slide projector," and a variable is "visual presentation capability," the variable would be applicable. Using the same logic, each

element is then rated in terms of all of the variables for the training program

function. The value's applicability or nonapplicability is recorded in the

appropriate cell as a nominal, ordinal, or ratio value, depending upon the degree of accuracy with which the assessment can be made. The complete matrix

thus represents a description of the set of training elements available to a training manager for that particular function. Such matrices are constructed

for each of the training prigram functions for which training literature indicates such pairings are practical. When the matrices are completed, the training elements will have been described in terms of variables.

A parallel problem, however, is how is task information described in terms

of the same variables? A similar procedure can be used. For example, suppose

that a task involves reading visual material from a radar scope. If the task

also requires a response to some kind of audio information, the variables applicable to the task description will not match the variables applicable to the

slide projector in the example above. It might, however, match some other piece

of equipment. How this is determined is where a matrix becomes valuable. One

difficult way in which a comparison can be performed is to individually examine

each variable for each task and compare it to each variable for each training

element in a process matrix, such as in Figure 3. Once done, the number of

matching variables can be totaled for each training element and the one having

the highest score taken as the "best fit" for that task. If the procedure is

repeated for all tasks and all variables in each of the training processes and

are combined into a single program, a training analyst has a starting point

from which to do predictive analysis. Needless to say, this procedure would be

extremely time consumihg, if done entirely by hand. Fortunately, such a task

is made easy through the use of rlatrix algebra and the procedure is well suited

to machine implementation.

A.3. DEVELOPING THE PROCESS MATRICES

Because the objective in the previous section is to choose the device or

method having the largest number of similar variables, the matrix operation of

premuitiplication is an ideal technique. Suppose that the task description is

characterized by only three variables. The corresponding matrix cells for

these variables are given a value of 1 when they apply. If variables do not

apply, they are valued as 0. Similarly, suppose there are only two training

elements to be considered in the process, and they are also coded with a 1 or

S
VIUJJ

03

:i

CC

e:C

12

3

i

n

ELEMENTS

1 2 3 j

aij

k

a..ij = A value in a matrix cell representing the degree of applicability

or non-applicability of variable i to element j. (a) may be a

nominal, ordinal, or ratio value.

Figure 3. Linking training variables

to string elements.

6
O. (In reality, any nominal, ordinal, or ratio value could be used.) To see

which element best "fits" the above task, the device matrix (A) is premultiplied

by the task variable vector (b), producing a row vector of column sums (c).

This vector represents the sum of the number of variables which match both the

task description and the device matrix. (See Fig. 4.) Therefore, the second

column element in the matrix is viewed as the "best fit" for properties in the

task description, i.e., it matches two, the most, task variables. Suppose,

however, that there are external factors, such as managerial constraints, affecting the suitability of elements described by matrix (A). If so, this implies that the values in (A) should be weighted. Use of matrix formats makes

this a simple operation. For example, if a weight is given to each element

that corresponds to its relative desirability, then each column of variables in

the matrix must be multiplied by the appropriate weight. Say that the element

in column one is four times as suitable as the element in column two. A column

multiplication of (A) by a vector of weights, w = (4, 1), produces:

Element

#1 #2

4 0

A = 4 1

0 1 The previous premultiplication by (b) now produces a summed vector c = (4, 2).

Element number one would now be the best choice considering these new conditions. The same logic can be used for variables in the task description, the

importance of variables in the matrix, or even individual items in the matrix

through row multiplication. column multiplication or matrix element replacement,

respectively.

Thus the use of matrix algebra presents a powerful analytic tool for applying a wide range of potential interactions and factors affecting the processes for formulation of training progi 1MS. A matrix process is for all practical purposes unlimited in terms of the number of variables it can consider,

as well as the number of intervening factors that can modify it. In these simple examples, only two or three elements were dealt with. In the computer program for this model, program elements alone number more than 35 for a single

process. The logic, however, remains the same in both cases.

B. APPLICATION OF THE METHOD

Up to this point, only the theoretical framework for training program formulation has been considered. It is now necessary to explore the specific ways

in which these ideas are implemented. First, generation of the training variables will be considered, since they are central for developing potential

training programs.

7
Step 1 - Recode task B into variable row vector b.

Let: b = 10, 1, 1)

Step 2 - Choose a device matrix such as in Figure 2, and call it A.

Elements

1 2

Let: 1 1 0

Variables 2 1 1 = A

3 10 1 Step 3 Pre-multiply A by b and call the result c.

Let: bA = (0, 1, 1) 1 0 = (1, 2) = c

1 1

,12 1

Step 4 Save c for later element choices.

Element

1 2

c= (1 , 2)

Figure 4. An example of a device matrix being premultiplied

by a task variable vector.

8

1j
B.1. GENERATING THE TRAINING VARIABLES

The number and kinds of variables affecting training programs that have

been considered in the psychological and training literature is extensive.

Anyone attempting to systematically reduce these variables into a usable format

is struck immediately by the fact that in each experiment and taxonomy variables

are confounded in at least two ways. First, there are no standard variable definitions. Second, there is little or no effort made to assure that the variables partition the space of training alternatives in an orthogonal, or at

least, non-overlapping fashion. Many variables given different names have

within their domain the same variances attributed to other variables. There

are, of course, exceptions to this trend, e.g., Finley, et al., 1970.6 These

papers, however, do provide a good starting point for the selection of variables

used in this procedure.

To overcome the difficulty noted above, this paper used the following selection strategy. First the summary literature was screened for suitable training variables. The result of this screening was the division of variables into

three basic groups: (1) Variables seeming to have effects on training, but

with little empirical support; (2) variables having effects on training and

having empirical support; and (3) variables have some empirical support, but

with very mixed results with regards to training. Due to time constraints, it

was not possible to conduct independent experiments on all the variables. Thus,

there was no way to assess their consistent impact upon the training processes.

Consequently, groups 1 and 3 were excluded from further consideration. It

should be noted, however, the matrix framework that is used does permit inclusion of new variables at any time. In addition, the authors felt that until

experience is gained with the entire procedure, the inclusion of too many variables could be counterproductive.

Within the group of remaining variables, i.e., group 2, there were still a

number of problems to be resolved. The remaining variables were divided into

smaller groups based on the functional role they played in the experiments from

which they were taken. However, before this could be done, a set of working

definitions had to be developed so that variables could be compared across

studies. Without working definitions, the variables could be interpreted in

any manner and could leave the whole procedure too ambiguous to use. The classification of variables was accomplished using the strin analogy discussed on

pages 3-4. Each variable was evaluated and revised, and a preliminary set of

variables was developed.

In general, the variables were divided irto four classes, depending on

their domain of influence. these four classes correspond to the program and

content level breakdowns shown on page 3. Variables dealing with physical

properties of real equipment were classified as media variables. Variables

dealing with interrelationships between media items or social structure were

placed under the heading of method variables. Variables dealing with the type

of information flowing through a training system were placed under the heading

of content variables. Finally, variables dealing with the interrelationships

between content el...aents were placed under the heading of structure variables.

The following examples may help clarify what role these classes of variables

play in the training process.

9

r,

4. 1
Example 1

Media variables refer to actual pieces of equipment (or even humans) that

can be used to present, accept, or evaluate training information. These are

listed in Appendix A. (The media variables were extracted, extensively modified, and combined from TAEG Report No. 16 7 and Meister's Behavioral Foundations

of System Development.) For example, a television set is a fairly sophisticated

piece of equipment having the capability of presenting color, sound, and movement information. A workbook is also a physical device, but with less capabilities. In another sense, even an instructor can be viewed as a training medium

with information handling capabilities.

Example 2

Training method variables (Appendix B) refer to the patterns and sequence

of interactions between students and media. For example, team teaching is a

pattern of interactions between more than one instructor and more than one

student. The variables describing this pattern of interactions are method variables. The generation of the methoe variables required developing new techniques because no literature was found characterizing methods in a form compatible with the string analogy. Three steps were required initially. First,

the five questions discussed earlier on page 4 had to be generated. Second, all narrative descriptions of methods had to be screened and rewritten to

reflect the key differences between time. (The narrative descriptions of methods were extracted and modified from W. R. Tracey's Designing Training and Development Systems.8) Third, all methods had to be coded in terms of their differences based on the five questions. (See Appendix B.)

Another technique required selecting classes of method variables that would

link the task environment to the training environment. This technique consisted

of examining the sources of variation in operational environments, selecting

variable names to represent that variation, examining sources of variation in

the method descriptions, selecting variables to represent that variation, and

then matching the two sets of variables. The variables that overlapped, i.e.,

linked methods and operational environments, were coded in matrix form and adjusted until the space of methods were orthogonally partitioned by variable

values. The result was a reduced set of methods which could be shown to be

operationally different from one another. This in turn reduced the total set

of methods and refined their definitions markedly. Five classes of method variables were developed and they included: (1) The role of the element, (2) the

function performed in the role, (3) the stability of the function over time,

(4) the effect of physical environment, and (5) the effect of psychological

environment. (The coding of these variables is discussed further in

Appendix B.)

Example 3

Content variables deal with the particular units of information being

taught within the system. These include information such as task type. An

example of a content variable is "mathematical manipulation" (Appendix C).

10
Example 4

Structure variables refer to the way in which content variables are related

to one anotner They are concerned with such things as repetition, time sequencing, similarity, and spacing (Appendix C).

To generate final sets of all variables, another problem had to be resolved. The problem was the way in which the variables were to be extracted

from task analysis descriptions. Since any predictive technique will probably

be operating with input information in a variety of forms in early states of

weapons systems, the procedure by which information is recoded into variables

becomes very important.

One approach is to recognize that all textual information is presented via

language, and, consequently, the information carrying elements can be analyzed

psycholinguistically. In linguistic circles, breakdown of text seems to follow

two tacts. The first, transformational grammar, generally ascribed to Noam

Chomsky,9 corresponds roughly to structure variables. The other dominant tact,

case structure grammar, emphasizes the meaning (semantics) of the information

and corresponds more directly to the content variables. Typical proponents of

this view are Fillmore and Fodor. -0 In general, formal linguistic schemes, although having considerable value for future applications, such as real time

language analyzers, (Winograd, 1972)11 do not have immediate applicability for

the processing of task information. In practice, the simple procedure of breaking down sentence structure into noun, verb, and object with adverbial or adjectival modifiers appears quite adequate and has been used with some success

in various classification schemes. Identification of critical information is

often quite easy for a trained analyst and it is not necessary for him to break

task information into linguistic components. However, if such a model were to

be entirely automated in the future, a linguistics approach might be of considerable value.

Generally, the analysis of task information for training falls into the

following categories. Task verbs are useful for identifying the category of

action taking place in the task. When paired to a series of task taxonomies,

such as used in the ARI Training Manager Mode1,12 considerable insight can be

gained into the types of skills and knowledges required for a group of tasks.

By pairing the task types to operational training guidelines, it is also possible to generate unique configurations of instructional programs. The ARI

Trainvice Model13 represents one study in this direction. Frequently, nouns

in task descriptions apply to pieces of physical equipment or units of information. The appropriate category is clarified by adjectives. Similar clarifications for verbs occur through adverbs and various other modifying phrases.

The wide variety of descriptive phrases found in task lists seems natural because of the flexibility of the English language structure. A given object or

phrase can have almost any combination or permutation of phrase modifiers. For

example, a slide projector can be a random access, high speed, black plastic,

carousel projector, or it can be a low speed, sequential, white plastic, individual slide projector. In this example, the modifiers set bounds upon the

equipment. But even more, in the case of a task analysis, the modifiers specify conditions for either a functional or physical fidelity match. The most

useful insights to be gained from linguistic breakdown concern the way in

which the variables are identified to allow maximum transfer of information

from the task descriptions into the variable description lists.

11

9
After the initial variable screening process mentioned earlier, it soon

became apparent that most variables fell cleanly into categories formulated by

the string analogy and could be identified by the breakdown of the task information in the simple linguistic model by verbs, nouns, and modifiers.

The final sets of variables are presented in Appendixes A, B, and C. Several things should be noted about these appendixes. For each major heading,

such as MEDIA RELATED VARIABLES, pages 41-43, the first class breakdown of variables is in terms of the sensory mode. Depending upon the training function,

the trainee's ability to perform in that function depends on two things:

(1) The sensory modality through which information is received, and (2) the

sensory modality by which he expresses his response to that information. For

example, task information can be presented to him through visual, auditory,

tactile, external motion, internal motion, olfactory, or taste senses. Similarly, he can respond auditorily, physically, or internally. Each task situation in which the trainee is involved must flow through one of these human information processing channels, and, consequently, it is these channels that

form the most logical basis upon which to break down the sets of variables for

the model. At the start of each set of variables is a list of sensory mediums

by which the training information can be exchanged. A given task or training

situation may use one or any combination of these sensory modalities. A list

of variables exists for each modality which corresponds functionally to the

previously discussed adjectival or adverbial information in the operational

environments. For example, six types of "Visual Form" listed under MEDIA RELATED VARIABLES--STIMULI CHARACTERISTICS are all variables for visual sensory

information.

Once the organizational structure was chosen, the final sorting of training variables from the literature became straightforward. The string analogy

was used to determine the type of process each variable represented, i.e.,

Methods, Media, Content, or Structure. The linguistic analysis was used to

determine whether it was directly related to a sensory modality or whether it

was a modifier, limiting the modality and its range of operation. The variables

were refined to make better distinctions between nouns (equipment) and modifiers

(variables). The result of this refinement was a list of training equipment

(Appendix A-36-40), a list of training methods (Appendix B-54-76), and four lists

of variables broken down by sensory modalities and modifiers (Appendixes A and

B). These lists underwent one last screening to assure consistency and accuracy of definitions.

B.2. IMPLEMENTING THE MODEL

Once the variables were chosen, it was possible to select a specific

method to implement the model.

The model begins with preparation of task descriptions. (Refer to Figure 1.) Task information, in whatever form, is broken into the sets of variables by the following procedure. The lists of variables are examined to determine if each is applicable to the task or not. Because of the way in which

the variables are presented (see Appendix A-41-43), the analyst performing the

initial coding need only scan the lists for sensory modalities and record those

that apply. If a sensory modality applies, the analyst records the applicable

modifiers for that particular modality. If a sensory modality does not apply,

12
he does not have to scan the modifiers for that modality because they are eliminated from consideration automatically.

The next step, which may be performed either serially or in parallel with

variable coding, is the grouping of tasks into functional units for Liocked

training. (See Figure 1.) (There are many ways this can be accomplished. At

present no one procedure has been chosen, although a probable course could be

to utilize an existing program developed in the ARI Training Manager Model.

The output of this model is a tree diagram of related tasks based on similar

personnel, tasks, and equipment functions.) The grouping of tasks is necessary

to assure that program generation processes do not combine tasks in totally unrelated fashions. The grouping should be based on logically related functional

areas, since each task does not operate in a vacuum, but rather, occurs within

some operational context or environment. Recall that it is the operational environment which has considerable implications for the instructional method to

be chosen.

When a group of tasks has been chosen by the analyst, the method and the

media to be used in the training process must be determined. (See Figure 1.)

At a detailed content level of analysis, the particular information which flows

through the system, as well as the structure which interrelates that information, must also be derived. For analysts working with relatively gross task

data, this level of analysis is not practical. ThE manner by which content information should be structured and its effect on performance is still highly

debatable. Consequently, although the variables for the content level of analysis were developed and are presented in Appendix C, they are not used in the

current version of the model. When specific performance predictions do become

possible, the matrix format discussed earlier will be capable of easily incorporating this information. Thus, the present paper focuses its attention on

those areas most sensitive to cost, namely training equipment and methods, and

their subsequent impacts on available resources.

After tasks are grouped, two processes begin. For each task grouping, an

appropriate set of training media must be chosen. (See Figure 1.) In addition, operational context within which the media will be utilized must be determined. Just as the trainee interacts with actual equipment within some

operational battlefield scenario, he interacts with the training equipment in

the scenario of an instructional method.

B.2.a. METHOD SELECTION

The main problem to be overcome in method selection was the identification

of suitable variables permitting a linkage between operational and training

context. Direct physical fidelity matching used in the construction of simulators or the selection of training equipment was not feasible. Although the

answer seemed to lie somewhere in the functional fidelity area, the way in

which this linkage could be accomplished was not obvious. Another complicating factor was that teaching methods were not clearly defined in the literature.

The problem encountered most often was that both methods and equipment were

combined and defined as only method. Thus, many methods were actually very

similar, but seemed different, because they used different types of equipment

to perform the same functions. The first step, therefore, was to examine the

list of methods discussed in the literature and to sort them. Methods having

13

9

"4.
similar patterns of interrelationships were grouped together under the same

general methodological headings. In order to quantify the methods, a formal

scheme for classifying methods was created.

The string analogy proved useful for accomplishing this task. (See the

earlier discussion on method, page 10.) Since many teaching methods center

around interpersonal relationships between instructors and students, it was

most useful to characterize the variables describing these relationships in

human terms, although many interactions going on in training may be between

man and machine. Five variable areas appeared to characterize the majority

of the differences among methods. These areas considered five characteristics

of the training program. They were:

1. Who or what is performing the action or processing?

2. How many times does this action or processing take place in one instructional method sequence?

3. When this performance takes place, in what order is the person or

machine in the sequence?

4. What is the percentage of total control which this person exercises

over the instructional process taking place at this time?

5. What percentage of the total instructional time is taken up by this

activity?

Defining methods in terms of these questions led to the schematic, tabular

definitions found in Appendix B-54-76. In all, 23 methods were analyzed and

given both narrative and quantifiable definitions.

Once these definitions were generated, it was then possible to establish a mapping between the operational environment and the training environment for a

given group of tasks. As anticipated earlier, a method's benefits served as a logical link between the actual operational environment and training method.

Each method generally has ascribed to it certain idealized characteristics.

These characteristics are the reason for choosing particular configurations

over others. (See Appendix B.) Because the trainee is the focus, one logical

way to link the functional variables and training methods is:

Step One: Identify the idealized characteristics which expert opinion

attributes to each method.

Step Two: Based on the characteristics which relate the trainee to his

functional context in the classroom, ider,tify common variables between the

functional classroom description and the operational context.

Step Three: Retain those variables common to both situations and code

them in the process matrix.

The result of this process can be seen in Appendix B-79. Along the Y-axis are

five variables that were found to overlap both situations. They are: (1) The

role of the performing trainee within the operational context of the system,

(2) the particular function performed by the trainee within that role, (3) the

14

V
stability of that function over time, (4) the significance of the physical environment upon performance of the task, and (5) the significance of the psychological environment upon performance of the task. Each of these classes of

variables could theoretically have an indefinite number of levels. The number

of levels chosen for this paper was somewhat arbitrary; however, the number selected appeared to oe adequate for classifying the methods. Role was divided

into three levels describing functions performed by the trainee in his actual

duties and each class was broken down as follows: (1) supervisory, where one

or more persons are involved; (2) team performance, where the trainee has equal

roles with other individuals or equipment; and (3) individual performance,

where the trainee performs a given task on an individual basis. The function

was broken into four levels corresponding to the type of operation being performed in the role. Many more levels could have been used, for example, Willis

and Peterson 's14 task taxonomy with 12 levels could have been chosen equally as

well. It was not clear, however, how the advantages attributed to specific

methods of instruction could be divided much finer, and, consequently, only

mental, physical, perceptual, and communicative functions were considered at

present. Stability, which reflects how often the function performed in the

operational context changed, was divided into two levels, high and low, due

to lack of precise causal relations. Physical context, referring to the effects which physical characteristics such as cramped space or chemical warfare

environments might have on performance, was also broken into two levels. Psy

chological context includes mental influences, such as peer pressure, which

could affect job performance in operational situations. Other examples are

time pressure and motivation. Psychological context was also broken into two

levels, high and low. Should other variables prove useful in the future, the

matrix structure of this procedure will permit their rapid inclusion. The entire process has been designed to accommodate such modifications and allow for

new training researci' and policy changes.

Each of the 23 methods mentioned earlier was coded in terms of its ability

to manipulate each of the five variables for a trainee. The matrix is found in

Appendix 3-79, with each column corresponding to one of the methods and each

row to one of the levels of the functional context variables. If a given method

includes more than one level of a variable, it is coded in each row where it

applies. As can be seen from Appendix B-79, each method differs somewhat in

terms of which functional context variable it manipulates. Thus, it is possible to match the fielded operational context within which the trainee will work

with the variables describing the method which can produce similar training environments. It is recognized that for various reasons a particular school may

not desire to train an individual in a manner that corresponds directly to his

or t,er field environment. In such a case, adjustments can be made to the matrix to represent differences by changing their relative weights as shown earlier. One advantage of the current approach is that training is oriented toward

battlefield training, and any change would require explicit changes in matrix

weights, thus highlighting cases where battlefield performance was not the final

goal.

As more is learned about the relative advantages or disad,,:Antages of each

training method from actual per rmance data, the matrix can be selectively

weighted to produce a better ar.. better fit.

15
B.2.b. MEDIA SELECTION

The selection of training media follows much the same logic as method se- lection, except that the media selection process involves three subprocesses

corresponding to different information flows taking place in a training program. These flows are stimulus presentation, response acceptance, and feedback

evaluation. Each complete instructional sequence includes one or all of these

components. The three subprocesses may b.: performed by a single piece of equipment, a single individual, or a combination of both. For example, stimulus information may be presented by a radar screen, the response of the operator to

that stimulus may be an audio reply through a radio net, and the evaluation of

the correctness of the response may be a verbal one made by a commanding officer standing nearby. On the other hand, the entire sequence may take place entirely between the operatc- and a computer controlled threat evaluation system

contained within the weapon itself.

This kind of flexibility points out a problem with matrix or network selection systems which have only a single media selection process. Because one

matrix then includes all the selection subprocesses, it is inherently biased in

favor of a single device having the largest number of capabilities, simply because that device is most flexible. The result is that real equipment or simulators are almost always chosen, even though several smaller, but less expensive components can be fitted to duplicate information flows performed in the

same task. In this paper each of the three subprocesses has its own set of

relevant variables. The complete set of variables for each subprocess is found

in Appendix A. Even though each subprocess has its own special characteristics,

there is variable overlap. The set of variables for media corresponds directly

to the sensory characteristics of the training situations, i.e., stimulus, response, or feedback. The process matches the elements having physical and

functional fidelity in the operational equipment with the elements of the training environment in contrast to method selection, where the determination of a method was more directly related to social interactions between students and

the persons or equipment performing the media functions.

Media selection is made by comparing coded task data with selection matrices in the same manner as the earlier matrix example. These selection matrices are shown in Appendix A-44, 45, 48, and 51. At this time, there is not

enough data to put specific weighting values in the matrix, so 1 and 0 are used

to represent applicability or nonapplicability, respectively.

B.2.c. CONTENT AND STRUCTURE SELECTION

The two sets of processes which have not been considered up to this point,

involve those variables relating to the information flowing through the training equipment and their configuration. These were called content and structure

variables. The entire area dealing with individual differences and their effects and interactions with course material is hotly contested. For a predictive methodology, it would appear to be premature to attempt anything more than

general statements in this area. In the present model, several procedures were

considered to handle these remaining areas. The procedure that seemed most applicable for future use consisted of using a verb dictionary to select information content presented in each task set. Each entry in the verb dictionary is

paired to a categorical heading or task type. Associated with each task type

16
were lists of guidelines or flowcharts which served as a coursewriter's aid for

generating courseware material. Currently, ARI has an ongoing research effort,

Training Managers Model, handling the problem in this manner. Another effort

aimed at the prediction of training device effectiveness, the Trainvice Model,

also contains guidelines which could be paired to the same categories. It was

felt that these efforts are not sufficiently developed to be useful for early

training prediction although they have important applications at other levels

of training development. Consequently, this paper strives only to predict the

grosser elements of the training program related to costing media and method

choices. Because of this limitation, predicting overall field effectiveness is

limited.

B.2.d. EFFECTIVENESS PREDICTION

Effectiveness prediction is probably the most difficult problem to solve

in CTEA. An important contributor to this difficulty is ambiguity about the

range of effects which should be considered. Specifically, how extensively

should effectiveness be defined for CTEA in contrast with COEA or battlefield

simulation? Although there is no doubt that training directly influences field

performance, the linkage between training programs and terminal battlefield

performance is much more tenuous. For example, studying 10 hours using a carousel projector instead of 12 hours using a textbook may have a later impact on

battlefield performance, however, the chain of causal events is contaminated by

so many other factors that it seems futile to attempt predictive links at such

a level. What can be done is to break the causal chain into a series of larger,

more controllable, sequences of events, and attempt to predict them. Later,

that information may be used in combination with other factors as the basis for

predicting overall battlefield performance. The implication of this for CTEA

is that a measure of effectiveness must at present be taken from the battlefield and focused on the school where the degree of control is the greatest.

The measure chosen in this paper is even more restricted. What can be controlled is the efficiency with which a projected training program is developed.

Theoretically, there are least five sources of information in this model from

which an effectiveness metric can be fashioned. All four process selection matrices and the variables themselves could be used as predictors using a form of

regression equation. Many other pieces of information, also possible, are not

presently being considered, such as managerial constraints, personnel characteristics, teacher characteristics and a host of aptitude by treatment interaction variables.15 A practical choice for the present is to record alternative

costs produced in the generation of a projected program. Individual decision

costs are then combined to determine the overall loss for a predicted, but constrained, training program versus an "ideal" program with no constraints. Although such a metric supplies no information about the absolute scalar properties of a number so generated, it does provide a relative standard of comparison

for several training programs generated using the same method. This has potential advantages not only for the comparison of alternative programs within a

given weapon system, but also between systems.

In this paper the metric is referred to as an efficiency ratio. It represents a value which is composed of the total matrix "fit" of a projected training program with real world constraints, divided by the fit of an idealized

program which is subject to no constraints. The value is generated in the following manner. First, the selection matrix scores for each media matrix are

17
recorded. (See vector c in the example in Figure 2.) Repeated over all tasks

this produces three new matrices of c vectors where the rows now correspond to

tasks and the columns correspond to the degree of fit which that task had with

each of the potential training media. If the largest sum in each of these rows

is chosen, a large number of different equipment choices would be produced

spread over all the tasks which are being considered. Thus, for a sample group

of six tasks there might be six different media choices for each of the three

media functions or a total of 18 different devices. If cost were no object,

and there were no resource or time constraints, the ideal choice of media would

be all 18 devices. The total of the equipment fit scores for those 18 pieces

then represents an ideal choice without constraints. This, of course, is not

realistic, since there are always constraints that enter into any decision. As

constraints are app.led, the viability of a particular piece of equipment for a

particular task may change, just as it did in Figure 2. For example, even

though it may not be optimal equipment, the training manager may have on hand a

particular training device and want to include that equipment in the program,

rather than buy a newer, more expensive one. Perhaps the piece of equipment

permits combining several tasks rather than individually training each. In the

above discussion, this corresponds to choosing one of the other sums in a column representing the particular piece of equipment the manager desires to use.

If he were now to do this for all six tasks, i.e., force the choice of the same

piece of equipment, the sum of the numbers across tasks would be much lower.

Thus, the efficiency of that particular decision would be lower than the ideal

case, as would all other potential combinations. If this value is then divided

by the ideal first value, what would be produced is a percent ratio of actual

to ideal, or in the terminology used above, an efficiency ratio. This is the

approach used in this model. (See Figure 5.)

In actuality this procedure becomes much more complicated, when many tasks,

media functions, and media types are considered. The actual program is somewhat analogous to the statistical technique of hierarchical grouping analysis,

where the ideal program is a starting point, and the number of potential media

is gradually reduced in a step-wise function while keeping track of the tasks

which are subsummed under each function and media type. As the number of media

is reduced, a plot is made of the efficiency ratios for each combination. The

result illustrated in Figure 5 is a graphic aid to allow an analyst to identify

the point at which further trade offs in equipment begin to have significant

effects on the training fidelity of the system. This point is represented by a

sudden drop in the ratio value. This will become more obvious in the example

given later.

B.2.e. THE COSTING PROCEDURE

A final problem consists of the costing procedure by which the output of

the method and media estimations are analyzed. This model uses a costing program originally developed by the Navy for this purpose.? Equipment and method

variables are costed in terms of a set of 37 variables specifically related to

a cost accounting procedure. These components are then put into a computer

program which calculates the costed output. This cost model is currently in

use by the U.S. Army Air Defense School for overall training estimates in the

ROLArD system and is being considered for Army-wide application. A modified

version of this program is proving helpful in aiding decision makers after a

training program has been developed. An example of a preliminary application

0

18 `-',/
Step 1_ - Group all (c) vectors (see Figure 4) into 3 matrices corresponding

to stimulus, response, and feedback functions for media selection.

Task 1

Task 2

Task n

Stimulus

C

11 C12Cln

C

nl

C

n2'

Cnn

Response

C

11

C

12

...0

ln

...

C

nl

C

n2'''

C

nn

Feedback

C

11 C12 .0

ln

C

n1

C

n2''

Cnn Step 2 - For each task choose the largest value of Cif, i.e., Lid and add

over all tasks. This produces 3 sums, one for each matrix: S

stimulus,

Sresponse , and S

feedback'

s stimulus LTask 1 + L

Task 2 +.. LTask n

L Task 1 LTask 2 1-"Task n

s response

sfeedback LTask 1 LTask 2

+'

'LTask n Step 3 Add the three S values together for a measure of the overall fit

of the Cif chosen for all tasks and all functions.

Overall Measure = S

-stimulus + S

response

+ S

feedback

Step 4 - Record the overall measure. Repeat Steps 1-3, after removing

the media in each stimulus, response, and feedback matrix (Step 1) having

the lowest column sum across all tasks. Record this new overall measure,

and repeat until all media have been considered.

Figure 5. The generation of a single efficiency ratio

and plot of possible ratios.

19
Step 5 - Plot the ratios of the overall measure values, using the first

value as the denominator for all the remaining values.

First Overall Measure

Ratio 1 = 1 First Overall Measure

Ratio 2 Second Overall Measure= First Overall Measure = Some Value ( 1

Last Possible Overall Measure

Ratio n = First Overall Measure

Ratio

Value

1.0

.5

0

ROO MI.

= Some Value << 1

6, Value Representing a Single Efficiency

'Ratio

1 2 3 4

Ratio Number

Figure 5 (Continued)

20

3;
of the Navy program is available as part of an ARI Technical Memorandum16 written in late 1975 as part of a preliminary feasibility study of matrix selection

concept and costing logic in COEA.

C. AN EXAMPLE

In order to pull together the prediction techniques, a simplified example

is given. Figure 6 should be continually referenced for a clearer picture of

the interactions among components. To make the example as simple as possible

only a single task is considered and traced through the model. This will, however, artificially constrain certain areas. These will be elaborated as they

are reached. The following is a sample task:

Assume the trainee is to be used in an Air Defense System consisting

of an interactive radar scope connected to a computer and a print out

device for typed material. He is to detect malfunctions in the system

resulting in unpredictable error messages appearing on the screen.

Upon detection of an error he is to use the keyboard printer to make

another display scope operational and is to verify the result of the

change on the keyboard printer output produced by the computer.

The task description is first compared to the variables listed under MEDIA

RELATED VARIABLES--STIMULUS CHARACTERISTICS, Appendix A-41-43. Note that comparison of the task and variable list numbers 1-7, shows only variable 1 (visual

symbolic) to be applicable. Because sensory modalities numbers 2-7 do not apply, they can be eliminated along with variables 25-30 which apply to audio

stimuli only. Variables 8-24 and 31-39 must now be examined for their applicability to the task description. If a variable is shown to be applicable it

is scored with a 1, and if it is shown to be non-applicable, it is scored with

a O. These evaluations are kept on separate task coding sheets, like the one

found on pace 39.

The example involves observation of a visual display. Normally, the "Visual Forms," variables 8-13, which can be presente! on displays, include alphanumeric, symbolic, and graphic information (8, 9, and 10), but do not include

other forms, such as pictorial, solid object, or environmental (11, 12, and

13). Variables 8, 9, and 10 would each be coded as 1 on the coding sheet,

while variables 11, 12, and 13 would be coded as O. A similar procedure is

used for all other relevant variables. For instance, a message found on a display may change at any time, and is therefore not static (31). Messages presented on the displays are also quite random in time (33), rather than ordered

(32). Therefore, variables 31 and 32 would be scored with a 0 and 33 would be

scored with a 1. Once the entire list of applicable variables has been scored

on the coding sheet in this manner, the task will have been described for the

stimulus situation which initiates the operator actions. (See the first row of

the Sample Coding Sheet for the applicable variables for this sample task on

page 39. This entire row of variables corresponds to the row vector for the

stimulus situation.)

The next step is to generate the row vector corresponding to the MEDIA

RELATED VARIABLES--RESPOWE CHARACTERISTICS, Appendix A-46-47. Only the physical properties of the equipment which receive the operator's response to the

stimulus are considered. In this example, note that the operator's response is

21

3,2
GENERATE TRAINING VARIABLE

DESCRIPTIONS FROM INFORMATION

CONTAINED IN TASK DESCRIPTIONS

AND GROUP

4,

RESPONSE

EQUIPMENT

SELECTION

FEEDBACK

EQUIPMENT

SELECTION

RECORD EFFICIENCY ----a

FOR EACH GROUP

CALCULATE IDEAL

PROGRAM SELECTION-Si

NU,

APPLY REAL WORLD

CONSTRAINTS TO MATRICES

'4;1,

ROGRAM EQUIPMENT FOR COSTING COSTING PROGRAM

CALCULATE REALISTIC EQUIPMENT RECODE EQUIPMENT FOR,..

J1

[RATIO

(CALCULATE EFFICIENCY CALCULATE COSTffiFICI.-7--ENOY'e

RATIO FOR EACH PROGRAM

NO )(FOR EACH

GROUP

DETERMINE OPERATIONAL CONTEXT

WITHIN WHICH TASKS ARE TO BE

PERFORMED

-4/

METHOD SELECTION

RECODE METHOD FOR

COSTING

/

)IRUN COST PROGRAM

Figure 6. The flow of training program formulation.
=la

=NM =II

EMEMOMME II II

1111111111111111 MCM3M

O., .B1311120311113131113313211

1 , , osmoonnono

',000,00 1 U

mu 0 o ono o in o I o olli

"Ono 01 NM MN IM

. II Mum=

.211 ImmMIMI 1111. II ME , =UM =Ea li

M

NEM II

111111 AM 111i .,Mil IER

pp

N 111 5 i

0 0 0 0 0 0

EMI

1

I I I

ME EMI

IMO 1111111 OM MO MO II mamma mom mmommom m
overt, i.e., visible, and involves the manipulation of buttons on a keyboard

printer. Thus, variables 1, 2, 4, and 5 are coded as a 0, and 3 is coded as a

1 on the coding sheet. Likewise, the responses are not found to require major

physical movements and are coded as weak intensity responses (6). Since the

trainee is manipulating equipment, the responses he produces will be dynamicordered (9) and the equipment must be capable of accepting such a response.

Similarly, the response rate would probably be fast (12), the number of response

channels, limited (13), and the response distribution, individual (15). (See

the second row of the Sample Coding Sheet for the applicable variables for

this sample task. This entire row of variables corresponds to the row vector

for the response situation.)

The task is now coded in the same manner for feedback variables, Appendix A-49-50. In this example, the medium of information is written form (3).

The source of feedback is intrinsic (10), the type of feedback is both response

consequences (14) and system status (15), and the feedback distribution is

again individual (16). (See the third row of the Sample Coding Sheet, page 39,

for the applicable variables for this example. This entire row of variables

corresponds to the row vector for the feedback situation.)

After the task information has been recoded into variables directly related to media choice, another evaluation is required. A group of tasks would

now be evaluated in terms of operational context variables for method selection.

Because this example only deals with a single task, this coding is artificial,

since normally a method would not be chosen for a single task situation, but

rather for the overall environment within which a cluster of related tasks

would be performed. The method variables are found in Appendix B-77. (Note:

In Appendix C-82-86 there is another set of variables broken down by stimulus,

response, and feedback functions. These variables were drawn from the summary

literature, but are not used in the method selection matrix. A mapping did not

seem possible between the operational context and these variables at this time.

However, they are included in Appendix C as a resource, having undergone the

same sorting and reduction processes as the other variables. These variables

might be useful in the future generation of a predictive regression equation

for training methods.)

Coding the method process requires the same mechanical procedure used to

code the media subprocesses. For this sample task, the role performed by the

operator in the real environment is individual performance (3). The functions

performed in the role are primarily mental, in terms of analyzing printouts

(4); physical, in terms of selecting display console options (5); and perceptual, in observing the displays (6). The function is basically stable (9),

since it does not change often. The impact of the physical context (10) and

psychological context (12) is low. (See the fourth row of the Sample Coding

Sheet for the applicable variables for this sample task. This entire row of

variables corresponds to the row vector for the functional context variables

for method.)

Because the model does not presently use content or structure information

no other codings are necessary for this sample task. (Content and structure

variables are listed in Appendix C-87-98 only as a future research resource.)
Selection processes can now be performed. First, the media-stimulus selection matrix, shown in Sample Figure 7, is premultiplied by the stimulus row

vector. The result of this multiplication is the vector of sums (a), which reflects how well each device fits the conditions described by the task vector.

Vector (a) can be found on the bottom row of the sample figure. The pieces of

equipment showing the "best fit" are carrel (19), dial access information retrieval system--audio/visual (20), and computer (32). Because only a single

task has been used in this example, more than one piece of equipment shows the

same value. However, if more tasks had been used, the selection process would

have narrowed down the number of "best fit" pieces of equipment. If there are

a number of pieces of equipment that have the same value, the equipment that

costs the least would be the one selected. Complete descriptions of the three

devices selected are found in Appendix A-36-40.

In a similar fashion, the response matrix, found in Sample Figure 8, is

premultiplied by the response row vector. The result of this multiplication is

the vector of sums (b). Vector (b) can be found on the bottom row of the sample

figure. The largest value in the vector is 6, which corresponds to carrels

(19), demonstrators (33), mock-up and panel (34), and operational equipment

(35). The description for each of these devices can be found in Appendix A-36-40.

Finally, the feedback vector is multiplied. Its vector of sums (c) can be

found on the bottom row of Sample Figure 9. The "best fit" media are computer

(32) and operational equipment (35).

Now the method can be selected. Using the operational context (method)

row vector, a vector of sums (d) is produced from premultiplication of the

method matrix. (See Sample Figure 10.) The method selected is tutoring (13).

Its description is given in Appendix B-66, both in the detailed string analogy

and narrative formats.

For this example, the complete package would consist of a carrel for a

stimulus presentation device, the carrel for a response acceptance device, and

a computer for a feedback evaluation device. The "best fit" method would be

tutoring. If "instructor" had been included as a type of media in the selection matrices for media, the probability that an instructor would have been

chosen over a computer for feedback evaluation is high, since he probably costs

less and is much more flexible. In addition, an instructor would fit well with

the tutorial method selected.

This example has been made artificially simple and, therefore, is limited.

Only one task at a time being considered in the selections of stimulus, response, and feedback equipment and method. Because only a single task is used,

graphic analytic aids for the manager cannot be shown. If several tasks had

been used, there would have been three vectors of sums for each media subprocess. Each one would have been grouped in a matrix and have been subjected to

the procedure that generates the efficiency ratios. When a large number of unrelated tasks is considered, a number of equipment choices may be generated,

but when a small number of highly related tasks is considered, the result is

often a single piece of equipment.

25
OM C EC MEM= M 20 som 24MI 28 29 i gang maw

mmmmmmmnrImmIrnmrmmommArmmm EOM OEM imummmummmuu ummuummumusumm =mum giummumum

MMINIMMOIRME mmNmmmmmammlmm MINIM

=MEM iii MMIUM MMMIUMENIMEMEMOMMUMM MUMMIN MMIMMIMME MUMEM

MIN MTN NOON NOM NINM OIMMIMMINI OMNI IMMUMNIMMOMMONIN MONNE OMM MMM MEW,M mmminmirmrpormminti OMEN MMM m:421

mmo m mREINmElmunnrcolmEnEmommmmriAmm

MIMMINImamnEurmumummuunammommummnam MUMUUMMONINEMINUMUMMIN M MINION ONINNIMNIMMIMMONINOMIUMMMIUMM MI NE EMMEIN MMINNOMMIUMMINIONN MEM

MI MM ,Enm =MOM

m =MOM MMEMMEMOMMEMMEMEMOMMEMM mum- mum mommuumm m M IMMO ill iii NM OM Ilili LIJ =ME I MNINNMIMNI MEN MOON MI Mu 1111111

MIMS= MEMMEMMOO O MIMI =MOM EOM MMMOMMEMM

MOMNUMNIMMINUMUNUMEREMMIMMINNOMM IMIMINI MU OMIMMINIMMUUNNUOIEW au MIN MOMMININNI MMOMMINNUMIM ONNIM IMO= ETIMMOMMEMOMOR4AMOMMMUUME MIME MM M MEM= immommumummummummenummommm mom mommi

6

4 6 9 16 18 19

14

20

.
MIME 1139100UHUMMUUMMIEBE M. R 2629to 113611

IBMENNEFIRMMM7MNIMEMMMMRMPAMEIMMEMO NM MM IMMENNMEEMMIRMEMMUMMUMMINIEWOMM . NO MMOOMOOMMOMMOMMMMMMOUNOM MI NE 10.00===OEMOMMOOMOMMEMME MEMNON.= MO a 00101111== .MM =NMI NEMO O 01 is a OON= =OM === 000OOM a 0 a =MEN= a " M mm M M OEM M 0 UMW M MOM 0 Min= NO MN 0 M= IM MOH= EM EMEENEE

EMMERNMEM nnsilmr ONEOMMOMMEMMMMESIMEIMM MEMMMEMMO ODOM UM MO MUM MM MN MN IMRE NM= MOMEE0 ME En NM MM ME MM

fflIMMEE 'MMMMESMLIMMME MMERVESEM MINIENNEMIgMB OMME .........M. Ii 110=======MMM

ommmmmmommgmmonmsim M MIMPIMMMEMMIIMMMOIME MEMENEME OUUMMUMUNIONUOMMEN=MEMEEMMM

==== = UN ====== =INN =MEN= E MINIMMINI

m m #

' 9 9 m' Ell 9 " 910 m6 ENIONOME 4 U

=NINE= MININNININEMINNIMINNINNME

MUM MI === EN======

IMM

MIN MM MO EM ME area ME NM OM MI MO EN UM

6

26

29

30

39

10 10 10 10 10
ambra 13, eTpapi uoploaTes wraqem 

otpny pue ue441.4m ALup -V im 1.1V ;nq 

opny aqua my . y = 

MINIMNIMMONNEMMOMMUIMMIMMOMMEMMUM M N 

MEMOMMEMEM INMEMMINIMMINIMMINIMMEM 

MOM MMEMIMINIMMEMI INIMINEMMIUMM 

0111 oilo momemen0000.1momm0000 9 9 

II MIIMMOIMM IMMOMMOMMEMMEME MOMMIN 

ONE AIMMIIMM INIMMUMMII 

OMMIMMEIMEMMEMMRI 111 INIMMEMMEMEM 

EMEMEOMMEMMEMEMMERM 'IMMO ENEM= 

WWW@OUWOMMIWUWEIMMUURNEM. MENEMEMEN 

MINIMMENMEMSEMEMEMMIMMEMEMMEMIMMOMMINM. 

UOMMROOWWEIRM=MMEELItiMMoEU- U 

mdmaLumiamam FAMMEMMMEMMMM MENEM MN 

imminummemmemimmummumemm: Lim mummerm 

empummilm- momminummmmlummommemmmemmumm tommwomiQuuumingiiiamau EINEM willummmemmommasnonm 

van 

mminimmumm MME 

IIMMEIMME MI EM NUMMMMEMMEMMENIM 

UQUOUQUEWWWWW M-0 MEMMEMEMMEMEhigiWWWEN 

UNONUMEME EMMEMEMEMIMMI 

ORMEMERE ME NUMMMMEMEIMME 

QWWWISWW4I16iWiii0 MONOWWWW MEMME 

MOM ENIMMEME INUNEME Mu ME 

EMU= EMEMMEMM OMMEIMMMEEMMIMEM ISM MMOMMOUOmuMMOMMUIPMMUMMUOmUUMUM2MME 
NIEMOMLIEDEMBEEMEZIMM20111631111101=30EBEINEME136EEMI

MMIMIMMIMMUMMI MIMMMMO MMAIMMMUMMMMIMM MIMMUMM MMEMEMOMMMMOM M MIN

u mmummommmimommuummummommummommumm _nnnmarnr77777771nrammmrmmonnL omm

MI M MMIMMIMMMIMMMUMMMIMMIMMOIMMWMMOMMM

UN EMUMMINIMMIUMUMM IMMINIMINOMMIIMIM MOMMUMM MMIMUUM IMI MM.=

AMIMMUMMU inimill...... MI MUM U U NUEMI IM MOMMUMMIUMMER memmmmurn7n77r77mmmnonntrTmommmtimommilmm

lu MUMMOMUUMMIMMUMMOOMMI NM MMINIMMIM

M MEMEMUMMIMUUMUMMUMMUUMM MMM MMIUMMIMMU m MIMMEMMEMMM EMME E MN= MMMIMMUM

MEMMMMOMMMMMMEMMMOMMM MMM MMMMM MIMMEMMI mmticumaucam=mmun =mom mqm OM BUM= ArlffIERTIFMRINI xi4MEMMEIMMM4MOMEEM MIMMIMERMMOOOMOOMOMM IMMMIUM MEM MMIEM UMW 11.111 MEMMI NMI MEM RN IUMMMIIIIM MOM OM ME

EUM= OOMONEMEMEIM MIN M=e OMMUOUBAKIWOOMMMINUOUMOUUM"M MMUMMM IUMMUMMEMMEMMI EMIUMENIM Iii EMINIM NEMMEMMEMEMMEMMIMMEMMEMMEMEM

0 0 0 0 0 0 0

Figure 9. Media selection matrix.
I 1

Su ervisory 1 1111111111111111111111111111111111111210111111

MINE111301111111M111111111111111 Mil7PrIarlrirom OMME

P111171, Mgi:4,11111

r tt,- 610. ;11:_if±_3 +1 "Wl

4F'1"4"M.,

111311111111101111111111111111311.101111011111

MINI111111111111111111111111111111111111111111

17177#!7:711E MCITIMFAMIR !NM Egrts:174E1 EfERFIrla

IMEN111111000.011101111111111 Minn

...41 .7P2 -ter AMIE 4.1 " AZIONIN ninaninallellarMIMIME

'40

4,1

titt

Alt Alt
Cost information would also be required for the example. First, a variable sheet, like that shown in Appendix D, would be filled out for each piece

of equipment. This procedure is included because of its implications for including the remaining variables, such as number of instructors, floor space,

student turnover, etc. This data is punched out and submitted for computer

analysis. The output of this analysis, combined with the efficiency ratio,

constitutes part of the input for a CTEA of a given weapon system.

In its current state, the model is now undergoing its first stages of

testing. The costing portion, as mentioned earlier, is being actively used by

the Army Air Defense School at this time. The selection portion has been coded

into a computer program suitable for running on an IBM 360-65 operating system.

It is currently being used by the TRADOC System Analysis Activity (TRASANA)

Training Effectiveness Analysis branch for initial estimation in a variety of

CTEA studies. Greater model specification should result as experience is

gained. It is anticipated that weighting vectors for the processes resulting

from managerial variables, gross cost factors, and personnel characteristics

will be included and con. 'rally refined.

Many problems still remain in the training program formulation area. It

is hoped that the present approach may provide a useful starting point for the

CTEA requirements dealing with the early prediction of training programs and

help with the many trade-off decisions that must be made prior to full scale

development.

11111111111

it

A )W
REFERENCES

1. Interservice Procedures for Instructional Systems Development, TRADOC

Pamphlet 350-30, Vols. 1-5, Aug. 1975.

2. Cost and Training Effectiveness Analysis, TRADOC Pamphlet 71-10, 1977.

3. Col Hufnagel, point of contact for Army Training Study Group,

Autovon, #354-1461, 1978.

4. D. Meister, Behavioral Foundations of System Development, John Wiley

and Sons, 1976.

5. E. Haverland, Transfer and Use of Training Technology: A Model for

Matching Training Approaches with Training Settings, HumRRO Interim

Report, Oct. 1974.

6. D.L. Finley, et al., Human Performance Prediction in Man-Machine Systems,

NASA Report CR-1614, Vol.1, August 1970.

7. R. Braby, J.M. Henry, et ii., A Technique for Choosing Cost-Effective

Instructional Delivery Systems, TAEG Report No. 16, Orlando, Florida:

Dept of the Navy, Training Analysis and Evaluation Group, April 1975.

8. W.R. Tracey, Designing Training and Development Systems, American

Management Association, Inc., 1971.

9. N. Chomsky, Syntactic Structures, The Hague: Mouten, 1957.

10. J.A. Fodor and J.J. Katz, Eds., The Structure of Language: Readings

in the Philosophy of Language, Englewood Cliffs, NJ: Prentice-Hall, 19 . 11. T. Winograd, "Understanding Natural Language," Cognitive Psychology,

Vol. 3, pp 1-191, 1972.

12. Development and Validation of a Model for 0 timization of Performance

Baseu Training and Evolution of Machine Ascendant MOS, ARI Contract

#DAHC-19-16-C-0043, 1978.

13. G.R. Wheaton, A. Rose, P. Fingerman, et al., Evaluation of the Effectiveness

of Training Devices: Literature Review and Preliminary Model, ARI Research

Memorandum 76-6, 1976.

14. M.D. Willis and R.O. Peterson, Deriving Training Device Implications

from Learning Theory Principles, U.S. Naval Training Device Center,

AD26436Y and AD26438L, Vols. I and II, 1961.

15. Canyon Research Group, Inc., ARI Contract # DAHC-19-77-C-0040, 1978.

16. C.C. Jorgensen, A Methodoloav and Analysis for Cost-Effective Training,

jn the AN/ISO-73 Missile Minder, ARI Research Memorandum 77-26, Feb. 1978.

33

A+
APPENDIX A

MEDIA

1. Descriptions of Training Equipment and Materials (A-2-6)

2. Media Related Variables

a. Stimulus Characteristics (A-7-9)

b. Media Selection Matrix - Stimuli Variables (A-10-11)

c. Response Characteristics (A-12-13)

d. Media Selection Matrix - Response Variables (A-14)

e. Information Feedback Logic (A-15-16)

f. Media Selection Matrix - Feedback Variables (A-17)

35
TRAINING EQUIPMENT & MATERIALS

Visual Only Systems

1. Case Stuck Folders - Folders with illustrative studies showing cause

and effect information, may include pictures, graphs, maps, charts, etc.

2. Flash Cards - A set or deck of cards designed to present information

to a group or individual student.

3. Printed Materials - Handouts - Handouts are a class of printed

materials issued to a student for his use and retention to augment regular

instructional materials. They are usually instructor prepared, machine

copied materials of one or two pages highlighting specific topics or

updating existing materials.

4. Printed Materials - Performance Aids - Performance aids are a class

of printed materials that display data to aid in job performance or to

identify facts or background information. They include conversion

tables, data charts, schematic diagrams, equipment test tolerance matrices,

checklist routines, maps, and the like.

5. Printed Materials - Reference Books - Reference books are a class of

prinIalthaterials used to identify certain facts or for background

information such as dictionaries, encyclopedias, technical publications,

guides or manuals.

6. Printed Materials - Workbooks and Exercises - A class of printed

materials used to augment or replace instructional texts by providing a

mix of text information and/or practice exercises and quizzes.

7. Printed Materials - Textbooks - Textbooks are a class of printed

material dealing with a subject of study and used as a principal source

of organized information.

8. Programmed Text - A printed text containing frames of information,

either in the form of questions requiring the trainee to construct simple

written responses, multiple choice questions, etc. The material is

carefully sequenced, tested, and revised to ensure that a specific student

population will achieve stated behavioral objectives with a predetermined

level or success.

9. Filmstrip Projection System - A single frame projector or attachment

thereto that will accept a filMstrip format and project the film

images upon a viewing screen.

10. Microform - Microimagery, such as microfilm, used as a medium of

instruction.

36
11. Overhead Projection System - A system consisting of a horizontal

stagiFijietor designed to use a vertical throw for focusing an

enlarged transparency image upon a projection screen.

12. Slide Projector System - 2". x 2" - A class of single frame picture

projectors that will accept a standard 2" x 2" slide and project the

contained image upon a viewing screen.

Audio Only Systems

13. Audio Disc System - An audio system that uses a record player and

sound recorded on a disc (record).

14. Audio Tape Stem - An audio system that uses a tape recorder/

reproducer to record sound on magnetic tape.

15. Language Laboratory - Audio - An audio presentational device that

distributes audio information via a control console to s%udent stations

equipped with headsets, and may have a microphone for console/instructorstudent intercommunication and a tape recorder. Student may interact

with taped instructional material, rewind and play back or store responses.

Student responses may be monitored or recorded at console.

16. Radio System - AM/FM - A passive audio system consisting of a

broadcast studio, transmitting station, and student radio receivers.

The system uses designated AM/FM frequency bands for information

transmission.

17. Radio System with Responders - A multi-channel two-way radio

communication system that operates within UHF or VHF-FM frequency bands

'limiting broadcast ranges. Network may be open or use encoding/

decoding techniques or responders for individual channel privacy.

18. Telephone System - A telephone system with switching matrix

capability that allows multiple station two-way audio communication

at two or more remote locations.

37
Audio-Visual Systems

19. Carrel - A small enclosure or alcove incorporating a desk, can

be uiiiiU-One or two trainees and can be equipped with AV equipment or tools or print materials or a combination of these.

20. Dial Access Information Retrieval System - Audio/Video - Dial

access information retrieval is an electronic system for distributing

audio and/or visual materials and programs which are stored in a location remote from where they are dialed and received.

21. Filmstrip Projection System with Audio - A sound filmstrip

projector represents a family of audio-visual devices using single

frame visual filmstrips with sound on magnetic tape or records.

Visuals and sound may be manually or automatically synchronized.

Commercial equipment options include front or rear screen projection,

remote and stop action capability, and cartridge loading models.

22. Microform with Audio - Microimagery, such as microfilm, used as a

medium of instruction with audio tape or disc, etc.

23. Motion Picture Projection System - 16MM and Super 8MM Films -

A motion picture projection system using professionally or locally

prepared 16mm or S-8mm sound motion picture films for training.

Appropriate 16mm or S-8i projector and projection screen are

included.

24. Sound Slide Projection System - A system for presenting information by means o an audio tape and a series of synchronized

projected visual slides.

25. Teaching Machine - Still Visual/Audio - An individualized instruction

system composed of programmed instruction still frames, such as large

step multiple choice or fixed linear sequence, and/or synchronized sound,

and a manually controlled device to display the audio and/or visual

information.

26. Teaching Machine - Motion Visual/Audio - An individualized instruction

system composed of programmed instruction motion frames, such as large

step multiple choice or fixed linear sequence, and/or synchronized

sound, and a manually controlled device to display the audio and/or

visual information.

27. Televideo System - A telecommunication system that allows audio

and visual two-way communication between two or more remote locations.

38
28. Television - Cable (CATV) - A hybrid CCTV system offering selective,

multiple channel, encoded programming to cable network patrons. A

typical system consists of a signal receiving antenna system for the

master station and relay of amplified signal channels via area substations

to system subscribers. Programmirg may also be generated and transmitted

between substations offering multiple options for conference or training.

Programs are encoded for privacy and control of viewing audience.

29. Television - Cartridge (CTV) - A cartridge television system (CTV)

consists of packaged video tape programs, video recorder, playback

and display units, and control equipment offering high selectivity

and availability for individualized programming. Program cartridges

may be prerecorded, locally produced, or recorded off-the-air.

30. Television - Closed Circuit (CCTV) - CCTV is an electronic transmission

system for images and sound using a coaxial cable distribution system.

System design includes one or more studios or control rooms, a signal

distribution center, and signal distribution cables terminating in

reception areas equipped with receiver/monitors. Off air, live or

video taped programs may be used.

31. Television - Non-Magnetic Video Disc System - An experimental

form of television, siffil4iTriTurietion to cartridge television, in which

the program is encoded on a thin plastic disc, distributed to users where

it is rotated at high revolutions per minute on a player which reads

the data and sends program signals into the antenna terminals of a standard color television receiver. Random access capability.

32. Computer - A programmable electronic device that can store, retrieve,

and process data. It may manage and display information to a student,

accept student responses, provide feedback, perform calculations, etc.

33. Demonstrator - A low fidelity simulator that demonstrates manipulative

prindpTes, movement in time or space, steps of a procedure, etc.

34. Mock-up and Panel - A training aid used to demonstrate relative

shape, size, composition or function of an object or system display.

It may have moving parts, such as dials, switches, levers, etc., or have

non-moving parts, such as cutaway or layout models, etc.

35. Operational Equipment - A unit of car the entire equipment used on

the job for training purposes where extreme fidelity is required.

36. Physiological Trainers (Hostile Environment) Auditory - training

devices designed to place controlled stress on the human hearing system

through use of a physiologically and/or psychologically adverse sound

environment, to enable a trainee to learn to function in this adverse

environment.

39
37. Physiological Trainers (Hostile Environment) Visual - A training

device designed to place controlled stress on the human visual system,

through the use of physiologically and/or psychologically adverse or

low threshold visual signals, to enable a trainee to learn to function

in this adverse environment.

38. Physiological Trainers (Hostile Environment) Surface and Internal

Senses - A broad category of training devices designed to provide the

cutaneous, kinesthetic and olfactory sensors with physiologically and/

or psychologically adverse signals, to enable a trainee to function

in adverse pressure, temperature, pain or disorientating motion environments.

40
MEDIA RELATED VARIABLES

STIMULI CHARACTERISTICS

Medium of Stimuli Presentation

1. Visual Cues - Signals received through the sense of sight.

2. Audio Cues - Signals received through the sense of hearing.

3. Tactile Cues - Signals received through the sense of touch, including

sensations related to texture, size, shape,'or vibration of the skin.

4. External Stimulus Motion Cues - The sensations felt by a person when

he is moved by some outside force in such a way that his body experiences

roll, pitch, yaw, heave,. sway and/or surge.

5. Internal Stimulus Motion Cues - The sensations felt by a person when

he moves his arm, leg, fingers, etc.

6. Olfactile Cues - Signals received through the sense of smell.

7. Gustatile Cues - Signals received through the sense of taste.

Visual Form

8. Visual Alphanumeric - Words and/or numbers presented visually.

9. Visual Symbolic - Symbols presented graphically.

10. Visual Graphic - Two - dimensional figures, such as maps, graphs,

mathematical curves, etc., presented visually.

11. Visual Pictorial - Two-dimensional images, such as photographs,

drawings, etc., presented visually.

12. Visual Solid Object - A three-dimensional image or reality that is

viewed from exterior perspectives.

13. Visual Environment - A three-dimensional image or reality that is

viewed from inside.

Visual Movement

14. Visual Still - A static visual field, as with a still photograph,

drawing or printed page.

15. Visual Limited Movement - A basically static visual field with elements

that can be made to move, as with an animated transparency or simple panel

with switches that move.

41

5
16. Visual Full Movement - A visual field in which all elements can move,

as with a motion pictUre, flight simulator, or operational aircraft.

17. Visual Cyclic Movement - A visual field which moves through a fixed

sequence and then repeats the sequence in a repetitive manner, as with a film loop.

Visual Spectrum

18. Black and White - A visual field composed of either black or white

elements, as with the printed page or line drawings.

19. Gray Scale - A visual field composed of black, white and continuous

gradations of gray, as with a black end white photograph or television

picture.

20. Color - A visual yield composed of various segments of the visual

spectrum, as with color television or motion pictures.

Visual Scale

21. Exact Scale - Actual visual field or a one-to-one replication of

that field as with a full-sized mock-up, simulator, or operational system.

22. Proportional Scale - A representation of reality in other than

full scale, such as-a scaled model map or photograph.

Visual Contrast

23. Dim - A visual object which blends in with its background, i.e.,

there-lea small or no luminance difference between an object and its

background.

24. Bright,- A visual object which is brighter than its background,

i.e., there is a large luminance difference between an object and

its background.

Audio Sources

25. Tonal Sound - A very limited source of sound or noise which is used,

rather than speech, for signaling or warning, e.g., horns, whistles, sirens,

bells, buzzers, etc.

26. Voice Sound - A limited source of sound which enables spoken words

to be used as the medium of communications, but not suited to more

demanding tasks, such as music or sound recognition exercises.

27. Full Sound - A source of sound that contains all the significant

elements of the sound and is suited to the demanding task of sound

recognition exercises.

42
28. Ambient Sound - A complex sound environment with sounds emanating

from various sources and from various directions, including background

noise and task significant sounds.

Audio Stimuli Intensity

29. Weak - Audio stimuli presented to the trainee which have weak intensity.

30. Strong - Audio stimuli presented to the trainee which have strong

intens y.

Stimuli Presentation

31. Static - A unitary stimuli situation, i.e., stimuli are presented to

the trainee "all at once", e.g., batch presentations.

32. Dynamic-Ordered - A sequential stimuli situation, i.e., stimuli are

presented to the trainee sequentially or in an ordered manner over time.

03. Dynamic-Random - A non-sequential stimuli situation, i.e., stimuli

are presented to the trainee randomly over time.

Stimuli Presentation Rate

'34. Slow Rate - A slow rate or speed of presentation of stimuli to the

trainee, the trainee a long or maximum stimulus analysis time.

35. Fast Rate - A'fast rate or speed of presentation of stimuli to the

trainee, al the trainee a short or minimum stimulus analysis time.

Number of Channels or Sources

36. Limited - A small number of sources, channels, or Instruments

through which stimuli are presented to the trainee.

37. Unlimited - A multiple number of sources, zhannels, or instruments

through which stimuli are presented to the trainee.

Stimuli Distribution

38. Individual - All information is presented directly to one individual

trainee.

39. Group - Information is presented to a group of trainees, allowing only

indirect access to the information for an individual.

43
M©©© BODEUEEMMEMMID EfIEBEEM26MILEICIEBEEILV ELICIEM =MUM =MU= NIMMUMUEOUMBOUUMOMMUIMMI

MMEMMIMIIMEMM IIIIIMUOMUUUDENIMMUMMEMMIMIN

MMIUMNEOM 11111111011111111.11111111111.11111 ME MMEMMIIIIMMUNINIMMNIUMMIAR EN CM EMINMERMIUMIMMU ME 10 1111MMIMM 11M N ME UMMMMEMIUMMMIMMIMMIM 11111111NMEMM II MEM= M nalummummummummunammummumm simmummm m momimommmemennammomemsmummomm

QUINIMMUMMUMUMIIIIMMUMMUUMMMUMMMIMMUMM W MWMEWMEMUUMMOMMMUMUMUUMMMMUWWIE MEN WIUMOMMEMINI OMMOMMIMMINMEMMO MOM WIMMEMIIMMI 111111111111111MINIMMUMMOIMMII MIIMMMMIIMMEOURIIMMINIDOUOMEMERM MUM= M EMMEMMMMIKOMMMIUMMIIMMEMOMMU MIMMIU M EMEMEMMINIMMERE 0111111111MMEIMMMEIMMI01

MEIMUMMIIMEMMUMIIMMOOMMUUMEIMMIIMMIIMILIMMI MMEMEMNUMBOUNIIMEMMOUBOONM MEIMIMMIMMEI MEMENUMENMUMENNENU OSIIMMILINOMMIIM

MMIBMIIMUUMMNIMMUIMMUOMMEMMMMEMOIM

MINNEMOMMINIMMEMINMEDIMMMMEMEEMIIMMI

IMIN =sump minimuumwmimmilmiummmin EMMIIIMMINIUMMOM IMMUDOMMIMIMMINIMMIUMMUM

20

20
EMEEEMEM gi NEMO3211 =GO OLEGmUCUMMIUM MMEMMENUMUU ISIMMOMMUUMEEMMENUMMumul MEMERNMMEO IUMMENMEMMIMEMINEMEMIN

MMIMMIMME MMMOUOOMOMMONOMMOMOMMOMMMOM ME =EN= 1110000MOMOUEOMMIMMEMMEIMMEME M IMMIM MI 000 01100MOMM0 MMUMMM MOMOMMOM 01100n 0 MIRO amass EM

OMOMME0IMM000000011 0000MMOOM EIMMME MMMEMMEMEADOMMNIMMON =ME MMEN MEM

MENMEMENRUOUOIONIUMBOOMMIMMEMMOMM MMM

-MEMIUMMEMEEMEMONOOMMOMOOMMOMMEMEMOIM ii

EUMMIMMIMMEMMUOUOMMOMMONOMMMOMMIMMMMM M

-INUMUMMIUMMIUMIUMBUBOUOUNNEMMECEINMMMMX LAMM MMUMOUUMUNU MUM MMINIMMIMMMIMMM OMM IUMMUM UMMINIMEMMEMEMMMEMMEM =MEM= UMMOUDIOUEOMME MIIMMUMENIMMINIMM

lIMMEMMIMUMUMOMUMUUMUMUNIMMOMMEMMEMEMM 111MOME0E IIMMUIMINIIMMEMMIMMUINEMNMEM MMEIMMI NIMINEMOMMIN OMMIMMEMEMOMMO 1110111.101111111MMO00 M NM IIMMEMER MMINIMMMEMIMMO MUM MME

MMIIMMOIMMMEMEM IMMIIMMO MO M= OMEMMORMOMME MMMMIN INI ME MMEMEMMEMENEMEEMEMMEMEMOMMMIOMM

30

10 26

A Audio Only V Visual Only
MEDIA RELATED VARIABLES

RESPONSE CHARACTERISTICS

Response Mode of Implementation

1. Overt Response - Verbal - A response which the trainee expresses in an audible (verbal) manner, such as a verbal short answer response to a question having a limited set of correct answers, a conversational response, or a verbal decision response.

2. Overt Response - Written - A response which the trainee expresses

in an observable (written) manner, such as a free style written response, a written multiple choice response, or a written fill-in-the blank response.

3. Overt Response - Manipulative Acts - A response which the trainee expresses

in an observable (manipulative) manner, such as the small movements of dials, switches, keys, or small adjustments to instruments or the large movements of

levers, wheels or use of hand held tools.

4. Overt Response - Tracking - A response which the trainee expresses in an

observable (tracking) manner, such as continuously controlling a constantly

changing system, e.g., steering an automobile.

5. Overt Response - Procedural Performance - A response which the trainee

expresses in an observable (procedural performance) manner, such as

performing a sequence of steps in a procedure, e.g., carrying out the items

on the checklist for preflighting an aircraft or turning on a ra6ar system.

Intensity of Response

6. Weak - Responses made by the trainee with weak intensity.

7. Strong, - Responses made by the trainee with strong intensity.

Response Implementation

8. Static - A unitary response situation, i.e., responses are made by the

trainee "all at once".

9. Dynamic-Ordered - A sequential response situation, i.e., responses are

made by the trainee sequentially or in an ordered manner over time.

10. Dynamic-Random - A non-sequential response situation, i.e., responses

are made by the trainee randomly over time.

46

5 ;:,
Required Response Rate

11. Slow Rate - A slow rate or speed of trainee response, i.e., a rate which

allows the trainee a long or maximum response time.

12. Fast Rate - A fast rate or speed of trainee response, i.e., a rate which

allows the trainee a short or minimum response time.

Number of Response Channels

13. Limited - A limited number of sources, channels, or instruments through

which required responses are made by the trainee.

14. Unlimited - An unlimited number of sources, channels, or instruments

through which responses are made by the trainee.

Response Distribution

15. Individual - One individual trainee makes the required response.

16. Group - A group of trainees make the required response.

47
t

MOMMEMEOU OrerererENEEMBECTIMITME MECMOMM

MMERMEMEMMOMMEMMOOMMMOMMUMMEIMMMOIMINMEMM MMEMINMMU1MU1MIUMMKM11 MUM MIMI MIIMMUMMEMMOMMIUMM

MMMOM MOMMIMMOMMEMMINOMMOMMEIMM

MIU MMEMMOMMEMEMMINIMMMEMMOMMOIMOM

=MO MMIMMIIME IMMINIUMNEMM

MOIMOMMOMMINIMMINM =MN

INIMIMMINUM ME

MMINMEIMEMOMMUMOMM MMIIMMUMMIMMOKIMM ME MIIMMMMUMUMMIN VI

MMUMUMUMUMMI II MINMEMMIIMMEMMUIMMMO Mil

.MMUMMMUMUMMEMMUMMUMMOMMEMIIMMIMMMEMME

MMIMIUMMOIMMOIMMOI ME

-MMUMUMUMUMMIM MOM MMOMMEMPIMMIUMMIMME MEUMUMUMUMMO NMI MEMENWOM MN MINER

,WMUMUNUEUMMIMMUMEMIUMIUMMIIIM . MMIUMMMAMM MMOMMIUMMIMMIMMIN IMMINIMMMEMMAMMIIM MOMMOMONMEMIIMMEMMM IMMIIMMIMMOIMMIMMIUM IMMIONIMMUMMIUMMIIMMOMOMMOM MIUMNIMMIMM MINIMMEMMEMUMMMOMMEMMIMI =ME 1111111 MINIMMEMEMIMM MMIMMOMMMOMM MUM OMMMOMMOMMUMME MOMMEMOMMOMMOMMIB

MINIMMIUMEMMOMMEMMINIMMUMMIUMMMIIMENEMMIM

OMMINIMMEMOMMEMMINIUMOMMIMMINIMMUMMINIM

IIMMINIMMEMINIMM MONMIUMMINOMMEMEMEMEIMEMMMOMMOMMU

IIIIIMMMINIMEMEMNINEMMINIMEM

10
MEDIA RhLATED VARIABLES

INFORMATION FEEDBACK LOGIC

Medium of Feedback Presentation

1. Visual - Feedback presented visually by means of a display, it may be

coded and transmitted visually to the trainee.

2. Aural - Feedback presented aurally by means of a display to the trainee.

3. Written Form - Feedback presented to the trainee in written form.

4. Face-to-Face Communication - Feedback presented by direct verbal means

to the trainee.

5. Indirect Communication - Feedback presented by indirect verbal means,

such as by intercom, telephone, or radio link.

6. Tactile - Feedback presented to the trainee through the sense of touch,

including sensations related to texture, shape, size, or vibration of the skin.

7. Kinesthetic - Feedback presented to the trainee by either internal or

external bodily movement, such as reaching, grasping, tilting, etc.

8. Olfactile - Feedback presented to the trainee through the sense of smell.

9. Gustatile - Feedback presented to the trainee through the sense of taste.

Source of Feedback

10. Intrinsic F r Information or cues built into the system from which the

trainee interprets feedback information.

11. Extrinsic F - Information or cues not inherent in the trainee action or

system operations but is supplied by an external source.

Type of Feedback

12. Research Correctness (Rcrl - Information about the correctness or

incorrectness of trainee's response, when several response alternatives are

possible and the correct choice is not known to the trainee in advance.

(Also known as augmented feedback.)

13. Response Correctness (Rcf) - Information provided to the trainee (or

others who need to know about his performance) that he has in fact performed

an operation, but does not say anything about the longer range consequences

of the action taken.

49

53
14. Response Consequences (Rcn) - Information about the consequences of the

action taken. It confirms the response made by the trainee, and the

correctness of a response can be inferred only from its consequences. May

also serve to cue the trainee to perform the next response in sequence.

15. System Status (Rss) - Information about the condition of one's own

or another system or the external environment, on the basis of which a

trainee or team must act. Information is not necessarily (or even frequently)

the immediate consequence of or follow-on to a specific trainee/team action;

it may reflect system events that have been put in motion by much earlier

trainee actions. Provides information that regulates trainee and system

actions in the sense that when a particular status condition occurs, the

trainee must often take action to maintain the integrity of his system.

Feedback Distribution

16. Individual - Feedback is presented to one individual trainee.

17. Group - Feedback is presented to a group of trainees, allowing only

indirect access for an individual.

50
BEIMEMBOUMEEMEMEMME MEMEGOMMEnMENEOMMMO

M MIMMOMMIMMUMMIUMMEMMBOOMMOIMMOMMOMMOMM

BEIRMEMEMMINMOUOIRIUMMEMOOMMMEMMMOMMIUM E MIIMIMOMIUMUMMIMMOMMMIMIMOMMIMIMMMi UNNNIN INUMNIMININU UNIMINMINMNIMINN

EI INNINMENNIMMIONEENNONNENEMENINN INM UMINIMIMIMMUMINNIMMI IIMIIIMINI ME

INNIMMINMIMIMMINIMINON MINNINIMIMIMIN UMMIIMMM MUMMWMEMMIOMMUNMUMEMOMMIU UMMEMMEM INEREMEMMINIMMEMEMMMIROMMIIM

OMMINEMUMMUMUUMMUOMMUMUMMIENIMMEMMMEMM

UMEIMMEMUMMUUMUUMUUUMUUMM IMIMMMIU 1111M

MEMOMMMUMUUMIUMMUMMINOMMUMINIMMOMMIMME

MIIIMMIUMMINEMOMUMMIMMM MMIIMOMMM MOM

-MIIMMIIIMINI 1111111.11111MMEMOM MINIMMIMMIINM Mull IMMEM IMMUIMOMMEMMIIIMM MINIMMIMMEM mum mmolummenammenimmimmmmmmmmmem mommumm mennemmenmemmommmunimmomm

INNI IMNINUMNIINIIMNIMMIIIMIMMIMI

INN. INNNIMONIUMMINNIMINNEMMEIMMN ' MINIMUM UMM MMINNINNINININNUMNI IIIIMMIIIMOMINNIMNIMIMMI =MO INNIMIMINNIMNONINNIMNI NNU UM ONENNENENENUNNUNNNENNNI EN ENNE

10

20
APPENDIX B

METHOD

1. Descriptions of Training Methods (B-2-24)

2. Functional Context Variables (B-25-26)

3. Method Selection Matrix (B-27)

DESCRIPTIONS OF TRAINING METHODS -- CODING EXPLANATIONS

1. Who is Performing?

a. I- Instructor

S=Student

M=Media

b.

1, S1, or M1 First individual or group

S2, or M2 Second individual, or group

Etc.

c. (N4)=Expected number of individuals or more

(N-)Expected number of individuals or less

2. How Many Times is He Performing?

a. 1-Once

2-Twice

Etc.

N- Number of times

3. Where is His Performance in the Sequence?

a. 1=First

2=Second

1,3=First and third

Etc.

....=Cyclical sequence

53

C'

(N=Number in cell)
Jl

L t S iNioti

DESCRIPTION:

. A semiformal discourse in which the instructor present a series of events, facts, concepts, or principles; explores a problem; or explains relationships. Trainees participate in a lecture mainly as listeners. A lecture is basically a means of "telline'trainees information

they need to know. Not all talking done by an instructor during a class period can be termed a lecture, the term describes a more formal presentation used to achieve an instructional objective.

METHOD VARIABLES STIMULUS RESPONSE FEEDBACK

WHO IS PERFORMING? 1

1 S1 (150) Ii

HOW MANY TIMES IS HE

PERFORMING?

1 1 1

. WHERE IS HIS PERFORMANCE IN THE SEQUENCE?

_1 1

WHAT IS HIS DEGREE

OF CONTROL? (%)

100 100 100

W)W Lnmn IS HIS

PERFORMANCE? (% TIME)

100 100 100
TRATNINn mminn. LECTURE-TEAMTEACH (2) .

DESCRIPTION: A semiformal discourse in which two or more) instructors present a series of events,

facts, concepts, or principles; explore a problem; or explain relationships. Trainees

participate in a lecture'aainly as listeners. A lecture is basically a means of

"telling" trainees information they need to know. Not all the talking done by the

instructors during a class period can be termed a lecture, the team describes a more

formal presentation used to achieve an instructional objective.

METHOD VARIABLES STIMULUS RESPONSE FEEDBACK

WHO IS PERFORMING?

ii

2

s1

(15 +) I1

HOW MANY TIMES IS HE

PERFORMING?

2 1 1

WHERE IS HIS PERFORMANCE IN THE SEQUENCE?

1,

2

3 1 1

WHAT IS HIS DEGREE

OF CONTROL? (%)

60

40

100 100

Hnw LONG IS HIS . PERFORMANCE? (% TIME)

40

60

100 . 100

6,3
I I I 1, I I

DESCRIPTION: Group discussion techniques are used to reach instructional objectives. The techniques inclI,Je questions, answers, and comments from the instructor in combination with answers, contents, and questions from the trainees, and are directed toward attainment of learning goals. The objective in directed discussion is to help trainees acquire better understanding and the ability to apply known

cies, cr procedures, facts, principles, concepts, poli- or to provide trainees with an opportunity to apply their knowledge. The function of the instructor is to guide the discussion in such a way that the facts, principles, concepts, or procedures are clearly articulated and applied.

METHOD VARIABLES STIMULUS RESPONSE FEEDBACK

WHO IS PERFORMING?

1

(10-) 1

(10-)

1

S1 (10-)

HOW MANY TIMES IS HE

PERFORMING?

N

N

N

N

N

WHERE IS HIS PERFOR-

"DANCE IN THE SEQUENCE?

4,... 2, 4,...

1 , 3,...

4,..,

_

WHAT IS HIS DEGREE

OF CONTROL? (%)

60

40

50

50

60

40

HOW LONG IS HIS

PERFORMANCE? (% TIME)

60

40

50

50

,..

60

40
TRATNINr, MFTHOn! rrINFFRFvF-FMUL12(4

DESCRIPTION: Group discussion techniques are used to reach instructional objectives. These techniques include

questions, answers, and comments from the instructor in combination with answers, comments, and questions from

the trainees, and are directed toward attainment of learning goals. The purpose of the seminar is to find an

answer to a question or a solution to a problem. The instructor does not have an answer or a solution; in fact

there is no known best or correct solution. Rather, he is seeking an answer, and he uses the group to develop

one. The primary functions of the instructor are to describe the problem as he understands it and to encourage

free and full participation in a discussion aimed at (1) identifying the real problem, (2) gathering and analyzing data, (3) formulating and testing hypotheses, (4) determining and evaluating alternative courses of

action, (5) arriving at conclusions, and (6) making recommendations to support or arrive at a solution or a

decision.

METHOD VARIABLES STIMULUS RESPONSE FEEDBACK

WHO IS PERFORMING? I1

S1 (10-)

1

(10-)

1

1

I1

S1 (10-)

N N N

HOW MANY TIMES IS HE N N N

PERFORMING?

1, 3,... --I, 3,...

WHERE IS HIS °ERFOR- 2, 4,... 2, 4,... 2, 4,...

MANCE IN THE SEQUENCE?

50 50 50

WHAT IS HIS DEGREE 50 50 so OF CONTROL? (%)

HnW LONG IS HIS

PERFORMANCE? (% TIME)

so

50

50

50

30

50
r; DESCRIPTION The instructor actually performs an operation or does a. job, thereby showing the trainee

what to do and how to do it; he then uses explanations to point out why, where, and when it is done. Usually, the trainee is expected to be able to repeat the job'or operation

after the demonstration.

METHOD VARIABLES

. STIMULUS RESPONSE FEEDBACK

WHO IS PERFORMING? Il sl (10-)

HOW MANY TIMES IS HE

PERFORMING?

WHERE IS HIS PERFORMANCE IN THE SEQUENCE?

1

2

1 1

2

WHAT IS HIS DEGREE

OF CONTROL? (%)

60

40

100 40

60

HOW LONG IS HIS

PERFORMANCE? (% TIME)

60

40

100 40

60
fr

TRATNTNV METHOD! PERFORMANCE-MANUAL-INDEPENDENT PRACTICE (6)

DESCRIPTION: A method in which the trainee is required to perform under controlled conditions, the operation, skill, or movement being taught. Performance is learning by doing. In independent practice, trainees work individually and at their own rate.

METHOD VARIABLES STIMULUS RESPONSE FEEDBACK

WHO IS PERFORMING?

MI

S1

mi

HOW MANY TIMES IS HE

PERFORMING?

N N N

WHERE IS HIS PERFORMANCE IN THE SEQUENCE?

1

1 1

WHAT IS HIS DEGREE

OF CONTROL? (%)

100 100 100

Hnw Lorin IS HIS

PERFORMANCE? (% TIME)

100

.

.

100 100

,

'13
TGAINTmn MFTHDO! PFRFARMANrr-MANHAI.STWAT MTN (7)

DESCRIPTION: A method in which the trainee is required to perform under controlled conditions, the oper- ation, skill, or movement being taught. Performance is learning by doing. In the student tutor method, trainees are paired and members of each pair perform alternately as instructor and trainee.

METHOD VARIABLES

.......1 ,-..........- .......-......14.,,e.....P...1 ......r Jo.. .. .. . STIMULUS

....., ... ....... .... ,--0.-.....

RESPONSE

.. -...........-........,.......

...,... 4 M1

S

FEEDBACK

/.. - . II .,4 4. .. P ,.. .....4 ... WHO IS PERFORMING? 14/

S

r .........,...

HOW MANY TIMES IS HE N

PERFORMING? N

N

N

1, 3, 5, 7,...

WHERE IS HIS PERFOR- 2, 6,... 2 2, 6,...

MANE IN THE SEQUENCE? 4, 8,... 4, 8,...

50 . . 50 50 WHAT IS HIS DEGREE 25 50 OF CONTROL? (%) 25

25 25

HOW LONG IS HIS 50

25

50

50

50

PERFORMANCE? (% TIME) 25

25 25

'1 *)
cr

20.1 I _ kl P P 11 C 8

DESCRIPTION: A method in which the trainee is required

tion, skill, or movement being taught. Performance is

of trainees perform an operation or function involving

to perform under controlled conditions, the operalearning by doing. In group performance, a group

teamwork.

METHOD VARIABLES STIMULUS RESPONSE FEEDBACK

WHO IS PERFORMING?

mi

S1 (5-)

S1

(5-) mi

s 1

(5-)

N N N

HOW MANY TIMES IS HE N N

PERFORMING?

1 1 1

WHERE IS HIS PERFOR.

MANCE IN 1HE SEQUENCE?

2 2

.... ..............., 50 100 50

WHAT IS HIS DEGREE 50 50

OF CONTROL? (%)

HOW LONG IS HIS

PERFORMANCE? (% TIME)

50 100 50

50

7 '3
cr

ThutLucuiEmm:EERFznELLLLALpaptosnaunpjmau_a)4 -t;

IN

DESCRIPTION: A method in which the trainee is required to perform under controlled conditions, the opera- tion, skill,. or movement being taught. Performance is learning by doing. In group controlled practice, trainees work together at the rate set by the instructor, step by step and "by the numbers".

METHOD VARIABLES STIMULI'S RESPONSE FEEDBACK

WHO IS PERFORMING?

11 s1 (5-)

1I

11

S (5-)

N N N

HOW MANY TIMES IS HE N N

PERFORMING? N N

1 1 1

WHERE IS HIS PERFOR- 2 2

. MANCE IN THE SEQUENCE? 3 3

40 100 50

WHAT IS HIS DEGREE 30 30 OF CONTROL? (%) 30 20

20 I. HOW LON(i IS HIS 40 20

PERFORMANCE? (% TIME) 40 30
TRATNTrim MFTHOfl prorrammo_ITIRtructim (10) .41111,

DESCRIPTION: A method of selfinstruction in which the trainee works through a carefully sequenced and

pretested series of steps leading to the acquisition of knowledge or skills representing

the instructional objectives. The trainee proceeds through the program at his own rate,

responds actively (or covertly) to each step in the sequence, and receives immediate

feedback on the correctness of his response before proceeding to the next step. Programs

are usually designed to permit the trainee to master the desired knowledge 1r skills.

METHOD VARIABLES STIMULUS RESPONSE FEEDBACK

WHO IS PERFORMING?

HOW MANY TIMES IS HE

PERFORMING?

N N N

WHERE IS HIS PERFORMANCE IN THE SEQUENCE?

1

WHAT IS HIS DEGREE

OF CONTROL? (%)

100 100 100

iinw LONG IS HIS

PERFORMANCE? (% TIME)

100 1.00 100

S
TRATNIN9 mFTHOn. STUDY ASSIGNMENT- INDIVIDUAL STUDY

DESCRIPTION: In the study assignment method, the instructor assigns readings in books, periodicals, manuals, or handouts; requires the completion of a project or research paper; _or prescribes problems and exercises for the practice of a skill. In independent study, the trainee carries out the assignment without instructor assistance or direct guidance.

METHOD VARIABLES

. ...... ,...... , ,. .., STIMULUS RESPONSE FEEDBACK

n.. . ... I Yai. .- / -.1 ..... o.,.......4

S1

S

I1

... . .. ............_.... .,_. 11

Ni

s1

.__. .4,.....,_ ... .., A... -,. 4.

Si

. WHO IS PERFORMING?

1 N N

HOW MANY TIMES IS HE N 1

PERFORMING? N

1

WHERE IS HIS PERFOR- 2 . 2

MANCE IN THE SEQUENCE? 3

20 100 70

WHAT IS HIS DEGREE 30 30

OF CONTROL? (%) 50

5 100 70

HOW LONA IS HIS 40 3o

PERFORMANCE? (S TIME) 55
0'Ui

TRAINJNO MFTHOO! STUDY ASSIGNMENT-SUPERVISED STUDY (12)

DESCRIPTION: In the study assignment method, the instructor assigns readings in books, periodicals, manuals,

or handouts; requires the completion of a project or research paper; or prescribes problems and exercises for

the practice of a skill. In supervised study, the, trainee carries out the assignment with an instructor

available for guidance and assistance.

METHOD VARIABLES STIMULUS RESPONSE FEEDBACK

WHO IS PERFORMING? I1

Mi

S1

HOW MANY TIMES IS HE

PERFORMING?

N

N

N

N

N

WHERE IS HIS PERFOR

MANCE IN THE SEQUENCE?

1,

2

4,

5,

6,..,

7,...

1

2

WHAT IS HIS DEGREE

OF CONTROL? (%)

30

20

50

80

20

HOW LONG IS HIS

PERFORMANCE? (% TIME)

20

;00 90

*lo
TRAINING, mFTHnn! TUTORING (13)

DESCRIPTION: An instructor works directly with an individual trainee.

exposition, demonstration, questioning, coaching, or guided practice.'

The method may involve

METHOD VARIABLES STIMULUS RESPONSE FEEDBACK

Si 1

1 WHO IS PERFORMING?

M1

S1 Z S

.

. . HOW MANY TIMES IS HE N N N

PERFORMING? N N N

N N

WHERE IS HIS PERFOR1

2

1

2

MANCE IN THE SEQUENCE? 3

3

60 70 60

WHAT IS HIS DEGREE 10 30 10 OF CONTROL? (%) 30 30

Hnw LONG IS HIS

PERFORMANCE? (% TIME)

50

10

40

70

30

50

10

40
TRATNINn MFTHOfl CASE STUDY (14)

DESCRIPTION: Involves in-depth group discussion .of real-life situations. It requires reading, study,

analysis, discussion, and free exchange of ideas as well as decision making and the selling of decisions

to others. A case report is distributed to trainees. The report contains a factual and:accurate

picture, based on firsthand observation, of a situation that portrays people acting, interacting, and

reacting. Trainees study the case report and discuss it in depth.

METHOD VARIABLES STIMULUS RESPONSE FEEDBACK

0

WHO IS PERFORMING?

(15-)

HOW MANY TIMES IS HE

PERFORMING?

WHERE IS HIS PERFORMANCE IN THE SEQUENCE?

1

WHAT IS HIS DEGREE

OF CONTROL? (%)

20

80

100 100

Hnw LONA IS HIS

PERFORMANCE? (% TIME)

20 100 100
MFTWD: TNCTDENT mpulop (15)

DESCRIPTION: A variation of the case method in which only a brief sketch of the climax of a case is

presented. Trainees are given a few minutes to study the incident and attempt to determine the information they need to find out what is going on. Trainees then get these facts by asking questions of the

leader. Time is usually limited, and questions must be of the type that can be answered "yes" or "nov

or by a simple factual statement. The group then determines the nature of the problem and the decisions

needed to solve it. Each member writes an individual decision with supporting reasons. Group discussion

of the case in general and the decisions reached follows.

METHOD VARIABLES STIMULUS RESPONSE FEEDBACK

WHO IS PERFORMING?

1

1 (15-)

1 (15-)

1

1 (15-)

1 N N

HOW MANY TIMES IS HE N N

PERFORMING? N

1

WHERE IS HIS PERFORMANCE IN THE SEQUENCE?

3, 5,...

2, 4,...

2

.... 10 70 100

WHAT IS HIS DEGREE 40 30

OF CONTROL? (%) 50 . 10 70 100

HOW LONG IS HIS 30 30

PERFORMANCE? (% TIME) 60

.
TRATOND METHOD! ROLF PLAYING (16)

DESCRIPTION: A laboratory method of instruction that involves the spontaneous dramatization or acting

out of a situation by two or more persons under the direction of a trainer. The dialog grows out of the

situation developed by the trainees :cisigned to the parts. Each person acts his role as he feels it

should be played. Other trainees serl as observers and critics. Following the enactment, the group engages in discussion.

METHOD VARIABLES STIMULUS RESPONSE FEEDBACK

WHO IS PERFORMING?

1

S

1

(4-)

S

2 (30-)

-

I1

s

2 (30-)

,, *1

S1 (4-)

2 (30-)

HOW MANY TIMES IS HE

PERFORMING?

1

N

N

N

N

N

N

N

N

WHERE IS HIS PERFORMANCE IN THE SEQUENCE?

1

2

3

1

2

3

1

2

3

WHAT IS HIS DEGREE

OF CONTROL? (%)

30

50

. 20

50

30

20

40

30

30

HOW LONG IS HIS

PERFORMANCE? (S TIME)

30

50

20

. 50

30

20. 40

30

30

.....-
MOB mFTHnol SENSITIVITY (LAWRAtORY OR T-GROUP) TRAINING (17)

DESCRIPTION: A deliberate effort to apply behavioral science to problems of motivation, co...munication,

problem solving, and teamwork. Basically, sensitivity training is small-group interaction under stress

in an unstructured group composed of learners and a trainer. The objective is behavioral change. To

attain the objective, a permissive or supportive environment is established by the trainer. Participants

are encouraged to act their on roles, receive feedback, examine their concepts of self, experiment

with and practice new patterns of behavior, and learn how to maintain changed behavior back on the job.

METHOD VARIABLES STIMULUS RESPONSE FEEDBACK

WHO IS PERFORMING?

1

1

SI (5-)

s (5-) 1

1

1

1

1

(5-)

HOW MANY TIMES IS HE

PERFORMING?

N

N

N

N

N

WHERE IS HIS PERFORMANCE IN THE SEQUENCE?

1

WHAT IS HIS DEGREE

OF CONTROL? (%)

30

70 . 70

30 so

70

HOW LONG IS HIS

PERFORMANCE? (% TIME)

30

70

70

30

30

70
TRAINTNA mrTHnn! GAMES (18)

DESCRIPTION: Games include a set of structured decision-making tasks typical of a real-life situation

and provide a systematic means of observing and evaluating trainees' decisions. These, then, are fed

back to the trainees so that they can judge their appropriateness. Most games are played by one or

more teams, each composed of from one to twenty participants. There may or may not be interaction

between teams. Games are usually played in periods-- an interval of time which may represent a month,

a quarter, or a year. Trainees are given information in the form of reports or a scenario and allowed

time to study the situation and make decisions. These decisions are processed either by a control group

of judges or by a computer. The resulting data or scenario projections are returned to the team for

analysis, and another decision.

METHOD VARIABLES STIMULUS

. .. RESPONSE FEEDBACK

1

WHO IS PERFORMING? M1 xl

Sl (20-)

N

HOW MANY TIMES IS HE N

PERFORMING? N

1

YHERE IS HIS PERFOR- 2

MANCE IN THE SEQUENCE? 3

10 100 70

WHAT IS HIS DEGREE 30 30

OF CONTROL? (%) 60

HOW LONG IS HIS 10 100 90

PERFORMANCE? (% TIME) 3°

60

10

--.......i....-----........,
t AW 104.4u; DESCRIPTION: The in-basket situation is composed of a representative sample of a full year's performance

in all aspects of a job. Trainees are given background materials, organization charts, policy manuals,

financial statements, reports, and position papers to study before the exercise begins. Each trainee is

then exposed to a structured array of memos, reports, letters, telephone calls, visits, and meetings. In

his role as a manager, the trainee makes decisions on the incoming "mail." In each instance he commits

himself in writing to specific courses of action. Time limits are established to introduce realism and

cause strew; The decision - making phase is followed by discussion and critique of the actions taken and

decisions reached. All actions are analyzed, evaluated, and fed back to participants.

METHOD VARIABLES STIMULUS RESPONSE FEEDBACK

WHO IS PERFORMING?

Il

Ni . S1

(20-)

S1 (20-)

II 1 (20-)

1

HOW MANY TIMES IS HE N

PERFORMING? N

WHERE IS HIS PERFORMANCE IN THE SEQUENCE?

1

23 ._ 10 80 60

WHAT IS HIS DEGREE 40 20 40

OF CONTROL? (%) 50 .

. 10 80 60 HOW LONG IS HIS 40 20 40

PERFORMANCE? (% TIME) 50 . L.....
TRATNTN( mrrimn! BRAINSTORMING (20)

DESCRIPTION: A small, carefully selected group is given a "how to" question or problem and is asked to

produce as many ideas or solutions as they can generate. Usually a time limit is set. The technique

of free association is encouraged. Quantity of ideas or solutions takes precedence over quality.

Judgments about the worth of ideas or solutions are deliberately postponed until a later time. Ideas

are written on a chalkboard or flip chart as fast as they are called out.

C

METHOD VARIABLES

,...._-___ ,.....m..,..- _

STIMULUS RESPONSE FEEDBACK

1

i1

. ,...______ (10-)

__

S (10-)

1

. __.., ,_......- , (10-)

1

WHO IS PERFORMING?

1

HOW MANY TIMES IS HE N

PERFORMING?

WHERE IS HIS PERFORMANCE IN THE SEQUENCE?

1

2

1

. 1

__ 10 100 100

WHAT IS HIS DEGREE 90

OF CONTROL? (%)

HOW LONG IS HIS

10 100 100

PERFORMANCE? (% TIME) 90 . 10i to 5 1 4
TRATNTN1 mFunn! COMITTEES (21)

DESCRIPTION: A group of trainees, ranging in size from three to seven, is given a special assignment

in the form of a problem. The group is asked to investigate the problem, reach conclusions, and

recommend a solution or a course of action. The committee may produce a report which is often presented

orally to the larger group.

METHOD VARIABLES STIMULUS

41.minimair

RESPONSE FEEDBACK

gco=========w

WHO IS PERFORMING? 1

S (7-) 1 1

1 (7-) S

2

(20-)

1 N N

HOW MANY TIMES IS HE M N

PERFORMING?

O ERE IS HIS PERFOR- 1

2 MANCE IN THE SEQUENCE?

10 100 70

WHAT IS HIS DEGREE 9C 30

OF CONTROL? (%) . . HOW LONI IS HIS 10 100 60

PERFORMANCE? (S TIME) 90 40
103

TRAiNIMr MFTHOOe YIELD ,TRIPS [ ?.21

DESCRIPTION: A carefully planned visit or tour to a place away from the training activity. The purpose

of the trip is to provide firsthand observation of objects, processes, operations, and situations not

transportable to, or reproducible in, the training facility. The field trip may tate less than an hour

when it involves a visit to an adjoining plant, office, or shop; or it may consume several days or weeks

as would be the case with a visit to a distant plant or an overseas installation.

METHOD VARIABLES STIMULUS RESPONSE FEEDBACK

FeI 2=1"123=1Vr

WHO IS PERFORMING?

Ml S1 (20-)

1

MI

S

1 (20-)

N N N

HOW MANY TIMES IS HE N

PERFORMING?

W IS HIS PERFOR1 1

2

MANCE IN THE SEQUENCE?

WHAT IS HIS DEGREE

100 100 50

50

OF CONTROL? (%)

100 100 50

HOW LONG IS HIS 50

PERFORMANCE? (% TIME)
TRAINTNI MF I. I PANELS 23

DESCRIPTION: Three to ten people, under the direction of a moderator, present their views on a particular

subject or problem, or present assigned phases of a broad topic. Usually, panelists represent different

kinds of expertise, experience, or perspectives. Often they are drawn from operating and staff elements.

At times experts from outside the enterprise are invited to participate. Sometimes trainees themselves

serve as members of panels. Following the presentations by the panelists, trainees are encour..ged to participate through questions directed to individual panelists.

METHOD VARIABLES STIMULUS RESPONSE FEEDBACK

WHO IS PERFORMING?

1

1

S

1

1

2 (10-)

S

2

(20+) 1

3

s

2

(20+)

N N N

HOW MANY TIMES IS HE N N

PERFORMING?

WHERE IS HIS PERFOR- 1 1 1

MANCE IN THE SEQUENCE? 2 2 ... 60 100 60

WHAT IS HIS DEGREE 40 40

OF CONTROL? (%)

H^W LONA IS HIS 30 100 30

PERFORMANCE? (% TIME) 70 70
FUNCTIONAL CONTEXT VARIABLES

ROLE OF ELEMENT - The social function performed by the trainee within the

system's operational context.

1. Supervisory - The trainee's function is unequal to functions being

performed by other individuals; the role is basically one of overseeing or .:. directing.

2. Team Performance - The trainee's function is equal to functions being

performed by other individuals; the function is basically one of teamwork

and cooperation.

3. Individual Performance The trainee's function is one of performing

alone, usually without supervision or team assistance.

FUNCTION PERFORMED IN ROLE - The primary actions performed within each role.

4. Mental - An action occurring or experienced in the trainee's mind, as

contrasted with overt physical activity.

5. Physical - An overt bodily action performed by the trainee.

6. Perceptual - An action by the trainee involving perception or observation.

7. Communicative - An action by the trainee in which he transmits either a

written or verbal message.

STABILITY OF FUNCTION - The function's state, quality, or degree of being

constant overtime.

8. Unstable - The trainee's function is not constant or regular, it is

characterized by continual change and fluctuation.

9. Stable - The trainee's function has little change or fluctuation over time.

iPHYSICAL CONTEXT - The significance of the physical environment upon

performance of the task.

10. Low Impact - The physical environment has little or no significant

impact on performance of the task.

11. High Impact - The physical environment has a large or significant impact

on performance of the task.

PSYCHOLOGICAL IMPACT - The significance of the psychological environment

upon performance of the task.

77

1 Ii

1. VI 1,1
12. Low Impact - The psychological environment has little or no significant

impact on performance of the task.

13. High Impact - The psychological environment has a large or significant

impact on performance of the task.
Psychol --Physical ogical context Stability Context of junction Function Per- formed in Role Role of Element = 8 G" O. 7' CSre k .... r- -5 2 Os c, et...a"3 - 35 ..2;* O. 7" C) reaa r- -,3 2 C. C) ce)0 of r;CT+CD40 c.CY. Cr .CO e) 07.OP 4*M4 -o1C) CD "I74-1. OP4 Ch -o4. 4O-C) DI C.71 3I' 44. Co 41. ,1 Z. 4's a 0 <,.a. 7 OP C) . CO41 - 44 34 >4 >4 >C >4 Lecture 34 >C >C >C 34 34 34 , Conference - 1 4 I 1 )4 )4 .st . )4 Conference - Seminar >4 >C >C 4 >C >C Demonstration )4 )4 )4 )4 Performance-Man. Ind. Practice )4 )4 )4 )4 )4 Performance-Man. Student Tutor )4 )4 )4 )4 )4 )4 Performance-Man. Group Practice Performance-Man. Group Contr. Prac. Programed Instruction )4 )4 1 )4 )4 )4 )4 4 NC 4 ,4C >< 1o 4 )4 )4 )4 )4 Study Assignment Individual Study )4 NC >e NC >4 'Study Assignment Supervised Study yc )4 )4 )4 )4 )4 )4 )4 )4 )4 Tutoring )4 )4 )4 )4 )4 Case Study )4 )4 )4 )4 )4 )4 Incident Method )4 )4 )4 )4 24 Role lavino 4 )4 )4 )4 , )4 , )4 )4 Sensitivity Trainina 3ames In-Basket -..: xe r c 1 s e )4 )4 )4 )4 )4 )4,4C NC )4 NC ,,C )4 )4 )4 )4 3rainstorming )4 )4 )4 )4 )4 >4 :ornmittees )4 )4 )4 )4 Field Trips )4 )4 )4 )4 )4 )4 Panels Col COto0am/ roUI 0 4 C O toON
APPENDIX C

FUTURE RESEARCH RESOURCE VARIABLES

1. Method Related Variables

t

a. Stimuli Characteristics (C-2-3)

b. Response Characteristics (C-4-5)

c. Information Feedback Logic (C-6)

2. Content Related Variables

a. Stimuli Characteristics (C-7-8)

b. Response Characteristics (C-9-10)

c. Information Feedback Logic (C-11-12)

3. Information Structure Related Variables

a. Stimuli Characteristics (C-13-14)

b. Response Characteristics (:-15-16)

c. Information Feedback Logic (C-17-18)

81 1' '' 1Ao
METHOD RELATED VARIABLES

STIMULI CHARACTERISTICS

Medium of Stimuli Presentation

1. Visual Cues - Signals received through the sense of sight.

2. Audio Cues - Signals received through the sense of hearing.

3. Tactile Cues - Signals received through the sense of touch, including

sensations re ated to texture, size, shape, or vibration of the skin.

4. External Stimulus Motivation Cues - The sensations felt by a person

when he is moved by some outside force in such a way that his body

experiences roll, pitch, yaw, heave, sway and/or surge.

5. Internal Stimulus Motion Cues - The sensations felt by a person when

he moves his arm, leg, fingers, etc.

6. Olfactile Cues - Signals received through the sense of smell.

7. Gustatile Cues - Signals received through the sense of taste.

Number of Channels or Sources

8. Limited - A small number of sources, channels, or instruments through

are presented to the trainee.

9. Unlimited - A multiple number of sources, channels, or instruments

through which stimuli are presented to the trainee.

Stimuli Pacing

10. Self-paced - Stimuli are only presented upon the trainee's request.

11. Forced-pace - Stimuli are presented at some predetermined rate,

frequency of change, etc.

Stimuli Distribution

12. Individual - All information is presented or displayed directly to

one individual trainee.

13. Group - Information is presented or displayed to a group of trainees,

allowing only indirect access to the information for an individual.

82
Response Criterion

14. Determinate - The response criterion has a fairly explicit or implicit

response requirement as a direct reflection of the training or system

requirement, and makes the stimulus (visual, audio, etc.) possess a certain

or determinate value or payoff for the trainee.

15. Indeterminate - The response criterion is highly dependent on the

trainee's interpretation of the response requirement, and makes the stimulus

(visual, audio, etc.) possess an uncertain or indeterminate value or payoff

for the trainee.

83
METHOD RELATED VARIABLES

RESPONSE CHARACTERISTICS

Response Mode of Implementation

1. Overt Response - Verbal - A response which the trainee expresses in an

audible (verbal) manner, such as a verbal short answer response to a question

having a limited set of correct answers, a conversational response, or a

verbal decision response.

2. Overt Response - Written - A response which the trainee expresses

in an observable (written) manner, such as a free style written response,

a written multiple choice response, or a written fill-in-the blank response.

3. Overt Response - Manipulative Acts - A response which the trainee expresses

in an observable (manipulative) manner, such as the small movements of dials,

switches, keys, or small adjustments to instruments or the large movements of

levers, wheels or use of hand held tools.

4. Overt Response - Tracking - A response which the trainee expresses in an

observable (tracking) manner, such as continously controlling a constantly

changing system, e.g., steering an automobile.

5. Overt Response - Procedural Performance - A response which the trainee

expresses in an observable (procedural performance) manner, such as

performing a sequence of steps in a procedure, e.g., carrying out the

items on the checklist for preflighting an aircraft or turning on a radar

system.

6. Covert Response - A response which the trainee creates in his mind but

does not express in an observable or audible manner.

Response Pacing

7. Self-paced - Responses are made by the trainee at his own rate.

8. Forced-pace - Responses are made by the trainee at some predetermined

rate or frequency.

Response Distribution

9. Individual - All responses are expressed by one individual trainee.

10. Group - Responses are expressed by a group of trainees, allowing

only indirect responses for an inaividual.

84

iI-t. t)
Response Determinancy

11. Determinant - A required response which can be specified in advance

of the stimuli that calls the response out, i.e., the response is

preprogrammed or determined before the operation begins.

12. Indeterminant - A required response which cannot be specified in

advance of the stimuli that calls the response out, i.e., the response

is highly dependent on the immediate stimuli situation and cannot be

predicted in advance, hence is indeterminate.

85
METHOD RELATED VARIABLES

INFORMATION FEEDBACK LOGIC

Medium of Feedback Presentation

1. Visual - Feedback presented visually by means of a display, it may be

coded and transmitted visually to the trainee.

2. Aural - Feedback presented aurally by means of a display to the trainee.

3. Written Form - Feedback presented to the trainee in written form.

4. Face-to-Face Communication - Feedback presented by direct verbal means

to the trainee.

5. Indirect Communication - Feedback presented by indirect verbal means,

such as by intercom, telephone, or radio link.

6. Tactile - Feedback presented to the trainee through the sense of touch,

inclaVgiensations related to texture, shape, size, or vibration of the skin.

7. Kinesthetic - Feedback presented to the trainee by either internal or

exteliiichlri Vi movement, such as reaching, grasping, tilting, etc.

8. Olfactile - Feedback presented to the trainee through the sense of smell.

9. Gustatile - Feedback presented to the trainee through the sense of taste.

Source of Feedback

10. Intrinsic F - Information or cues built into the system from which the

trainee interprets feedback information.

11. Extrinsic F - Information or cues not inherent in the trainee action or

system operations but is supplied by an external source.

Feedback Pacing

12. Self-paced - Feedback presented to the trainee only at his request.

13. Forced-pace - Feedback presented to the trainee at some predetermined

rate, frequency, etc.

Feedback Distribution,

14. Individual - Feedback is presented to one individual trainee.

15. Group - Feedback is presented to a group of trainees, allowing only indirect

access for an individual. 86
CONTENT RELATED VARIABLES

STIMULI CHARACTERISTICS

Medium of Stimuli Presentation

1. Visual Cues - Signals received through the sense of sight.

2. Audio Cues - Signals received through the sense of hearing.

3. Tactile Cues - Signals received through the sense of touch, including

sensations related to texture, size, shape, or vibration of the skin.

4. External Stimulus Motion Cues - The sensations felt by a person when

he is moved by some outside force in such a way that his body experiences

roll, pitch, yaw, heave, sway and/or surge.

5. Internal Stimulus Motion Cues - The sensations felt by a person when

he moves his arm, leg, fingers, etc.

6. Olfactile Cues - Signals received through the sense of smell.

7. Gustatile Cues - Signals received through the sense of taste.

Significance of Stimuli

8. Significant - The stimuli are consequential or important in terms of

their mearing for problem solving, e.g., the identification of enemy

bombers or missiles may direct the kind of weapons released against them.

9. Insignificant - The stimuli are inconsequential or unimportant in terms

of their meaning for problem solving, e.g., identification of a reconnaissance

aircraft may be less significant than the identification of an enemy bomber.

Relevancy of Stimuli

10. Relevant - Accurate, complete, or relevant stimuli which the trainee

is cu-R-TZETTspond to because of his instructions or other system criteria.

11. Irrelevant - Inaccurate, distorted, incomplete or irrelevant stimuli

which the trainee must discriminate from relevant stimuli, e.g., "clutter"

or "noise".

Stimuli Cost

12. Low - A low cost of securing information (visual, audio, etc. stimuli)

to helT-Make a decision e.g., little delay or effort, or loss of equipment

or men.

13. High,- A high cost of securing information (visual, audio, etc. stimuli

to help make a decision, e.g., much delay or effort or loss of equipment

and men. 87
Stimuli Similarity

14. Similarity - Stimuli presented to the trainee which resemble other

stimuli.

15. Variability - Stimuli presented to the trainee which vary or differ

from other stimuli.

88
CONTENT RELATED VARIABLES

RESPONSE CHARACTERISTICS

Response Mode of Implementation

1. Overt Response - Verbal - A response which the trainee expresses in an

audiffirverbal) manner, such as a verbal short answer response to a question

having a limited set of correct answers, a conversational response, or a

verbal decision response.

2. Overt Response - Written - A response which the trainee expresses

in an observable (writtehrmanner, such as a free style written response,

a written multiple choice response, or a written fill-in-the blank response.

3. Overt Response - Manipulative Acts - A response which the trainee expresses

in an observable (manipulative) manner, such as the small movements of dials,

switches, keys, or small adjustments to instruments or the large movements of

levers, wheels or use of hand held tools.

4. Overt Res onse - Tracking - A response which the trainee expresses in an

observa e tract ng manner, such is continuously controlling a constantly

changing system, e.g., steering an automobile.

5. Overt Response - Procedural Performance - A response which the trainee

expresses in an observable (procedural performance) manner, such as

performing a sequence of steps in a procedure, e.g., carrying out the items

on the checklist for preflighting an aircraft or turning on a radar system.

6. Covert Response - A response which the trainee creates in his mind but

does not express in an observable or audible manner.

Response Similarity

7. Similarity - Responses made by the trainee which resemble other responses.

8. Variability - Responses made by the trainee which vary or differ from

other responses.

Significance of Responses

9. Significant - The responses are consequential or important in terms of

their meaning for problem solving, e.g., pressing a "fire" button after

identification on any enemy aircraft.

10. Insignificant - The responses are inconsequential or unimportant in

terms of their meaning for problem solving, e.g., social communication

arong team members, which are not related to task accomplishment.

89
Relevancy of Responses

11. Relevant - Accurate, complete,. or relevant responses which the trainee

expresses, according to his instructions or other system criteria.

12. Irrelevant - Inaccurate, distorted, incomplete, or irrelevant responses

whichTF7ER.Fee expresses, e.g., a response which gives altitude

information only, but which was to include both altitude and azimuth

information.

Response Cost/Value or Payoff Relationship

13. Low Cost/High Value - The relationship between response cost and value, where

the cost of responding a particular way is low and the value is high, e.g.,

a decision to retreat from one area to help reinforce another, may mean

that one will lose some men and equipment in the area from which they are

retreating, but will save more men and equipment in the long run by reinforcing

the other area.

14. High Cost/Low Value - The relationship between response cost and

value, where the cost of responding a particular way is high and the value

is low e.g., a decision to delay an advance may cost more lives and loss

of equipment-than a decision to advance immAiately.

41

A

90
CONTENT RELATED VARIABLES

INFORMATION FEEDBACK LOGIC

Medium of Feedback Presentation

1. Visual - Feedback presented visually by means of a display, it may be

coded transmitted visually to the trainee.

2. Aural - Feedback presented aurally by means of a display to the trainee.

3. Written Form - Feedback presented to the trainee in written form.

4. Face-to-Face Communication - Feedback presented by direct verbal means

to the trainee.

5. Indirect Communication - Feedback presented by indirect verbal means,

such as by intercom, telephone, or radio link.

6. Tactile - Feedback presented to the trainee through the sense of touch,

including related to texture, shape, size, or vibration of the skin.

7. Kinesthetic - Feedback presented to the trainee by either internal or

external bodily movement, such as reaching, grasping, tilting, etc.

8. Olfactile - Feedback presented to the trainee through the sense of smell.

9. Gustatile - Feedback presented to the trainee thrOugh the sense of taste.

Type of Feedback

10. Research Correctness (Rcr) - Information about the correctness or

incorrectness of trainee's response, when several response alternatitres are:

possible and the correct choice is not known to the trainee in advance.

(Also known as augmented feedback.)

11. Response Correctness (Rcf) - Information provided to the trainee (or

others who need to know about his performance) that he has in fact performed

an operation, but does not say anything about the longer range consequences

of the action taken.

12. Res onse Conse uences (Rcn) - Information about the consequences of the

action a en. It con irms the response made by the trainee, and the

correctness of a response can be inferred only from its consequences. May

also serve to cue the trainee to perform the next response in sequence.

91

1 0.
13. System Status (Rss) - Information about the condition of one's own

or another system or the external environment, on the basis of which a

trainee or team must act. Information is not necessarily (or even frequently)

the immediate consequence of or follow-on to a specific trainee/team action;

it may reflect system events that have been put in motion by much earlier

trainee actions. Provides information that regulates trainee and system

actions in the -sense that when a particular status condition occurs, the

trainee must often take action to maintain the integrity of his system.

Feedback Specificity

14. Specific - Quanitative - Quanitative (outcome) information provided

to the trainee, which states a relationship between the trainee's response

and the desired correct response; for example, the percent error, the

number of correct or incorrect responses, or rate data.

15. Nonspecific - Qualitative - Qualitative (outcome) information provided

to the trainee about his performance; for example, "there were more correct

responses than last time," "almost all ammunition was expended," etc.

16. Specific - Detailed - Detailed, fractionated categories of information

or a wide range of feedback dimensions provided to the trainee about his

performance.

17. Nonspecific - Nondetailed - Whole unfractionated categories of

information or a narrow range of feedback dimensions provided to the trainee

about his performance.

Properties of Feedback

18. Motivational - Feedback provided to the trainee which has motivational

(stimulating, goal adjusting, and criteria selecting) properties;

motivational feedback has few or no cue properties, i.e., feedback that neither

informs the trainee of the nature or locus of his errors nor suggests

how these might be corrected.

19. Directive - Feedback provided to the trainee which has informational

or directive properties; directive feedback serves a guidance function to the

trainee about his performance, i.e., feedback that informs the trainee of the

nature and locus of his errors and suggests how these might be corrected.

92
INFORMATION STRUCTURE RELATED VARIABLES

STIMULI CHARACTERISTICS

Medium of Stimuli Presentation

1. Visual Cues - Signals received through the sense of sight.

2. Audio Cues - Signals received through the sense of hearing.

3. Tactile Cues - Signals received through the sense of touch, including

sensations related to texture, size, shape, or vibration of the skin.

4. External Stimulus Motion Cues - The sensations felt by a person when

he is moved by some outside force in such a way that his body experiences

roll, pitch, yaw, heave, sway and/or surge.

5. Internal Stimulus Motion Cues - The sensations felt by a person when

he moves his arm, leg, fingers, etc.

6. Olfactile Cues - Signals received through the sense of smell.

7. Gustatile Cues - Signals received through the sense of taste.

Stimuli Presentation

8. Static - A unitary stimuli situation, i.e., stimuli are presented to

the trainee "all at once", e.g., batch presentations.

9. Dynamic-Ordered - A sequential stimuli situation, i.e., stimuli are

presented to the trainee sequentially or in an ordered manner over time.

10. Dynamic=Random - A non-sequential stimuli situation, i.e., stimuli

are presented to the trainee randomly over time.

Stimuli Presentation Rate

11. Slow Rate - A slow rate or speed of presentation of stimuli to the

trainee,Tiing the trainee a long or maximum stimulus analysis time.

12. Fast Rate - A fast rate or speed of presentation of stimuli to the

trainee, wing the trainee a short or minimum stimulus analysis time.

Frequency of Stimuli Change

13. Infrequent - A low frequency of stimuli change, i.e., stimuli presented

to the trainee change from one to another infrequently.

14. Frequent - A high frequency of stimuli change, i.e., stimuli presented

to the trainee change from one to another frequently.

93
Number of Stimuli

15. Few - A small number of separate streams of signals that occur

independently of each other, but require simultaneous consideration.

16. Multi le - A large number of separate streams of signals that occur

independent y of each other, but require simultaneous consideration.

Stimuli Format

17. Formatted - Formatted or organized stimuli, such as alphanumeric

displays.

18. Unformatted - Unformatted or unorganized stimuli, such as raw video

displays.

Patterning of Stimuli

19. Simple - Simple or common patterns of stimuli, such as the repetition

of the same event at high frequencies of occurrence.

20. Complex - Complex. detailed, or circumstantial patterns of stimuli, such

as discrete sequenced patterns at low frequencies of occurrence.

I `)...t,,

94
INFORMATION STRUCTURE RELATED VARIABLES

RESPONSE CHARACTERISTICS

Response Mode of Implementation

1. Overt Response - Verbal - A response which the trainee expresses in an

audible (verbal) manner, such as a verbal short answer response to a question

having a limited set of correct answers, a conversational response, or a

verbal decision response.

2. Overt Response - Written - A response which the trainee expresses

in an observable (written) manner, such as a free style written response,

a written multiple choice response, or a written fill-in-the blank response.

3. Overt Response - Manipulative Acts - A response which the trainee expresses

in an observable (manipulative) manner, such as the small movements of dials,

switches, keys, or small adjustments to instruments or the large movements of

levers, wheels or use of hand held tools.

4. Overt Response - Tracking - A response which the trainee expresses in an

observable (tracking) manner, such as continuously controlling a constantly

changing system, e.g., steering an automobile.

5. Overt Response - Procedural Performance - A response which the trainee

expresses in an observable (procedural performance) manner, such as

performing a sequence of steps in a procedure, e.g., carrying out the items

on the checklist for preflighting an aircraft or turning on a radar system.

6. Covert Response - A response which the trainee creates in his mind but

does not express in an observable or audible manner.

Response Implementation

7. Static - A unitary response situation, i.e., responses are made by the

trainee "all at once".

8. Dynamic-Ordered - A sequential response situation, i.e., responses are

made by the trainee sequentially or in an ordered manner over time.

9. Dynamic-Random - A non-sequential response situation, i.e., responses

are made by the trainee randomly over time.

Required Response Rate

10. Slow Rate - A slow rate or speed of trainee response, i.e., a rate

which allows the trainee a long or maximum response time.

11. Fast Rate - A fast rate or speed of trainee response, i.e., a rate

which allows the trainee a short or minimum response time.

95
Response Frequency

12. Infre9uent - A low frequency of response change, i.e.

by the trainee change infrequently.

13. Frequent - A high frequency of response change, i.e.,

by the trainee change frequently.

Number of Required Responses

14. Few - A small number of responses that the trainee is required to

express, each independent of the other.

15. Multiple - A large number of responses that the trainee is required to

express, each independent of the other.

Response Format

16. Formatted - Formatted responses, e.g., responses to a programmed text.

17. Unformatted - Unformatted responses, e.g., responses made by team

members during an emergency situation.

Patterning of Responses

18. Simple,- Simple or common patterns of responses, such as repetition of

the same response at high frequencies of occurrence.

19. Complex - Complex, detailed, o- circumstantial patterns of responses,

such as discrete sequenced patterns of responses at low frequencies of

occurrence.

, responses made

responses made

96

19
INFORMATION STRUCTURE RELATED VARIABLES

INFORMATION FEEDBACK LOGIC

Medium of Feedback Presentation

1. Visual - Feedback presented visually by means of a display, it may be

coded and transmitted visually to the trainee.

2. Aural - Feedback presented aurally by means of a display to the trainee.

3. Written Form - Feedback presented to the trainee in written form.

4. Face-to-Face Communication - Feedback presented by direct verbal means

to the trainee.

5. Indirect Communication - Feedback presented by indirect verbal means,

such as by intercom, telephone, or radio link.

6. Tactile - Feedback presented to the trainee through the sense of touch,

inclaiTriensations related to texture, shape, size, or vibration of the skin.

7. Kinesthetic - Feedback presented to the trainee by either internal or

external bodily movement, such as reaching, grasping, tilting, etc.

8. Olfactile - Feedback presented to the trainee through the sense of smell.

9. Gustatile - Feedback presented to the trainee through the sense of taste.

Time Schedule for Feedback

10. Pre-F Interval - !mediate - Feedback provided immediately after the

trainee's antecedent response, i.e., there is a small or no interval of time

between the antecedent response and the feedback for that response.

11. Pre-F Interval - Delayed - Feedback provided subsequently after the

trainee's antecedent response, i.e., there is an interval of time or

delay between the antecedent response and the feedback for that response.

12. Post-F Interval - Immediate - Presentation of the next stimulus

inmediateTy after the occurrence of feedback for the last response, i.e.;

there is a small or no interval of time between the occurrence of feedback

for the last response and the presentation cf the next stimulus.

13. Post-F Interval - Delayed - Presentation of the next stimulus subsequently

after the occurrence of feedback for the last response, i.e., there is an

interval of time between the occurrence of feedback for the last response and

the presentation of the next stimulus.

97
Feedback Regularity

14. Regular - Feedback provided to the trainee at regular intervals, such as

after every or every other response or at established or fixed periods.

15. Irregular - Feedback provided to the trainee at variable intervals,

which may change as a function of stage of training or level of performance.

This includes the provision for intermittent presentations to permit

probabilistic schedules for reinforcement.

Frequency of Feedback

16. Frequent - Any feedback given to the trainee that is provided at least

for 20-30% of all his responses.

17. Infrequent - Any feedback given to the trainee that is provided less

than for 20% of all his responses or is not provided at all.

Feedback Format

18. Formatted - Formatted feedback, e.g., alphanumeric feedback displays.

19. Unformatted - Unformatted feedback, e.g., raw video feedback displays.

98

1 r,
APPENDIX D

SAMPLE COST SHEETS

COST DATA COLLECTION FORM

Instructional Delivery System 16P

Run ID SAMPLE SHEET 1

SELF PACE

SYMBOL VARIABLE DESCRIPTION VALUE UNITS

FACILITIES

FACOST

Total tacilities acquisition

and/or refurbishing,:costs 175,000 Dollars

LOFFA Expected years oelife of

S . - a i - a r

- 0."'"'

SQFTIN

Tota square eet require.

o I o 64 F

SQFTST

ota square teet require.

fier student nosition 100 So_ Ft

SQFTAH

Total square feet requires

for admidistrative overhead

for all student positions

2,500 Sq Ft

4 EQCISP

Equip. implementation costs

independent of stud. pos. 125.243 Dollars

LOFEQ1 Expected years of life of

EGCLSP gss is 10 Years__

EQIMPC

Equip. implementation costs

apr stibipnt pnciti0O 6F c nnliarc

_Years

_Percent

LOFEQ

Ex pected years of lite of

EQIMPC assets (in whale numhers) 10

TSPOSD Al

Percent of operating time

student nosition down

99
SAMPLE COSTING SHEET 2

SYMBOL 1 VARIABLE DESCRIPTION VALUE 1 UNITS

Material (IM)

% of TLENGH (i.e., time spent

in training medium) for which 1.00

new instructional material

must be developed

Percent

.InItructional

UIMD

UPDATE % of original development cost

required each year to maintain .20

instructional material

Percent

EVIH % of original de lopment.cost

remaining at end of planning 0

Period

Percent

CIMD Average coat of developing

one hour of instructional 188.

material

Dollars

_Rerspnnel Instructor to student

postion ratio .20

Decimal

IHTSPO Ratio

SALINR

nnua sa ary ana .enetits o

one instructor .640. Dollars

Cost of expendable supplies for

each student while enrolled in

course 1,168 Dollars

SUPPLY

STUDSL

Annual salary and benefits of

one student 5,656. Dollars

STCSTI Average student travel cost

to and from school 0 nllarc

I

Average per student travel cost

STCST2 n to I Misceilaneous Number of years in planning period 10 ttears

PIRATE _Attrition rate 04 PrrPnt

anim.mmoii 1' TLENGH

...m3umra_ eeks

Average time spent in training

medium per student 7

non-rec cled students

TLEGTH Average hours per week student

DP I i II-4 II 4 . ARCYTM Average time the recycled student

spends repeatin material 0 Weeks

ercentage o excess stuent

positions required to provide

for fluctuations in input 0.05 Percent

100

f)
0

SAMPLE COST SHEET 3

SYMBOL

Yr. 1 Yr 2 Yr 3 Yr 4

4 Yr 5 Yr 6 Yr 7 Yr 8 Yr 9 Yr 10

VARIABLE

i'r 11 Yr 12 Yr 1.3 Yr 14 Tr 15 Yr 167(r 17 Yr 18Yr 19 Yr 20

Facilities

Cost/Ft

2

for Facilities

Per Year (Dollars)

3 3 3 3 3 3 3 3 3 3

CPSQFT(I)

cE-174717617'

Equipment Acquisition 50 50 50 50 50 50 50 50 50 50

CAQSP(I) ......

LOFEQ(I) Expected Life of CAQSP(I)

Assets (Years)

10 10 10 10 10 10 10 10 10 10

COPMT(I) Operation and Maint. Cost

of Equipment Per Student

Position for Each Year

(Dollars)

0 & M Costs of Fixed

Equipment (Dollars)

30 30 30 30 30 30 30 30 30

1 30

-----,---.------......

OMFEQ(I) .2...E1U-M?.(100 LW.=2.4211 2 .flonn 7.0nr-2_nnn?,nnn

,LIALIElts

UIMDYR(I) Unique Hours of IMO

Per Year (Hours)

0 0 0, 0 0 0 0 0 0 0

Instructional Material r

No. of Graduates

Required for Eac Duo

1800

......................---.......- 1800 1800 1800 1800 1800 1800,1800_ 1800 1800

GRAD(I)
DISTRIBUTION

I US ARMY WESTERN CDHmANU AITN: MPPL

1 OEPAmTMENI OF IHE NAVY ImAIN1t4u ANALYSIS ANu EVALUATION GROUP

1 HQOA ATTN: DA4L,-ED

1 US PACIFIC FLEE: Human RESOURCE mANAGENF-4 DETACHMENT

1 H4. ICATA ATTN: AILAT-UP-u

? HNUA RESEARCH AN.) STUUlts OFL

1 MILIsARY UCCUPATIONAL utvtLoRpFNT Oils bApc-FISR-U, HM p52C HOFFMAN FIL9G 1 4 DASD (mRA AND L)

I HQ TLATA TECHNICAL LIBRARy

1 HUTA ODCSPLR

I HOLTA ATTN: DAm1-1S1

1 USA 0,11ATION S.01IF.MS CUMU Ai1N; UHSAV-ii.4

1 IISA LORAUDM ATTN: AmS[L-PA-RH

1 HEA0wUARTERS US MARINE CORPS ATTN: CJUE MTMT

I HEAOwOARTEPS. US MARINE LORPS ATTN: LOoF HPI-20

/ US ARMY EUROPE AN) SEVLNTH ARMY

1 1ST INFANTRY 0141510N ANO FT. RILEY ATTi: AF1N-UpT-T

1 CHIEF, ATTITUuE OPINION SURVEY OIVIsION ATTN: ATLIwNcii-PA. HOFFvAN BLDG 11

I USA INTELLIGENLF AND SECURITY LOMMAND ATTN: IA0Psd.TNG-T

? HI TmAOUL TECm,ICAL LIHkAmy

1 NAVA1 TRAINING EUJIPMENI LEN ATTN: TECHNICAL LIBRARY

1 MILIIARY OCCUPAIIJNAL ULVLLOPMFN1 DIRECTORATE ATTN: Ayzl-NCR-MS -M. Wm 3N33 HOFFmAN BLDG II

I DATA ANALYSIS DIVISION AIIN: ATLI-NCR-Ho, HoFFmAN BLDG II

I USA mILOLRCEN ATTN: DAPC-MOO -T

1 RTH INFANTRY OilSION

1 HQOA ARMY FORCE MODERNILAITON COORDINATION OFFICE

1 HQOA ATTN: DASD, -NTH

1 1230 USARCOM IRFSERVE CENTER

I US ARMY SOLDIER sJP120141 CENTLk / 1 USA FORCES COMMAND AFL(, - DtPuiY LHIEF OF STAFF FOR LOGISTICS

1 OS ARMY AIR DEFENSE SCHOOL ATTN: ATSA-OTO

1 DIRECTORATE OF TRAINING ATTN: MILD -T 1 OIkEt.TORATE OF COMHAT DEVELOPMENTS ATTN: AT240

1 HODARCOM MARINt CORPS LIAISON OFC

1 DEPARTMENT OF Imt ARMY US ARMY INTELLIGFNCE SECURITY COMMAND

1 USA MISSILE COMMAND ATTN: Ultsml-NIN

1 u5 ARMY CLCUM ATTN: DHSLL-ATOo

I USA FORCES COMMAND

1 PM TRADE /

1 US MILITARY DISTRICT OF wASHINGION OFC of EuLIAL OPPORTUNITY

I NAVAL CIVILIAN PERSONNEL LUMD SOUTHERN FLO DIV

2/ ARI LIAISON OFFICE

1 7TH ARMY TRAINING COMMAND

1 ARMY TRAINING SuPoORT CENTER INDIVIDUAL TRAINING EVALUATION

1 NOUA. COCSOPS INDIVIDUAL IRAININb

1 HODA. OCSOPS TRAINING DIRECTORATE

I HQOA. OCSLO(, mqINTLNANCt MANAGEMENT

1 HQOA. OCS STUDY OFFICE

1 U.S. NAVY TRAININ(, ANALYSIS LVALUATIJN GROUP

1 USACIAC ATTN: ATEC-EX-E HUMAN FACTORS

I ATTN: SM-ALC/DPCR

1 OSAFALIOS/TAC SENIOR ARMY ADVISOR

1 INTER -UN1V SEMINAR ON AF(mtO FORLtS SOC

1 OAsA (RUA) DEPLOY FOR SCIENCE AND TECHNOLOGY

1 OFC OF NAVAL RLSLARCM / I AFHRL/LRT

1 AFHRL/LRLG

1 AIR FORCE HUMAN RESOURCES LAD ATTN: AFHPL/TSR

1 AFAMNL /bb

1 NAVY PERSONNEL R AND U CENTER u1RECT3W OF PROGRAMS

1 NAVY PERSONNEL k ANU U CtNIER / 103
E OFC uF NAVAL RESEARCH PERSONNEL ANU THAINIING RESEARCH PROGRAMS

I NAVAL PERSONNEL H + U CENTER / I UEC ut. NAVAL RtsEARCH PROJECT UFFICER. EDvIRONMENTAL PHYSIOLOGY

I mCFANN-GRAY ASSOCIATES. INC. /

I NAVAL ALROSPACL MEUICAL RSCH LAb AIRBOKNE RANGER RESEARCH

I DEPT. OF NATIONAL DEFENCE UEFENCE ANJ CIVIL INSTITUTE OF ENVIR .

1 NAVAL AEROSPACE MEDICAL RSCH LAD AERJSPACE PSYCHOLOGY DEPARTMENT

I USA 1RAWC SYSTEMS ANALYSIS ACTIVITY ATTN: ATAA-TCA

I HEAOwUARTERS. COAST GUARD CHIEF. PSYCHJLOGICAC RcCH HP

1 NSA mESEARCH AN f CHNULOUY LAH /

I USA mOHILITY fuluIPmENT R ANU U CORO ALT,,: OkUME-T( (SCHOOL)

I NI6Hi VISION LAH ATTN: oNSLL-roP.SOU

1 USA IMAINING ROAN) ATTN: ATTG.ATb-TA

1 USA mATER1EL SYSTEMS ANALYSIS ACTIVITY ATTN: upXsy-C

1 NAFEL HUMAN ENNINEEM4ND BRANCH

I FTWELLE-COLUMboS LAkuHATumILS TACTICAL TECHNICAL ()EC

1 USA ARCTIC TEST LEN ATTN: AMST:.-..PL-TS

I USA LULU REGIONS TEST LEN ATTN: SIECR-Du

1 DEFENSE LANGUAGE INSTITUTE FOoLIGN LANGUAGE CEN

1 HU MmA1R DIV OF vEUROPSYCHIATRY

1 USA ELECTRONIC .4%.4FAHE LAb CHIEF. INTELLIGENCE MATER DEVEL SUP OFF

I USA mSCH UEVEL * STANUARULLA GP. U.K.

I USA PESEARCH AND AvELu?mENT LADS CHILE, REHAV SCIENCES UIV FOOL) SC1 LAb

I NAVAL AIR SYSTEMS COMMANU ATTN: AIR-5311

I ITSACuEC TECHNICAL INFORMATION CENTER

I USAAmL LIBRARY

I HUHAN RESOURCES RSCH ONG IHUMRRU) /

I SEVILLE RESEARCH CORPORATION

I USA IRAUUC SYSTEMS ANALYSIS ACTIVITY ATTN: ATAA-SL (TECH LIBRARY)

I ONIEuRMEu SERVICES UNIT OF THE HEALTH SCI ULPARTmLNT of PSYCHIATRY

I HUMAN RESOURCES RSCH °KU iMUMRPU)

1 HUMRpO LIBRARY

I RAITELLE REPORTS L1DRARY

I RANO CORPORAIIuv ATTN: LIDRAmY U

I GRONLNGER LIBRARY ATTN: ATVHS ..1_ REX., 1311

I CE4T04 FUR NAVAL ANALYSIS

I NAVAL HEALTH RSCH CEN LinHARY

I NAVAL PERSONNEL H ANU U LEN LIHRARY ATTN: CUUE P106

I AIR FORCE HUMAN RESOURCES LAb ATTN: AFHRL /OTS

I Ht. FT. HUACHULA ATTN: TECH REF 0Iv

I USA aCADEMY OF HEALTH SCIENCEs STIMsDN LIBRARY (DOCUMENTS)

1 SCHOuL OF SYSTEMS ANU LOGISTICS /

I ERIC PROCESSING AND REFERENCE FAC ACaUfsITIONS LIBRARIAN

I DEPAmTMENT OF THE NAVY TRAINING ANALYSIS ANU EVALUATION GP

I NATIONAL CENTER FOR HEALTH STATISTICS /

1 USMA DEPT OF BEHAVIORAL sLI AvU LEADERSHIP

I OLD) uOMINION UNIVERSITY PERFORMANCE ASSESSP:NT LABORATORY

I USA LOMMAND Amu GENERAL STAFF COLLEGE ATTN: LIBRARY

I USA TRANSPORTATION SCHOOL USA IHANSP IECH INFO ANO RSCH LEN

I NMROL PROGRAM HAN/4%6ER FOR HUMAN PERFORMANCE.

1 USA ADMINCEN TECHNICAL NESEARch BRANCH LIBRARY

1 LISA F1ELU ARTY HU /

1 NAT LLEARINGHDUSE FON MENIAL HEALTH INFO PARKLAvIN BLOC

1 U OF TEXAS CEN FaH COMMUNICATION RSCH

I INSTITUTE FOR UEFENSE ANALYSES

I USA TRAINING SUPPORT CENTER ATTN: ATIC-oST-PA

1 AFFIRL TECHNOLOGY AFC (H)

I PURDuE UNIV DEPT OF PSYCHOLOGICAL SCIENCES

1 USA MOBILI1Y EUuIPMENT R AND U LUMMANJ ATTN:. URUML -Z(;

I DA US ARMY RETRAINING MIL RESEARCH . .VALUATION IVISION

I ESSE* CORPORATION /

I HAZELTINE CORPORATION / 104

1 i-)t./ ,j
1 USA AROMLUICAL RESEARCH LAd SLIENTIFIC INFORMATION CENTER

I kArTREON SERVILE COMPANr SPENctk LAb3RATORY

I HU "1Ai RESOLRCE RANAGtMtN1 LEN. SAN uIEW

I USAF,. DEN! uF 'Li SC! LtAotQSHIP

I US MILITARY AcAuENY (EPI. Or HISTORY, OLuG 601

I US4 INTELLIGtNCF CEN ANU SCM AITN; SCHOOL Ur/MARY

1 054 INTELLIGENLE Ct-N ANU SCM AIIN: ATsl-uP

I mArtINE CORPS INSTIWIE

I NAVAL SAFtTY CENTER /

I 05 CuAST GUARD TN3 CEN AIIN: KUUCATIONAt SVCS OFFICER

1 GALLAUDET COLLtot LEARNING CtNItR

uS4AwNC AND FT. HjLKLR AIIN: AILU -ES

I US ARMY AVN TNo c11.14404y AITN: CHIEF LIBRARIAN

1 054 MIR OEFINSt SCHOOL AIIN: AISA-UT

I USAA.NC ATTN: ATILI-U

I U5 MILITARY ACAtwY UIRLL10k oF INSTITUTIONAL RSCH

I OSAAuS-LIBRARY-uuCuRtNIS

I USA SERGEANTS MAJ)k ACAutmY AT1N: LEARNING RESOURCES CENTER

1 USA INFANTRY BUAMJ ATIN: ATZd.1B-AE

I US INTELLIGENCE CEN ANU SCM alTN: ATSI-OT-sFL

I USA uRONANCE Ctv AND SCM AiTN: ATSL-TO-TAC

I U54 ARMOR SCHOOL ATM: Affix-1u

I 05a ARROW CENTtw DIRELTORATt OF COMBAT oFvtLORMtNTS

I NAVAL POSTGRADUAIE SCM AIIN: oUULtY KNOX LIBRARY (COuF 14?4)

I USA IRANSPoRlATIoy SCHOOL UtPulY ASST. COMMANDANT EUUCA. TECHNOLOGY

I USA SIGNAL SCHOOL ANU FT. oOkuoN ATTN: ATZH-ET

I USA QUARTERmASItO SCM ATTN: AISM-TNG-IM-tT

I USA MILITARY POLICE SCHOOL AIM: LIBRARy

u54 uRMOR cENTtk FT. KNOX oFrICt OF ARMOR FORCE MGT 5TANUARUIZAIION

I CHIEr OF NAVAL EuJLATIoN Awl) Tub /

I 'USA SIGNAL SCHOOL FT. GuMUON tLOCAIIONAL TECHNOLOGY DIVISION

I HQ AIC/XPTO TRAINING 5YSIOIS utvELOPMtNT

I USA INSTITUTE FuH MILITARY ASSISIANCE ATTN: ATSU-TU-TA

I US ARMY ARMOR CENTER AITN: AT/K-TU-PNO

I USA MIR UEFENSt CENTER. FT. BLISS ATTN: ATZL-DIM

I USA QUARTERMASTER SCHOOL uIREcIORATE OF TRAINING DEVELOPMENTS

1 u5 CuAST WARD ACADEMY /

I USA IRANSPORTAIION SCHOOL DIRECIORATE OF TRAINING DOCTRINE

I USA INFANTRY SCHODL LIBRARY /

I USA INFANTRY SLHODL ATTN: ATsR-I-V

1 US ARMY INFANTRY SCHOOL ATTN;

I USA INFANTRY SCHOOL MTN: AISH-UOT-LRU

I USA INFANTRY SCRIM ATTN: ATSH-tV

I USA RP CHEM SCH/TNG CEN FT. MCCLELLAN ATTN: ATLN-pTS

I USA RP CHEM SCH/ING CEN FT. MCCLELLAN UPI: COMBAT DEVELOPMENT

I USA RP CHEM ScH/TNG CEN FT. MCCLELLAN DIR. TRAINING DEVELOPMENT

I USA RP CHEM SCH/ING CEN FT. MCLLELLAN ATTN: ATZN-MP-ACE

I USA INSTITUTE OF ADMINISTRATION ATTN: RESIDENT !RAINING MANAGEMENT

1 USA rIELD ARTILLERY SCHOOL MORRIS SWETT LIBRARY

I USA INSTITUTE OF ADMINISTRATION ACADEMIC LIBRARY

I USA wAR COLLEGt ATTN; LIBRARY

I USA eNGINEER SCHOOL LIBRARY ANU LEARNING RESOURCES CENTER

I USA ARMOR SCHOOL (USARMS) ATTN: LIBRARY

I US CuAST WARD ACADEMY LIBRARY

I USA IRANSPORTAIION SCHOOL TRANSPORTATION SCHOOL LIBRARY

I ORGANIZATIONAL EFFECTIVENtSS CEN SCri ATTN: LIBRARIAN

1 US ARMY INTELLIGENCE CLKITti4 SCHOOL ATTN: AT$I -TP

1 US ARMY INTELLIGENCE CENTtk SCHOOL ATTN: ATSI -RM -M

1 US ARMY INTELLIGENCE CtNTtk SCHOOL ATTN: AM-ES

I US MARINE CORPS EDUCATION CENTO4

I USA rIELU ARTILLERY SCHOOL UIRUTORATE OF COURSE UEV TRAINING

I DEPARTMENT OF THE AIR FORCE AIR UNIVERSITY LIBRARY (ATC)

105

1 xi 1/4)
1 ()SA LHAPLAIN CENTEN + sCHUUL AIIN: AISC-TD-OU

I aSA LHAPLAIN CLNID, ScrIUUL ATTN: Alse-Tu-tU

I USA LHAPLAIN CL.41EN + SChUUL AITN: AISC-TO-SF

1 USA LHAPLA1N Ct4IER + SCHLAJL Aiiw: Atsc-i)ob-Li_c

I Ht4 TmAUOC IkAlvIVG DEvELOI,mtNT INSTITUT+

+TRITSH EMNASSY 3HITISH utFLNit. SIAFF

2 CA11101AN JOINT ,5141-1.

I COLS (w) LIHRAky

I FRENCH ARMY ATTALlt

I AWKIAN EMBASSY UttLNSL, MILITANT ANIU AIM WACHL

1 CAJAulAN UEFENLt _IA1SUN 110.1- ATIN: LOW6ELLOR, ULFENCE R ANU U

I ROYAL NETHERLANoS OlciA5hY MILITARY AllACmE

1 CAvAulAN FORCES HOSE CORNAALLP, AITN: PERSONNEL SLLLCT1ON

/ CA4Au1AN FORCES PERSONNLL APPL IISCM UV1T

1 ARmY PERSONNLL 4LSEAHCH LIAULISHMtNT

h LIHR.RY OF CUNk,-(ESS LXLMANA,L ANL) GIFT Oh/

1 OLFE,4SE TLCHNILAL INtOmmAlION (GN ATIN: 0T1L-UUA-2

14n LImRARY OF CONG,4tSS UNII UOCumENTS EAPtuITING PROJLLT

I US GuVERNMENT PRIV:TING Otl. LTHHAHY, PUtILIC UUCUMLNTS DEPARTMLNT

I US GuVERNMFNI PRIVIING UPL LImhARY AVU STATUTORY, LIM DIV (SLL)

I THE aRMY LIR/4144 ATTN: ARMY SIUUltS StC

4 / /

1 / /

NUmmER ut ADURESSLL ?utI TOTAL NUMBER OF CURLS 381

106

1 v4'''-'