Research Note 84-81

EARLY TRAINING ESTIMATION SYSTEM (ETES)

FINAL REPORT

APPENDIX H: USER'S GUIDE: MEDIA SELECTION PROCRAN1

Lawrence H. O'Brien, Donya Boylston and Rose White

Dynamics Research Corporation

0

submitted by

Michael H. Strub, Chief

ARI FIELD UNIT AT FORT BLISS, TEXAS

and

Jerrold M. Levine, Director Dl

SYSTEMS RESEARCH LABORATORY L E C

B

Uj Research Institute for the Behavioral and Social Sciences

CJ Approved for Public resleasi distribution unlimited.

e This report, as submitted by the contractor. hall boon cleared for oons to Defense Technical Information Center I(DTIC) to comply with regulatory requirementsl. It has been given no primary distribution other then to DTIC

an ilbeaalbeolthog TCr other reference services such as the National Technical Information

Sevc NI) h ies pnes ido findings contained in this report are those of the authorisl and

shudntbIosruda nofca 0eartmeink of the Army osition, policv, or deision unloe so designated

by other official documnentation.

J..,..... 84 -06 26 161.
PAGE READ INSTRUCTIONS

REPORT DOCUMENTATION BEFORE COMPLET NG FORM

. REPORT.NUMBER 2. GOVT ACCESSION NO. 3. RECIPIENT'S CATALOG NUMBER

Research Note 84-81 /__/ -. __, _

A. TITLE'(and Subtitle) S. TYPE OF REPORT & PERIOD COVERED

EARLY TRAINING ESTIMATION SYSTEM: FINAL REPORT FINAL

APPENDIX H: USER'S GUIDE: MEDIA SELECTION 6. PERFORMINGORG. REPORT NUMBER

PROGRAM

7. AUTHOR(e) 8. CONTRACT OR GRANT NUMBER(e)

Lawrence H. O'Brien, Donya Boylston, Rose White MDA-903-80-C-0525

9. PERFORMING ORGANIZATION NAME AND ADDRESS I0. PROGRAM ELEMENT. PROJECT, TASK

AREA & WORK UNIT NUMBERS

DYNAMICS RESEARCH CORPORATION

60 Concord Street

Wilmington, Massachusetts 01887 2Q162722A791 I. CONTROLLING OFFICE NAME AND ADDRESS 12. REPORT DATE

Army Research Institute June 1984

5001 Eisenhower Avenue 13. NUMBER OF PAGES

Alexandria, Virginia 22333 193

14. MONITORING AGENCY NAME & ADDRESS(I1 different from Controlling Office) IS. SECURITY CLASS. (of this report)

US Army Research Institute for the Behavioral

and Social Sciences 1S.. DECLASSIFICATION/OOWNGRAOING Fort Bliss Field Unit, P.O. Box 6057 SCHEDULE

Fort Bliss, TX 79916 16. DISTRIBUTION STATEMENT (of thie Report)

Approved for public release; distribution unlimited

17. DISTRIBUTION STATEMENT (of the abstract entered in Block 20, Ut different from Report)

IS. SUPPLEMENTARY NOTES

Charles Jorgensen,Contracting Officers Representative

13. KEY WORDS (Continue on roverse elde If neceeaaur and Identify by block number)

Training Training Estimation

Instructional System Development Simulation

Data Base Management Training Effectiveness

Task Analysis Training Effectiveness Analysis

Front End Analysis

23, ABSTRACT (Aot.we v erie a e 1I nuewaiy and ldertlf by block numowb)

This report describes the research and development activities conducted under

the Early Training Esti.,ation System (ETES) development project. The Early

Training Estimation System (ETES) is an integrated set of procedures and

automated tools for estimating training requirements during the earliest

phases of the weapon system acquisition process. The ETES has three major

components; a System Description Technology (SDT), Early Training Estimation

Aids and Procedures (TEAP), and Evaluative Technology. The SDT is a data

-- base management system for storing and trackin task and training-related

DD I J, 143 EDITION OF' NOV 6S IS OBSOLETE UNCLASSIFIED

SECURITY CLASSIFICATION OF THIS PAGE (When Date Ent.red)
Unclassified

SECURITY CLASSIFICATION OF THIS PAGE;(en Date Ented)

"data. The data in the SDT is used in the TEAP to estimate training requirements for a new system. These training requirements include estimates of

task requirements, course requirements, and resource requirements as well as

estimates of training costs, training efficiency, and training effectiveness.

In the Evaluative Technology, the integrated impacts of training requirements

are assessed, training alternatives are evaluated,, tradeoff and sensitivity

analyses of key parameters/are conducted, and the relationships between ETES

outputs and key Army acquisition documents and processes are specified.

This report provides an overview of the components of ETES, describes the

research conducted under each of the five ETES study tasks; and outlines

future directions for improving ETES.

The final report and Appendixes are published as separate volumes as follows:

Final Report: ARI Research Note 84-78 (includes Appendixes A through E)

Appendix F, User's Guide: ARI Research Note 84-79

Appendix G, User's Guide, System Description Technology: ARI Research Note

84-80

Appendix H, User's Guide, Media Selection Program: ARI Research Note 84-81.

Appendix I, User's Guide, Automated Resource and Cost Estimation Technique:

ARI Research Note 84-82

Appendix J, User's Guide, Automated Planning and Scheduling Technique for

Individual and Collective Training Plan: ARI Research Note 84-83

SFor

0 06, .....

* Av!

Dist -. 1 1

UNCLASSIFIED

,'- SECURITY CLA&SSIFICATION OF T.IS IAGE($47en D)ate Entitred)

:!( .. . . ..-..-... . . . . .. . . . . . . ..- 6- ,-. . . ..
- "PREFACE

This user's guide is part of the Early Training Estimation

System (ETES). Development of the ETES was sponsored by the

Army Research Institute (ARI) under contract No. MDA-903-80

C-0525. Dynamics Research Corporation (DRC) of Wilmington,

Massachusetts was the contractor. The contract monitor for

the project was Dr. Charles Jorgensen. The conceptual

framework for the Media Selection Program was developed by

Dr. Lawrence O'Brien and Ms. Donya Boylston. Ms. Rose White

developed the software for the Media Selection Program.

. .

4.,

' I :. % .. . .\. .._, . . . ... .,V\ " " -,, , ' v. : . ., . _. .. -.- 4....-,. .. '.j-..-'
TABLE OF CONTENTS

Section Title Page

1. Introduction 1-1

1.1 Objectives 1-1

1.2 Potential Users 1-1

1.3 Conceptual Overview 1-2

1.4 Procedural Overview 1-3

1.5 Products of Media Selection Program 1-9

1.6 Guidalines for Using this Guide 1-12

* 2. Basic Operational Procedures 2-1

2.1 Hardware Requirements 2-1

2.2 Software Requirements 2-2

2.3 Program Menus 2-7

2.4 Activating the Program 2-9

2.5 Preparing Input Data 2-15

3. Menus and Frames 3-1

3.1 Introductory Frame (MS-I) 3-6

3.2 Criteria Selection (MS-2) 3-8

3.3 Task Selection (MS-3) 3-10

3.4 Selection of Psychological Variables (MS-4) 3-14

3.5 Stimulus Variable Selection (MS-5) 3-18

3.6 Response Variable Selection (MS-6) 3-22

3.7 Feedback Variable Selection (MS-7) 3-26

3.8 View Data for Psychological Variables (MS-8) 3-30

3.9 Data Review-i (MS-9) 3-34

3.10 Criticality Weights (MS-10) 3-38

3.11 View Criticality Data (MS-11) 3-42

3.12 Data Review-2 (,MS-12) 3-44

3.13 Media Category Selection (MS-13) 3-46

3.14 Computation of Efficiency/Effectiveness

(MS-14) 3-50

3.15 Option for Viewing Efficiency/Effectiveness/

Utility Scores (MS-15) 3-52

3.16 Selection of Media Categories for Display

(MS-16) 3-54

3.17 Graphical Screen Display (MS-17) 3-56

3.18 Utility Weights (MS-18) 3-60

3.19 Current Criteria Scores (MS-19) 3-64

3.20 Selection of Objective (MS-20) 3-70

3.21 Selection of Constraint (MS-21) 3-74

3.22 Constraint Level (MS-22) 3-76

3.23 Rerun Option (MS-23) 3-82

V

.1D

k"• ,' .% .-. ' ' , ,' , S" -' - -* .,' -- -. ....-.- .'-.. . .. . .-€ -. '
W1. 4-. AF.%7

TABLE OF CONTENTS (continued)

Section Title Page

4. Example Application 4-1

5. Evaluating and Using Results 5-1

5.1 Step 1: Conduct Sensitivity Analysis 5-1

5.2 Step 2: Identify Specific Media within

Categories 5-4

5.3 Step 3: Assess Impacts of Practical Factors 5-8

5.4 Step 4: Enter Data Into SDT 5-8

A Technical Discussion of Media Selection Program

Algorithms A-i

B Definitions for Key Variables B- 1

VVi

e A
LIST OF TABLES

Table Pag e

1-1 Possible Assignment Strategies 1-7

1-2 Example of Efficiency Score Listing 1-10

1-3 Example of Task-Media Assignment Listing 1-11

2-1 Advantages and Disadvantages of Different

Criteria 2-18

2-2 Psychological Variables 2-20

2-3 Media Selection Program Planning Worksheet 2-21

*2-4 Task Data Worksheet 2-26

2-5 Media Categories 2-27

2-6 Possible Assignment Strategies 2-29

3-1 Menus and Frames 3-2

3-2 Media Categories 3-48

3-3 Example Printout of Criteria Scores 3-66

3-4 Media Categories 3-67

*3-5 Possible Assignment Strategies 3-72

3-6 Example of Printout of Constraint and Objective

Variables Scores 3-78

3-7 Guidelines for Selecting Constraint Levels 3-79

3-8 Example of Assignment Printout 3-84

5-i Alternatives for Sensitivity Analyses 5-3

5-2 Examples of Specific Media Categories 5-5

5-3 Worksheet for Documenting Media Assignments 5-7

5-4 Practical Factors to Consider in Making Media

Assignments 5-9

A-i Example Printout of Constraint and objective

variable Scores A-i

A-2 Example of Assignment Printout

B-i Definitions for Psychological Variables B-2

*B-2 Definitions for Criticality Variables B-9

B-3 Relative Cost Values B- 10

Vii
LIST OF FIGURES

Figure Page

!-1 Overview of Media Selection Program 1-5

2-1 Apple III Keyboard 2-3

2-2 Inserting a Diskette 2-6

2-3 Typical Menu 2-8

2-4 SDT Introduction (SDT-l) 2-11

2-5 System Selection (SDT-2) 2-12

2-6 System Description(SDT-3) 2-13

2-7 Mode Select (SDT-4) 2-14

2-8 Select Applications Program 2-16

2-9 Preparing Input Data 2-17

3-1 Overview of Media Selection Program 3-3

3-2 Detailed Description of Logic in Media

Selection 3-4

3-3 Introductory Frame (MS-i) 3-7

3-4 Criteria Selection (MS-2) 3-9

3-5 Task Selection (MS-3) 3-11

3-6 Selection of Psychological Variables (MS-4) 3-15

3-7 Stimulus Variable Selection (MS-5) 3-19

3-8 Response Variable Selection (MS-6) 3-23

3-9 Feedback Variable Selection (MS-7) 3-27

3-10 View Data For Psychological Variables (MS-8) 3-31

3-11 Data Review-i (MS-9) 3-35

3-12 Criticality Weights (MS-10) 3-39

3-13 View Criticality Data (MS-11) 3-43

3-14 Data Review-2 (MS-12) 3-45

3-15 Media Category Selection (MS-13) 3-47

3-16 Computation of Efficiency/Effectiveness (MS-14) 3-51

3-17 Option for Viewing Efficiency/Effectiveness/

Utility scores (MS-15) 3-53

3-18 Selection of Media Categories for Display

(MS-16) 3-55

3-19 Graphical Screen Display (MS-17) 3-57

3-20 Utility Weights (MS-18) 3-61

3-21 Print Criteria (MS-19) 3-65

3-22 Selection of Objective (MS-20) 3-71

3-23 Constraint Selection (MS-21) 3-75

3-24 Constraint Level (MS-22) 3-77

3-25 Select Rerun Option (MS-23) 3-83

Viii
. . . . . . o . -- w r r -.

LIST OF FIGURES (continued)

Figure Page

4-1 Introductory Frame (MS-I) 4-2

4-2 Criteria Selection (MS-2) 4-4

4-3 Task Selection (MS-3) 4-5

4-4 Selection of Psychological Variables (MS-4) 4-6

4-5 Stimulus Variable Selection (MS-5) 4-7

4-6 Selection of Psychological Variables (MS-4) 4-9

4-7 Response Variable Selection (MS-6) 4-10

4-8 Selection of Psychological Variables (Ms-4) 4-11

4-9 Feedback Variable Selection (MS-7) 4-13

4-10 Data Review 1 (MS-8) 4-14

4-11 Media Category Selection (Ms-13) 4-15

4-12 Computation of Efficiency/Effectiveness (MS-14) 4-16

4-13 Option for Viewing Efficiency/Effectiveness

Scores (MS-15) 4-18

4-14 Print Criteria Scores (MS-19) 4-19

4-15 Select Rerun Option (MS-23) 4-20

5-1 Guidelines for Evaluating and Using Results 5-2

Jq

ix

• "A'i", " - " " - , -. . -, -. . . z . . ,,. ., : . .- . . . . - .. . - . , . .. . . . . . . . . - ..-.-.

€ or,, ",, , ' ,' .:r - _' ' . , . ," ;' ', , .,, - .,, . ,' ," . ,,. -.. ,-. .- ,.' - . - -*. - . • . ..... _-
SECTION 1 -INTRODUCTION

1.1 OBJECTIVES

This guide provides detailed instructions for using the

Media Selection Program. This program is an automated aid

for (1) assigning tasks to media, and (2) determining the

efficiency and effectiveness of media-task combinations.

The Media Selection Program is part of the Early Training

Estimation System (ETES). The Media Selection Program is

I' designed to use input data from the ETES data base management system which is called the System Description Technology (SDT). A description of the other components of ETES

is provided in The -ETES User's Guide. A description of the

SDT is provided in the SDT User's Guide.

1.2 POTENTIAL USERS

The Media Selection Program has been designed to be used by

Army technical analysts who are directly concerned with

training development for new Army Weapon Systems. The

primary user organizations are expected to be: (1) the

Training Development Directorates in The Army schools

associated with development of new systems, (2) Program

Manager's office for new systems, particularly those

individuals concerned with training development, (3) other

Army organizations concerned with training development such

as the TRADOC Systems Analysis Activity (TRASANA) and PM

Trade, and (4) contractors who must develop training

requirements for new systems. It is assumed that users are
familiar with the basic Army tra ini ng development

terminology and processes.

1.3 CONCEPTUAL OVERVIEW

The Media Selection Program is an automated tool for (1)

assigning tasks to media and (2) calculating the efficiency

and effectiveness of various task-media combinations. The

media Selection Program will allow you to assign tasks to

media in a manner that maximizes overall efficier

maximizes overall effectiveness, or minimizes ove 11

-6 cost.l In addition, it would allow you to assign task to

media in a manner that optimizes various combination

these variables, including an overall "utility" meas..re

which combines either cost and efficiency or cost and

effectiveness.

S.Efficiency, in the Media Selection Program, is determined by

comparing the stimulus, response, and feedback characteristic of the tasks to the stimulus, response, and feedback

characteristics of potential media. More specifically, a

score is calculated which describes the match between media

and tasks on these characteristics. Efficiency for each

task-media combination is calculated by dividing this score

by the maximum match that may be achieved for the task.

1The program uses a relative cost and not actual cost to

measure the potential cost requirements of media. The

relative cost values used in the program are listed in

Appendix B.
Total efficiency for a set of tasks is the sum of the

efficiency score for individual tasks. 2

Effectiveness is calculated by weighting the efficiency of

each task by a task criticality score. The task criticality

score is a user-defined weighted combination of the eight

task factors typically used in selecting tasks for training.

These eight factors are task frequency, per cent members

performing, per cent time performing, task delay tolerance,

consequences of inadequate performance, task learning

difficulty, probability of deficient performance, and time

between entry and performance.

A matrix of relative cost values is stored in the program

for each major media category. (See Appendix B for a

listing of these cost values). In addition, a built-in set

* of algorithms is used to produce the utility measure which

combines cost with either effectiveness or efficiency. (See

Appendix A for a description of these algorithms).

1.4 PROCEDURAL OVERVIEW

The Media Selection Program is an interactive menu-driven

- ~ system. This means that you do not have to know or use a

computer language to run the program. Instead, you can run

through the program by selecting options from a series of

menus. The program will tell you what information to enter

*12

2 This concept of media efficiency was originally

developed by Jorgensen (1978) and was incorporated into

Training Efficiency Estimation Model (TEEM) described in

Jorgensen, Kubula, and Atlas (1981).

1-3
and when to enter this information. It will tell you when

you have data missing and will indicate when you have

entered data that is of the wrong type or when you have

entered data that is out of the acceptable range for the

variable you are trying to enter.

An overview of the procedures for using the Media Selection

Program is provided in Figure 1-1. To begin the procedures,

you must rate each task on (1) the psychological variables

to be used to assess the match between tasks and media, and

(2) the variables to be used to assess task criticality (the

latter is only necessary if effectiveness is being

calculated). These scores must be then entered into the

ETES data base management system, the System Description

Technology. When this is completed you must enter the

Applications Program mode in the System Description

Technology, and select the Media Selection Program. Once in

the Media Selection Program, you must then select criteria

to be used in making Media assignments. Seven options for

selecting criteria are provided: (1) efficiency, (2)

effectiveness (3) relative cost, (4) cost and efficiency,

(5) cost and effectiveness, (6) a utility measure, combining

efficiency and cost, and (7) a utility measure combining

effectiveness and cost.

Once the criteria have been selected, you must select the

tasks to be included in the analysis (only tasks already in

the SDT may be selected). Typically, the tasks for single

course module will be selected for each analysis.

" With the analysis criteria identified, the psychological

variables to be used in calculating the match between tasks

.g' and media must be selected. You may select from twelve

1-4
-~~~~ - ... --

INPUT DATA

ON PSYCHOLOGICAL

VARIABLES (SOT)

SELECT

CRITERIA

SELECT

TASKS

SELECT PSYCHOLOGICAL

VARIABLES

SELECT/WEIGHT

TASK

CRITICALITY

FACTORS

I COMPTE SCORES OF I DISPLAY MEDIA ON

SCORES PSYCHOSI LOGICAL

L - ~ -. J VARIABLES

WEIGHT

CRITERIA

SELECT

OBJECTIVES/

CONSTRAINTS

ASSIGNIPY RERUN

IRELATIVE TASKS TO RESULTS ANALYSIS

.3, I COST OF MEDIA E TA

MEDIA

3.' CATEGORIE L

Figure 1-1 Overview of Media Selection Program *,,.-1 ": . '€ ;;-4 ' . ::...,-.-.'.,," ,".'f --...- ,., .:.', ., , v . -. ,",,...-..'4-. , .";', -, . . ,, " -. . . ,". ."
* variables assessing stimulus characteristics, six variablesj

assessing response characteristics and four variables

* assessing feedback characteristics.3

If you have selected a set of criteria involving effective-]

ness (effectiveness, cost and effectiveness, or utility with

effectiveness), the weights for the component criticality

variables must be entered so that a composite task

criticality score can be computed. Eight variables may be

used in the calculation of task criticality: (task

frequency, per cent member performing, per cent time

performing, task delay tolerance, consequences of inadequate

performance, task learning difficulty, probability of

deficient performance and time between entry and

performance).

At this point, the program has all of the information needed

to calculate efficiency and/or effectiveness. It will use

this information to calculate the relevant measure and it

will display the results.

If you have selected one of the two utility measures

(combining either cost and efficiency or cost and effectiveness) you will then be required to enter the weights to be

used in the computation of utility. Following this you must

select the objectives and constraints to be used in assigning tasks to media. A listing of the possible combinations

of objectives and constraints is displayed in Table 1-1.

(More information on strategies for selecting objectives and

These psychological variables were taken directly from

the TEEM Model. (Jorgensen, Kubula, and Atlas; 1981).

1-6
r. .

 r .. . . .. .. 

. > . 

1 S S.. 7

r.T UWT 

. 

A.umA,4,T~ln x __ jx x ___ __ >x, x__ 

ISOD pu

AouiOr;;3 X

4-' 0 (a44- 0 W- .4.

0 .-4 -o .- 4 U) -4 -4 0 -4 w- .-4 -,0 to

4-4 m~ a ~ (o(o0 U) (a r (a) fa-4 0 (a m4 (4) .4.1 4jC .4- 4.) 4) .- ) (a JC 41( 44 .4j 4 .4 4 .10

>1 0C 0(4 0 0 X.C 04 0 ~ 0 0 0 fa 0 Aj

u) .4.1.J .4j.. .4.4-4 4 4.j .) .4j.. .4J 4.' .4-.) .J

C .4-'( > 4.d4) .j -4

4 s.6~ .4.1 .4.j -4 A.14w .4. 414 .4. 4J14 .41

- 44) M w4. (a)O 4.j (a)4a) m44- 6 M w --4 (a). m44~

4- CC~ 4C) C u a. C41 =C4) .C 0 -4 aC4-1 .C0) .

-. 4 .4J 4 M .4.1. 4J a) 4j (a 4d 4-J 4j -4 4j.i4.) .J41 4. 4)

44 a w ) z 4) c 0 44-4 c w4 ) 4) 4

4j) 0O w W )~ w) . m0 w. M 04- to ( 04) C.O. co0

En 4U U) .Cul -4 W) U) U) w- U) (a)

w) a U) a -4 'aJ 0j ) a V-4 10OW 0) Vm. a- a z

w) w4 0)00 0 0D .4j ) >4 ) 04) .J 4 )) 0) W 4j U) 44) fA. C .4.1: .4.4En (4a .- ) - .4j z .4j 4 4u .4.U) * 4J U) - .4.)

4)) 4))0) 0V CO>, 0_ f40 () >1 (o), (0 ()4a > (UU) >4 fa ( 4- 4 C> w4j4 .4) w 4) U) W 4. w 4U) W- 4-4 w. 4 M-) 

0.D 0. )--4 ( 4 )>1 - w. 0-C c * 4>4 w 4- 0)' 0 >,C aw Ccw 0) aC> 4 c-1 C4jc C C a) -4- C add CaO uwcwwC

) (D1 0--.) U )V-4 -4 G)0) W0-1- 4)) )-M 4 0)4 4j ) ( dC - Q 0.> -4 W)

-(4 '4. ) M 4j -4 M .'s4 a cr L4.UC ) .- 4J 4 al U) .C m.'0) u 4-4 r (.u) u 44 3 -4j 4 -Z -4 0 .4- .- 4 4-4-4 M (D- 44- U) 44-o4j4.DLW U J luU)u 4 ) 4 E

z -f W ) -4 U :3 .- -4 c -4 )4443.4 0)4 4) -4. -)C4 U ) 44 U) 4

4j4J. 4.4 -4 0 .4. -140) -4:- -4 .4. 4-44) 44.4a) En

m. ()4))4.4 (a(4 (a m ) u -4 (0(403 (a (a '( ( -4 (0444 M344 M 00 - ~ C--4 0 .4 4. 44 *.4 -4 4.j44 -4 .4.j 4.4 *-4 4j -14. - 444 --q 0>

41 VOO 0 1 V000 0 o 01 0 a )

0- 0> Vla 0 -4 aVO) V1 l 'aA V.)- -4 .4J1) .4j) .4J.-4

rnV 0 A-'0) 0Oa> 0.41 a V Oc 0Oa> 0CU 0 00> 0 0 >0 00

U) 4) .- -4 4j4)) .41 4) 4)) .4.14u )) .4.(M4W .4 M 0 4)) .4-14J4) 4J 4J4) .4J WJ---4 M z V VS V ) r2 -4 -4 u S r-4 '-4 4-4

t cV 10 EnC 10 ( )V C V wV En U)V '0 M 0 W 4-1

a) (a ) j w- UN4 wO wC 0)N ) V) N4) to N 0 .C W M a4) w)4)) m 0w w

1-4 4.) (aV'4-w (a 4*14 (a V 44 .4. M -4 -. 4 (a -.4*-4 M4 -4 .4. (a *-4 m4 .- I (4) 44

V0 4. 4) -4 .4 r kW4J W4.40 .4E44 .4r=44 4 r=-4 .4. rO4.4 4J 044 .4-J0O

U) 1 N u .1-4 N V .- 1-4 -4 -.4 -4a) a Q0).-I a) -M 0) En) -4 W IX - -4-.4 M)4 0 4)> -4 - ) -4)4 -44> N0 r-4 N -410 N -4

Ol C -2 E4 -4 M4)w -44) a fa ) '-q(4M4w -4(0m)w c -4 -4 ) -4 -4 ) -4..-4 (D 0.ON (a -i rn ( m (a -> 0t" ( S04 '4) m '0-4 E a' m E m eoO 4)2 E> .j x I U) x 0) -4 ul ) -- 4 -U) -4U) *,4))

E -' C) t.4(4M w 44W 1 4- 4 - C (U 4.4( 44U FA 44U)V' U) 44 X 1 4-4 X 1 X

rn C F 4 CU) - sw- C-=- (n) 0U) . CU0-)w CU- C mU) w 0 U) w 0Cm 

E -4 4)) 0l 4j)w w 4)) w - *( 4j)4))w )14w)w 0)40 *- 4d)* w 4j1. in j.)).44

-4 0 U)44)U .4 (a. w ) U)4. ... ) W)4J- .- 1 U)0 C-aU

>OO W0 CO >1 -4 1 U0 U) O )U U- )

CO 0.. (0 -4 4) (5 -4-44.) (-4 4 M CU - -, -( -4))4 )4J -4)OU -41 (0M -) 4 J (-4 44 4J 0

4J) 04J U _44 3 U 4-)40 .4-4U) 4- 4-4V0 41-4 ) 4-4W 4.)4) t -) . ,q to.4 4 w)

E-4w C444 t w0. C4.4 w44U 1C4 (0 -4 (n U-4 U)u4)-- 4 ) -4O -4O CU) -

.4- to- U) 4 d 4 M44 = 4 3( .4-1 J 3 U)P A 3 4

W- (0 =C4Wr-4 0 C L ) (a0 r- 44 t(0 a~ 440 w=c3 c no 4c W -a j w) 0 : ( 3 W 4 (D a) 4-4 0 <0

E-4 04 0 0 r-

'44 4) U) 4-4 4) (P 44 4-a U)

O 0 '.' : 0 0 44 z 0 0 4-4 44 0

z Z ) C ):C Q. C 0 )C

U)

.,q 4) -4 X4 44X 44 X)4J

20 42-o4

-. 50 )444. X44. (044. (044 .2
Isoo X ___

SS9Ue9vToeaj

4lTM wA~tTIfl. -xx

ISOZD pue * SS8UaA43a;ja XX_

qpue ADuorT;; ___ x

0 0

0 - 0 ~ 0

> 4J 41 0 .. .4 0 .J4

0a a 0

1 0 .) .) .4.) )

OAi 41

a) ~ . to4.) (a. .~

tr. ty, U) 4) tr t

>4 4) c U) .14 C.

Al a) U~ l c)L a)

41 1 40 > - 4 )

W 39 - 4 0 -4-0 -

41 -) 4)~ --4Li S4 WO -4 4 4 .* a)L

.1 4 a) a)244 a

V4. 0>4) 0 0) 0

4 -4. 0 r-1 -4 0

Li V) to ) (a >-4 t)

U) D e J 4 J - 4 V 4) -44

0 0 to) 0 ) (n0

C ~ .41 44 ) 4 ) 4 4

-- 4

.. 4 C .- 4 . 13 -4r0V 10 -4 010

tr (4) (aa'4 . W). m- a).

to W4 ON'-' 0 4 -4 ON"64' t U40 0 *0 N 0

(a ~ C) U r 41 4 a) 4 )

0- c- U) ) c44 U) *4 ) C (

- 4 trl U2. U.4)

to -.U 4 a ( . m F U) M r) 

04 - . 0) 0 0 u) to -4.i 

U W) ) ul U)

E- cl to cOO0c 0O 0O a)

W) U ..4

to)(

4.) -i C 4) (

4) -4 4 a)

01 -4 0 -4 4)

0 'i 4-1

c a)

.4.

uC

1A . . . . . . . . . . . . .4

it.
constraints is provided in Section 2.5). Once this

information has been put into the computer, the program will

optimally assign the tasks to training media. For instance,

if you selected "maximize etfectiveness" as your objective

and "minimize cost" as your constraint, the program would

determine the assignment of tasks to media which gives the

highest overall score on effectiveness and still remains

..- under a user-specified level of overall cost. Once the

S' initial assignments have been examined, you may wish to

examine the effects of several alternatives including

changes in (1) objective, (2) constraint, (3) criteria, (4)

task criticality variables, (5) psychological variables, (6)

task criticality variable weights, and/or (7) utility

weights.

After you have explored these alternatives, the task-media

assignments from the Media Selection Program should be

compared against practical factors, such as resource availability, and schedule and budget constr "nts and a final set

of assignments should be constructed and entered into the

SDT.

1.5 PRODUCTS OF MEDIA SELECTION PROGRAM

The Media Selection Program will produce two major products:

(1) a listing of the efficiency, effectiveness or utility

scores for each media-task combination and (2) a listing of

the optimal media-task assignments. Examples of both of

* these products are provided in Tables 1-2, and 1-3

. respectively.

1-9

-'*.'. ' ' .-.' ''. ' L -"", , .. "- ", . " ." " " . - "-"- . .i/-___* . -.'. .'. - -.- * .- .*. ". .- . ".
4. 0

0* 1 t7 0 0D 0

41 -4 r

U) u

1

en c

0c 0

--4

44-)

00

4)

-44

00

4 1 1

E4 41

C9 0

a4)

.0 0.

'41 44I

0

1-10
9',

41 4-)

4...-

4.-.4

t.,-

W 0 a

N/ - 0,,

4,..1

24 o

0

444

0 0

404

W 0

-4 9-4 -)

-l 2-

-' , -. .- ) u-.

SOCO
1.7

1.6 GUIDELINES FOR USING THIS GUIDE

The remainder of this User's Guide is divided into four

sections. Section 2 describes the basic operational

procedures for using the Media Selection program and the

hardware and software which are required to run the program,

and procedures for developing the input data needed to run

the program. Section 3 provides a detailed description of

the menus and frames in the Media Selection Program.4

Section 4 contains an example application of the Media

Selection Program. This example is designed to provide

first-time users with some hands-on experience in using the

program. No experience is required to run this application.

Section 5 contains guidance on evaluating and using the

results of the Media Selection Program. Appendix A provides

a technical description of the Media Selection Program

assignment algorithms and procedures. Appendix B provides a

detailed description of the psychological variables,

criticality variables, media categories and relative cost

values which are used in the Media Selection Program.

4 1-12
.- 77- -7.--

SECTION 2 -BASIC OPERATIONAL PROCEDURES

This section describes basic operational procedures for

using the Media Selection Program. In addition, it

.4 describes the hardware/software required to use the program

and the input data which must be developed before the

program can be run. This section is divided into five

subsections. Section 3.1 describes the hardware required to

use the program. Section 3.2 describes the software

required to use the program. Section 3.3 describes

procedures for using the program menus. Section 3.4

describes procedures for loading the Media Selection

Program. Section 3.5 describes the input data which must be

developed to run the program.

2.1 HARDWARE

To use the Media Selection Program, you must have an Apple

III computer with:

0 128K Bytes of RAM memory.

0 A video monitor. The monitor can be black and

white or color. However, the program does not

produce color images.

o A 5 megabyte Profile hard disk.

0 An additional floppy disk drive (that is, over and

above the disk drive built into your Apple III).

2-1
4.;

T - .. .7 - 7 Y .

This drive is needed to make a local floppy disk

copy of an SDT data base and to load the System

Description Technology (SDT) software which

contains the SDT data base management system and

the Media Selection Program.

o A printer. A printer is needed to get hard copy

output.

If you are using the Apple for the first time, be sure to

follow the APPLE III Owner's Guide for instructions on how

to connect the computer, the monitor, and an extra floppy

disk drive. Also, follow the manufacturer's instructions on

how to connect your PROFILE hard disk to your system.

Procedures for insuring that your printer and other peripherals are properly linked to SDT software are described in

the SDT User Guide.

2.2 SOFTWARE REQUIREMENTS

To use the Media Selection Program, you will need the

following software:

o SDT Program Diskettes. The program diskettes

contain software for both the SDT and the Media

Selection Program.

1 For information on obtaining SDT software, contact Dr.

Lawrence O'Brien, Dynamics Research Corporation, 7 Lopez

Road, Wilmington, MA 01887.

2-2

S21 ~ v
LLim. . . .~ ! . 4 .1 . . .. I , [ .7 ; .s .I. . 1 : . , 9 1 , , :

m1 1 6 V 

Fiur 2-1 The AppleIIIII Keyboard

-2-3

-. 47

.% '.9 ..

n I.9-. - - t -

*

i I 2 4,

4. B S S ' " Figure~~ 2-1 Th pleIIKebad

,, - ,.4 .",. , 'I_______''.. . . . .. ---.. ---. .- ., . .-. - ., ..-, . .- , • .-.- -. . , . . . . . . . .
o SDT Boot Diskette - This diskette is used to bout

(that is, activate) the SDT software.

--- 4

,.-

4.

4:.*

2-4
o SDT Backup Diskette (Optional) -These backup

diskettes are used to make a local copy of an SDT

data base.

2.2.1 General Instructions for Handling Diskettes

To insert a diskette, open the disk drive and slip the

diskette into the slot with the label facing upward as shown

in Figure 2-2. The edge of the diskette with the oval1

Ncutout should enter the drive first; the edge with the label

should enter face up and last. Gently push the diskette

into the drive; do not bend it. Close the drive door

firmly.

To remove the diskette, open the door and pull the diskette

straight out of the slot. If you leave the diskette in a

drive for long periods of time without use, it is a good

idea to open the door so the read/write head does not rest

on the diskette. NEVER REMOVE A DISKETTE WHILE THE RED

LIGHT UNDER THE DOOR IS ON. This can permianently damage the

diskette and is almost certain to destroy the information on

4., it. You may be able to reuse such a diskette, but you will

not be able to recover the lost data.

2.2.2 Loading Program Software

The software which contains the SDT and Media Selection

programs should be stored on your PROFILE hard disk.

To load the SDT program into your hard disk use the

following procedures:

2-5

5..%
.

I

1I

S.

.4
0 Load the SDT Boot Disk into the built-in drive.

Turn the computer off and then on again using the

V switch at back left of the Apple.

0 Follow the instructions which appear on the

screen, and load the SDT Program Diskettes into

the additional floppy disk drive.

Once the SDT Program Diskettes have been loaded into your

PROFILE hard disk, you will not have to enter the SDT

Program Diskettes again.

2.3 PROGRAM MENUS

Unlike many computer programs, you do not have to know a

computer language to use the Media Selection Program.

Instead, you can make commands and move through the program

by selecting options from a series of menus. An example

menu is listed in Figure 2-3.

To select a menu option, you must highlight it with the

cursor and press the RETURN key. The cursor is a

highlighted area on the screen, as shown in Figure 2-3.

You can move the cursor by hitting the up or down arrow

keys. Moving the cursor to a menu option causes no action

to take place. To select a menu option, you must move the

cursor to the option and press the RETURN key.

5 Some menus in the program may have so many items that not

11 of these items will be able to be shown at the screen at

2-7
, 

y 

WW U WY W

WW U 1. UX 

 .

 V 

 . 

 ~ -- . S.. - *. -. .-- .. -~

*4~I

-' . WELCOME TO

MEDIA SELECTION PROGRAM

.4.

- Return to SDT

- Continue with Program

V..5.

-'4.

-S

~ .1

.5..6

--- S

-S

*4~t

-. 4 .4.

--. 4

-. 4

-a.-'.

*4* 4.

%;qm

4.

4.

-a,

%* *4

Figure 2-3. Typical Menu

2-8

-F
any one time (such menus will have the phrase MORE at the

bottom). To see these additional items, you must scroll the

menu up or down. To scroll down, move to the bottom of the

menu by hitting the down arrow key. Continue to hit this

key and the menu will move up showing you the additional

items in the menu. You can then scroll back up by hitting

the up arrow key.

2.4 LOADING THE MEDIA SELECTION PROGRAM

To run the Media Selection Program, you must do three

things. First, you must load the SDT Boot Diskette into

your built-in drive. Second you must load the f ive SDT

Program Diskettes onto your PROFILE hard disk. Procedures

for accomplishing these first two steps were described in

Section 2.2. When you have successfully accomplished these

two steps you will see the f irst menu of the SDT data base

management system on your screen.

In the third step, you must enter the SDT program and move

your way through the program to the Applications Program

mode where you can select the Media Selection program and

load this program into "active memory." To do this you must

* run through four SDT menus. On the pages which follow these

four menus are listed and the specific options you must

select on each menu are described. When you are finished

with these four menus, the first menu or frame of the Media

Selection Program will appear on the screen (This frame will

say "Welcome to Media Selection Program" at the top).

Details for using the menus and frames in this program are

provided in Section 3.

2-9
1 dV

Action 1: Examine SDT Introduction Frame

The SDT introductory frame (see Figure 2-4) will appear on

the screen after you have loaded the SDT Program Diskettes

and SDT Boot Diskette (see Section 2.2). The frame will

describe instructions for using SDT menus. Read the

instructions and use what you have learned to move to the

next frame. More specifically, move the cursor up and down

until you get to the option Skip to Start of Program, then

hit RETURN.

Action 2: Select System

A menu will appear on the screen asking you to select the

system to be examined (See Figure 2-5 for an example). Move

the cursor to the system you want to work with and hit

RETURN. (See the SDT User's Guide for instructions on

setting up an SDT data base for your weapon system).

Action 3: Examine System Description

A description of the data base you selected will appear on

the screen (see example in Figure 2-6). Read the

description, then move the cursor to Continue and hit

Return.

Action 4: Select SDT Mode of Operation

A menu will appear on the screen asking you to select one of

the seven SDT modes of operation (see Figure 2-7). Move the

cursor to Applications and hit Return.

2-10
V X

WELCOME TO THE

SYSTEM DESCRIPTION TECHNOLOGY

(SDT)

Whenever the right arrow appears, press the up or down

arrow keys on the keyboard or use the joystick until the

desired menu action is highlighted. Then press the RETURN

key on the keyboard or push the joystick button.

For a further explanation at any time, hold down the escape

key (upper left hand corner) or toggle the auxiliary switch

on the joystick. The explanation will remain on the screen

for as long as the key or button is held down.

More instructions

Skip to start of program

Wait until later time

Fiqure 2-4. SDT Introduction (SDT-I).

. 2-11

".-- "" ',4 - "- Lt,%' . %' . ." - - "-" .-. -. - ....-..
SYSTEM SELECTION

Select system for this session.

-SDT Example

- ARI Example

.

4'.-Figure 2-5. System Selection 2-12
System Selected:

SDT EXAMPLE

DESCRIPTION GOES HERE.

UP TO 10 LINES

.'21

.4r

-'%

2-1

4, -' -; , , - - , , . - . . . , . , - . . . . - . - ., . . - , . . , . . ,

'h.* l
SDT MODE SELECTION

SELECT SDT OPERATION MODE:

-EXIT SDT- (Terminate the Program)

-AUDIT/UPDATL- (Examine Audit Trail/Update on Honeywell at DRC)

-INPUT DATA- (Add Tabular Data into the SDT data files)

-CORRECT DATA- (Change Specific Attribute for a Particular Entity)

-OUTPUT DATA- (Display or Print Existing Data)

-SAVE/RESTORE- (Copy this system to or from diskette)

-APPLICATIONS- (Execute the Applications Programs)

-AUXILIARY- (Execute Auxiliary SDT Programs)

Figure 2-7. Mode Select (SDT-4).

2-14
Action 5: Select Media Selection Program

A menu will appear on the screen as asking you to select a

SDT applications program (see Figure 2-8). Move the cursor

to Media Selection Program and hit Return.

2.5 PREPARING INPUT DATA

Figure 2-9 outlines the steps which must be taken to prepare

input data for the Media Selection Program. More details on

each of these steps follow.

Select Tasks: You must select the tasks to be used in the

analysis. (You can only select tasks which are already in

the SDT). The best strategy is to select all of the tasks

associated with a particular course module. This

information may be obtained from the standard SDT output

report labeled "Quasi Program of Instruction-Part 2". This

report is listed under the SDT entity Courses. (See the SDT

User's Guide for instructions on obtaining this report). A

maximum of fifty tasks may be examined in any one

application of the Media Selection Program.

Select Criteria: You may select from one of seven criteria

for assessing the "goodness" of a Media task assignment (1)

efficiency, (2) effectiveness, (3) cost and effectiveness,

(4) cost and efficiency, (5) relative cost, (6) a utility

measure combining cost and efficiency and (7) a utility

measure combining cost and effectiveness.

Table 2-1 summarizes the advantages and disadvantages of

each criteria. More details on the advantages and

disadvantages of the different criteria are contained in the

remaining portion of this discussion for inputting data.

2-15

S.-.
SDT APPLICATIONS PROGRAM

Lelect Applications Program:

-Terminate Session- (Terminate the Program)

-Return to SDT (Return to the SDT Main Program)

-Media Selection Program (Perform Media Selection)

-Resource and Cost Estimation (Interface with VISICALC Program)

Technique

Figure 2-8. Select Applications Program.

2-16

'm % . " .'. ' • .. • .% .'. . . . . . .. "- " . . .' " .. . ". " - . " .' .. '. " •. '- " " . . "- " " .e , % "
SELECT

TASKS

SELECT

CRITERIA

SELECT STIMULUS, RESPONSE, AND

FEEDBACK

VARIABLES I SELECT

CRITICALITY

VARIABLES

ENTER DATA

INTO SDT

SELECT

MEDIA

CATEGORIES

SELECT

OBJECTIVES

AND CONSTRAINTS

'II

RUN MEDIA

PROGRAM

Figure 2-9 Preparing Input Data

2-17
4.) Q) a)

3.1 Lo 4.1 4-1 A4-i (13 U) >4 U) > (a -).

a) ~~Q 4U)~ (1) o 4- a .0 41 QC a) 4- 4 (1)

4-I .0 C -4 - - .~

>1 4 0 >904 4 >4.1 -) W a 0 .4 Q $J -4 (1 .4 0 (3) '-

0=-u E (tS-4 r=4-4 0 U) ul

4.) (aJ( 2 (32W ) :3 Ql)

WO U) a)0

0 - 0 .C 0 MV W 0 a)4- a

04-1 u .4.) 4 C C 4-' w 1.

.C- a a) (1) t C _ (a l0r

a) w ~ n U) (a -- 4 V a' 0

WOU) .14- ~ -4 4.J .- 4 4-J ~W

a) a) WVa a) En >WU a) 4-J 4J

-- e U) Ena) .e 00 0 0 C.) 4~C-4 

w U 2W r U) WE fa >1~ $4u >4~ -H--

(0( a (a 0 (ri 4J 4. . 4) 4 O - (a =

E-4 4-) V -4 0' m" - 4-H w U

41 41 4-) 4J r 4iJ -I4JC 41 a 4J -4 4-J O 4J -4

a) 0 41 ' (-q -4 O (ao W U)~ W4-4

U)) -4 H'- .4 H-400 0 --4

W)O a) WW) W-.QJ Wa)) >1

4O WW-4U.J.0 Wl U)-0 o 00u 0 0( fa w0 0 0 0~( W 4M4. L

u zU)

414J f 4 4 J a)a

0' U).~) 0) UW W1-

UU)

(04. ca. . ~ u .1 a). a)

Ca 4 (a U) 4- n 04L

Q) V.0 4-'. -H W W 4)4

a)) a) 4. 1 2 20

"(WU)C a)' :3- a) r

Vla - 4( r_4- Z) m >-I a )(

QV a) 0 41U~' o

C1 3:3 .- 1 -H () J-4 C 0 ) (a 41 (1

a) 4U) 4.) W-1 4J W(n C C)E

a) a Q)-~ 0~r 0 0,- a 0 440

M .0W41) 0 z L )u

) 4 0 H>41 .J4 U) 4-J Wn W1

C1>4 r 4J~ 040 U 0)2 0) 0 0

f(0 >4 a) CUn>)a .0

(n 4J E a) w0 to -4( a)CW

- 1 V -f 4JWU)

2n U)0 1" >u U a) ) > C >(0 .0

a)U a)4 fa)( 2l a) a)a).

E- ) U n-I U.4 U 4 .0 fn4-4U . U)CO' 00 U

(D U) (a0 0 --U) 4 o 4-4 0W3 a 3 4-4 :(C 

4 ) a )W-i( Q)0 a) .'.4..0 a) 0..j -0-a)

>- En U) ro-4 U) r)4-4 W W 4- U)4 WW-4U

U) U) w~0 U) - ( rv 44( U)1cnrIU

U) W0- ra u U) -t 0 a (a'-i

U) 4-) U) U)

U) U) U) M .0W

a) 0 a) 4 ~4-a)

>4 C u >4 C, a9

a) 0 a) 3:u3:a a ) VC Va>>

a) -'q > Ca C-4i > >,.H

- -H4 a) (o 4.I-.-4 4-J.) a) -1 -4 1)

t4 4.4 4.4- U) 4.4 U) 4-4 .- 4 44 4.4

L ' 4-4 a) 0 4-4 04-' 4.J 4-4 - 6

2-18

-..- ~. . . . . - . . . . . . . . . . . . . -7 . . . . . .
Select Stimulus, Response, and Feedback Variables: if you

have selected an option involving efficiency or effectiveness (this will include all but the cost only option) , you

must determine what Stimulus, response and feedback

variables will be used to calculate the match between M'edia

and tasks. A listing of the stimulus response, and feedback

variables in the SDT and the Media Selection Program is

provided in Table 2-2.1 Definitions and descriptions of

each of these variables is provided in Appendix B. Unless

you have extensive experience in Media selection, it is

recommended that you use all of these variables. The

criteria you select should be documented in the Media

Selection Program planning worksheet listed in Table 2-3.

Select Criticality Variables: Typically, there are eight

potential variables which are used to determine task

criticality in the Army: (1) task frequency, (2) pec: cent

members performing, (3) per cent time performing, (4) task

delay tolerance, (5) consequences of inadequate performance,

(6) task frequency, (7) probability of deficient

performance, and (8) time between entry and performance.

Definitions for each of these variables is provided in

Appendix B. During the earliest portion of the acquisition

process, it is recommended that three factors--task learning

difficulty, consequences of inadequate performance, and task

frequency--be used to assess criticality. In later phrases

of the acquisition process, it is recommended that all eight

variables be used to assess criticality. More detailed

descriptions of strategies for assessing task criticality

1 These categories were taken directly from the TEEM

* model (Jorgensen, Kubala and Atlas; 1981).

2-19
Table 2-2. Psychological Variables

STIMULUS VARIABLES

MEDIUM

VISUAL FORM

VISUAL MOVEMENT

VISUAL SPECTRUM

VISUAL SCALE

VISUAL CONTRAST

AUDIO SOURCES

AUDIO STIMULI INTENSITY

STIMULI PRESENTATION

STIMULI PRESENTATION RATE

NUMBER OF CHANNELS

STIMULI DISTRIBUTION

RESPONSE VARIABLES

RESPONSE MODE

INTENSITY OF RESPONSE

RESPONSE IMPLEMENTATION

REQUIRED RESPONSE RATE

# RESPONSE CHANNELS

RESPONSE DISTRIBUTION

FEEDBACK VARIABLES

FEEDBACK MEDIUM

FEEDBACK SOURCE

FEEDBACK TYPE

FEEDBACK DISTRIBUTION

2-20
* . * . *..*.%*

Table 2-3. Media Selection Planning Worksheet (MSEL).

Date ________Analysis ______

Criteria (Pick one) Utility Weights

Efficiency

Effectiveness

___Cost

___Cost and Efficiency

* - Cost and Effectiveness __

Utility with Efficiency

Utility with Effectiveness

Criticality variables Criticality Weights

Task Frequency

Per Cent Members Performing __

Per Cent Time Performing

__Task Delay Tolerance __

Task Learning Difficulty

-Probability of Deficiency

Performance

Time Between Entry and Performance __

Consequences of Inadequate

Performance

Objective Constraint

Maximize Efficiency __

Maximize Effectiveness __

Minimize Cost_____

Maximize Utility __
4.

Table 2-3. (cont'd)

TASKS ] -4.

I.

q

4

I

F

N.

p

.4'

.4

4'

.4.

I

4'

'S *

* .1~~~~

* *1

2-22

* '.,..-.... . ~ 4** *~.* V
- - 4 S ** -. a * - - * . * v a - - a -

Table 2-3. (cont d)

PSYCHOLOGICAL VARIABLES

STIMULUS VARIABLES

MEDIUM___

VISUAL FORM

VISUAL MOVEMENT___

VISUAL SPECTRUM___

* ~VISUAL SCALE___

VISUAL CONTRAST___

AUDIO CONTRAST___

AUDIO SOURCES___

AUDIO STIMULI INTENSITY ___

STIMULI PRESENTATION___

STIMULI PRESENTATION RATE ___

4%*'~ NUMBER OF CHANNELS___

STIMULI DISTRIBUTION____

RESPONSE VARIABLES

* ~RESPONSE MODE __

INTENSITY OF RESPONSE___

RESPONSE IMPLEMENTATION ___

REQUIRED RESPONSE RATE___

# RESPONSE CHANNELS___

RESPONSE DISTRIBUTION____

FEDBC VARIABLES

FEEDBACK MEDIUM___

FEEDBACK SOURCE___

FEEDBACK TYPE___

FEEDBACK DISTRIBUTION___

2-23
K7

Table 2-3. Media Categories. (continued)

CAT GROUP

MEDIA CATEGORIf

* 1 1.1 PRINT INSTRUCTIONS W/O FEEDBACK ___

2 1.2 PRINT INSTRUCTIONS WITH ORAL FEEDBACK ___

3 1. PRINT INSTRUCTION - 'WITH WRITTEN FEEDB3ACK ___

4 1.4 PRINT - REFERENCE MATERIAL ___

5 1.5 IMAGE PROJECTION - STILL

6 2.1 PASSIVE AUDIO___

7 2.2 ACTIVE AUDIO ___

8 3.1 AUDIOVISUAL - STILL WITH FEEDBACK___

9 3.2 AUDIOVISUAL - STILL WITHOUT FEEDBACK___

l0 3.3 AUDIOVISUAL - MOTION WITH FEEDBACK___

11 3&4 AUDIOVISUAL - MOTION WITHOUT FEEDBACK___

12 4.1 STATIC DISPLAY WITHOUT FEEDBACK___

* *.,13 4.2 DYNAMIC DISPLAY WITHOUT FEEDBACK ___

14 4.3 PHYSIOLOGICAL TRAINER - INTERNAL ___

is 4. PHYSIOLOGICAL TRAINER - AUDIO ___

16s. PHYSIOLOGICAL TRAINER - VISUAL ___

* 17 4.6 SYMBOLIC SIMULATION WITH FEEDBACK ___

18 4.7 TRAINER ___

19 48 SIMULATOR

20 4.9 OPERATIONAL EQUIPMENT ___

21 4.10 OPERATIONAL EQUIPMENT WITH FEEDBACK ___

22 5.1 VISUAL COMPUTER - STILL WITH FEEDBACK ___

23 5.2 VISUAL COMPUTER - STILL WITHOUT FEEDBACK ___

24 5.3 VISUAL COMPUTER - MOTION WITH FEEDBACK ___

25 5.4 VISUAL COMPUTER - MOTION WITHOUT FEEDBACK ___

26 5.5 AUDIOVISUAL COMPUTER - STILL WITH FEEDBACK ___

27 5.6 AUDIOVISUAL COMPUTER - STILL WITHOUT FEEDBACK ___

28 57 UIVSALCMUE OIO IHFEBC 29 5.7 AUDIOVISUAL COMPUTER - MOTION WITHOU FEEDBACK ___

30 6.1 INSTRUCTOR

* * 2-24
are provided in The ETES User's Guide. The criteria you

select should be documented in the Media Selection Program

planning worksheet (see Table 2-3).

Enter Data Into SDT: once the stimulus, response, and

feedback variables ind task criticality variables have been

selected, each task must be scored on each of these

* variables and the resulting data must be entered into the

SDT. Table 2-4 displays a worksheet which can be used to

assist users *in inputting this data into the SDT.

To enter data into the SDT, enter the INPUT mode of the SDT

and select the Task entity. The psychological variables and

the criticality variables are stored under the Task entity.

Additional assistance can be provided by successively

selecting four built-in input/output formats for data

entry: (1) Training Emphasis Ratings - this format contains

the seven criticality variables, (2) Stimuli (Parts 1 and 2)

- this format contains the stimulus variables, (3) Responses

* ~- this format contains the response variables, (4)eedback-

*this format contains the feedback variables. By selecting

these formats, you can avoid the process of selecting the

variables to be entered into the SDT. More details on

inputting data into the SDT are provided in Section 7 of the

SDT User's Guide.

Select Media Categories: After entering the task data into

the SDT, you must decide what potential media categories

will be considered during the assignment process. Table 2-5

* lists the media categories which are available in the Media

Selection Program. More detailed descriptions of each of

these categories is provided in Appendix B. To select a set

of media categories, start with the full set of categories

~-\-;-.~.~:..-;~2-25 *..
Jed -U1 go -SUOD

>M~led -ac go qoi --

;jjTa 6UTUleaq-

'-4 uIT 1j%

siaqwelq %

Aouanbaij

ast paea

adAl paea

w aoinoS paad

~~. ~ Pal paed

E4 ;STa *dsaU

U *ueqo *dsaH *ON

e~~ dsei- *pbeai

- 0

apoW *dsaU - - - - -- -

-ISTU -UIT~ - -- - - - - - 4.4

-ueqo -ON

9eui -said *IUV S

-said *U1TIS-----

*4uzi *mit4s *pnv--- --

.3 saoinos *pny 1-

.0 1. 1 - 1 

E-0 

ioadS 'STA------------------------ ------

*aAOW STA-

*UIIOA STA------

E-
d. 7. -71a

* Table 2-5. Media Categories.

CAT GROUP

MEDIA CATEGORY

1 1.1 PRINT INSTRUCTIONS W/O FEEDBACK

2 1.2 PRINT INSTRUCTIONS WITH ORAL FEEDBACK

3 1.3 PRINT INSTRUCTION -WITH WRITTEN FEEDBACK

4 1.4 PRINT - REFERENCE MATERIAL

5 1.5 IMAGE PROJECTION - STILL

6 2.1 PASSIVE AUDIO

7 2.2 ACTIVE AUDIO

8 3.1 AUDIOVISUAL - STILL WITH FEEDBACK

9 3.2 AUDIOVISUAL - STILL WITHOUT FEEDBACK

10 3.3 AUDIOVISUAL - MOTION WITH FEEDBACK

11 3.4 AUDIOVISUAL - MOTION WITHOUT FEEDBACK

12 4.1 STATIC DISPLAY WITHOUT FEEDBACKI

13 4.2 DYNAMIC DISPLAY WITHOUT FEEDBACK

14 43 PHYSIOLOGICAL TRAINER - INTERNAL

15 4.4 PHYSIOLOGICAL TRAINER - AUDIO

16 4.5 PHYSIOLOGICAL TRAINER - VISUAL

17 4.6 SYMBOLIC SIMULATION WITH FEEDBACK

18 4.7 TRAINER

19 4.8 SIMULATOR

20 4.9 OPERATIONAL EQUIPMENT

21 4.10 OPERATIONAL EQUIPMENT WITH FEEDBACK

22 5.1 VISUAL COMPUTER - STILL WITH FEEDBACK

23 5.2 VISUAL COMPUTER - STILL WITHOUT FEEDBACK

24 5.3 VISUAL COMPUTER - MOTION WITH FEEDBACK

25 5.4 VISUAL COMPUTER - MOTION WITHOUT FEEDBACK

28 5.5 AUDIOVISUAL COMPUTER - STILL WITH FEEDBACK

27 5.8 AUDIOVISUAL COMPUTER - STILL WITHOUT FEEDBACK

28 5.7, AUDIOVISUAL COMPUTER - MOTION WITH FEEDBACK

29 5.8 AUDIOVISUAL COMPUTER - MOTION WITHOUT FEEDBACK

30 6.1 INSTRUCTOR :

2-27
listed in Table 2-4 and eliminate any categories which (1)

are not available at the setting where the course associated

with tasks is likely to be taught, and (2) are clearly

inappropriate for the particular subject area covered by the

course (the latter criteria should be used with extreme

caution since you may unnecessarily eliminate potential

* . media).

Once the media categories have been selected, they should be

documented in the Media Selection Program Planning Worksheet

listed in Table 2-5.

Select Criticality and Utility Weights: You must decide

what weights will be used in aggregating: (1) the

criticality variables and (2) the components of utility

(that is, cost and efficiency, or cost and effectiveness).

* - It is recommended that you use equal weights for both of

these areas.

* . If you (1) suspect the validity of some of the variables or

(2) feel that some of the variables are less important to

your particular analysis, you may want to assign these

variables smaller weights. However, if you decide to do

this you must compensate by increasing the weight of some of

A the other variables since the sum of any set of weights must

be 100. The worksheet listed in Table 2-4 should be used to

document the criticality weights you develop.

*Select Objectives and Constraints: Table 2-6 displays the

objectives-and instruments that you can use to assign tasks

to media, the constraints that can be used to qualify each

objective and the type of assignments that will be produced

by each objective-constraint combination. Examine this

2-28
....... ......... C F V . - . - w . - w r

SAUa!O;3;3

f4TA A(TiT~n. x x ____ __ _ __ _ _ _

Is0o pup

ssauaATj~a;;3 x x x ______

pup AuOTOT;;a X x____ ___

s sauaiATq3a;;a x

1-4 0

0-w- ~ Ad ~ '.U 

* - 0 -4 ~ rn 0 - -0 -44 -4 . 0 (a c- 0U w c0 (a 0 to 4 0 m (a

to.J0 .. c 444 A ) m44 .1.JC .1.4 M - 1-J 41 r 40

>4 0- = o0 0 C 0C 0 0 o 000 0 0 0~.

0 1 4 1. .)..l. 4- a) A4j4. -1-. = -0 : -444j.

m4( > 4. ) >5C4

a 41- .1. .1.4 = -4 4. w 4 4 44 1 44'U (a 414, t

O 4 (a 414) Ad ) 4 4.) .CO AI.)4 .j &J

Q) .14.. (a' .41 a)6.04.. 4 444 4

44.4 a) 0U ti a)0C 44..'

a)u 0 U- 0 ) 00ON L 61 C 04 u i U w 0

020 m4 44 :3 0202 =2 024 :3 (a m1. ma m

a) V ) W2 V-D c 0) 4 V) VA wa) '0' -

a2.- 4-1 4-1a a) .1.4 a ) 4-) 41 4n aa4 a) a)

Um a) 4a)a -4 4) -1>4 00a a) -14> >.: 4) -) tp w > 4) cf> c w2 r_ 1 > 41.c4--I. c 1-a0 _ U) a 1--4 * 1-0 a- u- 0 1a O w c

.1 C>.-'.) 0 C--f -4 C)t ).. . Q)a -.4 4 W 4Ja Ca4 ) I (a -r4 ) ON4 a -4 a)0-4 a) a.4.4 U a) 1 .- 4 m ) a) a -4 u-- a)w W. .4.) U4I - 44 4-4 -4-. 0 A.4 *A4.44 A.) W -) 44 M- 02 U 0a )44 4. : .. J 020 02044 020 ,4 02

.1. 4-4 .0 .4.) *-a 4 - 4.4 4-Ia) 4-ia) 02 **5* -4 CC a) 164 to' (a -'u( 0 4 -4 (0 tW '4.1 0 -

a) -.4 0 . %6.41 -4- -4 .4.44 .. 4Ad44 .- 1A 4 -4.. (a)1. a44I 0>1

a 0- 0O V- VO 0 0V O 0 0 0 T 0 V

E 4A) E E- 1. 14 rI E (aI -2 -E 4U E ma) .. 41 0 > V ) 0 a V ) V 1 4 -- 41a .4 Ad.

a2 0 014) 0C 04 V 0 O> 0OC> 0Oaw V 0 0> 0 0> 0 00 4 A2i) .140 41 44 440 ) 4 0 44 0 w) .. Ai) L444Ai) 4. Ad--I

V) '-I V V02 2 -4 0 E 1-4 1-4 44

02r-V WV T mC 0 to Wu(AV wV Laa WV oV 02)4.0

W) Moa W ~ %fo ) x a ) W.4 44 0 a) ~ CV . C C

1C 02) .4j WN ) 02 C U)Na 20 02 0 C 0'a 2 2

(a LV 44 I o-4 -- 4 mUV4M 44 Ito-.4 -. to -4 -- 4 4 ' -- 'U4. 4-i

.. 4 4 Ia)-1 A.) a.w 44)0 44 E44 A4 44 4)F 'iA. 444 -L)V4.4 .6 140

02 V NO 4- .. N V -.. 4 .. 4 -4 . ) V0 a)..4 ..-- 4 a) 024 ) ..-4-.Xr.) -- 444 0) -4X X 0 'x -4 x> 0) - N r- -4 U -4 N

0m Da -4 CU -C r=) 0a) -40a aa MW 40W -4 C) r 4-4 ) -i4a)4 -- 0) (a M0 C-0 odto-4 > m to S0 0 4 to a-4 0-' (ar D o 0 0 ( 2> -. 4 XI 02) -- 4 w2 02 -4I .402 -.402 *- 0j

w %64 21 w4m40-4 0 t4-)4 IV2 1 44 U02 0V 02 4 x 1 4.4x 1 4, X

1-. 02 E 4 0 s4L 02 02 0 -4 s 0-.44-w 0 -. 4 ) 02 0'U- 00- 0'

02W 02 4V 0 ) a) 0) w wa --4 -4 2

02 C- >4 c02 m -4- >42 020 C0 C2. 02 C02 eu2 E0 f-.4 trc 0 - a m) 0 a) 44 A.) 0) >0 w 0 0 O a ) -. ) U) r 4 0 a)-40 m 1Q

-44

to 00to a)4)Q -.-(a 44 -. -. 4 to -. 44 -M' 4-4 --. A4-4 44-4-4 --. -4 A J -4 A j 4 C 41 A.) n 04j 2-0 .4 44. 044) 024 A.) to i w 4444 02-- U)-- fn--4 0200w' 00-4 020 I2.- 44 020t.UrA -I t2a ) 02D -4 -4 rn - -I n- 0 1

41 m4 " mU0 0()a0 1 a o4 t--i(a 4C (a) (a 00 Q) ) C4-j '- 0 to --i m U-1-i

ca] 00( 44 W-4 4 ) 4w--44 4 4 0Z ( -W 4j 4j a4)

44 02 W c4 w4 ( 0 C4 4t 44c 02cw =

ra Cj4w0 0 >44 0 >5 C 4

I A. 4 > 9>1 44 C 544j C

to -4 z( C A-4C ..

0 

4 .. 4 

jU a - 44) 1 0 0) -4 0 0 1-4 0 0 0 "-..v U1 0 ) -4 A4 0 -4 A.440).4 41

440 LO4 4- 4J 02 %64 4-i W

44-0 0 0 4-' 0 0 4-4 4.4 0

Jz 0) £ )30 )a

S - - _________- -

2-02
*~ ;soo

W aJ o; r) n PUL x

SSaUDAT0BO __

puv AuOTO;;3

ssaueAl~.;

P- -4~

0 0a = 0 0'

0 0. Gtri u 4.j 4~ 414

V > *I . J0

0. Uc' sw u

-W 4 =4 0)L m

EU 0 mU 'U

a) - w w)O * 64- )

GjW ~ 0)02a

w. 'U > -uc

4.' V 44 V -.40 - S 3 44 (D a)t)w 4) -4E

v 0 w0 00 0

r4 0 -4O 0 -

41 0) 41 1"O W -

0aJEU~ 0J 'UJ0

W EU a) EU 0) 44 44

0 2 0 N .4 Ota 0 N -4

'U 41 " a .0 *. U - "4)

4Ja) .''E ) .IU0

mo c 02"c 0 "0 0-0

>4 41

E-4u 0 C)02020

~ ~ 4. ~ 4.4 41 ' 0 4444 

14 ) U)

.. a)

I 1~ 1 9 -3
table, determine the types of assignments you would like to

produce, and select the appropriate combination of objective

and constraint. The combinations that are most likely to

produce meaningful and easy-to-interpret results are starred

in Table 2-6. It is recommended that first-time users

select one of the following combinations;

OBJECTIVE CONSTRAINT

Maximize Efficiency Minimize Cost

Maximize Effectiveness Minimize Cost

Maximize "Utility" No Constraint

You should document your selection in the Media Selection

Planning Worksheet (see Table 3-5).

Table 2-6 also lists the criteria that must be selected to

support each objective-constraint combination. Make sure

you have selected a set of criteria which can support the

type of assignment and objective-constraint combination you

desire.

Run Media Selection Program: At this point, you are ready

to run the program. The menus and frames for running the

Media Selection program are described in Section 6. Don't

forget to bring the completed Media Selection program

worksheet (Table 2-3) and task data worksheets (Table 2-4)

with you when you decide to run the program. Procedures for

evaluating the results of the Media Selection Program are

listed in Section 5.

2-31
V

a.

a. THIS PAGE INTENTIONALLY LEFT BLANK

a'

V

a.

~1

2-32
SECTION 3 -MENUS AND FRAMES

This section provides a detailed description of the menus

and frames in the Media Selection Program. Table 3-1

describes the individual menus and frames. An overview of

the logic among these menus and frames is presented in

Figure 3-1. A detailed description of this logic is

presented in Figure 3-2.

In the sections which follow, detailed descriptions of each

individual menu or frame are provided. These descriptions

list (a) the purpose of each menu, (b) the procedures for

A using it, (c) the frames which will follow the frame when

you have completed the action required in'the frame, and (d)

the frame where the you will go if you select the abort

option which is available in most frames. In addition, an

example of each frame is provided.

-C3-1
*a~ *.r~ ~

'a.

V *

V...

4~

BLANKPfi~

4~Rg~.'

'.~ -~

4$

a.. * 4;

V

3-41z a * .... .... . a *****,.* %.~ *a~.N~
INPUT DATA

ON PSYCHOLOGICAL

VARIABLES (SOT)

.2.

SELECT

CRITERIA

SELECT

TASKS

.. _

SELECTI

PSYCHOLOGICAL

VARIABLES i'I

SELECT/WEIGHT

TASK

CRITICALITY

FACTORS

COMPUTE1 SCORES OF

DISPLAY MEDIA ON

$CORESPSYCHOWEIGHT

CRITERIA

SELECT

OBJECTIVES/

CONSTRAINTS

SASSIGN DISPLAY RERUN

RELATIVE TASKS TO RESULTS ANALYSIS

COST OF MEDIA SMDMEDIA

CATEGORI1E -------

Figure 3-1 Overview of Media Selection Program

'I. 3-3
iS

0

* .e.

ge

43-

.
-'-C,

-4.,

.4.

~

'-4,

4-

4.

.1'* -~ * 4. THIS PAGE INTENTIONALLY LEFT BLANK

:k~i

-q 4.,

4.,

-4,

"'p..

I -'4

-41

3-5

- Vp %"..' .%V*. .~. VV%,h.,'.V.VCC,~4-*%C - ~ '~~' ~' ~ -~ ~:c-§~ ~:
'" 3.1 INTRODUCTORY FRAME (MS-i)

Description. This is the first frame you will see in the

Media Selection Program. The purpose of this frame is to

let you know that you have successfully entered the Media

Selection Program.

The frame is listed in Figure 3-3. You have the option of

either returning to the SDT or continuing with the Media

Selection Program. To select an option, move the cursor

(that is the highlighted area) to the desired option and hit

Return. (The cursor is moved by hitting the up or down

arrow key.)

Abort Options

o Exit to SDT-By selecting this option, you will return

to SDT Applications Program Select (Frame SDT-5)

Follow-on Frames

o Criteria Selection (MS-2)

43-6

', '.-, . -. . . . , , -..- , -,---,,. - --- .,-,- .. , . :. .? . :.< .: .. . - .. .<
'.a-',

' WELCOME TO MEDIA SELECTION PROGRAM

- Return to SDT

- Continue

-3-7

* 4

a'.

-S

-.'o

.p.

Figure 3-3. Introductory Frame (MS-i).

3-7
3.2 CRITERIA SELECTION (MS-2)

Description. This menu will allow you to select one of the

seven sets of analysis criteria used in the Media Selection

Program (see Figure 3-4). The seven criteria are (1)

efficiency, (2) effectiveness, (3) efficiency and cost, (4)

effectiveness and cost, (5) "utility" with efficiency, (6)

"utility" with effectiveness, and (7) cost. Guidelines for

selecting criteria are provided in Section 2.5.

To select a set of criteria, move the cursor to the desired

-option and hit Return.

Abort Options

O SDT Applications Program Select (SDT-5)

Follow-on Frames

o Task Selection (MS-3)

43-%

~3-8

4..
-. 07

Select criteria for analysis.

ABORT

Efficiency

Effectiveness

Efficiency and Cost

Effectiveness and Cost

"Utility" With Efficiency

"Utility" With Effectiveness

Cost

igure 3-4. Criteria Selection (MS-2).

3-9
-. J.

3. 3 TASK SELECTION (MS-3)

Description. This menu allows you to select the tasks to be

included in the analysis (see Figure 3-5). A maximum of

fifty tasks may be selected. The tasks selected should be

associated with a course module or other logical unit of

instruction.

To select tasks, move the cursor up and down on the screen

and press Return when it is on the item you want to select.

To select another task move the cursor to the desired item

and hit Return again. If all of the tasks do not fit on the

screen (this is indicated by the phrase "more" on the bottom

of the screen), you can view~ and select the remaining tasks

by "scrolling" the screen up (see Section 2.2). When you

have selected all of the tasks you desire, hit No More. You

may select all tasks by moving the cursor to this option and

hitting Return. The All Tasks option can be used to reverse

V the current status (highlighted or non-highlighted) of the

tasks selected. (Thus you can use this option to "unselect"

the selected items which are highlighted on the screen or to

select the items not currently highlighted).

3--

b:. ix * SJ"t--N_
,4..

7" Select tasks for analysis.

- ABORT

- All tasks

PERFORM PREFLIGHT INSPECTION OF XXX FIGHTER AIRCRAFT

TAXI XXX FIGHTER AIRCRAFT

PERFORM A SHOREBASED VFR TAKE OFF

PERFORM A SHOREBASED VFR LANDING

OPERATE XXX HEADS-UP DISPLAY

EXECUTE FORMATION MANEUVERS

COMPLETE POST-FLIGHT REPORT

Figure 3-5. Task Selection (MS-3).

3-11
' .. o ,,, . . , < . l.l.: . ., L o. . -. . .. - .. . 4 . .- - . -, -. - - . . -, °- , , , , .

17.77 7 7. . .7..-1

Abort Options

O SDT Applications Program Select (SDT-5)

N.

' . Follow-on Frames

o If you have selected the cost only option as your

analysis criteria, you will go Media Category Selection

(MS-13).

o If you select any of the other criteria, you will go to

Selection of Psychological Variables (MS-4).

3...,2

'p.

,.

3-12
-. 4.

-I...

1

THIS PAGE INTENTIONALLY LEFT BLANK

S.,

-V.

.4"

WV

J

3-13
3.4 SELECTION OF PSYCHOLOGICAL VARIABLES (MS-4)

Description. This menu allows you to select one of the

three categories of psychological variables (stimuli,

responses, and feedback). This is the first step in a twostep process for selecting psychological variables. In

subsequent menus you will be able to select specific

variables within these three categories. Guidelines for

selecting categories are provided in Section 2.5.

An example of this frame is provided in Figure 3-6. To

select a category, move the category to the desired option

and hit Return. If you have completed your selection of

psychological variables, move the cursor to the No More

option and hit Return.

The latter action will take you to the Data Review-i Frame

(MS-9) where you will have the option of selecting a

different set of tasks, or exiting to the SDT mode select

frame (SDT-4). (These options are designed to let you deal

with any missing data identified during the selection and

examination of psychological variables.)

Abort Options

o SDT Applications Program Select (SDT-5)

Follow-On Frames

O If you select the No More option, you will go to Data

Review-i (MS-9)

3-14

" ." . . . . .
Select Psychological Variables

- ABORT

- No More

.- STIMULI

- RESPONSES

- FEEDBACK

'4

-4

Figure 3-6. Selection of Psychological Variables (MS-4).

3-15
o If you select the Stimuli category, you will go to

Stimulus Variable Selection (MS-5)

o If you select the Response category, you will go to

Response Variable Selection (MS-6)

o If you select the Feedback category, you will go to

Feedback Variable Selection (MS-7).

3,16

4-..,-'
.'.

-.-

,,

--

• 4.
3.5 STIMULUS VARIABLE SELECTION (MS-5)

This menu allows you to select which of the twelve variables

which describe stimulus characteristics will be included in

the analysis (see Figure 3-7). Guidelines for selecting

psychological variables are provided in Section 2.5. Until

you gain experience with the program, it is recommended that

you select all of the stimulus variables.

To select stimulus variables, move the cursor up and down on

the screen and press Return when it is on the item you want

to select. To select another variable move the cursor to

the desired item and hit Return again. When you have

selected all of the variables you desire, hit No More. You

may select all variables by moving the cursor to this option

and hitting Return. The All Attributes option can be used

to reverse the current status (highlighted or non- highlighted) of all variables on the screen. Thus, you can use

this option to "unselect" the selected items which are

highlighted on the screen or to select the items not

currently highlighted.

Once you have selected all of the stimuli variables you

desire (this is indicated by hitting the No More Attributes

option) a message will appear on the right hand side of the

screen asking you if you want to view the scores of the

tasks you selected in Task Selection (MS-3) on the stimulus

variables you have just selected in this menu. If you hit

yes, you will go to View Data for Psychological Variables

(MS-8) where the scores of each task on the psychological

variables will be displayed.

3-lb
Psychological variable: Siui

Select Attributes:

- ABORT

- All Attributes

* MEDIUM

VISUAL FORM

VISUAL MOVEMENT

SVISUAL SPECTRUM

VISUAL SCALE

VISUAL CONTRAST

AUDIO SOURCES

AUDIO STIMULI INTENSITY

-~ STIMULI PRESENTATION

STIMULI PRESENTATION RATE

NUMBER OF CHANNELS

STIMULI DISTRIBUTION

Figure 3-7. Stimulus Variable Selection (MS-5).

3-19
If you hit no, the program will check for missing data for

each task and indicate where data is missing by displaying

the phrase Missing Data beside each stimulus variable. If

there is no data missing, no such message will be displayed.

As the program performs this check for each task, the name

of the task will be displayed at the top of the~ frame. Once

the program has checked for missing data you will go to

Select Psychological Variables (MS-4), where you will have

* the option of selecting another category of psychological

variables or continuing with the analysis.

Abort Options

0 SDT Applications Program Select (SDT-5)

Follow-on Frames

0 If you select the option for viewing data you will go

to the View Data for Psychological variables (MS-8).

0 If you do not select the option for viewing data you

will go back to Selection of Psychological Variables

(MS-4) where you can select another category of

psychological variables or go on to the next step in

the analysis.

3-20
Iwrr- rjrq-~rrrrr r r

 rjww,. . -~ c c - r rvw w-j r~u~rwtrrkrrr.rkw S 5 

rQvhr.: cx. T% : t~. -.. -

it.-.- - --- ' - - --- -~' -- - . - - - - - - - . - . . . . . - - - - - . -

LI

P

- 4..

-4 A

'P.,

'P

-- 'S .1'.

4.

'S..

%0

THIS PAGE INTENTIONALLY LEFT BLANK

'5-,

4- *4 4

.4

.44.

A,

P4

*4

a-.

P4

'Sb;

(a

.4

- 5

4*

4%~

3-21

* 2

~ - ' 4 ap-*..,. *.p- - - ~ . . . -- . - .~ .1 * %~ t\?t - ~.j ~ & . . - -

9 RkPkEa&tX42..aLa&.LaLk1Ikara2atat. aLa~a~PakaLaL~A.Afl .P%..mM
: 

L . . . - .  t . . . - -  . C . .  --  C o . .C - . . -. . . - -.

3.6 RESPONSE VARIABLE SELECTION (MS-6)

This menu allows you to select which of the six variables

describing response characteristics will be included in the

analysis (see Figure 3-8). Guidelines for selecting psychological variables are provided in Section 2.5. Until you

gain experience with the program, it is recommended that you

select all of the response variables.

To select a response variable, move the cursor up and down

on the screen and press Return when it is on the item you

want to select. To select another variable move the cursor

to the desired item and hit Return again. When you have

selected all of the variables you desire, hit No More. You

may select all variables by moving the cursor to this option

and hitting Return. The All Attributes option can be used

to reverse the current status (highlighted or non-highlighted) of the variables selected. Thus, you can use this

option to "unselect" the selected items which are highlighted on the screen or to select the items not currently

highlighted.

Once you have selected all of the response variables you

desire (this is indicated by hitting the No More Attributes

option) a message will appear on the right hand side of the

screen asking you if you want to view the scores of tasks

you selected in Task Selection (MS-3) on the response

variables you have just selected in this menu.

If you hit yes, you will go to View Data for Psychological

Variables (MS-8) where the scores of each task on the

psychological variables will be displayed.

3-22
Psychological variable: Responses.

Select attributes

- ABORT

- All Attributes

RESPONSE MODE

V. INTENSITY OF RESPONSE

RESPONSE IMPLEMENTATION

REQUIRED RESPONSE RATE

# RESPONSE CHANNELS

RESPONSE DISTRIBUTION

Fiue38.RsoseVraleSlcio M-)

.13-23
If you hit no, the program will still check for missing data

for each task and indicate where data is missing by

displaying the phrase Missing Data beside each response

variable. If there is no missing data no such message will

be displayed. As the program performs this check for each

task, the name of the task will be displayed at the top of

the frame.

Once the program has checked for missing data, you will go

to Select Psychological Variables (MS-4), where you will

have the option of selecting another category of

psychological variables or continuing with the analysis.

Abort Options

o SDT Applications Program Select (SDT-5)

Follow-On Frames

o If you select the option for viewing data you will go

to View Data for Psychological Variables (MS-8).

.,-.v o If you do not select the option for viewing data you

- will go back to Selection of Psychological Variables

(MS-4) where you can select another category of

psychological variables or go on to the next step in

the analysis.

* -.

3-24

1% "
~ -. .~ ~fl.. 

.r .r, - - - - - - - - -

--- 4

4.

Pd..

- P.

P.

-. 4'

S

.4..

* it,

it

.4

4%. A

.4. N THIS PAGE INTENTIONALLY LEFT BLANK ~4

4*

.me

4*.

4%,

4%

4-.

4*.4

-A

-4 4 -4-

44

V.

4~S'!

-p.

'-i.e

3-25

Sj

&VJ'4 * ~ ~ <S ~ ~ Z- ~& I I
* - a . - 3.-.. - a - .. - * . . . . . . . .- . --. 77

3.7 FEEDBACK VARIABLE SELECTION (MS-7)

This menu allows you to select which of the twelve variables

describing feedback characteristics will be included in the

analysis (see Figure 3-9). Guidelines for selecting psychological variables are provided in Section 2.5. Until you

have gained experience with the program, it is recommended

that you select all of the feedback variables.

To select a feedback variable, move the cursor up and down

on the screen and press Return when it is on the item you

want to select. To select another, move the cursor to the

desired item and hit Return again. When you have selected

all of the variables you desire, hit No More. You may select

all variables by moving the cursor to this option and

hitting Return. The All Attributes option can be used to

reverse the current status (highlighted or non-highlighted)

of the variables selected. Thus, you can use this option to

Munselect" the selected items which are highlighted on the

screen or to select the items not currently highlighted.

Once you have selected all of the feedback variables you

desire (this is indicated by hitting the No More Attributes

option) a message will appear on the right hand side of the

screen asking you if you want to view the scores of tasks

you selected in Task Selection (MS-3) on the feedback

variables you have just selected in this menu.

If you hit yes, you will go to View Data for Psychological

Variables (MS-8) where the scores of each task on the

- psychological variable will be displayed.

3-26
- Psychological Variable: Feedback.

Select Attributes

- ABORT

-__ALL__ATTRIBUTES_

FEEDBACK MEDIUM

FEEDBACK SOURCE

FEEDBACK TYPE

FEEDBACK DISTRIBUTION

Fiue39 edbc aibeSeeto M-)

3-2
If you hit no, the program will still check for missing data

for each task and indicate where data is missing by

8 'displaying the phrase "missing data" beside each feedback

variable. If there is no data missing, no such message will

be displayed. As it performs this check for each task, the

name of the task will be displayed at the top of the frame.

Once the program has checked for missing data, you will go

to Select Psychological Variables (MS-4), where you will

have the option of selecting another category of

psychological variable or continuing with the analysis.

Abort Options

0 SDT Applications Program Select (SDT-5)

Follow-On Frames

o If you select the option for viewing data you will go

to View Data for Psychological Variables (MS-8).

.p." .

O If you do not select the option for viewing data, you

will go to Select Psychological Variables (MS-4) where

you can select another category of psychological

variables or go on to the next step in the analysis.

3-2

~3-28
- 

r. ~ 

U..

-~ -~ - -- -. -. - - -- - - - . .. . ~ .~qj.

F..

-'S....

-'S

.4'

A

THIS PAGE INTENTIONALLY LEFT BLANK

a;

4.

4.

4.

.4

V

.4.

A

.4

p.'

.4,

3-2~

* !~CA~. ~ ~2. i
3.8 VIEW DATA FOR PSYCHOLOGICAL VARIABLES (MS-8)

If you have selected the option for viewing data, in the

- Stimulus Variable Selection (MS-SO, Response Variable

- Selection (MS-6), or Feedback Variable Selection (MS-7)

frames, the data for the variables you have selected will

appear on the screen to the right of the variables names.

An example of this frame is presented in Figure 3-10. The

data will begin appearing immediately after you hit the yes

option for viewing data in the Stimulus Variable Selection

(MS-S), Response Variable Selection (MS-6), or Feedback

Variable Selection (MS-7) Frame. Data for one task will

appear on the screen, then data for the next task will

appear on the screen, etc. until data for all the tasks you

have selected has been displayed. The title of each task

will be displayed at the top of the screen. To move to the

next task, you must hit Continue. If there is data missing

for any of the tasks, the term Missing Data will appear on

. *the screen beside each variable with data missing.

You can abort the viewing of data by hitting the Abort

option. If you select this option, the program will still

search for missing data for all of the selected tasks but

individual data elements for each jariable will not be

displayed on the screen.

After all the data has been displayed you will go back to

Select Psychological Variables (MS-4) where you will have

the option of selecting another type of psychological

variable, or continuing with the analysis.

3-30
Psychological Variable: Feedback for Task Remove and Replace

AN RM/Q15.

Select Attributes: SDT VALUE:

- ABORT

- ALL ATTRIBUTES

FEEDBACK MEDIUM VISUAL

FEEDBACK SOURCE INTRINSIC

FEEDBACK TYPE RESPONSE CORRECTNESS

FEEDBACK DISTRIBUTION INDIVIDUAL

o.

- Abort viewing these values

- Continue

Figure 3-10. View Data for Psy. Variables (MS-8).

3-31

I.e..
Abort Options

o If you select the Abort option, the program will search

for missing data and then return you to Select

Psychological Variable (MS-4)

Follow-On Frames

o Select Psychological Variables (MS-4)

3-3

"IV mI

3-32 '
THIS PAGE INTENTIONALLY LEFT BLANK

3-33
3.9 DATA REVIEW-I (MS-9)

This purpose of this frame is to allow you to deal with any

missing data for psychological variables you selected and

examined in Stimulus Variable Selection (MS-5), Response

Variable Selection (MS-6), Feedback Variable Selection (MS7) and View Data for Psychological Variables (MS-8).

The frame provides you with three options (1) continuing

with the analysis, (2) selecting a different set of tasks,

or (3) exiting the program to enter the missing data into

the SDT.

If you have data missing, you should either select a new set

of tasks for which you are sure there is data, or you should

exit and enter the SDT where you can enter data for the

tasks with data missing. Only if there is no missing data

should you select the continue option. If you continue, the

program will assign a value of zero to each psychological

variable with missing data.

Abort Options

o If you select the Abort option you will go back to the

SDT Applications Program Select (SDT-5). If you select

this option, all of your work from the current session

of the Media Selection Program will be destroyed.

o If you select the option for exiting to enter missing

data into the SDT, you will go back to the SDT

Introduction (SDT-l).

43

a3-34 ,.

,.;- . ~- . . , . . ,,, -.. , -.. .. , . . . , . . . . . . -. .4 . -. .. .:-. A
.'.%-

If you would like to pick a different set of tasks, please

select the appropriate option.

-ABORT

-Continue

-Different tasks

-Exit to enter missing data into SDT

%6'.

• ,.1~

-3-3

* q

~Figure 3-11. Data Review-i (MS-9).

,

%3-35

*.
.- L7

Follow-On Frames

o If you have selected the Continue option and you have

selected as your analysis criteria any of the options

which involve effectiveness (effectiveness, cost and

effectiveness or utility with effectiveness) you will

go to Criticality Weights (MS-10).

o If you have selected the continue option and have not

selected a criteria which involves effectiveness you

will go to Media Category Selection (MS-B).

o If you have selected the option for choosing different

tasks you will go to Task Selection (MS-3).

3-36

S.% %

.. -

: 4,- ,5- .: ; .;. --.,* -.. -. . .. ' --- , .2 .- " ' . -. .- .: .: . -: .. - .; - . 4 -' -- " ." -, . " - .: ." . " . -'.. ., - .. . -
p.. - p p -p.

... * .-. * . .. p p* ..... 4, . - . -

'p..

.4.

-4

~p4

U'

"4

p.

THIS PAGE INTENTIONALLy LEFT BLANK

1.

'4

.1'

"U,

is.'

-U

-U

'U

4-,

I

3-3-/

* - *' ?~.' .'x*.*'.~',V..*~.*I . . .
3.10 CRITICALLY WEIGHTS (MS-b0)

This frame allows you to enter weights for the eight

variables used to assess task criticality (task frequency,

percent members performing, percent time performing, task

* -. delay tolerance, consequences of inadequate performance,

task learning difficulty, probability of deficient

performance, and time between entry and performance).

The frame will begin by asking you whether you would like to

use equal weights or not. Unless you are an experienced

user, you should select the equal weights option. If you

select the equal weights option, the program will assign

equal weights to all eight variables and you will be asked

if you want to review the scores of the tasks on the eight

psychological variables.

If you select the option for inputting weights, spaces will

appear in the screen for entering weights for each variable

(see Figure 3-12). To enter a weight, use your keyboard to

K:-:type in the weights .You must input a number between 0 and

100. The program will automatically convert your input to

P-t fractional weights. For example, if you type in 50 it will

assign a weight of .50. If you make a mistake you can

correct it, by moving the cursor back and retyping the

information before you hit Return. You can eliminate a

criticality variable from the analysis by assigning it a

":2 value of 0.

The weights you assign must sum to 100. If you assign a set

of weights and they do not sum to 100, you will get an

error message and the program will ask you to enter the

weights again, starting with the first criticality variable

4
Weights for criticality variables must be between 0-100 and

sum to 100.

Please enter weight for

PER CENT PERFORMING

1 4_

THEN PRESS RETURN

(MAKE NO ENTRY TO ABORT)

Fiue31.Crtclt eihs(S1.

~~3-3c,
-Percent Performing. After you have entered the weights

the program will give you the option of continuing or

reentering the weights. if you hit continue a message will

appear on the screen asking if you would like to view the

criticality data for the tasks you have selected. If you

select the yes option for viewing criticality data you will

go to View Criticality Data_ (MS-il). If you select the no

option for viewing criticality data, and there is missing

criticality data you will go to Data Review - 2 (MS-12). If

you select the no option for viewing criticality and there

is no missing data, the program will calculate criticality

scores. This will be indicated by the message computing

.please wait. After the scores are calculated you will go

to Media Category Selection (MS-13)_

Abort Options

0 If you make no entry for any particular criticality

variable, you will go to the form for inputting data on

the next criticality variable.

0 If you select the option for aborting criticality

values which appears after all tLhe weights have been

entered, you will return to the form for inputting data

A for the first criticality variable.

Follow-on Frames

0 If you select the option for viewing the criticality

data, you will go to View Criticality Data (MS-4,

0 If you do not select the option for viewing criticality

data and there is missing criticality data, you will go

to Data Review-2 (MS-12)

3-40
o If you do not select the option for viewing criticality

data and there is no missing criticality data, the

program will calculate criticality scores and then you

will go to Media Category Selection (MS-13)

C.

V%

-D

-4.-4

.4
.- *?

3.11 VIEW CRITICALITY DATA (MS-11)

During this frame, you can view the criticality data for

each of the tasks you selected. Data will be displayed for

each criticality variable. An example of this frame is

provided in Figure 3-13. If there is data missing, the

message No Data will be displayed beside the variable with

data missing. When you have completed your review of the 4"

criticality data for the first task move the cursor to 44

Continue and hit Return. The data for the second task you

have selected will appear on the screen, followed by the

data for the next, etc., until the data for all tasks has

been displayed.

Abort Options

o If you abort viewing criticality values, the program

will compute criticality scores. If there is no

missing data, you will then go to Media Category

Selection (MS-13). If there is missing data, you will

go to Data Review 2 (MS-12)

Follow-on Frames

o If there is missing data, you will go to Data Review 2

(MS-12)

o If there is no missing data, the program will compute

criticality scores and you will then go to Media

Category Selection (MS-13)

3-4 2
* i4

REPLACE AN RM/Q15 - Criticality Values.

PER CENT PERFORMING 58

PER CENT TIME PERFORMING 3

CONSEQ. INADEQ. PERFORM. 3

TASK DELAY TOLERANCE 3

TASK LEARNING DIFFICULTY 3

PROB. OF DIFICIENT PERFORMANCE 3

TIME FROM ENTRY TO PRFRM 3

- Abort viewing criticality values

- Continue

Figure 3-13. View Criticality Data (MS-I1).

3-43
3.12 DATA REVIEW -2 (MS-12)

The purpose of this frame is to allow you to deal with any

missing data for the criticality variables.

The frame is displayed in Figure 3-14. The frame provides

you with three options: (1) continuing with the analysis,

(2) selecting a different set of tasks, or (3) exiting to

enter the missing data into the SDT.

If you have data missing you should either select a new set

of tasks for which you are sure there is data, or you should

exit and enter the SDT to input data for the tasks you have

selected.

Only if there is no missing data should you select the

continue option.

Abort Options

o Exit to enter missing data into SDT. By selecting this

option you will go back to the SDT Introduction Frame

(SDT-1). If you select this option all of your work

from the current session of the Media Selection Program

will be destroyed.

Follow-on Frames

o If you select the option for selecting different tasks

you will go to Task Selection (MS-3).

343-/44
There is criticality data missing. Please select the appropriate

option.

- Different Tasks

- Exit to enter missing data into SDT

43-45

°a

.',-

~Figure 3-14. Data Review-2 (MS-12).

N.,0

i 3-45

- .4-
3.13 MEDIA CATEGORY SELECTION (MS-13)

This frame allows you to select which of the twenty nine

media categories hard-coded in the program will be used in

the analysis. A listing of the media categories is provided

in Table 3-2. The frame for selecting media categories is

presented in Figure 3-15. Guidelines for selecting media

categories are provided in Section 2.5. Unless you are

familiar with the program, you should select all media

categories.

To select media categories, move the cursor up and down on

the screen and press Return when it is on the media category

you want to select. To select another category move the

cursor to the desired item and hit Return again. Since all

of the categories do not fit on the screen (this is

indicated by the phrase more on the bottom of the screen),

you can view and select the remaining categories by

"scrolling" the screen up (see Section 2.2). When you have

selected all of the categories you desire, hit No More. You

may select All categories by moving the cursor to this

option and hitting Return. The All Media Categories option

can be used to reverse the current status (highlighted or

non-highlighted) of the categories selected. Thus, you can

use this option to 'unselect" the selected items which are

highlighted on the screen or to select the items not

currently highlighted.

Abort Options

o SDT Applications Program Select (SDT-5)

3A'
Please select built-in media categories for analysis:

- ABORT

- All Media Categories

.Prnt Ins w/o Fd

Prnt Ins w/o Fd

Prnt In w/oral Fd

Prnt In w/writ Fd

Prnt - Ref Material

Image Proj - Still

Passive Audio

Active Audio

AV - Still w/Fd

AV - Still w/o Fd

AV - Motion w/o Fd

AV - Mot w/o Fd

Stat Disp w/o Fd

*" .5 Dyn Displ w/o Fd

Phys Train - inter

* Phys Train - audio

Phys Train - visual

- more -

.5...

Figure 3-15. Media Category Selection (M-13).

3-47
Table 3-2. Media Categories.

* CAT GROUP * ___MEDIA CATEGORY

1 1.1 PRINT INSTRUCTIONS WIO FEEDBACK

2 1.2 PRINT INSTRUCTIONS WITH ORAL FEEDBACK

3 1.3 PRINT INSTRUCTION - WITH WRITTEN FEED1BACK

4 1.4 PRINT -REFERENCE MATERIAL

5 1.5 IMAGE PROJECTION - STILL

8 21 PASSIVE AUDIO

7 2.2 ACTIVE AUDIO

8 3.1 AUDIOVISUAL - STILL WITH FEEDBACK 9.. UIVSA TILWTOTFEBC

9c 3.2 AUDIOVISUAL - STIL WITHU FEEDBACK

11 3.4 AUDIOVISUAL - MOTION WITHOU FEEDBACK

12 3.1 SADIOVISPAYOT WITHOUT FEEDBACK

13 4.2 DYAIC DISPLAY WITHOUT FEEDBACK

14 4.3 PHYSIOLOGICAL TRAINER - INTERNAL

15 4.4 PHYSIOLOGICAL TRAINER - AUDIO

16 4,5 PHYSIOLOGICAL TRAINER - VISUAL

17 4.6 SYMBOLIC SIMULATION WITH FEEDBACK

is1 4.7 TRAINER

19 48 SIMULATOR

20 4.9 OPERATIONAL EQUIPMENT

21 4.10 OPERATIONAL EQ3UIPMENT WITH FEEDBACK

22 & I VISUAL COMPUTER - STILL WITH FEEDBACK

23 5.2 VISUAL COMPUTER - STILL WITHOUT FEEDBACK

24 5.3 VISUAL COMPUTER - MOTION WITH FEEDBACK

25 5.4 VISUAL COMPUTER - MOTION WITHOUT FEEDBACK

26 5&a AUDIOVISUAL COMPUTER - STILL WITH FEEDBACK

27 5.6 AUDIOVISUAL COMPUTER - STILL WITHOUT FEEDBACK

28 5.7' AUDIOVISUAL COMPUTER - MOTION WITH FEEDBACK

29 !.8 AUDIOVISUAL COMPUTER - MOTION WITHOUT FEEDBACK

30 6.1 INSTRUCTOR

3-48
.. . . . . . --..

Follow-on Frames

o If you have selected cost only as your analysis

criterion the program now has all the information

.. needed to make task-media assignments. The program

will print out the assignments and you will go to

-7 Select Rerun Option (MS-23).

O If you did not select the cost only criterion option,

you will go to Computation of Efficiency/Effectiveness

(MS-14).

3-4'

,%9.

h'.,.

4.°..

"4.

-. °.

-.4 ',, v -...- ., - - - .... . - -. m v . , '"v . 2".-, -.-.- , . . . . . - , -"v ","v -, -" "-• " _ ' . - " ' . .
I.*1

-I

3.14 COMPUTATION OF EFFICIENCY/EFFECTIVENESS (MS-14)

.1

During this frame the message Computing-please wait will be

displayed on the screen (see Figure 3-16). This message

will remain on the screen until the program has calculated

all efficiency and effectiveness scores.

Abort Options

None.

Follow-on Frames

o Option for View Efficiency/Effectiveness Scores (MS-15)

3-5

'4

'S

• 'S~

4%%

S. ~~ ;: Z~2~ ; ~, ~>~z~ ~.'.0''~ - -.... 1
b'

SComputing. .. please wait

-. 4

;."

4%,~

-.,

4

a'

4

i Figure 3-16. Computation of Efficiency/Effectiveness (MS-14).

a-)-... v ''-'-" . . . - - ' \ .q ,,. , . ' ',,, -" " . - . .
3.15 OPTION FOR VIEWING EFFICIENCY/EFFECTIVENESS SCORES

(MS-15)

During this frame, you can select the option for examining a

histogram of efficiency scores on your computer screen (see

Figure 3-17).

U.

To select an option, move the cursor to the desired option

and hit Return.

Abort Options

o SDT Applications Program Select (SDT-5)

Follow-on Frames

o If you select the option for viewing graphs of

efficiency scores on your computer screen you will go

to Selection of Media Categories for Display (MS-16).

o If you do not select the option for viewing efficiency

scores and have not selected "utility" as an analysis

criterion, you will go to Print Criteria Scores (MS19).

o If you do not select the option for viewing efficiency

scores and you have selected "utility" as an analysis

criterion, you will go to "Utility" Weights (MS-18).

3-52

UV.- -
i' - . - . . . . - . -- ". -9 1; " ' - • .

Would you like to view bar graphs for efficiency scores?

- yes

- no

-5

'p

, %

~Figure 3-17. Option for Viewing Efficiency/

i'!i Effectiveness Scores (MS-i5).

3-53
3.16 SELECTION OF MEDIA CATEGORIES FOR DISPLAY

During this frame, you can select which of the media

categories will be included in the graphical displays of

efficiency scores on your computer screen (see Figure 3-

18). No more than ten media categories may be selected tor

display.

To select media categories, move the cursor up and down on

the screen and press Return when it is on the item you wane

to select. To select another category move the cursor

the desired item and hit Return again. When you h

selected all of the categories you desire, hit No More. I)

may select All Media Categories by moving the cursor to t

option and hitting Return. The All Media Categories option

can also be used to reverse the current status (highlighted

or non-highlighted) of the categories selected. Thus, you

can use this option to "unselect* the selected items which

are highlighted on the screen or to select the items not

currently highlighted.

Abort Options

0 SDT Applications Program Select (SDT-5)

Follow-on Frames

o Graphical Screen Display (MS-15)

d1%

3-54

I.- .
Please select media categories for bar graph.

-ABORT

________Ins ___w/o _____

Pint Ins w/ora Fd

Pint Ins w/writ Fd

d Figure 3-18. Selection of Media Categories For Display (MS-16).

3-55
3.17 GRAPHICAL SCREEN DISPLAY (MS-17)

During this frame, graphs displaying the efficiency of each

selected media category will be displayed on the computer

screen. An example of one of these graphs is displayed in

Figure 3-19. The efficiency scores displayed in this

graphic will range from a minimum score of 1 to a maximum

score of 100. After completing your examination of a graph,

hit Return and bar graph for next task will appear on the

screen, until graphs have been displayed for each of the

tasks you selected. You may abort out of the graph displays

by hitting Abort. After the last graph has been displayed,

the program will ask you if you want to choose a different

set of media categories. If you select this option you will

return to Selection of Media Categories, For Display (MS16). If you do not select this option you will go to

Utility Weights (MS-18) or Print Criteria (MS-19).

Abort Options

o If you selected a criteria involving effectiveness you

will go to Utility Weights (MS-18)

o If you did not select a criteria involving effectiveness you will go to Print Criteria (MS-19)

Follow-on Frames

o If you select the option for choosing new media

categories you go to Selection of Media Categories For

Display (MS-16).

3-56

*1l
Replace AN RM/Q15

Print Ins w/o fd

Print Ins w/oral fd

0 10 20 30 40 50 60 70 80 90 100p

Efficiency

-Abort

-Continue

Figure 3-19. Graphical Screen Display (MS-17).

3-5.j
o If you did not select the option for choosing new media

J..

categories and you selected a criteria involving

effectiveness you will go to Utility Weights (MS-18)

0 If you did not select the option for choosing new media

categories and you did not select a criteria involving

:-, -effectiveness you will go to Print Criteria (MS-19)

i'S.

3- L

a..._

b o ,-

a,,

"."

3-5Lb
-~1

1

.1 -~!1

.4

j

I

'S

I

'S

THIS PAGE INTENTIONALLY LEFT BLANK 'I

-a.

4,

S..

.4

.4

.4

4

4.

a..

4.

I

I

4,

4. I .4 Sq.-,.. * S. ~ ~'.' . ... s . , * a i ~ '~ 4 '~ ~SS..A.SPJ
3.18 "UTILITY" WEIGHTS (MS-18)

This frame allows you to input weights for the two criteria

- used in the construction of an overall "utility" measure (a

detailed description of the algorithm used to compute

"utility" is provided in Appendix A).

An example of the frame is presented in Figure 3-20. To

enter a weight for a criterion, type in an integer between 0

and 100. The program will automatically convert your input

to fractional weights. For example, if you type in 50 the

program will assign the variable a weight of .5. If you make

a mistake, you can correct it by moving the cursor back and

retyping the information. (This must be done before hitting

Return.) The weights must sum to 100. until you have

gained experience with the program, it is recommended that

you use equal weights of 50 for both criteria.

Once you have entered the weight for the first variable,

space will be provided for entering the weight for the

second variable. When you have entered the second weight,

the program will ask you if you want to stay with the

weights you have just entered or select a new set of

weights. If you select the option for choosing a new set of

* weights, the form for entering the first utility weight will

appear on the screen. If you do not select this option, you

will go to Print Criteria (MS-19).

Abort Options

0 If you select the Abort option which will appear after

* . all the weights have been entered you will go back to

the form for inputting weights on the first variable.

3-60
Weights for effectiveness and cost must be between 0-100 and

sum to 100.j..

Please enter weight for

Effectiveness

THEN PRESS RETURN

(MAKE NO ENTRY TO ABORT)

4. "

4..

I

Figure 3-20. Utility Weights (MS-18).

p

3-%
.~ - -

'-a

Follow-on Frames

o Print Criteria Scores (MS-19)

V.

.4.

a-.

.4.

'4

I,..

4.*~~'4

4.!~

-a'.

4~

.4..,

a..'

a.-

'-a

4%,'

-V

4,-

.4

.4

4%

3-b2

4~14

~4
2 * *~ ~ ~* . ~ - -.-- - .- .- ~*~* . . .

1.*-,

.1*~

4...4' .4

.4,

4,f

.4.

.4

'SV

.4

4-..

.4,-

9.

*4*~

THIS PAGE INTENTIONALLY LEFT BLANK '4

5%

.9..

5,

4~*

.4.

4.

5.

Vb

S..

4,

3-~; 3

.4

4

~~%*%~.2%.4 .*X*%..**-**.. ...................................... ~ ~'.* *,.. 4-* kA..~ 4- 4 * 4-. ,~* ~. ~ 3. 4- 4-
3.19 PRINT CRITERIA (MS-19)

A This frame will allow you to print one of the three

individual analysis criteria calculated by the program (see

Figure 3-21). The three options are (1) print efficiency,

(2) print effectiveness (3) print "utility."

-. .4 ~ To select an option, move the cursor to the desired opt ion

and hit Return. The scores will then be printed out for

that option. The Print Criteria frame will appear again

giving you the option of obtaining printed output for

another criterion, or leaving the print option (this is

indicated by hitting No More.) An example printout is

displayed in Table 3-3. Each row represents one task, and

*% ,each column represents a media category A key for inter-

.4 preting the media category numbers is displayed in Table 3-

4.

Once you either have selected the No More option indicating

that you are finished printing criteria scores or the

kContinue option indicating that you want to skip the

printing of scores, a message will appear on the screen

asking you if the results you obtained were satisfactory.

If you select the yes option, you will then go to the first

frame in the task assignment process Selection of Objective,

MS-20. If you select no, you will go to Criteria Selection

(MS-2), where you can start your analysis over again.

Abort Option

.40 None

3-64
B L ANK PAE

79k ~ ..... ..e~ .('A .Z. 1 9
0 -4 

..

N 

N

-- ,-

i

0

0

040

o D0 

44 0 0 

0

4

z 

.

0-0

'IIn

E

C. 4

"Sn

04 > 04

00 00P

cnz

.4, 0

3-66
Table 3-4. Media Categories.

CAT GROUP

-- * MEDIA CATEGORY

1 1.1 PRINT INSTRUCTIONS W/O FEEDBACK

2 1.2 PRINT INSTRUCTIONS WITH ORAL FEEDBACK

3 1.3 PRINT INSTRUCTION - WITH WRITTEN FEEDBACK

4 1.4 PRINT - REFERENCE MATERIAL

5 1.5 IMAGE PROJECTION - STILL

6 2.1 PASSIVE AUDIO

7 2.2 ACTIVE AUDIO

8 3.1 AUDIOVISUAL - STILL WITH FEEDBACK

9 3.2 AUDIOVISUAL - STILL WITHOUT FEEDBACK

a .13.3 AUDIOVISUAL - MOTION WITH FEEDBACK

11 3.4 AUDIOVISUAL - MOTION WITHOUT FEEDBACK

12 4.1 STATIC DISPLAY WITHOUT FEEDBACK

13 4.2 DYNAMIC DISPLAY WITHOUT FEEDBACK

14 4.3 PHYSIOLOGICAL TRAINER - INTERNAL

15 4.4 PHYSIOLOGICAL TRAINER - AUDIO

16 4.5 PHYSIOLOGICAL TRAINER - VISUAL

17 4.6 SYMBOLIC SIMULATION WITH FEEDBACK

18 4.7 TRAINER

19 4.8 SIMULATOR

20 4.9 OPERATIONAL EQUIPMENT

21 4.10 OPERATIONAL EQUIPMENT WITH FEEDBACK

22 5.1 VISUAL COMPUTER - STILL WITH FEEDBACK

23 5.2 VISUAL COMPUTER - STILL WITHOUT FEEDBACK

24 5.3 VISUAL COMPUTER - MOTION WITH FEEDBACK

25 6.4 VISUAL COMPUTER - MOTION WITHOUT FEEDBACK

26 5.5 AUDIOVISUAL COMPUTER - STILL WITH FEEDBACK

27 5.6 AUDIOVISUAL COMPUTER - STILL WITHOUT FEEDBACK

28 5.7 AUDIOVISUAL COMPUTER - MOTION WITH FEEDBACK

29 5.8 AUDIOVISUAL COMPUTER - MOTION WITHOUT FEEDBACK

30 6.1 INSTRUCTOR

3-67

-,.
.. 

.   .    . .  . . 7 7W._. W. . . 1._ .W7 07 _ W. W70 . , , .7:

Follow-on Frames

o If you hit yes, indicating that your results are satisfactory you will go to Selection of Objective (MS-20).

o If you hit no, indicating that your results are not

satisfactory, the program will ask you if you want to

abort to SDT Applications Program Select (SDT-5) or

select different criteria. If you choose different

criteria, you will go to Criteria Selection (MS-2). If

you select this option, all of your work from the

current session will be destroyed.

-I

.4,,

-o,

3-68

*1 ,

Up#
THIS PAGE INTENTIONALLY LEFT BLANK

3 -6 9

Will"... . . .. . ... .
3.20 SELECTION OF OBJECTIVE (MS-20)

This frame will allow you to select the objective which will

guide the task assignment process. There are four possible

objectives: (1) minimize cost, (2) maximize efficiency, (3)

maximize effectiveness, and (4) maximize "ucility." Table

3-5 provides guidance for selecting objectives. This table

describes the types of assignments that will result from

each combination of an objective and a constraint.

(Constraints will be selected in the next frame.) To select

an objective, examine the column labeled "assignment

strategy," pick the type of assignment that meets your needs, and select the objective that matches that assignment

strategy. Make sure that the objective you selected is

, congruent with the analysis criteria you selected. Some

objectives will not be possible with certain criteria. The

assignment strategies which are the most informative and

easiest to interpret are stated in Table 3-5.

To select an objective move the cursor to the desired

objective and hit Return.

Abort Options

o SDT Applications Program Select (SDT-5)

Follow-on Frames

o Selection of Constraint (MS-21).

3-70

*1
* Please select objective.

-ABORT

Minimize Cost

Maximize Efficiency

Maximize Effectiveness

Maximize Utility

3-7
Isoo '

SSaaT4a a

qSOD pue

* Isoo

pue ADuaTOr;3 ____

ssauaATjoaj; x

04. O 0 0 0 -0

k&4 0~ 00 00 0

.4. c Cd A.) w) 4 w9 0 

0'lG 04. 0UO 0) 0UC m 04. :4 0 0C 0. 0

4) W ... ) VV- V ) V ~ &j4 V4.) .4) * 4') aC u. =.C) a 4.41. 0.W44 4' U .C U 4

4J -WC Ad> C) O j m) C)-> C~ ..4 A.)> .6C C

-C~4 W 4) C)4 -4 W:4) WU rn4 MU44 U

4C -44 C 0 -4 *W .w 4 4. -4 M *-' aC M -C

4

4 .

44 En) w 4 0 41 >.4C a-: 4) ) w4JC ... w

C )U -4 04) -1 w -44 c 4) 4 )C a )4 >..0 2 C4 -4 -4 2 t- (- w C 4J a -. O 4 0 -.VC 4 .. ).4 4)C 4.0 >. .4C.M V 0. 0C OC> r* 0.. m OC> MC OC Vj 004 00> 00

4d U~ %w = .4M 4.) ) % .4 4)(UC )(0 C) 4 )) ). )

W- 2n a 4j w W 20 ~ 44 f - . U m -u w 4.u'w

0~~~4 m 4(U . m m0mt4 mU -'. (U 4.

.4. C).. 2(a .4. I4. C) 0j 4)44 4 4 )- a) )V4. ~ . 4)V 0- r_-4 4

C -40 09 'IC 0(C .-C - 0 0 0 .- U ) -40 Ci Q44) 4JC C) 0 ij 0)4 0 > 0U4 10 0 C > 02a0> 02 c 0 0204> 0 0 > 0 0

(UW r- C) Vj (U W C) l Cw ) - . 0*C 4 U - 4 ) 4 2Cd ) 2V4

W C-- a~ VW c*- 0 W CU) V) U)- V U)! CU)! CU) ' C) m- 0 V C) c0 ' .( w. C)) 9 V )C x C)C) W . C4 - C4 e 0)~4. > -4 NC 1D>L -- ' 4) 0 NC w w CC N 0 0 2

% )04 WWI 4- - W4 O xC > u~> x~C ~> x)> wW 49 N U ~ 9 9NCU

Ea > 00 _ -460 C)f 6 ) c 00 r4m . ) 0 m w 00 *4 -4) ~4 -4 ( . 4

WWO W.4 4 W Ux ) -4. W to) wO- W-C W-4 n -4 W .Iw

g ui 4 0 rC) 0 0 0 E 00 (U )( -4.4. ( k, -U QU) ) to ) 0 m L( (a w( 0 (a.

(1) U(U '.4 '4C to. '4' '.! 4. m4. U 4) E 4) E)M

E- C ~C j4 0 AC)) r C) WC 4C) rz4. Aid( 4z)4 M: M Cw1.) 41 C) 41 Q)

4) W) 4)) W-4 )

Ce We )r C )0 eX r C W

(a' C)4 .4C -. 

4 .4.1 Ad (n - 3 M4. 4V 41 C >9 nw w 4) W Aj C)A U )- h

fn4)c cau U) I4 C) w u ) 4 W C)wu 4-4 w- 4 W -

cu -M 0 M ) to - C m. a o4) f E Ca 1 4) - o m-4 m ( -

U LW44Z 4 ) 6, 4 n 4 = U) w M n4- 41 W 1a

0 0 44t - W& 0 c. C4 0wCw 0 a. tr . , 0 '4-' c 0 m w

U) C)

C)C C) 0> >

... .. ' Ai 4) . -44 

-44 4 C) .. .. 1

'4 . A4. X4). L 0 04. 4 U'44 0 (4U! 41 0 IC CC 2:U

3-7
tR/ -Aiin -

1Pue ADuOTOT;;3 _

-4 -4 ~ -4

0. z~ 0 0 m

0 0 0 tr

x0~ C to c0

0 0

m U WC W'UJ

02 4 00 m- M D* pE

0' .,j 0 -4 '20 A 0U

0 to 0..t 4v 'a 4.e

4.3w, wo 4) M -0)2 0 03 v >0

F* 0 i4 0 . .0. 0

A) w *0 0 w 0 4) 0 4

'a , 4 '00 '0 4

-w c)1' ww -4.4J

0 *..4 '4 1

04 0 N4 -4 0~

tr U%2 4)0 al0

.-4 -40' 0V w

Em 0 0 c 0 0 c 0 0

U) la 4 0 Ai 4 0-4

-S.. * ) 0 -4~ W w

0a U) 0 0 0' ' 0

00 t2

* 0

0 £s U

49 3-73
*~~~~~ . .~ P. 7%b -r%- - r ~ .- r r~- ,, *.." , , * -' . . . .'," " ' - - .- . . .. e . .-. r . - . -. ...- -" . .. . "-' - . 4.

3.21 SELECTION OF CONSTRAINT (MS-21)

This frame allows you to select a constraint to guide the

task assignment process. To select a constraint, review

the assignment strategies listed in Table 3-5, and select a

constraint which is appropriate for the assignment strategy

you selected during the selection of an objective in the

previous frame. After you select a constraint, the program

will automatically print two columns listing the scores on

the objective and the constraint variables. This

information will be used in the next frame to select a

constraint level. (See Constraint Level, MS-21.) If you do

not select a constraint, that is you select the No

Constraint option, the program will automatically print the

task-media assignments and you will go to Select Rerun

Option (MS-23).

An example of this frame is provided in Figure 3-23. To

select a constraint, move the cursor to the desired item and

hit Return.

Abort Options

.4'

o SDT Applications Program Select (SDT-5)

Follow-on Frames

o If you select the No Constraint option you will go to

Select Rerun Option (MS-23).

o If you select a constraint, you will go to Constraint

Level (MS-22).

J.

3-74.

.'-,,.-': .. . ----. 4 . . . . ... . ..--.. ..- ... .-...-..-. ,.-.. ...- .-.- ..... -,...- ... .-.......--. -. .... ... . . . . .. - .. -' -C= ' . "- " :- . ," "" -,' ". -' ; ; ', - - . ,- - = : _. - ,' '".'k _" " " " -"- .. .
Please select constraint

-ABORT

Restart Selection

Minimize Cost

Maximize Effectiveness

No Constraint

Fiue32.CntantSlcin(S2)

... .3-75
3.22 CONSTRAINT LEVEL (MS-22)

During this frame, you will be required to enter the minimum

possible total score on the constraint variable that you are

willing to accept (or maximum possible if you have selected

minimize cost as you constraint). To select the appropriate

level, you must examine the printout of scores on the

objective and constraint values that was printed after your

selected a constraint in the previous frame (Selection of

Constraint MS-21). An example of one of these printouts is

presented in Table 3-6. The printout will list the best

objective value obtainable for each of a range of constraint

levels. Associated with each of the constraint levels

listed is an assignment set. An assignment set is a

specific assignment of each task to exactly one media

category. The score of the objective variable for the

assignment set is determined by aggregating the scores of

the individual tasks on that variable. Similarly, the score

of the constraint variable for the assignment set is

determined by aggregating the scores on the individual tasks

for the constraint variable. The constraint levels for the

assignment sets will be listed in ascending order with the

A assignment sets with the lowest constraint levels listed

first.

To select a constraint level, you must first determine how

% A low of a score in the constraint variable, you are willing

to tolerate. (If you have selected to minimize cost as your

constraint you must determine how high of a score you are

* -willing to tolerate.) Table 3-7 provides guidelines to

assist you in making this decision. If you are willing to

accept a low score, select a score that is about 25% of the

way between the minimum and maximum of the constraint scores

N 3-76
Please enter constraint level:

Numbers must be between 10 and 500

''1

THEN PRESS RETURN

(MAKE NO ENTRY TO ABORT)

V.

Figure 3-24. Constraint Level (MS-22).

3-77

* S.-.%.--. ..
Table 3-6. Example of Printout of *

Constraint and Objective Variable Scores

OBJECTIVE: MAXIMIZE EFFICIENCY

CONSTRAINT: MINIMIZE COSTI

CONSTRAINT LEVEL OBJECTIVE VALUE

50 500

60 500

70 550

80 550

90 550

100 670

110 670

-~120 670

130 700

140 700

150 700

3-78
C C

> U

0) (a U

c~ 0

CC

U E-4

UE

(D LA2 E2 tn Ln Z

dxdP X U)

0 CJ E -*E2

fE-4

u .0

r. 0 0 U)

E-i E04

r: 0

E-4I
mi 17 .4 R.7

(the minimum and maximum will be listed at the top of the

f rame.) f you are willing to accept a moderate score on

the constraint variable select a score that is about half

way between these two scores. If you are willing to accept

a high score, select a score that is 75% of the way between

the minimum and maximum. If you have selected Minimize Cost

as constraint this logic would be reversed (see Table 3-7).

When you are selecting a specific constraint level, you

should examine the printout to select a constraint score

- which is associated with a significant jump in the score for

the objective value which is close to your targeted level

25%, 50%, 75%.

To enter the constraint level, simply type it the number you

have selected in the spaces provided. You must input an

integer which is between the minimum and maximum constraint

level.

Abort Options

0 -Select Rerun Option (MS-23)

Follow-On Frames

0 Select Rerun Option (MS-23)

.43-80

.. ......
- ~v'~-~* ~ rr; ~ r-r* T r.-.

A.

J

'p

C..

44

THIS PAGE INTENTIONALLY LEFT BLANK

-p.4'

-p

S...'.

J*S

-U

4*

* 4

* '-4,.

-'p

* 4.

-'p

'.4

-. 4

U. 4,

3-81
3.23 SELECT RETURN OPTION (MS-23)

After you have selected the constraint level, the task-media

assignment will automatically be printed. When this

printing is complete, the Select Return Option (MS-23) will

appear on the screen. (See Figure 3-25.) This frame will

allow you the option of (1) exiting to the SDT where you can

terminate your session in the computer or use the SDT data

base management system, (2) selecting different criteria, or

(3) selecting different objectives.

To select an option, move the cursor to the desired item and

hit Return. An example of the Task-Media Assignment listing

is provided in Table 3-8.

Abort Options

By selecting the Abort option, you can exit to the SDT

Applications Program Select (SDT-5). If you select this

option all of your work from the current session of the

Media Selection Program will be destroyed.

Follow-On Frames

o If you select the option for choosing different

criteria you will go to Criteria Selection (MS-2). If

you select this option all of your work from the

current session of the Media Selection Program will be

destroyed.

o If you select the option for choosing a different

objective you will go to Selection of Objectives (MS20).

3-62

I.o
N. -. ; . .. N . . . .

Select rerun option:

- Abort

- Select different criteria

-Select different objectives

-3-8
.44

0

4JJ

01 0 -

ro -1

U) (J 40 W4

z

( a r z E4

.44 E- 0- u

01'

14 0 1

4-) 0

W4 N 00

2-4 >Z

E- ~ E0

0 0 4

N-4 N4 a

a4 a4

.p a) c

.52.
4 SECTION 4 EXAMPLE APPLICATION

This section presents an example application of the Media

Selection Program. The application is designed to provide

first-time users with hands-on experience in using the

program. The example will allow you to assign a set of

example tasks to media in manner that will maximize training

efficiency.

• .. On the pages which follow each specific action which you

must take to run through the example is described. More

specifically, the frames which you will encounter during the

example application are listed in sequence and the specific

actions which you must take during these frames are

described.

,jI. The first frame you will encounter during the example is the

Introductory Frame (MS-l) to the Media Selection Program.

To get to this frame you must load the SDT software, enter

the Applications mode of the SDT and select the Media

Selection Program. Procedures for performing these actions

are described in Section 2.4.

4.1 ACTION 1: INTRODUCTION FRAME (MS-i)

After you select the Media Selection Program option in the

SDT applications mode, this frame will appear on the screen.

Move the cursor (or highlighted area) to the Continue with

Program option and hit the Return key. (The cursor is moved

by hitting the up or down arrow keys.)

4-1
WELCOME TO MEDIA SELECTION PROGRAM

- Return to SDT

- Continue

q 4

.. .i

Figure 4-1. Introductory Frame (MS-i).

k

a . -. ..-. -a
4.2 ACTION 2: CRITERIA SELECTION (MS-2)

This frame will allow you to select the criteria that will

be used in the analysis. Move the cursor to the Efficiency

option and hit Return.

4.3 ACTION 3: TASK SELECTION (MS-3)

This frame will allow you to select the tasks used in the

analysis. Move the cursor to the All tasks option and hit

Return. Then move the cursor to No More and hit Return.

4.4 ACTION 4: SELECTION OF PSYCHOLOGICAL VARIABLES MS-4)

This frame will allow you to select one of the three

categories of psychological variables (stimuli, responses,

and feedbacks). In subsequent frames, you will select

specific variables within each of these three categories.

(Efficiency is determined by calculating the match between

tasks and media on these psychological variables).

Move the cursor to Stimuli and hit Return.

4.5 ACTION 5: STIMULUS VARIABLE SELECTION (MS-5)

This frame will allow you to select the specific stimulus

variables to be included in the analyses. Move the cursor

to All Attributes and hit Return. Then move the cursor to

No More and hit Return.

4-3

,- -.- " ". .-
. .. .

• ,. °-?

Select criteria for analysis.

- ABORT

Efficiency

Effectiveness

Efficiency and Cost

Effectiveness and Cost

"Utility" With Efficiency

"Utility" With Effectiveness

Cost

4%J

.-

Figure 4-2. Criteria Selection (MS-2).

4-4

*.,4 .,% :, ,, . ,. ," -- •. .. :- . ... . . . . . .-. ... ..... . . .- .. .
Select tasks for analysis.

- ABORT

- All tasks

PERFORM PREFLIGHT INSPECTION OF XXX FIGHTER AIRCRAFT

TAXI XXX FIGHTER AIRCRAFT

PERFORM A SHOREBASED VFR TAKE OFF

PERFORM A SHOREBASED VFR LANDING

OPERATE XXX HEADS-UP DISPLAY

EXECUTE FORMATION MANEUVERS

COMPLETE POST-FLIGHT REPORT

4,

4-

.4.

-4;

g.

4',

4-5

% °, - .4 -, . ,.., <,,. ,, . , . ;.... .: , . . , . _.,,, , , ... ,2. . .. ,.. . . ,
Select Psychological Variables

- ABORT

- No More

- STIMULI

- RESPONSES

- FEEDBACK

Figure 4-4. Selection of Psychological Variables (MS-4).

4-6

.5/,.
Psychological variable: Stimuli.

Select Attributes:

- ABORT

- All Attributes

MEDIUM

VISUAL FORM

VISUAL MOVEMENT

VISUAL SPECTRUM

VISUAL SCALE

VISUAL CONTRAST

AUDIO SOURCES

AUDIO STIMULI INTENSITY

STIMULI PRESENTATION

STIMULI PRESENTATION RATE

NUMBER OF CHANNELS

STIMULI DISTRIBUTION

Figure 4-5. Stimulus Variable Selection (MS-5).

4-7
After you hit Return, a message will appear on the screen

asking you if you would like to view the psychological

variable data. Move the cursor to the No option and hit

Return. The program will check for missing data. As it

does so the name of each task in the example will appear at

the top of the screen. (There is no data missing in this

example.)

4.6 ACTION 6: SELECTION OF PSYCHOLOGICAL VARIABLES (MS-4)

You will return to the frame asking you to select another

category of psychological variables. Move the cursor to

Responses and hit Return.

4.7 ACTION 7: SELECTION OF RESPONSE VARIABLES (MS-6)

This frame will allow you to select response variables.

Move the cursor to the All Attributes option and hit

Return. Then move the cursor to No More and hit Return. A

message will then appear in the screen asking you if you

would like to view the response variable data. Move the

cursor to No and hit Return. The program will check for

missing data. As it does so the name of each task in the

example will appear at the top of the screen. (There is no

data missing in this example.)

4.8 ACTION 8: SELECTION OF PSYCHOLOGICAL VARIABLES (MS-6)

You will return to the frame asking you to select a category

of psychological variables. Move the cursor to Feedback and

hit Return.
.* Select Psychological Variables

ABORT

No More

Stimuli

Responses

Feedback

4-9

U.

.1-

Figure 4-6. Selection of Psychological Variables (MS-4)

aJ

.

, . .,o , -,,, . . *. %* .. ,. . ... . .,. .. ,. ,.v .. , .- - -. ,.-.", .*d. .--..-. '- . -. .- -. "-....

' /- ",',. ,,'"," ,, '. ,..-" , '- ,.' ' -', . s ,t, '. _', , '..., ..-- -- -' ' .' ." "-*"
Psychological variable: Responses.

Select attributes

- ZBORT

- All Attributes

RESPONSE MODE

INTENSITY OF RESPONSE

RESPONSE IMPLEMENTATION

REQUIRED RESPONSE RATE

# RESPONSE CHANNELS

RESPONSE DISTRIBUTION

Figure 4-7. Response Variable Selection (MS-6).

4-!.
Select Psychological Variables

- ABORT

- No More

- Stimuli

- Responses

- Feedback

Figure 4-8. Selection of Psychological Variables (MS-4).

4.

. 4i
4.9 ACTION 9: SELECTION OF FEEDBACK VARIABLES (MS-7)

This frame will allow you to select Feedback variables. Move

the cursor to the All Attributes option and hit Return.

Then move the curosr to No More and hit Return. A message

will then appear on the screen asking you if you would like

to view the feedback variable data. Move the cursor to No

and hit Return. (There is no data missing in this example.)

4.10 ACTION 10: DATA REVIEW I (MS-8)

This frame is designed to allow you to deal with any missing

data identified during the previous frame. The frame will

give you the option of selecting a new set of tasks or

* exiting to enter data into the SDT.

Since you have no data missing, move the cursor to Continue

and hit Return.

4.11 ACTION 11: MEDIA CATEGORY SELECTION (MS-13)

This frame allows you to select the media categories to be

used in the analysis. Move the cursor to Select all media

categories and hit Return. Then move the cursor to No More

and hit Return.

4.12 ACTION 12: COMPUTATION OF EFFICIENCY SCORES (MS-14)

During this frame, a message will appear on the screen,

indicating that the program is calculating efficiency. You

do not have to do anything during this frame. When the

scores have been computed, you will go to the next frame.

4-12
Psychological Variable: Feedback.

Select Attributes

-ABORT

-ALL ATTRIBUTES

FEEDBACK MEDIUM

FEEDBACK SOURCE

FEEDBACK TYPE

FEEDBACK DISTRIBUTION

Figure 4-9. Feedback Variable Selection (MS-7).

4-13
7. 7. 7. r7 . -

You h-ave just reviewed psychological variables. Please select

the appropriate option.

-ABORT

-Continue

-Different tasks

-Exit to enter missing data into SDT.

.9 Figure 4-10. Data Review-1 (MS-9).

4-14
_JPlease select built-in media categories for analysis:

- ABORT

- All Media Categories

-SPrnt Ins w/o Fd

Prnt In w/oral Fd

Prnt In w/oral Fd

" Prnt In w/writ Fd

Prnt - Ref Material

Image Proj - Still

Passive Audio

Active Audio

-AV.- Still w/Fd

AV - Still w/o Fd

AV - Motion w/o Fd

AV - Mot w/o Fd

Stat Disp w/o Fd

4-' Dyn Displ w/o Fd

Phys Train - inter

Phys Train - audio

Phys Train - visual

- more -

4..

A Figure 4-11. Media Category Selection (M-13).

4-15

.*/ 4. * - ' " -" ," , -'' , ." ' ' - - . . . , . . . ... . . . ... . .... . . .-. . . .- . ... .. . , . - , . .
.- -J- ,

Computing. . . please wait

N'..,

-..

Figure 4-12. Computation of Efficiency/Effectiveness (MS-14).

4-16

* **.* **: * 4 -* . ..
4.13 ACTION 13: SELECTION OPTION FOR VIEWING EFFICIENCY/

EFFECTIVENESS/"UTILITY" SCORES (MS-15)

This frame provides you with the option of viewing

efficiency scores.

Move the cursor to No and hit Return.

4.14 ACTION 14: PRINT CRITERIA SCORES (MS-19)

This frame will allow you to print out the efficiency

scores. Move the cursor to the Efficiency option and hit

Return. The efficiency score for each media - task

combination will then be printed on your printer.

4.15 ACTION 15: PRINT CRITERIA SCORES (MS-19)

Move the cursor to No More and hit Return. A message will

then appear on the screen asking you if the results are

satisfactory. Move the cursor to yes and hit Return.

At this point, the program will automatically print out the

task-media assignments which maximize efficiency. In so

doing it will assign each task to the media which produces

the highest efficiency score.

After the results being printed the Rerun Option (MS-23)

frame will appear on the screen giving you the option of

selecting a different set of criteria and starting your

analysis over again.

4-17
- . - - .~~~~~~~ w . - . .~ .**~ .....w . .- . .* .* .

Would you like to view bar graphs for efficiency scores.)

-Yes

r -No

Fiue41.OpinfrVein fiiny

EfetvnesSoes(S1)
Select criteria to be printed.

-' -Continue

-Efficiency

4.-1

.4v
.J.

Select rerun option:

-ABORT-

-Select different criteria

-Select different objectives

'~

Figure 4-15. Select Rerun Option (MS-23).

4-20
SECTION 5 -EVALUATING AND USING RESULTS

Av overview of the process for evaluating and using the

results of the Media Selection Program is presented in

Figure 5-1. There are four steps in this process. In the

f irst step, sensitivity analyses are conducted. In the

second step, specific types of media within each media

category are identified for each task - media category

assignment. In the third step, the media assignments are

reviewed with respect to a set of practical factors which

may restrict media assignments. In the fourth step, the

final set of media assignments are documented in the SDT.

More details on each of these four steps are provided in the

sections which follow.

5.1 STEP 1 CONDUCT SENSITIVITY ANALYSES

A~s the first step in analyzing your results, you may want

to see how robust your analyses are to changes in key

J analyses alternatives and parameters. The more consistent

J your result are over a range of reasonable alternatives and

parameter values, the more likely it is that your results

are correct. One of the advantages of the automated Media

Selection Program is that it allows you to quickly determine

the impact of these sensitivity analyses. Table 5-1

displays the different alternatives that may be explored in

4<a sensitivity analysis. The three most likely alternatives

you will want to explore are: (1) alternative criteria, (2)

alternative objectives, and (3) alternative constraints.

5-1
. U .. - -

,- .-

STEP 1:

CONDUCT SENSITIVITY

ANALYSIS

.STEP 2:

IDENTIFY

SPECIFIC MEDIA

WITHIN

CATEGORI ES

.STEP 3:.

ASSESS IMPACTS

OF PRACTICAL

FACTORS

STEP 4:

ENTER

DATA INTO

SDT

Figure 5-1 Guidelines for Evaluating and Using Results

5-2
. . ............... . . . . . . . ... 

Np

Table 5-1. Alternatives for Sensitivity Analyses

ip

0 CRITERIA

0 PSYCHOLOGICAL VARIABLES

0 MEDIA CATEGORIES

* CRITICALITY VARIABLES

* OBJECTIVES

• CONSTRAINTS

0 CRITICALITY WEIGHTS

0 UTILITY WEIGHTS

5-3 '

4~
[ii~~~- -. --.- -07 7 -_1 -77--T 

*In planning new analyses, make sure to use the Media

Selection Program planning worksheet listed in Table 2-3 to

* document your analyses.

5.2 STEP 2: IDENTIFY SPECIFIC MEDIA WITHIN CATEGORIES

The Media Selection Program will identify the generic

category of media which should be used to train a task. To

develop more detailed media requirements, specific media

within each of these categories must be identified. Table 5-

2 displays some examples of specific media.

Media may be divided into two general groups (1) CourseRelated Media - those that are developed specifically for a

particular course and (2) Course- Indepe ndent Media those

that are supplied by the local school and used in wide range

of courses. Table 5-3 displays a worksheet which can be

used to describe the specific media that is used in number

of different courses. Relevant media categories within each

4..of these two groupings are identified in Table 5-2. To

identify specific media for Course- Indepe ndent Media, you

must examine the specific media that is likely to be

available within each m-dia category at the school where.-the

S course will be taught. To select specific media for- Course-

:4:Related Media, you must identify the most comparable

existing task that was taught via the same media category

and identify the specific media that was used to train that

task. (Procedures for identifying comparable tasks are

described in the ETES User Guide.) Specific media

identified for the Simulation-Three Dimensional Media

Category should be reviewed with PM TRADE since this

organization is likely to have the most up-to-date listing

of media in this category.
2'7. . .w

0 0 0 

J. 0 C 

- ca

Is A C

x 2 0 C)

UU 2 b6

IICL 4 l: .3

x r o .2 C>0

E .. I c 

C Z .2 .2 - .2- 

M6~ ~ ~ 3: .C - - .

X . 0 xVIW

LL a

4U Uc W

c C~ C.

Nb~ en 0 e 0I N

3Ybh.~-s ri

C.)

ccI *2M Iic

U.a

-U- -5 ,
L

U. -S

LU w

-IA C O

5 - E E

cr ?z 0. "0 I 0 xUL Z I"

.2 '.2

- C

C4 N- N- N

a.-
-4

4

U) 0)

U)

En

4: 4

41-

CU

Q))

.0

00

-r E-4

0 - 44

41 In

.'J

.00

0 g

jr5-7

C'M
5.3 STEP 3: ASSESS IMPACTS OF PRACTICAL FACTORS

Before making the final assignment of tasks to media, you

should review these potential assignments against practical

factors which may limit or in some cases prohibit making the

task-media assignments produced by the Media Selection

Program. More specifically, each task-media assignment must

be reviewed in terms of the practical factors listed in

Table 5-3. After reviewing the impacts of these factors,

the final assignments of tasks to media can be made.

5.4 ENTER DATA INTO SDT

Once the final task-media assignments have been made, these

assignments must be entered into the SDT data base

management system. To do this you must enter the Input mode

of the SDT, select the entity Media and enter the titles for

all of the specific media you have identified in your

analysis. While still in the Input Mode you must then

select the entity Course, select the subentity Module and

the attribute media, and enter the information required to

link course modules to media. The worksheet listed in Table

5-4 can be used to help in entering media selections into

the SDT.

V

14.

5-.

a-'

a.. . . .. . m . . . . - . . . . ... ..

a""... :.". v < k ' "'-'v -' . - ".-.. -- / -.- . .-.. " . ". • " v" "" -"
Table 5-4. Practical Factors to Consider in Making

Media Azsignments

1. Marginal Technical Solutions - The learning guidelines

and algorithm cannot be easily carried out with the

* system.

2. State-of-the-Art - The system is under development at

best and may not be available for practical application

by the time it is required.

3. Size of the System - Some approaches are useful within

large training programs. Others are suited only for

small programs and therefore, may not be suited to the

size program being considered.

4. Interface with Existing Program - Many new courses must

be designed to fit into existing programs, which place

constraints on the new courses; e.g., equipment on hand,

available classrooms, scheduling practices.

5. Time to Produce System - Approaches which require long

lead times for development may not be useful when

scheduled ready-for-training dates do not allow a long

development cycle.

6. Budget Cycle Constraints -While the application of some

of the powerful training approaches, such as CAI (Computer

Assisted Instruction), may result in low costs per student

graduate, the initial investment is substantial. Unless

these resources appear in existing budgets, the applications of these techniques to an immediate problem may not

be feasible.

7. Adoption of Innovations - Instructors frequently resist

innovations. If the proposed technique is significantly

different from existing techniques, either adequate resources must be focused upon gaining acc- otance for the

innovation, or a more traditional approack, must be selected.

8. Courseware Development - if the courseware is to be locally

developed, skilled personnel, equipment, time, and dollars

must be made available.

*9. High Cost Alternatives - The projected life cycle cost of

a media approach may be significantly higher than other

equally useful alternatives. Reject high cost alternatives

when others are available.

10. Learning Style of Trainees - If the trainee has a low

reading ability or would be limited in his ability to

use certain kinds of systems then reject these systems

* as inappropriate.

11. Other Constraints - A variety of other practical factors

should be considered; e.g., command policy and existing

investment in production facilities.

*Training Device Requirements and Document Guide.

5-9
- -.--.

.1k,

J.

.4

THIS PAGE INTENTIONALLY LEFT BLANK

A

p

$4'

.4,

4..

4..

.4

5-10
APPENDIX A -TECHNICAL DESCRIPTION OF MEDIA

SELECTION PROGRAM ALGORITHMS

v ~ A.1 COMPUTING THE SCORES

In order to produce an optimal assignment of media to tasks,

I some measures of the appropriateness of possible assignments

must be used as criteria. The four basic measures used by

the Media Selection Program are training efficiency,

training effectiveness, relative cost, and "utility."

A.1.1 Training Efficiency

Training efficiency provides a measure of the match between

the characteristics of a training medium and the characteristics of a task to be trained, based on the percentage of

-~ the psychological variables involved in the task which are

provided by the training medium. This approach, taken from

the TEEM model (Jorgenseon, Kubula, and Atlas, 1981), is

detailed below. Given T tasks, M media, and P psychological

variables, we make the following definitions:

1 if task t involves psychological variable p

t~p -t0 otherwise

mm~p 1 if medium m involves psychological variable p

14~

A-i
.. .

LL

The [T] matrix is an input from the SDT; the [ ] matrix is

hard-coded in the program. From these matrices, the

efficiency of each possible task-medium combination (e ,m)is

computed as follows:

1) A preliminary score (pst m) is computed as

P

Pst m - t,p m p p=l

This is equal to the number of psychological variables that

task t and medium m have in common.

2) An ideal score (it ) is computed for each task as

P it T

p=l t'p

This is the number of psychological variables involved in

task t, which is equal to the best score possible for any

medium with task t.

3) Finally, the efficiency is computed as

ps

et - 100

where "<>" denotes the greatest integer function (the

computed number is rounded down to the nearest integer) .

Thus training efficiency is an integer between 0 and 100

(inclusive) which measures the appropriateness of a given

training medium for training a given task. Training

A-2
71. -7 -7-7y Z-. ;P- 7 0-70 7

efficiency is computed for each possible task-medium

* combination.

A.1.2 TRAINING EFFECTIVENESS

Training effectiveness (~t ) ,a weighted measure of

training efficiency based on the criticality of the task, is

computed as the product of efficiency and task criticality

(kt). Task criticality is a composite of eight component

criticality variables: frequency, percent performing,

:,4 percent time performing, task delay tolerance, consequences

of inadequate performance, task learning difficulty, time

from entry to performance, and probability of deficient

performance. The overall criticality of a task, kt is an

i integer between 1 and 7 (inclusive) where 1 represents

minimum criticality and 7 represents maximum criticality

* (see A.1.2.1 for a detailed explanation of how the kt are

computed). Effectiveness is computed for each task-medium

combination as

et'm k

e

4'' 4'7

Thus training effectiveness is an integer between 0 and 100

(inclusive).

.pA-3
A.1.2.1 Task Criticality

Overall task criticality is a composite of eight component

criticality variables from the SDT (see Table B-2). Six of

the eight components (frequency, percent time performing,

consequences of inadequate performance, task learning

difficulty, time from entry to performance, and probability

of deficient performance) are scaled from 1 to 7, with a

score of 7 representing maximum criticality.

One of the components, task delay tolerance, is scaled from

7 to 1, with a score of 1 representing maximum criticality.

To convert this score from a 7-to-i scale to a l-to-7 scale,

the converted score (c) is computed from the unconverted

score (u) as follows:

c 8 -U.

Another of the component criticality variables, percent

performing, is scaled from 0 to 100, with a score of 100

representing maximum criticality. To convert this score from

A01. a 0-to-1QO scale to a 1-to-7 scale, the converted score (c)

* is computed from the unconverted score (u) as follows:

C (u* 7) + 99

100 /

Once the component criticality scores are on the same scale,

V a composite criticality may be computed for each task as a

weighted average of the eight components. Eight criticality

ii variable weights (wi) are entered by the user; each weight

is between 0 and 100 ( inclusive) , and the sum of the eight

A-4

~~~~~LA~~~~~~~~ *.**. *. . . . . . .- P #*-.. d~ . s
-a -7z 77.

weights must be 100. Then the criticality of task t (kt ) is !8computed as

i- (l it * wi) kt

100

where is the criticality score of task t on the ith

component of criticality, and wi is the user-specified

weight for the ith criticality component.

A.1.3 Relative Cost

Each of the media in the Media Selection Program is assigned

- a relative cost (cm ) . The cm are between 0 and 100

(inclusive) where 100 represents maximum cost. Clearly,

these costs are not in actual dollars; but they are based on

the dollar costs of the media.

A.1.4 "Utility"

In considering possible task-medium assignments, it is

useful to combine efficiency or effectiveness with cost into

one measure of the overall goodness (or "utility") of taskmedium combinations. The user specifies weights (wc and we)

which reflect the relative importance of cost and

efficiency/effectiveness; the weights must be between 0 and

100 (inclusive) and must sum to 100. Then "utility" is

computed for each task-medium combination as follows:

'

, A- 5

b °1 ! ~~ -.. .• . • . ..
K1

[wc * (100 - Cm) + [We I .. ~m= e t,m] o

100

[w * (100- cm)] + [w *et,m]

Ut,m = 100

depending on whether "utility" (Ut,m) is to be based on

efficiency (etm) or "utility" ('t,m) is to be based on

effectiveness (etm).

A.2 OPTIMIZATION

Two forms of optimization are available to the user of the

Media Selection Program: unconstrained and constrained.

A.2.1 Unconstrained optimization

An unconstrained optimization will occur whenever the user

selects an objective and "no constraint." For example, if

the user selects "maximize efficiency" as his objective, and

"no constraint," the program will return the unconstrained

maximum efficiency assignment. Mathematically this problem

is formulated and solved as follows.

Given T tasks and M media, and an efficiency etm for each

task-medium combination, each task will have assigned to it

A-6
the medium with the greatest efficiency for that task

according to the following rule:

Assign medium m to task t only if:

etm > et' i for all ie (1,2,...,M}.

The rules for the other optimizations are similar to that

for efficiency: To maximize effectiveness, assign medium m

to task t only if

i1t,m e e. >e et,i for all i; {1,2,...,M}.

To maximize "utility" assign medium m to task t only if

UP)Ut,m u u t for all i {1,2,...M}.

To minimize relative cost, assign medium m to all tasks only

if

cm 4 c i for all ie (1,2,...,M}. *

* Note that relative cost is task-independent.

A-7

*i - - - -,-w-.., .
A.2.2 Constrained Optimization

A constrained optimization will occur whenever the user

selects both an objective and a constraint. The

optimization is performed using a dynamic programming

algorithm which specifies the optimal assignment of exactly

one medium to each task for each of a discretized set of

levels of the constraint. Detailed discussions of the

formulation, the dynamic programming algorithm, and the

output follow.

A.2.2.1 The Formulation

Assume for example that the user selects "maximize

efficiency" as the objective, and "cost" as the

constraint. The mathematical formulation for this option is

* as follows:

T

N max Z et,m

t=1 tm

T

st E cm 4 c0

t=l t

where mt is the medium assigned to task t. A solution must

give the value of the objective function (later defined

as fT(co)) and a list of the optimal assignment dt =M i

(i~~ 1,2,...,M}) for each task t = 1,2,...,T.

A-
A.2.2.2 The Dynamic Programming Solution Approach

For T tasks and M media, there are efficiencies (etm) for

each task-medium combination, and costs (cm) for each

* *'"medium. The optimization must assign exactly one of the M

. media to each of the T tasks. For task 1 and a level of

cost, z, define the set Dl(z) as

Dl (z) {mlcm4z}.

Then Dl(z) is the set of media with costs not greater than

z, or the set of feasible media for task 1, cost z. Define

fl(z) as

fl(z) B-maximum fel,mlmeDl(z)}.

Then fl(z) is the greatest attainable efficiency for an

assignment of task 1 to a medium with cost not greater than

4,., z.

Next, define the set D2 (z) as

D2 (z) [mIDl(z- cm) O.

where 0 indicates the null set.

A-9
Then D2 (z) is the set of feasible media for task 2, cost

z. Note that D2 is dependent only on z, Dl, and the cm.

Next, define f2 (z) as

f 2 (z) - maximum {e2,m + fl(z - cm) ImeD 2 (z)}.

Then f2 (z) is the greatest attainable efficiency for an

assignment of tasks 1 and 2 to media with a total cost not

greater than z.

*..

Then, in general, Dt(z) is defined as

Dt(z) - (mDtl(z - cm) *O}.

and ft(z) is defined as

ft(z) z maximum feint + ft-l(z - cm)jmeDt(z)}.

Again, note that Dt is dependent only on z, Dt I , and the

cm.

This function, f, is a dynamic programming recursion in

canonical form. In particular, fT(co) gives the maximum

attainable efficiency for an assignment of all 'T tasks to

training media with the total cost of the assignment not

greater than co . The optimal assignment of media to tasks

A-10
is then obtained by back-substitution ("unwinding") in the

above equation (see A.2.2.3).

The above example gives the formulation for efficiency as an

objective, and cost as a constraint. In general, given T

tasks, M media, a matrix of zbjective values for each taskmedium pair (ot,m),and a matrix of constraint values (St,m)

for each task-medium pair, the dynamic programming recursion

may be formulated as follows:

m z{mlsl, z} if the constraint is cost,

Let D1 (Z)- I{mlSl,m > z} otherwise,

and for te{2,3,...,T}, define Dt(z) as

.

N' ,[. Dt (z) mDtl(z - st, m) *0

Then fl(z) is defined as

>'

f l (z) optimum {ol,mmEDl(z) },

and for te {2,3,...,T}, ft(z) is

A-11

-.. *.** '-.:%v:'- .- ** *. **>
ft(z) optimum {ot,m + ft-i (z - Sm,t) ImeDt(z)},

where "optimum" means maximum or minimum depending on the

objective selected, minimum for cost and maximum otherwise.

A.2.2.2. The Algorithm

As stated earlier, the Media Selection Program determines

the optimal assignment for a discretized set of 50 levels of

the constraint within the range of feasible constraints.

This is accomplished by execution of the following steps:

Step 1: Determine the minimum constraint level (Smin ) for

which an assignmeat of any task to any medium is possible as

Smi m=rin iS tFt{l,2, ... ,T}, m{,,.M}

Step 2: Determine the maximum possible constraint level

(Smax ) as

T

S a _= t=l (max {St,mIme{l,2,...,M}}).

Step 3: Compute a step size a as

Smax Smin

50

A-12

5i
Step 4: For zE;{Smin, Smin + a, Smin + 2 a, ... , Smin + 50a},

make a table of triples (z,fl(z), a,(z)) where a,(z) = m

only if fl(z) = olm. Then al(z) is the optimal medium for

task 1 with a constraint level of z.

Step 5: Perform the following for t=2,3,...,T:

For ze{Smin , Smin' + a, Smin + 2 a,,..., smin + 50Oa}, , make a

table of triples (z, ft(z), at(z)). Where at(z) = m only if

ft(z) = ot, m .

When step 5 is complete, the table for t=T contains a triple

for each of 50 levels of the constraint. Each triple gives

a constraint level (z), the optimum objective value

. attainable under the constraint (ft(z)), and the optimal

assignment for task T under the constraint (at(z)).

A.2.2.3 The Output

Once the optimization has been performed, the Media

Selection Program prints out a part of the table generated

in step 5 above for T=t. Table A-1 is an example of the

printout. The printout shows the user how much objective

value he can "get," given the associated constraint level,

i.e. ft(z) vs. z. Associated with each constraint level is

an assignment of all tasks to media. When the program

queries the user for the constraint level for which he would

like to see the assignment, the user may enter a number

between Smin and Smax . The program then "unwinds" the

tables to produce the optimal assignment Eor the userspecified constraint level as follows:

A41

A- 13

.. . ... .5* -. 5..* * - *%: , S .5:% .:. 
aW -ITable A-i. Example of Printout of

Constraint and Objective Variable Scores

OBJECTIVE: MAXIMIZE EFFICIENCY

CONSTRAINT: MINIMIZE COST

. CONSTRAINT LEVEL OBJECTIVE VALUE

50 500

60 500

70 550

S80 550

90 550

100 670

110 670

' 120 670

130 700

140 700

150 700

. .

A-14

.4 AIX 5,. " ..
 

 . .7 :i]

Let c o be the constraint level entered by the user. Then

task T is assigned to media at(co). Define rT to be the

remaining amount of constraint after task T is assigned, or

rT c - aT (c(, 0 )

Then task T-1 is assigned to media aTl(rt). -nd

iA

rl Zr -s T - T T-1, aT- (rT)

-T

- is the remaining amount of constraint after tasks T and T-1

4 have been assigned. Similarly, rt for t (1,2,...,T-I} is

defined as

rt rt+ 1 - St,a (r

*t t+l

Then, in general, the medium to be assigned to each of the

tasks t {T-1,T-2,...,l} is at(rt+l) = dt. These are the

assignments that are printed out in Table A- 2 .

A.3 TECHNICAL NOTES

A.3.1 The "Utility" Function

The function used to combine cost and efficiency/

effectiveness (see A.1.3) cannot legitimately be called a

A-15

• - % -.- _% .% .. % . .,- % - . . , . • . % ,. % , ... .. . . . -. . ., - - . , , . ,
- 4 0

00 0 a4

a, N. .,, ,,

• " " -- 

-4 0 -C

00 CD E00

-4 0 r24

a E- 0 0 4.4

0

* E-4

E-'

0x 

wU 0

.4 24

ad u

E-4 U)- (nA-1
utility function in the vonNeumann-Morgenstern sense; no

method is provided to elicit from the user the data

necessary to construct a function which satisfies the axioms

of u'.ility theory. Rather, the user is provided with the

most straight-forward preference curve: the straight line of

expected value.

A.3.2 Accuracy of the Dynamic Programming Solution

In order to keep main memory requirements within Apple III

limits, all possible constraint levels are not considered.

Instead, the feasible region is represented by 50 equallyspaced constraint levels (see A.2.2.2). This means that a

worst case error of up to a*T may be introduced into the

constraint levels reported for the possible assignments (see

Table A-1), the average error will be much smaller.

However, this is not a severe problem, as the exact

constraint level is recomputed and printed for any "unwound"

assignment (see A.2.2.3).

A.3.3 Integers vs. Real Numbers n the Apple III

An Apple Pascal library unit is available to process real

numbers, but it requires a great deal of main memory to

run. Since the memory requirements of the Media Selection

Program are quite large, the Apple library unit for real

numbers cannot be used and all computations are done using

integer arithmetic. Therefore all values computed using a

division are subject to a relatively small truncation error.

A-17
. . ....--

'S

*

N'S

-p.

p-N

N

THIS PAGE INTENTIONALLY LEST BLANK

'S

SS~

'F

-p

5'

'm~1

'.4

2

I~S

A- 18

-C-:-~ Ai~. *%** ~. *555 %*~ .S*~~ ~. * ~ S
. ._ , ,,,-- ; .

B"- W". 

.2." APPE NDIX B

This appendix provides definitions for the psychological and

criticality variables used in the Media Selection Program.

In addition, it lists the relative cost values for each

media category used in the program.

4.

Table B-i provides definitions for the psychological

variables. Those definitions are taken directly fran

Jorgensen, Kubulas, and Athus (1981). Most of these

definitions were derived fran the TRADOC Task Analysis

* -. Handbook. Table B-2 provides definitions for the

criticality variables. Table B-3 lists the relative cost

values for each media category used in the program.

B-1
Table B-i. Psychological Variables *

STIMULUS VARIABLES

Medium of Stimuli Presentation

1. Visual Cues - Signals received through the sense of sight.

2. Audio Cues - Signals received through the sense of hearing.

3. Tactile Cues - Signals received throuqh the sense of touch, including

serisations related to texture, size, shape, or vibration of the skin.

4. External Stimulus Motion Cues - The sensations felt by a person when

he is moved by some outside force in such a way that his body experiences

roll, pitch, yaw, heave, sway and/or surge.

5.- Internal Stimulus Motion Cues - The sensations felt by a person when

he moves ais arm, leg, fingers, etc.

6. Olfactile Cues - Signals received through the sense of smell.

7. Gustatile Cues - Signals received through the sense of taste.

Visual Form

8. Visual Alphanumeric - Words and/or numbers presented visually.

9. Vi'sual *Symbolic - Symbols presented graphically.

10. Visual Graphic - Two-dimensional figures, such as maps, graphs,

mathematical curves, etc., presented visually.

11. Visual Pictorial - Two-dimensional images, such as photographs,

drawings, etc., presented visually.

12. Visual Solid Object - A three-dimensional image or reality that is

viewed from exterior perspectives.

13. Visual Environment - A three-dimensional image or reality that is

viewed from inside.

Visual Movement

14. Visual Still - A static visual field, as with a still photograph,

drawing or printed page.

15. Visual Limited e*icvement - A basically static visual field with elements

that can be madeito move, as with an animated transparency or simple panel

with switches that move.

B-2
Table B-I. (continued)

16. Visual Full M.ovement - A visual field in which all elements can move,

as with a motion picture, flight simulator, or operational aircraft.

17. Visual Cyclic Movement - A visual field which moves through a fixed

sequence and then repeats the sequence in a repetitive manner, as with a

film loop.

Visual Spectrum

18. Black and White - A visual field composed of either black or white

elements, as with the printed page or line drawings.

19. Gray Scale - A visual field composed of black, white and continuous

gradations of gray, as with a black and white photograph or television

picture.

20. Color - A visual yield composed of various segments of the visual

spectrum, as with color television or motion pictures.

Visual Scale

21. Exact Scale - Actual visual field or a one-to-one replicaiion of

that field as with a full-sized mock-up, simulator, or operatidnal system.

*22. Proportional Scale - A representation of reality in other than

full scale, such as'a scaled model map or photograph.

Visual Contrast

23. Dim - A visual object which blends in with its background, i.e.,

there i-s a small or no luminance difference between an object and its

background.

24 Briqht - A visual object which is brighter than its background,

i.;., there is a large luminance difference between an object and

its background.

Audio Sources 4

25. Tonal Sound - A very limited source of sound or noise which is used,

rather than speech, for signaling or warning, e.g., horns, whistles, sirens,;1

bells, buzzers, etc.

I

26. Voice Sound - A limited source of scund which enables spoken words

to be used as the medium of communications, but not suited to more

demanding tasks, such as music or sound recoqnition exercises.

27. Full Sound - A source of sound that contains all the significant

elements of the sound and is suited to the demandinq task of sound

recognition exercises.

B-3
Table B-I. (continued)

28. Ambient Sound - A complex sound environment with sounds emanating from various sources and from various directions, including background noise and task significant sounds.

Audio Stimuli Intensity

29. Weak - Audio stimuli presented to the trainee which have weak intensity.

30. Strong - Audio stimuli presented to the trainee which have strong

intensity.

Stimul i Presentation

31. Static - A unitary stimuli situation, i.e., stimuli are presented to the trainee "allI at once", e.g., batch presentation .'

32. Dnamic-Ordered - A sequential stimuli situation, i.e., stimuli are

presented to the trainee sequentially or in an ordered manner over time.

33. Dynamic-Random - A non-sequential stimuli situation, i.e., stimuli

are presented to the trainee randomly over time.

Stimuli Presentation Rate

34.. Slow Rate - A slow rate or speed of presentation of stimuli to the

trainee, allowing the trainee a long or maximum stimulus analysis time.

35. Fast Rate - A fast rate or speed of presentation of stimuli to the

trainee, allowing the trainee a short or minimum stimulus analysis time.

Number of Channels or Sources

36. Limited - A small number of sources, channels, or instruments

through which stimuli are presented to the trainee.

37. Unlimited - A multiple number of sources, channels, or instruments

through which stimuli are presented to the trainee.

Stimuli Distribution

38. Individual - All information is presented directly to one individual

trai nee.

N, 39. Group - Information is presented to a group of trainees, allowing c-Ily

indirect access to the information for an individual.

B-4
Table B-I. (continued)

RESPONSE VARIABLES

Response Mode of Implementation

1. Overt Response - Verbal - A response which the trainee expresses in an

audible (verbal) manner, such as a verbal short answer response to a question

having a limited set of correct answers, a conversational response, or a

verbal decision response.

2. Overt Response - Written - A response which the trainee expresses

in an observable (written) manner, such as a free style written response,

a written multiple choice response, or a written fill-in-the blank response.

Overt Response - Manipulative Acts-- A response which the trainee expresses

in an observable (manipulative) manner, such as the small m6vements of dials,

switches, keys, or small adjustments to instruments or the large movements of

levers, wheels or use of hand held tools.

4. Overt Response - Tracking - A response which the trainee expresses in an

observable (tracking) manner, such as continuously controlling a constantly

changing system, e.g., steering an automobile.

5. Overt Response - Procedural Performance - A response which the trainee

expresses in an observable (procedural performance) manner, such as

performing a sequence of steps in a procedure, e.g., carrying out the items

on the checklist for preflighting an aircraft or turning on a radar system.

Intensity of Response

Weak - Responses made by the trainee with weak intensity.

7. Strong - Responses made by the trainee with strong intensity.

Response Implementation

8. Static - A unitary response situation, i.e., responses are made by the

trainee "all at once".

9. .Dynamic-Ordered A sequential response situation, i.e., responses are

made by the trainee sequentially or in an ordered manner over time.

10. Dynamic-Random - A non-sequential response situation, i.e., responses

are made by the trainee randomly over time.

B-5

-.° . , € o_..'' .' ,'.'. ¢ ,., '.. .".-....'. .-.....-.', ... ,..'..' ,-.-.. ,.' '. , ~ m,,, , . i, .'i " "" "" r ,, - -.-'.,
Table B-1. (continued)

- Recqu1red Response Rate

11. Slow Rate - A slow rate or speed of trainee response, i.e., a rate which

allows the trainee a long or maximum response time.

12. Fast Rate - A fast rate or speed of trainee response, i.e. , a rate which

allows the trainee a short or minimum response time.

N1umber of Response Channels

-S. 13. Limited - A limited number of sources, channels, or instruments through

which required responses are made by the trainee.

14. Unlimited -An unlimited number of sources, channels, or instruments

through which responses are made by the trainee.

Response Distribution

15. Individual - One individual trainee makes the required response.

16. Group -A group of trainees make the required response.

-'S

B-6

L'6 V
- . . .,- . . . . . . . . . . . .. . .,..

Table B-i. (continued)

•,FEEDBACK VARIABLES

Medium of Feedback Presentation

1. Visual - Feedback presented visually by means of a display, it may be

coded and transmitted visually to the trainee.

2. Aural - Feedback presented aurally by means of a display to the trainee.

3. Written Form - Feedback presented tc the trainee in written form.

" 4. Face-to-Face Communication - Feedback presented by direct verbal means

to the trainee.

-• Indirect Communication - Feedback presented by indirect verbal means,

such as by intercom, telephone, or radio link.

6. Tactile - Feedback presented to the trainee through the- sense of touch,

including sensations related to texture, shape, size, or vibration of the skin.

7. Kinesthetic - Feedback presented to the trainee by either internal or

external bodily movement, such as reaching, grasping, tilting, etc.

8. Olfactile - Feedback presented to the trainee through the sense of smell.

9. Gustatile - Feedback presented to the trainee through the sense of taste.

Source of Feedback

% 10. Intrinsic F - Information or cues built into the system from which the

.rainee interprets feedback information.

11. Extrinsic F - Information or cues not inherent in the trainee action or

system operations but is supplied by an external source.

Type of Feedback

12. Research Correctness (Rcr) - Information about the correctness or

incorrectness of trainee's response, when several response alternatives are

possible and the correct choice is not known to the trainae in advance.

(Also known as augmented feedback.)

13. Response Correctness (Rcf) Information provided to the trainee (or

)thers who need to know about his performance) that he has in fact performed

I in operation, but does not say anything about the longer range consequences

j f the action taken.

B-7
Table B-I. (continued) t

14. Response Consequences (Rcn) - Information about the consequences of the

action taken. It confirms the response made by the trainee, and the

correctness of a response can be inferred only from its consequences. May

also serve-to cue the trainee to perform the next response in sequence.

15. System Status (Rss) - Information about the condition of one's own

or another system or the external environment, on the basis of which a

trainee or team must act. Information is not necessarily (or even frequently)

the immediate consequence of or follow-on to a specific trainee/team action;

it may reflect system events that have been put in motion by much earlier

trainee actions. Provides information that regulates trainee and system

actions in the sense that when a particular status condition occurs, the

trainee must often take action to' maintain the integrity of his system.

Feedback Distribution

16. Individual - Feedback is presented to one individual trainee.

17. Group - Feedback is presented to a group of trainees, allowing only

indirect access for an individual.

B8

*B-8
Table B-2. Criticality Variables 1

1. Frequency - The number of times the task is performed per

week.

2. Percent Performing - The percentage of soldiers within an

MOS who are performing the task.

3. Percent Time Performing - An estimate of percentage of the

average workweek that soldiers in a given MOS spend in

performing the task.

4. Task Delay Tolerance - A measure of how much delay can be

tolerated between the time the need for task performance

becomes evident and the time actual performance must

begin. (TRADOC MPA 350-30)

5. Consequences of Inadequate Performance - The possible adverse

effects, in terms of personal safety, mission success

and/or damage to the system, which could result from

inadequate performance of a task.

6. Task Learning Difficulty - An estimate of the time, effort,

and assistance required by a student to achieve

performance proficiency.

7. Time from Entry to Performance - Amount of time between job

entry and task performance.

8. Probability of Deficient Performance - An estimate of the

frequency that a task is performed poorly.

iMore details on obtaining values for these variables are listed

in the ETES User's Guide.

B-9
Table B-3. Relative Cost Values.

CAT ROUPRELATIVE

* CT GOUPMEDIA CATEGORY COST

1 1._RNNSRCIN /OFEBC

1 1.1 PRINT INSTRUCTIONS WITHOR FEEDBACK 1

j3 1.3 PRINT INSTRUCTION - WITH WRITTEN FEEDBACK 1

4 1.4 PRINT - REFERENCE MATERIAL 5

5 1.5 IMAGE PROJECTION - STILL 1

6 2.1 PASSIVE AUDIO 1

7 2.2 ACTIVE AUDIO 4

8 3.1 AUDIOVISUAL - STILL WITH FEEDBACK 3 --

9 3.2 AUDIOVISUAL - STILL WITHOUT FEEDBACK 2

icr 3.3 AUDIOVISUAL - MOTION WITH FEEDBACK 5

11 3&4 AUDIOVISUAL - MOTION WITHOUT FEEDBACK -4

12 4.1 STATIC DISPLAY WITHOUT FEEDBACK 1

13 4.2 DYNAMIC DISPLAY WITHOUT FEEDBACK 5

14 4,3 PHYSIOLOGICAL TRAINER - INTERNAL 50

is1 4.4 PHYSIOLOGICAL TRAINER - AUDIO 37

16 45 PHYSIOLOGICAL TRAINER - VISUAL 37

*17 4.6 SYMBOLIC SIMULATION WITH FEEDBACK 5

18 4.7 TRAINER .55

19 4.8 SIMULATOR 72

20 4.9 OPERATIONAL EQUIPMENT 82

21 4.10 OPERATIONAL EQUIPMENT WITH FEEDBACK 95

22 51 VISUAL COMPUTER - STILL WITH FEEDBACK 12

23 5.2 VISUAL COMPUTER - STILL WITHOUT FEEDBACK8

24 5.3 VISUAL COMPUTER - MOTION WITH FEEDBACK 24

25 5.4 VISUAL COMPUTER - MOTION WITHOUT FEEDBACK 14

26 5.5 AUDIOVISUAL COMPUTER - STILL WITH FEEDBACK 22

27 5.6 AUDIOVISUAL COMPUTER - STILL WITHOUT FEEDBACK 18

23 5.7' AUDIOVISUAL COMPUTER - MOTION WITH FEEDBACK 34

29 5.8 AUDIOVISUAL COMPUTER - MOTION WITHOUT FEEDBACK 24

30 6.1 INSTRUCTOR

B-l0