Research Note 84-79

EARLY TRAINING ESTIMATION SYSTEM (ETES)

FINAL REPORT

APPENDIX F: USER'S GUIDE

Lawrence H. O'Brien, Michael Wagrer and Beth Modica

Dynamics Research Corporation

Charles Jorgensen, Contracting Officer's Representative

TSubmitted by

I IMichael H. Strub, Chief

ARI FIELD UNIT AT FORT BLISS, TEXAS

and

Jerrold M. Levine, Director D T l -I 

SYSTEMS RESEARCH LABORATORY ELECTE

JUL9 W4

U. S. Army

Research Institute for the Behavioral and Social Sciences

June 1984

* Approved for public release; distribution unlimited.

This report, as submitted by the contractor, has been cleared fr release to Defense Technical Information Center

I(DTIC to I comply with regulatory requirenents. It has been given no primary distribution other than to OTIC

and will be available only through DTIC or other reference services such as the National Technical Information

Service (NTIS). The vicws, opiniens, and/or findings contained in this reptn are those of the author(sl and

should not be construed as an official ,epatmeni of the Army position, policy, or decision, unless so designated by r other off itial documentation. B4 06 -
.. 

THIS DOCUMENT IS BEST 

QUALITY AVAILABLE. THE COPY 

FURNISHED TO DTIC CONTAINED 

A SIGNIFICANT NUMBER OF 

PAGES WHICH . DO NOT 

REPRODUCE LEGIBLY. 
REPORT DOCUMENTATION PAGE - -EREAD INSTRUCTIONS BEFORE COMPLETING FORM

1. REPORT.NUMBER OVT ACCESSION NO. 3. RECIPIENT'S CATALOG NUMBER

Research Note 84-79 L 9AJLY .. ,9 .

4. TITLE'(&d Subtitle) "S. TYPE OF REPORT & PERIOD COVERED

EARLY TRAINING ESTIMATION SYSTEM: FINAL REPORT FINAL APPENDIX F: USER'S GUIDE 6. PERFORMING ORG. REPORT NUMBER

-7. AUTHOR(s) 8. CONTRACT OR GRANT NUMBER(a)

Lawrence H. O'Brien, Michael Wagner, Beth Modica MDA-903-80-C-0525

S. PERFORMING ORGANIZATION NAME AND ADDRESS 10. PROGRAM ELEMENT. PROJECT. TASK

AREA & WORK UNIT NUMBERS

DYNAMICS RESEARCH CORPORATION

60 Concord Street

Wilmington, Massachusetts 01887 2Q162722A791

11. CONTROLLING OFFICE NAME AND ADDRESS 12. REPORT DATE

Army Research Institute June 1984

5001 Eisenhower Avenue I. NUMBEROF PAGES

Alexandria, Virginia 22333 412 14. MONITORING AGENCY NAME & ADDRESS(II diffe,rnt from Controlling Office) 13. SECURITY CLASS. (*I this report)

US Army Research Institute for the Behavioral UNCLASSIFIED

and Social Sciences

Fort Bliss Field Unit, P.O. Box 6057 ISC. NCkASSILICAT

ION/DOWNGRAOING

Fort Bliss, TX 79916 16. DISTRIBUTION STATEMENT (of this Report)

Approved for public release; distribution unlimited

1?. DISTRIBUTION STATEMENT (of the abstract entopedin Block 20, fi different (rout Report)

I6. SUPPLEMENTARY NOTES

Charles Jorgensen, Contracting Officers Representative

19. KEY WORDS (Continue on reverse side If necessary ad Identfly by block number)

Training Training Estimation

Instructional System Development Simulation

Data Base Management Training Effectiveness

Task Analysis Training Effectiveness Analysis

Front End Analysis 2". AIISRACT (Cootfie a reverse 610 If n0eewar m Identilfy by block number)

'This report describes the research and development activities conducted under

the Early Training Estimation System (ETES) development project. The Early

Training Estimation System (ETES) is an integrated set of procedures and

automated tools for estimating training requirements during the earliest

phases of the weapon system acquisition process. The ETES has three major

components; a System Description Technology (SDT), Early Training Estimation

Aids and Procedures (TEAP), and Evaluative Technology. The SDT is a data

L base management system for storing and tracking task and training-related A

DDFR 1473 EDITION OF I NOV 6S IS OBSOLETE D J, A.4 UNCLASSIFIED

SECURITY CLASSIFICATION OF THIS PAGE (When,, Date Ent-red)
PREFACE

This user's guide is part of the Early Training Estimation

System (ETES). Development of the ETES was sponsored by the

Army Research Institute (ARI) under contract No. MDA-903-80-

C-0525. Dynamics Research Corporation (DRC) of Wilmington,

Massachusetts was the contractor. The contract monitor for

the project was Dr. Charles Jorgensen. The guide was

written by Dr. Lawrence O'Brien, Dr. Michael Wagner, and Ms.

Beth Modica.

{; ,', '.; :;,, or

iii
THIS PAGE INTENTIONALLY LEFT BLANK

iv.
TABLE OF CONTENTS

Section Page

I. Introduction and Summary I-i

I.1 Objectives 1-2

1.2 Organization 1-3

1.3 Need for Early Training Estimation 1-3

1.4 Intended Users of ETES 1-4

1.5 ETES Components 1-6

1.5.1 The SDT 1-8

1.5.2 Training Estimati Aids/Procedures

(TEAP) 1-18

1.5.3 Evaluative Technology 1-28

1.6 Key Concepts Underlying ETES 1-32

1.6.1 Comparability Analysis 1-34

1.6.2 Phased Versus Steady-State Training

Resource Requirements 1-40

1.6.3 Limitations of Current ETES 1-41

1.7 Guidelines for Managing ETES Application 1-42

1.7.1 ETES Data Management 1-42

1.7.2 ETES Study Plan Development 1-44

1.8 Format Used to Describe ETES Procedures 1-62

v

*V
TABLE OF CONTENTS (continued)

Section Page

1. Conduct Functional Requirements Analysis 1-1

1.1 Identify Required System Functions and

Determine MPT Study Requirements 1-2

1.1.1 Identify Acquisition Characteristics 1-2

1.1.2 Identify/Describe System Missions 1-6

1.1.3 Determine High-Level Functions 1-11

1.1.4 Document Data 1-37

1.2 Identify System Performance Measures and

Goals 1-37

1.2.1 Identify System Performance Measures 1-41

1.2.2 Identify System Performance Goals 1-41

1.2.3 Identify Operational, Organizational,

and Support Concepts 1-45

1.2.4 Document Data 1-49

1.3 Allocate Required System Functions and

Identify Lower Level Functions 1-49

1.3.1 Allocate Major System Functions/

Subfunct ions 1-49

1.3.2 Establish Generic Equipment Structure 1-52

1.3.3 Determine/Describe Lower Level

Functions 1-57

vi

i
TABLE OF CONTENTS (continued)

Section Page

1.3.4 Describe Function Sequence 1-61

1.3.5 Document Data 1-64

1.4 Establish The Baseline Comparison

System (BCS) 1-64

1.4.1 Establish BCS 1-67

1.4.2 Determine BCS Operational and Support

Concepts 1-71

1.4.3 Describe Function Sequence and

Information Flow 1-73

1.4.4 Document Data 1-73

1.5 Identify Required System Improvements/

New Technologies 1-78

1.5.1 Identify Required System Improvements 1-78

1.5.2 Identify New Technologies 1-80

1.5.3 Document Data 1-84

1.6 Format New System 1-84

1.6.1 Describe Design Differences 1-85

1.6.2 Describe New System Hardware/Software 1-87

1.6.3 Document Data 1-87

vii
TABLE OF CONTENTS (continued)

Section Page

2.0 Generate Tasks 2-1

2.1 Identify, Collect, and Format Task Data 2-]

2.1.1 Identify MOSs Currently Performing

Similar Functions 2-3

2.1.2 Collect and Catalog Candidate MOS

Task Data 2-6

2.2 Develop Baseline Comparison System (BCS)

Task List 2-6

2.2.1 Select Subsystem from BCS Hardware/

Software List 2-11

2.2.2 Identify Non-Army Occupational

Specialty 2-11

2.2.3 Identify MOSs Associated with

Equipment 2-11

2.2.4 Identify Comparable Army Equipment 2-12

2.2.5 Identify Tasks Associted with

Equipment 2-12

2.2.6 Assess Adequacy of Task Descriptions 2-14

2.2.7 Revise Task Descriptions 2-16

2.2.8 Enter Data Into Data Base 2-16

2.3 Assign Baseline Comparison System (BCS)

Tasks to MOS, Duty Position, and Skill

Level 2-16

2.3.1 Identify Duty Position Structure 2-19

viii
TABLE OF CONTENTS (continued)

Section Page

2.3.2 Assign Tasks to Duty Position and

Skill Level 2-21

2.3.3 Assign Tasks to MOS 2-21

2.3.4 Assess Integrated Impacts of

Assignments 2-22

2.3.5 Document Data 2-23

2.4 Develop New System Task List 2-23

2.5 Assign New System Tasks to MOS, Duty

Position, and Skill Level 2-35

2.5.1 Identify Duty Position Structure 2-38

2.5.2 Assign Tasks to Duty Position and

Skill Level 2-38

2.5.3 Assign Tasks to MOS 2-39

2.5.4 Assess Integrated Impacts of

Assignments 2-40

2.5.5 Document Data 2-40

3.0 Training Requirements Determination 3-1

3.1 Select Tasks for Training 3-3

3.1.1 Choose Task Selection and Aggregation

Methods 3-5

ix
TABLE OF CONTENTS (continued)

Section Page

3.1.2 Rate/Score Tasks on Relevant

Characteristics 3-19

3.1.3 Apply Algorithm/Aggregation Method 3-40

3.1.4 Review Current Task Selection 3-52

3.1.5 Document Data 3-55

3.2 Assign Tasks to Training Settings 3-55

3.2.1 Select Setting Assignment Method 3-57

3.2.2 Obtain Task Data 3-58

3.2.3 Document Data 3-66

3.3 Identify Skills and Knowledges 3-66

3.4 Develop Target Population Description 3-72

3.5 Sequence Tasks/Skills For Training 3-76

3.6 Construct Quasi-Program of Instruction 3-80

3.6.1 Identify Courses and Course Modules 3-85

3.6.2 Identify Methods Within Modules 3-87

3.6.3 Identify Module Training Times 3-87

3.6.4 Determine Course Length 3-89

3.6.5 Document Data 3-89

3.7 Assign Tasks to Media 3-89

3.8 Construct Training Paths 3-91

x
TABLE OF CONTENTS (continued)

Section Page

4. Estimate Training Resources 4-1

4.1 Develop Course Operating and Support Plan 4-3

4.1.1 Determine Course Location 4-3

4.1.2 Determine Course Frequency 4-7

4.1.3 Determine Usage Rates 4-7

4.1.4 Document Data 4-10

4.2 Determine Number of Students to be Trained 4-10

4.3 Determine the Number of Instructors 4-20

4.4 Determine Facilities Requirements 4-20

4.5 Determine Training Device and Training

Equipment Requirements 4-26

4.6 Determine Requirements for Other Training

Resources 4-32

4.6.1 Determine Number of Training Aids/

Instructional 4-32

4.6.2 Determine Ammunition Requirements 4-38

5. Determine Training Cost 5-1

5.1 Estimate Individual Course Cost 5-1

5.2 Aggregate Course Costs 5-5

xi

xi
TABLE OF CONTENTS (continued)

Section Page

6. Estimate Training Efficiency/Effectiveness 6-1

6.1 Estimate Training Efficiency 6-1

I| 6.1.1 Estimate Training Efficiency 6-1

6.1.2 Estimate Media Efficiency 6-4

6.1.3 Estimate Training Method Efficiency 6-6

6.2 Estimate Training Effectiveness 6-28

7. Evaluate Training Program 7-1

7.1 Develop Figures-of-Merit 7-1

7.1.1 Assess Congruence with Training

Development Guidelines 7-7

7.1.2 Assess Congruence with Program

Requirements 7-l1

7.1.3 Assess Training Complexity 7-16

7.1.4 Assess Feasibility 7-18

7.1.5 Construct Summary Evaluation Score 7-19

7.2 Identify Likely Problem Areas 7-25

7.3 Identify Likely Causes of Problem Areas 7-30

7.4 Identify Alternatives 7-37

7.5 Evaluate Alternatives 7-45

7.6 Evaluate Impact of System Changes 7-52

xii

. . .. . . .. * , s4m• i~ .. ..
TABLE OF CONTENTS (continued)

Section Page

8. Develop Inputs to Acquisition Process 8-1

8.1 Develop/Monitor Training Development Schedule 8-1

8.2 Develop Impacts to Acquisition Processes

and Documents 8-1

8.2.1 Develop Inputs to OICTP/ICTP 8-2

8.2.2 Develop Inputs to Cost and Training

Effectiveness Analyses 8-6

8.2.3 Develop Inputs to Logistics Support

Analysis 8-14

8.2.4 Develop Inputs to Operational Testing 8-14

8.2.5 Develop Inputs to Training Device

Requirements Documents and Processes 8-19

8.2.6 Develop Inputs to New Equipment

Training 8-20

8.2.7 Develop Inputs to System Requirements

Documents 8-21

8.2.8 Develop Inputs to Request for Proposal

Development/Evaluation 8-21

8.2.9 Develop Inputs to Personnel Documents/

Processes 8-23

8.2.10 Develop Inputs to System Level

Documents and Processes 8-24

8.2.11 Develop Inputs to Mission Area

Analysis/Soldier Analysis 8-28

8.2.12 Relationship Between ETES and LCSMM 8-28

xiii
S

TABLE OF CONTENTS

Section Page

References

04 Appendix A: Overview of Media Selection Program A-i

Appendix B: Overview of Automated Resource and Cost

Estimation Technique B-i

Appendix C: Overview of Automated Planning and

Scheduling Technique C-i

Appendix D: Description of Projects Related to ETES D-1

Appendix E: Data Source Index E-1

Appendix F: List of Performance Measures F-i

Appendix G: Description of TRAINVICE G-I

Appendix H: Training Concept Descriptions H-i

Appendix I: HRTES Procedures for Identifying OT

Test Issues I-i

xiv
LIST OF FIGURES

Figure Page

I-i System Definition as a Function of Acquisition

Process 1-5

1-2 ETES Components 1-7

1-3 ETES Architecture 1-13

1-4 System Development Process for SDT 1-15

1-5 ETES Training Estimation Aids/Procedures 1-19

1-6 Relationship Between ISD and ETES 1-29

1-7 ETES Evaluative Technology 1-30

1-8 Overview of Comparability Analysis Procedures 1-35

1-9 ETES Data Management. 1-43

1-10 Overview of Audit Trail Procedures. 1-48

I-ll Constructing a Study Plan. 1-58

1-12 Procedures for Developing Data Source Index 1-60

1-1 Overview Diagram: Identify Required System

Functions and Determine MPT Study Requirements

(1.1) 1-3

1-2 Acquisition Information Worksheet (ACQ) 1-7

1-3 Mission Profile Worksheet (MIS) 1-9

1-4 System Performance Specification Worksheet (PRF) 1-12

1-5 Overview Diagram: Identify System Performance

Measures and Goals (1.2) 1-40

1-6 System Performance Specification Worksheet (PRF) 1-42

1-7 Organizational, Operational, and Support Concepts

Worksheet (OSC) 1-46

1-8 Operational Environment Worksheet (ENV) 1-48

xv
LIST OF FIGURES (continued)

Figure Page

1-9 Overview Diagram: Allocate Required System

Functions and Identify Lower Level Functions

(1.3) 1-51

1-10 System Performance Specification-Worksheet (PRF) 1-53

1-11 Generic System Worksheet (GEN) 1-58

1-12 Function Sequence Worksheet 1-62

1-13 Example Function Sequence Diagram 1-63

1-14 Overview Diagram: Establish the Baseline

Comparison System (BCS) (1.4) 1-66

1-15 Baseline Comparison Equipment Selection

Worksheet (EQP) 1-68

1-16 BCS Equipment Description Worksheet 1-72

1-17 Function Sequence Worksheet (FSQ) 1-75

1-18 Overview Diagram: Identify Required System

Improvements/New Technologies (1.5) 1-79

1-19 Baseline Comparison Performance Shortfall

Worksheet (PSH) 1-81

1-20 Overview Diagram: Format New System (1.6) 1-86

1-21 Design Difference Worksheet 1-88

1-22 New System Equipment Description Worksheet (EQP) 1-89

2-1 Overview Diagram: Generate Tasks (2.0) 2-2

2-2 Identify, Collect, and Format Task Data 2-4

2-3 Candidate MOS Worksheet (CND) 2-5

2-4 Overview Diagram: Develop BCS Task List (2.2) 2-8

2-5 BCS Task Association Worksheet (BCST) 2-10

xvi
. . . .

LIST OF FIGURES

Figure Page

2-6 Overview Diagram: Assign BCS Tasks to MOS,

Duty Position, and Skill Level (2.3) 2-20

2-7 Overview Diagram: Develop New System Task

List (2.4) 2-25

2-8 New System Task Generation Worksheet (NTA) 2-27

2-9 Overview of Relationship Between Subsystem

Functions, Collective Tasks, and Individual

Tasks 2-30

2-10 Overview Diagram: Assign BCS Tasks to MOS,

Duty Position, and Skill Level (2.5) 2-37

3-1 Overview Diagram: Estimate Training 3-2

3-2 Overview Diagram: Select Tasks for Training 3-4

3-3 Choose Task Selection and Aggregation Methods

(3.1.1). 3-7

3-4 DIF Model 3-11

3-5 Detailed DIF 3-12

3-6 Rate/Score Tasks on Relevant Characteristics

(3.1.2) 3-20

3-7 DIF Data Collection Format 3-21

3-8 Apply Algorithm/Aggregation Method (3.1.3) 3-46

3-9 Review Current Task Selection (3.1.4) 3-54

3-10 Overview Diagram: Assign Tasks to Training

Settings (3.2) 3-56

3-11 Overview Diagram: Make Setting Assignments

(3.2.2.3) 3-63

xvii

. . . . . . .. 0 .. . . .. . . . . . . .. . . . . . . . m i
ALIST OF FIGURES (continued)

Figure Page

3-12 Overview Diagram: Identify Skills and

Knowledges (3.3) 3-68

3-13 Overview Diagram: Sequence Tasks/Skills for

Training 3-78

3-14 Overview Diagram: Construct QUASI-POI (3.6) 3-84

3-15 Overview Diagram: Construct Training Paths 3-93

3-16 Example of Training Path for Institutional

Courses 3-94

4-1 Overview Diagram: Determine Course Operating

and Support Plan 4-2

4-2 Operating and Support Plan Worksheet (OPSP) 4-4

4-3 Media Usage Rate Worksheet(MUS) 4-5

4-4 Overview Diagram: Determine Course Locations 4-6

5-1 Overview Diagram: Determine Training Cost

(5.1) 5-.

5-2 Individual Course Cost Worksheet (ICOST) 5-3

5-3 Course Aggregation Worksheet (ACOST) 5-7

6-1 Overview Diagram: Estimate Training Efficiency/

Effectiveness 6-2

6-2 Overview Diagram: Estimate Training Efficiency 6-3

6-3 Overview Diagram: Determine Training Efficiency 6-5

6-4 Overview Diagram: Trainvice Procedures 6-8

6-5 Overview Diagram: Generalized Procedures for

Assessing Efficiency of Training Methods 6-12

xviii
LIST OF FIGURES (continued)

Figure Page

6-6 Overview Diagram: Estimate Training

Effectiveness 6-30

7-1 Overview Diagram: Evaluate Training Program

(7.0) - 7-2

7-2 Overview of Relationships Among Training

Figures of Merit 7-6

7-3 Overview Diagram: Construct Measure of

Congruence between Actual and Prescribed

Training Development Practices 7-8

7-4 Overview Diagram: Assess Feasibilty of

Training Program Events 7-20

7-5 Overview Diagram: Identify Problem Areas (7.2) 7-27

7-6 Relationship between Two Types of Causes

of Training Problems 7-31

7-7 Overview Diagram: Identify Likely Causes (7.3) 7-33

7-8 Overview of Causal Structure Underlying

Training Cost 7-34

7-9 Overview of Causal Structure Underlying

Training Complexity (Training Time) 7-35

7-10 Overview of Causal Structure Underlying

Early Measure of Effectiveness 7-36

7-11 Overview Diagram: Identify Alternatives (7.4) 7-39

7-12 Overview of Procedure for Evaluating Training

Alternatives 7-46

7-13 Overview Diagram: Conduct Sensitivity Analysis 7-48

xix

. . . . .. . . . ... . . . . . . . . . . . .. . . 4 
LIST OP FIGURES (continued)

Figure Page

8-1 Basic HRTES Flow 8-17

8-2 LCSMM Events 8-30

S xx
LIST OF TABLES

Table Page

I-I SDT Data Elements 1-10

1-2 Major Army Acquisition Processes and Documents

Related to Early Training Estimation 1-33

1-3 Sample Page From Data Source Index 1-45

1-4 Example Page from Library Index 1-46

I-5 Example of ETES Worksheet - System Performance

Specification Worksheet (PRF) 1-59

1-6 List of ETES Worksheets 1-5

I-7 ETES Study Plan Outline (OUT) 1-53

1-8 Logical Sequence for Applying ETES Functions/

Procedures 1-59

1-9 Organization of Remaining Sections 1-63

1-1 Outline of Information Elements Contained in

JMSNS 1-4

1-2 List of Program Documents for a New System 1-5

1-3 Systems and Functional Area Proponents 1-16

1-4 Doctrinal Literature Program Publications and

Proponents 1-27

1-5 Guidelines for Entering Data Into SDT (1.1) 1-38

1-6 Definition of Key Concepts 1-39

1-7 Guidelines for Entering Data Into SDT (1.2) 1-50

1-8 Example of Generic Equipment Structure 1-56

1-9 Guidelines for Entering Data Into SDT (1.3) 1-65

1-10 Example of an Equipment Taxonomy 1-70

1-11 Format of the Operational Concept and

Concept Statement 1-74

xxi
LIST OF TABLES (continued)

Table Page

1-12 Guidelines for Entering Data Into SDT (1.4) 1-76

1-13 Guidelines for Entering Data Into SDT (1.6) 1-90

2-1 LCSMM Documents and Processes Related to Early

Training Estimation 2-7

2-2 Relationship Between OICTP and ETES Functions 2-13

2-3 Relationship Between CTEA and ETES Functions 2-15

2-4 Summary of Standardized Reports Related to

Early Training Estimation 2-17

2-5 Organizations Interviewed During Task 1 2-17

2-6 Potential Problem Solving Aids 2-24

2-7 Task Deletion/Modification/Addition Codes 2-29

2-8 Maintenance Actions 2-31

2-9 Example Maintenance Allocation Chart 2-34

2-10 Guidelines for Entering Data Into SDT (2.4) 2-36

2-11 Guidelines for Entering Data Into SDT (2.5) 2-41

3-1 Overview of Task Selection Methods 3-6

3-2 Data Sources for 4-Factor Model 3-14

3-3 Potential Algorithms for Aggregating Task

Factors 3-17

3-4 Appropriate Aggregation Algorithms for

Alternative Task Selection Models 3-18

3-5 Sample Data Set Using 4-Factor Model 3-28

3-6 Sample Data Collection Forms for 8-Factor

Model 3-29

xxi i
LIST OF TABLES (continued)

Table Page

3-7 Sample Data Collection Forms for 8-Factor

Model 3-41

3-8 Data Summary Worksheets 3-48

3-9 Worksheet for Selecting Tasks for Training

(TSEL) 3-53

3-10 Data Summary Worksheet for Training Setting

Assignment (SUMSET) 3-61

3-11 Aggregation Methods for Assigning Tasks to

Training Settings 3-65

3-12 Guidelines for Entering Data into SDT (3.2) 3-67

3-13 Skill and Knowledge Worksheet (SKILL) 3-69

3-14 Skill & Knowledge Data Sources 3-71

3-15 Guidelines for Entering Data into SDT (3.3) 3-74

3-16 Procedures for Developing Target Population

Description Elements 3-75

3-17 Target Population Worksheet (POP) 3-77

3-18 Task/Skill Sequence Worksheet (SEQ) 3-81

3-19 QPOI Worksheet - PARTS 1 & 2 (QPOI-l, QPOI-2) 3-82

3-20 Course Modification Worksheet (MOD) 3-86

3-21 Instructional Methods 3-88

3-22 Guidelines for Entering Data into SDT (3.6) 3-90

3-23 Guidelines for Entering Data into SDT (3.7) 3-92

3-24 Training Path Worksheets (PATH) 3-95

3-25 Guidelines for Entering Data into SDT (3.8) 3-96

4-1 Algorithm for Determining Number of Course

Starts and Sessions Per Site 4-8

4-2 Methods of Instruction and Associated Student/

Instructor Ratios 4-9

xxiii

I
LIST OF TABLES (continued)

Table Page

4-3 Guidelines for Entering Data into SDT (4.1) 4-11

4-4 Algorithm for Determining Steady-State Training

Input Requirements 4-12

4-5 Number of Students - Steady-State Worksheet

(NSSS) 4-13

4-6 Worksheet for Phased Course Input Requirements 4-14

4-7 Number of Students to be Trained Worksheet

(NSTUD) 4-21

4-8 Guidelines for Entering Data into SDT (4.2) 4-22

4-9 Algorithm for Determining Number of Instructors 4-23

4-10 Number of Instructors Worksheet (NINS) 4-24

4-11 Guidelines for Entering Data into SDT (4.3) 4-25

4-12 Algorithm for Determining Training Facilities

Requirements 4-27

4-13 Worksheet for Determining Training Facilities

(DFAC) 4-28

4-14 Facilities Requirements Worksheet (NFAC) 4-29

4-15 Algorithm for Determining Number of Training

Devices/ Equipments Per Site 4-30

4-16 Worksheet for Determining Number of Training

Devices/Equipments (TDEV) 4-31

4-17 Training Device/Equipment Worksheet (DEEQ) 4-33

4-18 Guidelines for Entering Data into SDT (4.5) 4-34

4-19 Worksheet for Determining Number of Media and

Instructional Aids (NMED) 4-36

4-20 Number of Media/Aids (MEDSUM) 4-37

4-21 Guidelines for Entering Data into SDT (4.6) 4-39

4-22 Ammunition Requirements Worksheet (AMREQ) 4-40

4-23 Ammunition Requirements Summary Sheet (AMSUM) 4-41

xxiv
LIST OF TABLES (continued)

Table Page

5-1 Guidelines for Entering Data into SDT (5.1) 5-6

6-1 Training Summary Worksheet (TVICE) 6-9

6-2 Guidelines for Entering Data int. SDT (6.1.2) 6-10

6-3 Instructional Methods 6-13

6-4 Methods Related Variables 6-14

6-5 Method Rating Worksheet (METH) 6-25

6-6 Task Rating Worksheet for Instructional Methods

(TAMOD) 6-26

6-7 Worksheet for Calculating Method Efficiency

(METHEFF) 6-27

6-8 Form for Rating Effectiveness 6-31

6-9 Guidelines for Developing Elements in Training

Effectiveness 6-33

6-10 Worksheet for Calculating Overall Effectiveness

Score (EFF) 6-34

7-1 Overview Diagram: Evaluate Training Program

(7.0) 7-3

7-2 Overview of Relationships Among Training

Figures of Merit 7-5

7-3 Overview Diagram: Construct Measure of

Congruence between Actual and Prescribed

Training Development Practices 7-10

7-4 Overview Diagram: Assess Feasibilty of

Training Program Events 7-12

7-5 Overview Diagram: Identify Problem Areas (7.2) 7-14

7-6 Relationship between Two Types of Causes of

Training Problems 7-15

XXV
LIST OF TABLES (continued)

Table Page

7-7 Overview Diagram: Identify Likely Causes (7.3) 7-17

7-8 Overview of Casual Structure Underlying

Training Cost 7-21

7-9 Overview of Casual Structure Underlying

Training Complexity (Training Time) 7-22

7-10 Overview of Causal Structure Underlying

Early Measure of Effectiveness 7-24

7-11 Overview Diagram: Identify Alternatives (7.4) 7-28

7-12 Overview of Procedure for Evaluating Training

Alternatives 7-29

7-13 Overview Diagram: Conduct Sensitivity Analysis 7-38

7-14 Training Section Alternatives 7-40

7-15 Training Method Alternatives 7-42

7-16 Training Media Alternatives 7-43

7-17 Training Project Elements 7-44

7-18 Listing of Candidate Parameters 7-50

7-19 Worksheet for Describing Results

of Sensitivity Analysis (ETES) 7-53

7-20 ETES Procedures Which Must be Reapplied In

Response to System Changes 7-54

8-1 LCSMM Documents and Processes Related to Early

Training Estimation 8-3

8-2 Relationship Between OICTP and ETES Functions 8-7

8-3 Relationship Between Institutional Training

Concept Elements and ETES Functions 8-9

8-4 Relationship Between CTEA and ETES Functions 8-12

8-5 LSAR Elements Included in ETES SDT 8-15

8-6 Relationship Between HRTES Function and ETES

Function 8-18

xxv i
LIST OF TABLES (continued)

Table Page

8-7 ETES-LOA Relationships 8-22

8-8 ETES-QQPRI Relationships 8-25

8-9 ETES-IPS Relationships 8-26

8-10 ETES-Soldier Analysis Relationships 8-29

xxvii
THIS PAGE INTENTIONALLY LEFT BLANK

xv ii i
SECTION I - INTRODUCTION AND SUMMARY

I.1 OBJECTIVES

This guide describes the manual procedures in the Early

Training Estimation System (ETES). The objective of ETES is

to provide an integrated set of procedures and automated

tools for estimating training requirements for emerging Army

weapons systems during the earliest phases of the

acquisition process (Mission Area Analysis, Concept

Exploration, and Demonstration and Validation).

Detailed descriptions of ETES automated tools are available

in the following documents:l

O User's Guide: ETES System Description Technology

(approximately 300 pages)

0 User's Guide: ETES Media Selection Program

(approximately 200 pages)

o User's Guide: ETES Resource and Cost Estimation

Technique (approximately 100 pages)

o User's Guide: ETES Automated Training, Planning

and Scheduling Technique (approximately 100 pages)

01

For information on obtaining ETES documents, contact Dr.

Lawrence O'Brien, Dynamics Research Corporation, 7 Lopez

Road, Wilmington, MA 01887.

I-1

0 , l I • | -I l I I I. ......
A description of the research which led to the development

of ETES is provided in the ETES Final Report. Additional

descriptions of ETES tools are provided in the following

technical papers:

Jorgensen, C., and O'Brien, L. The Early Training Estimation

System: An automated training needs assessment technique.

(Paper to be published in the first issue of the Training

Technology Journal).

O'Brien, L. Automated System Description Technology. Paper

presented at the Second Annual Conference on Microcomputers

in Education, Washington, June 1982.

Boylston, D. An automated decision aid for the assignment

of tasks to training media in early training estimation.

Paper to be presented at the 51st Symposium of the Military

Operations Research Society, September 27-29, 1983.

O'Brien, L. Early Training Estimation System (ETES).

Proceedings of the TRADOC Developments Institute Chiefs of

Analysis Seminar, October 21-13, 1981.

1.2 ORGANIZATION

The remainder of this section describes (a) the requirements

which led to the development of ETES, (b) the intended users

of ETES, (c) the components of ETES, (d) key concepts

underlying the ETES procedures, and (e) guidelines for

* managing an ETES application.

Sections 1 thru 6 describe the ETES Training Estimation Aids

and Procedures. Sections 7 and 8 describe the procedures in

* the ETES Evaluative Technology.

1-2

6 I I I I . .. . . . .. . . . .. . . H - I I I
Overviews of the three ETES Automated Training Estimation

Tools -- the Media Selection Program, the Resource and Cost

Estimation Technique (RCET), and the Automated Planning and

Scheduling Technique are provided in Appendices A, B, and C,

respectively. An overview of other Army Research Institute

projects related to ETES is provided in Appendix D. A data

source index describing the most common data sources which

are used in an ETES application is provided in Appendix E.

1.3 NEED FOR EARLY TRAINING ESTIMATION

The Early Training Estimation System provides a capability

for systematically estimating training requirements for

developing Army weapon systems during the earliest phases of

the acquisition process (Mission Area Analysts, Concept

Exploration - Phase I, and Demonstration and Validation -

Phase II). These estimates of training requirements include

specification of the system's task requirements, training

course requirements, resource requirements, and estimates of

training cost and "effectiveness."2

There are two major reasons why early estimates of training

requirements are needed. First, by developing earlier and

more accurate estimates of training requirements, the

training planning process can begin earlier. As a result,

the training products associated with a system, many of

which require a long lead time, are more likely to be

available when the system is fielded.

2 ETES only provides gross high level estimates of

training effectiveness. These estimates are only

appropriate during the earliest phases of the acquisition

process.

1-3

0,, , , , nn I I N | - - i . . . . ... . . . . . . . . . . .. .
Second, by developing estimates of training requirements for

the various design. alternatives which are considered during

early phases of the acquisition process, the training

developer can provide the information needed to effectively

influence system design.

The importance of obtaining training projections during the

earliest phases of system acquisition cannot be overestimated. Most of the major design decisions related to a

new system are made during the early phases of the

acquisition process (see Figure I-1). Thus, if training is

to influence design, it must impact these early design

decisions. And there is good reason for ensuring that

training-related considerations do, in fact, impact

design.

In most weapon systems, manpower costs, including training

costs, are the largest component of the system's operation

and support costs. Because these costs are the result of

demands generated by the design characteristics of a system,

acquisition policies have been established by the Federal

Government to ensure that support requirements are

accurately determined and evaluated in conjunction with

system development (for example, the DoDD 5000 series on

major systems acquisition). ETES is specifically designed to

provide the Army with the capability for meeting the

training-related requirements specified in these acquisition

policies.

1.4 INTENDED USER OF ETES

As part of the review of existing Army procedures conducted

during Task 1 of the ETES development study, potential users

1-4
z

CA

z L"

0L -

-2

6u 2 

2

LU -A

.j.

> LU 

'

a 

z

S 4

z

2 .2 LU - 4

> 

z

LU UL.

Xe C.L >

0 -j

0 LU LU u-

>. U.1

L6 

>

U

C ,Lz

fAO

1-5
for an early training estimation system were identified.

The primary user organizations of ETES are expected to be

(1) the training developers in the Army schools associated

with the development of new systems, (2) program management

offices (PMOs) for new systems, particularly those individuals concerned with training development or Integrated

Logistics Support, (3) the TRADOC System Manager (TSM), (4)

other Army organizations concerned with training development

such as the TRADOC System Analysis Activity (TRASANA) and PM

TRADE, and (5) contractors who must develop training

requirements for new systems.

1.5 ETES COMPONENTS

ETES has three major components; a System Description

Technology (SDT), Early Training Estimation Aids and

Procedures (TEAP), and Evaluative Technology (see Figure I2). The SDT is a data base management system for storing

and tracking task and training-related data. The data in

-' the SDT is used in the TEAP to estimate training

requirements for a new system.-' These training requirements

include estimates of task requirements, course requirements,

and resource requirements as well as estimates of training

costs, training efficiency, and training "effectiveness".

In the Evaluative Technology component, the integrated

impacts of training requirements are assessed, training

alternatives are evaluated, tradeoffs and sensitivity

analyses of key parameters are conducted, and the

relationships between ETES outputs and key Army acquisition

a documents and processes are specified.

More details on the three components of ETES are provided in

the sections which follow. A detailed description of the

1-6

S i l " U B a I n I . . . . . . . . . . . . . . . .. .
UA

44

2

6. >u U'

0Z2Z CC g-~ 00

Z10 4 

o'-

-- Is.

0-

Z cc

lA1~.C. C.0

ow;! L1 -J

CU' 

-C C0w O

S4w

2w z = z U22w

UA ZC 0 

VbU

0-1-7L z
SDT is provided in User's Guide: ETES SDT. Detailed

descriptions of the Training Estimation Aids and Procedures

are provided in Sections 1 thru 6 of this guide. Detailed

descriptions of the Evaluative Technology are provided in

Sections 7 and 8 of this guide.

There are three automated aids associated with the Training

Estimation Aids/Procedures and Evaluative Technology; (1)

the Media Selection Program which is an automated tool for

assigning tasks to media, (2) the Resource and Cost

Estimation Technique which is an automated tool for

estimating instructor requirements and course costs, and (3)

the Automated Planning and Scheduling Technique (APST) which

is an automated tool for describing and monitoring the

training development schedule. Detailed descriptions of

these three automated tools are provided in User's Guide:

Media Selection Program; User's Guide: Resource and Cost

Estimation Technique; and User's Guide: Automated Planning

and Scheduling Technique.

1.5.1 The SDT

The SDT is a microcomputer-based data base management system

for (1) describing actual and projected system elements

including system functional requirements, system hardware

and software concepts, tasks, skills, and training program

elements and their associated resource and cost requirements, (2) for storing the above information, (3) for

changing and updating this information, and (4) for transmitting the information among the participants in the

acquisition process. The -)T data base management system is

designed to allow training developers to keep track ot the

numerous system changes which occur during the early phases

1-8
CC

0 zo

2 z 0

to

U- .J ..

2

0 CL2 242

CA tto

w C2

0 W_ W us L I

E P, 4 4

4 0 5U 2

a. >~0z

0 o C.C > .

4o, 0 =o -

0j

I- 2a w

0 z 0

-a go c U

w w cc o LU ~ ~ ca a

0 z cc L 0 w co z0 - 0 0 Z ta

Lu-~ U.w :3MU ) 0 uOj~

Wco Cu z4x a to to o ZI 

toLu D - L c 0 0

w = z<. .w U wcaL a

0w 0 0

U to (

z to 2: 2 F 4ca o2 .

- < E w a to 44 -j

w Z O O2 0.

CC U. ZL LU to 1 2 t

=- 1 U, - . ca 0 z 0 0 Z zw

2U Ca 2 Z 44 a z w 0 2 w o<w

0. 21--w - <0
of the acquisition process. In addition, it provides a

centralized data base, thus eliminating the redundant data

collection efforts which typically take place among the

numerous training development organizations within the Army.

The computerized SDT also facilitates the use of automated

aids for estimating training program elements and their

associated resource and cost requirements. These automated

aids allow the training developer to quickly develop

training requirements estimates for alternative training and

system concepts. This capability greatly increases the

probability that training requirements will be effectively

considered during the early phases of the weapon system

development process.

1.5.1.1 SDT Data Elements

To provide an effective communication vehicle for training

developers and other participants in the acquisition

process, the SDT has been designed to describe (a) training

programs and their associated resources, (b) the tasks which

drive these training programs, (c) the personnel who will be

required to perform the tasks, (d) the system designs which

generate the task requirements, and (e) the functional

requirements for which the system designs have been

developed. An overview of the major data elements currently

included in the SDT is provided in Table I-1. Although the

data elements in the SDT are designed to remain fixed for

any particular application, the SDT software allows the SDT

data base manager to quickly change the data elements to

meet the needs of particular users and/or models.

1-9
1.5.1.2 Characteristics of SDT Data Base

Data base management systems such as the SDT use a

specialized "language" to describe the relationships among

data elements. Unlike many other data base management

systems, the SDT does not require the user to explicitly use

or even know about this specialized data language. In the

SDT, the data language is made "invisible" to the user

through "user-friendly" human-computer dialogue techniques

such as menu selection and question-and answering.

There are four major types of variables in the SDT data

language:

o Entities - Major system elements. Entities are

roughly equivalent to nouns in the English

language. The entities in the SDT are functional

requirements, system missions, equipment, tasks,

courses, training media, and personnel.

o Subentities - Lower level system elements.

Subentities are linked to entities in a

hierarchical fashion. For example, "task

conditions" are subentities of "tasks".

o Attributes - Descriptors that delimit or specify

important properties of entities. Each attribute

has a set of values associated with it. Attributes

are used to describe both entities and

subentities. For example, one attribute for the

entity task is "task frequency."

I-li
O Pointer Variables - Variables used to specify the

relationship which exist between different

entities, between entities and subentities and

between entities, subentities, and attributes. The

relationships specified by the pointer variables

determine the SDT data structure. (The SDT uses

elements of both hierarchical and relational data

base structures.)

1.5.1.3 SDT Configuration

When fully implemented, the SDT will utilize a distributed

processing architecture (see Figure 1-3). A centralized

data base for each weapon system (or weapon system

alternative) will be stored on a mainframe computer. At

periodic intervals, users will transfer a copy of the data

base from the mainframe to a local microcomputer. Once on

the micro, users can perform standard data base management

functions (input, output, modify). Thus, all major data

base management functions can be performed independently.

Once users have completed their activities, they can

transfer the updated version to the mainframe. A detailed

"audit trail" will be kept for each weapon system so that

users can systematically track and assess system changes.

The current version of the SDT is designed to operate on (1)

an APPLE III microcomputer with a modem, printer, monitor,

floppy disk drive, and a 5 megabyte hard disk and (2) a

Honeywell DPS 8/32 mainframe computer. The mainframe

computer is actually only needed when there is more than one

user and these users are located at several different

sites. The SDT has built-in security features which allow

the SDT manager to restrict data input, modify, and output

capabilities to a limited set of users.

01

1-12
IOR

= LU

4..l)1-

a. 0 L

z 

z

00

cm) U.

I-

(1,U

C,,

SL
1.5.1.4 SDT Operation a

To provide a user-friendly interface, dialogue on the micro

computer relies on menu selection techniques for data item

and command selection and data output. Form-filling and

question and answer dialogue techniques are used for data

input. A "help" key is providtl to allow users to obtain

guidance on all SDT commands and data elements. This "help"

capability can be activated at any time during the operation

of the SDT by pressing the Escape key on the APPLE keyboard.

1.5.1.5 Generation of Data for SDT

In order to provide a capability for early training

requirements estimation, the SDT will be used to describe

system elements during the earliest phases of the

acquisition process. To generate data during these early

*phases, comparability analysis procedures, which are part of

the ETES Training Estimation Aids/Procedures, will be

employed.

During the early phases of the system acquisition, when only

information on a system's functional requirements is

available, comparability analysis techniques will be used to

identify existing subsystems which are similar to those of

the new system. Historical data for these subsystems will

then be collected and modified to (1) meet the differential

characteristics of the new system and (2) correct any

inherent deficiencies in the historical data base. By

* utilizing design and task data from comparable, existing

systems, meaningful early estimates of training requirements

can be made when only functional information on the

projected system is available (see Figure 1-4). Later, as

I- 14
6

z.'< - z 

~Z L

- - 0.

C CL

0 cnS

~< Z

0 Cd UJ f-.~.- .. W '

~~In

CAWLZ~l I_
actual design concepts are developed, comparability analysis

can be used to develop estimates of tasks and training

program elements. Still later, when the actual system tasks

are available, only the training program elements must be

estimated.

Thus, by adding comparability analysis procedures to SDT

data base management capabilities, the SDT will be capable

of (1) describing alternative system concepts during the

earliest phases of the acquisition process, (2) describing

projected system elements, (3) relating alternative system

concepts to a common framework so that meaningful

comparisons can be made, and (4) refining system information

as more accurate and more detailed data are developed.

1.5.1.6 Application of SDT in the System Acquisition

d Process

In its initial application to a particular weapon system

development process, the SDT can be used to describe the

system functional requirements which are generated during

functional analysis. These requirements specify the

functions which must be performed if the system is to

satisfy its designated mission need. The SDT can be

employed in a functional analysis as soon as the need for a

particular system has been specified. Formally, this

activity occurs after the approval of the requirements

document at Milestone 0, which initiates the Conceptual

phase of the acquisition process. However, the SDT could

probably be used to describe functional requirements even

prior to Milestone 0 if alternative system concepts were

identified earlier.

1-16

. . . . . S|. .. . .. .. i l l ... I
Once the functional requirements for a system have been

developed and described via the SDT, the SDT can be used to

generate and describe system designs. These designs specify

possible mechanisms for performing the desired functions.

These mechanisms include equipment, personnel, and software.

Once developed, the system designs can also be described

with the SDT.

Once the mechanisms for accomplishing the functions have

been identified in the design concepts, the human tasks

which must be performed to utilize the system designs can be

specified. -These tasks, which are the key building blocks of

training development, can be documented in the SDT. After

the tasks are identified and specified in the SDT, training

estimation aids and procedures can be used to determine

training program elements, estimate training resources, and

develop training products. The resulting training program

and its associated resources can then be documented in the

SDT.

The SDT, like the other components of ETES, is primarily

designed for applications during the Concept Exploration

phase of the acquisition process, which runs from Milestone

0 to Milestone 1. However, the ETES can also be used during

subsequent phases of the acquisition process. The primary

uses of ETES during these later phases are to (1) make more

detailed estimates of task and training resource requirements, (2) determine the impact of subsequent design changes

on task and training requirements via the data base management capabilities of the SDT, and (3) conduct trade-off

studies of proposed solutions to identified training

problems.

1-17

.... .. S... . . .. .. . . ..
1.5.2 Training Estimation Aids/Procedures (TEAP)

The Training Estimations Aids/Procedures are an integrated

set of procedures and automated aids for performing six key

early training estimation functions: (1) Functional Requirements Analysis - Systematic description of the functions

which the system must perform and, where necessary, estimation of the hardware/software design concepts needed to

achieve these functions, (2) Task Generation - identification of the tasks required to operate or maintain the

system, (3) Training Program Estimation - estimation of

where and how the tasks should be trained, (4) Training

Resource Estimation - estimation of the training resources

needed to implement the training program, (5) Training Cost

Estimation, and (6) Training Efficiency/"Effectiveness"

Estimation - estimation of the adequacy with which the

training program can be expected to train personnel.

A listing of the ETES Training Estimation Aids and

procedures is contained in Figure 1-5.

The TEAP includes new techniques as well as existing

methodologies from other ARI research projects such as the

Training Efficiency Estimation Model (Jorgensen, Kubala, and

Atlas, 1981), recent work on cost and training effectiveness

analysis by Dawdy, Chapman, and Frederickson (1981), and the

Hardware/Procurement - Military Manpower (HARDMAN) methodology (O'Brien, 1983; Mannle, 1981). These existing methodologies were modified to meet the specific requirements of

early training estimation. Brief descriptions of the

Training Estimation Aids and Procedures follow.

1-18
zoo z 02 

F II

mm

oi~

U.K&

*z 4

4..

M, s

Lei

.o

LL

V 00

<..

* I I I~I>

In Q 014

Z __ "O

1193

S0
1.5.2.1 Functional Requirements Analysis

In order to estimate training requirements for a new system,

the training developer must have information on the system's

hardware/software design, and its functional requirements.

Unfortunately, this information is often not available to

training analysts during the early phases of the acquisition

process. Consequently, it is often the case that no

systematic estimates of training requirements are developed

during these phases. The TEAP is designed to assist the

training analyst in overcoming these problems by providing

systematic techniques for estimating the system's functional

requirements and hardware/software design during the early

phases of system acquisition. The TEAP procedures for

estimating system functions, and hardware/software design

are at a general level. They are designed to provide the

minimum amount of information needed for early training

estimation. As the development of the system progresses,

and actual information on system functional requirements and

hardware/software design are developed, these system

elements no longer need *to be estimated by ETES procedures

but can be obtained directly from the combat developer

(functional requirements) or materiel developer (hardware/

software design and manpower requirements).

ETES functional requirements analysis procedures provide a

description of the information which should be generated

during the functional requirements analysis and the steps

that one must go through to generate these elements.

Early estimates of system hardware/software design are

generated via comparability analysis. Quite simply,

comparability analysis involves (1) identifying comparable

1-20
existing systems, (2) collecting data on the comparable

systems, and (3) modifying these data to reflect the

differences between the comparable existing system(s) and

the new system.

The most -;cent version of the DoD standard on Logistics

Support Analysis (LSA) (MIL-STD-1388) identifies comparability analysis as the preferred method for estimating key

system elements during the early phases of the acquisition

process.

Identification of comparable equipment in ETES takes place

in three major phases: (1) identification of the

Predecessor equipment system(s), (2) identification of the

Baseline Comparison System equipment system, and (3)

identification of the New equipment system.3 The BCS is

constructed by (a) adding subsystems to the Predecessor

subsystem to reflect additional capabilities required in the

New system (i.e., the system under development), and (b)

subtracting subsystems to reflect capabilities no longer

required in the New system. The BCS provides a baseline for

comparing the New system training requirements to other

similar systems. The BCS concept is directly congruent

with the new version of MIL-STD-1388 which specifically

calls for the construction of a BCS during the early phases

of the acquisition process.

3 The procedures described in this section were derived

from the HARDMAN Methodology. A description of the HARDMAN

Methodology is presented in Appendix D.

1-21
1.5.2.2 Task Generation Procedures

In the present version of ETES, comparability analysis is

used as the principal means of estimating task requirements.

In this approach, task data for the comparable existing

system(s) are collected and modified to reflect the

differences in design and/or employment between the New and

comparable system.

The task generation process begins with the collection of

task data for the equipments and MOSs in the BCS system. The

BCS is composed of the existing subsystems (including the

Predecessor sutsystems) which come closest to meeting the

New system functional requirements. The primary data source

for existing task data is the Soldier's Manual since it, by

definition, contains all tasks associated with an MOS. The

task data from existing systems are updated to reflect any

equipment or doctrinal changes that were made since the

source was published.

Once the BCS task data has been collected and updated, this

data is examined and compared to the New system hardware/

software design and functional requirements. BCS tasks

associated with equipments or functions which are no longer

needed are eliminated. New tasks are added to reflect new

equipment or functions. Descriptions of new tasks are

developed in accordance with existing Instructional Systems

Development (ISD) guidelines.

Identification of New system operator tasks is facilitated

by examining the subsystem functions identified during the

functional requirements analysis.

1-22
By adding and deleting BCS tasks, a New system task list can

be constructed. The reason for each deletion/addition is

documented using a systematic set of modification codes.

It is possible that a New subsystem may require the same

task as a BCS subsystem but that the essential characteristics of this task must be changed to reflect the New

system requirements. For example, the same task may be

required for both the BCS and New system but the frequency

with which the task is performed may differ.

To account for these changes to essential task characteristics, all tasks with major modifications are identified

and the reason for each task modification is documented.

Once tasks have been identified, their conditions and

standards can be developed using existing ISD procedures.

These same procedures are used to assess the adequacy of any

new task descriptions which are generated during comparability analysis.

1.5.2.3 Training Program Estimation4

Algorithms are provided to assist the training analyst in

determining the tasks to be trained and assigning these

tasks to training settings. To provide input to these

algorithms, tasks are rated on a series of task

characteristics (e.a., frequency, learning difficulty).

Several different algorithms are provided to meet the needs

of different phases of the acquisition process.

The current version of ETES only contains procedures

for estimating training programs for individual

institutional training courses.

1-23
Quasi-programs of instruction (QPOI) are constructed by (1)

modifying or deleting modules from existing courses to

reflect the task deletions and modifications made during

task generation and (2) adding modules to reflect the unique

requirements of the New system. As part of the QPOI

construction process, the instructional methods and

curriculum hours which must be devoted to each module are

determined.

Media for the training program are selected by the

application of an automated aid, the Media Selection and

Efficiency Estimation Program. This program is an extension

of the Training Efficiency Estimation Model (TEEM) produced

by Jorgensen et al. (1981). The Media Selection and

Efficiency Estimation Program significantly expands the

capabilities of TEEM by recasting media selection as a

dynamic programming problem and automating these procedures

on the Apple III microcomputer. This automated program

permits the user to employ the SDT to input and store the

data needed to feed the media selection procedures. Using

dynamic programming techniques, the program can assign tasks

to media in a manner that optimizes efficiency, relative

cost, or efficiency and relative cost.

Efficiency is determined by comparing the stimulus,

response, and feedback characteristics of the individual

task to the stimulus, response, and feedback characteristics

of potential media categories. More specifically, a score

is calculated which describes the match between media and

task characteristics. Efficiency for each task-media

combination is calculated by dividing this score by the

maximum match that may be achieved for the task.

1-24
Total efficiency -for a set of tasks is determined by

aggregating the efficiency score for individual tasks. An

additional efficiency measure can be calculated by weighting

the efficiency of each task by the task criticality score.

This task criticality score is calculated by aggregating the

task factors typically used in selecting tasks for training

(e.g., task frequency, percent members performing, task

delay tolerance, etc.).

1.5.2.4 Estimation of Training Resources

These procedures estimate the training resources needed to

implement the training program. The training resources

encompassed by these procedures include (a) the number of

students to be trained, (b) number of instructors and

support personnel, (c) facilities requirements, (d) training

device and training equipment requirements, and (e)

ammunition requirements. Included among the procedures are

techniques for using off-the-shelf automated worksheet

software (e.g., VisiCalc) for storing and applying several

of the resource determination algorithms.

1.5.2.5 Estimation of Training Costs

These procedures estimate the costs of the resource

requirements and aggregate these costs into a total course

cost. The procedures include techniques for using off-theshelf automated worksheet software to calculate course

costs. The procedures also describe how to use modified data

from comparable existing courses to assist in the cost

estimation process.

1-25
1.5.2.6 Training Efficiency/Effectiveness Estimation

Procedures are provided for determining the training

efficiency of selected aspects of the training program

(e.g., media). Training efficiency is defined as a measure

of the extent to which the characteristics of the training

program element match the task characteristics of the New

system. For example, media training efficiency is

determined by comparing the stimulus, response, and feedback

characteristics of the tasks to the stimulus, response, and

feedback characteristics of potential media categories. An

additional variant of this efficiency measure may be

obtained by weighting each task by its "criticality" where

criticality is determined by aggregating weighted scores of

key task characteristics (e.g., frequency, consequences of

inadequate performance).

Training "effectiveness" is determined by estimates obtained

from subject matter experts. More specifically, a group of

subject matter experts is presented with the following

information for each task: (a) the target population

description of the personnel who will perform the task, (b)

a description of the task including its associated

conditions, job performance standards, and general skills

and knowledges, (c) a description of the training program or

training program elements (e.g., course length, methods,

media) that will be used to train the task, and (d) the

criterion which must be achieved for the task (if different

from the job performance measure). Each subject matter

expert is then asked to estimate the expected percentage of

0

1-26

Si
soldiers in the target population who will pass the

5

criterion given that training program.

It should be noted that the ETES TEAP is designed to

estimate training requirements for a New system. The TEAP

is not designed to provide techniques for actually

developing instructional materials or programs (e.g.,

instructor packages, student packages, training literature,

training devices). Thus, the TEAP will assist training

analysts in determining what instructional

materials/programs should be produced, what the content of

these materials/programs should be, and in estimating the

cost of these materials/programs.

This focus on training requirements estimation rather than

on training product development is one of the unique

characteristics of the TEAP. This focus on training requirements distinguishes it from other existing training development methodologies such as the Army's Instructional Systems

Development (ISD) process and the Army Research Institute's

Training Developer's Decision Aid (Hawley, 1979).

The training requirements produced by the TEAP provide

front-end information needed for the early planning and

analysis of training programs. The training requirements

information produced by the TEAP provides the foundation for

the actual construction of training products in ISD and

5 It is recognized that this measure of effectiveness,

may differ from other conceptualizations of training

effectiveness. It is also recognizcd that this approach

only provides a gross measure of "effectiveness" that should

only be used during the earliest phases of the acquisition

process.

1-27

-
other related methodologies. Techniques and methodologies

which can be used to develop training products are provided

in TRADOC Pam 350-30, Schulz and Farrell (1980), Hawley

(1979), and Fink (1981). An overview of the relationship

between ISD and ETES is provided in Figure 1-6.

1.5.3 Evaluative Technology

The Evaluative Technology is an integrated set of procedures

and automated tools for (1) developing figures-of-merit for

assessing the integrated impacts of the training requirements developed in the Training Estimation Aids/Procedures,

(2) identifying potential problem areas for system training

and the likely sources of these problems, (3) identifying/

evaluating training problems, (4) developing trainingrelated input to key acquisition documents, and (5)

determining/evaluating training development schedules (see

Figure 1-7). A summary of the procedures in each of these

areas is provided below.

1.5.3.1 Development of Figures-of-Merit

Procedures are included for identifying figures-of-merit

which summarize the essential features of the training

requirements. Eight potential training figures-of-merit are

utilized including (1) cost, (2) training efficiency, (3)

training "effectiveness", (4) congruence with training

development guidelines, (5) congruence with program

requirements, (6) training complexity, (7) feasibility, and

(8) summary evaluation score which aggregates the scores on

the most critical figures-of-merit (cost effectiveness, and

complexity). This summary evaluation score provides a

global measure of the "goodness" of a training program.

1-28
ISD BLOCKS IN EACH OF THE FIVE ISD PHASES

THE BLOCKS IN EACH PHASE ARE:

EE1.2 CONSTRUCT NAYZE ISELECT

PHASE I ANALYZE AS/ LECT JOB JO TPK,' rXERF=ORM~ANCE • INST~rN J"RUCTIONAL HO- FUNCTIONS MEASURES COURSES SEYT G

x x x x x

wp _. 1__ U.2~ ~ DlbE SC R IB E 

E:MN PHASE i 1 I DEVELOP T ESE ENTRY SEOUENCE&

-OLEIvES -- TESTS BEHAvIoR ST.JCTURE

x x

m .2 m~~i SECIFY mm 1 SP=ECIFY INSTRUCTION =,3IWS~.£ I 41 * Sf 

PHASE III LFMI1RIG MANAGEMENT ReVIEWSELECT DEVELOP VALIDATE r EVENTSI PLAN & E XISTING INSTRION INSTRUCTION

ACTIVITIES DELIVERY MATERIALS

X SYSTEM X

PHASE IV 0"SRC IMPLEMENT I 2 IONL CNO

P MANAGEMENT V INSTRUCTION

I .2

CONDUCT •CONDUCT 2=.3

REVISE

PHASE V - INTERNAL •EXTERNAL SYSTIM

EVALUATION EVALUATION I

X = Function Covered by ETES

FIGURE 1-6. RELATIONSHIP BETWEEN ISD AND ETES

1-29
02

fA

0

1-30
1.5.3.2 Identification of Problem Areas

J

Procedures are included for identifying training problem

areas. These problem areas consist of the Army Military

Occupational Specialties (MOSs) which have high figure-ofmerit scores relative to (1) the Predecessor system, and/or

(2) the other MOSs in the New system. Procedures are also

provided for identifying the courses, equipments, and tasks

which contribute to the high figures-of-merit and

identifying the training program elements which are likely

causes of the high figure-of-merit scores.

1.5.3.3 Identification and Evaluation of Alternatives

Guidelines are included for (1) identifying training

alternatives which can address the training problems, (2)

evaluating the training alternatives through selected

*reapplication of TEAP and Evaluative Technology procedures,

(3) conducting sensitivity analyses of key parameters, and

(4) assessing the impact of non-training system changes

(e.g., hardware/software changes) on training.

1.5.3.4 Development and Evaluation of Training

Schedules/Plans

Construction of training development schedules for emerging

systems is a difficult process. Over 100 developmental

events are listed in TRADOC Reg 351-9, the Army regulation

governing training plan development. The sequential

relationships among these events are complex and are not

described in any systematic and integrated manner in TRADOC

Reg 351-9. To assist users in developing training

schedules, procedures are provided for using automated, off0

1-31

-6 I i i l I I I I I I.. . . . I i l l . . .... . . . . .
jthe-shelf scheduling software (e.g., VisiSchedule) to track

and monitor the training development schedule. By using

this software, the training developer can quickly and

efficiently respond to changes in the training development

schedule. Use of off-the-shelf scheduling scftware is

facilitated by the inclusion of an input data diskette which

(a) describes the events in the training development process

(as specified in TRADOC Reg 351-9), (2) describes the

temporal/sequential relationships among these events and key

acquisition milestones, and (3) lists the expected duration

of these events for a "typical" major Army weapons system.

This data diskette significantly reduces data input

requirements. In addition, it eliminates the need for an

analysis of the complex sequential relationships among

training development events which are either implicitly or

explicitly specified in TRADOC 351-9.

1.5.3.5 Develop Inputs to Acquisition Processes and

Documents

Procedures are provided for using ETES products to provide

inputs to the major Army acquisition processes and documents

related to early training estimation. These documents and

processes are summarized in Table 1-2.

1.6 KEY CONCEPTS UNDERLYING ETES

This section provides a description of several of the key

concepts underlying the ETES. Section 1.6.1 describes

comparability analysis, which is a systematic set of

procedures for using data from comparable existing systems

to estimate requirements for new systems. Comparability

analysis is particularly appropriate during the early phases

of the acquisition process.

1-32
I

Table 1-2. Major Army Acquisition Processes

and Documents Related to Early

Training Estimation

• Cost and Training Effectiveness Analysis (CTEA)

. Outline Individual and Collective Training Plan (OICTP/ICTP)

. Logistics Support Analysis (LSA)

. Operational Testing (OT)

. Training Device Requirements, Documents, and Processes

- Training Device Letter of Agreement (TDLOA)

- Training Device Requirements (TDR)

- Training Device Letter Requirement (TDRL)

- Training Device Study (TDS)

. New Equipment Training

. System Requirements/Documents

- Justification for Major System New Start (JMSNS)

- Letter of Agreement (LOA)

. Request for Proposal (RFP) Development/Evaluation

. Personnel Documents/Processes

- Tentative Quantitative and Qualitative Personnel

Requirements Information (TQQPRI)

- Integrated Personnel Summary (IPS)

. System Level Documents/Processes

- Concept Formulation Package (CFP)

- Cost and Operational Effectiveness Analyses (COEA)

- Tradeoff Determination (TOD)

- Best Technical Approach (BTA)

- Decision Coordinating Paper (DCP)

- Defense System Acquisition Review Council (DSARC)

- Ary System Acquisition Review Council (ASARC) 4

. Mission Area Analysis

1-33
Section 1.6 describes the differences between steady state

and phased resource requirements estimation. Section 1.6.3

describes the limitations of the current ETES.

1.6.1 Comparability Analysis

ETES is designed to determine training requirements during

the earliest phases of the acquisition process. However,

during these early phases detailii data on system equipment

and key training elements (e.g., tasks) are typically

unavailable. This poses an initial problem, because without

such data, estimates of training requirements cannot be

made. To circumvent this problem, comparability analysis

can be used to identify these key system elements. Quite

simply, comparability analysis involves (1) identifying

comparable existing systems, (2) collecting data on the

comparable systems, and (3) modifying these data to reflect

the differences between the comparable existing system(s)

and the New system.

The most recent version of the DoD standard on Logistics

Support Analysis (LSA) (MIL-STD-1388) identifies comparability analysis as the preferred method for estimating key

system elements during the early phases of the acquisition

process.

An overview of the basic process that is employed in comparability analysis procedures is outlined in Figure 1-8. The

process begins by determining if the necessary system

element (e.g., New system equipment configuration, tasks)

has already been developed. If this system element does not

exist, a comparable existing system element is identified.

A comparable existing system element is a system element

1-34

0
SYSTEM YE

ELEMENT

SCOMPARABILITY "17 N

ANALYSES ,

I IDENTIFY BEST

COMPARABLE

EXISTING

SYSTEMIS)

ELEMENT

COLLECT DATA FROM

COMPARABLE

SYSTEMIS)

II

I MODIFY DATA TO REFLECT

INHE RENT

RINACCURACIES IN IN DATA

NORMALIZE DATA

TO REFLECT SCENARIO

IAND OPERATIONAL CONCEPT

DIFFERENCES II

S MODIFY DATA TO

REFLECT DIFFERENCES BETWEEN COMPARABLE

EXISTING SYSTEM

AND NEW SYSTEM IjI

FORMAT

RESULTS

FOR

ANALYSIS

Figure 1-8. Overview of Comparability Analysis Procedures.

1-35

S
which performs the same function as the New system element.

There may be more than one comparable system element. In

these cases, the "best" comparable existing system is

selected. The "best" comparable element is the system

element which (1) has performance requirements of the New

system, (2) has available data, and (3) has been used in an

enviroTLment which closely matches the scenario, organizational, and operational and support concepts of the proposed

New system.

Once the comparable existing system element is identified,

the data associated with that element is collected. This

data is then modified to reflect any inherent inaccuracies

associated with the data base(s) from which the data was

collected. The data is also adjusted to reflect differences

between the comparable system and the New system in terms of

scenarios and operational concepts. In the final step in

the comparability analysis process, data on the existing

system is modified to reflect the differences between the

functional capabilities of the existing system element and

the functional requirements of the New system element.

The comparability analysis process outlined in Figure 1-8

may be used to identify functional requirements, the system

hardware/software design, system manpower requirements, and

system task and skill requirements. It may also be used to

provide high level estimates of system training prograin

requirements, training resources, and training costs.

The key applications of comparability analysis is in the

identification of comparable equipment which occurs in TEAP

Function 2.0.

1-36
A more detailed description of the procedures used to

identify comparable equipment is presented in the subsection

which follows.

1.6.1.1 Identification of Comparable Equipment6

Identification of comparable equipment takes place in three

major phases: (1) identification of the Predecessor

equipment system(s), (2) identification of the BCS equipment

system, and (3) identification of the New equipment

system. The BCS system is constructed by (a) adding

subsystems to the Predecessor subsystem to reflect

additional capabilities required in the New system (i.e.,

the system under development), and (b) subtracting

subsystems to reflect capabilities no longer required in the

New system. The BCS provides a baseline for comparing the

New system training requirements to training requirements

for other similar systems. The BCS concept is directly

congruent with the new version of MIL-STD-1388 which specifically calls for the construction of a BCS during the early

phases of the acquisition process.

Predecessor Equipment System(s). The Predecessor

equipment system is the system(s) which is currently

performing the mission(s) which will eventually be performed

by the New system. Thus, the Predecessor equipment system

will be replaced by the New system. By definition, the

Predecessor equipment is currently in the DoD inventory. In

some cases, a Predecessor equipment system may not exist.

6 The procedures described in this section were derived

from the HARDMAN Methodology. A description of the HARDMAN

Methodology is presented in Appendix D.

1-37

Iq
BCS Equipment System. The BCS equipment system is a

notional design composed of the existing subsystems which

most closely match the functional requirements of the New

system. The BCS equipment system is designed to meet each

of the general functional requirements required by the New

system. However, since it is composed solely of existing

subsystems from the DoD inventory, it will not fully meet

all of the performance goals associated with the New system

functions.

In order to be selected for the BCS equipment system, a

subsystem must meet the following criteria: (1) it must be

an existing subsystem currently in the DoD inventory

(preferably in the Army inventory), (2) it must be the

existing subsystem which most closely meets system performance requirements, and (3) it must have mature task data

(the latter set of data is critical for the training

requirements determination).

New Equipment System. The new equipment system is an

integrated system which is designed to fully meet all system

performance requirements. During the early phases of the

acquisition process, prior to the receipt of the contractor's design concept, a notional New equipment system is

constructed by (1) estimating the new technologies that must

be added to their BCS equipment subsystems so that the new

system performance requirements can be fully met, and (2)

modifying the BCS equipment data to reflect the addition of

these technologies.

During the later phases of the acquisition process, the

contractor's design(s) can be used as the New system(s).

Thus, while there is one BCS equipment system, there may be

1-38
several different New equipment systems during the development of any one weapon system.

1.6.1.2 Development of Predecessor, BCS, and New Systems

Training Subsystem

Predecessor, BCS, and New training subsystems, (i.e., a

system which includes task training programs and training

resources and costs) may be constructed to correspond the

Predecessor, BCS and New equipment systems. The Predecessor,

BCS, and New training subsystems may be defined as follows.

Predecessor System. The predecessor training subsystem

is composed of the tasks, and training programs, associated

with the predecessor equipment and the training resources

and cost associated with these training programs.

Baseline Comparison System (BCS). The BCS training

subsystem contains tasks, training programs, and training

resources and costs associated with the BCS equipment. Since

the BCS equipment subsystems are existing subsystems, the

training elements of these subsystems can also be directly

obtained from existing data. However, BCS task and training

data may be modified to reflect (a) any inherent inaccuracies in the data bases from which it is derived, and (b)

differences between the BCS subsystem scenarios and operational concepts and the New system scenarios and operational

concepts which may impact task requirements or training

resource usage.

New System. The training subsystem for the New system

contains the tasks, training programs, and training

resources and costs associated with the New system equipment. During the early phases of the acquisition process,

New system task requirements are derived by modifying the

1-39
BCS task requirements to reflect the design differences

between the BCS and New equipment systems. During the later

phases of the acquisition process, task data may be supplied

by the contractor(s) or cognizant government agencies.

Once task requirements have been deter-mined, it is possible

to apply a wide range of analytical techniques to determine

the training program requirements and training resources and

costs for the New system.

1.6.2 Phased Versus Steady-State Training Resource

Requirements

In estimatina training resource requirements for a

developing system, it is important to distinguish between

phased and steady-state resource requirements. Phased

resource requirements (e.g., NET resource requirements) are

the specific resources that are required while the weapon

system is being placed into the inventory, or while the

weapon system is being removed from the inventory (the

latter case is generally not relevant during system

development). Phased resource requirements are displayed by

calendar year. In order to determine these phased resource

requirements during system development, information on the

number of weapon systems introduced into the field in each

0 xlendar year must be available and/or estimated.

Steady-state resource requirements are the estimated average

yearly resource requirements that are needed to maintain the

weapon systems in the field once all of the systems have

been fielded. To estimate steady-state resource requirements, all that is required is an estimate of the total

number of systems that will be in the field once system

1-40

. .... .. . . . . .. ..
installation has been completed. Steady-state requirements

are also presented in yearly units.

ETES provides the capability for estimating both steadystate and phased MPT requirements. This is accomplished by

first estimating the steady-state requirements and then

modifying these requirements to meet the specific phased

requirements that are required to install the system. This

approach differs from the usual approach of developing the

phased requirements first and then modifying the phased

requirements to produce the steady-state requirements. The

ETES approach (steady-state, then phased requirements) has

an obvious advantage in that it starts with the end objectives (steady-state requirements) and then identifies the

intermediate products (phased requirements) needed to meet

those objectives.

1.6.3 Limitations of Current ETES

The current version of ETES has the following limitations: 7

0 It does not estimate training programs for

individual unit training or collective training.

0 It does not estimate acquisition-related training

costs.

These limitations are expected to be addressed in followon research conducted by the Army Research Institute. A

more detailed description of the limitations of the current

version of ETES is provided in the ETES Final Report.

1-41
o It provides relatively crude estimates of training

effectiveness which are only appropriate for the

very earliest phases of the acquisition process.

1.7 GUIDELINES FOR MANAGING ETES APPLICATION

The section describes general guidelines for managing an

ETES application. Guidelines are provided in two key

areas: data management and study plan construction.

1.7.1 ETES Data Management

To successfully manage the data associated with an

application of ETES the user must; (1) track and monitor all

input data and data products associated with each ETES

procedure, (2) establish and maintain an audit trail of the

products produced during each procedure, and (3) store

d relevant input and output data in the System Description

Technology (SDT) data base management system.

An overview of the elements in the ETES data management

process is provided in Figure 1-9. A more detailed

description of these elements is provided below.

Raw Input Data. The raw input data needed to implement

ETES procedures may come in a variety of different forms

(including hard-copy documents, interviews, mag."-tic tapes,

magnetic discs, and direct on-line data transmissions).

Data Source Index. The data source index is a

systematic table describing the source from which each of

the raw input data were obtained. The data sources in the

data source index are grouped according to major functional

1-42
~L6

CLC

LLLL

um

IL

1-433

S W 
data categories. An example of a page from a data source

index is presented in Table 1-3. A complete data source

index listing most of the major data sources that will be

employed in an ETES application is provided in Appendix F.

ETES data is obtained from toth system-specific sources

(i.e., data bases directly related to the system under

investigation) and nonsystem-specific sources (i.e., data

bases containing data on a wide range of systems). The data

source index can be manual. However, it should be automated

to facilitate construction and subsequent update.

Library Index. The library index is a systematic

document describing and cataloging (a) each raw input data

element and (b) each ETES worksheet. The worksheets

describe the intermediate products of ETES analytical

procedures. The descriptions in the library index include

standard bibliographic information on the input data. They

also indicate exactly where the input data or worksheet is

currently located (e.g., who has the document, worksheet,

etc.). Table 1-4 also provides an excerpt from a page in a

library index.

Extraction Procedures. The raw data from existing Army

data sources may not always be in a form which is appropriate for ETES procedures. In these cases, systematic

procedures for extracting the required data elements must be

developed as part of the data management process. The

specific extraction procedures that are employed will vary

depending on the data base and the particular focus of the

ETES application. Specific details on data extraction

procedures must be obtained from the organization controlling the data base. The organizations associated with the

major data sources are listed in Appendix E.

1-44
0

a' EC1 (n C)-- W(nM

inc 4r) m U .- U .

E- E- . U4J CL CC

a ( r- (l CN4

ii r- M C) Clv m c.

o -4 -C .. zC *-c - 14 4

M-- w4~t (a4- -. O> N ww

4-.E-. mcCC r O1 1c

C) u

0 Co

-4"-

E< CN-4 -

2 Im z 

cc-4-0 0~

0 04C 0a2

E

-- 4

a0 > 0Nlt

- -, 0 -l

1o E- m a

U-~Q 0)r~ -- -40f r_ r -

E. UC u CO ,' 3 0 r

E 0H0 0c M0--a

>l (n0O 0 E ) m4- 0 C

<0 0X

14.1

4--~C)C 41-45
Table 1-4. Example Page from Library Index.

Date Entered: ETES Procedure:

Data Input No.: Data Output No.:

Worksheet No.: Responsible Agency:

Reference:

Document Type:

Journal: Tape:

Article: Disc:

Book: Comp. Cards:

Report: Direct Comp. Link:

Other-Print: Other:

1-46
SDT. The SDT provides a system-specific data base for

describing the basic system elements which are required to

apply ETES procedures. The system-specific data base

includes both initial input data and output products from

ETES procedures, as well as changes to these elemencs which

are made during subsequent applications of the procedures.

The SDT contains the key data elements needed to characterize a developing system with an emphasis on the task and

training components of the system. The SDT is congruent

with the system description provided in the Logistics

Support Analysis Record.

Audit Trail Procedures. The ETES audit trail

procedures provide a systematic mechanism for (a) tracking

the development of training requirements and (b) monitoring

changes to the training requirements. The audit trail

procedures permit another analyst to replicate and/or

validate the study results. The audit trail procedures

docunent the data, procedures, and assumptions underlying

each step of the methodology.

An overview of the audit trail procedures is outlined in

Figure 1-10. First, as data is obtained, its source and

major features must be systematically documented in the data

source index and library index. Furthermore, as data is

distributed among analysts, its location must be tracked in

the library index.

After being collected and entered in the SDT, the data will

be analyzed in the ETES procedures. As each procedure is

ccnpleted, the analytical steps in these procedures should

be documented in the ETES worksheets. A description of each

worksheet and its current location should also be entered

into the library index.

1-47

. . . . . . 0, . .. . . . . . . . .. , ,
0U

<C

0 0

LU'

uj 0

C-) x A

x L

1-48-
As the system progresses, there will be frequent changes to

its data elements. These changes should be documented in

the SDT.

Worksheets. Worksheets are forms designed to describe

the intermediary results of the analytical steps and

algorithms in the ETES procedures. An example of a

worksheet is presented in Table 1-5. To assist the analyst

in monitoring the audit trail, each worksheet should be

assigned a unique identification number describing (1) the

ETES function in which it is applied, (2) the equipment with

which it is associated, (3) the MOS/ASI/skilI

level/duty position with which it is associated, and/or (4)

the course with which it is associated. A listing of the

ETES worksheets is presented in Table 1-6.

1.7.2 ETES Study Plan Development

Prior to the application of the ETES TEAP and Evaluative

Technology, a systematic study plan must be established. A

potential outline for an ETES study plan is provided in

Table 1-7. There are four major questions which have to be

asked during the development of a study plan:

(1) What resources will be available to conduct the

ETES application?

(2) What is the scope of the study (i.e., what ETES

procedures will be applied)?

(3) What is the focus of the study (i.e. on what

elements of the system will the ETES procedures be

applied)?

1-49
a 9 .~s *~

~ '~ ~ ____

o

J

-

C

NO o I - e *

i~ *~ ~ Io c. - ~ N o~ ~~* 1 -CI -

Ca'

E C ~IU * C - '..

w

'CIt T

~. 111 JR

o- CS

C

-. * Ca, -

LI,

4IC'

C

CE ~ - 2

C ~J 0

C

* C

S - a ~ U

C C

C a,',

U I C -, C C 3

~Ii~ I -

'a

- - -

p LZ --

I-50
Table 1-6. List of ETES Worksheets.

ETES Study Plan Outline (OUT)

Acquisition Information (ACQ)

Mission Profile (MIS)

System Performance Specification (PRF)

Organizational, Operational, and Support Concepts (OSC)

Operational Environment (ENV)

System Performance Specification (PRF)

Generic System (GEN)

Function Sequence (FSQ)

Baseline Comparison Equipment Selection (EQP)

BCS Equipment Description (BCS)

Baseline Comparison Performance Shortfall (PSH)

Design Difference (DD)

New System Equipment Description (EQP)

Candidate MOS (CND)

BCS Task Association (BCST)

New System Task Generation (NTA)

DIF Data Collection (DIF)

Task Data (PSYCRIT)

Select Tasks for Training (TSEL)

Media Assignment (MAS)

Media Selection Planning (MSEL)

Data Summary for Training Setting Assignment (SUMSET)

Skill and Knowledge (SKILL)

Target Population (POP)

Task/Skill Sequence (SEQ)

OPOI - Parts 1 & 2 (QPOI-1, QPOI-2)

Course Modification (MOD)

Training Path (PATH)

Number of Students - Steady-State (NSSS)

Phased Course Input Requirements (PHASE)

Number of Students to be Trained (NSTUD)

Number of Instructors (NINS)

Determine Training Facilities (DFAC)

Facilities Requirements (NFAC)

Determine Number of Training Devices/Equipments (TDEV)

Training Device/Equipment (DEEQ)

1-51
Table 1-6. List of ETES Worksheets (Continued).

Determine Number of Media and Instructional Aids (NMED)

Number of Media/Aids (MEDSUM)

Ammunition Requirements (AMREQ)

Ammunition Requirements Summary Sheet (AMSUM)

Operating and Support Plan (OPSP)

Media Usage Rate (MUS)

Individual Course Cost (ICOST)

Course Aggregation (ACOST)

Training Summary (TVICE)

Method Rating (METH)

Task Rating for Instructional Methods (TAMOD)

Calculate Method Efficiency (METHEFF)

Calculate Overall Effectiveness Score (EFF)

1-52
TABLE 1-7 ETES STUDY PLAN OUTLINE (OUT)

I. Available Resources and Costs

Labor FY FY FY FY FY

Category 1

Category 2

Category 3

Category 4

Category 5

Total

*in person-months

Costs

Category 1

Category 2

Category 3

Category 4

Category 5

Total

Computer Resources

Category I

Category 2

Category 3_

Category 4

Category 5

Total

1-53
Table 1--7 Study Plan (continued)

III. Study Focus

0System Functions to be included

General

Function Operation Maintenance ___Support

All Functions All Maint Functions All Sppt Fun.

Subfunctions _________________________

System Equipments to be included

All__Equipments _____________ ___________

1-54
* Table 1-7 (continued)

* Figures-of-Merit (FOM)

- Cost

- Efficiency

- Effectiveness - Congruence with Training Development Guidelines

- Congruence with Program Requirements

- Training Complexity

- Training Capacity

- Feasibility

- Impact on System Performance - Summary Evaluation Score (Place Check Beside FOM

Included in Score)

Teap Procedures to be Applied

Function Subfunction Procedures

* ____ ~~1.12_ _ _ _ _ _ _ _ _ _ _ _ _ _ _

___ 1.23______ ________________

____ 1.5 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_____ 1.6 _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _

_ _ _ 1.7 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ 2.1 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 2.2__ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 2.3 _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _

6.1.1 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 6.1.2 _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _

* _____ ~6.1.3 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 6.1.4 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 6.2.1 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 6.2.2 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

* _____ ~6.2.3 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

6.2.4 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 6.2.5 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 6.2.6 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

* ~~~~6.2.7__ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 6.3.1 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 6. 3.2 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

0 1-55
Table 1-7 (continued)

Evaluative Technology Procedures to be Applied

Function Subfunction Procedures

_ _ _ 7.1.1 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 7.1.2 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 7.1.3__ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 7.1.4__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 7.1.5 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 7.1.6__ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 7.1.7 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 7.1.8__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 7.2 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 7.3.1 _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 7.3.2 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 7.4.1 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 7.4.2 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 7.4.3__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 7.4.4 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

7.6.1 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 7.6.2__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 7.6.3 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 7.6.4__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 7.6.5__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

__ 7.6.6 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 7.6.7 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 7.6.8 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ 7.6.9 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

____ 7.6.10 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

1-56
(4) What data must be collected to support the study?

An overview of the procedures for developing the study plan

is shown in Figure I-l. The analyst must first determine

the total amount of resources available to conduct the

analysis (see Table 1-7). The resource and cost elements

which are considered include labor, computer resources, and

total available budget. The available resources place

constraints on what can be accomplished in any particular

ETES application.

For some functions, ETES provides alternative procedures for

accomplishing several key training estimation functions.

Resource constraints may place limits on which of these

procedures the analyst may employ. Generally, the less data

intensive procedures are the least costly. The analyst must

select the procedure which fits his/her "pocketbook".

After the study resources have been determined the scope of

the analysis, or more specifically, the specific ETES

functions and procedures to be applied during the proposed

application must be determined. ETES is an iterative set of

procedures which can be applied repeatedly during the acquisition process. However, it may not be feasible to apply

all of the ETES procedures during each application. Indeed,

there is a logical sequence for applying the ETES procedures, (see Table 1-8). Thus, as can be seen in Table 1-8, a

logical starting point for the application of the ETES would

be to identify system functional requirements. If additional

resources are available, the user would proceed to identify

the system hardware/software design, and system task

requirements and then to estimate training programs and

paths, etc.

1-57

' .. . . . . . I I . . . . . I I 1 1 . .. . .. I I I
STUDY PLAN

OUTLINE

/RESOURCESREO CS

IDNTF STUD

FOU

Figue Ii. COnstRcigaStdEln

PLA WIT8
Table 1-8. Logi:al Sequence for Applying

ETES Functions/Procedures

Function Sequence Grouping

1.0 Conduct Functional Requirements Analysis 1

2.0 Generate Tasks 2

3.0 Estimate Training Program 3

4.0 Estimate Training Resource 4

5.0 Determine Training Cost 4

6.0 Estimate Training Efficiency/Effectiveness 5

7.0 Evaluate Training Program 6

8.0 Develop Input to Acquisition Process 7

1-59
Following the identification of the study scope the study

focus must be determined. Study focus refers to the level

of detail to which the individual ETES procedures will be

applied. Specification of the focus requires identifying

(a) system functions to be examined, (b) system equipments

to be examined, (c) MOS/duty positions to be examined, (d)

types of training to be examined, and (e) figures-of-merit

to be used in evaluating the training programs. Eight

figures-of-merit are used in ETES: (1) cost, (2) training

efficiency, (3) training effectiveness, (4) congruence with

training development guidelines, (5) congruence with program

requirements, (6) training complexity, (7) training

capacity, and (8) a summary evaluating score combining the

cost-effectiveness, and complexity measures.

The last step in the development of the study plan is the

construction of the data source index. The data source

index lists the major categories of data elements which must

be collected during the study and the data sources from

which these elements can be obtained (see Section 1.7.1 for

a more detailed description of the data source index and its

role in overall ETES data management). A generic ETES data

source index encompassing all ETES functions is provided in

Appendix E.

An overview of the procedures for constructing a ETES data

source index is presented in Figure 3-12. The user begins

the construction of the data source index by examining the

data categories listed in the generic data source index in

Appendix E. Using the information from the ETES study plan,

the user then selects the generic data elements needed to

support the particular ETES functions to be applied. The

user subsequently identifies lower level data elements by

1-60

S I III I I H II - I .. ..... .
GENERIC DATA

SOURCE INDEX

i4

INDEX

IDENTIFY

ETES PROCEDURES

TO BE APPLIED

STUDY

CDENTIFY SYSTEM

ELEMENTS TO

WHICH ETES

PROCEDURES WILL

BE APPLIED

CONSTRUCT DATA

SOURCE INDEXb

FOR ETES

APPLICATION

Figure 1-12. Procedures for Developing Data Source Index.

1-61
£ examining the system elements, functions, equipments and

MOS/duty positions to which the procedures will be applied.

These system elements are described in the description of

the study focus of the ETES study plan.

As soon as the data source index is constructed, the user

should begin to identify interface requirements for all

manual and automated data bases which must be accessed

during the ETES application (see Figure 1-12). Users should

contact the relevant POCs for each data source to obtain upto-date versions of their standard operating procedures

(SOPs) for data requests. (Since SOPs change frequently,

users should make sure they have up-to-date SOPs from the

cognizant organizations.)

1.8 FORMAT USED TO DESCRIBE ETES PROCEDURES

The remaining sections of this guide (Sections 1-8) provided

detailed descriptions of the ETES procedures. Table 1-9

lists the procedures which are presented in each section.

As one would expect from a training-oriented system, ETES

procedures for identifying training-related produces

(Procedures 2-8) are described in more detailed than the

procedures for identifying functional requirements and other

related information (Procedure 1).

1-62
Table 1-9. Organization of Remaining Sections

1.0 Conduct Functional Requirements Analysis

1.1 Identify Required System Functions

1.2 Identify System Performance Measures and Goals

1.3 Allocate Required System Functions and Identify

Lower Level Functions

1.4 Establish Baseline Comparison System

1.5 Identify Required System Improvements/New Technologies

1.6 Format New System Description

2.0 Generate Tasks

2.1 Identify, Collect and Format Data

2.2 Develop BCS Tasks

2.3 Assign BCS Tasks to MOS, Duty Position and Skill Level

2.4 Develop New System Task List

2.5 Assign New System Tasks to MOS, Duty Position and

Skill Level

2.0 Estimate Training Program

3.1 Select Tasks for Training

3.2 Assign Tasks to Training Settings

3.3 Identify Skills and Knowledges

3.4 Develop Target Population Description

3.5 Sequence Tasks/Skills for Training

3.6 Construct Quasi-Program of Instructions

3.7 Assign Tasks/Skills to Media

3.8 Construct Training Paths

4.0 Estimate Training Resource

4.1 Develop Course Operating and Support Plan

4.2 Determine Number of Students to be Trained

4.3 Determine Instructor Requirements

4.4 Determine Facilities Requirements

4.5 Determine Training Device and Training Equipment

Requirements

4.6 Determine Requirements for Other Training Resources

5.0 Determine Training Cost

5.1 Estimate Individual Course Costs

5.2 Aggregate Course Costs

6.0 Estimate Training Efficiency/Effectiveness

6.1 Estimate Training Efficiency

6.2 Estimate Training Effectiveness

1-63

.. . .. i . *.. . . ii i i i i 6 
Table 1-9 (Continued)

7.0 Evaluate Training Program

7.1 Identify/Construct Figures of Merit

7.2 Identify Likely Problem Areas

7.3 Identify Likely Causes

7.4 Identify Alternatives

7.5 Evaluate Alternatives

7.6 Assess Impact of System Changes

8.0 Develop Input to Acquisition Process

8.1 Develop/Monitor Training Development Schedule

8.2 Develop Inputs to Acquisition Processes and

Documents

1-64

0
SECTION 1.0 - CONDUCT FUNCTIONAL REQUIREMENTS ANALYSIS

OVERVIEW

During this procedure, system functional requirements are

determined and functions are allocated to man, machine, and

software. To assist in the identification of manpower,

personnel, and training requirements, design concepts for a

Baseline Comparison System (BCS) and New system are

established.

The results of the functional analysis are entered into the

System Description Technology (SDT). The SDT serves as the

repository for information and data developed during

functional analysis and insures that subsequent analyses of

MPT requirements will be based on a common set of data.

PROCEDURE

This procedure consists of six lower level procedures.

During the first procedure, high level system functional

requirements are identified. In the second procedure,

performance measures and goals are constructed for these

functions. In the third procedure, lower level system

0 functions are identified. In the fourth procedure, the

Baseline Comparison System (BCS) is constructed. In the

fifth procedure; required system improvements and new

technologies are identified. In the sixth procedure, these

new technologies are integrated into a systematic design

concept for the New system.

1-1
0

The last two procedures are not necessary if a New system

design concept is available from the Program Office.

However, relevant information on the New system should be

entered into the SDT using the guidelines contained in

Procedures 1.5.3 and 1.6.3.

Potential data sources for functional requirements analysis

procedures are listed in Appendix E.

1.1 IDENTIFY REQUIRED SYSTEM FUNCTIONS AND DETERMINE MPT

STUDY REQUIREMENTS

OVERVIEW

0

The first step in conducting system functional analysis is

to identify system functional requirements. This step is

initiated by obtaining and reviewing any available

functional requirements information on the New system such

as that contained in the Justification for Major System New

Start (JMSNS) and threat studies as well as similar

information on the predecessor or other comparable systems.

This information is used to identify and describe system

missions and to generate high level functions and

subfunctions.

PROCEDURE

An overview of procedure 1.1 is provided in Figure 1-1.

1.1.1 Identify Acquisition Characteristics

During the earliest phases of the acquisition process,

information on system functional requirements can be

1-2

,0 l , .. . . . . . .... .. . . . . . . .
JWNSMA~ Pf-RMSYTMS

IDOCUTIFT

IDOENTY

DACIB T M

MISIN

FIGURE 1-1 Overview DiagrM: dniyRqie

System Functins1Nd DterineMP

Study RequiROEns(.)

DETEMIN

HIHLEE
obtained from the Directorate of Combat Developments (DCD)

I at the TRADOC School associated with the developing

system. Later, after a Program Manager (PM) has been

appointed, this information can be obtained from the Program

Office. The best source of information for functional

requirements information during the early phases of a new

acquisition is the Justification of a Major System New Start

(JMSNS), formerly called a Mission Element Needs Statement

(MENS). The purpose of the JMSNS is to identify a new

threat and the requirement for a new weapon system. The

JMSNS also explains the system's general operational and

support envelope (as required in Office of Management and

Budget (OMB) Circular A-109, DoD Direction 5000.1, and DoD

Instruction 5000.2). An outline of the information elements

contained in the JMSNS is presented in Table 1-1.

* Table 1-1. Outline of Information Elements

Contained in JMSNS

o Defense Guidance Element

o Mission and Threat

o Alternative Concepts

o Technology Involved

o Funding Implications

o Constraints

o Acquisition Strategy

* The JMSNS must be supplemented with additional information

such as feasibility or technology base studies, operation

and support plans, mission area analysis results or other

requirements documents. The availability of this kin

1-4

. . . .. . . . . .I0UI III,. . . .
documentation depends on the phase of the acquisition

process the system has reached. Examples of documents which

may contain information on general weapon system require-

* ' ments are provided in Table 1-2. The documents listed in

Table 1-2 should be available from (a) the Directorate of

Combat Developments prior to the establishment of a PM or

(b) the weapon system Program Management Office (PMO).

Table 1-2. List of Program Documents

For a New System

o Justification for Major System New Start (JMSNS)

o Mission Area Analysis (MAA)

o Technology base studies

O Army "86" and "90" studies

o Cost and Operational Effectiveness Analyses (COEA)

o TRADOC Scenarios

o Concept Analysis Agency Studies

o Battlefield Development Plan (BDP)

o Scenario Oriented Recurring Evaluation System

(SCORES)

0 Threat studies by OACSI/INSCOM

o Operational and Organizational Concepts

o Analysis of Technological Opportunities by TRADOC/

DARCOM

o Operation and Support Plans

Following the collection of available program documentation,

a list of points of contact for obtaining additional system

and acquisition program information should be developed.

1-5

,,-7ZZ
m-."".

L_. -.- ,.- -

Identification of points-of-contact begins with the

collection of up-to-date organizational charts for the PMO,

DCD, and major contractors (if a contractor has been

*--"selected). Points of contact are then interviewed (either

in person or by phone) to obtain additional system information, program documentation, or further points of contact.

The weapon system requirements documentation will often list

acquisition objectives and constraints which are not related

to the performance of the system (e.g., cost and schedule

constraints). These objectives and constraints may impact

MPT requirements. The system acquisition schedule is of

particular concern in this regard. The rate at which new

systems move through the acquisition process and are fielded

directly impacts the requirements for manpower, personnel,

and training resources. MPT requirements can be developed

for either steady state (constant number of systems

operating per year) or phased situations (accounting for

installation/retirement schedules). Each situation requires

different types of acquisition schedule information. All

J acquisition objectives and constraints including the acquisition schedule should be documented on an Acquisition

Information Worksheet (ACQ). Figure 1-2 contains a sample

ACQ worksheet.

J..-

1.1.2 Identify/Describe System Missions

This step involves developing a mission profile/operational

mode summary which contains a list of the New System

mission(s) and the corresponding operational requirements.

The mission profile consists of two parts; (1) a narrative

. description of potential New system missions and (2)

quantitative data describing the operational requirements of

1-6
-~ -,-~-... - - - .E

OIZ ~ ~ N

C. - ~

1o -

I I -r

cc ..-

.~c 0

22 c

0 0

U1-7
- o.

%b .° . .

those missions. A worksheet for documenting mission profile

data is presented in Figure 1-3.

Mission profile source documents may be obtained from the

Directorate of Combat Developments (DCD) associated with the

New system or the system program office. However, if this

information is not available, the mission profile must be

estimated via comparability analyses utilizing mission

information for the predecessor or other comparable systems

(procedures for identifying comparable systems are described

in the Introduction and in Procedure 1.4).

Program documentation is used to identify and match New

system missions (columns c and d) with predecessor/

comparable system missions (columns a and b). If information on the New system functions is already available,

columns (a) and (b) may be left blank. Columns (e) and (f)

describing the system functions should be filled in when

Procedure 1.1.3 (Determine High Level Functions) is

completed.

Quantitative data describing each mission function should be

entered into columns (g) thru (k). These columns contain

information on:

o percent operating time (column g),

o annual number of missions (column h),

O annual operating days (column i),

O mean duration (column j), and

o operating requirements for new system, in terms of

key performance measures (e.g., miles driven per

year) (column k).

4', 1-8

'" - • , ..'- * . °.* .... . ...- . .-. .- .-.-.... -. -...-.-.-. ... .,'-. '-..-. ."... .. -. . .... " .'. ... .',,, .,... .v.. . ," ,,
Figure 1-3 Mission

System DSWS Wodcsheet

Procedure 1.1

% PredecessorlComparable System New System

Mission Mission Mission MsgnCwcaitc Name Characteristics Name*Mno hreitc

ia) (b) (c)

- J Table 1-3 Table 1-3

* Provide Fire Short/Medium Range Provide Long/Short Range tire for purposes of:

Support fire for purposes of Fire Support (1) target engagements

(1) target engagements (2) counterfire

(2) counterfire (3) air defense suppression

(4) Interdiction/deep fires

Survive In a Periodic movement Survive in a Rapid and frequent movement to confuse enemy

Battlefield Secondary armament Battlefield counterfire targeting ability and markc forc

* Environment Environment Defend against anticipated NBC environment

secondary armament

Sustain Weapon Field resupply capability Sustain Weapon Provide for continuous resupply of ammunitic

Operations Operations

*Fill maintenance capability Field maintenance capability

Tactical Positioning Rapid and frequent movement over relatively

distances while maintaining continuous firt

operations

*1I9
ofile Worksheet (MISI # 1 of 1

*SDT Input/Output

Major System

Functions Required Quantitative Data

(d) by Mission

* Functio

Function Functibn (f) Pecn Operating Time (g) Number of annual Annual Operating Mean Duationl Operating Req (k) Number___ (________)___ Missions (h) Days (i) ___________________

diminish

strecngth

hoc t

-suppot
If quantitative mission data is not available for the New

system, this information should be estimated by using data

from the predecessor or other comparable systems.

1.1.3 Determine High-Level Functions

In this step, major system functions are identified and a

function hierarchy is developed. Major system functions

should be listed in the program documentation identified in

Procedure i.1.2. These major functions form the framework

for the system functional hierarchy. These major functions

are then developed into a hierarchy, and listed in the first

two columns of the System Performance Specification worksheet (PRF), displayed in Figure 1-4 (the remaining columns

in this worksheet are completed in Procedures 1.2 and

1.3). The entire system functional hierarchy goes beyond

identification of the major functions; it is a hierarchical

listing of the functions which the new system must perform

from the most general functional level (e.g., fire weapon at

target) down to the level where specific subsystem functions

(e.g., locate targets) are identified. Identification of

functions below the subsystem level is accomplished in

Procedure 1.3. High level functions should describe what

the system must do not how it will do it. Examples of high

level functions are presented in Figure 1-4.

Descriptions of system functions are likely to be available

in the program documentation. However, program documentation and information are often incomplete and do not

provide a systematic listing of system functions. Often, for

instance, the program information focuses on those functions

which are associated with potential improvements of the

existing system. This treatment ignores existing functions

i-11

P4 . C .
T T I O

0

o- o w MN0 u

0, 00~

9u I

* 8 ~41 -.

o to

-CI 3c r C

634. 63

CA ~ .

0414 411 1-12
E0

0

0 u

U) 96 0 U

E

.3 6

I03U .

00 c 

ul * J

J.

W) _______________________ 1-13_ 
10 c

0

EE

'i E -E-E

m (n

Cc c) 2

;I .- r- C.

1-14

It L0
07

which still must be performed by the system but not necessarily in an improved manner. Therefore, the program

information may have to be supplemented in order to develop

".- a complete set of hierarchical system functions. To accom-

" plish this, the user may also have to obtain functional

information on the predecessor system and/or other existing

systems which perform functions similar to those required of

the New system. More details on the identification of

predecessor and comparable system data and the identification of high level functions are provided in the

paragraphs below.

O IDENTIFICATION OF PREDECESSOR AND OTHE COMPARABLE

EXISTING SYSTEMS

The Predecessor system (that is, the system which the new

system is designed to replace) should be identified in the

requirements documents identified during Procedure 1.1.

Comparable existing systems are identified in a two-step

process. First, the functional area associated with the New

system is identified. A list of functional areas and the

major systems within these functional areas is provided in

Table 1-3. Second, the existing system within the functional area which most closely matches the requirements of

the New system is identified. If the user is not familiar

with the systems in the functional area, he/she should

contact subject experts from the school within the designated functional areas to obtain assistance in identifying

comparable existing systems.

1-15

I" ' . ' . k ' ' / ' ' - ..... .... "' ..... * " '" .. , " " "I ' "" " .... ' . "
TABLE 1-3. SYSTEMS AND FUNCTIONAL AREA PR0P1,)ENTS*

PROPONENT SYSTEMS AND AREAS

US Army Air System Peculiar Equipment:

Defense School HAWK Missile (Operator/Organizational

maintenance)

HERCULES Missile (Operator/Organizational

maintenance)

CHAPARRAL Missile (Operator/Organizational

maintenance)

REDEYE Missile (Operator/Organizational

maintenance)

ROLAND Missile (Operator/Organizational

maintenance)

VULCAN Gun (Operator/Organizational

maintenance)

M42 (40mm Self-Propelled)

Command and Control Radar Systems

NA/TSQ-73 (Organizational, DS/GS

maintenance)

AN/TSQ-38 (Organizational, DS/GS

maintenance)

FAAR (Organizational maintenance)

PATRIOT (Operator/Organizational

maintenance)

DIVA-) Gun (Operator/Organizational

maintenance)

Functional Areas:

Air Defense System Employment and Operations

US Army Armor System fuliar Equipment:

School Techniques of Firing:

Submachinegun (M3Al)

Coaxial Machinegun

M85-50 Machinegun

Armored Vehicle Operations

XMI Tank

M551 (Armored Reconnaissance Vehicle)

M557 (Command Post Vehicle)

M578 (Light Recovery Vehicle)

M88 (Tank Recovery Vehicle)

Calvary Fighting Vehicle (CFV)

* Taken from TRADOC Cir 351-1. Common Job and Task Management.

L1-16
TABLE 1-3. SYSTEMS AND FUNCTIONAL AREA PROPONENTS (continued)

PROPONENT SYSTEMS AND AREAS

US Army Armor Functional Areas:

-. School (Cont) Combat Reconnaissance

US Army Chaplain Functional Areas:

Center and School Religious Activities

"" ' Pastoral Counseling

US Army Engineer System Peculiar Equipment:

School Prime Power Generation Equipment

Combat Engineer Vehicle (CEV)

Armored Vehicle Launched Bridge (AVLB)

Engineer Construction Equipment

Engineer Support Equipment

Tactical Utility/Precise Power Electric

Generation Equipment (Operator/Crew,

DS/GS maintenance)

Topographic/Cartographic Equipment

Welding Equipment - ARC (DS/GS

maintenance)

Night Vision Material (other than

Common Thermal) (DS/GS maintenance)

Functional Areas:

Mine/countermine Operations

Military Bridging

Map Reading

Camouflage Operations

Demolitions

Field Fortifications

Route Reconnaissance

Obstacles

Fire Protection/Fire Safety

Refrigeration, Air Conditioning, Heating

Environmental Protection

US Army Field System Peculiar Equipment:

Artillery School Cannons (Operator maintenance)

LANCE Missile (Organizational

*; maintenance)

PERSHING Missile (Organizational

maintenance)

Field Artillery Target Acquisition

systems

e.

' ' ' '''' '" " . ' o", "''''" ,'.' "1,-7 ". ' '"" ,,'". . " ' ' '" "," . % ,
TABLE 1-3. SYSTEMS AND FUNCTIONAL AREA PROPONENTS (continued)

PROPONENT SYSTEMS AND AREAS

US Army Field Functional Areas:

Artillery School Observed Fire

(Cont)

US Army Infantry System Peculiar Equipment:

School Rifles

Machineguns

Grenade Launcher M203

Rocket Launcher M2OAl, M20B1

HEAT Rockets

Flame Weapons

Weapons Oriented Night Vision Devices

PVS-2

PVS-5

TVS-2

TVS-4

Anti-Tank Weapons

TOW (Operator maintenance)

DRAGON (Operator maintenance)

LAW

106 RCLR

90 RCLR

Hand Grenades

Infantry Fighting Vehicle (IFV)

MII3Al APC

M114 APC

Improved TOW Vehicle (ITV)

Functional Areas:

Operations

Ranger Operations

Physical Fitness

Visual Signals

Leadership

Anti-Armor Warfare

Jungle Operations

Air Mobile Operations

Air Assault Operations

Airborne Operations

Land Navigation

Security and Combat Patrols

Early Warning Devices

1-18
TABLE 1-3. SYSTEMS AND FUNCTIONAL AREA PROPONENTS (continued)

PROPONENT SYSTEMS AND AREAS

US Army Intelligence System Peculiar Equipment:

School (Devens) Electronic Warfare/Signal Intelligence

(EW/SIGINT) Analysis

EW/SIGINT Non-Morse Interception

EW/SIGINT Morse Interception

EW/SIGINT Voice Interception

EW/SIGINT Emitter Identification/Location

EW/SIGINT Noncommunications Interception

EW/INTERCEPT Equipment Repair

Signal Security

" US Army Intelligence Functional Areas:

Center and School Imagery Intelligence

(Huachuca) Interrogation

Ground Surveillance Radar Operations

Remote Sensor Operations

Combat Area Surveillance Radar Repair

Aerial Surveillance Sensor Maintenance

Aircraft Survivability Equipment Repair

Intelligence Analysis

Counter-Intelligence

Aerial Sensor Operations

Safeguarding Classified Documents

(except storage, receipt, and transfer)

Intelligence

US Army Military Functional Areas:

Police School Civil Disturbances

Physical Security

Rear Area Protection

Crime Prevention

Criminal Investigation

Techniques of Firing:

.45 Caliber Pistol

.38 Caliber Pistol

Prisoner of War

US Army Missile System Peculiar Equipment:

and Munitions LANCE Missile (DS/GS maintenance)

Center & School PERSHING Missile (DS/GS maintenance)

HAWK Missile (DS/GS maintenance)

HERCULES Missile (DS/GS maintenance)

1-19

..............................- .. ~
TABLE 1-3. SYSTEMS AND FUNCTIONAL AREA PROPONENTS (continued)

PROPONENT SYSTEMS AND AREAS

US Army Missile CHAPARRAL Missile (DS/GS maintenance)

and Munitions REDEYE Missile (DS/GS maintenance)

Center & School ROLAND Missile (DS/GS maintenance)

(Cont) PATRIOT Missile (DS/GS maintenance)

DIVAD Gun (DS/GS maintenance)

VULCAN Gun (DS/GS maintenance)

SHILLELAGH (DS/GS maintenance)

TOW (DS/GS maintenance)

DRAGON (DS/GS maintenance)

Forward Area Alerting Radar (FAAR)

(DS/GS maintenance)

LCSS

Functional Areas:

Munitions Handling, Storage, Inspection,

and Maintenance

Explosive Ordnance Disposal & Equipment

Missile Test Measurement and Diagnostic

Ecuipment

Aviation Life Support Equipment

(Pyrotechnics-R&D, storage, and issue)

Calibration and Related Support Equipment

US Army Ordnance System Peculiar Equipment:

and Chemical Center Wheeled Vehicles (maintenance)

and School Tracked Vehicles (maintenance)

Power Generating Equipment (maintenance)

Materials Handling Equipment (maintenance)

Small Arms (maintenance)

Artillery Maintenance (Non-Missile)

Ground Support Equipment (maintenance)

Cannons (DS/GS maintenance)

Chemical Equipment (maintenance)

Turret Maintenance

Recovery (Less Aircraft/Watercraft)

Compressors (Shop) (Organizational/DS

maintenance)

Laser - General Purpose (Organizational/

DS maintenance)

Laundry Equipment (OrganizatJonal/DS

maintenance)

Metal Working and Machining Equipment

(Organizational/DS maintenance)

Tire Repair Equipment (DS maintenance)

* Welding Equipment - ARC (Organizational

maintenance)

Welding Equipment (Other) (Organizational/

DS maintenance)

1-20

Zi * _ _ -7 
TABLE 1-3. SYSTEMS AND FUNCTIONAL AREA PROPONENTS (continued)

I.4

PROPONENT SYSTEMS AND AREAS

US Army Ordnance Functional Areas:

and Chemical Center NBC Defense

and School (Cont) SMOKE Equipment and Employment

NBC Defensive Materiel

Chemical Support Equipment

(Organizational/DS maintenance)

US Army System Peculiar Equipment:

Quartermaster School Supply (Automated & Manual)

Food Service

Functional Areas:

Airdrop Equipment Supply and Rigging

Exchange, Textile, and Leather Support

V.- Fabric Material and Repair

Subsistence and Food Service Equipment

Memorial Activities

Laundry, Clothing Exchange, and Bath

Operations

Materials Handling Equipment (Operation)

Office Machines and Repair

Organization Maintenance and Storage

of Small Arms

Packaging, Packing, and Crating

Petroleum Supply, Equipment, and

Operations

Procurement and Production

Property Disposal

Shelters (General Purpose - Soft Wall)

J. .. Storage

Supply and Supply Management Operations

(Except: Medical, Ammunition, ADP,

and Cryptographic)

The Army Maintenance Management Systems

('rAM~S)

Materiel Management 4.a.

, US Army Signal System Peculiar Equipment:

School Radio/Cable

Functional Areas:

Audiovisual Equipment

Antennas

Message Processing

TMDE

4%

#4

1-21
.%4

TABLE 1-3. SYSTEMS AND FUNCTIONAL AREA PROPONENTS (continued)

S. PROPONENT SYSTEMS AND AREAS

US Army Signal Functional Areas:

School (Cont) Television Equipment

TV Production

ADP Equipment

ADP Peripheral Equipment

Data Communication Equipment

Avionic Navigation and Flight

Control Equipment

Avionic Communications Equipment

Tactical FM Radios

Manual Switchboards

C-E Maintenance Administration

Electronic Switchboards

Telephones

Generators (Signal)

COMSEC Equipment

Patching Panels

Converters

Central Office Equipment

Multiplexers

Combat C-E Planning

C-E Engineering

Power Supplies

System/Technical Control Equipment

Printed Circuit Boards

HF Radios

Public Affairs

Radio Set Control Groups

Electronic Switchboards

Facsimile Equipment

Fixed Telecommunications Systems Planning

Photographic Equipment

Watercraft Navigational Aids

Land Navigation Positioning Equipment

Ground Control Approach Equipment

Illustration

Multichannel Equipment

Message Processing

Material Management (for Signal Equipment)

Radio Wire Integration (RWI)

Radiotelephone Procedures

Radio Teletypewriter Equipment (RATT)

Signal Security

Signal Records and Reports

Specialized C-E Skills

Strategic Microwave Equipment

1-22
TABLE 1-3. SYSTEMS AND FUNCTIONAL AREA PROPONENTS (continued)

PROPONENT SYSTEMS AND AREAS

US Army Signal Strategic Satellite Equipment

School (Cont) Specialized Signal Equipment

Teletypewriter Equipment

Tactical Satellite Equipment

Tactical Microwave Equipment

Tabulating Equipment

Wire and Cable Communications/Operations

US Army Functional Areas:

Transportation Wheeled Vehicle Operations (Except

School Special Use Equipment)

Aviation Maintenance

4Marine Operations and Maintenance

Rail Operations and Maintenance

US Army Institute Functional Areas:

of Military Assistance Civil Affairs

Survival, Evasion, Escape, and

Resistance (SEER)

Psychological Operations

Special Forces Operations

Security Assistance

Foreign Area Specialty

Low Intensity Conflict (LIC)

Civil Military Operations

US Army Participation in Counter

Terrorist Operations

Defense Against Enemy Psychological

Operations

* US Army Admin Center Functional Areas:

Administrative Services

Records and files management

Correspondence management

Routing of official mail

Micromation

Word processing

Forms management

Printing and reproduction

Publications supply

Command publications and orders

Postal operations

12

". 1-23

I"%

" "" '%"%" " ."%" ,* %''-"". ".."". .'.."".,-"-"". .-. ...
S ..- . .

.' TABLE 1-3. SYSTEMS AND FUNCTIONAL AREA PROPONENTS (continued)

PROPONENT SYSTEMS AND AREAS

US Army Admin Functional Areas:

Center (Cont) Storage, transfer, and control of

classified documents

Reports control

Unit mail operations

Article 15*

Summary courts Martial*

Special courts Martial*

Article 32 investigations*

Claims*

Court reporting*

Finance services

Military pay and allowances

Travel allowances

Commercial accounts

Civilian pay

Disbursing operations

JUMPS Army management

Comptrollership

Planning, programming and budgeting

Program review

Internal review (auditing)

Accounting systems and procedures

Management analysis of programs

Manpower management

.. Computer Science

ADP management

ADP systems analysis

Computer programming

ADP equipment operations

*. Personnel Management

Recruiting and retention

Personnel management techniques

Profesdional ethics

Club and restaurant management

Morale and recreation services

Personnel Services

SIDPERS

Casualty reporting

SOERs

*Enlisted tasks only

---- 1-24
TABLE 1-3. SYSTEMS AND FUNCTIONAL AREA PROPONENTS (continued)

PROPONENT SYSTEMS AND AREAS

US Army Admin EERs/SEERs

-" Center (Cont) Personnel assignments/utilization

X.A Personnel classification/reclassification

MOS classification/reclassification

Awards and decorations

Separations

Promotions/reductions

Personnel requisitioning/PERDDIMS

Personal affairs

Personnel records

VTAADS

Other

Equal opportunity

Personnel and financial management

information systems

Academy of Health Functional Areas:

Sciences, US Army First Aid

Lifesaving Measures

Medical Evacuation

Emergency Medical Care

Dental Care & Services

Veterinary Care & Services

Health Care Management, Patient Care,

Rehabilitation

Clinical Counseling

US Army Aviation System Peculiar Equipment:

Center and Fort Rucker Air Traffic Control

Flight Operation Coordination

Functional Areas:

Aviation Safety

Aviation Standardization

US Army Training Functional Areas:

Board Management of Military Training in Units

The Judge Advocate Functional Areas:

General School Military Justice

Geneva/Hague Convention and Laws of War

1-25

Q0/

".4. . .
. .

O IDENTIFICATION OF HIGH-LEVEL FUNCTIONS

High-level functions must be identified down to the

subsystem level (i.e., the level at which further

identification of functions is not possible without

allocating functions to people, hardware, or software).

Sources for the identification of system functions include

system doctrinal literature, system operational concepts,

mission area analyses, ARTEPs and the Battlefield Development Plan for the functional area involving the New system.

The Battlefield Development Plan contains the umbrella

operational concept for the functional area. A list of

doctrinal literature for each functional area is provided in

Table 1-4.

4"

Once a list of potential high level functions is identified,

these functions should be sorted and arranged in hierarchical order.

Each function should then be assigned a number which

specifies its place in the functional hierarchy. For

example, the highest level functions would be designated

1.1, 1.2, 1.3 or 2.1, 2.2, 2.3, etc. Lower level functions,

then, would be designated with 3 digits, etc.

When the identification of high level functions has been

c-mpleted the system function numbers titles should be

entered into columns (a) and (b) of the PRF worksheet (see

Figure 1-4).

.'

1-26
TABLE 1-4

DOCTRINAL LITERATURE PROGRAM PUBLICATIONS AND PROPONENTS

ACADEMY OF HEALTH SCIENCES

Publications Title Remarks

FM 8-10 Health Svc Spt in T.O.

FM 8-15 Health Svc in CZ

FM 8-21 Health Svc in Comm Zone

FM 8-25 Cbt Medic

FM 8-26 Dental Svc

FM 8-27 Vet Svc

FM 8-999C The Med Plt

FM 8-999D The Div Med Rn

FM 8-999F The Combat Zone Hospital

FM 8-999G Ground Evac Units

FM 8-999H Air Evac Units

SFM 8-9991 The Clearing Co

FM 8-999J C & C Combat Zone Health Svc Ops

FM 8-999K The MEDSON

FM 8-999L MED Intel

FM 8-999M Dispensary Detach

FM 8-999N The COMMZ Hospital

FM 8-9990 Convalescent Center

FM 8-999P Preventive Medicine Detach

FM 8-999Q C & C COMMZ Health Svc Ops

FM 8-999R Med Lab

FM (J) 8-8 Med Spt in Jt Ops TRADOC Approved

FM (J) 8-9 NATO Handbook on the Medical TRADOC Approved

Aspects of NBC Defensive Ops

AIR DEFENSE

FM 44-1 ADA Employment

FM 44-1-1 Air Defense Artillery Ops

FM 44-1-A ADA Materiel

FM 44-1-B ADA Materiel (Nike Herc)

FM 44-2 ADA Employment (Auto Wpns)

FM 44-3 ADA Employment, CV

FM 44-10 Roland AD Bn Ops

FM 44-11 DIVAD Gun ADA Bn Ops

FM 44-15 Patriot ADA Bn Ops

FM 44-18 Stinger Plt and Section Ops

FM 44-23 Redeye Employment

FM 44-70 C&C System AN/TSQ-73

FM 44-72 C&C of SHORAD Wpns

FM 44-90 ADA Employment, HAWK

FM 44-95 Nike/Herc Employment

1-27

' " ° " " • ' " °' - " " """ - °", "" "'° • ' " .i '..i- -' . ." .' ' .' i",

° -

." " ." ". €
TABLE 1-4

DOCTRINAL LITERATURE PROGRAM PUBLICATIONS AND PROPONENTS (continued)

Publications Title Remarks

ARMOR

FM 17-47 Air Cav Bde Combat Bde

FM 17-50 Attack Helo Ops

FM 17-95 Cavalry

FM 71-1 Tank & Mech Inf Co Team

FM 71-2 Tank & Mech Inf Bn Task Force

FM 90-3 Desert Ops

AVIATION

FM 1-2 Aircraft Battlefield Countermeasures and Survivability

FM 1-60 Army Airspace Mgt and Air

Traffic in a Combat Zone

FM 90-1 Employment of Army Aviation

Units in a High Threat Environment

TC 1-3 Army Aviation Employment on an

Integrated Battlefield

CHAPLAIN

FM 16-5 The Chaplain

FM 16-22 Conducting Military

Funerals and Memorials

CHEMICAL SCHOOL

FM 3-4 Collective Protection

FM 3-5 NBC Decontamination

FM 3-8 NBC Support Operations

FM 3-9 Military Chemistry & Biology

Defense

FM 3-10-1 Employment of Chemical Agents

FM 3-10-2 Chemical Effects Data

FM 3-10-3 Field Behavior of Chemical Agents

FM 3-12 Operational Aspects Radiological

Defense & Fallout Predictions

FM 3-15 Nuclear Accident Contamination

Control

FM 3-50 Deliberate Smoke Ops

FM 3-87 NBC Reconn/Decon Ops

FM 3-100 NBC Opns

41-28
'Jo.

TABLE 1-4

DOCTRINAL LITERATURE PROGRAM PUBLICATIONS AND PROPONENTS (conLiuued)

Publications Title Remarks

COMBINED ARMS CENTER

M 7-30 The Infantry Airborne and Air

Assault Bc. Ops

M 22-100 Military Leadership TRADOC Approved

M 22-101 Leadership Counseling TRADOC Approved

M 71-3 Armored & Mech Bde Ops

M 71-100 Armored & Mech Div Ops TRADOC Approved

M 71-101 Inf, AB and Air Assault Div Ops

M 90-2 Tactical Deception

M 90-6 Mountain Ops

M 90-10 Military Ops in Urbanized

Terrain

M 90-11 Winter Ops

M 90-13 River Crossing Ops

M 90-14 Rear Area Cbt Ops

M 100-1 The Army TRADOC Approved

100-5 5 Operations TRADOC Approved

M 100-15 CORPS TRADOC Approved

M 100-16 EAC TRADOC Approved

M 100-26 The Air/Ground Ops System

M (J) 100-27 USA/AF: Doctrine for Theater

Airlift and Jt Airborne Ops TRADOC Approved

M (J) 100-28 Doctrine and Procedures for

Airspace Control in CZ TRADOC Approved

M (J) 100-40 Armed Forces Doctrine for Chem

Warfare and Bio Wpns Defense TRADOC Approved

M (J) 100-42 USA/AF: Airspace M-t in AO TRADOC Approved

M (J) 100-43 The Landing Force TRADOC Approved

M 101-5 SOFM: Staff Org and Procedure

M 101-5-1 Operational Terms & Graphics

*,M 101-10-1 SOFM: Org, Tech, Log Data

M 101-10-2 SOFM: Org, Tech, Log Data

M 101-10-3(S) SOFM: Org, Tech, Log Data

COMMUNICATIONS COMMAND

M 11-23 Theater Army Commo Cmd

M 11-25 Signal Co Troposcatter

M 11-26 Signal Hq Ops Co, Signal Spt Co

M 11-27 Signal Cable Construction Bn

M 11-28 Signal Long Line Co

M 11-29 Sig Msg Co, Sig Radio Ops Co

1-29

• --
TABLE 1-4

DOCTRINAL LITERATURE PROGRAM PUBLICATIONS AND PROPONENTS (continued)

Publications Title Remarks

DEFENSE INFORMATION SCHOOL

FM 45-1 Public Affairs Ops TRADOC Approved

ENGINEER

FM 5-100 CBT Eng Ops

FM 5-101 Mobility

FM 5-102 Survivability

FM 5-103 Countermobility

FM 5-104 General Engineering

FM 5-105 Topographic Ops

FM 5-106 ADM Ops

FM 5-107 Engineer Combat Missions

* FM 5-108 Eng Special Ops

FIELD ARTILLERY

FM 6-1 TACFIRE Ops

FM 6-20 Fire Support in Combined

Arms Ops

FM 6-20-1 FA Cannon Bn

FM 6-20-2 DIVARTY, Corps FAS, FA Bde

FM 6-42 Lance Bn

FM 6-42-1(C) Lance Bn

FM 6-50 FA Cannon Btry

FM 6-60 MLRS

FM 6-121 FA Target Acquisition

TC 6-1-2 Battery Computer System

TC 6-20-3 Fire Support Operations in

Brigade Sized Units

TC 6-20-5 FASCAM

TC 6-20-7 FAC/FIST

TC 6-30-1 GLLD/Copperhead

TC 6-45-1 MLRS

TC 6-50-1 Direct Spt Cannon Bn

TC 6-50-2 FA Cannon Unit Nuclear Ops

1-30
TABLE 1-4

DOCTRINAL LITERATURE PROGRAM PUBLICATIONS AND PROPONENTS (continued)

Publications Title Remarks

S- ." INSTITUTE FOR MILITARY ASSISTANCE

FM 31-22 Command, Control and Spt of SF

Ops

FM 31-26 SF Selected Ops Tech

FM 33-1 PSYOPS

FM (J) 41-5 Joint Manual for CA TRADOC Approved

FM 41-10 Civil Affairs Ops TRADOC Approved

FM 100-20 Low Intensity Conflict

INFANTRY

FM 7-7 The Mechanized Infantry

Platoon/Squad

FM 7-7-1 The Mechanized Infantry Platoon

and Squad (IFV)

FM 7-8 The Infantry Platoon/Squad

FM 7-10 The Rifle Co

FM 7-20 The Infantry Battalion (Inf,

Abn, Air Asslt, Rgr)

FM 7-999A Tactical Employment of Mortars

FM 21-75 Combat Training of the Soldier

FM 31-18 Long Range Reconnaissance

Ranger Company

FM 57-38 Pathfinder Operations

FM 71-1 The Tank and Mechanized

Infantry Co Team

FM 71-2 The Tank and Mechanized

Infantry Battalion Task Force

FM 90-4 Airmobile Operations

FM 90-5 Jungle Operations

a FM 90-8 Counter-guerrilla Operations

FM 90-10-1 An Infantryman's Guide to

'Urban Combat

INTELLIGENCE - DEVENS

FM 34-3 Intel Analysis

FM 34-4 CEWI Commo & Data Clinics

FM 34-12 Collection and Jamming Co

FM 34-13 Plt Ldr/Tm Ch Hdbk

FM 34-23 TAC Exploitation Bn

1-31

4 . . . . . . . . . .
TABLE 1-4

DOCTRINAL LITERATURE PROGRAM PUBLICATIONS AND PROPONENTS (continued)

Publications Title Remarks

INTELLIGENCE - DEVENS (Cont)

FM 34-31 CEWI Co/Detachment

FM 34-32 CEWI Spt for ADA

FM 34-40 EW Ops

FM 34-41 Jamming Hdbk

FM 34-51 SIGINT/EW Templating

FM 34-62 SIGSEC Tech

FM 90-2A Electronic Deception

TC 34-41 Jamming Handbook

INTELLIGENCE - HUACHUCA

FM 34-1 Intel & EW Ops

FM 34-2 Mgt of Intel & EW Ops

FM 34-10 CBT EW and Intel Bn (Div)

FM 34-11 Ground Surveillance Co CEWI Bn

FM 34-14 Svc Spt Co CEWI Bn-Div

FM 34-20 CBT EW and Intel GP (Corps)

FM 34-21 Ops Bn, CEWI GP (Corps)

FM 34-22 Aerial Exploitation Bn, CEWI GP

FM 34-30 CEWI Co (ACR/SEP Bde)

FM 34-33 CEWI Spt to Arty

FM 34-52 Intel Interrogation

FM 34-53 Recon & Surveillance

FM 34-60 Tactical Counterintel Ops

FM 34-61 Counterintel Special Ops

FM 34-65 OPSEC Spt

FM 34-80 BN/Bde S2 Hdbk

FM (J) 34-81 Weather Intel in Spt of Army TRADOC Approved

Tactical Ops

TC 34-50 Recon/Surveillance Handbook

LOGISTICS CENTER

FM 29-20 Maintenance Mgt in T.O.

FM 31-82 Base Development

FM 54-6 Theater Army Area Cmd

FM 54-7 Theater Army Logistics

FM 63-1 CSS Ops-Sep Bde

FM 63-2 CSS Ops-Div

* FM 63-3 CSS Ops-Corps

FM 100-10 Combat Service Support TRADOC Approved

1-32

.°~
-7 47-7- -1

TABLE 1-4

DOCTRINAL LITERATURE PROGRAM PUBLICATIONS AND PROPONENTS (continued)

Publications Title Remarks

MISSILE AND MUNITIONS CENTER AND SCHOOL

FM 3-20 Tech Escort Ops

FM 3-21 Chem Accident Contamination Ctl

FM 9-6 Ammo Svc in the T.O.

FM 9-13 Ammo Hdbk

FM 9-15 Explosive Ord Disposal Unit Ops

FM 9-16 Explosive Ord Recon

FM 9-38 Conventional Ammo Unit Ops

FM 9-47(84) Special Ammo Unit Ops

FM 9-59 Missile Spt Unit Ops

% FM 29-27 Calibration Svc in the T.O.

MILITARY POLICE

FM 19-1 (Capstone) Operational Concepts of MP on

the Battlefield

FM 19-4 MP Team, Squad, Platoon Cbt Ops

FM 19-10 MP Operations

FM 19-15 Civil Disturbances

FM 19-16 Counterterrorism

FM 19-20 Law Enforcement Investigation

FM 19-30 Physical Security

FM 19-40 Enemy POW, Civilian, Internees

& Detained Persons

FM 19-60 Confinement and Correctional

Treatment of Military Personnel

NUCLEAR AND CHEMICAL AGENCY

FM (J) 101-31-1 SOFM: Nuke Wpns Employ TRADOC Approved

Doctrine and Procedures

FM (J) 101-31-2 SOFM: Nuke Wpns Employ TRADOC Approved

Effects Data

" FM (J) 101-31-3 SOFM: Nuke Wpns Employ TRADOC Approved

Effects Data

1-33

z7°
TABLE 1-4

DOCTRINAL LITERATURE PROGRAM PUBLICATIONS AND PROPONENTS (continued)

Publications Title Remarks

ORDNANCE CENTER

FM 29-2 Organizational Maintenance Ops

FM 29-23 Direct Spt Maintenance Ops

(nondivisional)

FM 29-24 General Spt Maintenance Ops

FM 29-30-1 Div Maintenance Bn

FM 29-35 Maint Spt in Separate Bde

FM 29-999 Division Maintenance Ops

FM 43-1 Org Maintenance Managers Guide

FM 43-1-1 Org Maintenance Managers Checklist

QUARTERMASTER

FM 10-27 General Supply in T.O.

FM 10-60 Subsistence Supply and

Management in T.O.

FM (J) 10-63 Handling of Deceased Personnel TRADOC Approved

in T.O.

FM 10-67 Petrol Supply in T.O.

FM 29-10 Supply Mgt in the Field Army

FM 29-19 Repair Parts Supply in T.O,

FM 29-45 GS S&S in the Field Army

FM 29-51 Div Supply Ops

FM 29-52 Supply and Field Svc Ops in Sep

Bde and Armored Cav Regiments

FM 29-999B Forward Supply Co

SCHOOL OF MUSIC

FM 12-50 The Military Band

SIGNAL

FM 11-40 TAC Audiovisual Doctrine

FM 11-44 ADA Signal Ops Bn

FM 11-50 CBT Commo w/i the Div

FM 11-92 CBT Commo w/i the Corps

FM 24-1 CBT Commo

FM 24-2 Radio Freq Mgt

FM 24-17 TAC Comm Ctr Ops

1-34

,-.' - , ':,, . *, . . . *.'. . *. , - . ._ , - ,- . .. . . . .," . . .. ,, ,.,, . ..
TABLE 1-4

DOCTRINAL LITERATURE PROGRAM PUBLICATIONS AND PROPONENTS (continued)

Publications Title Remarks

SIGNAL (Cont)

FM 24-18 Field Radio Tech

FM 24-20 Field Wire/Field Cable Tech

FM 24-21 TAC Multichannel Radio Commo Tech

FM 24-22 Commo Electronic Mgt System (CEMS)

FM 24-26 TAC Auto Switching (TTC-38)

FM 24-27 TAC Auto Switching (TTC-39)

FM 24-28 TAC Auto Msg Switching

FM 24-29 TAC Commo Center Facility & Ops

(TSQ-IlI)

FM 24-32 TAC Tele Sys Planning, Eng,

Integration and Control

FM 24-999 Cryptologistics Spt to the Army

in the Field

FM 32-30 EW, Tactics of Defense

TC 11-999E Camouflage of Signal Sites

TC 11-999G Airborne Radio Relay

TC 11-999H Unit Level Switching

TC 24-1 The CEOI

TC 24-3 Radio Wire Integration

TC 24-18 Commo in "The Come As You Are

War"

TC 32-11 How to get out of a Jam

SOLDIER SUPPORT CENTER

FM 12-3-1 Company/Bn Level P&A Doctrine

FM 12-3-2 DIV/SEP Bde Level P&A Doc

FM 12-3-3 CORPS Level P&A Doc

FM 12-3-4 EAC P&A Doc

FM 12-15 Wartime Casualty Reporting Sys

FM 12-16 Replacement Ops

FM 14-6 Comptroller/Finance Svc in T.O.

FM 14-7 Financial Services

FM 22-999A Unit Cohesion

FM 22-999B Human Performance in Continuous

Ops

FM 26-999A Mgt of Stress in Army Ops

1-35

-S.- - - - - . -) -. " .--- - - -"'- . , , . -' -- ".- - , . .• -. -". - . .- - , - . .-.
TABLE 1-4

DOCTRINAL LITERATURE PROGRAM PUBLICATIONS AND PROPONENTS (continued)

Publications Title Remarks

TRANSPORTATION SCHOOL

FM (J) 20-12 Amphib Embarkation TRADOC Approved

FM 29-39 Marine Equip Maintenance Mgt

FM 55-1 Army Trans Svc in T.O.

FM 55-2 Div Trans Ops

Procedures

FM (J) 55-12 Movement of Army Units in AF

Aircraft TRADOC Approved FM (J) 55-19 Spt of Contingency Forces

FM 55-20 Army Railway Units and Ops

FM 55-30 Army Motor Transport Units and Ops * FM 55-40 Cbt Svc Spt Air Transport Ops

FM 55-41 Aircraft Org Maintenance Mgt - ** FM 55-42 Army Aviation Intermediate

Maintenance (AVI )

FM 55-45 Aviation Spt Bn

FM 55-50 Army Water Transport Ops

FM 55-60 Army Terminal Ops

FM 55-999B LACV-30 Units and Ops

02 1-36

d ',4 e - . - . . * .- . - . .. °- - - --- - -

• 

-

-. * * 'V *V V-
., .

1.1.4 Document Data

...- Relevant data collected in this procedure must be entered

into the SDT. This data should be entered into the SDT

immediately after the worksheets contained in Figure 1-2,

1-3, and 1-4 are completed. Table 1-5 describes the

-.0.., sequence with which these data should be entered into the

SSDT. Additional guidelines for entering data into the SDT

* is provided in the SDT User's Guide.

1.2 IDENTIFY SYSTEM PERFORMANCE MEASURES AND GOALS

OVERVIEW

Once system functions have been defined, system performance

measures and goals and system organizational, operational,

and support concepts must be identified. These data

"* elements help to further define the functional requirements

of the New system.

Many new concepts and terms (for example, performance

measure, performance goal) are introduced in this procedure.

The reader should examine the definitions of key terms which

are listed in Table 1-6 before beginning this procedure.

PROCEDURE

An overview of the steps in this procedure is presented in

Figure 1-5.

1-37

%** .** * .. . .. . . . . . . .
0 0

00

U,4 En MN C4 MN mN (A M

-- %- 4

000

C, 1.-1 .,~ 4 0/ 0 ~ 0 4 4- -

F- 4 1 -4 .- 4 -4 -4 -4 -,4 4. 4.) 41 4

'U

--

.1-,q

I.4 *4. .- 4 *.4 . Lo- r- rz (a 4 .1~

a) - - - w -4 

01 (a0)0 4 4 .0z0 0 UU ) > -

(a4 sw C 0) 1-4 . E- 0 0 41

0' 0) CU .,1 Q

u C U) - u . U) 4) ".4 w C 0- t&4 LOU

4 -4 -5.' ".4 0~- C C)4I 

04 OctU 01 Cu N4 0

in 04 -4 U . 4JC 

4.)~-3 U-U U 0) -- .J4 ------- .
Table 1-6. Definition of Key Concepts

o System Performance Measures - A listing of the measures

which are used to describe the performance capabilities

which must be achieved by each system function. These

measures are usually described in the program

documentation (e.g., speed might be measured in miles

per hour (MPH) or kilometers per hour (KPH)).

o System Performance Goals - A description of the goals

performance measure. Goals may be stated in general

terms (e.g., increase by 10%) or as specific

quantitative values (e.g., the exact goal is listed-50

MPH or 30.8 KPH).

o Organizational Concept - A general textual description

of the way in which the new system will be manned (in

strictly an organizational sense) and the way in which

it will fit into existing Army organizational

structures.

o Operational Concept - A general textual description of

the doctrinal (how to fight) concepts that will guide

the operation of the system in both wartime and

peacetime and a description of how the system will fit

into more general concepts.

o Operational Environment - A textual description of the

specific environment, geographic, and threat conditions

which the projected system will encounter.

o Support Concepts - A general description of how and

where each individual Integrated Logistics Support

Concept should be performed and which Army organizations will perform these elements. As outlined in DoDD

5000.39, the elements of ILS are (1) maintenance, (2)

manpower and personnel, (3) supply support, (4) support

and test equipment, (5) training and training devices,

(6) technical data, (7) computer resources, (8)

packaging, handling, storage, and transportation, and

(9) facilities.

1-39

" "" °." "o° ". °*,',......................................................................... '"" "
-- 'a..,.-

OAGAWMEASURES

cONCEPTS CONCEPT

4AU!E

-'IMF

OENa.

FIGURE~~~S P-OevERDiaRAm: dniCEyse SPerfrmanceMeasurs ndGOALS (.)

S1-40

IDETIF
- 1.2.1 Identify System Performance Measures

You may identify potential performance measures by examining

the program documentation (for example, the JSMNS) collected

in Procedure 1. 1. 1. Any measure describing the potential

performance capabilities of the system should be treated as

a potential performance measure. To aid the user, a list of

* .common performance measures is presented in Appendix F.

Examine the list of system functions identified in Procedure

1.1.3 and identify the performance measures associated with

each f unction. Enter the description of each performance

measure in Column C of the System Performance Specifications

Worksheet (PRF) contained in Figure 1-6. (Columns a and b

of this worksheet, describing system functions should have

been completed in Procedure 1.1.3.) Each function may (1)

have more than one performance measure associated with it or

(2) not have any associated performance requirements and

therefore have no performance measures.

1.2.2 Identify System Performance Goals

Examine the program documentation collected during procedure

1.1.1 and identify the quantitative goals and standards

associated with each of the performance measures identified

in Procedure 1.2.1. Enter the data in Column (d) of the PRF

Worksheet listed in Figure 1-6. A goal may be stated in

general terms ( for example, increase 10%) or in terms of

* specific quantitative values (for example, fire 200 rounds

per minute). Each performance measure should have at least

one goal.

1-41
0-c

E a

c I I c0.

00

0 M a, 0 a

* a,

I = .- =m N n

*~~ 0 uEf

- u 

4) 0;.m,4

-C w r 0mm ~ r >Jo~ I ~ -

i4WC-0 m0~-

~.-*- 3t mm U -- 414GI~ 4 o96 *4 O'' - wUW ~ Cc- I oo mm--~ min 0 -

a ~ EEE -~-m

A.-42
0-~ ~ 0c~ _

0 E~

0.0

L6

E E

CL. 01 V )

o *0 0 -

-> ,. 0 I

* 4) 4)0 1-43
-. -. -.

0- 1

-w0

~0 (., 0"

o ° - > 

. .0

00

, 

0

a. 0 a

ca (U -

•~~ - ..- c--. w 

o & u . 0 I . . .

C1 0 0x

CD 0

7EE

0 0) u.

0'-.7i--4

.'0.

%A 0000

e~d 9~ -CJ C0L 0

t 00 ~

m- - ,- - -fl- E "I . "0

. - -
1.2.3 Identify Operational, Organizational, and Support

Concepts

Information on the organizational concept, operational

concept, and operational environment should be available in

the program documentation collected during Procedure i.]..

However, support concepts for many of the elements of ILS

are likely to be unavailable during the early phases of the

acquisition process. If the operational and support concepts

are unavailable, the user should obtain and use the operational and support concepts for the predecessor or another

similar system. These concepts should then be modified to

reflect the demands of the new system, subject to

coordination with, and verification by, personnel from the

Program Management Office (PMO). Organizational, Operational and Support Concept data is recorded on an Organizational, Operational and Support Concept Worksheet (OSC),

Figure 1-7. As the examples in Figure 1-7 indicate, the

organizational, operational, and support concepts consist of

a combinatioi of brief textual descriptions and quantitative

estimates of usage.

To further document system functional requirements the user

may use the worksheet listed in Figure 1-8, which briefly

describes the impact of the threat and the operational

environment on system functions. This information is

optional. However, some users find it helpful in developing

*- - descriptions of task conditions and standards for use in

Training and Soldiers Manuals. Information in system threat

and operational environment impacts may be obtained from (1)

SCORES, which contains general descriptions of threats and

scenarios, (2) the threat analyses conducted during system

development, and (3) the program documentation collected

during Procedure 1.1.

1-45
V. > 

I~ >~ .

-1 C 0 a .c V) 4t

z - .

E- E10 o 41 2 t 6 x- 0 z E> .

C w44 D c c E- PC,

CL 0 m4>

- s~ - -.N- cc G

r u 00 a m~ 0 ea .2 0n 0 0 0 -

0' 010 EN O E0 w 

0. SON 4 0 0 E

C44 CL .. u4-4 C 4 CCC - o oiE-4

TC 00 cc) CX u u

X 0.4

-1-4
E UE

0 0 La(

V)C

C. - E 3

c 'C

E- U

o a

-0C

)C W 

2 im- uCC) Q mC

C. U 1-47

j.% I
Cc

00

cn -

~. .

0 0 u

z z

CW - 3

4' 00 

*~~ 2 .2 - 0 v ... 4, U 0m - 0 C d c - 11. u 

4 u 0 C :.4 , 4

10

E

0 Z 00

-C

- 0) -4 I . U 44to

T -- . z

c- c c 4,

00 0~

p 0' j ~ 0 0 u~

im0. 00

.00

04, c0 -~3 4 0 0V u C 0 0 , - C 4 O 0

=c = c 

N 4, 34

.2 -A8
'0"

1. 2.4 Document Data

Relevant data collected during Procedure 1.2 must be entered

into the SDT. This data should be entered into the SDT

immediately after the worksheets listed in Procedures 1.2.1

to 1.2.3 have been completed. Table 1-7 describes the most

desirable order for entering data into the SDT. Additional

guidelines for entering data into the SDT is provided in the

SDT User Guide.

1.3 ALLOCATE REQUIRED SYSTEM FUNCTIONS AND IDENTIFY LOWER

LEVEL FUNCTIONS

OVERVIEW

This procedure is designed to allocate system functions in

Procedure 1.2 to people, equipment, and software based on

the allocation of functions in the predecessor and other

comparable existing systems. This information is used to

develop a generic equipment structure and to identify lower

level functions and place them in a hierarchy.

- ". PROCEDURE

An overview of the steps comprising this procedure is

depicted in Figure 1-9.

1.3.1 Allocate Major System Functions/Subfunctions

In this step, you must allocate the functions identified in

Procedure 1.2 to people, software, and equipment (or some

combination thereof). High level functions can be allocated

to people, hardware, and software by (a) examining how

1-49

%°-."
-4 C4 C4Ce V

0 0

o to

E-4 -4/

z

-4

t- -4 ) ( a a (

E4 0 0 0 0 0 0 0 0 -4 z l .1- *d ..-4 *r4- - 5'.0 ~ -~J41 4.j 4.J j. 41 -4 4 4-1

-- 4

00 co $

co c E-4> "a p W

I(D M(%a C -4 i c > - -u (a >.0 4 0

ao r_. 4) Q) a)

w1 -41J 4 Q() 41 a -H4.) a

0 0 co ua w ua( - $4 4- u

4. co a) m, ua cc~ 4aQ4 ma 

(a4 (al ( a4 C14 (a1a UOci * 1 jill1-50
0SYSTEM HIGH LEVEL SYSTEM /OPRIONA7 SYSTEM PERFORMANCE ORGANIZATIONAL, MISSIONS FUNCTIONS MEASURES AND SUPPORT/

(1.12) (.1.1 (1211CONCEPTSII.23/

ALLOCATE

MAJOR SYSTEM

FUNCTIONS

SUBFUNCTIONS

HARDWARE SOFTWARE HUMAN

FUNCTIONS FUNCTIONS FUNCTIONS

EQUIPMENOCUMTANTED

SEEENC DOCUENTDATADAT

FIGU ERI 1-9GOe EwEIaga: AlcaeRqie

Syste FuctosOnFIeTAR DWAR owrLveEuntos 13

COMPONNTS COPON-NT
'0:

similar functions were allocated in the predecessor system

or other similar systems, and (b) updating these allocations

" to account for the performance goals associated with the New

system.

Once identified the allocations should be entered in the

last three columns of the System Performance Specification

Worksheet (PRF) shown in Figure 1-10.

1.3.2 Establish Generic Equipment Structure

A generic equipment structure describes the generic

equipment subsystems (e.g., engine) which must be included

in the New system. A example of a generic equipment

structure is provided in Table 1-8. The generic equipment

structure can be developed by (a) examining the equipments

*-.p in the predecessor system, (b) adding equipments to achieve

the additional capabilities required by the New system, and

(c) deleting equipments which are not needed in the New

system (as indicated by the New system functional

requirements and functional allocation).

The generic equipment structure for the predecessor system

can be identified by examining the maintenance manuals

associated with the system. To identify the generic

equipments which must be added or deleted from the

predecessor, present the predecessor equipment list and

function allocations along with information on the system

functional requirements to subject matter experts from the

schools associated with the predecessor system and ask them

to make the proper modifications to the predecessor

equipment list.

1-52

0.....................................-*.*.-
I co

t II 

W cI

E m 

-t M -M

oU) E E

'n 01-.0

10-e f -

ca . Z -

. 0. 

mU 02m 0 -

-00 N C C

Z ---- 0.1

LL ~C 1'c 2

cr.~ -

a

0. 1-5

- 0.~
. x x

0-0

~M

. c 0 

u .a, 0

E E (A

E- U

o I

a. u u.

Ga Ga -uG. G - C)

Q E u L

*~ c - G

* v

* CC

U.U

a.2

0 u c 

411

CL. -0 41- - U1

cm 0

* ffl ! 4IA. ZG 1 -54

%-L~ 0.-A2
-I 4,

- 0 -~---.

I K .~ ~

_____ ________________________

r.. .9

0

0.

0

- E

C)

C)

0

C, . - C) C £2 - U . . 0 E

S -

e .~N C) -- I I 4,

£2 CO -. N

0. ~ N~.C) C)

E£2 4, C 0 C) - C) 4, C -- C' C) o ~ N N ~ C --

N I C)C) ~C'V '~ C) - 0 C)'C I-4,~. 0 ~ 0 C) 4,~-C £2 ., - ~ ~n >E-I.. '4 ~ 0

0

.2- £2 0 U o 0

0.

U

0.

£2 £2 £2

- 0 I 2£ ii *0 -b. £2 4,

C £2 0'-- o u 0' - C- :~£2 £2 2£ 0 4, .0 C).0 2£ - E 0' ~ 4,~ U * ~ o. cC. - ~ ,~ ... ~ 0 U - .~ ~ U UC)~ C

£2 0. .- C 0' * - 4, C) C) £2C, C)

£2 .~ U S .~ o ~ £2 .0 ' C. ~ -.

~' N. 0 0 L~ 0.

-2 0. ~' - .0 S

- a~ z U U~ 0

0.

S ______________________________________________________________________ £2

.0

a

U

LI.

S ~. 4,

4, 0. I C) C 02 .0 ~' .2 Z a - I- 0 - C) CC) ~ o C) C 0 (/)

- 2.-C C - 2£ - - C,

- - 0 0. j .2 u. C). C 0. £2 - 4, C.. -

- U 0 3 ~2~~ 4,

.'~~2 U. 4.' 4, 5 C, C,

o - 4, C) C ~C - 2£ C) 'V C --

0. z .0 0 0 u 4, 2

~.J 0. ~. 0. 2£

11 __________________________________________________________________

3 - -

In I

C 2£

'4 - C, -

C - - .~ '2~

4, 0=

* ~ u.Z -- 1-55

* - . - . - *7 - - - . . -
0

M2 Eo

41 'A w =- cr-=L Y4

EL ra.c E bE oEE E>-*A

*- r Cu0 -0 w- c-~ o o

.0c E L L 0.

r.=X4 C L - 0 w C

4- m x .

S U. L.<EU c u I.L

2i

* * E

Go

m

2r U - - -

1-5
'.o

Once the generic equipment structur-e has been identified it

should be documented in the Generic System Worksheet (GEN).

Columns (a) and (b) of the worksheet should list the number

and title of the system functions as described in columns

(a) and (b) of the PRF worksheet. Column (c) of the GEN

worksheet should be used to describe the generic equipment

structure.

Once the generic equipment has been identified at the

subsystem level, lower level equipments must be identified

and the resulting lower level functions must be described,

Information on lower level functions should be entered into

columns (a) and (b) of the GEN and PRF worksheets.

Information on generic equipment should be entered into

column (c) of the GEN worksheet.

As part of the identification of generic equipment, you must

allocate functions to hardware, software, or humans. These

allocations should be documented in column (e) of the PRF

worksheet and columns (d), (e), and (f) of the GEN worksheet. In the GEN worksheet (column f), functions allocated

to humans are further broken down by classifying these

functions as (a) generic operator functions, (b) generic

maintenance functions, or (c) generic support functions,

1.3.3 Determine/Describe Lower Level Functions

Subsystem level functions are identified in the same way as

high-level functions. Lower level functions from the

predecessor or other comparable systems are identified and

modified to reflect the New system requirements. Information

on the predecessor and other comparable equipments is

contained in the Generic System Worksheet (Figure 1-11).

1-57
0X . X XX X X 

x x

ui CL

CU

00

CDC

010

-4-V

a1 -14 r

01 -C

C U)

o)- 0 4u z

0. w 41 C) 1

E- E

0 004

4. 14.413

a* 041

cC

U-Z -5
A..

X °=

S0 10

o.2 0

0 2x x x r00

-- ~Q - 0 -4 m 

0, E- -4

Cal

:1

6. 0 ' . z : : :

z E

-. C u I r- Ea

~d j U -4 E 0 -4 - r-

-4 u z ~ r 0

w -- - .4. . ."_ In a) E - .

E. 0 ~ N-H u .,

4 .:3 C) HH -l -4 4)

oL u U C C $44 :3 pro E Q Q)) -4 C) Q) ,

c4 (*H r( a) In ") 

0 - 0 0

00 W0 04- u Q

L) 0 ) C. H ~ ~ C cu c0) 4Q 4J-4

Q4 rz

4-1 -4

) 0 C)0 41 

4-. IQ C) C

0 a

E.o~C

a)u.

Z~4 a) 
70x Q x x x x x×

-"I

- 0 0.2-

• < >.

.

0 :3

E 4 xo w - r II, _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

UU

r- E

0 ,-I 0 . .'-. J 0

UU

&U x

C El C EC 2-

-r a 01 0 4 rc > : >

r 0 0 0 0. 0 ,- " 0

s 4 0 .,- 0

Z N N C)r 'C : "

•W-" - . t • ) 0 n C 

ua ~ u r4' 

3. C. ) C) c3 oh 4J 4 C E o

moc

Q-4 E

LL -4 *. D 'C4 .. C

".) : o ,-. 4.) ,.-4 .-.- 

. ,c', . o % ' , , ', ' . . ' . , . ' " .U- - . . . . . . U ' . . ' • . , 4 . , - . . -4 -4., .:. - , -

C 00 0 ~~~~- 0 ' C C

E u ~.4 01 rq M4 0 'C a)-

(D-4

0. 0) 0) C 4. 44 u) 

0 .0 L

C)~~~~- Sn* '4 -

o~~~0 'C ' *- )*.

0' 0 0' '0 E' 

(n 0 j 41

-41 .4 Q 4. ~ 4J ~C 4.'

0 ) ) ~ 4) ~ )'2~

Z) C). 4)0)'

'C 460
S.i 1.3.4 Describe Function Sequence

Identification of function sequence is accomplished by

identifying function sequences from the predecessor or other

comparable existing systems and then modifying these

sequences to reflect the New system operational concept and

functional requirements. Function sequence information

describes (a) what functions must be performed prior to,

after, or concurrently with a given function during mission

- performance. The best sources for this function sequence

information are the doctrinal manuals (e.g., the How-toFight manuals and ARTEPS) associated with the existing

systems.

A worksheet which can be used to describe function sequence

is contained in Figure 1-12. Identification of function

sequences can be facilitated by drawing a function sequence

diagram such as that contained in Figure 1-13, which

contains a high level function sequence diagram. A

* ipreceding - function is a function which must be performed

before the designated function. A succeeding function is a

function which must be performed after the designated

function. A concurrent function is a function which must be

performed at the same time as the designated function.

Whenever comparability analysis procedures are employed

during system function requirements identification, the

- comparable existing systems and the modification to these

systems should be included in the audit trail. In addition,

all information on functional requirements generated via

comparability analysis should be presented to the Program

,..--.Office/Combat Developments Directorate for review and

ultimate approval.

1-61 S ," q*~ . * .~'% * * .
cJz

C

CD

0 T)

CDC

00

c E) .0 m

CD u

00

*1-6 EU

UL
LU L

U E

4-

0

CLC

0. C)

LU CA

UL

CLL

LLU

ZuZ

02

LUU

6~ -I

00

1-63-
1. 3. 5 Document Data

Relevant data collected in this procedure must be entered

into the SDT. This data should be entered into the SDT

immediately after the worksheets contained in Figures 1-10,

1-11, and 1-13 are completed. Table 1-9 describes the

- sequence with which these data should be entered into the

. SDT. Additional guidelines for entering data into the SDT

are contained in the SDT User's Guide.

1.4 ESTABLISH THE BASELINE COMPARISON SYSTEM (BCS)

OVERVIEW

The BCS is a notional design consisting of the existing

subsystems which currently come closest to meeting the

functional requirements of the New system. Since the BCS

only contains existing subsystems, the BCS may not meet all

of the performance goals established for the New system.

In order to be selected for the BCS, a subsystem must meet

the following criteria: (1) it must be an existing

subsystem currently in the DoD inventory, preferably in the

Army inventory, (2) it must be the existing subsystem which

most closely matches system performance requirements, and

(3) it must have mature task data. The latter set of data is

critical for training requirements determination.

. iPROCEDURE

An overview of procedure 1.4 is contained in Figure 1-14.

1-64

.'..............................................
0 0

04 y 04 0 

E- u- 4 ';Z

EJ -W, 4J a.4

00H 41__ - _____ 

4 

41 4

41 u

~u

>3 >

CQ

z

cn m m c n r

IL I' I'

1-6
SYSTEM PREDECESSORq/

'TPERFORMAC COMPARABLE r TMEASURES EQUIPMqENTS

DOCERENT4D

DMOI ATIAL

Baseline Comprso & COCStm BC

% & OPERAT1-66 UPR -CSCE

-~~~O TS- - . A A

DESCRIBE
70-1

1.4.1 Establish The Baseline Comparison System (BCS)

This step is initiated with the construction of a BCS

equipment list. The BCS is comprised of comparable existing

subsystems (predecessor or other) from the Army inventory

which (a) best meets the functional and performance

requirements of the New system and (b) have mature task

data.

A worksheet for describing the BCS equipments is presented

in Figure 1-15. New system functions and performance goals

are listed in columns (a) and (b). This information can be

taken from the System Performance Specification Worksheets

developed in procedures 1.1 through 1.3 (See Figures 1-4, 1-

6, and 1-10).

Second, generic equipments identified in Procedure 1.3.2 and

listed in Figure 1-11 (Generic System Worksheet) are entered

in Columns (c) and (d). These generic equipments designate

the type of existing equipment to be considered for the

BCS.

Once columns (a) thru (d) have been completed, the worksheet

should be presented to subject matter experts from the functional areas associated with each major subsystem in the

generic equipment structure. These analysts should be asked

to indicate what equipments in the predecessor or other

comparable system can (1) best meet the New system performance requirements, (2) has available mature reliability,

availability, maintainability (RAM), workload, training and

other data, and (3) where design information for the new

system exists, has technology as similar as possible to that

projected for the New system. Additional guidance for the

1-67
0' 0

CL'

S. x x

- ,s/s0 d *x

0.S

w EnM c C z

UU

0(5

uo ~l 0 

c4 '4 '4 '4

2~ - > >GG r. -* 7. 

- wu 

o w o

04E4 4) :N, 

-0 SC o - u* -

u5' 0 0 a

so 1 2 -: i

U. 0

w 'V1-68
0I

identification of BCS equipment s can be provided by

examining existing equipment taxonomies in the funct ional

areas associated with the New system. Equipment taxonomies

list the specific equipments associated with each category

in the generic equipment structure. Such taxonomies are

likely to be found in configuration or logistics management

directories associated with the New system functional area.

Table 1-10 presents an example page from the equipment

taxonomy which is included in the Aviation Electronics

Configuration Directory.

- Once potential BCS equipments have been identified, the

. subject matter experts should be asked to note the degree to

which each potential BCS subsystem matches the performance

requirements of the New system on a one to seven scale (onenot all and seven-identical). One rating should be obtained

for each potential BCS equipment. These ratings should be

entered in column (h) of the BCS worksheet. Column (i) of

this worksheet should be used to document the availability

of BCS data. Columns (k) and (1) may be used to describe

the Functional Group Code (FGC) Work Breakdown Code (WBC),

Work Unit Code (WUC), or Manufacturer's Part Number (MFG's

P/N) of the selected BCS equipments.

After the BCS equipment list has been constructed, detailed

data on the BCS equipment may be obtained. Some of this

information may be obtained during the manpower requirements

analysis. The BCS Equipment Description Worksheet contained

in Figure 1-16, should be used to document this data. The

major focus should be on the identification of equipment

characteristics which are directly related to task identification and analysis. However, additional equipment

characteristics may also be identified and included in the

1-69

S"
CM V- -W~--

'4t to

00 N 00N -~ 

c. m

-< - - 0 CD 0 . a go - - - 0 , , 0 "

6 0 i i 6 i i i

o 0 -am

" N

C4 ..

z

Of m

ca w0 CIO coel 0 

CD en 00 N 0

... -W , t - ca 'm m In La ,.. 8 8- t- m.

C4 o c w 10c - r IlC3 

,... - -

-n C N -N 4 ; C n

Le go m CO - eq -z a I In I9 

*~0 42 COC O C

C3" p LI 10 Lo -n 4) M 1' 0 V13 I' n In 10 = : :) LM In 1 -n

ca cc (D (D W ~ 14 W t.: D C- 

" 1

- C - , 0. 0- 0. - . 0.0" t" .. 0. . "0. ",0. . - 0 . " . 0.- ," . '

0~~~a 0 00 C 0 00 0 C 0 -0 C C o 0

Z - z. -

-~ ~ ~~~ - <O O <O CC <O <O O <O C CC <O <OCC OO C O

1- 70
SDT. Figure 1-16 displays the addi t irnal equipment

characteristics which may be estimated and iindicates which

of these characteristics are used in task identification.

Identification of task elements can be facilitated by the

types of controls and displays which are associated with the

equipment. Displays and controls provide the focal point of

the man-machine interface. Another important set of equip--

ment characteristics for task identification are descriptions of the information inputs and outputs which are

associated with each major equipment item. This input/output

information is critical for those equipments which perform

some analytical/software function. These equipment information inputs/outputs play a key role in determining human

task inputs/outputs.

In addition, available data on equipment reliability and

costs can be entered on this worksheet.

1.4.2 Determine BCS Operational and Support Concepts

After construction of the BCS equipment list, information on

the operational and support concepts of the BCS should be

obtained. (Definitions of the terms operational and support

concepts are provided in Procedure 1.2).

Operational concepts are documented in the TRADOC Pam 525

series. As described in TRADOC Reg. 11-7, the operational

concept describes "the performance of one or more combat,

combat support, or combat service support functions. Each

operational concept defines what needs to be done; why it

needs to be done; how it is to be done; where it needs to be

" . done; and who does it" (TRADOC Reg 11-7, pg. 3).

,Sq

..-- 7.

0-7
0-

/

0. C

0~~ - C. .~

o C -

C~

a -

C =

~

-C C -

I -

0~wo00

.0

-V I --

o I- C0 0 0. ~ U0. 

-

U --

-0

C _______________________________

CO -

-0 0 C

& E.~ 0. - - w

I- rJ 0~

0. -

w -

0

-I

CC

0. LL C

z --

0

C.,

~ 0

0 - -' _________ _____________

0

I- = ~-.

-~ I 

C I _________ ________ C

oT U. 0.

t 

0

I C

0 --- ------- ______ 

C

I 0

~~zI -

~

.1....~~

1-72
An overview of the information contained in the operational

concept is presented in Table 1-] 1,

Support concepts for BCS subsystems should be available in

(a) the program documentation for that subsystem,

particularly logistics support analysis documentation and

(b) maintenance and operator manuals for the equipment.

* BCS operational and support concepts should be documented in

the OSC worksheet listed in Figure 1-7.

1.4.3 Describe Function Sequence and Information Flow

BCS function sequence information can be obtained from Howto-Fight manuals and ARTEPs associated with the BCS

equipment. Function sequence information should be entered

into the worksheet listed in Figure 1-17.

Information flow data describing the inputs and outputs

associated with each BCS equipment may also be identified

(this infor-.3tion is particularly valuable in identifying

task requirements for automated systems). Information flow

data should be available in the operator and/or maintenance

manuals associated with the BCS equipments. Once identified,

the BCS information flow data should be entered into columns

* i(j) and (k) of the EQP worksheet listed in Figure 1-16.

1.4.4 Document Data I<.

Relevant data collected in this procedure must be entered

into the SDT. This data should be entered as the worksheets

contained in Figures 1-15, 1-16, and 1-17 are completed.

Table 1-12 describes the sequence with which these data

1-73
TABLE 1-11. FORMAT OF THE OPERATIONAL CONCEPT

AND CONCEPT STATEMENTk

Edition (i.e., lst Draft

2nd Dr a ft, etc.)

Date __ _

1. Purpose:

- WHY is the concept needed?

- WHAT needs to be accomplished?

- Are there influences of geography, climate, threat, or technology?

If there are, state them in general terms. If necessary, give a

detailed explanation in an annex.

- If the concept is one of a set, common environmental parameters may

be stated in a preface to the set. Individual concepts or concept

statements need only reflect that they are a part of the set.

2. Limitations:

- State any limiting assumptions of facts.

. For example, a concept may not apply to joint or combined

operations, or,

* A concept may be applicable only to the Mideast or Central

Furope.

3. The operational concept:

- HOW is the task to be done?

* How the operation will be performed and how it relates to

other operations performed simul. eously or in support.

- WHERE is it to be done on the battlefield?

• Covering force area, the mnain battle area, etc.

- WHEN is it to be done in relation to other battlefield activities?

- WHO does it?

* What echelon of command performs this task?

4. ANNEXES (As appropriate)

- Glossary of terms/acronyms

- Bibliography

*Taken from TRADOC Reg 11-7. 1-74
LL

C.)

I.. C

L 0 

E

4-7
00

E-- -c .

C,~~ C,, ~ l CY '-O c,

W~ wr w w -.

M E00

-

141

Ca

E- 4Jwcn w:3 C

EH 4J 4 w 0 0 0 W

a) ~ 4J 0 .j

-4-

C.,

1-7
W0 .W.WCL

00 00

C.

C 0 

-'n

w~ C 4-

C4.

00

C) H

E-4

z 0 0

H1--
should be entered into the SDT. Additional guidelines for

entering data into the SDT are contained in the SDT User's

Guide.

1.5 IDENTIFY REQUIRED SYSTEM IMPROVEMENTS/NEW TECHNOLOGIES

OVERVIEW

The purpose of this procedure is to identify system improvements/new technologies required for the New system. The New

system is defined as the best estimate of New system design,

incorporating modified or improved design features,

technological advances, new operating and support concepts,

and changes to other system elements. As with the BCS, the

New system must fulfill all functional requirements. Unlike

the BCS, however, the New system is expected to meet all

system performance criteria. The objective of this

procedure is to identify and describe functional areas where

the BCS system fails to meet required performance goals or

standards.

PROC EDURE

An overview of the steps comprising this procedure is

presented in Figure 1-18.

1.5.1 Identify Required System Improvements

This procedure begins with the comparison of the functional

requirements for the New system and its associated

performance requirements, to the capabilities of each of the

BCS subsystems. This comparison was initiated during

Procedure 1.4 (See Figure 1-14). In this procedure, a more

1-78
SYSTEMSYSTEM

SYSTEM PERFORMANCE C FUNCTIONS MEASURES H1.4)

0.1.3, 1.13

IDENTIFY

REQUIRED SYSTEM

IMPROVEMENTS

REQUIRED

SYSTEM

IMPROVEMENTS

I DENTI FY

NEW TECHNOLOGIES

NEW

TECHNOLOGIES

DOCUMENT DATA

DATA

rDCMNE

FIGURE 1-18. overview Diagram: Identify Required

System improvements/New Technologies ( 1.5).

1-7()
detailed comparison is conducted to identify areas where the

BCS does not meet the performance standards required by the

New system. If a new system description and/or equipment

list is available from the contractor, it would be utilized

to identify performance improvements (over the BCS) which

are part of the contractor's design. Some of these

improvements may exceed stated system performance goals.

These additional performance improvements would also be

documented. The results of this analysis are documented on

a Baseline Comparison Performance Shortfall Worksheet (PSH),

as shown in Figure 1-19.

First, the BCS equipments are entered into columns (a), (b),

and (c). Second, the New system performance requirements

are entered in column (d). Third, performance gaps; that

is, areas where the BCS does not meet the performance

requirements of the New system, are identified and entered

into column (e).

The same subject matter experts who assisted in the

identification of BCS subsystems should be used to identify

the BCS performance shortfalls.

1.5.2 Identify New Technologies

The purpose of this procedure is to identify new technologies which can satisfy each of the required performance

improvements identified by the previous procedure. The

estimate of the New system design is made by incorporating

desired and feasible technological improvements. If a New

system equipment description is available from the

contractor, the identification of new technologies which

could meet each of the required design improvements is

1-80
00

s - 0

oo m' ea

-. u m0-0 "

0. 41 c 0. 0 CU a,

ou -

*~~ 0 4l1

C F6 r . 06-

-M 4 0> > 41 c0 >2 a .~ 4

2 2 0

0. CL L

CL 0

0. Al 0 0

- I- -

0~ 3 .-

41 0 . (U>. 

1. '4 0.4

~0.00

E. 0. C

0 41

L6 4. 0 0..Cx 4

0 0 0 0~ mC

"414 00ig 3O 

410~ >. 4

E40 0 1~ - 3C2 C

fn cc N. 0.L 44 

4100 0 0

o~~~~~ 4k. ~ 1 4 0.'' 0'4' a, u. m'>4

ILCL

cm

cc

~ S I-1-81
'00

uN 4,

9LID

41.1

LL a 0

0 0' U

Z4 0, 41

00

2.

o co

UU

1-8
C. 0

0~ cr

0 t

CLC

00

cmE

o- u~~

0 0

0 0

u m6

- C8
 . . .  . . , . o . . : r rr%r . . .t : . w i; . .,, ,i . , .

relatively straightforward. The required technology

information could be obtained directly from the New system

equipment description. If a New system equipment descrip-

- tion is not available from the contractor, other sources

C would be used to identify the new technologies. There are

two major sources of such information: (1) equipment related

literature and documents, and (2) i;-terviews with subject

matter experts. The primary source for the identification

of new technologies, and the source which will probably

prove to be the easiest to access, are interviews with

subject matter experts who have expertise in each of the new

technology areas associated with the BCS equipments.

More specifically, the Baseline Comparison Performance

Shortfall Worksheet with columns (a) thru (e) filled in

should be presented to hardware subject matter experts from

the functional areas associated with each major BCS

subsystem. These analysts should then be asked to indicate

what new technologies can be added to the BCS equipment to

eliminate the performance gaps. Once identified, the new

technologies should be entered into column (f).

1.5.3 Document Data

Information on new technologies should be documented in the

worksheet listed in Figure 1-19.

1.6 FORMAT NEW SYSTEM

O. OVERVIEW

. During this step, the new technolog ies to be incorporated in

the New system design(s) must be examined and differences

p.

S -

4 I- I
-

between the BCS and the New system which impact human task

performance must be identified. These design differences

will be analyzed in detail in subsequent procedures and used

to modify BCS data to reflect the infusion of new technologies identified in Procedure 1.5.

PROCEDURE

An overview of the steps comprising this procedure is

presented in Figure 1-20.

1.6.1 Describe Design Differences

A design difference (DD) worksheet has been developed to

assist the analyst in describing the equipment associated

with each design difference (see Figure 1-21).

The information in columns (a) and (b), describing the BCS

equipments, can be taken directly from columns (a) and (b)

of the BCS EQP worksheet. After these two columns are

filled in, column and listing the New system equipment

associated with each BCS equipment should be filled in.

These New system equipments are developed by combining the

new technologies listed in column (f) of the PSH worksheet

with the BCS equipments. Once the New system equipments

have been identified, subject matter experts who are

familiar with the BCS equipments should be presented with

the partially completed DD worksheet and the completed PSH

worksheet and asked to identify and describe any potential

hardware/software differences between the BCS and New system

equipments which might impact the task performance of

operators, maintainers, or support personnel. If possible,

they should be asked to assign a quantitative value to these

1-85

S ..
"7077

REQUIRED NEW

IMPROVEMENTS TCNLGE

B04 DATAI13

HI0WR/D~G

FIGURE1-20.OvAEriwDarmFomtewSte(16

DI1-FFRNE
design differences. The subjr:t matter r s ",hould b

asked to focus on hardware/soft ware featui (fo tr , xamr.

reliability, maintainability),

1.6.2 Describe New System Hardware/Software

During this procedure, a hierarchical equipment structuL:

for the New system is constructeu. The Der-;ign Differenc

worksheet described in Figure 1-21 wiij oti a listIng o

New system equipments. These equ-pmeits mrust be examinet

and placed in a hierarchical equipment structure. If a Ne,

equipment design is available from the contractor thi.

equipment structure can be taken directly from contracto

supplied data.

Af ter the New system equipment. structuei identifEied, th,

New system equipments may be described in more detail 'se

Figure 1-22) in a New system Equipment Description Work,

sheet. Many of the data elements on the worksheet may no

be available until later in the acquisition process (see th,

data source ine in Appni E f:ori a lis potentia

data sources).

1.6.3 Document Data

Relevant data collected in this procedure must be entere

into the SDT. This data shall be entered as the workshee

contained in Figure 1-22 is completed. Table 1-13 describe

the sequene with which these kata should be entered act

the SDT. Additional guidelines for entering data into th

SDT are contained in the Sac User's Guide.

acn Ei

daasucsL
0 L0

CO 0

a; Z

2~ 0

a) E

0 c

c- CO

LU.0

co E
* 0

CL

oo

-. E

CLC

-CL

0. 4Cc

UL

~1-8

a) Ul -

oj
01 4 1 -. 1

04 04 04 04 04 04 Q4 04 04 a 0 0 0 0 0 0 0 0 0 0

C.C

* E-4

C 0 -

00

wt in =t C4 CPr4 at 04 r C4 1L 0:4 4 04 104 04 at at 0t at C' Of

w 0I

a)4-1 4.) 41 4-) 41) 4.) .41 41) 41) 4- 4J 4 4- 

>4 a a C r_ C r_ c C (2 ) (1) (1) 1 () ) Q) aj Q) a) a)

E-0401 01 0 01 01 01 0 01 04 01

* uu

(1) -

-4E- 4

4J 0 4-1 Unw1-

z0 4-1.) 4. >1 Ix 0 04 r_ C r_

- 4 a) 4 H 04 M 0 0 0 4 04E- 04 .0 w-4 E- 0L 4.4) L fu 2U ~ U) 0 4. 41 i - :4 4) . U C44 -4 o4 W (1

3.fuI) a) >1) r- C C w4 C 2 24 "1 co4 'a-a)-404-4' 0 0 41 4 0 u -- 4 ~- En m 44 44 E CU rX .1 N4 0 ~ 0U 0-~C) 

U 04 H 1 La C

-4 W I CUL 
SECTION 2.0 - GENERATE TASKS

OVERVIEW

Dur -ng this procedure, task lists are generated for the

Baseline Comparison System (BCS) and the New System. In

* "."addition, the tasks in each of these systems are assigned to

duty position, skill level, and MOS.

PROCEDURE

This procedure consists of five lower level procedures (see

Figure 2-1). During the first procedure, relevant task data

from the Predecessor system and other comparable existing

systems is collected. In the second procedure, the BCS

tasks are generated. In the third procedure, the BCS tasks

are assigned to duty position, skill level, and MOS. In the

fourth procedure, a New system task list is generated.

Finally, in the fifth procedure, the New system tasks are

;* assigned to duty position, skill level, and MOS.

2.1 IDENTIFY, COLLECT AND FORMAT TASK DATA

OVERVIEW

During this procedure, task data are collected for all

Military Occupational Specialties (MOSs) which may be

relevant to the BCS or New system.

2-1

|'°
.2.

COLLECT AND

CATALOGUE

TASK DATA

2.3

ASSIGN BCS

TASKS TO

MOS,DUTY POSITION

AND SKILL LEVEL

4!.

DEVELOP.....................~~
PROCEDURE

An overview of the steps in this procedure is presented in

the flowchart in Figure 2-2. A worksheet which can be used

to support this procedure is provided in Figure 2-3 (CND).

There are two steps in the procedure. In the first step,

candidate occupational specialities are identified. In the

second step, task data on these occupational specialties are

collected and formatted.

2.1.1 Identify MOSs Currently Performing Similar Functions

The purpose of this step is to identify all existing

occupational specialties which may be relevant to the New

system.

To begin the step, information on the new equipment number

and name should be entered into columns (a) and (b) of the

CND worKsheet. This information can be obtained from

columns (e) and (f) of the BCS worksheet (see Section

"- '-1.41).

Second, if the BCS equipment is in the current Army

inventory, this information should be noted in column (c).

Third, the occupational specialties which currently operate

and maintain the BCS equipment should be entered into column

(d). Information on operator MOSs should be readily

available from the schools associated with BCS equipment.

Information on maintenance MOSs can be obtained from the

maintenance manuals associated with the BCS equipment.

Information on Navy and Air Force occupational specialties

can be obtained from the Navy Enlisted Manpower and

2-3

.,.-..-................... .....-..... ,......... ..-.. .......
,- .-57.

SYSTEM REQUIRED GENERIC

S Y S T E ME FUNCTIONS & /UMASYSTEM N

MPT STUDY PERFORMANCE JR~~ ~ EFUNCTIE FUNCTIONS ( O,

REQUIREMENTS REQUIREMENTS (FROM 1.3) (FROM 1..1) (FRO 1.2)

IDENTIFY

(MOS(s) CURRENTLY

PERFORMING , SMILA R

FUNCTIONS

LIST OF

CANDIDATE

MOSWs

12.1.2

COLLECT/CATALOG

CANDIDATE

MOS

TASK DATA

CATA LOG

TASK

DATA

-" -. 7

o..

Figure 2-2. Identify, Collect, and Format Task Data.

2-4 S'.
0j

4K

0

.- z

w wz

-2-
Personnel Classifications and Occupational Standards and Air

Force Regulation 39-1 (see Appendix E for a more complete

listing of data sources).

2.1.2 Collect and Catalog Candidate MOS Task Data

During this step, all available task data on the candidate

MOSs identified in step 2.1.1 must be collected. A listing

of potential sources of task data is provided in Table 2-1

(a more complete list is provided in Appendix E). The best

sources of task data for existing Army systems are Soldier's

Manuals and the Trainer's Guides.

As task data is collected, it should be documented in column

(e) of the CND worksheet listed in Figure 2-3 and in the

library index.

2.2 DEVELOP BASELINE COMPARISON SYSTEM (BCS) TASK LIST

OVERVIEW

During this procedure, a task list is generated for the

Baseline Comparison System. The BCS is composed of the

existing subsystems (including the predecessor subsystems)

which come closest to meeting the New system functional

requirements (see Section 1.4 for a more detailed

'. "'..description).

PROCEDURE

r-" An overview of the steps in this procedure is presented in

Figure 2-4. A worksheet which can be used to support this

procedure is provided in Figure 2-5 (BCST).

I,"_ <.-- 2-6
Table 2-1. Task Data So rces.

& Soldier Manuals"*

a Trainer's Guide"*

a Job Book**

* CODAP**

*LSAR*

*ARTEPs**

*Field Manuals**

*Maintenance Manuals**

*Skill Performance Aids**

* Maintenance Allocation Charts*

0 Training Development Information System CTDIS)**

* Contractor Supplied Training Programs*

0 Program of Instruction (POI)

*Data source for developing system

**Data source for existing system

2-7
1.5 S LECT 2'ZI

SCS, SUBSYSTEM

[HARDWARE/ FROM SCS

!T SOFTWARE RE HARDWARE/

LIST SOFTWARE LIST

BCS

HARDWARE/

SOFTWARE

SUBSYSTEM

IDFNTIFY

is NO NON-ARMY

SUBSYSTEM OCCUPATIONAL

IN ARMY SPECIALTY

INVENTORY ASSOCIATED

WITH SUaSYSTEM

2 YES

[2-2.3

IDENTIFY

MOS41 OCCUPATIONAL

ASSOCIATED WITH SPECIALTY

EQUIPMENT

RELEVANT

mosis)

-TE2- 4 is NO IDENTIFY COMPARABLE COMPARABLE S. TASK ARMY EQUIPMENT EXISTING

DATA ARMY WITH AVAILABLE AVAILABLE EQUIPMENT

TASK DATA

ESYES

12-2-5 2 IDENTIFY TASKS

ASSOCIATED

WITH

EQUIPMENT

PRELIMINAPY

LIST OF TASKS

ASSOCIATED

WITH

EQUIPMENT

A

Figure 2-4. Develop BCS Task List (2-2).

2-8
ADEQUACY

OF TASK

DESCRIPTIONS

TASK

DESCRIPTION

ADEQUACY

[2L2-

is NO REIS
.. . . . . .

0 1

w 2

3

m

re

g

C,,

C-,

- w ~0

a -

0

0

C..,

________________________ ________________ 0

a,

0,

of

a 2

30- K __ _ __

* 4 ___------- -~-- 2-IC I,,

U, - -~ ___

............................... ~.
I-. . |,_ 7 7,

.', 2.2.1 Select Subsystem From BCS Hardware/Software List

To begin the procedure, the BCS equipment number and name

should be entered into columns (a) and (b) of the BCST

worksheet (This information can be obtained from columns (e)

and (f) of the BCS worksheet in Section 1.4.1).

Once this information is entered, you must determine if the

BCS equipment is currently in the Army inventory, (This

information is available in column (g) of the CND worksheet

used in Procedure 2.1). If the BCS equipment is in the Army

inventory, you should proceed to step 2.2.3. If the BCS

equipmen._ is not in the Army inventory, you should proceed

to step 2.2.2.

2.2.2 Identify Non-Army Occupational Specialty

Information on non-Army occupational specialties can be

obtained from the Navy Enlisted Manpower and Personnel

Classifications and Occupational Standards and from Air

Force Regulation 39-1. A more complete list of data

sources is provided in Appendix E. Once the occupational

specialty is identified, it should be listed in column (c)

of the BCST worksheet.

2.2.3 Identify MOSs Associated With Equipment

The best data source for linking MOS to equipment is the

Table of Equipments Manpower Authorization Criteria (MACRIT)

found in army Regulation AR 570-2. However, new equipments

are not likely to be listed in MACRIT. For new systems, you

must identify the proponent for the equipment/subsystem or

the system to which the equipment/subsystem belongs. Once

2-11

".. 5-' ". ". ".-. -" - . "". .. -""'•; .k ''.V ,<' ',..' , ,- '' """",
• . .*- - - - . . - w -- .. - ... s.* . • . .- .. .. W W- 1 r 

'%

the system proponent has been identified, you must call the

proponent school (see Table 1-3 in Section 1.0) to obtain

MOS information. The most definitive data source for identifying MOSs for Army equipments is Army Regulation 611-201,

Enlisted.Career Management Field and Military Occupational

Specialties. This document describes the general types of

activities performed by each Army MOS.

2.2.4 Identify Comparable Army Equipment

Comparable Army equipment must be identified for each BCS

equipment that is not in the Army inventory. Procedures for

identifying comparable equipment are listed in Section

1.4.. Once identified, the comparable equipment should be

listed in column (d) of the BCST worksheet and occupational

specialties for these equipments should be identified using

information listed in Step 2.2.3.

2.2.5 Identify Tasks Associated With Equipment

The tasks associated with each BCS equipment, which is in

the Army inventory, should be obtained from the Army task

data sources listed in Appendix E (This task data should

have been collected during procedure 2.1). Most of the

time, these tasks should be used in the BCS task list

without modification. However, before including these tasks

in the BCS, they should be examined, and when necessary,

updated to reflect any equipment or doctrinal changes that

were made since the task data was published. A list of

sources for monitoring these changes is presented in Table

2-2.

2-12

4.

9. . - : . - ., , .. - - ; .. - .. - - . . . . .. _. . . .- - -. ,..> .. .. - -. .. - -. , .- ,
Table 2-2. List of Data Sources for Monitoring

Equipment/Doctrinal Changes for

Existing Systems*

- Tables of Organization and Equipment (TOE)

- Tables of Distribution and Allowances (TDA)

- Job Supervisor Interview

Subject Matter Experts (SME)

- Equipment Modification Work Orders

- Directorate of Combat Development

(Organization and Operations Concept)

- DA/TRADOC Circulars and Pamphlets

V."

- Army Regulations

- Technical Manuals

- Soldier's Manuals

'.

*Derived from Schulz and Farrel (1980)

2-'1

p.p

°02-13

[ .- S . S S - *
Similarly, if the BCS equipment is not in the Army

inventory, task information should be obtained from the

appropriate non-Army data source (see Appendix E for a

listing of these data sources). Input for this process is

provided by non-Army occupational specialties listed in

column (c) of the BCST worksheet. As with the Army tasks,

, the non-Army tasks should be updated to reflect any

-* equipments or doctrinal changes that were made since the

task data was published.

2.2.6 Assess Adequacy of Task Descriptions

Before entering the task statements obtained from existing

data sources into the BCST worksheet and the SDT, the user

must assess the instructional adequacy of these task

statements. Instructional adequacy refers to the extent to

which the task data has been developed in accordance with

existing Instructional System Development (ISD) processes.

Table 2-3 displays criteria for assessing instructional

adequacy as well as some data sources for obtaining more

detailed guidance on assessing the adequacy of task

descriptions. As indicated in the introduction to this

guide, it is expected that ETES users are thoroughly

familiar with ISD and therefore know what constitutes a

"good" task statement. Hence, detailed instructions on

constructing task statements are not provided in this guide.

If the instructional adequacy of a task statement is good,

you can enter the task statement into column (k) of the BCST

worksheet.

If a task statement is not adequate, you must generate a new

statement for that task (See Step 2.2.7 below).

2-14

" QA P ' * ) . s " ?-.
1*..

4/ -4

-a Q

On. asi A

%P. '404

a -4

0 4) 0 c~ t 9 a az

(d CO 44~

0 a 9 j .0

~~a w 0 W

0. 0 . 1S

-j E. 41 0

-, 4 -4-5 

- - DM 44

-,4 CD~

u -'a 3:' -

oc 0404 t

ma 9 0a

- E4 E- 4

aa 2 154 e, 

U.., It~~.L7
2.2.7 Revise Task Descriptions

Inadequate task statements must be revised in accordance

with existing ISD procedures. Guidelines for the development of task statements can be found in TRADOC PAM 350-30,

Phase I, Section 1.2.3. Additional guidelines are provided

* in the data sources listed in Table 2-2. Once the task

statements have been revised they should be entered into

column (k) of the BCST worksheet.

After the task statements have been entered, task numbers

should be assigned and entered into column (j) of the BCST

worksheet. Table 2-4 provides some general guidelines for

constructing task numbers. More detailed guidance is

. provided in TRADOC Cir 351-28 and AR 351-183.

2.2.8 Enter Data Into Data Base

Relevant data collected in this procedure must be entered

into the SDT. Table 2-5 describes the sequence with which

these data should be entered into the SDT. Additional

guidelines for entering data into the SDT are contained in

the SDT User's Guide. Remember that separate SDT data bases

should be maintained for the BCS and the New System.

2.3 ASSIGN BASELINE COMPARISON SYSTEM (BCS) TASKS TO MOS,

DUTY POSITION, AND SKILL LEVEL

OVERVIEW

During this procedure, the BCS tasks are assigned to MOS,

duty position, and skill level. Comparability analysis is

used as the primary mechanism for making these assignments.

.16

...

..: 2-16

*'1
%'" Table 2-4. Guidelines for Assigning Task Numbers

Enlisted Task Numbers (10 digits) - XXX-YYY-ZZZZ

XXX - Responsible Agency Code

YYY - System Subject Code

ZZZZ - Unique Sequence No. used by responsible

agency to identify each task

Officer Task Numbers (12 digits) - V-W-XXXX-YY-ZZZZ

V - Current Disposition Indicator

W - Prefix (1-Commissioned, 2-Warrant, 3-Both)

XXXX - Basic Subject Area

YY - Additional Subject Area

ZZZZ - Unique Sequence No.

4.

2-17.
L <~~~~~~..... ... . ... . ...... ,. ..... . -,,.._. .m ... ,: ,., .• , q

C14

"0", o*

c*'n

00

C1

E-4

0

.,.

41

o 4 MC o

09

-4

4.)4

o- 0-5-' 4-4E- -z2 C

0 n -4J

S. CE

-- 4

-4

C%4

E--

E. -4

'--1

.. :.::

Co 02 (fl

E..,Co C

-• CO,(

N ',,¢g,, ',' V,-.' , ,.),, ,g, )- -C. -t..-,. .. ,-. ,-- .... .-.. .... , .- ...- . ... ,. . . .
Comparability analysis provides rough-cut assignments that

are only appropriate for the earliest phases of the

acquisition process.l

PROCEDURE

An overview of the steps in this procedure is provided in

Figure 2-6. During the first step, a duty position

structure is identified for the BCS. In the second step,

the BCS tasks are assigned to duty position and skill

- level. In the third step, duty positions are assigned to

- Army occupational specialties. In the fourth step, the MOS,

duty position, and skill level assignments are reviewed to

take into account any manpower or human factors analyses

which may have been conducted during the early phases of

the acquisition program. The BCST worksheet used in

Procedure 2.2 to describe BCS tasks is also used in this

procedure to describe MOS, duty position, and skill level

-. assignments.

:2.3.1 Identify Duty Position Structure

S.. A duty position structure may be defined as a listing of the

number and type of each duty position associated with a

* system. Construction of the BCS duty position structure

occurs in two steps. First, the duty positions structure - for the predecessor system are identified. Second, duty

positions are added or deleted from the predecessor list to

.-. reflect the equipments that were added/deleted to the

@1 predecessor system to form the BCS system.

,4%

1 A description of comparability analysis procedures is

provided in O'Brien (1983).

2-19

* 4 .-
SYSTEM SCS DUTY POSITION

FUNCTIONS EQUIPMENTS TASKS STRUCTURE

12.3.1

IDENTIFY

BE DUTY

POSITION

STRUCTURE

PITIoN

MAKE

PRELIMINARY DUTY POSITION AND

SKILL LEVEL ASSIGNMENTS

"RLMNARY DUTY

KL L LEVEL ASSIGLNMENTS

TO MANPOWER

DETERMINATION

123-3MODEL

TOAANKWE

DESDETERMINATION

MODE

ASIGMET

Figue Oerviw 2-. Dagra: Asign CSTask toNOR

DutyLT Poiton andTSillMevelT2.3)

MANPOWE V2-2DE

% DETERINATIO

MO E ASES 123 * ** %' ...
2.3.2 Assign Tasks To Duty Position and Skill Level

Assignment of tasks to duty position and skill level is a

three-step process. First, the BCS task list must be

examined to identify any tasks which are associated with

duty positions (or billets if a Navy system is involved)

which are not in the BCS duty position structure identified

in 2.3.1. The user must then identify what BCS duty

positions would be most likely to perform the tasks

associated with each of these other duty positio-s. This is

accomplished by comparing the tasks already af )c.ated with

the BCS duty positions and the tasks associ ] with the

other duty positions.

Second, each task should be assigned to a BCS duty position.

If a task is already associated with a BCS duty position,

then that duty position should be assigned to the task. If a

-.. task is not currently assigned to a BCS duty position, it

should be assigned to the BCS duty position which is most

. similar to the duty position from which it was derived.

Third, each BCS duty position should be assigned a skill

level. This is accomplished by identifying similar duty

positions in the predecessor or other similar systems and

assigning the BCS duty position the skill level associated

with that duty position.

2.3.3 Assign Tasks to MOS

Tasks are assigned to an MOS by comparing the tasks

associated with the duty position to the task requirements

of existing MOS that are relevant to the new system. Task

requirements for MOSs can be identified by examining (a) AR

2-21
611-201 which describes the occupational specialties and (b)

the Soldier's Manual for each specialty which provides

detailed data on MOS tasks.

If none of the MOSs listed in AR 611-201 match the task

requirements of the BCS, it may be necessary to establish

requirements for a new MOS. Whenever possible, an existing

MOS should be used since the cost of establishing a new MOS

-. is significant. New MOSs should be designated by the terms

" Xl, X2, X3, etc. in place of the two digits normally used to

designate an MOS. More details on the requirements for

establishing a new MOS are found in AR 611-1.

2.3.4 Assess Integrated Impacts of Assignments

Concurrent with an early training estimation study, there

may be additional studies assessing the system manpower or

human factors requirements. 2 If there are such studies, the

MOS, duty position, and skill level assignments developed in

the previous step should be updated to reflect the more

detailed input that these manpower and human factors studies

typically provide. (Such studi~s typically assess the

workload of operators and maintainers and this workl.oad data

provides more detailed information on what tasks can be

performed by each duty position.)

After the BCS skill level, duty position, and MOS

assignments have been finalized, they should be entered into

* columns (e), (m), and (n) of the BCST worksheet,

" respectively.

- 2 At the present time, such studies are seldom, if ever,

conducted during the early phases of the acquisition

process.

2-22
2.3.5 Document Data

Relevant data collected or developed in this procedure must

. be entered into the SDT. Table 2-6 describes the sequence

Swith which these data should be entered into the SDT.

Additional guidelines for entering data into the SDT are

-". contained in the SDT User's Guide. Remember that separate

SDT data bases should be maintained for the BCS and New

system.

2.4 DEVELOP NEW SYSTEM TASK LIST

OVERVIEW

During this procedure, a task list is generated for the New

system. This is accomplished by adding, deleting, or

modifying BCS tasks.

PROCEDURE

An overview of the steps in this procedure is presented in

the flowchart in Figure 2-7. A worksheet (NTA) for

" documenting the procedure is provided in Figure 2-8.

To begin the procedure, the new technologies identified in

column (f) of worksheet PSH as part of Procedure 1.5 should

be entered in column (a) of the New System Task Generation

Worksheet (NTA). The New system equipment list should then

be entered into Columns (b) and (c) of the NTA worksheet.

Information on New equipment names and numbers can be

obtained from columns (a) and (b) of the Baseline Comparison

System Performance Shortfall Worksheet (PSH).

O"1 2-23
- --- - . - - - -- - . - _ _ - -

0 0 0

Cl 0

* -I,4 I . I. I

* - - ,

U, U Ul I

-

GJ 0 

,- E-4 0 E-4 0 .-0 4 0 0 0

1.. ~ ~ c (n U) U ) U J

0 0 0 0 0 0 0

* 0 04O I 04 I 04

* -4 >9 =,, >, >9 > >9 >9

U )• - 4;. ). 4 ,. ) 4 . 4 . .4

4) -4 - - -4 r q

-- ,

4.1

-4 -4 -4 4

-n ... 4 > ) U0) U0

E 04 0 0 0 0 0

0 0 ,-1 a . a 0

w 04 0- > -, .4 .,->

U) - 1 41 4 J 4.)

-, UIl' U)O 0 N

E-42-24

-4 .4 4

> 00U, >4 CU 

U) En 0

60

I. 0

•-"2-24
ZAJ

NEW SYSTEM SELECT SUBSYSTEM

HARDWAREI SYSTEM

J W -11 

FROM NEW

D" SOFTWARE LIST HARDWARE/SOFTWARE 4--0 LIST

NEW SYSTEM A11.AR1/

SOFTWARE

SUBSYSTEM

DOES

SUBSYSTEM NO INVOLVE 2 DESIGN

IMPACTS

YES2.4,2

DELETE

TASKS NO

LONGER

% REQUIRED

%

SCS

TASK

UNNECESSARY RELEVANT LIST

BCS a LT A FSJKUS TA a S

so

2.4-3 NEW SYSTEM

IMPACTS

0--- 

DESIGN

ADD TASKS

WHERE

REQUIRED

NEW SYSTEM I FUNCTIONAL

RE

4 . QUIREMENTI

ADDITIONAL

TASKS

scs AND NEW

SYSTEM

LHAROWARE/I, 2.4. SOFTWARE LIS % 4 T/

IDENTIFY/ % DESCRIBE

MODIFIED

TASKS

% __j

MODIFIED

TASKS

% 

IV A

4,1

Figure 2-7. Overview Diagram: Develop New System Task List (2.4).

2-25

% !e: e- e- 2 ! , --k -.
TAS

D ESSPTO

ADEQUACY

DESCRIPTIONS

AETEASVS TS

DDESCRIPTION

+ II

Figur 21 (cniud

2-2MN6UDAE

DAA AS

DESCRIPTIONS ~ ~ -.
Figure 2-8.

NTA Worksheet

2-27
° T 

It is only necessary to analyze tasks for those subsystems

L which are associated with design impacts. For all other

subsystems, the baseline tasks can be used without

mod i f i cat ion.

A task list can be generated for the subsystems related to

design impacts by adding or deleting tasks from the

corresponding BCS subsystem. These additions or deletions

should be documented using the codes listed in Table 2-7.

More specifically, the BCS task data should be examined and

. compared to the New system hardware/software design and

functional requirements. BCS tasks associated with

equipments or functions which are no longer needed should be

eliminated. New tasks should be added to reflect new

equipment or functions. New functions can be identified by

examining the functional requirements generated during

- .~Procedure 1.3. An overview of the relationship between

subsystem functions and individual tasks is presented in

Figure 2-9. As Figure 2-9 indicates, individual tasks can

be viewed as lower level functions and these functions can

provide a top-down perspective on task identification.

IIdentification of New system maintenance tasks can be

facilitated by examining the standard list of action verbs

that are used to describe maintenance tasks. A listing of

these maintenance action verbs is presented in Table 2-8.

As part of the system development, a Maintenance Allocation

Chart (MAC) may be constructed for the New system. If a MAC

is available, it can probably be obtained from the New

system program office. The MAC is an excellent source of

task data since it describes which of the standardized list

of maintenance actions will be performed on the system and

at what levels of maintenance these actions will be

performed. Information on the MOSs performing each action is

2-8

• .,2-28.

.'% ................
TABLE 2-7 TASK DELETION/MODIFICATION/ADDITION CODES

TASK DELETION

Code Reason for Deletion

ELI Elimination of subsytem

AUT Task Automation - task now performed

by equipment

RTF Reduced task frequency

MC Change in maintenance concept/doctrine

OC Change in operational concept/doctrine

TASK MODIFICATION

Code Type of Modification

NC No change in task

MIN Minor task modification - task

essentially the same. Only minor

change in equipment/procedure required.

SKI Skill level change - task essentially

the same but assigned to different

skill level.

FRE Frequency change - Same task but task

is performed more (less) frequently

due to change in reliability, etc.

MAJ Major task modification - Significant

change in skills and knowledges and/or

other task characteristics.

TASK ADDITION

Code Reason for Addition

ADD Addition of subsystem

MAN Automated task converted to manual

ITF Increased task frequency

AMC Change in maintenance concept/doctrine

AOC Change in operational concept/doctrine

IT Increase in technology

2-29

- . _ .. - , - : , -. , .- ' -: , ,- j. ,".",, , , -. ' . . - -.- , .*, .
HIGH

LEVEL

FUNCTIONS*

SUBSYSTEM

FUNCTIONS*

INDIVIDUAL TASKS

TASK ELEMENTS

*COLLECTIVE TASKS

Figure 2-9 Overview of Relationship Between

*Subsystem Functions, Collective Tasks 2-30

And Individual Tasks
TABLE 2-8 MAINTENANCE ACTIONS

a. INSPECT. To determine the serviceability of an item

by comparing its physical, mechanical, and/or electrical

characteristics with established standards, through

examination.

b. SERVICE. Operations required periodically to keep an

item in proper operating condition; i.e., to clean

(decontaminate), to preserve, to drain, to paint, or to

replenish fuel, lubricants, hydraulic fluid , or compressed

air supplies.

c. TEST. To verify serviceability and detect incipient failure

by measuring the mechanical or electrical characteristics

of an item and comparing those characteristics with prescribed standards.

d. ADJUST. To maintain within prescribed limits, by bringing

into proper or exact position, or by setting the operating

characteristics to specified parameters.

e. ALIGN. To adjust specified variable elements of an item

to bring about optimum or desired performance.

f. CALIBRATE. To determine and cause corrections to be made

or to be adjusted on instruments or test measuring and

diagnostic equipment used in precision measurement. Consists of comparisons of two instruments, one of which is

a certified standard of known accuracy, to detect and

adjust any discrepancy in the accuracy of the instrument

being compared.

g. INSTALL. The act of emplacing, seating, or fixing into

position an item, part, or module (component or assembly)

in a manner to allow the proper functioning of an equipment

or system.

h. REPLACE. The act of substituting a serviceable like type

part, subassembly, or module (component or assembly) for

an unserviceable counterpart.

i. REPAIR. The application of maintenance services or other

maintenance actions to restore serviceability to an item

by correcting specific damage, fault, malfunction, or

failure in a part, subassembly, module (component or

assembly), end item or system.

2-31
TABLE 2-8 (continued)

j. OVERHAUL. That maintenance effort (service/action)

necessary to restore an item to a completely serviceable/

operational condition as prescribed by maintenance

standards (i.e., DMWR) in appropriate technical publications. Overhaul is normally the highest degree of maintenance performed by the Army. Overhaul does not normally

return an item to like new condition.

k. REBUILD. Consists of those services/actions necessary

for the restoration of unserviceable equipment to a like

new condition in accordance with original manufacturing

standards. Rebuild is the highest degree of materiel

maintenance applied to the Army equipment. The rebuild

operation includes the act of returning to zero those

age measurements (hours/miles, etc.) considered in

classifying Army equipment/components.

,-.'

at/

2-32

.1

-V. € " . . " " ",
of ten included in the MAC. A sample MIAC is provided in

Table 2-9. If a MAC is not available, the user should

consider developing a preliminary MAC as an aid to

maintenance task identification. MACs from comparable

existing systems provide a starting point for development of

a preliminary MAC.

It is possible that a New subsystem may require the same

task as a BCS subsystem. The essential characteristics of

this task, however, must frequently be changed to reflect

the New system requirements. For example, the same task may

be required for both the BCS and New system but the

frequency with which the task is performed may vary

significantly between the BCS and New systems. The task may

have to be performed with greater frequency in the BCS than

in the New system due to use of superior materials and

lubricants in the New system. To account for these changes

to essential task characteristics, all tasks with major

modifications must be identified and the reason for the task

modification must be documented using the code listed in

Table 2-7. Tasks associated with major changes to skills

and knowledges (i.e., tasks) in the MAJ category are

* . analyzed further in Procedure 3.2 to identify the specific

skill and knowledge differences between the BCS and New

* . system tasks. Tasks with minor changes in task elements are

not examined further (it is assumed that their skills and

knowledges have not changed). The task addition, deletion,

* *and modi fication codes should be entered in columns (q) ,

(r), and (s) of the NTA worksheet.

Once tasks have been identified, their conditions and

* . standards should be developed using existing instructional

systems development procedures from the source documents

2-33

"*A
r

 -: -  V r W  r - r - r ,r, • , ••   -• ;   % ; :

TABLE 2-9 EXAMPLE MAINTENANCE ALLOCATION CHART

(21 (2) (41 (5) (6?

Maintenane~ level Tools anc

N U Cum C inPC no m %A sf "OIV Maintenance eQuip. Romwrks

be' f unction C N D IMent

015 COOLING SYSTEM-Con

0535 T Towe A smbiv I nsatct 0.2

Tomt 0.2

Replace 1.0 35 A

Rear4.5 37

Overhow I

06 ELE.TRICAL

05Ci1 Alternator Inspect 0.2

TeEt 0.2

Replace 2.0

Repair 8.0

Overhaul

-" 0602 VltaRegulator Inspect 0.2

Tort 0.2 0.7 %V RepLace 2.0

Repair 1.0

0603 motor, Starting I nspect 0.2

Tau 0.2

Replace 2.0 4

Repair 2.4

Overhoul ~

0607 ConTro i and I riictor I Impact 0.2 ~nel Pa Ter 0.2

Roptace 0.2

Repair 4.0

0608 Pammer Suply. I A fDriver) Inmpact 0.2 4 ' 1Tesn 0.2

Replace 2.0

Repair 4.0

S...l ,0 Interie. External I unsect 0.2

Replace 2.0

Repair 2.0

0609 Domeoligrts I nopec 0.1

Replace 0.7

Lights. Head and Tail I nspect 0.2

Adjue 0.5

elAce 1.5 .

Repair 2.0

OVOrktimes are inCluded in DMWR

2-34.

.~V %

V *lrlpdl4 V .:*
specified in Table 2-3. These same procedures can be used

to assess the adequacy of any new task descriptions which

are generated during comparability analysis.

The completed New system list should be presented to the

Program Office and cognizant school for review.

To complete this procedure, relevant task data should be

entered into the SDT. Table 2-10 describes the sequence

with which these data should be entered into the SDT.

Additional guidelines for entering data into the SDT are

contained in the SDT User's Guide. Remember that separate

SDT data bases should be maintained for the BCS and New

system.

2.5 ASSIGN NEW SYSTEM TASKS TO MOS, DUTY POSITION AND SKILL

LEVEL

OVERVIEW

During this procedure, the New system tasks are assigned to

MOS, duty position, and skill level. Comparability analysis

is used as the primary mechanism for making these assignments. Comparability analysis provides rough-cut assignments

. that are only appropriate for the earliest phases of the

acquisition process.

PROCEDURE

An overview of the steps in this procedure is provided in

Figure 2-10. During the first step, a duty position

structure is identified for the New system. In the second

step, the BCS tasks are assigned to duty position and skill

. 2-35

• .• . • '.-.,.' -, -. '', .. . - -, .. . - . -. ,.-".."° .'A ". -- - -, .. - , - -. -, .. • . . - . . ..
14 M~

044

Uz

.E-4

0 W

-4

C

Ul) Ul)

U)E-4 EU))

Q) Q)

z

04 .~2-36
. . .

!'o -..-.

NEW NEW NEW BCS DUTY SYSTEM SYSTEM SYSTEM POSITION FUNCTIN EQUIPMENTS TASKS STRUCTURE

IDENTIFY

NEW SYSTEM DUTY POSITION

STRUCTURE

NEW SYSTEM

DUTY POSITION

STRUCTURE

MAKE

PRELIMINARY

DUTY POSITION AND

SKILL LEVEL ASSIGNMENTS

RELIMINARY DUTY

POSITION AND

SKILL LEVEL

ASSIGNMENTS

STO MANPOWER

DETERMINATION

EXISTING MAKEMOE

SMOS PRELIMINARY

DESCRIPTIONSO MOS / . .ASSIGNMENTS

p , PRELIMINARY

ASSIGNMENTS MOS

• /REULTSOF /TO MANPOWER

/ oo Duty / Position an , klLv(. 5)

2-37ERMINATION

RESLT AOFIGNENTSL

/ /ANALYSIS/ MOER

OFASIGNMET

MENTS

Figure 2-10. Overview Diagram: Assign New System Tasks to MOS,

Duty Position, and Skill Level (2.5).

2-37
level. In the third step, duty positions are assigned to

MOS. In the fourth step, the MOS, duty position, and skill

level assignments are reviewed to take into account any

manpower or human factors analyses which may have been

. conducted during the early phases of the acquisition

program. The NTA worksheet used in Procedure 2.4 to

describe New system tasks is also used in this procedure to

describe MOS, duty position, and skill level assignments.

2.5.1 Identify Duty Position Structure

A duty position structure may be defined as a listing of the

number and type of each duty position associated with a

system. Construction of a New systema duty position

-- .structure occurs in two steps. First, the duty position for

the BCS system is examined. Second, duty positions are

added or deleted from the BCS list to reflect the equipments

or functions that were added/deleted to the BCS system to

form the New system.

2.5.2 Assign Tasks to Duty Position and Skill Level

Assignment of tasks to duty position and skill level

involves a three step process. First, the New system task

list must be examined to identify any tasks which are

associated with duty positions (or billets if a Navy system

is involved) which are not in the New system duty position

structure identified in 2.5.1. The user must then identify

what New system duty positions would be most likely to

perform the tasks associated with each of these other duty

positions. This is accomplished by comparing the tasks

already associated with the New system duty positions and

the tasks associated with the other duty positions.

2-38
Second, all tasks should be assigned to a New system duty

position. If the task is already associated with a New

system duty position, then that duty position should be

assigned to the task. If it is not currently assigned to a

New system duty position, it should be assigned to the New

system duty position which is most similar to the duty

position from which it was derived.

Third, each New system duty position should be assigned a

skill level. This is accomplished by identifying similar

duty positions in the predecessor of other similar systems

and assigning the New system duty position the skill level

associated with that duty position.

2.5.3 Assign Tasks to MOS

Tasks are assigned to MOSs by comparing the tasks associated

with the duty position to the task requirements of existing

MOSs which are relevant to the New system. Task requirements

for MOSs can be identified by examining (a) AR 611-201 which

describes the MOSs and (b) the Soldier's Manual for each

specialty which provides detailed data on MOS tasks.

If none of the MOSs listed in AR 611-201 match the task

requirements of the New system, it may be necessary to

establish requirements for a new MOS. Whenever possible, an

existing MOS should be used due to the significant cost of

establishing a new MOS. New MOSs should be designated by

the terms Xl, X2, X3 etc. in place of the two digits normally used to designate an MOS. More details on the requirements for establishing a new MOS are found in AR 611-1.

2-39

...
;0"-

2.5.4 Assess Integrated Impacts of Assignments

Concurrent with an early training estimation study, there

may be additional studies assessing the system manpower or

human factors requirements. If there are such studies, the

MOS, duty position, and skill level assignments developed in

the previous step should be updated to reflect the more

detailed input that these manpower and human factors studies

, typically provide. (Such studies typically assess the

workload of operators and maintainers and this workload data

provides more definite information on what tasks can be

performed by each duty position).

After the New system skill level, duty position, and MOS

assignments have been finalized, they should be entered into

columns (e), (m), and (n) of the NTA worksheet,

respectively.

2.5.5 Document Data

Relevant data collected or developed in this procedure must

be entered into the SDT. Table 2-11 describes the sequence

with which these data should be entered into the SDT.

Additional guidelines for entering data into the SDT are

contained in the SDT User's Guide. Remember that separate

SDT data bases should be maintained for the BCS and New

system.

2-40

1"" ' " """ ' ' ." -" " " -" " "4 ' ' " """ " " " " '""" " " '" " " . . - " " " " """"" "
0'0

0 0 0

0 0n 0 

czz

00

1-4~

4-i-

* d ~ C C C C

0)0 0 0 0 0 0 0 4-i.4 -4 .4 -4 i4 -

C4. .Aj 4.) 41) 41 41~ 4-

(n (n (n U) U ) U

-. 00 0 0 0 0 0 0

0- 04 0 0 a 04 0

-- 4

100

4-3 0

* 4 0 z- U)f * -4 3aC

*I 1 90 * N >1 -1 id4) -4 to cfn 4)

(D 0l (1 (1 (ae'to

.00)>1 0 r= E E-4 - *E-4 z a) W) 0) 0)4- 4.) 413 4.) 4.) C 3 X Ul U) U) U) 0 0 N -~ >1 >4 >I. .-

w Cl) En U) rn 41 41- -4 '

zU z- U)4

Ln LA LA ULA LA) LA L

C4 I C4 

* I 2-41
-- p

'4

V.'.

THIS PAGE INTENTIONALLY LEFT BLANK

S.

.4..

*%* .~

4.

V.

@2

5'.

'.5

.5. 2-42

-7

'5
SECTION 3 - ESTIMATE TRAINING PROGRAM

OVERVIEW

During this procedure, an estimate of the institutional

training program is developed for both the Bazeline

Comparison System (BCS) and New System. The major products

of this procedure are (1) quasi-program of instructions

which outline the contents, methods, and media to be

included in each institutional training course and (2)

training paths describing the sequence with which these

courses must be taken in an MOS.

PROCEDURE

This procedure consists of eight lower level procedures (See

Figure 3-1). During this first procedure, tasks requiring

training are identified. During the second procedure the

tasks are assigned to training settings. In the third

procedure, the general skills and knowledges associated with

the tasks to be trained are identified. In the fourth

procedure, a target population description is developed. In

the fifth procedure, the sequence in which the tasks will be

trained is identified. In the sixth procedure, quasiprogram of instructions are constructed. In the seventh

procedure, the automated Media Selection Program is used to

assign tasks to media. In the eighth procedure, training

paths are identified. It should be noted that the present

version of ETES only estimates training programs for

individual institutional training courses.

3-1

., . .. ,o .. < : - ... v-, .. ... X . F -, -,.-,
7 . ~ -- 

r

7-77-

t7,.

TRAINNoG

TRRAINING

ATASKSMENT

3-3

........................................... - - -

.*. -- ~ ~ ~ ~ ~ ~ ~e ~ ~ ~ .as ~ ~ ~ ~ SIL AND--- . . .
3.1 SELECT TASKS FOR TRAINING

Once a task inventory has been compiled, the tasks in this

inventory must be analyzed to determine which tasks require

training. Since the task inventory for the typical MOS is

composed of several hundred tasks, it is inappropriate to

even consider training all of these tasks. Thus, the Army

cannot formally train the individual soldier on all of the

tasks required for full job proficiency. Even if the

resources were available such a strategy would not be cost

efficient considering the relatively low rate of soldier

reenlistment and career progression.

This procedure describes how to select tasks for training

(or conversely, to identify tasks which will not require

training). This procedure provides several alternative

methods for making the train-no train decision. These

methods differ in terms of (1) the criteria used to select

tasks for training and (2) the techniques used to aggregate

the scores on the criterion variables. An overview diagram

depicting the steps contained in this procedure is presented

in Figure 3-2. Procedure 3.2, Assign Tasks to Training

Settings, will assign the tasks to be trained to training

settings. The major steps in the task selection process

include choosing a task selection and aggregation method,

obtaining task factor ratings/scores, applying criterion/

cutoff scores to develop a tentative task list for training,

reviewing these tasks before making final task selections,

and documenting the data associated with the procedure.

3-3

•4 . . . f , . . . . . . . • . . ,, .
- ASK SELECTION TASKS METDS

ALGORITHMST

CHOOSE

TASK SELECTION

& AGGREGATION

METHODS

RATE/

SCORE TASKS; ON

REVELANT

CHARACTERISTICS

TASK

FACTOR

SCORE$/

RATINGS

APPLY

ALGOAITHMf

AGGREGATIONI

METHOD

TENTATIVE TA KV

SELECTION

REVIEW

CURRENT TASK

SELECTION

FINAL TASK DLOCUMENTIED

SELECTION DOCUMENT DATA

DATA

Figure 3-2 Overview Diagram: Select Tasks for Training

3-4

" -.- .- * * . . . - - . * .

,. a:t.s..=o S. ~ oo. o.s .. ,oAat.a
3.1.1 Choose Task Selection and Aggregation Methods

OVERVIEW

There is no single task selection method which can meet the

needs of the different phases of the acquisition process and

be effectively applied across all occupational specialties.

Selection of tasks for training requires a management

decision based on the best data available. A number of

alternative methods have merit and, depending on the

characteristics of the specialty analyzed and the requirements of the analysis, may be appropriate for task selection. A summary of these methods is provided in Table 3-1.

An overview of procedure 3.1.1 is provided in Figure 3-3.

PROCEDURE

3.1.1.1 Identify Task Selection Requirements

In order to choose a task selection methodology for a

particular application the user must determine the (1)

accuracy requirements, (2) the data availability, and (3)

resources available to conduct the analysis.

During the earliest phases of system acquisition, when

several alternative system concepts are still being considered, a high degree of accuracy may not be required. The

optimal approach may be one with moderate accuracy and low

data requirements, that is, a "quick and dirty" approach.

During the later phases of system acquisition, when the

accuracy of results is critical for training resource

.., .A requirements analysis, more data intensive and accurate

methods may be preferable.

0%" W -
L .. CL. 0

0 C

0

0 CL

00

2, 0 0 0.

5 0 CL

ft' I- CL -- L to

.sa ~C 

.2 X

CL a C

c . a3 S .C .co m ,a

M E ~ m r E

cc 0 a E 0 5c

0 0- 0 000 0

4- 0 0 0

-a c.0 Co~ 0 rE

0 cc CD w

X .2

a- x (U 0 WE W~ E5 E U- uji . 1 LU- CIMC

o oe 0 SO 0 S 0 S 0

CL > . U. 0 LL

a 3-6
TASK

SELECTION TASKS MGGETON

ALGORITHMS

IDENTIFY

TASK SELECTION

REQUIREMENTS

3.1.1.2

CHOOSE

TASK SELECTION

METHOD

TASK SELECTIO

METHOD

3.1.1.3

SELECT

AGGREGATION

METHOD

METHOD

Figure 3-3 Choose Task Selection and Aggregation

Methods (3.1.1)

3-7
Thus, the determination of the data an d accuracy

requirements of a study must precede the sel- "ion of an

approach to task selection.

3.1.1.2 Choose Task Selection Method

Four task selection methods are presented in Table 3-1.

These methods are described in terms of (1) their data

requirements and (2) their estimated accuracy. For example,

comparability analysis has the fewest data requirements and

can therefore be applied early in the acquisition process.

However, the accuracy of the results does not compare

favorably with the alternative methods. Thus, comparability

analysis should be used only when a quick and inexpensive

analysis is required in the very earliest period of the

acquisition process. The other methods listed in Table 3-1.

have progressively increasing data requirements and

corresponding improvements in analysis accuracy. These

methods are described in the paragraphs which follow,

O Comparability Analysis

Comparability analysis may be used to provide crude task

- training assignments during the early phases of the

* acquisition process. The first step in the application of

comparability analysis is to identify a comparable existing

task for each of the tasks in the New System. A comparable

existing task may be defined as a task which (1) is

performed on comparable equipment or (2) involves the same

. type of task action (that is, the action verb for the

comparable task matches or is highly similar to the action

verb for the New system task).

3-8

0''1,
The second step in comparability analysi. , is to assign the

New system task to the same training categories (training,

no training) assigned to their comparable tasks.

0 DIF Model

* Another strategy for selecting tasks for training is a

scheme called the DIF (Difficulty, Importance, Frequency)

analysis. This methodology is a simple approach to a

complex problem, in that it integrates several different

criteria during the task selection process separately. The

supervisor and incumbent are asked to determine the

difficulty of the task in terms of learning and performing,

then asked to comment on the importance of the task, and

finally to estimate the frequency of task performance.

In determining the "difficulty" of the task, these responses

encompass both task learning difficulty and probability of

deficient performance. For the decision on "importance" the

respondents' conscious or unconscious responses may well

include factors such as the sequences of inadequate

performance, the task delay tolerance, percent performing,

and time spent performing. When estimating the "frequency",

both frequency data and the time between job entry and task

performance might be considered.

This approach to task selection implies that asking the

supervisor and the job incumbent three simple questions will

facilitate the task selection process. This approach has

1 The descriptions of the DIF, Four Factor Model and

Eight Model presented in this section were derived from

TRADOC PAM 351-4.

3-9

.. .
7,

merit because of its simplicity yet degree of precision it

may achieve. The survey would be conducted by the service

* ischool (field surveys) and requires a relatively small

*. sample (40 incumbents and 40 supervisors). A graphic

portrayal of this sequence is described in Figure 3-4.

After going through this branching process it can be seen

that at one end of the scale are different, important, and

infrequently performed tasks. Each of these instances has a

different rationale for whether to train the tasks. The

sophistication and sensitivity of this technique can be

increased by incorporating degrees of importance,

difficulty, and frequency, and by using more detailed levels

of training as shown in Figure 3-5.

o 4-Factor Model

The 4-Factor Model for task selection consists of four

criteria for task selection that might influence the

selection of tasks for training. These criteria are:

o Percent performing,

o Task delay tolerance,

o Task learning difficulty, and

o Consequences of inadequate performance.

Information on these four-factors is collected from job

holders. Information can be collected directly by the

analyst or from surveys administered by the Army

Occupational Survey Program (AOSP) of MILPERCEN. Once

information has been collected for each of the four factors

* for every task on the task inventory, there will be a large

data base to use for selecting tasks for training. These

3-10
T- P

YES 'TRI4

FREQUENT 

--

Y E S N O - - -' - - - 6OV E R T R A I N

,AI.IMPORTANT

YESNOY S~

TASKO DIFCL ~RAININ

YES

NO ~~,N FREQUENT.--.4

YESS

MPORTANT ~ ~ PFREQUENT-.-4 pNOTANG

NO YEYES

NO NO TRAINING

Train -Trainee must be able to demonstrat- proficiency in performing

* task at speed required on the jof..

Overtrain -Trainee must be trained to a high standard of retention.

Accomplished by reinforcement training.

-., No Training -Formal training not required. Skills can be acquired on

the job.

lei

Figure 3-4 DIF Model

* 3-11
zz z 0zD

z zzz z z z Z z z

<- I-- <. <<

wU Z wZ M Z m 2 cc Zr m x

0 Z 0Z z z

P UL LU LU I

. W .LU >. LU LU LU Ls LUc

cc a cc cc a 0 CC cr a CC cc 0 cr

w OLL LU 0 z LO UUo L. L L L a L L U LL

2 0 0 0 LU 0

LU LU LU LU LU LUJ

U. L. LL LL. U. U

LU ZL z u

In YYY 0AI- I I- LL

z z L

4 4 4

LU

LL.

IL

3-12
factors are summarized in Table 3-2 and are described in

more detail in Procedure 3.1.2.

o 8-Factor Model

The 8-Factor Model contains eight criteria including those

comprising the 4-Factor Model. The additional four criteria

are:

o Percent of time spent performing,

o Frequency of performance,

o Probability of deficient performance, and

o Immediacy of performance or time between job entry

and task performance.

As in the 4-Factor Model, information can be collected from

job holders directly by the analyst or from AOSP surveys.

The 8-Factor Model provides a more comprehensive basis for

selecting tasks for training than does the 4-Factor Model,

but, as stated earlier, requires more extensive data

collection. The factors listed above are described in more

detail in Procedure 3.1.2.

Selecting tasks for training is probably the most important

management decision made early in training design. The

tasks selected for training (1) provide the specialty base,

(2) drive the production of training materials, Soldier's

Manuals, SQTs, and (3) are the basis of the specialization

within each specialty. If all tasks cannot be trained, then

the tasks that are trained must be the right ones in terms

of job proficiency, transferability, and combat criticality.

Each of these models described above requires a different

amount of analysis effort and time. Each model has certain

3-13

4-
Table 3-2. Data Sources for 4-Factor Model

Task Selection Factor Source

-Percent Performing Percent of soldiers surveyed

% .who indicate they perform the

task

- Task Delay Tolerance Average of responses from l(low)

- Task Learning Difficulty - 7(high) indicated by soldiers

- Consequences of Inadequate surveyed who perform the task

Performance

3-14

'6° - .. . . . .** .
:. . . .

advantages and limitations such as ease of use, accept-

* , ability by field, comprehensiveness, timeliness, etc.

Based on the data requirements of the alternative selection

methods, the accuracy requirements of the task selection

process, and an understanding of the differences among the

alternative procedures, the analyst must choose a task

selection method.

3.1.1.3 Select Aggregation Method

In each alternative task selection procedure, some method

must be used to aggregate data. Thus, in the eight factor

model for example, ratings on eight criteria have been

obtained for each task. These ratings must be combined to

yield a single measure of the requirement for training.

Because of the comprehensiveness and complexity of the

factors, it is not easy to simply combine all factors and

pick the most important tasks for training. One aggregation

strategy requires the command to designate the most

important factor(s). A value is specified as a cutoff point

for that factor. If a task is above that cutoff point on

that factor, it is selected for training. All the factors

can then be considered, with respective cutoff scores

indicating the most important tasks.

For example, for combat tasks, the factor of Consequences of

Inadequate Performance may be important. Thus any task over

O 6.0 (series) would be selected. Perhaps the commander might

also feel that a large percentage of performers indicates a

task should be trained. Thus, only tasks with over 50% of

the incumbents performing would be selected for training.

3-15

., -- ,.'..- '_,,, _' - _'.. . . -
" . Combining these two cutoff scores would mean that all tasks

performed by 50% of the soldiers and/or scoring 6.0 or

higher on Consequences of Inadequate Performance would be

selected for training (Note that the establishment of the

cutoff point is somewhat arbitrary).

Alternatively, a method can be devised for aggregating a set

of ratings on task factors to form a summary index of

training priority for each task.

Tasks are then rank ordered by the training priority index

and cutoff values are established indicating the number of

tasks that will receive initial training in institutional

courses. Cutoff values can be developed by first examining

the number of tasks currently assigned to each training

setting in the MOS (this information can be obtained from

the Trainer's Guide for the MOS). For example, the Trainer's

Guide might indicate that 30 tasks currently receive initial

training in institutions. The cutoff values that are

currently employed in the MOS must then be examined and

updated to reflect (1) projected resource constraints in

each training setting, and (2) major changes in the learning

difficulty of the tasks. Initial assignments to institutional training and no training categories are made by

applying the cutoff values to the training priority

rankings.

Table 3-3 contains a list of alternative algorithms for

aggregating task factor data. Table 3-4 indicates which

algorithms for aggregating task factors are appropriate for

each task selection model.

3-16
Table 3-3. Potential Algorithms for

Aggregating Task Factors

Weighted Z Scores

n

TPI= Z W Z

%" I=i

where TPI is the training priority index; W is the weigt assigned to

the i the task factor; and Zi . is the Z score of the i task on the j

factor; and N is the number o' task factors.

zij =xi - xJ

SDj

th th

where X is the raw score of the i task on the j factor; X is the

mean of he X.. for the j factor; and SD is the standard deviation of

the X for t A jtn factor.

ij

Individual Task Faztor Criteria

In this method, cut points are established for individual task factors.

Thus, for example, a cutoff point of 6.0 could be established on the

factor, consequences of inadequate performance. As a result, any task

with task factor scores exceeding 6.0 would be selected for training.

Similar criteria could be established for other task factors.

e.g., Select for training if:

> 6.0 on Task Factor 1

> 5.0 on Task Factor 2

> 5.0 on Task Factor 3

Evaluate each task against criteria for each task factor.

Task Factor Criteria Algorithms

A task factor criteria algorithm specifies, for a particular combination

of values on task factors, whether a task should be trained. Such

algorithms are not really used to aggregate task factor scores as much

as to lead the analyst to a solution.

3-17
.L . . -. .. . ,

 . . . . . - . -- -. . - --  .-  -.   

Table 3-4. Appropriate Aggregation Algorithms for Alternative

Task Selection Models.

Aggregation Idvda

AlorthsWeighted Task Task Factor Al o1h sz Factor Simple Criteria

Task Slection Model Scores Criteria Weighting Algorithms

Comparability Analysis

DIP

4 Factor

-jJ

8 Factor

3-18

-1 6! A A
3.1.2 Rate/Score Tasks on Relevant Characteristics

OVERVIEW

During this procedure, methods are designed to collect data

on the task factors required for the method selected in

Procedure 3.1.1.2 and the data are collected and compiled.

An overview of the elements comprising this procedure is

depicted in Figure 3-6.

3.1.2.1 Design Methods for Rating/Scoring Tasks

PROC EDURE

In this step, methods for rating/scoring tasks are devised,

based on the task selection method being used. Procedures

for each task selection method are described below.

o Comparability Analysis

When using comparability analysis no ratings/scores are

obtained on tasks. The user can proceed to Procedure 3.1.3.

o DIF Algorithm

An example of a data collection format which can be used to

employ the DIF model is presented in Figure 3-7. A series

of three questions is presented to job incumbents/

supervisors in the sequence shown in the figure.

Of course, this format should be tailored by each proponent

school to meet its specific needs.

3 - 19

0:5-
TASK

TAS K SELECTION

ALGORITHM

3.=12. 1

DESIGN

METHODS FOR

RATING/SCORING

TASKS

METHODS FOR RAT ING/

SCORING

RATE!

SCORE TASKS

TASKDAT

Figure 3-6 Rate/Score Tasks on Relevant Characteristics (3.1.2)

3-20

. ~.. . . . . . . . . . . . . . . . . . . . . . . .
JOB UPERVISOR'S10 Z 4(

INCUMBENT' ~ - o RESPONSE 4( 4

RESPONSE- 4

4:AT 

o-4(- 4-oL$

TASK' COMMENTS: '~ .,,COMMENTS:

X

Y

* z

NOTES: 1 TOTAL NUMBER OF INCUMBENTS AND SUPERVISORS MUST BE

DESIGNATED BY PROPONENT SCHOOL.

2 IF POSSIBLE DETERMINE WHAT WAS DIFFICULT:

LEARNING? PERFORMANCE?

3 THIS SCALE MUST BE DETERMINED BY PROPONENT SCHOOL.

A NOTIONAL CATEGORY COULD BE:

a. VERY FREQUENT-NEARLY EVERY PROJECT.

b. MODERATE FREQUENCY-+1 X PER WEEK.

CINFREQUENT-+1 X PER MONTH.

Figure 3-7 DIF Data Collection Format

3-21
0 4-Factor Model

- Data must be obtained on the four factors in this model

l either from the Army Occupational Survey Program (AOSP) of

MILPERCEN or directly from surveys administered to job

incumbents/supervisors. Each of the four factors is

described in detail in the following paragraphs.

Percent Performing

The criterion, "percentage of job incumbents who perform the

task", indicates those tasks that are most often performed

on the job.

For example, one task for weather technicians is "to answer

telephone inquiries about the weather." If you found that

96 percent of all weather technicians performed this task,

the implications for training would be different than if you

found that only 10 percent performed it.

In the above example, if only 10 percent of job incumbents

perform a task, there is a strong probability that 90

percent of your training resources would be wasted if you

trained all weather technicians to perform the task. ]1h

order to obtain data for determining the percentage of job

incumbents performing each task, simply ask "Do you perform

this task?" or derive the percentage performing from answers

to other related questions. If, for example, in collecting

"frequency of task performance" data one of the possible

*I responses is "never" or "do not perform", you will have the

basic data for determining percent performing.

3-.

" ,.'.3-22

,e
Task delay tolerance

The delay tolerance of a task is a measure of how much delay

can be tolerated between the time the need for task performance becomes evident and the time the actual performance

begins. There are some tasks normally encountered by job

incumbents in which no such delays can be tolerated. The job

incumbent who encounters the task must be capable of doing

it then and there, without needing to read task procedures

or finding someone to advise him or take over completely.

For other tasks, a delay of a few minutes or perhaps half an

hour might be quite acceptable, or even mandatory, while the

job incumbent gets advice, checks technical orders,

regulations, etc. And for some tasks, there might be time

to assemble a group of experts to confer before

proceeding. The delay tolerance of a task is a measure of

how much delay can be tolerated between the time the need

for task performance becomes evident and the time actual

performance begins. The following are examples of low delay

tolerance tasks requiring immediate pe-formance.

EXAMPLES

1. Use artificial respiration to restore the breathing of

an accident victim.

2. Pull ripcord of emergency parachute if main parachute

fails.

3. Warn suspect of his legal rights before questioning.

4. Film historic occasion for official records.

4 5. Extinguish fire in aircraft engine during startup on

flight line.

Tasks determined to have a low delay tolerance should be

given relatively high priority for selection for training.

o 3-23
Examples of tasks having a higher delay tolerance, thereby

permitting performance delay, would include:

[" . EXAMPLES

1. Review books for unit library.

2. Refill fire extinguisher after use.

3. Advise major command of unit manning problem.

4. Fit microphones in aircrew oxygen masks.

A high delay tolerance does not exclude a task from

training, but indicates that other factors will be more of a

* basis for acceptance or rejection. To obtain data on this

criterion, individuals who are familiar with the job are

*. asked to rate the amount of delay that can be tolerated

before task performance begins, according to the categories

listed below.

(1) Extremely low - performance can be put off

indefinitely: almost never urgent.

(2) Low.

(3) Somewhat below average.

(4) Average.

(5) Somewhat above average.

(6) High.

O, (7) Extremely high - task performance must begin

instantly.

Task learning difficulty

The learning difficulty of a task refers to the time,

effort, and assistance required to achieve performance

proficiency. Some tasks encountered in each specialty are

O so easy or so familiar that they can be readily "picked up"

on the job without formal training. At the other extreme,

, . 3-24 .

O_ * 
some tasks are so complicated that a job incumbent can

perform them adequately only after a sustained period of

formal training. Other tasks lie somewhere between these

extremes and require different levels of training. Tasks

easy enough to be "picked up" on the job without training

might be:

EXAMPLES

(1) Sweep floors.

(2) Collect food trays from patients in hospital

wards.

(3) Distribute unclassified correspondence in an

office.

Tasks requiring lengthy, formal training might be:

EXAMPLES

(1) Diagnose malfunction in an airborne radar weapons

system.

(2) Defuse unexploded enemy bombs.

(3) Identify parasites in clinical specimens.

To obtain data on the factor of task learning difficulty,

job incumbents or supervisors might be asked to rate tasks

as to the training time required to achieve proficiency, or

as to the difficulty of "picking up" the task on the job

without formal training. The following rating scale might

be used:

(1) Extremely low - extremely easy to "pick up"

without systematic training.

3-25
(2) Low.

(3) Somewhat below average.

(4) Average.

(5) Somewhat above average.

(6) High.

(7) Extremely high - extremely difficult to learn

without systematic training.

Consequences of inadequate performance

The criterion of consequences of inadequate performance

points to the need for selecting tasks for training that are

essential to job performance, when needed, even if they are

seldom performed. The consequences of inadequate

performance on certain tasks could result in injury to

personnel, loss of life, or damage to equipment. Inadequate

performance could have a serious impact on the mission, thE

operation, the product, the equipment, or the operator.

EXAMPLES

(1) More and more electronic equipment is being

transistorized and is therefore less subject to

malfunction. This fact should reduce the amount

of maintenance training that an operator

requires. However, there may be a number of

malfunctions that, although occurring very

infrequently, would be extremely critical if

immediate corrective action were not taken by the

O operator. Severe damage to the equipment,

materials, or products might result. Under such

circumstances, the criticality of the infrequently

used tasks is so great that it must be considered

in choosing tasks for training.

3-26 ,-S-
(2) The probable consequences of inadequate

performance of such combat tasks as "identify

enemy aircraft" could be loss of life and

equipment.

(3) The probable consequence of inadequate performance

of the task of "write trip report" is

negligible. If this task were selected for

training, it would be on the basis of factors

other than probable consequences of inadequate

performance.

-%%

To obtain data on this criterion, individuals familiar with

the job are asked to rate probable consequences of

inadequate performance of each task according to the

categories listed below.

(1) Extremely low - if performed inadequately,

consequences are negligible.

(2) Low.

(3) Somewhat below average.

(4) Average.

(5) Somewhat above average.

(6) High.

(7) Extremely high-inadequate performance may result

in mission failure, injury, death, or damage to

important equipment.

N Tables 3-5 and 3-6 contain sample sets of data collection

forms for the four factor model.

O0.

3-27

.0
Table 3-5. Sample Collection Form Using 4-Factor Model.

z

-Cr

<I U3 

-1 0

0. '&.

0 0 CD TASKU TILEC

1.~~0 MA2VRA PRPR A 

3-28
(n~ 0

-H

H J . -

ai 0 ca

-H4. Q -4 -u X

0 . 0n 0 cu

C ca 0 W

01 0I A.- "~ 0

HI2: 0 w -

i Q- ) 0 >1 -H -W

>S 0 w0 C) .0 -4 4-

c C w 0 E r. 0

i 0 H H Li C. -H

4 .H 0 w- 0 0

0 o E-4 c. cc cJ C 0 E

4-) -4 u ui 0 C) 4cI

Z 0. Cd0. L

r-4 (V C . ) 0 ~

E- 0- ) ) L

4- E- -0 ca 1 0 *4- cn - E0 Zi -4 ~ I ~ > 0. 

~C 4 -H a ) .1 Q) E) 0 C4C0 (-4 0 C4)

.--4 M -4-4a P4 W ~ Lia 41 0 CL 

z OH 0. t

H Z E-' C4- 0 a) i

Iz-2 4. co,-

M ~ 0 0. (( >

0 (0 t . =

w) 41 -i

-4 0 ( ) Q).0 .

0 01 w EnI (4 (1 -H Q.. C) 0) M1 M)

ca m' 0 41 zi (2 .

1. H Z3 Zl t) Li

cm z n cu >N

c ~ 0 0a

-~ 3-29
JO"7

201

0.

>

VIS.

0

E 0

,z L,- Cc 

C.0

C

"3-30

0(0

ai -. 
.~%w.-.-.-.-.% .-.-- -

--------------------------------------.-.... .*-~ . -

p. -~

F~ ~ 4W, zai I

4-,

S.-.

4-

S

4.. -~

6

StJ/ , . . - . . . .

............................................. Q. -. ., - - -
L.U

0.

Ejr N" N

'3--

CuOc f" c

Co.C _ Co CoCo>C

! -:,

• , I I I 1 " I

*. -. tci

..

o~ I-

* CuN "N IN N N "N IN

*I- -

x

3-32

"'".-v ,-...., .-'-.- .'.-.-'...-.. .- '.-- ..'... _ .. ....-. " " " "" " - "" "'" - . " ,*
3:0 Co LO La0 Cn o M(

.u0

1C0

cc~

0 &

ECO0

Cu0

U)U

~-3-33
E-g>

co a a

NO NOCO C4 CO C14

to'

'U

I-3-34-
0

o 8-Factor Model

Data collection for the 8-Factor Model. is similar to that

.. for the 4-Factor Model. Those factors in the 8-Factor Model

. which are not part of the 4-Factor Model are described in

the following paragraphs.

Percent of time spent performing

The percent of time spent performing a task is a criterion

suggesting that training be provided to assist job

incumbents in efficiently performing their most time

consuming tasks.

EXAMPLE

In the Protective Equipment/Pressure Suit Specialty, 6.67

percent of average time of all members is spent performing

the task of "maintain rigid survival kits". Only 0. 16

percent of average time is spent on the task of "instal]

microphones in oxygen masks." If more effective training

could increase productivity 50 percent in the first task,

3.33 percent of the total time of all members might be

saved. This would represent a sicinificant savings. However,

if productivity were increased 50 percent for the second

task. Only 0.08 percent of the total time would be saved.

This represents a much smaller savings opportunity.

To obtain data for determining the percentage of time spent

performing the tasks in a speci al y, inputs are required

from a la:ge number of job incumbents. Usually they are not

asked to state the percentage of their time spent on each

task because such a question would be very difficult to

- 3-35
answer. Instead, they are usually asked to rate each task

as to the amount of time spent performing it as compared to

their other tasks.

EXAMPLE

You are to rate the relative amount of time you spend

performing each task in your present job. Relat ive time

spent means the total time you spend doing the task compared

with the time you spend on each of the other tasks in your

present job. The following scale might be used to rate the

relative amount of time spent on each task:

1 very much belw average.

2 below average time.

3 for slightly below average time.

4 for about average time.

5 for slightly above average time.

6 for above average time.

7 for very much above average time,

From the above information- a computer program such as

MILPERCEN's Comprehensive Occupational Data Analysis Program

(CODAP) can compute and print out the average percent of

time spent by members of the specialty who perform the task,

and the average percent of time spent by all members of the

specialty.

Frequency of Performance

- While the probable consequences of inadequate performance of

a particular task are serious and the task delay tolerance

in low, the task might still rate low for training priority

if it is rarely performed.

3-36

-. . ..- - . " ." " ' I . - - - : . ' : ". : I H" " "
EXAMPLE

For a medical corpsman, the task of "deliver baby" is so

rarely performed that it probably would not be trained in

spite of the serious consequences of inadequate performance

and the relatively low task delay tolerance.

On the other hand, if a task is pe.r-formed frequently, the

pay-off in terms of return on training dollars expended is

likely to be great, particularly if there is a known "best

way" to perform the task. A practical way to collect

frequency of performance data on tasks is to rate their

frequency of performance on a scale such as the following:

(1) Never perform.

(2) Once a year.

(3) Once every three months.

(4) Once a month.

(5) Once a week.

(6) Once a day.

(7) Several times a day.

Probability of deficient performance

The criterion of probability of deficient performance

insures that training is given in 1hose essential job skills

in which job incumbents frequently perform poorly. In any

job, some tasks are more difficult to accomplish (or easier

O to bungle) than others. By tabulating the judgments of

knowledgeable personnel regarding the probability of

deficient performance, a list of these poorly performed

tasks can be produced. Training of these tasks, regardless

* of their criticality, must be given serious consideration.

EXAMPLES

O 1. If equipment downtime iL, often caused by faulty

soldering, this skill may require additional

3-37
emphasis in a list of tasks selected for training

of repairmen.

2. If widespread theft of items guarded by military

police is a problem, the task of "guard packages,

materials, and property" and "prepare physical

security plans" may require additional emphasis.

To obtain data on the criterion of probability of deficient

performance, supervisors of job incumbents might be asked to

rate each task as to how often, according to the scale

below, subordinates in the MOS perform the task in an

unacceptable manner:

(1) Very much below average.

(2) Below average.

(3) Slightly below average.

(4) Average.

(5) Slightly above average.

(6) Above average.

(7) Very much above average.

Immediacy of performance or time between job entry and task

performance

The criterion of the time interval between completion of

training and initial performance of the task on the job has

some significance in selecting tasks for training. Here, the

determining factors are:

"0

(1) Whether or not there is a high probability of a

graduate encountering the task on the job fairly

soon after completing training. "Fairly soon"

means, in this context, that task encountered

0 within the first year after training would,

3-38

0
'O

everything else being equal, be weighted more

heavily for selection than those not encountered

until one to two years later and

(2) The predicted or measured amount of decay of the

skill that will take place during the time

interval.

EXAMPLE

The ability to send and receive Morse Code is a relatively

difficult skill to acquire. If the skill is not used, a

considerable amount of decay is certain to occur. If the

* skill is only rarely needed by personnel, it may be wise to

S- .exclude the task in the training given to all trainees.

However, if the skill is likely to be used immediately after

graduation by most graduates, it probably should be included

in the training for all trainees.

To obtain data on this criterion, job incumbents and others

might be asked to rate the time between job entry and task

performance on a scale such as that listed below.

(1) Task not yet performed.

(2) Task first performed more than 4 years after

assignment.

(3) Task first performed between 2 and 4 years after

assignment.

(4) Task first performed between 1 and 2 years after

*[ Oass ignment.

(5) Task first performed between 6 months and 1 year

after assignment.

.

0"- 3-39.

O .
(6) Task first performed between 3 months and 6 months

after assignment.

(7) Task performed during first 3 months of

assignment.

Sample data collection forms for those factors not included

in the 4-Factor Model are presented in Table 3-7.

3.1.2.2 Rate/Score Tasks

During this procedure, data collection takes place using a

minimum of 10 to 20 raters for each task. Additional raters

may be required if the total number of judgments required of

each rater becomes excessive. The number of judgments

required can be computed by multiplying the number of task

factors by tasks. Raters may experience fatigue after

making several hundred judgments and the reliability of

subsequent ratings may be unsatisfactory.

3.1.3 Apply Algorithm/Aggregation Method

OVERVIEW

During this procedure, data collected during Procedure 3.1.2

will be compiled and aggregated, task selection criterion/

cutoff scores will be developed, and tentative task

selections will be made. An overview of this procedure is

presented in Figure 3-8.

PROCEDURE

3.1.3.1 Apply Aggregation Method

In this step, data will be compiled and then aggregated

using one of the methods depicted in Table 3-3. The

3-40
(n 0

'H 4.J 4.)

'-4 Q) 0

oa 44 0 wl 0

0i 0 C

-Cl 4.)

oj 0

'0 W -2.0

Q) 0 P.,..

'0 .. l ) a) a.)

CC* E- S- m- S

0 co m0 > ) Lo 41) 4.4 00) 0 E 0 ~ 0

o 4 =) 4 Ia . ) 4.m ~ 1 z C. E-4O

Li~ ~~ C/2 cnx '4 0 ) 0 ~ 0

-* 4 C E- W 03 ) ) L

00 u Q)

E-4 0 O z- cw O o (u m

0) a) 0-4 E-4 > S- . o0

-440-4 0 41 o- 9 =) .') z - 4

J9'- E- . U m. 0 - L

En -4 z 41 0 CI) 0) C -

E-4 . w- . 0 ~ . 0

:0 E-4 co .

OA z Z ) U)02 0 w ) 0 41.)~ .

4~ 0 U- 0 0 Jm 

- S- 0 4.) E -. J i

4-4 (L) 4-1

O ~ ~ 0) 4 4) cc0 '0

* - ' - 0 ) 4 ) M ~ W ) .

</ 0. Q) '0 '0 C 0 4- 0W~ ~ 0 z - Zn 0) SC L 

Im C 0 4-1 zi 4-4 W~ ,. en5 C. 0 * *

3-41
to ( (a w0

'Co

o M)

>* a

C aD

%- NN N N N4

.0 0

CuL

* c03-4

va
1- 0 (0 (0i 

0 0<

c. >0 (0 C14 1.0 C%4

> 0<

3-4
(D to(

0

0

ca

0

C

0 )3-44
r-.M- T I. 7-77r

~0

N

m 0

CO -0 L.(0(0 r

0U .

0.

0

Oft,

a, __3-45
I0

AGGREGATION TASK DATAMTO

APPLY

AGGREGATION

METHOD

SUMMARY

TASK DATA

DEVE LOP

TASK SELECTION

CRITERION/CUTOFF

SCORE(s)

CRITERION /

CUTOFF

SCORE(s)

TASKS FOR TRAINING

Figure 3-8. Apply Algorithm/Aggregation Method (3.1.3).

3-46
07. 

. k

worksheets provided in Table 3-8 can be used to compile the

data obtained in Procedure 3.1.2. These data should consist

of mean ratings/scores on each task factor for each task.

Thus, each entry to one of the worksheets in Table 3-8 will

consist of the average rating/score obtained by one task on

one task factor (sum across all raters and divide by the

number of raters).

Secondly, an aggregation method (if appropriate) will. be

applied to the data contained in the worksheets, resulting

in a summary score/rating for each task. Summaries of each

of the data aggregation methods are contained in Table 3-3.

3.1.3.2 Develop Task Selection Criterion/Cutoff Score(s)

" Criterion/cutoff scores are identified in this procedure

that will be used to determine which tasks will be selected

for training. These criterion/cutoff scores can be

determined by:

(1) Using the number of tasks currently trained for a

similar system. After determining the number of

tasks on the similar system a cutoff point on the

summary measure is set which selects the

appropriate number of tasks for training.

(2) Examining available training resources to determine the available training time in various

settings. This examination may reveal a limit on

the number of tasks which can be selected for

training. As a result, a cutoff point can be set

which will produce the appropriate number ot

tasks.

3-47

-0 .
4 Table 3-8 Data Summary Worksheets

Comparability Analysis (Comp)

SYSTEM _____________

(a) NEW TASK TITLE (b) COMPARABLE TASK (c) COMPARABLE TASK

3-4 8

-. .. 7-w
Table 3-8 Data Summary Worksheets DIF Algorith m (Continued)

K> ~ ~SYSTEM ________

(e) SUMMARY MEASURE

z

cr OVERTRAIN 3

2 TRAIN =2

> 0

0 g-

(a) TASK TITLE -

3-49
Table 3-8 Data Summary Worksheets (Continued)

LU

0C6 

L.LL. I

Ca LUT

z 4j

LU -

s sa

3-00
Table 3-8. Data Summary Workshecis (8FAC).

4~ <o

0 1)A

~C>

q< t ' o

TAS TIL q

3 - 51
'0

(3) Using objective indices of criticality of training

such as individual task factor criteria or task

factor criteria algorithms (See Table 3-3).

(4) Using some combination of the above methods.

3.1.3.3 Select Tasks for Training

The criterion/cutoff scores determined in procedure 3.1.3.2

"" are applied in this procedure to produce a tentative list of

- tasks selected for training. The worksheet presented in

Table 3-9 can be used to assist in this process.

3.1.4 Review Current Task Selection

During this procedure, tentative task selections for

training are reviewed, revised if necessary, and finalized.

An overview of this procedure is presented in Figure 3-9.

3.1.4.1 Review Tentative Task Selection

The purpose of this step is to review the tentative task

selection accomplished in procedure 3.1.3.3. If the

tentative task selection is adequate, the user is finished.

If the number of tasks selected for training either exceeds

or falls short of the planned training load, adjustments

will need to be made to the task selection criterion/cutoff

scores.

3.1.4.2 Revise Criterion/Cutoff Scores

In this step, revised criterion/cutoff scores are developed

-? - so that the task selection list will match available

- training resources.

03
'00

LU,

cc0

00

E

3-5
ENTATIVE CRITERION/ CURRENT TASK SELECTIO CUTOFF SCORE TASK SELECTION (al.3 (3.1.3)

C3.1 1-3

REVIEW

K TENTATIVE TASK

SELECTION

NN

CRITERION/CUTOFF

SCORE(S)

RVSCRITERION/

CUTOFF SCORES

Figure 3-9 Review Current Task Selection (3.1.4)

3-54
3.1.4.3 Apply Revised Criterion/Cutoff Scores

Revised criterion/cutoff scores are applied in this step to

select tasks for training. The steps described in procedure

3. 1.3.3 are repeated and a new list of tasks selected for

training is developed.

3.1.5 Document Data

The train-no train decision should be documented in the SDT

after tasks have been assigned to training settings in

Procedure 3.2.

3.2 ASSIGN TASKS TO TRAINING SETTINGS

Once tasks have been selected for training, they should be

further analyzed to determine the most appropriate training

setting. The key decision is to allocate tasks between

formal school settings, (for example, advanced individual

training) and unit training settings (for example,

supervised on-the-job training). Once this decision is

made, assignments within these two general types of settings

can be made.

This procedure describes two basic approaches to assigning

tasks to training settings. One approach is comparability

analysis which bases training setting assignments on assignments made to comparable tasks. The second approach involves

selecting criteria for making training setting assignments,

collecting data, applying an aggregation algorithm and

making training setting assignments based on the aggregated

criterion scores. An overview diagram depicting the steps

in this procedure is presented in Figure 3-10.

3-55

..
/SELECT SSETTING AS SIGNMENT

• II METHOD

F COMPARAB~~IIYXFCO

ANALYSIS /APPROACH

J STRAINING TS SETN DTIN

ASSIGNMENTST

. // ASSIGNMENTS 1

%'" -. IJ DOCUMENT

Figure 3-10. Overview of Assignment Tasks to Training Settings (3.2).

3-56
3.2.1 Select Setting Assignment Method

There are two basic approaches to assigning tasks to

training settings: comparability analysis and the use of a

selection algorithm (X-factor method).

o Comparability Analysis

Comparability analysis may be used to provide crude training

setting assignments during the early phases of the acquisition process. The first step in the application ,)f comparability analysis is to identify a comparable existing task

for each of the tasks in the New System (if comparability

analysis was used in Procedure 3.1, the results will support

this analysis). A comparable existing task may be defined

as a task which (1) is perfoLmed on comparable equipment or

(2) involves the same type of task action (that is, the

action verb for the comparable task matches or is highly

-' similar to the action verb for the New system task).

. The second step in comparability analysis is to assign the

New system task to the same training setting (institutional

training, unit training) assigned to their comparable tasks.

O X-Factor Approach

In this approach, criteria are identified which are relevant

to the training setting assignment decision. These criteria

will typically include the same criteria used in the

procedure for selecting tasks for training. The six factors

most likely to be used are task delay tolerance, task

learning difficulty, percent time performing, percent

performing, time between initial MOS training and required

3-57

S.Y

" 4k .- ; .- ". - L g 'v - - .- -:> .- :'.- . -- -. C , -, . -. " > - - '- -",' < > - .- ." -
*. - job performance, and consequences of inadequate performance.

Data is collected (If 4-Factor or 8-Factor models were used

in Procedure 3.1, much of the required data may already be

available) directly by the analyst or from surveys

administered by the Army Occupational Survey Program (AOSP)

of MILPERCEN. Once information has been collected for each

of the X-factors for every task during Procedure 3.1, there

will be a large data base to use for assigning tasks to

training settings. The user should select the same subset

of factors used in Procedure 3.1.

The X-Factor approach is more comprehensive than the

- comparability analysis approach and has greater data

requirements. Therefore, the X-Factor approach and be more

* appropriate later in the acquisition process when more

accurate training setting assignments are required.

3.2.2.1 Identify Training Settings for Comparable Tasks

If the comparability analysis approach is selected, then

training settings -for comparable tasks are identified (if

comparability analysis was employed in Procedure 3. 1, this

- data may already be available). These existing training

setting assignments are then simply transfet ed to the New

system tasks. Once the assignments have beei made, the user

can advance to Procedure 3.2.3 (Document Data).

3.2.2.2 Obtain Task Data

"0 If the user selects the X-Factor approach, data must be

-. collected on the task factors. First, however, the user

must select the criteria to be included in the analysis.

Two of the more useful criteria for the training assignment

*/ decision are:

3-58-
0 Percent performing -The greater the proportion of

individuals in an MOS who perform a task on the

job, the greater the payoff for training them in

an institution. Thus, if a specific duty position

accounts for a substantial percentage of the

personnel within an MOS, then that duty position

is identified to be trained in the institution.

For example, for MOS 95B, Military Police, at

Skill Level 1, there are several different duty

positions, such as military police, desk clerk,

radio dispatch clerk, fingerprint clerk, machinegunner, investigator, prisoner-of-war processing

specialist, etc. The majority of these duty

positions account for a relatively small percent

of the total MOS strength. However, the training

assignment decision is easy because the position

of military police accounts for approximately 60

percent of the MOS for Skill Level 1.

0 Time between training and required job performance

-If a task is complex and requires practice to

maintain proficiency then tasks which are not

performed by soldiers within 6 months af ter

training should be trained in the unit.

o Consequences of Inadequate Performance - This

*criteria is important insofar as it indicates

which tasks may need concentrated or professional

instruction that can best be accomplished in an

* institutional setting.

Data on these or other selected criteria can either be

collected in surveys conducted directly by the analyst or

3-59

| ...-
through the Army Occupational Survey Program (AOSP) (more

details on procedures for collecting this data are provided

in Section 3.1.) These data can be summarized on a

worksheet like that shown in Table 3-10. Data entries to

this worksheet should consist of mean scores/ratings for

each task on each criteria.

As part of the data collection process, the user should also

examine the Trainer's Guide for the MOS. This guide will

define the specific training settings currently used for the

MOS. If a new MOS is involved, training settings for other

comparable MOSs should be used (see Section 2.0 for

information on identifying comparable MOSs).

3.2.2.3 Make Training Setting Assignments

Training setting assignments must be made in a two step

process. First, tasks should be assigned to the either

institutional or unit training setting categories. Second,

specific settings within each of these general categories

must be determined. It should be noted that within the

institutional training setting category, there is a close

correspondence between specific training settings and skill

level. Thus, if you know the skill level of a task and have

made the assignment to the institutional training category

the assignment of tasks to specific institutional training

settings is largely determined. Listed below are the

relationships between skill level and institutional training

settings.

o Skill Level 1 - Basic Training, Advanced

Individual Training (ATT), or One Station Unit

Training (OSUT).

3-60

*

o

. . . . . . . . . . . ..
a U,

oz

E E

x~o o

-4 w

~~cn

3-6
-[ -. -" -

0 Skill Level 2 -AIT, OSUT, or Primary Leadership

Course.

0 Skill Level 3 -Basic Technical Course.

0 Skill Level 4 -Advanced Noncommissioned Officer

Course.

o Skill Level 5 -Senior Noncommissioned Officer

Course.

Some of the specific training typically included under the

* unit training category are Supervised On-the-Job Training

(SOJT), Self-Study, and Scheduled Training (SCD). These

settings do not vary with skill level.

Training may be prescribed for a task in both institutional

and unit settings. In these cases, a distinction is made

*between initial training to proficiency (called

qualification training) and subsequent refresher training

(called additional training). This procedure is primarily

concerned with the identification cf settings of initial

train ing assignments since most. additional training takes

place in units.

The method employed to make initial setting assignments is

presented in Figure 3-11. Note that this method is very

similar to the method used to select tasks for training,

3.2.2.3.1 Select and Apply Aggregation Method

In this procedure, data contained in Table 3-j0 will be

aggregated using one of the methods depicted in Table 3-

* 3-62

* - --- ,"..
SUMMARY TASK DATA

3-2 _ 2-2)

1223.Z2

DETERMINE AND APPLY

SETTING ASSIGNMENT

CRITERION/CUTOFF

SCORES

AGGREGATED

DATA

CONSTRAINTS CRITERIAN/CUTOFF

Figure 3-11 Overview of Make Setting Assignments (3.2.2.3)

3-63
-o

1. This will result in a summary score/rating tor each

task. More details in applying these aggregation methods are

provided in Procedure 3.1.

3.2.2.3.2 Determine and Apply Setting Assignment Criterion/

Cutoff Scores

Criterion/cutoff scores are identified in this procedure

that will be used to assign tasks Lo training settings.

These criterion/cutoff scores must take into consideration

any constraints on training resources (e.g., time available

in AIT for training, availability of equipment at an

institution). The criterion/cutoff scores can be determined

by:

O Using the number of tasks currently trained in a

setting for a similar system. After determining

the number of tasks trained in a setting on the

similar system, a cutoff point on the summary

measure is set which selects the same number of

tasks.

O Examining available training resources to

determine the available training time in each

setting. A cutoff score can then be set which

will produce the appropriate number of tasks.

O Using objective indices of appropriateness of a

setting for a particular task su--h as individual

task factor criteria or t.ask factor criteria

algorithms (Sre Table 3-11).

o Using some combination of the ahov,- meth< ° .

3-64

!JR
Once tasks have been assigned to the institutional training

category, they should be assigned to specific settings

within each category. As indicated in Section 3.2.2.3, this

[%- ,. •

is largely determined by the skill level of the task.

.--

Additional guidance can be provided by examining the

. .assignments made to similar tasks.

- The resulting task assignments will determine the location

of qualification training, but in most instances additional

* -. t-iining will be required. For example, if AIT is the

setting for qualification training, SOJT would likely to be

selected for additional training. This procedure does not

specifically address the question of training sites for such

additional training since most of this training takes place

in the unit.

3.2.3 Document Data

In this procedure, tasks which are selected for training and

assigned to training settings are entered into the SDT as

shown in Table 3-12.

3.3 IDENTIFY SKILLS AND KNOWLEDGES

OVERVIEW

During this step, skills and knowledges are determined for

the BCS and New system tasks which were selected for

training in Procedure 3.2.

PROCEDURE

An overview of this procedure is presented in Figure 3-12.

A worksheet which can be used to support this procedure is

presented in Table 3-13.

3-66 0."

%.. ' ..- "% .. - , .5... ...... . ... . . . 5
Table 3-11. Aggregation Methods for

Assigning Tasks to Training Settings

Weighted Z Scores

n

TPI= Z W Z ij I=i 1

where TPI is the training priority index; W is the weight assigned to th

, the i the task factor; and Zi . is the Z score of the i task on the j

factor; and N is the number o task factors.

..., z z i = X' --x- Xj

SD. .. .. J

Sth th

where X is the raw score of the i task on the j factor; X. is the

mean ofhe X. f the J factor; and SD. is the standard deviation of

the X for t j factor.

Individual Task Factor Criteria

In this method, cut points are established for individual task factors.

Thus, for example, a cutoff point of 6.0 could be established on the

factor, consequences of inadequate performance. As a result, any task

with task factor scores exceeding 6.0 would be selected for training.

Similar criteria could be established for other task factors.

e.g., Select for training if:

> 6.0 on Task Factor 1

5.0 on Task Factor 2

> 5.0 on Task Factor 3

Evaluate each task against criteria for each task factor.

Tpsk Factor Criteria Algorithms

A task factor criteria algorithm specifies, for a particular combination

of values on task factors, whether a task should be trained. Such

algorithms are not really used to aggregate task factor scores as much

as to lead the analyst to a solution.

3-65

F" - ,-"X" " ,'v ,-'. -," ,.-5.':-.-. *.,',~.- .~~*-".-' ,,- -- . " .. "-. '.. ...- • , .'."i"..". m. f '".. i .".., -'.'.'. . > &-
W ('4

000

*3 u

-4 IC1

4E-

-4-

ocU

*0

0

E-4 ' U

.C. 4J

*~ ) zE-4 cn .)

S -4 E-4 ~ * .0 oc c )

* E-4 CC 4 -- 1 -4 -4-

E-4 I E- 4

0 (4 ('4

0

3-67
.'. -S.

e434

IDENTIFY ADDED

TASKS AND TASKS

WITH MAJORIMAJ.I

MODIFICATIONS

ADDEDTASKSWITHOTHER

TASKSTASJS

IDENTIFYUSE EXISTING

COMPAABLESKIL LS AND TAS KNOWLEDGES

- SKILLS AND KNOWLEDGES COMPAABLEFOR

TASKSUNMODIFIED

TASKSI

-ADD

DEET

SKILSAN

KNWLDGE

SKILSAN

KNOLEGE

FO DDE

- DOCUMENT

-. DATA

DOUMIENTED

%- DA

*Figure 3-12. Overview Diagram: Identify Skills and Knowledges (3.3).

*.. *%*.**.**. ~S... .... .- 68
Table 3-13 Skill and Knowledge Worksheet (Skill)

SYSTEM MOS/3KILL LEVEL

(a) TASK:

bi COMPARABLE TASK:

COMPARABLE/UNMODIFIED TASK SKILLS MODIFIED/NEW TASK SKILLS

(c) SKILL (d) SOURCE (e) SKILL (f) SOURCE

3-69
To begin the procedure, you must examine the BCS and New

system task list. These tasks must be then categorized into

two groups, (1) added or modified tasks and (2) unmodified

existing tasks. For unmodified tasks, existing skill and

knowledges can be utilized. Table 3-14 lists some of the

data sources which can be used to obtain information or

skills and knowledges for existing tasks. As skill and

knowledge information on existing tasks is obtained, it

should be entered into columns (a) and (b) of the Skill and

Knowledge (SKILL) Worksheet.

To estimate skills and knowledges for added or modified

tasks, comparability analysis techniques must be used. More

specifically, a comparable existing task must be identified

and information on the skills and knowledges of this task

must be collected. Skills and knowledges from tne

comparable task must then be added or deleted to reflect the

differences between the equipments and functions associated,

with the comparable task and the added or modified task

(Information on equipments and functions should be available

from Procedure 1.0)

- A comparable existing task may be defined as a task which

(1) is performed on comparable equipment (procedures for

identifying comparable equipments are listed under Function

1.0) and (2) involves the same type of task action (that is,

. the action verb for the comparable task matches or is highly

similar to the action verb for the New System task).

In addition, if it is possible, the comparable task should

be selected from the same occupational specialty (MOS) as

the New system task.

3-70

-.7.:

.1'.e .,r ' . .- , .i- n ,-',-. ,' .. ,.. , .- . .- . .--- . .- . . ." ._. - ' ' ,. ". ..". " z - ' . ' .- ' .-.• .. ' ,' .,
i'-"-

Table 3-14. Skill and Knowledge Data Sources

" Soldier Manuals**

" Trainer's Guide**

o Job Book** ..•

."

" CODAP**

" LSAR*

a ARTEPs**

a Field Manuals**

0 Maintenance Manuals**

" Skill Performance Aids**

" Maintenance Allocation Charts*

" Training Development Information System (TDIS)**

a Contractor Supplied Training Programs*

" Program of Instruction (POI)

* Data source for developing systems

** Data source for existing systems

A,

'.

A,

5%

-,I

,-. 3-71

,

'.'.: ': "; ' ,'"" ,"-. /'. ." '"-" - ""c -~' . ','..'*.; ; *.'." .v 5 , , ', - .5. . - '5.. .5 -..'.? , ,< .: '..",.
All skill and knowledge statements should be constructed in

accordance with existing ISD guidelines. (See Schulz and

Farrell, 1980 and TRADOC Pam 350-30 for a description of

these guidelines).

Once the skills and knowledges for the new or modified tasks

have been identified, they should be entered into column (e)

and (f) of the SKILL worksheet.

Once the SKILL worksheets have been completed, relevant data

from these worksheets should be entered into the SDT. Table

3-15 describes the most desirable order for entering data

into the SDT. Additional guidance for entering data into

the SDT is provided in the SDT User's Guide.

During the later phases of the acquisition process you may

want to do a more detailed task analysis and enter some

additional data on tasks in the SDT. Some of the types of

data typically collected during these types of more detailed

task analyses are:

o Task Elements

o Preceding and Succeeding Tasks (in terms of

mission performance)

o Tools/Test Equipment

o Failure Modes (maintenance tasks only)

o Initiating Cues

* O Terminating Cues

o Learning Objectives

o Performance Measures

,3-.-

" 3-72

2,
O Task Frequency and Duration Measures

0 Work Area/Task Area

o Amount of Supervision

o Number of Performing

If necessary, these items can be generated by the same

comparability analysis techniques used to estimate skills

and knowledges. The sequence for entering this data into

the SDT is displayed in Table 3-15.

3.4 DEVELOP TARGET POPULATION DESCRIPTION

OVERVIEW

Target population descriptions must be developed for each

job within an MOS for the New system. A job is a set of

related duty positions within an MOS (see TRADOC Pam 351-4,

Job and Task Analysis Handbook, for a more detailed description of what constitutes a job). The elements of a jobdescription for a new systems, per TRADOC Pam 351-4, are (a)

skill level, (b) occupational specialty, (c) educational

level, (d) prerequisite training, prerequisite education,

(including reading grade level), (e) performance data (if

available), (f) enlistment rates, and (g) overseas tour

1 e ng t h.

PROCEDURE

Table 3-15 describes how each element of the target

population description can be developed during the early

phases of the acquisition process. Skill level and

occupational specialty are determined during Procedure

2.0. For the remaining elements of the target population

N, 3-73

.--..................................
U U (3 ('3 m' m (13 (o3 (U (

0 0 0 0 0 0 0 0 0 0 0

04 0 4 04 a, 404 04 04 04

U)0 0 0 0 0 0 0 0 0 0 0

-I N N CN CA N (N CN N N

rn 0

cco

E-.

00

°,--4

0 w w "0 E-'. g . . oI 0E4E-4

U) U) Cf U U U) (n U U) U) U -, "4 .Id -4 - -_x .-

&4 Z) U) U) ( U) U) cc ", wU (U (U (U mU mU fu (m mU m m m

E- &I E4 &4 &4 &4 -' &4 &" F4 E- E-0

00

E-47U

0) 0'E-4 0

vo E) 4 1 0

4) "-4 04 U 4--4

0'. . 1- CU - ) (U. - ( , --4,.

- -- - .4 .) .- .. - r-4 w) w44 U) - w

41' a) E-4 w) C3 3 0 ."V C E-4 10 m j E-) %0 4)0'0 N 0 U4)

C C0 m C:( 4-4 a)

E-4 -4 4-' ( ~ C 0)* 

l (U-4 $4 C4-4

'U - U -4) C 0 c ) E-4U 1-4 C1) 44 .. e :3 .0

U) 0 -4 a) * ) ( 4- U 4 0 E

X U 0 m 4 ~ 4 aU ) (U 0 E

U E-4 E-4 '-C) C 4E-4

S0

3-74
.. -

Table 3-16. Procedures for Developing

Target Population Description

Elements

ELEMENT METHOD FOR DEV' 'PING

Skill Level Obtain from P *(:tdure 2.0

Occupational Specialty Obtain from P, ...ure 2.0

Educational Level Use Data From Existing or

Comparable MOS

* *Prerequisite Training Use Data From Existing or

Comparable MOS

2.Performance Data Use Data From Existing or

Comparable MOS

Enlistment Rates Use Data From Existing or

Comparable MOS

Overseas Tour Length Use Data From Existing or

Comparable MOS

3-75

Q" .-

5'e=
description, current data for the ocrupational specialty

should be employed. If a new occupational specialty is

involved, data from a comparable MOS shold be employed.lI

Procedures for identifying comparable MOSs are I isted under

Procedure 2.0.

A worksheet for documenting the target population

description is provided in Table 3-17.

3.5 SEQUENCE TASKS/SKILLS FOR TRAINING

OVERVIEW

During this procedure, the tasks/skills associated with each

setting are placed in the sequence in which they must be

trained.

PROC EDURE

An overview of the procedure for sequencing tasks for

training is provided in Figure 3-13. First the tasks

associated with each setting are identified. This

information will be available from Procedure 3.2.

Once tasks for each setting have been identified, these

tasks must be sorted into groups of "functionally similar"

1 This procedure only provides gross estimates of the

target population description elements that are appropriate

for the earliest phases of the acquisition process. More

specifically, the procedure does not attempt to adjust the

target population elements to account for task modifications

@1 or additions within an MOS.

3-76

q - .".. .
0

444

i

0 cc

I40 IL

j

0.0

00

4Aa

U..c

0C

3-7
I. .47

7TASKS IN

EACH

SETTINGN

SORT TASKS

INTO FUNCTIONAL

GROUPS

FUNCTIONAL

GROUPS

SEQUENCE GROUPS

BY LEARNING

DEPIENDANCI ES

GROUPS

SEQUENCED BY

LEARNING

OEPFNDANCI ES

SEQ2UENCE GROUPS BY LEARNING

* DIFFICULTY

GROUPS

SEQUENCED BY

* LEARNING

DIFFICULTY

F SEQUENCE

TASKS WITHIN

GROUPS

IwoTASK SEQUENCES

WITHIN

GROUPS

Figure 3-13. Overview Diagram: Sequence Tasks/Skills for Training (3.5).

3-78

. .-
. : r

, tasks. There are two criteria for identifying functionally

similar groups of tasks.

. (1) Action Verb Similarity - Tasks should be sorted

into groups which have similar task action verbs.

(2) Equipment Similarity - Tasks should be sorted into

groups which involve comparable or similar equipments. Procedures for identifying comparable

existing equipments are provided under Procedure

1.0.

Note that the criteria for identifying functionally similar

tasks is the same as the criteria for identifying comparable

existing tasks, as described in Procedure 3.3.

Once tasks have been sorted into functional groups, the

groups must be sequenced to reflect the learning dependency

relationships among the groups (that is what groups of tasks

represent prerequisite skills and knowledges which must be

learned before other groups of tasks). Detailed guidance

for identifying these types of learning dependencies is

provided in Schulz and Farrell (1980).

Once the groups have been sequenced to reflect learning

dependencies, they must be rank-ordered to reflect the

estimated learning difficulty of the tasks within the groups

(Guidelines for estimating learning difficulty were

presented in Procedure 3.1).

4K After the functional groups have been sequenced, the tasks

within each group should be sequenced using the same

criteria to sequence the groups (i.e., learning dependency

and learning difficulty).

3-79

.°* * --. * - * - * ~ - ; .* . - .* *
° .

W. .

A worksheet which can be used to document the results of

this procedure is presented in Table 3-18.

3.6 CONSTRUCT QUASI-PROGRAM OF INSTRUCTIONS

During this procedure, quasi-program of instructions (OPOIs)

are developed for individual institutional training courses.

The quasi-POI provides a convenient vehicle for summarizing

the content of a course/instructional program during the

early phases of development. It provides the information

needed to determine the resource and cost requirements of a

course/program, to estimate its efficiency/effectiveness,

and to plan for its development. A worksheet which can be

used to support the procedure is provided in Table 3-19.

This worksheet summarizes the essential elements of the

- QPOI. It should be noted that ETES currently contains

procedures for developing QPOIs for individual institutional

training courses.

ETES does not currently include procedures for developing

QPOIs for individual unit training or collective training or

for developing outlines for reference material.

PROCEDURE

4An overview of the procedure for constructing quasi-POIs for

instructional training is provided in Figure 3-14.

There are five steps in the construction of quasi-POIs for

individual institutional training courses: (1) identification of courses and course modules; (2) identification of

the methods to be used within each course module; (3)

identification of the contact hours to be devoted to each

3..8 ~3-8 0
LU

LU

z

LLU

C00

IL

to M.

caa

CL.

z

LUA

S3--
. . --u -Z 7. 

0 1,

.5..

S.'U

wsw

0-

0

'7'

S U'

% I- PA

W, ,

3-8
LUL

ca

A

t ua

Z 'o -d(

3-8
TASK GROUPS

SEQUENCED

TOR TRAINN

3.IL

IDENTIFY

COURSE

MODULES

COURSE

MODU LES

STANDARM

LITO DETF OMAAL MEHDSMTHD CU

3.6.

IDETIF

MODUL

TRANIG TME

MODUL

FCNT.C

"OUR

*34.

FORATED

Figure 3-14. Overview Diagram: Construct QUASI--POI (3.6).

3-84
method within a module; (4) determination of course length,

and (5) documentation and formatting of the QPOI.

More details on each of these steps is provided *in the

sections which follow.

3.6.1 Identify Courses and Course Modules

The training task sequence relationships which were

identified in Procedure 3.5 for all the tasks within a

setting are examined and logical break-points in the

sequence are identified where tasks may be grouped together

to form courses and course modules. Guidance for the

construction of courses and modules is provided by the

examination of the types of courses currently used in the

MOS and/or predecessor system and the modules within these

courses which deal with similar tasks. Similar tasks may be

identified by examining (e) the task action verb and (b) the

equipment on which the task operates. (Procedures for

identifying "similar" tasks were identified in Procedure

3.3. Identification of comparable modules can be

facilitated by identifying the comparable existing course

which most closely matches the task requirements of the new

system. (Usually this comparable course is the course is

the course from the predecessor system).

Modules may be added to or deleted from the comparable

course to reflect the New system task requirements. An

example of a worksheet which can be used to document

modifications to an existing course is presented in Table 3-

20. Data sources for identifying comparable courses are

listed in Appendix E.

3-85

, * . * .
24 Z3

2d.

E-I 0

212

zS 4

-C N4~ 2. ;jx

- - SA.

-q4 -

3-86e
As part of the module identification process, the user must

identify non-academic modules. Non-academic modules are

modules which are not associated with task requirements.

For example, testing modules are usually included at the end

of each major section of a course. In addition, modules are

often added to a course for non-academic subjects such as

commandant's/commander's time, in-processing/out-processing,

chaplain's orientation, etc. Again, these types of modules

can be identified by examining the modules included in a

*comparable course.

After the modules have been identified, each module is

assigned a name which summarizes its content.

3.6.2 Identify Methods Within Modules

Table 3-21 displays the instructional methods which are used

in institutional training. The instructional methods which

muqt be used to train each module are identified by

examining the instructional methods used in modules from

existing courses dealing with similar tasks. Methods must

be added or deleted to reflect the differences between the

tasks included in the module from the comparable course.

3.6.3 Identify Module Training Times

Module training times are identified in a two-step process.

. In the first step, the total training time for a module is

determined. This is accomplished by modifying the training

__ time from the comparable existing module to reflect the

differences in task requirements between the proposed and

existing module.

3-87

.%%7
TABLE 3-21 INSTRUCTIONAL METHODS

AT Audio Tape

C Conference/Lecture

CAI Computer Assisted Instruction

CS Case Study

D Demonstration

DF Dual Flight Hours (only aviator courses)

(do not include in ICH computations)

El Hardware Performance Examination

E2 Nonhardware Performance Examination

E3 Nonhardware Performance Examination

EL Elective (in-house only, except for CGSC)

F Film

GS Guest Speaker

IS Independent Study

NC1 Non-contact Instruction with an Instructor

Available in Classroom

NC2 Non-contact Instruction without an

Instructor Available

PEl Hardware Oriented (hands-on) Practical

Application

PE2 Nonhardware Oriented (non-classroom)

Practical Application

PE3 Classroom Practical Application

PI Programmed Instruction (using programmed text)

PM Printed Materials

QC Besseler Cue See

S Seminar

SF Solo Flight Hours (only aviator courses)

(do not include in ICH computations)

SI Simulation Instruction

SP Self-Paced Instruction

ST Slide Tape

TV Television

4WC1 Instructor Led Work Group

WC2 Student Led Work Group

Sources: DA Pam 570-558 Staffing Guide

for U.S. Army Service Schools

and TRADOC Cir 351-12 Format

for Programs of Instruction

3-88
I<.-

7 7 7.

- In the second step, training times for each method within a

S'module at- determined. This is accomplished by modifying

the training times for the methods from the comparable

existing module to reflect task differences betwepn the

proposed and existing module.

3.6.4 Determine Course Length

. -Total course length is determined by summing the module

training times.

3.6.5 Document Data

Information developed in the previous step is placed in the

QPOI format listed in the QPOI Worksheet. The QPOI format

is divided into two general parts. The first part provides

summary information on modules within each course. The

second part provides more detailed information on each

module.

After the QPOI worksheets ha,"e been completed, relevant data

from these worksheets should be entered into the SDT. Table

3-22 displays the most desirable index for entering data

" into the SDT. Additional guidance for entering data into

the SDT is provided in the SDT User's Guide.

.3.7 ASSIGN TASKS TO MEDIA

Procedures for assigning tasks to media are described in the

User's Guide: Media Selection Program. This program is an

automated aid for (1) assigning tasks to media, and (2)

determining the efficiency and effectiveness of media-task

combinations. Relevant data from the Media Selection

3..9

,i-- 3-89

S

* *... ...

b - . -,- -*
-4 -1 4.1 4.

CL)U

Cu 0U U) 0CU) 0Cu)c

00 w~ 04 Q4 C w 4 . 4.j co 00 0 0 0 0 Co 0 0

:r. u -4 E4 E- E-4 U-

*0 0 0 0 0 0 0 0 0

*-'a4 0) a)4 a) X) a) a) a) 1a)

C) U ul InU U) U) (

La ~E-4 3:1=

E-4 -t

a)i

C

-4 0) -4-4u 1

4 Q1 41 1 Q

E-4)

*l 4-) <) 0.~UU

W a)4) aw ) 4-I ra) .

.04 a) ) En ( a 4 - r 0 o Q)

P* el W U O00 41 3 - -l C-JQ 

=3 U)i EI F= 10' ) 4-) C: U) u

00 0U 0 3 0 0 1 3(

1. 14 1; r4 r)(a 4 14 C'U "1

3-90
jO °

program must be entered into the SDT. Table 3-23 displays

the most desirable order for entering data into the SDT.

Additional guidance for entering data into the SDT is

provided in the SDT User's Guide.

3.8 CONSTRUCT TRAINING PATHS

OVERVIEW

During this procedure, training paths are constructed for

each of the MOSs associated with the BCS and New system. The

training paths provide a convenient framework for describing

the training courses required to train each MOS. The

training paths represent the possible flow of individuals as

they progress along the career path in their MOS.

PROC EDURE

An overview of the procedure for constructing training paths

is provided in Figure 3-15. The first step in this procedure

is to examine the non-system related courses in each MOS

which were identified as part of the target population

' identification function (Procedure 3.4). The courses within

each MOS are then sorted by skill level. When this is complete, the courses are sequenced to reflect task and skill

learning dependencies (what must be learned before what)

within skill level. Training paths are then developed to

represent the course sequences required for each MOS. An

example of training path is provided in Figure 3-16.

A worksheet which can be used to support the development of

training paths is provided in Table 3-24. Table 3-25

. . displays the training path data elements which must be

O entered into the SDT.

3-91

4'.
0 0

E-44 U

E-c 0 W4- E- E4J E-4

W~At u u u u

bd >4 >4 w w

0- a4 P04

414

E-4 U)

C 02 U) 3

W - E-4 E-4 u cz 0-

00

* U) C 0

.-- 4

U) 4

ul U

( 0 En

0-j U)

E (v 0

U)E-4

E-4 C -Uq

L 41J 4) () a) w)

E-4 (n . : ;

0

3-92
.. 

 

. 

~7r7.rrr.-vr. r rwr,

'0:

-

,-, I.' urn

it BLKA rAG

IC.-..

'VI'.

K

.3e?3
0Figure 3-16. Example of Training Path for Institutional Courses.

)X21 1B NOT!: ATTENDANCE IN THE 13T30 COURSE IS

IWO) PREREQUISITE FOR THIS COURSE.

- - - - - - - - --- - - - - - - - - -

- -SUPERVISED NOTE: THE TRAINING PROGRAM BEYOND E6 ON-THE-JOB WAS NOT PART OF THE STUJDY TRAINING

-. E6 (SOJT)-

E6 9------------------ -- ~---- -------------- -

XXXX-1P NO3TRO30NY SAHIVDB

MAINTANNER ONLYINNSSNE

LAE)APDATRSANSHINPIER

SUPRINE SYSTEEOPEASEO

DRINN TTIIGPORASIN

BYP STUDYOICECYI XECIIE AHEVDB

* (I) ~TRAIN ING PROGR AMS NT 39

SPECIFIED BY STUDY
d ...

I...

4too.

0

U

%.2

E. 4-

ID

a)

a...

00 U

* 3-9
E-44

000

00

4-

CC

(U)

C zE- a) a)

C)E- Z U) U)

0o CE-4 0 0

4-4 In U u u

E-4

.44

U)

1.4 U0

E 0 U0

C:) 0

co- co

a3-9
SECTION 4.0 - ESTIMATE TRAINING RESOURCES

OVERVIEW

During this procedure, training resource requirments are

determined for the Predecessor, Baseline Comparison (BCS),

and New systems.

PROCEDURE

This procedure is composed of six lower level procedures

(See Figure 4-1). During the first procedure, an operating

and support plan is developed for each modified or

additional course. In the second procedure, the number of

students who must be trained in each course is determined.

In the third procedure, the number of instructors required

in each course is determined. In the fourth procedure,

training facilities requirements are determined. In the

fifth procedure, training device and training equipment

requirements are determined. In the sixth procedure,

requirements for other types of training resources are

determined. Both phased arid steady-state resource

requirements are determined during this procedure.

Phased resource requirements are the specific resources that

are required while a weapon system is being phased into or

out of the inventory (the latter case is generally not

relevant during systems development). Phased resource

requirements are displayed by calendar year. In order to

determine these phased resource requirements, information on

the number of weapons systems introduced to the field in

each calendar year must be obtained or estimated.

4-1

*.o'*~*'p .*- '.
- 7. 71 -7 7

ANO0

Q,0k

*PREDECESSOR

*NEW SYSTEM

DEVE LOP

OPERATING

AND SUPPORT

PLAN

PLA

DETERMINE DEVICERADRQIEET

E UMB O REQUIREMENTSRSO CE

4 Figure 4-1. Overview ~STDiagrTm DeemnToreOeatnOn upr ln

85 TRA4-2

S- NUMBER OF . -. ~ ~ ~~ . ~ . ~ . ~ ~ . SUET . TO**.-.-~- . . . *. .
Steady-state resource requirements are the estimated average

yearly resource requirements that are needed to maintain the

weapon system in, the field once it has been completely

fielded. To estimate steady-state resource requirements,

all that is required is an estimate of the total number of

systems that will be in the field once system installation

has been completed. Steady-state requirements are also

presented in yearly units. Because they require less

information to calculate, only steady-state requirements are

* generally calculated during the very earliest phases of the

acquisition process.

* - 4.1 DEVELOP COURSE OPERATING AND SUPPORT PLAN

OVERVIEW

During this procedure, an operating and support plan is

developed for each new or modified course in the BCS and New

systems. The operating and support plan specif ies (a)

course locations, (b) course frequency, and (c) usage rates

* - for training recruits.

PROC EDURE

This procedure consists of the following three steps: (1)

determine course location, (2) determine course frequency,

and (3) determine usage rates. Worksheets which can be used

to support this procedure are provided in Figures 4-2 and

4-3.

4.1.1 Determine Course Location

An overview of the process for identifying course locations

is provided in Figure 4-4. MOS and setting are the primary

factors for determining course location. Since these

4-3

I... . . . . . . . . . . . . ..-.
1.2

.9 ca

00

L6

CC

ft..

* IL 4 4-4
Course Number___________ System

Course Titl e_ __ _

Comparable Media

(a) (b) (c) (d) (e)

Media Usage Rate Media Usage Rate Course

Figure 4-3. Media Usage Rate Worksheet (MUS).

4-5
1,

 7 w 

. 

IsID NTF

CORENE OAIOSFO

NEW O

EXSIN*OPAAL

?.OUSE

Fig re 4- O eriew of Pr ct duEW f r D tr ii g C us o ai n

EXITIN

EXI IN 6
information items are identifie<d as part of the QPOI

construction process (Procedure 3.6), identification of

course locations is relatively straightforward. Additional

guidance for course location identification can be provided

by examining (a) the course locations used in comparable

existing courses and (b) the projected locations where the

weapon system will be fielded.

4.1.2 Determine Course Frequency

Course frequency (i.e., the number of starts and sessions

per site) is determined by applying the algorithm listed in

Table 4-1. To estimate course frequency, you must determine

the number of students to be trained (Procedure 4.2). The

algorithm listed in Table 4-1 must be applied for each

location and year in which the course will be given.

4.1.3 Determine Usage Rates

Usage rates are determined for instructors and training

media. The usage rate for instructors is called the

student/instructor ratio. The usage rate for media is

called the student/media ratio.

Student/instructor ratios are determined for each

instructional method used in a course. Table 4-2 displays

the student/instructor ratios that are provided for each

* .-type of instructional method in DA Pam 570-558 and TRADOC

Cir 351-12.

Student/media rates are determined by identifying the rates

.- that were used for similar types of media in comparable

* courses. This information may be obtained by examining the

4-7

0.
IAP

TABLE 4-1. ALGORITHM FOR DETERMINING NUMBER OF COURSE

STARTS AND SESSIONS PER SITE.

1. Establish number of courses required per site

STUDIN (STDINE ) NMAX = NCOURSE

NSITE

STUDIN = Student Input Requirements

NSITE = Number of Training Sites

NMAX = Maximum Student-Instructor Ratio

for Course

NCOURSE = Number of Courses Required Per

Site

2. Determine maximum number of starts per year

50 5 CMAX

CLENGTH

CLENGTH = Course Length in week.

CMAX = Maximum Number of Course Starts

per year (NCON)

3. Select the smaller of NCOURSE or CMAX as the

number of course starts (NCON) reqoired per

site.

4. Determine maximum number of sessions needed per

start (NSES)

If NCOURSE CMAX, NSES = 1

If NCOURSE CMAX, NCORS NSES

MAX

Round NSES up to next highest unit.

4-8
TABLE 4-2. METHODS OF INSTRUCTION AND ASSOCIATED

STUDENT/INSTRUCTOR RATIOS

AT Audio Tape 20:1

C Conference/Lecture 1 per class

CAI Computer Assisted Instruction 20:1

CS Case Study 20:.

D Demonstration 20:.

DF Dual Flight Hours (Only Aviator Courses) -

El Hardware Performance Examination 6:1

E2 Nonhardware Performance Examination 6:1

E3 Nonhardware Performance Examination 1 per class

EL Elective (In-House Only, Except for

CGSC) 1 per class

F Film 1 per class

GS Guest Speaker 1 per class

IS Independent Study Local Appraisa]

NCl Non-contact Instruction with an

Instructor Available in Classroom

NC2 Non-contract Instruction without an

Instructor Available

PEI Hardware Oriented (Hands-On) Practical

Application 6:1

PE2 Hardware Oriented (Non-Classroom)

Practical Application 6:1

PE3 Classroom Practical Application 20:1

PI Programmed Instruction (Using Programmed Text) 20:1

PM Printed Materials 20:1

QC Besseler Cue See 20:1

S Seminar 20:1

SF Solo Flight Hours (Only Aviator Courses) -

SI Simulation Instruction Local Appraisal

SP Self-Paced Instruction 20:1

ST Slide Tape 20:1

TV Television 1 per class

WCI Instructor Led Work Group Local Appraisal

WC2 Student Led Work Group Local Appraisal

Sources: DA PAM 570-358

Staffing Guide for U.S. Army Service Schools

TRADOC Cir 351-12

Format for Programs of Instruction

4-9
POIs and other associated course documentation from the

comparable course. Student/media rates should not exceed

the student/instructor ratio for the instructional methods

with which they are associated.

4.1.4 Document Data

As course frequency and location information is developed,

it should be documented in the operating and support plan

worksheets (see Figure 4-2). Media usage rates should be

documented in the Media Usage Rate Worksheet (see Figure 4-

3). Student/instructor usage rates should be documented in

the QPOI Part I worksheet which is listed in Procedure 3.6.

After the worksheets have been completed, relevant data from

these worksheets should be entered into the SDT. Table 4-3

describes the most desirable order for entering data into

the SDT. Additonal guidance for entering data into the SDT

is provided in the SDT User's Guide.

4.2 DETERMINE NUMBER OF STUDENTS TO BE TRAINED

OVERVIEW

During this procedure, the number of students to be trained

in the steady-state and phased situations is determined.

PROCEDURE

An algorithm for determining the number of students to be

* trained in the steady-state condition is provided in Table

4-4. A worksheet which can be used in support of this

algorithm is provided in Table 4-5. A worksheet which can

be used to calculate the number of students to be trained in

the phased condition is presented in Table 4-6. A worksheeL

4-10 • el
'0:

.° 4 :, 

.'2:

r-4

00 0 l

*l 0

E -4 

-

,-E

'aa

° 0 00

U) U) U) r

- 4)

r I CE-4 0

OCl)

-

0

C4 0 4

E-4

-4 r- 4a -

Vu

0

". . C 4 -11

E-" 4 ,.i ,- U)

".",0 0 a)

0 ,4

0 4-i
Table 4-4. Algorithm for Determining Steady-State

Training Input Requirements

". Inflate the given manpower requirements by the TTHS rates:

M = MR x (1 + T

n n n

where Mn = Adjusted manpower reruirements for paygrade n

MR = Manpower requirements for paygrade n

n

Tn = Trainee, Transient, Holdee, and Student rates.

2. Calculate levels of X which satisfy the following eauations

for all n:

Xn x (An + U n U_, and X= M + 0 n" n- n - - n n.

where X = Personnel requirements for paygrade n

Ann = Yearly attrition rate for paygrade n

U n = Yearly upgrade rate for paygrade n

-n = Overhead rate for paygrade n.

- 3. Calculate the number of personnel to be trained per year:

TR =X n x (A + Un )

*.",where TRn = Yearly replacement personnel to be trained

for paygrade n.

5' 4-12

,..-. * *. * , ....
CL

eC

0 E

ui CL

Ucc

E 2

z

.0 vi

4--
. . .. ~ .z

z

Z

02 L)

WZ

0 +U

0- 

4

0 Lu 0

0 1

z 0

>02

0A z w

0~

0 C

z 0 - R0-

00

a.~ OZ

- - Z C

.j R coCaL CC~O

0 -1 0

LU~~ CA - 0 4 1

CDA

LLi ca=
TABLE 4-6. WORKSHEET FOR PHASED COURSE INPUT REQUIREMENTS (cont'd)

INSTRUCTIONS FOR COMPUTING ANNUAL TRAINING INPUT

REQU IREMENT S

This guide provides the procedural steps to be used in computing the final adjusted annual training input requirements to

a single course of instruction. The development of training input requirements involves the application of adjustments to the

number of personnel required to be trained in order to compensate

for losses through rating and school attrition and to accomodate

Transient, Trainee, Holdee, and Student (TTHS) requirements.

PROCEDURAL STEPS

STEP 1. BLOCK ONE, COLUMN (1)

Enter the fiscal years of the planning period.

STEP 2. BLOCK ONE, COLUMN (2)

Enter the cumulative number of trained personnel required.

STEP 3. BLOCK ONE, COLUMN (3)

Enter the product of the Total Unadjusted Trained Personnel

required in Column(2)and the TTHS factor to determine the

number of TTHS Adjusted Trained Personnel Required.

Transient, Trainee, Holdee, and Stu'dent (TTHS) Factor- a

factor calculated from historical data used to inflate the

cumulative total trained personnel to account for personnel

in a transient, trainee, holdee, or student status who are

therefore unavailable to occupy a duty position. A factor

of 1.09 should be used in the absence of a historical figure.

4-15

@.-
TABLE 4-6. Worksheet for Phased Course Input Requirements (cont'd)

STEP 4. BLOCK TWO, COLUMN (1)

Enter the fiscal years affected from Block One, Column (1).

STEP 5. BLOCK TWO, COLUMN (2)

Enter the TTHS Adjusted Trained Personnel Required from

Block One, Column (3) as the Starting Inventory for the

following fiscal year. These data are entered one year

later since the number trained in one year represents the

starting inventory for the next year.

STEP 6. BLOCK TWO, COLUMN (3)

Enter the product of the starting inventory in Column (2)

and the Loss Factor to determine the number of losses per

year.

Loss Factor - a factor calculated from historical data used

to inflate the number of personnel required to be trained

to account for losses due to non-reenlistment and other

attrition from the service. Loss is a function of several

different loss factors; a factor of 0.15 should be used

in the absence of an empirical figure.

STEP 7. BLOCK THREE, COLUMN (1)

Enter the fiscal years affected from Block One, Column (1).

- STEP 8. BLOCK THREE, COLUMN (2)

Enter the TTHS Adjusted Trained Personnel Required from

Block One, Column (3).

STEP 9. BLOCK THREE, COLUMN (3)

Enter the Starting Inventory from Block Two, Column (2).

STEP 10. BLOCK THREE, COLUMN (4)

Enter the difference between TTHS Adjusted Trained Personnel

Required in Column (2) and Starting Inventory in Column (3)

to determine Training Growth. The Training Growth may be

positive or negative.

Training Growth - the difference between training requirements

and starting inventory between one fiscal year and another.

Inventory in excess of requirements will result in a negative

growth; however, growth is normally positive.

STEP 11. BLOCK THREE, COLUMN (5)

Enter losses in the appropriate fiscal year from Block Two,

Column (3).

4-16

- 4 . - . -.- *.* 

. - . -.
TABLE 4-6. Worksheet for Phased Course Input Requirements (ccnt'd)

STEP 12. BLOCK THREE, COLUMN (6)

Enter the sum of Training Growth from Column (4) and Losses

from Column (5) to determine Course Output Requirements.

STEP 13. BLOCK THREE, COLUMN (7)

Enter the product of Course Output Requirements from

Column (6) and the Course Attrition Factor to determine

Course Attrition.

Course Attrition Factor - a factor calculated from historical

data used to inflate the Course Output Requirements to

account for trainees who fail to complete a course of instruction for a variety of reasons. Course Attrition Factors

differ from course to course; however a factor of 0.10 should

be used in the absence of an empirical figure.

STEP 14. BLOCK THREE, COLUMN (8)

*Enter the sum of Course Output Requirements from Column (6)

and Course Attrition from Column (7) to determine Adjusted

-. Course Input Requirements.

. STEP 15. BLOCK THREE, COLUMN (9)

Enter the Backout in appropriate (+) or (-) column in accordance with the following procedure.

a. Compute the Backout Factor as a percentage

by multiplying course length in weeks by

two (2). (Note: This number is not entered

on the worksheet.)

b. Multiply the Backout Factor by the Adjusted

Course Input Requirements for the second

fiscal year in Column (8). A Backout Factor

is not computed for the first fiscal year.

c. Enter the backout computed in b. as a positive number for the first planning year in

Column (9) and as a negative number for the

second planning year in Column (9). This

figure is a double entry as it represents

both an additional input for the preceeding

fiscal year and a backout for the current

fiscal year.

d. Repeat steps b. and c. above for each fiscal

year requiring Backout.

Backout Factor - a calculated figure applied to the Adjusted

Course Input Requirements to account for overlapping of a

course from one fiscal year to another. Courses with lengths

in excess of two weeks and convened on a continuous basis re4-17
TABLE 4-6. Worksheet for Phased Course Input Requirements (cont'd)

quire a portion of the input to be admitted the previous

fiscal year in order to meet the required output of the

following year. The Backout Factor is a percentage equal

to twice the course length and is applied to each fiscal

year with the exception of the first. The Backout Factor

is not applied to courses less than two weeks in length.

STEP 16. BLOCK THREE, COLUMN (10)

Enter the sum of the Adjusted Course Input Requirements

from Column (8) and the net Backout in Column (9) to determine the Final Adjusted Annual Training Input Requirements.

The example on the following page uses a Starting Inventory of

zero, a four week course length, and the following factors and unadjusted requirements by fiscal year:

TOTAL UNADJ TRAINED PERS REQ'D

FY88 61

FY89 103

FY90 148

FY91 183

FY92 196

FY93 196

TTHS factor: 1.09

Loss factor: .15

Course Attrition Factor: .10

'-.

5 4-18

- .0 .
+

<2 U

Ni

0 L -uDU

-- m. 0

< +U E C

ot C4 n 00s 

0 0 '

Cl,,

LU LI -

U, 'n- - - -

CM 2

00

C0 -

0 > >-iN V

1-o cncc +

o 0 cca

04 N1 P:P

-'LU Z~ U ; I-an 

(0 m - 0e

Z. (a C .j )Oc 0")c 

CL~~~L -UC z )u

>l >

C, > 2U. 

U. ) 4-1

1- LL c
which can be used to document the results of the steadystate and phased algorithm applications is provided in Table

4-7.

After the worksheets have been completed, relevant data from

these worksheets must be entered into the SDT. Table 4-8

displays the most desirable sequence for entering data into

the SDT.

4.3 DETERMINE THE NUMBER OF INSTRUCTORS

OVERVIEW

- During this procedure, the number of instructors required in

* -each course is determined.

PROCEDURE

An algorithm for determining the number of instructors is

presented in Table 4-9. This algorithm has been automated

in the ETES Resource and Cost Estimation Techniques (see

User's Guide: Resource and Cost Estimation Techniques). The

algorithm should be applied for each location and year in

which the course is given. A worksheet for documenting the * . results of the RCET is contained in Table 4-10. Table 4-11

displays guidelines for entering data into the SDT.

4.4 DETERMINE FACILITIES REQUIREMENTS

, .OVERVIEW

During this procedure, facilities requirements are

* - determined.

-.

'S.-.-

,4-2

<''.".'.. '. ... ..?.. .-. . .. ,-, A*. f .. 2._._,. " __ ., . - ' , , - - " . -,., , , . . , • • , -. , , ,',"
LL.

2

U,

IU,

z -

a, LI..

a,

U, 0

a, -~

(U

LL

00

U,

a, -~

C,, U.. 0

a,

0

(U

z 4-

C0

I.-: 3> 'It (Ua,

4-

a, C,,

-o

(U __________________________ I0 C0 h. - a,

-~ U0

-J -J

ma,

.4.4 I- * ~- ~.- 0

00

4-21

0
OlO

W c"I

4Js

Ei2 -4

0

1" 0 0

0 - -

-EC4 41s Q) a

-4 E- 44 f

0 0

N-,.

a)U Q)

.'--i U"

E

0

-~ L44-22

)U 0 0

-_ C

... _ (U

p+.. Z,

-..

l' " - N ---- .-. - .. .- '.--- 4--_ - 2-- 2 -- - "- --.-. . --- - -_ - - -
TABLE: 4--9. ALGORITHM FOR DETERMINING

NUMBER OF INSTRUCTORS

". Determine the number of Instructor Contact Hours

(ICH) per class.

NM

ICH per = hours of method i x class size

class student/instructor ratio for method i i=l

where NM is the number of methods.

2. Determine the number of instructors.

number of ICH per course frequency

instructors class x 1250

4-23)

:S
LL

E

U,

a a

C,, * zzw

* U,

* 0

* U,

b*. -

* 0 * .. ' * * U UU,

* C

'4-0E

z U.0

* 

0

S

S

.0 5 S C,)

*4

S

4-

C,,

*0

-S

C

* 0

S

.0 U

SE 0

* -j -J

.5

U, ~4 I- ~.. h.. 0

00

* 4-24
=li

- - 00

ou

E--4

z

-E0-4

E-i

00

Li ~ zE-4 0

00

44-2
PROCEDURE

An algorithm for determining facilities requirements is

displayed in Table 4-12. A worksheet for applying the

algorithms to a specific course location and year is

contained in Table 4-13. A worksheet for summarizing the

results for a specific course is displayed in Table 4-14.

4.5 DETERMINE TRAINING DEVICE AND TRAINING EQUIPMENT

REQUIREMENTS

OVERVIEW

During this procedure, resource requirements are determined

for training devices and training equipments.

PROCEDURE

An algorithm for determining the number of training devices

is presented in Table 4-15. A worksheet for applying these

algorithms is displayed in Table 4-16.

It should be noted that a training device can, by itself, be

considered a large system acquisition with its own associated resource requirements. Estimation of these additional resource requirements is beyond the current scope of

ETES. However, it is expected that these resource requirements could be estimated using comparability analysis.

Input to the determination of training device resource

-; requirements is provided by the task requirements associated

N- with each device which are determined in Procedure 2.0 and

_. the instructional programs developed during Procedure 4.5.

4

"* 4-26

- . . . . . . . -
- - < - -. ' -.--- - -

Table 4-12. Algorithm For Determining

Training Facilities Requirements

FR. = MST x FRS.

where FR. is the total square footage of facilities requirements

1- th required for the i facility type, MST is the maximum student

throughput and FRS. is the number of square feet required per student 1 th - for the i type of facility.

Maximum student throughput (MST) is calculated by the following

equation

MST = CS x MNC

where CS is the class size and MNC is the maximum number of courses

conducted concurrently during the year.

M1 If FRS. is not available, it may be estimated through comparability

analysis. More specifically, it can be calculated as follows:

FRSi TSF.

CS x MNC

where TSF is the total square footage in the comparable course for

th the i type of facility, CS is the class size for the comparable

course, and MNC is the maximum number of classes conducted concurrently

for the comparable course.

,-.4 -27

4 ..

° * - . S-
7.......

K*.~ ~ ~ ~~~r ____________________________

cu

U.

4-

E -L

cc

CdL

4-28
'0'

LI.

LI.

E0

4-

IA

Z -~

U. -

4- LI. o0

-C

U,

h.

-p... 0

C -

0E

* 0 LI.

0~0

0U,

4.~U

Cu

ULI.

* . I.-

1+

0 04-

.0 Cu Cu 4- ICuCu

4- (0

0:

* .C0

o CUcu

.0 U0

~1 *4* I-z -J

I-0

00 I.-

4-29

C . * *> . ..
... Ir - .r 

Table 4-15. Algorithm for Determining Number

of Training Devices/Equipments Per Site.

1. Determine number of hours to be spent on device

per day.

-NMAX (NCHOURS NSES *CLENGTH NHRSD NSTU-D) LN

NSES = Maximum number of sessions per start

NMAX = Maximum student/instructor ratio for course

NCHOURS = Number of hours to be spent on device for

each student

CLENGTH = Course length, in days

NSTUD = Number of students who can be trained on

device simultaneously

NHRSD = Number of hours to be spent on device per day

2. Determine number of devices required.

NHRSD - NDEVICE

SHIFT

NHRSD = Number of hours spent on device per day

SHIFT = Maximum operating hours per day per device

NDEVICE = Number of devices

3. Round NDEVICE TO NEXT HIGHEST INTEGER

4-30

.-

.- .- '
I.j

p-~

CL E

cz

LU.

cm LU

• L.

0a

ca

,-.--3

" .

0*, -

U .0
The same algorithm used for training devices can be used for

training equipment. Use of actual weapon system equipment

for training will generate additional resource requirements.

Again, determination of these resource requirements is

beyond the present scope of ETES. However it is expected

that comparability analysis could be used to determine such

additional resource requirements.

A worksheet for summarizing the results of the application

of the training device/equipment algorithms is provided in

Table 4-17. Table 4-18 displays guidelines for entering

data into the SDT.

4.6 DETERMINE REQUIREMENTS FOR OTHER TRAINING RESOURCES

OVERVIEW

During this procedure, resource requirements for media and

instructional aids and ammunition are determined.

PROCEDURE

Procedures for determining resource requirements for media

instructional aids and ammunition are provided in tho

sections which follow.

4.6.1 Determine Number of Training Nids/Instructional Media

There are two general algorithms for determining the number

of instructional aids and other forms of instructional media

directly assignable to a course: one algorithm for determining the number student-issued media and another algorithm

for determining the number non-student-issued media.

4-32

14- .
077 :

CL

4-

a,

U.

'7

2 E,

a 0

4-3
-- E-4

00

E--c

0 0E0

W w - - - - - _ _-

E0n Cl

J-j t40

CEC4 -40

4a 4 

-EaU)

-4

*z 0

C)4

* U,

0

4-3
Student-Issued Media

Media issued to students (i.e. , students consume the media

'" during the course or take it with them when they leave) is

':< determined by the following equation:

TMR i - NS x UR i

where TMR i is the total media requirements for the ith

medium, NS is the number of students taking the -ourse per

year, and UR i is the number of the ith media issued to each

student (this number is usually one).

Non-Student Issued Media

Media which is not issued to students may be determined by

the following equation:

TMR i = MST x URi

where TMR i is the total resource requirements for the ith

course, MST is the maximum student throughout the course and

URi is the usage rate for the ith media. Maximum student

. throughput (MST) is calculated by the following equation:

MST = CS x MNC

where CS is the class size and MNC is the maximum number of

courses conducted concurrently during the year. A worksheet

O. for determining the number of media and instructional aids

is presented in Table 4-19. A worksheet for summarizing the

results of this algorithm is presented in Table 4-20. Table

4-21 displays guidelines for entering data into the SDT.

4-35 S"
* -___

0 -

0

IE0

w

z

0

o~0

0

h..

0

0

0.0

C -~

0

-~

0 -

U,

0 0

2 0 *~ .0

z 3 ~ 0

C 0~/,

C .0

x ~ 0

* Z 0

C

o 0 0N L.

0 C 0 0.0 9. 0 0

U,

-~ C..,

0 ____________________________________ _____________________________

0

0

I- .~ I

0

0

~ C,,

0

o

C.,

0

4-36

~.. ~ -
LL

E >

I"

LL.

LL

u,

22 E

'4-3

'i: :i0

. U

i - :' .U -2 2 CU.

= - . . ,,i

* .

I.. L

"O,° 00

°" %°-,0

-"" 4-37 ..S z.
L

4.6.2 Determine Ammunition Requirements

To determine ammunition requirements for developing systems

during the early phases of the acquisition process, comparability analysis procedures must be employed. More specifically, comparable courses must be identified and data on the

ammunition requirements for these courses must be obtained.

This data should be available in the ammunition annex in the

Program of Instruction (POI) for each course. Ammunition

requirements for a comparable course should be modified to

reflect the task, methods/media, and operating and support

plan differences between a comparable course and a new

course. A comparable course should have been identified

during the POI instruction process (see Procedure 3.6).

Table 4-21 lists a worksheet which can be used to support

the modification of ammunition data from a comparable

course.

Table 4-22 lists a worksheet which can be used to summarize

ammunition requirements. Table 4-23 provides guidelines for

entering data into the SDT.

4-38

- - , *>I. - .z , -.. - -. " = _.
0 41

a~

z

J90

a0

CD 0 s 

.2.

C) CL

E 0 4

Coc

0 M

0 0

2L u ) 4

E .2 w

E EU

0 0 0 0

44-3
0

LL

S ____________________________________________________________________

~j, C,,

S

I LL.

U,

S

S

U,

0

S

S

S

.2 - 4-

U- S

S ___________________________

0

4-

CJ (U

(/,

O .0). .0 -*0 (U (U S

I- 4-

Cd,

0

£

.2

S

.0

0

-I

* ZI.- -J

000

I-. ~ 5 0 oo0w I- o~D~

* 4-40

- -. -........-. .. ~. .
71

E-4

x cn

0 0

0u

*s-J

-* >4

00

C

"-4 U

C4

r-44

0

- E- E

-Q 4-41
THIS PAGE INTENTIONALLY LEFT BLANK

4-42
SECTION 5 - DETERMINE TRAINING COST

OVERVIEW

During this procedure, costs are estimated for the BCS and

New system courses, and aggregate cost estimates are

developed for the Predecessor, BCS, and New systems.

PROCEDURES

An overview of the procedures in this step is provided in

Figure 5-1.

5.1 ESTIMATE INDIVIDUAL COURSE COST

OVERVIEW

During this procedure, costs are estimated for the BCS or

New system courses. A worksheet which can be used to

support this procedure is provided in Figure 5-2.

PROCEDURE

The first step in this procedure is to obtain the cost per

graduate for each course and to enter this information into

column (a) of the ICOST worksheet. For BCS or New system

courses which are unmodified existing courses, the cost per

*1 graduate can be obtained directly from the Cost Analysis

Program - MOS Training Costs (RCSATRM-159RI). Outputs from

this program may be obtained from the Resource and Economic

Analysis OPC, ATRM-R, in the DCS Resource Management - ATRM

at TRADOC Headquarters. All course costs should be obtained

from the latest version of the cost analysis program.

5-1
NUMBER COST OF ANALYSIS STUDENTS TO PROGRAM

BE TRAINEDPRGA

*PREDECESSOR

BCS NWSYSTEM

SOT

e- PREDEESSO
~0

LL.

z

I--
-.- .

Cost per graduate must be estimated for any modified or new

course which is part of either the BCS or New system. These

estimates can be obtained by applying the automated ETES

Resource and Cost Estimation Technique (RCET). Using RCET,

costs for a new or modified course are estimated by

identifying a comparable existing course, obtaining cost

data for this course from the ATRM system and then modifying

this data to reflect the differences in key resource

requirements (for example, number of instructors) between

the comparable and new course. An overview of RCET is

provided in Appendix B. A more detailed description of RCET

is provided in the User's Guide: RCET.

Once the cost per grad has been determined for each course,

the yearly number of students to be trained for both the

steady state and phased conditions must be taken from the

-" NSTD worksheet which was completed in Procedure 4.2 and

entered into the appropriate columns in the ICOST worksheet.

Yearly costs can then be calculated by multiplying the cost

per grad (from column (a)) by the number of grads in each

year.

Total costs per course can be obtained by summing over the

years in which the course will be taught (do not include

steady state costs in this calculation). In some

situations, it may not be possible to estimate the phased

graduate requirements (see Section 4.2). In these cases, it

is not possible to estimate phased course costs.

It should be noted that the cost estimates developed during

this step are only designed to provide input for training

evaluation; these costs may not be appropriate for budgeting

estimates.

5--4
After the ICOST worksheets have been completed, relevant

data form these worksheets should be entered into the SDT.

Table 5-1 describes the most desirable order for entering

data into the SDT. Additional guidance for entering data

into the SDT is provided in the SDT User's Guide.

5.2 AGGREGATE COURSE COSTS

OVERVIEW

During this step, courses costs are aggregated across MOSs

S,to provide total cost measures for the Predecessor, BCS, and

New systems. A worksheet (ACOST) which can be used to

support this procedure is provided in Figure 5-3.

PROCEDURE

Total steady state course costs for each MOS are obtained

directly from the subcost total under column (c) of the

ICOST worksheet and entered into the appropriate column in

the ACOST worksheet.

Similarly, the phased costs per year are obtained from the

subtotals listed in column (m) of the ICOST worksheet and

entered into the appropriate column in the ACOST worksheet.

These costs are then summed across MOSs to provide a total

cost value for the Predecessor, BCS and New systems.

-•a

:-O,

-.' 5-5

6
0

0

- 0

--

o - -

* C 0

u

E-4(3

00

-5-
ca

cc 

c

cnl

.00

0.)

~CL

0->
THIS PAGE INTENTIONALLY LEFT BLANK

-5-
SECTION 6.0 -ESTIMATE TRAINING EFFICIL-.NCYEFECI'

During this procedure, training "eff iciency" meas.

developed for three training program elements: (1) m

* . training devices (the most expensive $yeof medi

training methods (see Figure 6-1). In addition, e

of the effectiveness of an entire training prog

developed.

6.1 ESTIMATE TRAINING EFFICIENCY

OVERVIEW

Training efficiency provides a measure of how w

characteristics of the tasks to be trained mal

characteristics of a training prograt-(- eics .ent (for

media),1 Training efficiency measures are easily esi

even during the early phases of the acquisition proc(

PROC EDULRE

An overview of this procedure is presenLed in Fiqu,

three efficiency measures are developed durn

*procedure (training media efficiency, training

efficiency, and training method efficiency). Me

method efficiency are assessed using procedures f

Training Efficiency Estimation Model (TEEIM).

1 Th e concept o f t r a in ing efficiency (congruen

task characteristics) was initially devoloped '3y J(

(1979) in the TEEM project.

i-s-I
FTRAINING PROG RA M

ESTIMATE

TRAINING

EFFICIENCY

EFFICIENCY

& MEDIA

* TRAINING DEVICES

0 METHODS

ESTI MATE

TRAININGEFFECTIVENESS

ESTIMATED * TRAINING

EFFECTIVE NE7SS

Figure 6-1 Overview Diagram: Estimate Training Efficiency/Effectiveness

* 6-2
* W -:6 W 4 '. *- -7' --- - -

joMEDIAJ TASKS METHODS

ESTIMATE ESTIMATE ESTIMATE

MEDIA TRAINING MTO

MEDIA TRIIGMETHOD

EFFICENCYDEVICE EFFICIENCY EFFICIENCY

EFIIEC

MEI RANN MTO

Figure 6-2 Overview Diagram: Estimate Training Efficiency

6-3
device efficiency is measured using procedures from the

TRAINVICE methodology.

" 6.1.1 Estimate Media Efficiency

OVERVIEW

To estimate training efficiency for media, procedures from

the TEEM model are employed. These procedures have been

automated in the ETES Media Selection Program. An overview

of the Media Selection Program is provided in Appendix A. A

detailed description of the program is provided in the

User's Guide: Media Selection Program. This guide contains

. -. a detailed description of the procedures for estimating

media efficiency.

PROC EDURE

An overview of the procedure for estimating media efficiency

is presented in Figure 6-3. To begin the procedure, tasks

are rated on a set of variables describing the stimulus,

- response, and feedback variables. These ratings indicate

which variables can be applied to each task (indicated by

assigning a 1 or 0 to each task-variable combination). The

maximum possible match that is appropriate for each task is

then determined by adding up the number of 1's. (The

applicability of task variables to each media category has

already been determined and has been incorporated into the

Media Selection Program).

The match between media and tasks is determined by counting

2 .the number of psychological variables they have in common.

Training efficiency of each media is determined by dividing

the actual match by the maximum possible match.

6-4 ,S_
RATE TASKS

ON

VARIABLES

ATNG

--------------------------- .. &~~DTEMN z A?- ~ S. . A~ .. MAXIMUMAA *. *f -. S .V .< . < ' x~ .-
As an added feature, each task can be weighted by its task

criticality during the calculationi of efficiency using the

* following formula:

n m

E. E E C. m j 1 j1l k1 ~lj

where E1 is the efficiency of the ith MEDIUMN, Cj is the

criticality of the jth task, Mij is the match between the

ith medium and the jth task on the kth variable, N is the

number of tasks assigned to the medium, and M is the number

of psychological variables. Procedures for calculating task

criticality are described in Procedure 3.1.

* - It should be noted that the Media Selection Program can also

* be used to assign tasks to media as well as to evaluate the

efficiency of any given assignment, An overview of

* procedures for using the program to make media assignments

- . is presented in Procedure 3. 7.

6.1.2 Estimate Training Device Efficiency

OVERVIEW

Training devices are often the most expensive type of

* . training media to provide more ref ined estimates of the

efficiency of this critical type of training device,

procedures from the TRAINVICE methodology may be employed.2

2A detailed description of TRAINVICE is provided in the

Training Device Requrmns Guide.

6-6

\ . . .... .. . . . ..

- - - - - - - - - - . - -
PROCEDURE

Process: An overview of the steps in the TRAINVICE

procedure is presented in Figure 6-4. First, a measure of

task commonality (TC) is constructed by counting the number

of tasks which can be trained on the device. Second, a

measure of physical similarity (PS) is constructed by rating

-. how well the physical features of the controls and displays

of the device match the features on the control and devices

of the operational equipment. Third, a measure of functional

similarity (FS) is constructed by rating how well the information flow presented to students in each control displays

matches the information flow for the same controls and

displays in the operational equipment. Fourth, the improvement in skill and knowledge (SK) which can be expected for

each device is determined. Fifth, the task training

difficulty (TTD) of each task element is determined.

Finally, the total efficiency (called index of training

device effectiveness in the TEEM model) of a training device

is determined by the following formula:

TC + PS + PS + SK + TTD

3 2

2

A detailed description of the TRAINVICE procedure is

provided in Appendix G. A worksheet which an be used to

summarize the results of the TRAINVICE procedure is provided

in Table 6-1. Table 6-2 describes guidelines for entering

these summary results into the SDT.

6-7

( 0-
> U

- G)

Su..

cn
77 .V

0C

0 0

E-4-

(124

E-4 4-.

41 4

=0

0' >4

E-4ZV '

Li 4

E14 E0

41 0

a) U) x (0

4- (2l U2) En

C. 4~4 4-1 (1)

4- 44. E >E

0 0 ti -4 -

W 1*4 -4 41

*~ (( X.u(2 ro 

0 wo

C14 (N CN

cz
" .6.1.3 Estimate Training Method Efficiency

OVERVIEW

The same general procedures and algorithms used to assess

media efficiency can be used to assess the efficiency of

training methods. The major difference between the

application of the procedures to methods rather than media

is that a different set of varibles must be used to assess

the match between methods and tasks.

PROCEDURE

An overview of the procedures for assessing the efficiency

of training methods is presented in Figure 6-5. More

details on the steps in this procedure are provided in the

sections which follow.

o Rate Methods on Variables

Table 6-3 lists the instructional methods which are

typically used in the Army. Any one course is likely to use

some subset of these methods. The methods used in each

course must be rated on set of variables which assess the

match between the methods and tasks. A candidate listing of

such variables is presented in Table 6-4. 3 The user should

select five to twenty of these variables. The variables

selected should De those which are most likely to

discriminate between the methods included in the course.

3 These are the variables used by Jorgensen (1979) in the

TEEM project tu assess the efficiency of training methods.

%

6-11

•. 2
1.0em

RATE

METHODS RATE TASKS

VARIABLES 

ON 

VARIABLES

ON J

METHOD ASHTAS

DETERMIDETERMINEN

METHODTASK ATAMU

FOR TAASKTEATI 

CALCULATE

Er FICIENCY TASK

OF EACHCRITICALITY

METHOD

ITRAINING EFFICIENCY

METHOD

ACROSS

METHODS

TOTA L /T R AIN I NG S EFFICENCY

Figure 6-5 Overview Diagram: Cienerailieed Procedures for

* Assessing Efficiency of Trainling Methods

6- 12
Table 6-3. Method Alternatives,

AT Audio Tape

C Conference/Lecture

CAI Computer Assisted Instruction

CS Case Study

D Demonstration

SDF Dual Flight Hours (only aviator courses)

(do not include in ICH computations)

El Hardware Performance Examination

E2 Nonhardware Performance Examination

E3 Nonhardware Performance Examination

EL Elective (in-house only, except 'or CGSC)

F Film

GS Guest Speaker

iS Independent Study

"CI Non-contact Instruction with an Inst:rctor

Available in Classroom

IC2 Non-contact Instruction without an

Instructor Available

PE1 Hardware Oriented (hands-on) Practical

Application

PE2 Nonhardware Oriented (non-classroom)

Practical Application

PE3 Classroom Practical Application

PI Programmed Instruction (using programmed text)

PM Printed Materials

QC Besseler Cue See

S Seminar

SF Solo Flight Hours (only aviator courses)

(do not include in ICH computations;

SI Simulation Instruction

SP Self-Paced Instruction

ST Slide Tape

TV Television

WC! Instructor Led Work Group

WC2 Student Led Work Groun

Sources: DA Pam 570-558 3::: .

for U.S. Army Serv/ce Sc-.cc z

and TR.ADCC Cir 351-12 F-*7::

for Programs of

S;

S":
Table U-4. ehd-li&

Method Variah1es

STIMULI CHARACTERISTICS

Medium of Stimuli Presentation

1. Visual Cues - Signals received through the sense of sight.

2. Audio Cues - Signals received through the sense of hearing.

3. Tactile Cues - Signals received through the sense of touch, including

sensations related to texture, size, shape, or vibration of the skin.

4. External Stimulus Motivation Cues - The sensations felt by a person

when he is moved by some outside force in such a way that his body

experiences roll, pitch, yaw, heave, sway and/or surge.

5. Internal Stimulus Motion Cues - The sensations felt by a person when

he moves his arm, leg, fingers, etc.

6. Olfactile Cues - Signals received through the sense of smell.

7. Gustatile Cues - Signals received through the sense of taste.

Number of Channels or Sources

8. Limited - A small number of sources, channels, or instruments through

which stimuli are presented to the trainee.

9. Unlimj - A multiple number of sources, channels, or instruments

throL ahich stimuli are presented to the trainee.

Stimuli Pacing

10. Self-paced - Stimuli are only presented upon the trainee's request.

11. Forced-pace - Stimuli are presented at some predetermined rate,

frequency of change, etc.

Stimuli Distribution

12. Individual - All information is presented or displayed directly to

one individual trainee.

13. Group - Information is presented or displayed to a gioup of trainees,

allowing only indirect access to the information for an individual.

* Taken from Jorgensen and Hoffer (1978)

6-14

. " . --.-.-- , v., -- - - - - - - - . . . . .,.. .-.... - -- -- - ,.
Information Structure-Related VariablesSTIMULI CHARACTERISTICS

Medium of Stimuli Presentation

1. Visual Cues - Signals received through the sense of sight.

2. Audio Cues - Signals received through the sense of hearing.

3. Tactile Cues - Signals received through the sense of touch, including

sensations related to texture, size, shape, or vibration of the skin.

4. External Stimulus Motion Cues - The sensations felt by a person when

he is moved by some outside force in such a way that his body exper-

. iences roll, pitch, yaw, heave, sway and/or surge.

5. Internal Stimulus Motion Cues - The sensations felt by a person when

he moves his arm, leg, fingers, etc.

6. Olfactile Cues - Signals received through the sense of smell.

7. Gusatile Cues - Signals received through the sense of taste.

Stimuli Presentation

8. Static - A unitary stimuli situation, i.e., stimuli are presented to

the trainee "all at once", e.g., batch presentations.

9. Dynamic-Ordered - A sequential stimuli situation, i.e., stimuli are

presented to the trainee sequentially or in an ordered minne- over time.

10. Dynamic-Random - A non-sequential stimuli situation, i.e., stimuli

are presented to the trainee randomly over time.

Stimuli Presentation Rate

11. Slow Rate - A slow rate or speed of prescrntation of stimuli to the

trainee, allowing the trainee a long or maximum stimulus analysis time.

12. Fast Rate - A fast rate cr speed of presentation of stimuli to the

trainee, allowing the trainee a short or minimum stimulus analysis

time.

Frequency of Stimuli Change

13. Infrequent - A low frequency of stimuli change, i.e., stimuli presented

to the trainee change from one to another infrequently.

" 14. Frequent - A high frequency of stimuli change, i.e., stimuli presented

to the trainee change from one to another freuently.

* Tien from Jorgensen and Hoffer (1978)

6- 1)

S 0

",-°" •.
- 6- V - 

y

Table 6-4 (Continued)

Number of Stimuli

15. Few - A small number of separate streams of signals that occur

independently of each other, but require simultaneous consideration.

16. Multiple - A large number of separate streams of signals that occur

independently of each other, but require simultaneous consideration.

Stimuli Format

17. Formatted - Formatted or organized stimuli, such as alphanumeric

displays.

18. Unformatted - Unformatted or unorganized stimuli, such as raw video

displays.

Patterning of Stimuli

19. Simple - Simple or common patterns of stimuli, such as the repetition

of the same event at high frequencies of occurrence.

20. Complex - Complex, detailed, or circumstantial patterns of stimuli,

such as discrete sequenced patterns at low frequencies of occurrence.

RESPONSE CHARACTERISTICS

Response Mode of Implementation

1. Overt Response - Verbal - A response which the trainee expresses in an

audible (verbal) manner, such as a verbal short answer response to a

question having a limited set of correct answers, a conversational

response, or a verbal decision response.

2. Overt Response - Written - A response which the trainee expresses

in an observable (written) manner, such as a free style written

response, a written multiple choice response, or a written fill-inthe-blank response.

3. Overt Response - Manipulative Acts - A response which the trainee

expresses in an observable (manipulative) manner, such as the small

movements of dials, switches, keys, or small adjustments to instruments

or the large movements of levers, wheels or use of hand held tools.

4. Overt Response - Tracking - A response which the trainee expresses

in an observable (tracking) manner, such as continuously controlling

a constantly changing system, e.g., steering an automobile.

5. Overt Response - Procedural Performance - A response which the trainee

expresses in an observable (procedural performance) manner, such as

performing a sequence of steps in a procedure, e.g., carrying out the

items on the checklist for preflighting an aircraft or turning on a

radar system.

6-16

.*". " . .- * - " " - "- " "'" " " " %. - ° . . . ", ... / .. . ' . . • .- "', ,' -/ ,' " _ , " .' o"%- • • -. = -- . ., , . . . . .
Table 6-4 (Continued)

6. Covert Response - A response which the trainee creates in his mind

but does not express in an observable or audible manner.

Response Implementation

7. Static - A unitary response situation, i.e., responses are made by

the trainee "all at once".

8. Dynamic-Ordered - A sequential response situation, i.e., responses

-are made by the trainee sequentially or in an ordered manner over time.

9. Dynamic-Random - A non-sequential response situation, i.e., responses

are made by the trainee randomly over time.

Required Response Rate

10. Slow Rate - A slow rate or speed of trainee response, i.e., a rate

which allows the trainee a long or maximum response time.

11. Fast Rate - A fast rate or speed of trainee response, i.e., a rate

which allows the trainee a short or minimum response time.

Response Frequency

12. Infrequent - A low frequency of response change, i.e., responses made

by the trainee change infrequently.

-V.: 13. Frequent - A high frequency of response change, i.e., responses made

V.,' by the trainee change frequently.

Number of Required Responses

4 14. Few - A small number of responses that the trainee is required to

4.. express, each independent of the other.

15. Multiple - A large number of responses that the trainee is required

to express, each independent of the other.

Response Format

16. Formatted - Formatted responses, e.g., responses to a programmed text.

17. Unformatted - Unformatted responses, e.g., responses made by team

members during an emergency situation.

od Patterning of Responses

18. Simple - Simple or common patterns of responses, such as repetition of

the same response at high frequencies of occurrence.

19. Complex - Complex, detailed, or circumstantial patterns of responses,

such as discrete sequenced patterns of responses at low frequencies of

occurrence.

~3% 6-17

A f&4f2A..
-"

4%

Table 6-4 (Continued)

SINFORMATION FEEDBACK LOGIC

Medium of Feedback Presentation

1. Visual - Feedback presented visually by means of a display, it may

be coded and transmitted visually to the trainee.

2. Aural - Feedback presented aurally by means of a display to the

trainee.

3. Written Form - Feedback presented to the trainee in written form.

4. Face-to-Face Communication - Feedback presented by direct verbal

means to the trainee.

5. Indirect Communication - Feedback presented by indirect verbal means,

such as by intercom, telephone, or radio link.

6. Tactile - Feedback presented to the trainee through the sense of touch,

including sensations related to texture, shape, size, or vibration

of the skin.

7. Kinesthetic - Feedback presented to the trainee by either internal or

external bodily movement, such as reaching, grasping, tilting, etc.

8. Olfactile - Feedback presented to the trainee through the sense of smell.

9. Gustatile - Feedback presented to the trainee through the sense of taste.

Time Schedule for Feedback

10. Pre-F Interval - Immediate - Feedback provided immediately after the

trainee's antecedent response, i.e., there is a small or no interval

of time between the antecedent response and the feedback for that response.

11. Pre-F Interval - Delayed - Feedback provided subsequently after the

trainee's antecedent response, i.e., there is an interval of time or

delay between the antecedent response and the feedback for that response.

12. Post-F Interval - Immediate - Presentation of the next stimulus

immediately after the occurrence of feedback for the last response,

i.e., there is a small or no interval of time between the occurrence

of feedback for the last response and the presentation of the next

stimulus.

13. Post-F Interval - Delayed - Presentation of the next stimulus subsequently

after the occurrence of feedback for the last response, i.e., there is

an interval of time between the occurrence of feedback for the last

response and the presentation of the next stimulus.

6-18

•.
Table 6-4 (Continued)

Feedback Regularity

14. Regular - Feedback provided to the trainee at regular intervals,

such as after every or every other response or at established or

fixed periods.

15. Irregular - Feedback provided to the trainee at variable intervals,

which may change as a function of stage of training or level of

performance. This includes the provision for intermittent presentations

to permit probabilistic schedules for reinforcement.

- , Frequency of Feedback

. -16. Frequent - Any feedback given to the trainee that is provided at least

for 20-30% of all his responses.

17. Infrequent - Any feedback given to the trainee that is provided less

than for 20% of all his responses or is not provided at all.

Feedback Format

18. Formatted - Formatted feedback, e.g., alphanumeric feedback displays.

19. Unformatted - Unformatted feedback, e.g., raw video feedback displays.

• .- o.

6-19

°..7-
F. r FT

Table 6-4 (Continued)

Response Criterion

14. Determinate - The response criterion has a fairly explicit or implicit

response requirement as a direct reflection of the training or system

requirement, and makes the stimulus (visual, audio, etc.) possess a

certain or determinate value or payoff for the trainee.

15. Indeterminate - The response c:iterion is highly dependent on the

trainee's interpretation of the response requirement, and makes the

stimulus (visual, audio, etc.) possess an uncertain or indeterminate

value or payoff for the trainee.

RESPONSE CP 'kCTERISTICS

Response Mode of Implementation

1. Overt Response - Verbal - A response which the trainee expresses in

an audible (verbal) manner, such as a verbal short answer response to

a question having a limited set of correct answers, a conversational

response, or a verbal decision response.

2. Overt Response - Written - A response which the trainee expresses

in an observable (written) manner, such as a free style written

response, a written multiple choice response, or a written fill-inthe-blank response.

3. Overt Response - Manipulative Acts - A response which the trainee

* expresses in an observable (manipulative) manner, such as the small

movements of dials, switches, keys, or small adjustments to instruments

or the large movements of levers, wheels or use of hand held tools.

* 4. Overt Response - Tracking - A response which the trainee expresses

in an observable (tracking) manner, such as continuously controlling

a-.. a constantly changing system, e.g., steering an automobile.

5. Overt Response - Procedural Performance - A response which the trainee

expresses in an observable (procedural performance) manner, such as

performing a sequence of steps in a procedure, e.g., carrying out the

items on the checklist for preflighting an aircraft or turning on a

radar system.

6. Covert Response - A response which the trainee creates in his mind

but does not express in an observable or audible manner.

6-20
Table 6-4 (Continued)

Response Pacing

7. Self-paced - Responses are made by the trainee at his own rate.

8. Forced-pace - Responses are made by the trainee at some predetermined

rate or frequency.

Response Distribution

9. Individual - All responses are expressed by one individual trainee.

10. Group - Responses are expressed by a group of trainees, allowing

only indirect responses for an individual.

Response Determinancy

11. Determinant - A required response which can be specified in advance

of the stimuli that calls the response out, i.e., the response is

preprogrammed or determined before the operation begins.

12. Indeterminant - A required response which cannot be specified in

advance of the stimuli that calls the response out, i.e., the response

* is highly dependent on the immediate stimuli situation and cannot be

predicted in advance, hence is indeterminate.

INFORMATION FEEDBACK LOGIC

Medium of Feedback Presentation

1. Visual - Feedback presented visually by means of a display, it may be

coded and transmitted visually to the trainee.

2. Aural - Feedback presented aurally by means of a display to the trainee.

3. Written Form - Feedback presented to the trainee in written form.

4. Face-to-Face Communication - Feedback presented by direct verbal means

to the trainee.

5. Indirect Communication - Feedback presented by indirect verbal means,

6. such as by intercom, telephone, or radio link.

6 Tactile - Feedback presented to the trainee through the sense of touch,

including sensations related to texture, shape, size, or vibration of

- the skin.

7. Kinesthetic - Feedback presented to the trainee by either internal or

external bodily movement, such as reaching, grasping, tilting, etc.

8. Olfactile - Feedback presented to the trainee through the sense of smell.

9. Gustatile - Feedback presented to the trainee through the sense of taste.

6-21
.' n

Table 6-4 (Continued)

Source of Feedback

10. Intrinsic F - Information or cues built into the system from which

the trainee interprets feedback information.

11. Extrinsic F - Information or cues not inherent in the trainee action

or system operations but is supplied by an external source.

Feedback Pacing

12. Self-paced - Feedback presented to the trainee only at his request.

13. Forced-pace - Feedback presented to the trainee at some predetermined

rate, frequency, etc.

Feedback Distribution

14. Individual - Feedback is presented to one individual trainee.

15. Group - Feedback is presented to a group of trainees, allowing only

indirect access for an individual.

5..

.,

6-22

did
V'.

Table 6-4 Functional Context Variables*

ROLE OF ELEMENT - The social function performed by the trainee within

the system's operational context.

1. Supervisory - The trainee's function is unequal to functions being

performed by other individuals; the role is basically one of overseeing or directing.

2. Team Performance - The trainee's function is equal to functions being

performed by other individuals; the function is basically one of

teamwork and cooperation.

3. Individual Performance - The trainee's function is one of performing

alone, usually without supervision or team assistance.

FUNCTION PERFORMED IN ROLE - The primary actions performed within each role.

4. Mental - An action occurring or experienced in the trainee's mind, as

contrasted with overt physical activity.

5. Physical - An overt bodily action performed by the trainee.

6. Perceptual - An action by the trainee involving perception or observation.

7. Communicative - An action by the trainee in which he transmits either

a written or verbal message.

STABILITY OF FUNCTION - The function's state, quality, or degree of being

constant over time.

8. Unstable - The trainee's function is not constant or regular, it is

characterized by continual change and fluctuation.

9. Stable - The trainee's function has little change or fluctuation over

time.

PHYSICAL CONTEXT - The significance of the physical environment upon

performance of the task.

10. Low Impact - The physical environment has little or no significant

impact on performance of the task.

% 11. High Impact - The physical environment has a large or significant

impact on performance of the task.

% . PSYCHOLOGICAL IMPACT - The significance of the psychological environment

upon performance of the task.

12. Low Impact - The psychological environment has little or no significant

impact on performance of the task.

, 13. High Impact - The psychological environment has a large or significant

impact on performance of the task.

* Taken from Jorgensen and Hoffer (1978)

6-23

'4'
Once the variables have been selected, each method must be

rated in terms of whether or not the variable applies to

that method. Table 6-5 contains a worksheet which can be

used to make these ratings. The entries in the worksheet

would be one if the variable applies to the method and zero

if it does not.

- o Rate Tasks on Variables

Each of the tasks to be trained must be rated on the same

variables used to assess training methods. Table 6-6

contains a worksheet which can be used to make these

ratings. The entries in the worksheet should be one if the

variable applies to the method and zero if it does not.

o Determine Method-Task Match

The match between a task and an instructional method is

determined by counting the number of times the same variable

is rated as applying to both the method and the media (see

Table 6-7).

o Determine Maximum Possible Match Between Each Task

The maximum possible match for a task is determined by

counting the number of variables which were rated as

applying to a task (see Table 6-6).

o Calculate Efficiency of Each Method

Total efficiency for a method is determined by dividing the

method-task match score by the maximum possible match (see

the worksheet listed in Table 6-7) to produce an efficiency

6-24

*8 -- , ' -, ' ----- -".- . . .. ". . -" , v -v. " . .'.- -- .,
. .-

0; 1

" . I

-'-. -°- - t -- - -1

\ I

I i

o

- -"" Ki7- 3 - - - -o.- - - -

. - -.---- - - - - - - - - - - - -

" - - - - -- - - ".:. 6-25

I.: ,,-,-.....:,,v.' ... . .. ...- '. ,,.' '.::',': -. , ., .'- "-- --- "-,-- - -- -.-.- - _

• .'" " " " ,'; "&" '" " ' r " ', - -" -" "

- -> -" - - '

" " '

" ' - - "- -" - "
- . -

*6~4 -. - - -

iii~ --

1~~7V

o I------------------------------------ 6/6 4- @1 - - - - - - - - - -1-------- .0

Cu Cu

I-

.2

4- .~

0 ------ - -

4-

U,

C - - - ---- -

o 3

4-a,

6/6

0

4-C

'U

U

I- - - - - - - - - - - - -

S

.0

'U

I.-

4

a .0

~

I.-

CU

00>

'-'(-'C',

6-26 _____________
Joi

LUU

LUU

C4

0

LL L

I-J

'0

LU

U 0U

01

CC

0 0 0-L0

0e L) C a

6-2
score for each method-task combination and then summing

across tasks.

o Determine Task Criticality

As an added feature, each task can be weighted by its task

criticality during the calculation of total efficiency for a

method (see Table 6-7). Procedures for calculating task

criticality are described in Procedure 3.1.

o Sum Across Methods

Total efficiency for a course can be determined by summing

across the efficiency scores for each method.

6.2 ESTIMATE TRAINING EFFECTIVENESS

OVERVIEW

This procedure can be used to develop a rough measure of the

effectiveness of a training course during the early phases

of the acquisition process. In this procedure, subject

matter experts are used to estimate the percentage of

soldiers who will pass the criterion for each task. More

specifically, a group of subject matter experts is presented

with the following information for each task: (a) the target

population description of the personnel who will perform the

task, (b) a description of the task including its associated

0 conditions, job performance standards, and general skills

and knowledges, (c) a description of the training program or

training program elements (e.g., methods, media) that will

be used to train the task, (d) the criterion which must be

O achieved for the task (if different from the job performance

5,2 6-28
1 L

measure), and (e) the training time to be devoted to the

task. Each SME is then asked to estimate the expected

percentage of soldiers in target population who will pass

the criterion given that training program. A total

effectiveness measure for a course can be constructed by

averaging the effectiveness scores associated with each

- - task.

PROCEDURE

An overview of the procedure for estimating effectiveness is

presented in Figure 6-6. A more detailed description of the

steps in this procedure is provided in the sections which

follow:

o Select Subject Matter Experts

To provide usable estimates of effectiveness, a subject

matter expert must (1) have at least two years experience in

performing the task or other comparable tasks and (2) have

at least two years experience in Army training developments.

As many individuals as possible who meet the above criteria

should be selected (admittedly, this number will not be

large).

o Complete Rating Forms

Table 6-8 presents a rating form which can be used to elicit

the estimates of effectiveness. One of these forms should

be filled out for each task in the training program.

Obtaining the information to put in the forms should be

relatively straightforward since all of this information

should have been developed in previous ETES procedures.

6-29

" . . . . . . . .-
SELECT

SUBJECT

MATTER

EXPERTS

COMPLETE

RATING

FORMS

PRESENT

RATINGS-TO

SMEs

CONSTRUCT

OVERALL

MEASURE OF

EFFECTIVENESS

Figure 6-6 Overview Diagram: Estimate Training Effectiveness

6-30
V1 '

Table 6-8. Form for Rating Effectiveness.

Task Description

Task Number:

Task:

Comparable Task:

Conditions:

Standards:

Major Skills and Knowledges:

Training Course Module

4 Training Methods Training Time Media

Total

*Crierion

What percentage of soldier can be expected to pass the criterion given the training program?

.4 6-31

• ; , " - "= I - ,, .
.::

Table 6-9 describes the ETES procedures where each item in

the rating form are generated.

O Present Ratings to SMEs

Each subject matter expert should be presented with the

rating forms for all of the tasks in the course. Prior to

the presentation of these forms, the SME should be given the

QPOI (Procedure 3.6) and target population description

(Procedure 3.4) for the course. Each SME should be

instructed to leave the form bl&nk if he/she is not familiar

with the task or the comparable existing task.

o Construct Overall Measures of Effectiveness

Table 6-10 provides a worksheet which can be used to develop

an overall effectiveness measure for a course. Training

effectiveness estimates should be taken from the

effectiveness rating form and entered into columns (a) and

(b), respectively. Mean scores should be calculated across

SMEs and entered into column (c). The total of these mean

scores (item d) should then be calculated and divided by the

number of tasks to provide an overall measure of course

effectiveness.

6-32

'p:
Table 6-9 Guidelines for Developing Elements

in Training Effectiveness Form

N

ELEMENT RELATED ETES PROCEDURE

Task Number and Title 2.2, 2.4

Comparable Task 2.2, 2.4

Conditions, Standards 2.2, 2.4

Skills and Knowledges 3.3

Training Methods and 3.6

Media

Training Times 3.6

Course 3.6

Setting 3.2

Criterion 3.6

6-33

..

.1

I

4.:•-" ; ' :'" " ,% " ,"" "" . , " -" - " - -" - " - - - ".. S .-. " -- ".:. "- , i
00

U)*

CO) PA2

402

cw

30, LQ LU

00

IL.

0 a 4- go

J2E

4. ~' I

6-3
SECTION 7.0 EVALUATE TRAINING PROGRAM

OVERVIEW

This procedure provides techniques for evaluating the

training program which was developed in the earlier ETES

*procedures.

PROCEDURE

This procedure consists of six lower level procedures. An

overview of these procedures is presented in Figure 7-1.

In the first procedures, eight figures-of-merit for

evaluating the "goodness" of a training program are

developed. In the second procedure, these figures-of-merit

are used to identify likely problem areas in the training

program. In the third procedure, the likely causes of the

problem areas are identified In the fourth and fifth

procedures, methods are provided for identifying and

Sevaluating training alternatives. In the sixth procedure,

the impact of the system changes are determined

7.1 DEVELOP FIGURES-OF-MERIT

During this procedure training Figures-of-Merit are

developed to evaluate the training courses developed in the

previous ETES procedures. A listing of the Figures-of-Merit

which may be used to evaluate the training course estimates

during the earliest phases of the acquisition process is

presented in Table 7-1. A training Figure-of-Merit provides

"."" 7- 1

• -. ,, : ~ . ,' . ... . .. , . ,.. , . . ." ... . .. " . . . .. .. . ,
.1.1

IDENTIFY/

CONSTRUCT

FlOUR ES-OP-MERHIT

FIGURES.

*COST

*TRAINING EFFICIENCY

*TRAINING EFFECTIVENESS

*CONGRUENCE WITH TRAINING PROGRAM GUIDELINES

*CONGRUENCE WITH PROGRAM REOUIREMENTS

0 TRAINING COMPLEXITY

PEASIBIUTY

*SUMMARY EVALUATION SCORE

7.2

IDE NTIFY

LIKELY PROBLEM

AREAS

AREAS

IDENTIFY

LIKELY CAUSES

LIKELY CAUSES

* IDENTIFY ALTERNATIVES

LIKELY

ALTERNATIVES

IMPACT OF

SYSTEM

EVALUATE CHANGES

ALTERNATIVES

67

Figure 7-1. Overview Diagram: Evaluate Training Program (7.0).

7-2
Table 7-1. Potential Training Figures of Merit

1. Cost - the cost of the training program or training program

element as determined in ETES.

2. Training Efficiency - a measure of how well the training

program matches the task requirements of the system.

Training efficiency is determined by comparing the

characteristics of the tasks to be trained with the

characteristics of the training program element.

3. Training Effectiveness - training effectiveness is measured

by assessing the proficiency of the soldiers who experience

the training program element.

4. Congruence with Training Development Guidelines - a measure

of the degree to which the training program has been constructed in accordance with recommended or specified principles

of Army instructional system's development.

5. Congruence with Program Requirements - a measure of how well

the training program matches requirements outlined in requirements documents (e.g., JMSNS, LOA, TDLOA) or in request for

proposal3. This measure is not applicable unless meaningful

statements of training requirements have been developed.

6. Training Complexity - a measure of the training complexity

of a particular system or system element. Training complexity

is determined by summing the training man-days associated for

each course associated with the system.

7. Feasibility - a composite measure assessing (a) the technical

risk associated with development of the training program and

(b) the likelihood that the training program can be developed

within existing resource, cost, and schedule constraints.

8. Summary Evaluation Score - a summary measure combining the

three key training figures-of-merit -- cost, effectiveness,

• ..'.and complexity.

7-3
. . 7

a summary measure of the "goodness" of a training program or

course. In most cases, the training Figures-of-Merit will be

developed to summarize the "goodness" of course. In some

cases, however, they may be applied to the elements of a

course. Table 7-2 lists the training program or course

elements which are likely to be evaluated during the early

phases of the acquisition process and describes which of the

figures-of- merit are most appropriate for evaluating each

course element.

The training figures-of-merit can be used to evaluate the

training-related impacts of changes to non-training system

elements such as changes in manpower and personnel

assignments and requirements, changes in system hardware/

design, and the overall system schedule and budget. Table

7-2 indicates which training figures-of-merit can be used to

assess the impacts of each of these non-training system

elements. A more detailed discussion of procedures for using

training figures-of-merit to assess changes in overall

system design is presented under Procedure 7.6.

Of course, the individual training figures-of-merit are not

independent of one another. Figure 7-2 presents an overview

of the casual relationships among the training figures of

merit. A more detailed discussion of these casual

relationships is presented under Procedure 7.3.

Procedures for estimating training cost, efficiency, and

effectiveness were presented under Procedures 5.0, 6.1, and

6.2, respectively. During this procedure, procedures are

provided for estimating the other Figures-of-Merit listed in

Table 7-1.

7-4

* - . . . . .. . ~**..**..*.**- . . . ... *.. ~ - * ~ - Z_ - - - .
Table 7-2. System Elements and Applicable Figures of Merit.

E E o CL- 0~. ..

E ~ .)uu~IW0 wj

SyOe ElmE 0

Training Settings 0 00

Methods 0 0 0

Media 0 0

Course Sequencing

(between course)________________________

Training Times 00 0 0 0 0

* Course Scheduling

(within course),

Freuencv and Locations ________________

Usage Rates for 0 Training Resources

* .-. *.Assignment of Tasks to

MOS, Duty Position, and 0 0

Skill Level________

MOS/Selection

Requirements

*Hardware/Software Design 0 0 0

System Schedule

and Budget

7-5
w

0

a C.7

0

LUUUJ CC

Lu, u

00 Cd, 

-~~ l20 .

w zw ______7-60
w w . .. , _.. < . ¢r_.r< . .< . 'f' , -- ,.- . < .- - - r Z .. r W ... - - - r- ,.-- - - .. - ,- . .- I--"

The user should select which figures-of-merit I.1 1. be

applied to each training program element as part of

development of the ETES study plan, (see Section 1.6),

7.1.1 Assess Congruence with Training Development

Guidelines

OVERVIEW

Several documents have been produced to guide Army training

developers in developing training programs for emerging

systems. These documents represent the collective wisdom

and expertise of the training development community. One

can assume that training programs which have been developed

in accordance with these guidelines are likely to be more

effective than training programs which are not. This

figure-of-merit provides a measure of the congruence between

I the procedures actually used to develop the training program

element and the procedures prescribed in existing

, guidelines. This figure-of-merit can play an important role

* -in assessing the training products developed by outside

contractors. Obviously, if this measure is to have any

credibility, it should not be assessed by the same

organization which developed the training program.

PROCEDURE

An overview of the procedure for measuring the congruence

between the actual procedures used to develop the training

0 program element and the prescribed procedures is presented

in Figure 7-3. First, existing documents containing

guidance for developing the particular type of training

program elements under investigation are identified. A

7-7

%S -
IDENTIFY

SOURCES

SOURCES

CONSTRUCT

CHECKLIST

CHECK LIST

RATE TRAINING

PROGRAM

ELEMENT(S)

RATED

TRAINING

PROG RAJM

ELEMENT(S)

WEIGHT CONGRUENCE

_Figure 7.3 Overview of Diagram: Construct Measure of Congruence

Between Actual and Prescribed Training Development Prartices

7-8
preliminary list of such sources can be generated by

examining the sources listed in Table 2-3 and the list of

references provided at the end of this guide. Additional

guidance is provided in the Army ISD job aids developed by

Schulz and Farrell (1980).

Once the sources have been identified, a checklist must be

developed specifying the observable features that the

training program element would display if it was developed

in accordance with the guidelines specified in the sources.

An example checklist is provided in Table 7-3.

Once the checklist has been developed, each program element

must be rated as to whether it displays the characteristic

required in each item on the checklist. Following this, the

importance of each item on the checklist should be assessed

using the following scale:

1. Not at all Important

2. Somewhat Important

3. Moderately Important

4. Important

5. Extremely Important

Completion of the checklist and the importance ratings

should be done by training analyst who have at least three

years experience in Army training developments. Importance

ratings should be averaged across rates. An item should be

rated as meeting existing guidelines if a majority of the

analysts rate the item as meeting requirements.

Once the importance of the items has been determined, a

general measure of the congruence between actual and

7-9

Io

.............
* . . -• -. S-•

,.* 5' .* . 5. -.-. *. . * -_- * .

0C

/CO

C@

00

. j CC

0 .C

'D

*.-.' .. :

E - _-. U 

a. 0

0.

CM

CD

73

00

oE

~cn -

0o 0

07-10
,4-.

prescribed training development procedures can be developed

using the following algorithm:

M

CTD= E Ik Dk

k=i

where CTD is the general congruence measure, Ik is the

importance of the kth checklist item; Dk equals 1 if there

is a match between the actual and prescribed practice for

the kth checklist item and 0 otherwise; and k is the number

of checklist items.

Table 7-4 displays a worksheet which can be used to support

the development of the training congruence Figure-of-Merit.

7.1.2 Assess Congruence with Program Requirements

It is possible that requirements relating to the training

program may be included in system requirements documents

(e.g., JMSNS, LOA). Such requirements may also be included

in the request for proposals (RFPs) associated with the

system. One measure of the "goodness" of the training

program is the extent to which it meets these requirements.

This figure-of-merit can play an important role in

evaluating contractor developed training program elements.

It should be noted that this measure is less meaningful when

the organization setting the requirements is also the same

organization producing the training program element.

7-11

. . --
>. -- . .

00

a CA

z

0

CD

S0

0

o m

CE~ a--

cc

0

7o1
PROCE DURE

The first step is to identify training-related requirements

by examining existing system requirements documents (JSMSN,

LOA, ROC, T DLOA, TDR) and/or contractor Request for

Proposals. Second, a checklist must be developed specifying

the observable components of each requirement. Third, each

training program element must be rated on the extent to

which it meets the specified requirements. Fourth, the

importance of each requirement should be assessed using the

following scale.

1 = Not At All Important

2 = Somewhat Important

3 = Moderately Important

4 = Important

5 = Extremely Important

Congruence with requiremen.ts can then be determined using

the following algorithm:

,-.- M

CR= Ik Rk t'. k=l k k1

where CR is the general congruence measure; Ik is importance

of the kth requirement, Rk equals 1 if the training program

meets the requirement and 0 if it does note, and K is the

number of requirements.

A generic checklist is provided in Table 7-5. A worksheet

which can be used to support the development of this figureof-merit is displayed in Table 7-6. Completion of the

checklist and the importance ratings should be done by

training analysts who have at least three years experience

in Army training developments. Importance ratings should be

averaged across raters. A item should be rated as meeting

7-13
0

'S.

@3

Yb

4- *.1 E

U3E

(U

030

h.. 0.

.0

4.'

@3 a.-@3 *

I03 

E

C (U

o a.. C., 030 03

0 0.

Yb 03CC

(Ua.. a.. I- 0A9

.-

S . 4-

V. U

@3

.0C.,

* .. U

a..

@3 4.. C C

@3 @3

C, E

@3

I';

0:1

(U E I-. (Ua.-

032

E

@3

Co

'A

01 7-14 9- a

~N". -A.........- -........... S ... S.SS S 9 .'*~'\ .-.- 

A
LM

CD usa

CD,

Eo aD0))

721
existing guidelines if a majority of they analysts rate the

item as meeting requirements.

7.1.3 Assess Training Complexity

•% .'..

OVERVIEW

* DuRring the early phases of the acquisition process, it is

important to identify the system elements which. generate

excessive or complex training requirements. By constructing

a measure of training complexity of each system element

training developers can help materiel developers in

identifying potential design problems. A relatively single

measure of training complexity which can be used during the

* early phases of the acquisition process is training time

(course length).

PROCE DURE

.-

Training complexity is initially determined for each course

associated with the system. A summary complexity measure

for each MOS is determined by summing the length of the

courses associated with the MOS. The training complexity of

4 a system may be determined by summing the length of the

courses for all MOSs required to operate or maintain each of

the component subsystems.

The training complexity measures for each MOS can be

9examined to identify the system areas with excessive

requirements (see procedure 7.2 for more guidance on

procedures for identifying problem areas). A worksheet

which can be used to summarize the training complgexity

measure is provided in Table 7-7.

7-16

* ... *4P .OCE DURE . . . - - .. - .. - . . . . . . .
~O

a,

-J0

a,

* Cu a.

Cu Cu Cu .4- 4-

o 0 

0

a, IC,,

o 0 

0

a. C.) 0

4-K

a, 4-

C.

C E a,

o .4

C.) C

* .~ 

~

Cu ) 

0

I- 

~C

C a, 

- 4-. N -

* : 

(

* 

EE

Cd,

-. 

0

-4-

4- Cu Cu Cu

a~ 4- 4- 4-

a, 0 0 

0 a, k IA

Cd,

o 0 0 

0

0

rZ 4-

0~ * a, C

a, -~

* Cu

0-' 

0

77*

7%~

* 4-

IA

Cu Cu Cu 4- 4-

o 0 

0

-p 4,,

b.. Cd, C,, Cd, I-.

o 0 0 0 

0

C.) ~ I- 0 - - -

7-17

......... * 7 

- 7 * 7 7 

. S7 . . 7 . 7 7 - ....................................... 7 *.......*................~ ~
7.1.4 Assess Feasibility

OVERVIEW

One important factor to consider in assessing the "goodness"

.2-,..!of training program is its feasibility - i.e., the estimated

likelihood that the training program element can be

developed within existingj technical, cost, resource, and

schedule constraints. The feasibility of a training program

consists of four components:

(1) Technical Feasibility the likelihood than any

new or modified technologies in the training

program can be developed given the current stateof-the-art in trining technology. If existing

technologies are employed, this factor is not

relevant.

(2) Cost Feasibility the likelihood that a training

program or training program element can be

developed and maintained within the existing

ftbudget allotted to training development and

operation.

(3) Resource Feasibility the likelihood that a

training program can be developed and implemented

given the availability of critical training

resources (e.g., instructors).

(4) Schedure Feasibility - the likelihood that a

training program can be developed in accordance

with required system schedule milestones.

7-18 ° .
PROCE DURE

An overview of the procedures for assessing the feasibility

of a training program or set of training program elements is

provided in Figure 7-4.

First, the training program elements are rated on the

feasibility scales listed in Table 7-8. Second, weights are

developed to reflect the user's perception of the importance

of each of the scales. (The sum of the weights assigned

* should equal 1.0). Third, an overall feasibility score is

determined using the following algorithm.

N

F = Z Wi S.

i=l

where F is the overall feasibility score, W is the weight

attached to the ith scale, S is the score on the ith scale,

and N on the number of scale items. A worksheet to support

the construction of the feasibility score is provided in

Table 7-9.

The feasibility ratings should be obtained from training

analyses who have at least three years experience in Army

training developments. Ratings should be averaged across

raters.

7.1.5 Construct Summary Evaluation Score

OVERVIEW

During this procedure, a summary evaluation score is

constructed which aggregates scores on the three most

critical training figures-of-merit: cost, effectiveness,

and training complexity. To accomplish this, scores for the

7-19

" .-. .> -.-.- . . - -- .. .- - " -. .-. " . ..- -- - "" . . . -; ' " ." - .-." - .--'
. Q.. 7-- - T wv7- - -

TAINING COST AND SCEUESTATE-OF-ART

PROGRAM RESOURCE COSCRIUTS IN TRAINING

ELEMENT(S) CONSTRAINTS CNTATSTECHNOLOGY

RATE TRAINING

PROGRAM ELEMENT(S)

ON FEASIBILITY

SCALES

FEASIBILITY

RATINGS

WEIGHT

FEASIBILITY

SCALES

SCALE

WEI GHTS

CALCULATE

OVERALL

FEASIBILITY

SCORE

OVERALL FEA SIBILITY

SCORJE

Figure 7-4 Overview Diagram Assess Feasibility of 7- 20

Training Program Elements
w -

-4 .-

Lj w

= -4 =

4-M

00

00

0.) tz ~j 0 > 4- 0 >

~~0 CL 4-j c- 4h$

0 C: - -

4-i C*'Ji 4. M ( *~- >4--

r *H - -1 4. !e-J W -4 t co w.0 z 0 4) M 0 j > J -

a0c - aM v - - .0-0 ci W j j 7

= - r 0 f.nx -

4. 10 0 0 )b C 0 C: ) ( 4.J r- - c: 4 5.H - 4- -Si 4-j~ 030 a - U u r- - -

(U a.-C -40 E .5. *) -4 U

4-j *r4 (1)-~. 4.D 4)o .,-4- 

-A -44- -4-. 0.A 0 T a4i Ca). 0 44 ;>. Z -

7-21O C~J0 4~ ~ jiig
0L

E

<0

LU

* LL

41

02

Ucc

CuD

I7-2
New system on these three figures-of-merit are divided by

the Predecessor score on the same figures-of-merit. This

converts the figures-of-merit to a common scale. Weights

are then assigned to each FOM and the weighted scores are

then summed to provide a summary evaluation score.

PROCEDURE

A worksheet which can be used to construct the summary

evaluation score is presented in Table 7-10. A separate

summary evaluation score can be constructed for each course

or training program element. To begin the procedure, the

user should enter information on the Predecessor and New

system cost, effectiveness, and training complexity

(training time) into the appropriate column in the

worksheet. Cost values are produced in Procedure 5.1.

Training effectiveness scores are produced in Procedure

6.1. Training time is calculated in Procedure 3.5. After

these values have been entered the Predecessor cost and

training complexity scores should be divided by the New

system scores on these two variables. (See Table 7-10).

The New system effectiveness score should be divided by the

Predecessor system effectiveness score.l Weights should be

developed for each of the three f igures-of-merit. The

weights should reflect the user's estimate of the individual

importance of these three figures of merit. The weights

should dum to 1.0 and should be entered into the appropriate

columns in Table 7-10. The user will probably want to use

the same weight for each course.

.The divisions are designed to produce a summary evaluation

score where high scores are "good" and low scores are "bad".

7-23

* * .- * *. ** * .,- * *.
c Xex

E.2

cm

ca

* w

*j cn z 

U,.

CC

0c

~___ _ ____ ___ ____ ___ ____ ___ ____ ___ ____ ___ _____-7
A weighted score fore ach FOM can be obtained by multiplying

each of the predecessor/New system ratios by the weight for

the FOM. A summary evaluation score is calculated by

summing the weighted scores for the three FOMs.

7.2 IDENTIFY LIKELY PROBLEM AREAS

OVERVIEW

During this procedure, training problem areas are

identified. A problem area is defined as an MOS with a high

score on the three figures-of-merit which provide the most

critical diagnostic information on training for New

systems. These figures-of-merit are total training cost,

effectiveness, and training complexity (training time).

In ETES, problem areas are identified in a two step process.

First, all areas where the figure-of-merit for the New

system exceed the figures-of-merit for the Predecessor

System (that is where the New system does not fall within

the footprint of the Predecessor) are considered to be

problem areas. These are the areas which should be given

primary considerations during tradeoff analyses and

identification of training alternatives.

Second, the areas within the New system which have the

highest scores on the figures of merit are identified. These

areas represent secondary problem areas which should be

O. examined after the primary areas (i.e., the areas where the

New system does not fall within the footprint of its

predecessor) have been examined.

7-25 -:S.[7
PROC EDURE

An overview of the steps in this procedure is outlined in

Figure 7-5. During the procedure, both primary and

secondary problem areas are identified. A primary problem

area is the occupational specialty (and associated

equipments) where the New system training figures-of--merit

exceed the Predecessor system figures-of-merit (i.e, where

the New system does not fit within the footprint of the

predecessor). Three figures-of-merit are used for this

Predecessor-New system comparison: (1) total cost, (2)

effectiveness, and (3) training complexity (training time).

Secondary problem areas are identified by rank ordering the

MOSs in the New system by their associated figure-of-merit

scores. This process identifies the five MOSs, with the

highest scores on each of the three figures-of-merit

described above.

This procedure begins by collecting the scores on the three

figures-of-merit (total cost, effectiveness, and complexity)

for all MOSs in the Predecessor and New system. A worksheet

for describing this data is presented in Table 7-11. Any

MOS where the figure-of-merit scores for New system is 15%

greater (or 15% less in the case of cost) than the Predecessor system may be considered to be a primary problem area.

Secondary problem areas are identified by rank/ordering the

MOSs within the New system by the three figures-of-merit The

MOSs with the five highest lowest in the case of effectiveness scores on each figure-of-merit are identified. Table 7-

12 displays a worksheet for conducting this analysis.

7-26
PRDEE-O

FIUEO NWSSE

MEI FGREO

FiurC7OErve DiagrT SCOiigPobe RES(72

LI-27O~

.. ~ ~ ~ . ~ .. N. . ~ . ~ . ............................................-.- B . .
IL..L __ ___ ___ ___ ___ ____ ___ ___ ___ ___ ____ ___ ___ ___ ___

Cl.

up)

I.-

LU

b..

U.

b.)

o

43 .; i- 4

aa.

aca

EL

2-2
LL

I-0

co0

0

0. 

z

Cu 00

w

0>

fi)i

U.

*0

0-

cc

N 

L

I7--
7.3 IDENTIFY LIKELY CAUSES OF PROBLEM AREAS

OVERVIEW

During Procedure 7.2 problem areas (i.e., MOSs) related to

three figures-of-merit (cost, effectiveness, and training

complexity) were identified. In order to identify

alternatives for ameliorating these problem aresa, the

likely causes of the high scores in these three figures of

merit must be determined.

Typically, the term "cause" of a training problem is used in

two distinct ways (see Figure 7-6).

First, it is used to refer to courses, equipments, and tasks

which are associated with the high figure-of-merit scores.

In this sense, task x or equipment y can be said to be the

cause of the high training figure-of-merit. For instance,

one MOS may have an exceedingly high training cost and this

high cost can be traced back to the course associated with

one particular equipment subsystem.

Second, the term "cause" of training figure-of-merit can be

used to refer to the underlying variable which leads to high

figure-of-merit scores. For example, the large number of

students to be trained can be identified as the "cause" of

high training costs.

The first type of casual analysis identifies "where" the

problem lies while the second types of casual analysis

provides information on the nature of the problem. The

*"- second type of casual analysis is more difficult since it

J 7-30
uj to

-100

020

cr..

LU W Cn

uju

ccu

7-31-
.7 7

requires the identification and examination of complex

interrelationships among variables.

The procedure described in this section deals with both

types of casual analysis.

PROCEDURE

An overview of the steps in this procedure is provided in

Figure 7-7. First, the courses within each problem MOS that

are the major contributors to the high figures-of- merit are

identified. For cost and training complexity (i.e., course

length) this is relatively straightforward since courses are

the basic units for aggregating these figures-of-merit. For

training effectiveness, which uses task as the basic unit of

analysis, this is accomplished by examining the tasks

associated with each course.

Following the identification of problem courses, problem

equipments (subsystems) are identified by examining the task

descriptions associated with each course. These same task

descriptions are used to identify the tasks within each

course which are the major contributors to high cost, low

effectiveness, or high training complexity.

Once the problem tasks have been determined, the casual

factors likely to produce the high figures-of-merit are

identified. This is accomplished by first examining the

casual model underlying each figure-of-merit. These casual

models are presented in Figures 7-8, 7-9 and 7-10. To

determine likely casual factors, the user should start with

the figure-of-merit and move backwards in the casual

structure (from right to left in Figures 7-8, 7-9 and 7-

10). Examining the values of the variables in the casual

structure and noting where "excessively high" values occur.

To identify an excessively high score, the user should

7--32
IDENTIFY

PROBLEM

COURSES

• PROBLEM

COURSES

IDENTIFY

PROBLEM

EQUIPMENTS

F PROBLEM

EQUIPMENTS

IDENTIFY

PROBLEM

TASKS

PROBLEM

TASKS

• .. ," "MODELS FigCAUSAL CAUSAL STRUCTURE

.0

LIKELY " - - . "C A U S A L

_:::::FACTORS

",:':Figure 7-7 Overview Diagram Identifying Likely Causes (7.3)

..- 7-33
* 2

0,

cu

00

Z -z z

LL

<~ co

01 0

PU m

> >0

7-34

......................................................
z -z

00

W0

0,

zC

z 0,

7-35
00

Z u

7-

4, <

7-36-
compare the value of the New system on that variable to the

value of the Predecessor (or BCS) system on that variable. A

difference of 15% or greater can be considered excessive.

Variables with excessively high difference scors which are

not preceded in the casual structure by other variables with

high difference scores can be considered potential causes of

the high figure-of-merit. Alternatives which may reduce the

values of these potential causes are identified in Procedure

7.4 A worksheet which can be used to document the

identification of causes is presented in Table 7-13.

7.4 IDENTIFY ALTERNATIVES

OVEnRVIEW

During Procedure 7.2 potential training problem areas were

identified. During Procedure 7.3 likely sources of these

problems were identified and prioritized. During this

Procedure, alternatives for the training-related sources are

identified. In addition, non-training related sources of

training problems (e.g., hardware/software design) are

highlighted and brought to the attention of the Program

Management Office.

PROCEDURE

An overview of the procedure for identifying training alternatives is presented in Figure 7-11. First the sources of

training problems are sorted into two categories: trainingrelated sources and non-training related sources.

Altecnatives are not identified for the training related

sources since these are beyond the purview of the training

analyst. However, the non-training related sources are

presented to the program office for review.

7-37

......................
00

cc4

w

cc

C-4-

.00

0

U

w (U

7-3
SOURCES

OF TRAINING

PROBLEMS

IDENTIFY

TRAINING AND

NON-TRAINING

SOURCES

NON- TRAINING

TRAINING SOURCES

SOURCES

PRESENT TO IDENTIFY

PROGRAM RELEVANT

OFFICE FOR TRAINING

REVIEW PROGRAM ELEMENT

RELEVANT

TRAINING

[-E-1PROGRAM

ELEMENT

ALTERATIVE IDENTIFY PRIORITIZED TRAINING

ALTERNATIVES ALTERNATIVES

POTENTIAL PRIORITIZE

ALTERNATIVES ALTERNATIVES

'. •ASSESS IT

,FEASIBILITY LTOF

ALTERNATIVES

* Figure 7-11 Overview Diagram: Identify Alternatives (7.4)

7-39
The training-related sources are examined further to

identify the relevant training program element associated

with the alternative. During the early phases of the

S- .acquisition process, nine major categories of training

alternatives may be examined (1) settings, (2) methods, (3)

media, (4) course sequencing (between courses), (5)

scheduling of modules within courses, (6) the amount of

time spent in training tasks within a course, (7) course

frequency, (8) course location, and (9) usage rates for

training resources.

The general alternatives for training settings, methods, and

media are well defined. These alternatives are presented in

Tables 7-14, 7-15, and 7-16. The user may determine which

" - method or media alternative is likely to reduce the training

problem by constructing training efficiency scores for each

alternative (see Procedure 6.1).

The alternatives for the other categories of training

program element are less well-defined. Users can obtain

guidance for selecting alternatives in these areas by

examining the procedures that were used to originally create

these training program elements. A list of the functions

associated with the score categories of training programs

elements is provided in Table 7-17. After potential

alternatives have been identified, the feasibility of these

alternatives must be assessed using the procedures listed

under Procedure 7.1.4. Subsequently, the alternatives must

O be prioritized, taking into account (a) their feasibility,

(b) the priority of problem source with which they are

associated (as determined in 7.2), and (c) the cost of

rerunning the appropriate ETES steps to assess the

alternatives in Procedure 7.5. The user must take care to

7-40
':9,

Table 7-14. Training Setting Alternatives

Institutional Training Settings Skill Level

. Basic Training (BT) 1

Advanced Individual Training (AIT)

One Station Unit Training (OSUT) 1

* Primary NCO Course (PNCOC) 2

Basic NCO Course (BNCOC) 3

., Advanced NCO Course (ANCOC) 4

Senior NCO Course (SNCOC) 5

Primary Technical Training Course (PTC) 2

Basic Technical Training Course (BTC) 3

US Sergeants Major Academy (USASMA) --

Support School (SPT SCH)

Primary Leadership Course (PLC) 2

Unit Training Settings

Supervised On-the-Job Training (SOJT)

Self-Study

Scheduled Training

Training Extension Courses (TEC)

Army Correspondence Course Program (ACCP)

S.. 4

.

. 7-4"

"-- -S-S-
"."

Table 7-15. Training Method Alternatives

AT Audio Tape

C Conference/Lecture

CAI Computer Assisted Instruction

CS Case Study

D Demonstration

DF Dual Flight Hours (only aviator courses)

(do not include in ICH computations)

El Hardware Performance Examination

E2 Nonhardware Performance Examination

A E3 Nonhardware Performance Examination

EL Elective (in-house only, except for CGSC)

* -:F Film

GS Guest Speaker

IS Independent Study

NCT Non-contact Instruction with an Instructor

Available in Classroom

NC2 Non-contact Instruction without an

Instructor Available

PEI Hardware Oriented (hands-on) Practical

Application

PE2 Nonhardware Oriented (non-classroom)

Practical Application

PE3 Classroom Practical Application

" " P Programmed Instruction (using programmed text)

PM Printed Materials

.1 QC Besseler Cue See

S Seminar

SF Solo Flight Hours (only aviator courses)

(do not include in ICH computations)

SI Simulation Instruction

SP Self-Paced Instruction

ST Slide Tape

TV Television

WCl Instructor Led Work Group

WC2 Student Led Work Group

Sources: DA Pam 570-558 Staffing Guide

for U.S. Army Service Schools

and TRADOC Cir 351-12 Format

for Programs of Instruction

-* 7-42
Table 7-16. Training Media Alternatives.

Media Category

Print Instructions w/o Feedback

Print Instructions with Oral Feedback

Print Instruction - with Written Feedback

Print - Reference Material

Image Projection - Still

Passive Audio

Active Audio

Audiovisual - Still with Feedback

Audiovisual - Still without Feedback

Audiovisual - Motion with Feedback

Audiovisual - Motion without Feedback

Static Display without Feedback

Dynamic Display without Feedback

Physiological Trainer - Internal

Physiological Trainer - Audio

[-- .Physiological Trainer - Visual

Symbolic Simulation with Feedback

Trainer

Simulator

Operational Equipment

Operational Equipment with Feedback

Visual Computer - Still with Feedback

Visual Computer Still without Feedback

Visual Computer - Motion with Feedback

Visual Computer -Motion without Feedback

Audiovisual Computer - Still with Feedback

Audiovisual Computer - Still without Feedback

,Audiovisual Computer - Motion with Feedback

- Audiovisual Computer - Motion without Feedback .,

Instructor

7-43

02.-Q:::
Table 7-17. Training Project Elements

Training Program Element Function

Settings 3.2

Methods 3.6

Media 3.6

Course Scheduling (Within course) 3.5

Training Time 3.6

Course Frequency 4.1

Course Location 4.1

Training Resource Usage Rates 4.1

N

0 -'

0-. - al -7 riigPoetEeet

/ 4

.42...t - -- C.a.'.. A i.
restrict the alternatives to a reasonable number and to

those areas where he/she can get the "biggest bang for the

buck" since the cost of evaluating alternatives can be

expensive. Cost savings can be achieved by combining

alternatives into one or two major alternative training

programs.

7.5 EVALUATE ALTERNATIVES

OVERVIEW

During this procedure the training alternatives selected for

analysis in the previous step are evaluated by reapplying

selected ETES procedures and sensitivity analysis are

conducted of key parameters.

PROCEDURE

Procedures for evaluating alternatives and conducting

sensitivity analyses are presented in the sections which

follow.

7.5.1 Evaluate Alternatives

An overview of the procedure for evaluating alternatives

presented in Figure 7-12. First, the Training Estimation

Aids and Procedures which must be reapplied to assess each

alternative must be identified. This is accomplished by

0 examining Table 7-17 which lists the ETES procedures which

must be reapplied to reflect changes or alternatives in nine

major categories of training program elements.

Once the ETES Procedures which must be reapplied have been

7-45

.,,. '
TRAI NING

ALTERNATIVES

IDENTIFY

TEAP PROCEDURES

TO BE APPLIED

T EAP

PROCEDURES

TO BE

APPLIED

SELECT

FIGURES OF

MERIT

FIGURES

OF MERIT

IDENTIFY

EVALUATIVE

TECHNOLOGY

PROCEDURES

TO BE

APPLIED

SUMMARY

EVALUATIVE EVALUATION TECHNOLOGY SCORE 'FOR

PROCEDURES EACH ALTEROBE APPL.IED NATIVE

REAPPLY SCORES OfETES ALTERNATIVES PROCEURESON FIGUREFS PROCEURESOF MERIT'

Figure 7-12. Overview Procedure for Evaluating I raining Alternatives.

7--46
identified, the Evaluative Technology Procedures which must

be reapplied are determined. To accomplish this, the

figures- of-merit that will be used to evaluate the

alternative must be identified.

Once all of the required ETES Procedures have been

identified, they must be reapplied. As the steps are

applied, the worksheets should be complete to document the

results of each reapplication of an ETES procedure.

7.5.2 Conduct Sensitivity Analyses

Sensitivity analyses can be conducted to determine how

changes in key variables impact the results of the training

requirements analysis. Sensitivity analyses should be

conducted for variables for which (a) there is concern about

the accuracy of the values assigned to the variables, or (b)

there are several strong competing alternatives which may

significantly change the value of the variable.

The procedure for conducting sensitivity analys is is

outlined in Figure 7-13.

First, the variables on which the sensitivity analyses will

be conducted are determined. Sensitivity analyses are

typically conducted for two types of variables. First,

sensitivity analyses are conducted for variables where there

is concern about the accuracy of the values assigned to the

variable. For instance, a questionable value has been

assigned to a cost element for a course and the user is

interested in seeing how changes in this acquisition cost

would impact total training cost. Second, sensitivity

analyses are conducted for variables for which there are a

47-4 7

o-
IDENTIFY

VARIABLES

TO BE

VARIED

[SENSITIVITY VARIABLES

IDENTIFY

VARIlAB LES

TO BE

IMPACTED

VARIABLES

To BE

IMPACTED

DETERMINE

FUNCTIONS

WHICH MUST

BE REAPPLIED

FUCTIONS

WHICH MUST7

BE REAPPLIED

* IDENTIFY

RANGE AND

INTERVAL

VALUES

RANGE AND

INTERVAL

VALUES

REAPPLY SENSITIV7Y

SELTED RESULTS

PROCE DUiRES

SFigure 7-13 Overview Diagram: Conduct Sensitivity Analysis
* number of strong alternatives which may change t -cva "ic -,I

the variables. For instance, if the user bo]ieves that the

number of students which must be trained could range from

2000 to 3000 and he/she is interested in determini ng how

training resources and costs will be impacted by these

variations, then a sensitivity analysis would be conducted

on this variable.

Once the sensitivity variables have been identified, the

user must identify the variables he/she will use to assess

the sensitivity impacts. This is accomplished by examining

Table 7-18 which describes the major variables associated

with each function in ETES. Additional information is

provided in the causal models described under Procedure

7.3. In general, the user may select any variable(s) that

are associated with a higher numbered function (that is any

variables which occur later in the causal structure

underlying training requirements), Most typically, the user

will examine the impact of changes in sensitivity v;rjahies

on the training figures-of-merit. All procedures listed at

or between the function associated with the sensitivity

variable must be reapplied.

After the functions to be reapplied have been determined,

the user must identify the range and interval values to be

used during the sensitivity analyses. The range of values

must encompass the likely alternatives for that variable.

Once the range has been determined, specific values at

" equally-spaced intervals within the range must be

identified. The intervals between the values should be as

large as possible while still capturing the likel

Sa It ernat ives.

7-49

SO

-C,.i-,?-"- - -..-- Ct : .,-,-. • .* -.. -. .- '.-. .- .- -: v ..- ;..,. ... x., r. .- .- . ,p~ , - a
Table 7-18. Listing of Candidate Parameters

for Sensitivity Analyses

Parameter ETES Procedure

Task Selection Criteria 3.1 Select Tasks for Training

Task Selection Criteria 3.2 Assign Tasks to Training Settings

Training Times Per Method 3.6 Construct QPOI

-" Training Times Per Module 3.6 Construct QPOI

Training Times Per Course 3.6 Construct QPOI

Criticality Weights 3.7 Assign Tasks to Media

Utility Weights 3.7 Assign Tasks to Media

Constraint Levels 3.7 Assign Tasks to Media

Student-Instructor Ratios 4.1 Develop Course Operating and

Support Plan

Media Usage Rate 4.2 Develop Course Operating and

Support Plan

Attrition Rates 4.2 Determine Number of Students

to be Trained

Upgrade Rates 4.2 Determine Number of Students

to be Trained

Manpower Requirements 4.2 Determine Number of Students

to be Trained

' Number of Students 4.3 Determine Instructor Requirements

Instructor Contact Hours 4.3 Determine Instructor Requirements

-, Per Method

Cour:,e Frequency 4.3 Determine Instructor Requirements

Student/Instructor Ratios 4.3 Determine Instructor Requirements

7-50

0 . Z N
.7

Table 7-18 (Continued)

Parameter ETES Procedure

Number of Students 4.4 Determine Facilities Requirements

Facilities Requirements per 4.4 Determine Facilities Requirements

Student

Number of Students 4.5 Determine Training Device and

Training Equipment Requirements

Instructor Contact Hours 4.5 Determine Training Device and

Training Equipment Requirements

Course Length 4.5 Determine Training Device and

Training Equipment Requirements

Media Usage Rate 4.5 Determine Training Device and

Training Equipment Requirements

Number of Students 4.6 Determine Requirements for Other

Training Resources

Number of Students 5.1 Estimate Individual Course Costs

Course Length 5.1 Estimate Individual Course Costs

Number of Instructors 5.1 Estimate Individual Course Costs

Course Frequency 5.1 Estimate Individual Course Costs

Reference/Unit Training Costs 5.1 Estimate Individual Course Costs

• "Criticality Weights 6.1 Estimate Efficiency

7-51

S7
Once the range and values have been determined, the selected

ETES functions must be reapplied. Table 7-19 presents a

worksheet which can be used to summarize the results of

sensitivity analyses.

7.6 EVALUATE IMPACT OF SYSTEM CHANGES

OVERVIEW

As the development of the system progresses there are likely

to be changes to non-MPT system elements such as the

hardware/software design and manpower requirements. During

this subfunction, the impacts of these changes on training

are determined.

PROCEDURE

There are two steps in this proced,.res. First, the ETES

procedures which must be reapplied in response to system

changes must be determined. This is accomplished by

examining Table 7-20 which lists the functions which must be

* .*"reapplied to reflect changes in non-MPT system elements such

as the hardware/software design or manpower requirements.

Second, the identified procedures must be reapplied. All

changes and results should be entered into the ETES audit

trail so that a complete record of the system development

process is maintained.

". . . 7-52 • -". .
- . r - r - r . -- ,-- .

0

Cl, CuoC

U,.

• <4.4"

*! C

.* Cu

• .'-

-. - U,

*O _ _ __ 7-5__ _ _ __ _ _ __ __3_ _ _

.,- . , ., ...., C, -*- , , --- -. . ., - .-, --. .-- . .. - .. -. . . .. . . . - .. . - . ,.

" I, % ". . 0, , . , , - , ,- , = ,7 ., ,- . ". ., ,o" . - . , . . . , - . . ,
Table 7-20. ETES Procedures Which Must Be Reapplied

In Response to System Changes

System Variable Changed ETES Functions Which Must

Be Reapplied

Design Concepts 1.6, 2.0 to 8.0

Weapon System Requirements 1.1 to 1.6, 2.0 to 8.0

Organizational & Operational

Concepts 1.1 to 1.6, 2.0 to 8.0

System Functions 1.1 to 1.6, 2.0 to 8.0

Mission Profile & Operational

Mode Summary 1.1 to 1.6, 2.0 to 8.0

Threat and Scenario 1.2 to 1.6, 2.0 to 8.0

System Acquisition Objectives

and Constraints 1.1 to 1.6, 2.0 to 8.0

System Acquisition Schedule 1.1 to 1.6, 3.0 to 8.0

MOS/Duty Position Assignments 2.3, 2.5, 3.0 to 8.0

Skill Level Assignments 2.3, 2.5, 3.0 to 8.0

Manpower Requirements 4.2 to 4.6, 5.0, 6.0, 7.0, 8.0

.. 754

i- -.--- . --. --.. A
SECTION 8.0 - DEVELOP INPUTS TO ACQUISITION PROCESS

OVERVIEW

During this procedure, a training development schedule is

constructed and ETES-related inputs to Army acquisition

processes and documents are specified.

8.1 DEVELOP/MONITOR TRAINING DEVELOPMENT SCHEDULE

Procedures for developing a training-development schedule

are provided in the Automated Planning and Scheduling

Technique (APST). An overview of APST is provided in

Appendix C. A more detailed description of APST is provided

in the User's Guide: Automated Planning and Scheduling

Technique.

APST can be used to assist training developers in describing

and monitoring the training development schedule for

developing Army weapon systems. APST is designed to be used

with the Visischedule software, which is an automated

program for describing, monitoring and reporting schedule

information and for conducting critical path analyses of

schedule events.

8.2 DEVELOP INPUTS TO ACQUISITION PROCESSES AND DOCUMENTS

e OVERVIEW

This procedure describes how the products of the other ETES

aids/procedures can be used to support Army acquisition

documents and processes.

* 8-1
L

Table 8-1 lists the documents and processes which have the

most relevance to early training estimation.1

The two documents/processes which are most critical to early

training estimation were (1) the Outline Individual and

Collective Training Plan (OICTP) and (2) Cost and Training

Effectiveness Analysis (CTEA). More details on the

relationships between ETES procedures and the Army acquisition documents processes are provided in the sections

8.2.11. An overview of the relationship between ETES and

the Army Life Cycle System Management Model (LCSMM) is

provided in Section 8.2.12.

8.2.1 Develop Inputs to OICTP/ICTP

The OICTP and ICTP are plans which support the development

and implementation of new or revised individual and

collective training programs at institutional and unit

levels. The OICTP and ICTP are the major resources and

planning documents for developing training for new Army

systems. An approved OICTP is sufficient justification to

enter manpower and funding requirements into the Army's

programming and budgeting processes for inclusion of the

TRADOC Review of Manpower (TRM). As specified in existing

regulations TRADOC Reg. 351-9), the OICTP/ICTP contains a

proposed training concept and training strategy. The

concept and strategy are typically generated by the

proponent school responsible for the New system's training

requirements.

I Overviews of the Army acquisition process are provided in DA

Pam 11-25 and in Wagner (1982).

8-2

... .- ...,,-,.. . . . . .-... ,,.. .... .. .. .. ,. - .. ...... ..... . .. ., ., .. -< .;: .,,-.-.-,
4-)

-. 4 41 Q 4

41a -- U -. 0 U) 41 C V)

cO nC -U) W mfL 'aa ) > - 4J (1J) - -2>,W 42 0 a) U)( Ca

0.- a) > C- -. (n ) qZEeu : 1 w0 3 a W u .) : c

E . 0 0 W .C , U- a'-' C fl WU) rn 0 mU aC0 a)'- C Qu es- 4Js-- 4Ea) 0 n - JC C-4 4. CL

Cf>1-- UC 41 ( 0 4-~- :f0 co LO oa) M ) 4 .*.-4 Cfu) fu 4..C 4 1 .S-JL.

0 41 ) n CLow ~ >1a LU)D 0 O . C a

0 OW a-a -4 u 41a, U cn 4-1 (540 oa

z- W aC--4 W.,- -fC rU .- 4 ~ wu 0

E-0 -SS.w.5w 4 ()V C 5 U ) . 4 a( .14

4 -4 wa~L U) E ) 0 Cz 'a cn 4'-1

E-4 nC 0 ~~ GS) 4~ J 41 :r4c 'O c'

434 *-tr' a, 4 O'.-S .'-' 1 u ()

E- L) 41 ) QU)Z I--5 w W) 4- W) W-S 5-' W - ' U f4 O j~.4.j 

ca u- 5- . 0 a) 0 M~ . E-, E. Cra a 14 w ' U <(nl- aWO '-0 W1 Wa0 w) r- =1 aC W a)'-w a) a) Ea' 41' C) z >U E Z0' .4. E4 a 4.9. W) C-s ' s- ) (nt-- 'as to- C)W >a C *) 5 z z 0> 0 4'-''C L . ~ E- (1 ) *-s ,)- W 0a) -

z < 01-4 a"S) (14a C-S a)~U 0 Cn W) .) .

z 4J Mr 1 C)r* 0aU >S. a 61 4.) w''~ 4 ~U) Aj

IxS W.' M (V ) 4-' C W'- *- ' 4.)U- a4.0'1 -4 4 ) C a Ca .- E-s-- ~ C ) -4 -C C 46

fml s- w 4J 'C -"- 'a t6 a -4 a4 m 0 ma)' a)U(na)O0 >4E- 'aU)4 0 -S 0 O-. w L64 I w '->U Z 4.) E > - 4O a'c E- 0OE-4 4-.0 Ws&4

>4U) 41 -4s 4 U ca CU2 (n. E 0 w C) c u -4(Ca ( *- -. 4O 0 ~ C C))' CW).-C Ea a) C.-4 'C roEr >. a ~ 0s w-~- E.s>' I- z

4 s J r'a * s-s. -. %654a- m ..I .~ 'a C) -- E' (a u rO

0 41 4 s)-cC 1 *.5 W a)) .C 4 a)wC

Cfl) -w C C W > 4)'J- )-U (o) a) 4..-g4'0

0 as~. 1) w a) cCC :J- C: 'a 0 )

u 4J w w a. - m- )-SW E-s -J ) . cr-s a>

U). a. ty 0 M E-'- V r'>-4 (4s) 14.4 = (n iCa()

. E) 4s-1 a 0 *a U4 ~- 10 0 4- C04 U a.CC -) M Z Cn U: K-C CC- 4dC'- C: 0- ca- 'Es zJ''4 

C) *-*-S -. (as 0 a) Wa)'(a CC c .UaC 0-4 a. 'a to 4 5 'a~O ~ CC c ~)O0 -a) U) 4j

E. E-Sa'' Ac 4 -4' a 0) 0 0E - A..aC7,sW-' . E4 'a

U)4 U CE-4 C4E-H5(0.-4 W) 4. s$4 0- 4JO.5 E.S 0 X to 4

* )U - E-s >C) (n (W () 41 00' coC -4

cr, 1- u- U>1 W W (1) )> rz.= r E -4 - - -1 0) -4 U Ea) O 41' w ~ a)A - J( J4 -

a. *r-)' a) w aa) 0 ()~ 3 U a) Q) >- Oa U) E a CJ )W

U C (.a r- MCL. w w -0 cO. .Ca) .C..,4 C> >1 (a

s- El E- 2: E. WU)'a(a lo.) w*-' z-~- E- E-4U0 E& cn m 54

0

z

cn- < 0 z

Zo -)E-4 U) 0). U) Z 4 . 4 E-44) 4-J

CD CI -C) ()

En) C)rs r

C1- 0 -4) C) C)

C *d _4)4 E- Ai0

U) 41 a 0 x <E- z U cn W) 0 U)sS .- ) <)

C) 'C r-5 c) W-5 0 1

-4-- 0U a.r C)144 a) 'a 4oIU E--') . 0 E 0

'a " C) 'n >' 0 '-'C a'

r - , s " l 4 -4 C ' - 9)-4a a C 1

'a 4 SU 4r CN WW -- o

E) ZP-- M')' 0 .0 'a 6.~ 

M) ~ 0C .- U 41 C W 0C -

* - 4 %) (n' Lf5

'- AO - .~ -() m)

- I >O. >-- E-I o0-

*~4 U )) S S a ;0 C .). ) Ca 

WC C - a 0 I- I C E 41. 

Cvr_-4 cal-" C J .& 5I 0 anyc 4-S 1S- 0

*-SCI C >U' U)0 C)t 'a0 U -'-'--4 C) ~- - -C (Ur 4-55- 0)

r - l a' r ) an (U " (a (o L. Ctj s ' a)

0 E- 04 '3 &E- U

8-3
V) j a2 .. 4.-2w C

w~~ ~ ~ 0>,aW .1

0 U 0 >~r > u .4 .J

4J 2 1444 owE- ..- 0 -

-. CE. 0 0 w~ W 4J:3 W

.1 E- 0 &0 ~ wc u>

.) w En U)C~- (0 4JE" U)E 4J - 4 - 4

4) 4J-4 >

S z 0 ma

U 0.J)U Fa4~ ra tJo<

z0 ) -4 -4 0 r )

>4 ~ ~ ~ Ai4 n . o- (n (a -4L C-J 44O O4) u4- a

4 U--4 3-4 ) C 0 wM- ;

WZ* U) 0 P, 4-j o cu Z 0.4 U 41 U (U 4

c 4J 0) 4xJ V~U C:).

0~ M C ~~4- a U) Q) -0r M>

Cfl (0 ( ) ul 0)CfV U) rc) 0 a 4

0) 04a0 0 ) - ~> CO 4 --4c F - 0 -4 > 4) 0 V) 0) C 4. i

*E- E 41 () E w W .~>O- .

AC 0( ) ea . a) a. U) O0 w am4W

c - E-.4 61 4 - 4 10 0 &4 CE- a 4

*0 ~ :3 0 14 n : 1-4 04 z -4 ~ (a -4

u U) 0)O4-4 -4 C4 tr :3U) Q) m (1 )V a m. 1-4 6 z Q) aC) . E- Q) 0 ) a - C -4 *0) :

-4

-4

-- EC 0E

>1 4 r

VI - CU 0 En ) (aU) 0

4J141r- Ur) 4.j AiJ-4 Q) CJ-4 -4

0 Q)0.U) U) 04-4 1

m 4 0 C00) DJ2C 0 di w r-. 4C)4-4 : ( - 6..I.j a 0. - 0 )- 0 -41 U

N. O 0 to U u n

0 '-4 4 -

< U4 P..- -C -4 .

-4 a 0 M 0)w~ U) -4 Q)-4 -4 U

'JC a - ) 0 u0

(o C U) ra = al 3tC r- c >4 0

41- 00C. >1c u(o 0 ) 0 4

w. 4oW0 01 0E0U > E c (D Q " w0w0. a. Q) -

(oU C)(vm0. 4OE 

>0 0' aC 0

U) 0-4.J E 0 (a OE -4- U 

Z J -4 0 ., : aJ~ -0(aC4 :3A)a t )- wJC. U(a 0o-4-40-4ru)C oe-.u Q t

~'4 U0 - 0 0) ~ 0 J0' Ewix -

* 8-4
The OICTP/ICTP is an evolving document that increases in

specificity (via appropriate updates) as the system under

development is further defined. The OICTP/ICTP provides

significant feeder data for the Tentative Qualitative and

Quantitative Personnel Requirements Information (TQQPRI),

the Training Effectiveness Analysis (TEA), and the New

Equipment Training Plan (NETP) . After LOA approval, the

OICTP can be used by the contractor to facilitate the

identification of training requirements conducted as part of

the logistic support analysis.

The OICTP/ICTP is intended to guide the development of

training subsystem requirements and to provide the general

framework for their incorporation into the existing training

base. The regulations specifically emphasize the intention

to use the proposed training concept in the OICTP/1CTP to

"...identify the constraints which training requirements and

resources may impose on the design of the materiel system"

(TRADOC Reg 351-9, pg. 5). The regulation also mandates use

of the OICTP/ICTP as the vehicle to describe "... the

integration of the training subsystem into the development

of the total system and the integration of the developing

system into ongoing training systems."

TRADOC Reg 351-9 further stipulates that the OICTP/ICTP must

incorporate the principles of Army Training 1990 (AT 90)

into training for a new system, for both institutional and

unit training and at all skill levels for the MOSs affected.

The OICTP/ICTP is intended to develop and describe a

systematic and feasible strategy for training, ranging from

the development of "initial qualification" training to the

"sustainment of the proficiencies" needed for the successful

fielding and operational deployment of the system being

acquired.

8-5
As specified in existing regulations, the OICTP/ICTP

provides information on the training required to integrate

replacements from the training base into the unit, and to

qualify personnel for higher level tasks as they advance in

grade. The OICTP/ICTP is further directed to provide

information on the identification, quantification, and need

" * for training devices, simulator, documentation/publications,

- -'itraining aids, support facilities, instructors, costs, and

all other support and logistic considerations necessary for

the implementation and test of the proposed training plan.

Table 8-2 provides an overview of the elements in the

OICTP/ICTP and describes which ETES procedures can provide

input to the development of these elements.

The TRADOC Training Effectiveness Analysis Handbook produced

by TRANSANA specifies the elements which must be included in

an institutional training concept that is part of the OICTP.

Most information elements which are required in the

institutional training concept can be produced by applying

ETES procedures.

Table 8-3 lists the information elements of the

institutional concept and describe which ETES procedure can

be used to produce these elements. A more detailed description of information elements in the training concept is

- provided in Appendix H. This description is taken directly

from the TRADOC Training Effectiveness Analysis Handbook.

S0,-

8.2.2 Develop Inputs to Cost and Training Effectiveness

Analysis (CTEA)

The CTEA process is potentially the most critical LCSMM

process related to early training estimation since it

O '8-6
0o

Iat

1 ' -*

o ' I C- C-C.

4~1 C 01 

In 0 I

8- 7
I - - I I I * I . -~----------~-..--.~-- -' -'. -' -. -.

0

I I I T7

gILl

o~ ). * - C -

.1

O.O~ 0) 1.1 0) -' .-. ,-

_

w 0)) {.,1

.2

o 2 ~ - i __________

I .~_____ I ______

I'

- .-.-. ~~~1~~~*

*

I I'

S ~

*

- I
Table 8-3. Relationship Between Institution::l Trainig

Concept Elements and ETES Fr:cedures

TRAINING CONCEPT ETES

ELEMENT' PR0 CE rXJRE

1.0 Courses Impacted 3.6

1.1 Modification or New Course 3.6

1.2 Changes to Current Courses 3.6

2.0 Student Load per Class per Year 4,2

2.1 Average Grade of Student

2.2 Student Source 3.4

3.0 Class Frequency Per Year 4.1

3.1 Length of Class 3.6

3.2 Fiscal Year of Course Start 4.1

4.0 Instructor Reqs. Per Course 4.3

5.0 Support Personnel Requirements 4.2

6.0 Expended Equipment Per Class 4.6 (partiall,)

7.0 Non-Expended Equipment Per Class 4.5

8.0 Exportable Training Used* Not Covered in

Current ETES

8.1 Exportable Software* Not Covered in

Current ETES

8.2 Training Terms 3,6

8.3 Exportable Hardware* Not Covered in

Current ETES

9.0 Facility Requirements 4.4

• These elements refer to institutional training elements

sent to units in the field.

As specified in TRADOC Training_ Effectiveness Anilvsis W'indbook.

6:,:a 8-9-9-
requires the user to not only estimate what the training

program will look like (as does the OICTP/ICTP), but also to

evaluate these training programs and to provide trainingrelated input into the overall system development and

evalua tion process.

The new TRADOC Reg 351-9, governing the OICTP process,

contains the most current definition of the CTEA. It

defines CTEA as " a methodology which involves documented

* investigation of the comparative effectiveness and costs of

alternative training systems for attaining defined

performance objectives." The definition further specifies

that a CTEA can focus on any one or a combination of the

following:

" - o Training concepts

0 Training strategies

o Training equipment/devices

o Programs of instruction

o Training impacts of - New materiel - Organization

- Tactics

- Employment techniques

"- Families of systems

Regardless of the specific focus of the CTEA, TRADOC

Requlation 351-9 stipulates that the CTEA should include an

. analysis of the attainable levels of proficiency and the

costs associated with each alternative. In addition, a CTEA

should include a cost effectiveness trade-off analysis of

th( feasible alternatives. This regulation further

specifies that a CTEA must:

m'." .i
T0

0 Insure that training development is initiated

early in the life cycle of hardware systems and is

accomplished in coordination with combat

developments,

o Optimize soldier-hardware subsystem interface,

o Insure that all feasible training subsystem

alternatives are considered,

o Optimize soldier-training subsystem interface,

o Recommend the preferred training alternative based

on cost and training effectiveness, and

o Provide decision makers with more precise

information at critical points in the acquisition

process concerning the total system (comprising

the training, hardware, and other subsystems).

These objectives demonstrate that the CTEA, unlike the OICTP

and the QQPRI/BOIP, is an early training-related document

intended, in theory at least, to influence the hardware

system design. This is a very significant di fference from

the OICTP, which is, by regulation, primarily an MPT

planning and resource document.

The steps in the CTEA process and the corresponding ETES

procedures which can provide input to this process are

listed in Table 8-4.

S

8-11
10

I'I

6z c

<a Z"-

- C. C, 

C:C

(U I4j 0, c

C',Q

- - 0,

* . o~EI..L

* C C,

C - CU V I7
I~ : r 

u u

c I

I C o c I 0 0 Cr *C 0" Su a

0 IF
8.2.3 Develop Inputs to Logistic Support Analysis

As defined in AR 700-127, Logistics Support Analysis (LSA)

is the use of analytical tools and models to (1) develop and

evaluate alternative support concepts, (2) project manpower

and personnel impacts, (3) perform tradeoff- between system

design and ILS elements and tradeoffs among ILS elements,

integrated support planning and design, and (5) measure life

cycle impact of materiel and support system alternatives.

The outputs of LSA are used to (a) influence materiel design

or selection, (b) develop the required logistic support

system, (c) provide the manpower and logistics analysis for

the intergrated program summary, and (d) record the LSA

during the materiel acquisition phase to compare it with

operational use.

Training is one of the Integrated Logistics Support (ILS)

elements. Hence, all of the analyses requirements required

by the LSA apply to training.

One of the key components of LSA is the Logistics Support

Analysis Record (LSAR). The LSAR sheets provide a formal

mechanism for describing the support elements of a

developing system. As such, LSAR sheets, like the ETES SDT,

* can be reviewed as a tool for describing developing support

concepts. Table 8-5 describes the LSAR elements which are

included in the SDT and the ETES procedures in which these

elements are generated.

8.2.4 Develop Input to Operational Testing

As defined in DA Pam 11-25, operational testing is the

. testing and evaluation of materiel systems which is

8-14

* * .*
0-

C,)L

aO 43 0 N) 

00 cm, 1. 2-

LU - m I

0 E,

0: E Z E- 40

4- F- m Cm 

.2L LU Z (h 0 . crU

ca0 0 * ~ .L a
accomplished with typical user operators, crews, or units in

as realistic an operational environment as possible to

provide data to estimate the military utility, operational

" effectiveness, and operational suitability (including

compatability, interoperabiltiy, safety, reliability,

availability, and maintainability, supportability,

operational man (soldier)-machine interface and training

requirements) of new systems. Training is, and should be,

one of the key elements considered during OT. The Army

Research Institute has developed a procedure called HRTES

(Human Resource Test and Evaluation System) which is

* specifically designed to provide human factors and trainingrelated input to OT testing.

An overview of the functions performed in HRTES is presented

in Figure 8-1.

Table 8-6 displays the relationships between ETES functions

and HRTES functions. The HRTES system index assists

analysts in identifying comparable existing systems which

may be used to identify system mission func.tions. Systems

mission functions are then identified and documented in a

mission chart. The term "mission" in HRTES is equivalent to

the concept of function described in ETES Function 1.0.

Once the missions (i.e., functions) have been identified,

system performance measures for these functions are

identified. HRTES also contains procedures for evaluating

the results of operational testing.

The HRTES procedures for identifying OT training issues

(Human Performance Functions, Human Performance Measures,

Test Conditions, Human Resource Issues and Measures) can be

used to identify OT test requirements within the total ETES

.16

• 0 8-16

**. .**'-% .*-*** . *. .* *
SYSTEM

INDEX

* MISSION

CHARTS

SYSTEM

PERFORMANCE

ISSUES

HUMAN-PERFORMANCE

FUNCTIONS

KMANPERORMANCE

* -* MEASURES

TEST _

CONDITIONS

HUMAN RESOURCES

ISSUES AND

MEASURES

OPERATIONAL

TEST

EVALUATION OF

RESULTS

DIAGNOSIS OF

PERFORMANCE

INADEQUACIES

Figure 8-1. Basic HRTES Flow.

8-17
Table 8-6. Relationships Between HRTES Function

and ETES Function.

HRTES Function ETES Function

System Index 1.4 Establish Baseline Comparison

System

- Mission Charts 1.1 Identify Required System

Functions

System Performance Issues 1.2 Identify System Performance

Measures and Goals

Human Performance Functions 2.4 Develop New System Task List

Human Performance Measures

Test Conditions

Human Resources Issues and

Measures

Operational Test

Evaluation of OT Results

Diagnosis of Performance

Inadequacies

8-18

. .-. . . . . . . .. - ' ' i " " " ?, . . I : " !' ' , ' "P b,' % "k ,7
framework. Direct input for these procedures is provided by

the system functions (missions in HRTES terminology) and

performance measures identified in Function 1.0. An overview

of HRTES procedures for identifying OT test items and issues

is provided in Appendix 1.

8.2.5 Develop Inputs to Training Device Requirements,

Documents and Processes

A Training Device Requirement (TDR) is a document which

states concisely the minimum operational, technical,

logistical, and cost information necessary for development

of a training device. A TDR is required if the funding

thresholds exceed $2 million in RDTE funds and/or $3 million

for any fiscal year or more than S15 million for the 5 year

program in procurement funds.

Training device requirements which are not covered by TDRs

are specified in the Training Device Letter Requirement

(TDLR).

The Training Device Letter of Agreement is a jointly

prepared and authenticated document in which the

ccm bat/training developer (TRADOC) and the materiel

developer outline the basic agreements for further

investigation of a potential training device system.

During ETES Procedure 3.7, general requirements for training

devices are identified. More specifically, the tasks which

are most likely to require training via devices are

identified. This information provides the essential input

data needed to initiate a Training Development Study

(TDS). A TDS is a cost and training effectiveness analysis

. . . .- .
conducted to support the development of training devices.

(See TRADOC Reg 351-9 for a description of TDS procedures).

In addition, the resource requirements estimates (e.g.,

" number of students to be trained) made during Procedure 4.0,

provide input to the estimation of cost and resource

requirements for the training device. The TRAINVICE

methodology described in Procedure 6. 1 and in Appendix G

provides a mechanism for calculating the training

"efficiency" of an emerging training device concept.

8.2.6 Develop Inputs to New Equipment Training

New Equipment Training (NET) provides for the initial

transfer of knowledge from contractor and/or materiel

developer to tester/user that is needed for operator

maintenance and logistic support during testing and initial

introduction of new materiel into the Army.

In the past, training developers have often produced initial

resident training courses by modifying the NET courses, many

of which were developed in an unsystematic fashion. Using

the ETES procedures, a different concept of course

-evelopment can be employed. More specifically, using ETES

procedures, an early description of the system training

program (that is, the training program that will be used

when the system is fielded) can be developed first. Task

differences and target population differences between the

NET courses and the system training program can then be

identified u-ing the procedures described in Procedures 2.0

and 3.4. NET courses can be generated using Procedure 3.6

(Construct Quasi-Program of Instruction). Resource and cost

estimates can then be generated usin the Procedures 4.0

(Estimate Training Resources) and 5.0 (Estimate Training

8-20
Costs). This strategy has an obvious advantage in that it

starts with the end objective (system training program) and

then identifies the intermediate products (NET courses)

needed to meet those objectives.

8.2.7 Develop Inputs to System Requirements Documents

The Justification for Major System New Start (JMSNS) is used

to identify and support the need for a new or an improved

mission capability. The JMSNS is required only for

justifying the initation of a new major acquisition ($75

million or more RDTE or $300 million or more in acquisition

costs). If an ETES analyses is conducted during mission

area analysis, it will be possible to develop an initial

training concept which may be included in the JMSNS.

The LOA is a requirement document jointly prepared by the

combat developer and materiel developer which outlines the

agreements for further investigation of a potential materiel

system during the demonstration and validation phase.

Table 8-7 summirizes the relationships between ETES products

and the LOA.

8.2.8 Develop Inputs to Request for Proposal (RFP)

Development/Proposal Evaluation

There are four ways in which ETES related products can be

used to assist in the development and/or evaluation of the

proposals which are generated for system development.

First, the results of ETES can be used to specify the

minimally acceptable criteria which the training program

which is developed by the contractor must meet. This

8-21
aw

00

75 M.

C

E _

o L ) 0C

Ac E

C) C

cg -

00

*i W CL a)~) a

ev "a) 04' Cua 0

cu > > a)

4) 0 0

cc. Ca(a) CU 0 0 -J0 Z, 0~A CLU "aL wi CL 0

'A + C C

gn U 0 :3 0 00

W..LL) C7) C

C.) L) m C-' 4.,' >

8-2
 . .  .   .  . . 4 . .. % . ,,. .; 

criteria can be included in the Reques: For ProposaL )FP;

which is generated for the system.

Second, the ETES Evaluative Technology )Procedure 7. 2) can

be used to identify likely training problem areas which must

* be addressed in detail by the contractor. Third, LhKm

Baseline Comparison System (BCS) and New System training

- programs generated by ETES provide a standard with which Lo

evaluate the training programs generated by contractors. For

.* example, if the length of a contractor generated course is

significantly less than the length of an FTES generated

course, the training developer may want to examine the

A contractor course in more detail to determine if the

contractor is underestimating training requirements to make

their desijn concept look less training intensive.

Fourth, if the user is going to apply ES or some portion

of ETES and wants the contractor to supply the input data

needed to feed the ETES procedures, ETES input data requirements must be included in the RFP. Correspondingly, if the

*user wants the contractor to apply ETES procedures, ETES

output requirements (at the procedural step level) must be

included in the RFP. It the latter approach is taken, the

useL must insure that ETES input data from existing Army

data bases is included in the government furnished

information (GF'I) specified in the RFP.

8.2.9 Develop Inputs to Personnel Documents/Processes

The Qualitative and Quantitative Personnel Requircment--

I.nformation (QQPRI) describes the personnel skills reuir-*< "

to operate and support a specific materiel systeff; tHl.

reconmmended placement within the curr nrt, revi.<,

S. ".
Army Military Occupational Specialties (MOS), including a

* listing of duties and tasks; new or revised training

requirements, and projected annual manpower requirements.

* . Table 8-8 displays the relationship between the elements of

the QOPRI and ETES procedures.

According to TRADOC Reg 600-4, Integrated Personnel Support

(IPS) is the process by which personnel considerations are

integrated into the development effort for a materiel

s stem. Accomplishment of the

personnel consideration be evaluated in terms of its impact

on the total personnel support requirements and system

design. A goal of this evaluation is a quantitative/

qualitative statement of personnel requirements which will

insure the development of the personnel support necessary to

achieve the desired operational capability of the materiel.

The goals of the IPS roughly parallel the goals of the Cost

and Training Effectiveness Analysis (CTEA) which is

conducted to assess training. However, the IPS has a strong

emphasis on personnel planning and in this sense has

objectives similar to the Outline Individual and Collective

Training Plan (OICTP). Table 8-9 lists the key IPS products

which are generated during the first two phases of the

acquisition process and the relationships between these

products and the ETES procedures.

.* 8.2.10 Develop Inputs to System Level Documents and

' cProcesses

The Concept Formulation Package (CFP) documents the studies

conducted during the exploration of alternative systems

concepts conducted during the first phase of the

acquisition. The CFP consists of four elements:

8-24

W• V . rcse A. A. -

-**. °

Th v. Co cep For ula ion Packge CFP) doc mens th stdie
- -. . ~- -*. U~*V U

9,U

CE a E

LUL

0 .2

o 0'-

0

C CD

0. E E

CU 0. 0

4.C CA )

E2 M M

2 L o vov d iU L. 0U L

LUz

0 0

0 0

* )- 0 ~ D 0 0 C'J.

CL -

E .2 c

CD 0 09 mn

6) C C

0 8-25
c* .0

- - -8 - -E-

.2C 1Cb

0.0E

X0 C

0.L U 

m >

CU m

.2 0.

4.' 'a 7E' 0

a 80 - 0

all (A.. LU 4. L

:2 .M n 0 0 N 4.. 0 C-4 Yb z -ic -- l C Ybi IA - C

.2 0 LU).~ ~ .U 0

CLC CL L . .2 .2 . .2

u LU ig .LU 

IA o C C

a.0.0. ~ 0 0 00 0.0 . 0.

LU~ ~~ .SS4 * ~ 0. 

C C C a

.2c .2 .2 .2 . 0 0 LU r3 44 EC Cm

0 2 0 . C c

0.2 0. 0 .L 2 . 0 .2.

8-26

op *ILU LU L
(1) Cost and Operational Effectiveness Analysis (COEA)

provides essential information on , the cost and

effectiveness of materiel system alternatives to

permit evaluation and decision on the courses of

action open for acquisition. The CTEA is one of

the key components of the COEA.

(2) Tradeoff Determinations (TOD) describe the

apparent technical feasibility of a potential

system, including technical risks associated with

each approach, estimated RDTE, and procurement

" '- costs.

(3) Tradeoff Analyses (TOA) determine what technical

approach offered in the TOD is best.

(4) Best Technical Approach consolidates the results

of the TOD and TOA.

The CFP is included in the Decision Coordinating Paper (DCP)

which is a management summary document to support the Army

System Acquisition Review Council (ASARC) and the Defense

System Acquisition Review Council (DSARC). ASARC/DSARC

reviews are major management reviews conducted prior to

entry into successive phases of the materiel acquisition

process. DSARCs are for major systems requiring Secretary

of Defense approval. ASARCs are for major systems requiring

Secretary of the Army approval.

ETES does not provide direct input to the high level

acquisition documents/processes described above. However,

ETES does provide input to the CTEA (see Section 8.2.1)

which is in turn included in the CFP and higher level

documents.

8-27
-2.. ,-" . . . . ..

8.2.11 Develop Inputs to Mission Area Analysis/Soldier

Analysis

Mission Area Analysis (MAA) is the continuing analyses of

mission areas which is conducted to identify those mission

elements for which existing or projected capability is

S..deficient and to identify opportunities to enhance the

"* capability to achieve and sustain combat operations through

• ., more ef. -tive and less costly methods and systems. As part

of MAA, w system concepts may be identified. ETES provides

a mechanism for evaluating the training requirements of

these early concepts.

Soldier Analysis provides the initial mechanism for

consideration of soldier factors in the MAA. It requires

MAA proponents to identify, prioritize, and evaluate soldier

factors early in the development of a solution to a

deficiency and provides a timely and quantified basis for

further studies to evaluate the effects of soldier factors

upon solution effectiveness. Table 8-10 lists the products

which are generated by Soldier Analysis and the ETES

procedures which can provide input to the generation of

these products.

8.2.12 Relationship Between ETES and LCSMM

Figure 8-2 describes the LCSMM events which are directly

- related to ETES. ETES inputs to the LCSMM events are

described in Sections 8.2.1 to 8.2.1.1.

8-28

V~ ~~~~~! . I,- *-.. . . ~ 4.*\..... .-. **' * *****
7-

*0

0.

CLu

0e

0

b. Lu 4-4 'AM - M UL 5a. Cc,~U, ~ 4

0 PO

0. 00.

< .4-c

'c; a 00 In0 0ol L

0 LI) zI C4C;ciA- CA '-coU U~, L ~ L U C

(A L C-L 

dCl

w~~ 2 L u Lb.~. 

0 a u -1U SaU 0

MA X U. -Y CA << 0 cc U

*~~c 4' 4-4 CA4 

I- 0 ~ ~ )L~ C80-09

LU Z Z ccJCd N ~
. . .. . & . .c

Z4

C1

£M

aL

4 -0

.A.-

00 w w

8-30,
REFERENCES

ARMY REGULATION TITLE

1-1 Planning, Programming, and Budgeting

within the Department of the Army (MAY

76)

10-4 Organization and Functions, United States

Training and Doctrine Command

10-38 United States Army Concepts Analysis

Agency (CAA)

11-18 The Cost Analysis Program (DEC 75)

15-14 Systems Acquisition Review Council

Procedures (MAY 78)

70-1 Army Research, Development and

Acquisition (FEB 77)

70-8 Personnel Performance and Training

Program (PPTP) (OCT 78)

70-10 Test and Evaluation During Development

and Acquisition of Materiel (JAN 76)

70-15 Product Improvement of Materiel (AUG 80)

70-27 Outline Development Plan/Development

Plan/ Army Program Memorandum/Defense

Program Memorandum/Decision Coordinating

-* Paper (MAR 75)

71-1 Army Combat Developments (SEP 68)

71-2 Basis of Issue Plan (JUN 76)

-S

71-3 User Testing

71-5 Introduction of New or Modified Systems/

Equipment

71-6 Type Classification/Reclassification of

Army Materiel

R-1

-. A2 -!~ e. %2 - *-.** S** 4" - -
REFERENCES (continued)

ARMY REGULATION TITLE

71-7 Military Training Aids and Army Training

Aids (OCT 73)

71-9 Materiel Objectives and Requirements (APR

75)

71-11 Total Army Analysis (MAY 80)

310-1 Publications, Blank Forms, and Printing

Management

310-3 Preparation, Coordination, and Approval

of Department of the Army Publications

310-34 Equipment Authorization Policies and

Criteria, and Common Tables of Allowances

310-49 The Army Authorization Documents Systems

(TAADS)

350-1 Army Training (JUL 78)

350-35 New Equipment Training and Introduction

(NOV 81)

351-1 Individual Military Education and

Training (MAR 81)

351-183 Service School Training Reports Control

Symbol (JULY 76)

570-2 Organization and Equipment Authorization

Tables - Personnel (SEPT 78)

600-200 Enlisted Personnel Management System (FEB

81)

602-1 Human Factors Engineering Program (JUN

76)

611-1 Military Occupational Classification

Structure Development and Implementation

(JUN 76)

R-2

4I..
REFERENCES (continued)

ARMY REGULATION TITLE

611-201 Enlisted Career Management Fields and

Military Occupational Specialties (JUNE

82)

700-5 Total Logistics Readiness/Sustainability

Analysis (TLR/S) (MAY 78)

700-35 Product Improvement of Materiel

700-78 Production and Post-Production Testing of

Army Materiel

700-90 Army Industrial Preparedness Program

700-127 Integrated Logistic Support (ILS) (APR

81)

702-3 Army Materiel Reliability, Availability

and Maintainability (RAM) (JAN 77)

715-6 Proposal Evaluation and Source Selection

(SEP 70)

725-1 Special Authorization and Procedures for

Issues, Sales, and Loans

750-1 Army Material Maintenance Concepts and

Policies (JUN 72)

750-4 The Army Materiel Plan-Part II Depot

Materiel Maintenance and Support

Activities

1000-1 Basic Policies for Systems Acquisition

(JUN 81)

4.

TRADOC REGULATION TITLE

11-1 Manpower Analysis and Force Structuring

in Combat Development Forces (OCT 78) %"

11-5 Cost Analysis Program (MOS Training

Costs) (NOV 77)

11-7 Operational Concepts and Army Doctrine

(DEC 80)

R-3
REFERENCES (continued)

TRADOC REGULATION TITLE

11-8 Combat Development Studies (FEB 81)

71-4 TRADOC Standard Scen L,3 for Combat

Developments (MAR 77

71-5 Scenario Oriented RE rring Evaluation

System (SCORES) (MAI

. 71-10 Integration of the TOE development

Process and the Scenario Oriented

Recurring Evaluation System (SEP 77)

71-12 Total System Management - TRADOC System

Manager (SEP 78)

310-2 Development, Preparation and Management

of Army Training and Evaluation Program

(ARTEP) (DEC 79)

350-2 Development, Implementation and

Evaluation of Individual Training (FEB

79)

350-4 The TRADOC Training Effectiveness

Analysis (TEA) System (June 79)

350-7 A Systems Approach to Training

351-4 Job and Task Analysis (MAR 79)

351-9 Individual and Collective Training Plan

for Developing Systems: Policy and

Procedures

. -600-4 Integrated Personnel Support (IPS) (JUN

78)

700-1 Integrated Logistic Support (ILS) (JUL

77)

4R-4S . ". . . . ' , ' t ' ' : '. - ' ' .- .. " ' " . '" .'" ._ i " , , . '' , , ', . "' / / , ,
REFERENCES (continued)

TRADOC HANDBOOK TITLE

Mission Area Analysis (DEC 79) (Draft)

Training Device Requirements Documents

Guide (MARCH 79)

Soldier Analysis (JULY 81)

TRADOC Training Effectiveness Analysis

Handbook

TRADOC PAMPHLET TITLE

11-8 Cost and Operational Effectiveness

Analysis Handbook

71-8 Analyzing Training Effectiveness (FEB 76)

310-8 Collective Front-End Analysis (CFEA) for

Development of Army Training and

Evaluation Program (ARTEP) (Draft)

350-30 Interservice Procedures for Instructional

• .Systems Development (ISD) (AUG 75)

351-4 Job and Task Analysis Handbook

* 70-1 Training Device Development (FEB 79)

(Expired FEB 80)

350-2 Officer Job/Task Analysis and Training

Development (MAR 79) (Expired MAR 80)

350-3 Individual/Collective Training and

Development Glossary (Dec 79) (Expired

DEC 80)

351-1 Common Job and Task Management (JAN 80)

(Expired JAN 81)

" 351-3 Training Requirements Analysis System

(TRAS)/Individual Training Plan (ITP)

(DEC 79) (Expired Dec 80)

6 351-7 Job Training Program (JTP) (APR 80)

351-8 Individual and Collective Training Plan

. for Developing Systems - Policy and

Procedures (MAY 80)

4i R-5 . . . . . . . . . . .

[" . " .. . . . . . . . . . . . . . . . . . . . . . . .
. . .

REFERENCES (continued)

TRADOC CIRCULAR TITLE

351-12 Format for Programs of Instruction (POI)

(Expired APR 81)

351-28 Soldier's Manuals, Commanders Manuals and

Job Books: Policy and Procedures

(Expired Dec 79)

DARCOM/TRADOC PAMPHLETS TITLE

PAM 70-2 DARCOM/TRADOC Materiel Acquisition

Handbook (JAN 80)

DARCOM CIRCULAR TITLE

700-4 Logistics-TailorinQ Procedures

Guide (FEB 80)

DA PAMPHLETS TITLE

11-25 Life Cycle System Management Model

for Army Systems (May, 1975)

570-558 Staffing Guide for US Army Service

Schools (DEC 79)

* ;* MILITARY STANDARDS TITLE

MIL-STD XYZ Task Analysis (JAN 80) (Proposed)

MIL-STD 1388A Weapon System ard Equipment Support

Analysis (Nov 81) (Proposed)

DEPARTMENT OF

DEFENSE DIRECTIVES TITLE

DODD 5000.1 (Draft) Major System Acquisitions (DEC 81)

DODD 5000.1 Major System Acquisitions (MAR 82)

DODD 5000.2 Major System Acquisition Procedures

(MAR 80)

DODD 5000.39 Acquisition and Management of

Integrated Logistic Support for

Systems and Equipment (JAN 80)

R-6 ..$.
REFRENCES (continued)

DA USASC & FG PAMPHLETS TITLE

US ARMY SIGNAL CENTER Skill Performance Aids (SPA)

FT. GORDON Management Plan (FEB 79)

UNDERSECRETARY OF TITLE

DEFENSE MEMO

USD (R&E) Memo Major Defense System Acquisition

Program Documentation Format (APR 82)

DEPARTEMENT OF DEFENSE TITLE

SYSTEMS MANAGEMENT DOD Acquistion Improvement Program

COLLEGE (OCT 81)

-. R-7
REFERENCES (continued)

TECHNICAL REPORTS

Bonder, Seth, A Review of Army Force Modernization and

Associated Manpower, Personnel and Training Processes

(Technical Report 81-6). Alexandria, VA: U.S. Army

Research Institute for the Behavioral and Social Sciences,

January, 1981.

Boylston, D. An Automated Decision Aid For the Assignment of

"* Tasks to Training Media In Early Training Estimation. Paper

presented at the 51st Symposium of the Military Operations

Research Society, September 27-29, 1983.

Braby, J., Parrish, H.; and Swope, W. A Technique For

Choosing Cost-Effective Delivery Systems, Navy Training and

Analysis Group Report No. 16, April 1975.

Dawdy, E.; Chapman, W.; and Frederickson, E. REMBASS Final

Report. El Paso, Texas: Applied Science Associates, June

1981.

- Fink, C. D. and Carswell, W. A., Integrated Personnel and

Training Information for TRADOC Systems Managers (TSM):

. Technological Gaps (ARI Research Report 1238). Alexandria,

. VA: U.S. Army Research Institute for the Behavioral and

Social Sciences, February 1980.

Fink, C. D. and Carswell, W. A., Handbook for Action

Officers and Training Developers for New Materiel Systems.

Alexandria, VA: U.S. Army Research Institute for the

Behavioral and Social Sciences, June, 1981.

Frederickson, E., Hawley, J., Whitmore, P., & Wood, M. Air

Defense Training Developer Decision Aid: Model Extension

Research Requirements. Alexandria, VA: U.S. Army Research

Institute for the Behavioral and Social Sciences, November

1980.

Gagne, R., Foster, H., and Crowley, M. The Measurement of

Transfer of Training, Psychological Bulletin, 1948, 43, 97-

130.

GAO Report, Effectiveness of U.S. Forces Can Be Increased

Through Improved Weapon System Design. Washington, D.C.:

Comptroller General of the United States, January, 1981.

Handbook for Human Resources Test and Evaluation Systems

Vols 1 and 2. (In Press)

R- 8

S..- . v.......-...... .... .. ,....-. -...-. -
w00

REFERENCES (continued)

Hanson, Vernon, L., and Purifoy, George, R., TSM Guide to

Training Development and Acquisition for Major Systems, (ARI

TR-78-A7). Alexandria, VA: U.S. Army Research Institute

for the Behavioral and Social Sciences, March 1978. (TSM

Guide)

Hawley, J. Evaluation of a Training Developer's Decision Aid

(TDDA) For Optimizing Performance-based Training In Machine

MOS. Alexandria, VA: U.S. Army Research Institute for the

Behavioral and Social Sciences, November 1979;

Jorgensen, C. A Methodolgoy and Analysis For Cost-Effective

Training in the AN/TSQ-73 Missile Minder (Research

Memorandum 77-26). Alexandria, VA: U.S. Army Research

Institute for the Behavioral and Social Sciences, February

1977.

Jorgensen, C., Hoffer, P. Prediction of Training Programs

For Use in Cost Training Effectiveness Analysis (Technical

Report 472). Alexandria, VA: U.S. Army Research Institute

for the Behavioral and Social Sciences, January 1978.

Jorgensen, C., Kubala, S., Davis, J., and Atlans, M. How to

Use the Training Effectiveness Estimation Model (Working

Paper 01-81). Ft. Bliss, TX: U.S. Army Research Institute

for the Behavioral and Social Sciences, January 1981.

Jorgensen C., and O'Brien, L. The Early Training Estimation

System: An Automated Training Needs Assessment Technique.

(Paper to be published in the first issue of the Training

Technology Journal).

Kane, J. J., Personnel and Training Subsystem Integration in

an Armor System, (ARI Research Report 1303). Alexandria,

VA: U.S. Army Research Institute for the Behavioral and

Social Sciences, January 1981.

Kamal, A., Eid, Aly, S., & Mahmoud, M. Multistage

Clustering: An Efficient Technique in Socioeconomic Field

Experiments. IEEE Transactions on Systems, Man, and

Cybernetics, December 1981, 11.

Kerwin, W.T. , and Blanchard, G.S., Man/Machine Interface - A Growing Crisis. Army Material Systems Analysis ActivityArmy Top Problem Areas Discussion Paper Number 2, Aberdeen,

MD, August, 1980.

R-9

",%

a

'", . .> 4 - "" '" .- " ". - -" " '. .""" " " " '"-' "",- - ' ' , , - ". . .. .- .. , "".
REFERENCES (continued)

Kinton Incorporated Handbook for Action Officers and

Training Developers for New Material Systems. Alexandria,

VA: Kinton Inc., June 1981.

Lenzycki, H. P. and Finley, D. L., How to Determine Training

Device Requirements and Characteristics: A Handbook for

Training Developers. Alexandria, VA: U.S. Army Research

Institute for the Behavioral and Social Sciences, May 1980.

Mannle, T., & Balcom, J. Estimating the Manpower, Personnel

and Training Requirements of the Army's Corps Support Weapon

System Using the HARDMAN Methodology_. Wilmington, MA:

Dynamics Research Corporation, September 1982.

Matlick, R. K., Berger, D. C., Knerr, M., and Chiorini, J.

*[ R. Cost and Training Effectiveness in the Army Life Cycle

System Management Model (Volume I and II). Alexandria,

VA: U.S. Army Research Institute for the Behavioral and

Social Sciences, May, 1980.

Matlick, R., Berger, D., & Rosen, M. Cost and Training

Effectiveness Analysis Performance Guide (Research Product

81-1). Alexandria, VA: US Army Research Institute for the

Behavioral and Social sciences, July 1980.

Northrup Services, Inc., Study to Improve the Development of

U.S. Army Maintenance Manpower Authorization Criteria

(MACRIT) Data: Office of the Army Deputy Chief of Staff for

Logistics, Washington, D.C., December 1981.

O'Brien, L. Early Training Estimation System (ETES)

Proceedings of the TRADOC Developments Institute Chiefs of

Analysis Seminar, October 21-12, 1981. 1 %:

O'Brien L. Automated System Description Technology. Paper

presented at the Second Annual Conference on Microcomputers

in Education, Washington, June 1982.

Office of Management and the Budget Circular A-109. Major

*System Acquisition, April 1976.

Pieper, W., Elliot, T., & Hawley, J. Training Developers

Decision Aid For Optimizing Performance Based Training in

Machine Ascendant MOS (ARI Research Report 1223).

Alexandria, VA: U.S. Army Research Institute for the

Behavioral and Social Sciences, November 1978.

".?

R-10

:-[NR
REFERENCES (continued)

Pieper, W., Elliot, T., & Hawley, J. Tryout of a Training

Developer's Decision Aid For Optimizing Performance-based

Training In Machine Ascendant MOS. Alexandria, VA: U.S.

Army Research Institute for the Behavioral and Social

Sciences, September 1979.

Price, H. E., Fiorello, M., Lowry, J. C., Smith, M.G., and

Kidd, J. S., The Contribution of Human Factors in Military

System Development: Methodological Considerations

(Technical Report 476). Alexandria, VA: U.S. Army Research

Institute for the Behavioral and Social Sciences, July,

1980.

Rhode, A. S., Friedman, F. L., O'Connor, F. E., Integration

of Manpower, Personnel and Training Issues from the Material

System Acquisition Process into the Planning Programming and

Budgeting System, (ARI TR 526). Alexandria, VA: U.S. Army

Research Institute for the Behavioral aand Social Sciences,

March, 1981.

Rhode, A., S., Skinner, B. B., Mullin, J. L., Friedman, F.

L.,. and Franco, M. M., Manpower, Personnel and Training

Requirements for Materiel System Acquisition. Alexandria,

VA: U.S. Army Research Institute for the Behavioral and

Social Sciences, February 1980. (ARI MPT Study)

Schulz, R., and Farrell J., Job Aids: Descriptive Authoring

Flowcharts for Phase IV Analyze of the Instructional Systems

Development Model, Research Product 80-13. Alexandria,

VA; U.S. Army Research Institute for Behavioral and Social

Sciences, May, 1980.

Skinner, B. B., and Friedman, F. L., A Study to Identify and

Consolidate Manpower, Personnel and Training Requirements

for Materiel Systems at ASARC Milestones. Alexandria, VA:

U.S. Army Research Institute for the Behavioral and Social

Sciences, August, 1981.

Skinner, B. B., Friedman, F. L., Mueller, E. H., A Study to

Validate A Sample Set of Questions and the General Approach

in Their Development for an Army Systems Acquisition Review

Council (ASARC) III System, (Draft ARI Technical Rport).

Alexandria, VA: U.S. Army Research Institute for the

Behavioral and Social Sciences, June 1981.

Weddle, P., & Fulkerson, G. Forecasting the Human Resource

Costs of Navy Weapon Systems. Defense Management Journal,

1980, 16, 6-12.

R-11
REFERENCES (continued)

Wheaton, G., Fingerman, P. Rose, A., and Leonard, R.

Evaluation of the Effectiveness of Trainin Devices:

Eluboration and Application of the Preditive Model.

Research Memorandum 76-16. Alexandra, VA: U.S. Army

Research Institute for the Behavioral and Social Sciences,

July 1976.

VisiCalc: User's Guide for the Apple III, Visi Corp, 1982.

VisiSchedule: User's Guide for the Applie III, Visi Corp,

1982.

.1

q.

.4.

_'4
Appendix A: Overview of Media Selection Program

This appendix provides a brief overview of the Media

Selection Program. A more detailed description is provided

in the User's Guide: Media Selection Program. The Media

Selection Program is an automated tool for (1) assigning

tasks to media and (2) calculating the efficiency and

effectiveness of various task-media combinations. The Media

Selection Program allows users to assign tasks to media in a

manner that maximizes overall efficiency, maximizes overall

"effectiveness," or minimizes overall cost.' In addition,

it allows users to assign tasks to media in a manner that

optimizes various combinations of these variables, including

an overall "utility" measure which combi..es either cost and

efficiency or cost and effectiveness.

Efficiency, in the Media Selection Program, is determined by

comparing the stimulus, response, a nd feedback

characteristic of the tasks to the stimulus, response, and

feedback characteristics of potential media. More

specifically, a score is calculated which describes the

match between media and tasks on these characteristics.

Efficiency for each task-media combination is calculated by

dividing this score by the maximum match that may be

1_ The program uses a relative cost and not actual cost to

measure the potential cost requirements of media. In

addition, the program does not use a measure of

"effectiveness" that fits the most common usage of that

terms. Rather effectiveness is actually efficiency weighted

by task criticality.

p.'-A-1
achieved for the task. Total efficiency for a set of tasks

is the sum of the efficiency score for individual tasks.2

"Effectiveness" is cal.culated by weighting the efficiency of

each task by a task criticality score. The task criticality

* score is a user-defined weighted combination of the eight

task factors typically used in selecting tasks for training.

These eight factors are task frequency, percent members

* performing, percent time performing, task delay tolerance,

consequences of inadequate perf ormance, task learning

difficulty, probability of deficient performance, and time

between entry and performance.

A matrix of relative cost values is stored in the program

*for each major media category. In addition, a built-in set

of algorithms is used to produce the utility measure which

combines cost with either effectiveness or efficiency.

PROCEDURAL OVERVIEW

The Media Selection Program is an interactive menu-driven

system. This means that users do not have to know or use a

computer language to run the program. Instead, they can run

through the program by selecting options from a series of

menus.

|o" 2

-.-

- ac2 his onteptaof. meda efficiency woa se oigially

-ideveloed bym Jorghensicenc(178 andr was incorprateds into

•- . .

| ..° . . .. . . . . . . . . . . . . . . . . . . .
:.

An overview of the procedures for using the Media Selection

Program is provided in Figure A-I. To begin the procedures,

users must rate each task on (1) the psychological variables

to be used to assess the match between tasks and media, and

(2) the variables to be used to assess task criticality (the

latter is only necessary if effectiveness is being

calculated). These scores must be then entered into the

- ETES data base management system, the System Description

Technology. When this is completed, users must enter the

Applications Program mode in the System Description

Technology, and select the Media Selection Program. Once in

the Media Selection Program, users must then select criteria

to be used in making media assignments. Seven options for

selecting criteria are provided: (1) efficiency, (2)

"effectiveness" (3) relative cost, (4) cost and efficiency,

(5) cost and "effectiveness," (6) a utility measure,

combining efficiency and cost, and (7) a utility measure

combining "effectiveness" and cost.

Once the criteria have been selected, users must select the

tasks to be included in the analysis (only tasks already in

the SDT may be selected). Typically, the tasks for single

course module will be selected for each analysis. With the

analysis criteria identified, the psychological variables to

be used in calculating the match between tasks and media

must be selected. Users may select from 12 variables

assessing stimuls characteristics, six variables assessing

response characteristics and four variables assessing

feedback characteristics.3

3 These psychological variables were taken directly from

the TEEM Model (Jorgeson, Kubula, and Atlas; 1981)

A-3
INPUT DATA

ON PSYCHOLOGICAL

VARIABLES iSOTI

.'I .*%SELECT

CRITERIA

SELECT

TASKS

N..

SELECT

PSYCHOLOGICAL

SELECTIWEIGI4T

TASK

CRITICALITY

FACTORS

I- - TE SCORES OF

DISPL~AY MEDIA ON

SCORES PSYCHO- LOGICAL

j VARIABLES

-e, WEIGHT

CRITERIA

SELECT

OBJECTIVESI

CONSTRAINTS

ASSIGN DISPLAY RERUN

. .RELATIVE TASKS TO RESULTS AMA LY.IS

COST OF MEDIA

CATEGORIE L

9' Figure A-i Overview of Media Selection Program

4. A-4
If the user has selected a set of criteria involving

effectiveness (effectiveness, cost and effectiveness, or

utility with effectiveness), the weights for the component

criticality variables must be entered so that a composite

task criticality score can be computed. Eight variables may

be used in the calculation of task criticality: (task

frequency, percent member performing, percent time

performing, task delay tolerance, consequences of inadequate

performance, task learning difficulty, probability of

deficient performance and time between entry and

performance).

At this point, the program has all of the information needed

to calculate efficiency and/or effectiveness. It will use

this information to calculate the relevant measure and it

will display the results.

If the user has selected one of the two utility measures

(combining either cost and efficiency or cost and

effectiveness) the user will then be required to enter the

weights to be used in the computation of utility. Following

this, the user must select the objectives and constraints to

be used in assigning tasks to media. A listing of the

possible combinations of objectives and constraints is

..* displayed in Table A-1. Once this information has been put

into the computer, the program will optimally assign the

tasks to training media. For instance, if the user selected

"maximum effectiveness" as an objective and "minimize cost"

as a constraint, the program would determine the assignment

of tasks to media which gives the highest overall score on

effectiveness and still remains under a user-specified level

of overall cost. Once the initial assignments have been

examined, the user can examine the effects of several

.A-

..
'09

so:)pu

~so:)

co4 0(

>. 0. OIt 0 C 0. C' 0 O0= 0 0 0' 0~ o 6 4- ~ -. 4.- W 1J. 4-j 0' Ad4.

C..'U > (U >1 c A) w- b 4.) 41) -- 1 W.'. .4. 4.4 U. 4.4' f 4

.. (v'. (v0 4.j (0 w aU. mU to (U C wU ro :

.2j Z U2 01 Z2 z~ .0 =C -4 = 41' a)W aa, - ' 4 W 4.4 1 4 a) 41' 41 4j 41) Ad4 4-. 4-4

-44 ) mU 4-' 4 ~ 41C 4'

44' JC w v0 L U. ene * wn eU 44 0)e en w n r- en en6C

n u-e n n ( el en en V)Q en t

W) v n - ' a) a a- .- 0 U' ) 0w-a'0.

en-&1 a~n w a -6) ( w>1 a) W)0 w 4 a a) vA U) ' 1 M..en A. a Aj U) (a 41.' U .4. 41 -4 'U -''e 41 c.'n A

61 'Uw a 4U. w 0) 'UC> w) , wU 'U w) U> wU w > U ew

a)0- >' W . c' c' QW'vC W > w' -4 a)2 >. I Wa)c 4J~ a)i -OW C .4-4 Q Cr44 .. Cn( vA ) C .- 0.) Cr4

(0- a)O 0-4-4 wn -4C WM' L a-4 % ) W a)- cC- 0)-- U W f

n a)44 O"j U) 4-4 '4.)~ (-' %W U2 4J f44 U.1- 4-4

M-4. LW 4 W*-en -- U2 :3 . UW*4 4 1W 4.4 .- 41

4- 41- '4 4 0 41 -4WQ ~ -- -4 41.-' w.a %&. W) en

44 0- M 0U C) ..4 fa' mU' to' ea' 'U-4 (U 14.4 m O

C *4 0 .6J .4 -- x -- 4 -W 4 Aj'. *-4 41 - LW4- . - 4 0I>-

'0- ' a) 0 a '000 a 000 00 0 0 a0 0 '0

2 2 WU 04) ( 2 W41 4 a) V41 c-2 ) 4 a) -4 -42

C 24 .6 v0 E - = 2-1 E. -E E z E4 2 m - It E '-4 2eUM

o 0' > V0 0 * 0 '41 rjW~ 41 W 41 0)

en -. '-4 Aj ri0 .4. L4. W) 41 -4 (a-U )O 4-) (0 0 a) 4J 4 J QW .4J .J ) .'444.W

an' a (0~V n 2 -4 - 0 2 - -1 -4 '4.4

he M 4' )be wV d'mU -1 () -1 a)W -~',d44-4. a) .C e ~C 10 xC W

Wa .2 e -4 nNW a) en V) eNa) n Na W e)Ne0 (n mn tVU'o a) Mn'm 44 a a 4.4 (0 .U- -4 'a'0 k.4 41 ( 'U-f-I4 M - - m. U- 'U - (a' - (a %.1

414-- 4.4 %W. .4J 0)0 444' JE w 42J- E4 -4V' % 4 A.. .4 J- 0

enV 44 -O~ N a . -- .. - - a)(V)4 a n W UW -4 -,1 4 W) -4x0U x0 -4 x> W -4N -4 N U0 -4

C 'Uv E0 C) m-20 m-U vU 1-'' a)- Cu aW -.4- 0. ) 14 a ON m- '--n '2. U-4 > 'l E . U EU 0j MU- E' M ECL M 0 C MU 'U > xl en x (v en en .. 4 -e -4e U -- 4v

( n %-W U.. w u-a e M -4 en 164U 1 V 4. V) a-e Ui en U-'e 1-> 4- x X -

en 02 0-m44. 02 en 0-. 0--- 0-Ha) en 0 ru w- 0 (0 w 0 m en Va (a *) W '= EU 2W W) 20

41 (n~ :3 A.'enen 41 co W .e 4.4 e .1 Lenen 4-' C/'4 414 en (. n 4J

cj n o -. .Cen - ."- enn (a aen (en- n Men a U en Cen- 0)~~( W) 0 0 '44 -- U Wa) Wa) WWv U -4 a)Q W )-'.

* 2>-. U 2U-- U - C' QC' 2 U 2 ' a2 z 

fn CO Cr_ m~ 0a COO (v Ca >0 CW- )U C: C.) % 0 0 C0 a

t.1 41. n (n 0C = w A 0'C en4. -a W'> 44e > LO>. 4JW ene A 0J>. -4 UWU

'U ~ U U V)' W--U4 (n 0U 'U' *1 V) U-4 -44U M~ I U 4) m 4 -4 (n-44- 4

Z 1 m44. In- env .w''. (a4 enme mn4 t W- a )U 4.-.a0 C4 m e-- en- (a-en

U ~ OU 4 W 44 44 a) U L4 V-4~ 3 4.4~ M %W= J- U m 414 4J 4140v

(al aU. c44 C".0a' C- 4e w U C(aO %". "~0 Ca- Ct0 : Cc0 a3Caa W0 Cs4J ~0 0 w0 0W n %'.4~W 0 W 0 )wUm .J 9,: a ) Z: 0) ~

41 en -6) 4.4 e C enC e --4- a) .-- 4 4 a)

E- 'U C (a >.'U > C:

.4 > >.4- C> 4.J C en -- n U) Q4e -

C A4-- -4C ' 4.)

U'0 0 ' 0 U 'A 0 0 U U ~ ) 0 - 44 -- 4-. 0 '-4) 41

z U) e k4 .4-1 en en o 0 '.0 0 1w- 0 0 ka-4 14.4 0

U a)U

*' U4 a

N 4-

'.- 'U E

0 W kW C:

M ,I M-:
SSaUaA43aj3

4soo pue

4soz)x

Su sua-lT~zJ3

U) , 4J Wi 41 :

0. 0 0~ *U 2 41O 41i (v

O r- 000 > 41i 4-1 0.

*~ ' C1 4 (

m u = U (.ro U 

41 t4. 4C) U) 4

41 41. w

Cf ) 4. (13 4J4 01 41. &

aC Oc"-' VN OU ) tr V) 4i414> 4) r -4 2

u>U c 0C) > c,- > c C) "- ' >, 4 

to ) -4 m -4 I. 0 J3 -4 -4 U ~4J U aO U~C wOe 4-' ) U 

1 4 a)C4- C),41-

E )- 44 a) 4-4 a) 2:

a%' 4i 4. Ad - 44.4 4-i %&A

E 0~ -4 0 40

U) d 4JO -4 (12C) 4 ' .60) -

-6) C 4J' a) .6 a)

a4 MO 4O r-4

C -4 4C 4 C 4 >4 C - fo 44~ r'oa

V) w V '- '-4 E* t" a) tW 4 t44) 4644V 4. 4

4 (V 4

4 0 U) N 0 ON-4 0 N --

EiC 4-12 41 E (v

C12 CC 4 CL C~O -44

Wz CC In Q) c' (n 0) a E

4-4 4- V) -4~ 3 cn-4~ m C-4 U) (12 (12

u- ta(nU) U

C tC m

C 4-

-. 0 u u -4 -

c/2 U Q)- ) z44 LW. A4-L

0 041i0

U~ C d C 0C

___ _ E_ 

A-7
'0

. alternatives including changes in (1) objective, (2)

constraint, (3) criteria, (4) task criticality variables,

(5) psychological variables, (6) task criticality variable

weights, and/or (7) utility weights.

After users have explored these alternatives, the task-media

assignments can be finalized and entered into the SDT.

0

°F.

A-8

*.7 *.e- A 

*.. .- I 
-'0

Appendix B: Overview of Automated Resource and Cost

Estimation Technique

."" '2

This appendix provides detailed instructions on how to use

the Automated Resource and Cost Estimation Technique

(RCET). A more detailed description of RCET is contained in

the User's Guide: Resource and Cost Estimation Technique.

The purpose of RCET is to provide Army Training analysts

with an automated tool for estimating instructor

--requirements and institutional training costs during the

earliest phases of the acquisition process.

The Automated Resource and Cost Estimation Technique (RCET)

is designed to use input data from the ETES data base

management system, the System Description Technology (SDT).

Actual calculation of instructors and institutional training

course cost in RCET is accomplished by using the VISICALC

automated worksheet software developed by Visicorp. The

VISICALC worksheet is also used to conduct sensitivity

analyses of key parameters.

B.1 CONCEPTUAL OVERVIEW

The Resource and Cost Estimation Technique has three

components:

(1) SDT Interface Software - this software is used to select

and remove data from the SDT and to format the data for use

in the VISICALC program. In addition, it is used to copy

the results of the VISICALC program back into the SDT.

B-1

1,
'0:

(2) Tailored VISICALC Worksheet - this worksheet contains

the equations for determining number of instructors and

course costs. In addition, it contains all of the commands

needed to load and unload the SDT input file, and to conduct

sensitivity analyses. This tailored worksheet saves the

user from the somewhat tedious process of setting up a

VISICALC worksheet and command structure.

(3) Manual Procedures - these procedures describe how to

develop input data and how to use the SDT interface software

and the tailored VISICALC worksheet.

There are two major products of RCET: (1) a listing of the

number of instructors required in the course and (2) a

listing of projected costs for the course. An example of

the cost elements estimated by RCET is listed in Table B1 . These are the same cost elements user in the Cost

Analysis Program of the Army TRADOC Resource Management

(ATRM) system.

B.1.1 Calculation of Course Costs'

Costs for a new course are estimated by identifying a

comparable existing course, obtaining cost data from this

course data to reflect the differences in key resource

requirements (for example, number of students and number of

instructors) between the comparable course and the new

course.

This procedure provides estimates of course costs that are

(1) empirically based and (2' suitable for the types of high

level analyses which are conduc.ted during the early phases

of the acquisition process.

B- 2
N. 1.4

I =1

I cc u I 4zI w0 :

I x CL- I

I u

5~. 4z 0

IC)0aJ4 )L x L I

u I LA -j Lz0L I L1 4 CL 0I.crW 4I = :3 W . I0- w I C cc LI I-u LILI 0 Im 0 - InI- o n 1 Z Z i 1-4" 40 .L 1 M IJL - M Lz . i J - 0aJ I C) W _j = -oc Z = i a a IL =- C3 C In .0 )0 w I x1 -

CCX: I. - O 3 I C3ax

* IB-3
"77 7-"2~

. - u

B.l.2 Calculation of Number of Instructors

The number of instructors required in a course is calculated

by an automated version of the algorithm listed in the

Staffing Guide for U.S. Army Services Schools (DA PAM 570-

538).

B.2 PROCEDURAL OVERVIEW

An overview of the procedures in the Resource and Cost

Estimation Technique is provided in Figure B-I.

The first step in the application of RCET is the

identification of the "reference" course or the comparable

* - existing cousre, which most closely resembles the task and

target population requirements of the new course.

Procedures for identifying a reference course are contained

in the ETES User Guide Manual Procedures. Once the reference

course has been identified, cost data for this course is

obtained from the Cost Analysis Program (MOS Training Cost)

and entered into the SDT.

Reference course information is also used in the

construction of the quasi-program of instruction (QPOI) for

the new course. Included in the QPOI is a description of

the methods to be used in each module in the course, and the

student-instructor ratio and instructor contact hours

associated with each method. Procedures for constructing a

QPOI are contained in the ETES User's Guide. This same

guide contains procedures for determining the number of

students to be trained. This value is a critical factor in

the determination of course costs.

B-4 1 .. . .
1.0 3.05.0

D0ETERM IN E

IDENIFY10 CNSTUCTREFERENCE AND

RFRNEQUASI-.PROGRAM NEW COURSE

% OREOF INSTRUCTION RESOURCE

FACTORS

COWOLSECE AND ENE

VISICALC LINPUT FILES

WOKHE ANSO ROEDR

V- ISCAL POCEUR

CNPUT COFIESLTERA

4.E

CostUREstmto ehi~ RE)

B-TA
Once the QPOI has been instructed for the new course,

information from the QPOI on instructional methods, student/

instructor ratios, and contact hours must be entered into

the SDT. When this is completed the the user milst enter the

- SDT, enter the Applications mode, select the Resource and

Cost Estimation Techniques (RCET), and copy the reference

- cost data and new course QPOI data onto files which can be

read into the VTSICALC program.

Once the VISICALC input files have been developed, the user

must remove the SDT software diskette, put in the VISICALC

software diskettes and enter the VISICALC program. Once into

the VISICALC software, a few simple crmands may be used to

load the SDT input files and RCET worksheet into the

VISICALC. When this is completed, a small selected set of

resource data for the reference and new course must be

entered into the SDT. Course costs and instructor

*! requirements may then be calculated by executing a simple

command built into the RCET worksheet.

- ' -" After examining the initial estimates of course costs and

instructor requirements, the user can then use a few

cammands built into the RCET worksheet to conduct

sensitivity analyses of key parameters. When these analyses

are complete, the final set of costs for the new course can

be copied onto a VISICALC output file. The user can then

exit the VISICALC software, enter the SDT software and copy

O the output file into the SDT data base.

B'-6

7-. P - 4~
Appendix C - Overview of Automated Training Planning

and Scheduling Technique

This appendix provides an overview of the Automated

Scheduling and Planning Techniques (APST). A more detailed

description of APST is provided in the User's Guide:

Automated Planning and Scheduling Technique. APST is

designed to assist training developers in describing and

monitoring the training development schedule for developing

Army weapon systems. APST is designed to be used with the

Visischedule software, which is an automated program for

describing, monitoring and reporting schedule information

and for conducting critical path analyses of schedule

events. A data input diskette, describing the events

required in the Army's Individual and Collective Training

Plan (ICTP), is included in these techniques. This data

input diskette contains detailed information on the

sequential relationships among the events in the OICTP.

APST is designed to make it relatively easy for training

developers to track and monitor the complex relationships

among the events in the training development schedule. In

addition, by providing an automated capability to monitor

the training schedule, it should aid training developers in

responding quickly and efficiently to the frequent schedule

changes which occur during the develpoment of Army weapon

systems.

C-I

i 4 . '. -. - -*- .
T- 67°

I:

C.1 CONCEPTUAL OVERVIEW

Construction of training development schedules for emerging

systems is a difficult task. Over 100 developmental events

are listed in TRADOC Reg 351-9. The sequential

relationships among these events are complex and are not

described in any systematic and integrated manner in TRADOC

Reg 351-9.

Further, the training scheduling process, particularly

during the early phases of system development, is

characterized by frequent changes and updates.

Determination of the impact of these changes is a tedious

and time consuming process.

APST contains techniques for using automated VisiSchedule

software to track and monitor training develpoment schedule.

By using VisiSchedule, the training developer can quickly

and efficiently respond to changes in the training

development scheudle. Use of the VisiSchedule program is

facilitated by the inclusion of an input data diskette which

(a) describes the events in the training development process

(as specified in TRADOC Reg 351-9), (2) describes the

temporal/sequential relationships among these events, and

(3) lists the expected duration of these events for a

"typical" major Army weapon system. This data diskette

significantly reduces data input requirements In addition,

it eliminates the need for an analysis of the canplex

sequential relationships among training development events

which are either implicitly or explicitly specified in

TRADOC 351-9.

C-2
,.

.. ooViichdl

o Capabilities of VisiSchedule Software

.

As applied to. the training development process, the

VisiSchedule software can be viewed as providing the

following capabilities:

(1) Allows users to systematically describe an

integrated training development schedule including

information on training development events, the

sequential relationships among these events, the

duration of these events, the manpower (by labor

category) required to accomplish each event, and

the costs (that is, salaries) of this manpower.

(2) Allows users to quickly determine the impact of

changes to any of the above information.

(3) Allows users to identify the "critical path" in

the training development schedule. A "critical"

event is one whose delay would impact completion

of the whole project.

(4) Allows user to aggregate events to determine total

manpower requirements (by paygrade or occupational

specialty) and to determine total training

development costs. -4

P4.

C.2 PROCEDURAL OVERVIEW

An overview of the procedures for using the automated

planning and scheduling techniques is provided in Figure CS

.4I.

* ~C- 3

"", " " " i , .- " o- " ,-- ." " , -",'v -. . . . • . . - ... ... - . - •. . . .
.77 Ell I.. 'NY W.. -9

SIT-Up

"ROWAN11

SO$ rWARIE

REVIEW

EVENTS

No

ACCEPTAfLE INPUI/ S.FY

EVEN

YES

REVIEW

TIE %Ak.

AILMIONAMPS

3.0

140

4CCIPTABLE INPUT/MOO4FY

RIELATIONSORM

YES

1-3

DUMATIONU RIVIIIN COMPLETION

DATE$

ACCEPTASCE No IMPLI111MODIFY

DURATIONS/

COMPLETION

OATES

YIS

3.0

INCLUDE YES

MAN"9111 INPUT

INFORMATION

< 

INFORNIAT1101% 

MANPOWER

NO

-Lai

SILICTIMINT

OUTPUT

OUTPUT

REPORT

ADDITIONAL S.

OUTPU T

ROU,0190

No

ARE

AOOirjON:L

MPUTIMOOIF.C TIONS

STOP

C-4 Figure C-1 Overview of Procedures for Using Automated Planning

and Scheduling Techniques.
The first procedure involves sett ing up the hardware and

software needed to run the automated techniques. As part of

this procedure, the VisiSchedule sottware and the

accompanying input data diskette, describing the ICTP events

and their interrelationships are entered into the computer.

In the next three procedures (2.1 and 2.3), the data on the

., input data diskette describing the ICTP events, their

interrelationships, and their durations is reviewed by the

user. If the user feels that the existing data is

acceptable and, thus, is an accurate description of his/her

training development schedule, the user can proceed directly

to the output report procedure (4.0). However, it is more

likely that the user will want to change the durations or

completion deadlines of some of the events; the events

themselves and their relationships are less likely to

require change.

If changes are required, these changes may be made using the

methods described in procedures 3.1 to 3.4. Procedure 3.4

allows users to enter and/or modify data on manpower

requirement and costs. Information on manpower requirements

must be entered by the user since this information is too

system specific to include in the input data diskette.

In procedure 4.0, the user may select from one of four

ditferent output reports to describe the training

* development schedule. After examining these outputs, the

user may wish to conduct tradeoff analyses or sensitivity

analyses of the schedule input variables. This can be

accomplished by changing the input parameters through

procedures 2.0 and 3.0.

.o

C'-

.t),-5
THIS PAGE INTENTIONALLY LEFT BLANK

C-6
APPENDIX D

DESCRIPTION OF PROJECTS RELATED TO ETES

D-
This appendix provides descriptions of the six ARI projects

- which are directly related to ETES (1) the Hardware

Procurement-Military Manpower (HARDMAN) Methodology, (2) the

Man Integrated System Technology (MIST), (3) the Training

Efficiency Estimation Model (TEEM), (4) the Training

Developer's Decision Aid (TDDA), (5) the Model for the

prediction of the effectiveness of Training Devices

(TRAINVICE), and (6) the Army Manpower and Personnel

Requirements Process (ARMPREP). More details on these six

. projects are provided in the subsections which follow.

D.1 MILITARY MANPOWER VERSUS HARDWARE PROCUREMENT (HARDMAN)

The HARDMAN methodologies provide the Navy and the Army with

a systematic means of estimating human resource requirements

(manpower, personnel, and training) during the earliest

"' phases of the Weapon System Acquisition Process (WSAP). The

integrated models and data bases of the methodologies are

designed to (1) assess manpower, personnel, and training

-"requirements for proposed sytems; (2) determine the impacts

of manpower, personnel, and training requirements; and (3)

identify and evalute tradeoffs which would alleviate

unfavorable impacts. DRC has also extended the capability

of HARDMAN through increased automation and the development

of proprietary software packages. These advances have

.- . significantly cut cost and response time and more closely

aligned the output products of HARDMAN with the information

needs of Program Managers, resource sponsors and their

staffs.

- -- )- 2
.•* ~ 7. L ~ - • -. 

' In its nresent form, DRC's tiARD'IAN Methcdology is composed

of six interrelated steps which can be iterated in a timely

fashion for the analysis of alternative design proposals.

* " The first step is conerned with variDus aspects of data

collection, generation, formatting, and analysis. It also

includes the establishment of an automated audit trail or

program record of all input data, analyses and output

products. The final five steps involve data evaluation with

respect to acquisition program goals and constraints.

The establishment of the consolidated data base triggers the

methodology and ensures that all analyses pertaining to a

new weapon system use a common data source. Consequently,

consistent definitions and collection procedures are

employed in assembling data on operator and maintainer

functions, the weapon's operational :ission/scenario and

maintenance concept, its cost factors, and its manpower and

- - training sytem support requirements. The proposed weapon

system's demand, in terms of manpower, personnel, and

training is determined in Steps 2 through 4 and compared to

the projected supply of such resources at system Initial

Operational Capability (IOC) in Step 5. Significant

shortfalls and equipment sources of high resource demand are

identified. In Step 6, the methodology is iterated to

examine alternative designs, training methods and media, and

total force tradeoffs. Thus the program manager can

participate in the design process with a heightened

awareness of the training and manpower demands of a new

weapon system, thereby ensuring its supportability as well

as its mission capability.

DRC's HARDMAN Methodology has been used on a wide range of

all three Military Services, including the Shipboard

Intermediate Range Combat (SIRCS) ; the ISD-41, a new ci iss

of guided missile destroyer (DDG-51); the Submarine Advanced

* D-3
Combat System (SUBACS); the Army's Divison Support Weapon

System (DSWS); the Corps Support Weapon Systtm (CSWS); and

the Remotely Piloted Vehicle (RPV); and in modi tied form

with the Air Force's Combat Identification System (CIS).

These applications have fully exercised all of the analytic

capabilities of the methodology. Additionally, they have

demonstrated its capability to provide timely, accurate

support for program management and review. Specifically,

significant or problematic human resource demand generated

by equipment design has been targeted early in program

development; alternative system and subsystem configurations

have been evaluated through tradeoff studies; and the

.. methodology's consolidated data base, supported by carefully

"* . documented resource design analyses, has been used to build

an audit trail of manpower, personnel, and training assessments and tradeoffs.

D.2 MAN INTEGRATED SYSTEMS TECHNOLOGY (MIST)

The United States Army Research Institute for the Behavioral

and Social Sciences (ARI) is developing a technology which

integrates manpower, personnel and training (MPT) considerations throughout the Weapon System Acquisition Process

(WSAP). Its goal is to ensure the effective planning for

and utilization of projected human resources for operational

readiness. The techniuai activity required to achieve this

objective has been organized into a major research and

development program entitled Man Integrated Systems

Technology (MIST). DRC has been selected as the contractor

*[ for MIST which is an extensive (5 year, 32 man-years) multimillion dollar effort.

MIST will consist of the necessary technology and management

procedures to address the following considerations: (1) the

SD-4

r . ., r_, _, ,. . . ,:,:,_v ",: ,....,.' .. '.-. 2,<.,' ', .<-.-'? ?4'..-..'> ? -i.,.<-. .... ... ',.,,,;..;-.-. . -. -.
treatment of human resources as a performance and cost

factor during system concept formulation, (2) the planning

and forecasting of manpower information, (3) the parallel

development of associated training systems with weapon

systems, and finally (4) the specification of test and

evaluation issues to ensure human resource accountability,

and to support the ASARC/DSARC review at. each of the major

development milestones. MIST in particular wil) integrate

and demonstrate the technical relationships among these

considerations, and provide the necessary methodology to

ensure their effective treatment during the design process

from a performance, manpower, training and cost point of

view. In addition, MIST will. demonstrate how these

technical considerations are related and responsive to the

Army's Life Cycle System Management Model (LCSMM), and to

relevant Army regulations and responsible agencies. MIST

will further provide a comprehensive framework for the

assessment and integration of other relative technologies

and information as they are identified or: developed.

When fully developed, MIST will serve as an integrating

mechanism providing efficient interfaces for information

exchange and feedback among concurrent activities and/or

processes involving industry and government, system design

engineers and support planners, Army acquisition managers,

and the Army's oversight bureaucracics in the Department of

Defense and the Congress. Hence, MIST will provide the

design and implementation of more cC)mprehensive and

responsive data bases and technical/management procedures

employed in prime and support system development,

acquisition and operation.

D
  

 

7 

.

I' ,

When completed, MIST will be composed of three principal

components:

4q.

O Analytical Tool Designed to Support Program

Managers, TRADOC System Managers, Acquisition

Managers (e.g., Weapon System Manager (WSM) in

DARCOM), and their staffs/supporting agencies;

o A system-specific data base to provide a single

point source for all man-machine data (input to

the tools, output from them and decision support

information for program documentation and reviews)

collected for an emerging system.

0 A management information system to link MIST users

and the data base in an effective communication

network within the framework of the Life Cycle

System Management Model.

Hence, MIST will provide an important new capability within

the Army. This capability is embodied in a cost-and timeefficient system for human resource requirements analysis

early in weapon system development. MIST will capture the

most recent advances in both man-machine technology and

automated data processing to provide a system which is userfriendly at the operational level and responsive to the

analysis and information requirements of users at several

levels within the chain of command. Finally MIST will "fit

into the real world" by providing effective interfaces to

existing management information systems and data bases for

logistic support analysis, training system development and

personnel planning in DARCOM, TRADOC and the Army Staff.

D-6
D.3 TRAINING EFFICIENCY ESTIMATION MODEL (TEEM)

TEEM is an interactive computer-based aid designed to assist

training developers in early evaluation and generation of

training program components. At the foundation of TEEM is a

set of psychological variables which reflect the training

information contained in a system task description. This is

in contrast to the normal textual form in which task

descriptions are usually developed. A representation by

variables was chosen bec use early task descriptions are

often not in standard formats, may change quickly, and may

require elaboration by design engineers or other subject

matter experts. A predetermined set of standard psychological variables provided a starting point which could be

depended upon for input into the fixed operational code of a

computer program.

Initial task groups corresponding to procedures that should

be trained together are generated through an algorithm that

clusters tasks based upon internal similarity along the

psychological variable dimensions. After clustering is

complete, a task group has maximum difference between task

clusters. The clusters serve as an automated estimate for

use during the initial formulation of a training program.

Task groups are then modified, if needed, based on the

expertise of the analyst or outside mitigating factors such

as management and cost constraints that would normally not

be included in the psychological variables.

Selection of concrete, costable components such as media and

method of training are based upon the similarity of a group

of task variable requirements to the ability of a media or

method to manipulate psychological variables during

training. Selection is accomplished by representing each

training program component by a description in the same set

D-7

. .. .. ,
of psychological variables. A task is also coded as a

vector of variables with an assocated variable value for

each dimension representing applicability or nonapplicability. Selection of a program component is

performed by automatically matching each task to each

potential component and selecting the component with the

higher match of critical variables.

If an already existing training component is being

evaluated, the matching process is utilized in reverse to

generate a figure of aggreement between the theoretical

maximum possible under the task and hardware conditions and

the actual match under a particular alternative. The

normalized numeric value of this match is called the

efficiency ratio.

After efficiency ratios have been generated for potential

training programs a final selection for the best training

approach is made based on pair values of gross dollar costs

and their efficiency ratios. Together the values represent

an early form of cost-effectiveness ratio for each approach.

The best candidate programs are selected for detailed cost

analysis through a much more complex accounting program and

given increased program specification.

TEEM was constructed with the realities of the early

training situation in mind. It does not assume extensive

and accurate information. It is "user friendly" with stepby-step guidance to individuals unfamiliar with microcomputers. It is designed to accommodate local school needs

and training development procedures without major program

. changes. It is also flexible enough to be quickly modified

based upon improvements in psychological research and field

experience.

* D-8

. _-.............................................
D.4 TRAINING DEVELOPER'S DECISION AID (TDDA)*

The Training Developer's Decision Aid (TDDA) was designed to

assist training development specialists in applying the

Instructional Systems Design (ISD) process. The ISD process

is a comprehensive technique for the systems approach to

development training. Applied Science Associates, Inc.

- .(ASA), working in collaboration with the Army Research

Institute Field Unit at Fort Bliss, Texas, has developed,

tested, revised, computerized, and conducted tryouts of the

model at five Army schools.

The TDDA is intended to be an aid to training developers who

need technical assistance in designing or redesigning

training and to streamline the process of making training

decisions. The TDDA is procedural in nature; it leads

- training developers through a series of procedures which

specify: (1) what decisions are to be made, (2) a rational

'-. order for decisions, and (3) the nature and sources of

* information required to make valid training decisions. The

model also serves to organize and retain information that is

gathered and processed during the course development

process.

The TDDS model can be used with partial or complete task

lists or performance objectives. It is not necessary to

redo the job analysis in order to use the TDDA process.

This reflects a training development philosophy that a

thorough analysis of the job early in the training

development process is essential for making valid training

development decisions.

Derived from Frederickson, et al (1981).

01 D- 9

;, .- ;.". C.....* *. .. ._. . . . .. . .. . .- -" .- • -""•. . . . . . - . .. "
The Job Analysis Module requires the greatest input form

subject matter experts (SMEs). Those who hold the MOS being

analyzed and selected to serve as SMEs are expected to be

,'. , expert job performers with at least eighteen months of

recent on-the-job experience at an operational site. This

'. is an essential requirement for the successful application

of the model. The model is sensitive to quality of the

input data and information, since all three major decision

(what, where and how to train) are directly tied to input.

There are two separate parts to the SME input in the Job

Analysis Module. First, the initial task list is generated

by exercising function analyzers. Then, the task list is

verified as correct and complete. This should be carried

out by a second set of SMEs. At the time the tasks are

- ."verified, the remainder of the input information is obtained

from the SMEs. The output of this module provides the

answer to the question, "What should be trained to prepare

someone to perform in their MOS?"

The Functional Learning Requirements Module requires inputs

from two sources, MOS SMEs and course development instructor

personnel in order to make decisions of how to train. The

information generated in this module is used by the course

developer to specify the functional characteristics of the

training program. The emphases in specifying these

.- characteristics are on how best to communicate the learning

objectives, job context and consequences of task

performance, and on providing for efficient practice for

acquiring special job skills.

Development of the course requires that the relationships

" " between tasks be determined. The last SME input provides

the information for describing these relationships. Tasks

are related in three ways. First, the same skills may be

* D-10

* '* *.- ..
mo . ior r . -~7~ -. p -. - - * -_ i -°. - . .. .• .

required in the performance of different tasks. Knowing

which skills are redundant allows for more efficient

scheduling and sequencing of instructional elements.

Second, some tasks must be performed on the job before other

tasks can be performed. For example, a piece of equipment

cannot be repaired before the cause of the malfunction is

isolated. Or, an intelligence report. cannot be prepared

before the intelligence information and data are atnalyzed.

The output of these tasks are inputs to the subsequent

tasks. The third relationship is merely chronological. One

task must be performed prior co a second one but there is

not direct output-input dependency. For example, a weapon

system has to be emplaced (a collective task) and made ready

for action, before it can be used to engage hostile targets,

which may or may not occur.

The dependency relationship information is provided by SMEs

and is then used to build a task dependency hierarchy. The

product is considered to be a general course structure upon

which the course map can be built. The task dependency

information is generated in the Functional Learning

Requirements Module, but is analyzed in the Structure

Designation Module.

Information and data from several earlier inputs are used in

the last module to make decisions as to where the training

should be conducted. The task criticality data is one of

the primary inputs to this decision. Additionally, the

instructional setting characteristics and school context

information are used in reaching these conclusions.

The TDDS model then is used to make three major decisions:

what to train, how to train, and where to train. Some

decisions are made algorithmically and others are

heuristically determined.

.2 . --
D.5 A MODEL FOR THE PREDICTION OF THE EFFECTIVENESS OF

TRAINING DEVICES

TRAINVICE is a methodology for the systematic assessment of

the characteristics of training devices under development.

The model is based on the assumption that certain attributes

to be assessed in the training situation will lead to

transfer of training to the operational situation. Therefore, the higher the rating on the assessment factors, the

higher the transfer that will take place and the more

effective the device. The model provides a framework for

the making of these judgments. The three variables entering

into the assessment are: (1) the transfer potential of the

device, (2) the learning deficit to be overcome and (3)

instructional effectiveness. As with any model, its

effectiveness depends on the adequacy of the input data.

Inputs into the model consists of descriptions of tasks and

subtasks represented in the operational situation, as

circumscribed by the training objective, and those

represented in the training device. The controls and

displays and their functions for both situations are listed.

In addition, the skills and knowledges involved in each

subtask in the operational situation are formulated for use

in the model. Using these inputs, judgments are made using

rating scales. The subtasks in the two situations,

operational and training, are compared to ascertain if

provision is made for representation of the subtasks in the

" training device in the commonality analysis. Next, the

displays and the controls for both situations are compared

.[ on physical and functional similarity. The more similar the

display or control in the training device is to the

* Derived From Narva (1979).

0_ D-12
7 

* 7

operational situation, the higher the score. This is based

on the premise that the greater the physical or functional

similarity, the greater the transfer of training that will

result. Physical similarity refers to the appearance and

physical aspects of the displays and controls involved;

i.e., their "fidelity"; functional similarity involved in

the operation of control, in information processing terms.

The learning deficit analysis is based upon (1) the assessment of the level of proficiency in each skill or knowledge

for the students upon entering the training situation, (2)

the desired level of proficiency in each skill or knowledge

for the students upon leaving the training situation, and

(3) the difficulty (in terms of training time) of training

in the skills or knowledges involved in a subtask. This

analysis yields a weighted learning deficit for each

subtask. The judgments concerning the level of each skill

or knowledge are made using scales adapted from Demaree

(1961). The last analysis involved in the TRAINVICE model

is an assessment of how well the training device adhers to

"good" training techniques. In order to perform this

analysis, each of the subtasks is cast into one or more

categories of behavior. These categories are those of

Braby, et al (1972), which are derived for an earlier

* - behavioral categorization by Willis and Peterson (1961).

- . For each of the behavioral categories represented in the

subtask, a list of guidelines, also those of Braby, et al.

(1972), are consulted and judgments made of the degree to

which the guidelines are followed, or not followed, relative

to the manner in which the subtask is represented in the

* training device. The guidelines are broken up into those

dealing with the stimulus, response, and feedback aspects of

the training situation. For each subtask, the lowest

obtained score on each of the three aspects is used to

derive an average training technique score. All of the

preceding rating, are then fed into an equation to formulate

D-13
O1 x 

an index of prediction of training effectiveness, ranging

from 0 to 1. This equation is as follows:

(Ci x Si x Ti x Di)

Di

where C is task commonality, S similarity, T training

techniques, and D the training deficit scores for each

subtask. The equation was derived from a transfer of

training equation of Gagne, Foster and Crowley (1943), which

was for use with empirical data, while the TRAINVICE extrapolation deals with judgments made concerning aspects of a

device assumed to bring about subsequent transfer of

training.

A validation study has been performed on the model,

utilizing data obtained during the course of two field

studies as criteria against which to compare the predictions

derived from use of the model (Wheaton, et al., 1976). The

devices were tank gunnery trainers involved with burst-ontarget techniques and tracking with the main gun of the

M60Al tank. In each case, the. prediction of no differences

between the training devices involved was found to be

consistent with the equivalence in transfer actually found

in utilization of the various devices. This was felt to be

a promising but not definitive finding.

In order to obtain additional validation data on the model,

and also to obtain experience in utilization of the model to

determine if there were aspects that might be changed in

*. order to enhance the practicality of utilization of the

model, the Army Research Institute personnel applied the

* * model to two maintenance trainers undergomn; evaluation at

D-14

7 . - . , . , . ."
the Army Ordnance Center and S(-hoo]. This afforded the

opportunity to obtain data within a ditferent context than

that dealt with by gunnery trainers.

These trainers were concerned with automative troubleshooting. No difference in training effectiveness was

predicted for the two trainers, which agreed with the

results of the empirical evaluation. Varlous aspects of the

model which caused difficulty in its utilization were noted

and influenced the development of the modified version Tn

addition, ARI conducted a three-day workshop, in which the

developers of the original model and individuals who had

utilized the model or had an interest in its utilization

participated, and this furnished further ideas for possible

modification.

D.6 ARMY MANPOWER AND PERSONNEL REQUIREMENTS PROCESS

(ARMPREP)

The Systems Manning Technical Area of the Systems Research

Laboratory, US Army Research Institute, is promoting a

research thrust known as Manned System Integration. One

major component of this program is the development of

improved procedures for the manpower planning process. The

major objective of this proposed research is the development

of new, innovative and effective techniques to support the

*-. determination of manpower and personnel requirements.

- ARMPREP will include: methodologies to accurately estimate

manpower quantity and skill level of personnel through

SO derivation of behavioral requirements and subsequent

translation into Military Occupational Specialty (MOS) and

other relationships; techniques to aggregate manpower demand

of new systems for comparison with available supply;

requirements and attributes of a manpower requirements

management information system; and finally, the specificaD-15

S7.

' ,'V ." ,' ,'. ." . . . -.-.-. . . . . '.-. . . . . . ,. . '- . ' . .",- ..- -" ' '- -" '- .--- -'- .- -. - .", "- '. -"... . . .. '...,
tion for a computer interactive system. ARMPREP is intended

to provide tools for potential integration into the current

manpower planning process by cognizant Army agencies. In

essence, ARMPREP has four principal components each with its

own identifiable research products. These four components

are:

o Manpower and Personnel Requirements Determination

Methodologies (MANPERS).

o Aggregation Procedures of Manpower Demand (TOTAL

MANPERS).

o Requirements for a Manpower Requirements

Management Information System (MARMIS).

o Computer Interactive System for determination of

Manpower arid Personnel Requirements (AUTO

MANPERS).

MANPERS

The MANPERS component will provide the techniques and

methodologies to formally quantify manpower and personnel

requirements. These formal methodologies will aid in the

timely and accurate determination of manpower and personnel

requirements. In addition, these methodologies are intended

to standardize the manpower and personnel requirements

determination process to provide information of a consistent

nature of use to Army personnel subsystem developers.

Specific aspects of the MANPERS component will consist of

the following attributes:

(a) The development of a taxonomic model to derive

behavioral requirements based upon new system Task

* D-- 16
Descriptive Data (TL)D) and translation of these

behavioral requirements into MOS and associated

decisions. The model will define the requisite

data to be included This input data will be

obtained from sources such as LSA, Task and Skill

Analyses (TASA), system requirements and doctrine

documents, engineering design data and training

plans. The model wil] specify the ]eve) of

specificity of data congruent with each phase of

the LCSMM. In addition, the model will identify

the process by which the behavioral requirements

are derived (i.e., the taxonomy) and the means by

which the level of data specifity is embraced by

the taxonomy. Finally, the MANPERS methodologies

will guide the selection of personnel for a given

job in a new system based upon the behavior (or

performance) expected of him, as opposed to the

current method which involves the preselection of

MOS category, previous to and/or without rigorous

documentation to support such a choice.

(b) Integration with events and activities of the

LCSMM, ILS (LSA) and TSD, The MANPERF; process

will provide the necessary manpower input to

decision makers at critical system development

milestones to encourage optimization of manpower

planning. Where possible it will support the

determination of manpower related life cycle and

support costs relative to specific systems.

Through involvement with the ISD model, the

MANPERS component will support a margin of

requirements for new equipment training (of the

Material Developer) with training requirements (of

the Combat Developer) as identified by specific

schools. Finally, the MANPERS component will

D-17

.
serve to provide increased accuracy of information

used in development of draft plan TOEs (as well as

throughout the TOE planning process).

(c) A final aspect of the MANPERS component is the

development of a MANPERS manual to include: job

aids, examples, procedural models, instrument

formats and presentation methods for thorough

manpower and personnel requirements determination.

The MANPERS manual will provide a tool for quick

responses to needs to generate quality information

on a timely and economical basis. The MANPERS

manual will facilitate the standardized determination and presentaiton of requirements to increase

the utilization of information by decision makers.

Finally, the MANPERS manual will provide a

competent training tool to producers of manpower

and personnel requirements (especially for those

new to the job and content area).

TOTAL MANPERS

The TOTAL MANPERS component will provide the techniques for

aggregation of manpower demand for comparison with available

supply data. Manpower demand is defined as quantity of

manpower by quality of personnel (e.g., skill level) both

within and across systems. In short, it is a quantity by

quality of manpower comparison between demand and supply.

Specific aspects of the TOTAL MANPERS component include:

(a) Development of technqies which can provide

quantity by quality loadings of manpower demand

for a specific system. The data used by TOTAL

MANPERS procedures will be an extension of the

D-18

" . . "-. . "' . .. - -. - ".W', '% '.,..' ...-. " • .o • . .--. ."%"-. ," .".- . ". ' .' . .. . '. .- '. -.-.-- . .. -. "... , "
basic MANPERS tools by relating some paradigm of

personnel performance or behavioral requirements.

(b) Another aspect of this component will be to

provide techniques which aggregate information for

quantity by quality of manpower across systems

through comparison of manpower demand loadings

held in common with personnel performance or

behavioral requirements.

(c) A final aspect of the TOTAL MANPERS component will

be its coordination with events and activities of

the LCSMM, the PPBS (through SACS and TOE

planning) and ILS to insure the timely

determination of requisite manpower information,

and authorizations, as well as changes to force

structure and composition.

MARMIS

The MARMIS component will provide the manpower management

procedures to integrate the MANPERS and TOTAL MANPERS

components with the Army manpower related documentation

process. In addition, current manpower management and

documentation procedural deficiencies and recommended fixes

will be identified. The most significant activity in this

* ..' component is to support current Army actions in management

* information systems developments relating to the manpower

- planning process through the specification of the requirements and attributes of MARMIS. Specific aspects of the

MARMIS component include:

(a) Determination of manpower management and

documentation procedures to facilitate the

utilization of manpower and personnel requirements

D-19

-. - 4 . -- .. . i . .. -. . ; .4-.' - .. '-j .. . -* - -. -.-- - . - S . . "-A" - '-
, . .

methodologies and aggregation techniqes so as to

improve the timeliness and quality of requirements

generated.

(b) Identification of means to improve accountability

of manpower and personnel requirements documentation to identify potential economies in management

and documentation flows. This will take into

account organizational and technical complexities

which serve to work against desired results. In

addition, these efforts will facilitate improved

manpower planning as well as to focus upon

deficiencies for possible resolution.

(c) Means to control manpower related information will

be developed so that many activities concerned

with other interests (e.g., LCSMM, SACS and TOE

planning, and ILS) can be fully coordinated with,

such that appropriate milestone are achieved on a

timely basis and the requirements of other

activities are met as they relate to manpower

concerns.

AUTO MANPERS

The AUTO MANPERS component will provide the basis for

development of a computer interactive system for determining

manpower and personnel requirements by developing the

requirements, attributes and specification for AUTO MANPERS.

AUTO MANPERS is intended to provide a basis for MANPERS an

TOTAL MANPERS methodologies. Specific aspects of the AUTO

MANPERS component will support development of a proposed

computer interactive system by addressing the following:

D-20

-0 r. *- - .*.-*- . .'~
(a) Ease of documentation, update and edit of requirements through use of an online, interactive ADP

system which provides: continuous a'vailability to

authorized users, embedded training or job aids,

and quick response cross-referencing capabilities

with related manpower documentation. These

capabilities should lead to increased accuracy and

timeliness of requirements production and

utilization as well as potential economies in

administrative support.

(b) Another potential aspect of this component is the

development of a built-in analytic model and

mathematical capability to increase the rigor of

manpower requirements determination. For example,

whenever mathematical calculations or transformations are required the formulas should be readily

accessed on the system.

It should be noted that a training device can be considered

to be a large system in and of itself. Thus, it has its own

associated resource requirements. Estimation of these

additional resource requirements is beyond the current scope

of ETES. However, it is expected that these resource

requirements could be estimated by comparability analysis.

Input to the determination of training device resource

requirements is provided by the task requirements associated

with each device which are determined in Procedure 2.0 and

the instructional programs developed during Procedure 4.5.

D-21

°ii
*

*

Jo

THIS PAGE INTENTIONALLY LEFT BLANK

-a

a,-.'

U.'

N.

U.

B)

-U

'a.

I'

'4.

I.

D-22

- UU.* . .a

. a . . . . ~U*-
< >1

2Z'

<N <

E <I

-n lo a,

<ul a.l

-~~ . S 

'EN Z:

'.(I - I = -: C

2 -0 -

m E- I - o E- Z:

*. 0, E5~CJ

W- E- .c-

* - . jl

0lc

Z 1 Z

'73
Q) E

% E 0 ,j- ,

10 3 - >, w L 0

4 c ) V) :3 C) c l (U> ol > - 00

-1 CP O(V- -4U (n (n

rO m4e-r fuC COC 4 4w

a0 -Cr wM0t ,=(

0- a-414> '- 4 .-4

(0 L"0 J -4 0 01M (

-4E-4 wo OC4a-4 o s4><

E-4 4 E.4> U OrC' -4-4

z4J (a4C r_ d4 c0 0 C0 CDO (a-4O 0 (7 E0 -

(a 0 . a(a7 cr ax C - 04 In (P. I 4 )- o 4

uC -4 U 

1-4 U)'CDa

0 a 0

)0-4 Co )a

0n~ >4 0a24U

4 U 0

C00 E0 a-'U j40U

E4 U)0 U) 4 (0)4'Ua) E

'U Z (UU) UU) C .' 

-,qUU 40') c 4 - 4- -4 e0A

'0- -4 N *--4- .0- '04- U) .&J .0 4 U

c 1-4 Cl EU4~U -4 mm - 4-0E U

04 041 -4 FaN ON E.' (-4' a - E

rEU '00C4I a)Uf 0 0 0- ~ 0 -,4 C zCU aO CU u~0

. U dA 0U 0CC4 E- C 4-4: 40 (3 r ,

-4 0 wC-

-4

o C

44I

* E-4

E--
C) u

0 V

41 0 U -

(a .0 -4 fu~ U) Q) ' Au j C >. -4- r_- w 4 E

CD r-P--- -44. M 0 - > C) aC0 

1 ) - * W-U&-(0 &J - 4 1-( ul 1- ~ - CC44.

04 (ncp -4~- fu> Q -1 - a (uQ -4V -HO :

C-..-1 01 3~ :-4 .- ~ :3 %0H E IJ- '0> 1 a

-4JW r a 4. wj~r .- Ue a)U) 5 cn)4),4U)%"

m 4.4 u Q C4 0 C,>, .r' W o>Cs4AU

S001

C) 0

E-4 > U -J

o 1 0V 0 )

-4 .. a 1 ' 0 4J0 Q)

r-4 4J rc 14 

E uC 0.CO0.

Q)4J04a -'4 (0 -4 4 a w~

Cl C:co C0 Cl .0 rM0

0 0 0 0 4 3 L4

o u 01 E000 aC

43 4 l >C 0

a C 40 0m 0 C 0( 0n -4 14~4 0) 1( 0* U) 4

0-' L~~*HU) O C 0CU U>

m to~0 4 ena -4r

U4J U)0 1 a~) 0'-0U O-4 0 44

-4 MU 0 1 0 u 0 , G E 0 

o 4 4>0J W 0 m> 4) 1 2 10UL)U) r

%Dl -4 U): Ua u I a a a>

0 0 >*w a

N ,-4 4-4 *.E-4 w w a

ge2J U4 EnCE n0 - l ,HH r i

0 ~ . 4 .4 >4.

:3 W 4.. U

4-4 4 U) 0 . w a

44rC. .) C :3 -4 0 4

> C ) (nCU 00 a V 3 > 0 10

4 ) r -,a aW -. 4 NC .-4 ra

.0 j"E0 U) 4 0 a .J -JC~ 04J

0 r-4 4 (0 0nCP40 4 )O41 >-(0 04 3 40

-4- 4 4N' 4- E - *'4' r-4 ,-4 .4.I) V

- 4 t U) A U)I-W E -4Wt -0 V)a- U)

C le0 fa Cn ) 0 0 (n0 r_ ( 0 >0

41

0U) t (a

C) U) w40

E-3
LL'.

1 -4 0W,-- .

0 0' - 0 0 0

C ~ ~ U '-cr--U 0Q~~ 4 f -

r"-U 41 4 - a) fa C- O C0 .)-4 44 1> 0 m

14 g U E -. E~- E-4

E. -4 0 U 0 0C C >- f

SU .C E X U 0' -4 a) 0 0 a) r_ e

r.1- ." z f n

W =$ E:,:.m; w E

m o 0 0a x m0 - -14 .1-4 a U, c 4 a, -- c0 z00 4 . )U ( (a 0 Z0 EJ >> 0U EUU

I..l

0

ot

[.--

C.C E- KC EC E

0 t ( 1 :3 z. i

0 C 4X

,.....W , ,, ', -4 ,

>" -

WV 0-' -1 0 0 44

0 a,- e <I+ w 0 u" 0 u

W_. ,,-, 

E- e-4 " to . )e ,4 e

0" > c 43 (1 tr Q) V) 0" , a 

-4 -4 U4

W"- W - W 4

'-o 0 c = o 1 a) , u4 Q

U2

%C

-4 C

I.0

0E 04

OU(O

. 0 ' 0' (V 4I )

In 4.J W .. J -) I-4C

I.'-, .'; C'-U C U0 o

?. I-E40-4cr o. J
0

4.) 0'u- 0

c) 0 (a0 w14C wC)u j

a 4 (73 (04 E- 4

4.4 4 0 --- -4- 0- Ce r

Z) fn CL0 :3 0 -4en C - 14

0 r0 -4 'a 4 -r(

w 44 -4 U) I -4 a)

U) --. >4)0d -W >4 04 (a "0

0 0f~- =1a A J4. -4

00 -4 c~- a V'm)

InU fl a, Q (4 :..

Q) a4J .

0 c 0 0

0 (a C 0) (aE)

a4 E) I En tl- 40

o-0 040

u ( O0(U 0 0-

to 0 (a 0.s- w u0 1

0Z 4)- 4 U0' 0 0u

Uz 0 - -I

0-4k-4 w~~c E

0 4 0IJO. C: -40 4) 1Sa a) - Vra

Lnw lLn: n -4 c nC C C C Q

M04a .'10a M04 n0' 00 (a (0- UcflU~~- 00,(~UI U'

U -4L- 0 N -4 ME4( 4 o> 111c w ) f 04 -u- P

4J : :1 40' a > M -I- 4 C: U-OC U) -C) W ( - =- 4) 0 EV -1 U) 4

-0~ -- 0 4: *4r (UE( > 0 > C: a)L 1 j- 0r C C

-4&~. -fl 4JC- 4U 4 0 '4 - 'a ) -4>,0 : 0 a)(z

UC0> M '0)U -4 E-4E--- U- 4 M (n -40 I( Q)r )0E

04U 0u)-4 0 C 0 aC >4)W )0a rj-UC~4 X w

E-4 r4 0 E 4 04 r= U) E4 E-4- U &4L4 ~ a l,Z - 4

cn 0

a 0

(D a) a)CO)--

U) a) U) 0 4)

s- 0- 0( -4)

0) a) 0u 0~ 0 04

U u Cl, U 0 M--~

0 a

~u) u. 4j Vw 0J Uc )U)U>

N0- a)C/ 0a) 00 ~WC.

0 ru
C I u -4

-- 4 C) U) r- EOLI 

>e m (a4 3 aU) r- .0 C- iI cc wkI 41- 'x -a 0 0 41 U fa 41 C 1) U) I' - 4 Ln w

E- U C E-4 C 4-J E0 LI4- '-- 1

~fCD C: C ULI l . Q-4 91 U) L .

fn 0 00C -. 41 O -4 n- 4 Cl W 4 'a t O U 1.4''

V] -4 u-E u 0*4E -a-~- zW E- C- U 4-. t,)

C. W 1 4-jE W 4-L w wQL C4-4 <0

< ~ (a~~E' ~ w U~ &JJU' o-Ua0 , -'U nu*1,k -> 

S -4 E (n

0~ 0

o C. o 'U W J1

u) a s. . C C C C < - ,: 0 0 0 0 0

Q 1-4W CL. z z Z

u -44

1 0

'Iq

U '4-4 U) 4J

EU

U) 1, U

* 0

4.) 41 :

LI-4 L) m.- 'ULl-4 044 0 14~ r_ C . wL C>,O W -- H 0 -4

:34 0 "-3 CO -E-4 -4 0 0 .J 4J

U'u- E0 fa 0U O'L 4J LW (n U)C Cr

U4 aI wL u 4JLWU C Lu , W I44

4.IE-4Lw 44 E4 LW )E-4 - 0C : Q)-444OU 0 0CCra 0 0 0 CUr) wI ;c4 WC04-JLII O- :3I W CL C4 0C 0 CU 4 J~ Z)0-f -) 4)0 O)-4

E- t 0UC00 WC 4fJ~ .'-'4 w r w n u Cn U4.J ) 4-J

-4- U --- 4 ) - 0--1u 0 4JO0C L0) ) WI.-4 u0M.0 m (n W U)0) U1LU0 Va -)4-00 &> un. 04 C r-U) a.~f a n LI=U 4 LCU)04. L wa- 0'U OLaE-4LW

C0 WI4J 0)L (v w WLI4 j ~ '44JL MLI W WW D ~ " 4-

C4d =3 -4 r-4J~ :3 r-104-1 3 U) :3 -4 0 ZO0 Q Z- U U) w~ 

0 (

C- =j Ci

O w CWUw-% "

E- w E0 "-' )

L-6
0 . 0n 0 0 cr u 0 - Z - uU f)0

0 - C 0 " x 0 4J X

f-W-4 40 0 C: W

Ln.C 0 : (U-t-4 ( U U a)

0 0 0 I

0 m E 0-U . 0 nw L.4

c) 0 w w r-4 - -4 ,C

""- E, 0 . f 0 c ci.E

Lo Cw ol c lm >, 4J >*.-4 E<4

r0 >1 w-4t u2l- 41a U) C

E.- .- E w U u Q)( ( < E,4

C/ 14 w 0 ~ 00 J ) - *U)c 0 " c u

o3 ea.- o oE w m to 0 ,E0 oL

0 04 0 0 0 M 0 1 a) wU)

0

E-4 00

*0 u

- fu 4

M 44

U 0

0m 0 O3 0

- 41 0 0 0.-4 0 _) zC C'f- U -c

0 0 L W r- E-. 0 0

z u < z z

1 U

C1 0 I U 0 0 

.4U20 ,-, fl L C: -oC --. 4 a) --4 a u20 0 t

o 0 r r_ 0 wC w 0 " w 0

-% . 0 0 .C

C/) 0)a

0: U- Wo

• "-w W U-4 z 0 4 41i 0 C u uCf 0-

-I0 0 v ' -. 4

C u 04 0- 00) ro (1) W xW a 1) 4.) 41)0u 0: _--. w lz w4 r -- 4Q0 0 <*--f 4Jw 0 ( 1~4J Ca)4G u

clE0 1-1U o .0 '0 014-4 -dU

w r_-4~ 0Q 4 X- -L :3 (L) 4.J .U)-4

0 4 .C 1U -4 Z U) ZP'V02U . J'~0 COJ -aU W4 oj- LW W . ~ uU) r0 Wc 0m 4.

P E- M U) 0 E (a (a 0U)

C 4-

0

"0 -"-(10) a) 4&J 4J n

0 0"4t2

E-7
Uo a4 0 m -o E I'

GN 0 Q) m-- c .4 U) 

<a >. -~ -1 0U '0 A C

E- > (2C W 0 04M > .C0 

C C E~ 0- ) )- 0 E CZ C: CU)LO 0) 

a r u 0l u) W V) C - a -a a4 0 V- -) a~ UUd)a -) .. i

0 ~ ,U c~,)OC U) C: . X fu- 0

tEL~ 0.4 UCCJME n 14 E-U 

a.,-- a) 0C- al. ) > 0--4-.' E-e(0 ,a4 (a C4. C)"> 'Ci> ' > 0 . 44

U)

D>

0 -0

U - 4 <'

u ' E >

0D 71 e2 j (

a' <' a' a'

--- --- 0)0 Cl 0)0n

C. 0M *4-

443 WO 0 0 -4 w 0 '-4 U

0 E w- w-" E-' QO~ ) a4 E 0-. aC

C U) C 0 r_ U WU VC r_ n -I 2E E

0 -- 4 ) -4 ) -f -40 a).0) 0)- .0(D>CO(a

40 .40 Aj4J)J CC r_ aC .400 0

U)-. U) -4 m w r)- C-4 r- -4 C-- .,I-a- (v > .- > u .> (a' 0> ra tu -4 > a0 m a x ( Q 0W Ox ) r-40a) 1- v:

0

0 00

.40
.a c 

0 Q- , '

00

C 0

-i x C:c

U- > C- N

0 0 J co f - "

0 m a . 0

OD Ia O 4- 0 O n .

E I > 0 0 EC m x 0 -.. a r -4 o6

u a*J-~ CL~ u 41 1

".- 1 0

%U..

U C:

U2

0 E-

,',.- .l ''"-",..

U 0-4

J , 0 C 4" 

0 0

z

-. C.4

cn

W -W

u )(a a 0 '0

E-4 P0(3 a -

0 0 0 w4 rOIO) 0)

0 ~ ' c - 1

u Ln 0

u UO
THIS PAGE INTENTIONALLY LEFT BLANK

4E-.
L

Appendix F. List of Potential Performance Measures.

Speed

Acceleration

Range

Vertical Manueverability

Horizontal Manueverability

Carrying Capacity

Rate of Fire

Firing Capacity

Firing Payload Range

Firing Accuracy

Communication Range

Communication Rate

Communication Capacity

Communication Distortion/Accuracy

Detection Range

Detection Capacity

Detection Accuracy

Recovery Time

Probability of Survival

Memory Capacity

Processing Speed

Processing Accuracy

Processing Capacity

Operational Availability

Achieved Availability Inherent Availability

Availability

Mean Time Between Failures (MTBF)

Summary Measures - Elapsed Maintenance Time

Mean Corrective Maintenance Time

Mean Preventive Maintenance Time

Mean Active Maintenance Time

Maintenance Downtime

Mean Time Between Maintenance Actions (MTBMA)

Mean Active Corrective Maintenance Time

Mean Active Preventive Maintenance Time

Maximum Active Corrective Maintenance Time

Turn Around Time

Summary Measures - Manhours

Mean Maintenance Manhours per Equipment Operating Unit

Mean Maintenance Manhours per Maintenance Action

Mean Maintenance Manhours per Month of System Operation

Mean Corrective Maintenance Manhours . Mean Preventive Maintenance Manhours

Detailed Measures (per Task and Task Type)

Elapsed Time

Maintenance Hours

Number of People Performing

Frequency of Maintenance Action

F-1
APPENDIX G

DESCRIPTION OF TRAINVICE

This appendix provides a description of the TRAINVICE

procedures, The description is taken from Training Device Requiremnents Document Guide.

G-1I
Evaluate Theoretical Device Concepts--If no existing devices approximate

the proposed device's requirements or if alternative existing device concepts are available, one may wish to derive new alternative concepts that

have no existing hardware representation. Theoretical device concepts are

those device concepts for which no actual training device equipment now

exists but for which such equipment could theoretically be constructed. As

with existing device concepts, theoretical device concepts are specified by:

.G

.4.

6 .. . .... .. .. .. ... .. .. .: .: ... .... .. .: .-..-. ,.-, ... , , . , : : , . . : : :.
* ...- ,-.

- Tasks

"' * Task elements

9 Controls and displays

For theoretical device concepts, these data are all defined in the task training requirements already developed. A theoretical device concept specified

.y all of the task training requirements would represent the "ideal" device;

by definition, it would meet all task training requirements. Because of

practical constraints (e. g., cost, development time), it may not be possible

to develop the "ideal" device. Some trade-offs must be made. These affect

tasks, elements, controls, and displays which specify a device. Thus, for

practical reasons, it is desirable to be able to evaluate alternative theoretical device ccncepts. The procedures described below allow this; regardless of whether existing or theoretical alternative device concepts are being

evaluated, th, procedures are applied in the same way.

" .Analyze Alternative Device Effectiveness

Task Co.nmonality--Table 27 is used to analyze each task requiring training

on a training device. The table is used in deriving a task commonality index

for each training device or training device concept under consideration. The

following procedure is applied to one task at a time.

0 1. List the task elements down the left column of Table 27.

"2. List the training devices or training devicc concepts to be

- - assessed across the top of Table 27.

G-3

- - . - -. , -. .'- - .- -. .- ,,.. . . . - -. . . . • -. -
TABLE 27. TASK COMMONALITY ANALYSIS

NAME OF TASK_______________ _____

REQUIRED TASK ELEMENTS DEVICE A DEVICE B DEVICE C

2.

3.

4.

- . 5.

6.

d 7.

8.

- 9.

- 10.

Total number of 1's

Task Commonality

Index

Task Commonality -t of 1's for Device

- Index (t; of Rq'd Task Elements) + ( of Unique Element in Device)

G-4
3. For each device or device concept under consideration list

Son a separate piece of paper the elements of the task as they

would be trained on that device.

. 4. For each task element in the left column of Table 27

<" determine whether a device enables the trainee to prac-

*[ tice that task element by examining the separate list of

task elements for that device.

5. If the training device does allow practice of that operational

task element, then enter a "" in the appropriate cell for

that device.

6. If the particular task element is not represented in the

Az training device, either because the task is truncated or

*simplified, then enter a "0" in the appropriate cell.

d 7. Repeat this procedure for each task element of the training

task.

8. Repeat the entire procedure for each training device or

device concept under consideration.

9. For each training device or device concept under consideration, total the number of "l"s (i.e., the number of task

elements for which the training device allows the trainee

to actually practice the training task).

10. Enter the total number of "l"s for each device in Table 27.

.11. Derive the task commonality index value by dividing the

-" total number of "l "s by the total number of task elements

in the training task plus the total number of task elements

*2 G-5

.* *

I" - ..... ;... '? .... ;: .. . : . . . .J . .. .. . .... % ... ....-. ., -, -.-. . . ,:,... , - ..
unique to a device which are not related to the training

requirements. The number of elements unique to a device

and not needed is included in the formula to reduce the final

effectiveness score for those devices offering greater capability (and therefore cost more) than actually required.

A separate task commonality table is used for each task. How faithfully the

element is represented in a training device is considered separately.

Physical Similarity--A physical similarity analysis is based on the physical

similarity or fidelity of displays (cues) and controls (responses) in a training

device relative to those in the operational equipment. The following procedure is applied to each task:

1. List the controls and displays involved in task performance

down the left column of Table 28.

2. List the training devices or training device concepts to be

assessed across the top of Table 28.

3. For each control or display listed in the left column of

Table 28, rate how well it is physically represented in each

training device or device concept under consideration. Base

your ratings of physical similarity on the four-point scale

given in Table 29.

In some cases, additional elements which aid training may be present (for

example, augmented feedback). The present use of "unique" element does

not include these training-related extra elements. If extra elements are

*present which enhance training of the required task, these should be noted

for later reference in overall evaluation of the device concept.

- G-6

. .. . . . . . . . -.-
0 0 C7T 77 w

TABLE 28. PHYSICAL SIMILARITY ANALYSIS

NAM: OF TASK_____________ _____

REQ 'JIRED TASK

CON 'ROLS AND DISPLAYS DEVICE A DEVICE B DEVICE C

2.

3.

* - 4.

5.

6.

7.

9.

10.

* ~~~~~Sum of Physical Simi~arity Ratings _______ ________ ________

Physical Similarity Index

Physical Similarity = _________ Sum of Similprity Ratings for Device

*Index 3(= of Ra'd Cont. 3t. Disolaysl = of Controls & Discilays Uniaue to Device)

G- 7
TABLE 29. PHYSICAL SIMILARITY RATING SCALE

Rdting Definition

3 Identical. The trainee would not notice a difference between

the training device control or display and the operational

control or display when he moves from the training to the job

situation. Include for consideration the location, appearance,

feel, and any other physical characteristics. Ignore the amount

and quality of inforuation transmitted.

2 Similar. There would be a small noticeable difference for the

t'rainee between the training device control or display and the

operational control or display, but he would be able to perform

the task. There might be a decrement in performance, but any

such decrement would be small and readily overcome.

-. 1 Dissimilar. There would be a large noticeable difference quite

apparent to the trainee, between the training device control or

display and the operational control or display and a large performance decrement, given that the trainee could perform at all.

Specific instruction and practice would be required on the operational equipment after practice on the training device to overcome the decrement.

--.- 0 Missing. The control or display is not represented at all in the

training device.

0

- .G-.°

* - * - .- - -- - -- .--- -- - --
i."7

4. Repeat the procedure for each training device or training

- "' device concept under consideration.

5. For each training device or device concept under consideration add the ratings for each control and display listed

to obtain a total score.

6. Enter the total score for each device in Table 28,

7. Total the number of controls and displays unique to a

device or device concept in addition to the required set

of controls and displays. These are controls and displays

unrelated to the training requirements (see previous

footnote).

8. Derive the physical similarity index value by dividing the

sum of physical similarity ratings by three times the number

of controls and displays plus the nurber of controls and

displays unique to a device or device concept.

9. Enter the physical similarity index value in Table 28.

A separ: te physical similarity table is used for each task. How faithfully

the training device represents the functioning of the operational equipment

is considered below.

Functional Similarity--The functional similarity analysis compares the

"*" operator's behavior in terms of the information flow from each display to

- the operitor, and from the operator to each contro-. The assessment

is made in terms of the amount of information transmitted from each

display to each control and the type of inforrnation--rocessing activity

perform -d by the operator. The issue is not the physical fidelity of a

* G-9

- "- .* • , .. . . . . . . . . . . ..
-. ontrol or display, but whether the operator acts on the same amount of

information in the same way in both operational and training situaions.

Controls and displays are considered in conjunction with task elements to

determine the type, amount, and direction of information flow occurring

during task performance. Each situation in which a display transmits

information to the operator (e. g., reads it) is defined as a stimulus function.

while each situation in which the operator transmits information to a control

(e. g., operates it) is termed a response function.

In each, the amount of information may be estimated by the number of

states or discrete values which the display or control may assume. The

- - functional similarity is assessed by comparing the number of discrete

values or states in the training situation to the corresponding number

- in the operational situation. The range of values may be partitioned into

four major levels: 1) continuous (essentially unlimited), 2) multivalued

- (some discrete value greater than two), 3) binary (two values), or 4) not

represented. Table 30 is used in performing functional similarity

analyses.

1. List the controls and displays involved in task performance on the

operational equipment down the left column of Table 30.

- 2. List the training devices or training device concepts to be

assessed across the top of Table 30.

3. For each control or display listed in the lcft column of Table 30

note how well it is functionally represented in each training

device under consideration. Base your ratings of functional

*Q similarity on the four-point scale given in Table 31.

SG-] )

. -- .7
TABLE 30. FUNCTOCNA L S n-1IIL-ARI TY ANA

NAME OF TASK________________ _____

REQUIRED

TASK CONTROLS AND DISPLAYS DEVICE A DEVICE B

2.

3.

4.

5.

6.

7.

* 8.

9.

10.

Sum of Functional Similarity Ratings

Functional Similarity Index

FucinlSimilarity Sum of Similarity Ratings for D-.viceIndex 3(1: of Rq'd Controls & Displays) (:t of Controls & Displays
'0

,° !0 

TABLE 31. FUNCTIONAL SIMILARITY RATING SCALE

Rating Definition

3 Identical. The number of states in the training situation

is the same as the number of states in the operational setting.

2 Similar. The number of states in the training situation is at

least half of the number of states in the operational setting.

1 Dissimilar. The number of states in the training situation is

less than half of the number of states in the operational setting.

0 Missj. The control or display is not represented at all in

te training device.

!-G1

• - - .o---
4. Repeat the procedure for each training device or training

device concept under consideration.

5. For each training device or device concept under consideration add the ratings for each control and display listed

to obtain a total score.

6. Enter the total score for each device in Table 30.

7. Total the number of controls and displays unique to a device

or device concept in addition to the required set of controls

and displays, if any such controls or displays exist (see

previous footnote).

8. Derive the functional similarity index value by dividing the

- sum of functional similarity ratings by three times the number

.- of controls and displays plus the number of controls and

displays unique to a device or device concept.

* 9. Enter the functional similarity index value in Table 30.

A separate functional similarity table is used for each task.

0 Skills and Knowledges Requirements--This analysis assesses the skills

and knowledges in the student's repertory before training and compares them

to the skills and knowledges required for successful performance of the

training tasks. Table 32 is used in performing this analysis.

1. List the skills and knowledges required fo,- successful task

performance down the left columi of Table 32.

0" G- 13

F. '; ..-..... - -. .* .'. ;" .- - - -." )" - " " '- - " - " .-- " - *""
Table 32. Skill and Knowledge Requirements Analysis.

NAME OF TASK__________________

RE~QUIRED TASK AFTER BEFORE DIFFERENCE

SKILLS AND KNOWLEDGE TRAINING TRAINING (AFTER - BEFORE)

2.

N 3.

4.

5.

6.

7.

9.

* . 10.

Total Differerice Score -

* Skill & Knowledge--

Requirements Index

Skill & Knowledge = tQ!i Duffersce Score

Requiremnents Index 4 (Number of Skills & Knowledge)

* G- 14
'.

2. For each skill or knowledge listed in the left collinmn of

Table 32, estimate how "much" of the skill or knowledge

the average trainee could be expected to have upon first

exposure to the training device, and estimate how much of

the skill and knowledge he must possess at training's completion. Use the two scales in Tables 33 and 34.

3. Compute the differences between skill and knowledge requirements before and after training by subtracting the

before-training value from the after-training value for each

skill or knowledge. (Negative differences are set equal to

zero, because they indicate the trainee enters with more

skill than necessary).

4. Add the difference scores ("after" minus "before" ratings) for

the skills and knowledges listed to obtain a total difference

score.

5. Enter the total difference score for each device in Table 32.

6. Derive the skill and knowledge requirements index by

dividing the total difference score by four times the number

of skills and knowledges.

7. Enter the skill and knowledge requirements index value

in Table 32.

A separate skill and knowledge requirements table is used for each task.

Task Tr dning Difficulty--A task training difficulty analysis estimates the

difficult,. (in terns of training time) of training soldiers to successfully

G- 15

€ ............................................
°. 77

TABLE 33. RATING SCALE FOR SKILLS AND

KNOWLEDGES BEFORE TRAINING

Rating Definition

4 Has a complete understanding of the subject or skill. Can do

the task completely and accurately without supervision. Has

received "skill" training.

3 Understands the subject or skill to be performed. Has applied

part of the knowledge or skill either on the actual job or a

trainer. Has done the job enough times to make sure he can do it,

although perhaps only with close supervision. Has had "proce- dural" training.

2 Has received a complete briefing on the subject or skill. Can use

the knowledge or skill only if assisted in every step of the

operation. Requires much more training and experience. Has

received "familiarization" training only.

1 Has only limited knowledge of this subject or skill. Has not

actually used the information or skill. Cannot be expected to

perform. Has had "orientation" only.

0 No experience, training, familiarity, etc., with this skill or

knowledge. Cannot perform a task requiring this skill or

knowledge.

G-16
.. ..,, . ,, . . . ... . .. .. .. ,7 

7 1.

TABLE 34. RATING SCALE FOR SKILLS AND

KNOWLEDGES AFTER TRAIING

Rating Definition

4 Should have a complete understanding of the subject or be

highly skilled. Is able to perform the task completely,

accurately, and independently. Has had "skill" training.

3 Should have an understanding of the subject or skill to be

performed. Has applied part of the knowledge or skill on the

actual job or a trainer. Has done the job enough times to

make sure he can do it although perhaps only with close supervision. Needs more practice under supervision. Has had

"procedural" training.

2 Should have received a complete briefing on the subject or

task. Is able to use- the knowledge or skill only if assisted in every step of the operation. ReqLires much more training and experience to be able to perform the task independently.

Has had "familiarization" training.

1 Should have limited knowledge of the subject or skill. Has

not actually used thi information. Is not expected to perform

the task. Has completed "orientation" training.

0 At the end of training, the trainee should have no experience

or training.

0 G-17
-. . . - .

'- 77",

uxuL training occurs on the operational equipment. Use Table 35 in performing

this analysis.

1. List the task elements down the left column of Table 35.

2. For each task element listed, rate the amount of training time

for surmounting its associated deficit relative to the most timeconsuming task element. Table 36 is used.

3. Add the training difficulty ratings for the task elements listed

to obtain a total training difficulty score.

4. Enter the total training difficulty score in Table 35.

* . 5. Derive the task training difficulty index by dividing the total

training difficulty score by four times the number of task

elements.

6. Enter the task training difficulty index value in Table 35.

A separate task training difficulty table is used for each task; however, the

rating scale is used by evaluating training time for each task element relative to the most time-consuming element for all tasks in the current analysis.

Index of Training Device Effectiveness--The five analyses just completed

are used to derive overall indexes of effectiveness for each training device

or training device concept under consideration.

O" Task commonality

0 Physical similarity

. Functional similarity

G-18

. . .. . - . .. .: ... - .:. . : . .. . . . . .,: -. * -. , - -,: , -, .

• .,,,.......... ........... .....:....... .. .. . . . . . . . .
-. 7 7 e

TABLE 35. TASK TRAINING DIFFICULTY ANALYSIS

NAME OF TASK __________________

REQUIRED TASK ELEMENT TRAINING DIFFICULTY

2.

3.

4.

5.

6.

7.

8.

9.

10.

Toaorinnzifcut cr

TsTrnngTotal Training Difficulty Score

Difficulty Index 4(Number of Task Elements)

- - .
TABLE 36. TASK TRAINING DIFFICULT' RATING SCALE

Rating Definition

4 Requires as much time to train as the most

time-consuming task element, considering all

task elements for all tasks in the current

analysis

3 Requires substantial training time, but less

than above

2 Requires a moderate amount of training time

relative to the most time-consuming task element

1 Requires only minimal training time relative

to the most tirme-consuming task element

0 Requires no training time

..

0.: G-2 0
1W

. Skill and knowledge requirements

* . Task training difficulty

The task commonality, physical similarity, and functional similarity analyses

were conducted on a task-by-task basis for each training device or training

device concept. The skill and knowledge requirements and task training

difficulty analyses were independent of any particular device and addressed

specific personnel and training requirements. The following procedure

- - is used to derive task level training effectiveness indexes.

1. List the tasks requiring training down left column of

Table 37.

2. Obtain task commonality, physical similarity, and func-

* tional simlarity index values de-ived in previous analyses

(Tables 27, 28, and 30) for each task listed in Table 37.

3. Enter these values in the appropriate columns of Table 37.

4. For each task, add the listed task commonality, physical

similarity, and functional similarity scores and divide by

'hree. The obtained value indicates the degree of correspondence between the operational equipment and the

particular training device for that task.

5. Repeat this analysis for each training device.

G-21

,'-'. A,,,?: - ,?b -.--. , .- - .. -- - ,- -,,
TABLE 37. TRAINING DEVICE AND OPERATIONAL

EQUIPMENT TASK LEVEL CORRESPONDENCE

NAME OF DEVICE___________________

TASK PHYSICAL FUNCTIONAL TC PS ~-FS

COMMONALITY SIMILARITY SIMILARITY 3

REQUIRED TASKS (TC) (PS) (FS)

- *2.

3.

4.

5.

7.

9.

-0 10.

G-22
Fhe next analysis summarizes the personnel and training requirements

Lnlyses.

1. List the tasks requiring training down the left column of Table 38.

2. Obtain the skill and knowledge requirements and task training

difficulty index values derived in previous analyses (Tables 32 and

35) for each task listed in Table 33.

3. Enter these values in the appropriate columns of Table 38.

4. For each task, add the listed skill and knowledge requirements and

task training difficulty index values and divide by two. The obtained value indicates the extent of training required based on

skills and knowledge requirements and abilities of the typical

trainee.

The final analysis derives overall training device effectiveness indexes for

each device or device concept by combining the task level training device

and operational equipment correspondence analysis and the task level

personnel and training requirements analysis.

1. List the tasks requiring training Jown the left column of Table 39.

2. List the training devices or traini,:,E device concepts to be assessed

across the top of Table 39.

3. Obtain the TC+PS+FS and SK.R+TTD values for each task and for

3 2

for each training device from Tables 37 and 38 and enter them in

the appropriate columns of Table 39.

, -23
TABLE 38. TASK LEVEL PERSONNEL AND

TRAINING REQUIREMENTS

NAME OF DEVICE__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

SKILL&S KNOWLEDGE TASK TRAINING

REQUIREMENTS DIFFICULTY SKR + TTD

TASKS (SKR) (TTD) 2

2.

3.

4.

5.

7,

* 8.

9.

10.

G2 4
Table 39. Example Training Device Effectiveness Index.

TASKS DEVICE A DEVICE 8 DEVICE C

2.

Ad ust Fire .43 .8o .34~ .70 .B0 .5(o*9 s~a So,66

3.

4.

7.

6.

-. 7.

* 0

Su opPodct

Su-o SY - - - --

* . 62

TriigDvc 5 8 Efetvns Inde

9. iin evc

Effctve-s - - of - e -eu -a-ks

Ine10.o qdTss +( fTssUiu oAlentv eieCnet

N u fPou~ 8~-.I~~9~ .3 _ _-25
4. Multiply the TC + PS + FS and SKR + TTD. vaLues for each

32 task and enter in the appropriate coUmn labeled product.

5. Add Ulhe product values for all tasks and enter the sum in the

sum of products location for that device.

6. Add the SKR + TTD values for each task and enter the sum in

the sum of Sh~ + TTD location for that device.

2

7. Divide the value from Step 5 ("A") by the value from Step 6

V ("'B"). This value is the training device effectiveness index

for that training device concept.

8. When comparing alternative existing device concepts, an

* effectiveness index is adjusted by multiplying it by:

,-

# ofTRequiredoTasks

"tof Required Tasks) + (# of Tasks Unique to the Alternative)

This factor accounts for capabiities within an existing device

that are not required for the device under consideration. Capabilties not required mean additional cost, and this correction

factor adjusts the effectiveness score to reflect a loss of effectiveness due to unnecessary cost. Where a theoretical device

concept is being considered, there probably will be no tasks

unique to it and the correction factor is not used.

9. Repeat the procedure for each training device or training

"- tdevice concept under consideration.

V G-26

• I .:
"h,. training device effectiveness index will take on a value of "I" if there

-s a perfect correspondence between the training device and the operational

equipment. The effectiveness index will take on a value of "0" when there

Is no correspondence between the device and the operational equipment.

Example 3f Training Device Concept Formulation

The following example illustrates application of the procedures for comparing alternative training device concepts. Three'devices are compared.

For purposes of this example assume that devices A and B are existing

devices and device C is a theoretical training device concept. Two task

training requirements (Table 40) are included in this example. Task

commonality (Tables 41 and 42), physical similarity (Tables 43 and 44) and

functional similarity (Tables 45 and 46) analyses are performed for each

device for each task. Skill and knowledge requirements (Tables 47 and 48)

and task training difficulty (Tables 49 and 50) analyses are performed for

each task. Adjustments are made for task elements, controls, and displays

unique to each existing device. Tables 51 through 55 compute effectiveness

scores for each device. The effectiveness score for device C is the highest.

* Therefore, the theoretical device concept should be developed further.

The effectiveness score for device B, an already existing device, is lower;

however, its score is sufficiently high to conduct further evaluation. Using

* an existing device may result in cost savings that would compensate for its

lower effectiveness score. Validation testing could be used to. determine

whether device B or device C is more appropriate.

G-27
.. , --..

4.

I

4.

.4

4.

-~ THIS PAGE INTENTIONALLY LEFT BLANK

V

4.1

4.'

4.

4..

S.

4., 4.

4.

4.

S.*

J4

S.'

G- 23

eli

.e~ ~S.'.-.~??-~ .4'.-.. .- '.'X~ . . 4~*~ -..- ~.. ~ *
"I

APPENDIX H

TRAINING CONCEPT DESCRIPTIONS*

V-..

*Taken from TRADOC Training Effectiveness Analysis Handbook,

Appendix 1.

H-1

..

-p

, . - * * .. . ?*~. * * * ... .-.. -' * .. - - * * -* * 
General SECTION 2 - INSTITUTIONAL TRAINING CONCEPT

4% The data required to develop the

institutional training costs (ITC) are listed

below. This fomat is provided to facilitate

the definitional process required to develop

costs for institutional training concepts or

courses of instruction. The requirements herein should be completed for each alternative concept under consideration and for each course

impacted.

1.0 Course(s) Impacted

1.1 Modification or New Course

1.2 Describe Changes to Current Courses of Instruction

2.0 Student Load Per Class Per Year

2.1 Average Grade of Student

2.2 Student Source

3.0 Class Frequency Per Year

4.. 3.1 Length of Class

3.2 Fiscal Year of Course Start

4.0 Instructor Requirements Per Affected Subcourse(s)

5.0 Support Personnel Requirements

', *6.0 Expended Equipment Per Class (Subcourse)

7.0 Non-expended Equipment Used and Usage Rate Per Subcourse

8.0 Exportable Training Used

, 8.1 Exportable Software

8.2 Training Teams

8.3 Exportable Hardware

9g. Facility Requirements

Exampl e

An outline and an example of a hypothetical institutional

training concept is shown below.

H-2

. ',f . '- '.. --. , . . ,_, ,.,, '., ., . .- . . . . . . . ..
74. 1.7 - 771

SAMPLE: INSTITUTIONAL TRAINING CONCEPT (HYPOTHETICAL)

Training Device - X99 Mortar Crew Trainer (MCT)

1.0 Course(s) Impacted

The X99 MCT will be utilized in the 11CIO course.

1.1 Modification or New Course

This concept will modify the current course.

1.2 Describe Changes to Current Course of Instruction

The current live fire crew drill subcourse will be replaced by

"" instruction of the X99 MCT. This will reduce the time required

for this subcourse and will reduce the total rounds fired and

personnel support requirements.

2.0 Student Load Per Class Per Year

Year

1 2 3. 20

50 40 30 30

2.1 Average Grade of Student

The average class consists of 10 percent E3, 80 percent E2, and

10 percent El; therefore, the average grade of students is E2.

H4',

H-

_ . . * p*il%* ~ *'S 'p

* . - - . - * ' p ) * . P*"~ p . -p . '

4, 5
2.2 Student Source

Students in the 11C1O course are in AIT phase of OSUT.

3.0 Class Frequency Per Year

Year

1 .2 3_ .. 2

*520 17 12 12

3.1 Leivgth of Class

Current Modified

Full Course 10 weeks 9 weeks

Crew Drill Subcourse 3 weeks 2 weeks

3.2 Fiscal Year of Course Start

The modified 11C1O course utilizing the X99 MCT will be

feasible in FY 87.

4.0 Instructor Requirments Per Affected Subcourse(s)

Crew Drill subcourse instructors per Class:

Current Modified

11C40 E7 160 Manhours 110 Manhours

11C30 E6 160 Manhours 110 Manhours 16~.

118 03 80 Manhours 50 Manhours

118 04 80 Manhours 50 Manhours

H-4.
5.0 Support Personnel Requirements

The modified subcourse will require no support personnel;

however, since the simulated crew drill replaces several live

fire exercises, the following personnel, currently required,

wil l be freed for other purposes:

Current Per Class

11C30 E6 480 Manhours

11CZO E5 960 Manhours

11C20 E4 1920 Manhours

6.0 Expended Equipment Per Class (Subcourse)

Current Modified

Cartridge M374 800 (250) 550 (0)

Cartridge M57 200 (25) 175 (0)

Cartridge M301 50 (0) 50 (0)

Cartridge M362 85 (5) 80 (0)

Cartridge M375 100 (100) 0 (0)

Soldiers Manual lC 50 50

7.0 Non-expended Equipment Used Per Subcourse

Current Modified

M151 1/4 T 160 miles 10 miles

26 Passenger Bus 320 miles 20 miles

M29AI Mortar 380 RDS 0 ROS

M35 2 1/2 T 160 miles u miles

PU-619 Gen 80 hours 0 hours

H-5
*-Instructors Manual 11C 10 10

Instructors Manual X99 0 10

8.0 Exportable Training Used

None

8.1 Exportable Software

None. (This would be applicable to training packages sent to

the field such as manuals, training films, circulars,

bulletins, cassettes, etc.)

8.2 Training Teams

None. (If teams are sent fromt the institution to the units to

provide training on new equipment, doctrine, or tactics, this

section will reflect team composition, schedule, units trained,

personnel trained, and assets utilized during training.)

8.3 Exportable Hardware

N@..

None. (This section will describe quantities and types of

hardwere to be provided to units by institutions to support

training requireents for the alternative.)

*9.0 Facility Requirements

Classroom Ranges

Institution 3000 Sq Ft*

N .

Permanent brick None

Unit None None

*2,000 Sq Ft exist; 1,000 Sq Ft require construction.

or."8 IEpraleSfwr

N .le-s

& h il uha aultann imcruas

bultn, astes t.
SECTION 3 - UNIT TRAINING CONCEPT

%eneral

The unit training concept should also be defined by the proponent

and is outlined below. This format is provided as a means of capturing

pertinent unit training factors for use in calculating training costs.

The outline herein should be used to define a generic unit training

r- routine for each training alternative of interest.

1.0 Training Alternatives

2.0 Training Events (This section to be completed for ea-h

Generic Training Event)

2.1.1 Training Event 01 (Title, Soldiers Manual, or ARTE

Reference)

2.1.2 DescrIption of Training Conducted

2.1.3 Level of Training Aggregation (Squad, Plt, Bn, Division)

2.1.4 Instructor Personnel Required and Parent Unit (Organic to

Level of Training Conducted)

2.1.5 Personnel Trained (Quantity, Grade, MOS)

2.1.6 Location of Training, Distance from Unit Area, Means of

Transportation

2.1.7 Equipment Utilized and Rate Per Event (Miles, Hours, etc.)

2.1.8 Facility Requirements (Classrooms, Ranges, Motor Pool

Buildings, etc.)

H-7

. L •

|..........................
2.1.9 Time Requirenents Per Training Event

2.1.10 Frequency of Training Event Repetition (Times Per Month,

Quarter, or Year)

2.1.11 Support Personnel and Equipment Requirements (Personnel

Other than Students and Instructors)

2.1.12 TDY Requirements for Training Event #1

2.2.1 Training Event #2 (Title)

2.2.2 Description of Training Conducted

2.N.12 TDY Requirements for Training Event #N

Exampl e

The following is an example of a hypothetical unit training

concept.

i

I . SAMPLE: UNIT TRAINING CONCEPT (HYPOTHETICAL)

1.0 Training Alternative

The unit training concept definition is the program which is

anticipated to be used with the intensified simulation

training for REDEYE gunners (hypothetical).

2.0 Training Events #1 through #15

2.1.1 Training Event #1

Engage Hostile Aircraft; Task 1057, FM 44-16P

H-8

.. -
* '- 2.1.2 Description of Training Conducted

Gunners engage simulated hostile aircraft in a combat

environment utilizing the moving target simulator (MTS).

Each gunner will track 25 aircraft, scoring simulated hits

on no fewer than 80 percent of engageable targets.

2.1.3 Level of Training Aggregation

Training event will be conducted at the REDEYE section level

for each battalion.

2.1.4 Instructor Personnel Required and Parent Unit

Instructors will be those organic to the local or supporting

MTS and will not be part-of the unit trained. In general,

the instructor requirements will consist of one E6 16P30 per

section trained.

2.1.5 Personnel Trained (per battalion)

16P10, E4, 5 each

2.1.6 Location of Training, Distance from Unit Area, Means of

Transportation

MTS facilities are located in each major troop concentration

geographic area such as Ft Hood, Ft Riley, Ft Bragg,

Ft Bliss,- 7th Army ATC, Ft Lewis, and Camp Casey. The

average REDEYE section would be required to travel approximately 75 miles one-way to reach the MTS facility.

Organic section vehicles would be used to and from the MTS.

,0H

"% H- 9

- -. - - - . - - - - - - . - . *
. 2.1.7 Equipment Utilized and Rate Per Event

PMTS 8 Hours Per Event

Tracking Head Trainer 16 Hours

1/4T Truck 300 Miles Per Event

2.1.8 Facility Requirements

MTS Screen

2.1.9 Time Requirements Per Training Event

8 Hours Per Section

2.1.10 Frequency of Training Repetition

The engagement simulation training will be conducted at

least monthly for each REDEYE section or 12 times annually

for each gunner.

2.1.11 Support Personnel and Equipment Requirements

" GS-9 Electronics Technician 3 Hours Per Training Event

2.1.12 TDY Requirements Per Training Event #1

Three REDEYE sections are located in isolated areas from

the nearest MTS facility and will require three days per

diem pay for each training event plus commercial air

transportation costs of approximately $135 per man per

event.

H-10
2.2.1 Training Event #2

Visual aircraft recognition; Task 1040, FM 44-16P

a#

a

2.N.10 Frequency of Training Event Repetition

Task 1061, determination of aircraft category for REDEYE

ranging will be conducted weekly per section using the

modified GOAR kit.

0

2.N.12 TDY Requirements Per Training Event #N

None

SECTION 4 - TRAINING SYSTEM REQUIREMENTS SPECIFICATION (TSRS)

General

This format is provided to facilitate the definitional process

required to develop costs for training devices or systems. It is

intended as a means of standardizing required input to preclude both too

*~ little and too detailed input.

H-.

".aiomti rvddt alliaetedfntoa rcs

reurdt eeo ot o rann eie rsses ti

-aitne a en f tnadzngr.i..inu opeluebt o

1 itte toodetaledH-ilt an 

* a...a ''-a ***. . . . . . - i *:.. a 
. r- r- 7. r - r

1.0 Hardware Description

1.1 Physical Characteristics (dimensions, weight, space

requi rements)

1.1.1 Functional Characteristics (task flow, operating layout,

input/output description)

1.2 Power Requirements (kw/hr, volts, ac/dc)

1.3 Calibration Requirements (level, location, frequency)

1.4 Maintenance hours per unit of operation (hr/yr, hr/mile,

hr/round, etc.)

1.4.1 Maintenance Level and Location

1.5 Depot Overhaul Frequency (overhauls/time period)

1.6 Computer Requirments

1.6.1 Type (general or specialized, time share of dedicated,

available or required)

1.6.2 Storage Capacity (main core and peripheral)

1.6.3 Compiler (available or not available)

1.6.4 Word Size (bits per word)

1.6.5 Software Requirements

1.7 Hardware Replaced and Replacement Ratio

1.8 Personnel Repl aced

1.9 Projected Operational Life

2.0 Empl oyment Concept

.H,

H-12

.

.4.

,D,.'.Ve:r:, .'.-. '-:."".:T,//v .-. .' .;.. ....... -. .' -, . .""-."-.>/:,,-,.. . . . . .-.. . .. . .... . . . .". .-.-.. . . . . - .. - . . .-... . .".,'.".',-
2.1 Annual Operating Factor (hours, miles, rounds, use per

training event/frequency of events)

2.2 Operators Required (grade, MOS, quantity)

2.3 Level of Use

2.3.1 Institutional (BCT, AIT, NCOES, OBC, AOC, etc.)

2.3.2 Unit (individual, squad, platoon, etc.)

2.4 Basis of Issue

2.4.1 Institutions

2.4.2 Units (Active and Reserve)

2.5 Float Requirement Ratio (float devices per each operational

device)

The TSRS provides the minimum information required about the system

before cost analysis can take place. The source of this information

would primarily be the proponent with assistance from DARCC#4. The

general intent of the TSRS is to define the system in such detail as

to a allow the analyst to define and direct his cost effort.

Exampl e

An outline and example of a hypothetical TSRS Is shown below.

SAM4PLE: TRAINING SYSTEM REQU1REMENTS SPECIFICATI.ON (HYPOTHETICAL)

SYSTEM: Fire Control Simul ator BT33

JV1.0 Hardware Description

H- 13
1.1 Physical Characteristics

Screen Size: 6m x 1.5m

Central Unit: 4m x 1m x 1.5m

Slide Projectors: .5m x .5m .2m; 8mm (4 ea)

Space Requirements: 3000 sq ft classroom

Student Capacity: 50

1.1.1 Functional Characteristics

The BT33 simulates artillery fire and adjustment of fires on

stationary and moving targets. The battlefield panorama

including the target area is projected on the screen with a

slide projector. The observer (students) can see the

projected area from the simulated OP. The instructor

indicates the target display projector. The students

observe the target and, using simulated communications

procedures, make inputs to the device operator (simulated

fire distribution center). The operator inputs data to

,. implement the student's calls for fire. The central unit

calculates the position of the bursts in the furtherest

area and projects the burst symbols onto the screen with

.- the burst projectors. The student may then adjust fire,

lift fire, increase fire, or report damage assessment.

1.2 .Power Requirements

220 VAC, 50-60 Hz, I kw/hr

-H.1

D H-i 4
1.3 Calibration Requirements

System is aligned quarterly, total manpower requirements

equal one manhour per quarter. Cal ibration performed by

local unit at user location, level c.

1.4 Maintenance Hours Per Unit of Operation

The BT33 required an average of one hour of direct

maintenance for every 40 hours of operatioai

1.4.1 Maintenance Level and Location

Organizational, DS, and GS maintenance are authroized to be

conducted by the qualified operator on station.

1.5 Depot Overhaul Frequency

The ST33 will require depot overhaul every 10 years. Depot

overhaul will require the BT33 to be transported to Red

River Depot, Texas.

1.6 Computer Requirements

1.6.1 Type

Specialized. The BT33 is complete with its own processing

unit which Is dedicated.

2 1.6.2 Storage Capacity

The core memory will store 4,096 words, each containing 16 "; bits.

P..

H-15
1.6.3 Compilers

None

1.6.4 Word Size

16 bits

1.6.5 Software Requirements

The BT33 contains 8am film cassettes to simulate targets

plus pre-programned fire mission commands controlled by the

operator consol e.

1.7 Hardware Replaced and Replacement Ratio

In the institutional environment the BT33 will replace four

puffboards (terrain mock-ups) and possibly replace live fire

exercises if no effectiveness degradation results. The

replacement of the puffboard will enhance realism while

the replacement of the live fire exercises could result in

cost savings.

1.8 Personnel Repl aced

None

1.9 Projected Operational Life

Twenty years with one depot overhaul.

H-16
S". 77 - - " . "

2.0 Empl oyment Concept

2.1 Annual Operating Factor

Institution: (24 hr per week - 52 weeks) 1,248 hr per year

Unit: (80 hr per month - 12 months) g60 hr per year

- 2.2 Operators Required

One per device minimum

Institution: E7 13E40

Unit: 03 13F

2.3 Level of Use

2.3. 1 Institutional

OBC, Advanced NCOES (13E)

2.3.2 Unit

DIVARTY FIST; maneuver unit leaders courses

2.4 Basis of Issue

2.4.1 Institutional

USAFAS: 2

2.4.2 Units

2.5 Float Requirement Ratio

4H-1.

:,....H-i 7

," .' . . . .. . .- .. ' "'..''. "'"'. -.-. ''-:i b":.,... -.,-.,L T. - . * - - .
SECTION 5 - LIFE CYCLE COST ESTIMATE (LCCE).

The LCCE considers all costs incurred during the projected life of

the training system and/or subsystem. The LCCE includes the cost to

acquire, operate, and maintain the training hardware (device) over its

useful life. Materiel system life cycle cost includes all costs

associated with the three life cycle phases: research and development

(R&D), investment, and operations. Depicted in Figure I-i is a profile

of the life cycle of a materiel system and cost phases.

,.',..

V,.:

-,...;

. .-. 9

• .-. " -

i .

H..1

.%,

~H-i8

.. 'Z ./ - . -.--.. ', . . """ """ ' " - p. ''" "' "- * " " " " ' " "-"" ' "
APPENDIX I

HRTES PROCEDURES FOR IDENTIFYING CT TEST ISSUES

This appendix describes HRTES procedures for developing OT

test issues. The description of these procedures was taken

from the HRTES Prototype Handbook, pages 123 to 175.

*0

AAW
HUMAN PERFORMANCE MEASURES

In this section, you will identify the performance measures to take during

the operational test. Read the general section on performance measures

(time and accuracy). These will reference the human performance functions

you gathered from section 6. After completing this section you should

have developed specific measures of human performance which can be used in

the operational test.

NEXT: Turn to Test Conditions page 1-63).

1-2

9-. Ii I

'.. , * -. 
HUMAN PERFORMANCE MEASURE 1

TIME-DURATION OF HUMAN PERFORMANCE FUNCTION

RATIONALE

This is the==meisure of the length of time between the presentation

of that cue which marks the initiation of a Human Performance Function (HPF)

and the completion of that HPF, or thb cessation of HPF behavior, in the

absence of completion. This measure incorporates reaction time according

to the nature of the HPF-s measured in HRTES. If an HPF were "Target

Detection", then the duration of that HPF would be identical to reaction

time. Therefore, for the purposes of simplification and clarity, HPF's

are so structured that-duration is the only time measure required.

The length of time it takes to perform an HPF is important for two

reasons:

(1) Many HPF's can only be accomplished successfully as a

function of time. That is, if a given HPF were not completed

within a specified period of time, it would, by definition,

be conside-Rd a failure. In addition, the faster a given

HPF were completed, the greater would be the level of its

success.

(2) Comparab1 systems frequently have HPF's in common. Time to

perform an HPF is one measure of the difficulty of that HPF.

The larger the number of more difficult HPF's a given system

requi-reV,-Th ' ess desirable is that system, all other factors

being equA--Therefore, time to perform may be used to

differentiate the relative desirabilities of competing systems.

1-3
17

METHODOLOGY

T. Identify the cue which initiates HPF behavior,

1.1 In the first HPF of a series, this cue is normally the introduction

of a new stimulus, such as:

- 1.1.1 visual target

* 1.1.2 auditory signal

1.1.3 command

1.2 In later HPF's in a series, the cue is often the completion of

the previous HPF.

1.3 Example - Target acquisition:

1.3.1 target detection-presentation of target

1.3.2 target identification-completion of identification or, in

some cases, target-presentation

1.3.3 target location-presentation of target

1.3.4 aim/lock on-completion of target location

Frequently, the HPF's may not take place in as linear a manner as shown

above. For the purpose of the OT, it is necessary that recordable initiation

cues either be determined or be artificially introduced prior to the

start of actual testing.

-.... ......- -. * ,.- .
Jo.

2. Determine the smallest time interval required for measurement of the

given HPF.

2.1 For HPF's which consist entirely of reaction to a presented

stimulus, one tanth of a second is the smallest time interval

."-. required, unless otherwise stated.

2.2 For HPF's which consist of motor behavior or some combination of

motor and intellectual behavior, thesmallest time interval will

vary with the HPF. A reasonable rule of thumb is that the

smallest time interval required is one-tenth of the interval

used in stating the criterion.

3. Identify the incident which informs you that either the HPF has been

completed, or that HPF behavior has ceased.

*, 3.1 In the case of HPF completion, the end point is defined in

advance. Since this end point may be entirely intellectual

and not observable, provisions may be required to aid in end

point recognition.

3.1.1 target detection either operator switch closing, or

vocal signal upon detection, depending on the amount

of money and time available.

3.1.2 target identification - vocal signal of the nature of the

target, or prestructured switch coding indicating nature

of target.

3.1.3 target location - vocal signal of target coordinates,

measured from initiation of vocal response.

1-5
3.1.4 weapon aiming completion of each aiming incident,

measured to complEtion - not initiation of aiming.

3.2 In the case of the cessation of HPF behavior, in the absence

of completion, three general methods are available:

3.2.1 vocal signal

3.2.2 switch closing

3.2.3 observation - This is the most questionable technique,

but in the confusion which often accompanies cessation

without completion, it may be necessary to use this

approach to avoid complete loss of data.

4. Select conditions in which HPF is to be run.

NOTE: It may be desirable to append the most important of these

conditions directly to the selected SPI, thereby creating a

composite SPI which emphasizes that condition. See appropriate

Note accompanying the SPI's for further information on this

subject.

4.1 There are ten classes of conditions:

4.1.1 weather 4.1.6 ground surface

4.1.2 light 4.1.7 ground slope

4.1.3 target 4.1.8 obstacles/concealment

4.1.4 personnel 4.1.9 tactical

4.1.5 operational 4.1.10 training

For a more detailed analyis of these classes of conditions, and a system

for ranking them for criticality, see the Condition Ranking System on

page

1-6

* . . .
L -

5. Determine the number of subjects to be run in the test to collect the

measure.

6. Determine the number of trials to run per subje:t.

7. Determine the method for collecting the data.

7.1 There are three basic problems which impact the collection of

data: -.1

7.1.1 How much money is available?

7.1.2 How much time is available for preparation?

7.1.3 How much space is available in the system?

-,4 7.2 These three problems interact with each other. If neither space,

time, nor money is a serious issue, it is best to:

7.2.1 include a trained observer with each system in the test

7.2.2 fully instrument each system in the test so that HPF cues

indicating initiation and completion, as well as time

measures, would be automatically recorded.

7.3 If there is no room for an observer, instrumentation is a firm

requirement.

7.4 If there is room for an observer, but not sufficient funding for

full instrumentation, it is best to train the observer as fully

as possible.

7.5 If there is room but insufficient funding for full instrumentation,

and insufficient time for complete observer training, it is best

to select those individuals to be observers who are either

experienced in operational testing, or who learn quickly and are

adaptable.

1-7
7.6 One factor is clear: one must collect data. The more objective

the method of collecting the data, the better quality it is likely

to be. It is preferable to minimize reliance on operator reports

for objective data collection. Therefore, systemns to be

* * investigated shouli be instrumented and/or supplied with observers.

1.-..

a .

.

a, w

"...,:-:
HUMAN PERFORMANCE MEASURE 2

, ACCURACY OF HUMAN PERFORMANCE FUNCTIONS

RATIONALE

This is a group of measures relating to the accuracy with which

personnel carry out Human Performance Functions (HPF's). An HPF may be

either omitted or performed. If it is performed, the accuracy of its

performance is variable. Based on judgement, one point on this continuum

may be selected as the lowest level of acceptable performance. This

point becomes the criterion. All performance superior to the criterion

is desirable, but may cost more to achieve than is warranted by system

requirements. All performance inferior to the criterion is undesirable,

and that undesirability increases as the level of accuracy decreases.

:1

Determination of HPF accuracy serves three functions:

(1) Analysis of System Performance: It provides important data

for the analysis of total system performance, leading to an

understanding of the reasons that a system has not achieved

all expected performance criteria.

(2) Analysis of Human Performance: It provides data leading to

a.-~ an understanding of the bases of human performance inaccuracy

and the steps needed to reduce that inaccuracy.

(3) Human Performance Reliability: It provides data required

for the eventual projection of human performance reliability

in system operation.

NI4

1-9

I

. .,4,, ,- , , " -,, " ' - .' " " -,"-, ,', -, •"-" " .= L ° ' " . """"" '''
METHODOLOGY

1. Determine the criticality of the problem in question.

1.1 It would be iesirable if there were some objective method for

determining the criticality of problems. Unfortunately, this

can be done only through the use of judgment. It is clear that

some problems are more critical than are others, and that this

difference may provide some increased meaning to the output of

the OT. The following rating scale can be photocopies and given

-criticality.

, - --- - -niiul o -o -i prpit -h -eemnto -f -problem- in ----- ----

What effect would this problem have on performance of the HPF?

0 = No effect on performance

2

3 = HPF can be performed but has been moderately affected

4

5

6 = HPF cannot be performed adequately

It is desirable to have problem criticality rated by both a user in a job

classification equivalent to that of the individual involved in the problem

and by an individual in the position immediately supervisorial to the former.

2. Identify measures Which apply, in varying degree, to each HPF task.

NOTE: Each accuracy measure type is referenced by the appropriate problems

listed on the Human Performance Charts.

I-10
2.1 Measures of HPF accuracy deal with

(1) problems in performance

(2) performance effectiveness

(3) effects of inadequate performance. They include evaluative

measures which are applicable to other HPF measure areas

and will be so noted.

2.1.1 number of problems/deviations from accurate performance

2.1.2 size of problems/deviations

2.1.3 direction of problems/deviations

NOTE: The following 4 measures include omissions, sequence problems, and

accidents with accuracy measures:

2.1.4 system downtime due to all problems

2.1.5 percentage of HPF's run without problems

2.1.6 percentage of subcriterion trials of HPF's

2.1.7 number of repetitions required to perform HPF to

criterion

2.2 Measures of omission refer to the absence of either an entire HPF, or

one or more of its elements. Elements are.those finer pieces of

behavior which are performed to carry out the HPF.

2.2.1 number of HPF's totally omitted for a given System

Performance Issue

2.2.2 number of HPF elements omitted for a given HPF

2.2.3 number of omitted HPF elements for a given HPF by category:

2.2.3.1 cueing elements

2.2.3.2 perception elements

2.2.3.3 cognitive elements

2.2.3.4 motor elements

.j I-ii

:, , ,,,-.. o.- ,, , ... -,o .. &A: , . .. . j
- 2.3 Measures of sequence refer to the order in which HPF's and HPF

elements are performed. In some cases, sequence is a very important

,'.." measure, but in others it will be either meaningless or essentially

impossible to collect. If you are considering collecting sequence

-" measures, carefully consider if there really is a specific sequence

in which the HPF or its elements are to be performed.

2.3.1 number of HPF's performed out of sequence (percentage of

HPF's performed out of sequence for a given System Performance

Issue is also useful)

2.3.2 -number of HPF elements, for a given HPF, performed out of

sequence (percentage of HPF elements performed out of

sequence is also useful)

2.3.3 direction of sequence shift

2.3.4 number and sequence shift of HPF elements by category:

2.3.4.1 cueing elements

2.3.4.2 perception elements

2.3.4.3 cognitive elements

2.3.4.4 motor elements

2.4 Measures of accidents refer to both personnel and equipment

accidents..Accidents to personnel need not involve injury. They

do involve occurrences which lead to injuries. Accidents to

materiel need not involve damage. They involve incidents such as

04 dropping, striking, hitting, etc., which could be expected to

produce damage. Critical incidents are those occurrences in which

'"neithe- a personnel nor materiel accident takes place, but in which

such an accident almost takes place.

1-12

02
'I

2.4.1 number of accidents associated with HPF (personnel and

materiel)

2.4.2 number of injuries associated with HPF

2.4.3 number of man-hours lost, associated with HPF

2.4.4 number of materiel accidents resulting in damage, associated

with HPF

2.4.5 cost of replacement/repair of materiel involved in

accident associated with HPF

2.4.6 number of man-hours lost due to materiel accident associated

with HPF

2.4.7 number of critical incidents for personnel and for materiel

associated with HPF

3. Select conditions in which HPF is to be conducted.

NOTE: It may be desirable to append the most important of these conditions

directly to the selected SPI, thereby creating a composite SPI which

emphasizes that condition. See appropriate Note accompanying the SPI's for

further information on this subject.

' 1-13
* 3.1 There are ten classes of conditions:

3.1.1 training

3.1.2 weather

3.1.3 light

3.1.4 ground surface

- 3.1.5 ground slope

. 3.1.6 obstacles/concealment

3.1.7 target

3.1.8 personnel

3.1.9 operational

3.110 tactical

For a mori detailed analysis of these classes and a system for ranking them,

see the Condition Ranking System on page

3.2 It is assumed that OT will exercise the system under optimum

versions of probable conditions. It is also desirable for probable

worst case conditions to be exercised in OT. If one or more of

the conditions has been selected for inclusion in the System

Performance Issue, one should still examine the others. Inclusion

of a condition directly into a System Performance Issue does not

mean that no other condition is applied; it means that this

condition is of preeminent importance and must be tested.

4. Determine the method for collecting the data.

4.1 There are three general methods for collecting data:

4.1.1 observation

4.1.2 instrumentation

4.1.3 debriefing

4.2 Observation requires an observer who has some level of training

and space/time in which to work, as well as some sort of structured

data collection form.

1-14
L.

4.3 Instrumentation requires an understanding of the nature of

the data to be collected, a piece of appropriate performance

measuring equipment, a means for activating it at the right

times, a power source, and a place to put it. Debriefing

requires a subject with some information, a structured debriefing

form, a place/time for the debriefing and, in some cases, a

debriefing individual who has some experience and/or ability

in the area.

4.4 There are three basic problems which impact the collection of

data:

4.4.1 How much money is available?

4.4.2 How much time for preparation is available?

4.4.3 How much space, in the system, is available?

4.5 These three problems interact with each other. If neither

space, time, nor money is a serious issue, it is best to:

4.5.1 include a trained observer with each system in the

test

4.5.2 fully instrument each system in the test so that HPF

cues indicating initiation and completion, as well

as time measures, would be automatically recorded.

4.6 If there is no room for an observer, instrumentation is a

firm requirement.

1-15

-0
4-7 If there is room for an observer, but not sufficient

*-** funding for full instrumentation, it is best to train

the observer as fully as possible.

4.8 If there is room, but neither sufficient funding for full

instrumentation nor sufficient time for complete observer

training, it is best to select those individuals to be

observers who are (ither experienced in operational

testing or who learn quickly and are adaotable.

4.9 It is necessary that observers/instrumentation be so

placed and trained that they are able to record the

following sorts of information about HPF behavior:

4.9.1 What was the cueing stimulus? Did either incorrect

cueing, too general a cueing stimulus, too heavily

stressed an operator, or lack of operator

knowledge cause the HPF to be inaccurately performed?

4.9.2 Was HPF inaccuracy due to the operator's misperception

of either the environment or the components of the

system?

4.9.3 Did HPF inaccuracy originate in operator cognitive

processes?

4.9.4 Did HPF inaccuracy originate in the operator's

ability to apply his motor skills to the system,

as designed? That is, was the operator unable to

manipulate the system to accurately perform the HPF?

5. To maximize measure reliability, determine both the number of

subjects to be run and the number of trials per subject.

1-16

*A

';,

'* . . . .
SYSTEM

[mDEX

MISSION

CHARTS

SYSTEM

PERFORMANCE -

,SUES

EVALUTION VALUATICN CF

RF R ANSESTE.M/HUMAN SYSTEM DESIGNHUMAN PERFORMANCE EFRAC

% ~FUNCTIONS

DEVELOPMENT (RTTP XSS

OF FE NMHUMAN PERFORMNC

SYSTMDEIGUMASRESRS

EFFEAT OFNA

SYS~q DEIGNTEST

ON TRAINNVALONATIONSO

O 'T

-ESUTS

DIASNOSN ON

'PFRAIN

EVALEUAIOS

1-or

SIIAIYO6EUT
TEST CONITTONS

This section will help you to select the conditions under Which yvou should

* . measure performance. Notice that the conditions can apply to the System

Performance Issues, as well as to the test measures themselves.

11EXT: Turn to Human Resources Implications and Measures (page r.)

1-1
CONDITON CHART

The conditions, in the following ten condition classes; are those which

might negatively impact the operation of the system to be tested. It is

assumed that best case conditions will be tested in the OT. To insure

that probable and operationally significant worst case conditions are

tested, apply the Condition Ranking Scales to the appropriate conditions

on this Chart.

CONDITION CLASS CONDITION

Weather Clouds/Low Ceiling

Rain

Fog

Snow

Sleet

Head Wind

Cross Wind

Tail Wind

Continuous High Wind

Gusts

High Temperature/Low Humidity

High Temperature/High Humidity

Low Temperature/Low Humidity

Low Temperature/High Humidity

Light. Ambient Light/Dark Variation (Natural Light)

Flares and Other Artificial Light

Direct Glare (Natural Light)

Direct Glare (Artificial Light)

Reflected Glare from:

Water

Ice

Snow

Sand

Clouds/Fog

Ground Surface Sandy

Rocky

Vegetation Covered

Muddy

Loam Soil

Steppe

Frozen

Ice Covered

Snow Covered

Water Covered

Broken Ground

Plowed Fields

1-19

• " % ' , , ' . , " . ." - . " . . - ' - . .. - . .. - . - . . . -. . . . . - . . " . ., - . " . -. " . ,." . . , . .* . . . . - -.. " . .
CONDITION CLASS CONDITION

Ground Slope Flat

Hilly (Positive/Negative Slopes)

Mountains (Positive/Negative Slopes)

Obstacles/Concealment Dense Forest/Jungle

Hedgerows

Dense Brush

Sparse Vegetation

Water Obstacles/Concealment

Natural Rock

Holes/Di tches/Trenches

Dense Buildings/Walls

Bridges

Tunnels

Manmade Obstacles

Target Range

Size

Figure-Ground Contrast/Color

Sound

Speed

Direction

Altitude

Maneuver

Nature of Required Hit (Placement/Type/Number)

Operational Smoke

Concealed

Buttoned Up

100% Loaded

Damaged

Reduced Personnel

Personnel With low but acceptable aptitude

With low but acceptable demonstrated ability

With minimum acceptable characteristics

of size and strength

Extended periods of work without rest

(continuous combatfatiqued)

Wearing clothing and/or gear (e.g., winter

gear, NBC protective equipment)

O.

. . ." .... °
" CONDITION CLASS CONDITION

Training Following a period of time without training

or practice

Trained according to:

--- School Training

OJT

Unit Training

With little or not field experience with

this system

With little or no field experience with

any system in this generic class

Crew member not specifically trained for

that job

-.- Tactical:

Army systems are exercised according to the tactical theories -? prevalent at a given time. Tactics change, but the parameters

which they control remain relatively constant. The tactical

conditions to be exercised in OT will either be the complete

tactic or that sement of the tactic which is most significant for

the system under investigation. The following parameters of

tactics should be treated in the same manner as conditions and be

subjected to the Condition Ranking Scales. The resultant ranking

should aid in determining which tactics and tactic segments to

deal with in OT.

Movement in Attack

Movement in Defense

Movement in Travel

Utilization of Terrain/Weather/Light

Target/Target Part Selection

Weapon/Annunition Selection

Range Selection

Time to Fire

Unit Size

Unit Composition

Unit(s) Formation

Unit Coordination

Inter-unit Coordination

Length of Continuous Combat

i~i 1-. 1-21
CONDITION CRITICALTTY RANKING SCALES

To rank the relativecriticalityof conditions within each condition class,

each should be rated according to the following parameters. Assign the

numbers zero to six to each parameter, as indicated.

I. What is the probability that 0 = Probability of the system

the system will operate in operating in this condition

this condition for some period approaches zero

of time during the sort of 1

operation in which it is 2

expected to take part? 3 = Probability of some operating

in this condition is about

500.

4

5

6 = Probability of the system

operating in this condition

at some time approaches 100%

2. To what extent was this system, 0 System/components not designed

or a major component of this for this condition

system, specifically designed I

to operate in this condition? 2

3 = Design takes this condition

into account

4

5

6 = Reason for the design of this

system/component is the

ability to operate in this

specific condition

3. To what extent does this 0 No history of problems

condition have a history of 1

being associated with in- 2

adequate performance and 3 = History of a number of

O other problems in similar nonsevere problems

systems of this generic 4

class? 5

6 = History of severe problems

and/or frequent problems

1-22
4. To what extent does this 0 = No history of problems

condition have a history of 1

being associated with in- 2

adequate performance and other 3 : History of a number of nonproblems in the system in severe problems

question during prior OT's 4

and/or DT's? 5

6 = History of severe problems

and/or frequent problems

5. To what extent will this 0 = No unusual demands/stress

condition place unusual 1

demand or stress on the 2

operator and/or system? 3 = Average amount of demands/stress

4

6 = High level of demands/stress

'Ie

=.G.23

-2".23

- . ..................................- ° .- .--
CONDITION DIFFICULTY RANKING SCALES

To determine the difficulty of performing an HPF under a specific condition,

it is necessary to evaluate the difficulties of the condition present in the

input, decision making and output components of the HPF. These component

difficulties sum to an overall difficulty evaluation for that condition.

Assiqn the numbers zero to six to each of the following parameters, as

indicated.

1. To what extent does this condition 0 = Presents no additional difficulty

increase the difficulty of 1

perception of the external-system 2

environment for this HPF? 3 = Somewhat more difficult

4

5

6 = Very much more difficult

*q d

2. To what extent does this condition 0 = Presents no additional difficulty

increase the difficulty of the I

internal system environment, 2

including its displays and controls, 3 = Somewhat more difficult

for this HPF? 4

6 Very much more difficult

3. To what extent does this condition 0= Presents no additional difficulty

increase the difficulty of decision 1

making for this HPF? 2

3= Somewhat more difficult

4

5

6= Very much more difficult

,', 4. To what extent does this condition 0= Presents no additional difficulty

increase the difficulty of control 1

of motor responses for this HPF 2

3 = Somewhat more difficult

4

5

6 = Very much more difficult

1-24

I -..2a4 k - K.- a 
'0

5. To what extent does this condition 0 = Presents no additional difficulty

increase the difficulty of multiple 1

demands in this HPF? (Multiple 2

demands, in this context, relate 3 = Somewhat more difficult

to time sharing of perception, 4

decision making and motor 5

responding.) 6 = Very much more difficult

1-25

,6%

- * -1-25
WORKSHEET

RELATIONSHIP BETWEEN CONDITION

CRITICALITY-DIFFICULTY AND TRAINING TIME

Conditions

2 j , - - - - - - - -

3

- ,-

sum

22

33aa

PROC FSM1 I[ I-

-1-a
SYSTEM

IN4DEX

Mll SS I1M

CHARTS

SYSTEM

PERFORMANCE

ISSUES

EVAUNCTTONS

SYSTEM/HUPMANSYTEMLAT

DEVELOPMENT (PRorOryPE EXISTS)

HUMAN PERFORMAE I

ON TRINIPGCNDTIONS 44

ENECTO sso

DEINO AI*m 1CNRSU

REOUIeqWT n i'ATIO-2AN

- ~ ** - * * ~ *\E* '-URE
HUMAN RESOURCES IMPLICATIONS AND MEASURES

Human Resource Implications are those questions which yoU ask to aid in

- determining the causes(s) of operator/maintainer sub-criterion performance.

Human Resource Measures are a means of answering the Human Resource

Implications. The measures of Human Resource Implications can be taken

before, during, or after the OT exercise. Whil' the OT is in its planning

stage, you should determine when each measure should be taken. This section

includes a procedure designed to aid you in making this determination.

During OT planning, first read the allocation procedure on page :-ZA. Next,.

read the Human Resources Implications and associated Measures on pages -

and apply the allocation procedure to them. Include these allocated Measures

in your Test Plan. If you discover that you have left out important Human

Resource Measures, immediately following the OT exercise read through these

Measures again and select appropriate ones which you can still take.

W

1-28

I\
'p

'4

PROCEDURE FOR DETERMINING

WHICH HUMAN RESOURCE ISSUES

TO TEST BEFORE OR DURING OT

STEP 1: List all the human performance functions (HPF) you are going to

test in OT on the rows of the Human Resource Issue Work Sheet

Table 1 (pager-C). Make additional copies of the worksheet if

necessary.

STEP 2: Rate each HPF on each of the four rating items listed on page

Place the items' rating in the appropriate place of Table I

(page :1-30).

STEP 3: Sum the ratings for the four items for each HPF and enter the

value in the appropriate space in Table 1.

STEP 4: Rate each human resource area listed in Table 2 (page :r') on

each of the three rating items shown on page

STEP 5: Sum the ratings for the three items for each human resource area and

enter the value in the appropriate space in Table 2.

STEP 6: Transfer the sum values from Tables 1 and 2 to the margins of

Table 3 (page X-).

" STEP 7: Multiple the marginal values in Table 3 to obtain a product for

each cell in the task.

STEP 8: The cells with the highest values should be considered for

4 measurement before or during OT.

1

1-29

• . ,"..%........................,.. .. ,' .,...*"-. ... , .............. ..-.... *.-
HUMAN RESOURCE ISSUE WORK SHEET

TABLE 1

Items from Page

Human Performance Functions 1 2 34 Sm1t

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __1-30to
TABLE 2

Items from Page

Human Resource Items 1 2 J 3 Sum 1 to 3

Training

Hum a n Eng in e e r in g ___________________________

Manpower Planning___________1

Logi sti cs__ _ _ _ _ _ _ _ __ _ _ j__ _ _ _ _ _I

1-31
TABLE 3

Human Resource Areas

SUM ITEMS Training HFE Manpower Logistics

1-3 (Table 2) m m---

Human Performance Functions

SUM ITEMS

1-3 (Table 1)

1-32
- RATING ITEMS FOR HUMAN PERFORMANCE -'NC71 N S

*To what extent does the system involve a modification of improvements

which could impact this HPF.

* .2. To what extent have similar existing systams, or prior OT or DT activities,

* demonstrated problemns with this HPF.

3. To what extent is it anticipated t.hat this HPF will not be successfully

performed during OT.

4. To what extent will performance of this HPF be evaluated during 0T.

1-33
RATING ITEMS FOR HUMAN RESOURCE AREAS

1. To what extent have similar existing systems or prior OT or DT

activities demonstrated problems with this human resource area.

2. To what extent are the human resource measures associated with this

area obtainable after the OT .xercise has been completed.

3. What is the cost and/or feasibility of obtaining the measures associated

with this human resource area before or during OT relative to the

cost/feasibility of obtaining them after OT.

6

1-34
HUMAN RESOURCE ISSUES

- TRAINING -

The following measures apply to those Human Performance Functions (HPF)

for whi.h performance has been sub-criterion.

FOR THE HPF WHICH IS AT A SUB-CRITERION PERFORMANCE LEVEL APPLY THE

FOLLOWING ISSUES:

1. To what extent has adequate time been spent in training on this HPF?

1.1 Relationship between relative HPF criticality-difficulty and

relative percentage of time spent in training. (See procedure

for Diagnostic Measure #1, page .)

The relative amount of time spent in training should match the

relative criticality-difficulty of the HPF. This measure is an

attempt to determine if an adequate amount of training time was

spent on the HPF in question. If, in relation to other HPF's,

* .""less time was spent training an HPF than its criticality-difficulty

indicates, and if that HPF did not reach criterion performance

-i level, then one may infer that insufficient time was spent on it.

2. How well did students perform this HPF in training?

S.2.1 Mean exit score

Exit scores are those numerical ratings which are assigned to

* . each student for a given HPF as a result of OT training. OT

students should have been trained on selected, critical HPF's to

1-35
predetermined levels of performance. They should have been

given tests which, when performed at or above the predetermined

performance levels, permitted exit from that particular part

of the training. Exit scores must be kept as raw scores, that

is, not converted into pass-fail. All exit scores must be

recorded for this measure, regardless of the students' passing

or failing. If the students do better than the predetermined

exit criteria, this must be noted and not collapsed into a

homogeneous pass score. Exit scores should equal or exceed OT

HPF criteria for successful field performance.

2.2 Trainers' ratings of student ability

It is assumed that student ability has a significant impact on

training effectiveness in OT. It is also assumed that exit

scores, though important, do not also permit suffic-ient latitude

for an understanding of the students' level of ability in OT.

For this reason, a form is provided to aid trainers in the

assessment of student ability. Since the OT training package

trains personnel who come into the OT. with individual ability

levels, an .understanding of training effects necessitates an

understanding of pre and post training student ability. The

appropriate form may be found on page

2.3 Mean number of exit trials to reach exit criterion in training

It is assumed that all students, in the OT training package, may

not be able to reach HPF performance exit level on their first

attempt. The mean number of exit t-ials required to reach exit

criterion provides further information about both the

effectiveness of the training and the ability level of the

students.

1-36
2.4 Number of individuals who failed to reach exit level criterion

It is possible that, within the specified numbe~r of exit trials

permitted, some students may not be able to reach the required

level. The number and consequent percentage of such students

will provide further data to aid in the determination of the

effects of training on OT performance. If a large number of

students fail, it may indicate a potential difficulty with

the HPF.

3. To what extent were applicable training methods used to train this HPF?

3.1 Compatibility of training method to skill being trained

The skills required for the performance of HPF's may be trained

by a number of different methods. The nature of the skills ought

to determine which of the training methods is used. Sub-criterion

performance in the OT may be related to inappropriate training

method used. The compatibility between training method and skill

may be viewed as affecting OT performance in the following manner.

The less compatible these are, the greater the likelihood of

ineffective learning and consequent inadequate performance. For

the procedure to determine this compatibility, see pages _T-iI

4. How effective were the trainers in OT training?

4.1 Applicable training experience of trainers in months

There are certain assumptions one makes when assessing the

effectiveness of trainers. One of these assumptions is that

more training experience should increase the effectiveness

of training. Knowing the amount of applicable training

1-37
.4o ..

experience that a trainer has had may serve as an aid in

explaining student OT performance.

4.2 Operator/maintainer experience or training on current or similar

systems (in hours or months, as applicable)

It is recognized that individuals who are not highly proficient

at a particular skill may still be able to train others

effectively on that skill. In general, however, it is assumed

that there is some positive relationship between trainer

.operational ability and ability to train a skill effectively.

..3 MOS skill level of trainer in applicable specialty

This measure may be viewed as a subset of. 4.2. It is taken for

much the same reason.

4.4 Student ratings of trainers' abilities

Student ratings of trainers serve two possible functions:

(1) They can provide information about the trainers' abilities,

" =. which leads to a greater understanding of the effects of

training on OT.

(2) They can serve as an index of the students' attitudes toward

their trainers, and this itself may have a significant bearing

on the effect of the training package. A student rating form

may be found on page

5. What is the relationship between conditions applied in OT training and

OT exercises?

. Similarity of conditions applied in training to conditions

encountered in OT exercises

1-38

/.:." "--" . ,--"," -- .. ". "."." -- ', ." ' ", . . - .:.' . :_ __. , :
One of the goals of training is to prepare students for the

conditions they are likely to encounter in the operational

environment. If students, who otherwise pass satisfactorily

through an OT training course, find it impossible to reach

criterion levels in the field exercises of the OT, one may

discover that they could not make the transition from training

conditions to field conditions. For this reason, it is desirable

S;- to determine the similarity between those conditions which were

applied in OT training and the conditions actually seen in the

OT exercises. The greater the disimilarity between those two

sets of conditions, the more likely training is to be a cause of

sub-criterion performance in OT. Procedures for obtaining this

measure may be found on page--.

6. How effective was the organization and learning environment in OT

training?

6.1 Student evaluation of training exoerience

. * As in 4.4, student ratings of the training experienc serve two

possible funtions:

(1) They can provide information about the training package, in

general, which may not be available through more standardized

quantifiable means. This information will tend to lead to a

greater understanding of the effects of training on OT.

(2) They can serve as an indication of students' attitudes toward

the OT training. These attitudes, whether valid or not, will

influence the effect of the OT training package on the students

r and, therefore, provide further information on the reasons for

sub-criterion OT performance. A student rating form can be

found on page J,-1,4.

1-39

I. "
• - _ -- ;

 ..  . . . . , .  .-.. , . ,  . I , -..- . ~ . .. +_ + .  i <  + -

6.2 Trainers' evaluation of the CT training experience

(1) They can provide information about the training package, in

.general, which may not be available through more standardization

quantifiable means. This information will tend to lead to a

greater understanding of the effects of training on OT.

(2) They can serve as an indication of trainers' attitudes toward

the CT training. These attitudes, whether valid or not, will

influence the effect of the CT training package on the trainers

and, therefore, provide further information on the reasons for

sub-criterion OT performance. A trainer rating form can be

found on page

1-40

,+'"-i

0..'

" ... ,

5 .* . . *. - * - *-* - - - * , - ..- ***Jn.*-. . .
.  : 

. 

,, 

L

PROCEDURE FOR DIAGNOSTIC MEASURE #1

(Relationship between relative HPF criticality-difficulty and

relative percent time spent in training).

Step 1. Determine the criticality for each HPF involved in OT training

(regerdless of whether performance in OT was sub-criterion).

1.1 Rate each HPF on each item of the criticality rating form

contained on page

1.2 Write the individual criticality ratings on the worksheet

contained on page (Photocopy a sufficient number of

worksheets to accommnodate all HPF's to be evaluated).

1.3 Total the ratings for each HPF and enter the values in the

appropriate spaces on the worksheet.

Step 2. Determine the difficulty for each HPF involved in OT training

(regardless of whether performance in OT was sub-criterion).

2.1 Rate each HPF on each item of the difficulty rating form

contained on page X-q7.

2.2 Write the individual difficulty ratings on the worksheet

contained on page z-q,

2.3 Total the ratings for each HPF and enter the values in the

appropriate spaces on the worksheet.

Step 3. For each HPF, multiply the criticality sum by the difficulty sum

to obtain the product of sums; then enter the values in the

appropriate spaces on the worksheets.

1-41
"F 7 P7

Step 4. Rank the product of sums, using "I" for the highest product;

this is the most critical/difficult HPF.

Step 5. Obtain from the OT training personnel the mean amount of training

time spent on each HPF.

Step 6. Enter the training time for each HPF in tne appropriate place on

the worksheet.

Step 7. Rank the training times, using "" for the most time.

Step 8. Compare the ranked product of sums with the ranked training time.

8.1 If the ranked product of sums is larger or equal to the

ranked training time, then one may conclude that adequate

training time was given on this HPF.

8.2 If the ranked product of sums is smaller than the ranked

training time, then one may conclude that insufficient

training time may have been given on this HPF.

1-42

-I-9-
HPF CRITICALITY RATING FORM

To compute the criticality of HPF's for training, each should be rated

according to the following parameters. Assign the numbers zero to six

to each of the following parameters, as indicated.

1. To what extent is this system, 0 = Not designed specifically for

or a component of this system, HPF

specifically designed to carry 1

out the HPF. 2

3 = Design takes the HPF into

account but is not specifically

designed for it

4

5

6 : Principle reason for the design

of this system is the performance

of this specific HPF.

2. In relation to other HPF's how 0 = Seldom if ever

often is it performed in this 1

system? 2

3 = About as often as any other

4

5

6 = This system will only perform

this HPF.

3. What would be the effect of failure 0 = No effect

to successfully complete this HPF 1

on survivability of the system/ 2

operator? 3 : Medium probability of destruction

4

5

6= High probability of destruction

or surrender of system

1-43

. . . .-.. . . . . . . . . . . . . . . . . . . . . . . . . .
4. To what extent have other systems 0 = No history of problems

in this generic class had a history 1

of inadequate performance or 2

encountered problems in carrying 3 = Has encountered a number of

out this HPF. severe problems

4

6 : History of severe problems

and/or frequent problems

5. To what extent have prior DT's/ 0 : No history of problems

OT's indicated problems in 1

successfully performing this 2

HPF. 3: Has encountered a number of

severe problems

4

5

6 : History of severe problems

and/or frequent problems

6. What would be the effect of 0 : No effect

failure to perform this HPF 7

on the effectiveness of the 2

system's mission? 3= Mission significantly impaired

but not a complete failure

4

5

6 High probability of total

mission failure

-4

6

.

• .%.1-44
HPF DIFFICULTY RATING FORM

To determine the difficulty of an HPF, it is necessary to evaluat e the

i difficulties of its input, decision making and output components. These

:omponent difficulties then sum to an overall difficulty evaluation.

Assign the numbers zero to six to each of the following parameters, as

indicated.

1. To what extent is perception of 0 = Presents no difficulty

the external-system environment 1

a source of difficulty to this HPF? 2

Factors contributing to this 3 = Somewhat difficult

difficulty include: 4 ' 5

(a) vibration 6 Very difficult

(b) inadequate illumination

(c) inadequate visual access

(d) too much noise

(e) many inputs to attend to

2. To what extent is perception 0 : Presents no difficulty

of the internal system environment, 1

including its displays and controls, 2

a source of difficulty in this HPF? 3 = Somewhat difficult

Contributing factors include: 4

5

(a) vibration 6 = Very difficult

(b) inadequate illumination

(c) poorly designed on placed

displays/controls

(d) inadequate visual access

(e) many inputs to attend to

(f) too much noise

1-45

' '- ", "; , ' o - '_- -" .' .t' ,- / " - - - ' '" . ,; . . . ,' '' : ''''' ' T' : 'I 3 _. '
- -. . . ..

3. To what extent is decision making 0 = Presents no difficulty

a source of difficulty in this I

HPF? Contributing factors include: 2

3 Somewhat di'fficult

(a) large number of decision 4

points 5

6 = Very difficult

(b) many alternatives per

decision

(c) lack of formal decision

rules

(d) tight time constraints

(e) important consequences

4. To what extent is the control of 0 : Presents no difficulty

motor responses a source of I

difficulty in this HPF? 2

Contributing factors include: 3 = Somewhat difficult

4

(a) vibration 5

6 = Very difficult

(b) improper design of a control

(c) poor position of control

(d) short time constraints

(e) close tolerences required

(f) large amount of strength required

(g) many responses required

5. To what extent are multiple demands 0 = Presents no difficulty

inherent in this HPF a source of I

difficulty? Multiple demands in 2

this context relate to time sharing 3 = Somewhat difficult

of perception, decision making and 4

motor responding. 5

6 = Very difficult

1-46 0'
'4S-.777. 77077 W. .

WORKSHEET

RELATIONSHIP BE,4EEN HPF

CRITICALITY-DIF!CULTY AND TRAINING TIME

HPF

/

S2

I'M

S - .. 1 

rM I IP4 IR 

i, J

POCTRANKED

rRAIMING 71ME

1-47

4.
DIAGNOSTIC TRAINING MEASURE 43

COMPATIBILITY OF SKILL AND TRAINING METHOD

Step 1. Enter percent of'time spent training this'HP in each

modality listed below:

Modality % Time Spent in Training

" 1. Classroom-lecture

2. Audio-visual presentation

3. Paper Pencil Test

4. Hands-on simulation

5. Field Experience

6. Observation of other

students.

Step 2. Compute a time index for each of the skills listed below

using the equations given.

where %Ml = % of time spent in training

this HPF using modality 1 (classroom-lecture)

%M2 = % of time spent in training

this HPF using modality 2 (audio-visual presentation)

and so-on for each modality listed in Step 2.

Skill 1 (Visual Discrimination) Index :

(%M1 x 1.1) + (%M2 x 5.8) + (%M3 x 4.0)

+ (%M4 x 3.0) + (%M5 x 2.8) ..

Skill 2 (Fine hand motor control) Index-

(%Ml x .1) + (%M2 x .2) + (%M3 x .1)

+ (%M4 x 6.4) + (%M5 x 6.8) + . .

1-48

60

-. o J
0

(-.

Skill 3 (Team Coordinator) Index =

Skill N ( ) Index =

Step 3. Enter the percent of HPF performance that involves each of the

skills listed below.

% of HPF Performance

Skill Involving the Skill

1. Visual

2. Fine hand motor control

3. Team Coordinator

N

Step 4. Rank the skill time indices (computer in Step 2) from lowest

(Rank 1 1) to highest (rank.= N) and enter them on the chart

below.

Rank the '0' of HPF performance involving each skill (computed

in Step 3) from lowest (rank - 1) to highest (rank = N) and

enter on the chart below.

Skill Time % HPF Performance

Skill Index Rank Rank

i. Visual Discrimination

2. Fine hand motor control

3. Team Coordinator

N

1-49

S - a,- -•
Step 5. Compute the degree of concordance between the two sets of ranks

in Step 4. The higher the value, the better the compatibility

of skills and modalities used to train this HPF.

1-50
'edi

TRAINING DIAGNOSTIC -MEASURE 5

CONDITIONS IN OT TRAINM:iG AND OT EXERCISE

Step 1. List the condition(s) under which performance on this HPF failed

in the operational test

Condition in OT

2.

3.

4.

Step 2. Determine the percentage of time this HPF was practiced in each

condition identified in Step 1.

Condition % Time Practiced

(from Step 1) in Training

1.

2.

3.

4.

Step 3. Was there any condition under which no practice on this HPF

took place in training (that is, are there any zero percentages

listed in Step 2)?

If the answer is then:

Yes may indicate that HPF performance was

suboptimal because no practice was given

under this condition in OT training.

No Continue with Step 4.

1-51
,,'.'

F Step 4. Compute the criticality of each condition using the HPF

condition criticality rating form on page Enter the values

below.

Condition Rating Form

(from Step 1) Rating Form Items Sum

1 2 3 4 5 5 1 to 5

j-.

2.

3.

Step 5. Compute the difficulty of each condition using the HPF condition

difficult rating form on page Enter the values below.

Condition Rating Form

(from Step 1) Rating Form Items Sum

1 2 3 4 5 1 to

.. 1 1.2. -__.. . .

3.

Step 6. Compute for each condition, a single criticality-difficult index

by multiplying the criticality and difficulty ratings from Steps

4 and 5. Rank these giving a rank of 1 to the lowest product.

Condition Step 4 Step 5 Product Rank

(from Szep 1) Sum 1-6 X Sum 1-5

2.

3. _

1-52

, . ... 
Step 7. ank the pe!-cef tae *4me v.aldes 4,1 2-n 2crr-'te - the design

*.of c ncord-anrce a-w-en tnese ra~l~s ,rl,, -: c~e coc e in Ste,

Pc rcentaseks

Condi tl n Time ;anks F5

3.

%.4
THIS PAGE INTENTIONALLY LEFT BLANK

1-5

4*-2 7* i